Friday 26 August 2022

download from URL Kubeflow pipeline

I have been trying to create a kubeflow pipeline in my local machine and created a simple one component pipeline to download data from a given url.

import kfp
import kfp.components as comp

downloader_op = kfp.components.load_component_from_url(path)

def my_pipeline(url):
  downloader_task = downloader_op(url = url)

This works fine if I execute this pipeline through python code

client = kfp.Client() 

client.create_run_from_pipeline_func(my_pipeline,arguments=
     {
        'url': 'https://storage.googleapis.com/ml-pipeline-playground/iris-csv-files.tar.gz'
     })

But It shows an error when I try to execute the same code through Kubeflow UI. First I execute the above code which create a yaml file using below line and then upload the yaml file in kubeflow UI to create a new run. kfp.compiler.Compiler().compile(pipeline_func=my_pipeline,package_path='pipeline.yaml')

It give below error

This step is in Error state with this message: Error (exit code 1): cannot enter chroot for container named "main": no PID known - maybe short running container

I am not able to understand why it creates problems through UI and what is the solution.



from download from URL Kubeflow pipeline

No comments:

Post a Comment