--- title: From notebooks to pipelines with Kubeflow KALE description: What is Kubeflow? Kubeflow is the open-source machine learning toolkit on top of Kubernetes. Kubeflow translates steps in your data science workflow into Kubernetes jobs, providing the cloud-native interface for your ML libraries, frameworks, pipelines and notebooks. Read more about Kubeflow Notebooks in Kubeflow Within the Kubeflow dashb […] url: https://canonical.com/blog/kubeflow-kale?format=md --- 1. [Blog](https://canonical.com/blog) 2. Article --- [aymen frikha](https://canonical.com/blog/author/aym-frikha "More about aymen frikha") 28 July 2021 # From notebooks to pipelines with Kubeflow KALE [AI/ML](https://canonical.com/blog/tag/ai-ml) [deep learning](https://canonical.com/blog/tag/deep-learning) [Kale](https://canonical.com/blog/tag/kale) [Kubeflow](https://canonical.com/blog/tag/kubeflow) [kubernetes](https://canonical.com/blog/tag/kubernetes) [machine learning](https://canonical.com/blog/tag/machine-learning) [MLOps](https://canonical.com/blog/tag/mlops) --- Share the article ## What is Kubeflow? Kubeflow is the open-source machine learning toolkit on top of Kubernetes. Kubeflow translates steps in your data science workflow into Kubernetes jobs, providing the cloud-native interface for your ML libraries, frameworks, pipelines and notebooks. [Read more about Kubeflow](https://ubuntu.com/ai/what-is-kubeflow) ## Notebooks in Kubeflow Within the Kubeflow dashboard, data scientists can spin up [notebook servers](https://www.kubeflow.org/docs/components/notebooks/setup/) for their data preparation and model development. Upon creating the server, users can select among default images, or use a custom image, including one with pre-installed KALE. ## What is Kubeflow KALE? KALE (Kubeflow Automated pipeLines Engine) extends notebooks within Kubeflow in order to allow for automated pipeline creation. All you have to do is to annotate cells (pieces of your code) within your Jupyter notebook and these tags will be automatically translated into your Kubeflow Pipeline. You can tag a cell as a *component* or *block* indicating that the code within represents a step in your pipeline, and indicate the dependencies of that step (*pre: )*. ## DEMO: Using KALE with Charmed Kubeflow, ElasticSearch and Ceph In this demo, an Elasticsearch cluster will be used as a data store, a Ceph cluster as object storage to store the resulting models, and a Kubeflow environment is used as a data scientist platform to develop the AI algorithm as well as run the pipeline. The following diagram shows an overview of the environment: To set up the environment demo, you can follow the steps mentioned in this [repository](https://github.com/aym-frikha/kale-demo). To set up the environment demo, you can follow the steps mentioned in this [repository](https://github.com/aym-frikha/kale-demo). After setting up the environment and making sure that everything is running, we can now open a *sshuttle* tunnel (or an SSH forward) from your machine to the AWS instance using this command: ``` $ sshuttle -r ubuntu@ 10.64.140.43/24 ``` Then we access the Kubeflow dashboard and create a new Jupyter notebook with the Kale container image. We will use the following parameters for the notebook demo: **Name:** kale-demo **Custom image:** localhost:32000/kale-demo **CPU:** 1 **MEM:** 2 Gi In a new terminal within this notebook we need to download the financial time series demo notebook: ``` $ wget https://raw.githubusercontent.com/aym-frikha/kale-demo/master/financial-time-series.ipynb ``` We need to do some changes to the following notebook parameters that reflect the current environment: * **The endpoint\_url** (IP of the ceph-radosgw unit) * **The access\_key** (access key for the ceph user: from ./create-rados-user.sh output) * **The secret\_key** (secret key for the ceph user: from ./create-rados-user.sh output) * **The elastic\_url**  (IP of the elastic unit) Now we can enable Kale for this notebook and, in the advanced settings, we need to change the docker image to “localhost:32000/kale-demo” Then we can run the Kubeflow pipeline by hitting the button **compile and run**. This will compile and run your Kubeflow pipeline. Finally, if you click on *View*, you will be redirected to the specific pipeline run just created. ## Serving models on Kubernetes Enterprise computing is moving to Kubernetes, and [Kubeflow](http://ubuntu.com/ai/what-is-kubeflow) has long been talked about as the platform to solve MLOps at scale. [KFServing](https://ubuntu.com/blog/what-is-kfserving), the model serving project under Kubeflow, has shown to be the most mature tool when it comes to open-source model deployment tooling on K8s, with features like canary rollouts, multi-framework serverless inferencing and model explainability. Learn more about KFServing in [What is KFServing?](https://ubuntu.com/blog/what-is-kfserving) ## Learn more about MLOps Canonical provides [MLOps & Kubeflow training](https://assets.ubuntu.com/v1/ee5a96f3-Kubeflow+Training+Curriculum.pdf?_ga=2.196159596.1187423123.1619085522-809525743.1610727282) for enterprises alongside professional services such as security and support, custom deployments, consulting, and fully managed Kubeflow – read Ubuntu’s [AI services page](http://ubuntu.com/ai/services) for details. ## Simplify your Kubeflow operations Get the latest Kubeflow packaged in [Charmed Operators](http://charmed-kubeflow.io), providing composability, day-0 and day-2 operations for all Kubeflow applications including KFServing. [Get up and running in 5 minutes](https://charmed-kubeflow.io/docs/install) ## Sign up for our newsletter Get the latest Canonical news and updates in your inbox. Work email: \*I agree to receive information about Canonical's products and services. By submitting this form, I confirm that I have read and agree to [Canonical's Privacy Policy](https://canonical.com/legal/dataprivacy). Sign up ## Share on --- ## Related posts [### Canonical joins the Open Secure AI Alliance](https://canonical.com/blog/open-secure-ai-alliance) Canonical is now part of the Open Secure AI Alliance, announced by NVIDIA with partners across cloud computing, cybersecurity, enterprise software, open source foundations, and... [Canonical](https://canonical.com/blog/author/canonical) 28 August 2026 [### Beyond tokens per watt – using Ubuntu 26.04 LTS for AI](https://canonical.com/blog/beyond-tokens-per-watt) Tokens per watt (TpW) – the measure of useful AI work produced per watt of energy consumed – is the metric at top of mind for CEOs, heads of AI, and infrastructure teams alike.... [Freyja Cooper](https://canonical.com/blog/author/freyja-cooper) 5 June 2026 [### Securing AI agent workflows on Ubuntu with the new NVIDIA OpenShell snap](https://canonical.com/blog/nvidia-openshell-ubuntu-announcement) By packaging OpenShell as a snap, Canonical is enabling enterprises to confidently run next-generation agentic workflows across local devices, hybrid environments, and private clouds. [Canonical](https://canonical.com/blog/author/canonical) 1 June 2026 [### Canonical announces fully Managed Kubeflow AI operations platform on the Microsoft Azure Marketplace](https://canonical.com/blog/managed-kubeflow-microsoft-azure-canonical-release) Canonical has announced the general availability of Managed Kubeflow on the Microsoft Azure Marketplace. This fully managed MLOps platform allows enterprise AI teams to deploy... [Massimiliano Gori](https://canonical.com/blog/author/massigori) 21 May 2026