MLOps with Kubeflow Training in India

In today’s world data science has penetrated across all industries and domains and has become ubiquitous. Most data scientists and machine learning engineers are able to come up with amazing models as proof of concepts but they are unable to deploy them in production and at scale. This has given rise to something called MLOps which basically means DevOps for Machine Learning. This chapter dwells deep into this need.

This module offers a complete overview of everything you need to know about Kubernetes. This module introduces Kubernetes and then explains why containers are required. The module explains the basic building blocks of Kubernetes such as pods and how they can be used in applications and finally wraps it up with an explanation of the Kubernetes API.

This module will introduce the need for Kubeflow even when Kubernetes is already existing. This module will also attempt to answer how Kubeflow should be installed. It details the security constraints, infrastructure requirements, scalability, and reliability.

This module will touch up some concepts that may be familiar to users in the DevOps space. It will introduce concepts such as public key infrastructure (PKI), authentication, authorization, and role- based access control, (RBAC), Kerberos, and transport layer security (TLS). It will also introduce service mesh management with Istio.

This module will begin with an introduction of Kubeflow pipelines - which is a platform that comprises:

• A user interface that tracks, manages, and executes pipelines
• A pipeline execution scheduling engine
• Python SDK for managing these pipelines
• Leveraging Jupyter Notebooks for using the Python SDK

Data preprocessing is a multi-stage process which consists of collecting data from disparate sources, augmenting it, calculating basic statistics, handling missing values, and outliers. Feature engineering is the process of deriving additional features or removing unnecessary features to add more predictive power to the ML model. This chapter will introduce participants to the methods provided by Kubeflow to construct easily repeatable data processing and feature engineering pipelines.

Model training is the process of creating logical relationships between ‘training data’ and using it to make predictions on ‘unseen’ data. This chapter will focus on how to train models on Kubeflow using two different frameworks - Tensorflow and Scikit-learn.

This chapter discusses how to deploy, serve models, and continuously monitor and update them. Model serving means hosting the model which can be interfaced via a service. The models can be served through two approaches - embedded serving and model serving as a service (MaaS).