A Beginner’s Guide to MLOps

So, what really is MLOps?

MLOps is a portmanteau of the words Machine learning (ML) and DevOps (Ops). It basically means DevOps for Machine Learning applications.

Being a MLOps Engineer is just a step away. Check out the MLOps Course Training in Bangalore at 360DigiTMG and get certified today.

Why MLOps?

There has been an incredible surge in the number of organisations attempting to use data science and machine learning to benefit their operations as a result of the explosion of data around us and the development of accessible, low-cost computing power in the form of cloud computing. The creation of new and better algorithms and their implementation as standalone libraries or as packages in the Python and R languages have accounted for the majority of advancements in data science and artificial intelligence.

Enterprises have, however, swiftly come to the realisation that although performing a POC on a laptop (or other on-premises device) is simple, deploying these POCs into production and at scale is really rather difficult. As businesses have grown and focused on enhancing their array of data science applications, it has become clear that they must find solutions to the following issues:

• Production deployment of models
• Infrastructure considerations - cloud, on-prem or hybrid?
• Scaling the models up and down
• Continuous (re)training
• Continuous deployment
• Model management
• Metadata Management

and several other similar factors. Companies are now realising that data science is more complex than first appears. The field of data science is in dire need of innovation, particularly in terms of getting the models to market rapidly.

Enter MLOps

Wikipedia defines “MLOps or ML Ops as a set of practices that aims to deploy and maintain machine learning models in production reliably and efficiently.” According to this report, a staggering majority (88%) of companies are struggling to put ML/AI models into practice. It also suggests that the companies that actually put ML/AI models into production are expected to see a 3-15% increase in their profits.

Businesses are beginning to see the enormous benefit in spending money to create a scalable MLOps platform. This problem has also been encountered by Google and Facebook, two leaders in the AI and ML industries. Some of them created technologies for internal use that they subsequently made open source. Google came up with and made open-source software like Kubernetes, Kubeflow, and Tensorflow Extended. With others following suit, there has been a huge snowball effect, resulting in an explosion of MLOps-based frameworks and tools.

The main goal of MLOPs is to automate the related pipelines and data engineering activities while also significantly enhancing the quality of ML models currently being used in production. While MLOps and DevOps and DataOps share a balanced emphasis and certain commonalities, MLOps has undergone an unexpected and extremely evolutionary endeavour. This is largely because MLOps has evolved into a comprehensive framework that covers the entire machine learning model development lifecycle (MDLC), which combines model development with continuous deployment, integration with software apps, and continuous training, governance, and diagnostics.

This has resulted in a purposefully independent path forward for managing the lifecycle of ML models which has spawned an engineering culture that unifies ML model development with operations by applying best practice guidelines from DevOps culminating in what we now know as MLOps.

Change below image to avoid copy right issues:

But, isn’t MLOps the same as DevOps?

Looking forward to becoming a MLOps Engineer Check out the Best MLOps Course in Pune and get certified today.

Yes and No. MLOps shares a lot of similarities with DevOps but it has distinctive features that separate it from DevOps in very significant ways.

• MLOps has an exploratory/experimental outlook : The core function of both DevOps and MLOps is to ensure that a software can be produced in a repeatable and fault tolerant workflow. But MLOps also has to contend with added uncertainty in terms of hyperparameters, managing data and code to ensure reproducible results etc.
• Automated Deployments: Continuous retraining requirements exist for MLOps which are not present for DevOps.
• Stateful applications: ML Models have the added complexity of ensuring that the data on which inferences are issued have comparable distributions relative to the training data. This is to avoid a phenomenon called Train/Serve skew.
• Monitoring requirements: In DevOps you simply needed to monitor code and/or infrastructure, but in MLOps the models need to be continuously monitored for performance decay etc and retrained periodically.

The following illustrates how the typical CI/CD pipelines in DevOps have taken on new significance in MLOps:

Integrating continuously (CI): Testing and verifying data, schema, and models are included in continuous integration (CI), in addition to testing and validating code components.\

Continuous Deployment (CD): CD develops into a framework for rolling back model deployments as needed.

Continuous Training: MLOps deployments are the only situations in which this idea is used. Depending on how well a model performs in applications in the real world, it might need to be updated. Retraining it to respond to time-based or event-based triggers can be necessary.

For copyright reasons, alter the below image with alternative colours. We may further cite this source:

MLOps maturity index: Based on the degree of automation involved, the sophistication of the pipeline architecture, and the implementation of various features like model management, experiment tracking, etc., MLOps implementations may be widely thought of as having three stages.

• Manual Processes (Level Zero)
• Pipeline Automation (Level One)
• Pipeline Automation with CI/CD/CT (Level Two)

Let us take a look at each of these phases in detail.

Level Zero :

Usually prevalent in companies that are just testing their ML models and applications. It is usually characterized by the following features:

• Manual processes : Highly dependent on Data Scientist involving use of multiple scripts etc, this level of maturity is notable for all of the steps being performed manually - data ingestion and preparation, model training, tuning and deployment etc. No concept of experiment tracking, model management etc
• No CI/CD/CT
• ML and Ops are highly disconnected resulting in non-smooth workflows during deployment and updates

Want to learn more about Best MLOps Course. Enroll in this MLOps Course in Hyderabad to do so.

Level One:

At this stage, automation of the ML pipeline streamlines continual training. When a customer's behaviour is constantly changing and the model needs to adjust quickly, this kind is often good.

• Continuous Training: The models could be refreshed and retrained continuously or as required
• Lightning fast experimentation: All the steps of the machine learning model development life cycle (MLDC) are automated
• Pipeline Automation: The entire ML training pipeline is deployed into production and serves the predictions automatically via pipelines

Level Two:

This is the most advanced MLOps framework, replete with an automated CI/CD system that aids in the quick and dependable deployment of ML pipelines. This is an end-to-end MLOps platform that includes a feature store, ML metadata store, source code version control, testing capabilities, deployment capability, option for maintaining model registries, and pipeline orchestrator tools like Kubeflow, Apache Airflow, etc.

• Model development: Different models are developed via experiment management
• Pipeline CI/CD : During the CI phase of the pipeline, the source code is built and tests are run. These artifacts are then deployed during the CD phase resulting in a new model deployment.
• Triggering the pipelines: This is automated in this level and could be time based or event based triggers
• Monitoring: Model is monitored for both data and concept drift
• Continuous Training: Model is being continuously refreshed based on the monitoring results from previous phase

MLOps Frameworks:

• TFX + Pipeline orchestrators (Kubeflow, Airflow etc): Tensorflow Extended is GOogle’s open source version to write production grade ML code and simplify the process of going to production. One drawback of this is that it needs a pipeline orchestrator such as Kubeflow or Airflow to fully automate and orchestrate its ML pipelines
• Kubeflow: Kubeflow is a wonderful open source project whose main goal is to deploy ML workloads on Kubernetes based infrastructure with ease and scale. It provides a best of the breed open source systems for machine learning in highly diverse environments such as Cloud, On Prem and Hybrid landscapes
• MLFlow: Another awesome open source framework which manages the end to end ML model lifecycle. It provides various steps crucial in ML lifecycle through its components namely, Tracking, Projects, Models and Model Registry
• AWS Sagemaker Studio (Pipelines): The full service solution of AWS cloud, it is quickly becoming one of the best managed services options provided by a major public cloud provider
• Google Vertex AI: The Google Cloud counterpart to Sagemaker, this has been recently released and trying to compete with Sagemaker
• DataRobot: A UI based (no/low code) alternative to perform MLOps at scale
• Cnvrg.io : A low code DAG based MLOps platform geared towards Data Scientists

You now have a comprehensive understanding of what MLOps is, why it is important, its advantages, platforms that offer MLOps capabilities, and their open-source equivalents. I hope you find this post beneficial.