MLOps Training

The Tensorflow Extended (TFX) library contains all the components that are needed to build robust ML pipelines. Once the ML pipeline tasks are defined using TFX, they can then be sequentially executed with an orchestration framework such as Airflow or Kubeflow Pipelines.

During this module, you will learn to install TFX and its fundamental concepts along with some literature which will make the future modules easier to understand. Additionally, you will learn Apache Beam which is an open-source tool that helps in defining and executing some data manipulation tasks. There are two basic purposes of Apache Beam in the TFX pipelines:

• It forms the base of several TFX components for data preparation/preprocessing and validation
• Is one of the orchestration frameworks for TFX components so a good understanding of Apache Beam is necessary if you wish to write custom components

In the previous modules, we set up TFX the ML MetadataStore. In this module, we discuss how to ingest data into a pipeline for consumption in various TFX components (like ExampleGen). There are several TFX components that allow us to ingest data from files or services. In this module we discuss the fundamental concepts, explore how to split the datasets into train and eval files and practically understand how to join multiple data sources into one all-encompassing dataset. We will also understand what a TFRecord stands for and how to interact with external services such as Google Cloud BigQuery. You will also learn how TFRecord can work with CSV, Apache Avro, Apache Parquet etc. This module will also introduce some strategies to ingest different forms of data structured, text, and images. In particular, you will learn

• Ingesting local data files
• Ingesting remote data files
• Ingesting directly from databases (Google BigQuery, Presto)
• Splitting the data into train and eval files
• Spanning the datasets
• Versioning
• Working with unstructured data (image, text etc)

Data validation and preprocessing is essential for any machine learning algorithm to perform well. The old adage ‘garbage-in, garbage out’ perfectly encapsulates this fundamental characteristic of any ML model. As such, this module will focus on validation and preprocessing of data to ensure the creation of high performing ML models.

Data Validation: This module will introduce you to a Python package called Tensorflow Data Validation (TFDV) which will help in ensuring that

• The data in the pipeline is in line with what the feature engineering step expects
• Assists in comparing multiple datasets
• Identifies if the data changes over time
• Identify the schema of the underlying data
• Identify data skew and data shift

Real-world data is extremely noisy and not in the same format that can be used to train our machine learning models. Consider a feature which has values as Yes and No tags which need to be converted to a numerical representation of these values (e.g., 1and 0) to allow for consumption by an ML model. This module focuses on how to convert features into numerical representations so that your machine learning model can be trained.

We introduce Tensorflow Transform (TFT) which is the TFX component specifically built for data preprocessing allowing us to set up preprocessing steps as TensorFlow graphs. Although this step of the model has a considerable learning curve it is important to know about it for the following reasons:

• Efficiently preprocessing the data within the context of the entirety of the dataset
• The ability to scale the data preprocessing steps efficiently
• Develop immunity to potentially encountering training-serving skew

As part of the previous modules, we completed data preprocessing and transforming the data to fit our model formats. The next logical step in the pipeline is to begin the model training, perform analysis on the trained models and evaluate and select the final model. This module already assumes that you have the knowledge of training and evaluating models so we don’t dwell fully in the different model architectures. We will learn about the TFX Trainer component which helps us in training a model which can easily be put into production. Additionally, you will also be introduced to Tensorboard which can be used to monitor training metrics, visualize word embeddings in NLP problems or view activations for layers in a deep learning model.

During the model training phase, we typically monitor its performance on an evaluation set and use Hyperparameter optimization to improve performance. As we are building an ML pipeline, we need to remember that the purpose is to answer a complex business question modelling a complex real-world system. Oftentimes our data deals with people, so a decision that is made by the ML model could have far-reaching effects for real people and sometimes even put them in danger. Hence it is critical that we monitor your metrics through time—before deployment, after deployment, and while in production. Sometimes it may be easy to think that since the model is static it does not need to be monitored constantly, but in reality, the incoming data into the pipeline will more likely than not change with time, leading to performance degradation.

TFX has produced the Tensorflow Model Analysis (TFMA) module which is a fantastic and super-easy way to obtain exhaustive evaluation metrics such as accuracy, precision, recall, AUC metrics and f1-score, RMSE, MAPE, MAE among others. Using TFMA, the metrics can be visually depicted in the form of a time series spanning all the different model versions and as an add-on, it gives the ability to view metrics on different splits of the dataset. Another important feature is that by using this module it is easy to scale to large evaluation sets via Apache Beam. Additionally in this module, you will learn

• How to analyse a single model using TFMA
• How to analyse multiple models using TFMA
• Checking for fairness among models
• Apply decision thresholds with fairness indicators
• Tackling model explainability
• Using the TFX components Resolver, Evaluator and Pusher to analyze models automatically

This module is in many ways the crux of the MLOps domain because the original question was - ‘I have a great ML model prototype, how do I deploy it to production?’. With this module, we answer that question with - here is how: using Tensorflow Serving which allows ML engineers and data engineers to deploy any TensorFlow graph allowing them to generate predictions from the graph through its standardized endpoints. TF Serving takes care of the model and version control allowing for models to be served based on policies and the ability to load models from various sources. All of this is accomplished by focussing on high-performance throughput to achieve low-latency predictions. Some of the topics discussed in this module are:

• How to export models for TF (TensorFlow) Serving
• Signatures of Models
• How to inspect exported models
• Set up of TF Serving
• How to configure a TF Server
• gRPC vs REST API architecture
• How to make predictions from a model server using
  • gRPC
  • REST
• Conduct A/B testing using TF Serving
• Seeking model metadata from the model server using
  • gRPC
  • REST
• How to configure batch inference requests

Pipeline orchestration tool is crucial to ensure that we are abstracted from having to write some glue code to automate an ML pipeline. Pipeline orchestrators usually lie under the components introduced in the previous modules.

• Decide upon the orchestration tool - Apache Beam vs Apache Airflow vs Kubeflow
• Overview of Kubleflow pipelines on AI Platform
• How to push your TFX Pipeline into production
• Pipeline conversion for Apache Beam and Apache Airflow
• How to set up and orchestrate TFX pipelines using
  • Apache Beam
  • Apache Airflow
  • Kubeflow