Path to production for ML models

Extracted from evanto elements

Let's start by going on machine learning workflow to show how automated pipelines fit into this process. Automated pipelines actually span all the workflow steps, including ingest and analyze, prepare and transform, train and tune, and finally deploy and manage.

MLOps builds on DevOps practices that encompass people, process, and technology. MLOps also includes considerations and practices that are unique to machine learning workflow. All of these practices aim to be able to deliver machine learning workloads. All of these practices aim to be able to deliver ML workloads quickly to production while still maintaining high quality consistency and ensuring end-to-end traceability.

Consideration to ensuring the model have a path to production.

1. Machine Learning Development Lifecycle (MLDC) is very different from the software development life cycle

• The model development life cycle is difficult to plan for from a project management perspective
• Typically, includes longer experimentation cycles than a standard agile software development process
• Development of Machine Learning models includes data tasks like feature engineering and data preparation.
• There are also a data processing code
• New inputs and artifacts to consider for versioning
• Additional pipeline task for production
• Dependencies that can be a bit more challenging

2. A Model may be a small part of an overall solution

• Reading prediction requests and getting responses through a batch process or even within your notebook on an ad hoc basis
• Need for other components that need to be built or integrated
• You also need to consider how that prediction will actually be used and potentially integrated into other existing applications
• Creating automation to initiate back-end processes that allow for customer support engineers to quickly react and respond to any negative reviews

3. Multiple personas span the machine learning development lifecycle, and all are really needed to ultimately be able to build, deploy, integrate, and operate a machine learning workload.

• There may also be skill gaps in building an operating machine learning workloads
• many teams have processes in place supporting different regulatory or even internal corporate requirements

Now that we have all the considerations, let’s start the workflow

Machine Learning Workflow

MLOps aims to provide that path to production by reducing manual hand-offs between the steps in your workflow, increasing automation within those steps in your workflow, and then going a step further to orchestrate the steps across your workflow. But you don’t want to just apply automation, you also want to improve the quality of your models. To do that, you need to establish mechanisms and quality gates inside your machine learning pipeline:

• Incorporates cross team collaboration with automation to reduce those hand-offs and delays in a workflow.
• Incorporate a centralized Model Registry, Model Registry holds key metadata and information about how your model was built, how the model performed with evaluation metrics
• Once that model has been registered and approved, it can then trigger that downstream automated pipeline to deploy your model
• That deployment pipeline typically includes the steps that are needed to package your model and then deploy it to one or more environments.
• for operating your model, the correct monitors need to be identified and put in place early through your cross-functional teams: model quality drift or data drift

With automation, you’re typically looking at automating the step and the tasks within that step, that are required to accept an input or inputs and then ultimately produce some type of output or artifact.

It’s important to not only automate these individual tasks, but also the end-to-end machine learning workflow as well.

our orchestration layer can also provide those quality gates that I talked about a bit before. You can use this orchestration layer to implement quality gates between your steps to determine when the pipeline should proceed to the next step in your machine learning pipeline.

For model building, you’re typically setting up a minimum threshold for your model evaluation metrics. This typically means establishing a metric that you’re trying to optimize for, so something like accuracy or F1 score.

On model integration, you want to make sure that the application that’s consuming your model is able to get prediction requests bag.

Then finally for model monitoring, you want to monitor for standard metrics, so things like CPU, GPU, or memory utilization.