How to Turn Your OpenShift Pipelines Into an MLOps Pipeline

Data scientists and machine learning engineers often face significant challenges in taking models from experimentation to production. Machine learning is the iterative process of data collection from various sources, data processing, model training, hyperparameter tuning, model evaluation, and model deployment. Performing these operations manually every time you make a change to your model or its dependencies (code, datasets, or configurations) will lead to inconsistent results because:

• There is no continuous integration and continuous deployment system to automate this workflow.
• Limited version control makes it difficult to track changes across model iterations.
• It is difficult to monitor the performance of your models in real-time.
• Deployment is inconsistent across environments, leading to unpredictable results.

For these reasons, the demand for MLOps will keep expanding, and organizations will keep adopting it. The rapid growth of MLOps stems from organizations needing to reduce the friction between DevOps and machine learning teams. Organizations using MLOps pipelines gain a competitive edge by streamlining model deployment, monitoring, and scalability. Building an MLOps pipeline does not have to be tedious. With KitOps and OpenShift pipelines, you can quickly build an ML pipeline to get your AI models to production. This article will teach you how to easily build and deploy your machine learning model using KitOps and OpenShift.

TL;DR

• MLOps pipelines improve your machine learning workflows by incorporating automation.
• KitOps enables various teams to easily unpack models and their dependencies, such as code and datasets, into different directories.
• OpenShift pipelines and Quay make it easy to run KitOps packaged AI projects.

Steps to building an MLOps pipeline with OpenShift pipeline and KitOps

Prerequisites
To follow along with this tutorial, you will need the following:

• A container registry: You can use Jozu Hub, GitHub Package registry, or DockerHub. This guide will use Jozu Hub.
• A code hosting platform: Create a GitHub account.
• A HuggingFace account.
• KitOps: Here’s a guide to installing KitOps.
• OpenShift pipeline: Create a developer sandbox account.

Step 1: Install KitOps
First, you must make sure you have the Kit CLI installed locally. Once installed, run the command below to verify the installation:

kit version

Step 2: Create a JozuHub repository
Login to your Jozu Hub account and create a repository. Here, you create an empty repository called qwen-openshift.

To authenticate your local terminal to JozuHub, run the command:

kit login jozu.ml

This prompts for your username and password. Your username is the email address used to create your Jozu Hub account and the password. After authenticating, you’ll download a model from HuggingFace.

Step 3: Download a model from HuggingFace
Head over to the Qwen model on HuggingFace. You will see a list of files, including models, LICENSE and README.md. You can install all of these packages by running the command on your local terminal:

curl -L -O https://huggingface.co/Qwen/Qwen2-0.5B-Instruct-GGUF/blob/main/LICENSE
curl -L -O https://huggingface.co/Qwen/Qwen2-0.5B-Instruct-GGUF/blob/main/README.md
wget https://huggingface.co/Qwen/Qwen2-0.5B-Instruct-GGUF/resolve/main/qwen2-0_5b-instruct-q2_k.gguf

This installs the packages in your current working directory. Let’s make the directory structure more readable.

Currently, your directory structure should look like this:

|-- Kitfile
|-- models
  |-- qwen2-0_5b-instruct-q2_k.gguf
|-- docs
  |-- LICENSE
  |-- README.md

Author a Kitfile and organize your packages in two new folders: models and docs. Move the Qwen model into the models folder and the license and markdown document into the docs folder. Copy the following code in your Kitfile:

manifestVersion: 1.0.0
package:
  name: qwen2-0.5B
  version: 2.0.0
  description: The instruction-tuned 0.5B Qwen2 large language model.
  authors: [Emmanuel]
model:
  name: qwen2-0_5b-instruct-q2_k
  path: models/qwen2-0_5b-instruct-q2_k.gguf
  description: The model downloaded from hugging face
code:
  - path: docs/LICENSE
    description: License file.
  - path: docs/README.md
    description: Readme file.

Step 4: Pack your ModelKit
The next thing you want to do is to pack your ModelKit. To do that, run the command:

kit pack . -t jozu.ml/<your-Jozu-username>/<your-Jozu-repository-name>:latest

You created the Jozu repository in Step 2 above - qwen-openshift.

Tag the ModelKit as latest. After executing the kit pack command, you should see an output similar to the one below:

Step 5: Push your ModelKit
To push the ModelKit, run the command:

kit push jozu.ml/<your-Jozu-username>/<your-Jozu-repository-name>:latest

After executing the command, you should see an output:

With a successful push to your remote repository, you can view the packages you have uploaded to Jozu Hub’s container registry.

After pushing your model dependencies to Jozu Hub, deploy the model using OpenShift Pipelines.

Step 6: Create an OpenShift Pipeline
RedHat OpenShift is a platform that simplifies the building, testing, and deployment of applications at scale. With OpenShift Pipelines, you get a CI/CD framework that’s built into Kubernetes, allowing each step of the pipeline to run in its own container for better scalability.

Deploying your Qwen Modelkit to OpenShift pipelines is easy and can be done in a few steps. The first step is to create a developer sandbox account.

After creating your account, visit your dashboard, as shown in the image below.

To create an OpenShift pipeline, you can either use the pipeline builder on the OpenShift web console or a programmatic approach(YAML). In your GitHub repository, create a file called **.tekton/push.yaml** and add the following code:

apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: qway-openshift-pipeline
spec:
  finally:
    - name: pack-and-push
      params:
        - name: SCRIPT
          value:  cd <YOUR-GIT-REPO> && kit pack -t jozu.ml/<YOUR_JOZU_USERNAME>/<JOZU_REPO>:latest && kit push jozu.ml/<YOUR_JOZU_USERNAME>/<JOZU_REPO>:latest
        - name: VERSION
          value: '4.7'
      taskRef:
        kind: Task
        name: openshift-client
      taskRef:
        kind: Task
        name: openshift-client
      workspaces:
        - name: manifest-dir
          workspace: workspace
        - name: kubeconfig-dir
          workspace: workspace

  tasks:
    - name: install-kit
      params:
        - name: SCRIPT
          value: 'wget https://github.com/jozu-ai/kitops/releases/latest/download/kitops-linux-x86_64.tar.gz && tar -xzvf kitops-linux-x86_64.tar.gz && mv kit /usr/local/bin/ && kit version'
        - name: VERSION
          value: '4.7'
      taskRef:
        kind: Task
        name: openshift-client
      workspaces:
        - name: manifest-dir
          workspace: workspace
        - name: kubeconfig-dir
          workspace: workspace
    - name: login-to-jozu
      params:
        - name: SCRIPT
          value: kit login jozu.ml -u <JOZU_USERNAME> -p <JOZU_PASSWORD>
        - name: VERSION
          value: '4.7'
      runAfter:
        - install-kit
      taskRef:
        kind: Task
        name: openshift-client
      workspaces:
        - name: manifest-dir
          workspace: workspace
        - name: kubeconfig-dir
          workspace: workspace
    - name: move-models-to-folder
      params:
        - name: SCRIPT
          value: 'wget https://huggingface.co/Qwen/Qwen2-0.5B-Instruct-GGUF/resolve/main/qwen2-0_5b-instruct-q2_k.gguf && mv qwen2-0_5b-instruct-q2_k.gguf models'
        - name: VERSION
          value: '4.7'
      runAfter:
        - login-to-jozu
      taskRef:
        kind: Task
        name: openshift-client
      workspaces:
        - name: manifest-dir
          workspace: workspace
        - name: kubeconfig-dir
          workspace: workspace
    - name: git-clone
      params:
        - name: url
          value: '<YOUR-GIT-REPO>'
        - name: revision
          value: <YOUR-GIT-BRANCH>
        - name: refspec
          value: ''
        - name: submodules
          value: 'true'
        - name: depth
          value: '1'
        - name: sslVerify
          value: 'true'
        - name: crtFileName
          value: ca-bundle.crt
        - name: subdirectory
          value: ''
        - name: sparseCheckoutDirectories
          value: ''
        - name: deleteExisting
          value: 'true'
        - name: httpProxy
          value: ''
        - name: httpsProxy
          value: ''
        - name: noProxy
          value: ''
        - name: verbose
          value: 'true'
        - name: gitInitImage
          value: 'gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.40.2'
        - name: userHome
          value: /home/git
      taskRef:
        kind: Task
        name: git-clone
      workspaces:
        - name: output
          workspace: workspace
  workspaces:
    - name: workspace

This code creates an OpenShift pipeline of name qway-openshift-pipeline. This pipeline performs several tasks: it installs the Kit, logs in to the JozuHub repository, downloads the model from HuggingFace, packs the ModelKit, and pushes it to the JozuHub registry. When you navigate to the OpenShift web console, you’ll see an output similar to the image below.

You can trigger this pipeline manually or integrate with GitHub using webhooks. To run the pipeline manually, click on the run command. Once executed, you will see an output as shown below:

At this point, you’ve successfully built your ML pipeline. In the next section, we will show you how to deploy and interact with the chatbot you imported from Huggingface.

Step 7: Validating your deployment
By default, OpenShift creates two projects for you. On the developer portal, click on +Add. Under the Git repository, click on container images. Give your deployment a name. In this case, the deployment was named llama-cpp. Also, expose a port where the application will run.

Select the Qwen ModelKit you want to deploy from Jozu Hub and click on Create.

Once created, a domain will be assigned to your deployment. Behind the scenes, Kubernetes deployments, pods, service accounts, auto-scaling groups, and services are created for you. On the RedHat OpenShift dashboard, navigate to the Topology view to see a diagram representing your deployment.

Next, if you open the assigned domain, you'll see the deployed llama-cpp container. This includes a web UI where you can send prompts and view responses.

In the image above, the Qwen model was prompted to discuss the “French Revolution.” With a few steps, you've successfully built an MLOps pipeline for a chatbot.

Conclusion

Building an effective MLOps pipeline can be simple using the right tools. With KitOps and OpenShift pipelines, you can automate the building and deployment of your MLOps applications. This is also helpful because you can easily visualize your models and see how they perform in real-time.

KitOps plays a key role in packaging models and managing dependencies. OpenShift helps you automate deployment whenever a change is made. This results in faster, more reliable deployments and improved team collaboration.

If you have questions about integrating KitOps with your team, join the conversation on Discord and start using KitOps today!