This walk through will detail the technical configurations for using vRA Code Stream to deploy vSphere with Tanzu supervisor namespaces and guest clusters.
Requirement
For a recent customer proof-of-concept, we wanted to show the full automation capabilities and combine this with the consumption of vSphere with Tanzu.
The end goal was to use Cloud Assembly and Code Stream to cover several automation tasks, and then offer them as self-service capability via a catalog item for an end-user to consume.
High Level Steps
To achieve our requirements, we’ll be configuring the following:
- Cloud Assembly
- VCF SDDC Manager Integration
- Kubernetes Cloud Zone – Tanzu Supervisor Cluster
- Cloud Template to deploy a new Tanzu Supervisor Namespace
- Code Stream
- Tasks to provision a new Supervisor Namespace using the Cloud Assembly Template
- Tasks to provision a new Tanzu Guest Cluster inside of the Supervisor namespace using CI Tasks and the kubectl command line tool
- Tasks to create a service account inside of the Tanzu Guest Cluster
- Tasks to create Kubernetes endpoint for the new Tanzu Guest Cluster in both Cloud Assembly and Code Stream
- Service Broker
- Catalog Item to allow End-Users to provision a brand new Tanzu Guest Cluster in its own Supervisor Namespace
Pre-Requisites
In my Lab environment I have the following deployed:
- VMware Cloud Foundation 4.2
- With Workload Management enabled (vSphere with Tanzu)
- vRealize Automation 8.3
- A Docker host to be used by Code Stream
For the various bits of code, I have placed them in my GitHub repository here.
Configuring Cloud Assembly to deploy Tanzu supervisor namespaces
This configuration is detailed in this blog post, I’ll just cover the high-level configuration below.
- Configure an integration for SDDC manager under Infrastructure Tab > Integrations
- Enter the details of your environment
- Now click into your new account and select the Workload Domains tab
- Select your Workload domain name and Click Add Cloud Account
- Provide the infrastructure account details for vCenter and NSX
- Or let the system provision new dedicated service accounts for you.
Now we need to configure the Kubernetes configurations of the vSphere with Tanzu cluster so that I can be consumed by vRA Cloud Assembly.
So next I’ll walk through the following steps:
- Create a Kubernetes Zone and add in your Workload Management enabled cluster on the Cluster Tab
- Provide a tag to the cluster
In the same Cloud Assembly Infrastructure UI view:
- Go to Kubernetes Zones > New Kubernetes Zone
- Select your SDDC account
- Provide a name for your Kubernetes Zone
- Add a tag
- For my example files I used the tag “vwt-supc1”
- Click the Provisioning Tab
- Click add compute
- Select your cluster
- Click add
- Click save on the various dialog boxes to save your configuration.
Next, we need to link this Kubernetes Zone to our project.
- On your Project, select the Kubernetes Provisioning tab
- Click Add Zone
- Add the zone you’ve just configured
Now we have the infrastructure side of things configured, we will configure a Cloud Template to deploy our supervisor namespace.
Create your cloud template with the name “Deploy vSphere K8s Cluster in Namespace” and version of 1. This will mean importing the pipelines will be more seamless.
formatVersion: 1
inputs:
ns-name:
type: string
title: Enter Supervisor Name
description: Enter Supervisor Name
resources:
Cloud_SV_Namespace_1:
type: Cloud.SV.Namespace
properties:
name: ${input.ns-name}
contraints:
- tags: 'vwt:supc1'
- Create a version of this cloud template and release it, so we can consume it in Code Stream.
I recommend running a quick test to make sure a Supervisor Namespace is created successfully before you continue.
Creating a Code Stream Pipeline to deploy a Tanzu supervisor namespace and Tanzu guest cluster
Create the variables to be used
First, we will create several variables in the system, however as you see how these are used in the Code Stream pipeline, you could also specify them as User Inputs if you wanted.
- Create as regular variables
- vraUser
- vraFQDN
- vcf-vc
- vcf-username
- Create as secrets
- vraUserPassword
- vcf-password
Create a custom integration script to add Storage Policy to your Supervisor Namespace
vRA currently does not support this configuration item as part of the Cloud Assembly template, so we will use a Code Stream custom integration to run a python script to do this for us. You can grab the code from here. (Backup link)
- Go to Custom Integrations > New Custom Integration
- Set the Name as “CI-Script-Add-Storage-Super-Namespace”
- This will mean the pipelines pick everything up fully.
- Runtime as “Python3” and click Create.
- Copy and paste in the code into the pane
- Click to create a version
- Set a version number and enable the Release Version slider
Import the Delete Deployment Pipeline to enable rollback capabilities
The first Code Stream pipeline to create (import) is for deleting a deployment. We will use this as a rollback option if our main pipeline fails, passing through the deployment name used to create the Supervisor Namespace. Therefore, if a task fails such as create a Tanzu Guest Cluster, the pipeline will clean up the namespace and delete any lingering objects.
- Go to Pipelines View > Click Import
- Apply the pipeline code from this file
- Change the Project name on line 2 to map to your correct project.
- Click import
- Enable the Pipeline
If you open the pipeline, you can see it’s a simple task using the “Type: VMware Cloud Template”, with the action to delete the deployment, with an input of the deployment name, which will be passed through from our main pipeline, if the rollback criteria are met (failed task within the stage).
Import the Pipeline to Tanzu create a supervisor namespace and guest cluster
Now we will import the main pipeline and the core of the functionality we are creating.
- Perform the same actions as the last pipeline import
- Using this file
Click to open the pipeline so we can make some changes for your environment.
Explanation of the Pipeline
When we open the pipeline, we’ll see the Model view, and all our Stages (Red Box) and Tasks (Purple Box). I’m going to high level cover off what each part of the pipeline does.
- Stage: Get vRA API
- This authenticates to our vRA cluster and gets a refresh token we can use in the later stages where we configure the Kubernetes Endpoints
- Stage: Create Supervisor Namespace
- Task: Create Supervisor Namespace from Blueprint
- Creates a deployment using the Cloud Template we configured earlier
- Runs our custom integration, using mainly the variables we created earlier.
- Uses a Storage policy called “Management Storage policy – Thin” which is a default policy. You can change this to another policy name or even have the user specify on deployment if you wish.
In the below screenshot, I’ve also highlighted when you can see a rollback is configured on the stage.
- Stage: Create Tanzu K8s Cluster
- Task: Create Login Script to Supervisor Cluster
- Inside of our container, we run a number of commands to create a login script to be used by Expect, where we can securely pass our password to login to the supervisor namespace.
- Task: Create Tanzu Cluster
- Login to the Supervisor namespace we created earlier.
- Create a YAML file with our Tanzu Guest Cluster deployment configuration
- Use kubectl to apply this configuration to our environment, which triggers the Tanzu supervisor namespace to build the cluster.
- Task: Checking for Cluster Completion
- A simple loop waiting for the Tanzu Guest Cluster status to change to “Running”.
- Stage: Configure Tanzu Cluster
- Task: Create Login Script to Tanzu Cluster
- Create a login script for use with expect to login to the context of our new Tanzu Guest Cluster
- Task: Create Cluster Service Account
- Creating a Service Account to be used by vRA for the Kubernetes Endpoints
- Task: Role Binding Service Account
- Create a role binding for the service account
- Stage: Integration-to-vRA
- Task: Collect-Cluster-Information
- Collect all the information we need to be able to configure the vRA Kubernetes endpoints
- Task: Create Code Stream K8s Endpoint
- API call to vRA to create the endpoint
- Task: Create Cloud Assembly K8s Endpoint
- API call to vRA to create the endpoint
Configure the Pipeline
We need to make a few changes to ensure that our pipeline can run successfully.
First, we need to set our docker host.
- Click the Workspace tab
- Select the appropriate docker host
For this pipeline, we will be using my[K8s-ci container image](https://hub.docker.com/r/saintdle/k8s-ci) which is hosted in DockerHub. If your Docker host does not have internet access, then pull the image and host centrally via an image registry, or locally on the docker host.
Alternatively, you can build your own image, you will need installed:
- Python3 with the relevant modules used in the custom integration code
- kubectl
- yq
My container was built upon one provided by Chris McClanahan. Click [here](https://github.com/mcclanc/CodeStream-K8s-Blog/blob/master/DockerFile/Dockerfile) to get the Dockerfile/code to build the container. You can find the original container on DockerHub [here](https://hub.docker.com/repository/docker/vmwarecmbu/k8s-ci-container).
On the Input tab, we have a few inputs configured, some which have defaults set already some which do not and will need to be provided by the user.
Each input has a clear description. The first two are used as conditional values determining if the tasks should run to create the Kubernetes Endpoints in Code Stream and Cloud Assembly
Under the Model view.
- Click Task: Create Supervisor Namespace from Blueprint
Ensure the correct template is selected. For deployment name in vRA, you can change the inputs, as needed, but make sure it makes the rollback action input.
To check the rollback actions, click the Stage then the rollback tab.
- Click Task: Update Supervisor Namespace
- Check that the right Custom Integration task has been selected with the right input values provided. See the above note for the Storage Policy in the explanation of pipeline section.
- Click Task: Create Tanzu Cluster
- I haven’t set an input for the version of Kubernetes you can deploy, you can change this, or manually set it to a version available within WCP.
- You can also set the number of Control-Plane and Compute-Plane nodes as well.
- Click Task: Create Code Stream K8s Endpoint AND/OR Create Cloud Assembly K8s Endpoint
Here I just wanted to highlight where we use the Input value “Add_to_CS/CA” as a precondition to running this part of the pipeline.
- Save and then enable your pipeline
Running the Pipeline
- Run your Pipeline from the Pipeline screen or from within the Pipeline editing view itself.
- Provide values for the necessary inputs.
To view the execution for the pipeline, click within your Pipeline edit view (you’ll have a screen banner at the top), or click Executions on the left-hand navigation pane.
As you pipeline runs, you can refresh and see the progress. You can click each stage name and task name to find out more information, see the inputs passed through from each task for example, and the outputs as you can see below.
Eventually you should end up with a successful run below.
Creating a catalog item in Service Broker
Release the pipeline for use by Service Broker.
You will see the below.
Move into Service Broker. Create a content source of “type: Code Stream Pipelines”
- Set a name for the content source
- Select the project which your pipelines are attached
- Validate, you should see a green return status box
- Save and Import
Click the Content Sharing Tab on the left-hand navigation pane. Select your project the pipeline is attached to. Then click add items.
Select the Code Stream Pipelines and save.
Finally, if we click on the Catalog tab along the top row. We can see our catalog item, and the requested inputs when we request the item.
Wrap up and Resources
This was quite a lengthy blog post; however we used all the areas of vRealize Automation to its full potential, and hopefully along the way you also seen how you may implement other automation workflows using vRA as well.
This solution is a means to an end, to provide a catalog item which can be used by an end user to easily request and deploy a Tanzu Guest cluster inside of its own supervisor namespace. From here we could be more dynamic and add in the ability to select from either a new or existing supervisor namespace, change the YAML file inputs used for the cluster sizing, and even use Service Broker to set governance policies on controlling the requests from users.
Below are several resources I used to create this automation and content; you’ll see my Pipeline is very similar but not quite the same as the below. Each blog post including this one, does something slightly different, uses a different product or the outcome is different.
Thanks to the CMBU Tech Marketing team members at VMware as well who helped me along the way with this as well.
- vRealize Automation provides self-service Supervisor Namespaces
- vSphere with Kubernetes, vRealize Automation, and Tanzu…A Perfect Match!
- Deploying Tanzu Kubernetes Grid with vRealize Automation
Regards
The post Walk through – Using vRA to deploy vSphere with Tanzu Namespaces & Guest Clusters appeared first on vEducate.co.uk.
Top comments (0)