k3s is a lightweight Kubernetes distribution developed by Rancher which is suitable for edge, IoT, and resource-constrained computing environments.
Helm is a package manager for Kubernetes which enables users to find, share, and use software built for Kubernetes.
Traditionally, Helm is comprised of a client (helm
) and server-side component, Tiller, which manages the deployment and lifecycle of Helm charts. In k3s, Tiller is replaced by Helm Controller. Helm Controller defines a new HelmChart
custom resource definition, or CRD, for managing Helm charts.
You can verify the HelmChart
resource is present on a k3s cluster by using kubectl
to list the available API resources.
$ sudo k3s kubectl api-resources --api-group=helm.cattle.io
NAME SHORTNAMES APIGROUP NAMESPACED KIND
helmcharts helm.cattle.io true HelmChart
The HelmChart resource
Below is an example of a HelmChart
resource definition.
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata: # Kubernetes object metadata
name: # Chart name (required)
namespace: kube-system # Namespace Helm Controller is monitoring to deploy charts (required)
spec: # resource specification
The apiVersion
and kind
fields together identify this as a HelmChart resource to be managed by Helm Controller.
The name
field under metadata
specifies a name for the HelmChart resource. namespace
specifies the Kubernetes namespace in which Helm Controller is monitoring CRDs. For k3s, this must be kube-system
as Helm Controller is only configured to watch this namespace for new HelmChart resources.
Lastly, the spec
, or specification, field is used to specify the desired state of the HelmChart resource.
HelmChart resource spec
The HelmChart resource supports several configuration parameters. These parameters correspond to helm install
command line arguments and are passed to Helm during chart deployment.
-
chart
(string: null) - Chart reference or a URL. -
targetNamespace
(string: null) - Namespace to install the Chart into. Defaults to the namespace used by the k3s controller (i.e., kube-system) if omitted. -
version
(string: null) The exact Chart version to install. If this is not specified, the latest version is installed. -
repo
(string: null) - Chart repository URL where to locate the requested Chart. -
set
(string: null) - A dictionary/map consisting of configuration values to provide to Helm. Keys should be specified as strings. Values may be either Integers or Strings. If more complex values are required, usevaluesContent
. -
valuesContent
(string: null) - A list of configuration values to provide to Helm. This should be a multi-line string.
Below is an example of a HelmChart resource definition with all available configuration values defined.
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
name: example-helmchart
namespace: kube-system
spec:
chart: test-helmchart
targetNamespace: test-namespace
version: 0.0.1
repo: https://example.com/helm-charts
set:
key1: "value1"
key2: "value2"
valuesContent: |-
config:
application:
admin_username: admin
admin_password: insecure_password
server:
port: 8080
tls_enabled: false
The corresponding CLI arguments for this would be:
$ helm install test-helmchart --namespace test-namespace \
--version 0.0.1 \
--repo https://example.com/helm-charts \
--set-string key1=value1 \
--set-string key2=value2 \
--values values.yaml
Where values.yaml contains the content present under the valuesContent
field.
Note: Both set
and valuesContent
may be used to configure the chart. set
only supports strings or integers as values. If other data types are needed, use valuesContent
.
Creating a HelmChart
We will now walk through the process of creating a HelmChart resource definition to deploy Node-RED β a flow-based programming for the Internet of Things.
The following steps were performed on a Raspberry Pi 4 running the Raspian operating system. You may need to modify the commands if using a different environment.
First, create a file named node-red-helmchart.yaml
using your default editor.
$ editor node-red-helmchart.yaml
Copy and paste the following block of text into the file.
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
name: node-red
namespace: kube-system
spec:
chart: stable/node-red
targetNamespace: node-red
set:
image.tag: rpi
valuesContent: |-
ingress:
enabled: "true"
annotations:
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip
hosts:
- raspberrypi1.local
path: /node-red/
Note: Be sure to modify the ingress.hosts
field to contain the hostname and/or IP address of one or more node in your Kubernetes cluster.
The deployment of the chart has been customized by modifying several of the available configuration values. Particularly, Ingress support has been enabled which exposes Node-RED to hosts outside of the Kubernetes cluster at http://raspberrypi1.local/node-red/.
Installing a HelmChart
There are two methods to install a HelmChart in k3s.
- Install the resource definition using
kubectl apply
. - Copy the resource definition to the server's manifest directory at
/var/lib/rancher/k3s/server/manifests/
.
Installing via kubectl
Instruct Kubernetes to install the specified resource definition using kubectl apply
.
$ sudo k3s kubectl apply --filename node-red-helmchart.yaml
helmchart.helm.cattle.io/node-red created
This is the standard method of provisioning resources within Kubernetes.
Installing using Auto-Deploying Manifests
k3s includes a feature called Auto-Deploying Manifests which allows for the automated deployment of Helm charts (and Kubernetes manifests) placed in k3s manifest directory (/var/lib/rancher/k3s/server/manifests
). Helm Controller continually watches this directory for new CRDs. When it detects a new resource has been added, it will automatically deploy the chart. Therefore, you can initiate the installation by simply copying node-red-helmchart.yaml
into the server's manifest directory.
$ sudo cp node-red-helmchart.yaml /var/lib/rancher/k3s/server/manifests/node-red-helmchart.yaml
Installation process under the hood
Regardless of the installation method used, when a chart is deployed Helm Controller creates a Kubernetes Job which installs the chart into the cluster using helm install
.
Execute kubectl get jobs
in the kube-system namespace and search for Jobs associated with the deployed chart.
$ sudo k3s kubectl --namespace kube-system get jobs \
--selector helmcharts.helm.cattle.io/chart=node-red
NAME COMPLETIONS DURATION AGE
helm-install-node-red 0/1 21s 23s
You can view the command the Job is executing by re-running kubectl get jobs
, and appending additional arguments to select and display Job-specific information using kubectl's custom-column output.
$ export CUSTOM_COLUMN_OUTPUT="COMMAND:.spec.template.spec.containers[0].name,INSTALL_ARGS:.spec.template.spec.containers[0].args"
$ sudo k3s kubectl --namespace kube-system get job helm-install-node-red \
--output=custom-columns=$CUSTOM_COLUMN_OUTPUT
COMMAND INSTALL_ARGS
helm [install --name node-red stable/node-red --namespace node-red --set-string image.tag=rpi]
The Job installs the chart by executing helm install
with the list of installation arguments provided in the HelmChart resource, as noted earlier.
Verifying the chart installation
Verify the chart has been successfully deployed by ensuring the associated resources have been created within the node-red namespace.
$ sudo kubectl --namespace node-red get ingress,service,pods
NAME HOSTS ADDRESS PORTS AGE
ingress.extensions/node-red raspberrypi1.local 10.0.0.133 80 2m2s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/node-red ClusterIP 10.43.135.177 <none> 1880/TCP 2m2s
NAME READY STATUS RESTARTS AGE
pod/node-red-7c5c564cb6-sz8b8 1/1 Running 0 2m1s
Updating a HelmChart
The configuration for a chart may be updated using kubectl apply
.
$ sudo k3s kubectl apply --filename node-red-helmchart.yaml
Deleting a HelmChart
To delete a Helm Chart, use kubectl delete helmchart
.
$ sudo k3s kubectl --namespace kube-system delete helmchart node-red
Conclusion
You have learned about the HelmChart custom resource available in K3s, how to construct a HelmChart resource definition, install it on the cluster, and perform post-deployment operations (update/delete).
That's it for this walkthrough.
I have a written program, k3s-helmchart-generate
, which simplifies the process of building HelmChart manifests. You can find a link to the repo on GitHub below.
Top comments (1)
Great article. I'm curious if you had to pre-create the 'node-red' namespace for this to work. I'm trying to deploy cert-manager in the same way but it fails due to the name space not already existing.