DEV Community

Kaito Ii
Kaito Ii

Posted on

WASM workload on AKS

#azure #webassembly #kubernetes #todayilearned

Public preview of AKS supporting WASI (WebAssembly System Interface) workload in Kubernetes is out. And I thought I would try it out.

https://azure.microsoft.com/en-us/updates/public-preview-aks-support-for-webassembly-system-interface-wasi-workloads/

TOC

Register WasmNodePoolPreview feature

Register WasmNodePoolPreview feature flag to your subscription

$ az feature register --namespace "Microsoft.ContainerService" --name "WasmNodePoolPreview"
{
  "id": "/subscriptions/<YOUR SUBSCRIPTION ID>/providers/Microsoft.Features/providers/Microsoft.ContainerService/features/WasmNodePoolPreview",
  "name": "Microsoft.ContainerService/WasmNodePoolPreview",
  "properties": {
    "state": "Registering"
  },
  "type": "Microsoft.Features/providers/features"
}

$ az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/WasmNodePoolPreview')].{Name:name,State:properties.state}"
Name                                            State
----------------------------------------------  -----------
Microsoft.ContainerService/WasmNodePoolPreview  Registering

$ az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/WasmNodePoolPreview')].{Name:name,State:properties.state}"
Name                                            State
----------------------------------------------  ----------
Microsoft.ContainerService/WasmNodePoolPreview  Registered
Enter fullscreen mode Exit fullscreen mode

Install aks-preview extension 

$ az extension add --name aks-preview
$ az extension update --name aks-preview
Enter fullscreen mode Exit fullscreen mode

Add WASM/WASI node pool to AKS cluster 

# Create resource group
$ az group create --name wasmRG -l westus
# Create AKS cluster
$ az aks create --resource-group wasmRG --name myAKSCluster
# Add WASM/WASI node pool
$ az aks nodepool add --resource-group wasmRG --cluster-name myAKSCluster --name mywasipool --node-count 1 --workload-runtime wasmwasi
# Check the workloadRuntime
$ az aks nodepool show -g wasmRG --cluster-name myAKSCluster -n mywasipool | jq '.workloadRuntime'
"WasmWasi"
Enter fullscreen mode Exit fullscreen mode 
# Get credential for AKS cluster
$ az aks get-credentials -n myakscluster -g wasmRG

$ kubectl get nodes
NAME                                 STATUS   ROLES   AGE   VERSION
aks-mywasipool-11259355-vmss000000   Ready    agent   76m   1.0.0-alpha.1
aks-nodepool1-11259355-vmss000000    Ready    agent   80m   v1.20.9
aks-nodepool1-11259355-vmss000001    Ready    agent   80m   v1.20.9
aks-nodepool1-11259355-vmss000002    Ready    agent   80m   v1.20.9

# Get labels on node
$ kubectl get node aks-mywasipool-11259355-vmss000000 -o jsonpath='{.metadata.labels}' | jq
{
  "agentpool": "mywasipool",
  "beta.kubernetes.io/arch": "wasm32-wagi",
  "beta.kubernetes.io/os": "wasm32-wagi",
  "kubernetes.azure.com/agentpool": "mywasipool",
  "kubernetes.azure.com/cluster": "MC_wasmRG_myAKSCluster_westus",
  "kubernetes.azure.com/mode": "user",
  "kubernetes.azure.com/node-image-version": "AKSUbuntu-1804gen2containerd-2021.10.02",
  "kubernetes.azure.com/os-sku": "Ubuntu",
  "kubernetes.azure.com/role": "agent",
  "kubernetes.azure.com/storageprofile": "managed",
  "kubernetes.azure.com/storagetier": "Premium_LRS",
  "kubernetes.io/arch": "wasm32-wagi",
  "kubernetes.io/hostname": "aks-mywasipool-11259355-vmss000000",
  "kubernetes.io/os": "wasm32-wagi",
  "kubernetes.io/role": "agent",
  "node-role.kubernetes.io/agent": "",
  "storageprofile": "managed",
  "storagetier": "Premium_LRS",
  "type": "krustlet"
}

# Get Taints defined in node
$ kubectl get node aks-mywasipool-11259355-vmss000000 -o jsonpath='{.spec.taints}' | jq
[
  {
    "effect": "NoSchedule",
    "key": "kubernetes.io/arch",
    "value": "wasm32-wagi"
  },
  {
    "effect": "NoExecute",
    "key": "kubernetes.io/arch",
    "value": "wasm32-wagi"
  }
]
Enter fullscreen mode Exit fullscreen mode
  • label "type": "krustlet" shows that the node runs krustlet instead of kubelet
  • label "kubernetes.io/arch": "wasm32-wagi" shows the node architecture
  • taint on the wasi node pool will only schedule/execute pods with toleration of wasm32-wagi

Run WASM/WASI Workload

  • Create the following yaml as wasi-example.yaml
    • nodeSelector will choose the node with certain label (i.e. node with kubernetes.io/arch: wasm32-wagi)
    • toleration will tolerate the taint in the node (i.e. wasm32-wagi NoSchedule and NoExecute) 
apiVersion: v1
kind: Pod
metadata:
  name: krustlet-wagi-demo
  labels:
    app: krustlet-wagi-demo
  annotations:
    alpha.wagi.krustlet.dev/default-host: "0.0.0.0:3001"
    alpha.wagi.krustlet.dev/modules: |
      {
        "krustlet-wagi-demo-http-example": {"route": "/http-example", "allowed_hosts": ["https://api.brigade.sh"]},
        "krustlet-wagi-demo-hello": {"route": "/hello/..."},
        "krustlet-wagi-demo-error": {"route": "/error"},
        "krustlet-wagi-demo-log": {"route": "/log"},
        "krustlet-wagi-demo-index": {"route": "/"}
      }
spec:
  hostNetwork: true
  nodeSelector:
    kubernetes.io/arch: wasm32-wagi
  containers:
    - image: webassembly.azurecr.io/krustlet-wagi-demo-http-example:v1.0.0
      imagePullPolicy: Always
      name: krustlet-wagi-demo-http-example
    - image: webassembly.azurecr.io/krustlet-wagi-demo-hello:v1.0.0
      imagePullPolicy: Always
      name: krustlet-wagi-demo-hello
    - image: webassembly.azurecr.io/krustlet-wagi-demo-index:v1.0.0
      imagePullPolicy: Always
      name: krustlet-wagi-demo-index
    - image: webassembly.azurecr.io/krustlet-wagi-demo-error:v1.0.0
      imagePullPolicy: Always
      name: krustlet-wagi-demo-error
    - image: webassembly.azurecr.io/krustlet-wagi-demo-log:v1.0.0
      imagePullPolicy: Always
      name: krustlet-wagi-demo-log
  tolerations:
    - key: "node.kubernetes.io/network-unavailable"
      operator: "Exists"
      effect: "NoSchedule"
    - key: "kubernetes.io/arch"
      operator: "Equal"
      value: "wasm32-wagi"
      effect: "NoExecute"
    - key: "kubernetes.io/arch"
      operator: "Equal"
      value: "wasm32-wagi"
      effect: "NoSchedule"
Enter fullscreen mode Exit fullscreen mode 
$ kubectl apply -f wasi-example.yaml

$ kubectl get po -o wide
NAME                 READY   STATUS       RESTARTS   AGE   IP       NODE                                 NOMINATED NODE   READINESS GATES
krustlet-wagi-demo   0/5     Registered   0          78s   <none>   aks-mywasipool-11259355-vmss000000   <none>           <none>
Enter fullscreen mode Exit fullscreen mode
  • We can see that the pods has been scheduled on wasipool node 
# Get Internal IP for WASI Node
$ WASINODE_IP=$(kubectl get nodes aks-mywasipool-11259355-vmss000000 -o jsonpath='{.status.addresses[?(@.type=="InternalIP")].address}')

# Create values.yaml for nginx chart
$ cat << EOF > values.yaml
serverBlock: |-
  server {
    listen 0.0.0.0:8080;
    location / {
            proxy_pass http://$WASINODE_IP:3001;
    }
  }
EOF

$ helm repo add bitnami https://charts.bitnami.com/bitnami
$ helm repo update
$ helm install hello-wasi bitnami/nginx -f values.yaml

$ kubectl get service 
NAME               TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
hello-wasi-nginx   LoadBalancer   10.0.163.230   40.83.129.0   80:32513/TCP   3m11s
kubernetes         ClusterIP      10.0.0.1       <none>        443/TCP        152m

# Get the external ip for the hello-wasi-nginx service
$ EXTERNAL_IP=$(kubectl get service hello-wasi-nginx -o jsonpath='{.status.loadBalancer.ingress[].ip}')

$ curl $EXTERNAL_IP/hello
hello world
Enter fullscreen mode Exit fullscreen mode

Create my own WASM workload with Rust

  • Prerequisite: Rust, wasmtime
  • Set up the Rust environment 
$ mkdir aks-wasm; cd aks-wasm
$ cargo init 
$ rustup target add wasm32-wasi
Enter fullscreen mode Exit fullscreen mode
  • Create a simple app for wasm (src/main.rs) 
fn main() {
    println!("Content-Type: text/plain\n");
    println!("Hello from wasm!");
}
Enter fullscreen mode Exit fullscreen mode
  • Build the app and run it locally 
$ cargo build --release --target wasm32-wasi
$ wasmtime target/wasm32-wasi/release/aks-wasm.wasm
Content-Type: text/plain

Hello from wasm!
Enter fullscreen mode Exit fullscreen mode

Upload the WASM module to Azure Container Registry

  • Prerequisite: wasm-to-oci
  • Setup Azure ContainerRegistry 
# You may have to choose a different name for the registry 
$ az acr create -n MyWASMRegistry -g wasmRG --sku Standard

# login to Azure Container Registry. Make sure Docker is running
$ az acr login -n MyWASMRegistry

# Enable anonymous pull to allow pulling the WASM module from WASI node
$ az acr update --name myregistry --anonymous-pull-enabled
Enter fullscreen mode Exit fullscreen mode
  • Push the WASM module to Azure Container Registry 

# Push the WASM module to Azure Container Registry
$ wasm-to-oci push target/wasm32-wasi/release/aks-wasm.wasm mywasmregistry.azurecr.io/mywasm:1.0.0 

$ az acr repository list -n MyWASMRegistry
[
  "mywasm"
]

# Make sure the WASM module has been uploaded
$ az acr repository show -n MyWASMRegistry --image mywasm:1.0.0                                           [18:27:29]
{
  "changeableAttributes": {
    "deleteEnabled": true,
    "listEnabled": true,
    "readEnabled": true,
    "writeEnabled": true
  },
  "createdTime": "2021-10-21T09:21:15.8964565Z",
  "digest": "sha256:3cade5d310ece611b4d87df55c64e4e6d99ae08d052cfaef2e699df56db90fef",
  "lastUpdateTime": "2021-10-21T09:21:15.8964565Z",
  "name": "1.0.0",
  "signed": false
}
Enter fullscreen mode Exit fullscreen mode

Deploy my WASM module

  • Update wasi-example.yaml by adding the wasm module we created earlier 
apiVersion: v1
kind: Pod
metadata:
  name: krustlet-wagi-demo
  labels:
    app: krustlet-wagi-demo
  annotations:
    alpha.wagi.krustlet.dev/default-host: "0.0.0.0:3001"
    alpha.wagi.krustlet.dev/modules: |
      {
        "krustlet-wagi-demo-http-example": {"route": "/http-example", "allowed_hosts": ["https://api.brigade.sh"]},
        "krustlet-wagi-demo-hello": {"route": "/hello/..."},
        "krustlet-wagi-demo-error": {"route": "/error"},
        "krustlet-wagi-demo-log": {"route": "/log"},
        "krustlet-wagi-demo-index": {"route": "/"},
        "krustlet-wagi-mywasm": {"route": "/mywasm"}
      }
spec:
  hostNetwork: true
  nodeSelector:
    kubernetes.io/arch: wasm32-wagi
  containers:
    - image: webassembly.azurecr.io/krustlet-wagi-demo-http-example:v1.0.0
      imagePullPolicy: Always
      name: krustlet-wagi-demo-http-example
    - image: webassembly.azurecr.io/krustlet-wagi-demo-hello:v1.0.0
      imagePullPolicy: Always
      name: krustlet-wagi-demo-hello
    - image: webassembly.azurecr.io/krustlet-wagi-demo-index:v1.0.0
      imagePullPolicy: Always
      name: krustlet-wagi-demo-index
    - image: webassembly.azurecr.io/krustlet-wagi-demo-error:v1.0.0
      imagePullPolicy: Always
      name: krustlet-wagi-demo-error
    - image: webassembly.azurecr.io/krustlet-wagi-demo-log:v1.0.0
      imagePullPolicy: Always
      name: krustlet-wagi-demo-log
    - image: mywasmregistry.azurecr.io/mywasm:1.0.0
      imagePullPolicy: Always
      name: krustlet-wagi-mywasm
  tolerations:
    - key: "node.kubernetes.io/network-unavailable"
      operator: "Exists"
      effect: "NoSchedule"
    - key: "kubernetes.io/arch"
      operator: "Equal"
      value: "wasm32-wagi"
      effect: "NoExecute"
    - key: "kubernetes.io/arch"
      operator: "Equal"
      value: "wasm32-wagi"
      effect: "NoSchedule"
Enter fullscreen mode Exit fullscreen mode
  • Restart the AKS cluster since the WASM module does not get updated for some reason. 
# Delete current WASM workload
$ kubectl delete -f wasi-example.yaml

# Restart AKS cluster
$ az aks stop -n myakscluster -g wasmRG && az aks start -n myakscluster -g wasmRG

# Redeploy WASM workload
$ kubectl applt -f wasi-example.yaml

$ EXTERNAL_IP=$(kubectl get service hello-wasi-nginx -o jsonpath='{.status.loadBalancer.ingress[].ip}')

$ curl $EXTERNAL_IP/mywasm
Hello from wasm!
Enter fullscreen mode Exit fullscreen mode

Clean up 

$ helm delete hello-wasi
$ kubectl delete -f wasi-example.yaml
$ az aks nodepool delete --name mywasipool -g wasmRG --cluster-name myakscluster
Enter fullscreen mode Exit fullscreen mode

Reference

Top comments (0)

Read next

beppe90 profile image

The Home Server Journey - 4: Enter The Matrix

Beppe -

carrie_luo1 profile image

How to Run Open Source WAF SafeLine on K8s

Carrie -

jakepage91 profile image

Glasskube Launch week #2: The wrap up

Jake Page -

willvelida profile image

Deploying to Azure with Terraform and GitHub Actions

Will Velida -