DEV Community

Cover image for KWOK - Kubernetes without Kubelet
Ashok Nagaraj
Ashok Nagaraj

Posted on

KWOK - Kubernetes without Kubelet

Premise

Kubernetes is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. While kubelet is an essential component of Kubernetes, it is not necessary for running Kubernetes. Kubelet is responsible for managing the state of each node in the Kubernetes cluster and ensuring that containers are running on the node as expected. However, there are alternative ways to manage Kubernetes clusters without kubelet, such as using external node agents or third-party tools. These tools can perform similar functions as kubelet, including managing node resources and monitoring container health. Therefore, while kubelet is an integral part of Kubernetes, it is not a requirement for running a functional and efficient Kubernetes cluster.

KWOK, which stands for Kubernetes Without Kubelet, is a new experimental project that aims to manage Kubernetes clusters in a different way. Normally, Kubernetes uses a component called kubelet to manage the state of each node in the cluster and ensure that containers are running correctly. However, KWOK explores the possibility of replacing kubelet with other tools that can do similar jobs, to make Kubernetes clusters more lightweight and customizable.

Even though the KWOK project is still in early development and not ready for production use, it has shown promising results. By using KWOK, users can have more control over how they manage their Kubernetes clusters and tailor them to their specific needs. KWOK also allows for more compatibility with other containerization platforms and unique environments.


Component hierarchy

                 +-----------------------+
                 | Kubernetes API Server |
                 +-----------------------+
                            |
                            v
                  +----------------+
                  | Etcd           |
                  +----------------+
                            |
                            v
   +-------------------------------------+
   | Kubernetes Controller Manager       |
   +-------------------------------------+
                            |
                            v
   +-------------------------------------+
   | Kubernetes Scheduler                |
   +-------------------------------------+
                            |
                            v
   +----------------------------------------+
   | External Node Agent (replacing kubelet)|
   +----------------------------------------+
                            |
                            v
   +-------------------------------------+
   |    Container Runtime (containerd)   |
   +-------------------------------------+

Enter fullscreen mode Exit fullscreen mode

Architecture


Need

Try KWOK if you want to (one or more of):

  • simulate kubernetes resources (1000s of nodes, pods ...)
  • create ultra-fast clusters for testing/simulation
  • verify manifests across kube-api server
  • play around with taints and tolerations without actual nodes
  • just get a feel of kubernetes

Install - https://kwok.sigs.k8s.io/docs/user/install/

# Install the main cli
❯ brew install kwok
==> Downloading https://formulae.brew.sh/api/formula.jws.json
...
❯ kwok --version
kwok version v0.1.1

# Install the command to control cluster (m1 mac in my case)
❯ wget https://github.com/kubernetes-sigs/kwok/releases/download/v0.1.1/kwok-darwin-arm64
❯ kwokctl --version
kwokctl version v0.1.1
Enter fullscreen mode Exit fullscreen mode

Cluster administration

To create clusters and builds you will need to install docker.

# assumption: container images are already pulled on the hosttime kwokctl create cluster --name=c-one
...
Status: Downloaded newer image for registry.k8s.io/kube-scheduler:v1.26.0
registry.k8s.io/kube-scheduler:v1.26.0
Starting cluster                                                                                                                      cluster="c-one"
[+] Running 6/6
 ⠿ Network kwok-c-one                            Created                                                                                         0.0s
 ⠿ Container kwok-c-one-etcd                     Started                                                                                         0.5s
 ⠿ Container kwok-c-one-kube-apiserver           Started                                                                                         0.7s
 ⠿ Container kwok-c-one-kwok-controller          Started                                                                                         1.2s
 ⠿ Container kwok-c-one-kube-controller-manager  Started                                                                                         1.2s
 ⠿ Container kwok-c-one-kube-scheduler           Started                                                                                         1.0s
Cluster is created                                                                                                         cluster="c-one" elapsed=1s
You can now use your cluster with:

    kubectl config use-context kwok-c-one

Thanks for using kwok!
kwokctl create cluster --name=c-one  1.25s user 0.52s system 32% cpu 5.397 total

# images pulled
❯ docker image ls | grep registry.k8s
registry.k8s.io/kwok/kwok                                          v0.1.1    b4aa6a246bed   3 weeks ago     61.6MB
registry.k8s.io/kube-apiserver                                     v1.26.0   646e01ef5206   4 months ago    129MB
registry.k8s.io/kube-scheduler                                     v1.26.0   1c1f5ebdeb5a   4 months ago    54.8MB
registry.k8s.io/kube-controller-manager                            v1.26.0   8c4e5a3e4d95   4 months ago    120MB
registry.k8s.io/etcd                                               3.5.6-0   ef2458028240   4 months ago    181MB

# list clusters
❯ kwokctl get clusters
c-one

# get kubeconfig
❯ kwokctl get kubeconfig --name c-one > kubeconfig

# Delete cluster
❯ kwokctl delete cluster --name=c-one
Stopping cluster                                                                                                                      cluster="c-one"
[+] Running 6/6
 ⠿ Container kwok-c-one-kwok-controller          Removed                                                                                         0.2s
 ⠿ Container kwok-c-one-kube-controller-manager  Removed                                                                                         0.2s
 ⠿ Container kwok-c-one-kube-scheduler           Removed                                                                                         0.2s
 ⠿ Container kwok-c-one-kube-apiserver           Removed                                                                                         1.3s
 ⠿ Container kwok-c-one-etcd                     Removed                                                                                         0.2s
 ⠿ Network kwok-c-one                            Removed                                                                                         0.1s
Deleting cluster                                                                                                                      cluster="c-one"
Cluster deleted
Enter fullscreen mode Exit fullscreen mode

Using the cluster

❯ kubectl config use-context kwok-c-one
Switched to context "kwok-c-one".

# create nodescat /tmp/node.yaml
apiVersion: v1
kind: Node
metadata:
  annotations:
    kwok.x-k8s.io/node: fake
  name: node-02
spec:
  taints: []
❯ k apply -f /tmp/node.yaml
node/node-01 created
❯ k get nodes
NAME      STATUS   ROLES    AGE   VERSION
node-01   Ready    <none>   32s   fake

# run a pod
❯ k run nginx-pod --image=nginx
pod/nginx-pod created
❯ k get pods
NAME        READY   STATUS    RESTARTS   AGE
nginx-pod   1/1     Running   0          4s

# create a service
❯ k expose pod nginx-pod --port 8080 --target-port 80
service/nginx-pod exposed
❯ k get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP    40m
nginx-pod    ClusterIP   10.0.0.20    <none>        8080/TCP   5s

# access a service - is still TODO
❯ k port-forward svc/nginx-pod --address=0.0.0.0 8080:8080
Forwarding from 0.0.0.0:8080 -> 80
Handling connection for 8080
# from another terminal
❯ http localhost:8080
http: error: ConnectionError: ('Connection aborted.', BadStatusLine('TODO: PortForward("default/nginx-pod", \'P\')')) while doing a GET request to URL: http://localhost:8080/

❯ k delete pod nginx-pod
pod "nginx-pod" deleted
Enter fullscreen mode Exit fullscreen mode

Conclusion

Very interesting concept, instantaneous turn-around + some added goodies like monitoring, audits and admission control. It is not yet fit for end-user use, kind is preferred for now


Further reading

Top comments (3)

Collapse
 
davidv profile image
Davide

Hi Ashok, thanks for sharing this!
I'm doing some experiments using KWOK and it seems easy to create nodes and pods with it. After creating them, how is it possible to use these fake nodes? Can I assign them as real ones?

Collapse
 
ashokan profile image
Ashok Nagaraj

Probably not for actual workloads when I checked; it was more to simulate nodes and wiring. Do check on the latest in the project page

Collapse
 
dhruvgarg79 profile image
Dhruv garg

can we actually run application on these fake nodes for testing on local?