I had a blast learning about Linkerd lately. It's a service mesh designed to help with traditional issues regarding service to service communication
Many times we would have to write a lot of code and operational overhead to achieve:
- SSL between microservices
- Auto retry for failed requests
- Metrics (Request per second & Latency)
- Tracing
- and More
A service mesh can help here by injecting a discrete proxy next to every application which hijacks requests coming in and out of that service.
I cover these challenges and service mesh basics in my Introduction to Service mesh video on YouTube.
Step 1 : We need a Kubernetes Cluster
Lets create a Kubernetes cluster to play with using kind
Kind is great for running throwaway test clusters :)
kind create cluster --name linkerd --image kindest/node:v1.19.1
Make sure you have kubectl
installed and its pointing to your cluster. You should be able to contact your cluster like so:
#test cluster access:
kubectl get nodes
NAME STATUS ROLES AGE VERSION
linkerd-control-plane Ready master 26m v1.19.1
Step 2: Example Microservice Architecture
I have a few components that makes up a video catalog application.
This architecture consists of :
- Ingress controller: A gateway for receiving traffic
- Videos Web : A web interface to show video catalog
- Playlists API: An API to retrieve lists of playlists
- Playlists DB: A small redis database to store playlists
- Videos API: An API to retrieve video data by ID
- Videos DB: A small redis database to store video data
Architecture:
servicemesh.demo/home +--------------+
+------------------------------> | videos-web |
| | |
servicemesh.demo/home +------+------------+ +--------------+
+------------------>+ingress-nginx |
|Ingress controller |
+------+------------+ +---------------+ +--------------+
| | playlists-api +--->+ playlists-db |
+------------------------------> | | | |
servicemesh.demo/api/playlists +-----+---------+ +--------------+
|
v
+-----+------+ +-----------+
| videos-api +------>+ videos-db |
| | | |
+------------+ +-----------+
To follow along with this tutorial, all the code is in my GitHub repo
Clone the repo, and lets deploy all the applications to our cluster:
# ingress controller
kubectl create ns ingress-nginx
kubectl apply -f kubernetes/servicemesh/applications/ingress-nginx/
# applications
kubectl apply -f kubernetes/servicemesh/applications/playlists-api/
kubectl apply -f kubernetes/servicemesh/applications/playlists-db/
kubectl apply -f kubernetes/servicemesh/applications/videos-api/
kubectl apply -f kubernetes/servicemesh/applications/videos-web/
kubectl apply -f kubernetes/servicemesh/applications/videos-db/
Step 3: Make sure all our apps are up and running
kubectl get pods
NAME READY STATUS RESTARTS AGE
playlists-api-d7f64c9c6-rfhdg 1/1 Running 0 2m19s
playlists-db-67d75dc7f4-p8wk5 1/1 Running 0 2m19s
videos-api-7769dfc56b-fsqsr 1/1 Running 0 2m18s
videos-db-74576d7c7d-5ljdh 1/1 Running 0 2m18s
videos-web-598c76f8f-chhgm 1/1 Running 0 100s
Let's double check the Ingress controller too :
kubectl -n ingress-nginx get pods
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-6fbb446cff-8fwxz 1/1 Running 0 2m38s
nginx-ingress-controller-6fbb446cff-zbw7x 1/1 Running 0 2m38s
Next up we'll need a fake DNS name servicemesh.demo
Let's fake one by adding the following entry in our hosts (C:\Windows\System32\drivers\etc\hosts
) file:
127.0.0.1 servicemesh.demo
Now we have a DNS servicemesh.demo
, lets connect to our Ingress controller using port-forward
kubectl -n ingress-nginx port-forward deploy/nginx-ingress-controller 80
Now we can access our site under http://servicemesh.demo/home/
here
Step 4: Installing LinkerD
Firstly, I like to do most of my work in containers so everything is reproducible and my machine remains clean. Let's get a small lightweight Alpine Linux container so we can get to work:
docker run -it --rm -v ${HOME}:/root/ -v ${PWD}:/work -w /work --net host alpine sh
# install curl & kubectl
apk add --no-cache curl nano
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
chmod +x ./kubectl
mv ./kubectl /usr/local/bin/kubectl
#test cluster access:
/work # kubectl get nodes
NAME STATUS ROLES AGE VERSION
linkerd-control-plane Ready master 26m v1.19.1
Now let's install the linkerd
CLI in our container:
I grabbed the edge-20.10.1
release using curl
curl -L -o linkerd https://github.com/linkerd/linkerd2/releases/download/edge-20.10.1/linkerd2-cli-edge-20.10.1-linux-amd64
chmod +x linkerd && mv ./linkerd /usr/local/bin/
linkerd --help
Let's do some preflight checks
Linkerd has a great capability to check compatibility with the target cluster
linkerd check --pre
Let's grab the YAML from the linkerd
CLI:
# Grab YAML:
linkerd install > ./kubernetes/servicemesh/linkerd/manifest/linkerd-edge-20.10.1.yaml
# Deploy it:
kubectl apply -f ./kubernetes/servicemesh/linkerd/manifest/linkerd-edge-20.10.1.yaml
Let's wait until all components are running
watch kubectl -n linkerd get pods
kubectl -n linkerd get svc
You can also do a final check:
linkerd check
Step 5: Access the LinkerD Dashboard
Let's access the linkerd
dashboard via port-forward
kubectl -n linkerd port-forward svc/linkerd-web 8084
Feel free to have a browse around to familiarise yourself with the dashboard features.
Step 6: Mesh our video catalog services
There are 2 ways to mesh services:
1) We can add an annotation to your deployment to persist the mesh if our YAML is part of a GitOps flow:
This is a more permanent solution:
template:
metadata:
annotations:
linkerd.io/inject: enabled
2) Or inject linkerd
on the fly:
This may only be temporary as your CI/CD system may roll out the previous YAML
kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
playlists-api 1/1 1 1 8h
playlists-db 1/1 1 1 8h
videos-api 1/1 1 1 8h
videos-db 1/1 1 1 8h
videos-web 1/1 1 1 8h
kubectl get deploy playlists-api -o yaml | linkerd inject - | kubectl apply -f -
kubectl get deploy playlists-db -o yaml | linkerd inject - | kubectl apply -f -
kubectl get deploy videos-api -o yaml | linkerd inject - | kubectl apply -f -
kubectl get deploy videos-db -o yaml | linkerd inject - | kubectl apply -f -
kubectl get deploy videos-web -o yaml | linkerd inject - | kubectl apply -f -
kubectl -n ingress-nginx get deploy nginx-ingress-controller -o yaml | linkerd inject - | kubectl apply -f -
Step 7: Generate some traffic to our services
Let's run a curl
loop to generate some traffic to our site
We'll make a call to /home/
and to simulate the browser making a call to get the playlists,
we'll make a follow up call to /api/playlists
While ($true) { curl -UseBasicParsing http://servicemesh.demo/home/;curl -UseBasicParsing http://servicemesh.demo/api/playlists; Start-Sleep -Seconds 1;}
Step 8: Failure and Observability
In my video, I inject a fault to my videos-api
using some faulty code that can be enabled by setting Environment variable FLAKY=true
on the videos-api
deploy.yaml file. I redeploy that service once the environment variable is set.
kubectl apply -f kubernetes/servicemesh/applications/videos-api/
We can observe cascading failures in the linkerd
dashboard once failures start to occur.
Step 9: Service Profiles & Auto Retry
Now that we have a failure, we can test out the auto retry feature by creating a service profile. A service profile can be generated using the linkerd
CLI or you can just apply one I created already:
kubectl apply -f kubernetes/servicemesh/linkerd/serviceprofiles/videos-api.yaml
We can notice retries are occuring when effective RPS is different to the actual RPS
linkerd routes -n default deploy/playlists-api --to svc/videos-api -o wide
We will also see the playlist API recover in the linkerd
dashboard.
Step 10: Mutual TLS
A cool feature of Linkerd is every meshed service gets mutual TLS which means all internal meshed service traffic is encrypted using TLS. We can view this by running :
linkerd -n default edges deployment
SRC DST SRC_NS DST_NS SECURED
playlists-api videos-api default default √
linkerd-prometheus playlists-api linkerd default √
linkerd-prometheus playlists-db linkerd default √
linkerd-prometheus videos-api linkerd default √
linkerd-prometheus videos-db linkerd default √
linkerd-prometheus videos-web linkerd default √
linkerd-tap playlists-api linkerd default √
linkerd-tap playlists-db linkerd default √
linkerd-tap videos-api linkerd default √
linkerd-tap videos-db linkerd default √
linkerd-tap videos-web linkerd default √
You can also tap
deployments to get a tcpdump
style view which will also show tls on traffic between services:
linkerd -n default tap deploy
Hope you found this guide useful!
Be sure to checkout my LinkerD episode on YT
Peace!
Top comments (0)