Optimizing Kubernetes Resource Requests with Goldilocks : Day 25 of 50 days DevOps Tools Series

Welcome to Day 25 of our "50 DevOps Tools in 50 Days" series! Today, we will delve into Goldilocks, a lesser-known but incredibly useful tool for Kubernetes that helps you optimize your resource requests and limits. Properly setting these values is crucial for ensuring that your applications run efficiently and cost-effectively in your Kubernetes cluster.

What is Goldilocks?

Goldilocks is an open-source utility that runs as a controller in your Kubernetes cluster and provides recommendations for setting resource requests and limits on your deployments. By analyzing historical data collected by the Vertical Pod Autoscaler (VPA), Goldilocks helps you find the "just right" resource settings for your workloads, ensuring they are neither over-allocated nor under-allocated.

Why Use Goldilocks?

Setting appropriate resource requests and limits is critical for several reasons:

Efficiency: Prevents over-provisioning of resources, which can save costs and reduce wastage.
Stability: Avoids under-provisioning, which can lead to application instability and performance issues.
Scalability: Helps your applications scale more effectively by ensuring they have the right amount of resources.
Cost Optimization: By fine-tuning resource requests and limits, you can optimize your cloud spending.

Installing Goldilocks

Goldilocks can be installed using Helm, a package manager for Kubernetes. Below are the steps to install Goldilocks:

Add the Fairwinds Helm Repository:

helm repo add fairwinds-stable https://charts.fairwinds.com/stable
helm repo update

Install Goldilocks:

kubectl create ns goldilocks
helm install goldilocks fairwinds-stable/goldilocks --namespace goldilocks

Enable the Goldilocks Dashboard (optional):

kubectl port-forward --namespace goldilocks svc/goldilocks-dashboard 8080:80

Configuring Goldilocks

Goldilocks uses the Vertical Pod Autoscaler (VPA) to gather historical resource usage data. Ensure the VPA is installed in your cluster:

kubectl apply -f https://github.com/kubernetes/autoscaler/releases/download/vertical-pod-autoscaler-0.9.2/vpa-v1-crd.yaml
kubectl apply -f https://github.com/kubernetes/autoscaler/releases/download/vertical-pod-autoscaler-0.9.2/vpa-rbac.yaml
kubectl apply -f https://github.com/kubernetes/autoscaler/releases/download/vertical-pod-autoscaler-0.9.2/vpa-deployment.yaml

Using Goldilocks

After installation, you need to annotate the namespaces you want Goldilocks to monitor. Here’s an example:

kubectl annotate namespace <namespace> goldilocks.fairwinds.com/enabled=true

Replace with the name of your namespace. Goldilocks will start collecting data and provide recommendations for the workloads in the annotated namespaces.

Accessing Goldilocks Dashboard

If you enabled the dashboard, you can access it by navigating to http://localhost:8080 in your web browser after running the port-forward command mentioned earlier. The dashboard provides a user-friendly interface to view the recommendations.

Real-Life Example

Let's consider a scenario where you have a namespace called production with several deployments running your web application. You want to ensure these deployments have optimal resource settings.

Annotate the Namespace:

kubectl annotate namespace production goldilocks.fairwinds.com/enabled=true

Wait for Data Collection: Allow Goldilocks to collect historical data. This might take a few hours to a few days, depending on your application's activity.

View Recommendations: Access the Goldilocks dashboard or use kubectl to get recommendations.

kubectl get vpaconfiguration -n goldilocks

Apply Recommendations: Based on the recommendations, update your deployment manifests. For example, if Goldilocks suggests increasing CPU requests and limits for a deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
  namespace: production
spec:
  containers:
    - name: web-container
      image: my-web-app:latest
      resources:
        requests:
          cpu: "500m"
          memory: "256Mi"
        limits:
          cpu: "1000m"
          memory: "512Mi"

Benefits of Using Goldilocks

Cost Savings: By optimizing resource requests and limits, you can reduce unnecessary spending on cloud resources.
Improved Performance: Ensures your applications have the right amount of resources, leading to better performance and stability.
Ease of Use: The dashboard and automated recommendations make it easy to fine-tune your workloads without deep manual analysis.
Scalability: Helps you plan for scalability by providing insights into resource usage trends.

Limitations of Goldilocks

Data Collection Time: Requires historical data to provide accurate recommendations, so it may take some time to gather enough information.
Dependency on VPA: Relies on the Vertical Pod Autoscaler, which must be correctly configured and running in your cluster.
Not a Silver Bullet: While Goldilocks provides valuable recommendations, it's important to review and test them before applying changes in production.

Conclusion

Goldilocks is an invaluable tool for DevOps engineers looking to optimize their Kubernetes resource requests and limits. By providing clear recommendations based on historical data, Goldilocks helps you ensure your applications run efficiently and cost-effectively. Incorporating Goldilocks into your Kubernetes management practices can lead to significant improvements in performance, stability, and cost savings.

Stay tuned for more exciting tools in our "50 DevOps Tools in 50 Days" series. This was the last post related of this series related to the Kubernetes. From tomorrow, we'll explore additional tools that will enhance our DevOps practices.

👉 Make sure to follow me on LinkedIn for the latest updates: Shiivam Agnihotri