DEV Community

Cover image for Kubectl exec/port-forward with AWS ALB and nginx-ingress-controller
Mohamed Radwan for AWS Community Builders

Posted on • Updated on

Kubectl exec/port-forward with AWS ALB and nginx-ingress-controller

I was facing issues when I performed kubectl with exec or port-forward option on my Rancher clusters that used EKS and ALB, it was giving me this error

kubectl exec -it app -- bash

Error from server (BadRequest): Upgrade request required
Enter fullscreen mode Exit fullscreen mode

exec and port forward are using SPDY protocol and ALB does not support it.

The HTTPS request is going from the user to ALB, then SSL is terminated on the ALB, and the request is forwarded to the Nginx controller service after that forward to the rancher service.

See the part 1 to setup Rancher on EKS and ALB

After that, you need to do the following:

1- Install Nginx Ingress Controller

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
Enter fullscreen mode Exit fullscreen mode

Kubernetes v1.16+ use version 3.x.x

helm upgrade --install nginx ingress-nginx/ingress-nginx --namespace cattle-system --set controller.service.type=NodePort  --set controller.service.targetPorts.https=http --set-string controller.config.use-forwarded-headers="true" --version 3.12.0
Enter fullscreen mode Exit fullscreen mode

Kubernetes v1.19+ use version 4.x.x

helm upgrade --install nginx ingress-nginx/ingress-nginx --namespace cattle-system --set controller.service.type=NodePort  --set controller.service.targetPorts.https=http --set-string controller.config.use-forwarded-headers="true" --version 4.2.3
Enter fullscreen mode Exit fullscreen mode

2- Edit the rancher Ingress

kubectl edit ingress -n cattle-system rancher
Enter fullscreen mode Exit fullscreen mode

Change the host and name inside spec with the following:


spec:
   rules:
   - host: '*.example.com'
     http:
       paths:
       - backend:
           service:
             name: nginx-ingress-nginx-controller
             port:
               number: 80
         pathType: ImplementationSpecific
Enter fullscreen mode Exit fullscreen mode

3- Create a new ingress with the following:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
  name: rancher-exec
  namespace: cattle-system

spec:
   rules:
   - host: 'rancher.example.com'
     http:
       paths:
       - backend:
           service:
             name: rancher
             port:
               number: 80
         pathType: ImplementationSpecific
Enter fullscreen mode Exit fullscreen mode

You need to add ingressClassName: nginx only for rancher-exec

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
  name: rancher-exec
  namespace: cattle-system

spec:
   ingressClassName: nginx 
   rules:
   - host: 'rancher.example.com'
     http:
       paths:
       - backend:
           service:
             name: rancher
             port:
               number: 80
         pathType: ImplementationSpecific
Enter fullscreen mode Exit fullscreen mode

Note: It will remove the ALB if you add ingressClassName: nginx to rancher ingress

Test the kubectl exec and port-forward

kubectl create deployment nginx2 --image nginx:alpine
kubectl expose deployment nginx2 --port=80
kubectl exec -it nginx2-XXXXX -- sh
kubectl port-forward service/nginx2 --address 0.0.0.0 80:80
Enter fullscreen mode Exit fullscreen mode

Top comments (1)

Collapse
 
rez0k profile image
Rez0k • Edited

Great article!

I encountered another problem after implementing your solution, the terminal session in Rancher was closed after about 60 seconds.

The reason was that the nginx controller config was set to the default of 60 seconds.
to solve this add to the nginx helm command:

--set-string controller.config.proxy-connect-timeout: "1800" --set-string controller.config.proxy-read-timeout="1800"

This will increase the terminal session to half an hour (1800 seconds).