AWS Network Load Balancer, Terraform, and Go, as a cloud network engineer.

As a network engineer, it's important to stay up-to-date with the latest technologies and tools in the industry. NLBs are a critical component in modern cloud networking infrastructures, allowing for efficient distribution of incoming traffic across multiple targets, improving application availability and scalability. As traditional network engineer its important we understand how these logical cloud constructs work and how they are built.

TLDR;

• You can pull down this project and build an internet facing network load balancer that serves content to two instances running a Go server. Once the infrastructure is stood up, start the client application with the DNS FQDN of your load balancer.

• .\windows_client.exe -remote_host golang-nlb-123456.elb.us-east-1.amazonaws.com -go_routines 1000

• ./linux_client -remote_host golang-nlb-123456.elb.us-east-1.amazonaws.com -go_routines 1000

• Check out CloudWatch to investigate EC2, ENI flow logs.

The desired outcomes from this project:

1. Utilize Go to create a simply client/server application.
2. Use terraform to stand up the infrastructure.
3. Put load on the NBL and servers.
4. Use CloudWatch to view incoming traffic.

Creating a Go client server application

• This was written in Go because of the popularity of the language. Its becoming more and more popular in cloud native application in addition to network automation projects (If you are a network engineer looking to get into automation python is probably the first language to pick up, if you're still interested in another language after that, Go is probably what you should study).
• The first step was to see if a simple Go server could be created that listens on port 80 and returns a basic message to the caller/consumer.
• After the server had been written a client needed to be written, in this case go routines were chosen as a way to do some load testing.(for those unfamiliar, a go routine is Go's way of doing concurrency, similar to "threads" or "threading". However they are more light weight than threads. They can also maintain state between each other which is not traditionally something threads do well.) By using go routines you are able to asynchronously hit the load balancer with many requests.
• Find the corresponding code here: Client code Server code

Creating the supporting infrastructure with IaC and Terraform

• Once the basic server was working it was time to start working on the actual infrastructure. The high level objective was a Network Load Balancer (NLB) that takes requests from the internet and distributes those requests across a few hosts. In order to accomplish the outcome a few things needed to be built:
• VPC
• Subnets in two availability zones ( to test cross zone load balancing ). In this situation the subnets are public (ie. the hosts have public and private IP addresses). But they very well could have been private subnets.
• Custom route table.
• Internet gateway ( without an IGW and a route to it, you cannot route traffic in or out of your VPC to the internet).
• Network load balancer and three additional pieces which all need to be tied together correct. The load balancer itself with a designation of "Internet facing". A listener must be created, the listening is what directs requests to the a specific "target group". Lastly the target group is created, and it actually holds the logical grouping of hosts who also should be listening on the same ports as the "listener".
• Security groups needed to be configured and associated to the correct VPC. In this solution, a generous number of services were allowed for troubleshooting and illustration purposes. Port 80 was exposed to test the actual application, port 22 and ICMP were allowed for troubleshooting purposes.
• Compute instances were deployed, one instance stood up in each subnet in each AZ to simulate a highly available workload.
• One final thing to note, the server code was deployed in a rather crude way. It works but it was just for testing so therefore no concern was given besides getting it to work. In this project the "user data" was updated to pull in the binary server go file and run it as root. This runs the web server as root and starts it when the instance starts up. You can see the user data HERE and the actual binary file that gets run HERE.
• Lastly VPC flow logs were created on the ENI's attached to the EC2 instances and pushed to a CloudWatch log group. IaC Terraform code

Put load on the NLB

• In this last phase you can actually test whether the load balancer distributes the load across our targets. By running the client go application. It accepts a remote host address and a number of go routines to instantiate. I wanted to test from my PC and incrementally increased the number of go routines and eventually tried 100,000, I wouldn't recommend 100,000 though you will DOS yourself 🤭. 
• You can run the client like so:

.\windows_client.exe -remote_host golang-nlb-b21d59f5a66c91f5.elb.us-east-1.amazonaws.com -go_routines 1000

Evaluate your flow logs stored in CloudWatch

In this last step, the goal was to get flow logs working and put into CloudWatch. If you spin this project up in your environment you'll see two log streams in the log group created. It takes a while to populate after receiving the client traffic, but it works. You'll notice the traffic from the client application is successfully hitting the instances.

Cheers!