AWS Route Tables

Welcome to this guide on AWS Route Tables, in this blog post we will delve into the fundamentals of AWS route tables.

This article assumes basic knowledge of VPCs, AZ's, subnets and other basic network constructs. If unfamiliar it may be difficult to follow along with all the content.

Route Table basics

• Route tables have three different types of routes, local, static and propagated routes.
• Local routes are automatically instantiated with a VPC, static routes are configured by humans, and dynamic or propagated are added dynamically. Examples of propagated routes could come from VGW's(Virtual Private Gateway) (for example, routes learned from on-prem) or in the case of a transit gateway could come from the attachments hanging off of the transit gateway.
• Every VPC created is built with a "Main" route table. This is the default route table for a particular VPC and every subnet in the VPC will be associated with it upon initial creation.
• A VPC must have at least one mainroute table and up to 200 (at the time of writing) custom route tables per VPC.
• Cloud route tables can be likened to traditional route table, where all routes are in the default route table. Similarly one cloud route table may be used to do routing between every subnet in a VPC. Alternatively cloud route tables could be utilized like VRF's and each subnet could have its own individual route table and therefore its own completely isolated routes separate from a main route table.
• Subnets and route tables have a one to many relationship. One subnet can only be associated with one route table. But one route table can have many subnets associated with it.
• Every subnet MUST be associated with one route table, either custom or the main RT.
• If you disassociate the subnet from a custom route table it will automatically be associated with the main RT. You cannot actually disassociate a subnet from the main route table. Think of the the main RT as the catch all, if a subnet has no where to go the main route table will accept it.
• Regardless of the whether we're talking route tables on traditional routers or cloud routers, the concepts remain the same. A route table always analyzes the destination of the traffic if the destination is in the route table, it forwards the traffic to a target.

Cloud Route Table
| Destination | Target |
| ----------  | -------|
| 0.0.0.0/0 | igw-123 |
| 192.168.2.0/24 | 192.168.2.1 |
| 52.95.154.0/23 | vpce-123 |

• In the table above we see an example of a cloud route table with an array of different target types. An internet gateway as a default route out of the VPC, an actual IP address as a destination for traffic to the subnet 192.168.2.0/24, and lastly a VPC endpoint as a target for a public S3 endpoint vpce-123. Notice it differs from a traditional route table in that we can route to logical constructs like an IGW (Internet Gateway) or a VPC endpoint, not just physical end points and remote IP's.

Traditional Route Table
| Destination | Target |
| ----------  | -------|
| 192.168.0.0/24 | eth1-1 |
| 192.168.1.0/24 | 10.120.10.3/32 |

• In contrast with the cloud route table example above, here we see an example of a traditional route table for a physical router or switch. We see a physical interface as a target for outbound traffic to the 192.168.0.0/24 and an actual IP as the target for a destination of 192.168.1.0/24. This is what we are normally used to but cloud route tables offer more than we are used to as actual endpoints.

-- As mentioned, cloud route tables operate very similarly to traditional route tables. Its really just the type of targets that can be used to forward traffic to.

Ingress and Egress Route Tables

• Ingress and Egress Routing is an important topic and not something to be overlooked. Similar to traditional routing, when setting up routing or switching you can configure the network and your routing protocol or switching will handle ingress and egress without much intervention (obviously this is a gross over statement but true enough for the example). However when you need fine grain control over how traffic is traversing your network you need to do more specialized routing and switching. This too is something we need to considered in the cloud. Left at its defaults, you can probably fairly easily maintain connectivity between endpoints, but when you need to do anything beyond the basics you really need to understand both ingress and egress routing.

• With cloud routing we need to be more cognizant of the direction of our traffic and ensuring that both directions are accounted for. It would seem that cloud routing does take more consideration in determining the flow of traffic if you have specific routing requirements. Read on to find out more about ingress and egress routing...

• Route tables can be associated with more than just subnets. Route tables can also be associated with other logical constructs like IGW and VGW. This is great news for engineers because the route tables associated with subnets can only direct traffic in one direction (egress from the subnet). However route tables associated with internet gateways and virtual private gateways means we also control traffic in the opposite direction (ingress into our VPC).

Lets look at an example:

• In the first example we have a conundrum. We want traffic inspected by a firewall for ingress and egress traffic from the private subnet 10.0.0.0/24. (Ignore the second private subnet for this example.) The outbound routing has been set up correctly. We see a route table with the local route and a default route pointing to the FW's ENI(elastic network interface). So anything destined for an unknown destination will route into that ENI. Traffic returning from that source will not be forwarded to the FW though. The traffic will be analyzed and found to have a local route and forwarded directly to the target/host residing in 10.0.0.0/24 altogether skipping the FW. Ultimately we will end up with an asynchronous path to the internet and one that is only inspecting outbound traffic, not return traffic.

• In this second example we're able to resolve the issue above. The private subnet on the right has a default route to the nat-gateway in the public subnet. The private subnet on the left has a default route to the ENI of a firewall. Moving to the public subnet we see it has a default route to the internet through the IGW. At this point from an egress standpoint everything is set up properly. But ingress we would see an issue. The reason we would see an issue is because the IGW would have only seen a single route to "local" and would have natively routed directly to the end hosts who initiate outbound connections. What we desire is for a certain subset of hosts to have traffic inspected inbound and outbound on the firewall. Its only when we add a more specific route into the IGW route table to point to the ENI of the firewall for the 10.0.0.0/24 subnet that we get the desired in and outbound traffic flow. With this end to end set up, traffic for the private subnet 10.0.0.0/24 bound for an its default route is sent and inspected by the FW, upon returning to the VPC the IGW sees the ingress traffic originating from somewhere in the 10.0.0.0/24 subnet and forwards it to the firewall for inspection. The other private subnet gets less scrutiny simply traverses the nat-gateway and back to the hosts natively. No additional route manipulation required. The point being here that certain situations lend themselves better to specific ingress routing, if you require something specific to happen to your traffic upon re-entry into your VPC you need to have a VGWor IGWset up with its route table configured to handle routing.

• The above examples only mention ingress and egress routing inbound and outbound from a VPC but this could also very well be intra-VPC traffic that you need to control. In that case you'll need route tables to again control traffic to and from a destination where there is a requirement for specific routing.

Route Table Priority
There are three overarching ways that routes are prioritized within an AWS route table. The concepts are not that different from traditional routing and the administrative distances associated with different route types. In traditional routing we have static and connected routes with certain administrative distances. Then depending on the routing protocol used we have different admin distances that contend with each other. I want to make a special note about the way BGPworks and the path selection algorithm because it is similar to another concept that we'll cover below. When analyzing BGP routes we know there is a hierarchy within the protocol itself for path selection (Weight, Local Pref, Originate, AS Path, Origin, MED, eBGP > iBGP ... ect). This concept also exists for a subset of routes learned dynamically or via propagation within AWS.

Priorities:

Below is the way routes are prioritized in an AWS route table, MOST preferred to least preferred. Notice 1 - 3 are the high level route priorities. The table further breaks down prioritization when a route is learned via propagation. 1 is the highest priority within a propagated route, 4 is the lowest.

1. Longest prefix
2. Static routes
3. Propagated routes

What you should take away from this is that there is a hierarchy and based on how routes are learned or statically defined will determine how they are routed.

Lets dive into a few examples.

1. Two routes, one is a /24 the other a /25. The longest prefix match wins.

| Destination | Target |
| ----------  | -------|
| 192.168.0.0/24 | eni-123 |
| 192.168.0.0/25 | eni-123 | <- Wins due to longest prefix match

1. In this example we've got one route dynamically learned the other is a static route. Both are the same prefix length so the static route cannot be superseded by the longest prefix match.

| Destination | Target |
| ----------  | -------|
| 192.168.0.0/24 | eni-123 | <-- Dynamically learned
| 192.168.0.0/24 | eni-456 | <-- Static route: Higher priority over the route above it.

1. Finally we'll deal with propagated routes. This is much more similar to BGP and the path selection algorithm. In this example we've learned the same route from our on-prem router. All higher priority criteria matches (ie. they both have the same prefix length which would have made a difference in prioritization). So we're left with two routes both learned via propagation, now we need to figure out how they were learned. In this case we've got a route learned via a DX connection and a route learned via BGP over a VPN tunnel. If you look above you'll notice DX trumps routes learned via BGP over a VPN.

| Destination | Target |
| ----------  | -------|
| 192.168.0.0/24 | vgw-123(to DX connect) |<-Wins due to priority
| 192.168.0.0/24 | vgw-456(to VPN) | <- Lower priority than DX

In conclusion, I hope you gained some new understanding of AWS route tables, along with the fundamental principles of routing and path selection priorities. Feel free to share your thoughts, questions, and experiences in the comments below.

Cheers!