VPC Packet Flow Simulator, where the bill comes from.
Subnets, route tables, security groups, NACLs, gateways and endpoints. One packet, one hop at a time — every policy stop that can drop it, plus what each line of the eventual NAT bill represents.
| Dir | Proto | Port | CIDR |
|---|---|---|---|
| in | TCP | 443 | 0.0.0.0/0 |
| out | TCP | 0-65535 | 0.0.0.0/0 |
| # | Dir | Proto | Port | Action |
|---|---|---|---|---|
| 100 | in | ALL | ALL | allow |
| 100 | out | ALL | ALL | allow |
| Destination | Target |
|---|---|
| 10.0.0.0/16 | local |
| 0.0.0.0/0 | nat-gw |
The diagram is a small VPC: a public subnet holding the NAT Gateway and IGW, a private subnet holding an EC2 instance, plus the S3 gateway endpoint and a peering attachment. Choose a source and destination, set protocol and port, and hit Send. A packet then walks the policy stops one at a time — SG egress, NACL egress, the route table, then whichever gateway the route points at — and each hop lights green for allow or coloured for deny, with the trace log spelling out the reason. The rule tables below are editable; add an SG or NACL rule and the next packet is judged against it.
Run "Private → 1.1.1.1" first. The packet clears both rule layers, the route table sends it to the NAT Gateway, and only then does it reach the IGW and the internet — that NAT hop is the line on your bill. Now add a NACL egress rule that denies TCP and send again: the packet dies mid-path. What should surprise you is the asymmetry. Security groups are stateful, so the return trip is allowed automatically, while NACLs are stateless and evaluated per direction. Forget one layer and the packet vanishes with no error anywhere.
What a VPC actually is
Software-defined networking on EC2.
An Amazon VPC is a logically isolated virtual network you run inside an AWS region. You choose its IP range, you carve it into subnets, you decide what reaches the public internet and what stays private. None of this is physical. Every packet inside a VPC is encapsulated, routed, and policed by a fleet of services running on AWS Nitro cards in the hypervisor; there is no cable for you to trace.
The closest mental model is a routed overlay on someone else's data centre. When an EC2 instance sends a packet to another instance in the same VPC, the Nitro card on the source host wraps the packet in an envelope addressed to the Nitro card on the destination host, hands it to the underlying physical network, and unwraps it on arrival. The instances see a normal IP packet. The physical network sees encapsulated traffic between two Nitro cards. AWS's mapping service answers "which Nitro card owns this IP right now?" and is the single largest piece of infrastructure most people working on EC2 will never touch.
That overlay model is why VPCs are cheap to create (the limit per account is 5 by default, raisable to thousands), cheap to delete (no hardware reallocation), and have predictable per-byte costs (no over-provisioned pipes to amortise). It's also why all the policy hooks you have — security groups, NACLs, route tables, flow logs — work at any scale. The control plane never has to physically rewire anything. It just updates a table.
The simulator above shows the canonical small VPC: one public subnet with an IGW and a NAT Gateway, one private subnet with an EC2 instance, plus the policy stops on the packet path. Choose a source and destination, pick a protocol and port, hit Send and watch the packet traverse the hops. Toggle a NACL rule to deny and see the same packet drop. The interactive bit is meant to make the hop ordering concrete: SG → NACL → Route table → Gateway, every time.
CIDR planning — pick big, regret little
Pick big, regret little.
A VPC needs a CIDR block. The default suggestion in the AWS console is 10.0.0.0/16 — 65,536 addresses, more than most teams ever need. Take it. Bigger blocks cost nothing and avert future pain. The cardinal sin of VPC design is creating two /24 VPCs in different accounts, then trying to peer them later and discovering they overlap.
Inside the VPC, carve out /24 subnets per availability zone. A /24 gives you 251 usable hosts (AWS reserves 5 per subnet for the network address, broadcast, the implicit router, the DNS resolver, and a future-use placeholder). For most application workloads that's plenty. For Lambda-in-VPC or large EKS clusters where every pod gets an ENI IP, jump to /22 or larger per subnet. EKS in particular eats IPs fast; an under-provisioned subnet shows up as pod scheduling failures.
The renumbering problem is the killer. Once instances are launched, processes are bound to IPs, configs reference IPs, and Terraform state knows IPs. Renumbering a live VPC is roughly equivalent to rebuilding it; you spin up a parallel VPC with the new range, migrate workloads across, then decommission the old one. AWS now offers secondary CIDR blocks on VPCs which soften this — you can graft an extra /16 onto an existing VPC if you outgrow the original. But planning for that from the start is cheaper than learning it the hard way.
The minimum subnet AWS allows is /28 (16 addresses, 11 usable). The maximum is /16 (65,531 usable). Pick subnet sizes per use case: tiny /28 for a single bastion host, /24 for typical app workloads, /20 or larger for Kubernetes data planes.
Public vs private subnets — defined entirely by the route table
The subnet itself isn't tagged.
A common misconception is that subnets carry a public-or-private flag. They don't. A subnet is "public" if and only if its route table has an entry sending 0.0.0.0/0 through an Internet Gateway. Drop that entry and the same subnet becomes private. Add it to a previously private subnet and you've just exposed every instance with a public IP to the open internet. The only thing distinguishing the two kinds of subnet is the routing policy.
For an instance in a public subnet to actually reach the internet, three things must align: the route table sends 0.0.0.0/0 to the IGW; the instance has a public IPv4 address (auto-assigned or via Elastic IP); and security groups permit egress on the relevant port. Miss any of those three and the packet vanishes.
Private subnets reach the internet only through a NAT — either a managed NAT Gateway, a self-managed NAT instance, or in newer designs, AWS Transit Gateway routing through a central egress VPC. The instances themselves never get a public IP. The reverse direction is blocked: nothing on the public internet can initiate a connection to a private instance, because there's no route in to it.
The convention is to put load balancers, NAT Gateways, and bastions in the public subnet, and put all the application servers, databases, and internal services in private subnets. Web traffic enters via the ALB, the ALB forwards to private targets, and outbound traffic from those private targets flows back through the NAT Gateway. Most production VPCs look exactly like this, repeated across two or three AZs for high availability.
| Property | Public subnet | Private subnet |
|---|---|---|
| Route to internet | via IGW | via NAT |
| Inbound from internet | possible (with public IP) | impossible |
| Typical inhabitants | ALB, NAT GW, bastion | app servers, RDS, internal services |
| Public IPv4 auto-assign | often on | off |
| Cost per byte | data-out at standard rate | NAT GW: $0.045/GB extra |
Internet Gateway — the cheap exit
Stateful, no throughput limit, free.
The Internet Gateway (IGW) is a horizontally scaled, redundant component that sits at the edge of a VPC and connects it to the public internet. Conceptually it does two things: it translates between the public IP and the instance's private IP for instances that have a public IP (a 1:1 NAT, essentially), and it serves as the route target for the 0.0.0.0/0 entry in public subnets' route tables.
The IGW itself has no throughput cap, no hourly cost, and no per-GB cost beyond standard data-out pricing. It is, in AWS networking terms, the cheapest exit available. The catch is that to use it for outbound traffic, the instance needs a public IP. That's fine for load balancers and bastions. It's awkward for fleets of private application servers, which is exactly the use case NAT Gateway exists to handle.
The 1:1 NAT model means inbound packets to your public IP arrive at the IGW, get rewritten to your private IP, and are delivered to the instance. There is no port-to-port mapping like a home router's NAT — your public IP and your private IP have a one-to-one correspondence as long as the instance has the public IP attached. This is why allocating an Elastic IP (a static public IPv4 reserved to your account) gives an instance a stable external identity.
Egress-only Internet Gateways exist for IPv6: they allow outbound IPv6 traffic but block inbound, which is the IPv6 equivalent of "instances in a private subnet behind a NAT." IPv6 doesn't use NAT in AWS; addressing is end-to-end, so the egress-only IGW is how you replicate the inbound-blocked semantics.
NAT Gateway — the expensive exit
Managed, fast, and the line item that surprises you.
A NAT Gateway is AWS's managed network address translator. Instances in a private subnet send outbound traffic through it; the NAT Gateway rewrites the source IP to its own public IP, forwards the packet via the IGW, and tracks the connection so return packets get rewritten back. From the outside world, a fleet of a hundred private instances looks like a single public IP — that of the NAT Gateway.
NAT Gateways scale to 100 Gbps of throughput per NAT and support up to 55,000 simultaneous connections to a single destination IP. They are zonal — one per availability zone for high availability, since a NAT Gateway in us-east-1a doesn't survive an outage of that AZ. Properly designed VPCs run one NAT Gateway per AZ and route each private subnet through the NAT Gateway in its own AZ to avoid cross-AZ data transfer costs.
The pricing is where it gets interesting. NAT Gateway costs $0.045 per hour (around $33 a month) plus $0.045 per GB processed. The per-GB charge is on top of normal data-out pricing — so an outbound byte through NAT costs roughly twice what an outbound byte through an IGW costs. For high-egress workloads this becomes the dominant line item. A team running a service that pulls 10 TB a day of external API traffic through a NAT Gateway pays $450/day just for the NAT, on top of regular data-out. The same traffic through an IGW with public IPs on the instances would cost half.
This is why mature shops put as much outbound traffic as possible through VPC endpoints (free for gateway endpoints to S3 and DynamoDB), make sure cross-AZ NAT traffic is eliminated, and treat NAT Gateway processing GB as an OKR. Cloud cost engineers have favourite war stories about misrouted internal-only services that ran through NAT Gateway by accident, costing $50k a month before someone noticed.
An EKS pod hits S3 through the cluster's NAT Gateway. The S3 bucket is in the same region as the cluster. The team is paying NAT GB charges plus cross-AZ traffic for what should be a free, in-region S3 call. Adding a gateway VPC endpoint for S3 to the route table eliminates both — the traffic stays inside AWS's backbone and the bill drops.
VPC endpoints — Gateway and Interface
Two flavours, very different price tags.
VPC endpoints let an instance in a private subnet reach an AWS service without going through the public internet — no NAT, no IGW, no data-out charge, just an entry in the route table that says "52.218.0.0/16 goes to this endpoint." There are two kinds and their pricing differs by an order of magnitude.
Gateway endpoints exist only for S3 and DynamoDB. They are free. They are implemented as a special route in the subnet's route table that funnels traffic to the service over AWS's internal backbone. No ENI, no hourly charge, no per-GB processing fee. Every production VPC that uses S3 should have a gateway endpoint for S3; not having one is leaving money on the table.
Interface endpoints (PrivateLink) exist for everything else: SSM, ECR, KMS, SQS, SNS, the AWS API itself, plus arbitrary third-party services that publish their own endpoints. They are implemented as ENIs in your subnet with private IPs from your CIDR. They cost $0.01 per AZ per hour ($7.20/month/AZ) plus $0.01 per GB processed. The cost is significant but usually still cheaper than NAT-routed access — and the security posture is better, since traffic never leaves AWS's private network.
The combination — gateway endpoint for S3/DynamoDB, interface endpoints for the AWS APIs the workload needs — lets you build a no-IGW, no-NAT-Gateway VPC where private instances can still reach the AWS services they depend on. Cost-conscious shops use exactly this pattern: a "private" VPC with zero public network exposure and all AWS access via endpoints.
# Adding a Gateway endpoint for S3 — Terraform resource "aws_vpc_endpoint" "s3" { vpc_id = aws_vpc.main.id service_name = "com.amazonaws.us-east-1.s3" vpc_endpoint_type = "Gateway" route_table_ids = [aws_route_table.private.id] } # Adding an Interface endpoint for ECR resource "aws_vpc_endpoint" "ecr_api" { vpc_id = aws_vpc.main.id service_name = "com.amazonaws.us-east-1.ecr.api" vpc_endpoint_type = "Interface" subnet_ids = aws_subnet.private[*].id security_group_ids = [aws_security_group.endpoints.id] private_dns_enabled = true }
Route tables — one per subnet, most specific wins
One per subnet, most specific wins.
Every subnet is associated with exactly one route table. If you don't explicitly create one, AWS associates the subnet with the VPC's default route table. The table consists of entries mapping CIDR destinations to targets — IGW, NAT Gateway, peering connection, transit gateway attachment, VPC endpoint, network interface, or "local" (the implicit VPC-internal route, which is always present and can't be removed).
The matching rule is longest prefix match, the same rule used by every internet router on the planet. If the table has entries for 10.0.0.0/16 (local) and 10.0.1.0/24 (custom), a packet to 10.0.1.5 matches both, but the /24 wins because it's more specific. This lets you override the implicit local route for a portion of your CIDR — for instance, to send traffic for a specific subnet through a network appliance for inspection.
Subnets can share a route table; multiple subnets pointing at the same route table is the normal pattern. Most VPCs run with a "public" route table associated with all public subnets and a "private" route table associated with all private subnets. The public table has a 0.0.0.0/0 → IGW entry; the private table has a 0.0.0.0/0 → NAT-GW entry; both have the implicit local route.
The trick most often missed: peering connections need routes on both sides. Setting up VPC peering only attaches the connection — you still need to add routes in both VPCs' route tables for traffic actually to flow across. Forgetting this is a multi-hour debugging session, because peering connections show as ACTIVE while traffic silently disappears.
| Destination | Target | Why |
|---|---|---|
| 10.0.0.0/16 | local | implicit, the VPC itself |
| 0.0.0.0/0 | igw-abc123 | public subnet egress |
| 0.0.0.0/0 | nat-def456 | private subnet egress |
| 10.1.0.0/16 | pcx-789 | peered VPC |
| 52.218.0.0/16 | vpce-s3 | S3 gateway endpoint (auto-managed) |
| 192.168.0.0/16 | tgw-attach-1 | on-prem via Transit Gateway |
Security groups vs NACLs — the two firewalls
Stateful at the ENI, stateless at the subnet.
Every packet in a VPC passes two firewalls. Most engineers know about the first — security groups — and forget about the second, NACLs, until something breaks.
Security groups attach to ENIs (the elastic network interface of an EC2 instance, load balancer, RDS endpoint, Lambda-in-VPC pod, anywhere there's an interface). They are stateful: if a rule permits a packet outbound, the return packet is automatically permitted regardless of inbound rules. They are default-deny: no rule, no traffic. Rules are allow-only — you can permit, you can't deny. To "deny" something you remove the permitting rule.
NACLs attach to subnets, not interfaces. They are stateless: every direction must be permitted independently. A connection from 10.0.1.5 to an internet host on port 443 needs an egress rule permitting TCP 443 outbound and an ingress rule permitting TCP on the ephemeral port range (1024-65535 or 32768-60999, depending) inbound. They are default-allow: a fresh NACL permits everything. Rules are numbered and evaluated in order; first match wins. Allow and deny rules both exist.
The evaluation order on egress from an EC2 in a private subnet: SG (at the ENI) → NACL (at the subnet boundary) → route table → next hop. On ingress: NACL → SG → instance. A packet has to pass every check to make it through.
The right mental model: NACLs are a coarse-grained subnet-level perimeter, useful for blocking obviously bad traffic at the boundary (e.g., a known-bad IP range). SGs are the fine-grained per-workload firewall, where most of the actual policy lives. In practice, most VPCs leave NACLs at default-allow and put all the enforcement in security groups. The NACL is a defence-in-depth backstop.
You add a deny rule to a NACL for inbound port 22 to block SSH from the internet. Six hours later, a colleague reports that ELB health checks have started failing. The NACL was blocking the health check return traffic on the ephemeral port range, because NACLs are stateless and you forgot the return-traffic rule. Lesson: if you're using NACL deny rules, always think through the return path.
VPC peering vs Transit Gateway — pairwise vs hub-and-spoke
Pairwise vs hub-and-spoke.
VPC peering connects two VPCs directly. It supports cross-account, cross-region peering. Traffic flows over AWS's backbone, never the public internet. Costs nothing per hour. Pays standard data-transfer pricing. CIDRs must not overlap, or the connection silently drops packets to the overlapping range.
The killer limitation: peering is non-transitive. If A peers with B and B peers with C, A cannot reach C through B. You'd need a third peering between A and C. For three VPCs that's three peering connections; for ten VPCs that's 45; for a hundred it's 4,950. The combinatorial explosion is real.
Transit Gateway (TGW) solves the combinatorics. It's a regional hub: every VPC attaches to it once, and the TGW handles routing between attachments. Ten VPCs require ten attachments and one TGW. Routes are configured via TGW route tables, which give you fine-grained control over which attachments can talk to which (you can isolate dev from prod even when both attach to the same TGW). On-prem VPN and Direct Connect attach to the TGW too, making it the standard hub for hybrid topologies.
The pricing: TGW costs $0.05 per attachment-hour plus $0.02 per GB processed. For ten attachments that's $0.50/hour, around $360/month, before data transfer. For a small number of VPCs, peering is cheaper. For more than ~6-7 VPCs, TGW typically wins on both cost and operational simplicity. AWS Cloud WAN is the newer (2022+) successor pitched for multi-region, multi-account topologies; it's a TGW-of-TGWs essentially.
| Property | VPC peering | Transit Gateway |
|---|---|---|
| Topology | pairwise (mesh) | hub-and-spoke |
| Transitive routing | no | yes (via TGW route tables) |
| Cross-region | yes | yes (TGW peering) |
| Cross-account | yes | yes (via RAM) |
| On-prem (VPN / DX) | no | yes |
| Hourly cost | $0 | $0.05 per attachment |
| Per-GB processed | $0 (data transfer only) | $0.02 |
| Sweet spot | ≤5 VPCs, simple peering | ≥6 VPCs, hybrid, multi-account |
DNS in a VPC — the .2 resolver
Route 53 inside, at the .2 address.
Every VPC ships with an internal DNS resolver, the Route 53 Resolver, reachable at the second IP of the VPC CIDR. For a 10.0.0.0/16 VPC, that's 10.0.0.2. Instances configured with the VPC's DHCP option set use this resolver by default. It answers queries for public DNS names by recursing out to the internet, and answers queries for internal names (private hosted zones, EC2 private DNS names) directly.
The resolver has its own quirks. It rate-limits to 1024 packets per second per ENI; workloads doing huge volumes of DNS lookups (microservices with per-request DNS, certain Java HTTP clients with short-TTL caches, Kubernetes Services without conntrack) can hit the limit and see resolution failures. The standard mitigations: increase application DNS caching, deploy CoreDNS as a per-node cache in EKS, use Route 53 Resolver endpoints if you need higher throughput.
Private hosted zones let you define internal-only names for a VPC. Associate a private hosted zone with a VPC and any query inside that VPC for a name in that zone resolves to whatever you configured. The standard pattern is internal.example.com for internal services, with records pointing at internal ALBs or service discovery endpoints. The same name resolves to nothing from outside the VPC.
For hybrid setups — DNS queries from on-prem needing to resolve VPC private names, or VPC queries needing to resolve on-prem AD names — Route 53 Resolver supports inbound and outbound endpoints. An inbound endpoint exposes the VPC resolver to on-prem via Direct Connect / VPN; an outbound endpoint forwards specific zones from VPC queries to on-prem DNS servers. The two-endpoint pattern is how most hybrid AWS-plus-on-prem shops handle DNS.
What breaks — failure modes worth memorising
Failure modes worth memorising.
Overlapping CIDRs between peered VPCs. The peering attaches successfully; routes get added; traffic to the overlapping range goes nowhere. There's no error, no log entry, just silent forwarding loss. The fix is renumbering one of the VPCs, which is expensive. The prevention is picking non-overlapping CIDRs from the start, ideally with a documented IPAM plan covering everything you might ever peer.
Forgotten NACL deny rules. Someone adds a deny rule to a NACL to block a specific IP. The rule blocks more than intended because NACLs are stateless and the engineer forgot that return traffic on ephemeral ports also passes through the NACL. Health checks fail, deploys break, alarms fire.
NAT Gateway running in a dev subnet. Someone enables a NAT Gateway in dev for a one-off troubleshoot, forgets to remove it. $33/month for the hourly charge plus whatever processed bytes the dev workload generates. Multiplied across forgotten dev environments, this is a five-figure annual line item.
Missing return-route through TGW. Workload A in VPC-A talks to workload B in VPC-B via TGW. Route from A to B exists. Route from B back to A doesn't. A's request goes out, B receives it, B sends a response that has no return path. Connection times out. The fix is symmetric routing on both sides of every TGW attachment.
Public IP without an IGW. Instance auto-assigns a public IP because the subnet is configured to. The subnet's route table doesn't have an IGW entry. The public IP is functionally useless — no traffic enters. The instance still reports as having a public IPv4 in its metadata, which adds to the confusion.
Cross-AZ data transfer through NAT Gateway. Private subnet in us-east-1a routes its 0.0.0.0/0 through a NAT Gateway in us-east-1b. Every outbound byte pays cross-AZ data transfer ($0.01/GB each way) plus NAT Gateway processing ($0.045/GB). Per-AZ NAT Gateways with per-AZ private route tables eliminate the cross-AZ leg.
Further reading on VPC and AWS networking
Primary sources first.
- AWS docsAmazon VPC User GuideThe canonical reference. Long, but exhaustive. Start with "How Amazon VPC works" and the routing chapter.
- AWS Architecture BlogVPC design patternsHub-and-spoke, multi-account, shared services VPC, ingress VPC patterns. The most-referenced shapes.
- James HamiltonPerspectives — AWS networking talksA decade of essays on the AWS network from a former VP/Distinguished Engineer. The re:Invent 2014 talk on the AWS network is the definitive history.
- Stripe engineeringNetworking at StripeReal-world VPC architecture from a payments-scale shop. Egress hardening, private service endpoints, and lessons learned from a production bot storm.
- AWS newsEC2 network bandwidth scalingHow modern instance types get 100+ Gbps of network capacity. Background for understanding what NAT Gateway's 100 Gbps ceiling means.
- Dalton et al · NSDI 2018Andromeda — Performance, Isolation, and Velocity at Scale in Cloud Network VirtualizationGoogle's overlay-network paper, the closest published analogue to AWS's VPC implementation. Worth reading alongside any VPC reference.
- re:Invent 2019NET407 — Networking deep dive: route tables, peering, transit gatewayThe advanced VPC routing talk that goes through the same material as this page but with AWS staff drawing diagrams live.
- Corey QuinnLast Week in AWS — NAT Gateway pricing postsThe most useful catalogue of NAT Gateway war stories on the public internet. Many production NAT bills started as someone reading one of these posts.
- Semicolony guideVPC deep dive (codex companion)The codex page that walks through every concept on this simulator with worked examples and Terraform.
- Semicolony simulatorLoad balancerWhat's running in your public subnets — ALB vs NLB, health checks, sticky sessions.