Auto-Scaling Group Simulator: target tracking in slow motion.

What an Auto-Scaling Group actually does

An Auto-Scaling Group is a desired-state controller for a fleet of EC2 instances. You declare a launch template, a min, a max, and a desired count. The ASG keeps the fleet at the desired count, replacing failed instances and re-balancing across Availability Zones. That's the static job.

The dynamic job is to change the desired count based on a metric. A scaling policy attached to the ASG watches a CloudWatch metric (CPU utilisation, ALB RequestCountPerTarget, custom metric) and adjusts desired count up or down. The ASG then either launches new instances or terminates existing ones to converge.

Target tracking vs step scaling vs simple scaling vs predictive

Default choice: target tracking on either CPU or ALB request count. Move to step scaling when you need finer control, or pair predictive with target tracking for the daily traffic shape plus the unexpected spike.

The control loop

Five hops, all asynchronous:

instance emits metric → CloudWatch agent → CloudWatch service (60s lag)
                                            ↓
                                       alarm evaluator
                                            ↓
                                  fires when threshold met
                                            ↓
                                   scaling policy invoked
                                            ↓
                                  ASG adjusts desired count
                                            ↓
                            EC2 fleet management launches/terminates
                                            ↓
                                instance boot + warm-up (60-600s)

From a CPU spike on the instance to traffic being served by a new pod, you're looking at 90 seconds to 12 minutes. The metric publication interval is 60 seconds by default (1 second for detailed monitoring at extra cost). Alarm evaluation requires N consecutive datapoints, where N defaults to several. Then the ASG action runs, EC2 boots, the load balancer starts sending traffic. None of this is instant.

CloudWatch metric publication lag

Default EC2 instance metrics publish at 5-minute intervals — that's basic monitoring, free with EC2. Enable detailed monitoring ($2.10 per instance per month at the time of writing) and you get 1-minute resolution. Custom metrics via the CloudWatch agent can go to 1-second resolution with high-resolution custom metrics, at extra cost.

Whatever resolution you choose, there's a publication delay on top — the metric covering minute t isn't available until minute t+1 or later. A target-tracking alarm evaluates the most recent N datapoints, so you're effectively making decisions on data that's 2–5 minutes old. If your cooldown is shorter than this lag, the next decision fires on the same stale data and you get oscillation.

Rule of thumb: cooldown should be at least 2× the metric publication interval, and ideally at least 1.5× the instance warm-up time. Defaults exist for a reason.

Warm-up time and the dilution trap

When a new instance launches, it boots, runs user-data, pulls container images, registers with the load balancer, and starts taking traffic. During this time it reports CPU near 0% because it's not doing any work. The ASG's CPU metric averages across all instances in the group.

So: existing instances at 80% CPU, controller scales out, two new instances come up at 0%, fleet average drops to (80×4 + 0×2)/6 = 53%. If your target is 50%, the controller now sees "below target" and orders a scale-in, terminating the brand-new instances. Then the original four spike back to 80% and you're in a thrash loop.

The fix is the instance warm-up parameter on the scaling policy. New instances are excluded from the metric average for the warm-up duration (default 300s, often needs to be 600s or more for slow-booting apps). With proper warm-up configured, the controller ignores brand-new instances until they're ready, and the dilution trap disappears.

Cooldown periods

Cooldown is the minimum time between scaling actions. After a scale-out, no further scale-out or scale-in happens until cooldown elapses. The intent is to give the previous scaling action time to take effect before reacting again.

Default is 300 seconds. Tune up if your application has a long warm-up time (Java apps with JIT compilation, large container images) or if your metric has high natural variance. Tune down only if you've measured that 300s is too conservative for your shape of traffic — and only after confirming there's no flapping at the shorter setting.

Target tracking handles cooldown automatically — AWS picks values based on the alarm and warm-up settings. Step scaling and simple scaling require you to set it yourself. Get this wrong and you'll either thrash (too short) or fail to respond to real spikes (too long).

Scaling on the right metric

CPU is the easy default, and it's wrong for many workloads. Web servers that block on database queries don't scale with CPU — they scale with concurrent connections. Background workers that pull from SQS don't scale with CPU — they scale with queue depth. Auth gateways that fan out to many downstreams don't scale with CPU — they scale with active outbound connections.

The right metric is the one that hits a wall first. For ALB-fronted services, RequestCountPerTarget usually beats CPU because it isolates the per-instance work signal from variable request weights. For queue-backed workers, ApproximateNumberOfMessagesVisible with a custom metric math expression dividing by current instance count. For services bound on something exotic (file descriptors, database connection pool saturation, GPU memory), emit a custom CloudWatch metric and target-track on that.

The principle is "scale on the bottleneck, not the symptom." CPU is the symptom in most overload scenarios; the bottleneck is connection pool exhaustion or queue depth or memory pressure. Find the bottleneck once, scale on it forever.

Mixed instance policy and Spot

A modern ASG runs a mixed instances policy: a list of instance types (e.g., m5.large, m5a.large, m6i.large, c5.large) and a split between On-Demand and Spot. The ASG launches across the eligible types and capacity pools, falling back automatically when one pool is exhausted.

Target tracking still works the same — the metric is fleet-wide. Spot interruption is the wrinkle. When AWS reclaims a Spot instance you get a 2-minute warning, and a lifecycle hook can run drain logic before the instance dies. If the hook takes too long the instance is force-terminated; if it returns fast the ASG launches a replacement, which then has to warm up.

The recommended config: capacity-optimized allocation strategy (AWS picks the deepest Spot pool to minimise interruption), 70–90% Spot for stateless workloads, a Capacity Rebalancing setting on for graceful replacement before interruption.

Predictive scaling

AWS trains a model on 14 days of your historical load and forecasts traffic 24–48 hours ahead. The ASG then pre-launches instances ahead of the predicted curve, so the fleet is already warm when traffic arrives. Combined with target tracking, you get the best of both — predictive handles the steady diurnal/weekly shape, target tracking handles deviation from it.

Where it helps: workloads with strong daily or weekly cycles (e-commerce, B2B SaaS, content platforms with regional peaks). Where it doesn't: launch traffic, viral events, anything where the past two weeks don't predict the next 24 hours. Predictive scaling will happily overprovision for the wrong curve if your traffic shape just changed.

Run it in "forecast only" mode for a week first; check the predicted curves against actual; then flip to "forecast and scale" if the predictions are sane.

Lifecycle hooks

Hooks pause the ASG transition for an instance and let you run setup or drain logic. Two hooks: autoscaling:EC2_INSTANCE_LAUNCHING runs before the instance is in service; autoscaling:EC2_INSTANCE_TERMINATING runs before termination. Each has a configurable timeout (default 1 hour, max 48 hours) and a default action (CONTINUE or ABANDON).

Common uses. Launching hook: pull config from Parameter Store, register with service discovery, run health checks before adding to the load balancer. Terminating hook: deregister from the load balancer (connection draining), flush in-memory state, upload final logs to S3. The hook signals "done" via complete-lifecycle-action in the SDK; if it times out, the default action applies.

Most teams use EventBridge to trigger a Lambda from the hook event, which is cleaner than running anything on the instance itself for the termination case. For launching, user-data + cfn-signal is the older pattern; lifecycle hooks + Lambda is the modern one.

Common production failures

• Flapping. Cooldown too short relative to metric lag and warm-up time. The fleet oscillates between 4 and 8 instances every 10 minutes, never settling. Fix: bump cooldown to 600s, instance warm-up to match boot time, confirm with CloudWatch's scaling activity history.
• Runaway scale-out. A metric goes wrong (NaN, stuck at 100%) and the controller scales to max-instances. If you don't have a max-instances cap, you wake up to a billion-dollar EC2 bill. Always set a max that's no more than 3–5× expected steady-state, even if it means dropping requests during a real spike.
• AZ-imbalanced fleet. An AZ has limited capacity for your instance type. ASG launches everything in the remaining AZs and you lose the resilience that the multi-AZ ASG was supposed to give you. Mitigation: mixed instance policy across instance types, so the ASG can pick a different type in the constrained AZ.
• ASG fighting cluster-autoscaler. If you're running Kubernetes on top of an ASG with cluster-autoscaler, the cluster-autoscaler scales the node group based on pod resource requests. Don't also attach a target-tracking policy to the same ASG. The two controllers will disagree and one will keep undoing the other's actions.
• Stale launch template. A new launch template version (new AMI) is published but the ASG points at "version $Latest" without instance refresh. New instances boot with the new AMI; old ones keep the stale one. Either explicitly bump the launch template version + run instance refresh, or accept that mixed-AMI fleets are temporary.
• SNS subscription on every scaling event. If you wire scaling events to a Slack channel, expect 50+ notifications a day on an active fleet. Either filter to scale-failures only or send to a dashboard rather than a chat channel.

A worked target-tracking example

Imagine a web fleet of 4 c5.large instances behind an ALB. Target: 50% CPU. Min 2, max 16. Cooldown 300s. Warm-up 300s.

t+0:   load 60%, fleet at 65% CPU, no action (just above target, within damping)
t+1:   load 80%, fleet at 85% CPU
t+2:   alarm fires (3 consecutive > target + 10)
t+2:   ASG orders +3 instances (proportional to overshoot)
t+2:   3 new instances launching, warmup_remaining=300s
t+3-7: new instances at 0% but excluded from metric average via warm-up
t+8:   warm-up expires, fleet now at 7 instances, average CPU drops to ~48%
t+10:  cooldown expires, controller checks: avg=48%, near target, no action
t+30:  load drops to 30%, fleet at 25% CPU
t+33:  alarm fires (under target - 15)
t+33:  ASG orders -2 instances (proportional)
t+33:  oldest 2 terminate after draining

Notice the gaps. From the alarm firing to the new instances being effective is 8 minutes. From load dropping to instances being terminated is 3 minutes. The ASG is a slow controller; it suits steady-state workloads, not bursty ones. For bursty, layer a CloudFront cache or an SQS queue in front and let the queue absorb the spike while the ASG catches up.

Further reading

• AWS Auto Scaling user guide. The canonical reference. The scaling policies and lifecycle hooks chapters are the must-reads.
• Target tracking docs. Specifically the section on instance warm-up — most production issues trace back to misunderstanding this.
• Predictive scaling announcement. AWS's own framing of when ML-based forecasting helps.
• KEDA — Kubernetes Event-Driven Autoscaling. The right reference when comparing ASG to k8s scaling on event-source metrics (Kafka lag, queue depth).
• Large-scale cluster management at Google with Borg. The original bin-packing paper. Useful framing for what fleet sizing is really about.
• Load balancer simulator. The ALB sits in front of the ASG and routes the actual requests; tuning them together is the real production story.
• Circuit breaker simulator. What protects the upstream while the ASG is busy launching.
• EC2, EBS, and AMI codex. The longer-form reference that pairs with this simulator.