Every finance team asks the same question at the end of a quarter: why did this go up again? And every engineering team gives the same answer: we grew.
Sometimes that is true. Usually it is not.
When you break a growing bill into its actual line items, the pattern is almost always the same. A small number of charges — most of them created by accident, none of them owned by a named person — quietly compound month over month. Traffic growth is real, but it is rarely the largest contributor.
Here are the seven that show up most often, in roughly the order they surprise people.
1. Orphaned EBS volumes and snapshots
When an EC2 instance is terminated, its root volume is usually deleted with it. Volumes that were attached later, or created by hand, or provisioned by a Terraform run that was never applied cleanly, are not. They keep existing. They keep billing.
Snapshots are worse, because they are supposed to outlive the volume. A backup policy that creates a daily snapshot and never expires anything will accumulate indefinitely, and each one is charged for the storage it consumes.
The tell is a storage line that grows on a straight line while your instance count is flat.
# Volumes that exist but are attached to nothing
aws ec2 describe-volumes \
--filters Name=status,Values=available \
--query 'Volumes[].{ID:VolumeId,Size:Size,Created:CreateTime}' \
--output table
Run that in every region you have ever touched — not just the ones you use. Regions you enabled once for an experiment are the classic hiding place, because nothing in your normal dashboards points at them.
2. NAT Gateway data processing
This is the one that consistently shocks engineers, because the pricing model is not the one they assume.
A NAT Gateway costs money in two separate ways: an hourly charge for the gateway existing, and a per-gigabyte charge for every byte that passes through it. At the time of writing both are around $0.045 in us-east-1, though you should check current pricing for your region.
The hourly charge is predictable. The per-GB charge is not, because it scales with something nobody is watching: how much traffic your private subnets send to the internet. Container images pulled on every deploy. Package installs in CI. Logs and metrics shipped to a third-party SaaS. Backups written to an endpoint that resolves publicly.
A single chatty service in a private subnet can add thousands of dollars a month, and nothing in the service's own dashboards will show it, because from the service's point of view it is just making HTTPS calls.
Two structural fixes:
- VPC endpoints for AWS services. Traffic to S3 and DynamoDB through a gateway endpoint does not traverse the NAT Gateway at all. Interface endpoints for other services have their own hourly cost, so this is a calculation, not an automatic win — but for high-volume S3 traffic it usually pays for itself immediately.
- Check what is actually egressing. Enable VPC Flow Logs, then look at the destinations. Teams routinely discover their largest external destination is a service they forgot they were paying for twice.
3. Cross-AZ data transfer
Multi-AZ is the correct architecture. It is also billed on both ends: traffic leaving one availability zone and traffic entering another.
The cost is small per gigabyte and enormous in aggregate, because service-to-service chatter inside a cluster is invisible. A microservice mesh where every call is randomly load-balanced across three AZs sends roughly two-thirds of its internal traffic across an AZ boundary — by design, without anyone deciding it.
You do not fix this by abandoning multi-AZ. You fix it by making routing topology-aware, so a call prefers a healthy endpoint in its own zone and only crosses a boundary on failover. Kubernetes exposes this directly through topology-aware routing; most service meshes have an equivalent setting.
4. Load balancers with nothing behind them
An Application Load Balancer bills for every hour it exists, plus capacity units — whether or not a single request reaches it.
Delete a service, forget the ALB, and you have created a permanent line item with zero corresponding value. Multiply by every proof of concept your team has shipped in two years.
The same logic applies to Elastic IPs that are allocated but unassociated, and to Kubernetes Service objects of type LoadBalancer that were left behind when a namespace was partially torn down. The last one is especially common, because deleting a deployment does not delete the service, and the cloud-side load balancer belongs to the service.
5. Over-provisioned databases
RDS and its equivalents are the largest single line item on most SaaS bills, and they are almost never right-sized. The reason is rational: the person who picked the instance class picked it under uncertainty, and picking too large has no immediate consequence while picking too small has a very loud one.
So everyone rounds up, once, and nobody ever rounds back down.
Look at your database CPU utilisation over ninety days. If the peak — not the average, the peak — is under 40%, you are paying for a class you do not need. Right-sizing a production database is genuinely risky and needs a maintenance window, which is exactly why it keeps getting deferred. But the money is real, and it is recurring.
Storage has the same trap in reverse: RDS storage can be scaled up and cannot be scaled back down. A one-time load test that pushed you to 2 TB has permanently changed your monthly floor.
6. Log ingestion and retention
CloudWatch Logs bills for ingestion, for storage, and for queries. Ingestion is the one that grows without anyone approving it.
A service gets a new debug log line inside a hot path. It ships. Volume per request goes up by a few kilobytes. Multiply by request rate, multiply by 30 days, and a single log statement becomes a recurring four-figure charge.
Then there is retention. The default for a new CloudWatch log group is never expire. Every log group created by every Lambda, every ECS task, every experiment, keeps everything forever unless someone set a retention policy — and the setting is per log group, so "we fixed retention" almost always means "we fixed retention on the log groups we knew about."
# Log groups with no retention policy — these keep everything, forever
aws logs describe-log-groups \
--query 'logGroups[?!retentionInDays].[logGroupName,storedBytes]' \
--output table
7. Non-production environments
Staging, QA, demo, the environment built for a customer trial in March. Each was justified when created. Together they are often 30–40% of a bill, and unlike production they are idle for roughly two-thirds of every weekday and all of every weekend.
Nothing about this is a hard problem. A scheduled scale-down outside working hours is a well-understood pattern. It does not happen because it is nobody's job, and because the first time it breaks someone's late-night debugging session, the schedule gets disabled and never re-enabled.
The common thread across all seven: none of these are caused by a bad decision. They are caused by the absence of a decision — a resource that outlived its owner, a default that was never revisited, a schedule nobody enforced.
Why finding these late is the actual problem
Every item above is fixable in an afternoon once you know it exists. The expensive part is not the fix. It is the delay.
Cloud billing data is not real-time. Cost and Usage Reports land with a lag, monthly invoices land later still, and most teams only look during a quarterly review. That means a misconfiguration introduced on the 3rd of a month can easily go unnoticed until the middle of the next one.
By then you have already paid for it. Twice.
This is why cost work that depends on humans remembering to look does not hold. The audit finds real savings, everyone is pleased, and six months later the bill is back where it started — because the next orphaned volume was created the week after the audit ended, and no one was watching.
The thing that actually changes the trajectory is continuous detection: something that notices an untagged resource the day it appears, flags a log group without a retention policy at creation, and catches a NAT Gateway throughput change against last week's baseline rather than last quarter's invoice.
A practical starting order
If you want to work through this yourself, do it in this sequence — it is ordered by effort against return, not by size:
- Unattached volumes, unassociated Elastic IPs, empty load balancers. Pure waste, no architectural risk, deletable today.
- Log group retention policies. One command to find them, one to fix them, permanently lower floor.
- Non-production scheduling. Highest single return. Needs a rule for who can override it, or it will be overridden.
- VPC endpoints for high-volume S3 and DynamoDB traffic. Run the arithmetic first.
- Topology-aware routing. Real engineering work, real recurring savings.
- Database right-sizing. Largest number, highest risk, needs a window. Do it last, do it carefully.
Then decide who owns the number going forward. Not who reports it — who owns it.
DevLift's FinOps agent, Finly, runs this detection continuously across AWS, Azure and GCP: it flags idle and orphaned resources as they appear, tracks spend against a rolling baseline instead of a monthly invoice, and proposes right-sizing changes as reviewable infrastructure-as-code rather than console clicks. Book a walkthrough to see it against your own account.