The average Kubernetes cluster in 2026 runs at 8% CPU utilization — down from 10% the year before — while 69% of clusters are actively over-provisioning CPU and 79% are over-provisioning memory (PointFive). That's not a rounding error; it's most of the fleet paying for capacity it never uses. Organizations without a FinOps program waste 32-40% of cloud spend overall, while mature FinOps teams get that down to 15-20% (PointFive).
GPU waste is its own, more expensive problem: average GPU utilization sits at just 5%, and companies are paying $30,000-$50,000 per NVIDIA GPU for that — a waste bucket estimated at $10 billion industry-wide (PointFive).
The unmanaged-cluster waste range and the recoverable-savings range are, unsurprisingly, nearly the same number: unmanaged clusters waste 30-50% of spend, and structured optimization programs covering right-sizing, autoscaler tuning, and spot adoption typically recover 30-50% of cluster spend (Substack — Backend Developers).
Why cost ownership doesn't happen by default
Only about 14% of engineering teams implement chargeback for Kubernetes costs (PointFive). Without cost visibility and ownership at the team or namespace level, there's no incentive for any individual team to right-size their own requests or clean up unused services — the bill is someone else's problem, so nobody optimizes. Reducing waste is the #1 FinOps priority (47% of respondents in the FinOps Foundation's State of FinOps survey), but priority isn't the same as mechanism, and without per-team visibility the priority stays aspirational (PointFive).
Note
Right-sizing: the single highest-impact change
Right-sizing resource requests is consistently cited as the highest single-impact change for most clusters — typically a 30-50% cluster cost reduction, achievable in 1-2 weeks for a mid-size cluster (Substack — Backend Developers). The mechanism: the Vertical Pod Autoscaler (VPA) in recommendation mode analyzes 7-30 days of actual usage and recommends right-sized CPU/memory requests instead of the padded, guessed values most teams set at deploy time and never revisit.
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: my-app-vpa
spec:
targetRef:
apiVersion: "apps/v1"
kind: Deployment
name: my-app
updatePolicy:
updateMode: "Off" # recommendation-only mode: observe before you automate
Run VPA in "Off" mode first to gather recommendations without letting it actually restart pods — teams that jump straight to "Auto" mode often get surprised by unexpected pod evictions during business hours.
Spot instances: the biggest lever for stateless workloads
Spot instances remain one of the most powerful cost levers in 2026, especially for stateless, interruptible, or queue-driven workloads, cutting compute cost by 70-90% compared to on-demand pricing (Sedai). The tradeoff is real: spot instances can be reclaimed with as little as two minutes' notice, so anything stateful or latency-sensitive on the request path needs a fallback strategy (on-demand backup node groups, graceful pod eviction handling) before you put it on spot.
Karpenter has become the default autoscaler
Karpenter provisions nodes on demand across any instance type a workload can run on, offering faster scaling, better spot instance diversity, and more intelligent node selection than the older Cluster Autoscaler (Sedai). AWS made Karpenter the default autoscaler for EKS in 2024, and by 2026 it's described as running in every serious Kubernetes shop (Sedai) — if you're still on Cluster Autoscaler in an EKS environment, migrating is one of the more mechanical, well-documented upgrades available and directly compounds with spot savings.
Comparison: optimization levers ranked by typical impact
| Lever | Typical savings | Effort | Risk if misconfigured |
|---|---|---|---|
| Right-sizing (VPA-informed) | 30-50% | 1-2 weeks | Under-provisioning causes throttling/OOM kills |
| Spot instances | 70-90% on affected workloads | Moderate (needs fallback design) | 2-minute reclaim notice; not safe for all workloads |
| Karpenter vs. Cluster Autoscaler | Compounds with above | Moderate migration | Node selection misconfig can increase, not cut, cost |
| Namespace-level chargeback | Indirect (incentive effect) | Organizational, not technical | None — mostly a visibility/accountability gap to close |
The GPU-specific problem
GPU waste deserves separate treatment because the unit economics are so much higher. At 5% average utilization on $30-50k hardware, the fix isn't autoscaling in the traditional sense — it's workload scheduling and multi-tenancy: sharing GPU capacity across jobs via time-slicing or MIG (Multi-Instance GPU) partitioning rather than dedicating a full GPU to a job that uses a fraction of its capacity. Teams running AI/ML training or inference workloads on Kubernetes in 2026 should treat GPU utilization as its own tracked metric, separate from general CPU/memory dashboards, given the scale of the waste documented industry-wide.
Actionable takeaway
Start with visibility, not tooling: get per-namespace cost attribution running before buying an optimization platform, since only 14% of teams currently have that and it's the precondition for every other fix having teeth. Then run VPA in recommendation mode for two weeks to get real right-sizing data — this alone typically recovers 30-50% of cluster spend and is lower-risk than jumping straight to automated scaling changes. Once requests are right-sized, move interruptible/stateless workloads to spot instances (70-90% savings there) with Karpenter handling provisioning. If you run GPU workloads, treat utilization there as a separate, urgent metric — 5% average utilization on $30k+ hardware is the most expensive waste category in the entire stack.
Sources: PointFive — Kubernetes Cost Optimization: A Practical Guide 2026, The Backend Developers (Substack) — Kubernetes Cost Optimization in 2026, Sedai — Spot Instances in Kubernetes: Architecture & Cost Guide 2026, Sedai — Kubernetes Autoscaling Explained: HPA, VPA & Best Practices 2026
Get new posts as they publish
No spam — just the next post, straight to your inbox.