Our cloud bill is skyrocketing and I need to optimize our resource usage. I am considering moving our non-critical workloads to Spot/Preemptible instances while using HPA to handle traffic spikes. Are there specific pitfalls when mixing HPA with cluster autoscalers on Spot nodes, especially regarding pod eviction and graceful termination during node reclamation?
3 answers
Mixing HPA with Spot instances is the ultimate way to save up to 80% on compute costs, but it requires a very robust architecture. You must implement a 'Pod Disruption Budget' (PDB) to ensure a minimum number of replicas stay alive during a mass reclamation. Also, ensure your application handles SIGTERM signals gracefully; you only get a 30-second warning before a Spot node is taken back. We use a "fallback" node pool of On-Demand instances with lower priority so that if Spot capacity disappears, our HPA can still schedule pods on reliable hardware. This "weighted" approach saved us $5k last month.
Have you tried using Vertical Pod Autoscaler (VPA) in 'Recommendation' mode alongside HPA to see if your initial resource requests are just set too high to begin with?
I recommend looking into 'Karpenter' if you are on AWS. It handles the bin-packing and node selection much more efficiently than the standard Cluster Autoscaler.
I second the Karpenter recommendation. It's much faster at spinning up new nodes during HPA triggers compared to the old-school autoscaler logic.
Kevin, that's a brilliant suggestion. Many teams over-provision because they don't know their app's actual footprint. By using VPA for insights and HPA for scaling, you get the best of both worlds. It prevents the cluster from requesting massive nodes for pods that only actually use 10% of their allocated CPU. This "right-sizing" is often more effective than just switching to cheaper instances.