We have traffic spikes that require us to scale quickly. How does the Horizontal Pod Autoscaler (HPA) decide when to add more replicas? Also, how do we use Requests and Limits to ensure one "greedy" container doesn't crash an entire Worker Node by consuming all available CPU or Memory?
3 answers
HPA monitors metrics (usually CPU or Memory). When a Deployment exceeds a defined threshold (e.g., 70% CPU usage), HPA tells the Deployment to spin up more Pods. Regarding resources: Requests are what the container is guaranteed, and Limits are the maximum it can take. If a container tries to exceed its memory limit, K8s will terminate it (OOMKilled) to protect the node.
Be precise with your Limits. If you set them too low, your app will be throttled or killed unnecessarily. If you set them too high (or don't set them at all), a single bug causing a memory leak could starve every other Pod on that node, leading to a "cascading failure" across your cluster.
Scaling isn't just about Pods; it's about Nodes too. If HPA tries to add Pods but the nodes are full, those Pods will sit in "Pending." You need a Cluster Autoscaler to talk to your cloud provider (AWS/GCP/Azure) to provision new virtual machines automatically when the cluster hits capacity.
This is where the cost-efficiency of K8s really shines. When traffic drops, Cluster Autoscaler removes the extra nodes, ensuring you only pay for what you actually use.
We also use the Vertical Pod Autoscaler (VPA). While HPA adds more Pods, VPA actually recommends (or changes) the size of the Pods themselves based on historical usage. It’s great for "right-sizing" your infrastructure costs.