I’ve noticed that several of my Pods have suddenly stopped running and are now showing a status of "Evicted" in the dashboard. When I check the logs, there isn't much information, but the node itself seems to be under heavy load. What exactly triggers this state in a Kubernetes cluster, and is there a way to prevent the scheduler from terminating my active containers when the node reaches its resource limits? I need to understand if this is related to memory, disk space, or something else entirely.
3 answers
Pod eviction typically occurs when a node experiences "Node Pressure," which means it is running out of critical resources like Disk Space (DiskPressure), Memory (MemoryPressure), or PID availability. When the kubelet detects that the node has crossed a specific threshold—for example, less than 10% of disk space remaining—it begins terminating Pods to reclaim those resources and keep the node stable. This is a failsafe to prevent the entire node from crashing. To diagnose the specific cause, you should run kubectl describe pod [pod_name] and look at the "Status" or "Reason" field, which will often explicitly state if it was due to DiskPressure or node.kubernetes.io/memory-pressure. Setting proper resource requests and limits in your deployment YAML is the best way to ensure your mission-critical Pods aren't the first ones chosen for eviction.
I've checked the descriptions, and it seems like my pods are being evicted because of imagefs.available thresholds. Does this mean my Docker images are taking up too much space, or is it related to the logs and temporary data my application is writing during its execution?
You should also check your "Eviction Hard" and "Eviction Soft" thresholds in the kubelet configuration. Sometimes the default values are a bit too aggressive for smaller nodes.
I agree with Michael. In many Cloud Technology setups, especially on managed services like EKS or GKE, the defaults are set to protect the control plane. Tuning these thresholds, combined with Kimberly's suggestion of setting resource limits, is the only way to gain true stability for your cluster.
Steven, imagefs.available usually refers to the storage used by your container images and the writable layer of your containers. If your application writes large log files to the container's internal filesystem instead of a persistent volume or a centralized logging service, you will hit this limit very quickly. You should consider using an emptyDir volume with a size limit or moving your logs to an external mount. This keeps the container's root filesystem clean and prevents the kubelet from identifying your Pod as the "offender" that needs to be evicted to save the node's disk