I’m struggling with data persistence in my kubernetes cluster. Every time a pod restarts, I lose my database changes. I know about PersistentVolumes, but I'm confused about the best StorageClass to use for high-availability databases like PostgreSQL. How do you ensure data doesn't vanish?
3 answers
For stateful applications, you must use a StatefulSet instead of a Deployment. This ensures that pods maintain a persistent identity and stay attached to the same PersistentVolume (PV) even after a restart. Regarding the StorageClass, it depends on your cloud provider. On Azure, we use Managed Disks with the "premium-ssd" tier to ensure low latency for our SQL workloads. You should also look into the Container Storage Interface (CSI) drivers. In mid-2023, we implemented snapshots via the CSI driver, which allowed us to take point-in-time backups of our volumes directly within the kubernetes API.
Have you explored using an operator like the CloudNativePG for your databases? It automates a lot of the manual storage and failover logic within kubernetes.
Use "Retain" as your ReclaimPolicy in the kubernetes PV. This prevents the cloud provider from deleting the physical disk if the PVC is accidentally deleted.
That "Retain" tip is a lifesaver. I also agree with Deborah; moving to StatefulSets is the first and most important step for any kubernetes database setup.
I’ve heard of operators but haven't tried them yet. Bradley, does the operator handle the PersistentVolumeClaim creation automatically, or do I still need to define those manually in my kubernetes YAML files? We really want to automate as much of the database lifecycle as possible to avoid human error during a node failure.