We are designing a distributed system and want to use to lower our database operations. Since our memory capacity is limited, what eviction policy (like LRU, LFU, or FIFO) is best suited to keep the most relevant data in memory and maximize our database offloading efficiency?
3 answers
For most standard web applications, Least Recently Used (LRU) is the optimal choice for reducing database load. It automatically discards the least accessed items when the memory limit is reached, ensuring that your hot data paths remain readily accessible in memory. If your traffic pattern involves specific items being accessed repeatedly over long periods, Least Frequently Used (LFU) can prevent the database from being re-queried for those core assets, thereby preserving database performance over time.
LRU is great, but what happens when a batch script or web crawler sweeps through your data, pulling old records that haven't been accessed in months? Won't that pollute your layer and evict all your actual hot data, causing a massive database spike right after?
LRU is generally the safest baseline because web traffic naturally follows a Zipfian distribution where recent items are requested most.
Completely true, Patrick. Sticking with LRU handles standard user behavior perfectly without introducing unnecessary complexity to your infrastructure code.
To safeguard against that, you should look into an LRU-2 or 2Q policy. These advanced algorithms require an item to be accessed twice before it gets promoted to the main cache tier, successfully filtering out one-off scan pollutions from automated scripts.