I am managing a pool of servers where the processing time for each request varies significantly. We currently use Round Robin, but some servers are getting overwhelmed while others stay idle because some sessions take 30 minutes to complete. Should I switch to Least Connections or Weighted Round Robin? How do these algorithms handle session persistence (sticky sessions) in a production cluster?
3 answers
If your request times vary, Round Robin is definitely the wrong choice because it assumes all requests are equal. You should switch to "Least Connections." This algorithm tracks how many active sessions each server is currently handling and sends new traffic to the server with the lowest count. This naturally balances the "heavy" 30-minute sessions across your fleet. For session persistence, you'll need to enable "Sticky Sessions" using cookies. This ensures that once a user is assigned to a server, they stay there for the duration of their task, preventing state loss in the middle of a transaction.
Have you checked if your backend servers have different hardware specs, or are they all running on identical virtual machine instances?
Least Connections is the standard for long-lived tasks. Round Robin is only good when all your tasks take roughly the same amount of time to finish.
I agree with Susan. We had the same issue with our data export service until we switched. Round Robin was killing our "unlucky" nodes that got hit with multiple heavy jobs.
Thomas, if the hardware is different, he should definitely look into "Weighted Least Connections." It allows you to assign a higher weight to more powerful servers so they can handle a larger portion of the active sessions. It's a lifesaver when you're in a transition period moving from older on-premise hardware to newer, faster cloud instances without decommissioning the old ones.