We recently moved to AWS Lambda, thinking serverless would solve all our capacity problems. However, during our recent flash sale, we hit "Concurrent Execution" limits that caused our checkout to fail. How do you plan capacity for a system that is supposed to be "Infinite"? Is it just about raising the service limits, or should we be architecturaly "smoothing out" the spikes using SQS queues to manage the flow?
3 answers
Serverless" doesn't mean "unlimited." You always have to plan for your "Regional Concurrency Limit." For flash sales, raising the limit is step one, but the real solution is "Asynchronous Decoupling." We use an SQS queue in front of our Lambda functions. When a burst hits, the messages sit in the queue, and the Lambdas process them at a steady, controlled rate that doesn't trigger the "Throttling" errors. This turns a "Spiky" capacity requirement into a "Flat" one over a slightly longer period. It’s better for the customer to wait 2 seconds for a confirmation than to see a 503 error.
Does your database tier have the same "elastic" capacity as your serverless front-end, or is the DB becoming the real bottleneck during these bursts?
You should also look into 'Provisioned Concurrency' for your critical paths to eliminate any "Cold Start" latency during the initial burst.
Good point, Thomas. Provisioned Concurrency ensures that your "Baseline" capacity is always warm and ready to handle the immediate wave of traffic.
James, you hit the nail on the head. Our RDS instance was red-lining at 100% CPU while the Lambdas were just fine. We are looking into Aurora Serverless v2 to see if it can keep up with the burst, but the "Scaling Speed" of the DB is still slower than the Lambda invocation speed. I think the SQS approach Patricia mentioned might actually save our database as much as our compute tier.