We are debating migrating our microservices from EKS to AWS Lambda. While the "pay-per-use" model sounds great, I am worried about the cost of Lambda at scale and the dreaded "Cold Start" issues affecting our API response times. Is serverless actually viable for high-throughput apps?
3 answers
Serverless is highly cost-effective until you hit a certain "inflection point" of constant high traffic. For a steady-state API with thousands of requests per second, EKS (Kubernetes) is almost always cheaper because you aren't paying the premium for the abstraction. However, for "bursty" workloads, Lambda is unbeatable. To mitigate cold starts, use "Provisioned Concurrency," though this adds to the cost. If your team is small, the operational savings of not managing a cluster often outweigh the raw compute costs. You need to do a total cost of ownership (TCO) analysis that includes the salary of the DevOps engineers required to maintain EKS.
What programming language are you using? Java and .NET have much longer cold starts on Lambda compared to Go or Python, which might influence your decision.
Don't forget the API Gateway costs. Sometimes the gateway itself ends up costing more than the Lambda executions when you are dealing with millions of small requests.
Excellent point, Scott. People often hyper-focus on the compute cost and forget that networking and gateway fees can become the majority of the serverless bill.
We are primarily a Java shop. That's a major concern for us, as our startup times are already slow. We might look into GraalVM native images to see if that helps before we commit.