Our AWS bill for CloudWatch is spiraling because we are publishing custom metrics for every single user session. We use the PutMetricData API call in our Python backend. Are there any best practices for batching these calls or using alternative methods like the Embedded Metric Format (EMF) to save money?
3 answers
You should definitely look into the CloudWatch Embedded Metric Format (EMF). Instead of making an expensive synchronous API call every time you want to record a metric, you simply print a JSON object to your logs in a specific format. The CloudWatch service then extracts these metrics from your logs automatically at no extra "per-call" cost beyond standard logging fees. This is significantly cheaper than calling PutMetricData thousands of times per hour. If you must use the API, ensure you are batching up to 20 metrics or 150 data points per call, which is the maximum limit allowed by AWS for a single request.
Do you find that EMF adds too much latency to your application logs, or is the asynchronous nature of the metric extraction working well for your dashboarding needs?
If you are on EC2, you can also use the CloudWatch Agent to aggregate metrics locally before sending them, which is much more efficient than individual SDK calls.
I agree with Linda. The agent handles all the batching and retries for you, so your application code stays clean and doesn't need to manage the logic of metric buffering.
Michael, the latency impact is negligible because writing to stdout is extremely fast. The real trade-off is the slight delay in the metric appearing on the dashboard. It’s not "instant" like PutMetricData, but for most business KPIs and session tracking, a 30-second lag is a small price to pay for the massive reduction in API request charges.