I am setting up custom metrics for a high-frequency trading application. I need to decide between standard 1-minute resolution and high-resolution 1-second metrics. Aside from the obvious cost difference, are there specific limitations on how long high-resolution data is retained before it gets aggregated into larger time buckets?
3 answers
The decision really depends on your required recovery time objective. High-resolution metrics allow you to see sub-minute spikes that standard 60-second monitoring would simply smooth over as an average. However, you must remember that high-resolution data (anything with a period less than 60 seconds) is only kept for 3 hours. After that, CloudWatch aggregates it. If you need to perform a post-mortem on an incident that happened yesterday, you’ll be looking at 1-minute averages regardless of how granular your initial push was. For a trading app, the immediate visibility is worth the cost, but for general web traffic, it’s often overkill.
That is a great point about retention, but have you considered how high-resolution metrics affect your alarm evaluation periods?
Stick with standard resolution for 90% of your metrics to keep costs down. Only use high-resolution for your most critical "Golden Signals" like latency and error rates.
Jennifer is right. I once accidentally set all our application heartbeats to 1-second resolution and the bill at the end of the month was a very unpleasant surprise for our finance team!
Thomas, high-resolution alarms are actually a huge benefit here. Because you can evaluate the metric every 10 or 30 seconds, your "Time to Alert" drops significantly. Instead of waiting 3 to 5 minutes for a standard alarm to trip, you can trigger a Lambda function or an SNS notification in under a minute, which is critical for the trading environment Robert mentioned.