Why CUR-Based Cost Analysis Beats Cost Explorer
Why CUR-Based Cost Analysis Beats Cost Explorer
When building Guardian Pro's cost intelligence module, we made a deliberate architectural decision: use AWS Cost and Usage Reports (CUR) as the primary data source instead of the Cost Explorer API. Here is why.
The Cost Explorer Limitation
The AWS Cost Explorer API is the most common starting point for cost analysis tools. It is convenient -- a few API calls and you have cost data. But it comes with significant limitations:
- Aggregated data only: Cost Explorer returns pre-aggregated data. You cannot drill down to individual resource costs without significant filtering gymnastics.
- API rate limits: The Cost Explorer API has strict throttling (5 requests per second). At scale, this becomes a bottleneck.
- Limited granularity: Daily or monthly granularity only. No hourly breakdowns without CloudWatch integration.
- 14-month retention: Historical data beyond 14 months requires separate archival.
- Cost: Each Cost Explorer API request costs $0.01. For a platform scanning multiple accounts continuously, this adds up.
The CUR Advantage
Cost and Usage Reports deliver line-item billing data as Parquet files to S3. Guardian Pro syncs this data into DynamoDB for fast, indexed queries.
Resource-Level Granularity
CUR data includes individual resource IDs, usage types, and pricing details. When Guardian Pro tells you an EC2 instance costs a specific amount per day, that figure comes from actual billing line items, not estimates.
Correlation with Security Findings
Because we have resource-level cost data, Guardian Pro can answer questions like: "How much are we spending on resources with critical security findings?" This is the foundation of our Infrastructure Cost of Exposure (ICoE) analysis.
No API Rate Limits
CUR files are delivered to S3 on a schedule. Guardian Pro's sync process pulls them into DynamoDB without competing for API rate limits. This means consistent performance regardless of how many accounts you monitor.
Full Historical Data
CUR data persists in S3 indefinitely (subject to your retention policies). Guardian Pro maintains daily cost records in DynamoDB, enabling long-term trend analysis and forecasting.
Guardian Pro's 8-Step Pipeline
Our cost analysis uses a Step Function pipeline that processes CUR data through 8 stages:
- Initialise -- Validate that CUR is active and has synced data.
- Fetch Data -- Query the cost-resource-daily DynamoDB table for the analysis period.
- Process Data -- Generate service breakdowns, region breakdowns, trends, and cost drivers.
- Generate Recommendations -- Parallel analysis for reserved instances, savings plans, rightsizing, and idle resources.
- Check Budgets -- Evaluate spend against defined budgets with 80% warning and 100% critical thresholds.
- CUR Sync -- Sync latest Parquet files from S3 to DynamoDB.
- Detect Anomalies -- Run four complementary anomaly detection methods.
- Save Results -- Consolidate everything into the analysis table and broadcast results via WebSocket.
Anomaly Detection
Guardian Pro runs four complementary detection methods:
- Z-score analysis: Statistical deviation detection with a 2-sigma threshold. Catches gradual drift.
- Day-over-day: Triggers on >50% daily increases. Catches sudden spikes.
- Service-level: Compares individual service costs against historical baselines. Catches service-specific issues.
- New service detection: Alerts when a previously unused AWS service appears in billing. Catches shadow IT.
The Trade-Off
CUR-based analysis requires initial setup: you need to configure a Cost and Usage Report in your AWS account and wait for the first delivery (up to 24 hours). Guardian Pro's onboarding wizard guides you through this process, and our CUR status monitoring shows you exactly where you are in the setup flow.
The setup investment pays for itself immediately. The depth and accuracy of CUR-based analysis is in a different league from Cost Explorer API responses.
Conclusion
If you are evaluating cost management tools, ask what data source they use. Cost Explorer is fine for a quick dashboard glance, but for production cost governance -- anomaly detection, resource-level attribution, security-cost correlation -- CUR is the only serious option.
Guardian Pro's cost intelligence is built on this foundation from the ground up. Start your free trial to see the difference.