TL;DR
CAFE-AQI is a lightweight, "coverage-aware" machine learning framework designed to forecast next-day air quality in Bangladesh while accounting for significant real-world data quirks like duplicate sensor trajectories and sudden network expansions. By combining a dual-head model for regression and early warnings with conformal prediction, it provides accurate forecasts and reliable uncertainty intervals, significantly outperforming standard baselines.
What Problem Does This Paper Address?
Forecasting the Air Quality Index (AQI) is critical for public health, especially in highly polluted regions like Bangladesh. However, real-world environmental data is often "messy." The authors discovered two major issues in existing Bangladesh AQI datasets:
- Data Redundancy: While the dataset lists 103 cities, an audit revealed that many share identical data patterns. In reality, there are only 61 unique AQI trajectories. Treating all 103 as independent leads to "over-representation" during training and overly optimistic evaluation results.
- Coverage Shifts: The monitoring network expanded significantly in August 2022, changing how data was reported.
- Evaluation Pitfalls: Standard "random splits" in time-series data often leak future information into the past or fail to account for spatial dependencies, leading to models that look great on paper but fail in the real world.
Methods
The authors propose CAFE-AQI (Coverage-Aware Day-Ahead AQI), a framework built on a Histogram-based Gradient Boosting model. Its architecture is notable for several reasons:
- Dual-Head Design: Instead of one model doing everything, CAFE-AQI uses two separate "heads." One predicts the exact maximum AQI for the next day (regression), while the other specifically estimates the probability of the air reaching "Unhealthy" levels (AQI $\ge$ 151).
- Duplicate-Aware Weighting: To handle the 61 unique trajectories hidden within 103 city labels, the authors applied weights to the training loss. This ensures that a single trajectory shared by four cities doesn't exert four times the influence of a unique city trajectory.
- Feature Engineering: The model uses 30 features, including causal temporal data (lagged values, rolling averages), pollutant levels (PM2.5, PM10, etc.), and a "five-neighbor" spatial context to capture regional pollution trends.
- Conformal Prediction: To provide a "safety margin," the model uses risk-conditional split-conformal calibration. This generates prediction intervals (e.g., "The AQI will be $160 \pm 20$") that have a guaranteed mathematical coverage, helping users understand the forecast's reliability.
Datasets
The study utilizes a large-scale Bangladesh AQI dataset containing over 3.19 million hourly records spanning from 2000 to 2025.
- Audit Finding: The authors identified a critical transition in August 2022 where the data regime changed.
- Chronological Partitioning: To prevent data leakage, they used a strict temporal split: 2022–2023 for training, 2024 for calibration, and the "untouched" 2025 data for final testing.
Key Insights and Ideas
- The "Random Split" Trap: The authors demonstrated that using a standard random split resulted in an MAE of 6.15, which is 37% lower (better) than the valid chronological result of 9.76. This highlights how easily researchers can be fooled by data leakage if they don't respect temporal order.
- PM2.5 is King: Feature importance audits showed that PM2.5 measurements are the primary drivers of the model's accuracy. Removing PM2.5 features caused the error to spike significantly more than any other feature group.
- Spatial Generalization: Even when an entire geographic block of cities was hidden during training, the model's performance remained robust (MAE of 10.66), suggesting it learns generalizable pollution patterns rather than just memorizing specific locations.
Key Results
On the untouched 2025 test set, CAFE-AQI delivered impressive performance:
- Accuracy: Achieved a Mean Absolute Error (MAE) of 9.76 and an $R^2$ of 0.910, outperforming the "persistence" baseline (simply predicting today's air for tomorrow) by 27.1%.
- Early Warning: The dedicated warning head achieved an $F_1$-score of 0.908 and an AUROC of 0.982 for predicting unhealthy air conditions.
- Uncertainty: The conformal intervals provided 92.09% coverage, successfully capturing the true AQI within the predicted range the vast majority of the time.
Limitations and Future Work
While CAFE-AQI is highly effective, the authors note several areas for improvement:
- Data Source Dependency: The model was trained and tested on a single data product. Future work should involve validation against independent, ground-truth physical monitoring stations.
- Missing Variables: The current model does not incorporate meteorological data (wind speed, humidity) or specific emission data (factory locations, traffic patterns), which could further refine forecasts during sudden pollution events.
- Explainability vs. Causality: While the model identifies which features are important, it doesn't necessarily explain the underlying physical causes of pollution spikes.
Summary updated