CAFE-AQI: Coverage-Aware Day-Ahead AQI Forecasting with Early Warning and Conformal Uncertainty in Bangladesh

Md Rafid Islam, Rafsan Jany, Rafid Bin Zaman, Mohammad Fahim Islam, Hafiz Abdur RahmanarXiv 2026 · arXiv

TL;DR

CAFE-AQI is a lightweight, "coverage-aware" machine learning framework designed to forecast next-day air quality in Bangladesh while accounting for significant real-world data quirks like duplicate sensor trajectories and sudden network expansions. By combining a dual-head model for regression and early warnings with conformal prediction, it provides accurate forecasts and reliable uncertainty intervals, significantly outperforming standard baselines.

What Problem Does This Paper Address?

Forecasting the Air Quality Index (AQI) is critical for public health, especially in highly polluted regions like Bangladesh. However, real-world environmental data is often "messy." The authors discovered two major issues in existing Bangladesh AQI datasets:

  1. Data Redundancy: While the dataset lists 103 cities, an audit revealed that many share identical data patterns. In reality, there are only 61 unique AQI trajectories. Treating all 103 as independent leads to "over-representation" during training and overly optimistic evaluation results.
  2. Coverage Shifts: The monitoring network expanded significantly in August 2022, changing how data was reported.
  3. Evaluation Pitfalls: Standard "random splits" in time-series data often leak future information into the past or fail to account for spatial dependencies, leading to models that look great on paper but fail in the real world.

Methods

The authors propose CAFE-AQI (Coverage-Aware Day-Ahead AQI), a framework built on a Histogram-based Gradient Boosting model. Its architecture is notable for several reasons:

  • Dual-Head Design: Instead of one model doing everything, CAFE-AQI uses two separate "heads." One predicts the exact maximum AQI for the next day (regression), while the other specifically estimates the probability of the air reaching "Unhealthy" levels (AQI $\ge$ 151).
  • Duplicate-Aware Weighting: To handle the 61 unique trajectories hidden within 103 city labels, the authors applied weights to the training loss. This ensures that a single trajectory shared by four cities doesn't exert four times the influence of a unique city trajectory.
  • Feature Engineering: The model uses 30 features, including causal temporal data (lagged values, rolling averages), pollutant levels (PM2.5, PM10, etc.), and a "five-neighbor" spatial context to capture regional pollution trends.
  • Conformal Prediction: To provide a "safety margin," the model uses risk-conditional split-conformal calibration. This generates prediction intervals (e.g., "The AQI will be $160 \pm 20$") that have a guaranteed mathematical coverage, helping users understand the forecast's reliability.
Fig. 1. CAFE-AQI workflow. The task-specific models are trained on 2022–2023 data, calibrated on 2024, and evaluated on the untouched 2025 test set.

Datasets

The study utilizes a large-scale Bangladesh AQI dataset containing over 3.19 million hourly records spanning from 2000 to 2025.

  • Audit Finding: The authors identified a critical transition in August 2022 where the data regime changed.
  • Chronological Partitioning: To prevent data leakage, they used a strict temporal split: 2022–2023 for training, 2024 for calibration, and the "untouched" 2025 data for final testing.
The chronological data split used to ensure valid, leakage-resistant evaluation.

Key Insights and Ideas

  • The "Random Split" Trap: The authors demonstrated that using a standard random split resulted in an MAE of 6.15, which is 37% lower (better) than the valid chronological result of 9.76. This highlights how easily researchers can be fooled by data leakage if they don't respect temporal order.
  • PM2.5 is King: Feature importance audits showed that PM2.5 measurements are the primary drivers of the model's accuracy. Removing PM2.5 features caused the error to spike significantly more than any other feature group.
  • Spatial Generalization: Even when an entire geographic block of cities was hidden during training, the model's performance remained robust (MAE of 10.66), suggesting it learns generalizable pollution patterns rather than just memorizing specific locations.
Feature importance analysis showing PM2.5 as the dominant predictor for AQI.

Key Results

On the untouched 2025 test set, CAFE-AQI delivered impressive performance:

  • Accuracy: Achieved a Mean Absolute Error (MAE) of 9.76 and an $R^2$ of 0.910, outperforming the "persistence" baseline (simply predicting today's air for tomorrow) by 27.1%.
  • Early Warning: The dedicated warning head achieved an $F_1$-score of 0.908 and an AUROC of 0.982 for predicting unhealthy air conditions.
  • Uncertainty: The conformal intervals provided 92.09% coverage, successfully capturing the true AQI within the predicted range the vast majority of the time.
Fig. 4. Coverage and interval width across true AQI categories.

Limitations and Future Work

While CAFE-AQI is highly effective, the authors note several areas for improvement:

  • Data Source Dependency: The model was trained and tested on a single data product. Future work should involve validation against independent, ground-truth physical monitoring stations.
  • Missing Variables: The current model does not incorporate meteorological data (wind speed, humidity) or specific emission data (factory locations, traffic patterns), which could further refine forecasts during sudden pollution events.
  • Explainability vs. Causality: While the model identifies which features are important, it doesn't necessarily explain the underlying physical causes of pollution spikes.

Summary updated

Abstract

Next-day air quality forecasting often assumes city records are equally complete and independent. However, an audit of a large Bangladesh dataset revealed a sharp coverage expansion after August 2022 and showed that 103 city labels correspond to only 61 unique AQI trajectories. To address these issues, this paper presents CAFE-AQI, a coverage-aware framework for forecasting next-day maximum AQI. It also estimates the probability of Unhealthy-or-worse conditions (AQI >= 151). Data were divided chronologically into training (2022-2023), calibration (2024), and testing (2025). To prevent duplicate records from dominating training and evaluation, identical trajectories were weighted. The framework combines temporal and pollutant features with a five-neighbor spatial context. It also includes a class-balanced warning head and risk-conditional conformal calibration aimed at providing early warnings and prediction intervals. Evaluation on the duplicate-balanced 2025 data gave an MAE of 9.76, an RMSE of 15.04, and an R-squared value of 0.910 for CAFE-AQI. Its MAE was 3.63 AQI lower than persistence (95% CI: 3.49-3.76), with lower errors in each of the 61 trajectory groups. For unhealthy-air warnings, the F1-score was 0.908 and the AUROC was 0.982. The nominal 90% intervals covered 92.09% of observations. When an entire spatial block was excluded from training, the MAE remained 10.66, showing that the lightweight model transferred reasonably well to unseen locations.
BibTeX
@misc{islam2026cafe,
  author = {Md Rafid Islam and Rafsan Jany and Rafid Bin Zaman and Mohammad Fahim Islam and Hafiz Abdur Rahman},
  title = {{CAFE-AQI: Coverage-Aware Day-Ahead AQI Forecasting with Early Warning and Conformal Uncertainty in Bangladesh}},
  year = {2026},
  eprint = {2611.23815},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23815},
}

Loading related content...

Figure