Vision-Language Models Meet Meteorology: Developing Models for Anomalies Analysis with Heatmaps
Its authors withdrew this paper, so there is no full text to read or summarize.
Lists
Only you see this
Abstract
Real-time analysis and prediction of meteorological anomalies protect human lives and infrastructure. Traditional methods rely on numerical threshold setting and manual interpretation of weather heatmaps with Geographic Information Systems (GIS), which can be slow and error-prone. Our research redefines Meteorological Anomalies Analysis(MAA) by framing it as a Visual Question Answering (VQA) problem, thereby introducing a more precise and automated solution. Leveraging Vision-Language Models (VLM) to simultaneously process visual and textual data, we offer an effective aid to enhance the analysis process of weather heatmaps. Our initial assessment of general-purpose VLMs (e.g., GPT-4-Vision) on MAA revealed poor performance, characterized by low accuracy and frequent hallucinations due to inadequate color differentiation and insufficient meteorological knowledge. To address these challenges, we introduce ClimateIQA, the first meteorological VQA dataset, which includes 8,760 wind gust heatmaps and 254,040 question-answer pairs covering four question types, both generated from the latest climate reanalysis data. We also propose Sparse Position and Outline Tracking (SPOT), an innovative technique that leverages OpenCV and K-Means clustering to capture and depict color contours in heatmaps, providing ClimateIQA with more accurate color spatial location information. Finally, we present Climate-Zoo, the first meteorological VLM collection, which adapts VLMs to meteorological applications using the ClimateIQA dataset. Experiment results demonstrate that models from Climate-Zoo substantially outperform state-of-the-art general VLMs, achieving an accuracy increase from 0\% to over 90% in MAA verification.
BibTeX
@misc{chen2025vision,
author = {Jian Chen and Peilin Zhou and Yining Hua and Dading Chong and Meng Cao and Yaowei Li and Zixuan Yuan and Bing Zhu and Junwei Liang},
title = {{Vision-Language Models Meet Meteorology: Developing Models for Anomalies Analysis with Heatmaps}},
year = {2025},
url = {https://openreview.net/forum?id=kWELXBKl54},
howpublished = {OpenReview},
note = {Submitted to ICLR 2025},
}