HumBug Zooniverse: a crowd-sourced acoustic mosquito dataset

Ivan Kiskin, Adam D. Cobb, Lawrence Wang, Stephen Roberts

Abstract

Mosquitoes are the only known vector of malaria, which leads to hundreds of thousands of deaths each year. Understanding the number and location of potential mosquito vectors is of paramount importance to aid the reduction of malaria transmission cases. In recent years, deep learning has become widely used for bioacoustic classification tasks. In order to enable further research applications in this field, we release a new dataset of mosquito audio recordings. With over a thousand contributors, we obtained 195,434 labels of two second duration, of which approximately 10 percent signify mosquito events. We present an example use of the dataset, in which we train a convolutional neural network on log-Mel features, showcasing the information content of the labels. We hope this will become a vital resource for those researching all aspects of malaria, and add to the existing audio datasets for bioacoustic detection and signal processing.

Introduction

Malaria remains one of the world’s deadliest public health challenges, claiming hundreds of thousands of lives annually. Effective "vector control"—managing the mosquito populations that spread the disease—is the primary defense, but it requires accurate surveillance data. We need to know where the mosquitoes are and which species are present.

While automated acoustic detection (identifying mosquitoes by their unique flight tones) is a promising solution, it faces a major bottleneck: a lack of high-quality, labeled data. Existing datasets often rely on expensive optical sensors that don't translate well to the microphones found in cheap mobile phones. Furthermore, labeling thousands of hours of audio is a Herculean task for a small team of experts.

Methods: Citizen Science and Deep Learning

The authors turned to citizen science to bridge the labeling gap. They hosted their data on the Zooniverse platform, where over 1,300 volunteers listened to two-second audio clips and viewed their corresponding spectrograms. Volunteers classified clips into three categories: {Yes, No, Not Sure} regarding the presence of a mosquito.

The Zooniverse interface used by volunteers to classify mosquito sounds based on audio and spectrograms.

To ensure accuracy, the audio clips were designed with a 50% overlap (1 second), meaning every segment of audio was reviewed by multiple people. To demonstrate the dataset's utility, the researchers implemented a baseline Convolutional Neural Network (CNN). They transformed the raw audio into log-Mel spectrograms—a standard feature in audio processing that represents sound in a way similar to human hearing—and trained the model to distinguish mosquito "buzz" from background noise.

The HumBug Zooniverse Dataset

The primary contribution is the HumBug Zooniverse dataset. It consists of:

  • 195,434 labels covering 80,101 unique audio segments.
  • Four distinct sources (Groups A-D): Laboratory colonies from the UK (Culex), Kenya, and the USA (CDC), plus wild-captured mosquitoes from Thailand.
  • Diverse species: Includes major malaria vectors like Anopheles gambiae and Anopheles stephensi, as well as vectors for West Nile, Yellow Fever, and Dengue.
Distribution of labels (Mosquito, Not Sure, Noise) across the four distinct data collection groups.

The paper contrasts this with the existing "Wingbeats" dataset. While Wingbeats is larger, it uses optical sensors; HumBug uses conventional microphones, making it more applicable for real-world deployment on mobile devices in resource-constrained settings.

Key Insights

  • The Power of the Crowd: The study proves that non-experts can effectively label bioacoustic data. Even with "noisy" labels from volunteers, the sheer volume of data allows for robust model training.
  • Handling Uncertainty: The researchers dealt with "Not Sure" labels by assigning them a numerical value (0.5) and using a majority voting scheme to create a "ground truth" for the baseline model.
  • The Imbalance Challenge: In real-world recordings, mosquitoes are rare. Only about 10% of the dataset contained mosquito events. Addressing this class imbalance (e.g., through weighted loss functions) is critical for any practical detection system.
  • Overlap for Reliability: By overlapping clips by one second, the researchers ensured that even if a mosquito buzz occurred at the very edge of a clip, it would be captured and labeled in the subsequent segment.

Results and Performance

  • Scale: The project generated a 22-hour dataset of unique, labeled audio.
  • Baseline Performance: Using equal class weights, the CNN baseline achieved a 39% detection rate (True Positive Rate) for mosquito events with only a 2% false alarm rate (False Positive Rate).
  • Tunability: By adjusting the weights in the loss function (weighting the mosquito class 10x more than noise), the researchers showed they could significantly increase the detection rate, though this comes at the cost of more false positives (increasing from 2% to ~15%).

Limitations and Future Work

The authors acknowledge that the current dataset, while large, is just the beginning. The "Not Sure" labels and disagreements between volunteers (found in about 18% of multi-labeled clips) suggest that some acoustic environments are still very challenging.

Future work involves expanding the dataset with more species-specific information and more detailed metadata regarding the recording devices used. The ultimate goal is to integrate these models into a global, real-time mosquito surveillance network powered by the smartphones already in people's pockets.

Summary updated

Abstract

Mosquitoes are the only known vector of malaria, which leads to hundreds of thousands of deaths each year. Understanding the number and location of potential mosquito vectors is of paramount importance to aid the reduction of malaria transmission cases. In recent years, deep learning has become widely used for bioacoustic classification tasks. In order to enable further research applications in this field, we release a new dataset of mosquito audio recordings. With over a thousand contributors, we obtained 195,434 labels of two second duration, of which approximately 10 percent signify mosquito events. We present an example use of the dataset, in which we train a convolutional neural network on log-Mel features, showcasing the information content of the labels. We hope this will become a vital resource for those researching all aspects of malaria, and add to the existing audio datasets for bioacoustic detection and signal processing.
BibTeX
@inproceedings{kiskin2020humbug,
  author = {Ivan Kiskin and Adam D. Cobb and Lawrence Wang and Stephen Roberts},
  title = {{HumBug Zooniverse: a crowd-sourced acoustic mosquito dataset}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=d6MiFEpm05Q},
}
Figure