Spatiotemporal reconstruction of emergent flash synchronization in firefly swarms via stereoscopic 360-degree cameras

Raphaël Sarfati, Julie C. Hayes, Élie Sarfati, Orit PelegbioRxiv 2020 · DOI

TL;DR

Researchers have developed a novel 3D reconstruction technique using stereoscopic 360-degree cameras to precisely track firefly flashes in swarms. This method reveals that synchronized flashing, a hallmark of firefly mating, emerges from local interactions and requires a minimum group size, offering new insights into collective behavior and insect communication.

What Problem Does This Paper Address?

Synchronized flashing in firefly swarms is a fascinating natural phenomenon that has inspired numerous mathematical models aiming to explain how global patterns emerge from local interactions. However, a significant challenge has been the lack of detailed, three-dimensional experimental data to validate these models. Previous studies often relied on 2D observations or limited field-of-view cameras, making it difficult to capture the full spatial and temporal dynamics of these swarms, especially given their often dense and extended nature. This paper aims to bridge this gap by providing a method for high-resolution 3D reconstruction of firefly flash occurrences, enabling a deeper understanding of their collective behavior and communication.

Methods

The core of this research lies in a novel 3D reconstruction method utilizing a stereoscopic setup of two 360-degree cameras. Unlike traditional stereoscopic vision that uses planar cameras with limited fields of view, 360-degree cameras offer a full spherical view, allowing them to be placed directly within the swarm.

The methodology involves several key steps:

  • Camera Setup: Two 360-degree cameras (GoPro Fusion) are positioned close to each other, with their relative position and orientation calibrated.
  • Data Acquisition: Footage is captured in the natural habitat of Photinus carolinus fireflies and in a controlled tent environment.
  • Calibration: The precise spatial relationship (translation and rotation) between the two cameras is determined using matched points from calibration targets or observed trajectories. Two methods are proposed: an optimization search and a fundamental matrix approach.
  • Matching: Corresponding flash points from the two camera views are identified using an algorithm that minimizes the distance between potential correspondences.
  • Triangulation: Once matched points and camera poses are established, the 3D coordinates of each flash are calculated by triangulating the corresponding 2D projections from the two cameras. This process allows for the reconstruction of flash occurrences in 3D space.
  • Trajectory Reconstruction: Individual firefly trajectories are inferred by linking consecutive flash occurrences from the same individual over time.

The use of 360-degree cameras is a key innovation, enabling a more comprehensive view of the swarm compared to traditional methods that might only capture a small portion of the activity.

Datasets

The paper utilizes two primary datasets:

  • Natural Habitat Data: Recordings of Photinus carolinus flashing displays in the Great Smoky Mountains National Park (GSMNP). This data was collected in the fireflies' natural environment.
  • Controlled Environment Data: Recordings made within a large tent where a controlled number of Photinus carolinus males were introduced. This allowed for experiments with varying densities of fireflies (1, 5, 15, and 40 individuals).

These datasets are made available for visualization using a provided software tool (FireflyNavigator).

Key Insights and Ideas

  • Emergence of Synchrony: The study demonstrates that synchronized flashing, a complex collective behavior, emerges from local interactions and is not simply a coordinated effort.
  • Minimum Group Size for Periodicity: Periodic flash bursts, a key characteristic of synchronized swarms, only appear when the group size exceeds approximately 15 individuals. This suggests a threshold for the emergence of this specific temporal pattern.
  • Long-Range Correlations: Flash occurrences are correlated over several meters, indicating that interactions are not limited to immediate neighbors, suggesting a broader communication network within the swarm.
  • Behavioral Differentiation: Individual firefly trajectories reveal distinct patterns, hinting at competitive mechanisms and behavioral differentiation. For instance, "early flashers" appear to be more mobile and flash for longer durations than "late followers."
  • Inexpensive and Accessible Technology: The use of readily available 360-degree cameras makes this 3D reconstruction technique accessible for a wider range of researchers studying collective behavior in various species.

Key Results

  • 3D Reconstruction Accuracy: The method achieves sub-centimetric precision for flash localization within a few meters of the cameras.
  • Spatial Distribution: In their natural habitat, firefly flashes are concentrated in a layer approximately 2 meters above the ground.
  • Temporal Patterns:
    • Flash bursts occur at regular interburst intervals (T₀ ≈ 12.5s).
  • Within bursts, flashes occur at a well-defined interflash interval (Tƒ ≈ 0.5s).
  • The characteristic "triangular shape" of flash bursts (slow increase and decrease in flash frequency) was consistently observed, suggesting underlying propagation mechanisms.
  • Controlled Experiments:
    • A single firefly flashes continuously with a regular interflash interval but lacks burst periodicity.
  • With increasing numbers of fireflies, a well-defined interburst frequency emerges, confirming it as an emergent property of collective behavior.
  • The interflash interval (≈ 0.45s) remained consistent across different group sizes.
  • Flight Kinematics: Analysis of individual trajectories revealed a range of motilities and flight patterns, including sharp turns and straight flights, with some evidence of behavioral differentiation.

Limitations and Future Work

  • Stitching Discrepancies: Small stitching errors in 360-degree camera footage can lead to localization "jumps" along FOV edges.
  • Stereographic Distortion: Projections of a sphere onto a plane can cause stretching and reduced accuracy near the poles. The authors note that the projection orientation can be adjusted to mitigate this.
  • Triangulation Resolution: Triangulation accuracy decreases with increasing distance from the cameras and along the line connecting them. However, the authors report good resolution (below 1cm up to 8m away) for their experimental setup.
  • Visual Occlusion: In natural environments, vegetation and other obstacles can lead to visual occlusion, limiting the detection of flashes at larger distances and affecting density estimations.
  • Future Work: The authors suggest that this technique is extensible to studying other firefly species and tracking flight trajectories in other insect swarms. They also highlight the potential for using this data for large-scale firefly density monitoring programs and for developing more sophisticated mathematical models that incorporate the observed behaviors and correlations. Further classification of firefly trajectories using statistical methods is also proposed.

Summary updated

Abstract

During mating season, males of synchronous firefly species flash in unison within swarms of thousands of individuals. These strongly-correlated collective displays have inspired numerous mathematical models to explain how global synchronous patterns emerge from local interactions. Yet, experimental data to validate these models remains sparse. To address this gap, we develop a method for three-dimensional tracking of firefly flashes, using a stereoscopic setup of 360-degree cameras. We apply this method to record flashing displays of the North American synchronous species Photinus carolinus in its natural habitat as well as within controlled environments, and obtain the 3D reconstruction of flash occurrences in the swarm. Our results show that even a small number of interacting males synchronize their flashes; however, periodic flash bursts only occur in groups larger than 15 males. Moreover, flash occurrences are correlated over several meters, indicating long-range interactions. While this suggests emergent collective behaviour and cooperation, we identify distinct individual trajectories that hint at additional competitive mechanisms. These reveal possible behavioural differentiation with early flashers being more mobile and flashing longer than late followers. Our experimental technique is inexpensive and easily implemented. It is extensible to tracking light communication in various firefly species and flight trajectories in other insect swarms.
BibTeX
@misc{sarfati2020spatiotemporal,
  author = {Raphaël Sarfati and Julie C. Hayes and Élie Sarfati and Orit Peleg},
  title = {{Spatiotemporal reconstruction of emergent flash synchronization in firefly swarms via stereoscopic 360-degree cameras}},
  year = {2020},
  doi = {10.1101/2020.03.19.999227},
  howpublished = {bioRxiv},
}

Loading related content...

Figure