Reconstructing illusory camouflage patterns on moth wings using computer vision

Laurent Valentin Jospin, John N. Porter, Farid Boussaïd, Mohammed Bennamoun, Jennifer L. KelleyJournal of The Royal Society Interface 2025 · DOI

TL;DR

This paper explores how moth wing patterns might create illusory depth cues to fool predators, using computer vision to reconstruct these patterns. By applying both traditional intrinsic image decomposition and modern deep learning methods, the researchers found that different algorithms interpret these patterns differently, highlighting the role of prior experience in visual perception.

What Problem Does This Paper Address?

Animals often use camouflage to avoid predators. One fascinating strategy is to create the illusion of depth or shape where none truly exists, a phenomenon known as "pictorial cues." This can make an animal appear more three-dimensional, potentially disrupting predator recognition. While humans are known to be susceptible to such visual illusions, understanding how non-human predators, like birds hunting moths, perceive these cues is incredibly challenging. This paper tackles this by using computer vision, a field adept at reconstructing 3D information from 2D images, to analyze and reconstruct the potential depth information encoded in moth wing patterns, aiming to understand how these patterns might deceive a predator's visual system.

Methods

The researchers employed two primary computer vision approaches for monocular depth estimation (reconstructing depth from a single image):

  • Intrinsic Image Decomposition (Closed-form solution): This method, based on the Retinex theory, aims to separate an image into its reflectance (the inherent color of the surface) and shading (how light falls on the surface) components. By assuming a single light source and applying constraints for smoothness and edge orientation, they can estimate surface normals and then reconstruct a height map, effectively a depth map. This approach is sensitive to both real depth cues and high-contrast patterns.
  • Deep Learning Models: They utilized several state-of-the-art deep learning architectures for monocular depth estimation, including BinsFormer, BTS, AdaBins, and DepthFormer. These models learn from vast datasets of 3D scenes to predict depth, implicitly incorporating prior knowledge about how objects typically appear in the real world.

Additionally, for one species, they performed a Multi-view Three-Dimensional Reconstruction using the Meshroom software. This technique uses multiple images of the same object from different viewpoints to create a more accurate 3D model, allowing them to determine the "true" flat shape of the moth's wing and compare it to the illusory depth perceived from a single view.

Datasets

The study used two types of datasets:

  • Natural Three-Dimensional Surfaces: Images of natural objects like curled mandarin leaves, marri bark, damaged bark, and eucalypt bark were used to test how the computer vision algorithms performed on real-world textures and shapes that moth patterns might mimic.
  • Moth Wing Patterns: Images of six species of moths (Eudocima salaminia, Anthela sp., Spodoptera litura, Syneora sp., Dasypodia selenophora, and Ardices glatignyi) were used. These species were selected for their varying degrees of potential pictorial depth cues. Additional moth images were used for further exploration.

The paper does not explicitly mention the creation of a new benchmark dataset for training or evaluation, but rather uses existing moth images and natural surfaces to evaluate the performance of established computer vision techniques.

Key Insights and Ideas

  • Computer Vision as a Proxy for Predator Vision: The study demonstrates that computer vision algorithms can serve as a valuable tool to approximate how animal visual systems might interpret complex patterns, offering a way to study visual illusions in non-human contexts.
  • Differential Sensitivity of Algorithms: The Retinex-based closed-form solution is sensitive to high-contrast patterns, reconstructing them as depth variations. In contrast, deep learning models, trained on more general scene data, tend to smooth over these high-contrast features, only responding to patterns with strong, inherent pictorial depth cues.
  • The Role of Prior Experience: The difference in how the two types of algorithms interpret moth patterns highlights the importance of prior experience. Deep learning models, having learned from vast datasets, may have implicit priors that help them distinguish between genuine depth cues and patterns that merely mimic depth.
  • Illusory Depth vs. True Shape: The multi-view reconstruction of Eudocima salaminia revealed that the wing surface is physically flat, starkly contrasting with the perceived 3D curvature suggested by its wing pattern in single-image views. This underscores the "illusory" nature of the depth cues.
  • Camouflage Strategies Beyond Crypsis: The research touches upon how patterns might achieve camouflage not just by blending in (crypsis) but by mimicking inedible objects (masquerade) or by creating deceptive visual information.

Key Results

  • Closed-form solution: Reconstructed depth maps that were consistent with real surface relief and also showed depth variations for high-contrast moth wing patterns, even where no actual depth existed. This method was sensitive to both natural textures and moth patterns.
  • Deep learning models: Showed more variable results. For natural surfaces, they performed well on whole objects but struggled with fine textures. When applied to moth wings, they generally produced smooth depth maps, with noticeable depth variations only for moths with very strong pictorial depth cues (like E. salaminia and Uropyia meticulodina).
  • Multi-view reconstruction: Confirmed that the wings of E. salaminia are physically flat, demonstrating that the perceived depth from single images is indeed an illusion.
  • Bird Vision Adaptation: Calibrating images for bird vision did not significantly alter the reconstruction results from the closed-form model, suggesting that for these specific algorithms and patterns, the fundamental depth cues are interpreted similarly across RGB and simulated bird vision.

Limitations and Future Work

  • Generalizability of Deep Learning Models: The deep learning models showed a tendency to over-smooth patterns, particularly those with fine-scale textures, likely due to being trained on datasets that don't extensively feature such detailed animal camouflage. This limits their ability to reconstruct the fine illusory depth cues in many moth patterns.
  • Lack of Predator Experience in Models: The current deep learning models are trained on general RGB images and do not explicitly incorporate the specific visual priors or "experience" a predator might have with certain objects or patterns.
  • Behavioral Validation: The authors acknowledge that these computer vision approaches cannot replace empirical behavioral studies with real animals.
  • Future Work:
    • Incorporate images of moths photographed under different illumination conditions and backgrounds.
  • Extend the analysis to a broader range of species and camouflage functions (e.g., signaling).
  • Combine computational models with behavioral studies using 3D-printed reconstructions.
  • Investigate the influence of environmental factors like shadows and background visual cues on depth perception.
  • Develop specialized datasets for training deep learning models that better represent animal camouflage patterns.
  • Explore incorporating predator-specific priors into deep learning models.

Summary updated

Abstract

Monocular depth cues, such as shading, are fundamental for resolving three-dimensional information, such as an object's shape. Animal colour patterns may potentially exploit this mechanism of depth perception, generating false illusions for functions such as camouflage. Reconstructing the potential percept produced by false depth cues is challenging, especially for non-human, animal viewers. Here, we provide a novel step towards solving this problem, taking advantage of state-of-the-art computer vision algorithms typically used for three-dimensional scene reconstruction. We used two approaches for single-image monocular depth estimation: intrinsic image decomposition and deep learning. We first examined how these models performed using images of natural three-dimensional surfaces that moth wing patterns may mimic. We then applied these models to the wing patterns of six species of moth (Lepidoptera) with varying amounts of potential depth information. For one species, we then performed a multi-view reconstruction of the wing pattern to reveal the true (flat) wing shape. Intrinsic image decomposition, which is based on Retinex theory, was sensitive to both real depth cues and high contrast patterns, while the deep-learning models only responded to moths with strong pictorial depth cues. Both approaches reveal how the interpretation of visual cues depends not only on the information available, but also on experience with the natural world.
BibTeX
@article{jospin2025reconstructing,
  author = {Laurent Valentin Jospin and John N. Porter and Farid Boussaïd and Mohammed Bennamoun and Jennifer L. Kelley},
  title = {{Reconstructing illusory camouflage patterns on moth wings using computer vision}},
  journal = {Journal of The Royal Society Interface},
  year = {2025},
  doi = {10.1098/rsif.2024.0757},
}
Figure