Tracking Phenological Status and Ecological Interactions in a Hawaiian Cloud Forest Understory using Low-Cost Camera Traps and Visual Foundation Models

Luke Meyers, Anirudh Potlapally, Yuyan Chen, Mike Long, Tanya Berger-Wolf, Hari Subramoni, Remi Megret, Daniel RubensteinarXiv 2026 · arXiv

TL;DR

This paper introduces a zero-shot, annotation-free computer vision pipeline that pairs low-cost camera traps with visual foundation models to track plant phenology and animal interactions simultaneously. By combining models like DepthPro and BioCLIP with traditional color-space segmentation, the framework achieves individual-level monitoring accuracy comparable to intensive in-person field surveys while capturing rapid ecological interactions that conventional methods miss.

What Problem Does This Paper Address?

Understanding plant phenology—the timing of life cycle events like leafing, flowering, and fruiting—is critical for tracking ecosystem responses to climate change. While temperate zones are well-monitored, tropical ecosystems remain heavily understudied due to their complexity and density.

Currently, networks like the National Ecological Observatory Network (NEON) rely on biweekly in-person surveys or fixed-interval "pheno-cams." However, these methods are labor-intensive, lack the temporal resolution to catch rapid transitions, and fail to capture fine-scale plant-animal interactions (like pollination or seed dispersal) at the individual plant level. Camera traps offer a high-frequency alternative, but they generate massive, irregular datasets that typically require expensive, manual expert annotation to analyze.

Methods

To bypass the need for custom-trained models and manual labeling, the authors developed a hybrid pipeline combining zero-shot foundation models with lightweight, traditional computer vision techniques.

The physical deployment uses a dual-mount system: one camera on a tripod at chest height to capture the canopy and visiting birds, and another on a cinder block at ground level to track terrestrial fauna.

Fig. 1: Top Left: Crop showing an ’amakihi (Chlorodrepanis virens) visiting an ohelo flower. Bottom Left: An ’omao (Myadestes obscu- rus) consuming a pukiawe berry. Right: Camera-trap setup—tripod and cinderblock-mounted—targeting a Rubus hawaiiensis individual along NEON’s phenology transect. Below: Mini-overview of general workflow.

The analysis pipeline consists of three main components:

  1. Canopy Leafing Estimation: To isolate a target plant (e.g., Vaccinium calycinum) from a dense tropical background, the pipeline uses DepthPro for zero-shot monocular depth estimation. A distance threshold isolates the foreground plant. The isolated pixels are converted to the LAB color space, where the 'a' channel segments green foliage to calculate a greenness score over time. Anomalies are filtered using DBSCAN.
  2. Berry Production Tracking: Pre-trained deep learning models struggle to detect tiny fruits like pukiawe berries. The authors instead used a dual color-space segmentation approach (HSV and LAB thresholding) targeted at red-hue pixels, followed by contour matching to count berries per frame.
  3. Avian Visitor Detection: The authors evaluated three zero-shot object detection models—MegaDetector, Grounded DINO, and OWLv2—prompted to find "birds." Detections were passed to BioCLIP (leveraging its TreeOfLife classifier) to filter out false positives and classify avian visitors down to the class level (Aves).
Fig. 2: Canopy Leaf Cover Estimation (a) Raw image of the focal plant against a complex background, making individual leaf counts difficult. (b) Depth map from DepthPro; applying a 2 m thresh- old isolates the plant foreground. (c) Extracted foreground showing only the focal plant with background removed. (d) Segmented leaves obtained by converting the foreground to LAB space and isolating green-chromaticity pixels.

Datasets

The framework was deployed and evaluated using data from NEON Domain 20 at the Pu‘u Maka‘ala Natural Area Reserve in Hawaii:

  • Total Dataset: 12,484 images collected between January 27 and April 3, tracking ten individual plants.
  • Evaluation Subset: A manually annotated subset of 4,625 images from four cameras containing 221 confirmed bird visits was used to benchmark the zero-shot detection models.
  • Ground Truth Validation: The pipeline's phenological trends were validated against NEON’s official biweekly manual field observations.

Key Insights and Ideas

  • Foundation Models Eliminate the Annotation Bottleneck: Off-the-shelf vision models can perform complex tasks like depth-based foreground isolation and taxonomic filtering out-of-the-box, making automated ecological monitoring accessible without massive domain-specific training sets.
  • The Power of Hybrid Pipelines: Foundation models excel at macro tasks (like depth estimation and object detection), but traditional pixel-level techniques (like color-space thresholding) are still superior and more efficient for fine-grained tasks like counting tiny berries.
  • Revealing Ecological Cause-and-Effect: High-frequency camera data allows researchers to link animal behavior directly to plant phenology. For instance, the data revealed that visits from the ’oma’o bird peaked precisely as berry abundance leveled off, directly capturing the consumption and subsequent decline of the fruit.

Key Results

  • Phenology Tracking Accuracy: The automated leaf-cover estimates closely mirrored NEON’s manual field records, capturing the sharp density increase from late January to late February and its stabilization in March ($R^2 \approx 0.91 - 0.97$).
  • Zero-Shot Detection Performance: For bird detection, OWLv2 combined with BioCLIP and low-level motion filtering yielded the best overall balance, achieving an F1 score of 0.786 (Precision: 0.873, Recall: 0.715).
  • Recall vs. Precision Trade-offs: Grounding DINO achieved the highest raw recall (0.855) but suffered from high false-positive rates before being cleaned up by BioCLIP filtering.

Limitations and Future Work

Despite the pipeline's success, the authors noted several challenges:

  • Taxonomic Bias in Missed Detections: The detection models missed significantly more ’amakihi than ’apapane birds, likely due to the ’amakihi's darker plumage blending into the foliage. This kind of systematic error can skew downstream ecological conclusions if not accounted for.
  • Spatial Bias from Fixed Views: A single camera trap only captures a partial view of a plant. The authors observed that their automated greenness curves plateaued slightly before manual records, likely because the camera was focused on a faster-developing section of the plant.

Future work will explore deploying multiple cameras per plant and using randomized orientations to build a less biased, whole-plant phenological profile.

Summary updated

Abstract

Plant phenology, the study of cyclical events such as leafing out, flowering, or fruiting, has wide ecological impacts but is broadly understudied, especially in the tropics. Image analysis has greatly enhanced remote phenological monitoring, yet capturing phenology at the individual level remains challenging. In this project, we deployed low-cost, animal-triggered camera traps at the Pu'u Maka'ala Natural Area Reserve in Hawaii to simultaneously document shifts in plant phenology and flora-faunal interactions. Using a combination of foundation vision models and traditional computer vision methods, we measure phenological trends from images comparable to on-the-ground observations without relying on supervised learning techniques. These temporally fine-grained phenology measurements from camera-trap images uncover trends that coarser traditional sampling fails to detect. When combined with detailed visitation data detected from images, these trends can begin to elucidate drivers of both plant phenology and animal ecology.
BibTeX
@misc{meyers2026tracking,
  author = {Luke Meyers and Anirudh Potlapally and Yuyan Chen and Mike Long and Tanya Berger-Wolf and Hari Subramoni and Remi Megret and Daniel Rubenstein},
  title = {{Tracking Phenological Status and Ecological Interactions in a Hawaiian Cloud Forest Understory using Low-Cost Camera Traps and Visual Foundation Models}},
  year = {2026},
  eprint = {2603.07817},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.07817},
}
Figure