Introduction: The Critical Need for Urban Forest Monitoring
Urban forests are essential infrastructure, providing cooling, carbon sequestration, and significant public health benefits. However, maintaining an accurate tree census is a massive logistical and financial challenge. For instance, a single census in Los Angeles can cost upwards of $2 million and take years to complete. Consequently, many urban forest records are outdated or non-existent, particularly in under-resourced areas.
Computer vision (CV) offers a potential solution for automated monitoring, but existing datasets have historically been limited in scale or geographic scope. The primary technical hurdles include geographic domain shift—where models fail to generalize across different cities due to variations in architecture, lighting, and local flora—and the long-tailed distribution of tree species, where a few common genera dominate the data while hundreds of others are rarely seen.
The Auto Arborist Dataset: Scale and Diversity
To address these challenges, the authors introduce the Auto Arborist Dataset, a benchmark that is over two orders of magnitude larger than previous urban forest datasets. It consists of 2.6 million tree records across 23 North American cities, spanning 344 unique genera.
The dataset provides a multimodal perspective for each tree, merging public census data with approximately 9.1 million images. This includes 512x512 aerial crops and up to three 768x1152 street-level panoramas per tree. This multiview approach allows models to capture both the canopy structure from above and the trunk/leaf details from the side.
Methodology: Multimodal Fusion and Regional Evaluation
The researchers developed a pipeline using ResNet-101 backbones to process both aerial and street-level imagery. To effectively combine these modalities, they employed a Mixture of Experts (MoE) approach, enabling the model to dynamically weigh the importance of aerial versus street-level data based on the specific genus or geographic region.
Evaluation was conducted hierarchically to test generalization:
- Per-city: Learning within a single urban environment.
- Regional: Generalizing across broad geographic areas (West, Central, East).
- Full Dataset: Testing a global model on entirely unseen holdout cities.
Key Insights: Domain Shift and the Long Tail
The study revealed several critical findings regarding urban tree classification:
- Street-level Dominance: Street-level imagery is the most informative modality, providing a >20% boost in Class-Averaged Recall (AR) compared to aerial data alone.
- Regional Generalization: Models generalize significantly better to new cities within the same geographic region. This suggests that local "terroir"—including climate and urban planning styles—creates a distinct visual signature that models can learn.
- The Challenge of Rarity: Despite the massive scale of the dataset, rare genera (those with fewer than 100 examples) remain difficult to classify accurately, highlighting the need for improved few-shot learning.
Results and Future Directions
The best-performing model, utilizing the full dataset and a regional MoE, achieved a 49.96% Class-Averaged Recall (AR). While top-1 accuracy reached as high as 93.28% in some cities (e.g., Vancouver), the variance across cities highlights the difficulty of handling diverse local distributions. Moving from a single street-level view to a multiview + aerial approach provided a consistent performance boost of 3-5%.
Future work will focus on addressing label noise from public censuses, moving from genus-level to species-level identification, and improving privacy-preserving techniques for large-scale street-level imagery. The ultimate goal is a "virtual census" capability that can monitor urban forests globally using only available remote sensing data.
Summary updated