Abstract
Accurate measurement of roadside tree morphological parameters, such as Diameter at Breast Height (DBH) and Tree Height (TH), is essential for quantifying urban ecological benefits. While street-view images offer a low-cost alternative to traditional field surveys or LiDAR scanning, they suffer from scale ambiguity and poor generalization in complex urban environments. This paper introduces VLM-TMN (Tree 3D Measurement Network), a novel framework that adapts Large Vision Language Models (VLMs) for geometric reasoning. By integrating a Depth Projector Module and a "Magnifying Glass" strategy into a LLaVA-7B backbone, VLM-TMN achieves a significant reduction in DBH measurement error (from 24.39 cm to 7.08 cm RMSE). Our results demonstrate that VLMs can be effectively repurposed for precise 3D metrology, providing a scalable solution for sustainable urban planning.
Introduction
Quantifying the ecological contributions of urban forests—such as carbon sequestration and temperature regulation—requires precise data on tree structure. Traditionally, this data is collected via manual field surveys, which are labor-intensive, or mobile laser scanning (LiDAR), which is prohibitively expensive for city-wide deployment. Street-view imagery presents a promising middle ground, yet automated 3D measurement from 2D images remains a challenge due to the "scale ambiguity" problem: without depth context, a small foreground tree can appear identical to a large background tree.
Existing image-based methods often rely on rigid reference objects (e.g., traffic signs) or manual interpretation, limiting their utility in diverse urban landscapes. While modern VLMs like LLaVA exhibit impressive reasoning capabilities, they are primarily trained for semantic tasks and lack the inherent spatial understanding required for physical measurement. VLM-TMN bridges this gap by explicitly teaching the model to reason about depth and fine-grained geometry.
The VLM-TMN Framework
The VLM-TMN architecture enhances the standard LLaVA-7B model through two primary technical innovations designed to improve spatial perception and detail extraction.
1. Magnifying Glass Strategy
Standard visual encoders often operate at resolutions that obscure the fine textures of tree trunks and the subtle structures of canopies—details critical for estimating DBH. The Magnifying Glass strategy implements a dynamic cropping mechanism. The model identifies the target tree's bounding box, expands it to include context, and generates an "Enhanced Image." This high-resolution sub-patch is processed alongside the original image, forcing the visual encoder to capture the granular details necessary for accurate metrology.
2. Depth Projector Module
To resolve scale ambiguity, the model must understand the distance between the camera and the tree. The Depth Projector Module extracts implicit depth features from the visual tokens using cross-attention. During training, the model is subjected to explicit depth supervision, learning to predict the distance to the tree center as an intermediate step. This information is then injected into the Large Language Model (LLM) as a specialized <DEPTH> token.
3. Reasoning-Based Prompting
The final measurement is not a simple regression but a reasoning task. The text prompt provides the LLM with the tree's image coordinates and the estimated depth. By combining these geometric cues with the visual features of the tree's shape and perspective, the LLM "reasons" the physical dimensions (DBH and TH) based on its internal world model.
Dataset and Training
The model was trained and evaluated using the WHU-RSTree dataset, a comprehensive multimodal benchmark collected in Nanjing, China. The dataset includes over 12,000 panoramic images and 68 km of point cloud data, providing ground-truth measurements for over 20,000 trees. For this study, a subset of 19,613 samples was utilized, pairing street-view images with precise DBH and TH values derived from LiDAR.
Key Results
VLM-TMN significantly outperforms both vanilla VLMs and traditional automated segmentation methods.
- Accuracy: The system achieved a DBH Root Mean Square Error (RMSE) of 7.08 cm, a dramatic improvement over the 24.39 cm error seen in standard LLaVA-7B fine-tuning. Tree Height was measured with an RMSE of 2.25 m.
- Efficiency: The pipeline is highly efficient, processing individual trees in approximately 0.27 seconds, making it suitable for large-scale urban inventory.
- Robustness: Qualitative analysis shows that the model maintains high accuracy across diverse scenarios, including varying lighting conditions, different tree species, and partial occlusions by urban infrastructure.
Conclusion and Future Work
VLM-TMN demonstrates that the reasoning power of Large Language Models can be successfully extended into the domain of 3D physical measurement. By providing explicit depth cues and high-resolution visual focus, the framework overcomes the traditional limitations of image-based metrology.
Future research will focus on enhancing the model's generalization to unseen geographic regions and exploring "Chain-of-Thought" prompting to further refine the geometric reasoning process. Additionally, integrating the system into a fully autonomous end-to-end pipeline—from tree detection to 3D reconstruction—will be a key step toward real-time urban ecological monitoring.
Summary updated