Unleashing the Reasoning Capabilities of Vision Language Models for Effective Image-based Roadside Tree 3D Measurement

Long Chen, Zhen Dong, Bisheng Yang2025 · DOI

Abstract

Accurate measurement of roadside tree morphological parameters, such as Diameter at Breast Height (DBH) and Tree Height (TH), is essential for quantifying urban ecological benefits. While street-view images offer a low-cost alternative to traditional field surveys or LiDAR scanning, they suffer from scale ambiguity and poor generalization in complex urban environments. This paper introduces VLM-TMN (Tree 3D Measurement Network), a novel framework that adapts Large Vision Language Models (VLMs) for geometric reasoning. By integrating a Depth Projector Module and a "Magnifying Glass" strategy into a LLaVA-7B backbone, VLM-TMN achieves a significant reduction in DBH measurement error (from 24.39 cm to 7.08 cm RMSE). Our results demonstrate that VLMs can be effectively repurposed for precise 3D metrology, providing a scalable solution for sustainable urban planning.

Introduction

Quantifying the ecological contributions of urban forests—such as carbon sequestration and temperature regulation—requires precise data on tree structure. Traditionally, this data is collected via manual field surveys, which are labor-intensive, or mobile laser scanning (LiDAR), which is prohibitively expensive for city-wide deployment. Street-view imagery presents a promising middle ground, yet automated 3D measurement from 2D images remains a challenge due to the "scale ambiguity" problem: without depth context, a small foreground tree can appear identical to a large background tree.

Existing image-based methods often rely on rigid reference objects (e.g., traffic signs) or manual interpretation, limiting their utility in diverse urban landscapes. While modern VLMs like LLaVA exhibit impressive reasoning capabilities, they are primarily trained for semantic tasks and lack the inherent spatial understanding required for physical measurement. VLM-TMN bridges this gap by explicitly teaching the model to reason about depth and fine-grained geometry.

The VLM-TMN Framework

The VLM-TMN architecture enhances the standard LLaVA-7B model through two primary technical innovations designed to improve spatial perception and detail extraction.

1. Magnifying Glass Strategy

Standard visual encoders often operate at resolutions that obscure the fine textures of tree trunks and the subtle structures of canopies—details critical for estimating DBH. The Magnifying Glass strategy implements a dynamic cropping mechanism. The model identifies the target tree's bounding box, expands it to include context, and generates an "Enhanced Image." This high-resolution sub-patch is processed alongside the original image, forcing the visual encoder to capture the granular details necessary for accurate metrology.

2. Depth Projector Module

To resolve scale ambiguity, the model must understand the distance between the camera and the tree. The Depth Projector Module extracts implicit depth features from the visual tokens using cross-attention. During training, the model is subjected to explicit depth supervision, learning to predict the distance to the tree center as an intermediate step. This information is then injected into the Large Language Model (LLM) as a specialized <DEPTH> token.

Architecture of VLM-TMN, highlighting the Magnifying Glass strategy (Enhanced Image) and the Depth Projector module.

3. Reasoning-Based Prompting

The final measurement is not a simple regression but a reasoning task. The text prompt provides the LLM with the tree's image coordinates and the estimated depth. By combining these geometric cues with the visual features of the tree's shape and perspective, the LLM "reasons" the physical dimensions (DBH and TH) based on its internal world model.

Dataset and Training

The model was trained and evaluated using the WHU-RSTree dataset, a comprehensive multimodal benchmark collected in Nanjing, China. The dataset includes over 12,000 panoramic images and 68 km of point cloud data, providing ground-truth measurements for over 20,000 trees. For this study, a subset of 19,613 samples was utilized, pairing street-view images with precise DBH and TH values derived from LiDAR.

Key Results

VLM-TMN significantly outperforms both vanilla VLMs and traditional automated segmentation methods.

  • Accuracy: The system achieved a DBH Root Mean Square Error (RMSE) of 7.08 cm, a dramatic improvement over the 24.39 cm error seen in standard LLaVA-7B fine-tuning. Tree Height was measured with an RMSE of 2.25 m.
  • Efficiency: The pipeline is highly efficient, processing individual trees in approximately 0.27 seconds, making it suitable for large-scale urban inventory.
  • Robustness: Qualitative analysis shows that the model maintains high accuracy across diverse scenarios, including varying lighting conditions, different tree species, and partial occlusions by urban infrastructure.
Qualitative comparison between VLM-TMN predictions (Pred) and ground truth (GT) measurements for DBH and tree height across various urban scenes.

Conclusion and Future Work

VLM-TMN demonstrates that the reasoning power of Large Language Models can be successfully extended into the domain of 3D physical measurement. By providing explicit depth cues and high-resolution visual focus, the framework overcomes the traditional limitations of image-based metrology.

Future research will focus on enhancing the model's generalization to unseen geographic regions and exploring "Chain-of-Thought" prompting to further refine the geometric reasoning process. Additionally, integrating the system into a fully autonomous end-to-end pipeline—from tree detection to 3D reconstruction—will be a key step toward real-time urban ecological monitoring.

Summary updated

Abstract

The Diameter at Breast Height (DBH) and Tree Height (TH) are key morphological parameters of roadside tree, their accurate measurement is conducive to quantifying various ecological benefits of trees. Compared with traditional field survey or laser scanning methods, using low-cost street view images as an alternative data source is a promising measurement method. However, existing methods rely on preset reference systems or manual interpretation, which results in poor generalization and low efficiency. Recently, Visual Language Models (VLMs) have shown potential in mimicking human visual reasoning, but their direct application fails to address 3D measurement tasks. To tackle this, we propose a VLM-based Tree 3D Measurement Network, named VLM-TMN. Our key idea is to adjust VLMs to focus on the semantic and geometric information of trees to achieve effective measurement. Specifically, it contains several designs: 1) A Depth Projector Module integrating explicit depth supervision and implicit depth encoding to enhance geometric understanding. 2) A Magnifying Glass Strategy that amplifies visual perception by dynamically focusing on critical tree regions. Built upon LLaVA-7B, our method reduces DBH measurement errors from 24.39 cm to 7.08 cm RMSE, achieving a 7.57% improvement over standard supervised fine-tuning approaches and significantly outperforming existing methods (7.08 cm &lt; 15 cm). The results demonstrate how VLM-TMN can be effectively repurposed for urban ecological parameter quantification, providing a cost-effective solution for sustainable city planning.
BibTeX
@inproceedings{chen2025unleashing,
  author = {Long Chen and Zhen Dong and Bisheng Yang},
  title = {{Unleashing the Reasoning Capabilities of Vision Language Models for Effective Image-based Roadside Tree 3D Measurement}},
  booktitle = {˜The},
  year = {2025},
  doi = {10.5194/isprs-archives-xlviii-1-w5-2025-69-2025},
}
Figure