TL;DR
The PVIS-10K dataset introduces over 10,000 high-resolution drone images with 3.5 million instance-level polygon annotations for rooftop solar panels. By utilizing a novel 3D-to-2D annotation workflow, it provides a massive benchmark for identifying individual photovoltaic (PV) modules across complex, multi-angle urban scenes.
What Problem Does This Paper Address?
As solar energy deployment shifts toward distributed rooftop installations, automated inspection becomes critical for maintenance. However, most existing computer vision datasets for solar panels focus on "array-level" detection (finding the whole group of panels) or semantic segmentation (finding the "solar pixels").
For detailed maintenance—like identifying a single cracked module—we need component-level instance segmentation. This is exceptionally difficult in drone imagery due to:
- Geometric Variation: Different flight heights and oblique viewing angles change the projected shape and scale of panels.
- Dense Layouts: Panels are often packed tightly, making it hard for models to distinguish the boundary between two adjacent modules.
- Environmental Noise: Shadows, reflections (glare), and partial occlusions from building structures further obscure these boundaries.
Methods
The authors developed a rigorous 3D-to-2D annotation workflow to ensure consistency and efficiency:
- 3D Reconstruction: They used Agisoft Metashape to create 3D models of 22 different rooftop stations from multi-view UAV imagery.
- 3D Annotation: Instead of labeling every 2D image, experts delineated the outer boundaries of physical components directly on the 3D model.
- Reprojection: Using camera poses and intrinsics, these 3D polygons were reprojected back into the original 2D images.
- Manual Correction: The reprojected labels underwent a two-stage manual review to handle occlusions and fine-tune boundaries.
This approach ensures that a single physical panel has a consistent identity and boundary across multiple images taken from different angles.
Datasets
The primary contribution is PVIS-10K, which consists of:
- 10,065 images (5280 × 3956 resolution).
- 3,516,675 polygon annotations representing 63,632 physical modules.
- Data from three Chinese cities (Ningbo, Shaoxing, Shenzhen) to test cross-city generalization.
The paper compares PVIS-10K against existing datasets like Bradbury, BDAPPV, and PV01. Unlike these predecessors, which mostly offer orthographic (top-down) views and array-level labels, PVIS-10K provides high-density instance labels in oblique views.
Key Insights and Ideas
- The "Instance vs. Semantic" Gap: The researchers found that while models are excellent at identifying the general area covered by solar panels (high IoU), they struggle significantly with "instance matching"—correctly separating one panel from its neighbor.
- Zero-Shot Limitations: Even advanced foundation models like SAM 3 (Segment Anything Model 3) struggle with this task. SAM 3 often merges multiple panels into a single instance or misidentifies boundaries in densely packed rows.
- Station-Disjoint Evaluation: The authors emphasize evaluating models on entirely unseen stations. Performance can vary wildly (from ~45% to ~92% Mask AP) depending on the specific rooftop's geometry and lighting, suggesting that "solving" one rooftop doesn't mean a model is ready for the next.
Key Results
The authors benchmarked four supervised models (Mask R-CNN, HTC, SOLOv2, Mask2Former) and one zero-shot model (SAM 3):
- Top Performer: The Hybrid Task Cascade (HTC) model achieved the best results, reaching a Mask AP of 72.72% in Ningbo and 90.49% in Shenzhen.
- Zero-Shot Performance: SAM 3 struggled with instance separation, yielding a low Mask AP (e.g., 4.75% in Ningbo), though it maintained a high foreground IoU (~89%), meaning it knew where the panels were but not where one ended and the next began.
- Qualitative Errors: Common failure modes included "merged instances" in bright glare and "missed instances" under heavy structural shadows.
Limitations and Future Work
- Persistent Identity: While the annotations were derived from 3D models, the released COCO-format dataset treats each image observation as an independent instance rather than tracking the same physical panel across a video sequence.
- Data Release: The authors released the images and 2D annotations but did not include the raw 3D models or camera parameters, which might limit researchers interested in photogrammetric refinement.
- Future Directions: The authors suggest that future work should focus on improving boundary detection in "over-exposure" (glare) conditions and developing better cross-view association techniques.
Summary updated