TL;DR
This paper introduces PhotoCity, a hybrid "game with a purpose" (GWAP) and alternate reality game (ARG) that leverages players' real-world photography to build detailed 3D models of urban environments. By gamifying the data collection process, PhotoCity addresses the challenge of acquiring comprehensive and uniformly distributed photographic data needed for accurate 3D reconstruction, proving that crowdsourced efforts can effectively tackle large-scale data acquisition tasks.
What Problem Does This Paper Address?
Building detailed 3D models of cities is crucial for applications like navigation, visualization, and urban planning. While computer vision techniques can automatically reconstruct 3D models from large photo collections, existing datasets (like those scraped from Flickr) often suffer from incomplete coverage, with popular viewpoints of well-known landmarks being overrepresented, while less common angles and less popular areas are sparse. This lack of uniform, dense photographic data limits the quality and completeness of the resulting 3D reconstructions. The authors aim to solve this by finding a way to collect a massive amount of photographic data, distributed across various viewpoints, from real-world locations.
Methods
The core of PhotoCity's method is its gamified approach to data collection. It combines elements of Games with a Purpose (GWAPs), where players contribute to a computational task, and Alternate Reality Games (ARGs), which blend virtual and real-world elements.
- Gameplay Loop: Players are presented with a map of virtual buildings overlaid on real-world locations. They are tasked with capturing virtual "flags" placed on these buildings by taking photographs from specific viewpoints.
- 3D Reconstruction: The collected photos are used to generate and refine 3D models of buildings. Initially, models are "seeds" generated from a batch of photos. As players contribute more photos, especially from underrepresented viewpoints, the models grow and become more detailed.
- Player Motivation: Players are motivated by accumulating points for their photos, capturing flags for their teams, and ultimately owning virtual models of buildings. This competitive and ownership-driven system encourages players to explore different angles and contribute data where it's most needed.
- Iterative Refinement: The game actively directs players to focus on gaps and fringes in existing reconstructions by spawning new flags in these areas, ensuring that the data collection process is guided towards improving the overall model quality.
- Computer Vision Integration: The system relies on computer vision algorithms for 3D reconstruction from images. The paper notes the need for new algorithms that can interact with players and effectively direct their efforts.
Datasets
The paper does not explicitly list or introduce a new benchmark dataset. Instead, it describes the data collection process as an ongoing effort using player-submitted photographs of real-world locations around the University of Washington campus. The "dataset" is effectively the collection of images generated by players during the game trials.
Key Insights and Ideas
- Gamification for Data Acquisition: The most significant insight is that by transforming a tedious data collection task into an engaging game, it's possible to motivate a large number of people to contribute high-quality, diverse photographic data.
- Hybrid GWAP/ARG Success: Combining the purpose-driven nature of GWAPs with the immersive, real-world integration of ARGs creates a powerful framework for large-scale data collection.
- Emergent Player Strategies: The game mechanics led to unexpected player strategies, such as players developing tactics to reclaim flags by expanding nearby models, demonstrating how abstract game rules can lead to complex emergent behavior.
- Directing Data Collection: The game's ability to direct players to specific areas needing more data is crucial for building complete 3D models, overcoming the "popular viewpoint" bias of passively collected datasets.
Key Results
- Data Collection Volume: In the second trial, 25 players submitted over 11,000 photos, generating over 3,000,000 sparse-model points. Since the publication of the full paper, over 130,000 photos have been collected, with a high proportion providing useful reconstruction data.
- Model Growth: The paper shows examples of significant model growth, with players circumnavigating buildings, expanding models to include adjacent structures, and even contributing entirely new building "seeds."
- Player Engagement: Interviews and observations indicated that players found the game fun, with positive reinforcement for scoring points and a sense of accomplishment from contributing to the reconstruction.
Limitations and Future Work
- Scalability and Interactivity: The authors acknowledge that PhotoCity is "vastly more data intensive than other GWAPs and more process intensive than traditional ARGs," requiring the development of new algorithms and a robust distributed system for interactivity.
- Player Overwhelm: Players who joined when many flags were already captured felt overwhelmed and less able to make a significant impact, suggesting a need for mechanisms to ensure continued engagement for new players.
- Future Work: The paper implies that further development of algorithms for interacting with players and improving the distributed system is necessary. The ongoing effort to reconstruct the world in 3D through player submissions is also a clear direction for future work.
Summary updated