Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Vision

Yuping He, Yifei Huang, Guo Chen, Lidong Lu, Baoqi Pei, Jilan Xu, Tong Lu, Yoichi SatoarXiv 2025 · DOI · arXiv

Introduction: The Multi-Perspective Challenge

In computer vision, research has traditionally been siloed into specific camera perspectives. Egocentric (Ego) vision, captured from wearables like AR glasses, offers an intimate look at human-object interactions but suffers from instability and a narrow field of view. Conversely, Exocentric (Exo) vision, from static surveillance or overhead cameras, provides stable environmental context but lacks fine-grained detail.

This survey addresses the critical gap in cross-view collaborative intelligence—the ability of AI systems to synthesize these disparate perspectives. By bridging the actor's detailed view with the observer's contextual view, researchers aim to create AI that perceives the world with human-like multi-perspective synergy.

Overview of the cross-view collaborative intelligence landscape, including applications, research tasks, and datasets.

Research Landscape and Taxonomy

The authors categorize the current research into three primary collaborative directions:

  1. Exocentric for Egocentric (Exo-for-Ego): Leveraging broad third-person knowledge to assist first-person systems. This includes using "teacher" videos for robotic learning and grounding "affordances"—identifying how a wearer should interact with tools based on external observations.
  2. Egocentric for Exocentric (Ego-for-Exo): Enhancing third-person understanding with first-person details. A prominent task is View Birdification, where egocentric data helps estimate top-down trajectories in crowded scenes.
  3. Joint Learning: Frameworks that process both views simultaneously. Key tasks include Cross-View Retrieval (matching moments across feeds) and Egocentric Wearer Identification (identifying the wearer within a crowd from a third-person feed).

The field is currently transitioning from traditional CNN architectures toward Diffusion Models for video generation and Vision-Language Models (VLMs) for cross-perspective reasoning.

Key Datasets and Benchmarks

The survey reviews numerous datasets essential for training these multi-view models:

  • Ego-Exo4D: A massive dataset of skilled activities (cooking, dancing) with synchronized views.
  • Assembly101: Focuses on procedural tasks using 12 synchronized views (8 exo, 4 ego).
  • H2O: Specialized for hand-object interactions in multi-view environments.
  • Synthetic Datasets (YOWO, CSRD): Crucial for camera localization where real-world synchronized data is scarce.

A significant trend is the move toward "pair-free" learning, reducing the reliance on perfectly synchronized video pairs to leverage the abundance of unsynchronized web video.

Insights: Biological Inspiration and the Detail-Context Trade-off

The research is grounded in Mirror Neuron Theory, suggesting that biological brains use similar neural mechanisms for performing and observing actions. This serves as a "North Star" for cross-view AI. Integrating these views requires resolving the geometric disparity between a head-mounted camera and a ceiling-mounted one. Affordance Grounding acts as a vital bridge, mapping interaction regions from exo-videos onto a robot's ego-view to teach manipulation skills.

Field Growth and Results

The field has seen explosive growth, with citations rising from near zero in 2015 to over 1,640 in 2024.

The rapid growth of citations and key research milestones in the egocentric-exocentric vision field from 2015 to 2024.

Key findings include:

  • Knowledge Distillation: Training on high-quality egocentric data and distilling it into third-person models significantly improves action recognition.
  • Generative AI: Diffusion models can now "hallucinate" first-person views from third-person video (Exo2Ego), providing vital data for robotic simulations.

Future Directions and Limitations

Despite progress, several hurdles remain:

  • Data Scarcity: There is a lack of data in specialized domains like surgery and search-and-rescue.
  • Hardware Constraints: Models must eventually run on lightweight wearables with limited power, making current heavy VLMs impractical for edge deployment.
  • Foundation Models: The authors advocate for the development of perspective-agnostic foundation models to reduce the need for task-specific fine-tuning.

Summary updated

Abstract

Perceiving the world from both egocentric (first-person) and exocentric (third-person) perspectives is fundamental to human cognition, enabling rich and complementary understanding of dynamic environments. In recent years, allowing the machines to leverage the synergistic potential of these dual perspectives has emerged as a compelling research direction in video understanding. In this survey, we provide a comprehensive review of video understanding from both exocentric and egocentric viewpoints. We begin by highlighting the practical applications of integrating egocentric and exocentric techniques, envisioning their potential collaboration across domains. We then identify key research tasks to realize these applications. Next, we systematically organize and review recent advancements into three main research directions: (1) leveraging egocentric data to enhance exocentric understanding, (2) utilizing exocentric data to improve egocentric analysis, and (3) joint learning frameworks that unify both perspectives. For each direction, we analyze a diverse set of tasks and relevant works. Additionally, we discuss benchmark datasets that support research in both perspectives, evaluating their scope, diversity, and applicability. Finally, we discuss limitations in current works and propose promising future research directions. By synthesizing insights from both perspectives, our goal is to inspire advancements in video understanding and artificial intelligence, bringing machines closer to perceiving the world in a human-like manner. A GitHub repo of related works can be found at https://github.com/ayiyayi/Awesome-Egocentric-and-Exocentric-Vision.
BibTeX
@misc{he2025bridging,
  author = {Yuping He and Yifei Huang and Guo Chen and Lidong Lu and Baoqi Pei and Jilan Xu and Tong Lu and Yoichi Sato},
  title = {{Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Vision}},
  year = {2025},
  eprint = {2506.06253},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.06253},
}
Figure