TL;DR
This paper presents a novel approach to teaching robotics and applied AI by integrating mixed reality (MR) into the curriculum. By having students actively contribute to the MR framework itself, the authors found increased motivation, deeper understanding of AI concepts, and valuable software engineering experience.
What Problem Does This Paper Address?
Teaching robotics and artificial intelligence (AI) often involves grappling with complex real-world systems. While robotics offers a hands-on, motivating way to learn AI concepts, the steep learning curve associated with perception (especially vision) and other systems-level challenges can be a significant barrier. This often leads to simplified, less rich domains that don't fully leverage the potential of robotics for AI education. Furthermore, replicating experimental conditions and comparing student projects can be difficult due to the inherent variability of physical systems. The authors aim to address these challenges by using mixed reality to abstract away complexities, provide scaffolding, and ensure repeatability, while also making the MR infrastructure itself a subject of student learning.
Methods
The core of the authors' approach is a mixed reality environment that augments a physical robot's world with virtual elements. This is achieved through a global vision system (Ergo) that captures both physical and virtual objects, allowing students to interact with a unified perception. The system allows for adjustable levels of abstraction, enabling instructors to simplify or expose complex aspects of perception and control as needed.
A key innovation is the integration of the MR framework development into student assignments. Instead of just using the MR system for robotics tasks, students are tasked with extending and improving the MR infrastructure itself. This includes:
- Ergo Vision Server Extensions: Students implement new features for the vision server, such as object recognition and tracking algorithms (e.g., Viola-Jones Haar features) and automated camera calibration systems. This exposes them to computer vision concepts and software maintenance.
- Robotics and AI Assignments: Students then use this enhanced MR environment to tackle increasingly complex robotics and AI problems, including:
- Aucklandianapolis: A path-tracking problem for mobile robots, introducing the full sensor-reasoning-action cycle.
- Treasure Hunt: A multi-agent coordination task involving virtual treasures and dynamic environments, requiring point stabilization and planning.
- Obstacle Run and Obstacle Pong: Assignments focusing on global and local path planning in dynamic environments, and introducing competitive multi-agent scenarios.
- Capstone Assignment (Soccer, Hockey, Pac-Man): A competitive multi-agent domain where students apply learned concepts, often using behavior trees for agent architecture.
Datasets
Not discussed in the paper. The paper focuses on a simulated and augmented physical environment rather than pre-existing datasets.
Key Insights and Ideas
- Student-Driven MR Development Enhances Learning: By making students contribute to the MR framework, they gain a deeper understanding of the abstracted elements (like vision) and develop valuable software engineering skills. This approach is more motivating than simply using a pre-built system.
- Mixed Reality as a Powerful Scaffolding Tool: MR allows for controlled abstraction of complex robotic perception and control problems, making them accessible to students without deep prior knowledge in those areas. This scaffolding can be dynamically adjusted.
- Repeatability and Comparability: Moving elements of the environment to the virtual plane allows for precise randomization and repeatability of experiments, making student projects easier to compare and evaluate.
- Bridging Theory and Practice: The hands-on nature of robotics, augmented by MR, grounds abstract AI concepts in concrete, verifiable solutions, fostering a deeper understanding.
- Software Engineering Skills are Integral: The assignments naturally expose students to challenges like code maintenance, documentation, debugging complex systems, and the importance of modularity and reuse.
Key Results
The paper doesn't present quantitative results in terms of specific metrics like accuracy improvements or speedups. Instead, it focuses on qualitative findings:
- Increased Student Motivation: The authors report that the approach better leverages student motivation.
- Valuable Software Engineering Experience: Students gain practical experience in maintaining and extending existing codebases, dealing with documentation issues, and understanding the impact of design choices.
- Deeper Understanding of AI and Robotics: By actively contributing to the MR infrastructure, students gain a more profound understanding of the underlying principles of robotics, AI, and computer vision.
- Successful Integration: The approach has been successfully employed for four years, teaching AI and robotics to undergraduates.
Limitations and Future Work
- Limitations:
- The paper acknowledges that not all student extensions are successful.
- The cost of hardware (LCD panels, robots) can be a consideration.
- The complexity of the MR system itself requires maintenance and setup.
- Future Work:
- The authors plan to ask students to replace the aging configuration scripting language with a more modern one (e.g., LUA or Python).
- Reducing the number of patterns needed to track robots is another planned extension.
- The paper suggests that the MR approach can be applied to a wide variety of dynamic multi-agent domains, implying potential for further expansion of assignment types.
Summary updated