Exploring the Use of Collaborative Robots in Cinematography

Pragathi Praveena, Bengisu Cagiltay, Michael Gleicher, Bilge MutluarXiv 2023 · DOI · arXiv

TL;DR

This paper explores the untapped potential of collaborative robots (cobots) in cinematography through exploratory interviews with industry practitioners. By moving beyond rigid industrial automation, the authors identify how safe, flexible, and lower-cost cobots can enable new creative workflows like remote directing, precise shot repeatability for VFX, and human-robot co-production on set.

What Problem Does This Paper Address?

While robotic systems like high-speed industrial arms and drones have become staples in high-budget filmmaking, they remain inaccessible to most creators due to high costs, safety requirements (like cages), and steep learning curves. Furthermore, these systems are often designed for rigid automation rather than fluid, creative collaboration.

The authors address the gap between existing robotic technology and the actual needs of cinematography practitioners. They seek to understand how "cobots"—robots designed to work safely alongside humans—can be integrated into the creative process without encroaching on the cinematographer’s artistic control.

Methods

The researchers conducted a qualitative study consisting of two-hour exploratory interviews with three cinematography practitioners (ranging from independent filmmakers to commercial videographers). The study was structured into three distinct parts:

  1. Background Interview: Understanding the participants' professional experience and existing workflows.
  2. System Demonstration & "Free-play": Participants interacted with Periscope, a prototype robotic system. They tested capabilities such as physical posing (kinesthetic teaching), mouse-based control for orbiting objects, automatic hand tracking, and saving/recalling camera positions.
  3. Storyboarding Task: Participants designed a "how-to" video scenario, sketching how they would specifically utilize the robot’s unique capabilities in a real production.
Figure 1: We interviewed three cinematography practitioners to investigate the design space of robot-supported cinematogra- phy. The study was divided into three parts. (Left) Part 1: Illustration of the interview setup. (Middle) Part 2: A participant in- teracting with our prototype robotic system while describing how they envision such technology in media production. (Right) Part 3: An example storyboard sketched by a participant that incorporated the robot into their filming process.

The data was analyzed using reflexive thematic analysis to identify key design factors for future robotic cinematography tools.

Datasets

As this is a qualitative human-computer interaction (HCI) study, no large-scale machine learning datasets were used. The "data" consists of interview transcripts, observation notes, and participant-generated storyboards from the three practitioners.

Key Insights and Ideas

  • The Trust Gap in High-Stakes Scenarios: Practitioners expressed a "lack of trust" in robotic reliability for high-stakes shoots where only one or two takes are possible. They preferred a human operator with "10 years of experience" over a robot that might require "re-programming" mid-scene.
  • Repeatability as a Creative Layering Tool: Beyond just getting a "perfect" shot, practitioners saw immense value in the robot’s ability to repeat a trajectory exactly. This allows for "building multiple layers in post-production," such as filming the same move with different lighting or subjects to be composited later.
  • Democratizing Remote Expertise: A major "aha moment" was the potential for remote cinematography. A Director of Photography in Los Angeles could remotely operate a robotic camera in Cincinnati, reducing travel costs and allowing for rapid, high-quality remote interviews.
  • Spatial Awareness is the Remote Bottleneck: Participants noted that controlling a robot remotely feels like being "blind." They suggested integrating 360-degree "witness cameras" or VR rigs to provide the spatial context necessary for creative decision-making.
  • Robots as Secondary Crew: Instead of just holding the camera, practitioners envisioned robots taking over tedious secondary tasks, such as holding a boom microphone perfectly out of frame while tracking a subject.

Key Results

The study identified three primary categories of findings that impact the adoption of cobots in film:

  • Practical Constraints: Cost remains a barrier (equipment is usually rented), and technical issues like "white noise" from robot fans or micro-vibrations can ruin audio and video quality.
  • Workflow Compatibility: Automation is a double-edged sword; it’s beneficial for meticulous planners but can feel restrictive for those who prefer improvised, "run-and-gun" styles. Tactile, multi-modal interfaces (joysticks, physical posing) are preferred over abstract software controls.
  • New Creative Frontiers: The ability to perform perfect "orbits" around objects or use hand-tracking for smooth, intentional-looking shots was highly praised. Participants felt these features could produce shots that "look different from what anybody else is doing right now."

Limitations and Future Work

The authors acknowledge the small sample size (N=3), which limits the generalizability of the findings across the entire diverse film industry. Additionally, the participants' ideas were naturally bounded by the specific features demonstrated in the Periscope prototype.

Future Work involves:

  • Designing more advanced interactive prototypes based on these insights.
  • Conducting co-design sessions with a larger, more diverse group of practitioners.
  • Exploring how to encode expert cinematographic knowledge directly into robot motion profiles to assist non-experts.

Summary updated

Abstract

Robotic technology can support the creation of new tools that improve the creative process of cinematography. It is crucial to consider the specific requirements and perspectives of industry professionals when designing and developing these tools. In this paper, we present the results from exploratory interviews with three cinematography practitioners, which included a demonstration of a prototype robotic system. We identified many factors that can impact the design, adoption, and use of robotic support for cinematography, including: (1) the ability to meet requirements for cost, quality, mobility, creativity, and reliability; (2) the compatibility and integration of tools with existing workflows, equipment, and software; and (3) the potential for new creative opportunities that robotic technology can open up. Our findings provide a starting point for future co-design projects that aim to support the work of cinematographers with collaborative robots.
BibTeX
@misc{praveena2023exploring,
  author = {Pragathi Praveena and Bengisu Cagiltay and Michael Gleicher and Bilge Mutlu},
  title = {{Exploring the Use of Collaborative Robots in Cinematography}},
  year = {2023},
  eprint = {2304.07892},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2304.07892},
}
Figure