Love in Action: Gamifying Public Video Cameras for Fostering Social Relationships in Real World

Zhang Zhang, Da Li, Geng Wu, Yaoning Li, Xiaobing Sun, Liang WangarXiv 2024 · DOI · arXiv

Introduction

In an era of hyper-connectivity, digital social media often fails to address the growing epidemic of loneliness. While "smart cities" are increasingly blanketed by public video surveillance, these systems are typically perceived as tools for security or control. The "Love in Action" (LIA) project explores a provocative alternative: can we repurpose this "all-seeing" infrastructure to foster positive, real-world social interactions? By gamifying public cameras, LIA encourages users to engage in non-verbal communication—gestures, postures, and clothing choices—to strengthen existing friendships and improve social cohesion.

System Design and Methodology

LIA is implemented as a location-based game via a WeChat mini-program, facilitating asynchronous social interactions grounded in physical space. The game defines two primary roles:

  • Requesters: Users who issue specific social challenges, such as asking a friend to perform a "love sign" at a designated camera location while wearing specific attire.
  • Performers: Users who navigate to the specified location using an AR map and perform the requested action in view of the camera.
The LIA system architecture, illustrating the interaction between Requesters, Performers, and the AI-enhanced video analysis pipeline.

To validate these performances, the researchers developed an AI-enhanced video analysis pipeline. This system integrates several state-of-the-art models:

  1. Pedestrian Detection: YOLOv5 identifies and tracks the player within the video feed.
  2. Attribute Recognition: The DeepMar model identifies fine-grained attributes like clothing color and gender.
  3. Action Recognition: A multi-stage pipeline utilizing AlphaPose and VideoPose extracts 3D skeletal joints, which are then classified by an EfficientGCN model into specific gestures (e.g., hugging, waving, bowing).

The system calculates a matching score; if the performance meets the requester's criteria, the performer earns Experience Points (EPs), and the requester can review the footage to award "social medals."

Experimental Results

A two-week field study involving 27 participants on a technology institute campus demonstrated the system's efficacy. The AI pipeline achieved a high precision of 90.3% for action recognition, although attribute recognition was more challenging at 65.9% due to variable lighting conditions.

More importantly, the study measured social impact using the Social Relationships Index (SRI). The results showed a statistically significant increase (p < 0.001) in friendship positivity. Interestingly, the game was most effective at bridging "weak ties"—colleagues or acquaintances who interacted regularly but lacked emotional depth. Participants reported a high mean social satisfaction score of 4.16/5.

Discussion: The "Embodied" Experience

Qualitative feedback highlighted several unique aspects of LIA:

  • Asynchronous Physicality: Unlike real-time location-based games, LIA allows for meaningful interactions that don't require simultaneous presence, yet remain tied to a shared physical environment.
  • Performance and Presence: Participants described a sense of "embodied performance," finding that acting for a friend in a public space created a stronger sense of connection than sending digital emojis.
  • Social Lubricant: For introverted users, the game provided a low-pressure, indirect way to express kindness and affection, mitigating the potential awkwardness of direct face-to-face interactions.

Limitations and Future Directions

While promising, the current implementation faces challenges regarding scale and AI robustness. Inconsistent recognition across different camera angles can lead to user frustration. Future work aims to integrate Large Vision-Language Models (LVLMs) to support more natural, open-ended requests and provide better explanations for AI decisions. Additionally, deploying the system in semi-public spaces like office recreation areas could help address privacy concerns associated with wider urban deployment.

Summary updated

Abstract

In this paper, we create "Love in Action" (LIA), a body language-based social game utilizing video cameras installed in public spaces to enhance social relationships in real-world. In the game, participants assume dual roles, i.e., requesters, who issue social requests, and performers, who respond social requests through performing specified body languages. To mediate the communication between participants, we build an AI-enhanced video analysis system incorporating multiple visual analysis modules like person detection, attribute recognition, and action recognition, to assess the performer's body language quality. A two-week field study involving 27 participants shows significant improvements in their social friendships, as indicated by self-reported questionnaires. Moreover, user experiences are investigated to highlight the potential of public video cameras as a novel communication medium for socializing in public spaces.
BibTeX
@misc{zhang2024love,
  author = {Zhang Zhang and Da Li and Geng Wu and Yaoning Li and Xiaobing Sun and Liang Wang},
  title = {{Love in Action: Gamifying Public Video Cameras for Fostering Social Relationships in Real World}},
  year = {2024},
  eprint = {2411.10449},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.10449},
}
Figure