Discussion and recommendations from the 2023 "Gaze Meets ML" workshop breakout session

Dario Zanca, Alexandros Karargyris, Shuaib Ahmed, Ryan Anthony J. de Belen, Bonny Banerjee, Timur Ibrayev, Satyananda Kashyap, Elizabeth A. Krupinski, Chenyi Kuang, Paul Madu, Amarachi Madu, Silvia Makowski, Athul Mathew, Tim Rolff, Bert Shi, Joy T. Wu, Danca ZarioGaze Meets ml 2023

TL;DR

This paper summarizes discussions from the 2023 "Gaze Meets ML" workshop, highlighting the need for standardized eye-gaze datasets and a more cohesive research community. It proposes concrete steps for improving dataset interoperability and fostering collaboration between diverse fields like computer vision, machine learning, and cognitive psychology.

What Problem Does This Paper Address?

The paper addresses the challenge of effectively integrating eye-gaze data into machine learning (ML) research and applications. While eye-gaze offers rich insights into human attention and cognition, its potential is hindered by fragmented and non-standardized datasets, a lack of community cohesion across disciplines, and the difficulty in translating these insights into practical AI advancements. The authors aim to bridge the gap between eye-tracking expertise and ML methodologies to unlock new possibilities in areas like human-computer interaction, medical diagnosis, and personalized experiences.

Methods

The paper doesn't present a novel algorithm or experimental method in the traditional sense. Instead, it's a summary of discussions and recommendations from a workshop. The "method" here is a collaborative, community-driven approach to identify challenges and propose solutions. The workshop involved breakout sessions focused on three key areas: Datasets, Community, and Vision and Actions for the Future. The findings were then synthesized and expanded upon.

Datasets

The paper discusses the current landscape of publicly available eye-gaze datasets, noting that over 30 have been surveyed. A key recommendation is the creation of a Common Data Model (CDM), inspired by formats like COCO, to standardize dataset structure and documentation. This proposed CDM would include:

  • A metadata schema integrating information about license, contributors, applications, eye tracker details, gaze data collection protocols, participant demographics, citations, code repositories, preprocessing workflows, model weights, and other documentation.
  • Inclusion of raw (unprocessed) gaze data with reformatting to a common structure.
  • A standard way to index input images or other non-gaze data.
  • Annotation data (e.g., bounding boxes, points, masks) in a COCO-like format.

No new datasets were introduced in this paper; rather, the focus is on improving the usability and interoperability of existing and future datasets.

Key Insights and Ideas

  • Standardization is Paramount: The most significant takeaway is the urgent need for standardization in eye-gaze datasets. Without it, research reproducibility, data sharing, and leveraging the full potential of these datasets are severely hampered.
  • Bridging Disciplinary Gaps: The field of eye-gaze research involves diverse communities (cognitive psychology, computer vision, neuroscience). These groups often use different terminology and methodologies, making collaboration difficult. The workshop aimed to foster a shared understanding and common ground.
  • Gaze for AI Evaluation and Enhancement: The paper explores novel applications of eye-gaze, such as using it to evaluate AI outputs (e.g., heatmaps in medical imaging), assess the impact of AI on clinical decision-making, and determine optimal ways to present AI information to users.
  • Community as a Catalyst: Building a strong, collaborative community is seen as crucial for advancing the field. This involves defining clear goals, establishing governance, utilizing effective communication platforms, organizing events, and promoting knowledge sharing.
  • Sustainability Beyond Sponsorship: The authors suggest exploring diverse revenue streams and membership models to ensure the long-term sustainability of community initiatives.

Key Results

This paper is not about experimental results but rather about community consensus and recommendations. The "results" are the proposed actions and directions for the field:

  • Recommendation 1: Develop a database of state-of-the-art methods in gaze-assisted ML to serve as a repository for advancements, techniques, and software.
  • Recommendation 2: Create a taxonomy for gaze-assisted ML to improve understanding and consistency in terminology.
  • Recommendation 3: Establish a community portal (e.g., a wiki) to foster community engagement, host webinars, share datasets, and disseminate knowledge.

Limitations and Future Work

  • Dataset Completeness: The surveyed dataset table was not exhaustive, aiming only to provide an initial view of available resources.
  • Adoption Challenges: The authors acknowledge that achieving community consensus on specifications and ensuring their adoption will be a significant challenge.
  • Future Work: The paper implicitly calls for action on its recommendations: developing the CDM, building the community portal, and actively engaging researchers from various disciplines to contribute to these efforts. The future work lies in the community's collective effort to implement these proposed solutions.

Summary updated

BibTeX
@inproceedings{zanca2023discussion,
  author = {Dario Zanca and Alexandros Karargyris and Shuaib Ahmed and Ryan Anthony J. de Belen and Bonny Banerjee and Timur Ibrayev and Satyananda Kashyap and Elizabeth A. Krupinski and Chenyi Kuang and Paul Madu and Amarachi Madu and Silvia Makowski and Athul Mathew and Tim Rolff and Bert Shi and Joy T. Wu and Danca Zario},
  title = {{Discussion and recommendations from the 2023 "Gaze Meets ML" workshop breakout session}},
  booktitle = {Gaze Meets ml},
  year = {2023},
}
Figure