TL;DR
This paper explores how real-time chat in the citizen science game Eyewire impacts player engagement and performance. The study reveals that chat is crucial for collaboration, knowledge sharing, learning, and community building, ultimately fostering long-term participation. The authors offer design recommendations for future citizen science platforms to leverage real-time communication effectively.
What Problem Does This Paper Address?
Citizen science projects, which harness the collective intelligence of volunteers for scientific research, face challenges in maintaining participant engagement and ensuring effective collaboration. While many platforms exist, understanding how social features, particularly real-time communication, influence the user experience and scientific contribution remains an open question. This paper specifically investigates the role of a real-time chat interface within Eyewire, a successful gamified citizen science platform, to understand its impact on player interactions, learning, and overall participation.
Methods
The researchers employed a mixed-methods approach. They collected survey data from 1,365 Eyewire players responding to a question about their use of the real-time chat console. Additionally, they analyzed approximately 53,090 chat messages from a two-week period. This chat data was manually coded by two researchers to identify themes related to player interactions. The coding process involved an initial independent coding of a sample, followed by agreement on a final set of codes, with inter-coder reliability assessed using Cohen's Kappa coefficient (achieving 85% for survey responses and 80% for chat messages). The analysis focused on understanding how players used chat for collaboration, knowledge sharing, learning, and social interaction, and how these uses impacted their overall experience.
Datasets
- Eyewire Survey Data: Responses from 1,365 players to the question: "How do you use the real-time chat console?"
- Eyewire Real-time Chat Data: Approximately 53,090 chat messages collected over a two-week period in February 2015. This dataset included player messages, system-generated bot messages, and status updates.
Key Insights and Ideas
- Chat as a Multifaceted Tool: Real-time chat in Eyewire isn't just for asking questions; it's a dynamic space for collaboration on tasks, sharing knowledge and strategies, learning about the science behind the game, and fostering social connections.
- Beyond Task Completion: The study highlights that chat's value extends beyond direct task performance. It significantly contributes to player retention and engagement by creating a supportive community.
- "Lurking" is Valuable: Even players who don't actively participate in chat benefit from observing discussions, gaining insights and tips that improve their own experience. This underscores the importance of accessible communication channels.
- Informal Learning Hub: Chat serves as an informal learning environment where players can ask detailed questions about the science and platform, often receiving quick and helpful responses from more experienced users.
- Community Emergence: Eyewire's chat interface, despite lacking explicit social networking features, has facilitated the emergence of a strong, self-organized community where experienced players mentor newcomers.
Key Results
- Facilitates Engagement: Real-time chat was found to facilitate collaboration on tasks, knowledge sharing, learning, socializing, and supporting other community members, all contributing to sustained long-term participation.
- Qualitative Themes: Three cross-cutting themes emerged from the analysis: collaboration and discussion around joint tasks, learning and sharing knowledge, and the opportunity to help or receive help and advice.
- Conversation Characteristics: 80% of the analyzed chat messages were included in the study. Conversations typically involved 3-5 players and ranged from short bursts to discussions lasting up to two hours, often focused on help-seeking or information exchange.
- High Participation: 63% of all players in the corpus were identified as participating in at least one conversation.
Limitations and Future Work
- Data Subset: The study acknowledges that the survey responses and chat messages represent a subset of the total active Eyewire membership, potentially limiting the generalizability of findings.
- Self-Reported vs. Actual Behavior: While the chat data helps bridge the gap between what players say they do and what they actually do, the selection of a specific two-week period might not capture the full spectrum of discussions.
- Researcher Bias: The use of a limited number of researchers for coding and interpretation could introduce bias, although inter-coder reliability measures were employed.
- Future Work: The authors suggest extending the investigation to classify the types of expertise gained and their transferability to other domains. They also propose exploring structured capabilities for peer support and feedback, and experimenting with modifications to solo workflows to address concerns about cheating and collusion in microtask platforms.
Summary updated