Solving Pictorial Jigsaw Puzzle by Stigmergy-inspired Internet-based Human Collective Intelligence

Zhi Jin, Fellow of Ieee and Ccf, Bo Shen, Wei Zhang, Haiyan Zhao, Zhi Jin, Yanhong WuarXiv 2018 · arXiv

Abstract

This paper presents a novel framework for solving pictorial jigsaw (PJ) puzzles through internet-based human collective intelligence. Inspired by biological stigmergy, the system coordinates distributed players via a continuous "Exploration-Integration-Feedback" (EIF) loop. By aggregating individual efforts into a Collective Opinion Graph (COG), the framework achieves near-linear speedup in solving time relative to group size, significantly outperforming solo efforts and traditional collaboration methods while maintaining high solution quality.

Introduction

Pictorial jigsaw puzzles serve as an ideal proxy for "bottom-up" complex problems—tasks that are difficult to decompose into independent sub-problems. While automated solvers exist, they often struggle with "human-sensitive" content or degraded image data. Conversely, human collaboration is typically limited by the communication overhead of face-to-face interaction. This research addresses the challenge of organizing large, distributed groups to solve such non-decomposable problems efficiently without direct communication.

The EIF Framework: A Stigmergic Approach

The core of the proposed system is the Exploration-Integration-Feedback (EIF) loop, which leverages stigmergy—environment-mediated indirect interaction.

The Exploration-Integration-Feedback (EIF) loop framework for collective puzzle solving.

The loop consists of three main phases:

  1. Exploration: Players work independently in private virtual workspaces. Their actions, such as piece placements and rejections, are recorded by the system.
  2. Integration: The system aggregates these individual actions in real-time to construct a Collective Opinion Graph (COG). This graph represents the group's consensus on piece adjacencies.
  3. Feedback: The system provides personalized hints back to players based on the COG. Responsive feedback automatically snaps pieces into place when group confidence is high, while stimulative feedback highlights potential matches to assist stalled players.

Collective Opinion Integration

The integration phase is crucial for turning disparate individual efforts into a coherent solution. The system builds a graph where nodes are puzzle pieces and edges represent perceived adjacencies (e.g., Left-Right or Top-Bottom).

Integration of piece relationships into a connected graph with deduced edges.

As shown in the figure above, the system can deduce new relationships from existing ones, strengthening the collective model even when individual players have only explored partial connections. This "digital pheromone" guides the group toward the global solution without requiring direct verbal coordination.

Experimental Results

The framework was tested using a custom platform, "Crowd Jigsaw Puzzle," with 52 participants across 350 game rounds. Puzzles ranged from 16 to 100 pieces, with group sizes up to 10. To increase difficulty, edge pixels were erased to thwart simple color-matching algorithms.

Key findings include:

  • Efficiency Gains: Groups of 2-10 players reduced solving time by 31.36% to 64.57% compared to the best solo players.
  • Scalability: Solving time followed a near-linear relationship with the reciprocal of the group size ($1/n$), demonstrating excellent scalability compared to face-to-face methods.
  • Solution Quality: Human collectives achieved perfect solution quality (1.0), far exceeding state-of-the-art automated solvers (0.52) on complex images.
  • Feedback Utility: 64% of participants found the stigmergy-based hints helpful, confirming the effectiveness of the feedback mechanism.

Discussion and Future Directions

The success of the EIF loop demonstrates that the environment can effectively mediate complex human collaboration. By eliminating the "too many cooks in the kitchen" bottleneck, stigmergic systems can scale collective intelligence to larger groups and more complex problems.

Future work will focus on addressing server-side processing bottlenecks and exploring the application of the EIF loop to other domains, such as knowledge graph construction and distributed software development.

Conclusion

This research provides a scalable model for human collective intelligence. By mimicking the indirect coordination seen in social insects, the EIF framework enables distributed groups to solve complex, non-decomposable problems with high efficiency and accuracy.

Summary updated

Abstract

The pictorial jigsaw (PJ) puzzle is a well-known leisure game for humans. Usually, a PJ puzzle game is played by one or several human players face-to-face in the physical space. In this paper, we focus on how to solve PJ puzzles in the cyberspace by a group of physically distributed human players. We propose an approach to solving PJ puzzle by stigmergy-inspired Internet-based human collective intelligence. The core of the approach is a continuously executing loop, named the EIF loop, which consists of three activities: exploration, integration, and feedback. In exploration, each player tries to solve the PJ puzzle alone, without direct interactions with other players. At any time, the result of a player's exploration is a partial solution to the PJ puzzle, and a set of rejected neighboring relation between pieces. The results of all players' exploration are integrated in real time through integration, with the output of a continuously updated collective opinion graph (COG). And through feedback, each player is provided with personalized feedback information based on the current COG and the player's exploration result, in order to accelerate his/her puzzle-solving process. Exploratory experiments show that: (1) supported by this approach, the time to solve PJ puzzle is nearly linear to the reciprocal of the number of players, and shows better scalability to puzzle size than that of face-to-face collaboration for 10-player groups; (2) for groups with 2 to 10 players, the puzzle-solving time decreases 31.36%-64.57% on average, compared with the best single players in the experiments.
BibTeX
@misc{ccf2018solving,
  author = {Zhi Jin, Fellow of IEEE and CCF and Bo Shen and Wei Zhang and Haiyan Zhao and Zhi Jin and Yanhong Wu},
  title = {{Solving Pictorial Jigsaw Puzzle by Stigmergy-inspired Internet-based Human Collective Intelligence}},
  year = {2018},
  eprint = {1812.02559},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1812.02559},
}
Figure