Collaposer: Transforming Photo Collections into Visual Assets for Storytelling with Collages

Jiayi Zhou, Liwenhan Xie, Jiaju Ma, Zheng Wei, Huamin Qu, Anyi Rao

TL;DR

Collaposer is an AI-powered creativity support tool that automates the tedious process of preparing object-level visual assets (cutouts) from large photo collections for digital collage storytelling. By coupling multi-model computer vision with an LLM, it extracts, selects, and hierarchically organizes assets based on a user's text narrative, allowing creators to seamlessly transition from abstract concepts to static or animated compositions.

What Problem Does This Paper Address?

Digital collage is a powerful, expressive art form, but creating one is notoriously labor-intensive. Creators must manually sift through massive photo collections, execute precise and tedious image cutouts (segmentation), and organize hundreds of fragments before composition can even begin. This mechanical overhead disrupts creative momentum.

Furthermore, traditional asset management systems operate at the level of whole images and rely on rigid keyword searches. This makes it difficult for creators to translate loose story ideas into effective search queries or discover contextually relevant assets that aren't explicitly named. Collaposer addresses these bottlenecks by automating object-level extraction, semantic asset selection, and hierarchical organization.

Methods

Collaposer introduces a story-driven asset preparation pipeline divided into three distinct stages:

  1. Photo Collection Preprocessing (Stage I): The system processes raw images using a suite of vision models. It utilizes the Recognize Anything Model (RAM) for zero-shot image tagging, Grounding DINO for open-set object detection bounding boxes, and the Segment Anything Model (SAM) to generate clean, object-level RGBA cutouts.
  2. Element Selection and Clustering (Stage II): Using GPT-4o, the pipeline analyzes the user's text narrative against the extracted asset tags. Crucially, it performs divergent selection, identifying both "central labels" (explicitly mentioned in the text) and "related labels" (contextually implied assets to enrich the scene). The LLM then classifies these assets into functional roles (characters, backgrounds, accessories) and organizes them into a nested semantic hierarchy. For character assets, Sapiens is used to perform 2D part segmentation and keypoint estimation, automatically defining rotation centers for future animation.
  3. Asset Presentation (Stage III): Assets are displayed in a user interface featuring a linked tree view and canvas layout. Elements are adaptively sized based on a composite selection score that balances content diversity, story consistency (via CLIP embedding similarities), and image resolution.
The three-stage Collaposer pipeline: Preprocessing, Selection & Clustering, and Asset Presentation.

Datasets

The system was evaluated using a fixed photo set of 116 images preprocessed into 2,427 distinct visual elements. The images were randomly sampled from public-domain historical and scientific digital archives:

  • The NASA Archives
  • The Olympics Archive
  • The Library of Congress

No new machine learning benchmarks were introduced; these datasets served as the testing ground for the user evaluation and application scenarios.

Key Insights and Ideas

  • Semantic Inference Sparks Serendipity: Traditional search limits creators to what they can explicitly name. By leveraging an LLM to infer contextually relevant assets (e.g., suggesting "clouds" and "trees" for a landscape story), Collaposer expands the creative space and provides unexpected visual inspiration.
  • Functional Role Classification Reduces Cognitive Load: Displaying thousands of cutouts in a flat grid is overwhelming. Sorting assets into structural roles (characters, backgrounds, accessories) mirrors the natural workflow of artists, making asset evaluation significantly more intuitive.
  • Pre-Rigging for Animation: By integrating part-level parsing and keypoint estimation directly into the asset preparation phase, the tool bridges the gap between static graphic design and dynamic animation tools like Adobe After Effects.

Key Results

The authors conducted a within-subjects user study ($N=12$) benchmarking Collaposer against two ablated baselines: Ablated-Select (no semantic inference/related labels) and Ablated-Present (no semantic clustering or adaptive sizing).

  • 100% One-Pass Success: Under a strict 15-minute task budget, all participants using Collaposer achieved a satisfactory asset set on their very first prompt submission, requiring zero re-prompting iterations.
  • Superior Usability and Alignment: Quantitative 7-point Likert scale ratings showed that Collaposer significantly outperformed Ablated-Select in prompt alignment and visual expectation, and significantly beat Ablated-Present in ease of element identification.
  • Downstream Integration: The exported structured JSON files and layered cutouts were successfully imported into commercial animation software via scripts, enabling rapid production of complex, articulated stop-motion animations.
Figure 1: Based on the input photo collection and story descriptions, Collaposer supports visual assets preparation for collage- based storytelling by automatically selecting, cutting out, and presenting diverse visual assets. Given a curated overview of the extracted visual elements, users can compose them into a static collage, and animate them to create expressive visual stories. Photo collections included are illustrative examples from public-domain image collections [33, 40].

Limitations and Future Work

  • Text-Based Constraints: Relying solely on text prompts can occasionally cause misalignments between a user's mental image and the system's output. Incorporating multimodal inputs (like sketch-based queries) is a suggested next step.
  • Lack of Transparency: Users occasionally found the LLM's semantic inferences confusing, pointing to a need for explainable AI features where the system explains why an asset was recommended.
  • Single-Pass Workflow: The current pipeline operates linearly. Future work aims to support continuous, iterative refinement loops, allowing users to dynamically rename, regroup, or re-segment assets on the fly without restarting the pipeline.

Summary updated

Abstract

Digital collage is an artistic practice that combines image cutouts to tell stories. However, preparing cutouts from a set of photos remains a tedious and time-consuming task. A formative study identified three main challenges: 1) inefficient search for relevant photos, 2) manual image cutout, and 3) difficulty in organizing large sets of cutouts. To meet these challenges and facilitate asset preparation for collage, we propose Collaposer, a tool that transforms a collection of photos into organized, ready-to-use visual cutouts based on user-provided story descriptions. Collaposer tags, detects, and segments photos, and then uses an LLM to select central and related labels based on the user-provided story description. Collaposer presents the resulting visuals in varying sizes, clustered according to semantic hierarchy. Our evaluation shows that Collaposer effectively automates the preparation process to produce diverse sets of visual cutouts adhering to the storyline, allowing users to focus on collaging these assets for storytelling. Project website: https://jiayzhou.github.io/collaposer-website/
BibTeX
@inproceedings{zhou2026collaposer,
  author = {Jiayi Zhou and Liwenhan Xie and Jiaju Ma and Zheng Wei and Huamin Qu and Anyi Rao},
  title = {{Collaposer: Transforming Photo Collections into Visual Assets for Storytelling with Collages}},
  booktitle = {CHI},
  year = {2026},
  doi = {10.1145/3772318.3791160},
}
Figure