Fusionista2.0: Efficiency Retrieval System for Large-Scale Datasets

Huy M. Le, Dat Tien Nguyen, Phuc Binh Nguyen, Gia-Bao Le-Tran, Phu Truong-Thien, Cuong V. Dinh, Minh Nguyen, Nga Nguyen, Truong Thao Nguyen, Gia Huy Ngo, Tan Nhat Nguyen, Binh T. Nguyen, Thao Nguyen TruongLecture Notes in Computer Science 2026 · DOI · arXiv · OpenAlex

Introduction

Interactive video retrieval systems are essential for navigating thousands of hours of video content based on diverse user queries, including text, images, and audio. In competitive environments like the Video Browser Showdown (VBS), the challenge lies in balancing retrieval accuracy with extreme time constraints. Previous iterations, such as Fusionista and Fustar, utilized state-of-the-art AI models for tasks like keyframe extraction and question answering. However, these heavy models created significant computational bottlenecks when scaled to multi-terabyte datasets. Fusionista2.0 addresses these limitations by re-engineering the pipeline for maximum efficiency without sacrificing retrieval effectiveness.

System Architecture

Fusionista2.0 is divided into two primary phases: Indexing and Searching. The indexing phase focuses on rapid data preparation and feature extraction, while the searching phase utilizes a multi-modal back-end to process user queries in real-time.

Overview of the Fusionista2.0 architecture, illustrating the indexing and searching pipelines.

Data Preparation and Extraction

To optimize the indexing stage, the authors replaced heavy AI-based scene detection with a deterministic ffmpeg workflow. This approach extracts intra-coded frames as structural keyframes, significantly reducing memory demands. For feature extraction, the system employs an ensemble of two CLIP variants (CLIP-Sig400M and CLIP-ViT-5B) with an empirically tuned weighting ($\alpha=0.7$) to better capture semantic nuances.

Lightweight Components

Efficiency is further gained by swapping standard models for streamlined alternatives:

  • OCR & ASR: Vintern-1B-v3.5 is used for multilingual text recognition, while faster_whisper accelerates audio transcription by 4x, focusing on speed for ambient-heavy datasets.
  • Vision-Language Models (VLMs): Massive models were replaced with compact versions ($\le$ 1B parameters) to handle frequent, simple queries like object counting or color recognition instantly.

Interactive Reranking and UI/UX

A novel post-processing pipeline was introduced to refine search results. Using GPT-4o, the system generates clarifying yes/no questions based on the user's query. A lightweight VLM then evaluates retrieved candidates against these questions to automatically rerank them.

To improve the user experience, the frontend was migrated to Vite, incorporating virtual scrolling and sidebar navigation. These changes, combined with WCAG accessibility compliance, minimize redundant browsing and streamline the search process for the human operator.

Results and Discussion

The transition to lightweight components yielded a 75% reduction in overall retrieval time. The authors validated their choice of models using a custom benchmark of 200 video-question pairs, focusing on counting, image information, and video information.

Performance comparison of various lightweight Vision-Language Models on the custom 200-question benchmark.

As shown in the performance table, the chosen model (InternVL-1B-ffn6-Seq) achieved high accuracy (84% for counting) with an average inference time of just 4.80 seconds. While these lightweight models may lack the deep reasoning of 7B+ parameter models, their speed allows for a more fluid interactive search where the human remains in the loop for complex decision-making.

Conclusion

Fusionista2.0 demonstrates that for real-time, large-scale video retrieval, deterministic preprocessing and sub-1B parameter models can outperform heavier state-of-the-art alternatives in practical utility. By reducing computational overhead and optimizing the user interface, the system provides a robust framework for the upcoming VBS 2026 competition and broader real-world applications.

Summary updated

BibTeX
@inproceedings{le2026fusionista,
  author = {Huy M. Le and Dat Tien Nguyen and Phuc Binh Nguyen and Gia-Bao Le-Tran and Phu Truong-Thien and Cuong V. Dinh and Minh Nguyen and Nga Nguyen and Truong Thao Nguyen and Gia Huy Ngo and Tan Nhat Nguyen and Binh T. Nguyen and Thao Nguyen Truong and Tan M. Nguyen},
  title = {{Fusionista2.0: Efficiency Retrieval System for Large-Scale Datasets}},
  booktitle = {Lecture Notes in Computer Science},
  year = {2026},
  doi = {10.1007/978-981-95-6963-2\_17},
}
Figure