Data Storage and Management for Image AI Pipelines

Utku Sirin, Stratos Idreos

Abstract

Image AI is essential for various applications, such as self-driving cars, medical imaging, and smart farming. Data management is key for efficient image AI, from how to store images to how to manage data while processing the images. This tutorial overviews the emerging area of image AI pipelines by combining approaches from various cross-disciplinary areas such as data management, digital signal processing, computer vision, and machine learning. We specifically focus on image storage and data management. The tutorial first gives an overview of image AI pipelines step by step, how they work and the main the challenges. We then describe the main approaches to making image AI pipelines more efficient. We first cover how image AI pipelines store images based on stan- dard storage formats, learned formats, task-specific learned formats, and self-designed formats. Second, we cover how state-of-the-art approaches manage data within image AI pipelines. We identify and describe three main approaches to making image AI pipelines more efficient by efficiently managing data within the pipeline: (i) compressing intermediate data, (ii) materializing and re-using data objects, and (iii) parallelism for better hardware utilization. Lastly, the tutorial covers open data management and systems problems and future directions in making image AI pipelines more efficient.
BibTeX
@inproceedings{sirin2025data,
  author = {Utku Sirin and Stratos Idreos},
  title = {{Data Storage and Management for Image AI Pipelines}},
  booktitle = {Sigmod Conference Companion},
  year = {2025},
  doi = {10.1145/3722212.3725640},
}

Loading related content...

Figure