A Large-Scale Film Style Dataset for Learning Multi-frequency Driven Film Enhancement

Zinuo Li, Xuhang Chen, Shuqiang Wang, Chi-Man Pun

Abstract

This paper introduces FilmSet, the first large-scale, high-quality dataset specifically designed for film-style color grading, featuring over 5,000 images across three distinct film aesthetics. To accompany the data, the authors propose FilmNet, a deep learning framework that uses a Laplacian Pyramid to separately enhance image features across different frequency bands. This approach significantly outperforms state-of-the-art methods in both accuracy and visual appeal, bridging the gap between digital convenience and analog aesthetics.

Introduction

While digital photography offers unparalleled convenience, many photographers still seek the unique "soul" of analog film—characterized by specific color science, grain, and a timeless aesthetic. Currently, achieving this look digitally relies on Look-Up Tables (LUTs) or labor-intensive manual editing. These methods often result in artifacts because they apply global changes without considering local image details.

A major hurdle in automating this process has been the lack of dedicated, large-scale datasets for film stylization. Existing benchmarks like MIT-Adobe FiveK focus on general enhancement rather than specific stylization. Furthermore, standard deep learning models often struggle to balance global color shifts with the preservation of fine textures and edges.

Methods

The authors propose FilmNet, a novel architecture based on the principle of multi-frequency decomposition. The model operates in several key stages:

  1. Laplacian Pyramid Decomposition: FilmNet breaks the image into different frequency bands, allowing it to treat color (low frequency) and texture/edges (high frequency) as separate tasks.
  2. Nonlinear Stylization Remapping (NSR): Low-frequency components are processed through a lightweight UNet-style architecture to handle complex color transformations.
  3. High-frequency Refinement: A Multi-Scale Reconstruction Module (MSRM) uses global-aware convolutions and attention mechanisms to protect important textures and edges during stylization.
  4. Triple Trilinear Regulator (TTR): A learned 3D LUT approach performs final global refinement, ensuring the output maintains a consistent tone characteristic of specific film stocks.

Datasets

The primary contribution is FilmSet, a dataset containing 5,285 high-resolution images. Each image is paired with three "ground truth" versions corresponding to Fujifilm-inspired styles: Cinema, Classic Negative, and Velvia. These were generated using professional-grade recipes in Capture One to ensure high fidelity to analog looks.

Sample of a film-style output from the FilmSet dataset, demonstrating the target aesthetic for color and tone.

In addition to FilmSet, the authors utilized the MIT-Adobe FiveK and HDR+ datasets to evaluate the model's general enhancement and high dynamic range capabilities.

Key Results

FilmNet achieves state-of-the-art (SOTA) performance across multiple benchmarks. Quantitatively, on the FilmSet "Cinema" style, FilmNet achieved a PSNR of 40.07, significantly higher than other SOTA methods like LPTN (36.55) or 3D-LUT (35.49). It also consistently demonstrated the lowest $\Delta E$ (color difference) scores, indicating high faithfulness to target film recipes.

Qualitative comparison of FilmNet against other methods. FilmNet avoids the over-exposure and flat looks of competing models, maintaining deep shadows and vibrant highlights.

Qualitatively, visual comparisons show that FilmNet avoids the "muddy" or over-exposed look common in standard CNN-based enhancers. By separating frequency bands, the model preserves the organic sharpness of the original image while applying sophisticated color grading.

Conclusion and Future Work

FilmNet successfully bridges the gap between modern deep learning and traditional color grading. By creating FilmSet, the authors provide the necessary "vocabulary" for AI to understand analog aesthetics. Future work may involve expanding the dataset to include iconic stocks from Kodak and Agfa, as well as integrating explicit grain synthesis to further enhance the organic feel of high-ISO film simulations. Current limitations include hardware constraints for ultra-high-resolution processing due to the depth of the Laplacian Pyramid.

Summary updated

Abstract

Film, a classic image style, is culturally significant to the whole photographic industry since it marks the birth of photography. However, film photography is time-consuming and expensive, necessitating a more efficient method for collecting film-style photographs. Numerous datasets that have emerged in the field of image enhancement so far are not film-specific. In order to facilitate film-based image stylization research, we construct FilmSet, a large-scale and high-quality film style dataset. Our dataset includes three different film types and more than 5000 in-the-wild high resolution images. Inspired by the features of FilmSet images, we propose a novel framework called FilmNet based on Laplacian Pyramid for stylizing images across frequency bands and achieving film style outcomes. Experiments reveal that the performance of our model is superior than state-of-the-art techniques. The link of our dataset and code is https://github.com/CXH-Research/FilmNet.
BibTeX
@inproceedings{li2023large,
  author = {Zinuo Li and Xuhang Chen and Shuqiang Wang and Chi-Man Pun},
  title = {{A Large-Scale Film Style Dataset for Learning Multi-frequency Driven Film Enhancement}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.24963/ijcai.2023/129},
}
Figure