Introduction
Traditional Chinese ink wash painting is characterized by "freehand brushwork"—a style defined by high-contrast black strokes, delicate contours, and the strategic use of "white space" to evoke an ethereal atmosphere. While video style transfer (VST) has advanced significantly, general-purpose methods often struggle with this specific medium. This paper introduces a specialized framework designed to handle the unique aesthetic and temporal requirements of ink wash animation.
Challenges in Ink Wash Video Transfer
Existing VST methods, primarily optimized for Western oil paintings, face two critical hurdles when applied to ink wash styles:
- Temporal Instability: The stark contrast between black ink and white paper makes minor misalignments in optical flow highly visible, resulting in distracting flickers.
- Semantic Ignorance: Standard models often treat images as uniform textures. In ink wash painting, the distinction between the subject (detailed strokes) and the background (intentional void) is vital. Without semantic awareness, results often appear cluttered and messy.
Methodology
The proposed solution consists of a two-pronged approach: a multi-frame fusion scheme for temporal coherence and an instance-aware stylization process for artistic fidelity.
Coherent Video Transfer
To ensure temporal stability, the model employs a multi-frame feature fusion strategy. Instead of a simple sequential approach, it aligns and fuses features from $k$ reference frames to stylize the target frame.
- Non-Sequential Processing: To prevent error accumulation (drift), the authors use a "binary tree-like" order for frame generation, ensuring diverse and stable reference points.
- Consistency Loss: A spatial-transformation consistency loss ($L_{cons}$) is introduced to make the GAN-based generator robust to small spatial changes, further minimizing jitter.
Instance-Aware Stylization
By integrating instance segmentation, the model gains a semantic understanding of the scene, allowing for more authentic artistic choices.
- White Space Constraint ($L_{whiten}$): This constraint encourages the model to leave the background blank, concentrating ink only on detected subjects.
- Contour Contrast Constraint ($L_{contour}$): Using Haar wavelets, the model sharpens high-frequency components at subject boundaries to mimic traditional contour brushwork.
- Adaptive Scale Selection: To ensure brushstrokes remain proportional regardless of subject distance, the model "normalizes" the scale of detected instances before processing.
Experimental Results
The model was trained on Chinese ink wash paintings from the ChipPhi dataset and a custom collection of 115 horse videos. Quantitative evaluations showed the method achieved a Fréchet Inception Distance (FID) score of 213.16, outperforming state-of-the-art models like CycleGAN and AdaIN.
Qualitative user studies confirmed a strong preference for this method, noting its superior temporal consistency and authentic "ink wash" look. The results are significantly cleaner than previous methods, with well-defined contours and appropriate use of negative space.
Conclusion
This framework successfully bridges the gap between neural style transfer and the specific nuances of Chinese ink wash painting. By treating white space as a semantic element and addressing scale-dependent brushwork, the model produces stable, artistically faithful videos. Future work will focus on reducing dependency on segmentation accuracy and expanding the model's repertoire to include landscapes and architecture.
Summary updated