Iterative applications of image completion with CNN-based failure detection
Lists
Only you see this
Abstract
Highlights • A framework of uman-in-the-loop style image completion is proposed. • Failure inpainted regions are automatically detected using a deep neural network. • A set of features tailored for patch-based image completion are used. • Our system is comparable to human-in-the-loop completion by real human annotators. Abstract Image completion is a technique to fill missing regions in a damaged or redacted image. A patch-based approach is one of major approaches, which solves an optimization problem that involves pixel values in missing regions and similar image patch search. One major problem of this approach is that it sometimes duplicates implausible texture in the image or overly smooths down a missing region when the algorithm cannot find better patches. As a practical remedy, the user may provide an interaction to identify such regions and re-apply image completion iteratively until she/he gets a desirable result. In this work, inspired by this idea, we propose a framework of human-in-the-loop style image completion with automatic failure detection using a deep neural network instead of human interaction. Our neural network takes small patches extracted from multiple feature maps obtained from the completion process as input for the automated interaction process, which is iterated several times. We experimentally show that our neural network outperforms a conventional linear support vector machine. Our subjective evaluation demonstrates that our method drastically improves the visual quality of resulting images compared to non-iterative application.
BibTeX
@article{tanaka2018iterative,
author = {Takahiro Tanaka and Norihiko Kawai and Yuta Nakashima and Tomokazu Sato and Naokazu Yokoya},
title = {{Iterative applications of image completion with CNN-based failure detection}},
journal = {J. Vis. Commun. Image Represent.},
year = {2018},
doi = {10.1016/j.jvcir.2018.05.015},
}Loading related content...