Frank-Wolfe Network: An Interpretable Deep Structure for Non-Sparse Coding
Lists
Only you see this
Abstract
The problem of L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> -norm constrained coding is to convert signal into code that lies inside an L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> -ball and most faithfully reconstructs the signal. Previous works under the name of sparse coding considered the cases of L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> and L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> norms. The cases with p > 1 values, i.e. non-sparse coding studied in this paper, remain a difficulty. We propose an interpretable deep structure namely Frank-Wolfe Network (F-W Net), whose architecture is inspired by unrolling and truncating the FrankWolfe algorithm for solving an L p-norm constrained problem with p ≥ 1. We show that the Frank-Wolfe solver for the L p-norm constraint leads to a novel closed-form nonlinear unit, which is parameterized by p and termed pool p. The pool p unit links the conventional pooling, activation, and normalization operations, making F-W Net distinct from existing deep networks either heuristically designed or converted from projected gradient descent algorithms. We further show that the hyper-parameter p can be made learnable instead of pre-chosen in F-W Net, which gracefully solves the non-sparse coding problem even with unknown p. We evaluate the performance of F-W Net on an extensive range of simulations as well as the task of handwritten digit recognition, where F-W Net exhibits strong learning capability. We then propose a convolutional version of FW Net, and apply the convolutional F-W Net into image denoising and super-resolution tasks, where F-W Net all demonstrates impressive effectiveness, flexibility, and robustness.
BibTeX
@article{liu2020frank,
author = {Dong Liu and Ke Sun and Zhangyang Wang and Runsheng Liu and Zheng-Jun Zha},
title = {{Frank-Wolfe Network: An Interpretable Deep Structure for Non-Sparse Coding}},
journal = {IEEE Trans. Circuits Syst. Video Technol.},
year = {2020},
doi = {10.1109/tcsvt.2019.2936135},
}