Unsupervised Coding with LOCOCODE

Sepp Hochreiter, Jürgen SchmidhuberICANN 1997 · OpenReview

Abstract

Traditional approaches to sensory coding use code component-oriented objective functions (COCOFs) to evaluate code quality. Previous COCOFs do not take into account the information-theoretic complexity of the code-generating mapping itself. We do: “Low-complexity coding and decoding” (LOCOCODE) generates so-called lococodes that (1) convey information about the input data, (2) can be computed from the data by a low-complexity mapping (LCM), and (3) can be decoded by a LCM. We implement LococoDE by training autoassociators with Flat Minimum Search (FMS), a general method for finding lowcomplexity neural nets. LococoDE extracts optimal codes for difficult versions of the “bars” benchmark problem. As a preprocessor for a vowel recognition benchmark problem it sets the stage for excellent classification performance.
BibTeX
@inproceedings{hochreiter1997unsupervised,
  author = {Sepp Hochreiter and Jürgen Schmidhuber},
  title = {{Unsupervised Coding with LOCOCODE}},
  booktitle = {ICANN},
  year = {1997},
  url = {https://openreview.net/forum?id=Nx27qlJxeN},
}

Loading related content...

Figure