High-Dimensional Dataset Entropy Estimation via Lossy Compression

Ivan Butakov, Sofia Malanchuk, Anna Neopryatnaya, Alexander Tolmachev, Kirill Andreev, Stanislav Kruglik, Evgeny Marshakov, Alexey Frolov

Abstract

This paper proposes a method for estimating a high-dimensional dataset entropy based on compression via autoencoder. We give theoretical proof of such estimation relevance and illustrate it by experiments with synthetic data and the MNIST digits dataset.
BibTeX
@article{butakov2021high,
  author = {Ivan Butakov and Sofia Malanchuk and Anna Neopryatnaya and Alexander Tolmachev and Kirill Andreev and Stanislav Kruglik and Evgeny Marshakov and Alexey Frolov},
  title = {{High-Dimensional Dataset Entropy Estimation via Lossy Compression}},
  journal = {Journal of Communications Technology and Electronics},
  year = {2021},
  doi = {10.1134/s1064226921060061},
}

Loading related content...

Figure