Low Precision Representations for High Dimensional Models
Lists
Only you see this
Abstract
The large memory footprint of high dimensional models require quantization to a lower precision for deployment on resource constrained edge devices. With this motivation, we consider the problems of learning a (i) linear regressor, and a (ii) linear classifier from a given training dataset, and quantizing the learned model parameters subject to a pre-specified bit-budget. The error metric is the prediction risk of the quantized model, and our proposed randomized embedding-based quantization methods attain near-optimal error while being computationally efficient. We provide fundamental bounds on the bit-budget constrained minimax risk that, together with our proposed algorithms, characterize the minimum threshold budget required to achieve a risk comparable to the unquantized setting. We also show the efficacy of our strategy by quantizing a two-layer ReLU neural network for non-linear regression. Numerical simulations show the improved performance of our proposed scheme as well as its closeness to the lower bound.
BibTeX
@inproceedings{saha2023low,
author = {Rajarshi Saha and Mert Pilanci and Andrea J. Goldsmith},
title = {{Low Precision Representations for High Dimensional Models}},
booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
year = {2023},
doi = {10.1109/icassp49357.2023.10095529},
}Loading related content...