Variance reduction with neighborhood smoothing for local intrinsic dimension estimation
Lists
Only you see this
Abstract
Local intrinsic dimension estimation has been shown to be useful for many tasks such as image segmentation, anomaly detection, and de-biasing global dimension estimates. Of particular concern with local dimension estimation algorithms is the high variance for high dimensions, leading to points which lie on the same manifold estimating at different dimensions. We propose adding adaptive 'neighborhood smoothing' - filtering over the generated dimension estimates to obtain the most probable estimate for each sample - as a method to reduce variance and increase algorithm accuracy. We present a method for defining neighborhoods using a geodesic distance, which constricts each neighborhood to the manifold of concern, and prevents smoothing over intersecting manifolds of differing dimension. Finally, we illustrate the benefits of neighborhood smoothing on synthetic data sets as well as towards diagnosing anomalies in router networks.
BibTeX
@inproceedings{carter2008variance,
author = {Kevin M. Carter and Alfred O. Hero III},
title = {{Variance reduction with neighborhood smoothing for local intrinsic dimension estimation}},
booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
year = {2008},
doi = {10.1109/icassp.2008.4518510},
}Loading related content...