Lazy exact deduplication
Lists
Only you see this
Abstract
During data deduplication, on-disk fingerprint lookups lead to high disk traffic, resulting in a bottleneck. In this paper, we propose a “lazy” data deduplication method which buffers incoming fingerprints and performs on-disk lookups in batches, aiming to reduce the disk bottleneck. In deduplication in general, prefetching is used to improve the cache hit rate by exploiting locality within the incoming fingerprint stream. For lazy deduplication, we design a buffering strategy that preserves locality in order to similarly facilitate prefetching. Experimental results indicate that the lazy method improves fingerprint identification performance by over 50% compared with an “eager” method with the same data layout.
BibTeX
@inproceedings{ma2016lazy,
author = {Jingwei Ma and Rebecca J. Stones and Yuxiang Ma and Jingui Wang and Junjie Ren and Gang Wang and Xiaoguang Liu},
title = {{Lazy exact deduplication}},
booktitle = {MSST},
year = {2016},
doi = {10.1109/msst.2016.7897081},
}Loading related content...