In-Memory Big Data Management and Processing: A Survey

Hao Zhang, Gang Chen, Beng Chin Ooi, Kian-Lee Tan, Meihui Zhang

Abstract

Growing main memory capacity has fueled the development of in-memory big data management and processing. By eliminating disk I/O bottleneck, it is now possible to support interactive data analytics. However, in-memory systems are much more sensitive to other sources of overhead that do not matter in traditional I/O-bounded disk-based systems. Some issues such as fault-tolerance and consistency are also more challenging to handle in in-memory environment. We are witnessing a revolution in the design of database systems that exploits main memory as its data storage layer. Many of these researches have focused along several dimensions: modern CPU and memory hierarchy utilization, time/space efficiency, parallelism, and concurrency control. In this survey, we aim to provide a thorough review of a wide range of in-memory data management and processing proposals and systems, including both data storage systems and data processing frameworks. We also give a comprehensive presentation of important technology in memory management, and some key factors that need to be considered in order to achieve efficient in-memory data management and processing.
BibTeX
@article{zhang2015memory,
  author = {Hao Zhang and Gang Chen and Beng Chin Ooi and Kian-Lee Tan and Meihui Zhang},
  title = {{In-Memory Big Data Management and Processing: A Survey}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2015},
  doi = {10.1109/tkde.2015.2427795},
}

Loading related content...

Figure