Optimization Model of Reliable Data Storage in Cloud Environment Using Genetic Algorithm

Feng Liu, Haitao Wu, Xiaochun Lu, Xiyang LiuInternational Journal of Grid and Distributed Computing 2014 · DOI

TL;DR

This paper tackles the challenge of optimizing data storage in cloud environments by developing a multi-objective model that balances storage cost and reliability. They propose a genetic algorithm (GA) to find optimal storage solutions, demonstrating that storing data files individually can significantly reduce costs while maintaining acceptable reliability, offering a more flexible approach for cloud providers.

What Problem Does This Paper Address?

The explosion of digital data means cloud storage is becoming increasingly critical, but it presents a dual challenge for service providers: managing ever-growing storage capacity and associated costs, while ensuring data reliability and availability. Different applications have varying needs for storage space and reliability, leading to inefficient resource utilization where allocated space isn't fully used. Furthermore, data loss in cloud environments can have severe consequences, from financial disaster to legal disputes. Existing solutions often struggle to find the right balance between these competing objectives, leading to either high costs or insufficient reliability. This paper aims to address this by creating a model that explicitly optimizes for both storage cost and reliability simultaneously.

Methods

The core of this paper is a multi-objective optimization model for reliable data storage. The model considers two primary objectives:

  • Minimizing Total Storage Cost: This includes the cost of data storage itself, data migration between servers, and communication costs for data transfer.
  • Maximizing Storage Reliability: This encompasses the reliability of data transmission, hardware dependability, and server intrinsic reliability.

To solve this complex optimization problem, the authors employ a Genetic Algorithm (GA), specifically the NSGA-II (Non-dominated Sorting Genetic Algorithm II). GA is chosen for its robustness and ability to handle multi-objective problems. The paper details the GA's parameters, including population size, crossover probability (set high at 0.95 for faster convergence), mutation probability (set high at 0.9 to aid in finding feasible solutions in a constrained problem), selection strategy (roulette selection with elitism), and stopping conditions.

Datasets

The paper uses publicly available data from two cloud service providers: Amazon EC2 and GoGrid. This data includes information on storage instance types, CPUs, memory, and pricing across different geographical regions (US-east, US-west, EU-west). Network data transmission prices and available bandwidth from UA to cloud service providers are also utilized. No new datasets were introduced; rather, existing commercial cloud service data was used to simulate and validate the proposed model.

Key Insights and Ideas

  • The Trade-off is Real: The experiments clearly show that as storage reliability increases, storage costs tend to rise sharply. This highlights the fundamental challenge in cloud storage optimization.
  • Individual File Storage is Key: Storing data files individually (as opposed to as a single integral file) offers significant cost reduction while still achieving acceptable reliability. This provides greater flexibility and more storage solution options.
  • Migration Frequency Impacts Reliability: Higher migration frequency of data files directly leads to a decrease in storage reliability. This underscores the importance of network stability and minimizing unnecessary data movement.
  • Safety Levels Matter: Data files with higher safety levels require more resources (like CPU time for encryption) and thus incur higher storage prices, impacting the overall cost.
  • GA is Effective: The NSGA-II algorithm successfully generates multiple Pareto-optimal solutions, offering a range of cost-reliability trade-offs for users to choose from.

Key Results

The experimental results demonstrate the effectiveness of the proposed model and GA.

  • When storing data as an integral file, the model found storage costs between $10,000-$16,000 for reliability greater than 0.5.
  • When storing sub-files individually, the optimal storage cost was found to be between $5,500-$9,500 with a reliability range of 0.60-0.95. This represents a significant cost saving.
  • The study showed that for a given reliability level, individual sub-file storage yields a noticeably lower storage cost compared to integral file storage.
  • The GA was able to produce multiple representative solutions, with storage reliability distributed uniformly between 0.63-0.95 in many scenarios, providing clients with diverse options.

Limitations and Future Work

  • Computational Complexity: While GA is effective, it can be computationally intensive, especially for very large datasets or complex environments.
  • Parameter Sensitivity: The performance of GA can be sensitive to the chosen parameters (population size, crossover/mutation rates).
  • Future Work: The authors suggest that further research could explore more diversified storage solutions to enhance attractiveness in the storage service market. They also imply that the model's ability to provide multiple solutions is helpful for service providers to effectively utilize storage resources in data centers. The paper also mentions that in the time of migration, all servers are required to be traversed, which might be an area for optimization in future work.

Summary updated

BibTeX
@article{liu2014optimization,
  author = {Feng Liu and Haitao Wu and Xiaochun Lu and Xiyang Liu},
  title = {{Optimization Model of Reliable Data Storage in Cloud Environment Using Genetic Algorithm}},
  journal = {International Journal of Grid and Distributed Computing},
  year = {2014},
  doi = {10.14257/ijgdc.2014.7.6.14},
}
Figure