A Survey on AI Sustainability: Emerging Trends on Learning Algorithms and Research Challenges

Zhenghua Chen, Min Wu, Alvin Chan, Xiaoli Li, Yew-Soon Ong

Introduction

As artificial intelligence models grow exponentially in size and capability, the field faces a dual crisis: a massive environmental carbon footprint and mounting social concerns regarding bias and privacy. Historically, the "state-of-the-art" in AI has been synonymous with "bigger is better." However, this progress has reached a tipping point of diminishing returns regarding sustainability. This survey provides a comprehensive technical roadmap for "Sustainable AI," categorizing the algorithmic innovations that aim to make high-performance machine learning both environmentally "green" and socially responsible.

The Sustainability Crisis

The authors highlight two major gaps in current AI development:

  1. Environmental Cost: Training a single large Transformer model can emit over 600,000 pounds of CO2—roughly equivalent to the lifetime emissions of five cars. Compute requirements for AI are currently doubling every 3.4 months, far outstripping Moore’s Law.
  2. Social Friction: As AI permeates critical sectors like healthcare and law, "black-box" models raise urgent questions about fairness, data privacy, and safety against adversarial attacks.
Comparison of CO2 emissions between daily activities and training large-scale AI models.

A Taxonomy for Sustainable AI

To move the conversation from high-level ethics to concrete technical solutions, the landscape of Sustainable AI is organized into four technical pillars. This taxonomy allows researchers to target specific inefficiencies in the model lifecycle, from data collection to deployment.

The four technical pillars of Sustainable AI: Computation-efficient, Data-efficient, Responsible, and Rationalizable & Resilient AI.

1. Computation-efficient AI

These techniques aim to shrink models for deployment on low-power devices. Key methods include:

  • Pruning: Removing redundant neurons or weights to reduce model size.
  • Quantization: Using lower-bit precision for weights and activations.
  • Knowledge Distillation: Training a small "student" model to mimic the performance of a large, energy-intensive "teacher" model.

2. Data-efficient AI

Reducing the "data hunger" of AI is critical for sustainability. The paper reviews Active Learning (selecting only the most informative samples for human labeling), Few-shot Learning, and Self-supervised Learning, which allows models to learn from vast amounts of unlabeled data, reducing the human and energy cost of annotation.

3. Responsible AI

Sustainability includes social longevity. This pillar covers Fairness (integrating bias-removal metrics directly into loss functions) and Privacy. A major focus is Federated Learning (FL), where models are trained across decentralized devices without ever exchanging raw user data, preserving privacy by design.

4. Rationalizable & Resilient AI

Improving trust is essential for sustainable adoption. This involves Interpretability (making "black boxes" explainable) and Safety (defending against adversarial perturbations and data poisoning that could render a model useless or dangerous).

Key Insights and Results

  • The 3.4-Month Doubling Rate: AI compute requirements are doubling nearly seven times faster than the hardware improvements predicted by Moore’s Law. This makes algorithmic efficiency a necessity, not an elective.
  • The "Smart Data" Shift: Sustainability requires moving from "Big Data" to "Smart Data." Techniques like Active Learning can significantly reduce the carbon footprint by ensuring energy is only spent training on the most valuable data points.
  • Carbon Benchmarks: The survey provides sobering quantitative data, noting that a GPT-3 model (175B parameters) produces over 1.2 million pounds of CO2, equivalent to the lifetime emissions of 10 cars.
  • Disciplinary Expansion: A visualization of the last decade shows AI papers exploding not just in Computer Science, but in traditional fields like Physics and Environmental Science, emphasizing that AI sustainability is now a global scientific concern.

Limitations and Future Directions

Despite progress, several hurdles remain. The authors note a Training vs. Inference Trade-off: while compressed models are efficient during use (inference), the process of pruning or distilling them often requires significant training time and energy upfront.

Future research must move toward Energy-Aware Evaluation, where energy consumption and resource utilization are reported alongside traditional accuracy metrics. Furthermore, the authors advocate for Model Sharing Communities to prevent redundant energy waste, encouraging a "safe-sharing" ecosystem where pre-trained models are reused and fine-tuned rather than trained from scratch by every research group.

Summary updated

Abstract

Artificial Intelligence (AI) is a fast-growing research and development (R&D) discipline which is attracting increasing attention because of its promises to bring vast benefits for consumers and businesses, with considerable benefits promised in productivity growth and innovation. To date it has reported significant accomplishments in many areas that have been deemed as challenging for machines, ranging from computer vision, natural language processing, audio analysis to smart sensing and many others. The technical trend in realizing the successes has been towards increasing complex and large size AI models so as to solve more complex problems at superior performance and robustness. This rapid progress, however, has taken place at the expense of substantial environmental costs and resources. Besides, debates on the societal impacts of AI, such as fairness, safety and privacy, have continued to grow in intensity. These issues have presented major concerns pertaining to the sustainable development of AI. In this work, we review major trends in machine learning approaches that can address the sustainability problem of AI. Specifically, we examine emerging AI methodologies and algorithms for addressing the sustainability issue of AI in two major aspects, i.e., environmental sustainability and social sustainability of AI. We will also highlight the major limitations of existing studies and propose potential research challenges and directions for the development of next generation of sustainable AI techniques. We believe that this technical review can help to promote a sustainable development of AI R&D activities for the research community.
BibTeX
@misc{chen2022survey,
  author = {Zhenghua Chen and Min Wu and Alvin Chan and Xiaoli Li and Yew-Soon Ong},
  title = {{A Survey on AI Sustainability: Emerging Trends on Learning Algorithms and Research Challenges}},
  year = {2022},
  eprint = {2205.03824},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.03824},
}
Figure