Data Science: Challenges and Directions

Longbing CaoCommunications of the ACM 2017 · DOI · OpenReview

TL;DR

This paper argues that "data science" is a nascent, interdisciplinary field facing significant challenges due to the inherent "X-complexities" (data, behavior, domain, social, environment, learning, deliverables) and the need for "X-intelligence" (data, behavior, domain, human, network, organizational, social, environmental) to derive actionable insights. It proposes a complex systems perspective to navigate these challenges and outlines future research directions, particularly in handling non-IID data and developing human-like machine intelligence.

What Problem Does This Paper Address?

The paper addresses the burgeoning field of "data science," which has seen widespread adoption but lacks a clear, systematic understanding of its intrinsic complexities and future directions. While many discussions focus on the hype and the transformation from existing disciplines like statistics and computer science, there's a critical gap in understanding the low-level complexities embedded in data science problems and the intelligence required to tackle them. The authors aim to provide a structured view of these challenges and opportunities, framing data science as a complex system that requires a holistic approach to transform data into knowledge and intelligence for decision-making.

Methods

The paper doesn't present a specific algorithmic method or empirical study with datasets in the traditional sense. Instead, it offers a conceptual framework and a high-level analysis of the field. The "methods" described are more about a way of thinking and approaching data science problems:

  • Complex Systems Perspective: Viewing data science problems as complex systems with interconnected "X-complexities."
  • X-Intelligence Synthesis: Emphasizing the need to integrate various forms of intelligence (data, behavior, domain, human, etc.) for effective problem-solving.
  • Qualitative-to-Quantitative Metasynthesis: A proposed methodology for exploring open complex systems by engaging diverse intelligences, moving from qualitative understanding to quantitative insights.
  • Metasynthetic Computing and Engineering (MCE): An approach that provides a systematic framework for engineering complex intelligent systems.

Datasets

Not discussed in the paper. The paper is a conceptual overview and does not present empirical results based on specific datasets.

Key Insights and Ideas

  • Data Science as a Complex System: The core idea is that data science problems are not isolated tasks but intricate systems with multiple interacting complexities (X-complexities).
  • The Ubiquity of "X-Intelligence": To navigate these complexities, a broad spectrum of intelligence (X-intelligence) is crucial, extending beyond just data analysis to include domain expertise, human intuition, and societal context.
  • The "Known-to-Unknown" Progression: Data science is framed as a journey from known states (where capabilities match visibility) to unknown states (where capabilities lag behind the complexity of the problem), highlighting the need for discovery and innovation.
  • Non-IID Data as a Major Hurdle: A significant challenge is the prevalence of non-Independent and Identically Distributed (non-IID) data, which violates assumptions of many traditional statistical and machine learning methods.
  • The Revolution of Human-like Machine Intelligence: The paper posits that data science is driving a shift towards machine intelligence that mimics human imagination and curiosity, potentially transforming human-machine roles.
  • The Need for Interdisciplinary Synthesis: Effective data science requires integrating knowledge and methodologies from a wide array of fields, moving beyond siloed disciplinary approaches.

Key Results

This paper is primarily a conceptual framework and a call for future research, rather than a report of empirical results. Therefore, there are no quantitative results (accuracy, speedup, etc.) to summarize. The "results" are the insights and directions outlined:

  • A structured categorization of "X-complexities" and "X-intelligence" in data science.
  • Identification of key research challenges, including handling non-IID data, developing new analytical frameworks, and addressing social and ethical issues.
  • A proposed conceptual landscape for data science research.
  • A vision for the future of data science, emphasizing human-like machine intelligence and the synthesis of diverse intelligences.

Limitations and Future Work

Limitations:

  • The paper is largely theoretical and conceptual, lacking empirical validation of its proposed frameworks or methodologies.
  • While it identifies challenges, it doesn't offer concrete, ready-to-implement solutions for many of them.
  • The "X-complexities" and "X-intelligence" are broad categories, and their precise operationalization and measurement may require further definition.

Future Work:

  • Developing New Theories and Tools for Non-IID Data: This is highlighted as a critical area, requiring new similarity and evaluation metrics, feature analysis, and learning algorithms.
  • Advancing Human-like Machine Intelligence: Research into simulating human imagination, curiosity, and creative thinking in machines is essential.
  • Integrating Diverse Intelligences: Further development of methodologies like metasynthesis to effectively combine various forms of intelligence for complex problem-solving.
  • Addressing Data Quality and Social Issues: Continued focus on privacy, security, trust, and ethical considerations in data science.
  • Bridging the Gap Between Theory and Practice: Developing practical tools and frameworks that can be applied to real-world complex data science problems.
  • Refining the Conceptual Landscape: Further exploration and refinement of the data science landscape and its constituent research areas.

Summary updated

Abstract

While data science has emerged as a contentious new scientific field, enormous debates and discussions have been made on it why we need data science and what makes it as a science. In reviewing hundreds of pieces of literature which include data science in their titles, we find that the majority of the discussions essentially concern statistics, data mining, machine learning, big data, or broadly data analytics, and only a limited number of new data-driven challenges and directions have been explored. In this paper, we explore the intrinsic challenges and directions inspired by comprehensively exploring the complexities and intelligence embedded in data science problems. We focus on the research and innovation challenges inspired by the nature of data science problems as complex systems, and the methodologies for handling such systems.
BibTeX
@article{cao2017data,
  author = {Longbing Cao},
  title = {{Data Science: Challenges and Directions}},
  journal = {Communications of the ACM},
  year = {2017},
  doi = {10.1145/3015456},
}
Figure