A Perspective on Future Research Directions in Information Theory

Jeffrey G. Andrews, Alexandros G. Dimakis, Lara Dolecek, Michelle Effros, Muriel Médard, Olgica Milenkovic, Andrea Montanari, Sriram Vishwanath, Edmund M. Yeh, Randall Berry, Ken R. Duffy, Soheil Feizi, Saul Kato, Manolis Kellis, Stuart Licht, Jon Sorenson, Lav R. Varshney, Haris VikaloarXiv 2015 · OpenReview

TL;DR

This paper, authored by an ad hoc committee of the IEEE Information Theory Society, surveys the current landscape and future potential of information theory. It argues that information theory's rigorous framework is poised to expand its impact beyond traditional communications into diverse fields like neuroscience, genomics, and economics, driving innovation and solving complex problems.

What Problem Does This Paper Address?

Information theory, a foundational mathematical science, has historically revolutionized telecommunications. As the field approaches its 70th anniversary, this paper addresses the critical question of its future trajectory. It seeks to identify promising new research directions, anticipate areas where information theory will have the most significant impact in the next 10-20 years, and explore how the Information Theory Society can foster high-risk, high-reward research while maintaining its intellectual rigor. The paper aims to bridge the gap between information theory's established strengths and its emerging applications in a rapidly evolving scientific and technological landscape.

Methods

This paper is not a typical research paper presenting novel algorithms or experimental results. Instead, it's a comprehensive survey and forward-looking perspective based on extensive discussions and detailed inputs from experts across various disciplines. The authors, an ad hoc committee, engaged in debates and solicited insights from specialists in fields including genomics, biology, economics, and neuroscience. The "methodology" is essentially a structured synthesis of these collective discussions, aiming to articulate current and future connections of information theory with other domains.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Information Theory as a Universal Language: The paper posits that information theory's abstract and rigorous framework is increasingly relevant to fields that have traditionally relied on observational or heuristic approaches. It's becoming an indispensable tool, akin to calculus or computer programming.
  • Beyond Communications: While communications remains a core area, the paper strongly advocates for the deepening and broadening of information theory's boundaries into new fields. This expansion is seen as crucial for its future growth and impact.
  • Interdisciplinary Synergy: The report highlights numerous areas where information theory can offer profound insights and drive breakthroughs. These include:
  • Neuroscience: Understanding the brain as an information processing organ, from sensory systems to consciousness.
  • Genomics and Molecular Biology: Applying information-theoretic tools to analyze genetic information, evolutionary processes, and biological systems.
  • Economics and Finance: Bridging the gap between information theory and economic models, particularly in areas like investment theory, rationality, and the economic value of information.
  • Theoretical Computer Science: Exploring the intersection of communication complexity, coding theory, and graph theory.
  • Physics: Leveraging mathematical tools from physics to tackle complex information-theoretic problems and vice-versa.
  • Embracing New Abstractions: To address the challenges in these emerging areas, information theorists will need to develop and adopt new abstractions, models, tools, and methods, pushing the boundaries of the field.

Key Results

This paper does not present quantitative results in the traditional sense. Its "results" are the identified future directions and the articulation of the potential impact of information theory across various disciplines. The key outcome is a roadmap and a call to action for the information theory community to actively engage with and contribute to these emerging fields.

Limitations and Future Work

Limitations: * The authors acknowledge that the report is by no means exhaustive and makes no attempt to cite the vast literature in each area. * The paper focuses on identifying potential connections and future directions, rather than providing in-depth technical solutions for each area.

Future Work: The paper explicitly outlines numerous future directions within each discussed domain, including:

  • Communications: Massive MIMO, finite block length coding, delay-tolerant feedback, and industry collaboration.
  • Networks: Multi-terminal information theory, network coding, graph properties, nano-circuit design, and reliable data storage.
  • Control Theory: Feedback channels, cognitive radios, control over communication channels, and human-in-the-loop systems.
  • Neuroscience: Sensory systems, cognition, brain as a controller/data network, predictive information, and consciousness.
  • Signal Processing: Interference alignment, full-duplex communication, and human information acquisition.
  • Statistics and Machine Learning: Computational efficiency, high-dimensional statistical theory.
  • Genomics and Molecular Biology: DNA processing with noise, adapting IT tools to genomics, evolutionary biology, data compression, and bio-inspired algorithms.
  • Theoretical Computer Science: Communication complexity, locally repairable codes, and index coding.
  • Physics: Mathematical tools from physics, intersection of theory and empirical research, and physics-inspired algorithms.
  • Economics and Finance: Investment theory, rationality, economic value of information, optimal pricing, and information economics.

The overarching future work suggested is for information theorists to actively develop new abstractions, models, tools, and methods to tackle the complex problems in these interdisciplinary areas.

Summary updated

Abstract

Information theory is rapidly approaching its 70th birthday. What are promising future directions for research in information theory? Where will information theory be having the most impact in 10-20 years? What new and emerging areas are ripe for the most impact, of the sort that information theory has had on the telecommunications industry over the last 60 years? How should the IEEE Information Theory Society promote high-risk new research directions and broaden the reach of information theory, while continuing to be true to its ideals and insisting on the intellectual rigor that makes its breakthroughs so powerful? These are some of the questions that an ad hoc committee (composed of the present authors) explored over the past two years. We have discussed and debated these questions, and solicited detailed inputs from experts in fields including genomics, biology, economics, and neuroscience. This report is the result of these discussions.
BibTeX
@misc{andrews2015perspective,
  author = {Jeffrey G. Andrews and Alexandros G. Dimakis and Lara Dolecek and Michelle Effros and Muriel Médard and Olgica Milenkovic and Andrea Montanari and Sriram Vishwanath and Edmund M. Yeh and Randall Berry and Ken R. Duffy and Soheil Feizi and Saul Kato and Manolis Kellis and Stuart Licht and Jon Sorenson and Lav R. Varshney and Haris Vikalo},
  title = {{A Perspective on Future Research Directions in Information Theory}},
  year = {2015},
  eprint = {1507.05941},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1507.05941},
}
Figure