Recurrent Expert Networks

AAAI 1996

TL;DR

This paper introduces "recurrent expert networks," a novel hybrid approach that combines the pattern recognition strengths of neural networks with the explicit knowledge representation of expert systems. By incorporating recurrent connections, these networks can maintain "state" information, enabling them to tackle problems requiring context and sequential reasoning, a limitation of traditional feed-forward expert networks.

What Problem Does This Paper Address?

Traditional expert systems, while powerful for encoding explicit knowledge, can be brittle and difficult to scale due to the "knowledge acquisition bottleneck." On the other hand, neural networks excel at pattern recognition but struggle to provide interpretable "reasons" for their classifications, leading to skepticism from human experts. Hybrid systems aim to bridge this gap by combining both approaches. However, existing hybrid systems, particularly those based on expert networks (which translate expert rules into network topologies), are typically feed-forward. This architecture limits their ability to handle problems where the solution requires maintaining and utilizing "state" information – the context that changes as a problem is solved. The authors identify this limitation and propose a solution for problems that inherently depend on such contextual information.

Methods

The core innovation is the introduction of recurrent connections into expert networks. Expert networks, as previously developed, translate expert system rules into a network topology where nodes represent parts of the knowledge base and trainable weights represent certainty factors. Data is fed through these networks, and learning algorithms like backpropagation adjust the weights.

The paper proposes extending this by adding recurrent connections. These connections allow information from previous steps (i.e., the network's "state") to be fed back into the network at subsequent steps. This mechanism is analogous to how recurrent neural networks handle sequential data and state maintenance. The authors envision this enabling expert networks to retain context, which is crucial for problems where the interpretation of current information depends on past events or states.

Datasets

Not discussed in the paper. The paper focuses on the conceptual framework and potential application domain rather than empirical evaluation on specific datasets.

Key Insights and Ideas

  • Bridging the interpretability-performance gap: Recurrent expert networks offer a path to combine the accuracy of neural networks with the explainability of expert systems, addressing a long-standing challenge in AI.
  • Stateful reasoning for expert systems: The introduction of recurrent connections directly tackles the limitation of feed-forward architectures in handling problems that require maintaining and utilizing contextual "state" information.
  • Protein folding as a prime application: The authors highlight the protein folding problem as a domain where state information (e.g., predictions for earlier amino acids) is critical for predicting the structure of subsequent amino acids, making it an ideal testbed for recurrent expert networks.

Key Results

Not discussed in the paper. The paper is a theoretical proposal and does not present empirical results or quantitative evaluations.

Limitations and Future Work

  • Theoretical Framework: The paper presents a conceptual extension of expert networks. The practical implementation and empirical validation of recurrent expert networks are left for future work.
  • Application Domain: While the protein folding problem is identified as a promising application, the paper does not detail how the translation from protein folding rules to a recurrent expert network topology would be performed or evaluated.
  • Learning Algorithms: The paper mentions backpropagation-like algorithms for learning in expert networks but doesn't explicitly detail how these would be adapted for recurrent connections in this hybrid context.

Summary updated

BibTeX
@inproceedings{1996recurrent,
  title = {{Recurrent Expert Networks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1996},
}
Figure