TL;DR
This paper introduces KBANN, a hybrid machine learning system that combines symbolic knowledge with neural networks to improve gene recognition in DNA. By initializing neural networks with existing biological rules, KBANN significantly reduces the amount of training data needed to achieve high accuracy, making it a powerful tool for analyzing complex biological sequences.
What Problem Does This Paper Address?
The paper tackles the challenge of identifying specific DNA sequences, known as splice-junctions, which are crucial for gene expression in eukaryotic cells. While these sequences have identifiable patterns, they are often ambiguous and can be mimicked by non-functional sequences. Traditional methods struggle to reliably distinguish true splice-junctions from these false positives. The authors aim to develop a more accurate and efficient method for this task by leveraging existing biological knowledge, which is often in the form of "rules of thumb" or inference rules, and integrating it with the learning capabilities of artificial neural networks (ANNs).
Methods
The core innovation presented is the KBANN (Knowledge-Based Artificial Neural Networks) system. KBANN works by:
- Knowledge Base Translation: It takes a knowledge base of domain-specific inference rules (similar to PROLOG clauses) that are "roughly-correct" and translates them into an initial artificial neural network.
- ANN Structure and Initialization: The structure of the ANN directly mirrors the hierarchical structure of the rules. Units in the network represent antecedents and consequents of the rules, and links represent dependencies. Weights are initialized based on whether the dependency is necessary (positive weight) or prohibitory (negative weight).
- Empirical Refinement: This pre-initialized network is then trained using actual DNA sequence examples. The learning algorithm adjusts the weights and biases of the network (except for those derived from definitional rules) to refine the initial knowledge and improve classification accuracy.
This approach is novel because it doesn't start with a blank slate (randomly initialized network) but rather with a structured network informed by prior knowledge, which is then fine-tuned.
Datasets
- Training and Testing Data: The study used a dataset of DNA sequences derived from documented split genes in primate genomes.
- Positive Examples: 751 examples of acceptor splice-junctions and 745 examples of donor splice-junctions. Each example was a window of 60 nucleotides centered around the junction.
- Negative Examples: A comparable number of sequences that did not cross intron/exon boundaries, sampled randomly.
- Total Dataset Size: 3190 examples.
- Benchmark Comparison: For comparison, a standard, randomly-initialized, fully-connected, two-layer ANN with 24 hidden units was also evaluated.
Key Insights and Ideas
- Knowledge as a Starting Point: Pre-existing, even if imperfect, biological knowledge can be a powerful starting point for machine learning, significantly accelerating the learning process.
- Bridging Symbolic and Connectionist AI: KBANN offers a concrete method for integrating symbolic reasoning (rules) with connectionist learning (neural networks), a long-standing goal in AI.
- Efficient Learning: By providing a structured initial network, KBANN dramatically reduces the number of training examples required to reach a certain level of accuracy compared to traditional ANNs.
- Interpretable Initialization: The initial KBANN network's structure provides some insight into which features are considered important by the domain theory.
Key Results
- Faster Learning: KBANN-initialized networks learned significantly faster than randomly-initialized ANNs. They made less than half the errors when presented with 100 or fewer training examples.
- Improved Accuracy (with limited data): For smaller training sets, KBANN achieved substantially lower error rates.
- Comparable Accuracy (with large data): When a very large number of training examples were provided, the randomly-initialized ANNs achieved a slightly lower error rate (5.5% vs. 6.4% for KBANN), though the paper notes all differences were statistically significant. This suggests KBANN's primary advantage is in scenarios with limited labeled data.
- Network Size: The final KBANN-created network for the splice-junction problem had 286 units (3 output, 240 input, 31 fixed-weight hidden, and 12 tunable hidden units).
Limitations and Future Work
- Limitations:
- KBANN is currently restricted to non-recursive, propositional (variable-free) sets of rules.
- The paper acknowledges that understanding what a trained neural network has learned can be difficult.
- Future Work:
- Investigating methods for automatically translating trained KBANN networks back into symbolic rules to improve interpretability.
- Exploring the use of richer, more complex splice-junction domain theories to further improve KBANN's accuracy.
- Applying KBANN to other biological sequence analysis tasks.
Summary updated