Background Knowledge Injection for Interpretable Sequence Classification

Severin Gsponer, Luca Costabello, Chan Le Van, Sumit Pai, Christophe Guéret, Georgiana Ifrim, Freddy LécuéarXiv 2020 · arXiv

Abstract

Sequence classification is vital in domains like bioinformatics and activity recognition, yet practitioners often face a trade-off between the high accuracy of "black-box" deep learning models and the interpretability of linear models. This paper introduces Emb-SEQL, a sequence classification algorithm that integrates external background knowledge—such as WordNet or specialized protein databases—into linear models. By grouping semantically related symbols during training, Emb-SEQL maintains high predictive accuracy while enhancing interpretability. To quantify this improvement, the authors propose Semantic Fidelity, a novel objective metric that measures the alignment between model features and domain-specific concepts. Experimental results on Human Activity Recognition (HAR) and protein classification tasks demonstrate that Emb-SEQL outperforms traditional baselines in both performance and interpretability.

Introduction

Sequence classification involves assigning labels to ordered series of symbols, such as amino acid sequences or sensor data streams. While deep learning models like LSTMs and CNNs achieve state-of-the-art accuracy, their internal logic remains opaque. Conversely, linear models are interpretable but often fail to capture semantic relationships between different symbols that represent similar underlying concepts. Furthermore, the field lacks a quantitative, human-independent metric for interpretability. This work addresses these gaps by proposing Emb-SEQL and the Semantic Fidelity metric.

Methods

The proposed framework, Emb-SEQL (Embedding-enriched SEQL), builds upon the Sequence Learner (SEQL) algorithm. While standard SEQL relies on exact $k$-mer matches, Emb-SEQL leverages external knowledge graph or word embeddings to group semantically similar symbols. For instance, in a kitchen activity dataset, "knife" and "spoon" can be grouped under a "Kitchen Tool" abstraction. This allows the model to learn generalized rules that are more meaningful to human observers.

Overview of the Emb-SEQL framework and interpretability evaluation pipeline.

To evaluate the interpretability of these models objectively, the authors introduce Semantic Fidelity (SF). SF calculates the distance in an embedding space between the features selected by the model and the target class concept. A higher SF score indicates that the model's logic aligns more closely with established domain knowledge.

Datasets and Background Knowledge

The researchers evaluated Emb-SEQL on two distinct domains:

  • OPPORTUNITY (HAR): A Human Activity Recognition dataset containing sensor data from wearables. The task was to predict top-level activities (like "Coffee time") from low-level sensor symbols.
  • Protein Classification: An excerpt from the PhosphoELM database, classifying amino acid sequences into specific protein kinase groups (PKA vs. SRC).

Background knowledge was sourced from several large-scale repositories, including WordNet, YAGO-41, GloVe, ConceptNet, and ChEBI-ChEMBL.

Statistics of the knowledge graphs used for background knowledge injection.

Key Results and Insights

  • Accuracy: Emb-SEQL achieved an F1-score of 0.96-0.97 on the HAR dataset, significantly outperforming LSTM (0.76) and SVM (0.50) baselines. Importantly, the injection of background knowledge did not compromise accuracy compared to standard SEQL.
  • Interpretability Boost: On the HAR task, using WordNet increased Semantic Fidelity by 4.6% over the standard model. On the protein dataset, Emb-SEQL reached an SF of 0.719, outperforming the standard SEQL (0.708) while maintaining comparable accuracy (~0.90).
  • Beyond Exact Matches: Standard sequence models are often brittle. By allowing "OR" combinations of symbols (e.g., A or B followed by C), Emb-SEQL captures the underlying intent of a sequence rather than just literal noise.

Conclusion

This research demonstrates that background knowledge injection can bridge the gap between performance and interpretability in sequence classification. By moving beyond exact matches to semantic groupings, Emb-SEQL captures the underlying intent of sequences while remaining computationally efficient. Future work will focus on automating the selection of grouping parameters and extending the Semantic Fidelity metric to other model architectures.

Summary updated

BibTeX
@misc{gsponer2020background,
  author = {Severin Gsponer and Luca Costabello and Chan Le Van and Sumit Pai and Christophe Guéret and Georgiana Ifrim and Freddy Lécué},
  title = {{Background Knowledge Injection for Interpretable Sequence Classification}},
  year = {2020},
  eprint = {2006.14248},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2006.14248},
}
Figure