Abstract
Rule-Constrained Networks (RCNs) represent a novel hybrid architecture designed to bridge the gap between high-performance "black box" deep neural networks and interpretable rule-based systems. By forcing a model to make decisions by selecting from a set of human-readable "if-then" rules, RCNs ensure that every prediction is backed by a logical justification without sacrificing predictive accuracy. This paper evaluates RCNs across time-series and sentiment classification tasks, demonstrating that they achieve performance parity with standard neural networks while providing inherent interpretability.
Introduction
Deep neural networks (DNNs) have achieved state-of-the-art results in various domains, yet their internal logic remains notoriously difficult to interpret. While post-hoc explanation methods like LIME or SHAP attempt to explain decisions after the fact, they cannot guarantee that the explanation perfectly reflects the model's internal reasoning. Conversely, traditional interpretable models like decision trees often fail to capture complex, latent patterns in raw data, leading to lower accuracy. RCNs address this trade-off by utilizing the feature-learning power of DNNs to select the most appropriate human-readable rule for any given input.
Methodology
Instead of a standard output layer predicting a class label, an RCN predicts a specific rule from a predefined ruleset. The architecture incorporates several key technical components to ensure validity and efficiency:
- Filtered Softmax: This mechanism restricts the network to selecting only those rules whose "if" conditions (antecedents) are satisfied by the current input. This guarantees that the outputted rule is always valid for the specific instance.
- RCN+A (Attention Mechanism): This variant represents rules as latent vectors in a high-dimensional space. By calculating similarity between rules, the model better exploits logical relationships, improving accuracy while reducing parameter count.
- Joint Ruleset Optimization: Using a generalized Expectation-Maximization (EM) algorithm and Metropolis-Hastings sampling, the system prunes massive, redundant rulesets down to a distilled set of the most useful rules.
Experimental Setup
The RCN framework was evaluated on two primary sequence classification tasks:
- Time-Series Classification: Five binary datasets from the UCR archive, including spectrograph data (Strawberry) and bone outline datasets (Prox and Phar).
- Sentiment Classification: Three large-scale text review datasets, including IMDB, Elec, and Yelp.
Rules were pre-mined using standard techniques such as Eclat or extracted from the paths of a Random Forest model.
Results and Discussion
RCNs achieved error rates nearly identical to standard "black box" neural networks. For instance, on the Strawberry dataset, the base CNN recorded a 4.1% error rate, while the RCN+A achieved 3.8%. Furthermore, RCNs significantly outperformed traditional interpretable baselines like CART (Decision Trees) and SBRL (Bayesian Rule Lists).
Qualitatively, the model demonstrates "interpretability by design." As shown in the figure below, the RCN provides a specific logical justification for its classification of a time-series signal, whereas a standard base network only provides a probability score.
Beyond accuracy, the ruleset optimization successfully reduced complexity. On the IMDB dataset, the model maintained high accuracy while distilling thousands of initial rules down to approximately 100 essential ones. This succinctness allows for a direct trade-off between the model's logical complexity and its predictive power.
Conclusion
RCNs provide a robust framework for building models that are interpretable by construction. While the current implementation depends on pre-mined rules and focuses on classification, it offers a clear path toward "human-in-the-loop" AI where human-validated logic can be prioritized. Future work will explore extending RCNs to regression tasks and conducting human subject studies to quantify the practical utility of these rule-based justifications in real-world applications.
Summary updated