Weakly Supervised Neural Symbolic Learning for Cognitive Tasks

Jidong Tian, Yitian Li, Wenqing Chen, Liqiang Xiao, Hao He, Yaohui JinAAAI 2022 · OpenReview

Abstract

Despite the recent success of end-to-end deep neural networks, there are growing concerns about their lack of logical reasoning abilities, especially on cognitive tasks with perception and reasoning processes. A solution is the neural symbolic learning (NeSyL) method that can effectively utilize pre-defined logic rules to constrain the neural architecture making it perform better on cognitive tasks. However, it is challenging to apply NeSyL to these cognitive tasks because of the lack of supervision, the non-differentiable manner of the symbolic system, and the difficulty to probabilistically constrain the neural network. In this paper, we propose WS-NeSyL, a weakly supervised neural symbolic learning model for cognitive tasks with logical reasoning. First, WS-NeSyL employs a novel back search algorithm to sample the possible reasoning process through logic rules. This sampled process can supervise the neural network as the pseudo label. Based on this algorithm, we can backpropagate gradients to the neural network of WS-NeSyL in a weakly supervised manner. Second, we introduce a probabilistic logic regularization into WS-NeSyL to help the neural network learn probabilistic logic. To evaluate WS-NeSyL, we have conducted experiments on three cognitive datasets, including temporal reasoning, handwritten formula recognition, and relational reasoning datasets. Experimental results show that WS-NeSyL not only outperforms the end-to-end neural model but also beats the state-of-the-art neural symbolic learning models.

Introduction

Deep Neural Networks (DNNs) excel at perception tasks like image recognition or text transcription. However, they often struggle with complex logical reasoning. Neuro-Symbolic Learning (NeSyL) attempts to bridge this gap by combining neural perception with symbolic logic. Despite its promise, NeSyL faces three primary challenges: a lack of supervision for intermediate reasoning steps, the non-differentiable nature of discrete symbolic systems, and the difficulty of aligning probabilistic neural outputs with deterministic logic rules.

Methods

The WS-NeSyL framework addresses these challenges through two main innovations. First, it utilizes Back Search Sampling based on the Metropolis-Hastings algorithm. Since ground-truth labels for intermediate reasoning steps are typically unavailable, the model starts from the final answer and searches backward through logic rules to identify plausible intermediate paths. These paths serve as pseudo-labels to guide the neural network.

Second, the model incorporates Probabilistic Logic Regularization (PLR). PLR uses gradient semirings to create a differentiable loss function, ensuring that the network's predicted probabilities for intermediate steps remain mathematically consistent with the final logical outcome. This bridges the gap between "soft" neural predictions and "hard" symbolic logic.

The unified Encoder-Decoder framework of WS-NeSyL, featuring specialized decoders for perception (Dec-P) and reasoning (Dec-R).

The architecture employs a unified Encoder-Decoder framework, as shown above, where separate decoders handle perception (identifying symbols) and reasoning (applying rules).

Datasets and Evaluation

WS-NeSyL was evaluated on three distinct cognitive benchmarks:

  • MHTBD (Multi-hop TimeBank-Dense): A temporal reasoning task focused on extracting time-based information from text.
  • MHHWF (Multi-hop Handwritten Formula): A vision task requiring the recognition of handwritten digits and operators followed by mathematical computation.
  • CLUTRR: A relational reasoning benchmark where the model infers kinship relations from short family narratives.

Key Results

Experimental results demonstrate that WS-NeSyL consistently outperforms both pure end-to-end models and existing state-of-the-art neuro-symbolic methods.

On the MHTBD dataset, WS-NeSyL achieved 72.9% accuracy, significantly surpassing the DeepProbLog baseline. For MHHWF, it reached a peak accuracy of 96.7%, showing strong generalization in out-of-domain tests.

Experimental results on the CLUTRR dataset, demonstrating WS-NeSyL's superior performance in relational reasoning across different training settings.

As shown in the table above, WS-NeSyL maintained high accuracy on the CLUTRR dataset even when reasoning across more "hops" than encountered during training. This suggests that the model effectively learns underlying logical rules rather than merely memorizing patterns. A key insight is the model's ability for "perception correction," where logical constraints help correct misidentified symbols if they lead to mathematically inconsistent results.

Conclusion

WS-NeSyL provides a robust framework for integrating neural perception with symbolic reasoning under weak supervision. By leveraging back-search sampling and probabilistic regularization, it overcomes traditional barriers in neuro-symbolic learning. Future work will explore automatic rule selection and addressing "cold start" issues in highly complex reasoning scenarios.

Summary updated

Abstract

Despite the recent success of end-to-end deep neural networks, there are growing concerns about their lack of logical reasoning abilities, especially on cognitive tasks with perception and reasoning processes. A solution is the neural symbolic learning (NeSyL) method that can effectively utilize pre-defined logic rules to constrain the neural architecture making it perform better on cognitive tasks. However, it is challenging to apply NeSyL to these cognitive tasks because of the lack of supervision, the non-differentiable manner of the symbolic system, and the difficulty to probabilistically constrain the neural network. In this paper, we propose WS-NeSyL, a weakly supervised neural symbolic learning model for cognitive tasks with logical reasoning. First, WS-NeSyL employs a novel back search algorithm to sample the possible reasoning process through logic rules. This sampled process can supervise the neural network as the pseudo label. Based on this algorithm, we can backpropagate gradients to the neural network of WS-NeSyL in a weakly supervised manner. Second, we introduce a probabilistic logic regularization into WS-NeSyL to help the neural network learn probabilistic logic. To evaluate WS-NeSyL, we have conducted experiments on three cognitive datasets, including temporal reasoning, handwritten formula recognition, and relational reasoning datasets. Experimental results show that WS-NeSyL not only outperforms the end-to-end neural model but also beats the state-of-the-art neural symbolic learning models.
BibTeX
@inproceedings{tian2022weakly,
  author = {Jidong Tian and Yitian Li and Wenqing Chen and Liqiang Xiao and Hao He and Yaohui Jin},
  title = {{Weakly Supervised Neural Symbolic Learning for Cognitive Tasks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  url = {https://openreview.net/forum?id=NbRxY89773},
}
Figure