A Prior-information-guided Residual Diffusion Model for Multi-modal PET Synthesis from MRI

Zaixin Ou, Caiwen Jiang, Yongsheng Pan, Yuanwang Zhang, Zhiming Cui, Dinggang Shen

TL;DR

This paper introduces FGeo-HyperGNet, a novel neural-symbolic system for solving plane geometry problems. By integrating a formal symbolic system (FormalGeo) with a hypergraph neural network (HyperGNet), it achieves human-like, traceable, and verifiable geometric problem-solving, setting new state-of-the-art results on the FormalGeo7K dataset.

What Problem Does This Paper Address?

Geometric Problem Solving (GPS) is a long-standing challenge in AI due to its reliance on both symbolic reasoning and numerical computation, often involving complex cross-modal information (text and diagrams). Existing approaches often fall into two camps: deductive database methods that struggle with scalability and traceability, or program sequence generation methods that lack readability and guaranteed correctness. A key limitation is the difficulty in creating formal geometric systems that are both expressive and easily integrated with neural networks, often requiring code modifications for new theorems. This paper aims to bridge this gap by proposing a system that can automatically solve geometric problems, generate human-readable solutions, and ensure correctness through a tight integration of neural and symbolic components.

Methods

The core of FGeo-HyperGNet is a neural-symbolic architecture that operates in a Predict-Apply Cycle (PAC).

  • Symbolic Component (FormalGeo): This system formalizes geometric problems and their solutions as a directed hypergraph. Conditions are represented as hypernodes, and theorems as hyperedges. FormalGeo handles geometric relational reasoning and algebraic calculations, ensuring mathematical rigor and traceability.
  • Neural Component (HyperGNet): This is a hypergraph neural network designed to predict the next theorem to be applied. It consists of:
  • Hypernode Encoder: This module, inspired by the Transformer architecture, encodes the semantic and structural information of hypernodes and their neighboring hyperedges into fixed-length vectors. It uses a self-supervised approach to learn rich representations.
  • Theorem Predictor: This component takes the encoded hypergraph and goal information to predict the most relevant theorem for the current problem state. It uses a task-specific attention mechanism.
  • Predict-Apply Cycle (PAC): The neural component predicts a theorem, and the symbolic component applies it, updates the hypergraph, and generates new conditions. This cycle repeats until the problem is solved.
  • FGeo-Parser: A component that converts raw geometric problem descriptions (images and text) into a formalized language, enabling the system to handle multimodal inputs.
  • Solution Generator: A rule-based system that leverages the structured problem-solving process to generate human-like solutions.

Datasets

The paper evaluates FGeo-HyperGNet on three datasets:

  • FormalGeo7K: The primary dataset used for evaluation, containing formalized geometric problems.
  • Geometry3K: Used for comparison with other methods.
  • GeoQA: Also used for comparison.

The authors also describe a process of generating training data by topologically sorting theorem DAGs derived from the FormalGeo7K dataset, creating pairs of (hypergraph, applicable theorems).

Key Insights and Ideas

  • Hypergraph Representation for GPS: Representing the problem-solving process as a hypergraph, where conditions are nodes and theorems are edges, effectively captures the structured nature of geometric reasoning.
  • Neural-Symbolic Integration for Traceability: Combining neural theorem prediction with a symbolic execution engine is crucial for achieving both high performance and verifiable, human-readable solutions.
  • Hypergraph Neural Networks for Formal Language Encoding: Standard NLP models struggle with the structured and symbolic nature of formal languages. HyperGNet's hypergraph encoder is specifically designed to capture this structural and semantic information.
  • Predict-Apply Cycle (PAC): This iterative process of neural prediction and symbolic application is key to the system's ability to progressively solve complex problems.
  • Importance of Pretraining and Graph Structure: Ablation studies highlight that pretraining the hypernode encoder and explicitly leveraging the hypergraph structure significantly boost performance, underscoring the value of these components.

Key Results

FGeo-HyperGNet achieved state-of-the-art results on the FormalGeo7K dataset, with:

  • Theorem Prediction Accuracy (TPA): 93.50%
  • Problem-Solving Success Rate (PSSR): 88.36%

The system demonstrated superior performance across various difficulty levels, particularly excelling on more challenging problems where other methods saw significant performance drops. For instance, it achieved 56.67% on the hardest level (L6), compared to 36.59% for the second-best method. The paper also shows that FGeo-HyperGNet has a significantly smaller number of parameters (20.38 million) compared to other large language models.

Limitations and Future Work

The authors acknowledge several limitations:

  • Equation-Solving Timeouts: The system can experience timeouts when solving complex systems of equations, especially those involving trigonometric functions.
  • Rare Problem Types: For very rare problem types, the model may struggle to learn adequate solution methods due to insufficient training data.

Future work includes:

  • Reinforcement Learning: Integrating reinforcement learning into the neural component to further optimize theorem selection.
  • Auxiliary Construction: Enhancing the symbolic component with auxiliary construction capabilities to tackle more complex geometric constructions.
  • IMO-Level GPS: Aiming for automatic IMO-level geometric problem solving without human supervision.

Summary updated

BibTeX
@inproceedings{ou2024prior,
  author = {Zaixin Ou and Caiwen Jiang and Yongsheng Pan and Yuanwang Zhang and Zhiming Cui and Dinggang Shen},
  title = {{A Prior-information-guided Residual Diffusion Model for Multi-modal PET Synthesis from MRI}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure