Beyond path selection: Better LLMs for Scientific Information Extraction with MimicSFT and Relevance and Rule-induced(R2)GRPO

Ran Li, Shimin Di, Yuchen Liu, Chen Jing, Yu Qiu, Lei ChenarXiv 2025 · DOI · arXiv

Abstract

This paper introduces a two-stage training framework—MimicSFT and $R^2$GRPO—designed to enable Large Language Models (LLMs) to outperform specialized BERT-based models in Scientific Information Extraction (SciIE). While LLMs often struggle with the domain-specific reasoning and precise boundary detection required for SciIE, the proposed method allows them to achieve state-of-the-art results. Our findings suggest that reinforcement learning with verifiable rewards (RLVR) enhances a model's reasoning capacity beyond simple "path selection," leading to significant performance gains on the SciER benchmark.

Introduction

Scientific Information Extraction (SciIE) is a complex task involving the identification of entities (e.g., "Machine Learning") and their relationships (e.g., "Used-For") within dense academic texts. Despite the success of reasoning-heavy models like DeepSeek-R1, they frequently underperform compared to smaller, specialized models in SciIE due to the dual requirements of domain-specific memorization and systematic reasoning. This research addresses this gap by proposing a novel training pipeline that enhances both the structural reasoning and precision of LLMs in scientific domains.

Methodology: MimicSFT and $R^2$GRPO

The authors propose a two-stage training pipeline to refine LLM performance:

  1. MimicSFT (Supervised Fine-Tuning): To circumvent the need for expensive, human-annotated Chain-of-Thought (CoT) data, the researchers utilize "pseudo-reasoning templates." These templates guide the model through a structured process—identifying entities, considering potential relations, and then formulating the final extraction. This "mimicry" helps the model internalize the logical structure of reasoning even without high-quality logical content.
  2. $R^2$GRPO (Reinforcement Learning): Building on Group Relative Policy Optimization (GRPO), this stage introduces a "Relevance and Rule-induced" composite reward function. The model is rewarded based on four factors: F1 accuracy, entity span precision (using Jaccard overlap), adherence to logical rule patterns (e.g., using keywords like "leads to"), and relevancy to the source text to prevent hallucinations.
The R2GRPO framework (right) and its comparative performance (left), showing significant improvements over base models and competitive results against specialized architectures.

Results and Key Insights

The proposed $R^2$GRPO* framework achieved state-of-the-art results on the SciER benchmark, which contains 24k entities and 12k relations. Key performance metrics include:

  • NER (Named Entity Recognition): Achieved an F1 score of 88.90, surpassing the previous best supervised model, HGERE (86.85).
  • Relation Extraction (Rel+): Achieved an F1 score of 74.03, a massive jump over standard SFT (41.01) and the specialized HGERE model (61.10).

A critical finding of the study is that RLVR is more than just "path selection." By analyzing "Best@K" scores—the best result out of $K$ attempts—the researchers found that the model's upper-bound performance improved. This suggests the model is not just getting better at picking the right answer from its existing knowledge but is actually acquiring new domain-specific reasoning capabilities. Furthermore, the research highlights that the structure of the thought process and its relevance to the text are more important than the length of the "thinking" block; excessively long reasoning often led to "noisy" thinking and degraded performance.

Detailed F1 score performance on the SciER benchmark for NER and Relation Extraction across different training sample sizes.

Conclusion

The combination of MimicSFT and $R^2$GRPO successfully bridges the gap between general-purpose LLMs and specialized scientific extraction models. By focusing on structured reasoning and multi-faceted rewards, the framework achieves superior precision and generalization, even on out-of-distribution data. Future work will explore the scalability of this approach to larger models and its applicability to other specialized fields such as legal or medical information extraction.

Summary updated

Abstract

Previous study suggest that powerful Large Language Models (LLMs) trained with Reinforcement Learning with Verifiable Rewards (RLVR) only refines reasoning path without improving the reasoning capacity in math tasks while supervised-finetuning(SFT) with distillation can. We study this from the view of Scientific information extraction (SciIE) where LLMs and reasoning LLMs underperforms small Bert-based models. SciIE require both the reasoning and memorization. We argue that both SFT and RLVR can refine the reasoning path and improve reasoning capacity in a simple way based on SciIE. We propose two-stage training with 1. MimicSFT, using structured reasoning templates without needing high-quality chain-of-thought data, 2. R$^2$GRPO with relevance and rule-induced rewards. Experiments on scientific IE benchmarks show that both methods can improve the reasoning capacity. R$^2$GRPO with mimicSFT surpasses baseline LLMs and specialized supervised models in relation extraction. Our code is available at https://github.com/ranlislz/R2GRPO.
BibTeX
@misc{li2025beyond,
  author = {Ran Li and Shimin Di and Yuchen Liu and Chen Jing and Yu Qiu and Lei Chen},
  title = {{Beyond path selection: Better LLMs for Scientific Information Extraction with MimicSFT and Relevance and Rule-induced(R2)GRPO}},
  year = {2025},
  eprint = {2505.22068},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.22068},
}
Figure