Introduction
In online education, providing "similar exercises" is a critical pedagogical tool for reinforcing concepts after a student makes an error. However, the "Finding Similar Exercises" (FSE) task is challenging due to the massive scale of exercise banks, the multi-modal nature of content (text, LaTeX, diagrams), and subtle semantic nuances where similar-looking text may require different logic. Tencent researchers have introduced the Exercise Semantic Representation Model (ESRM), a robust retrieval framework designed to address these challenges by moving beyond simple keyword matching toward deep semantic understanding.
System Architecture: Recall, Ranking, and Re-rank
The FSE problem is framed as a multi-stage retrieval pipeline to balance efficiency and accuracy across millions of items.
- Recall: Filters ~10 million exercises down to the top ~100 candidates using a mix of traditional text search (Elasticsearch) and vector-based retrieval.
- Ranking: Uses a sophisticated model to narrow the list to ~20 items. This stage employs Multi-Task Learning (MTL) and a Mixture of Experts (MoE) layer to dynamically weight semantic features most relevant to a specific pair of exercises.
- Re-rank: Finalizes the top ~10 results by applying personalized filters and specific logic to identify "variant" exercises that test the same concepts with different parameters.
ESRM: Multi-modal Multi-task Pre-training
At the heart of this system is the ESRM backbone. Unlike standard language models, ESRM is designed to understand the structured components of an exercise: the stem, options, answer, and analysis.
The model undergoes a three-stage training process:
- General Text Pre-training (GTP): Initializing with a BERT-style transformer trained on general corpora.
- Multi-modal Multi-task Pre-training (MMP): The core innovation where the model learns via auxiliary tasks simultaneously:
- Contrastive Learning: Aligning "stem + options" with "answer + analysis."
- Metadata Prediction: Learning to predict difficulty, exercise type, and knowledge concepts directly from the text.
- Image Prediction: Learning to associate exercise text with its corresponding diagrams or geometric figures.
- Supervised Fine-tuning (SFT): Refining the model on a high-quality, teacher-labeled dataset of exercise pairs.
Addressing Label Noise and Semantic Nuance
A significant hurdle in FSE is the subjectivity of "similarity." Human experts often disagree, leading to noisy training data. To mitigate this, the researchers implemented Confidence Learning to identify and prune inconsistent labels, preventing the model from overfitting to poor data.
Furthermore, the model prioritizes "problem-solving logic" over surface-level text similarity. By analyzing the "Analysis" section of an exercise, the model can distinguish between problems that look identical but require different mathematical approaches, or problems that look different but share the same underlying logic. This structural awareness allows for effective "cross-modal" matching, such as linking a question to its solution logic.
Experimental Results
The framework was evaluated using a dataset of ~350,000 junior high school math exercises from the Tencent Education Platform.
- Offline Performance: ESRM achieved a Recall (R@100) of 86.48% and a Ranking Precision (P@5) of 83.83%, significantly outperforming standard baselines like SBERT (75.91%).
- Online Impact: Real-world A/B testing showed a 15.98% increase in user acceptance rates, proving the effectiveness of the semantic retrieval approach in a production environment.
Conclusion
Tencent's ESRM framework demonstrates that effective exercise retrieval requires moving beyond text matching toward a structural and multi-modal understanding of educational content. Future work will explore Graph Neural Networks (GNNs) to better model relationships between knowledge concepts and further personalize the learning experience for individual student paths.
Summary updated