Introduction
Automated resume parsing—the process of transforming unstructured PDF documents into structured data like job titles, graduation dates, and skills—is a critical component of modern recruitment technology. While deep learning has significantly advanced this field, existing state-of-the-art (SOTA) models face two primary challenges. First, most operate at a token (word) level, which is computationally expensive for long-form resumes and often misses the broader document structure. Second, they frequently fail to capture the inherent hierarchical nature of resumes, where segments (e.g., a specific job entry) are nested within larger blocks (e.g., "Work Experience").
To address these issues, the authors propose ERU (Efficient Resume Understanding), a multi-modal framework that shifts the focus from fine-grained token-level analysis to a structural segment-level approach. By treating lines or blocks of text as the primary unit of analysis, ERU better aligns with how humans visually navigate documents.
The ERU Framework
ERU processes resumes as a sequence of segments, integrating textual, visual, and layout information through a specialized architecture.
Multi-Modal Embedding and Fusion
For each segment, the model extracts three types of features:
- Textual: Encoded using a 6-layer BERT model.
- Visual: Extracted via a Faster R-CNN applied to the segment's specific image region.
- Layout: Incorporates both 1D reading order and 2D spatial coordinates (position, width, and height).
These features are integrated using a Layout-aware Multi-Modal Fusion Transformer. This graph-based transformer utilizes Absolute Position Bias to locate segments on a page and Relative Position Bias to model the relationships between neighboring segments (e.g., identifying that a date typically follows a company name).
Pre-training and Fine-Tuning
The model undergoes hierarchical pre-training on a massive corpus of 169,286 unlabeled resumes using three self-supervised objectives:
- Masked Language Model (MLM): Predicting missing text using layout context.
- Visual-Position Alignment (VPA): Predicting the relative spatial direction between segments based solely on visual features.
- Masked Segment Prediction (MSP): A contrastive learning task that aligns textual and visual representations of the same segment.
During fine-tuning, ERU employs a multi-granularity approach, simultaneously predicting labels for individual segments and broader document blocks, ensuring the model respects the document's hierarchical structure.
Key Results and Performance
ERU was evaluated against a manually annotated dataset of resumes, which averaged approximately 100 segments and 2.4 pages per document. The model achieved state-of-the-art performance, significantly outperforming established document AI models and generative Large Language Models (LLMs).
As shown in the results, ERU achieved an F1-score of 87.75%, surpassing LayoutLMv3 (84.36%). Notably, the "LLM IE" baseline (based on Baichuan-13b) struggled with an F1-score of only 61.24%, highlighting the difficulty purely text-based generative models have with complex, layout-dependent documents. Ablation studies further confirmed that the Masked Segment Prediction (MSP) task was the most critical component in achieving these performance gains.
Conclusion
By moving to segment-level granularity and utilizing multi-modal fusion, ERU provides a more efficient and accurate solution for long-form document understanding. The framework's ability to capture hierarchical relationships and visual layout cues allows it to outperform much larger models, setting a new benchmark for automated resume parsing.
Summary updated