Introduction
Machine Reading Comprehension (MRC) requires models to synthesize local phrase-level patterns with global document context. While Recurrent Neural Networks (RNNs) excel at long-range dependencies and Convolutional Neural Networks (CNNs) capture local features efficiently, traditional architectures often stack these layers vertically, potentially losing nuance or efficiency. The "Rception" network introduces a novel architectural fusion inspired by the "Inception" modules in computer vision, arranging these components in a horizontal, multi-branch structure to capture both fine-grained details and broad context.
Architecture
The core innovation is the Rception module, which serves as the primary interaction layer between the question and the passage.
The model consists of three main stages:
- Encoding Layer: The model uses a weight-fixed BERT-base backbone to generate contextualized embeddings, augmented with linguistic features like Part-of-Speech (POS) tags, Named Entity Recognition (NER) tags, and attention scores.
- The Rception Module (Horizontal Fusion): Instead of sequential stacking, the model employs five parallel branches to process the same input from multiple perspectives:
- The "Deep" Part: Three CNN branches using 1D dilated depth-wise separable convolutions with varying kernel sizes. This approach increases the receptive field while remaining computationally efficient.
- The "Wide" Part: Two RNN branches (one unidirectional, one bidirectional) combined with self-attention to model long-distance relationships across the text.
- Aggregation and Answer Layer: Outputs from all five branches are integrated via max pooling. This acts as a selection mechanism, allowing the model to dynamically choose the most relevant feature for every word. Finally, bilinear functions predict the start and end positions of the answer span within the passage.
Experimental Results
The Rception model was evaluated on the SQuAD 1.1 benchmark and adversarial datasets (AddSent and AddOneSent) to test robustness against distractor sentences.
As shown in the results table, the Rception model achieved an Exact Match (EM) of 84.4% and an F1 score of 90.6% on the SQuAD 1.1 test set. This performance significantly outperforms BERT-Base and is competitive with the much larger BERT-Large model.
Ablation studies (bottom of the table) confirm the necessity of the multi-branch approach. Removing either the "deep" (CNN) or "wide" (RNN) components led to a measurable drop in performance, validating the hypothesis that both local phrase-level features and global context are critical for MRC. Furthermore, the model demonstrated superior robustness on adversarial tests, achieving 40.0% EM on AddSent and 52.0% EM on AddOneSent, outperforming many existing baselines.
Discussion and Conclusion
The primary insight of the Rception architecture is that horizontal fusion is more effective than vertical stacking for MRC. By processing input from multiple perspectives simultaneously and using max-pooling as a selection mechanism, the model mimics the human tendency to consider multiple interpretations of a sentence. While the parallel structure introduces some computational overhead, the use of depth-wise separable convolutions maintains efficiency. Future work may explore scaling this architecture with larger transformer backbones or applying the Rception framework recursively for even more granular feature extraction.
Summary updated