Introduction
Multi-author writing style analysis involves identifying the precise locations where a document transitions between different authors. This task is foundational for authorship verification and plagiarism detection. A primary challenge in training deep learning models for this purpose is catastrophic forgetting, where the model's adaptation to new stylistic patterns or topics causes it to overwrite previously learned weights. To mitigate this, the authors propose the Entropy-Based Stability-Plasticity (ESP) method, which dynamically balances knowledge retention and new learning.
Methodology
The proposed architecture employs a BERT encoder and a classifier decoder. The ESP mechanism manages the stability-plasticity trade-off by adjusting learning rates at the layer level based on information entropy.
The ESP Framework
The ESP method follows a three-step process:
- Entropy Calculation: During the forward pass, Shannon entropy ($H_i$) is calculated for each layer based on neuron activations.
- Plasticity Factor Derivation: A plasticity factor ($P_i$) is derived by normalizing the layer's entropy against the total entropy of the network.
- Adaptive Gradient Scaling: During backpropagation, the gradient update for each layer is scaled by its plasticity factor: $\Delta \theta_i = P_i \cdot \nabla \theta_i$.
This mechanism allows layers with higher entropy—typically the task-specific upper layers—to exhibit greater plasticity, while maintaining stability in the lower layers to preserve foundational stylistic features. Furthermore, the authors utilized a reversed curriculum learning strategy, training the model on datasets in the order of Hard → Medium → Easy.
Experimental Setup
The model was evaluated using the PAN 2024 Multi-Author Writing Style Analysis dataset, categorized into three difficulty levels:
- Easy (Task 1): Paragraphs cover diverse topics, allowing topical shifts to serve as proxies for author changes.
- Medium (Task 2): Topical variety is reduced, requiring the model to focus more on stylistic nuances.
- Hard (Task 3): All paragraphs share the same topic, necessitating detection based purely on authorial style.
Results
The ESP approach, submitted under the team name emerald-callable, outperformed the provided baselines across all difficulty tiers.
The results indicate that the ESP framework effectively maintains performance across sequential tasks, showing clear improvements over "Predict 1" (always change) and "Predict 0" (never change) baselines.
Discussion and Conclusion
The Entropy-Based Stability-Plasticity method offers a robust, data-driven approach to the stability-plasticity dilemma. By using entropy as a metric for layer-wise adaptation, the model autonomously determines which components should remain stable and which should be plastic. While the performance gains on the most challenging tasks were incremental, the ESP framework provides a scalable solution for lifelong learning in text analysis. Future research will focus on applying this mechanism to diverse neural architectures and broader domains.
Summary updated