Introduction
Author profiling—the automatic identification of latent attributes such as age, gender, or personality from text—is a critical component of social media analytics and forensic linguistics. A persistent challenge in this field is "topic bias," where models rely on lexical clues related to what is being discussed rather than the stylistic markers of how it is written. The TAG-it (Topic, Age, and Gender prediction) task at EVALITA 2020 was established to address these issues within the context of the Italian language, creating a new benchmark for multi-dimensional author profiling.
The TAG-it Challenge
The competition introduced two distinct subtasks designed to push the boundaries of current profiling systems:
- Task 1 (Multi-dimensional Profiling): A multi-label classification problem requiring systems to simultaneously predict an author's gender (Male/Female), age (categorized into five bins), and the topic of their writing (across 14 classes).
- Task 2 (Topic Control): A cross-topic evaluation where systems predicted gender or age for posts belonging to topics entirely absent from the training set. This tested the models' ability to disentangle stylistic features from topical content.
Methodology and Dataset
The dataset was curated from ForumFree, a popular Italian forum platform, comprising 2.5 million posts from over 7,000 users. To ensure sufficient textual evidence, only users with at least 500 tokens were included, resulting in a final set of 2,458 unique users. Forums were manually grouped into 31 broader topics, and age was binned into five groups: 0-19, 20-29, 30-39, 40-49, and 50-100.
Participants utilized various approaches, ranging from classical Support Vector Machines (SVMs) using n-grams to advanced neural architectures. The most successful systems leveraged UmBERTo, a RoBERTa-based model specifically pre-trained on Italian text, which significantly outperformed multilingual alternatives like mBERT.
Key Insights: The Age Prediction Gap
A primary finding of the challenge was the significant difficulty in predicting age compared to gender or topic. While gender and topic prediction achieved F-scores in the 0.90 range, age prediction struggled to exceed 0.40. This disparity suggests that linguistic markers of age are more subtle and continuous, making discrete binning a difficult task for current models.
Analysis of the confusion matrices (see Figure 1) reveals that errors are not random. When the models misclassify an author's age, they typically predict an adjacent age bin. This indicates that the models are capturing a general signal of "maturity" or "seniority," even if they cannot pinpoint the exact category.
Results and Cross-Topic Robustness
In Task 1, the top-performing model (ItaliaNLP-STL-UmBERTo) achieved a score of 0.735 on a partial credit metric and 0.331 on an exact match metric for all three dimensions. In Task 2, despite the lack of topical overlap between training and testing data, performance remained remarkably robust.
As shown in Figure 4, the models maintained their predictive patterns even in the cross-topic scenario. The best gender prediction reached a 0.925 Micro-F score on unseen topics, suggesting that modern Transformer-based models are beginning to successfully isolate stylistic markers from lexical topic bias.
Conclusion
The TAG-it challenge highlights the superiority of monolingual Transformer models for Italian author profiling and underscores the persistent difficulty of age prediction. Future research should explore moving from age bins to regression models for more granular predictions and further investigate the integration of explicit stylistic features—such as punctuation patterns or verb tenses—with deep learning embeddings to enhance performance across diverse genres.
Summary updated