Médias traditionnels, médias sociaux : caractériser la réinformation (Traditional medias, social medias : characterizing reinformation)

Cédric Maigrot, Ewa Kijak, Vincent ClaveauJEP-TALN-RECITAL (Posters) 2016

Abstract

As social media platforms become primary conduits for news, the distinction between established journalistic outlets and "reinformation" sources—alternative media that often manipulate facts to serve specific ideologies—becomes increasingly blurred. This paper presents a supervised machine learning approach to automatically characterize the editorial nature of Facebook posts. By analyzing a dataset of over 700,000 messages, the study finds that while surface-level stylistic markers offer some discriminative power, the integration of deep linguistic content from linked articles is essential for high-accuracy classification, reaching up to 88%.

Introduction

Traditional news organizations now compete directly with "reinformation" sites on fast-moving social media feeds. These alternative sources frequently claim to expose "hidden truths" while serving political or religious agendas. For researchers and platforms, the challenge lies in the visual similarity of these posts. This research aims to bridge this gap by developing a system that characterizes the source's editorial nature, serving as a critical precursor to real-time veracity checking.

Methodology and Datasets

The researchers employed a supervised machine learning framework, testing algorithms such as Naive Bayes, Decision Trees (J48), Random Forests, and Support Vector Machines (SMO). The study utilized a substantial new dataset of 735,764 Facebook messages collected since 2015, divided into French ($C_f$) and English ($C_a$) subsets. Sources were manually labeled by three annotators (achieving high agreement, $\kappa = 0.874$) based on their institutional links or explicit "reinformation" claims.

Two primary feature sets were extracted:

  1. Surface Features: 13 descriptors including message length, punctuation (exclamation/question marks), capitalization, and pronoun distribution.
  2. Content Features: Lemmatized text from posts and—crucially—the full content of linked web pages, processed via TF-IDF to identify the 1,000 most discriminative n-grams.

Characterizing the "Shape" of Information

Initial analysis revealed distinct stylistic differences between the two media types. Traditional media tend to produce longer, more structured "teasers" compared to the often shorter, more punchy style of reinformation sources.

Comparison of surface features across French ($C_f$) and English ($C_a$) corpora.

As shown in the table above, traditional media in the French corpus ($C_f$) average significantly more words per message (4118.89) than their reinformation counterparts (1173.64) when linked content is included. Conversely, reinformation sources often show higher relative frequencies of certain punctuation marks, such as question marks and exclamation points, suggesting a more emotive or provocative editorial style.

Key Results and Performance

The experimental results highlight a clear hierarchy in feature effectiveness. Surface features alone typically yielded accuracies between 50% and 65%. However, the inclusion of content features significantly boosted performance across all models.

Classification results for the French corpus ($C_f$) across different feature sets.

In the French corpus, the SMO (SVM) classifier achieved the highest accuracy (Taux BC of 88.62%) when utilizing content features. Interestingly, the system proved exceptionally precise at identifying traditional media; for instance, using Naive Bayes, the precision for the "traditional" class reached 94.75%, meaning the model rarely misidentifies a mainstream source.

Discussion: The Professional Signature

The study suggests that traditional media are defined by a "professional signature." Terms related to institutional norms—such as "subscription," "privacy," and "opinion"—serve as strong mainstream indicators. Furthermore, the researchers observed that reinformation sources are often treated as a "default" category by classifiers; if a post lacks the specific markers of professional journalism and editorial rigor (including standard orthography), it is likely to be flagged as reinformation.

Conclusion

This research demonstrates that the editorial nature of a news source can be identified with high reliability by looking beyond the surface of a social media post. While the current model focuses on text, future work will explore social propagation patterns and multimodal data (images and video) to further refine the characterization of information in the digital age.

Summary updated

BibTeX
@inproceedings{maigrot2016medias,
  author = {Cédric Maigrot and Ewa Kijak and Vincent Claveau},
  title = {{Médias traditionnels, médias sociaux : caractériser la réinformation (Traditional medias, social medias : characterizing reinformation)}},
  booktitle = {JEP-TALN-RECITAL (Posters)},
  year = {2016},
}
Figure