Introduction
Social media platforms are under increasing pressure to moderate content effectively. In Germany, the Network Enforcement Act (NetzDG) mandates that "evidently illegal" content be removed within 24 hours. However, a significant gap exists between current Natural Language Processing (NLP) research and legal reality. While NLP models typically focus on broad, subjective categories like "hate speech" or "abusive language," German law follows a deductive Roman law tradition based on abstract, codified rules. This paper addresses the challenge of translating these abstract legal concepts into a concrete annotation schema suitable for machine learning.
Methodology: Translating Law to Logic
To bridge the gap between legal theory and technical implementation, the researchers deconstructed the German Criminal Code (StGB)—specifically sections regarding insult (§ 185) and defamation (§ 186/187)—into a sequence of binary decisions. This flowchart allows annotators to systematically determine if a tweet constitutes a criminal offense.
The schema requires answering four primary questions:
- Defamatory Object: Is the post directed at a living individual or a specific, distinguishable group?
- Disparaging Statement: Does the content express contempt or reduce the victim's social standing?
- Value Judgment vs. Factual Claim: Is the statement an opinion or an assertion of fact? (Factual claims require external evidence, which the authors treat as a separate category).
- Balancing Rights: For value judgments, does the "right to honor" outweigh "freedom of expression"? This involves distinguishing between "abusive insults" (Schmähkritik), which contribute nothing to public discourse, and comments on "topics of public interest."
Experimental Setup
The study utilized 1,100 tweets from the GermEval 2018 dataset, a well-known collection of German political tweets. These were annotated by two fully qualified German lawyers to create a gold-standard Legal Reference Set. To test the feasibility of crowdsourcing such complex tasks, the researchers also collected annotations from 1,000 crowd workers and student assistants.
For the technical component, the authors employed a Bi-directional LSTM with pre-trained German word embeddings to evaluate whether a machine could learn the individual binary steps of the legal decision tree.
Key Results: The Gap Between Offensive and Illegal
The most significant finding of the study is the stark contrast between what laypeople consider "offensive" and what the law deems "punishable." Out of 844 tweets previously flagged as containing an offense by laypeople, legal experts found that only 8% (65 tweets) were actually punishable under German law.
Several factors contribute to this discrepancy:
- Public Interest as a Shield: In political contexts, the threshold for a punishable statement is exceptionally high. If a tweet addresses a topic of public interest, freedom of expression usually prevails unless the statement is a pure "abusive insult."
- Layman Reliability: There is a significant "disparaging disconnect." Laypeople often label statements as "non-disparaging" that a lawyer would identify as a legal violation of honor. Student assistants achieved a Cohen’s κ of 0.250 compared to experts, while general crowd workers fell to 0.050, suggesting that legal annotation requires specialized training.
- Data Sparsity: Because punishable tweets are so rare (the "needle in a haystack" problem), the data is extremely skewed. This makes training a direct "punishable vs. not punishable" classifier nearly impossible with current small datasets.
Automated Classification Performance
The Bi-LSTM model showed promise in identifying objective elements, such as "factual claims" (97.7% accuracy). However, it struggled with more nuanced categories like the "defamatory object" (74.4% accuracy). Most notably, the F1-scores for the "punishable" class were near zero because the model rarely encountered enough positive examples to learn the subtle legal boundaries involved in balancing rights.
Conclusion and Future Work
Automating legal judgment remains a formidable challenge. The "balancing of interests" step is deeply contextual and requires a level of world knowledge and common sense that current LSTMs lack. The authors suggest that future research should focus on building larger, more reliable datasets by better instructing lay annotators on legal boundaries. They also propose expanding the framework to include incitement to hatred (§ 130 StGB), a more complex but critical area of digital law enforcement.
Summary updated