Raw-to-End Name Entity Recognition in Social Media

Liyuan Liu, Zihan Wang, Jingbo Shang, Dandong Yin, Heng Ji, Xiang Ren, Shaowen Wang, Jiawei HanarXiv 2019 · arXiv

Abstract

Traditional Named Entity Recognition (NER) systems often struggle with the informal and noisy nature of social media text due to their reliance on brittle pre-processing pipelines. This paper presents Neural-Char-CRF, a "raw-to-end" framework that eliminates the need for tokenization by processing raw character sequences directly. By leveraging contextualized character representations and a novel string-matching alignment strategy, the model achieves state-of-the-art performance on major social media benchmarks, demonstrating significant resilience to the linguistic irregularities common on platforms like Twitter.

The Fragility of the Pipeline

Named Entity Recognition—the task of identifying names of people, organizations, or locations in text—usually follows a rigid "pipeline" approach. First, a tokenizer splits a sentence into discrete words; then, an NER model labels those words. While effective for clean news articles, this approach falters in the "wild west" of social media, characterized by typos, hashtags, and creative punctuation.

Comparison between the proposed Raw-to-End system and traditional pipeline systems.

The authors found that in the Broad Twitter Corpus, over 45% of named entities had incorrectly identified word boundaries when using standard tools like spaCy. For instance, if a tokenizer fails to split Lakers-Raptors, the NER model is forced to assign a single label to the entire string, missing two distinct entities. These early errors propagate through the pipeline, capping the performance of even the most advanced models.

Methodology: Neural-Char-CRF

The proposed Neural-Char-CRF shifts the focus from words to characters. Instead of labeling words, the model assigns a label (using the IOBES schema) to every single character in the raw string.

Robust Word Embedding Alignment

To retain the benefits of powerful word-level embeddings (like GloVe) without being bound by tokenization errors, the authors developed a String Matching-based approach. This tokenization-free method searches a dictionary for all possible word matches within a string. To resolve overlaps, it uses Inverse Document Frequency (IDF) to prioritize the most "informative" matches.

The string-matching alignment process using IDF to select the most relevant word representations for characters.

Architecture and Pre-training

The model utilizes contextualized character representations pre-trained on a massive corpus of 700 million tweets. These representations are fed into a Bi-LSTM, followed by a Conditional Random Field (CRF) layer to ensure the final sequence of labels is globally optimal. This allows the model to understand the specific "flavor" of social media language at the most granular level.

Experimental Results

The model was evaluated on two primary public benchmarks: the Twitter Name Tagging (TNT) dataset and the Broad Twitter Corpus (BTC). Neural-Char-CRF significantly outperformed existing state-of-the-art (SOTA) pipeline systems:

  • TNT Dataset: Achieved an F1 score of 86.96, a 3.70 point gain over the previous best model (Flair).
  • BTC Dataset: Achieved an F1 score of 74.98, a 6.65 point gain over SOTA.

Ablation studies confirmed that the character-level language model is vital, as its removal led to a performance drop of approximately 6 points. Furthermore, the system proved remarkably stable; while switching tokenizers in a traditional pipeline could cause a drop of over 20 F1 points, Neural-Char-CRF remained resilient.

Conclusion and Future Work

Neural-Char-CRF demonstrates that building high-performing NLP tools for noisy domains does not require the "human endeavor" of designing complex, language-specific tokenizers. By operating directly on raw text, the model effectively "sees inside" hashtags and concatenated words that baffle traditional systems. Future work aims to apply this "raw-to-end" philosophy to other tasks like part-of-speech tagging and to expand into multi-lingual settings where tokenization is even more challenging.

Summary updated

Abstract

Taking word sequences as the input, typical named entity recognition (NER) models neglect errors from pre-processing (e.g., tokenization). However, these errors can influence the model performance greatly, especially for noisy texts like tweets. Here, we introduce Neural-Char-CRF, a raw-to-end framework that is more robust to pre-processing errors. It takes raw character sequences as inputs and makes end-to-end predictions. Word embedding and contextualized representation models are further tailored to capture textual signals for each character instead of each word. Our model neither requires the conversion from character sequences to word sequences, nor assumes tokenizer can correctly detect all word boundaries. Moreover, we observe our model performance remains unchanged after replacing tokenization with string matching, which demonstrates its potential to be tokenization-free. Extensive experimental results on two public datasets demonstrate the superiority of our proposed method over the state of the art. The implementations and datasets are made available at: https://github.com/LiyuanLucasLiu/Raw-to-End.
BibTeX
@misc{liu2019raw,
  author = {Liyuan Liu and Zihan Wang and Jingbo Shang and Dandong Yin and Heng Ji and Xiang Ren and Shaowen Wang and Jiawei Han},
  title = {{Raw-to-End Name Entity Recognition in Social Media}},
  year = {2019},
  eprint = {1908.05344},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1908.05344},
}
Figure