Abstract
This paper describes a system for the analysis of Chinese language that incorporates several heuristic techniques for reducing ambiguities. Heuristic knowledge about "characteristic" words is used to predict a partial syntactic structure of a sentence before doing global analysis. Semantic constraints are used for translating phrase structure into case structure, organized via an object-oriented scheme. A preference score is calculated at each step to steer a priority-driven parser towards the most plausible sentence structure. Experimental results on over one hundred sentences show a significant reduction in the number of trees required for correct analysis, typically from several hundred to under ten.
Introduction: The Challenge of Chinese Parsing
In the 1980s, natural language processing (NLP) for Chinese faced a unique hurdle: a lack of morphological cues. Unlike English, where suffixes like "-ed" or "-ing" signal a word's role, Chinese words rarely change form. This leads to a "combinatorial explosion" during parsing; a single string of characters can often be interpreted in hundreds of syntactically valid but semantically nonsensical ways.
The author specifically highlights the "Serial Verb Construction" (SVC)—sentences with multiple verbs where it’s unclear which is the main predicate and which belongs to a subordinate clause. Without a way to introduce constraints early in the process, parsers get bogged down in a sea of improbable possibilities.
System Architecture
The proposed system utilizes a four-stage pipeline designed to "fail fast" and prioritize the most likely interpretations. The flow of information and the interaction between different knowledge bases (dictionaries and rule sets) are illustrated in Figure 1.
- Segmentation: Translating character strings into words using Chinese, terminology, and idiom dictionaries.
- Preprocessing (Heuristic Prediction): The system identifies "characteristic words" (about 200 functional words like prepositions or auxiliary verbs). It uses roughly 40 Augmented Transition Networks (ATNs) to predict partial syntactic structures. If multiple predictions exist, a "preference score" helps choose the most compatible combination.
- Phrase Structure Analysis: A Context-Free (CF) parser builds the global structure, restricted to the partial structures predicted in the previous step. If the parser hits a dead end, it backtracks to the preprocessing stage for the next best prediction.
- Semantic (Case Structure) Analysis: The system maps phrase structures to semantic roles (e.g., Agent, Object, Location) using an object-oriented rule scheme.
Semantic Mapping and Case Structure
The final stage of the pipeline involves translating the syntactic phrase structure into a semantic case structure. Rules are written in independent modules for word classes and organized into layers according to their priorities. This object-oriented approach allows for an elegant "inheritance" of constraints, where a specific verb can have its own unique rules while still inheriting general properties from its broader verb class (e.g., "Emotional" or "Causative" verbs).
By checking if a subject semantically "matches" its complement, the system can assign negative scores to improbable structures, effectively pruning bad branches of the search tree. The resulting case structure, as seen in Figure 2, provides a deep representation of the sentence's meaning beyond mere syntax.
Experimental Results
The system was evaluated using a custom dataset of over one hundred sentences curated from Chinese physics textbooks, scientific papers, and grammar books. The heuristic approach proved highly effective at reducing computational overhead:
- Ambiguity Reduction: In one example sentence regarding physics ("In the perfect situation without friction..."), the system reduced the number of possible syntactic trees from 552 down to just 8.
- Preprocessing Accuracy: The heuristic preprocessing stage produced the correct partial structure as its first choice 94% of the time.
- Efficiency: For a subset of 20 complex sentences, the correct analysis was the system's first choice in 14 cases and the second choice in the remaining 6. The number of trees required to reach a correct analysis was typically reduced from several hundred to under ten.
Conclusion
By combining early partial syntactic predictions with hierarchical semantic constraints, this system demonstrates that syntax alone is insufficient for efficient Chinese parsing. The use of "characteristic words" as anchors and the application of preference scores allow the parser to navigate the "fog" of ambiguity effectively. While the system relied on manual rule engineering and a limited dictionary, it established a successful framework for integrating multiple levels of linguistic knowledge to solve the combinatorial explosion in NLP.
Summary updated