TL;DR
This paper introduces a novel method for reconstructing the evolutionary history of languages using typological features, moving beyond traditional lexical-based approaches. By representing languages in a continuous latent space and incorporating a "typology evaluator," the authors achieve more accurate reconstructions of language families and even attempt to infer the characteristics of a common ancestor of all human languages.
What Problem Does This Paper Address?
Reconstructing the evolutionary history of languages, or phylogenetic inference, has traditionally relied on comparing lexical items (words) to identify cognates and trace their divergence. However, this approach struggles with language isolates or small language families that lack sufficient shared vocabulary. Linguistic typology, which classifies languages based on structural features (like word order or syllable structure), offers a promising alternative because it allows for comparisons between any pair of languages.
The challenge with typological data is that changes in these features over time are less intuitive than sound changes in words. It's hard to model how a language might switch from one grammatical structure to another. Furthermore, typological features are not independent; they exhibit dependencies (e.g., word order often correlates with adjective-noun order). Existing methods often treat features independently or use simplified binary representations, which can lead to the reconstruction of "typologically unnatural" ancestral states. This paper aims to address these limitations by developing a method that accounts for feature dependencies and reconstructs typologically plausible evolutionary histories.
Methods
The core of the proposed method lies in two key components:
- Continuous Space Representation (Autoencoder): The authors use an autoencoder to map each language's typological features into a continuous latent space. This allows for smoother transitions between feature states and better captures dependencies between them. The autoencoder learns a compressed representation (latent components) for each language.
- Typology Evaluator (Energy-Based Model): To ensure that reconstructed ancestral languages are "typologically natural," an energy-based model is integrated. This model learns to distinguish between plausible combinations of typological features and those that are unlikely to occur in natural languages. It acts as a constraint during the phylogenetic inference process.
Phylogenetic inference is then performed in this continuous latent space. The model uses known language families for supervision, learning how feature dependencies evolve. For inference, a Gibbs sampling approach is employed with several operators (CAT, COMP, SWAP, MOVE) to adjust feature values, latent components, and tree topologies.
Datasets
- World Atlas of Language Structures (WALS): Used for its extensive collection of typological features across many languages.
- Ethnologue: Used to obtain hierarchical classifications of languages and provide tree topologies for phylogenetic inference.
The authors processed these datasets by selecting a subset of features with sufficient coverage and mapping WALS languages to Ethnologue using ISO 639-3 codes. They also constructed family trees from the available data, removing intermediate nodes with only one child and treating language isolates as their own family trees.
Key Insights and Ideas
- Continuous space is key for natural transitions: Representing languages in a continuous latent space allows for smoother, more "typologically natural" evolutionary paths, avoiding the deep "valleys" of unnatural states that can occur with discrete representations.
- Dependencies matter: Explicitly modeling dependencies between typological features, rather than treating them independently, leads to more accurate phylogenetic reconstructions.
- Typological naturalness as a constraint: The "typology evaluator" is crucial for ensuring that inferred ancestral languages are plausible, preventing the reconstruction of logically impossible or highly improbable feature combinations.
- Typology for deep time: Typological features, being more stable than lexical items, hold promise for tracing language evolution further back in time, potentially breaking the ~6,000-7,000 year limit of traditional historical linguistics.
Key Results
- Improved Reconstruction of Known Families: The proposed method outperformed baseline methods (average-link agglomerative clustering) in 5 out of 8 metrics for reconstructing known language family trees, particularly in capturing subtree relationships.
- Reconstruction of a Common Ancestor: The paper successfully reconstructed a single tree representing the world's languages, inferring that the common ancestor was moderate in phonological typology, used suffixing morphology, and preferred SOV word order. This aligns with some previous speculations.
- Missing Data Imputation: The method achieved a 64.6% accuracy in predicting missing feature values, significantly outperforming random and most-frequent-value baselines.
Limitations and Future Work
- Underspecificity of Typology: The authors acknowledge that typological data alone might be underspecified, meaning two languages with identical feature vectors might not be the same language. Incorporating geospatial information is suggested as a future direction.
- Training on Modern Languages: The typology evaluator is trained on modern languages, which might not perfectly reflect the linguistic capacities or constraints of ancestral languages from very distant pasts.
- Data Sparsity: Despite using large databases, there are still gaps in typological data, which the imputation method helps to address but doesn't entirely solve.
- Future Work: The authors suggest extending the model to handle real-valued features (like consonant inventories), exploring more complex feature dependencies, and incorporating other types of linguistic data. They also propose using their framework to test hypotheses about specific language families (e.g., Altaic).
Summary updated