TL;DR
This paper introduces novel VAE-based deep learning architectures that learn character representations for unknown scripts, outperforming previous methods on deciphered scripts while using significantly fewer parameters. The models are then applied to the undeciphered proto-Elamite script, revealing new potential relationships between signs and validating expert intuitions.
What Problem Does This Paper Address?
Deciphering ancient scripts hinges on identifying the distinct characters (types) used. This is challenging for human analysts, especially with large numbers of tokens and subtle visual variations. Previous work has automated this "script clustering" by training neural networks to generate character representations and then clustering them. However, these methods often rely on complex, over-parameterized models (like ResNet) and may not fully leverage contextual information within the script. This paper aims to develop more efficient and effective deep learning models for script clustering, particularly for undeciphered scripts where ground truth is unavailable.
Methods
The authors propose four VAE-based architectures that build upon a variational autoencoder (VAE) framework. The core idea is to learn dense vector representations of character images.
- VAE: A baseline VAE model that reconstructs input character images from a latent representation.
- VAE+Neighbors: Extends the VAE by adding an auxiliary task to reconstruct a character image using the representations of its immediate left and right neighbors. This introduces local context.
- VAE+LSTM: Incorporates an autoregressive LSTM to capture longer-range sequential context from a sequence of character representations.
- VAE+Transformer: Replaces the LSTM with a Transformer encoder, allowing for more flexible attention-based contextual modeling. This model uses a masked language modeling task to recover masked character images.
All models are trained end-to-end using a combination of reconstruction loss, an auxiliary task loss (for VAE+Neighbors, VAE+LSTM, VAE+Transformer), and a pseudolabel loss for soft clustering. A key innovation is the use of lighter-weight convolutional encoders compared to the ResNet used in prior state-of-the-art methods.
Datasets
The authors evaluate their models on four datasets:
- English: 33,000 alphanumeric tokens extracted from the WikiText-2 corpus, with characters rendered from 55 handwritten English letter images. This dataset aims to mimic the detail level of ancient scripts.
- Japanese: 33,000 tokens from the Japanese Tatoeba corpus, using glyphs from two Japanese fonts. Evaluation focuses on the syllabaries (hiragana and katakana).
- Cypro-Greek: A known dataset of manually-segmented images from artifacts, used for direct comparison with prior work.
- Proto-Elamite: The primary focus of the paper, using a work-in-progress transliteration of ~1500 signs from ~1581 clay tablets. This is an undeciphered script, so evaluation is qualitative and collaborative with domain experts.
Key Insights and Ideas
- Lighter models are sufficient: The paper strongly suggests that complex, deep architectures like ResNet are over-parameterized for script clustering. Simpler convolutional encoders can achieve comparable or better results with far fewer parameters.
- Context matters: Models incorporating contextual information (VAE+LSTM, VAE+Transformer) generally outperform the contextless VAE, especially on scripts with more visual variation or longer-range dependencies.
- Allography vs. Character Count: The difficulty of script clustering is more influenced by the degree of visual variation between different forms of the same character (allography) than by the sheer number of distinct characters in the inventory.
- Proto-Elamite Discoveries: The models, particularly VAE+LSTM and VAE+Transformer, can identify plausible new groupings of proto-Elamite signs that align with expert intuitions and suggest novel relationships, demonstrating their utility for decipherment research.
Key Results
- Cypro-Greek Performance: The VAE+LSTM and VAE+Transformer models achieve state-of-the-art V-measure scores on Cypro-Greek, outperforming Sign2Vec and DeepCluster-v2, while using approximately 98% fewer parameters than Sign2Vec.
- English Performance: Contextual models (LSTM, Transformer) show the highest performance on English, supporting the hypothesis that wider context aids script recovery.
- Japanese Performance: On synthetic Japanese data with low allography, differences between models are less pronounced, with contextual models sometimes underperforming the baseline VAE. This highlights the importance of allography.
- Proto-Elamite Qualitative Findings: The models propose several new groupings of proto-Elamite signs, some of which suggest plausible relationships (e.g., "gunuification" variants, potential links between signs associated with agriculture and document headers) that have not been previously noted.
Limitations and Future Work
- Proto-Elamite Ground Truth: The primary limitation for proto-Elamite is the lack of a known ground truth, making quantitative evaluation impossible. The authors rely on qualitative assessment in collaboration with domain experts.
- Generalizability: While the paper covers a range of scripts, it acknowledges that it cannot cover all possible writing systems and their variations.
- Future Work: The authors are optimistic that their models can continue to provide insights for proto-Elamite decipherment as experts investigate the proposed sign groupings. They also suggest that exploring different contextual modeling approaches and further tuning hyperparameters could yield additional improvements.
Summary updated