This paper provides a sobering "reality check" for the application of unsupervised machine learning to archaeological datasets. By re-analyzing the Khipu Field Guide (KFG) database, the author demonstrates that what previously looked like meaningful structural patterns in Inka khipus (knotted-string records) are largely artifacts of physical size and the idiosyncratic habits of the researchers who first cataloged them.
TL;DR
Generic clustering algorithms fail to find meaningful "types" of khipus once you control for two massive confounds: the physical size of the object and the identity of the person who recorded the data. However, targeted, domain-specific heuristics—like looking for spatial blocks of knot directions—successfully identify potential "calendar khipus" where generic methods fail.
What Problem Does This Paper Address?
For decades, researchers have sought to "crack the code" of the Inka khipu, a complex system of knotted strings used for accounting and potentially narrative recording. Recent efforts have turned to data science, using unsupervised clustering to find distinct "typologies" or classes of khipus.
The challenge is that khipu data is "messy" and archival. This paper addresses a critical gap in prior work: the failure to account for confounds. If a large khipu naturally contains more patterns simply because it has more strings, or if one researcher consistently forgot to record knot directions while another didn't, an algorithm will "discover" these differences as structural clusters, even if they have nothing to do with how the Inka actually used the device.
Methods
The author employs a mix of statistical "sanity checks" and machine learning:
- OLS Regression: Used to determine how much variance in structural features (like arithmetic sum relations) is explained by simple physical size (cord count).
- UMAP + HDBSCAN: A standard pipeline for non-linear dimensionality reduction and clustering, used here to search for khipu typologies.
- Null Model Testing: Crucially, the author compares clustering results against two null models: a permutation null (shuffling features) and a Gaussian null (random noise with the same covariance). This determines if a cluster is "real" or just a lucky grouping of noise.
- Spatial-Block Heuristic: A targeted algorithm that looks for "moments" (mean positions) of S-direction vs. Z-direction knots to find khipus where knot types are organized into distinct physical blocks.
Datasets
The study uses the Khipu Field Guide (KFG) database, which contains digitized records of 650 khipus. This dataset is a cleaned and expanded version of the older Open Khipu Repository (OKR). It includes metadata, cord counts, knot types, and arithmetic relations.
Key Insights and Ideas
- The "Cataloger Confound": The two-letter prefixes in khipu names (e.g., UR for Urton, AS for Ascher) identify the researcher, not the archaeological site. This is vital because different researchers had different "data quality" standards. For example, the Aschers (AS) left 84-89% of knot directions undocumented, while Gary Urton (UR) documented almost all of them. An algorithm will see "AS-style khipus" as a distinct group simply because they are missing data.
- Size as a Mechanical Driver: Khipu size explains 42–70% of the variance in structural counts. Larger khipus provide more "surface area" for patterns to appear by chance. Any model that doesn't log-scale or control for cord count is essentially just clustering khipus by size.
- The Reproducibility Trap: The author found that UMAP results were not reproducible across different computing environments (e.g., local machine vs. Kaggle), even with fixed random seeds. This non-determinism led to "significant" results in one environment that vanished in another, serving as a warning against over-interpreting specific embeddings.
Key Results
- Clustering = Noise: When pooling all 650 khipus, the clustering results were statistically indistinguishable from a random Gaussian cloud (p = 0.45–0.65).
- The "HP" Difference: After controlling for size and documentation, khipus cataloged by Hugo Pereyra (HP) showed significantly fewer "Ascher sum-relations" (p = 0.044) than those cataloged by Urton. This suggests a genuine difference in either the accounting style of those specific khipus or Pereyra’s detection protocol.
- Calendar Khipu Discovery: While generic clustering failed, the targeted spatial-block heuristic recovered 8 out of 10 previously known "calendar khipu" candidates and identified 34 new candidates for further study.
Limitations and Future Work
- AS Data Gap: The "Ascher effect" remains unresolved because their data is too incomplete (collinear with missingness) to determine if their khipus were truly different.
- Validation Needed: The 34 new calendar candidates are "quantitative leads" only; they require manual, visual inspection by khipu specialists to confirm if the spatial blocks are meaningful.
- Environment Pinning: Future work must use containerized environments (like Docker) to ensure that the non-deterministic nature of UMAP doesn't lead to more "false positive" discoveries.
Summary updated