Introduction
As machine learning architectures become increasingly standardized and accessible, the "secret sauce" of artificial intelligence is shifting. The competitive advantage in AI development is no longer found in the design of neural network layers, but in the management and curation of the data that feeds them. Bill Howe argues that we are entering a new paradigm—"curation as programming"—where the primary challenge is finding, organizing, and cleaning high-quality data for specialized, high-stakes applications.
The Curation Dichotomy: On-Read vs. On-Write
A fundamental divide exists in how AI systems are currently deployed. Most modern breakthroughs, particularly Large Language Models (LLMs), operate under a "curation-on-read" regime. In this setting, models are trained on massive, weakly curated "convenience samples" from the internet. The burden of identifying errors, biases, and intellectual property violations is shifted to the end-user at the time of output.
However, in specialized fields such as science, law, or urban planning, this "fix it later" approach is insufficient. These domains require "curation-on-write": a regime where models must be trained on precise, objectively correct inputs to produce reliable outputs. Curation-on-write is notoriously difficult and expensive, characterized by limited data availability, complex multi-modal features (e.g., combining text with sensor data), and conflicting labels from different subject matter experts.
Curation as Programming
To address these challenges, Howe proposes a shift toward "curation as programming." Rather than focusing on tweaking model hyperparameters, the goal is to design new tooling that empowers domain experts to "program" AI systems through data curation. Key technical approaches include:
- Enabling Technologies: Developing tools to drastically reduce the cost of finding and organizing training and evaluation data.
- Ontology Integration: Investigating methods to incorporate expert-provided ontologies—structured frameworks of knowledge—directly into the learning process.
- Bias Mitigation: Developing techniques to "unlearn" specific biases during the fine-tuning phase, ensuring models adhere to the strict requirements of specialized domains.
Specialized Applications and Physics-Based Learning
This curation-centric approach is being applied across several real-world domains where accuracy is paramount:
- Urban Mobility: Recovering missing data points in complex transportation networks.
- Governance: Identifying speakers and agenda items within local council meeting records to improve transparency.
- Legal: Extracting structured, actionable information from dense legal documents.
- Meteorology: In a striking development, deep learning models are now poised to outperform traditional physics-based models. By learning the underlying laws of physics directly from observational data, these models bypass the need for explicit fluid dynamics programming, effectively becoming "physics learners."
The Expert in the Loop
The commoditization of models means that the developer's role is evolving. In the "General AI" space (e.g., ChatGPT), the user acts as the filter. In "Specialized AI" (e.g., fluid dynamics or legal analysis), the developer must be the filter. The future of AI for specialized tasks depends on empowering domain experts rather than replacing them, providing them with tools that translate their nuanced knowledge into curated data.
Challenges and Future Work
Despite the promise of this shift, several significant hurdles remain:
- Data Scarcity: Curation-on-write settings often lack the massive datasets available to general-purpose LLMs.
- Complexity and Ambiguity: Handling multi-modal features and resolving conflicting labels from different experts remains a technical bottleneck.
- Logic and Consistency: There is a pressing need for better methods to ensure models maintain logical consistency and utilize expert ontologies effectively.
- The Tooling Gap: Current software ecosystems lack specialized tools designed specifically for the "curation as programming" workflow.
As AI continues to mature, the focus will inevitably move away from the model and toward the data. The success of the next generation of AI will be defined by how well we can curate the knowledge of experts into a format that machines can reliably understand.
Summary updated