Documenting the Middle of the Research Lifecycle

21 concepts10 papers1 proposal

Scientific communication has historically focused on the "bookends" of research: the initial hypothesis and the final results. However, a growing body of evidence suggests that the most critical information for reproducibility and validation—the "messy middle" of the research process—is frequently omitted. This research direction focuses on moving beyond the publication of final outputs toward a comprehensive documentation of the generating process, or procedural provenance.

The Reporting Discrepancy

A recurring theme across disciplines is a sharp divide between how well researchers describe their motivations and how poorly they document their technical execution. In a systematic review of public health dissertations, Dai et al. found that while students were proficient at reporting study backgrounds and objectives, they consistently failed to document critical analytical steps such as sensitivity analyses or the management of missing data.

This isn't merely a training issue for early-career researchers; it persists in large-scale knowledge bases. Moodley et al. and the authors of A model for capturing provenance of assertions about chemical substances observed that even when the authorship of a scientific claim is known, the specific experimental or analytical procedures used to derive that claim are often missing. Without this "generating process metadata," the scientific community cannot evaluate the rigor of an assertion or reproduce the result.

Mapping the Research Ecosystem

To address this "black box" problem, researchers are developing frameworks to make the intermediate steps of science machine-readable and citable. Daquino et al. and Daga et al. propose the concept of a Research Ecosystem. Instead of treating a paper as an isolated artifact, this framework uses Semantic Web technologies to map the "data journey"—the complex interdependencies between software versions, data cleaning scripts, and the tacit knowledge that usually stays trapped in a researcher's head.

This need for process transparency is particularly acute in fields involving complex data integration. For example, Zhang et al. highlight how health research often lacks standardized documentation for variable mapping and data merging. When these "middle" steps are opaque, other researchers cannot verify the logic used to prepare the data, potentially masking systematic errors.

Transparency in Human and Expert Inputs

The "middle" of the research process often involves human judgment, which is notoriously difficult to document. In machine learning, this appears in two main areas:

  1. Knowledge Elicitation: When researchers interview experts to build models, they often fail to report the medium of interaction or how they validated the expert's responses. Kerrigan et al. found that a third of surveyed papers lacked transparency regarding their elicitation methods, making the resulting models impossible to critically evaluate.
  2. Data Annotation: We often treat datasets as objective truths, but they are the result of complex interactions between researchers and annotators. Zhu and Rzeszotarski argue that we must report the specific constraints and agency given to annotators, as many data quality issues stem from misalignments in the collection process rather than "lazy" workers.

From Open Science to Science in the Open

The most radical approach to solving the transparency deficit is to conduct the entire research process in public view. Phang et al. describe this as "Science in the Open." By moving the research lifecycle—including failed experiments, informal trade-offs, and debugging—to public platforms like Discord, organizations like EleutherAI allow the community to see the full context of a breakthrough.

This shift suggests that true methodological transparency isn't just about better checklists or data dictionaries, such as those proposed by Bagg et al. for neurological research. Rather, it is about capturing the iterative, often messy path of discovery that leads to the final result. When we document the middle, we don't just show what we found; we show exactly how we got there.

Key Foundational Work

A model for capturing provenance of assertions about chemical substances (2018) — contributed 3 concept elements to this direction.
Evaluation of the reporting quality of observational studies in Master of Public Health’s dissertations in China: a systematic review (2019) — contributed 3 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_methodological_transparency_f9d7b401,
  title = {methodological transparency},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/methodological-transparency/rdr_f9d7b4011d084bb4ab1f45e54415c258},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_methodological_transparency_f9d7b401_v1,
  title = {methodological transparency},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/methodological-transparency/rdr_f9d7b4011d084bb4ab1f45e54415c258/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake