Abstract
Within neuroscience, models have many roles, including driving hypotheses, making assumptions explicit, synthesizing knowledge, making experimental predictions, and facilitating applications to medicine. While specific modeling techniques are often taught, the process of constructing models for a given phenomenon or question is generally left opaque. Here, informed by guiding many students through modeling exercises at our summer school in CoSMo (Computational Sensory-Motor Neuroscience), we provide a practical 10-step breakdown of the modeling process. This approach makes choices and criteria more explicit and replicable. Experiment design has long been taught in neuroscience; the modeling process should receive the same attention.
Introduction: The Need for a Structured Approach
While specific computational techniques—such as training neural networks or solving differential equations—are widely taught, the actual process of constructing a model to answer a specific scientific question is usually left opaque. Students typically learn how to model through informal mentoring and imitation, which is often an ineffective way to train researchers to tackle entirely new problems.
Just as experimental design is explicitly taught in the life sciences, the step-by-step logical process of model building needs a formalized framework. Without it, researchers often fall victim to "mission creep," unstated assumptions, and the premature adoption of trendy mathematical toolkits. This paper introduces a practical, 10-step guide to building models from scratch, breaking down the workflow from framing the initial question to publishing the final code.
The 10-Step Modeling Framework
The authors propose a pipeline divided into four main phases, illustrating the process with an analogy of an archeologist trying to reverse-engineer the purpose of a mechanical clock.
Phase 1: Framing the Question
- Identify the phenomenon: Define a precise question (e.g., "How do the angles of the clock hands predict the time of day?").
- Literature review: Understand the current state of the art.
- Determine ingredients: Identify the necessary variables, parameters, and mechanisms.
- Formulate hypotheses: Define specific, mathematically grounded hypotheses.
Phase 2: Implementing the Model
- Select the toolkit: Choose an appropriate framework (e.g., kinematics vs. mechanics) based on the question rather than popularity.
- Plan the model: Draft diagrams and equations before coding.
- Implement: Write the code, using unit tests for individual components.
Phase 3: Model Testing
- Complete the model: Adhere strictly to the stopping criteria defined in Step 1 to avoid unnecessary complexity.
- Evaluate: Test the model using statistical comparisons and generalization tests.
Phase 4: Publishing
- Publish: Ensure the code is clean, open-source, and accompanied by the relevant data.
Key Insights for Effective Modeling
- Start with the Question, Not the Toolkit: A common pitfall is choosing a modeling framework (e.g., deep learning) before defining the exact scientific question. The toolkit should be dictated by the hypotheses and the required level of abstraction.
- Know When to Stop: A well-defined modeling goal must have well-defined stopping criteria. This prevents "mission creep," where researchers add unnecessary parameters that fit data better but reduce explanatory power, violating Occam's razor.
- Models are Not Beauty Contests: A model should be judged on whether it achieves its specific, stated goals—whether that is explaining data, making predictions, or providing mechanistic insight—rather than its complexity or fashionability.
- Preregister Modeling Plans: Similar to clinical trials, modelers should consider preregistering their hypotheses and structures. This prevents "HARKing" (Hypothesizing After Results are Known) and stops researchers from tweaking architectures just to "win" a benchmark.
Impact and Practical Application
The qualitative success of this 10-step framework has been demonstrated at the CoSMo (Computational Sensory-Motor Neuroscience) summer school. By using this systematic approach, instructors have successfully guided hundreds of students through the full "from scratch" modeling process in just two weeks. This success occurred across a vast range of model types, from spiking neural networks to normative behavioral models, proving the pipeline's versatility.
While the process is presented linearly for educational purposes, real-world modeling remains highly iterative. Drafting equations or writing code often reveals weak questions or flawed assumptions that require returning to earlier steps. By formalizing these steps, the authors aim to make model construction as rigorous and replicable as experimental design.
Summary updated