Reproducibility is a threshold, not a slogan

Our position

Computational biology can produce a coherent narrative from nearly any sufficiently large dataset. Dimensionality reduction will separate cells, enrichment tools will return pathways, and network methods will connect genes. None of those outputs is automatically false, but none is a mechanism merely because it is visually persuasive.

REELD treats reproducibility as an admission threshold for mechanistic interpretation. Before a result can anchor a therapeutic hypothesis, it should survive reasonable changes in data source, preprocessing, label resolution, background universe, ontology representation, and statistical model. The required tests depend on the claim, but the obligation to define them does not.

Reproduction, robustness, and replication are different

Computational reproduction asks whether the same code and inputs generate the same output. Analytical robustness asks whether the conclusion persists under reasonable alternate choices. Biological replication asks whether compatible evidence appears in independent samples or systems. A study can pass the first and fail the other two.

These distinctions should appear in the manuscript and release record. Containerizing an unstable analysis makes it repeatably unstable. Conversely, an independent dataset can support the same biological interpretation even when its platform and labels prevent exact numerical reproduction. The standard should match the inferential level of the claim.

• Reproduction: same inputs, code, environment, and deterministic output.
• Robustness: stable interpretation across predefined reasonable analytical choices.
• Replication: compatible evidence in independent biological data or experimental systems.

The multiverse should be designed before the preferred result

Every analysis contains a multiverse of plausible decisions: filtering thresholds, normalization, covariates, cell labels, gene universes, similarity functions, integration methods, and exclusion rules. If those decisions are explored after the result is visible, the analyst can unintentionally optimize for coherence. A convincing final figure may be one selected member of a much less stable result family.

We favor an analysis charter that names the primary model, the reasonable alternatives, the negative controls, and the failure conditions before final interpretation. Not every possible pipeline deserves equal weight. The purpose is to define the scientifically defensible neighborhood around the primary analysis and measure whether the conclusion lives throughout it.

Stability is multidimensional

A result need not reproduce with identical effect size to remain useful. Direction, rank, cell-context neighborhood, leading-edge genes, pathway interpretation, and experimental implication can each be stable or unstable. Reporting one p-value or overlap coefficient compresses these dimensions and can hide a mechanistically important disagreement.

For rare-disease mechanism work, we ask at least four questions. Does the candidate remain supported? Does the relevant cell or developmental context remain compatible? Does the molecular direction remain the same? Does the proposed experiment remain appropriate? A signal that preserves rank but reverses direction has not replicated in the way translation requires.

Negative results define the transport boundary

Failed transfer is not an embarrassment to be moved into supplementary material. It defines where the mechanism does not travel. A program present in fetal cortex but absent from adult references may be developmentally restricted. A signal present after dissociation but absent from nuclei may be technically contingent. A phenotype ranking that changes after one term is removed may depend on ascertainment quality.

The correct response is not always rejection. It may be a narrower claim. Scientific value increases when the boundary is explicit because another group can choose the right tissue, stage, assay, and comparison rather than attempting to generalize the result blindly.

What reviewers should demand

Review should move beyond asking whether code is available. Reviewers should be able to reconstruct source accessions and releases, understand the unit of inference, inspect biological-replicate handling, evaluate the background model, see sensitivity to key choices, and identify the observation that would weaken the conclusion. Availability without auditability is insufficient.

A result that fails these tests may still be an exploratory observation. It should be labeled accordingly. The problem is not exploration; it is presenting exploration with the rhetorical certainty of confirmation.

What replication cannot establish

Agreement across public datasets does not establish causality, clinical validity, target safety, or therapeutic efficacy. It reduces the space of fragile explanations and identifies a more precise experiment. That narrower claim is both defensible and useful.

The final bridge remains perturbational. A computational mechanism earns stronger status when a mechanism-matched intervention changes the predicted molecular and cellular readouts, and when correction or rescue reverses the effect. Replication improves the target; it does not replace the test.