Statistical Methods Section From Your Code

Start from the reporting guideline for your study design, and check whether your journal wants the completed checklist with your submission. Observational studies use STROBE, whose item 12 asks you to describe all statistical methods, explain how missing data were addressed and describe any sensitivity analyses². Randomized trials use CONSORT 2025³, which replaced the 2010 checklist and now asks who is included in each analysis.

Say which method produced each result rather than listing every technique in one block. Then follow your journal's headings and order: the guideline says what to include, not where.

Analyses drift while you work. A covariate gets added late, a filter moves into a cleaning script, and defaults settle questions you never asked. So before you write a sentence, list what the script actually does.

1. 1

   List every model

   Give each model the script fits a row, in run order, whether or not it reached the paper.

2. 2

   Copy the specification

   Record the outcome and covariates exactly as written, the estimator, and every option changed from its default.

3. 3

   Count the rows each model used

   Read the number from the fitted model (nobs() in R, e(N) in Stata, .nobs in statsmodels), not from the dataset.

4. 4

   Match tables to models

   Trace each table to the fitted object that produced it, and note any you can't trace.

Check what your software does with missing values before you describe it. R's lm() drops any row with a missing value in the model's variables, because its documented "factory-fresh" default is na.omit⁴. So every covariate you add can shrink the sample again, and two models in the same table can rest on different people.

In the BMJ, Jonathan Sterne and colleagues note that the cumulative effect of missing data in several variables often leads to exclusion of a substantial proportion of the original sample⁵. They ask for the number of missing values in each variable, or of complete cases in each important part of the analysis. If you imputed, name the method and its assumptions (for example, missing at random).

Observations often come in groups whose members resemble each other, such as employees within firms or patients within clinics. In that setting, economists A. Colin Cameron and Douglas Miller warn in the Journal of Human Resources that default standard errors can greatly overstate estimator precision⁶. Cluster-robust errors correct for this, but only if the code requests them, for example with vce(cluster) in Stata.

Confirm which version each table shows. In R, clustered errors often live in a second object, and a table built from the original fit still prints the classical ones.

Report the clustering variable and the number of clusters. With few clusters, unadjusted cluster-robust tests produce too many false positives. Cameron and Miller say "few" can mean anything from under 20 to under 50 equal-sized clusters, and more when sizes vary. Say whether and how you corrected for it.

Your list will often hold more models than the paper reports. Sort the extras: sensitivity analyses, subgroup or interaction models, superseded build-up steps, mis-specified fits you abandoned, and duplicates. The first two belong in the methods section, since STROBE asks you to describe any methods used to examine subgroups and interactions² and any sensitivity analyses.

Then label each analysis as planned or not. ICMJE asks authors to distinguish prespecified from exploratory analyses, including subgroup analyses¹, and CONSORT 2025 asks trials to describe any additional analyses, distinguishing prespecified from post hoc³.

Mis-specified fits and exact duplicates usually need no mention. Keep the list anyway: it answers the reviewer who asks how you reached the final model, and it makes a response to reviewers quicker to write.

ICMJE also asks you to specify the statistical software package(s) and versions used¹. Versions matter because defaults change between releases. R 3.6.0 changed how sample() draws random numbers, and R's release notes say the old method can be requested if necessary for reproduction of old results⁷. A bootstrap rerun with the same seed can give different intervals on either side of that release.

Read versions from the session, not from memory: sessionInfo() in R, the about command in Stata, or a lock file or pip freeze in Python. Name every package that changed the numbers, such as the one that computed clustered errors. Keep that record with the code and data, wherever your research data management plan says they will live.

Write the introduction after this section, so it promises only what these analyses can support. To draft the methods from your own script, start with the tool at the top of this page.