ADR 0002: Statistical defaults and boundary policies

  • Status: Accepted
  • Date: 2026-07-15
  • Amendment: the prediction-interval variance decision is superseded by ADR 0004.
  • Amendment: the decision to defer Mantel-Haenszel risk differences is superseded by ADR 0005.
  • Amendment: the decision to defer Peto pooling is superseded by ADR 0006.

Context

Several defensible meta-analysis conventions differ across software and can produce materially different results. PyMetaAnalysis needs explicit, stable choices for random-effects confidence intervals, risk-difference boundary studies, heterogeneity statistics, Mantel-Haenszel pooling, and prediction intervals. These choices must be visible after fitting and independently testable.

Decision

Random-effects confidence intervals

ci_method="normal" remains the default. hartung_knapp implements the unmodified method, including the possibility of a variance below the classic variance. hartung_knapp_adhoc is a separate opt-in method that applies max(classic_variance, hk_variance). No data-dependent default or silent substitution is used.

Summaries recommend a Hartung-Knapp sensitivity analysis when tau-squared is positive and more than three studies are included. With three or fewer studies they recommend comparing methods and interpreting both cautiously.

Risk differences with zero raw sampling variance

The default rd_zero_variance="correct" retains double-zero, double-all, and opposite-boundary studies. Their study effect is the raw risk difference; continuity-corrected counts are used only for sampling variance.

rd_zero_variance="exclude" is available for protocols that exclude these studies. Exclusion occurs before pooling, Cochran's Q, tau-squared estimation, and weights. The result retains each row, its exclusion reason, the resolved option, and a provenance transformation.

Mantel-Haenszel scope

Mantel-Haenszel pooling remains a common-effect estimator for OR and RR. It uses raw tables by default and has a correction setting separate from the one used for individual-study effects. Random-effects Mantel-Haenszel, RD Mantel-Haenszel, and Peto pooling were outside the scope of this original decision; ADRs 0005 and 0006 supersede the latter two deferrals.

Heterogeneity

Cochran's Q, degrees of freedom, and p-value always use common-effect inverse- variance weights. Common-effect and Mantel-Haenszel analyses use Q-based I-squared and H-squared. ADR 0006 adds Peto's estimator-specific Q while retaining those Q-based inconsistency transformations.

Random-effects analyses use:

C = sum(w_i) - sum(w_i^2) / sum(w_i), where w_i = 1 / v_i
v_typical = (k - 1) / C
I^2 = tau^2 / (tau^2 + v_typical)
H^2 = 1 + tau^2 / v_typical

The result and report record i2_method as q_based or tau2_typical_variance. I-squared remains a proportion internally and a percentage in human-readable output.

Prediction intervals

The original decision used the Higgins-Thompson-Spiegelhalter prediction interval with k - 2 degrees of freedom and the classic variance of the pooled mean for every confidence-interval method. ADR 0004 replaces that variance rule for Hartung-Knapp fits while retaining the study-count boundary and small-sample warning.

Validation

Committed fixtures generated by R metafor validate common and random inverse-variance fits, tau-squared estimators, Hartung-Knapp variants, Mantel-Haenszel pooling and heterogeneity, prediction intervals, and repeated- fit workflows. Sparse RD variances are cross-checked against metafor; the intentional raw-effect difference for a single-zero study is asserted separately.

These references are independent software comparisons, not a claim that the package has undergone an external statistical audit.

Consequences

  • defaults remain reproducible and do not change according to observed data;
  • non-default statistical choices are explicit in public arguments;
  • summaries, reports, method text, study tables, and provenance expose the resolved conventions;
  • changing a convention requires a new ADR, reference updates, and regression tests.