Sensitivity analysis¶

Sensitivity workflows repeatedly refit the stored model. They help reveal how the pooled estimate and heterogeneity depend on individual studies or on the order in which evidence accumulated. They do not automatically diagnose why a study is influential or replace protocol-specified sensitivity analyses.

Leave-one-out analysis¶

Call leave_one_out() on any fitted MetaAnalysisResult:

import pandas as pd
import meta_analyze as ma

studies = pd.DataFrame(
    {
        "effect": [0.12, 0.35, -0.08, 0.21, 0.42, 0.28],
        "variance": [0.04, 0.06, 0.03, 0.05, 0.07, 0.045],
        "publication_year": [2001, 2004, 2004, 2008, 2011, 2015],
        "dose": [0, 1, 2, 3, 4, 5],
        "region": ["A", "A", "A", "B", "B", "B"],
    }
)
result = ma.meta_analysis(studies, effect="effect", variance="variance")
influence = result.leave_one_out()

print(influence.to_dataframe())

The returned LeaveOneOutResult contains:

• original, the fitted result supplied to the workflow;
• results, one omission-aligned immutable refit or None per included study;
• table, summary(), and to_dataframe(), which return defensive copies;
• failed, the rows for reduced models that could not be estimated;
• warnings, for workflow-level notes.

The table identifies omitted_row_id and omitted_study and reports each refit's estimate, standard error, confidence interval, tau-squared, Q, I-squared, and H-squared. It also records refit_success, error_type, and error_message. An unestimable deletion remains aligned in results as None, retains a table row with unavailable numeric values, and does not stop the remaining deletions.

Originally excluded rows are never omission candidates. Common-effect analysis requires at least two included studies so each refit retains one. Random-effects analysis requires at least three so every refit retains the two studies needed to estimate tau-squared.

Meta-regression leave-one-out analysis¶

MetaRegressionResult.leave_one_out() repeats the fitted regression while omitting each included study:

regression = ma.meta_regression(
    studies, effect="effect", variance="variance", moderators=["dose"]
)
diagnostics = regression.leave_one_out()

print(diagnostics.table)
print(diagnostics.coefficients)

The returned MetaRegressionLeaveOneOutResult retains original and a results tuple aligned with the omitted-study rows. Its model table reports the refitted residual heterogeneity, global moderator test, condition number, and warnings. Its long-form coefficient table reports every refitted term and estimate_change, defined as the deleted estimate minus the full-model estimate.

Deleting a study can make a categorical level disappear or otherwise make the design matrix unidentifiable. That deletion is retained with refit_success=False, an exception type and message, unavailable numeric fields, and None in the matching results position. Other deletions continue to be fitted. Use diagnostics.failed to inspect these rows. A failed deletion does not by itself label that study as influential; it shows that the fitted design depends on the study for identifiability.

The original model must have at least k >= p + 2, so deleting one study can still leave more studies than coefficients. Each successful refit re-estimates tau-squared and coefficient inference with the original model settings.

Meta-regression influence diagnostics¶

MetaRegressionResult.influence() builds on the same exact deleted-model refits:

influence = regression.influence()

print(influence.table)
print(influence.dfbetas)
print(influence.flagged)

MetaRegressionInfluenceResult contains the underlying leave_one_out workflow and its omission-aligned results. The case-level table adds:

• the deleted residual, its standard error, and the externally standardized residual;
• Cook's distance based on the full-model coefficient covariance;
• the largest absolute DFBETAS value across fitted terms;
• the original leverage and normalized precision weight;
• separate outlier and influence screening flags.

The long-form dfbetas table records the unstandardized dfbeta as full-model minus deleted-model coefficient, its deletion-based standard-error reference, the resulting DFBETAS value, and its threshold flag for each term. Failed deletion fits retain their exception type and message in table, use NaN diagnostics, and retain one unavailable row per term in dfbetas.

The screening references are explicit in both the result attributes and table:

• abs(externally_standardized_residual) > z(0.975) is a pointwise, asymptotic potential-outlier reference;
• Cook's distance is flagged above the median of a chi-squared distribution with p degrees of freedom;
• abs(DFBETAS) > 1 flags a coefficient-specific change.

potentially_influential combines only the Cook's-distance and DFBETAS rules; flagged also includes the residual outlier screen. These cutoffs are heuristic diagnostics, not hypothesis-test conclusions, multiplicity-adjusted decision rules, or instructions to exclude a study. Inspect subject-matter differences, data quality, protocol decisions, and model specification before acting on them.

Cumulative analysis¶

By default, cumulative analysis follows input order:

cumulative = result.cumulative()
print(cumulative.to_dataframe())

Supply a DataFrame column name or one-dimensional array-like to define another stable order:

cumulative = result.cumulative(
    order="publication_year",
    ascending=True,
)

A string selector must identify exactly one namespace. If the same column name exists in both the original source data and the result study table, it is ambiguous and raises InvalidStudyDataError. Pass the intended Series or array explicitly, for example order=studies["publication_year"].

CumulativeMetaAnalysisResult.results contains the estimable prefix fits, and final returns the last fit. Its table records the rows and study labels added at each step together with the same principal statistics as leave-one-out. If an otherwise eligible prefix is not estimable, it is skipped with an explicit warning. Its pending studies are included in the next estimable prefix row, so the cumulative path and final all-study fit remain complete.

When multiple studies have the same order value, collapse=True adds the tied studies in one step:

cumulative = result.cumulative(
    order="publication_year",
    collapse=True,
)

Random-effects cumulative analysis starts at the first two-study prefix. The unavailable single-study prefix is not silently fitted as a common-effect model, and the boundary is recorded in cumulative.warnings.

Reused analysis settings¶

Every refit reuses the original result's:

• common- or random-effects model;
• inverse-variance, Mantel-Haenszel, or Peto pooling method;
• tau-squared and confidence-interval methods;
• confidence level, absolute tolerance, and iteration limit;
• missing-data policy;
• binary continuity corrections and RD zero-variance policy, the continuous effect-size convention, or the Fisher's z correlation convention.

Meta-regression refits additionally reuse the intercept choice, inference method, moderator order, and complete explicit categorical level definitions.

Derived results retain provenance that maps their local calculations back to the original row_id values.

Repeated-fit tables include i2_method so a random-effects path cannot be mistaken for a Q-based inconsistency series.

Subgroup results¶

The same methods are available on SubgroupMetaAnalysisResult:

subgroups = ma.meta_analysis(
    studies, effect="effect", variance="variance", subgroup="region"
)
subgroup_influence = subgroups.leave_one_out()
subgroup_cumulative = subgroups.cumulative(order="publication_year")

These return composite objects with .groups and .overall workflows. Each subgroup is refitted independently. The paths do not reinterpret different numbers of subgroup steps as a new sequence of subgroup-differences tests.

Interpretation¶

Large changes after omitting a study warrant inspection of its population, design, outcome definition, risk of bias, and numerical leverage. They are not by themselves a reason to exclude the study. Likewise, a cumulative trend can describe the historical evidence path but does not remove time-related changes in methods, populations, or publication processes.