Meta-analysis in Clinical Trials

In medical research, information on the outcomes of a treatment may be available from a number of clinical studies with similar treatment protocols.  When the studies are considered individually, they may be either too small or too limited in scope to come to a general conclusion about the effect of the treatment.  Meta-analysis, defined as the statistical analysis of a large collection of analytic results for the purpose of integrating the findings, attempts to combine the results of multiple studies in order to gain statistical power, strengthen the evidence about possible treatment effects and, in adequately-powered studies, learn more about subgroups and possible interactions.

In its simplest form, meta-analysis uses what are known as frequentist fixed-effect approaches, with weighted averages of fully aggregated data (obtained from publications), to compare two treatment groups.  For example, this may use information on the mean weight loss and standard deviation for an active and placebo drug from five trials.  More advanced methodologies (such as random-effects models, meta-regression, and Bayesian approaches) allow the researcher to incorporate complex data structures (such as subgroup data, or individual patient data (IPD)).  Note that higher model complexity carries both advantages and disadvantages.

IPD is widely regarded as the gold-standard data structure for meta-analysis.  The use of IPD typically provides greater statistical power, by incorporating study-level covariates that represent the average covariate response of each study. However, issues relating to the ownership and accessibility of IPD need to be addressed, as well as the time and effort involved in the collection of data, and so it is often unclear whether the benefits gained from using IPD outweigh the extra costs involved.

In a fixed-effects meta-analysis model, it is assumed that the result from each individual study estimates the same quantity and any deviations are due to random sampling variability. Although this may sound reasonable, it does not take into account the possible heterogeneity between trials such as different patient characteristics, or different treatment regimes. Therefore, it may be unreasonable to assume that the differences between trials are due to random variation alone. A random-effects model relaxes this assumption, allowing the treatment effect-sizes to differ from each other under the assumption that they are drawn from a common distribution.  This allows random-effects models to account for between-study heterogeneity, and often leads to pooled estimates that are similar to those obtained from the corresponding fixed-effects models.  Random-effects models typically produce wider confidence intervals than their corresponding fixed-effects models, meaning that the effect estimates are more conservative.

In recent years, Bayesian approaches have gained popularity due to the flexibility of their associated models.  Whereas frequentist approaches to meta-analysis generally calculate a weighted average of results obtained from individual studies (where model parameters are assumed to be constant and unknown), Bayesian approaches consider both the data and the model parameters to be random quantities about which there is uncertainty.  The likely values of these parameters are described using a probability distribution that quantifies this uncertainty, which allows all parameter uncertainty to be automatically accounted for in the analysis.  In the context of meta-analysis, individual trials can then “borrow strength” from other trials, meaning that with every iteration the updated estimates take into account the results from all other trials in the analysis, giving a better estimate for the individual trial effects.

Bayesian approaches can incorporate varying levels of data aggregation (fully aggregated, subgroup aggregation and IPD), mixed modelling (more than one covariate parameter), and analyses involving more than 2 treatment groups (indirect comparisons and network meta-analysis).  Prior knowledge of an effect can also be incorporated into an analysis by prescribing a suitable prior distribution (note that a sensitivity analysis is always recommended, because the choice of prior distribution can change the conclusions).  Consequently, Bayesian approaches carry a higher computational cost than frequentist approaches, and do not produce a direct measure of statistical significance that is analogous to the frequentist p-value, but the interpretation of Bayesian credible intervals is more intuitive than that of frequentist confidence intervals, since direct probability statements can be made regarding the model parameters and the quantities of interest.