Article Text

Problems caused by heterogeneity in meta-analysis: a case study of acupuncture trials
  1. Stephanie L Prady1,
  2. Jane Burch2,
  3. Simon Crouch1,
  4. Hugh MacPherson1
  1. 1Department of Health Sciences, The University of York, York, UK
  2. 2Centre for Reviews and Dissemination, The University of York, York, UK
  1. Correspondence to Dr Stephanie L Prady, Department of Health Sciences, Seebohm Rowntree Building, Area 2, University of York, Heslington, York YO10 5DD, UK; stephanie.prady{at}


Objectives To illustrate the pitfalls of using meta-analysis to combine estimates of effect in trials that are highly varied and have a high potential for bias.

Methods We used a random-effects meta-analysis to pool the results of 51 sham-controlled acupuncture trials of chronic pain published in English before 2008 and explored the heterogeneity using meta-regression. We repeated the process on a subset of these trials that used a visually credible non-penetrating sham device as control (N = 12).

Results In both analyses there were high levels of heterogeneity and many studies were at risk from potential bias. The heterogeneity was not explained by meta-regression.

Conclusions Trials of interventions that have high potential for bias, such as many in the acupuncture literature, do not meet the assumptions of the statistical procedure that underlie random-effects meta-analysis. Even in the absence of bias, heterogeneity in meta-analyses is not accounted for by the CIs around the pooled estimate.

  • Acupuncture
  • Systematic Reviews
  • Statistics & Research Methods

Statistics from

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.