Statistics from Altmetric.com
We believe AiM readers will be interested to learn of the factors that led to negative reporting of the positive study by Hinman et al on acupuncture for knee osteoarthritis (OA) pain.1 The study showed the effects of acupuncture to be significantly superior to no acupuncture and consistent with the best current evidence, but the authors actually reported: ‘Our findings do not support acupuncture for these patients’.
Patients with OA knee pain are suffering the commonest cause of pain and disability in elderly people. More than half have inadequate pain relief.2 They face ‘a choice between ineffective paracetamol, non-steroidal drugs that can harm the heart, (kidneys) and gastrointestinal tract, gels that scarcely work, physiotherapy, opioids that cause dependency and lose effectiveness, arthroscopic washouts that do nothing or surgery’.3 They deserve a fuller, more considered answer to their question: should they try acupuncture?
The neat part of the Zelen design that Hinman et al used was that the control group, who were not given acupuncture, were not even aware that their pain scores were used in a trial of acupuncture so disappointment could not influence their scores, as was claimed for other studies. This ‘no acupuncture’ group was compared with acupuncture (manual) and with sham laser (and with real laser, which is not considered here, to keep things simple).
The problems started with the trialists’ choice of the threshold minimum clinically important difference (MCID) to estimate sample size. They chose a value based on one chosen by six self-styled ‘expert’ physicians,4 namely a 35% fall in baseline pain score (1.8/sample mean baseline 5.1). This is equivalent to an effect size (ES) of 0.6, calculated using their assumed baseline SD of 30 (the actual SD was 21, giving a higher threshold ES of 0.86). A different figure for MCID was generated by 192 patients with OA, who registered improvement scores as well as changes in pain.5 This showed a more modest MCID, equivalent to an ES of 0.39 (shown in figure 1). The National Institute for Health and Care Excellence (NICE) did not regard any value for MCID as valid6 and chose a generic value of 0.5 (see figure 1). Hinman et al chose a high threshold and also failed to discuss the effect that alternative threshold MCID values would have on the interpretation of their findings. We also note that the MCID for any treatment should be chosen to take account of acceptability, safety and cost-effectiveness,7 which would argue for a lower threshold for acupuncture for knee pain.
Next, Hinman et al applied this ‘clinically important’ difference to a ‘clinically irrelevant’ comparison—acupuncture versus sham laser. Sham laser is not an available therapy. The only reason for comparing acupuncture with sham would be to estimate the effects of the needles themselves, but this is already well known from the Cochrane review8 and an individual patient data meta-analysis (figure 1).9 It is known that the effect of needles alone is small, and so is unlikely to be identifiable reliably with sample sizes of less than about 800.10 The sample size in the study by Hinman et al (n=70) clearly appears to be inadequate for the question, according to the existing evidence, and not best use of resources. The resulting ES of acupuncture against sham that was actually found by Hinman is similar to that shown by the best evidence8 (see figure 1), although the wide CI means the data can only be of any importance when they are included in a meta-analysis in the future.
Despite the existing evidence that already indicates a large and useful difference between acupuncture and no acupuncture (figure 1), a new study was justified using the Zelen design on the grounds that it avoided any risk of response bias of previous studies. Reassuringly, the effect of acupuncture is consistent with previous evidence—reassuring in the sense that it seems unlikely that response bias has affected the validity of the evidence from other studies.
Hinman et al found that, after 12 weeks, knee pain was significantly reduced by acupuncture compared with no acupuncture control, with an ES of 0.6 (data from their table 2; see figure 1). The difference did not quite meet the MCID they had postulated—although the estimated ES is the same size as the MCID—but it more than meets the MCID chosen by patients themselves (ES 0.39) and that selected by NICE (ES 0.5). In interpreting this result, the secondary outcomes should also have been brought into thoughtful consideration: there were significant differences in favour of acupuncture for six out of eight secondary outcomes (see eTable 5 in their paper) and the response rate, which is the most patient-orientated measure of success,7 was 76% in the acupuncture group compared with 32% in the no acupuncture control group.
It should also be noted that Hinman et al did not apply optimal acupuncture. Use of electroacupuncture was shown to be superior to manual stimulation alone for knee pain in 2010.8
So, the correct message from the study by Hinman et al is that even suboptimal acupuncture gives clinically relevant benefits for patients with knee OA who have few options other than surgery. Their results give a powerful and positive result that is clearly consistent with the best data from other studies. Instead of concluding that their findings do not support acupuncture for these patients, they should have concluded that patients with knee OA should consider acupuncture as an option. Indeed, acupuncture is more likely to give relief than any other option: a network analysis comparing physical interventions for knee pain shows acupuncture to be best with an ES of 0.89; the nearest is warm baths (ES 0.65) followed by exercise (ES 0.55).11 The global evidence clearly shows that acupuncture offers real and meaningful benefits for these patients with real pain and disability.
(This letter is a modified version of a letter submitted for publication to JAMA but rejected).
Competing interests AW and MC are employed by the British Medical Acupuncture Society (BMAS) as journal editor and medical director respectively and have received travel expenses.
Provenance and peer review Not commissioned; internally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.