Background The masking properties of a new, non-penetrating, double-blind placebo acupuncture needle were demonstrated. Practitioners correctly identified some of the needles; if they were confident in this opinion, they would be unblinded.
Objective To investigate the clues that led to correct identification, and the confidence in this decision.
Methods Ten acupuncture practitioners, blindly and randomly, applied 10 each of three types of needle to the shoulder: blunt, non-penetrating needles that pressed the skin (‘skin-touch placebo needle’); new non-penetrating needles that penetrated soft material (stuffing) but did not reach the skin (‘non-touch control needle’); matching penetrating needles. Afterwards, practitioners were asked to judge the type of needle, their confidence in their decision and what clues led them to their judgements.
Results Of the 30 judgements made by each practitioner, the mean number of correct, incorrect and unidentifiable answers were 10.4 (SD 3.7), 15.2 (SD 4.9) and 4.4 (SD 6.1), respectively. There was no significant difference in the confidence scores for 104 correct (mean, 54.0 (SD 20.2)%) and 152 incorrect (mean, 50.3 (SD 24.3)%) judgements. Twelve needles were identified with 100% confidence—three correct, and nine incorrect. For needles correctly identified, the proportions of non-touch (p = 0.14) and skin-touch (p = 0.17), needles were no greater than chance, but the proportion of penetrating needles correctly identified exceeded chance (p < 0.01). 53% of judgements were made from the “feeling of needle insertion”, but 57% of these were wrong.
Conclusion Practitioners had a slight tendency to guess the penetrating needles correctly, but were uncertain about most of their judgments, posing only a very small risk to double blinding.
Statistics from Altmetric.com
Double-blind design, where both patient and practitioner are masked to the treatment condition, is critical for accurately measuring the efficacy of any treatment, including acupuncture, but blinding acupuncture practitioners is a great challenge.1 2 Without blinding practitioners, measurement of treatment effects may be influenced by bias, so acupuncture research may be seen as having less methodological rigor than conventional medical research.3,–,8
Blinding of acupuncturists was not considered feasible1 2 9 10 until we designed and validated a new double-blind needle.11,–,14 The blunt needle simply presses against the skin, but meets some resistance from soft material (lower stuffing) in the guide tube to give the impression that it penetrates.11,–,14 In this paper, we call this the ‘skin-touch placebo needle’ to distinguish it from another type of needle we describe later.
In our first validation study, 10 experienced acupuncturists each applied 23 non-penetrating placebo needles and 17 conventional penetrating needles to the LI4 point—a total of 400 applications. After removing each needle, they made a judgement about whether the needle was ‘penetrating’, ‘non-penetrating placebo’ or ‘unidentifiable’. They judged 170 needles correctly, 166 incorrectly and rated 64 as unidentifiable, which was a chance distribution.11 12 Subsequent investigations provided further validation: experienced practitioners made statistically an equal number of correct and incorrect judgements of the type of needle in one study; and, in another study, they made a larger number of incorrect judgements than correct ones.13 14 Therefore we concluded that the new needle is effective for blinding.11,–,14
However, because of the way we conducted those previous studies, we could not tell to what extent practitioners were able to truly identify any of the needles—for example, from the sensation in their fingers as they used them. If a proportion of needles can be truly identified with confidence, this would jeopardise the blinding.
For this study, we designed a new version of the placebo needle in which the tip does not reach the skin but still meets resistance from soft material. Our aim was to test whether this produced a different sensation for the practitioners, but we were also interested to determine whether a non-touch placebo needle was realistic, since researchers have questioned whether placebo needles that touch the skin might produce stimulation and therefore not be true placebos.15 16
The aims of this study were to explore to what extent practitioners can determine the true nature of these needles from the feeling during use; what these judgements were based on; and how confident they were in them.
We recruited 10 licensed and experienced acupuncturists (mean duration of acupuncture experience 10.8 (SD 12.9) years; mean age: 40.1 (SD 11.1) years; four men and six women) from the teaching or research staff of Japan School of Acupuncture, Moxibustion and Physiotherapy, The Educational Foundation Hanada Gakuen, Tokyo, Japan (table 1). Before the study, its purpose and format were explained and the participants provided written consent. The Showa University Ethics Committee gave its approval.
Design of double-blind needles
We used three types of needles in this study. (a) the non-penetrating placebo needle, the tip of which presses against the skin but cannot penetrate it (‘skin-touch placebo needle’); (b) the matching ‘penetrating needle’ with 10 mm insertion depth. These have been described in detail elsewhere.11,–,14 Additionally, in this study, we used (c) the newly developed needle (‘non-touch control needle’), the tip of which cannot reach the skin. The appearance of these three needles is indistinguishable (figure 1). The diameter of the needles was 0.16 mm.
Validation test for practitioner blinding
For each practitioner we prepared 10 of each type of needle and shuffled them to achieve a random order.
Practitioners were informed about the design of the needles, and instructed to apply the needles as if treating a stiff neck using any appropriate acupuncture or tender points they wished. The acupuncturists applied each needle separately, taken randomly from the shuffled set of 30 needles, into the shoulder of the author (NT), 15 on each side, using the ‘alternating twirling technique’—that is, insertion by rapid rotation of the needle. The acupuncturist then advanced the needle until the stopper made contact with the top of the guide tube and finally withdrew it (figure 1), while observing the reactions of the patient. No further needle manipulation was used in this study, reflecting one of the techniques most commonly used by practitioners in Japan. The acupuncturist then removed the entire needle assembly from the skin. An assistant, blind to the true nature of needles and the practitioner's judgements, numbered the needle and sealed it in an opaque envelope. The practitioner then reported his/her judgement as to whether the needle was non-touch control, skin-touch placebo, penetrating needle or unidentifiable. Confidence in making this judgement (ie, the degree of certainty about the decision) was then rated by the practitioner on a visual analogue scale, the end points of which were 0 for no confidence and 100 for complete confidence. Clues that led to the judgement of identity of the needle were then reported by the practitioner; the options were ‘feeling of needle insertion’, ‘feeling of needle removal’, ‘feeling in the left hand holding the guide tube (‘Oshide’ in Japanese)’ (figure 1), ‘body movement’, ‘bleeding’ and ‘lack of bleeding’.
The author (NT), blinded to the practitioners' responses, subsequently opened the opaque envelopes to record the type of needles. We took all possible precautions to ensure that the identity of the needle was not revealed to the practitioners, the patient or the investigators during the acupuncture trials.
A χ2 test was used to determine whether the numbers of needles fitted an expected probability. Practitioners' confidence scores for correct and incorrect identifications were compared using the Mann–Whitney U test; comparisons between three groups were made using Kruskal–Wallis test. Spearman's rank correlation coefficient was used to indicate the relationship between the years of practitioners' experience for acupuncture practice and number of correctly identified needles. All statistical analyses were performed using SPSS, version 15.0J (SPSS, Chicago, Illinois, USA).
All 10 acupuncture practitioners completed all judgements for all needles. The mean of the practitioners' overall confidence in making their judgements was 44.2 (SD 28.0)%. Of the 300 needles applied, the practitioners identified 104 (34.7%) correctly (non-touch control=34, skin-touch placebo=23, penetrating=47); 152 (50.7%) incorrectly (non-touch control=49, skin-touch placebo=64, penetrating=39); and reported 44 (14.7%) as unidentifiable (non-touch control=17, skin-touch placebo=13, penetrating=14), see table 1. The 104 correctly identified needles overall fitted the probability of one in three (χ2=0.24, p=0.62). There was no significant difference in the confidence scores for correct (mean 54.0 (SD 20.2)%) and incorrect (mean 50.3 (SD 24.3)%) judgements (p=0.09). Practitioners stated they were 100% confident in 12 judgements: three were correctly identified (one non-touch control and two penetrating needles), seven were incorrectly identified as penetrating (one non-touch control and six skin-touch placebo needles) and two were incorrectly identified as non-touch control (skin-touch placebo needles). For each of the non-touch control and skin-touch placebo needles, the proportion of correctly identified needles fitted the probability of one in three (χ2=2.18, p=0.14; χ2=1.86, p=0.17). The 47 penetrating needles that were correctly identified exceeded the chance probability of one in three (χ2=17.59, p<0.01) (table 1). However, there was no significant difference in the confidence scores for different types of needle (p=0.69) (figure 2).
Practitioners were more likely to judge the needles as ‘penetrating’ (119 out of 256 needles identified, χ2=19.9, p<0.01). However, 47 correct identifications of the 119 needles fitted the probability one in three (χ2=2.03, p=0.15) as did 23 correct identifications of 52 needles judged as skin-touch placebo (χ2=2.78, p=0.10) and 34 correct identifications of 85 needles judged as non-touch control (χ2=1.70, p=0.19). The mean confidence score for 47 correctly identified penetrating needles was 55.9 (SD 21.4)% which was higher, though not significantly (p=0.17), than 48.5 (SD 17.1)% of the correctly identified skin-touch placebo and 55.2 (SD 20.4)% of the correctly identified non-touch control needles (figure 2). Importantly, there was no significant difference in confidence between non-touch control, skin-touch placebo and penetrating needles identified as penetrating (p=0.15); or for those that were identified as non-touch control (p=0.34) and skin-touch placebo (p=0.70) (figure 2).
The practitioners made their judgements on the nature of the needles principally on the ‘feeling of needle insertion’ and were far less dependent on other clues. The mean score for confidence of the practitioners in 69 needles correctly identified from ‘feeling of needle insertion’ was 57.6 (SD 19.0)% (table 2).
The highest percentage of correct answers was 60.0% in the acupuncturist with 2.5 years of acupuncture experience, and the lowest was 13.3% in the acupuncturist with 1.5 years of acupuncture experience (table 1). There was no significant correlation between the years of experience in acupuncture and the numbers of correctly identified needles (r=0.42, p=0.23) (table 1).
In previous validation studies we found that recently developed double-blind needles have the potential to mask practitioners.11,–,14 But there still remained a question about whether practitioners were sometimes certain of their judgement about a needle, which could jeopardise blinding. Practitioners in this study judged a great majority of correctly identified needles with uncertainty, which indicates that the identity of the needles was well blinded from the practitioners. However, practitioners in this study had a tendency to judge needles to be penetrating compared with the previous studies.11,–,14 Further, a few (<3%) correctly identified needles were reported with 100% confidence. These results suggest a potential limitation in the success of perfect practitioner blinding in clinical acupuncture studies. Therefore, in future, confidence in the practitioner's guesses should be recorded to see whether the true identity of the needle is revealed.
The 60% and 50% correct identification obtained by practitioners No 5 and No 10 were striking scores, as was the lowest figure, 13% by practitioner No 3. However, the success rates of these three seeming outliers were within 2 SDs (±12.9) of the mean success rate of 34.7. Thus statistically the three practitioners did not perform much better or worse than the others. There was no significant correlation between the years of experience of acupuncture practice and the number of correctly identified needles. Duration of experience does not seem to affect judgement. In our previous validation studies, acupuncture points to which the needles were applied were located in the upper extremity.11,–,14 In this study acupuncturists applied the needles to the shoulder, to replicate a clinical setting. However, the double-blind needle still needs to be validated in a true clinical trial, and by other centres.
Although a non-penetrating needle that presses the skin may be physiologically active, we call our non-penetrating needle ‘placebo’ because it lacks a key ingredient of acupuncture: skin penetration. At present we are not aware of any conclusive evidence as to whether placebo, sham or another type of needle has a specific therapeutic effect on any medical condition obtained under double-blind conditions using an appropriate control. The new, non-touch needle designed for this study provides no physiological stimulation by the needle tip, and our results suggest that it would be suitable for practitioner blinding. It would be important to compare the effect of these two needles on the symptoms of a clinical condition, though it may be difficult to mask patients. Thus, the explanation given to patients will be the next challenge in adopting such non-touch control needles in research.
In conclusion, this study suggests that unblinding is not likely with this placebo needle, since practitioners were not certain about a large majority of their correct identifications. However, it showed that achievement of perfect practitioner blinding was limited because practitioners had a tendency to guess needles to be penetrating, and because the true identities of some needles may be revealed.
▶. We previously reported a new placebo needle which blinds the practitioner.
▶. Here we compared it with a genuine needle and another form of placebo needle.
▶. Practitioners were more likely to guess the ‘real needle’.
▶. But they were not confident in saying that, so probably remain correctly blinded.
The authors thank Ikuo Homma (Second Department of Physiology, Showa University School of Medicine, Tokyo) for his support. The authors also express their appreciation to all the participants of this study.
Funding The work of TJK was supported by NCCAM-NIH grant # K24 AT004095.
Competing interests NT and The Educational Foundation of Hanada Gakuen possess a US patent 6575992B1, a Canadian patent CA 2339223, a Korean patent 0478177, a Taiwan patent 150135, a Chinese patent ZL00800894.9 (Title: Safe needle, placebo needle, and needle set for double blind) and a Japanese patent 4061397 (Title: Placebo needle, and needle set for double-blinding) on the needles described in this manuscript. NT is a salaried employee of The Educational Foundation of Hanada Gakuen and has received research funding from The Educational Foundation of Hanada Gakuen.
Patient consent Obtained.
Ethics approval This study was conducted with the approval of the Showa University School of Medicine.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.