Article Text

PDF

Double blinding with a new placebo needle: a further validation study
  1. Nobuari Takakura1,2,3,4,
  2. Miho Takayama1,2,3,4,
  3. Akiko Kawase1,2,3,4,
  4. Ted J Kaptchuk5,
  5. Hiroyoshi Yajima1,2,3,4
  1. 1Department of Acupuncture and Moxibustion, Tokyo Ariake University of Medical and Health Sciences, The Educational Foundation of Hanada Gakuen, Tokyo, Japan
  2. 2Second Department of Physiology, Showa University School of Medicine, Tokyo, Japan
  3. 3The Foundation for Oriental Medicine Research, Tokyo, Japan
  4. 4Japan School of Acupuncture, Moxibustion and Physiotherapy, The Educational Foundation of Hanada Gakuen, Tokyo, Japan
  5. 5Osher Research Institute, Harvard Medical School, Boston, Massachusetts, USA
  1. Correspondence to Professor Nobuari Takakura, Department of Acupuncture and Moxibustion, Faculty of Health Sciences, Tokyo Ariake University of Medical and Health Sciences, 2-9-1 Ariake, Koto-ku, Tokyo 135-0063, Japan; takakura{at}t-ariake.ac.jp

Abstract

Background The masking properties of a new, non-penetrating, double-blind placebo acupuncture needle were demonstrated. Practitioners correctly identified some of the needles; if they were confident in this opinion, they would be unblinded.

Objective To investigate the clues that led to correct identification, and the confidence in this decision.

Methods Ten acupuncture practitioners, blindly and randomly, applied 10 each of three types of needle to the shoulder: blunt, non-penetrating needles that pressed the skin (‘skin-touch placebo needle’); new non-penetrating needles that penetrated soft material (stuffing) but did not reach the skin (‘non-touch control needle’); matching penetrating needles. Afterwards, practitioners were asked to judge the type of needle, their confidence in their decision and what clues led them to their judgements.

Results Of the 30 judgements made by each practitioner, the mean number of correct, incorrect and unidentifiable answers were 10.4 (SD 3.7), 15.2 (SD 4.9) and 4.4 (SD 6.1), respectively. There was no significant difference in the confidence scores for 104 correct (mean, 54.0 (SD 20.2)%) and 152 incorrect (mean, 50.3 (SD 24.3)%) judgements. Twelve needles were identified with 100% confidence—three correct, and nine incorrect. For needles correctly identified, the proportions of non-touch (p = 0.14) and skin-touch (p = 0.17), needles were no greater than chance, but the proportion of penetrating needles correctly identified exceeded chance (p < 0.01). 53% of judgements were made from the “feeling of needle insertion”, but 57% of these were wrong.

Conclusion Practitioners had a slight tendency to guess the penetrating needles correctly, but were uncertain about most of their judgments, posing only a very small risk to double blinding.

  • Accepted 12 March 2010

Statistics from Altmetric.com

Introduction

Double-blind design, where both patient and practitioner are masked to the treatment condition, is critical for accurately measuring the efficacy of any treatment, including acupuncture, but blinding acupuncture practitioners is a great challenge.1 2 Without blinding practitioners, measurement of treatment effects may be influenced by bias, so acupuncture research may be seen as having less methodological rigor than conventional medical research.3,,8

Blinding of acupuncturists was not considered feasible1 2 9 10 until we designed and validated a new double-blind needle.11,,14 The blunt needle simply presses against the skin, but meets some resistance from soft material (lower stuffing) in the guide tube to give the impression that it penetrates.11,,14 In this paper, we call this the ‘skin-touch placebo needle’ to distinguish it from another type of needle we describe later.

In our first validation study, 10 experienced acupuncturists each applied 23 non-penetrating placebo needles and 17 conventional penetrating needles to the LI4 point—a total of 400 applications. After removing each needle, they made a judgement about whether the needle was ‘penetrating’, ‘non-penetrating placebo’ or ‘unidentifiable’. They judged 170 needles correctly, 166 incorrectly and rated 64 as unidentifiable, which was a chance distribution.11 12 Subsequent investigations provided further validation: experienced practitioners made statistically an equal number of correct and incorrect judgements of the type of needle in one study; and, in another study, they made a larger number of incorrect judgements than correct ones.13 14 Therefore we concluded that the new needle is effective for blinding.11,,14

However, because of the way we conducted those previous studies, we could not tell to what extent practitioners were able to truly identify any of the needles—for example, from the sensation in their fingers as they used them. If a proportion of needles can be truly identified with confidence, this would jeopardise the blinding.

For this study, we designed a new version of the placebo needle in which the tip does not reach the skin but still meets resistance from soft material. Our aim was to test whether this produced a different sensation for the practitioners, but we were also interested to determine whether a non-touch placebo needle was realistic, since researchers have questioned whether placebo needles that touch the skin might produce stimulation and therefore not be true placebos.15 16

The aims of this study were to explore to what extent practitioners can determine the true nature of these needles from the feeling during use; what these judgements were based on; and how confident they were in them.

Methods

Participants

We recruited 10 licensed and experienced acupuncturists (mean duration of acupuncture experience 10.8 (SD 12.9) years; mean age: 40.1 (SD 11.1) years; four men and six women) from the teaching or research staff of Japan School of Acupuncture, Moxibustion and Physiotherapy, The Educational Foundation Hanada Gakuen, Tokyo, Japan (table 1). Before the study, its purpose and format were explained and the participants provided written consent. The Showa University Ethics Committee gave its approval.

Table 1

Acupuncturist's judgements on 30 needles (10 of each type)

Design of double-blind needles

We used three types of needles in this study. (a) the non-penetrating placebo needle, the tip of which presses against the skin but cannot penetrate it (‘skin-touch placebo needle’); (b) the matching ‘penetrating needle’ with 10 mm insertion depth. These have been described in detail elsewhere.11,,14 Additionally, in this study, we used (c) the newly developed needle (‘non-touch control needle’), the tip of which cannot reach the skin. The appearance of these three needles is indistinguishable (figure 1). The diameter of the needles was 0.16 mm.

Figure 1

Double-blind needles. (A) Means of needle insertion. (B) The non-touch control (left), skin-touch placebo (middle) and penetrating needles (right) comprise an opaque guide tube and upper stuffing to give resistance to the needle body during its passage through the guide tube. The needle body of the non-touch control needle is shorter than the guide tube, and the needle body of the skin-touch placebo needle is just long enough to allow its blunt tip to press against the skin, but the needle body of the penetrating needle is longer than the guide tube by an amount equal to the insertion depth when the needle body is advanced as far as possible. The non-touch control and skin-touch placebo needles contain lower stuffing to give a similar sensation to that of skin puncture and tissue penetration. Each needle has a stopper, which prevents the needle handle from advancing further when the tip of the needle reaches the specified position. The pedestal on each needle is adhesive, allowing it to stick firmly to the skin surface. The left and right needle for each type of needle is before and after needle insertion, respectively.

Validation test for practitioner blinding

For each practitioner we prepared 10 of each type of needle and shuffled them to achieve a random order.

Practitioners were informed about the design of the needles, and instructed to apply the needles as if treating a stiff neck using any appropriate acupuncture or tender points they wished. The acupuncturists applied each needle separately, taken randomly from the shuffled set of 30 needles, into the shoulder of the author (NT), 15 on each side, using the ‘alternating twirling technique’—that is, insertion by rapid rotation of the needle. The acupuncturist then advanced the needle until the stopper made contact with the top of the guide tube and finally withdrew it (figure 1), while observing the reactions of the patient. No further needle manipulation was used in this study, reflecting one of the techniques most commonly used by practitioners in Japan. The acupuncturist then removed the entire needle assembly from the skin. An assistant, blind to the true nature of needles and the practitioner's judgements, numbered the needle and sealed it in an opaque envelope. The practitioner then reported his/her judgement as to whether the needle was non-touch control, skin-touch placebo, penetrating needle or unidentifiable. Confidence in making this judgement (ie, the degree of certainty about the decision) was then rated by the practitioner on a visual analogue scale, the end points of which were 0 for no confidence and 100 for complete confidence. Clues that led to the judgement of identity of the needle were then reported by the practitioner; the options were ‘feeling of needle insertion’, ‘feeling of needle removal’, ‘feeling in the left hand holding the guide tube (‘Oshide’ in Japanese)’ (figure 1), ‘body movement’, ‘bleeding’ and ‘lack of bleeding’.

The author (NT), blinded to the practitioners' responses, subsequently opened the opaque envelopes to record the type of needles. We took all possible precautions to ensure that the identity of the needle was not revealed to the practitioners, the patient or the investigators during the acupuncture trials.

Data analysis

A χ2 test was used to determine whether the numbers of needles fitted an expected probability. Practitioners' confidence scores for correct and incorrect identifications were compared using the Mann–Whitney U test; comparisons between three groups were made using Kruskal–Wallis test. Spearman's rank correlation coefficient was used to indicate the relationship between the years of practitioners' experience for acupuncture practice and number of correctly identified needles. All statistical analyses were performed using SPSS, version 15.0J (SPSS, Chicago, Illinois, USA).

Results

All 10 acupuncture practitioners completed all judgements for all needles. The mean of the practitioners' overall confidence in making their judgements was 44.2 (SD 28.0)%. Of the 300 needles applied, the practitioners identified 104 (34.7%) correctly (non-touch control=34, skin-touch placebo=23, penetrating=47); 152 (50.7%) incorrectly (non-touch control=49, skin-touch placebo=64, penetrating=39); and reported 44 (14.7%) as unidentifiable (non-touch control=17, skin-touch placebo=13, penetrating=14), see table 1. The 104 correctly identified needles overall fitted the probability of one in three (χ2=0.24, p=0.62). There was no significant difference in the confidence scores for correct (mean 54.0 (SD 20.2)%) and incorrect (mean 50.3 (SD 24.3)%) judgements (p=0.09). Practitioners stated they were 100% confident in 12 judgements: three were correctly identified (one non-touch control and two penetrating needles), seven were incorrectly identified as penetrating (one non-touch control and six skin-touch placebo needles) and two were incorrectly identified as non-touch control (skin-touch placebo needles). For each of the non-touch control and skin-touch placebo needles, the proportion of correctly identified needles fitted the probability of one in three (χ2=2.18, p=0.14; χ2=1.86, p=0.17). The 47 penetrating needles that were correctly identified exceeded the chance probability of one in three (χ2=17.59, p<0.01) (table 1). However, there was no significant difference in the confidence scores for different types of needle (p=0.69) (figure 2).

Figure 2

Confidence level in the practitioners' judgements of ‘non-touch control’, ‘skin-touch placebo’ and ‘penetrating’ for non-touch control, skin-touch placebo and penetrating needles. The top, middle and bottom lines of the boxes correspond to the 75th, 50th (median) and 25th centiles, respectively. The whiskers extend from the 10th centile to the 90th centile. The circles indicate the arithmetic mean. Note: NS indicates that there was no significant difference.

Practitioners were more likely to judge the needles as ‘penetrating’ (119 out of 256 needles identified, χ2=19.9, p<0.01). However, 47 correct identifications of the 119 needles fitted the probability one in three (χ2=2.03, p=0.15) as did 23 correct identifications of 52 needles judged as skin-touch placebo (χ2=2.78, p=0.10) and 34 correct identifications of 85 needles judged as non-touch control (χ2=1.70, p=0.19). The mean confidence score for 47 correctly identified penetrating needles was 55.9 (SD 21.4)% which was higher, though not significantly (p=0.17), than 48.5 (SD 17.1)% of the correctly identified skin-touch placebo and 55.2 (SD 20.4)% of the correctly identified non-touch control needles (figure 2). Importantly, there was no significant difference in confidence between non-touch control, skin-touch placebo and penetrating needles identified as penetrating (p=0.15); or for those that were identified as non-touch control (p=0.34) and skin-touch placebo (p=0.70) (figure 2).

The practitioners made their judgements on the nature of the needles principally on the ‘feeling of needle insertion’ and were far less dependent on other clues. The mean score for confidence of the practitioners in 69 needles correctly identified from ‘feeling of needle insertion’ was 57.6 (SD 19.0)% (table 2).

Table 2

Number and confidence score (mean±SD %) in practitioners' identification of needles identified from clues

The highest percentage of correct answers was 60.0% in the acupuncturist with 2.5 years of acupuncture experience, and the lowest was 13.3% in the acupuncturist with 1.5 years of acupuncture experience (table 1). There was no significant correlation between the years of experience in acupuncture and the numbers of correctly identified needles (r=0.42, p=0.23) (table 1).

Discussion

In previous validation studies we found that recently developed double-blind needles have the potential to mask practitioners.11,,14 But there still remained a question about whether practitioners were sometimes certain of their judgement about a needle, which could jeopardise blinding. Practitioners in this study judged a great majority of correctly identified needles with uncertainty, which indicates that the identity of the needles was well blinded from the practitioners. However, practitioners in this study had a tendency to judge needles to be penetrating compared with the previous studies.11,,14 Further, a few (<3%) correctly identified needles were reported with 100% confidence. These results suggest a potential limitation in the success of perfect practitioner blinding in clinical acupuncture studies. Therefore, in future, confidence in the practitioner's guesses should be recorded to see whether the true identity of the needle is revealed.

The 60% and 50% correct identification obtained by practitioners No 5 and No 10 were striking scores, as was the lowest figure, 13% by practitioner No 3. However, the success rates of these three seeming outliers were within 2 SDs (±12.9) of the mean success rate of 34.7. Thus statistically the three practitioners did not perform much better or worse than the others. There was no significant correlation between the years of experience of acupuncture practice and the number of correctly identified needles. Duration of experience does not seem to affect judgement. In our previous validation studies, acupuncture points to which the needles were applied were located in the upper extremity.11,,14 In this study acupuncturists applied the needles to the shoulder, to replicate a clinical setting. However, the double-blind needle still needs to be validated in a true clinical trial, and by other centres.

Although a non-penetrating needle that presses the skin may be physiologically active, we call our non-penetrating needle ‘placebo’ because it lacks a key ingredient of acupuncture: skin penetration. At present we are not aware of any conclusive evidence as to whether placebo, sham or another type of needle has a specific therapeutic effect on any medical condition obtained under double-blind conditions using an appropriate control. The new, non-touch needle designed for this study provides no physiological stimulation by the needle tip, and our results suggest that it would be suitable for practitioner blinding. It would be important to compare the effect of these two needles on the symptoms of a clinical condition, though it may be difficult to mask patients. Thus, the explanation given to patients will be the next challenge in adopting such non-touch control needles in research.

Conclusion

In conclusion, this study suggests that unblinding is not likely with this placebo needle, since practitioners were not certain about a large majority of their correct identifications. However, it showed that achievement of perfect practitioner blinding was limited because practitioners had a tendency to guess needles to be penetrating, and because the true identities of some needles may be revealed.

Summary points

  • ▶. We previously reported a new placebo needle which blinds the practitioner.

  • ▶. Here we compared it with a genuine needle and another form of placebo needle.

  • ▶. Practitioners were more likely to guess the ‘real needle’.

  • ▶. But they were not confident in saying that, so probably remain correctly blinded.

Acknowledgments

The authors thank Ikuo Homma (Second Department of Physiology, Showa University School of Medicine, Tokyo) for his support. The authors also express their appreciation to all the participants of this study.

References

View Abstract

Footnotes

  • Funding The work of TJK was supported by NCCAM-NIH grant # K24 AT004095.

  • Competing interests NT and The Educational Foundation of Hanada Gakuen possess a US patent 6575992B1, a Canadian patent CA 2339223, a Korean patent 0478177, a Taiwan patent 150135, a Chinese patent ZL00800894.9 (Title: Safe needle, placebo needle, and needle set for double blind) and a Japanese patent 4061397 (Title: Placebo needle, and needle set for double-blinding) on the needles described in this manuscript. NT is a salaried employee of The Educational Foundation of Hanada Gakuen and has received research funding from The Educational Foundation of Hanada Gakuen.

  • Patient consent Obtained.

  • Ethics approval This study was conducted with the approval of the Showa University School of Medicine.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.