Objective To determine if acupuncture-exposed and naïve participants differ in their perceptions of real and sham acupuncture under blinded conditions.
Methods The setting was an outpatient clinic at the Colorado School of Traditional Chinese Medicine. Participants were between the ages of 18 and 90 years. Acupuncture-exposed participants had at least five prior acupuncture treatments, with one treatment in the month prior to the study date. Acupuncture-naïve participants had experienced no prior acupuncture treatments. Participants with dementia, cognitive impairment, or neuropathy were excluded. In total, 61 acupuncture-exposed and 59 acupuncture-naïve participants were blindfolded and received either real acupuncture or toothpick sham acupuncture treatment. Following treatment, participants completed a questionnaire rating the realness of the acupuncture and were asked how they made this determination. We used a previously developed scale rating treatments from 1 (definitely real needle) to 5 (definitely imitation needle) to assess outcome.
Results Perceptions of the real treatment were rated as more real than sham treatments for all participants. Further analysis revealed that prior acupuncture exposure did not influence ratings of real treatments, but exposed participants rated sham treatments as significantly less real than naïve participants.
Conclusions Acupuncture-naïve and exposed participants both reported different perceptions of real and sham acupuncture using a blindfolded toothpick protocol. This suggests that future trials should carefully monitor participant perceptions of treatments received, even for naïve individuals. Differences between groups further suggest that participants with significant and/or recent exposure to real acupuncture may introduce bias to blinded clinical acupuncture trials.
- Statistics & Research Methods
Statistics from Altmetric.com
There has been widespread and increasing use of acupuncture to treat many medical conditions in Western societies.1 ,2 This has fuelled a growing interest in developing well-designed research trials to assess the efficacy of acupuncture. Blinded randomised controlled acupuncture trials have been designed using an array of control groups, such as standard care,3 waitlists,4 acupuncture performed off of channels or known therapeutic points,5 and non-invasive sham acupuncture treatments, including sham needle devices6–8 or toothpicks.9 The use of non-invasive sham acupuncture is generally preferred to invasive sham acupuncture at non-acupuncture points because of the potential for therapeutic non-specific needling effects.5 ,10
When performing double-blinded trials of acupuncture, it is essential that blinding issues do not bias results and that the effectiveness of blinding is reported for the real and placebo groups.11 While it is acknowledged that prior acupuncture exposure may affect participants’ perception of research acupuncture protocols,12 there is a need for adequately powered empiric studies to guide researchers in their inclusion or exclusion of these participants. In the absence of guidelines of evidence, current studies are mixed in their inclusion of participants in trials that consider acupuncture exposure, ranging from exclusion of participants with any acupuncture exposure,13 inclusion of participants regardless of acupuncture exposure,14 or setting an arbitrary cut-off point (for example, no acupuncture in the past 12 months).4
The primary objective of the current study was to determine whether acupuncture-exposed and naïve participants differed in their perception of real or sham acupuncture treatments. This study used the blindfolded, toothpick sham acupuncture protocol developed by Sherman et al.9 The protocol is appealing because it is inexpensive, reproducible and easy to learn. On the basis of initial piloting and conduct of this protocol for a randomised controlled trial for fatigue in Parkinson's disease, we hypothesised that neither the acupuncture-exposed nor the naïve participants would be able to distinguish between real acupuncture and the toothpick sham acupuncture. A secondary objective of this study was to assess what strategies participants used to make their determinations with the goal of improving future sham methods.
Setting, acupuncturists and training
Two US-trained and licensed acupuncturists, with an average of 11 years of experience, provided real acupuncture and toothpick sham acupuncture sessions at the clinic of the Colorado School of Traditional Chinese Medicine. Prior to the current study, the acupuncturists were trained on the toothpick and real protocol to maintain consistency of needling techniques and point location for an ongoing randomised controlled trial of acupuncture for fatigue in Parkinson's disease. This included several pilot sessions to ensure consistency of insertion force, insertion depth, manipulation techniques and point location for the real and sham acupuncture treatments conducted on study personnel, as well as acupuncture-exposed and naïve volunteers until there was no consistently reported differences for either real or sham treatments.
Participants and preparation
The study was approved by the Colorado Multiple Institutional Review Board and informed consent was obtained from all eligible participants. Sham acupuncture was explained to participants as ‘non-true acupuncture’ that differs from acupuncture by using non-traditional point locations or not inserting needles at all. Participants age 18–90 were recruited by advertisement. Inclusion criteria for the acupuncture-exposed group included at least five prior acupuncture treatments and at least one acupuncture treatment within 1 month of the study date. Acupuncture-naïve participants included those with no previous acupuncture treatments. Exclusion criteria for both groups included dementia, cognitive impairment, or neuropathy as these conditions were expected to interfere with the ability to perceive tactile stimulation or report ratings.
A total of 61 acupuncture-exposed and 59 acupuncture-naïve participants were recruited for this cross-sectional study and randomised to receive either a single session of real or sham acupuncture in a pseudorandom and balanced fashion. The study was performed over 4 days, with 2 days each assigned for real or sham acupuncture to aid acupuncturists in consistency of their treatments. Acupuncturists were blinded to the participant's acupuncture exposure status.
Each treatment room was supplied with two opaque containers, one with acupuncture needles and one with toothpicks in 25.4 mm (1-inch) needle guide tubes. Additional items in each room included a covered treatment table, alcohol swabs, facial tissue, blindfold and infrared heat lamp. Participants were instructed to wear loose clothing, eat a snack before arrival and refrain from wearing lotion.
Real acupuncture protocol
The participant was instructed to roll up their sleeves above the elbows and trouser legs above the knees and lie face up on the table. Participants were then blindfolded. All acupuncture points were swabbed with alcohol. Acupuncture needles (Seirin, 36 gauge, 25.4 mm (1-inch)) were inserted to a depth of 6.35 mm (one-quarter of an inch) in the following order: GV20, GV24, CV6 (midline points), right LI10, right HT7 (upper extremity points), right ST36, right SP6 (lower extremity points), left LI10, left HT7 (upper extremity points), left ST36 and left SP6 (lower extremity points). For each point insertion, the needle was twisted three times to the right. These points were identical to those of the Parkinson's disease study and were chosen to treat fatigue in this population. The depth of 6.35 mm was chosen to maintain a standard sensation across points and practitioners.
After 15 min, the acupuncturist returned to the room to check on the participant. After 30 min, the acupuncturist removed the needles in the same order as they were inserted and swabbed each point with alcohol after removal, dropping needles into a sharps container after three removals.
The toothpick protocol used sham acupuncture points, located 12.7 mm (half an inch) lateral to the traditional acupuncture point, or to the right for midline points. After the participant rolled up their sleeves and trousers and was blindfolded, each point was swabbed with alcohol. Then, a toothpick in a guide tube was placed on the sham point and tapped on the skin. The guide tube was removed and the toothpick was twisted three times to the right. Then, the toothpick and guide tube were discarded into a waste container located in the treatment room and covered with tissues. The toothpicks were mock inserted the same order as the real acupuncture protocol.
After 15 min, the acupuncturist checked on the participant. After 30 min, the acupuncturist mock removed the toothpicks by slowly pressing a new toothpick into the sham point and quickly removing it. This was performed in the same order as they were mock inserted. The toothpicks were discarded three at a time into a sharps container to mimic the sounds heard in a real acupuncture treatment.
Post session questionnaire
Following each session, the participants completed a questionnaire rating their impression of the realness of the acupuncture received. We used the five-point rating scale from Sherman et al,9 which rates an acupuncture session as 1=definitely real needle, 2=probably real needle, 3=uncertain, 4=probably imitation needle and 5=definitely imitation needle. Subjects were also asked an open-ended question of how they made their determination. These answers were later classified into common themes and coded.
Statistical analysis was performed using SAS V.9.3 (SAS Inc, Cary, North Carolina, USA). All data was checked for outliers, distributions and missing values. Wilcoxon ranked sum tests were used to make group comparisons for imitation ratings (ordinal data) and Student's t test was used to make group comparisons for numeric data. χ2 tests (or Fisher's exact test if any category had less than five observations) were used to check for differences between categorical variables. Spearman's correlation was used for bivariate correlations. p Values<0.05 were considered significant.
We enrolled 120 subjects, 61 with prior acupuncture exposure. Table 1 summarises demographic information for all participants.
Table 2 summarises results of realness ratings from our post-treatment questionnaire over all participants and divided by prior acupuncture exposure. We found a significant difference in mean ratings over all subjects for the real (2.0±1.0) and sham (3.9±1.2) protocols (z=7.4, p<0.0001). This pattern of results was similar when examined within the acupuncture-naïve group (real (2.0±1.2) vs sham (3.2±1.1); z=3.9, p<0.0001) and within the acupuncture-exposed group (real (1.9±1.0) vs sham (4.6±0.6); z=−6.3, p<0.0001).
As groups differed on age and had a trend for difference on gender we checked to see if age or gender could have biased our results. When divided by real and sham treatments, age did not correlate with ratings (real; r=−0.08, p=0.5 and sham r=0.12, p=0.4) and ratings did not differ by gender for real (men 2.0±1.1 vs women 1.9±1.0; z=0.14, p=0.89) or sham (men 3.9±1.0 vs women 4.0±1.2; z=−0.77, p=0.44) treatments. Since there was no difference on gender, we did not proceed with adjusted models.
Looking at ratings by group assignment, overall ratings statistically differed for exposed (3.3±1.6) vs naïve (2.5±1.2) participants (z=−2.6, p=0.01). For those who received real treatments, there was no statistical difference in the distribution of ratings between naïve (2.0±0.9) and experienced (1.90±1.0) participants (z=−0.78, p=0.44). However, for those who received sham treatments there was a statistical difference between the rating distribution of naïve (3.2±1.1) and experienced (4.6±0.7) participants (z=−5.0, p<0.0001). Accuracy of ratings (rating real as ‘definitely real’ or sham as ‘definitely sham’) also differed between exposed (54%) and naïve (22%) participants (χ2, p<0.0003). This difference was seen with sham treatments (experienced (68%) and naïve (11%); χ2, p<0.0001) but not with real treatments (exposed (40%) vs naïve (31%); χ2, p=0.5).
Participants reported several common strategies for making their ratings including perceptions of needle insertion, sensations of de qi (energy), feelings of relaxation during the treatment, expectations of the treatment, pain, acupuncturist error and guessing. The only significant differences between exposed and naïve subjects were an increased proportion of naïve subjects guessing, failing to respond, or attributing results on acupuncturist error. Table 3 describes strategies the participants used to distinguish real and sham treatments. Table 4 shows results based on accuracy and reveal no significant differences in strategy between accurate and inaccurate responders.
This study investigated whether prior acupuncture exposure affected participants’ perception of real or sham acupuncture in blinded, controlled trials using Sherman et al's9 blindfolded toothpick method. We found that exposed and naïve groups overall both rated sham acupuncture as less real than real acupuncture. Exposed participants also differed from naïve participants in their overall ratings, but only for the sham procedure. It should be emphasised that these results apply only to the sham protocol studied and may not be generalised to other sham procedures.
This study elaborated on the blindfolded toothpick method from Sherman et al.9 While the Sherman study evaluated their participants’ perception of treatment they received, they did not perform the protocol on acupuncture-exposed participants and may have been underpowered (N=52) to evaluate small to moderate effect sizes. To the best of our knowledge, ours is the largest study to assess whether prior acupuncture exposure affects blindedness in acupuncture trials and to evaluate how participants made determinations of which treatment group they were assigned.
Our results suggest that future clinical trials of acupuncture should carefully consider the inclusion of acupuncture-exposed individuals, and should assess participant perceptions of what treatment they are receiving regardless of prior exposure status. Currently, there is no consensus on whether to include those with prior acupuncture exposure in clinical trials of acupuncture with trials ranging from exclusion of all acupuncture-exposed individuals,9 ,13 inclusion of individuals regardless of acupuncture exposure,14 or setting an arbitrary cut-off point.4 While exposed participants may introduce bias by their experience with acupuncture treatments, excluding these individuals may also limit recruitment efforts in areas where acupuncture is prevalent and exclude those participants most interested or likely to benefit from acupuncture research.
It is still unclear how participants assess their treatment assignment. We found no significant associations between self-reported strategies and accuracy, and very small differences even between acupuncture-exposed and naïve individuals. Notably, this study differed from our pilot work in that participants’ received full treatments rather than simply rating the sensations of being needled and ‘needle’ removal. This suggests that there may be subtle and potentially unconscious factors leading to participants’ decisions that occur during the treatment period. A previous study15 found that participants who received real acupuncture for acute pain relief after tooth extraction were more accurate in their determination than the sham group because they attributed a reduction in pain to a real acupuncture treatment suggesting that participants may make a post hoc decision based upon how they feel after the treatment.
In a meta-analysis of blinding effectiveness across acupuncture randomised, controlled trials there was less of a clear bias in the sham group than seen in our study.11 This may reflect two somewhat artificial aspects of our study: first, that we provided only a single treatment and perceptions of acupuncture may change over a full course of treatment and; second that participants were aware that this was a study of acupuncture research methods and thus may have been more focused on the treatment itself rather than a clinical condition. It is also possible that our practitioners, who given the demands of this protocol were necessarily not blinded, may have introduced unconscious performance bias. The same meta-analysis of blinding effectiveness across acupuncture randomised controlled trials does support the potential influence of blinding research staff to reduce bias.11
This study did not address acupuncture-exposed participants with more remote acupuncture exposure, that is, no acupuncture in the past 12 months. It is possible that these individuals may be less sensitive to differences in real and sham acupuncture than individuals with more recent exposure. However, empiric studies are needed to determine how recentness or total number of treatments influence acupuncture perception. Also, this study did not evaluate other sham acupuncture needles, such as Park et al,6 Streitberger and Kleinhenz,7 or Takakura et al,8 or invasive sham acupuncture protocols. Further research is needed to determine if other sham acupuncture needles or inclusion criteria are more effective at eliminating bias than the toothpick method and exclusion of any acupuncture exposure. Notably, Takakura et al's8 placebo needle has good evidence to support its ability to blind experienced acupuncture participants and even the treating acupuncturists.
Research on how prior acupuncture exposure affects blindedness in controlled trials of acupuncture is crucial for future trial design. It is recommended that all blinded, controlled trials of acupuncture address this issue in inclusion and exclusion criteria and evaluate participants’ perceptions of their treatment for real and placebo groups to improve the quality of acupuncture research.
We tested whether prior experience of acupuncture renders sham interventions to be perceived as less real, risking unblinding in clinical trials.
Overall, participants rated a toothpick sham less real than acupuncture.
Those with prior experience of acupuncture also rated the sham as less real than acupuncture naïve participants.
The authors would like to thank Anh Tran, Rebecca Wilson, Erika Holmes and Matthew Kester for their assistance on this project and the manuscript.
Contributors BMK contributed to conception and design, data analysis, drafted and revised the paper, and is the guarantor. MKC, NN and JB contributed to design, acquisition and final approval of the paper. CRD contributed to drafting, data acquisition and revisions. RC contributed to data acquisition and manuscript revision. RPB contributed to data analysis, drafting and revisions.
Funding This work was supported by the Michael J Fox Foundation, the National Institutes of Health/Colorado Clinical and Translational Sciences Institute KL2 program (NIH/NCATS 8 KL2 TR000156-05) and the Colorado School of Traditional Chinese Medicine.
Competing interests None.
Patient consent Obtained.
Ethics approval Colorado Multiple Institutional Review Board.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.