Background In trials, ‘therapist intensive’ complex interventions are typically delivered over time, during which a relationship between the practitioner and participant may develop. Such relationships are sometimes criticised as obscuring any ‘true’ treatment effect. Limiting interactions is one strategy that might be used to try to control for the effect of a therapeutic relationship.
Objectives We conducted systematic review into the rationale, methods and effects of constraining relationships in controlled trials and cohort studies of acupuncture, including studies published before 2008 with an update citation search in 2010.
Methods We searched six databases without keyword restrictions. Meta-analysis and meta-regression were used to explore the effect of relationship constraint on pain outcomes.
Results Eighty-one of 785 (10.3%) trials reported constraining relationships. Most did not state the reason for constraint, describe the nature of the limitation, provide information on how the constrained relationship was monitored or note protocol adherence. Where a reason was reported, this was primarily to maintain participant blinding, rarely was it stated that the constraint was to control the therapeutic relationship. We found no evidence of an effect of constraint on pain outcomes (percentage heterogeneity explained, p=0.89). These results were robust to variation in trial quality and design.
Conclusions Acupuncture trials appear to be constrained mostly to try to prevent participant unblinding to their allocated treatment, not to control the therapeutic relationship. The apparent lack of monitoring and negligible effects on pain outcomes of the included trials indicate the need for more high-quality randomised controlled trials investigating the effect of constraint.
- Systematic Reviews
Statistics from Altmetric.com
Delivery of ‘therapist intensive’ complex interventions such as acupuncture in trials is typically through a course of treatment rather than a single visit and over time a relationship between the practitioner and patient might be expected to develop. This relationship may be particularly strong in complementary medicine, particularly if a diagnosis can be established when conventional diagnostics previously failed or the practitioner provides an overview that explains and provides guidance through temporary symptom exacerbation.1 There is increasing interest in the effect of patient–practitioner relationships on outcomes in all areas of medicine, and particularly in patients with chronic conditions. Reviewers have mostly found positive consistent effects on outcomes in trials that provide cognitive and emotional enhancement2 ,3 or improve doctor–patient communication,4 independent of any treatment given. However in trials such interactions are often considered to be ‘noise’ that obscures the ‘true’ effect of the treatment under study. In acupuncture trials, for example, relationship factors are the focus for debate about which aspects of an acupuncture treatment are ‘specific’ to acupuncture and therefore should be delivered in trials and which are incidental to acupuncture and might potentially contribute to ‘placebo’ effects.5
Limiting the interaction between participants and the practitioner is one potential strategy that might be used to investigate the effect of relationships in clinical trials. The effect on outcomes could then either be compared within a trial if multiple relationship states were seen, or between trials in a pooled analysis. To the authors knowledge there has been no systematic review into the effect of constrained relationships in complex interventions that involve unblinded practitioners (the reviews cited above examined enhanced as opposed to restricted relationships). In this study we aimed to describe the use of constrained relationships in clinical studies of acupuncture and examine their effect on pain outcomes.
This section contains a précis of the methods. A complete description can be found in online supplementary appendix 1.
In November 2007 (supplemental search in 2010) we searched for published reports of randomised controlled trials (RCTs), quasi-randomised trials (controlled clinical trials) and controlled cohort studies of acupuncture across six databases. We included studies that were published in English and used acupuncture to treat a medical or psychological condition in adults. Two people read each paper and categorised studies as either constrained (any mention that communication between a practitioner and a conscious patient was constrained), or unconstrained (no mention of a constraint).
We extracted general information about the study, quality markers6 and details of the relationship constraint. Studies were divided into two categories: RCTs that used acupuncture to treat chronic musculoskeletal or neuropathic pain, and all other studies. We defined chronic (≥2 weeks’ duration) musculoskeletal or neuropathic pain as that associated with arthritic conditions, muscle pain (including tension-type headache),7 joint pain or fibromyalgia, any neuropathy or chronic postoperative pain, or pain of unknown origin.
We compared the characteristics of the constrained and unconstrained studies and described the relationship constraints.
To explore the effect of relationship constraint on outcomes, we conducted a meta-analysis on the outcomes of the subset of chronic pain RCTs. To further reduce heterogeneity (variability seen in the effects of each study),8 we included only those trials that reported a continuous pain measure after the end of the treatment and standardised the unit of outcome measurement into the standardised mean difference. RCTs that reported a relationship constraint were included in the meta-analysis as ‘constrained’; those that fulfilled the criteria but did not report a relationship constraint were included as ‘comparators’.
We conducted a random-effects meta-analysis on these trials,9 ,10 which produced both an estimate of overall average effect on pain, and a measure of heterogeneity (I2). A point value of I2 of ≤25% was considered to be low, 20–75% moderate and ≥75% high.11 We used the 95% confidence interval (CI) to interpret the precision of I2.8 ,12 We examined whether constrained trials reported different effects on pain outcomes by regressing the binary covariate (a factor common to the design of a subgroup of trials) of constrained/comparator onto the results of meta-analysis13 ,14 and noted whether heterogeneity changed. The change in I2 was measured in the following ways: the proportion of heterogeneity explained by the covariate (R2Adj) (the relative reduction in between-study variance),13 and the proportion of heterogeneity remaining after adjustment with the covariate (I2Res). As an interpretative note, it is possible for R2Adj to be negative, which indicates that the proportion explained is no greater than would be expected by chance.13 We also reported the regression coefficient and two-tailed p value (α=0.05). Owing to the moderate and high levels of heterogeneity present we reported prediction intervals to indicate the range of possible summary effect and not CI for the estimation of variance around the standardised mean difference.15
To explore the effect of different trial designs and as a marker of trial quality, we repeated the meta-regression also including the binary covariates of whether the trial was controlled by sham acupuncture, and the adequacy or otherwise of allocation concealment.6 The metan,16 heterogi17 and metareg13 modules of STATA V.12 (StataCorp LP, Texas, USA) were used to generate the meta-analysis and meta-regression.
A targeted citation search for additional publications associated with trials identified as constraining relationships identified one further report18 of a study located in the original search as an abstract.19
Eighty-one19–93 of 785 trials were identified as having a practitioner–participant relationship constraint—a prevalence of 10.3% (figure 1). Of these, 3120–22 ,27 ,28 ,32 ,37 ,39 ,42 ,47 ,50 ,51 ,58–61 ,63 ,65 ,67 ,72 ,78 ,80 ,86 ,88 ,89 ,93 of the 192 randomised trials of musculoskeletal or neuropathic pain (16.1%) were constrained. Four papers contained reports of multiple studies; three papers90 ,91 ,93 reported two trials each and one92 reported four trials. Several papers reported their findings in more than one paper.
Constrained trials were less likely to be open label, and more likely to have adequate allocation concealment and blinded outcome observers, and constrained sham-controlled trials were more likely to tell participants that there was a sham involved than non-constrained (table 1). Constrained trials were more likely to report who the treatment provider was and that provider was more likely to be an acupuncturist (vs physician, physiotherapist, other) than in unconstrained trials. Characteristics by trial are tabled in online supplementary appendix 2.
Fourteen41 ,44 ,45 ,48 ,49 ,56 ,57 ,66 ,76 ,77 ,84 ,87 ,90 of 81 constrained trials (17.3%) gave the treatments in a group setting compared with only one of the unconstrained RCTs of chronic pain. All constrained trials in a group setting investigated the role of acupuncture in treating substance abuse. It was unclear whether the large number of constrained relationships reported in these settings reflected the group setting common in this clinical situation or whether investigators seized on the norm of a quiet group environment to proscribe any communication.
Reasons for constrained relationships
The most common stated reason for constraining relationships was to maintain participant blinding24 ,25 ,28 ,32 ,34 ,40 ,42 ,45 ,47 ,58 ,59 ,65 ,70 ,75 ,82 ,85 ,93 (22.2% all constrained studies, 25.8% of chronic pain constrained studies) (table 2). Stating that the constraint was to reduce any effect of the therapeutic relationship was only rarely23 ,33 ,37 ,67 ,78 (6.2–6.5%) given as a reason and 2.5–6.5% of trials maintained it was done for both reasons.29 ,74 Most studies (61.3–69.1%) did not state any reason,19–22 ,26 ,27 ,30 ,31 ,35 ,36 ,38 ,39 ,41 ,43 ,44 ,46 ,48–53 55–57 60–64 ,66 ,68 ,69 ,71–73 ,76 ,77 ,79 ,80 ,83 ,84 ,86 ,87 ,89–92 although five mentioned bias reduction.54 ,62 ,64 ,76 ,81 One small factorial trial86 ,94 set up a constrained and an enhanced relationship contrast. The authors wished to examine the effect of negative and positive relationships; this was the only trial to do so in this review.
Descriptions of constrained relationships
The type of relationships permitted was not reported in 38.7–51.9% of trials19 ,20 ,24 ,29 ,30 ,33 ,34 ,36 ,40 ,41 ,43 ,44 ,46 ,48 ,49 ,52 ,54 ,56–58 60–62 ,64 ,65 ,67 ,69–71 ,78 ,82 ,87 ,90–93 (table 3). The most commonly reported type of interaction permitted was minimal talking22 ,23 ,28 ,32 ,38 ,50 ,68 ,75 ,81 ,95 (12.3–12.9%), followed by not talking about the treatment27 ,39 ,47 ,63 (4.9–12.5%) and silence59 ,80 ,83 ,85 ,89 (6.2–12.9%). Only one trial reported permitting casual talking.21 Other situations were reported by 19.4–23.5% of studies. These included answering questions,66 ,77 only talking about the treatment,37 ,73 conveying ‘essential’ information76 ,84 (not further described), not talking about the effects of treatment,51 not talking about the treatment or effects55 and not talking about the participant's condition.72 The one trial to incorporate two relationship states reported setting up the enhanced arm as being a supportive, warm and friendly relationship, setting up and enhancing positive expectations for treatment success.86 ,94 In contrast, those in the constrained arm were set up with negative or neutral expectations, and the practitioner stifled any social relationship. Fourteen of the 20 constrained substance abuse trials gave descriptions that made it unclear whether the limited interaction was a consequence of the setting or a deliberate attempt to constrain relationships—for example, ‘Interaction of the acupuncturist and the patient was limited to the time required for placement of the needles’ (Worner et al,56 p170).
Monitoring and fidelity
Few trials (6.5–12.3%) reported checking adherence to interaction protocols (table 4). Most who did report on the method of monitoring used an observer22 ,45 ,59 ,66 ,74 ,76 ,77 (6.5–8.6, but only one trial reported definitively that this person was blinded to treatment allocation76 and it was unclear in another whether the observer was there to monitor the interaction.74 Three trials reported that an observer monitored all sessions,59 ,74 ,76 one monitored at monthly intervals22 and it was unclear whether all sessions were observed for three other trials.45 ,77 ,96 Of the three trials (0–3.7%) which audiotaped sessions, two audited a random sample29 ,68 and it was unclear whether another selected the tapes at random.23
Only two trials (2.5%, 3.2%) reported the results of fidelity. One22 ,97 reported that the services of three acupuncturists were discontinued for failing to adhere to the protocol, although it was not clear whether any of the violations were interaction-based. The other29 reported no significant violations on review of randomly selected audiotapes.
Debriefing of participants and practitioners
Four trials reported recounting the participant or practitioner experience of treatment interaction. One trial twice administered a therapeutic alliance scale and reported the results on that scale.83 A second small trial, in which two relationship conditions were played out in a factorial design (verum, sham), asked participants to fill out four scales rating the physician on this enthusiasm, empathy, competence and friendliness.86 ,94 The sham group found the practitioner friendlier than the verum group, but there was no difference in ratings between the relationship groups. In a third sham-controlled trial,18 ,19 in which the sole acupuncturist had to treat both groups similarly, 10/30 participants were interviewed one or more times and the acupuncturist once. Participants reported being conscious of participating in an experiment and commonly restricted discussion with the practitioner, although this was not required. The acupuncturist was described as ‘quiet’ but friendly. The acupuncturist, for her part, was anxious that any responses to questions did not give away group allocation and to avoid this tended to shut down communication as the treatments began (Paterson et al,18 p 204).
Another trial,22 ,97 which limited discussions to those necessary to evaluate symptoms and change, reported debriefing 8/12 practitioners through open-ended questions given via a survey at the end of the trial. Although two questions related to interference in relationships associated with the prescribed manualised treatments and sham needles, practitioners were not directly asked about the influence of the reduced interaction demanded by the protocol and the authors did not report this as a theme.
Effect on outcomes
One hundred and eleven trials of chronic pain met the criteria for the meta-regression; 17 constrained20 ,21 ,27 ,32 ,37 ,39 ,42 ,47 ,51 ,59–61 ,63 ,65 ,67 ,72 ,78 and 94 unconstrained (see online supplementary appendix 3). Quality assessment showed that only 35.3% of constrained trials and 28.7% of unconstrained trials had adequate allocation concealment, and 82.4% and 55.3%, respectively, reported the outcome observer blinded. Sham controlled trials made up 88.2% of concealed trials and 61.7% of unconcealed. As would be expected in an analysis with a variety of different populations and treatment protocols, there was evidence of high heterogeneity (table 5 and see online supplementary appendix 3, figure A1). The covariate of constraint explained no more of the heterogeneity than would be expected by chance alone. Adding the covariates of control type (sham or non-sham acupuncture) and the adequacy of otherwise of allocation concealment did not change this estimate (R2Adj for all covariates=−3.5%, joint test for an effect p=0.78, coefficient of constraint −0.08, p=0.77).
To the authors’ knowledge this is the first systematic review to examine restricted relationships in healthcare that involves unblinded practitioners. One major finding was that most trials did not report why constraint was imposed. Constrained trials were more likely to be sham controlled and around a quarter of trials made reference to protecting participants from cues about group assignment from unblinded practitioners. This supposes that while acupuncturists would have a vested interest in ‘giving the game away’, and while lack of double blinding has been associated with increased outcome effects,98 there is a lack of studies providing evidence to suggest that unblinded practitioners deliberately or mistakenly unblind participants. Indeed, some methodologists caution against placing excessive weight on studies which do not blind investigators but focus more on the hazards of employing unblinded outcome observers.99 The process of how unblinded practitioners behave in sham-controlled trials needs to be further explored.
Around 10% of trials in this review aimed to suppress the therapeutic relationship in order to examine the intervention separately from the ‘placebo effect’ of treatment alliance. Arguments against constraining relationships in this way include disturbance to the process of change in a complex intervention that is dependent on a mutually understanding relationship, and because the alliance is a critical part of acupuncture.5 ,100 Some scholars have tried to promote the hypothesis that human relationships can enhance the well-being of patients through neuropsychoimmunological mechanisms over a wide range of treatments.101–103
Most trials in this review did not report the nature of the constraint, and those which did, reported a variety of experimentally imposed conditions, nearly all without accompanying justification. Of most concern, few trials monitored whether the constraint took place as planned and only four trials reported the results of monitoring.
Two acupuncture trials on the effect of restricting and enhancing relationships have been published since the completion of this review. A study on the effect of relationships for people with irritable bowel syndrome found that those receiving sham acupuncture with an ‘augmented’ friendly, empathic and confidence-building relationship with the acupuncturist had better outcomes than those who received the same sham treatment with a constrained relationship.104 Several covariates were found to be associated with response: prior trial experience, reclusiveness105 and extraversion106 in the augmented group, different practitioners despite all interactions being standardised and monitored,106 and constrained participants participating in a qualitative substudy, indicating the possibility that even the attention of a neutral interviewer can provide a therapeutic relationship.105 The second trial, of knee osteoarthritis, randomised participants to an acupuncturist who provided either a high expectation of success or neutral expectation, or to a waiting list, with those allocated to initial treatment further randomised to sham or verum acupuncture.107 Again, these trialists found that participants randomised to receive treatment from an acupuncturist with higher expectations did better on two out of the three primary outcome measures.
One study in this review with a similar set up to these two trials (two relationship states and an unconsenting placebo group) asked participants to rate the therapeutic alliance with the practitioner in four domains, but found no difference in ratings between groups. It is difficult to interpret this finding; the trial was small, which raises the possibility of a false-negative result, and the Likert measures used might have been relatively insensitive. However, as fidelity was not monitored it is also possible that the single practitioner did not consistently maintain the facade for the duration of the trial, weakening the difference between the enhanced and constrained conditions. Maintaining a constrained interaction over repeated visits could be difficult. Equally, participants might have put a positive spin on their own explanations for silence2 or they might have sought out a therapeutic relationship with other study staff.105
Attempts to improve practitioner communication may be less effective than those improving patients’ communication, perhaps owing to the difficulty of changing entrenched practitioner communication styles.3 Indeed, perhaps the difference between our results and those of the two more recent trials, and others that have found no effect of altering communication, is the difficulty of maintaining a prescribed relationship in the absence of monitoring fidelity. This review found that debriefing of acupuncturists or patients about their experience was extremely rare but, when carried out, offered useful insights into trial behaviour. Further knowledge about debriefing, unblinding and the impact of fidelity monitoring might be usefully gained from using mixed method research in trials.
Unlike others who have indicated an effect of enhanced communication in systematic reviews,2–4 the meta-regression used in this review did not discern any effect of constraint on pain outcomes, although there was substantial heterogeneity. This variation was not explained by general trial quality factors, indicating either clinical/methodological heterogeneity or, possibly, a lack of adherence to the constraint imposed. Lack of statistical power is a possibility, but it remains likely that stronger, competing factors account for the variability.
This review used established systematic review methods, searching a range of sources, developing a priori inclusion criteria and duplicating effort at each stage of the review process to reduce the potential for error and bias. There are, however, limitations. The context in which the constraint was applied in the original trial is lost when studies are ‘lumped’ together in systematic reviews of methods such as this.108 Thus our crude interpretation of effect only approximates an average, potentially missing the nuances of effects within individual trials. The update citation search in 2010 for additional papers that might provide more detail of the effects of relationship constraint was for papers that had already reported a constraint. Papers from authors who might have debriefed or otherwise expanded relationships but did not report constraint in the main trial publication would have been missed. Indeed, the extent to which constraint took place but was not reported is unknown. The finding that constraint in acupuncture treatment occurs more often in trials of substance abuse is tempered by incomplete reporting; it is difficult to know whether many of these trials imposed constraint over and above usual practice or simply reported the normative protocols of a group treatment setting common to the field. A major strength of the study is the hand-searching method used to identify constrained trials. We employed this process as such details would only rarely be indexed, and our technique allowed us to produce prevalence estimates with some confidence. The random-effects meta-regression technique we used is a robust method of quantifying moderators of average effects13 ,109 and, unlike other methods such as vote counting, adjusts for variation between studies.
To conclude, this review has highlighted a lack of application of theoretical underpinning for constraining practitioner–patient relationships in acupuncture trials amid a high prevalence of poor reporting. Trials that constrain relationships under the guise of bias reduction may be misleading if the theories that the relationship is a component part of acupuncture treatment are correct. We identified little discernible effect of constrained relationships, and further empirical evidence is needed before clear guidance can be provided about what constraint will achieve or how best to accomplish it. The results from two more recent trials using factorial designs of enhanced and constrained arms do not seem to concur with the substantive findings of this review, and we suggest that further studies of this design are carried out. After accumulation of additional data we recommend that this review is updated and the effect of relationships on outcomes expanded to include other clinical conditions and other therapist intensive treatment modalities.
The therapeutic relationship can affect outcome.
We investigated the effect of modifying it experimentally.
In RCTs of painful conditions, there was no evidence of any effect.
The authors thank Gillian Worthy, Alison Longridge, Laura Vanderbloemen and Ann Hopton who also contributed to the screening and general data extraction.
Review history and Supplementary material
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
Contributors SLP designed and analysed the study, which was advised upon by JB, SC and HM. JB also advised on, and contributed to, the screening and general data extraction.
Funding The Foundation for Traditional Chinese Medicine contributed financially to the project.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.