BY AARON HANWAY
Richard Feynman spoke the words above to a new generation of scientists graduating from the California Institute of Technology. In his address, Feynamen described Cargo Cults, indigenous tribes in the South Pacific that developed religious practices around planes bringing cargo. During World War 2, these tribes witnessed many planes landing on their Islands bringing a variety of goods, unlike anything they had seen. After the War, the tribes built replica airfields with landing strips and control towers, complete with bamboo headphones for the tribesmen stationed in the control tower. From the tribesmen's perspective, it seemed like logical thinking - in the attempt to bring back the planes and their magical cargo they replicated everything they witnessed when the planes first landed, but obviously they were missing something crucial. Taking the label, Feynman coined the term Cargo Cult Science to describe pseudo-scientific practices that have the semblance of being scientific but lead us to miss crucial variables and subsequently believe things that aren't true. His speech was a cautionary tale to the burgeoning scientists of Caltech on how easily we can be influenced by our irrationality and biases.
Is Psychotherapy a Cargo Cult Science?
We are no less victim to pseudo-scientific thinking in the field of psychotherapy than any other, in fact, with more than 240 competing theories developed since Freud (Corsini and Wedding, 1989), we may be more prone to it than others. But psychotherapy is no easy endeavour, for how does one continually and reliably measure the contents of someone else's mind, while correlating that measure to a variety of interventions and inputs, and at the same time, correct for countertransference that distorts your thoughts and emotions?
Wading into this quagmire, in an attempt to provide a standardized measurement across psychotherapy practises, are routine outcome measures (ROMs) such as the Outcome Rating Scale (ORS) and Session Rating Scale (SRS), developed by Barry Duncan and Scott Miller (Duncan and Miller, 2008), and the CORE system, developed by Pr. Michael Barkham (History, n.d.). But, support for the use of ROMs is divided.
Duncan and Miller (2008) state that ROMs lift therapy out of the realm of the subjective, by using an objective measure to improve therapy and subsequently improve the outcomes for clients:
“The basic principle behind this way of working is that our day-to-day clinical actions are guided by reliable, valid feedback about the factors that account for how people change in therapy...Monitoring the outcome and the fit of our services helps us know that when we are good, we are very good, and when we are bad, we can be even better" (Duncan & Miller, 2008)
However, others are cautious, stating that the promotion of ROMs is simply for that of economic and political aims to provide simple and quantitative results to an endeavour that is naturally complex and qualitative. Evans (2012) in an article for Canadian Psychology Journal writes:
"There are two levels to this problem. One is a quantitative issue of excessive focus on one particular stream of information from short self-report measures; the other issue is the more qualitative and philosophical danger of being pulled much more generally toward the quantitative, abstracted, reductive viewpoint and away from the relational and from the non-generalizable individuality of every single client and every single practitioner" (Evans, 2012, pg. 133)
These two statements sum up the argument nicely: on one side, the move towards outcome measures is a move towards a more objective and scientific process that helps psychotherapy come out of the shadow of Cargo Cult Science and subsequently improve outcomes for clients; on the other side, its a move towards an overly rationalistic space that ignores therapy's naturally individualistic and creative processes. For therapists, it would seem that they tend to support the latter, as research shows that therapists are resistant to using ROMs in their practice (Unsworth, Cowie and Green, 2012). For new and experienced therapists alike the question remains, should they use ROMs in their practice?
The Case Against ROMs
Like many students, I didn't pursue psychotherapy as a scientific endeavour, I pursued it in spite of that. I found my calling by being deeply moved when reading great philosophers and thinkers who wrote about the limits of reason and rationality and the power of the transcendent and spiritual to overcome the tragedy of human existence.
When Neitzsche (2006) famously proclaimed that “God is dead”, he wasn't making a technical claim, he was cautioning against throwing aside the transcendent and replacing it with reason and rationality alone. Nietzsche wrote about a cult of reason taking hold of Victorian society and worried greatly of the consequences of reducing everything to a rationalistic viewpoint.
This is a philosophical narrative in psychotherapy also. The great psychoanalyst Carl Jung regularly wrote about the limits of rationality. For him, the sciences failed to satisfy the deeper spiritual needs of his clients, elaborating that they could only ever occupy the "the commonplace and matter of fact, the merely average and normal...They afford, after all, no answer to the question of spiritual suffering and its innermost meaning" (Jung, 2009 pg.229). Jung cautioned that an overly scientific and rationalistic approach was limited as it placed rationality over the transcendent variables that produced it.
Carl Roger's revolutionising paper The Necessary and Sufficient Conditions of Therapeutic Personality Change (Rogers, 1957) echoed a similar process, postulating that only 6 conditions, later condensed into the three core conditions: empathy, congruence and unconditional positive regard, were necessary for positive change and that anything else added into the space impacted negatively on this change, "It is very clear that when a person is receiving an absolute minimum of external stimuli, he or she is opened to a flood of experiencing at a level far beyond the everyday." (Rogers, 1995, p. 123). His theory seemed so simplistic that at the time it was flippantly dismissed by the mainstream psychoanalytic schools. And on the surface, it seems simplistic, but Rogers believed that his method was the key to connecting the client with a universal process that could support them to overcome any problem, “That the individual has within himself vast resources for self-understanding, for altering his self-concept, his attitudes, and his self-directed behaviour--and these resources can be tapped if only a definable climate of facilitative psychological attitudes can be provided." (Roger, 1995, pg.49). And near the end of his career even Rogers, who was stridently anti-religious due to his conservative upbringing, became much more open to the spiritual nature of the therapeutic space, “Our experiences in therapy and in groups, it is clear, involves the transcendent, the indescribable, the spiritual. I am compelled to believe that I, like many others, have underestimated the importance of the mystical, spiritual dimension.” (Rogers, 1995, p. 130).
Nietzsche, Jung, Rogers, all great thinkers and influencers who seemed to be touching on the same theme. It's compelling to believe that there is an element of psychotherapy that goes beyond “the merely average and normal” as Jung writes, tapping us into a transcendent and universal reality as Roger's believed. In this space, there doesn't seem room to ask the client to “please fill out this questionnaire” without impacting negatively on that process, but a question remains: how do we know that we are not missing something?
The Case For ROMs
Despite the beautiful fluidity of their writing and its ability to touch something deeper in their readers, Nietzsche, Jung and Rogers didn't have the benefit of modern research, and when it comes to ROMs, research supports their efficacy. Reese, Nosworthy and Rowlands (2009) showed that clients of therapists using the ORS and SRS system showed a more positive and durable change, in fewer sessions than therapists who didn’t. Furthermore, Kluger and DiNisi (1996) showed that clients are more likely to improve when the results of outcome measures are openly shown and discussed with them by their therapists than for those with therapists who didn't.
Still unsure? Research also shows that the use of ROMs are justified not by identifying when therapy is working but when it is not working. Most therapists believe that they are good therapists, however, there is a large variation in outcomes of clients between individual therapists, with poor performing therapists often failing to recognize when their clients aren’t responding to therapy and leaving them worse off than when they began (Okiish, Lambert, Nielsen, & Ogles, 2003). Reese et al (2009) showed that the use of ROMs showed greater results with those not benefiting from therapy as it allows therapists to identify those not responding and make subsequent changes to their treatment. And it is here where Miller, Duncan and Hubble (2008) place much of the efficacy of ROMs - as a tool for identifying when problems are arising - so that therapists can analyse their performance and make the necessary changes.
Of course, “research says...” is the beginning of many false statements, but when it comes to supporting the case for the need to have some process that corrects for our failings, few could argue with the father of Behavioural Economics and Nobel prize winner Daniel Kahneman. In his book, Thinking Fast and Slow (2015), Kahneman outlines in detail the many ways in which our thinking is inherently flawed and in particular how immediate and accurate feedback is essential to developing expertise, “Whether professionals have a chance to develop intuitive expertise depends essentially on the quality and speed of feedback, as well as on sufficient opportunity to practice." (Kahneman, 2012, pg. 241). And on this topic, Kahneman uses psychotherapists as an example:
“Psychotherapists have many opportunities to observe the immediate reactions of patients to what they say. The feedback enables them to develop the intuitive skill to find the words and the tone that will calm anger, forge confidence, or focus the patient's attention. On the other hand, therapists do not have the chance to identify which general treatment approach is most suitable for different patients. The feedback they receive from patients' long-term outcomes is sparse, delayed, or (usually) non-existent, and in any case too ambiguous to support learning from experience." (Kahneman, 2012, pg. 242).
Another problem Kahneman outlines in his book is that confidence in our skill level can push us to believe we are better than we are, in fact, the higher the confidence the worse we are in recognising our flaws, because we are blinded by our self-belief. And it is here where Miller et al. (2008) state is the difference between a good therapist and a great therapist: those that are great, are always looking to see where they can improve and never assume they are good enough. And ROMs are the tools that provide the feedback they need to see where and when they need to make those improvements.
To Use or Not to Use
Like many students and burgeoning psychotherapists, as I hand a ROM over to a client I feel resistance, as if this piece of paper is a barrier to connecting with what Jung and Rogers were talking about. I echo Evan's (2012) concerns about ROMs moving psychotherapy towards an overly abstracted and reductive process, that focuses therapists on a narrow band of self-reported information which ignores non-quantitative variables. But, I know also that my thought process has to be grounded in a scientific methodology, otherwise, how do I know that I am not fooling myself?
When speaking to a live audience about the issue of cognitive biases, Daniel Kahneman stated that his knowledge and awareness of the biases he spent a lifetime researching and writing about afforded him no extra protection against them (Harris, 2019). That is because we can't overcome them by just being aware of them, they are bugs inherent in our biological hardware. What Kahneman advises is to have systems in place that correct for them. And this aligns directly with where the efficacy of ROMs lies, in the ability to identify problems and course-correct. Therefore, with the weight of the evidence presented, ROMs are necessary to help therapists improve themselves and the outcomes for their clients.
However, although ROMs may be necessary that doesn't mean they are sufficient. Much depends on how these tools are integrated into the therapeutic space, which is where the therapist's naturally creative and relational ability comes into play. Nor is this an endorsement of the CORE or the ORS and SRS system. This article is about the use of routine outcome measures, in which the CORE, ORS and SRS are some examples. The point is we need to have some system that can measure the process of therapy, but what that looks like or how that can be achieved may differ greatly, and in the future, we may have tools that can achieve this with minimal intrusion into the therapeutic space.
You Must Not Fool Yourself
I don't get inspired by reading overly technical and abstracted journals and research articles into the efficacy of various psychotherapy models. They define and explain every conceivable variable of the work into its principal parts and in the process drum all the life out of the practice. I get inspired by reading Rogers, Jung, Yalom, Frankl and all the greats of the field. I get inspired when I read about the power of the therapeutic alliance to help others overcome their problems. I get inspired when my clients tell me that just being able to come every week and express themselves honestly helps them feel at ease and normal again.
Nietzsche and Jung were not claiming in their arguments that the transcendent and spiritual was superior over the rational and logical. They were claiming that problems occur when you ignore one at the expense of the other. There is a place in therapy for the transcendent, but there is also a place for the rational and therefore we need to be open to the scientific element of our work as much as the spiritual element, otherwise, we are doomed to be a cargo cult science. If we claim to care about our clients, we must be open to our own limitations and never stop looking for the crucial thing we are missing.