VISTAS articles are made available for historical reference only and are presented "as is." ACA does not guarantee or represent that the information is current, accurate or indicative of the original or intended quality. These materials are not maintained or updated and may contain outdated or incomplete information. Readers should exercise discretion and verify information independently before relying on it. We assume no responsibility for the use or interpretation of this content.

Article 51

Do Counselors Use Evidenced-Based Treatments?

Results of a Pilot Survey

Thomas A. Field, Elizabeth B. Farnsworth, and Steven K. Nielsen

Download PDF

Paper based on a program presented at the 2014 American Counseling Association Conference, March 26, 2014, Waikiki, HI.

Correspondence about this article should be directed to Thomas A. Field, City University of Seattle, 521 Wall Street, Seattle, WA 98121. Email: [email protected]

For the past decade, the counseling profession has made concerted efforts to delineate best practices for the field. In 2005, the American Counseling Association’s (ACA) Code of Ethics included a recommendation to use therapies that “have an empirical or scientific foundation” (ACA, 2005, C.6.e). The Journal of Counseling and Development introduced a new journal feature in 2007, entitled “Best Practices.” In 2009, the Council for the Accreditation of Counseling and Related Educational Programs (CACREP) modified their 2009 Standards for Addiction Counseling (I.3., p. 22), Clinical Mental Health Counseling (I.3., p. 34), and Marriage, Couple, and Family Counseling (I.3., p. 39) to require that the student “knows evidence-based treatments.” As best practices such as evidenced-based treatments (EBTs) are being identified in the literature and disseminated through training programs, it has become necessary to examine the actual use of these practices by counselors in the field. While other best practices could be explored, the 2009 CACREP Standards imply that specific attention needs to be given to the utilization of EBTs.

Review of Related Literature

Evidenced-based treatment (EBT) is an important construct in the history of evidence-based practice (EBP) in counseling and psychotherapy. The definitions of EBP and EBT are easily confused, in part because of their historical context. EBP refers to the global definition of an intervention that has empirical or scientific support for its effectiveness. EBT refers to a specific intervention (e.g., cognitive-behavior therapy) matched with a specific disorder (e.g., generalized anxiety disorder) that is manualized (i.e., has a treatment manual).

The History of Evidenced-Based Treatment

In the mid 1990s, a misperception existed within healthcare that psychiatric medications were a superior intervention (and thus, first line treatment) to counseling and psychotherapy (LaRoche & Christopher, 2009). A Task Force of the American Psychological Association (APA) sought to address this concern by calling for research that directly compared the outcomes of psychiatric medication to psychotherapy (Task Force on Promotion and Dissemination of Psychological Procedures, 1995). In addition, the Task Force created a list of interventions, termed psychological treatments, which they claimed had demonstrated superior outcomes for treating certain disorders through at least two randomized controlled trials (RCTs). These RCTs either compared bona fide psychological treatments with each other, or compared psychological treatments with control groups that consisted of placebos (e.g., no treatment) or treatment-as-usual interventions (e.g., by master’s level therapists in the community). Additionally, a treatment manual had to exist in order for a psychological treatment to make the list. The resulting interventions were designated as EBTs, also referred to in the literature as “Empirically Validated Treatments” or “Empirically Supported Treatments” (Levant, 2004). Therefore, EBP in psychotherapy was initially reduced to matching manualized treatments to certain disorders (Wampold & Bhati, 2004).

Following dissemination of the EBT list, practitioner utilization of EBTs has been disappointing (Becker, Stice, Shaw, & Woda, 2009). Criticisms of the EBT paradigm include lack of generalizability from research to practice settings because subjects are excluded if they have comorbid mental disorders; respected theoretical orientations excluded from the EBT list due to lack of RCT studies, even when other credible non- RCT studies exist; and underrepresentation of minority groups in RCTs (Bernal & Scharron-del-Rio, 2001). Furthermore, meta-analyses have consistently found no salient differences in effectiveness when counseling interventions are directly compared to one another (Wampold & Bhati, 2004). Yet despite this, governmental insurance programs such as TRICARE and Medicaid, alongside private insurance programs, require that EBTs be used in order for services to be reimbursed, a practice that is anticipated to increase since healthcare expenses will need to be managed and cost-contained more carefully as the 2010 Patient Protection and Affordable Care Act reaches full implementation (Rozensky, 2014). Considering that the 2013 law allowing mental health counselors to practice independently within the Veteran’s Affairs and TRICARE system was informed by a report issued from the Institute of Medicine recommending that all health care professionals, including mental health counselors, use EBTs as a routine part of treatment (Institute of Medicine, 2010), it is unlikely that counseling professionals will be exempt from requirements to use EBTs in their direct work with clients.

Previous Studies in EBT Utilization

Little is currently known about the utilization of EBTs within the counseling profession. To date, no studies exclusively surveying counselor use of EBTs have been published. Only one survey study has been conducted on EBT utilization that included counselors as a significant part of the sample (Gray, Elhai, & Schmidt, 2007). In another master’s level discipline, Pignotti and Thyer (2011) surveyed 400 Licensed Clinical Social Workers (LCSWs) and found that the majority had used at least one EBT in the past year. One of the major limitations of the Pignotti and Thyer survey was failure to match treatments with a disorder. Both of these studies sampled attitudes toward EBP while measuring EBT utilization via a standardized instrument (Evidence-Based Practice Attitudes Scale, or EBPAS; Aarons, 2004).

Method

Participants

Since counseling does not have a national credential for licensure, participants were chosen who held the National Certified Counselor (NCC) credential in order to establish consistency in counseling preparation of participants. Participants were selected via the Counselor Find online search engine at the National Board of Certified Counselors (NBCC) Web site (http://www.nbcc.org). The search engine is comprised of NCCs who wish to advertise their services to the public. Potential NCC participants were screened by practice area and selected on the basis of identifying themselves as working in Addictions and Dependency, Clinical Mental Health, or Mental Health/Agency Counseling. This method was used to avoid contacting NCCs who work in non-clinical settings such as schools, counselor education, career development, and sports counseling. NCCs who worked in university counseling centers or identified themselves as working in Addictions and Dependency, Clinical Mental Health, or Mental Health/Agency Counseling in addition to school counseling were included in the sample. A cluster sampling method was utilized, with individual states in the United States being randomly selected. The nine states from which NCCs were randomly selected included Arkansas, Colorado, Indiana, Massachusetts, Montana, Pennsylvania, Tennessee, Texas, and Wyoming. Survey recipients were then randomly selected from a pool of NCCs who identified themselves as working in a clinical setting and residing in the above mentioned states.

Materials

An expedited review was requested from the Lynchburg College Institutional Review Board. After the research proposal was approved, an initial survey was constructed. The first section of the survey contained the informed consent statement. The second section contained questions pertaining to demographic information and a dichotomous question (true/false) asking participants whether they had utilized EBTs within the past year. Data from participants who responded negatively (false) were excluded from analysis of specific EBT utilization rates. The third section of the survey contained a checklist for EBTs used for specific disorders within the past year (taken in full from the current official EBT list on the APA Division 12 Web site, http://www.div12.org/PsychologicalTreatments/treatments.html). The fourth section contained Aarons’ (2004) EBPAS scale in an unchanged form.

The Evidence-Based Practice Attitudes Scale

The EBPAS standardized scale was chosen since it was the only standardized scale available for measuring attitudes toward EBP (Aarons, 2004; Aarons, McDonald, Sheehan, & Walrath-Greene, 2007). The EBPAS contains four subscales, including Appeal (the intuitive appeal of EBP), Requirements (the willingness to adopt EBP if so required by state, supervisor, or agency), Openness (the openness to change and adopting new practices), and Divergence (the degree to which EBP diverges from the practitioner’s usual practices). These subscale domains were chosen after a thorough review of the extant literature (Aarons, 2004). Item responses are measured on a Likert- scale, from 0 (“not at all”) to 4 (“to a very great extent”). In terms of reliability, the scale has acceptable internal consistency, although the subscale alpha for Divergence was below the generally accepted cut-off value of .70 (Aarons et al., 2007). Prior to statistical analysis, the Divergence subscale is reverse-coded. This may explain the low alpha for the subscale, since reverse coding can affect covariances between items. This subscale was retained since it measures an important construct in the literature (Aarons, 2004). Aarons et al.’s (2007) geographically diverse sample included counselors, grouped with psychologists (n = 100, 45.2%). Thus, this study’s sample can be somewhat compared to the instrument’s geographically diverse sample. One of the main concerns with the EBPAS is its apparent lack of external validity; past studies (Gray et al., 2007; Pignotti & Thyer, 2011) have shown a lack of relationship between scores on the EBPAS and reported utilization of EBTs.

Procedure

The survey was reviewed by professors at Lynchburg College and James Madison University. Initial practice administrations were performed among graduate students at Lynchburg College to ensure comprehension of questions and ease of use. Following several revisions, the survey was uploaded to Survey Monkey (http://www.surveymonkey.com) for data collection purposes. Dillman’s (2007) tailored method for mail and Internet surveys was consulted as a guide for dissemination. A cover letter, informed consent statement, and hard copy of the survey were sent to the initial 370 NCCs. The letter included an invitation to complete the survey online. A second mailing was sent out 6 weeks later via postcard, with a reminder to complete the survey online. Data was collected via online surveys and inputting hard copies into the Survey Monkey collector. The survey data was then analyzed using SPSS.

Using the sampling procedure described above, 400 recipients were initially selected. Thirty recipients were excluded due to inability to find a current mailing address. Three recipients opted out of the survey and 57 surveys were returned undeliverable. Seventy-six recipients completed the survey from the remaining 310 participants, resulting in a return rate of 24.5%. While this is a small response rate, it was comparable to similar mail surveys sent to practitioners regarding EBT utilization in which response rates ranged from 15% to 43%, with the majority below 30% (e.g., Freiheit, Vye, Swan, & Cady, 2004; Stewart & Chambless, 2007).

Results

The respondent sample consisted of 69.9% females (n = 51) and 30.1% males (n= 22). Regarding race/ethnicity, respondents primarily identified themselves as Caucasian (81.7%, n = 59), then as African-American (6.9%, n = 5), Hispanic/Mexican-American (5.6%, n = 4), Native-American (4.2%, n = 3), and Asian-American (1.4%, n = 1). Respondents were fairly experienced, averaging 12.87 years as an NCC (SD = 8.67, SEM = 1.05). The sample was overrepresented by private practitioners (68.1%, n = 49), with less than one third working in community mental health centers and outpatient clinics (29.2%, n = 21), and fewer working in a university counseling center (15.3%, n = 11). This was expected, since it was more likely that private practitioners would advertise their services on the Internet than NCCs who work in settings where advertisement is not as essential to recruiting clients. The sample demographics were not wholly representative of the population but were deemed adequately representative because the most recently available national study of NCC demographics found that 74% were female and almost half were over the age of fifty (NBCC, 2000).

Over two thirds (69.4%, n = 50) of NCCs reported using treatment manuals in their practice within the past year. Appendix A shows the most popular EBTs utilized. Of those using EBTs, the average reported number of manual-based interventions was higher than expected (M = 9.17, SD = 6.94, SEM = 0.97). Nearly two thirds of participants (61.5%, n = 32) reported utilizing more than five, 38.5% (n = 20) reported utilizing more than nine, and 9.6% (n = 6) reportedly utilizing 20 or more.

Responses on the EBPAS Scale

Responses to Aarons’ (2004) standardized instrument (EBPAS) were intriguing. The mean EBPAS item score was 2.45 (SD = 0.49), fairly close to Aarons et al.’s (2007) geographically diverse sample (M = 2.77, SD = 0.45). Table 1 reveals that the mean scores and standard deviations between this study’s sample and Aarons et al. (2007) were somewhat comparable. The only subscale with a significant difference between samples was Divergence (α = .19 vs. .61), whose subscale alpha was weak. According to Aarons (2004), a high total score on the EBPAS indicates a positive attitude toward EBP. This sample’s mean total score on EBPAS was 50.98 (SD = 7.46), which was above the neutral midpoint of 45 and comparable to Pignotti and Thyer’s (2011) mean score for LCSWs (M = 54.08, SD = 7.15). Therefore, this sample of NCCs appeared to hold positive attitudes to EBP.

These positive attitudes toward EBP did not appear to influence EBT utilization rates. Similarly to previous studies (Gray et al., 2007; Pignotti & Thyer, 2011), no significant correlation was found between EBPAS total scores and number of manual- based interventions used within the past year. It is therefore likely that the EBPAS has questionable external validity. It is also possible that attitudes toward EBP are distinct from EBT utilization rates.

Table 1.

Comparison of means, standard deviations, and Cronbach’s alpha on EBPAS subscale scores between Aarons et al. (2007) and Field et al. (2015)

Note. Total, subscale, and item mean scores range from 0 to 4, where 0 = not at all, 1 = to a slight extent, 2 = to a moderate extent, 3 = to a great extent, 4 = to a very great extent. N = 76.

Demographic Variables and EBT Utilization

On average, males (M = 6.64, SD = 4.99, SEM = 1.50) used fewer numbers of EBTs than females (M = 10.10, SD = 7.24, SEM = 1.15). This finding may be related to number of years as an NCC; male participants tended to be more experienced (M = 16.42, SD = 9.54, SEM = 2.19) than females (M = 11.49, SD = 7.99, SEM = 1.14). An independent samples t-test for differences in number of years as an NCC between males and females was significant (t66 = 2.16, p < .05), with a large effect size (d = .56). To further explore the relationship between years of experience and EBT utilization, the true/false question “within the past year, I have used treatment manuals in my counseling practice” was processed as a dichotomized group variable. The mean number of years as an NCC for those who utilized EBTs was 10.54 (SD = 8.10, SEM = 1.20), and the mean number of years as an NCC for those who did not was 17.73 (SD = 7.91, SEM = 1.69). An independent samples t-test for differences between groups of EBT users versus non- EBT users was significant (t66 = -3.45, p < .01). The computed effect size between these groups for years of experience was very large (d = .91). Therefore, the more experienced a participant, the less likely they were to use EBTs. This was particularly true for men.

A significant but weak negative correlation was also found between number of years as an NCC and EBPAS scores (r = - .32, p < .05). This suggests that attitudes toward EBP are also slightly influenced by years of experience. The more experienced an NCC, the less positive their attitude toward EBP. These results are consistent with earlier findings by Aarons (2004) and Gray et al. (2007), who also found significant negative relationships between years of experience and utilization of EBTs/attitudes toward EBP.

Discussion

Previous research has indicated that mental health professionals are beginning to use EBTs as part of their practice (e.g., Pignotti & Thyer, 2011). This study’s findings indicate that not only are counselors (NCCs) using EBTs (69.4%), they are using a great number of them (M = 9.17, SD = 6.94). Since the greatest source of variance was attributed to number of years as an NCC, further research is needed regarding why EBT utilization rates are lower among more experienced practitioners. Past studies have found that experienced counselors tend to rely more on intuition than manualized protocols (e.g., Rønnestad & Skovolt, 2003). Several explanations can be posited to explain this phenomenon. The first and simplest is that EBTs become less appealing to practitioners as they gain more years of experience. EBTs tend to be highly structured therapies, and newer practitioners may be more receptive to using a manual when conducting counseling. A second explanation involves the timeline of EBT dissemination. EBTs are a fairly new concept in the field, with the first list appearing in 1995. According to Levant (2004), it takes an average of 17 years before research findings are translated into routine practice behaviors within the health care system. It can be hypothesized that more experienced NCCs have received less exposure to EBTs than newer NCCs, which may have affected attitudes toward EBP and utilization of EBTs.

In Appendix A, a form of cognitive, behavior, or cognitive-behavior therapy (CBT) dominated the top-ten list of most utilized EBTs (7 out of 10). The majority of interventions on the official EBT list are CBT-related (Society of Clinical Psychology, 2013). Only 5.9% of the NCC sample (n = 68) that used EBTs did not report using a form of CBT. Therefore in the current climate, EBT utilization may be practically synonymous with CBT utilization. It could be argued that the emergence of the EBT movement has propelled CBT into first-place among interventions used in practice settings. The dominance of CBT may only solidify following the initiation of EBT training within counselor education programs.

Limitations

The study had several limitations. The survey sample was comprised of NCCs who advertised their services to the public via Counselor Find. NCCs who did not advertise their services through the Internet may have responded differently, and thus the sample cannot necessarily be considered representative of all NCCs in the United States. The majority of NCCs reported their work setting as private practice (68.1%), which, while expected, is not representative of the NCC population. The sample was small (N = 76), and it is difficult to generalize these findings to counselors as a whole. Further studies could seek to acquire a larger sample to support or discredit the findings of this study. The response rate (24.5%), while comparative to past studies in EBT utilization, only gathered data from one fourth of all possible respondents. Most respondents completed the survey following the initial mailing (97.4%). It is therefore possible that the sample represents NCCs who were more motivated to return surveys on EBT utilization than non-returners, which would bias the results. The mean number of EBTs utilized within the past year (9.17) was surprisingly high and might be explained by this potential response bias. In other words, NCCs who used a high number of treatment manuals within the past year may have been more likely to complete the survey than NCCs who used fewer treatment manuals.

The mean reported number of EBTs utilized within the past year brings us to the limitation of self-reported data. It is unknown whether self-reported number of EBTs used in the past year was an accurate reflection of actual EBT use in client sessions. It cannot be guaranteed whether NCCs actually used manual-based interventions, or if these interventions were paired with the actual disorder listed. Some attempt was made to maximize participant comprehension of an authentic EBT; the survey instructions included a disclaimer that directed respondents to only check interventions that were manual-based and paired with a specific disorder.

Implications for Counselors

While the small size of the sample makes it difficult to generalize findings, counselors can glean some important information from this survey. First, it seems likely that some counselors are utilizing EBTs with varying frequency, especially CBT. Second, attitudes toward EBP do not appear to influence rates of EBT utilization. Third, it appears that counseling experience may be negatively correlated with EBT utilization. Counselor education programs that already train students in EBTs may consider offering professional trainings for experienced practitioners as an area of development.

From this study, it seems that less experienced counselors are likely to use more EBTs than counselors with more experience. Thus, newer practitioners may have a greater desire for training and supervision in EBTs. Supervisors could ask novice supervisees about their interest in receiving EBT supervision. Future research could evaluate the impact of reimbursement on counseling interventions and the current frequency of EBT training in counselor education programs. Finally, this study marks the first endeavor to measure actual EBT utilization rates by counselors. Further studies could corroborate or disconfirm the findings of this study.

Appendix A

Utilization rates within the past year for participants reporting use of EBTs (n = 52)

Note: This list was taken directly from the official EBT list on the APA Division 12 Web site (http://www.div12.org/PsychologicalTreatments/treatments.html). EBTs with a frequency (n) of 0 were omitted from this table.