VISTAS articles are made available for historical reference only and are presented "as is." ACA does not guarantee or represent that the information is current, accurate or indicative of the original or intended quality. These materials are not maintained or updated and may contain outdated or incomplete information. Readers should exercise discretion and verify information independently before relying on it. We assume no responsibility for the use or interpretation of this content.

Article 32

The Effects of Observation on Self-Efficacy and Satisfaction of Pre-Licensed Counselors

Neal D. Gray, Paul Erickson, & Zareena Kehkashan

Download PDF

Paper based on a program presented at the 2009 American Counseling Association Annual Conference and Exposition, March 19-23, Charlotte, NC.

A review of regulations by the American Counseling Association (ACA) Office of Professional Affairs (2012) revealed that all 50 states require the practice of counseling under supervision for 2 or more years after the completion of the master’s degree prior to licensure. However, there are no unified national standards that govern this post-master’s degree supervision (e.g., ACA, 2012; Borders & Cashwell, 1992). Supervised counseling experience ranges from 1,500 to 4,000 hours (ACA, 2012), with the amount of face-to-face clinical supervision occurring either in an individual or group setting varying greatly (ACA, 2007). Furthermore, only two states (Arizona and North Carolina) specifically require supervision interventions that include the use of reviewing audio or videotapes, or live supervision modalities to help evaluate supervisee competence (ACA, 2012). Very few studies have investigated supervision in this counselor pre-licensure stage, including factors related to counselor effectiveness such as self-efficacy and satisfaction (e.g., Bernard & Goodyear, 2004; Fall & Sutton, 2004; Magnuson, Norem, & Wilcoxon, 2002). Thus, in an effort to better understand the impact that post-master’s degree supervised experience has on counselors, this study sought to determine whether counselors’ levels of self-efficacy (beliefs concerning competence to successfully counsel a client or clients) and satisfaction with supervision are affected by the frequency of clinical skill observation provided by their supervisors.

Literature Review

Supervision Background and Standards

ACA, the Association for Counselor Education and Supervision (ACES), and the American Association of State Counseling Boards (AASCB) have all attempted to define best practices in counselor supervision. The ACA Code of Ethics (2005), Standard F.1.a, discusses a primary obligation of supervisors in the role of monitoring services of counselors-in-training. This monitoring consists of case notes, samples of clinical work, or live observation of the trainee (Herlihy & Corey, 2006). In 1990, model legislation for licensed professional counselors was developed by ACA to promote acceptable professional standards within the realm of counseling (ACA, 1990). This proposed licensure bill recommended state licensure boards consider “what is the nature of the supervision co-therapy, direct observation, audio and/or videotaping” (Bloom et al., 1990, p. 520).

ACES and ACA also addressed this issue in two different documents. First, the Ethical Guidelines for Counseling Supervisors (1995) states that supervisors are responsible for “actual work samples via audio or videotape or live observation . . .” which “should be reviewed by the supervisor as a regular part of the ongoing supervisory process” (p. 272). Secondly, the document entitled, Standards for Counseling Supervisors (1990), outlines 11 core competencies necessary for successful supervision. This document recommends that effective supervisors be skilled in using appropriate methods and techniques to promote counselor development; included are the review of video and audiotapes and live supervision (ACA, 1990). Lastly, the AASCB’s Approved Supervisor Model (2007) recommends “some type of actual counseling session reviewed on a regular basis (i.e., videotaped session at least once a month)” (p. 2).

Methods of Supervision

While utilizing direct observation of counseling skills in supervision is recommended by these various counseling associations, numerous studies have shown the most common method of supervision is self-report (e.g., Borders & Cashwell, 1992; Borders, Cashwell, & Rotter, 1995; Borders & Usher, 1992; Coll, 1995; Culbreth, Woodford, Levitt, & May, 2005; Fall & Sutton, 2004). This method of providing information about the content of counseling sessions relies exclusively on the supervisee’s subjective beliefs (Noelle, 2003). A limitation of this method may involve a lack of observable information about the session that supervisors need to accurately evaluate the effectiveness of the counselor (Bernard & Goodyear, 2004). Rogers and McDonald (1995) found that when social work instructors in the field practicum experience used student self-evaluation as the primary content focus of supervision, they more often rated students as prepared for professional practice. However, when instructors employed direct observation of skills as the primary focus of their teaching and discussion surrounding the supervision session, they were less likely to assess the students as being prepared for clinical work.

Extant work also suggests supervision beyond self-report may enhance the supervision experience. Anderson, Schlossberg, and Rigazio-DiGilio (2000), in a study of family therapy trainees’ experiences in supervision, found live supervision and videotape review related to an enhanced supervision experience. In addition, Smith (1984), in a study of counseling practicum students, found supervisor observation of skills to be directly related to counselor effectiveness. Although highly recommended, some reasons the direct observation of clinical skills in supervision is not more commonly utilized could include a lack of time clinical supervisors have to provide adequate supervision due to other job related duties (i.e., administrative), limited contact the supervisor has with the supervisee (Borders & Usher, 1992; Magnuson, Norem, & Wilcoxen, 2000; Rogers & McDonald, 1995), or the availability of apparatus necessary to directly observe skills, such as videotaping equipment and one-way mirrors (Rogers & McDonald, 1995).

Counselor effectiveness has been linked to self-efficacy and satisfaction, important components of therapeutic skill. Little work has investigated these variables in the context of the post master’s degree supervisory experience, particularly related to the method of supervision.

Self-Efficacy

Assisting the pre-licensed counselor in producing efficacious actions with clients is a primary goal of the supervisor (Larson & Daniels, 1998). Cormier and Bernard (1982) stated that the most important goal of supervision is the protection of clients’ welfare and that directly observing supervisee skills is useful in meeting this goal. Abbott and Lyter (2008) posited that supervisor observation of the supervisee during field supervision, whether by direct observation or via audio or videotaped recordings, is essential for professional growth. Lent et al. (2006) stated one function of effective supervisors is that of efficacy builders, through support, encouragement, and observation of skills; direct observation of skills is related to confidence in skills, or self-efficacy.

Self-efficacy is a component of social cognitive theory, which partially is a theory of learning through observation. In terms of counseling, the theory posits that to successfully conduct therapy, counselors must believe they are capable of providing successful treatment and be able to master techniques and interventions (Bandura, 1986; Larson, 1998). Mastery is one of four factors that contribute to the development of self- efficacy (Carruth & Woodside, 2010). If counselors have experienced previous success with an intervention, they are more likely to engage in that behavior again. They better find out if interventions work through having their skills directly observed rather than by case conceptualization (Bandura, 1986; Bandura, 1997; Larson, 1998).

Direct observation of skills helps with gaining self-efficacy over more widely used methods of training in medical settings, such as through paper and pencil testing (DuPre, 2010). For instance, in a study of supervisory observation of medical trainees’ clinical skills by Kogan, Holmboe, and Hauer (2009), the authors found direct observation of skills is related to quicker attainment of clinical skills and more effective patient care. Most importantly, confidence and the ability to apply clinical skills in practice directly influences quality of services provided (Bradley & Fiorini, 1999). Self- efficacy is thus an important component of clinical skill building in counseling.

Satisfaction With Supervision

Ramos-Sanchez et al. (2002) found that the supervisory relationship was a critical factor in supervisee development. Patton and Kivlighan (1997) found that the bond between supervisee and supervisor was predictive of this same relationship in the supervisees’ relationship with clients. Larson (1998) stated that supervisor support and encouragement (in addition to structured learning situations such as direct observation of skills) would affect supervisee self-efficacy. Learning occurs through both skill practice and within a supportive, satisfactory relationship (Frymier & Houser, 2000). Thus, if the supervisee and supervisor have a satisfactory supervisory relationship, the supervisee is more likely to gain competency in clinical skills, and further, the production of self- efficacy through direct observation of skills within that relationship is likely to lead to a greater satisfaction with supervision.

Summary of Literature Review

Counselor performance has been found to be related to self-efficacy and the supervisory environment (Larson & Daniels, 1998); counselors who feel confident in their skills and have had adequate supervision have been shown to perform better clinically. Further, Kanno and Koeske (2010) found social work interns who rated the supervisory experience as positive (i.e., helpful, receiving positive feedback) felt more empowered and reported higher levels of self-efficacy; positive supervisory experiences are linked to self-efficacy and confidence. Observation and practice of skills are directly linked with self-efficacy (Bandura, 1986). Thus, it is likely that increased levels of direct observation during supervision are related to both counselors’ self-efficacy and satisfaction with the supervisory experience.

Hypotheses

Based on the literature reviewed above, the following hypotheses were tested:

H1: Supervisees whose clinical skills are observed more often during the supervision process will have higher levels of perceived self-efficacy than supervisees whose clinical skills are observed less often.

H2: Supervisees whose clinical skills are observed more often during the supervision process will have higher levels of satisfaction with supervision than supervisees whose clinical skills are observed less often.

Method

Procedure

Survey packets were mailed to professional counselors who had been licensed for 2 years or less in a large, Southern state. After obtaining addresses from the state’s licensure board, survey packets containing the introduction letter and the self-report paper-and-pencil instruments were mailed directly to the potential participants. Each survey packet contained a cover letter introducing the study, one document which included the following three instruments: (a) the Supervisory Satisfaction Questionnaire (SSQ) constructed by Ladany, Hill, Corbett, and Nutt, (1996); (b) the Counselor Self- Efficacy Scale (CSES) developed by Melchert, Hays, Wiljanen, and Kolocek (1996); and (c) the Post-Master’s Degree Supervision Questionnaire (PMDSQ) developed for the purpose of this research project, and a return envelope.

The potential participants were asked to complete and return the survey in a self- addressed, stamped envelope. Participation was completely voluntary. No additional contact with the participants was made, and no individual identification was assigned. Data analysis included generating descriptive statistics and analyses of the independent and dependent variables using ANOVA. Descriptive and inferential statistics were calculated using SPSS (Version 17.0).

Instrumentation

The Supervisory Satisfaction Questionnaire. The Supervisory Satisfaction Questionnaire (SSQ) is an 8-item self report measure that rates supervisee perceptions of the quality and outcomes of supervision, a modification of the Client Satisfaction Questionnaire (CSQ; Larson, Attkisson, Hargreaves, & Nguyen, 1979). Lower scores on this instrument reveal greater satisfaction with supervision (Ladany et al., 1996). Factor analyses reveal internal consistency estimates consistently ranging from .84 to .93 (Nguyen, Attkisson, & Stegner, 1983).

The Counselor Self-Efficacy Scale. The Counselor Self-Efficacy Scale (CSES) is a 20-item Likert-type scale that assesses respondents’ sense that they are competent and effective as counselors (specifically in the areas of individual and group counseling). Lower scores correspond to higher levels of self-efficacy.

The authors addressed content-related validity by having three expert judges, supervising trainees at a counseling center, appraise the CSES (Melchert et al., 1996). The Self-Efficacy Inventory (SEI; Friedlander & Snyder, 1983) was used to examine the convergent construct validity of the CSES. The correlation was found to be high (r=.83) for the scales measuring similar constructs (Melchert et al., 1996). The Cronbach alpha internal consistency correlation coefficient for the SEI was found to be .93, while the test- retest reliability coefficient for the total scale scores was .85.

The Post-Master’s Degree Supervision Questionnaire. The Post-Master’s Degree Supervision Questionnaire (PMDSQ) was developed for use in this particular research study. The instrument asks counselors questions concerning (a) race, (b) age, (c) sex, (d) type of supervision received, (e) matching with supervisor of specialty area, (f) matching with supervisor of theoretical orientation, and (g) credentials of supervisor (Gray, 2001).

Participants

Survey packets were mailed to 1,400 Licensed Professional Counselors (LPCs) licensed 2 years or less living in a large, Southern state. A total of 294 participants completed the survey instruments yielding an overall response rate of 21%. A total of 248 women and 42 men participated. Of the respondents, 70.3% reported their ethnicity as European-American, followed by Hispanic-American (11%), African-American (6.2%), Asian-American (1.4%) and Native American (.7%). A response of “other” accounted for 10.3%. Of the participants, 78.9% reported supervisor ethnicity as European-American, followed by African-American (6.2%), Hispanic-American (5.8%), Asian-American (.7%), and Native American (.3%). A response of “other” accounted for 8.2%. The participants were asked to identify the extent to which their supervisor listened to audiotapes, watched videotapes, or conducted live supervision of their counseling skills during the supervisory relationship (question #4 of the PMDSQ). Choices included (1) never; (2) seldom; (3) about half of the time; (4) almost always; and (5) always. The majority of the participants reported that their supervisor “never ” (111 participants or 38%) observed their skills directly; 108 participants (37%) reported “seldom” having had their counseling skills observed; 51 participants (17%) reported their supervisor observed their skills directly “about half of the time”; 16 participants (5%) reported their supervisor observed their skills directly “almost always”; and 8 participants (3%) stated that their supervisor observed their skills directly “always.” Therefore, most supervisors are depending on supervisees’ personal accounts and opinions about what occurs in counseling sessions rather than having the benefit of direct observation or review of their supervisees’ counseling skills.

Results

The first hypothesis was that supervisees whose clinical skills are observed more often during the supervision process will have higher levels of perceived self-efficacy than supervisees whose clinical skills are observed less often. An ANOVA procedure compared the scores on the CSES (range 20 - 100) to item #4 of the PMDSQ which included the following categories concerning the supervisors’ observation of supervisees skills: (1) Never; (2) Seldom; (3) About Half of the Time; (4) Almost Always; and (5) Always. Due to a low response rate, categories three, four, and five were collapsed into one category. Lower scores on the CSES indicate higher levels of perceived self-efficacy.

Table 1

Results of the ANOVA Counselor Self Efficacy Scale (CSES) Scores of Supervisees by Amount of Viewed Supervision

*p < .05

Counselors who reported their clinical skills never being observed during the supervision process (N = 103) recorded an overall mean score of 35.94 on the CSES. Supervised counselors who reported their skills being seldom observed (N = 104) recorded an overall mean score of 29.14, while counselors who reported their skills being observed in a range from half of the time, almost always, to always (N = 72) obtained an overall mean score of 25.23. The results of the ANOVA (Table 1) indicate a statistically significant difference among the CSES scores of participants depending on the amount of viewed supervision that occurred. The second table includes a listing of means and standard deviations for Hypothesis 1. A post-hoc comparison was made using the Scheffe test. This comparison indicated that the overall mean CSES scores for participants whose skills were never observed were significantly lower than those whose skills were observed in the category of seldom. Additionally, for those in the combined group whose skills were observed either half of the time, almost always, or always, levels of perceived self-efficacy were significantly higher than those whose skills were viewed in the groups categorized never or seldom.

Table 2

Means and Standard Deviations of Counselor Self Efficacy Scale (CSES) Scores by Viewed Supervision

Lower Scores Indicate Higher Levels of Beliefs Concerning Self-Efficacy

Table 3

Results of the ANOVA Satisfaction with Supervision Questionnaire (SSQ) Scores of Supervisees by Viewed Supervision

p < .05

Table 4

Means and Standard Deviations of Supervision Satisfaction Questionnaire (SSQ) Scores by Viewed Supervision

Lower Scores Indicate Higher Levels of Beliefs Concerning Satisfaction with Supervision

*Totals are not all equal across instruments due to missing data.

The second hypothesis was that supervisees whose clinical skills are observed more often during the supervision process will have higher levels of satisfaction with supervision than supervisees whose clinical skills are observed less often. An ANOVA procedure compared the scores on the SSQ (range 8 - 32) to item #4 of the PMDSQ which included the following categories concerning the supervisors’ observation of supervisee’s skills: (1) Never; (2) Seldom; (3) About Half of the Time; (4) Almost Always; and (5) Always. Due to a low response rate, categories three, four, and five were collapsed into one category. Lower scores on the SSQ indicate higher levels of satisfaction with supervision. Counselors who reported their clinical skills never being observed during the supervision process (N = 107) recorded an overall mean score of 14.98 on the SSQ. Supervised counselors who reported their skills being seldom observed (N = 107) recorded an overall mean score of 14.20, while counselors who reported their skills being observed in a range from half of the time, almost always to always (N = 75) obtained 13.02. Although the results of the ANOVA (Table 3) indicate greater satisfaction with supervision when more frequent direct observation of skills occur, the finding was not statistically significant among the three groups. Table 4 includes a listing of means and standard deviations for Hypothesis 2.

Discussion

This study investigated whether the frequency of clinical skill observation (live, video, audio) that occurs in the supervision experience affects pre-licensed counselors’ self-efficacy and satisfaction with the supervision experience. Results suggest pre- licensed supervisees whose clinical skills are observed more often during the supervision process have higher levels of perceived self-efficacy than supervisees whose clinical skills are observed less often. Specifically, supervisees who reported their skills were rarely observed reported significantly higher levels of self-efficacy than those who reported their skills never directly observed. Also, supervisees that reported their skills were observed from half of the time to always reported significantly higher levels of self- efficacy than those whose skills were observed rarely. From these findings it can be concluded that directly observing supervisees engaging in counseling is a positive factor in their clinical development.

The second hypothesis asked if pre-licensed supervisees whose clinical skills are observed more often during the supervision process have higher levels of satisfaction with supervision than supervisees whose clinical skills are observed less. Results found no significant differences concerning satisfaction with supervision. Fernando and Hulse- Killacky (2005) provided a possible explanation for this disparity in findings of self- efficacy versus satisfaction with supervision, stating “it is reasonable to believe that effective supervision is not always the most satisfying supervision, because the hard work that accompanies learning may not always be experienced as the most satisfying” (p. 302).

This study has shown that the pre-licensure supervision experience of counselors is influenced by the direct observation of supervisors, and thus, may stimulate supervisors and counselor educators to address professional development issues with supervisees and in their own development.

Implications for Supervisors

Results from this study indicate that supervisees rate their levels of self-efficacy higher when supervisors directly observe their skills. These findings have several implications for counselor supervisors interested in supervisee self-efficacy. While numerous studies indicate the most common form of conducting supervision is by self- report (e.g., Borders & Cashwell, 1992; Borders et al., 1995; Borders & Usher, 1992; Coll, 1995; Culbreth, et al., 2005; Fall & Sutton, 2004), this study indicates observing counselors through the use of audio or videotapes or live supervision is beneficial to supervisee growth. The findings further suggest the seldom observation of skills has a significantly greater effect on supervisees than no observation, and that observing supervisee skills at least half of the time in supervision is related to greater supervisee self-efficacy than rarely observing clinical skills. Self-efficacy is associated with counselor effectiveness and, thus, if direct observation of skills contributes to greater self- efficacy, such observation may be linked to more effective counselor performance.

It is paramount during discussions of expectations, roles, and responsibilities in the supervisory relationship that supervisors emphasize to supervisees that their skills will be observed during the pre-licensure stage (Remley & Herlihy, 2010). If not mandated by supervisors, it is doubtful supervisees will solicit direct observation of their skills. Borders and Usher (1992), in a study conducted to determine preferred supervison modalities of supervisees, found that supervisees preferred self-report over observation of skills. The authors stated “respondents may have considered other methods (e.g., live observation, videotaping) to be too inconvenient, intrusive or threatening” (p. 598). Furthermore, at the conclusion of the pre-licensure experience, supervisors will be required to formally report to state licensure boards a judgment of the supervisees’ competence to practice independently as a counselor (Cobia & Boes, 2000). Observing skills on a regular basis throughout the relationship can aid in this decision. Lastly, it is recommended if supervisors are not trained in using these methods of direct observation they consider completing continuing education training, workshops, or graduate courses (Borders et al., 1995; Culbreth et al., 2005).

Implications for Counselor Educators

The Council for Accreditation of Counseling and Related Educational Programs (CACREP) requires supervisors in master’s and doctoral practicum experiences to include during the experience “the development of program-appropriate audio/video recordings for use in supervision or live supervision of the student’s interactions with clients” (CACREP, 2009, p. 17). It is also common that evaluation for course success utilizes audio/videotapes or live observation of clinical sessions in practicum and internship. However, at the post-master’s degree level this is not a requirement, but usually a recommendation. Previous research studies point out that the observation of clinical work is a rarity and that self-report of clinical sessions is the norm (Borders & Cashwell, 1992; Borders et al., 1995; Borders & Usher, 1992; Coll, 1995; Culbreth et al., 2005; Fall & Sutton, 2004). In the current study, almost 75% of respondents reported during the pre-licensure phase of supervision having their skills observed never or rarely. That being said, the study’s findings emphasized that viewing supervisees’ skills significantly affects their beliefs positively concerning their levels of self-efficacy.

It is recommended that counselor educators in master’s and doctoral degree programs discuss the post-master’s degree process with students during their graduate school experience, specifically recommending that when selecting a supervisor, pre- licensed counselors broach the issue of how their counseling skills will be evaluated. It is also necessary that counselor educators encourage future counselors to choose supervisors committed to using direct observation as a method to enhance counselee growth. Studies such as this can be used to demonstrate to future pre-licensure supervisees that despite the possible feelings of anxiety associated with having their clinical skills examined, requesting this observation will greatly benefit their growth as counselors (Borders & Usher, 1992; Overholser, 2005).

It is also recommended that since counselor educators have experience and skill in using these methods of direct observation, they should provide supervision trainings or continuing education opportunities for supervisors in their communities to enhance their competence in supervision (Coll, 1995). Furthermore, as more states call for supervisors to become board certified, counselor educators should advocate for state licensure boards to require supervisory training in methods of clinical skill observation. In addition, it is also recommended that licensure boards should mandate a certain percentage of time that counselor’s skills be directly observed.

Limitations

A limitation is defined by Pryczak and Bruce (1998) as a “weakness or handicap that potentially limits the validity of the results” (p. 57). The limitations that could have threatened the internal and external validity of this study are discussed below.

Only one administration of the survey occurred. Strategies for increasing response rates such as conducting a second mailing of the survey, or sending a follow-up postcard/letter may have greatly enhanced the response rate (Bourque & Fielder, 1995; Dillman, 2000; Gay & Airasian, 2003).

Also, concerning the observation of skills, participants were asked how often their skills were observed using video or audiotapes or through live supervision. Other methods of observations such as computer-based live supervision (e.g., webcams or other technological methods utilizing distance supervision) were not offered as choices. Furthermore, supervisees were not asked to quantify the exact amount of times their clinical skills were observed so the categories assigned by the researchers (never, seldom, about half of the time, almost always, and always) may be perceived differently and vary from participant to participant.

In addition, information obtained from supervisee participants on characteristics of their supervisors was not verified and may be inaccurate. Acquiring information from a secondary source, as was done in this study, assumes that the participants honestly and knowledgeably answered the questions concerning characteristics of their supervisors. If supervisees were not accurate when reporting supervisor characteristics, the results of this study may not be accurate.

Threats to external validity are associated with the sample used in this study. Because the participants represent only one state located in the Southern United States, the sample may not be representative of LPCs in general. Finally, the results of this study may not be applicable to the supervision experience of LPCs in other regions of the United States.

Conclusion

While the post-master’s degree supervision experience is critically important in the development of competent counselors, very little research has been conducted to determine which factors in supervision produce more satisfied or capable counselors. Individual state licensure boards in the United States lack specific requirements concerning methods of supervision (e.g., verbal exchange, direct observation). As a result, it is necessary to conduct research in order to determine the relationship of supervision methods to supervisee satisfaction and self-efficacy levels.

This study found that the self-efficacy of post-master’s degree counselors seeking licensure is positively affected by having their skills directly observed through live observation, or through the examination of videotapes or audiotapes. The results of this general exploratory study may help to determine if current supervisory conditions are optimal for training future counselors, or whether changes within supervisory regulations are needed to enhance the supervision received by post-master’s degree counselor licensure candidates. The primary goal of this study was to provide information that will be used by counseling supervisors and supervisees to enhance the pre-licensure supervision experience. Findings from this study will also provide information that is useful to supervisors and counselor educators in determining better methods of addressing professional development issues in both supervisees and their own development.