VISTAS articles are made available for historical reference only and are presented "as is." ACA does not guarantee or represent that the information is current, accurate or indicative of the original or intended quality. These materials are not maintained or updated and may contain outdated or incomplete information. Readers should exercise discretion and verify information independently before relying on it. We assume no responsibility for the use or interpretation of this content.

Article 46

Introducing the Accountability Bridge Model

A Program Evaluation Framework for School Counselors

J. Kelly Coker, Randall L. Astramovich, and Wendy J. Hoskins

Download PDF

In 2001, the No Child Left Behind Act was passed which emphasized student testing and teacher effectiveness (U. S. Department of Education, 2001). This movement in education reform paved the way for the American School Counselor Association (ASCA) to develop a new framework which included an emphasis on accountability in school counseling (ASCA, 2003). Both of these endeavors have created increased pressures on school counselors to be able to demonstrate the effectiveness of their school counseling programs and their impact on student achievement.

Program Evaluation

A question not easily answered is, How then do school counselors provide evidence that their programs are making a difference? Program evaluation is one process that can be used to aid school counselors. Program evaluation is considered an applied research discipline, and is defined as a systematic process of collecting and analyzing information about efficiency, effectiveness, and impact of programs and services (Boulmetis & Dutwin, 2000). One reason program evaluation is suggested as a valuable tool for school counselors is that it can be considered as a type of action research geared toward monitoring and improving programs or services. These evaluations can be conducted on a smaller scale, can be planned and implemented by practitioners, and can be used to communicate the impact of programs on student achievement and other pertinent variables. There are some key questions program evaluations can help school counselors answer, such as

1. What methods, programs, and interventions are most helpful for students?

2. How satisfied are students and teachers with the services received?

3. How has student achievement been impacted because of a particular intervention or program?

4. How does class placement affect student achievement?

5. How well are the school counseling program objectives being met?

Barriers to Program Evaluation

There are some key potential barriers to conducting program evaluations that school counselors face. Literature related to counseling program evaluation has cited a lack of interest and ability to systematically evaluate counseling services (Whiston, 1996).

Conducting program evaluations does require some degree of expertise in basic research methods, yet counselors typically receive little training to prepare them to demonstrate research outcomes in their professional settings (Whiston, 1996). The authors conducted a study whereby 28 school counselors in the southwest answered questions that included an inquiry of the type of instruction, if any, they received in program evaluation during their training programs. The majority of training participants, 15 (53.6%), did not receive training in program evaluation during their graduate coursework. Of the remaining participants, 12 (42.9%) indicated receiving some program evaluation training in during graduate level coursework, and 1 (3.6%) did not specify (Astramovich, Coker, & Hoskins, in press).

Another potential barrier to conducting program evaluations is a lack of confidence school counselors may feel in regard to their ability to collect and analyze data and then utilize their findings to impact their school counseling program (Isaacs, 2003). Other counselors have acknowledged that even if they do possess the skills to conduct program evaluations, there can be an accompanying fear that their findings may not support the idea that their programs are effective (Lusky & Hayes, 2001). While a legitimate concern, the authors believe that by conducting program evaluations, counselors can also work more effectively by focusing more on prevention and intervention efforts that have proven to be effective rather than spending their time engaged in ancillary duties that do not directly benefit student outcomes.

Time is also a limitation identified in the authors’ research. When asked if program evaluations were too time consuming to be conducted, the 28 participants’ responses were split, with 9 (32.1%) agreeing or strongly agreeing, 9 (32.1%) expressing uncertainty/ neutrality, and 10 (35.7%) disagreeing or strongly disagreeing (Astramovich et al., in press). Counselors certainly experience the phenomenon of too much to do, too little time. One argument for including program evaluation into a comprehensive school counseling program is to help counselors streamline their programs and create opportunities to make a case to administrators for those aspects of the school counselor’s role that are indeed the most effective. If through the use of a program evaluation a middle school counselor can show that the study skills group he or she runs has decreased failure rates among participating students, his or her administrator may be more likely to encourage that activity over lunchroom duty or some other ancillary activity.

Some school counselors also experience what they perceive to be a lack of administrator(s) support for conducting program evaluations. The Transforming School Counseling Initiative (TSCI) sponsored by the Education Trust suggests that school counselors play a crucial and central role in increasing student achievement. The TCSI encourages administrators to view their school counselors not as support personnel whose role it is to react to crises as they occur, but instead as proactive and key players in a team of educators whose purpose it is to provide rigorous academic preparation for all students (Education Trust, 2002).

If school counselors are to overcome some of the barriers to conducting program evaluations, they must first have enough foundational knowledge to effectively utilize strategies to determine the impacts of their school counseling programs. The Accountability Bridge Model is a framework developed by the authors that can aid in the facilitation of both conducting program evaluations and communicating the results.

Accountability Bridge Model

The Accountability Bridge Model (see Figure 1) is designed to aid school counselors in the planning, delivery, and assessment of the effectiveness and impact of their services. In the model, counseling evaluation is organized into two reoccurring cycles (counseling program evaluation cycle and counseling context evaluation cycle) representing an ongoing refinement of services based on outcomes, stakeholder feedback, and needs of the population served. First, the counselingprogram evaluation cycle involves the planning and implementation of strategies, interventions, and programs, the monitoring and refinement of those programs, and the assessments of previously identified outcomes. Four stages are involved in this cycle.

Figure 1

Accountability Bridge Counseling Program Evaluation Model

In the program planning stage, information is gathered during needs assessments and identification of service objectives, and counseling programs and services are planned and developed. At this stage, school counselors identify interventions and programs to be implemented as well as the resources required to carry out the implementation. At this stage as well, school counselors need to purposefully plan their means to assess outcomes. Means for assessing outcomes could include pre-post instruments, performance indicators, and checklists. In addition, available school- based data, self-report data, and observable data can be used (Gysbers & Henderson, 2000; Studer & Sommers, 2000).

During the program implementation stage, school counselors initiate programs and services. This stage is sometimes referred to as formative evaluation because the delivery or services is shaped by input from the context evaluation cycle. In the program monitoring and refinement stage, school counselors determine if adjustments need to be made to the program or interventions based on preliminary data and feedback.

Then, in the outcomes assessment stage of the program evaluation cycle, school counselors collect final data and analyze them to determine the outcomes of interventions and programs. At this stage, school counselors with limited knowledge and training in research methods may need to consult with a colleague or supervisor for help with analysis. The use of software programs for data analysis (e.g., SPSS, SAS, Microsoft Excel) can help expedite data interpretation and presentation.

The accountability bridge in the model represents the communicating of program outcomes to key stakeholders. Administrators, parents, central office personnel, students, other school counselors, and teachers represent some of the stakeholders who may have a key investment in the success of students. Communicating to stakeholders represents taking a proactive stance designed to help school counselors maintain support for their services and increase demand for their services (Ernst & Hiebert, 2002). Communication of results can take several forms including reports, summaries, presentations, and discussions.

The counseling context evaluation cycle represents the second cycle in the model. It includes obtaining feedback from stakeholders and using that feedback as well as results obtained in the assessment to plan for ongoing programs. In addition, needs assessments are conducted during this cycle so the program objectives are tied to identified needs in the population being served.

During the feedback from stakeholders stage, school counselors actively solicit feedback based on communicated results. When invested parties feel they have a voice in the planning and implementation of needed services, they are more likely to be supportive of ongoing efforts to improve those services (Ernst & Hiebert, 2002). After feedback, school counselors engage in strategic planning which can include an examination and possible revision of the mission and purpose of the overall school counseling program. This stage is representative of the context in which school counseling programs occur, and takes into account the program’s impact on the overall mission and goals of the school environment.

The needs assessment stage can provide school counselors with critical information that redesigns and redefines the overall school counseling program and services offered within it. Needs assessments include not just needs of the identified population, in this case the students, but also needs of other stakeholders, such as administration, parents, and teachers. Comprehensive needs assessments gather information from multiple sources and are planned with a clear purpose in mind (Royse, Thyer, Padgett, & Logan, 2001).

The identification of service objectives should be based on prior outcomes of counseling services, stakeholder feedback strategic planning, and results of needs assessments. A key component of program implementation is building interventions and strategies that have a clearly designed purpose and objective. If implemented programs do not have identified goals, they cannot be adequately evaluated for effectiveness. Once service objectives have been established, the entire evaluation cycle begins again with information from the counseling context evaluation cycle feeding back into the program planning stage of the counseling program evaluation cycle.

Implications

With the emphasis on accountability both in education in general and school counseling, specifically, school counselors can no longer question the need for evaluating their programs. By conceptualizing program evaluation as a collaborate process, school counselors may be more interested and motivated to participate in program evaluations. The Accountability Bridge Model provides school counselors with a framework to involve key stakeholders, take a proactive stance when demonstrating the impact of their programs, and utilize practical evaluation and assessment methods to analyze outcomes.