VISTAS articles are made available for historical reference only and are presented "as is." ACA does not guarantee or represent that the information is current, accurate or indicative of the original or intended quality. These materials are not maintained or updated and may contain outdated or incomplete information. Readers should exercise discretion and verify information independently before relying on it. We assume no responsibility for the use or interpretation of this content.
Article 30
Web-Based Research Tools and Techniques
Paper based on a program presented at the 2007 Association for Counselor Education and Supervision Conference, October 11-14, Columbus, Ohio.
According to Sedwick (2004), “the Internet is gaining in popularity as a research tool through the use of e-mail and the World Wide Web” (p. 35). Most people who use e-mail or “surf the Web” have received a solicitation to complete some type of web-based survey. Many of the early web-based surveys were from companies conducting marketing research. However, as noted by Siah (2005), “the speed, ease and cost of conducting an internet-based study has attracted an increasingly large number of researchers to the medium for data collection” (p. 115).
Advantages of Web-Based Research Techniques
Like Siah (2005), numerous researchers have noted a plethora of advantages of using the Internet to collect research data. The following table briefly summarizes reasons noted by various authors for collecting data through a web-based interface.
Web-Based Research Data Collection: Advantages
Ability to reach larger population (Betz Hobbs & Farr, 2004; Wright, 2005)
Capability to reach participants with certain physical disabilities (Wright, 2005)
Ease of completion by participants (Ahern, 2005)
Flexibility and control over format (Granello & Wheaton, 2004)
Improved accuracy and simplicity of data entry (Ahern, 2005; Granello & Wheaton, 2004; Van Selm & Jankowski, 2006)
Increased participation (Sax, Gilmartin, & Bryant, 2003; Van Selm & Jankowski, 2006)
Incorporation of rich media such as audio, graphics, and video (Tourangeau, Couper, & Conrad, 2004)
Interactive nature of Web (Sax et al., 2003)
Methodological rigor (Ahern, 2005)
Lower cost (Betz Hobbs & Farr, 2004; Fricker, Galesic, Tourangeau, & Ting, 2005; Granello & Wheaton, 2004; Parks, Pardi, & Bradizza, 2006; Tourangeau et al., 2004; Sax et al., 2003; Van Selm & Jankowski, 2006; Wright, 2005)
Popularity among certain populations such as college students (Carini, Hayek, Kuh, Kennedy, & Ouimet, 2003; McCabe, 2004; Sax et al., 2003; Van Selm & Jankowski, 2006)
Rapid access to participants (Betz Hobbs & Farr, 2004; Parks et al., 2006)
Saves time (Ahern, 2005; Granello & Wheaton, 2004; Sax et al., 2003; Wright, 2005)
Simplicity of administration (Betz Hobbs & Farr, 2004; Wright, 2005)
Disadvantages of Web-Based Research Techniques
Despite the number of advantages cited for using the Internet to collect research data, several investigators have expressed concerns related to using this approach. The following table briefly summarizes reasons noted by various authors for not collecting data through a web-based interface.
Web-Based Research Data Collection: Disadvantages
Concerns with data integrity (Sax et al., 2003; Wright, 2005)
Initial development time or costs (Ahern, 2005; Sax et al., 2003; Van Selm & Jankowski, 2006)
Issues related to data security (Sax et al., 2003)
Limited Internet access for some sub-populations such as nursing home residents, people with certain disabilities (Granello & Wheaton, 2004; Sax et al., 2003)
Technical troubles experienced by users (Ahern, 2005; Granello & Wheaton, 2004; Sax et al., 2003)
Methodological Issues
Whether collecting data using a web-based survey or any other method, researchers need to consider methodological issues related to the data collection technique being employed. However, the use of web-based survey tools for data collection has forced researchers to address methodological concerns that are unique to this electronic medium. The following table briefly summarizes methodological matters noted by various authors that must be addressed when conducting web-based research.
Web-Based Research: Methodological Issues
Ability to follow-up with participants who did not complete survey (Granello & Wheaton, 2004)
Capability to provide participants with immediate feedback such as individual response summaries (Sax et al., 2003)
Eliminate interviewer bias (Parks et al., 2006; Van Selm & Jankowski, 2006)
Increased generalizability of data (Ahern, 2005)
Lack of control over test setting (Ahern, 2005)
Limited generalizability of data (Granello & Wheaton, 2004; McGothlin, 2003; Sax et al., 2003; Stafford &
Goiner, 2007; Wright, 2005)
Measurement errors (Granello & Wheaton, 2004)
Subject recruitment bias (Ahern, 2005; Wright, 2005)
A methodological issue not explicitly cited in the literature that has been expressed to the authors during conference presentations on this topic is related to the challenges of receiving approval from Institutional Review Boards (IRBs) for research involving web-based data collection. Over the years, the authors have received reports from conference attendees of not receiving approval for web-based research projects due to IRBs’ apprehensions related to study participants’ inability to complete institutionally required informed consent documents. In general, it appears that some IRBs do not comprehend the unique nature of web-based data collection. This conclusion has some support from a recent study by Kotzer and Milton (2007) which reported that many investigators believed that IRBs do not understand their studies.
A paramount issue for many researchers is the ability to generalize results from a sample to a larger population. As noted above, various authors disagree concerning the generalizability of results from research designs that utilize web-based data collection techniques. When developing the methodology for a web-based research project, it is essential for researchers to address this issue in the study’s design.
Ethical Issues
In addition to addressing methodological concerns unique to online data collection, researchers must consider ethical issues that are of special concern when conducting web-based studies. The following table briefly summarizes ethical issues documented by numerous authors that must be considered when conducting web- based research.
Web-Based Research: Ethical Issues
Ability to address sensitive topics such as alcohol abuse, eating disorders, HIV (Link & Mokdad, 2005; Parks et al., 2006; Van Selm & Jankowski, 2006; Wright, 2005)
Ensuring anonymity to participants (Ahern, 2005; Sax et al., 2003)
Lack of ethical guidelines related to conducting web- based research (Hamilton, 1999)
Participant concerns about confidentiality (Ahern, 2005; Madge, 2007; Sax et al., 2003)
Personalization of invitations to participate in surveys related to sensitive topics (Heerwegh,Vanhove, Matthijs, & Loosveldt, 2005)
Promotes increased access to cultural groups (Ahern, 2005; Betz Hobbs & Farr, 2004)
Underrepresentation of racial or ethnic minority groups (Sax et al., 2003)
All researchers should be concerned about these ethical issues. However, with the special attention given to research in the American Counseling Association’s Code of Ethics (2005), it is important that counselor educators and supervisors adhere to these ethical principles when conducting web-based research.
As noted above, various authors disagree concerning the impact of web-based research on inclusion of participants from racial or ethnic minority groups. As counselors, this issue should be of particular concern and must be considered prior to conducting web- based research.
Web-Based Survey Tools
The increasing availability of web-based survey tools now provides researchers with a variety of options when selecting a tool to best meet the data collection needs for a particular project. However, having a broad understanding of the types of web-based data collection applications available can assist in narrowing the options from many applications to a few tools, then ultimately selecting the best product for the project.
Based on the collective experiences of the authors in designing numerous web-based surveys using a variety of web-based data collection applications, the following matrix provides researchers with a relative comparison of web-based survey tools.
Web-Based Survey Tools: Relative Comparison
Type of Tool | Hosting Location | User Control | Ease to Customize | Turnaround Time | Analysis Tools | Researcher’s Cost |
|---|---|---|---|---|---|---|
Web Hosted Survey Wizard | External | Medium to High | Low to Medium | Low | Low to Medium | Medium |
Web Survey Wizard | Internal | Medium to High | Low to Medium | Low | Low to Medium | Low |
Custom Design | Internal | Low | High | Medium to High | Low | Low |
Types of Tools
Web-based survey applications can generally be divided into the following three broad categories with great variation of features available for each tool within a category.
Web Hosted Survey Wizard. These applications are generally housed on a web server that is outside of the researcher’s organization. Companies make these tools available for use by businesses for market research as well for individuals who use these products for data collection. Most of these companies charge investigators to use these tools on a per respondent basis (i.e., more responses = more money).
Web Survey Wizard. These products are generally housed on a web server that is inside the researcher’s organization. These tools are similar to “hosted” options except that the institution maintains the hardware and software application. Most of these applications are one-time purchases with the organization paying an annual service fee for software upgrades and support.
Custom Design. Rather than purchasing a web survey product, some institutions have built their own applications in order to complete an individual web-based survey. Conceptually, building this type of tool is fairly straightforward because the product has two components: (a) a database for collecting the data, and (b) a web-based interface for data entry. However, in reality, designing custom tools can be both expensive and time consuming.
Issues to Consider
The factors involved in selecting a web-based survey tool will vary greatly from research project to research project. However, five general matters need to be considered when differentiating between products.
User Control. The ability for the researcher to manipulate the “look and feel” of survey items is often an important consideration. For example, the ease of editing the contents of a survey item or manipulating the layout of the survey is imperative. Many survey applications include templates of standard survey items (e.g., select only one, select all that apply, text response boxes) that the researcher uses to build the survey.
Ease to Customize. In many situations, templates of standard survey items are suitable. However, due to the nature of some research questions, standard survey items may not be appropriate for collecting some types of data. Therefore, it might be important for the researcher to use non-standard data collection items (e.g., rank order a series of items) that cannot be created using templates.
Turnaround Time. The timeframe between when a researcher receives initial approval to conduct a study (e.g., IRB, funding agency), and the time that the investigator needs to begin data collection is often an important consideration. This is especially true if the researcher needs to conduct any type of pilot test during the early stages of the project.
Analysis Tools. Many researchers prefer to extract data from the survey tool and conduct the analysis using a stand alone software application (e.g., SPSS). However, some web-based survey tools have built in reports that allow the investigator to review summary results while the survey is still active (i.e., no need to export data from the survey application and import it into another product). Most of these internal reports provide basic demographics (e.g., number of respondents, percentage responding “yes” to a certain item).
Researcher’s Cost. The cost of using varying survey applications will be dependent upon the arrangement the researcher negotiates with external vendors or organizational policies related to cost recoupment for use of institutional resources. For many research projects, cost considerations become a primary factor in selecting a survey tool.
The ultimate selection of an application for a web-based survey might be determined by considerations other than those outlined above. However, identifying the best product to collect data for a research project must always be carefully contemplated. Following are examples of web-based data collection applications.
http://www.activewebsoftwares.com
http://www.advancedsurvey.com
http://www.askget.com
http://www.classapps.com/SelectSurveyOverview.asp
http://www.freesurveysonline.com
http://www.hostedsurvey.com
http://www.infopoll.com
http://www.prezzatech.com
http://www.questionpro.com/web-based-survey- software.html
http://www.raosoft.com
http://www.supersurvey.com/?sezbcom_software
http://www.vovici.com (was http://www.websurveyor.com)
http://www.zoomerang.com
Note: Inclusion in this set of examples does not represent the authors’ endorsement of the product.
Conclusion
As outlined above, web-based data collection is growing in popularity for a number of reasons. However, prior to diving into web- based methodologies, it is important for researchers to weigh the advantages and disadvantages of this approach as well as be prepared to address methodological and ethical issues unique to this medium. Ahern (2005) concluded that the advantages outweigh the disadvantages. Nevertheless, each investigator needs to consider these factors individually and reach his or her own conclusion. Finally, if web-based data collection is selected as the preferred method for a particular study, researchers need to identify the best application available to collect the type of data needed for the particular project.
References
Ahern, N. R. (2005). Using the Internet to conduct research. Nurse Researcher, 13(2), 55-70.
American Counseling Association. (2005). ACA code of ethics.Available from http://www.counseling.org/Files/FD.ashx?guid=ab7c1272-71c4-46cf-848c-f98489937dda
Betz Hobbs, B., & Farr. L. A. (2004). Accessing Internet survey data collection methods with ethnic nurse shift workers. Chronobiology International, 21(6), 1003-1013.
Carini, R. M., Hayek, J. C., Kuh, G. D., Kennedy, J. M., & Ouimet, J. A. (2003). College student responses to web and paper surveys: Does mode matter? Research in Higher Education, 44(1), 1-19.
Fricker, S., Galesic, M., Tourangeau, R., & Ting Y. (2005). An experimental comparison of web and telephone surveys. Public Opinion Quarterly, 69(3), 370-392.
Granello, D. H., & Wheaton, J. E. (2004). Using web-based surveys to conduct counseling research. In J. W. Bloom & G. R. Walz (Eds.), Cybercounseling and Cyberlearning: An Encore (pp. 287-306). Greensboro, NC: CAPS Press.
Hamilton, J. C. (1999, December 3). The ethics of conducting social- science research on the Internet. Chronicle of Higher Education, 46(15), p. B6.
Heerwegh, D., Vanhove, T., Matthijs, K. & Loosveldt, G. (2005). The effect of personalization on response rates and data quality in web surveys. International Journal of Social Research Methodology: Theory and Practice, 18(2), 85-99.
Kotzer, A. M., & Milton, J. (2007). An education initiative to increase staff knowledge of Institutional Review Board guidelines in the USA. Nursing and Health Sciences, 9, 103-106.
Link, M. L., & Mokdad, A. H. (2005). Effects of survey mode on self-reports of adult alcohol consumption: A comparison of mail, web, and telephone approaches. Journal of Studies on Alcohol, 66(2), 239-245.
Madge, C. (2007). Developing a geographer’s agenda for online research ethics. (2007). Progress in Human Geography, 31(5), 654-674.
McCabe, S. E. (2004). Comparison of web and mail surveys in collecting illicit drug use data: A randomized experiment. Journal of Drug Education, 34(1), 61-73.
McGothlin, J. M. (2003). The infusion of Internet-based surveys and postal mail surveys. Journal of Technology in Counseling, 3(1). Retrieved October 16, 2005, from http://jtc.colstate.edu/ vol3_1/McGothlin/McGothlin.htm
Parks, K. A., Pardi, A. M., & Bradizza, C. M. (2006). Collecting data on alcohol use and alcohol-related victimization: A comparison of telephone and web-based survey methods. Journal of Studies on Alcohol, 67(2), 318-323.
Sax, L. J., Gilmartin, S. K., & Bryant, A. N. (2003). Assessing response rates and nonresponsive bias in web and paper surveys. Research on Higher Education, 44(4), 409-432.
Sedwick, J. L. (2004). A comparison of three data collection survey modes among Southern Baptist youth workers. The Journal of Youth Ministry, 3(1), 35-46.
Siah, C. Y. (2005). All that glitters is not gold: Examining the perils and obstacles in collecting data on the Internet. International Negotiation, 10, 115-130.
Stafford, T. F., & Goiner, D. (2007). The online research “bubble”. Communications of the ACM, 50(9), 109-112.
Tourangeau, R., Couper, M. P., & Conrad, F. (2004). Spacing, position, and order. Public Opinion Quarterly, 68(3), 368-393.
Van Selm, M., & Jankowski, N. W. (2006). Conducting online surveys. Quality & Quantity, 40(3), 435-456.
Wright, K. B. (2005). Researching Internet-based populations: Advantages and disadvantages of online survey research, online questionnaire authoring packages, and web survey services. Journal of Computer-Mediated Communication, 10(3), article 11.
Appendix I
Authors and Titles of Additional Articles Accepted for Inclusion in the ACA VISTAS Online Library
Addressing Grief and Loss Issues With Children and Adolescents of Military Families
Jacqueline Melissa Swank and E. H. Mike Robinson
Arena for Success: Metaphor Utilization in Equine-Assisted Psychotherapy
Sandra L. Kakacek
Body Dissatisfaction Among Gay Men: A Cultural Phenomenon
Kristin Meany-Walen and Darcie Davis-Gage
The Boy Code Betrays Me: Addressing Societal and Sex-Based Trauma in the Lives of Gay and Bisexual Men
Stacee Reicherzer, Jason Patton, and Alessio Pisano
Bridging the Professional Gap: Mentoring School Counselors-in- Training
Kelly Duncan, Robin Svendsen, Tobin Bakkedahl, and Lisa Sitzman
The CACREP Standards: How Much Do Students Know?
Livia M. D’Andrea and Leping Liu
Cognitive Appraisal and/or Personality Traits: Enhancing Active Coping in Two Types of Stressful Situations
Ming-hui Li
Compulsive Gambling “Action” Inventory
Valerie C. Lorenz
Counseling Students Learn Adventure Counseling as an Additional Mode of Therapy to Increase Their Repertoire of Counseling Skills Louise B. Graham
A Counselor’s Guide to Child Sexual Abuse: Prevention, Reporting and Treatment Strategies
Kenneth L. Miller, Marianne K. Dove, and Susan M. Miller
‘Crash’: Modernism Meets Postmodernism
Jerry A. Mobley
Cyberbullying and Cyberbalance: Cultivating a Respect for Technology
Barbara Trolley, Connie Hanel, and Linda Shields
Developing a Regional Supervision Training Program for School Counselors
Alan Bakes
A Discussion of Coping Methods and Counseling Techniques for Children and Adults Dealing With Grief and Bereavement Candice N. Slate and David A. Scott
Ecotherapy: Theoretical Foundations Leading to Clinical Work With Images and Dreams for Individual, Community, and Planetary Transformation
Patricia A. Sablatura
Everything Counselors and Supervisors Need to Know About Treating Trauma
Lindsay Bicknell-Hentges and John J. Lynch
Exploring Racial Variations and the Impact of Parental Attachments and Psychological Health Among Diverse College Students
Deneia Thomas, Keisha Love, and Kenneth M. Tyler
A Flexible Pedagogy for Counseling Supervision
Jerry A. Mobley
Grief Work: Its Contributions to Healthy Living
Lori A. Russell-Chapin and Rachel B. Bridgewater
Group Therapy to Build Strong Relationships for Same Sex Couples
Lamerial Jacobson, John Super, and Kara Pappalardo
Horses as Healers: Equine Facilitated Therapy for Grieving Children
Laura Strom and Jennifer Wilson
The Impact of Cyber Bullying: A New Type of Relational Aggression
Jennifer M. Johnson
The Impact of Relaxation Techniques on Third Grade Students' Self-Perceived Levels of Test Anxiety
Colleen M., Johnson, Heidi A. Larson, Steven R. Conn, Lincoln A. Estes, and Amanda B. Ghibellini
The Impact of Skills-Based Training on Counselor Locus of Control and Emotional Intelligence
Jill Packman, Marlowe Smaby, Cleborne Maddux, Craig Farnum, Colin Hodgen, Elisabeth Liles, and Becky Rudd
Is There A Magic Bullet? Pharmacologically Assisted Addiction Management: What Counselors Should Know
Benjamin P. Kelch
The ISLLC Standards: A Unifying Force in School Administrator and Counselor Preparation
Gene Wright and Neal D. Gray
Joining Forces for Students: School and Community Counselors Unite!
Rebecca N. Earhart and Sharon Mindock
Leading Mutual Aid Support Groups: Difficult Members and Other Challenges
Lawrence Shulman
Living With Learning Disabilities: Strategies for Family Support
Carol J. Dolan
Multiple Addictive Behavior Questionnaire (MABQ) Validation Project
Victoria L. Bacon and Theresa A. Coogan
Narrative Approaches in Sand Therapy: Transformative Journeys for Counselor and Client
Dee Preston-Dillon
Out of the Ivory Tower and Into the School-Based Practicum
Marielle A. Brandt and A. Jonathan Porteus
Outcome Study of a Community Based Training Clinic: Are We Serving Our Clients?
Darlene Daneker
Pharmacological Treatment of Childhood and Adolescent Depression: What School Counselors and School Psychologists Need to Know…
Carrie Lynn Bailey
Physical Wellness Self-Monitoring Project
Darren A. Wozny and Julia Y. Porter
Possible Selves: Concepts, Applications, and Implications for Career Practice and Policy
Anne Marshall and Fran Guenette
Predictors of Body Dissatisfaction Among Adolescent Females
Melissa Hall
The Professional Counselor and the Diagnostic Process: Challenges and Opportunities for Education and Training
Mary Beth Mannarino, Mary Jo Loughran, and Deanna Hamilton
Promoting International Counseling Identity: The Role of Collaboration, Research, and Training
Wendy J. Hoskins and Holly C. Thompson
Promoting Professional Development Among Students and Professional Counselors
John McCarthy, Amy J. Thompson, and Teresa E. Fernandes
Racial Ethnic Identity and Career Development Concerns of College Students From Immigrant African and Hmong Families
Aneneosa A.G. Okocha
The Relationship Between Psychological Birth Order and Romantic Relationships
Nicole A. Healy, Tammy H. Scheidegger, Amy L. Ridley Meyers, and Karen Friedlen
Situating Supervision in the Context of a Social Justice Paradigm: A Convergence of Bernard’s Discrimination Model and Social Justice
Melissa Odegard, Linwood G. Vereen, and Nicole R. Hill
Social and Emotional Needs of Gifted Students: What School Counselors Need to Know to Most Effectively Serve this Diverse Student Population
Carrie L. Bailey
Social Justice, Advocacy, and Counselor Education Pedagogy
Kaye W. Nelson, Marvarene Oliver, and Rochelle Cade
Strategies for Effectively Teaching Career Counseling
Aaron H. Oberman and Jeannine R. Studer
Support Systems for Parents of Children with Special Needs
Chiharu Hensley
Taking a Multifaceted Approach to Retirement
Vera S. Maass
That’s Just Plain Silly! Channeling Outrage to Champion Change
Roberta Neault
A Tool of Facilitating Courage: Hope Is a Choice
Mark Blagen and Julia Yang
Training Counselors to Be Consumers of Supervision
Irene Mass Ametrano, Devika Dibya Choudhuri, and Diane L. Parfitt
Traumatic Brain Injuries and Substance Abuse: Implications for Rehabilitation Professionals
David A. DeLambo, Kananur V. Chandras, Debra Homa, and Sunil V. Chandras
Uses of Bibliotherapy for Adoptive Children and Their Families
Claire A. Kavanaugh and Jody J. Fiorini
Verification- To Check or Not to Check: The Use of Criminal Background Checks in Counselor Training Programs
Gloria Dansby-Giles and Frank Giles
Wellness and Academic Performance of Elementary Students
Mary A. Hollingsworth
Women Counselor Educators: Level of Job Satisfaction While Raising Children
Carrie Alexander-Albritton, Nicole R. Hill, and Brooks Bastian Hanks