Understanding Mental Health Scale

In 2014, over 20 million students from increasingly diverse backgrounds were pursuing degrees from a U.S. university or college (Kena et al., 2016). As expected, this heterogeneous population faces some psychosocial risk factors as they adjust to college life (Kitzrow, 2009). Some students exhibit complex mental health disorders (MHDs; Much & Swanson, 2010) stemming from a variety of biopsychosocial sources, including dysfunctional family relationships, substance abuse, evolving social norms, impacts of technology, unwanted sexual experiences, and college adjustment issues (Gallagher, 2012; Young & Calloway, 2015). With the rise in the number of students with MHDs, campus mental health professionals are further stretched to provide a variety of effective interventions (Holm‐Hadulla & Koutsoukou‐Argyraki, 2015; Kalkbrenner, 2016). Furthermore, the rate at which college students with MHDs seek counseling services is concerning (Holm‐Hadulla & Koutsoukou‐Argyraki, 2015; Hunt & Eisenberg, 2010).

College Student Help‐Seeking Behavior

The synthesized findings from past investigations (Holm‐Hadulla & Koutsoukou‐Argyraki, 2015; Hunt & Eisenberg, 2010) have indicated that a considerable proportion of college students with MHDs do not receive counseling or other forms of important treatment. Many college students lack information about the resources available for individuals with MHDs (Albright, Kognito, & Schwartz, 2017; Becker, Martin, Wajeeh, Ward, & Shern, 2002; Eisenberg, Golberstein, & Gollust, 2007; Hunt & Eisenberg, 2010). For example, Eisenberg et al. (2007) found that 59% of a sample (N = 2,785) of college students reported not knowing what counseling services were offered on campus. Similarly, Dobmeier, Kalkbrenner, Hill, and Hernández (2013) reported that 50% (N = 114) of their student sample were unaware of the university‐related mental health services available to them.

The positive relationship between students' awareness of the warning signs for mental health issues and positive well‐being outcomes has been documented in the literature (Becker et al., 2002; Clough & Casey, 2015). Recent research extended this finding to a more specific relationship between students' awareness of MHDs and the types of referrals they would make for a classmate who was at risk for mental health issues (Kalkbrenner & Hernández, 2017). Students with a high awareness of the signs of MHDs were significantly more likely to report a willingness to refer a classmate who was showing signs of an MHD to facilitative resources (e.g., counseling) compared with students with low awareness of MHDs. The characteristics of mental disorders can reveal themselves in various ways, making the classification of warning signs for MHDs a complex task. Achenbach (1978) was among the seminal researchers to address this complicated task.

In their influential work, Achenbach (1978) and Achenbach and Edelbrock (1979) found that the symptomatology of youth living with mental health issues tended to manifest in either internal or external dimensions. The internalizing dimension involves the inward expressions of mental health distress, including, for example, feelings of anxiety, depression, and withdrawal. The externalizing domain includes the outward expression of mental distress (e.g., aggression, oppositional defiance, delinquency, and hyperactivity). Achenbach's internal and external classification system has been applied and systematically evaluated in the literature for almost 40 years.

Achenbach's Internal and External Classification System

Achenbach's (1978) classification system for mental health issues is well documented in the counseling and clinical psychology literature with over 75,000 published peer‐reviewed articles that included internalizing or externalizing as keywords in studies with children (e.g., Achenbach, Ivanova, Rescorla, Turner, & Althoff, 2016; Cohen, Gotlieb, Kershner, & Wehrspann, 1985; Sink, 2011) and in studies with emerging adults (e.g., Ritchie et al., 2013; Trudeau, Spoth, Randall, Mason, & Shin, 2012). Achenbach's classification system was also used by the Diagnostic and Statistical Manual of Mental Disorders (5th ed., DSM‐5; American Psychiatric Association [APA], 2013) Task Force to cluster the symptomatology of some mental disorders “according to what has been termed internalizing and externalizing factors [which] represents an empirically supported framework” (p. 13). Furthermore, the internal and external classification system is an especially versatile framework for conceptualizing mental health issues in that the system is based on two “broadband dimensions” that are represented by “narrowband dimensional symptoms” (Achenbach et al., 2016, p. 648). Achenbach's internal and external classification system has also appeared in the measurement literature.

Achenbach et al. (2016) identified authors of 4,870 peer‐reviewed articles who used 49 different instruments to measure internal and external classification systems for mental health issues. However, among these 49 instruments, only six possessed adequate reliability, validity, and normative data. Also, the existing research on the internal and external dimensions of mental health appears limited to individuals with firsthand experience of mental distress symptoms. Furthermore, the college counseling literature appears to be lacking in research on Achenbach's (1978) classification system.

Rationale and Purpose of the Current Study

Findings from the literature indicate that regular mental health screening is related to positive mental health outcomes among high school and college students who are at risk for adverse social‐emotional problems (Alschuler, Hoodin, & Byrd, 2009; Hill, Yaroslavsky, & Pettit, 2015). Several college‐age mental health surveys exist (Wei, McGrath, Hayden, & Kutcher, 2015); however, the majority of these measures lack a strong theoretical background and psychometric support. Achenbach et al. (2016) identified a need for future psychometric researchers to develop and validate psychometrically sound scales based on the internalizing and externalizing dimensions. Furthermore, research is lacking on the extent to which the internalizing and externalizing framework of mental health symptoms can be used to measure and increase awareness about mental health, especially among the college student population. Achenbach's (1978) classification system might have a variety of benefits for promoting college student mental health.

Achenbach's (1978) classification system has the potential to provide a clear and simple structure for helping college students identify warning signs of mental distress. Specifically, Achenbach's classification system is especially valuable for classifying the symptomatology of comorbid mental disorders (Achenbach et al., 2016). Thus, Achenbach's classification system has the potential to apply to the college student population in that comorbid mental disorders are common on college campuses (Kalkbrenner, 2016). A validated measurement instrument based on Achenbach's classification system has the potential to provide a parsimonious framework for classifying the warning signs of mental disorders in college students. There is a clear need for this psychometrically sound instrument for measuring college‐age students' understanding of mental health issues because of the growing prevalence and complexity of MHDs among college students.

The purpose of the current study was to develop and validate the Understanding Mental Health Scale (UMHS), a user‐friendly questionnaire designed to assess college students' understanding of internalizing and externalizing mental health symptoms as well as campus mental health support services. To establish the psychometric properties of the UMHS, we sought to determine the measure's reliability and validity. In particular, the following research questions were pursued:

Research Question 1: What is the underlying factor structure or dimensionality of the UMHS?

Research Question 2: Are the scale and its subscales reliable?

Research Question 3: Are there demographic differences in participants' awareness of MHDs?

Method

Participant Characteristics

Participants (N = 350) ranged in age from 18 to 58 (M = 22, SD = 4.4) with 55.7% (n = 195) self‐identifying as female. Men (n = 154) composed 44% of the sample. One student marked “other gender” (0.03%). At the time of data collection, 26% of participants (n = 91) had previously attended at least one session of personal counseling. Respondent ethnicities were distributed as follows: Black or African American, 45.1% (n = 158); American White or Caucasian, 36.6% (n = 128); multiethnic, 6.9% (n = 24); Hispanic or Latino, 5.1% (n = 18); other, 4.3% (n = 15); American Indian/Alaska Native, 1.1% (n = 4); not reported, 0.6% (n = 2); and Native Hawaiian or Pacific Islander, 0.3% (n = 1). The demographic profile of our sample for gender and ethnicity is consistent with the demographic profile of the overall university. This university is highly ranked in ethnic diversity, with more than 50% of the student population identifying with minority ethnic backgrounds.

Procedure

The purpose of this survey is to improve communication between college/university students and university faculty and staff related to students' awareness about mental health issues below are examples of behaviors that may or may not be warning signs that someone is struggling with a mental health issue. Please read each statement carefully and select the response that most accurately reflects your view. There are no correct answers.

Instrument Development

The process for developing a new questionnaire is multistage (DeVellis, 2016). The initial set of 45 Likert response items, ranging from 1 (strongly disagree) to 5 (strongly agree), was generated based on Achenbach's (1978) internal and external classification system, the DSM‐5 (APA, 2013), and Sink (2011). Specifically, we reviewed the literature and identified the following DSM‐5 diagnostic categories that are common diagnoses among college students with MHDs: depressive disorders, anxiety disorders, alcohol use disorder, and eating disorders (Best Colleges, n.d.; Holm‐Hadulla & Koutsoukou‐Argyraki, 2015; Knopf, Park, & Mulye, 2008). We also developed the UMHS items by referring to Sink's description of how the symptomatology from the DSM‐5 diagnostic categories (listed earlier) fit within Achenbach's internal and external classification system. Five distractor items that were by design mainly not related to mental health issues (e.g., “feels bored at work”) were added as a validity check. We predicted that these items should not correlate strongly with the other items that were related to mental health.

Second, the items were sent to three expert reviewers to establish face and content validity. These scholars had more than 60 years of combined experience working in college counseling, school counseling, and student affairs settings. The reviewers suggested a variety of revisions to clarify and improve the wording of 30 items. The 50‐item UMHS (45 content items plus the five distractor items) was then pilot tested with a developmental sample of 19 undergraduate college students who were attending a local university. Pilot study participants were asked to complete the measure and offer any feedback on how the instrument could be improved. Respondents suggested further alterations to the questionnaire to improve readability. Third, demographic questions were created. The final measure, comprising 10 demographic items, 45 mental health–related items, and five distractor items, was then administered to a large sample of college students to maximize the respondents‐to‐items ratio (Mvududu & Sink, 2013).

Statistical Analyses

Following the recommendations provided by leading psychometricians (Nunnally & Bernstein, 1994) and counselor educators (Dimitrov, 2012; Mvududu & Sink, 2013), the dimensionality of the UMHS questionnaire was explored using principal factor analysis (PFA). Factor rotation criteria were eigenvalues (Λ) greater than 1, the percentage of variance accounted for by each derived factor (≥ 5%), the magnitude of item commonalities (≥ .40), a review of the scree plot, and parallel analysis results. An oblique rotation (oblimin, Δ = 0) was deployed, for it was assumed that the derived factors would have at least a low to moderate correlation (Achenbach et al., 2016). A follow‐up multivariate analysis of variance (MANOVA) was computed on derived factor scores using relevant demographic characteristics as independent variables. Descriptive discriminant analysis is generally recommended as the post hoc procedure for a significant main effect with three or more levels (Tonidandel & LeBreton, 2013). However, Spector (1977) suggested that univariate analyses of variance (ANOVAs) are the most appropriate follow‐up to significant MANOVAs, particularly when researchers seek to identify which variables yield the most significant contribution to group differences in the overall model. Hence, both post hoc analyses are reported on the basis of the recommendations of statisticians (e.g., Borgen & Seling, 1978; Field, 2013). Finally, Bonferroni adjustments were made to the alpha levels to minimize the potential for Type 1 errors.

Results

Upon completion of data collection, we examined unusual or problematic response patterns, missing data, and the parametric nature of the item distributions. An SPSS (Version 25) missing values analysis revealed that less than 2% of the data were omitted for all UMHS items. Missing values were replaced with the item mean (Field, 2013). Descriptive statistics were computed for the 45 UMHS items (see Table 1). Skewness and kurtosis revealed that all the items on the UMHS met the parameters of a normal distribution because values were all within the range of ± 1.

Table 1

Descriptive Statistics for Initial Awareness Items

Note. N = 350. The letter R following an item number indicates a reverse‐scored item.

Interitem Correlation Matrix

Interitem correlations were computed, ranging from –.34 to .88. Three UMHS items were removed from the data set, failing to correlate minimally (r ≤ .30) with at least half of the other items. As expected, the five distractor items correlated weakly (r < .30) with more than half of the other items. Preliminary internal consistency reliability was computed on the remaining 42 UMHS items, generating an overall Cronbach's coefficient alpha of .94. Factor analysis aims to condense the number of items on a questionnaire while maximizing the reliability of the measure (Mvududu & Sink, 2013). Reliability analysis indicated that removing 13 additional UMHS items improved the overall internal consistency to .96 (29 items). The removal of these items also reduced the length of the questionnaire, thus minimizing respondent fatigue. The remaining 29 items were renumbered in chronological order. Bartlett's test of sphericity, χ2(990) = 12,270.84 (p < .01), and the Kaiser–Meyer–Olkin test of sampling adequacy (KMO = .96) indicated that the correlation matrix was factorable.

Exploratory Factor Analysis

A PFA was conducted on the 29 items, revealing an initial six‐factor solution using the Kaiser criterion (Λ > 1.00). A total of 66% of the total variance was explained by these factors, with only two factors accounting for at least 5% of the variance. Inspection of the scree plot and parallel analysis results supported the appropriateness of rotating two factors. The following factor retention criteria (Beavers et al., 2013; Mvududu and Sink, 2013) were used: factor loading > .40, commonality (h2) > .30, and minimal cross‐loadings. The magnitude of the item commonalities was acceptable, ranging from .36 to .86. The correlation between the two factors was minimal (r = .07). The weak correlation between factors, coupled with the coherent factor pattern depicted in Table 2, provided preliminary support for the construct validity of the UMHS. The final factor pattern produced a simple structure with clearly interpretable factor loadings. We reproduced the correlation matrix to verify the 2‐factor solution.

Table 2

Principal Factor Analysis (Pattern Matrix) Results Using Oblique Rotation

Note. N = 350. Factor loadings over .40 appear in bold and mark the particular factor. RFA = Risk‐Factor Awareness; ESA = External Support Awareness.

We selected the pattern rotation matrix to interpret, for it depicted the most coherent factor structure with minimum cross‐loadings (see Table 2). The first dimension accounted for 48% of the variance of the correlation matrix and included Items 1 to 21. This dimension was named Risk‐Factor Awareness (RFA) because the items marking this factor described a warning symptom of a mental health issue. The second factor accounted for 13% of the variance and was marked by eight items (22 to 29). This dimension was titled External Support Awareness (ESA) on the basis of item content. In other words, each statement reflected some aspect of respondents' knowledge of resources for mental health issues or their willingness to seek support for mental distress.

Analyses on Demographic Variables

To further establish the reliability and validity of the final version of the UMHS, we rechecked awareness items for internal consistency. Overall score (29 items, α = .95) and individual factor reliabilities were strong (Factors 1 and 2, α = .96 [21 items] and .85 [8 items], respectively). Subsequently, a 2 (Gender) × 3 (Ethnicity) MANOVA was computed to answer the third research question (Are there demographic differences in participants' awareness of MHDs?). The independent variables were gender (male, n = 154, and female, n = 195) and ethnicity. Ethnicity categories were as follows: 45% Black or African American (n = 158) and 37% American White or Caucasian (n = 128). To ensure sample sizes were large enough to make group comparisons, the remaining 62 respondents from all other ethnicities (18%) were aggregated in a third group (other ethnicities). The dependent measures consisted of a mean composite score from each of the two derived factors. A Bonferroni correction was applied to protect against the familywise error rate.

A significant main effect emerged for gender, F(5, 338) = 6.01, p < .01, Wilks's Λ = .97, η2p = .034. Female participants (M = 3.6, SD = 1.16) scored higher on RFA compared with male participants (M = 3.3, SD = 1.04), F(1, 338) = 7.86, p <.01, η2p = .053. Women (M = 3.7, SD = 0.72) also scored higher on ESA compared with men (M = 3.5, SD = 0.85), F(1, 338) = 5.24, p <.05, η2p = .02. A significant main effect also emerged for ethnicity, F(5, 338) = 5.81, p < .001, Wilks's Λ = .96, η2p = .033. The post hoc univariate ANOVA revealed that participants who identified as Caucasian (M = 3.7, SD = 0.91) scored higher on RFA compared with participants who identified as African American (M = 3.2, SD = 1.20) and other ethnicities (M = 3.32, SD = 1.22), F(2, 338) = 9.5, p <.001, η2p = .053. As a post hoc analysis, the descriptive discriminant analysis supported these findings with two significant discriminant functions emerging from the analysis. The first function discriminated between groups, Wilks's Λ = .95, χ2 = 19.51, df = 4, canonical correlation = .23, p < .01. The first and second functions accounted for 93.5% and 6.5% of the variance, respectively. The correlations between the latent factors and discriminant functions showed that the RFA dimension loaded more strongly on the first function, r = .96, than the second function, r = .31. The ESA dimension loaded higher on the second, r = .93, than the first function, r = –.38. In summary, the RFA dimension contributed the most to ethnic group separation. The mean discriminant scores on the first function by participant ethnicity were as follows: Caucasian, 0.29; African American, –0.23; and other, –0.02.

A significant gender by ethnicity interaction effect was found, F(5, 338) = 2.56, p < .05, Wilks's Λ = .97, η2p = .016. Specifically, male participants who identified as African American (M = 3.78, SD = 0.88) scored higher on ESA compared with male participants who identified as Caucasian (M = 3.41, SD = 0.78) and non‐Caucasian (M = 3.40, SD = 0.90), F(5, 338) = 1.95, p < .05, η2p = .019.

Discussion

The current investigation examined the development and initial validation of the UMHS, a new instrument to appraise college students' awareness of mental health symptoms and potential coping resources. The results are promising in that a variety of factor retention criteria demonstrated a coherent 2‐factor structure. The first UMHS dimension, RFA, reflects the extent to which students are aware of the key warning signs of mental health issues (e.g., Item 9, “Takes a higher dose of prescription medication than is prescribed”). The second factor, ESA, refers to the degree to which students are aware of external resources available to them or specific action steps they may take. This latter factor is represented by these sample items: Item 23, “I know where to go in the local community to access counseling services” and Item 26, “I would seek personal counseling if I thought that I was experiencing symptoms of a mental health issue.”

As discussed earlier, Achenbach's (1978) bidimensional model of mental psychopathology was used, in part, to formulate the UMHS items. It was therefore anticipated that two factors representing internalizing and externalizing awareness dimensions would emerge from the PFA. Instead, the items reflecting Achenbach's internalizing and externalizing symptoms loaded on Factor 1 (RFA). For example, items reflecting an awareness of internalizing symptoms (e.g., Item 3, “Thinks a great deal about ending one's life,” and Item 19, “Restricts food intake”) marked this dimension. Similarly, externalizing awareness items (Item 14, “Performs actions to harm others deliberately,” and Item 16, “Intentionally destroys property”) loaded on the RFA factor. This finding suggests that college students did not differentiate symptoms into two categories as proposed by Achenbach. It is plausible that respondents viewed items representing different symptoms of MHDs as occurring within (internal to) the student. Thus, they loaded on a single RFA dimension. Not surprisingly, items that marked the ESA dimension were composed of respondents' knowledge or awareness of external resources (e.g., Item 22, “I know where to go on campus to access counseling services,” and Item 27, “Professors on this campus are supportive of students who are receiving mental health services”).

Reliability and Validity of the UMHS

We provided evidence for the reliability and validity of the UMHS. The range of interitem correlations (rs = .35–.80) and strong internal consistency coefficients (overall α = .95 [29 items], Factor 1 α = .96 [21 items], Factor 2 α = .85 [8 items]) suggest that the UMHS is a reliable questionnaire, measuring the two aspects of the same construct: understanding of mental health. Content validity was established in two primary ways. First, UMHS items were developed using the research based on the DSM‐5 and Achenbach's (1978) theoretical work. Second, pilot testing and expert reviews strengthened item quality and interpretability. Preliminary construct validity was evidenced by the coherent factor structure emerging from the PFA and the strong internal consistency coefficients.

The multivariate findings provided evidence for the measure's discriminant validity. Specifically, demographic group comparisons revealed that female students were significantly more aware than male students of warning signs of and external resources for MHDs. This finding is consistent with previous investigations, where female students had a greater understanding of mental health issues and available resources compared with male students (Becker et al., 2002; Dobmeier et al., 2013). Our results also revealed that students who identified as Caucasian scored higher on the RFA subscale when compared with students who identified as African American. Similarly, some previous investigators identified African American college students as a vulnerable population for poor mental health outcomes (Kingkade, 2017; McClain et al., 2016). In particular, African American college students are less likely to attend counseling and more likely to experience minority status stress (Kingkade, 2017; McClain et al., 2016). Our findings offer a potential explanation for African American students' underutilization of counseling services. It is possible that African American students' unawareness of warning signs for MHDs might be contributing, in part, to their underutilization of counseling services. Future research is needed to investigate possible explanations for this finding.

Implications for College Counseling Practice

The daily life of counselors involves ensuring the welfare of clients through support and empowerment (American Counseling Association, 2014). Both UMHS subscales can provide college counselors and administrators with valuable information about their students' understanding of mental health issues. The RFA subscale can be used to identify the extent to which college students at universities are aware of warning signs for mental distress. This might be particularly valuable considering that MHDs are especially likely to manifest themselves as students are adjusting to postsecondary education (Young & Calloway, 2015). The UMHS can be used as a screening tool for identifying warning signs and resources for MHDs.

Consequently, it is perhaps beneficial to administer the RFA subscale at new‐student orientations. College counselors can use the results to guide the content area(s) of awareness initiatives for increasing students' knowledge of warning signs for mental distress. The ESA subscale can be deployed by college counselors and administrators to identify the extent to which students are aware of both on‐campus and off‐campus resources for MHDs. However, logistically, surveying all incoming students at orientation may be challenging. Technology (e.g., mobile health communication) can be an effective method for communicating with college students about mental and physical wellness (Johnson & Kalkbrenner, 2017). Many online survey platforms (e.g., Qualtrics) might make distribution more feasible by allowing the administration of the UMHS using a cellular device and a quick response (QR) code. College counselors might post flyers or handouts with a brief description of the UMHS and a QR code for accessing the questionnaire at various locations during new‐student orientations (e.g., near the sign‐in/registration table, restrooms, dining/snack areas). See 2013) for a short video tutorial on creating QR codes. The QR code might also be used to distribute the UMHS during a designated time during the new‐student orientation day. Only a small amount of time would have to be reserved as the UMHS can be completed in 5 to 8 minutes. It might also be feasible for college counselors to ask incoming students to take the UMHS when they register for the new‐student orientation. This would give college counselors time to score and interpret the results before the day of orientation.

Previous investigators have identified peer‐to‐peer referral systems as an effective strategy for connecting students to college counseling services (Indelicato, Mirsu‐Paun, & Griffin, 2011). However, the findings of a recent national survey on college student mental health revealed that a substantial number of college students (51%) do not feel prepared to recognize warning signs of MHDs (Albright, Kognito, & Schwartz, 2017). The UMHS can be used by college counselors to structure more efficient peer‐to‐peer referral networks. Specifically, college counselors can administer the UMHS to a representative sample of students. The results can be used to identify and create a list of warning signs and resources for MHDs that can be disseminated to the college student population. The composite score on each subscale will provide college counselors with more representative information about students' understanding of mental health than any single item because each subscale represents a latent variable or overall score of students' awareness of warning signs and resources for mental distress. This information can help college counselors determine the extent to which it might be valuable to create a list or directory of warning signs (RFA subscale) and resources (ESA subscale) for MHDs of which students on their campus are especially unaware. Kalkbrenner (2016) provided a customizable resource list template for mental health support services. The list can be distributed in a variety of ways to increase the probability of student exposure (e.g., posted in the student union, included in course syllabi, and added to university counseling webpages; Kalkbrenner, 2016; National Alliance on Mental Illness, 2012). The results of the UMHS can also be used to devise programs that are intended to increase students' awareness of warning signs and resources for MHDs. Programs can be delivered in person or online and include short summaries of warning signs for MHDs (RFA subscale items) and resources for MHDs (ESA subscale items) of which students were unaware.

Group gender and ethnicity disparities in mental health are relatively well documented on college campuses (Cook et al., 2014; Dobmeier et al., 2013; Eisenberg, Hunt, & Speer, 2012; Lee, Xue, Spira, & Lee, 2014). However, there is a scarcity of research exploring the interaction between gender and ethnicity in college students' understanding and awareness of mental health. Although additional research is needed, the significant interaction effect reported in the present study provides useful information in this regard. Specifically, the mean scores on the ESA scale were virtually identical for all gender‐by‐ethnicity groups. However, male participants who identified as Caucasian or with the other ethnicity group scored significantly lower on the ESA scale. On the basis of this finding, college counselors may want to specifically target mental health awareness initiatives toward students who identify as male and certain minority groups.

Limitations and Suggestions for Future Research

The findings should be considered within the context of the study's limitations. In particular, systematic biases (e.g., social desirability, method variance, and monomethod) affect the validity of questionnaire scales (DeVellis, 2016). Our results revealed that 26% of the participants (n = 91) in our sample had attended at least one session of personal counseling before data collection. However, data were not collected about the number of participants in our sample who were living with a diagnosed MHD at the time of data collection. It is possible that living with MHDs and participating in treatment for MHDs might have a psychoeducational impact on students, enhancing their awareness of resources and warning signs of MHDs. Future research is needed to investigate this possibility. Reliance on self‐report attitudinal questionnaires for the measurement of dependent and independent variables precluded drawing causal conclusions. Moreover, the generalizability of findings is also restricted to the local student population. It is possible that our results captured participants' understanding of mental health on one specific campus.

A variety of college student subpopulations are especially vulnerable to mental health challenges. For example, community college students, international students, first‐generation college students, and distance learning students are at higher risk for developing mental health problems, encountering social obstacles, and experiencing academic impairments (Akanwa, 2015; Fortney et al., 2016; World Health Organization, 2012). Moreover, mental health issues remain pervasive among high school students and young adults in the general population (Eisenberg, Hunt, Speer, & Zivin, 2011; Fortney et al., 2016; Michaud et al., 2006). The UMHS and its conceptual framework may apply to these populations once its psychometric properties and construct validity are firmly established. Our findings suggest that the psychometric properties of the UMHS and its dimensions were estimated adequately with a large sample of college students who were enrolled in a 4‐year university at the time of data collection. As the next research steps, UMHS's factor structure and invariance should be confirmed (confirmatory factor analysis) with new groups of undergraduate and graduate students, and follow‐up application studies in college counseling centers should be initiated. Future researchers can also extend this line of research by testing the criterion validity of the UMHS. In particular, future investigations on the extent to which college students' scores on the RFA and ESA subscales predict their frequency of peer‐to‐peer referrals to the counseling center are needed. Future researchers can also test the convergent validity of the UMHS with other well‐established instruments, such as the Beck Anxiety Inventory and the Beck Depression Inventory.

Conclusion

This purpose of this study was to examine the psychometric properties of the newly developed UMHS. Findings reported here are encouraging and support the questionnaire's initial reliability, as well as its content, discriminant, and factorial validity. The measure's items yielded a coherent factor structure comprising two underlying dimensions related to college students' understanding of mental health issues. At this stage of its development, the UMHS can be used by college counselors and other related helping professionals to screen undergraduate students' level of awareness of the risk factors associated with mental health problems and potential external support options. With this information, college counselors can improve peer‐to‐peer referral systems and devise programs to increase students' awareness of warning signs and resources for MHDs.