Developing a Developmentally Sensitive and Unbiased Measure of Civic Orientation for Use With Undergraduate Students

• In Event: Leveraging Social-Emotional Learning in Higher Education for Inclusive Excellence and Civic Engagement

Abstract

Objectives
The objective of this research was to develop a brief scale of civic orientation that is valid for use with young adult undergraduate students from multiple demographic identities.

Perspectives
Higher education institutions are increasingly turning to wellbeing models and programming to ameliorate growing mental health challenges among undergraduate students. The models used to underpin these programming efforts vary widely, and they tend to be founded on objectivist frameworks that list a number of wellbeing “dimensions”. Because many schools’ missions include a civic focus, some of these models include eudaimonic constructs like civic orientation or social justice. Civically oriented wellbeing dimensions show promise for imparting lifelong benefits: engaging in civic behavior as an adolescent or young adult is associated with better income, education, and mental health outcomes later in life (Ballard, Hoyt, & Pachucki, 2018).

Schools that implement models with these eudaimonic constructs face a dearth of well-researched measures, which impairs their ability to develop and evaluate programming. A paucity of funding for measure development research means that most measure-development efforts are not able to subject their measures to the lengthy and repeated testing demanded by best-practice guidelines (e.g., Borsboom, Mellenbergh, & van Heerden, 2004; Hughes, 2018; Kane, 2013; Messick, 1995).

We will present results from the development of a large-scale, multi-dimensional measure of wellbeing with a focus on Civic Orientation scales. This measure has been in development since 2014 and has been able to undergo many of the complex validity evaluations described above. The scales are founded on Crystal and DeBell’s definition of civic orientation as a “favorable orientation toward, and participatory experiences in, the civic/political domain” (p. 114, Crystal & DeBell, 2002). This broad definition allows for the inclusion of items evaluating affect, behavior, and cognition around civic orientation.

Methods & Data Sources
We will present summary results from four rounds of cognitive interviewing and five pilot administrations that took place between Fall 2015 and Spring 2019. We will provide more detailed results from cognitive interviews conducted in 2017 and 2018 in which undergraduate students described their understanding of terms commonly used in measures of civic orientation (e.g., McFarland, Brown, & Webb, 2013; Pancer, Pratt, Hunsberger, & Alisat, 2007; Porter, 2013). We will also detail results from a Spring 2019 administration of these items to 11,920 undergraduate students across 28 institutions in the US. Presented results will include CFA (Kline, 2015) and measurement invariance (Chen, 2007; Cheung & Rensvold, 2002; Vandenberg & Lance, 2000) analyses demonstrating strong, unbiased factors across gender, race/ethnicity, first-generation, or sexual orientation groups.

Findings & Scholarly Significance
Our findings demonstrate that published measures developed without qualitative research can inadvertently incorporate language or constructs that are not meaningful to a young adult undergraduate audience. Our quantitative results provide validity evidence for the measure, which is designed to directly inform student affairs and other programming efforts. We will introduce some practices for translating the measure into equitable and inclusive programming, such as conducting disaggregated data analyses, allowing for multiple instantiations to civic orientation, and methods for sensitively collecting demographics.

Authors

• Nicole Brocato, Wake Forest University

• Laura Hix, Wake Forest University

• Parissa Jarhomi, University of California - San Francisco

• Xinyu Ni, Wake Forest University