Measuring Civic Orientation in Undergraduate Students

• In Event: Integrating Social-Emotional Learning in Higher Education for Equity, Well-Being, and Civic Engagement

Abstract

Theoretical Framework
Higher education institutions are increasingly turning to wellbeing models and programming to ameliorate growing mental health challenges among undergraduate students. The models used to underpin these programming efforts vary widely, and they tend to be founded on objectivist frameworks that list a number of wellbeing “dimensions”. Because many schools’ missions include a civic focus, some of these models include eudaimonic constructs like civic orientation or social justice. Civically oriented wellbeing dimensions show promise for imparting lifelong benefits: engaging in civic behavior as an adolescent or young adult is associated with better income, education, and mental health outcomes later in life (Ballard, Hoyt, & Pachucki, 2018).

Schools that implement models with these eudaimonic constructs face a dearth of well-researched measures, which impairs their ability to develop and evaluate programming. A paucity of funding for measure development research means that most measure-development efforts are not able to subject their measures to the lengthy and repeated testing demanded by best-practice guidelines (e.g., Borsboom, Mellenbergh, & van Heerden, 2004; Hughes, 2018; Kane, 2013; Messick, 1995).

Objectives
The objective of this research was to develop a brief scale of civic orientation that is valid for use with young adult undergraduate students from multiple demographic identities. We will present results from the development of a large-scale, multi-dimensional measure of wellbeing with a focus on Civic Orientation scales. This measure has been in development since 2014 and has been able to undergo many of the complex validity evaluations described above. The scales are founded on Crystal and DeBell’s definition of civic orientation as a “favorable orientation toward, and participatory experiences in, the civic/political domain” (p. 114, Crystal & DeBell, 2002). This broad definition allows for the inclusion of items evaluating affect, behavior, and cognition around civic orientation.

Methods & data sources
We will present summary results from four rounds of cognitive interviewing and five pilot administrations that took place between Fall 2015 and Spring 2019. Results from the cognitive interviews will describe undergraduate students’ understanding of terms commonly used in measures of civic orientation (e.g., McFarland, Brown, & Webb, 2013; Pancer, Pratt, Hunsberger, & Alisat, 2007; Porter, 2013). Results from the pilot administrations will focus on a Spring 2019 administration of these items to 11,921 undergraduate students across 28 institutions in the US. These results will include CFA (Kline, 2015) and measurement invariance (Chen, 2007; Cheung & Rensvold, 2002; Vandenberg & Lance, 2000) analyses demonstrating strong, unbiased factors across gender, race/ethnicity, first-generation, or sexual orientation groups.

Findings & Significance
Our findings demonstrate that published measures developed without qualitative research can inadvertently incorporate language or constructs that are not meaningful to a young adult undergraduate audience. Our quantitative results provide validity evidence for the measure, which is designed to directly inform student affairs and other programming efforts. We will introduce some practices for translating the measure into equitable and inclusive programming, such as conducting disaggregated data analyses, allowing for multiple instantiations to civic orientation, and methods for sensitively collecting demographics.

Authors

• Nicole Brocato, Wake Forest University

• Laura Hix, Wake Forest University

• Parissa Jahromi Ballard, Stanford University

• Xinyu Ni, Wake Forest University