Validation of Grit Scales Across Multiple Samples: Restricted Recalibration and Multiple-Group Analysis

• In Event: 61.013 - Co-Evolution of Substantive and Methodological Research: The Case of Grit, Self-Regulation, and Motivation

Abstract

Objectives
Grit, defined as “passion and perseverance for long-term goals” (Duckworth et. al, 2007), has attracted much attention. However, Muenks et al. (2017) noted that the Grit-S, the most frequently used measure (Duckworth & Quinn, 2009), does not include items focused on the long term. The purpose of this study is to develop such items and validate both the Grit-S and the newly-developed Grit-L. We examine the structural and predictive validity of scale scores and the replicability of the results across samples. The methodological objective is to compare multiple group multidimensional item response analysis (MIRT; Bock & Zimowski, 1997) and restricted recalibration (RR; Liu et. al, 2018).

Perspectives
Duckworth proposed that grit consists of two components, consistency of interests and perseverance of effort. To date research on its factor structure has been inconclusive because of inconsistency in analytical approaches and limited sample sizes in previous studies. Thus it is unclear whether a simple structure or a bifactor structure is better. Further, until Muenks et al. (2017) first applied MIRT to the Grit-S 5-point Likert scales, researchers treated item responses as continuous variables. Therefore, we further validated the factor structure and item parameters of the Grit-S found by Muenks et al. (2017), and did an initial validation of the Grit-L. We did this under two different scenarios - when item level response data are available or when only model parameters are available, utilizing both multiple group MIRT analysis and the RR method.

Methods
RR is a two-stage model building procedure in which conditional estimates of model parameters are obtained when some parameters are fixed at previous estimates. However, the estimate of the asymptotic covariance matrix (ACM) for model parameters under RR fails to account for sampling variability carried over from the original calibration and thus subsequent inferences may be misleading. We adopted the pseudo maximum likelihood estimation (Parke, 1986) to correct the ACM. For the multiple group analysis, we used full information maximum likelihood estimation for all tested models.
The sample included 1,242 high school students and 581college students as well as previously collected data by Muenks et al. (2017). Students completed the Grit-S in both studies and the Grit-L in the current study. Model parameter estimates reported in the literature were also used in the RR method.

Results
The RR method yielded comparable standard errors for the new parameters compared to the multiple group analysis after adjusting the ACM (Table 1). A bifactor model and a unidimensional model fit the best for Grit-S and Grit-L, respectively, in both samples. The predictive validity of Grit-L is well evidenced while Grit-S predicts only math subject grades among high school students

Significance
Grit has received much attention despite the fact that its measures have not been fully validated. This study provides clarity about the factor structure of the Grit-S, and provides initial validation of the Grit-L. The RR method used here can help researchers validate findings across studies at the measurement level even when the full response data are not available.

Authors

• Monica Morell, University of Maryland - College Park

• Ji Seung Yang, University of Maryland

• Yang Liu, University of Maryland - College Park

• Jessica R Gladstone, Virginia Commonwealth University

• Annette Ponnock

• Lara Turci

• Katherine Marie Muenks, University of Texas at Austin

• Allan L. Wigfield, University of Maryland - College Park