Paper Summary
Share...

Direct link:

Team Research Outcomes: Scale Validation

Fri, April 14, 2:50 to 4:20pm CDT (2:50 to 4:20pm CDT), Fairmont Chicago Millennium Park, Floor: B2 Level, Imperial Ballroom

Abstract

Although researchers have explored self-efficacy in online learning environments, little research exists on the impact of transitions from face-to-face to online learning. This research project expands on the existing literature related to self-efficacy and student perception of meaningful learning in a collaborative environment through development and validation of a scale that measures health professions students’ self-efficacy with online learning while exploring how cognitive, social, and teaching presence is experienced by students who transition from one learning environment to another. As learning environments continue to evolve, understanding the impact of learning modality transitions on student perceptions of their learning experience and self-efficacy can inform how educators consider curriculum design.

The measurement constructs (self-efficacy, attitudes towards online learning, teaching presence, social presence, and cognitive presence) and corresponding items related to student online learning were synthesized and developed to fit for measuring modality change purposes. An interinstitutional research team then ranked the items based on the necessity of each item and construct. Debriefing discussions and iterative revisions were made to ensure content and face validity prior to achieving consensus on the final 14 Community of Inquiry (CoI) and 13 self-efficacy items.

A sample of 205 students from 5 U.S. institutions participated in the study. The sample consisted of students from a range of programs with the majority enrolled in Nursing (18.5%), Doctor of Medicine (MD) (17.6%), and Physician Assistant (12.2%) programs. Approximately 75% of participants were female, 78% were graduate students and 82% were under 35 years old. Seventy five percent had online course experience prior to the pandemic while only 22% indicated “quite a bit” or “a great deal” of online learning experience.

The Learning Modality Change CoI and Self-Efficacy scales were validated separately as two independent scales to create two instruments of reasonable length and provide flexibility for future researchers to choose to use one scale or both scales in a single survey. To assess the validity of the Learning Modality Change CoI and Self-Efficacy scales, exploratory structural equation modeling (ESEM) was performed using Mplus 7.3.

The comparison of several ESEM models indicated that the ESEM of three correlated factors of the Learning Modality Change CoI scale with 14 items achieved an acceptable model fit (χ2(52) = 116.98, p < .001, CFI = .96, TLI = .93, RMSEA = .08, SRMR = .02). The ESEM of the three correlated factors in the 13-item self-efficacy scale failed to achieve a good model fit; however, removing Items 1 and 5 from the model, yielded a favorable final model with 11 items (χ2(25) = 48.00, p < .001, CFI = .98, TLI = .95, RMSEA = .07, SRMR = .02). The internal consistency (Cronbach’s alpha) of the Learning Modality Change CoI and Self-Efficacy scales were .92 and .95, respectively.

Overall, the two new scales are efficient yet comprehensive, demonstrate strong validity and reliability, and can be used by researchers to explore the impacts of learning modality changes on student learning experiences.

Authors