Predicting Online Student Outcomes From a Measure of Course Quality

• In Event: Unpacking Online Outcomes: Applying Research to Improve the Online Community College Learning Experience

Abstract

Online coursework is growing even more rapidly at community colleges than at four-year colleges. Yet many online courses in the community college setting are not thoughtfully designed; typically, faculty are provided with little training or support when designing online courses, such that some courses merely consist of face-to-face materials that have been ported to an online platform (Cox, 2006). It is not surprising, then, that community college students perform poorly in online compared to face-to-face courses (Jaggars & Xu, 2010; Xu & Jaggars, 2011). To improve course outcomes, some community colleges are beginning to consider using a peer-review process driven by online course quality measures. However, research has not yet established a clear link between specific aspects of course quality and concrete student-level outcomes.

The current study attempts to set forth a practical framework for the design of high quality online courses by identifying instructional activities and pedagogical practices that have a direct impact on student outcomes. To address this larger goal, we analyze data from 26 online courses across two community colleges, with the following research questions: 1) Are the 26 courses representative of the larger set of online courses within these two colleges, in terms of both student characteristics and overall course outcomes? 2) How does the overall measure of course quality, as well as each quality subscale, relate to student end-of-semester course outcomes?

We use a detailed rubric to render a holistic judgment on each of the 26 selected courses in terms of four quality subscales: i) organization and presentation, which examines whether the course has an easy to navigate interface and whether the course materials are clear and well organized; ii) specification of learning objectives and alignment, which evaluates whether the course clearly outlines goals and expectations; iii) course interaction, which assesses the effectiveness of interaction at reinforcing course content and goals, including student-content interaction, student-instructor interaction and student-student interaction; and iv) usage of technology, which examines the effectiveness of the chosen technology to support the course goals. Each of the four quality subscales, as well as the overall course quality measure, are used to predict individual student end-of-semester course outcomes, including course dropout and course grade, controlling for student- and course-level characteristics.

Preliminary impressions from the data imply that students perform at a higher level in courses that have: an easy to navigate interface that is generally self-explanatory; learning objectives that are clearly articulated and transparent, including an explicit description of how students’ performance will be measured; ample opportunities for students to interact with the instructor, as well as with course content; and a variety of ways to master and interact with course content that is engaging and useful. The paper will also provide concrete illustrations of how instructors can meet each of these objectives, as well as provide instructor and student reflections on the meaning and impact of each of these elements.

Author

• Di Xu, Teachers College, Columbia University