Recruiting and Data Sharing in a Study of Flipped College Calculus

• In Event: The Challenges and Successes of Conducting Large-Scale Educational Research

Abstract

Objectives. The U.S. Department of Education First in the World program has funded a project offering science, technology, engineering, and mathematics (STEM) faculty support to “flip” their instruction through workshops and organized Flipped Learning Communities (FLCs) for instructors. One goal of the project is to examine what works, for whom, under what conditions in first semester college calculus.

Perspective. There are many ways to “flip” a course. In a flipped course, students take in lecture-type information outside of class and then do active learning in class (e.g., labs, group activities, projects). Clear from the current research is that such an approach can result in better outcomes for students, both in terms of achievement and quality of learning experience [7,8,9,13,15,20]. However, there never has been a rigorous test of its effectiveness for improving learning in collegiate mathematics.

Methods. The study is a cluster-randomized controlled trial across first semester calculus classes at three large state universities. When funded, the original design of the study was vetted and approved by First in the World evaluation staff who relied on What Works Clearinghouse (WWC) standards. It called for participating instructors to be their own control comparison as they taught two sections of calculus each semester for two semesters (i.e., teach one section flipped and one as usual). However, realities of single-course assignments meant that fewer than half of instructors would be eligible. A revised plan, also vetted and approved based on WWC standards, led to the design shown in Figure 1 (see Appendix). Given that most calculus courses at two of the three sites are taught by non-tenure track or adjunct faculty, Group 1 includes those eligible in the original design and Group 2 was created for those teaching just one section each semester.

Data sources. The target sample is at least 36 sections across 17 instructors with about 25 students/section (900 students) in the study sample. Data sources included a common post-test across all classes, weekly teaching logs by instructors, and surveys by students and instructors about their mathematics instructional experiences before and during the year of the study.

Results. In preparing for instructor recruitment, study staff encountered strong negative reactions to the design by project leaders. They were sure no instructor would agree to be in Group 1. We created a Guide for Administrators to explain the design which has succeeded as a reference document for talking with department chairs. We also are creating a 5-minute video that a research staff member will show to instructors at each college during an informational recruiting session. Another challenge is acquisition and cleaning of de-identified data for hundreds of students across three colleges. From lessons learned on other projects, we have planned a test of the data sharing process with all three Institutional Research offices.

Significance. In the poster we will share challenges and solutions related to recruiting participants in Fall 2017 and for data exchange. Also, we will share lessons learned during Fall 2017 recruiting and data exchange tests.

Authors

• Shandy Hauk, San Francisco State University

• Bryan Matlen, WestEd

• Alma B. Ramirez, WestEd