If Truth Is Found in Evidence, What Evidence Do We Have?

• In Event: Co-Constructing Systemic Support for Sustaining Humanizing and Inclusive Computer Science Teacher Education

Abstract

Objectives: Education research provides evidence, but there are numerous decision points in research design that can make the evidence collected more sound than others. As a young field, computer science education research is still evolving and researchers are still building their repertoire of tools. As such, evidence produced may or may not be telling us what we think it is. The purpose of this paper/presentation is to explore the known issues with evidence and what steps can be taken to start to improve the quality of and representation within evidence.

Perspective: This position builds upon several years studying the education research quality in K-12 CS education, particularly with respect to known standards like the American Psychological Association (APA), CONSORT, and What Works Clearinghouse (Decker and McGill, 2019; McGill, Decker, and Abbott, 2018; McGill, Decker, McKlin, and Haynie, 2019). It also considers the focus on equity within the research, such as who is included as participants in studies.

Methods/Data: Through secondary data analysis of over 1,000 K-12 CS education papers in our K-12 CS Education Research Resource Center (McGill and Decker, 2017), we have examined the survey instruments used, the study participants, types of studies, topics studied and the types of data analysis performed.

Results/Findings: The data is reflective of the fledgling state of K-12 CS education research and its current lack of inclusiveness. Only 4% of papers with student participants are inclusive of students with disabilities, while in the U.S. the number of students with disabilities stands at 14%. Nearly 1 in 10 papers with student participants did not state the number of participants in the study. Further, the majority of survey instruments that are used in studies are created by the authors without evidence of reliability or validity, calling into question the evidence that these instruments generate.

Scholarly Significance: The scholarly significance of this data analysis will give us pause as we look at the data already generated to help define promising practices in teaching K-12 students about computer science. This is a clear call for a need to strengthen and build the capacity of quality CS education research that is inclusive of all students.

Conference theme: This presentation directly aligns with truth as it is found in evidence and disseminated throughout the community, particularly when this scant, questionable evidence is being integrated into promising pedagogical practices. As researchers, we must ask ourselves how we can strengthen our evidence so it more accurately reflects the authentic truth of students’ lived experiences.

Author

• Monica McGill, CSEdResearch.org
  Presenting Author