Making the Grade: Reporting Education Technology and Teacher Knowledge Research

• In Event: Education Technology, Teacher Knowledge, and Classroom Impact: Frameworks and Approaches to Research

Abstract

Objectives

We examine issues surrounding the design of research in education technology and teacher knowledge.

Theoretical Framework(s)

The six scientific principles from the National Research Council (NRC; Shavelson and Towne, 2002) can guide researchers in their pursuit of education research (Pose Significant Questions That Can Be Investigated Empirically, Link Research to Relevant Theory, Use Methods That Permit Direct Investigation of the Question, Provide a Coherent and Explicit Chain of Reasoning, Replicate and Generalize Across Studies, Disclose Research to Encourage Professional Scrutiny and Critique).Whitehurst (2002) laid out a hierarchy of six evidence levels from most valid for inference to least: “(1) randomized trial (true experiment), (2) comparison groups (quasi-experiment), (3) pre-post comparison, (4) correlational studies, (5) case studies, and (6) anecdotes” (p. 15).

Method(s)

This study synthesizes results from two systematic reviews of literature examining research in education technology, teacher knowledge, and the intersection of education technology and teacher knowledge.

Data Sources

The first systematic review examined a random sample of 145 articles from an initial sample of 1,785 manuscripts from five content areas (English/language arts, career and technical education, mathematics, science, and social studies) examining the integration of technology in teaching. The second review examined 203 manuscripts examining teacher knowledge across 14 content areas and 208 manuscripts examining teacher knowledge without specifying a particular content (e.g., nature of teacher knowledge, teacher quality).

Results

The first review found that the evidence relied upon in education technology research follows an exact reverse order to Whitehurst’s (2002) quality of evidence hierarchy. Instead, 50.34% provided anecdotal support for integrating technology into teaching, and another 12.93% proposed theories based on non-systematic reviews of literature for a total of 63.27% articles distributing “evidence” with no systematic research foundation. Qualitative studies (including case studies, ethnographies, interviews, focus groups, observations, and action research) accounted for 26.53% of the sample. Quantitative and mixed methods studies (including experimental, quasi-experimental, pre-post comparisons, and correlational studies) accounted for the remaining 10.20% of the sample.

The second review found that teacher knowledge studies suffer from a similar lack of rigorous evidence. Research on teacher knowledge has largely remained in its infancy due to a lack of coherent structures and organizing frameworks. Rather than developing a coherent framework for teacher knowledge, a myriad of frameworks were found that attempted to capture only some components of the knowledge needed for teaching. Unfortunately, no structure has yet organized these components into a structured whole that can be used to advance the field. As a result, few studies exist that empirically measure the effects of teacher knowledge on student outcomes. We present a checklist based aligned with the six NRC principles to guide future researchers in producing rigorous, scientific evidence.

Significance

A number of texts have been produced to provide guidance to researchers using particular research methodologies (e.g, Cooper, Hedges, & Valentine, 2009; Creswell, 2007). When used as a supplement to these excellent texts, the checklist may provide researchers with an overview of the issues to be considered when designing studies involving education technology or teacher knowledge.

Authors

• Robert N. Ronau, University of Louisville

• Christopher R. Rakes, University of Maryland - Baltimore County