Young Learners: An Examination of Early Literacy Assessment Responses From Multiple Perspectives

• In Event: Roundtable Session 46
  In Roundtable Session: Methodological Choices and Their Consequences in Education Research

Abstract

Objective
This paper contrasts the different theoretical perspectives that are reflected in the employment of different methods in analyzing early literacy assessment responses. In particular, through exploring ways in which the Early Literacy Knowledge and Skills (ELKS) instrument (Author et al., XXXX) can be used by teachers, this paper showed the importance of researchers being cognizant of their methodological choice and the associated epistemological position.
Perspective
The formulation of this study was based on the understanding that assessment is “a process of reasoning from evidence” (Pellegrino, Chudowsky, & Glaser, 2001, p. 43) and the recognition that theory, method, and practice are closely linked with each other. It draws upon the National Research Council assessment framework (Pellegrino et al., 2001) which suggests that the theory about the nature of early literacy determines the kinds of tasks, events, or settings that are selected or specified for observing the display of early literacy behaviors. To make informed decisions based on the data collected from an assessment therefore requires a thorough understanding of the limitations and constraints of different theoretical perspectives, the task or observation settings chosen for the assessment, and the analytical methods. This study examined the research question: What insights can be gained from examining the responses to the ELKS instrument using different methods that are underpinned by different theoretical perspectives?
Methods and data sources
The ELKS instrument was administered to 293 four- to six-year-olds who were at preschool or first year of school. The instrument was developed based on studies carried out by Ferreiro and Teberosky (1979/1982) on preschool children’s early literacy concepts. Adopting a mixed methods research design, the children’s assessment responses were analyzed using quantitative and qualitative methods at group and individual levels using item response modeling, thematic analysis, and multiple case studies.
Results
The combined findings suggest that the same set of responses when processed and analyzed in different ways can show seemingly contradictory characteristics. For example, an examination of the scored and raw responses of two children who were estimated to share the same ability level according to item response modeling found that the children demonstrated distinct patterns in the ways they responded to the assessment tasks. In particular, the children’s raw responses indicated that they focused on different levels of text features when attempting the tasks. Each analytical method focused on a particular aspect of early literacy development in the children’s responses. Rather than privileging a particular analytical method in assessment, it may be more appropriate for teachers to distinguish the group level and individual level inferences that can be drawn from the assessment results so that appropriate decisions can be made.
Scholarly significance
Through conceptualizing the ELKS instrument as a standardized assessment tool that can provide different views depending on the analytical method used, this study may help researchers and teachers to better utilize information that can be obtained from such tools to observe individual children’s early literacy development.

Author

• Man Ching Esther Chan, The University of Melbourne