Paper Summary
Share...

Direct link:

Developing an Inventory to Investigate Learners' Language Assessment Literacy

Thu, April 13, 11:40am to 1:10pm CDT (11:40am to 1:10pm CDT), Swissôtel Chicago, Floor: Event Centre, 1st Floor, Vevey 4

Abstract

Objectives
The objective of this paper is to document the development and validation of an inventory that identifies English language learners’ (ELLs’) language assessment literacy (LAL). LAL refers to education participants’ understanding and practices of language assessment (Davies, 2008; Fulcher, 2012; Inbar-Lourie, 2008). Although learners are major education participants, learners’ LAL and its measurement have received limited attention (Author & Colleague). This study is one of the first attempts to design an inventory to identify learners’ LAL.

Theoretical Framework
With an increased emphasis on learners’ active involvement in assessment in recent years, it has become critical to understand learners’ LAL. As research on learners’ LAL is still limited, we turned to teachers’ LAL as a starting point to identify its constructs. While the discussions are ongoing, research on teachers’ LAL has identified that LAL generally consists of assessment knowledge, assessment skills, and assessment principles (Author & Colleagues; Giraldo, 2018). Assessment knowledge refers to understanding of language assessment theories; assessment skills refer to creating, using, and analyzing assessment; and assessment principles refer to understanding of cultural, ethical, pedagogical, political, and social consequences of assessment. The literature also identifies affective factors including motivation and anxiety as the critical constructs to understand learners’ involvement in learning and assessment.

Data Sources & Methods
The inventory development consists of design and validation stages. At the design stage, we adopted a deductive approach where we followed literature on Likert-scale inventory design by creating items that are direct, concise, easy to comprehend for learners, and unidirectional. Through multiple iterations, the two authors came up with 52 items. The first section of the inventory collects learners’ demographic information and assessment experiences (14 items) while the second section is composed of items that measure learners’ assessment knowledge (13 items), assessment skills (11 items), assessment principles (7 items), and affective factors (7 items). The second author piloted the inventory on seven college-aged, intermediate-level ELLs to test the clarity and comprehensibility of the items.
At the validation stage, we employed both qualitative and quantitative approaches. Eight graduate students recently trained in language assessment validation were invited to conduct an expert review of the inventory to ensure construct validity. The authors will administer the inventory to 150 ELLs at community college and university English language programs in the USA. At this stage, more rigorous quantitative analysis, including factor analysis, will be employed to validate the inventory.

Results & Scholarly Significance
Based on the students’ results and feedback from the pilot, we found that intermediate-level ELLs did not experience significant difficulties completing the inventory. Nonetheless, we used their feedback to increase clarity by simplifying the item wording and instructions in the inventory. The expert reviewers generally agreed that the three LAL constructs and the affective factors are relevant to learners’ LAL. Although quantitative analysis is pending, this study is significant as it is the first of its kind to investigate learners’ LAL. The study will also expand the theoretical discussions of conceptualizing learners’ LAL.

Authors