Pros and Cons of Technology-Enhanced Items in a Digital Literacy Assessment

• In Event: AERA Poster Session 2
  In Poster Session: Poster Session 1 (San Diego)

Abstract

Technology-enhanced items (TEIs) have particular strengths in measuring Digital Literacy (DL), including increasing test-takers' engagement level and improving authenticity. However, TEIs also pose validity threats, including introducing nuisance factors. This study evaluated TEIs in a territory-wide DL assessment. The results (a) demonstrated that a bifactor model with a general DL factor with item type as bifactors fit the data best, (b) confirmed two contested advantages of TEIs, including that (1) some TEIs had higher levels of fidelity, which in turn had better discriminations in the general DL factor; (2) TEIs had fewer rapid guessing behaviors. The study contributes to a more comprehensive understanding of the impacts of TEIs and the development of computerised assessment of DL.

Authors

• Qianqian Pan, National Institute of Education - Nanyang Technological University

• Yuxiao Zhang, Purdue University

• Patrick Lam, The University of Hong Kong

• Zhehan Jiang, Peking University