PD24-07: Using R and Generative AI for NAEP Data Analysis

Session Type: Professional Development Course

Abstract

This 4-hour mini-course will introduce the unique design features of the National Assessment of Educational Progress (NAEP) data to researchers and provide guidance in the required data analysis strategies, including the selection and use of appropriate plausible values, sampling weights, variance estimation procedures (i.e., jackknife approaches) and the generative AI chatbot specifically designed for NAEP data analysis. The course will provide participants with hands-on practice training in analyzing public-use NAEP data files using the R package EdSurvey, which is developed for analyzing national and international large-scale assessment data with complex psychometric and sampling designs, and generative AI chatbot, which is fine-tuned with NCES-approved materials, such as technical documentation on the web, published technical memos and reports. Participants will learn how to perform the following:
•data process and manipulation,
•descriptive statistics,
•cross-tabulation and plausible value means, and
•linear and logistic regression.
•integrating their work with the chatbot, getting help with writing EdSurvey-R codes, and support from the chatbot.

The knowledge and analytic approach from this course can be applied to analyzing other large-scale national and international data with plausible values, such as PISA, TIMSS, PIRLS.

This course is designed for individuals in government, universities, private sectors, and nonprofit organizations interested in analyzing large-scale assessment data. Participants should have at least basic knowledge/skills of R software, statistical inference, and multiple regression. Working knowledge of Item Response Theory and sampling theory is preferred. Participants must have a computer preloaded with the latest R and RStudio software version to participate in the hands-on portion.

Sub Unit

• Professional Development and Training Committee

Director

• Emmanuel Sikali, U.S. Department of Education

Instructors

• Ting Zhang, American Institutes for Research

• Paul D. Bailey, American Institutes for Research

• Sinan Yavuz, American Institutes for Research