How NAEP Data Explain Aspects of Arts Learning

• In Event: Discussing Educational Policy and Implications of Efficacy in the Arts as School Subject Areas: NAEP to Teacher-Made Assessment to Subject Area Growth

Abstract

Paper explains statistical aspects of the Visual Arts NAEP data, both the data available to the public users and statistical analysis of restrictive data licensed under William Carey University.
The NAEP Data Explorer on the website of National Center of Educational Statistics has the capacity to provide versatile reports for its users. For example, take our subject of interest, Visual Arts. NAEPs online software can generate reports in multiple forms for users interested in a variety of criteria and variables, such as, charts and significant tests. These variables include but not limited to students factors (e.g., demographics, affective disposition etc.), school factors (e.g., demographics, organization, resources etc.) and factors beyond school (e.g., home regulatory environment and time use outside of school). Average scores for different levels of specific variables will be provided in cross-tabs and significance tests can be performed and provided in the report. Users can also have a glimpse of Visual Arts test items from the sample questions, both multiple-choice questions and constructed response questions, provided on the website. Despite all this capacity, however, users can’t have access to individual students’ responses and scores for background items and test items in the NAEP assessment, nor the full set of question items. No advanced statistical tests (e.g., factor analysis, HLM, path analysis) could be run without this information.
NAEP data require particular tool kits for analysis. AM software is one of them. It is a software for analyzing large-scale assessment data and can apply replicate weights for the estimation of sampling error. It runs descriptive and inferential statistics (e.g. t-tests, regression) for the dataset generated by NAEP data extraction software. It can also manipulate variables, e.g., recoding/collapsing categories and dummy coding variables. An updated version of AM is available on http://am.air.org.
This presentation will also discuss the replication results of path analysis of the portrait block of 1997 (unpublished graphic Diket & Thorpe, 2001) using 2008 data. Further, it will discuss the rerun of the Hierarchical Linear Modeling analysis that Diket and Thorpre (2000) reported last time in 2000 to explain within and between group variation in Visual Arts achievement for students taking and not taking arts classes with the 2008 NAEP Visual Arts data. These runs will be accomplished in late July, missing the proposal deadline by a few days.
All the questions administered in the 2008 NAEP Visual Arts assessment were selected from those in the 1997 assessment. However, the scoring procedures for questions with constructed responses were changed, which makes the direct year-to-year comparison impossible. Fortunately, this is not the case for multiple choice questions and direct comparison between 1997 and 2008 data are possible for evaluation of patterns.

Author

• Lihua Xu, University of Central Florida