The influence of writing skills on final-year assessment performance in New Zealand’s National Certificate of Educational Achievement

• In Event: Issues of Writing in Assessment and Feedback

Abstract

New Zealand’s secondary school qualification system, comprising the three levels of the National Certificate of Educational Achievement (NCEA) is assessed using units called standards. Typically, the assessment programme for each course of study entails about 5-8 standards, some assessed within schools, and others by external examination. Every standard describes what a student is expected to be able to do to receive credit for that standard. Three passing grades are available; in increasing order of quality, these are Achieved, Merit and Excellence. Results are reported individually for each standard attempted.
The NCEA assessment system affords opportunities for data analysis that would be more difficult in an environment in which results are reported only at the subject level. For example, standards can be grouped according to certain criteria, and quantitative performance variables can be fitted to these groupings using statistical techniques such as Rasch analysis. In the present paper, we use this feature of NCEA to investigate the impact of writing skills on performance in a range of subjects in the final secondary year, over and above any (valid) writing component inherent in each subject.
For each subject, a generalised writing performance variable was generated by applying Rasch analysis to a set of standards for the penultimate secondary year, for the 2009 cohort, drawn from diverse subject areas, identified by expert judges as having high writing requirements. The writing variable generated for each subject drew upon exactly the same set of standards, except that any standards from the subject in question were excluded. Also for each subject, two subject-level performance variables were generated for the same cohort, one for the penultimate year in 2009 and the other for the same cohort, for the final year in 2010.
A hierarchical liner regression approach was used to estimate the variance in final-year performance uniquely associated with prior performance in the same subject, and the variance in final-year performance uniquely associated with prior generalized writing performance. The results of the analysis are discussed in relation to the hypothesis that the relatively heavy writing requirements of NCEA assessment limit the validity of the assessment in some subjects.

Author

• Michael Johnston, Victoria University of Wellington