Definitions and Development of Assessments of Collaborative Skills

• In Event: Roundtable Session 60
  In Roundtable Session: Assessment of Technology-Based Collaborative Skills: Transforming Assessment for the 21st Century

Abstract

Objectives

This presentation reports on change in assessment and teaching linked to 21st century skills and their potential impact on curriculum. It reports on the use of classroom discourse, concept checking by teachers, and pilot and trial studies across seven participating countries. The results provide a platform for research to improve education. The presentation links with others to bring together academics from several countries and institutions to discuss the definition, measurement and implementation of the assessment of collaboration based skills. It also illustrates how these can be increased in scale, thus advancing new forms of assessment.

The framework

Labour market changes in developed economies have altered the skill demands of work environments that are now technology-rich. Workplace problems are frequently ill-defined and people work in (often multidisciplinary) teams to deal with them. Cisco, Intel and Microsoft have sponsored an international, multi-year research project to define collaborative skills in operational terms, to address methodological and technological barriers to their ICT-based assessment, and to do this in ways that take account of assessment needs, from classroom practice to national and international studies of achievement. This presentation describes the identification and operational definition of the skills on which the work is focused, it also addresses the work completed on methodological and technological developments to provide dynamic, ICT-based assessments.

Methods and sources

A series of collaborative tasks were prepared that addressed problem-solving and learning in digital networks. The research involves five phases: the first was conceptual, involving 250 academics in exploring, defining and writing a series of white papers for a volume published by Springer. The second involved the selection of skills and the formulation of hypotheses regarding their developmental nature. The third involved the development and refinement of the tasks through a series of cognitive laboratories and pilot studies. The fourth involved large-scale trials. The fifth phase involved dissemination and policy development for the governments involved in the study.

Data sources

Cognitive laboratories in four countries were followed by small pilot studies, then large scale trials in seven countries involving more than 100 schools and 2000 students.

Results and substantive conclusions

The five phases of the project have resulted in calibrated assessment tasks delivered online. Automatic coding and calibration resulted in a series of developmental continua. These were presented to teachers during field trials in order to elicit advice about suitable teaching intervention strategies leading to progress along the developmental learning progressions.

Scientific and scholarly significance

The study has opened up new forms of assessment of skills not previously assessed in on a large scale nor using technology. This has the potential to influence large-scale studies of assessment. The dissemination phase is yet to be completed. Once is has been completed, it is expected that a number of further studies will commence in these areas. It is likely these studies will be influenced by both the content and the methodologies employed in this project.

Authors

• Patrick E. Griffin, University of Melbourne

• Esther Care, University of Melbourne