The Development and Validation of the Measure of Original Thinking in Elementary Students: A Computerized Test of Original Thinking

• In Event: Classic and Novel Approaches in Creativity Assessment and Programming: Implications for Gifted Education

Abstract

The field of gifted education relies heavily on teacher nominations and IQ tests to identify gifted students, which often fall short in identifying students with high creative potential. Sincere efforts to include creativity in the identification process are limited by the cost of assessment tools, especially when universal screening is considered. We developed the Measure of Original Thinking in Elementary Students (MOTES) to help identify highly creative students without a heavy financial burden on the schools. MOTES aims to measure original thinking and originality is scored using text-mining methods, replacing costly and time-intensive human ratings. In this method, semantic distance is used as a marker of originality because originality is greater when unexpected responses are produced (Acar et al., 2022). The unexpectedness is evident when the open-ended prompts and responses given for them are more semantically distant. This feature allows testing a larger number of students, as would be the case in universal screening, with a limited cost, because the scoring system is computerized and automated.

Structurally, MOTES comprises three scales, each consisting of eight prompts: 1) How Do You Use It? 2) What is an Example? and 3) Complete the Sentence. Each prompt requires single response to avoid fluency confound while allowing for using a larger number of prompts. MOTES has a few key features. First, students are engaged in a playful, game-like context. Accordingly, test is presented to them as the “Game of Surprises”. Children were consistently asked to generate surprising responses. Further, each practice item is preceded by a quick video presenting an example of both a surprising and a non-surprising response. All instructions are presented in a playful, dynamic video animation where kids “play” the Game of Surprises along with a creative child character who can generate highly surprising responses. Last, individual prompts are randomized, and the test is timed at the scale-level (not the prompt level) without any mention of it to avoid any time pressure.

The current presentation will focus on the development and validation steps of MOTES. Following cognitive pilots with 32 elementary students and expert validation with 35 elementary school teachers, gifted coordinators, and creativity experts, a 24-item version of MOTES was administered to 387 schools at grades 3 through 5. Students also completed self-report measures of creativity and TTCT-Figural. Additionally, we collected teacher ratings of student originality. Finally, student responses were rated by expert judges for originality. For validation, the dimensional structure of MOTES is explored along with scale reliability using structural equation modeling approaches. Following these, criteria validity evidence is presented using the other indicators of creativity and originality described above. Further analyses focus on the consequential validity (test bias) of MOTES for students with ELL-status or lower socio-economic status.

Results are discussed in terms of implications for inclusive practices in gifted identification.

References

Acar, S., Berthiaume, K., Grajzel, K., Dumas, D., Flemister, C. T., & Organisciak, P. (2021). Applying Automated Originality Scoring to the Verbal Form of Torrance Tests of Creative Thinking. Gifted Child Quarterly, https://doi.org/10.1177/00169862211061874.

Authors

• Selcuk Acar, University of North Texas
  Presenting Author

• Denis Dumas, University of Georgia
  Non-Presenting Author

• Peter Organisciak, University of Denver
  Non-Presenting Author

• Kelly Berthiaume, University of North Texas
  Non-Presenting Author

• Tedd Flemister, University of North Texas
  Non-Presenting Author

• Katalin Grajzel, University of Denver
  Non-Presenting Author

• Michele Newman
  Non-Presenting Author

• David Eby
  Non-Presenting Author