Measurement Matters: Assessing Attributes Other Than Cognitive Ability

• In Event: Measuring Motivational Processes and Other Noncognitive Factors: Challenges and Advances in Connecting Research to Practice

Abstract

Educators, parents, and policymakers are increasingly interested in a broad category of competencies other than general cognitive ability (i.e., general intelligence) or more specific cognitive abilities such as memory or mathematical reasoning (e.g., Tough, 2012). This trend reflects growing dissatisfaction with current measures of competence in school-age children. The current theoretical article identifies three challenges to measuring attributes of school-age children other than their general or specific cognitive abilities: ongoing debates about nomenclature, the typical-vs.-maximal distinction, and cost-accuracy tradeoffs. Should policymakers and practitioners use what psychologists have developed to assess self-control, persistence, honesty, generosity, and the like? We argue that, yes, they might, but only if the advantages and limitations of these assessments are acknowledged and findings interpreted appropriately.

Challenge #1: Naming and defining the category
Various terms are used to define this new category of human attributes– a state of affairs familiar to most psychologists but particularly problematic when attempting to communicate a coherent state-of-the-science message to practitioners and the general public. These labels include “non-cognitive skills;” “personality or temperament;” “social and emotional skills;” “character;” “virtues;” “soft skills;” and others. Our conclusion is that there is no perfect term for this category of individual differences. Nevertheless, there is considerable agreement that this category would connote: positive valence (i.e., benefits to the self and to others); stability over time in the absence of exogenous forces; change in response to experience including learning, training, and practice; and conceptual independence from general and specific cognitive abilities.

Challenge #2: Typical vs. maximal behavior
In contrast to abilities (of any kind, cognitive or otherwise), “non-cognitive skills” refer not to what individuals could do if maximally motivated but, rather, what they typically do in everyday life. When performance measures of these skills optimize motivation, they may not represent behavior in situations to which one hopes to generalize. Similarly, when assessment settings increase threat or heighten mistrust, then behavior on these assessments might only be predictive of real-world behavior to the extent that the world replicates these situations. This issue will be illustrated with concrete examples.

Challenge #3: Tradeoffs
From a practical point of view, ideal measures would be like thermometers: cheap, quick, easy to use with high fidelity across diverse testing situations, non-fakeable, suited for repeated measures over time to detect age-related or intervention-induced change. However there is no thermometer for character. While no ideal measure exists, various imperfect measures with their particular strengths and limitations have been developed. Two conclusions may be drawn: The goals of assessment should dictate choice of assessment, and a sober evaluation of the state-of-the-art measures suggests the potential for further innovation. We conclude the paper with a comparison of the positive and negative features of the following types of measures: (a) self-report questionnaires; (b) informant-report questionnaires; (c) observer ratings of behavior; (d) situational judgment tasks; (e) experience sampling methods; and (d) performance tasks. We conclude with a reminder that the question is not whether a given measure is, itself, “good” but rather “good for some purpose.”

Authors

• Angela L. Duckworth, University of Pennsylvania

• David Scott Yeager, Stanford University

• Anthony S. Bryk, The Carnegie Foundation for the Advancement of Teaching