Assessment to Support Independence and Resilience in Workplace Communication

• In Event: Assessing Complex Constructs: Examining Construct and Consequential Validity and Implications for Realizing Educational Equity

Abstract

Objectives: Our research team collaborated on designing a large-scale formative assessment and instructional program intended to equip adult learners with communication and collaboration skills needed for in-demand entry- and intermediate-level jobs across high-growth occupations. Elucidating measures and training to support adult learners in acquiring workplace communication skills can potentially increase opportunities to learn (Moss, Pullin, Gee, Haertel, & Young, 2008).

Perspectives: Research consistently demonstrates a workplace-education disjuncture impeding the assembling of a competitive workforce for an increasingly global economy (see Deloitte Millennial Survey, 2018; Hart Research Associates, 2015). At the heart of this disjuncture are challenges to communicate, problem solve, and manage complex tasks that graduates of post-secondary programs face as they enter the workforce (Oliveri & Tannenbaum, 2017, 2019). There are few high-quality, research-based assessments and instructional units that accurately assess these competencies. There also is a well-known scarcity of training and learning materials to help close this gap (Oliveri & Markle, 2017). The preliminary construct model we have developed describes literacy/communications as a socio-cultural, bioecological, multimodal, transactional, critical, meaning-making process (see Appendix 1).

Methods: Extending contemporary models of assessment design, Slomp’s (2016) Integrated Design and Appraisal Framework (IDAF) calls for attention to the potential consequences of assessment design and use at each stage of an assessment design process. Our focus in this study is on phases 1-4 of the IDAF: Identify aims; Identify target domain; Analyze assessment design; Analyze scoring system. Our study examines (a) the workplace communications construct, (b) the alignment of a prototype assessment (Kitchen Design) designed to provide formative data relative to students’ mastery of this construct, and, (c) review of the scoring criteria, reporting, and procedures.

Data related to the alignment between the assessment program and construct model is drawn from several sources: (a) expert analysis of assessment tasks; (b) workplace communications instructors’ analysis of assessment tasks; (c) focus groups with students enrolled in business or workplace communications courses at the post-secondary level.

Results: The authors found the construct model for workplace literacies reflects a form of applied metacognition across multiple modalities of communication. Construct facets include attention to critical discourse knowledge, discourse knowledge, rhetorical knowledge, genre knowledge, subject matter knowledge, and process knowledge (See Appendix for an elaboration of these knowledge domains). Review of our experimental assessment program reveals a focus on genre knowledge, rhetorical knowledge, and process knowledge. Revisions include the need for greater attention to facets most closely linked to long-term development, independence and resilience of students: metacognitive knowledge, critical discourse and discourse knowledge.

Significance: The workplace-education disjuncture with respect to communications needs to be addressed because low-literacy levels can limit the opportunities available to individuals and create a non-diversified workforce for jobs that require further education (Kirsch, Braun, Yamamoto, & Sum, 2007; Sum, Kirsch, & Yamamoto, 2004). With its focus on construct representation and social consequences of assessment design and use, the formative assessment program and accompanying online instructional modules we are developing may foster greater access to high-quality instruction in workplace communications for diverse learners.

Authors

• David Hendrik Slomp, University of Lethbridge

• Maria Elena Oliveri, Buros Center For Testing

• Julie Corrigan, Concordia University - Montreal