Leading Across Borders: Perceptions of Western-based Assessment Models in Other Cultures

• In Event: Leadership and Culture

Short Description

How effective are Western-based assessments in other cultures? Two, the Cultural Intelligence Scale (CQS) and the Leadership Practice Inventory (LPI), were used in three countries to determine the extent of their transferability. This presentation will present the qualitative results of each assessment's effectiveness from the perceptions of national users.

Detailed Abstract

As organizations strive to compete in a global economy, understanding the financial benefits of growing globally is not enough. Often there is a need for a deeper cultural evaluation, followed by assessing individuals to determine a best development plan. But a problem is that assessments may not be transferrable to the host country’s culture. In order for assessments to be effective in other cultures, they must be reliable and valid and ensure meaning is not hindered by translation of words and phrases. Two assessments, the Cultural Intelligence Scale (CQS) and the Leadership Practice Inventory (LPI), were used in three countries to determine the extent of their transferability. This presentation will present the qualitative results of each assessment's effectiveness from the perceptions of national users.

Cultural intelligence (CQ) is a multidimensional concept that spans the mental, motivational, and behavioral components of an individual (Fang, Ling, & Linzi, 2013). The mental component of CQ is divided into two sub-components: cognitive and a metacognitive (Rockstuhl, Hong, Ng, Ang, & Chiu, 2010). Cognitive CQ relates to individual knowledge pertaining to cultural differences, practices, and norms while metacognitive CQ relates to higher level interactions with a consistent and conscious mindfulness to cultural differences (Amiri, Moghimi, & Kazemi, 2010; Ng, Van Dyne, & Ang, 2009; Rockstuhl et al, 2010). The motivational dimension of CQ relates to the attention given and directed towards learning about culturally diverse situations (Ng et al., 2009). The behavioral dimension relates to the behaviors demonstrated in culturally diverse situations.

Van Dyne et al. (2008) conducted six studies to determine if the cultural intelligence scale (CQS) is a reliable and valid measurement instrument of cultural intelligence. For validity the composite chi-square test statistics were reported as (x2= 822.26, p<.05). For reliability the composite Cronbach’s alpha was reported as .92 (Ang et al., 2007). Outcomes indicated that the instrument is both reliable and valid. More specifically, the results indicate that the structure of the CQS is meaningful, and is stable across samples, time, and countries (Van Dyne et al., 2008). Discriminate validity indicates if constructs that are assumed to be unrelated are in actuality distinguishable from one another. Additionally, the CQS was tested for generalizability across countries and the four factors of cultural intelligence was proven to be generalizable between Singapore and the United States.

In addition, the Leadership Practices Inventory (LPI) is an instrument that has been used globally to test for leadership actions and the impact of these behaviors on individuals and organizations. The Leadership Practices Inventory instrument, developed by Kouzes and Pozner (2007), investigates five practices of “exemplary leadership.” The LPI was developed through a triangulation of qualitative and quantitative research methods and studies. Interviews and case studies from personal-best leadership experiences were used to develop a conceptual framework, which consists of: Model the Way; Inspire a Shared Vision; Challenge the Process; Enable Others to Act; and Encourage the Heart.

Each of the five areas relates to an essential aspect of leading. Modeling the way reflects what leaders believe, how they set an example, and how their values are shared. Inspiring a shared vision reflects the way that leaders conceive of the future, relate to others, and share a common purpose. Challenging the process involves how leaders pioneer new things, create innovation, and build capacity and confidence for change. Enabling others to act is how leaders bring about collaboration toward goals, strengthen team members to contribute, and foster empowerment. Finally, encouraging the heart involves the ways that leaders celebrate contributions from followers, build community, and make sure that core values are expressed.

The actions that make up these practices have been adapted into behavioral statements. After several iterative psychometric tests, assessments were developed and administered to leaders and others in a wide variety of organizations.

The tool contains strong internal reliability coefficients—which show that the items are highly correlated within each scale. Test and retest reliability (that is, over time) also is high. In addition, results have demonstrated high face validity and predictive validity. This means that the results make sense to people and have accurately predicted effective leaders and moderate- and low-performing ones.

The CQS and LPI were administered to a variety of leaders and/or university students in China, Germany, and/or Ukraine. Participants also completed a questionnaire regarding their experience with the assessment. In China, participants viewed the CQS as understandable and applicable to their culture. There were phrases that drew concern: I check the accuracy of my cultural knowledge… and I alter my facial expressions… Participants in Germany noted the CQS was effective. There were no concerns with the words used. One participant noted that Germany’s population is diverse, and familiarity with other cultures is expected in the business world. In Ukraine, the CQS was translated into Russian and Ukrainian, as both languages are used in Ukraine. The CQS was administered as a pre-test and through pre-testing it was found to be reliable in both Russian and Ukrainian. An assessment of the results in this country revealed the strong impact of the historic isolation of the country during the Soviet era, resulting in a somewhat ambiguous attitude toward foreign contact and input.

Other issues that are significant include pre-testing an assessment prior to use in other countries. Based on the pre-test of the CQS in Ukraine, for example, it was determined that even though the CQS was reliable, there were several words and phrases that could not be translated into Russian and/or Ukrainian. Therefore it's important to understand the meaning of words, cultural and historical factors and assumptions that are part of the participants' worldview when using assessments in other cultures.

The attendees will be provided with some of the challenges faced by using the CQS and LPI in other countries and what was done to overcome those challenges. The presenters will also discuss the outcomes of the qualitative results and ideas on how research instruments should be assessed when used outside their country of origin.

Participants

• Joanne Barnes, Indiana Wesleyan University

• Boyd Johnson, Indiana Wesleyan University