Analysis of the effectiveness of Generative AI on the English-speaking proficiency of international students from non-English speaking countries in Higher Education

• In Event: Frontiers of AI and Immersive Technology in Teaching and Learning

Proposal

Abstract
Oral proficiency is vital for international students adapting to a new cultural environment, as it impacts both academic success and social integration. Despite being tested on English skills before arrival, many students still struggle with speaking. These challenges often stem from a lack of immersive practice environments, leading to anxiety, isolation, and limited engagement in academic and social settings when living abroad as a newcomer. In recent years, generative AI has emerged as a powerful language-learning tool, offering more communicative, interactive, and personalized practice opportunities for learners. However, research on its impact on oral proficiency and cultural integration for international students remains limited. In this article, Pre-and post-tests and semi-structured interviews are conducted to analyze 30 International freshmen from non-English speaking countries at Yale University. The result shows that generative AI improves their willingness to communicate and speaking abilities, including vocabulary, grammar, pronunciation, and colloquial expressions, and further impacts cultural understanding.

Key Words: Generative AI; language learning; international students; higher education; EFL.

Introduction
Effective oral communication is a key challenge for international students, as it is essential and influential for both academic and nonacademic acculturation even long after admission (Sawir et al., 2012). While international students are typically tested on their English proficiency before arrival, the importance of oral proficiency is usually underestimated, which often leads to difficulties in adjusting to the demands of their life abroad (Al Asmari, 2015). These pressures brought by the language barrier can negatively affect their academic performance (Leong, 2015), as well as their ability to form social connections, full engagement in campus life, and the development of intercultural friendships (Meng, Zhu, & Cao, 2018). Therefore, it is crucial to improve international students' English-speaking skills even after they enter university.

Traditional language-learning methods, like classroom instruction and workshops, can be effective but are often constrained by time, resources, and limited personalized practice. The rise of generative AI, such as ChatGPT, offers a new avenue for language learning by providing learners with customized, dynamic, and immersive practice opportunities (Yeh, 2024). AI-powered platforms can simulate conversations, provide real-time feedback (Geng and Razali, 2022), and offer a wealth of practice materials that are tailored to the learner's specific needs (Kasneci et al. 2023) in a relatively private space to help students practice. However, there is a lack of research on whether and to what extent generative AI can help international students improve their oral proficiency and how these improvements affect their cultural engagement.

This study will explore the impact of practicing with generative AI on oral English of international students from non-English speaking countries studying in English-speaking institutions and its further influence on university engagement through the following questions:

1.How does generative AI influence international students' oral proficiency, including vocabulary, grammar, pronunciation, colloquial language, and cultural awareness?
2.To what extent does generative AI enhance international students' willingness to communicate and their oral communication in comparison to traditional methods?
3.How does improved oral proficiency through generative AI impact international students' cultural engagement and adaptation to new environments?

Theoretical Framework
Social Constructivist Framework
This study is grounded in Vygotsky’s constructivist theory, which emphasizes that learning is a process in which students actively construct knowledge rather than passively receiving it. In oral English learning for international students in higher education, each student has their unique cultural background, language foundation, and learning style (Cobern, 1993). Generative AI offers personalized learning resources and interactive environments according to students’ individual differences, helping them build their own oral English knowledge system. By creating rich, diverse virtual contexts, such as simulated communication scenarios and role-playing, it enables international students to practice in context, enhancing their fluency and accuracy in oral expression.

Methodology
A mixed-methods approach will be used in this study, combining quantitative pre-and post-tests with qualitative semi-structured interviews. There are 30 international freshmen from Non-English Speaking Countries (China, South Korea, and Japan) who are studying at Yale University joining the pre-and post-tests which measure their speaking proficiency before and after using ChatGPT DAN to practice oral English, a chatbot and virtual assistant built on large language models (LLMs), allows users to guide conversations in terms of length, format, style, detail, and language with each response considering previous prompts as context (Roumeliotis & Tselikas, 2023; Lock, 2022).

This assessment will evaluate improvements in vocabulary, grammar, pronunciation, and the use of colloquial expressions. The semi-structured interviews will explore students' experiences with generative AI, focusing on how it has influenced their willingness to communicate and their ability to integrate into the new cultural environment.

1.Pre- and Post-tests: These will be conducted to assess changes in speaking proficiency after using generative AI tool. The tests will evaluate vocabulary expansion, grammatical accuracy, pronunciation improvement, and the ability to use colloquial language.
2.Semi-structured Interviews: These interviews will aim to gather deeper insights into students' experiences. Key questions will explore:
1)Changes in students' willingness to engage in English-speaking activities.
2)The perceived impact of improved speaking skills on their ability to adapt to the new environment and engage culturally.

Results
This research found that international students using ChatGPT DAN showed marked improvements in their oral proficiency, including vocabulary, grammar, pronunciation, and colloquial language. Additionally, their willingness to communicate in English increased, leading to greater participation in academic and social settings. As a result, students felt a stronger sense of belonging and a deeper understanding of the cultural nuances in their new environment. The study found that generative AI proved to be more effective than traditional language-learning methods in these areas due to its personalized, immersive, and adaptive nature.

Scholarly significance
This study examines the effectiveness of generative AI in oral English learning, offering insights into how this technology influences international students' learning processes and outcomes. It highlights the connection between technology and language learning, providing theoretical support for integrating educational technology with language teaching. Additionally, by focusing on international students, the study offers valuable insights for cross-cultural and international education, helping educators design tailored teaching strategies and resources to enhance educational equity and quality across diverse cultural backgrounds.

Author

• Xinyi Zhang, SUSTech