Does test preparation work? Grounded theory research of TOEFL-iBT preparation programme in a Chinese educational institution

• In Event: Poster session 2

Proposal

Test preparation is considered as one overt type of washback effect that can be brought forth in case of a high-stakes English test. The practice of test-prep and its effect have been scrutinised by researchers focusing on preparation courses (Alderson & Hamp-Lyons, 1996), teachers’ beliefs and practices (Gebril & Eid, 2017), influencing mechanism (Xie & Andrews, 2012), etc. However, most studies employed questionnaire as the main research instrument, which fails to portray the complexity of test preparation practices and beliefs and capture individual and contextual differences. This study attempts to address the aforementioned problems by adopting a grounded theory analysis of test takers’ practice and beliefs of test preparation for TOEFL in a Chinese educational institution. The study has significance for second language learning and teaching in the Chinese context, which has been traditionally permeated with the perception of tests as a major indicator of personal success and a means of aristocracy (Jin, 2016).

The participants (N = 54) were enrolled in a course aiming to better prepare them for taking TOEFL iBT, whose score will be used for applications to further study mainly in North American universities. After the duration of 3 months, participants voluntarily attended the focus group interviews, during which their motivations, learning experiences, and perceived effects were investigated. The transcription and coding were jointly conducted by the researcher and his colleague. The coding process followed the recommendations by Strauss (1987), which resulted in a desirable intra-coder reliability (Cohen’s kappa = .92). A tentative framework of test-prep practices and beliefs was developed based on the indicators and concepts emerging from the interviews. Given the nature of grounded theory research, no a priori theoretical framework was established before the coding and analysis. However, results were discussed in relation to the test-prep framework proposed by Xie (2013), which includes six dimensions (i.e. rehearsal of test taking skills, test preparation management, drilling, memorising, socioaffective strategies, and language-skill development strategies).

Results suggest that the test-prep course facilitates the overall test, especially in terms of the development of participants’ linguistic competence, study skills, cognitive abilities and self-efficacy, though the effect of test preparation is largely contingent on participants’ motivations and their initial English proficiency level. It should be noted that longitudinal analyses should be conducted in the future in order to ascertain the long-term impacts of test preparation on test candidates, for instance their language proficiency. Though test preparation has been under criticism for inflating scores and undermining scoring validity, its role in cultivating students’ socioaffective and language-skill development strategies is corroborated in this study. The effect of test preparation is still open to discussion, especially in a context where much importance is attached to success in passing major tests. This study is relevant to those engaged in test preparation and ELT in general, and holds implications for EAP teaching and test validation in China.

Author

• Yuanyue Hao, University of Oxford; New Oriental Education & Technology Group