Methodology used to create the MLE database enabling systematic review of multilingual education studies

• In Event: Building and sustaining a searchable, open-source database on l1-based multilingual education research

Proposal

This paper reports on the methodology used to create the MLE database, which is designed to enable systematic review of MLE studies. Articles, chapters and books published on MLE between 2007 and 2017 are being annotated and coded to facilitate analysis by our cross-institutional research team and, in the future, in an open-source format for anyone who is interested in research on non-dominant languages in education. This paper describes the three early phases of the methodology, which we hope will provide insights into database creation as well as soliciting feedback.

The first phase was the online data collection and extraction process. A set of search criteria were designed to extract English-language publications from two databases, ERIC and Google Scholar using Zotero, a reference management software. Our inclusion criteria were: “first language” OR “multilingual education” OR “mother tongue education” OR “bilingual education” OR “ethnic minority education” OR “non-dominant language” OR “medium of instruction” OR “language of instruction” AND “policy” OR “methods” OR “assessment.” There were also exclusionary items like “English as Second Language (ESL)” and “English Learners (ELL).” This enabled us to target a wide range of topics in MLE while excluding articles that focused only or mainly on dominant languages. Our initial document search yielded a total of 6,863 references from ERIC and 8,270 from Google Scholar, most of which are journal articles.

The second phase was to establish inter-rater reliability in terms of coding for inclusion. Two rounds of coding trials were run to help our five team members familiarize ourselves with the coding process and strengthen the consistency of our coding. Regular meetings were held to discuss emergent issues in regard to the meanings of each code between team members.

The last step was to code all entries according to our agreed categories. Items from the two databases needed to be treated differently depending on the availability of abstracts, which were included in ERIC but not in Google Scholar. The coding process involved a close reading of abstracts and locating obvious keywords. Once the coding of items from the ERIC database was finished, they were combined with Google Scholar references to sift out duplicates. For the remaining items from Google Scholar, those whose titles clearly indicated irrelevant themes were excluded, while abstracts for the remaining titles were manually searched for, cut and pasted and coded using the same standards applied to the ERIC coding.

This database is significant to the field in that it makes possible the tracing of historical trends as well as emerging topics in the literature of MLE. It will allow our team and others to map MLE research topics, methods and issues that have gained relevance in the last decade or so.

Author

• Lajiadou Lajiadou, Teachers College, Columbia University