Writing the Same Thing? (Poster 17)

• In Event: Promising Scholarship in Education Research: Dissertation Fellows and Their Research

Abstract

Most selective colleges require applicants to submit a counselor recommendation letter as part of their undergraduate admission processes. The letters are designed to support colleges’ holistic assessments of students, conveying important information that may not be gleaned from elsewhere in the application. However, there is limited understanding about what information these letters contain, and furthermore, how that information is conveyed. Leveraging a novel data set encompassing nearly 550,000 recommendation letters submitted by high school counselors, this study investigates at-scale a letter feature that has important equity implications: letter similarity. Using natural language processing techniques, I first document the degree of textual overlap among letters written by the same counselor and then construct a series of multi-level models to examine the relationship between letter similarity and school as well as student and counselor characteristics. Finally, I implement a topic model to uncover the types of information recycled across letters with high similarity. I find that measures of school resources such as the student-to-teacher ratio, counselor caseload size, and the percentage of low-income students enrolled have significant effects on the similarity of a counselor’s letters. I show that letters with high similarity tend to consist of boilerplate language and descriptions of student qualities that lack personalization. As such, these letters do not provide the type of detailed, contextual information that supports the holistic review process. These findings contribute to our understanding of recommendation letters, shedding light on one way these letters can perpetuate, rather than ameliorate, inequity in the admission process.

Author

• Tara P. Nicola, Harvard University
  Presenting Author