Teens’ Ethical Considerations When Building babyGPTs

• In Event: Can Education Stop Killer Robots? Exploring Values, Ethics, and Impacts in K12 Computer Science Instruction

Abstract

Objectives
As generative language models (GLMs) have gained popularity, high school students are increasingly encountering them in their everyday lives. While most research has focused on examining youth as productive users of GLM-powered systems, far fewer efforts have focused on how to engage high school students as designers of these models to foster better understandings of how these systems work. Building on the rich legacy of positioning learners as designers of computing systems, we explore the ethical considerations that teens engaged with when designing very small-scale GLMs, which we call babyGPTs.

Theoretical framework
The design process of computing applications also involves grappling with ethical issues (Iivari et al., 2023). However, how to engage young people with ethics and criticality when building ML models remains understudied. In model-building efforts, ethics are often discussed in relation to commercial ML-powered applications instead of being applied to learner-designed models (Morales-Navarro and Kafai, 2024). Promising efforts integrate ethical considerations into model design activities using cards with questions about design decisions (Bilstrup et al., 2020), having students discuss potential misinformation generated by their projects (Lin et al., 2020), or creating ethical matrices for their classifiers (Jordan et al., 2021). More recently, researchers have identified distinct ethical issues for young people to consider when learning about AI/ML, including algorithmic biases, surveillance, privacy, transparency, identity representation, misinformation, environmental issues, ownership, labor issues, and overreliance on AI/ML systems (Veldhuis et al., 2025).

Methods
We conducted a five-day school workshop with 35 high school students (ages 14-15) in a school located in the Northeastern US, where they designed very small GLMs, which we call babyGPTs, using the nanoGPT framework (Karpathy, 2024). In this poster, we present a case study (Yin, 2018) of one group of three students in which we examine their construction process, discussions, artifacts, and interviews to address the following question: What ethical considerations did participants engage with in the construction process of their own GLMs? We investigate how the students engaged with ethical considerations while (a) defining a design problem and conducting research, (b) ideating and building, and (c) arguing and reflecting.

Results
When developing their model, the team considered ethical issues related to ownership (copyright, authorship, and attribution) while designing a dataset for their babyGPT and exploring its outputs, sharing different ethical stances towards using screenplays to train a model. Some argued that copyrighted screenplays should not be used; others emphasized the importance of providing credit to the writers of the screenplays, and one student argued that authorship could never be attributed to a GLM. When reflecting on their model, the team discussed ethical implications (Veldhuis et al., 2025) related to misinformation (trustworthiness of GLM-generated text), ownership (copyright, authorship, attribution), environmental impact (energy consumption), harm, algorithmic bias, and reliance on AI.

Significance
This work contributes a case study showing how students engaged in ethical considerations in the construction of generative language models and outlines directions for future research to support young people’s engagement with ethics and societal implications in model design activities.

Authors

• Luis Morales-Navarro, University of Pennsylvania

• Daniel J Noh, University of Pennsylvania

• Yasmin B. Kafai, University of Pennsylvania