Alarm Bells or Just Smoke: An Evaluation of Plagiarism Detection of ChatGPT-Generated Essays in CJ.

• In Event: Diverse Findings from the University of Cincinnati Cybercrime and Behavior Online Research Group (UC CYBORG)

Abstract

OpenAI’s ChatGPT is an advanced large language model AI that has caused both excitement and concern in academia. TurnItIn and similar software programs are widely used in higher education to detect potential plagiarism. However, the extent to which such software can identify papers produced by ChatGPT remains unclear. The current study partially addressed this question by submitting five versions of short essays about criminological topics generated from ChatGPT to the TurnItIn software. Overall, the results indicated that TurnItIn adequately detected that the essays were not original works (mean percent plagiarized score = 31%). The analyses further illustrated that ChatGPT wrote at an exceedingly high level (mean Flesch-Kincaid Grade Level = 15.1) atypical of essays in higher education. Consequently, it appears that detection of AI-generated writing may be easier than initially assumed although the technology is bound to improve. Accordingly, possible strategies for approaching AI in higher education are proposed.

Authors

• Tim Engle, University of Cincinnati

• Joseph Nedelec, University of Cincinnati