Solving Multi-Class Classification with Large Language Models

• In Event: Comparative Agenda Setting: Messaging, Communicating and Legislating

Abstract

The goal of this article is to demonstrate how the latest developments in the NLP field can be leveraged by social science research. Recently developed Transformer or large language models allow for the fine-tuning of large pre-trained models which means lower overall costs and better performance when compared to traditional machine learning approaches. In this article we use a fine-tuned BERT model to apply transfer learning between Hungarian and Polish law texts using the coding scheme of the Comparative Agendas Project (CAP), an international research network which classifies public policy in a wide range of corpora from political speeches to laws. Results show that the fine-tuned model outperforms the traditional supervised learning benchmarks (Support Vector Machines, Naïve Bayes, and Multinomial regression) in precision, recall, and accuracy. This approach demonstrates a viable solution to apply a transfer learning framework for low-resource languages and achieve state of the art classification results.

Authors

• Miklós Sebők, Centre for Social Sciences / TK

• Akos Mate, Centre for Social Sciences, Hungarian Academy of Sciences Centre of Excellence