Classifying Newspaper Articles with Supervised Machine Learning: A New Approach

• In Event: Machine Learning for Measurement

Abstract

The paper offers a step-by-step presentation of our new methodological solution for coding public policy major topics in the content of media articles with no, deficiently reliable or limited training sets. We combine a quantitative text mining approach with supervised learning methods and limited human coding in ordet to classify the front-page articles of the two leading Hungarian daily newspapers, Népszabadság (NS) and Magyar Nemzet (MN), between 1990 and 2014. We first applied our research design to NS. In the second step we used these newly coded articles to recreate the process for MN. The initial database for NS consisted of 53,316 articles, 11,342 of which had been hand-coded for policy topic. In the first step our goal was to classify the remaining items in the dataset based on the codebook of the Comparative Agendas Project. Next, we set the coded NS corpora as the training set for categorizing MN articles. Our results show that for MN, our approach provided better precision results than what is customary for human coders. Nevertheless, this high precision came at the expense of low estimated recall - a task for future refinements for our workflow.

Authors

• Miklós Sebők, Centre for Social Sciences, Hungarian Academy of Sciences

• Zoltan Kacsuk, Institute for Political Science, Centre for Social Sciences, Hungarian Academy of Sciences