Unsupervised Clustering of Image Data: Overview and Assessment of Performance

• In Event: Advances in Deep Learning Methods for Text, Image, and Video Data

Abstract

Image data provide rich and unique symbolic information about political events and actors that are not captured by other types of data. Developing automated methods to analyze political image data has been a new trend in social science research. Most current methodology advances on image-as-data, however, focus on using supervised machine learning methods to predict image labels. Unsupervised learning methods aim to learn the underlying features of images without explicitly imposing categories of interest. Unsupervised learning can be especially helpful when scholars have a massive number of images, such that human judgement misses some image categories that are theoretically useful, but perhaps understudied or previously unknown. This study first introduces the steps to perform unsupervised image clustering. It then systematically evaluates the performance of different unsupervised image clustering methods on real, large political science image datasets. We find that deep learning-based methods that jointly learn image representations and categories outperform "bag-of-visual-words" methods that separately learn image representations and then cluster. We further investigate how different clustering methods, such as K-means and agglomerative clustering, may lead to different clustering performances. We last discuss the validation issues of image clustering methods. This study contributes to the growing new field of "image as data", and provides new insights to various social science disciplines.

Authors

• Han Zhang, The Hong Kong University of Science and Technology

• Yilang Peng, University of Georgia