Evaluating Natural Language Generation Model Evaluation: A Measurement Theory Perspective

• In Event: Assessment in the Advent of Process Data and AI

Abstract

Natural language generation (NLG) models, such as large language models (LLMs), have undergone rapid advancements. A pivotal issue in NLG research is the development of evaluation tasks and metrics to assess the capabilities and limitations of these models. Effective evaluation not only provides essential guidance for practitioners selecting models for downstream tasks but also serves as a roadmap for the scientific community, aiding the development of better models.

Similar to cognitive assessment, an NLG evaluation task comprises a set of questions designed to reflect domain-specific tasks such as translation and summarization. A model’s performance is generally quantified through an aggregate score, based on some evaluation metric, across these questions. The resulting score informs both domain-specific capabilities and the risk-benefit analysis necessary for real-world deployment decisions. Thus, the robustness of evaluation metrics has profound implications for both the validity and generalizability of any conclusions made regarding a model’s capabilities and foreseeable behavior.

Evaluation metrics aim to provide a quantitative representation of a model’s performance in specific tasks, such as summarizing news articles. These metrics are crucial for guiding development, benchmarking progress, and assessing generalizability across diverse tasks and domains. Effective metrics can capture valuable signals from model outputs, aiding in diagnostic procedures and facilitating the comparison of different models. However, poor metrics can lead to incorrect diagnoses and misguided development and deployment. Designing effective metrics remains a formidable challenge due to the inherent complexity of language, the open-ended nature of NLG tasks, and the multifaceted, context-dependent qualities that define good language generation. To confront these challenges, a plethora of NLG evaluation metrics have been developed, ranging from word-based metrics to embedding-based and end-to-end metrics. Despite this, the emergence of general-purpose LLMs has escalated the urgency for metrics that can assess model utility across a wide array of downstream applications.

Several researchers have critiqued existing metrics for their limitations, such as the inability to capture semantic richness and failure to be sensitive to textual perturbations. Further complicating matters, there has been a lack of principled methodologies for evaluating these metrics. While previous work has often sought to correlate automated metrics with human judgments, this approach has proven insufficient, mainly due to challenges in the validation, standardization, and consistency of human evaluation methods.

To bridge these gaps, our work introduces a Metric Evaluation Framework, rooted in the principles of educational and psychological measurement theory. This framework centers around the foundational concepts of reliability and validity, offering a structured approach to identifying sources of measurement error in existing metrics and proposing statistical tools for more rigorous metric evaluation. We demonstrate the utility of our framework through a case study, examining a variety of NLG evaluation metrics in the context of text summarization, revealing key issues related to the validity and reliability of both human-based and LLM-based metrics. Our framework aims to significantly contribute to the ongoing discourse by promoting the design, evaluation, and interpretative strategies that advance the development of robust, reliable, and valid metrics for NLG models.

Authors

• Susu Zhang, University of Illinois at Urbana-Champaign

• Ziang Xiao, Johns Hopkins University

• Vivian Lai, Visa Research

• Vera Q. Liao, Microsoft Research