Machine Learning with Explainable AI: Handling Imbalanced Panel Data through Resampling Techniques

• In Event: Analyzing Large Scale Data Systems: Methodology and Implications for Practice

Abstract

This study aims to solve the class imbalance problem that occurs when applying machine learning technology to large-scale datasets by evaluating the performance of various resampling methods (synthetic minority class oversampling technique (SMOTE), adaptive synthetic sampling (ADASYN), SMOTE combined with edited nearest neighbors (SMOTE+ENN), and SMOTE combined with Tomek links (SMOTE+Tomek) with ensemble models such as Extra Trees, Random Forest, XGBoost, and CatBoost. The optimal combination was found to be SMOTE and CatBoost. Additionally, explainable AI (XAI) techniques, particularly Local Interpretable Model-Agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP), were applied to identify key predictive variables, providing both local and global insights.

Authors

• Yeheun Baek, Chungnam National University

• Hyewon Chung, Chungnam National University