Utterance-Level Text Emotion Classification on MELD: A Performance and Inference-Efficiency Comparison of BERT-Base and DistilBERT

Abstrak

Text-based emotion detection presents a significant challenge in the development of modern interactive systems, given the lack of non-verbal cues in digital communication. This study evaluates two Transformer-based models, namely BERT-Base and DistilBERT, in the task of single-label multi-class classification emotion classification using the Multimodal EmotionLines Dataset, which consists of multi-utterance conversations from the TV series Friends. The research focuses on the models’ performance in understanding emotions from individual text-based utterances. BERT-Base is known for its high accuracy in capturing semantic context but requires substantial computational resources. In contrast, DistilBERT is a lightweight version offering better computational efficiency, though its performance relative to BERT-Base in the context of emotion detection needs further investigation. This study involved data collection and preprocessing, model training with consistent hyperparameter settings, and evaluation using accuracy, precision, recall, and F1-score metrics. Each utterance was classified independently without conversational context. BERT-Base achieved 65% accuracy, a macro-F1 score of 0.39, and an inference time of 9.448 ms, while DistilBERT achieved 63% accuracy, a macro-F1 score of 0.40, and an inference time of 5.238 ms. These results indicate that DistilBERT provides comparable classification performance with approximately 44.6% lower inference latency, although its suitability for resource-constrained devices requires further evaluation.