Multimodal Emotional Recognition for Human-Robot Interaction in Social Robotics

Resum

This study explores the enhancement of human-robot interaction (HRI) through multimodal emotional recognition within social robotics, using the humanoid robot Pepper as a testbed. Despite the advanced interactive capabilities of robots like Pepper, their ability to accurately interpret and respond to human emotions remains limited. This paper addresses these limitations by integrating visual, auditory, and textual analyses to improve emotion recognition accuracy and contextual understanding. By leveraging multimodal data, the study aims to facilitate more natural and effective interactions between humans and robots, particularly in assistive, educational, and healthcare settings. The methods employed include convolutional neural networks for visual emotion detection, audio processing techniques for auditory emotion analysis, and natural language processing for text-based sentiment analysis. The results demonstrate that the multimodal approach significantly enhances the robot’s interactive and empathetic capabilities. This paper discusses the specific improvements observed, the challenges encountered, and potential future directions for research in multimodal emotional recognition in HRI.