Designing Multimodal Emotional Expression for a Robotic Study Companion

Abstract

This thesis investigates the design, implementation, and evaluation of the emotional expression of the Robot Study Companion (RSC) intended to support university students’ motivation, wellbeing, and learning outcomes. Guided by a theoretical review of affective feedback in educational robotics, we identified six target emotions: Anger, Joy, Pride, Fun, Surprise, and Caring, each linked to educationally meaningful functions. We developed a fully functional digital twin of the RSC in ROS 2 to enable rapid prototyping and simulation, and built a modular emotional expression that coordinates color, motion, facial expression, and vocal behavior for each emotional response. To assess recognition accuracy and user experience, we conducted a video-based cross-cultural study with 47 participants. Participants distinguished positive versus negative valence with over 95% accuracy, and achieved an overall emotion-recognition rate of 58.16%. User Experience Questionnaire (UEQ) results were consistently positive, with Perspicuity and Attractiveness rated highest. Qualitative interviews highlighted the strengths of our multimodal design as well as opportunities for refinement. Cross-group analyses revealed systematic differences by gender, academic level, and nationality, underscoring the need for adaptive, personalized emotion profiles. All software developed in this work is released open-source, providing a robust foundation for future research in affective human–robot interaction within educational settings.