Robotic Vision of Social Humanoid Robot SemuBot

Abstract

Social robots can be used in areas such as healthcare, education, and customer service due to their ability to interact socially. They aim to interact customizably, emotionally, and intelligently to promote trust, comfort, and induce real engagement. For effective interaction with people, a social robot has to identify and interpret human facial features such as age, gender, gaze, and emotive expressions. In this thesis, I developed SemuBot’s vision system, a social humanoid robot which is built to operate in a hospital environment. SemuBot can perform face recognition using OpenCV, gaze engagement estimation using dlib, personal identification by 128- dimensional facial embeddings through the face_recognition library, and perform emotion analysis with DeepFace. This permits the robot to effectively interact with individuals looking at the robot and optimize the interaction based on collected demographic and emotional data, thus improving real-time responsiveness. Using gaze direction detection with NumPy and dlib enables the robot to track only those looking at it, which enhances efficiency further. This helps prevent processing overload and minimizes lag in crowded settings. Focused interactions, as opposed to wide-area surveillance of all individuals in view, enhance real-time responsiveness and accuracy.