College - Author 1
Orfalea College of Business
Department - Author 1
Industrial Technology
College - Author 2
College of Engineering
Department - Author 2
Computer Engineering Department
Advisor
Javier Gonzalez-Sanchez, College of Engineering, Computer Science and Software Engineering Department; Rafael Guerra Silva, Orfalea College of Business, Industrial Technology and Packaging department
Funding Source
Summer Undergraduate Research Program of the Orfalea College of Business - Cal Poly SLO
Date
10-2025
Abstract/Summary
This project explores the use of Extended Reality (XR) technologies to enhance human- robot interaction in industrial contexts. Building upon prior research in affective and cognitive state recognition during human-cobot collaboration, this study investigates how natural hand and head gestures, captured through Meta Quest passthrough mode, can be used to communicate human intent to a Universal Robotics e-Series collaborative robot. The XR system provides users with an immersive, real-world visual interface while tracking motion and position in real time. The captured gestures are interpreted through a custom software pipeline that integrates machine learning models and rule-based logic to trigger adaptive robot behaviors. This hands-free, intuitive control mechanism has the potential to lower barriers to cobot use, improve safety, and support both skilled and unskilled workers in manufacturing tasks. The project will prototype and evaluate an XR-based interface for gesture recognition, assess its usability and responsiveness in collaborative tasks, and explore its potential for integration with affective data gathered from complementary sensors. Ultimately, this work aims to contribute to the development of human-centric, adaptive robot systems that respond not only to physical commands but also to the human’s cognitive and emotional context.
October 1, 2025.
Included in
URL: https://digitalcommons.calpoly.edu/ceng_surp/98