Gesture-Controlled Robotic Arm for Real-Time Surgical Tool Assistance
Gesture-Controlled Robotic Arm for Real-Time Surgical Tool Assistance
TLDR
This research introduces a robotic manipulation system that leverages YOLOv5 for accurate surgical tool detection and Mediapipe for real-time hand gesture tracking to control a UFactory Lite 6 robotic arm, which achieves rapid response times essential for critical procedures.
摘要
This research introduces a robotic manipulation system that leverages YOLOv5 for accurate surgical tool detection and Mediapipe for real-time hand gesture tracking to control a UFactory Lite 6 robotic arm. Designed for the surgical environment, this system enables intuitive, gesture-based control of instruments, minimizing manual interaction and maintaining sterility. Key contributions include a multithreaded, GPU-accelerated framework optimized for low latency, which achieves rapid response times essential for critical procedures. Results demonstrate the system’s effectiveness in reducing procedure time and cognitive load, significantly enhancing surgical workflow efficiency. Real world vigorous testing validates the design and methodology.
