Surgical Tool Occlusion Detection and Segmentation Using Deep Transfer Learning: A Machine Vision Approach

The combination of deep transfer learning and mask region-based convolutional neural network (Mask-RCNN) technology offers a groundbreaking approach to enhancing machine vision for surgical robots. The objective of this work is to enhance the accuracy and precision of identifying tools in surgical environments by investigating the use of deep transfer learning for detecting and segmenting occluded surgical tools. Our investigation exhibits the model's robustness in successfully recognizing and segmenting obstructed surgical tools, as seen by an exceptional Mean Average Precision (mAP) score of 93.8%. By conducting thorough testing and inference studies, the model demonstrated an outstanding ability to accurately identify various instruments, such as “mayo,” “clamp,” and “scalpel,” even in difficult situations when objects were partially occluded. The use of Mask-RCNN enhances our investigation, representing a crucial advancement in enhancing surgical robotics with sophisticated segmentation skills. The research underscores the potential of Mask-RCNN in the field of machine vision for surgical robots, underlining the necessity for ongoing progress to tackle complexities for improved real-world implementation.