Development of an intelligent surgical training system for Thoracentesis

Highlights：

• We implemented the surgical process model with ontology and production rules.

• We implemented ontology-based and marker-based recognition of surgical instruments and materials.

• We evaluated individual system components i.e. knowledge module comprising ontology and production rules, computer vision module comprising tracking and recognition of surgical instruments and materials, and data monitoring and Graphical User Interface. We also evaluated the training system with 10 naïve participants.

• We validated the ontology using annotated surgical videos, where the system identified “Anaesthesia” and “Aspiration” phase with 100% relative frequency and “Penetration” phase with 65% relative frequency. The system tracked surgical swab and 50 mL syringe with approximately 88.23% and 100% accuracy in surgeon’s hands and recognised surgical instruments with approximately 90% accuracy on the surgical stand.

摘要

•We developed a novel context-aware software framework for intelligent surgical training system for Thoracentesis using knowledge representation, computer vision and semantic web technologies. The developed surgical software framework could be used as a new tool for context awareness and decision-making during surgical training.•We implemented the surgical process model with ontology and production rules.•We implemented ontology-based and marker-based recognition of surgical instruments and materials.•We evaluated individual system components i.e. knowledge module comprising ontology and production rules, computer vision module comprising tracking and recognition of surgical instruments and materials, and data monitoring and Graphical User Interface. We also evaluated the training system with 10 naïve participants.•We validated the ontology using annotated surgical videos, where the system identified “Anaesthesia” and “Aspiration” phase with 100% relative frequency and “Penetration” phase with 65% relative frequency. The system tracked surgical swab and 50 mL syringe with approximately 88.23% and 100% accuracy in surgeon’s hands and recognised surgical instruments with approximately 90% accuracy on the surgical stand.