Motion2Vec: Algorithm designed by Google Brain, Intel and UC Berkeley for robotic surgery

In the field of medicine, it is notable to highlight how technology has been making it possible to develop increasingly sophisticated resources and tools to optimize the work performed by health personnel working in hospitals.

Even virtual reality has been used as a means to facilitate the learning of medical students when it comes to performing surgery.

Focusing on this area, researchers from Google Brain, UC Berkeley and Intel AI Lab joined forces to carry out the development of Motion2Vec, an AI model endowed with the ability to obtain knowledge related to the performance of robotic surgery tasks, such as suture, pass and insert needles and tie knots. All this, as part of the training provided by a surgical video.

As a way to test the results, the researchers set out to implement the model in a two-armed Da Vinci robot, which passed the needle through a piece of cloth in the laboratory.

This is how Motion2Vec constitutes a representation learning algorithm that is subjected to training through the use of semi-supervised learning, thus following the same behavior pattern applied in previous models such as Word2Vec and Grasp2Vec.

On the other hand, the researchers expressed that their work is a sample of how video robotics used in surgery can be taken to a higher level, supporting it with demonstration videos from experts. This, as a way of providing knowledge for the acquisition of new robotic manipulation skills.

Regarding this, the researchers published in their article the following

The results suggest an improvement in segmentation performance over the last generation baselines, while introducing pose imitation in this dataset with a 0:94 cm error in the position per observation respectively. .

Motion2Vec Training

As for the details about Motion2Vec, these were posted last week in the arXiv prepress repository, and were later presented at the IEEE International Conference on Robotics and Automation (ICRA).

For the algorithm training, videos were used showing 8 human surgeons controlling the Da Vinci robots. These videos were obtained from the JIGSAWS data set, acronyms used to refer to the JHU-ISI Gesture and Skills Assessment Working Set, a project made up of videos from John Hopkins University (JHU for its acronym in English) and Intuitive Surgery, Inc (ISI).

These JIGSAWS videos showed the Motion2Vec motion-focused representations of manipulative skills through imitation learning.

In reference to this, the article points out the following

We used a total of 78 demonstrations of the suture data set []The style of suturing, however, is significantly different for each surgeon.