We are already experiencing a shortage of physicians and enduring the consequences for patient care. Can mobile surgical robots provide a solution?
Combining approaches from robotics with AI methods using the future mobile communications standard 6G, researchers at RPTU and the German Research Center for Artificial Intelligence (DFKI) are investigating this question. Their objective is an analysis to show the potential of remote-controlled robotics for surgical operations and requirements for AI and communication networks. The researchers will present their project at the Medica trade fair from November 13 to 16.
Using robotics during operations is not a new idea. What is innovative is the approach of making such systems operable remotely or mobile. Project leader Marc Ruffing, who is doing research at the Institute for Wireless Communication and Navigation at RPTU and in the research area “Intelligent Networks” at DFKI, said: “Until now, the person performing the operation has to be present near the operating room, because mobile use of remotely controlled robotics is still impractical due to the size and infrastructure of such systems. Moreover, the control of the systems is not intuitive. The person who operates uses joysticks and uses a shielded screen. Both natural motion and haptic feedback are missing. Unlike humans, robots are not sensitive enough.”
High latency requirement
Overcoming these hurdles is what Ruffing and his team have made their research task. To do so, they use a demonstrator that consists of two collaborative robot arms and thus represents a classic setup. One robot arm can be controlled by human hands through the guidance of the other. The system enables virtual haptic feedback in the form of force feedback, which is transmitted from the controlled robot arm to the controlling robot arm.
This allows the controlling person to sense what he or she is doing. Communication between the two robots takes place via a network. During this process, no large data packets are transmitted – rather, the tricky part is a different one: “Particularly with such highly sensitive activities as remote-controlled operations, there are especially high requirements in terms of latency times. Control commands from the person performing the operation must arrive at the treatment site without any delay,” said team leader Christoph Lipps.
“That’s why we’re using the test scenario to define the requirements that the future 6G mobile communications standard must meet in terms of real-time control.”
More natural control
At the same time, the team is researching how intelligent technologies can be used to improve the system’s operation. Among other things, they are testing a near-infrared-based motion capture system. This allows objects such as a hand and their movements to be captured with millimetre precision in space. This would eliminate the need for unnatural joystick control.
It is also possible to integrate a human-machine interface (brain-computer interface, BCI) into the system.
“By measuring an individual’s brain waves using electroencephalography or near-infrared spectroscopy, we can obtain data that provide information about their state of mind,” said Matthias Rüb, a research associate on the team.
“An artificial neural network, which is an application from the field of machine learning, is used for the evaluation. It scans the data measured by BCI and assigns states of mind to them. If the doctor’s attention decreases or their stress level increases, for example, a warning message could be sent.”
Contribution to “Open6GHub”
The team will use the findings from the project to contribute to the specifications for real-time remote control of surgical robots.
“We are not developing medical devices,” Ruffing explained.
“We are more concerned with defining requirements for 6G and for AI in order to bring the technology into use. For example, in the form of a mobile operating room or built into an ambulance.”
The researchers will use the demonstrator to convey the status of their research to visitors at the Medica trade fair.
The project is part of the “Open6GHub” which is coordinated by Professor Schotten, Head of the Institute for Wireless Communication and Navigation at the RPTU and Head of the Intelligent Networks Research Department at the DFKI. In addition to RPTU and DFKI, other universities and research institutes are also involved. The partners in the research alliance want to contribute to the development of an overall 6G architecture and also to launch end-to-end solutions in the following and other areas: Advanced network topologies with highly agile so-called organic networking, security and resilience, Thz and photonic transmission methods, sensor functionalities in the networks and their intelligent use and further processing, and application-specific radio protocols.
As they move forward, the researchers are open to dialogue and collaboration.
“We are looking for an early and interactive dialogue with the public and are equally ready for collaborations with industry and users,” Schotten said.
“To this end, we will install OpenLabs and open experimental fields. Last but not least, we want to promote an open innovation system by involving SMEs and startups and their results.”
Jim Cornall is editor of Deeptech Digest and publisher at Ayr Coastal Media. He is an award-winning writer, editor, photographer, broadcaster, designer and author. Contact Jim here.
