top of page

AI and multispectral imaging facilitate keyhole surgery

How can camera images be used during minimally invasive surgeries to assess whether the operated organ is sufficiently perfused with blood? Scientists from the German Cancer Research Center (DKFZ) and the Städtisches Klinikum Karlsruhe have now succeeded in automatic monitoring of ischemia during a kidney surgery using only the optical properties of the tissue and not relying on injection of contrast agent. To do this, they combined a special imaging technique with artificial intelligence methods. The new method has now been used for the first time in patients with kidney tumors.

While conventional devices (right) generate an RGB video image, the newly developed laparoscope (left) uses a multispectral camera. This also makes it possible to visualize functional properties of the tissue. © Leonardo Ayala / DKFZ

Minimally invasive surgical techniques, also known as "keyhole surgery", are increasingly replacing conventional open operations - also in cancer care. For example, certain tumors of the internal organs can be removed in such laparoscopic procedures. This involves the use of an instrument known as a laparoscope, which is inserted into the abdomen through a small incision in the skin and then allows the physician to view the affected organs.


The video camera connected to the laparoscope delivers an image signal consisting of the three color channels red, green and blue (RGB). This color space is also used, for example, in televisions or smartphones. However, it has been shown that human tissue has optical properties that cannot be represented with this image information alone. In the scope of the European Research Council (ERC) project NEURAL spicing and in collaboration with the Städtisches Klinikum Karlsruhe, Lena Maier-Hein (DKFZ) and her team therefore investigate the use of multispectral imaging in this context. These multispectral camera systems not only capture the three wavelength ranges of the optical spectrum mentioned above, but provide a multispectral measurement covering over 16 wavelength ranges. This makes it possible to visualize certain functional properties of the tissue that are invisible to conventional camera systems, such as the blood perfusion status of an organ.


This is relevant, for example, in patients with kidney cancer when the tumor has to be removed surgically. During this intervention, the blood flow to the affected areas must be interrupted by clamping the arteries. Only relying on the conventional laparoscopic RGB video makes it virtually impossible to assess the state of tissue perfusion. Only by using a fluorescent contrast agent, which is injected into the patient and subsequently accumulates in the perfused tissues, the physician is able to see whether the correct segments of the organ have been cut off from the blood supply. If this was not successful, the contrast medium has to be washed out for about 30 minutes so that the procedure can then be repeated.


"We were able to show that blood perfusion in the kidney can be visualized without contrast agents and in real time using a combination of multispectral imaging and artificial intelligence," says Lena Meier-Hein. The new technology has now been successfully used in a study in ten patients who had to undergo partial nephrectomy. "Because our approach allows the blood perfusion status of the tissue to be monitored live, the clamping of the blood vessels can be corrected immediately if necessary. This reduces the duration of the procedure, making it safer for patients," says Leonardo Ayala, lead author of the study. The new technology also eliminates the risk of an allergic reaction to the contrast agent.


Two aspects played a decisive role in the acquisition and processing of the image data. The use of multispectral imaging systems in minimally invasive surgery has failed so far, among other things, due to the time required by the systems to acquire and process the data. If several seconds are needed to calculate a new image, the information is practically unusable during the procedure. By using novel multispectral camera technology, the researchers have now been able to solve this problem. Their newly developed system delivers 25 frames per second and thus a fluid video image. On the other hand, the perfusion estimation for the first time could be accomplished using AI methods that do not rely on large amounts of training data. The special feature here is that the AI learns to distinguish perfused from non-perfused tissue for each patient anew. "This is necessary because the tissue is not the same in every patient, so the optical properties can also be very different," explains Leonardo Ayala. Therefore, at the beginning of a surgery, a short video sequence of the kidney is recorded for each patient with the help of a laparoscope.


"The algorithm then calculates, based on the patient's individual image data, how the state of blood flow affects the optical properties of the tissue, so that the degree of blood flow can be displayed during surgery," Tim Adler, second lead author complements.


Comparable methods could also be used for other surgical issues in the future. Dogu Teber, clinical lead of the project, sees great potential for this new technological approach: "Spectral imaging in combination with new AI-based analysis tools could become an important tool for fast, reliable and safe functional imaging in minimally invasive surgery."


Leonardo Ayala, Tim J. Adler, Silvia Seidlitz, Sebastian Wirkert, Christina Engels, Alexander Seitel, Jan Sellner, Alexey Aksenov, Matthias Bodenbach, Pia Bader, Sebastian Baron, Anant Vemuri, Manuel Wiesenfarth, Nicholas Schreck, Diana Mindroc, Minu Tizabi, Sebastian Pirmann, Brittaney Everitt, Annette Kopp-Schneider, Dogu Teber, Lena Maier-Hein: Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery, Science Advances 2023, DOI: 10.5281/zenodo.7529803


Caption: While conventional devices (right) generate an RGB video image, the newly developed laparoscope (left) uses a multispectral camera. This also makes it possible to visualize functional properties of the tissue.


Note on the use of images for press releases Use is free of charge. The German Cancer Research Center (DKFZ) permits one-time use in connection with reporting on the subject of the press release or on DKFZ in general. Please indicate as photo credit: "Source: Leonardo Ayala / DKFZ". Distribution of the photographic material to third parties is only permitted after prior consultation with the DKFZ Press Office (Tel. 06221 42 2854, e-mail: presse@dkfz.de). Use for commercial purposes is prohibited.


Kommentarer


bottom of page