STRATUM is a Horizon Europe multi-partner project developing a clinically validated, real-time 3D decision support tool for brain tumour surgery. The system integrates Hyperspectral Imaging (HSI), AI-based multimodal data fusion, and heterogeneous High-Performance Computing (HPC) architectures combining Graphics Processing Units (GPUs), Field-Programmable Gate Arrays (FPGAs), and Processing-In-Memory (PIM) technologies. A touchless augmented reality interface facilitates safe and intuitive intraoperative interaction. The distinguishing characteristic of STRATUM is its end-to-end co-designed approach, which integrates advanced computing, state-of-the-art imaging and clinical expertise into a unified Point-of-Care (PoC) platform. Utilising a structured co-creation methodology involving surgeons, engineers, and social scientists, the project ensures usability, safety and regulatory compliance from its early design stages to its clinical validation. The usability of STRATUM will be tested in three hospitals located in different European regions with diverse conditions and regulations. This will allow to collect advice and remarks from surgical staff in a continuous co-creation and co-tuning protocol. Beyond its clinical objectives, STRATUM contributes to the advancement of heterogeneous computing for real-time diagnostics, AI acceleration in critical medical environments and energy-efficient system integration. Furthermore, it delivers open datasets, validated AI pipelines, and performance benchmarks with a view to fostering future research and industrial innovation in digital surgery. The STRATUM project establishes a replicable model for intelligent, human-centred computing integrating microelectronics, AI and medicine.The paper presents an overview of the project in terms of aims, concepts and technologies and the description of the state of the work when approaching the end of the second of the five years planned. Specifically, the outcomes of the steps related to the collaboration with surgeons and medical staff (co-creation process) and the intelligent Graphical User Interface (GUI) development will be described. The latter allows for contactless interaction of the surgeon with several functions that have already been developed in the system.

Integrated digital diagnostics can support complex surgical procedures in many anatomical sites, brain tumour surgery being the most complex. STRATUM is a 5-year Horizon Europe funded project with the goal of developing an innovative 3D decision support tool for brain tumour surgeries, based on real-time multimodal data processing using artificial intelligence algorithms. The proposed tool is envisioned as an energy-efficient Point-of-Care computing system to be integrated within neurosurgical workflows to aid surgeons to make informed, efficient, and accurate decisions during surgical procedures. The expected long-term impact of STRATUM is to reduce the duration of surgical procedures, thus decreasing patients' risks, but also optimising the resources of European health care systems.

Hyperspectral Imaging (HSI) is a novel non-invasive, label-free technique for rapid medical diagnosis, capable of capturing a wide range of the electromagnetic spectrum in numerous narrow spectral bands. This comprehensive data acquisition poses challenges in clinical settings due to patient movement (e.g., breathing, tremors, heartbeat). This work presents a method to compensate inter-band movements during HSI acquisition using spectral scanning acquisition systems. The system used in this work is based on a Liquid Crystal Tunable Filter (LCTF), featuring a Kurios VBl filter (Thorlabs, USA) and a CSl35 MUN monochrome camera, capturing the visible range (420–730 nm) in successive frames at different wavelengths. The proposed method uses iterative image registration to align each frame with the initial reference frame, correcting for movements along the x-axis, y-axis, and rotational shifts. The system's efficacy was validated using two datasets: static and dynamic. In the first one, the HS images were captured with no motion applied and later they were digitally deformed. The second dataset was acquired employing a custom motion platform. Results show improved alignment and reduced motion artifacts in the resulting HS images, highlighting the potential of the proposed method to improve the reliability of HSI in clinical applications.

Hyperspectral imaging (HSI) has gained prominence in medical diagnostics due to its ability to capture and analyse detailed spectral information beyond human visual capabilities. Processing of HSI data is essential to enhance subsequent analysis and ensure the accuracy of results by reducing noise and unwanted artifacts. This paper provides an overview of state-of-the-art processing methods for HSI data, focusing on smoothing, normalization, and spectral derivatives. The efficacy of these methods is evaluated using root mean square error (RMSE) to compare pre-processed data with wavelength reference standard, alongside execution time considerations. Results indicate that certain algorithms, such as smoothing based on moving average, standard normal variate, and first spectral derivatives, yield superior performance across different medical HSI systems. Additionally, combining these processing techniques further improves data fidelity to the wavelength reference standard. Overall, this study offers insights into optimal processing strategies for enhancing the accuracy and reliability of HSI data.

This paper presents the design of an FPGA-accelerated application for skin cancer detection which uses both hyperspectral imaging and a k-means algorithm. The accelerator is designed employing 3 FPGA kernels. The first 2 kernels filter and normalize the hyperspectral image. Then, the last kernel runs k-means to segment the image into three different regions according to the distribution of the lesion. This application is developed following the HLS methodology, implemented as an embedded system in MPSoC, and runs under Linux OS. FPGA acceleration will improve the application's throughput and energy efficiency significantly when compared to pure software execution.

The purpose of this paper is to outline the advances in hyperspectral (HS) image fusion for intraoperative delineation of brain tumor tissue. HS images were acquired using an intraoperative acquisition system based on two push-broom HS cameras, one covering the visible and near-infrared (VNIR) [400–1000 nm] and the other the near-infrared (NIR) [900–1700 nm] spectral range. A dataset of a wide range of in-vivo brain cancer acquired during neurosurgical procedures with both HS cameras was used to compare the performance results of using the VNIR and NIR data, independently and combining the VNIR-NIR data. Classification maps obtained using the fused VNIR-NIR images provide more accurate classification, removing false positives that appear when the VNIR and NIR images are processed independently.

Chemotherapy-induced peripheral neuropathy (CIPN) is a common adverse reaction produced by chemotherapy drugs used to treat cancer. A few of the most common symptoms are pain, discomfort, tingling, numbness, and weakness in the hands, feet, and other parts of the body. The use of ozone therapy (O3T) is a novel therapy which aims to reduce these side effects. This study focuses on the visualization of oxygen saturation (StO2) in the extremities in a non-contact fashion employing hyperspectral (HS) imaging (HSI) with the goal of using HSI as a predictive value in the objective assessment of pain after ozone therapy application. A customized acquisition system composed of an HS camera (covering the 470–900 nm spectral range) and a halogen illumination system was developed to capture images of the extremities in a non-contact approach. An experimental clinical procedure was established to measure the evolution of StO2in hands and feet using a mathematical model based on information related to two wavelengths (660 and 880 nm). The preliminary results show that, in general, when analyzing the extremities of all unified patients, StO2slightly improves but significatively in the peripheral tissues after ozone therapy (O3T). HS technology allows the estimation of StO2values and allows the quantification of the effect of O3T.

Recently, several medical applications have relied on hyperspectral imaging. This technology enables both automated diagnosis and surgeon guidance. The employed algorithms adopt machine and deep learning methods to classify the images. In particular, Vision Transformers are a recent deep architecture that have been used to classify hyperspectral images of skin cancers achieving interesting results. However, deep architectures are computationally intensive and parallel architectures are mandatory to ensure a fast classification (depending on the application type even in real time). In this paper, we propose a parallel Vision Transformer architecture exploiting a low power GPU targeting the development of a portable diagnostic device. The classification time and power consumption of the low power board are compared with the performance of a desktop GPU. The results clearly highlight the suitability of the low power GPU to develop a portable diagnostic system based on hyperspectral imaging.

Today, hyperspectral (HS) imaging has become a powerful tool to identify remotely the composition of an interest area through the joint acquisition of spatial and spectral information. However, like in most imaging techniques, unwanted effects may occur during data acquisition, such as noise, changes in light intensity, temperature differences, or optical variations. In HS imaging, these problems can be attenuated using a reflectance calibration stage and optical filtering. Nevertheless, optical filtering might induce some distortion that could complicate the posterior image processing stage. In this work, we present a new proposal for reflectance calibration that compensates for optical alterations during the acquisition of an HS image. The proposed methodology was evaluated on an HS image of synthetic squares of various materials with specific spectral responses. The results of our proposal show high performance in two classification tests using the K-means algorithm with 97% and 88% accuracy; in comparison with the standard reflectance calibration from the literature that obtained 77% and 64% accuracy. These results illustrate the performance gain of the proposed formulation, which besides maintaining the characteristic features of the compounds within the HS image, keeps the resulting reflectance into fixed lower and upper bounds, which avoids a post-calibration normalization step.

In recent years, hyperspectral imaging has been employed in several medical applications, targeting automatic diagnosis of different diseases. These images showed good performance in identifying different types of cancers. Among the methods used for classification, machine learning and deep learning techniques emerged as the most suitable algorithms to handle these data. In this paper, we propose a novel hyperspectral image classification architecture exploiting Vision Transformers. We validated the method on a real hyperspectral dataset containing 76 skin cancer images. Obtained results clearly highlight that the Vision Transforms are a suitable architecture for this task. Measured results outperform the state-of-the-art both in terms of false negative rates and of processing times. Finally, the attention mechanism is evaluated for the first time on medical hyperspectral images.

The early detection of precancerous cervical lesions is essential to improve patient treatment and prognosis. Current methods of screening and diagnosis have improved the detection of these lesions but still present some critical limitations. Hyperspectral (HS) imaging is emerging as a new non-invasive and label-free imaging technique in the medical field for performing quick diagnosis of different diseases. This work describes the first step in the research and development process to present to the gynaecologist a new non-invasive tool to detect cervical neoplasia during routine medical procedures. This tool is based on a HS camera coupled to a colposcope, a primary tool already used in cervical examinations. The developed HS colposcope was validated by comparing the HS images obtained against the captures obtained with conventional optics. Results show the feasibility of the developed system to start a data acquisition campaign of cervical lesions targeting future developments of algorithms based on artificial intelligence.

Oxygen saturation (StO2) measurement allows to detect different clinical conditions related with the low oxygenation of tissues or is used to monitor the quality and safety of organ transplantation. This study is focused on the visualization and measurement of StO2using hyperspectral imaging (HSI) through non-contact skin captures, targeting a potential real-time monitoring application. A customized acquisition system composed by a hyperspectral camera (covering the 470-900 nm spectral range) and a thermal camera was developed to capture images of hands in a non-contact fashion. An experimental procedure was established to measure the evolution of StO2in healthy hands where a compression of the index finger or brachial artery were performed. StO2measurements were performed in normal, compression, and reperfusion states. Two mathematical models with different sets of wavelengths were evaluated. The results show the proposed models, which employed two wavelengths (660 and 880 nm), obtain reliable StO2values, providing a potential non-contact imaging tool for StO2measurement.