Reliable recognition and localization of surgical instruments in endoscopic video recordings are foundational for a wide range of applications in computer- and robot-assisted minimally invasive surgery (RAMIS), including surgical training, skill assessment, and autonomous assistance. However, robust performance under real-world conditions remains a significant challenge. Incorporating surgical context - such as the current procedural phase - has emerged as a promising strategy to improve robustness and interpretability. To address these challenges, we organized the Surgical Procedure Phase, Keypoint, and Instrument Recognition (PhaKIR) sub-challenge as part of the Endoscopic Vision (EndoVis) challenge at MICCAI 2024. We introduced a novel, multi-center dataset comprising thirteen full-length laparoscopic cholecystectomy videos collected from three distinct medical institutions, with unified annotations for three interrelated tasks: surgical phase recognition, instrument keypoint estimation, and instrument instance segmentation. Unlike existing datasets, ours enables joint investigation of instrument localization and procedural context within the same data while supporting the integration of temporal information across entire procedures. We report results and findings in accordance with the BIAS guidelines for biomedical image analysis challenges. The PhaKIR sub-challenge advances the field by providing a unique benchmark for developing temporally aware, context-driven methods in RAMIS and offers a high-quality resource to support future research in surgical scene understanding.

The field of computer vision applied to videos of minimally invasive surgery is ever-growing. Workflow recognition pertains to the automated recognition of various aspects of a surgery, including: which surgical steps are performed; and which surgical instruments are used. This information can later be used to assist clinicians when learning the surgery or during live surgery. The Pituitary Vision (PitVis) 2023 Challenge tasks the community to step and instrument recognition in videos of endoscopic pituitary surgery. This is a particularly challenging task when compared to other minimally invasive surgeries due to: the smaller working space, which limits and distorts vision; and higher frequency of instrument and step switching, which requires more precise model predictions. Participants were provided with 25-videos, with results presented at the MICCAI-2023 conference as part of the Endoscopic Vision 2023 Challenge in Vancouver, Canada, on 08-Oct-2023. There were 18-submissions from 9-teams across 6-countries, using a variety of deep learning models. The top performing model for step recognition utilised a transformer based architecture, uniquely using an autoregressive decoder with a positional encoding input. The top performing model for instrument recognition utilised a spatial encoder followed by a temporal encoder, which uniquely used a 2-layer temporal architecture. In both cases, these models outperformed purely spatial based models, illustrating the importance of sequential and temporal information. This PitVis-2023 therefore demonstrates state-of-the-art computer vision models in minimally invasive surgery are transferable to a new dataset. Benchmark results are provided in the paper, and the dataset is publicly available at: https://doi.org/10.5522/04/26531686.

Fetal growth restriction, affecting up to 10% of pregnancies, is a critical factor contributing to perinatal mortality and morbidity. Ultrasound measurements of the fetal abdominal circumference (AC) are a key aspect of monitoring fetal growth. However, the routine practice of biometric obstetric ultrasounds is limited in low-resource settings due to the high cost of sonography equipment and the scarcity of trained sonographers. To address this issue, we organized the ACOUSLIC-AI (Abdominal Circumference Operator-agnostic UltraSound measurement in Low-Income Countries) challenge to investigate the feasibility of automatically estimating fetal AC from blind-sweep ultrasound scans acquired by novice operators using low-cost devices. Training data, collected from three Public Health Units (PHUs) in Sierra Leone are made publicly available. Private validation and test sets, containing data from two PHUs in Tanzania and a European hospital, are provided through the Grand-Challenge platform. All sets were annotated by experienced readers. Sixteen international teams participated in this challenge, with six teams submitting to the Final Test Phase. In this article, we present the results of the three top-performing AI models from the ACOUSLIC-AI challenge, which are publicly accessible. We evaluate their performance in fetal abdomen frame selection, segmentation, abdominal circumference measurement, and compare their performance against clinical standards for fetal AC measurement. Clinical comparisons demonstrated that the limits of agreement (LoA) for A2 in fetal AC measurements are comparable to the interobserver LoA reported in the literature. The algorithms developed as part of the ACOUSLIC-AI challenge provide a benchmark for future algorithms on the selection and segmentation of fetal abdomen frames to further minimize fetal abdominal circumference measurement variability.

Radiation therapy plays a crucial role in cancer treatment, necessitating precise delivery of radiation to tumors while sparing healthy tissues over multiple days. Computed tomography (CT) is integral for treatment planning, offering electron density data crucial for accurate dose calculations. However, accurately representing patient anatomy is challenging, especially in adaptive radiotherapy, where CT is not acquired daily. Magnetic resonance imaging (MRI) provides superior soft-tissue contrast. Still, it lacks electron density information, while cone beam CT (CBCT) lacks direct electron density calibration and is mainly used for patient positioning. Adopting MRI-only or CBCT-based adaptive radiotherapy eliminates the need for CT planning but presents challenges. Synthetic CT (sCT) generation techniques aim to address these challenges by using image synthesis to bridge the gap between MRI, CBCT, and CT. The SynthRAD2023 challenge was organized to compare synthetic CT generation methods using multi-center ground truth data from 1080 patients, divided into two tasks: (1) MRI-to-CT and (2) CBCT-to-CT. The evaluation included image similarity and dose-based metrics from proton and photon plans. The challenge attracted significant participation, with 617 registrations and 22/17 valid submissions for tasks 1/2. Top-performing teams achieved high structural similarity indices (≥0.87/0.90) and gamma pass rates for photon (≥98.1%/99.0%) and proton (≥97.3%/97.0%) plans. However, no significant correlation was found between image similarity metrics and dose accuracy, emphasizing the need for dose evaluation when assessing the clinical applicability of sCT. SynthRAD2023 facilitated the investigation and benchmarking of sCT generation techniques, providing insights for developing MRI-only and CBCT-based adaptive radiotherapy. It showcased the growing capacity of deep learning to produce high-quality sCT, reducing reliance on conventional CT for treatment planning.

The alignment of tissue between histopathological whole-slide-images (WSI) is crucial for research and clinical applications. Advances in computing, deep learning, and availability of large WSI datasets have revolutionised WSI analysis. Therefore, the current state-of-the-art in WSI registration is unclear. To address this, we conducted the ACROBAT challenge, based on the largest WSI registration dataset to date, including 4,212 WSIs from 1,152 breast cancer patients. The challenge objective was to align WSIs of tissue that was stained with routine diagnostic immunohistochemistry to its H&E-stained counterpart. We compare the performance of eight WSI registration algorithms, including an investigation of the impact of different WSI properties and clinical covariates. We find that conceptually distinct WSI registration methods can lead to highly accurate registration performances and identify covariates that impact performances across methods. These results provide a comparison of the performance of current WSI registration methods and guide researchers in selecting and developing methods.

The early detection of glaucoma is essential in preventing visual impairment. Artificial intelligence (AI) can be used to analyze color fundus photographs (CFPs) in a cost-effective manner, making glaucoma screening more accessible. While AI models for glaucoma screening from CFPs have shown promising results in laboratory settings, their performance decreases significantly in real-world scenarios due to the presence of out-of-distribution and low-quality images. To address this issue, we propose the Artificial Intelligence for Robust Glaucoma Screening (AIROGS) challenge. This challenge includes a large dataset of around 113,000 images from about 60,000 patients and 500 different screening centers, and encourages the development of algorithms that are robust to ungradable and unexpected input data. We evaluated solutions from 14 teams in this paper and found that the best teams performed similarly to a set of 20 expert ophthalmologists and optometrists. The highest-scoring team achieved an area under the receiver operating characteristic curve of 0.99 (95% CI: 0.98-0.99) for detecting ungradable images on-the-fly. Additionally, many of the algorithms showed robust performance when tested on three other publicly available datasets. These results demonstrate the feasibility of robust AI-enabled glaucoma screening.

Domain Adaptation (DA) has recently been of strong interest in the medical imaging community. While a large variety of DA techniques have been proposed for image segmentation, most of these techniques have been validated either on private datasets or on small publicly available datasets. Moreover, these datasets mostly addressed single-class problems. To tackle these limitations, the Cross-Modality Domain Adaptation (crossMoDA) challenge was organised in conjunction with the 24th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2021). CrossMoDA is the first large and multi-class benchmark for unsupervised cross-modality Domain Adaptation. The goal of the challenge is to segment two key brain structures involved in the follow-up and treatment planning of vestibular schwannoma (VS): the VS and the cochleas. Currently, the diagnosis and surveillance in patients with VS are commonly performed using contrast-enhanced T1 (ceT1) MR imaging. However, there is growing interest in using non-contrast imaging sequences such as high-resolution T2 (hrT2) imaging. For this reason, we established an unsupervised cross-modality segmentation benchmark. The training dataset provides annotated ceT1 scans (N=105) and unpaired non-annotated hrT2 scans (N=105). The aim was to automatically perform unilateral VS and bilateral cochlea segmentation on hrT2 scans as provided in the testing set (N=137). This problem is particularly challenging given the large intensity distribution gap across the modalities and the small volume of the structures. A total of 55 teams from 16 countries submitted predictions to the validation leaderboard. Among them, 16 teams from 9 different countries submitted their algorithm for the evaluation phase. The level of performance reached by the top-performing teams is strikingly high (best median Dice score — VS: 88.4%; Cochleas: 85.7%) and close to full supervision (median Dice score — VS: 92.5%; Cochleas: 87.7%). All top-performing methods made use of an image-to-image translation approach to transform the source-domain images into pseudo-target-domain images. A segmentation network was then trained using these generated images and the manual annotations provided for the source image.

PURPOSE: Surgical workflow and skill analysis are key technologies for the next generation of cognitive surgical assistance systems. These systems could increase the safety of the operation through context-sensitive warnings and semi-autonomous robotic assistance or improve training of surgeons via data-driven feedback. In surgical workflow analysis up to 91% average precision has been reported for phase recognition on an open data single-center video dataset. In this work we investigated the generalizability of phase recognition algorithms in a multicenter setting including more difficult recognition tasks such as surgical action and surgical skill. METHODS: To achieve this goal, a dataset with 33 laparoscopic cholecystectomy videos from three surgical centers with a total operation time of 22 h was created. Labels included framewise annotation of seven surgical phases with 250 phase transitions, 5514 occurences of four surgical actions, 6980 occurences of 21 surgical instruments from seven instrument categories and 495 skill classifications in five skill dimensions. The dataset was used in the 2019 international Endoscopic Vision challenge, sub-challenge for surgical workflow and skill analysis. Here, 12 research teams trained and submitted their machine learning algorithms for recognition of phase, action, instrument and/or skill assessment. RESULTS: F1-scores were achieved for phase recognition between 23.9% and 67.7% (n = 9 teams), for instrument presence detection between 38.5% and 63.8% (n = 8 teams), but for action recognition only between 21.8% and 23.3% (n = 5 teams). The average absolute error for skill assessment was 0.78 (n = 1 team). CONCLUSION: Surgical workflow and skill analysis are promising technologies to support the surgical team, but there is still room for improvement, as shown by our comparison of machine learning algorithms. This novel HeiChole benchmark can be used for comparable evaluation and validation of future work. In future studies, it is of utmost importance to create more open, high-quality datasets in order to allow the development of artificial intelligence and cognitive robotics in surgery.

This paper proposes an automatic classification method based on machine learning in contrast-enhanced ultrasonography (CEUS) of focal liver lesions using the contrast agent Sonazoid. This method yields spatial and temporal features in the arterial phase, portal phase, and post-vascular phase, as well as max-hold images. The lesions are classified as benign or malignant and again as benign, hepatocellular carcinoma (HCC), or metastatic liver tumor using support vector machines (SVM) with a combination of selected optimal features. Experimental results using 98 subjects indicated that the benign and malignant classification has 94.0% sensitivity, 87.1% specificity, and 91.8% accuracy, and the accuracy of the benign, HCC, and metastatic liver tumor classifications are 84.4%, 87.7%, and 85.7%, respectively. The selected features in the SVM indicate that combining features from the three phases are important for classifying FLLs, especially, for the benign and malignant classifications. The experimental results are consistent with CEUS guidelines for diagnosing FLLs. This research can be considered to be a validation study, that confirms the importance of using features from these phases of the examination in a quantitative manner. In addition, the experimental results indicate that for the benign and malignant classifications, the specificity without the post-vascular phase features is significantly lower than the specificity with the post-vascular phase features. We also conducted an experiment on the operator dependency of setting regions of interest and observed that the intra-operator and inter-operator kappa coefficients were 0.45 and 0.77, respectively.

This paper presents a low power and low noise sub-harmonically injection-locked PLL using a 20GHz sub-sampling PLL (SS-PLL) and a quadrature injection locked oscillator (QILO). Lower in-band phase noise and out-of-band phase noise have been achieved through the sub-sampling phase detection and sub-harmonic injection techniques, respectively. Implemented in a 65nm CMOS, this work can support all 60GHz channels and achieves a phase noise of -115dBc/Hz at 10MHz offset while consuming 20.2mW and 14mW from the 20GHz SS-PLL and the QILO, respectively.

This paper presents a small area, low power, fully synthesizable PLL with a current output DAC and an interpolative-phase coupled oscillator using edge injection technique for on-chip clock generation. A prototype PLL is fabricated in a 65nm digital CMOS process, achieves a 1.7-ps integrated jitter at 0.9 GHz and consumes 0.78 mW leading to an FOM of -236.5 dB while only occupying an area of 0.0066 mm2. It achieves the best performance-area trade-off.

This paper proposes a high-accuracy image correction method using SIFT (Scale-Invariant Feature Transform) and POC (Phase-Only Correlation) for multi-projector display. The accurate correspondence between the projector and camera images is required to achieve seamless imagery in a multiprojector display. The conventional methods need to project and take special light patterns on a screen many times to obtain the correspondence. On the other hand, the proposed method needs to take only one snapshot of ordinary images so as to realize real-time geometric correction of projector images. Through a set of experiments, we demonstrate that the proposed method is effective for practical use of multi-projector display compared with the conventional methods.

We propose a video coding technique using super-resolution image processing. The proposed method reduces spatial resolution of specific pictures in input video sequence at first, namely it produces a mixed-resolution video sequence. And then the proposed method encodes the mixed-resolution video sequence. The spatially reduced pictures are converted to original resolution pictures with a super-resolution image processing. The proposed method can reduce the bit rate compared to conventional methods such as H.264, since it encodes the video sequence in which several pictures are spatially reduced. Moreover, the proposed method can prevent deterioration in subjective quality, since the spatially reduced pictures are restored with the super-resolution image processing. We evaluate the proposed method by implementing it on an H.264-based video coding scheme. Experimental results confirm that compared to H.264, our implementation of the proposed method can achieve a 3.5 dB improvement in PSNR at a low bit rate.

This paper proposes a new motion compensation method using "sliced blocks" in hybrid video coding. In H.264, a brand-new international video coding standard, motion compensation can be performed by splitting macroblocks into multiple square or rectangular regions. In the proposed method, on the other hand, macroblocks or sub-macroblocks are divided into two regions (sliced blocks) by an arbitrary line segment. The result is that the shapes of the segmented regions are not limited to squares or rectangles, allowing the shapes of the segmented regions to better match the boundaries between moving objects. Thus, the proposed method can improve the performance of the motion compensation. In addition, adaptive prediction of the shape according to the region shape of the surrounding macroblocks can reduce overheads to describe shape information in the bitstream. The proposed method also has the advantage that conventional coding techniques such as mode decision using rate-distortion optimization can be utilized, since coding processes such as frequency transform and quantization are performed on a macroblock basis, similar to the conventional coding methods. The proposed method is implemented in an H.264-based P-picture codec and an improvement in bit rate of 5% is confirmed in comparison with H.264.

We propose a new frame-rate up-conversion algorithm using a vector reliable analysis scheme that reduces artifacts caused by the use of inaccurately transmitted motion vectors (MVs). In conventional up-conversion algorithms using motion estimation (ME), ME is performed between two adjacent decoded frames to construct MVs that allow the frame to be interpolated, irrespective of the amount of calculation for ME required. On the other hand, in conventional up-conversion algorithms using transmitted MVs, the quality of the interpolated frame depends largely on the MV that is derived by the encoder. In our proposed scheme, transmitted MVs are first analyzed to decide whether or not they are usable for constructing interpolation frames. The interpolation method is then adaptively selected from three methods: local motion-compensated interpolation, global motion-compensated interpolation and frame-repeated interpolation. The proposed method provides high quality interpolated frames and decreases the volume of calculation needed, making it especially well suited to multimedia mobile devices that depend on low-power processing.

This paper proposes a new method to generate intra prediction images in intra coding, tree structured hybrid intra prediction. When generating intra prediction images, the proposed method selects the most suitable one from intra prediction images generated using two methods, pixel-based and block-based intra prediction. This selection is performed on macroblock or sub macroblock level based on a rate distortion optimization scheme. In addition, information indicating which prediction method is selected is described in the bitstream using a tree structure. The proposed method is initially described, and then the results of evaluation experiments are shown by comparison to conventional methods. Evaluation is carried out by implementing the proposed method in an H.264-based intra picture codec. The results of evaluation confirm that the proposed method improves PSNR by up to nearly 0.7 dB compared to conventional methods over a wide range of bit rates.