In recent years, video stabilization methods have demonstrated significant achievements. However, learning-based video stabilization methods needs large paired unstable and stable video data, which is hard to collect. Besides, lots of online video stabilization methods always estimate trajectory by information of previous and several future frames, which is disadvantageous for devices that capture real-time video. In this paper, we propose Unsupervised Multi-stage Online Video Stabilization Network (UMOS-Net) to adapt trajectory with variable dynamic scenes by accurately perceiving the connections between current and previous frames, which does not reference future frames. Here, UMOS-Net is divided into two stages: trajectory estimation and trajectory smoothing. In trajectory estimation stage, considering the registration errors caused by blur or deformation in frames, such as fast-moving scenes, multi-scale content-aware modules are combined with adaptive optical flow loss to increase adaptability in the scenes with low overlap rates. Furthermore, lots of methods not only use several future frames, but use only the original camera trajectory as prior that accumulate from the first frame. The original camera trajectory is difficult to effectively smooth the current frame trajectory for cumulative error caused by low correlation of long-term frames. Therefore, a window-based dual-branch input recurrent neural network is designed in trajectory smoothing stage, in which stabilized historical trajectory is added as a supplement to get better trajectory smoothing results even in fast-moving scenes. Experiments show that our proposed UMOS-Net can obtain competitive results both qualitatively and quantitatively comparing to current representative online methods, especially in fast-moving scenes.

Recently, diffusion-based image compression has achieved significant progress in terms of rate–distortion-perception trade-off, these approaches have replaced decoders with conditional diffusion models to enhance the visual quality of reconstructed images. However, diffusion models introduce noise into the input image during the initial stages of the diffusion process, which may cause the potential degradation of crucial image information. To address these limitations, we propose a Hierarchical Conditional guidance Diffusion model for perceptual Image Compression (HCD-IC) to ensure the fidelity of reconstruction, in which hierarchical features with selected typical context provide informative guidance during the denoising process of diffusion model to preserve both structural integrity and fine details. Specifically, we design a Gated Scale-Cross module (GSC) to integrate and select representative semantics and details, which leverages a hierarchical feature interaction architecture and dynamic gated strategy to ensure more robust and expressive representations. Furthermore, we present a Conditional Control Diffusion decode module (CCD) to integrate time-step information and latent features augmented by GSC into the diffusion model, which can dynamically acquire the required time-aware conditional features during different denoise stages. Extensive experiments conducted on multiple public datasets demonstrate that our method outperforms state-of-the-art approaches in various quantitative realism metrics.

Abstract Doctors typically use non‐contrast‐enhanced computed tomography (NCECT) in the treatment of kidney cancer to map kidney and tumour structural information to functional imaging single‐photon emission computed tomography, which is then used to assess patient kidney function and predict postoperative recovery. However, the assessment of kidney function and formulation of surgical plans is constrained by the low contrast of tumours in NCECT, which hinders the acquisition of accurate tumour boundaries. Therefore, this study designed a radiomics feature‐enhanced level‐set evolution (RFLSE) to precisely segment small‐sample low‐contrast kidney tumours. Integration of high‐dimensional radiomics features into the level‐set energy function enhances the edge detection capability of low‐contrast kidney tumours. The use of sensitive radiomics features to control the regional term parameters achieves adaptive adjustment of the curve evolution amplitude, improving the level‐set segmentation process. The experimental data used low‐contrast, limited‐sample tumours provided by hospitals, as well as the public datasets BUSI18 and KiTS19. Comparative results with advanced energy functionals and deep learning models demonstrate the precision and robustness of RFLSE segmentation. Additionally, the application value of RFLSE in assisting doctors with accurately marking tumours and generating high‐quality pseudo‐labels for deep learning datasets is demonstrated.

Data‐hunger is a persistent challenge in machine learning, particularly in the field of image processing based on convolutional neural networks (CNNs). This study systematically investigates the factors contributing to data‐hunger in machine‐learning‐based image‐processing algorithms. The results revealed that the proliferation of model parameters, the lack of interpretability, and the complexity of model structure are significant factors influencing data‐hunger. Based on these findings, this paper introduces a novel semi‐white‐box neural network model construction strategy. This approach effectively reduces the number of model parameters while enhancing the interpretability of model components. It accomplishes this by constraining uninterpretable processes within the model and leveraging prior knowledge of image processing for model. Rather than relying on a single all‐in‐one model, a semi‐white‐box model is composed of multiple smaller models, each responsible for extracting fundamental semantic features. The final output is derived from these features and prior knowledge. The proposed strategy holds the potential to substantially decrease data requirements under specific data source conditions while improving the interpretability of model components. Validation experiments are conducted on well‐established datasets, including MNIST, Fashion MNIST, CIFAR, and generated data. The results demonstrate the superiority of the semi‐white‐box strategy over the traditional all‐in‐one approach in terms of accuracy when trained with equivalent data volumes. Impressively, on the tested datasets, a simplified semi‐white‐box model achieves performance close to that of ResNet while utilizing a small number of parameters. Furthermore, the semi‐white‐box strategy offers improved interpretability and parameter reusability features that are challenging to achieve with the all‐in‐one approach. In conclusion, this paper contributes to mitigating data‐hunger challenges in machine‐learning‐based image processing through the introduction of a novel semi‐white‐box model construction strategy, backed by empirical evidence of its effectiveness.

Compared to computed tomography (CT), magnetic resonance imaging (MRI) is more sensitive to acute ischemic stroke lesion. However, MRI is time-consuming, expensive, and susceptible to interference from metal implants. Generating MRI images from CT images can address the limitations of MRI. The key problem in the process is obtaining lesion information from CT. In this study, we propose a cross-modal image generation algorithm from CT to MRI for acute ischemic stroke by combining radiomics with generative adversarial networks. First, the lesion candidate region was obtained using radiomics, the radiomic features of the region were extracted, and the feature with the largest information gain was selected and visualized as a feature map. Then, the concatenation of the extracted feature map and the CT image was input in the generator. We added a residual module after the downsampling of the generator, following the general shape of U-Net, which can deepen the network without causing degradation problems. In addition, we introduced the lesion feature similarity loss function to focus the model on the similarity of the lesion. Through the subjective judgment of two experienced radiologists and using evaluation metrics, the results showed that the generated MRI images were very similar to the real MRI images. Moreover, the locations of the lesions were correct, and the shapes of lesions were similar to those of the real lesions, which can help doctors with timely diagnosis and treatment.