Occluded person re-identification (ReID) poses substantial challenges in computer vision, primarily due to incomplete information and occlusion interference. Although Transformer architectures have become dominant in ReID due to their strong feature modeling capabilities, their lack of an adaptive weight allocation mechanism for multi-granularity feature processing limits their ability to extract generalizable and robust features. Recently, Masked Image Modeling (MIM) has demonstrated considerable promise in visual tasks, but its integration into ReID models remains underexplored. This paper presents AMFOR (Adaptive Multi-granularity feature Fusion and Occlusion Reconstruction), a novel framework combining MIM and Transformer architectures. AMFOR consists of three key components: AMFF-Encoder, HPR-Decoder, and Teacher-Student Decoder. The AMFF-Encoder enables adaptive fusion of multi-granularity features through learnable queries, allowing interaction between text-visual features and visual features extracted from multiple Transformer layers. The HPR-Decoder conceptualizes occluded regions in pedestrian images as reconstructable patches, guiding the encoder to extract more discriminative features through reconstruction. Additionally, the self-distillation teacher-student decoder is employed to refine pedestrian part features, further optimized by the proposed AMGDLoss. This paper represents the first successful implementation of the MIM mechanism in person ReID models. Empirical evaluations on five benchmark datasets, covering both occluded (Occluded-DukeMTMC, Occluded-REID, and P-DukeMTMC-reID) and complete (Market-1501 and DukeMTMC-reID) scenarios, demonstrate that AMFOR outperforms existing state-of-the-art methods in person ReID. Our code is available athttps://github.com/Guangdeng-Li/AMFOR

Single-angle plane wave (SAPW) ultrasound imaging has gained significant attention in ultrafast and wearable ultrasound systems due to its high frame rate and hardware efficiency. However, the lack of transmit diversity leads to severe degradation in image quality, limiting its clinical utility. In contrast, Coherent Plane Wave Compounding (CPWC) improves spatial resolution and contrast by aggregating data from multiple transmission angles, but at the expense of a substantially reduced frame rate. To overcome this trade-off, we propose TF-Fusion, a novel Time-Frequency Domain Feature Fusion framework that enhances SAPW imaging by jointly exploiting both time- and frequency-domain features extracted from raw in-phase/quadrature (IQ) data. Specifically, our model utilizes a dual-branch encoder to learn compact and complementary representations from each domain, followed by a learnable fusion module that adaptively integrates the multi-domain information. This design facilitates effective noise and artifact suppression while preserving fine anatomical structures. We evaluate TF-Fusion on two public benchmarks—PICMUS and CUBDL—and demonstrate that our method significantly improves image resolution and contrast, achieving performance comparable to multi-angle CPWC methods. Notably, TF-Fusion maintains the high temporal resolution of SAPW, making it well-suited for real-time and resource-constrained ultrasound imaging scenarios.

Left ventricular segmentation in echocardiographic videos plays a crucial role in assessing various heart functions and disease diagnoses. However, due to the dynamic nature of echocardiography, maintaining consistent target segmentation across context shifts between frames poses a significant challenge. This task becomes even more difficult in the presence of noise interference in ultrasound images. In this paper, we propose DiffSegMem, which leverages the generative advantages of conditional diffusion model and a contextual memory storage architecture to enhance the accuracy of cross-frame segmentation in dynamic echocardiographic videos. Specifically, we introduce a noise-frequency domain aware dual-branch conditional encoding that establishes noise-resistant conditions at each sampling step, providing a reliable mask for the first frame of the echocardiographic sequence. As a result, our method does not require any prompting for the first video frame. Additionally, we propose a dynamic memory propagation strategy that utilizes memory transfer switches to extract memory from deeply linked video frames using a positional switch, with memory blocks carrying contextual clues prompting segmentation frame by frame. We evaluate our method on publicly available echocardiographic segmentation datasets and demonstrate state-of-the-art performance compared to existing models, outperforming current supervised methods for prompt-based segmentation.

Vision-Language Models (VLMs) have great potential for advancing echocardiography (Echo) probe navigation, which is crucial for assisting sonographers in standardized view acquisition. However, the high clinical deployment costs and spurious correlations pose major challenges for VLM-based probe navigation. To address these challenges, we propose Contrastive Language-Image Navigator for Latent Hyperbolic Geometric Knowledge Distillation (CLINav-GKD), a novel VLM-based 6-DOF Echo probe navigation framework. Specifically, Contrastive Language-Image Navigator (CLN) proposes a lightweight VLM-based 6-DOF navigator, reducing deployment costs while improving sensitivity to quality variations. Latent Hy perbolic Geometric Distiller (LGD) models the global geometric-topology between samples, mitigating spurious correlations and enhancing robustness. We train CLINav-GKD on real-world data with probe motion trajectories. Experimental results show that CLINav-GKD outperforms other VLM-based distillation methods by 2.8%, 3.8%, and 3.6% in probe navigation, achieving a superior balance of accuracy, robustness, and deployability for real-world clinical use. Code and data are available at https://github.com/DaisyLi0516/CLINav-GKD.

In the treatment of coronary artery disease, the stent implantation strategy (SIS) plays a decisive role in both the procedural success rate and the long-term prognosis of patients. However, SIS determination is primarily based on the experience of the doctors, visual evaluation based on coronary angiography, and the real-time interpretation of intravascular ultrasound (IVUS) during the procedure. To improve the rationality of stent implantation and reduce postoperative adverse events, we propose an objective, automated, and efficient method to evaluate the rationality of SIS. Specifically, a time-series chunk encoder is constructed to extract temporal features from IVUS video, enabling the model to handle long-sequence feature extraction while accommodating input videos of varying lengths. Furthermore, a frequency domain feature encoder is employed to extract spectral characteristics from IVUS video. Meanwhile, stent, balloon, and placement information is input into the model in text form, with feature extraction performed using two pretrained BERT models in medical Chinese and medical English. In order to ensure the discriminability of text features, a text memory bank and text matching loss function are designed to participate in model training in conjunction with other commonly used loss functions. Finally, two Transformer Layers integrate the temporal, spectral, and textual modalities, and a fully connected layer is used to determine the rationality of the SIS. Validation in a private rationality evaluation dataset for SIS demonstrates that our method effectively performs rationality evaluation tasks, providing valuable auxiliary recommendations for clinical decision-making.

The Rydberg atomic receiver (RARE) has garnered increasing attention in quantum communication due to its capability for high-precision signal sensing and detection. Recent advancements have led to the integration of RARE into multiple-input multiple-output (MIMO) systems. Signal detection in RARE MIMO systems presents a distinct biased phase retrieval (PR) challenge compared to conventional MIMO detection problems associated with radio frequency (RF) chains, rendering many traditional high-performance MIMO detectors inapplicable. This paper investigates the multiuser RARE MIMO problem under M -ary phase-shift keying (MPSK) modulations. The central challenge is to jointly address the biased PR formulation and the discrete MPSK constellation—an area not extensively explored in existing literature. We develop a custom approach that employs a smoothing technique to alleviate the nonsmoothness in the biased PR objective and a penalty transformation to tackle the discrete MPSK structure. The resulting algorithm combines a Majorization-Minimization (MM) framework with a modified Wirtinger flow (WF) method. Numerical simulations demonstrate that our proposed approach achieves superior detection accuracy compared to state-of-the-art detectors while maintaining lower computational complexity.

Nowadays, personal privacy protection is extremely emphasised. Face de-identification is considered as an effective way to protect the visual privacy through disguising or replacing identity attributes. Existing methods compromise either high fidelity or reversibility. To address these issues, this paper proposes DiffDeid, the first diffusion-based face de-identification and recovery method. Leveraging recent diffusion inversion and control technicques, DiffDeid achieves both high quality imperceptible de-identification and exact recovery with passwords. DiffDeid has three attractions: (1) It can generate de-identified faces with the superior fidelity while maintaining other non-identity attributes for visual tasks. (2) The correct password is powerful enough to restore facial images with extreme details. Meanwhile, incorrect passwords can lead to vastly different decryption results. (3) DiffDeid demands minimal computing resources and instant training time compared to others. We conducted experiments on various face datasets to showcase the superiority of our proposed method. Additional experiments show that DiffDeid is powerful with diverse text prompts and control instructions even beyond human faces. Codes are available at project page.

Intravascular ultrasound (IVUS) not only reveals changes within the vascular lumen but also illustrates the cross-sectional structure, encompassing aspects such as plaques, vessel wall thickness, morphology, and composition. However, during the image acquisition process, ultrasound imaging of vessels can lead to intraluminal misalignment due to motion, resulting in inaccurate measurement outcomes. Current methods for motion artifact removal in IVUS face the following challenges: (a) Gating, which extracts key gating frames to form a new artifact-free sequence, but often results in the loss of substantial useful information; and (b) Direct artifact removal, which requires lumen segmentation followed by registration, where the accuracy of registration is highly dependent on segmentation accuracy. To address these challenges, this paper proposes a robust direct artifact removal method based on spatiotemporal representations. Specifically, to address the issue of information loss in gating methods, a spatiotemporal representation network is introduced, which primarily relies on temporal granularity normalization and spatial position compensation. To tackle the problem of direct artifact removal methods heavily relying on segmentation accuracy, this paper integrates the segmentation network with the artifact removal network, allowing for mutual supervision. This approach ensures effective motion artifact removal even when segmentation accuracy is not optimal. Experimental results show that our method not only achieves state-of-the-art (SOTA) performance in both quantitative and qualitative evaluations but also maintains robust motion artifact removal even when the segmentation network is replaced with one of lower performance.

Designing an effective representation learning method for multimodal sentiment analysis is a critical research area. The primary challenge is capturing shared and private information within a comprehensive modal representation, especially when dealing with uniform multimodal labels and raw feature fusion.To overcome this challenge, we propose a novel deep modal shared information learning module that utilizes the covariance matrix to capture shared information across modalities. Additionally, we introduce a label generation module based on a self-supervised learning strategy to capture the private information specific to each modality. Our module can be easily integrated into multimodal tasks and offers flexibility by allowing parameter adjustment to control the information exchange relationship between modes, facilitating the learning of private or shared information as needed. To further enhance performance, we employ a multi-task learning strategy that enables the model to focus on modal differentiation during training. We provide a detailed formulation derivation and feasibility proof for the design of the deep modal shared information learning module.To evaluate our approach, we conduct extensive experiments on three common multimodal sentiment analysis benchmark datasets. The experimental results validate the reliability of our model, demonstrating its effectiveness in capturing nuanced information in multimodal sentiment analysis tasks.

This article proposes a multimode neuromorphic event-frame integrated vision sensor that enables event detection (ED) with simultaneous brightness measurement based on the pulse width modulation mechanism. The logarithmic voltage is directly taken as intensity information. Brightness measurement involves in-pixel voltage-to-pulse conversion and out-pixel pulse coding. The maximum event bandwidth is improved to 366 Meps by pipelining the time-prior arbiter along with the address-events grouping circuit. A wide intensity dynamic range of 105 dB can theoretically be achieved through logarithmic photoelectric conversion and pulse coding. Our sensor supports a$128\times 64$frame-like image with an improved signal-to-noise ratio of 49 dB. The experimental results indicate that the log sensitivity of the optimized logarithmic photoreceptor was measured as 164 mV/dec. The equivalent frame rate for both event and intensity reaches kilo fps, making it a promising candidate in high-speed wireless sensing applications.

Thanks to its powerful ability to depict high-resolution anatomical information, magnetic resonance imaging (MRI) has become an essential non-invasive scanning technique in clinical practice. However, excessive acquisition time often leads to the degradation of image quality and psychological discomfort among subjects, hindering its further popularization. Besides reconstructing images from the undersampled protocol itself, multi-contrast MRI protocols bring promising solutions by leveraging additional morphological priors for the target modality. Nevertheless, previous multi-contrast techniques mainly adopt a simple fusion mechanism that inevitably ignores valuable knowledge. In this work, we propose a novel multi-contrast complementary information aggregation network named MCCA, aiming to exploit available complementary representations fully to reconstruct the undersampled modality. Specifically, a multi-scale feature fusion mechanism has been introduced to incorporate complementary-transferable knowledge into the target modality. Moreover, a hybrid convolution transformer block was developed to extract global-local context dependencies simultaneously, which combines the advantages of CNNs while maintaining the merits of Transformers. Compared to existing MRI reconstruction methods, the proposed method has demonstrated its superiority through extensive experiments on different datasets under different acceleration factors and undersampling patterns.

Human activity recognition (HAR) technology based on wearables has received increasing attention in recent years. The traditional methods have used hand-crafted features to recognize human activities, resulting in shallow feature extraction. With the development of deep learning, an increasing number of researchers have focused on studying deep learning methods. To achieve higher recognition accuracy, the majority of the current HAR research involves multisource and multimodal sensors (MMSs) data. However, due to the limitations in the receptive fields of single-dimensional convolutional kernels, these networks are still infeasible for extracting spatiotemporal features. In this study, a multidimensional parallel convolutional connected (MPCC) deep learning network based on MMS data for HAR is proposed that fully utilizes the advantages of multidimensional convolutional kernels. Moreover, multiscale residual convolutional squeeze-and-excitation (MRCSE) modules are proposed to enrich the diversity of feature information by combining squeeze-and-excitation (SE) blocks. A daily home activity (DHA) data set is constructed based on the requirements for HAR in certain scenarios, such as smart home, and we conduct experiments on the optimal combination of sensor locations on the DHA data set according to a weighted$\text{F}1~({\mathrm{ F}}_{\mathrm{ W}})$-score. Both tenfold and leave-one-subject-out (LOSO) cross-validations (CVs) are used to evaluate the performance of the proposed network. The MPCC-MRCSE network achieves${\mathrm{ F}}_{\mathrm{ W}}$-scores of 98.33% and 95.42% on the physical activity monitoring for aging people (PAMAP2) and OPPORTUNITY data sets using tenfold CVs, respectively, and achieves${\mathrm{ F}}_{\mathrm{ W}}$-scores of 81.47% on the PAMAP2 when applying an LOSO CV.

The recognition performance of existing vision-based human action recognition (HAR) methods is greatly reduced in the case of low camera resolution or occlusion. Wearable sensors can provide complementary information to alleviate this problem. It is challenging to construct a robust HAR model using multimodal wearable-sensor data. In this paper, we propose a cross-Attention-based Multimodal fusion Wavelet Knowledge Distillation Network (MAWKDN) method to guide recognition from video data by acquiring complementary information from wearable sensors and reduce the noise effects through wavelet knowledge distillation, which improves the robustness of the model. A multi-attention dilated convolution kernel residual network including dilated convolution and an attention mechanism is constructed to extract features from various sensor modalities and fuse the various modal data through the cross-view attention method to acquire additional information from different modalities. To reduce the modal differences between different modalities of the teacher and student networks and acquire similar semantic knowledge, we learn the information between different modalities by constructing a graph structure of convolutional layer features, and computing the semantic preservation loss between the teacher and student networks. To reduce the influence of noise in the input data, we construct the loss of wavelet knowledge distillation, which transforms the image through the discrete wavelet transform and only retains the low frequency features to extract the useful information. The top-1 accuracy achieved on the UTD-MHAD (99.31%), Berkeley-MHAD (99.40%) and the F1-score on the MMAct (85.26% based on cross-session) dataset prove the superior performance of MAWKDN compared with the state-of-the-art HAR methods. Moreover, we demonstrate the robustness of the MAWKDN approach on the noise-added UTD-MHAD dataset.

Clinical workflow of cardiac assessment on 2D echocardiography requires both accurate segmentation and quantification of the Left Ventricle (LV) from paired apical 4-chamber and 2-chamber. Moreover, uncertainty estimation is significant in clinically understanding the performance of a model. However, current research on 2D echocardiography ignores this vital task while joint segmentation with quantification, hence motivating the need for a unified optimization method. In this paper, we propose a multitask model with Task Relation Spatial co-Attention (referred as TRSA-Net) for joint segmentation, quantification, and uncertainty estimation on paired 2D echo. TRSA-Net achieves multitask joint learning by novelly exploring the spatial correlation between tasks. The task relation spatial co-attention learns the spatial mapping among task-specific features by non-local and co-excitation, which forcibly joints embedded spatial information in the segmentation and quantification. The Boundary-aware Structure Consistency (BSC) and Joint Indices Constraint (JIC) are integrated into the multitask learning optimization objective to guide the learning of segmentation and quantification paths. The BSC creatively promotes structural similarity of predictions, and JIC explores the internal relationship between three quantitative indices. We validate the efficacy of our TRSA-Net on the public CAMUS dataset. Extensive comparison and ablation experiments show that our approach can achieve competitive segmentation performance and highly accurate results on quantification.

The aim of this paper is to provide a comprehensive overview of the MICCAI 2020 AutoImplant Challenge. The approaches and publications submitted and accepted within the challenge will be summarized and reported, highlighting common algorithmic trends and algorithmic diversity. Furthermore, the evaluation results will be presented, compared and discussed in regard to the challenge aim: seeking for low cost, fast and fully automated solutions for cranial implant design. Based on feedback from collaborating neurosurgeons, this paper concludes by stating open issues and post-challenge requirements for intra-operative use. The codes can be found at https://github.com/Jianningli/tmi.

Magnetic resonance imaging (MRI) vertebral localization, identification, and segmentation are important steps in the automatic analysis of spines. Due to the similar appearances of vertebrae, the accurate segmentation, localization, and identification of vertebrae remain challenging. Previous methods solved the three tasks independently, ignoring the intrinsic correlation among them. In this paper, we propose a multi-task relational learning network (MRLN) that utilizes both the relationships between vertebrae and the relevance of the three tasks. A dilation convolution group is used to expand the receptive field, and LSTM(Long Short-Term Memory) to learn the prior knowledge of the order relationship between the vertebral bodies. We introduce a co-attention module to learn the correlation information, localization-guided segmentation attention(LGSA) and segmentation-guided localization attention(SGLA), in the decoder stage of segmentation and localization tasks. Learning two tasks simultaneously as well as the correlation between tasks can not only avoid the overfitting of a single task but also correct each other. To avoids the cumbersome weight adjustment for different tasks loss functions, we formulated a novel XOR loss that provides a direct evaluation criterion for the localization relationship of the semantic location regression and semantic segmentation. This method was evaluated on a dataset which includes multiple MRI modalities (T1 and T2), various fields of view. Experimental results demonstrate that both of the co-attention and XOR loss work outperforms the most recent state of art.

MOTIVATION: Advances in neuroimaging and sequencing techniques provide an unprecedented opportunity to map the function of brain regions and identify the roots of psychiatric diseases. However, the results from most neuroimaging studies, i.e. activated clusters/regions or functional connectivities between brain regions, frequently cannot be conveniently and systematically interpreted, rendering the biological meaning unclear. RESULTS: We describe a brain annotation toolbox that generates functional and genetic annotations for neuroimaging results. The voxel-level functional description from the Neurosynth database and gene expression profile from the Allen Human Brain Atlas are used to generate functional/genetic information for region-level neuroimaging results. The validity of the approach is demonstrated by showing that the functional and genetic annotations for specific brain regions are consistent with each other; and further the region by region functional similarity network and genetic similarity network are highly correlated for major brain atlases. One application of brain annotation toolbox is to help provide functional/genetic annotations for newly discovered regions with unknown functions, e.g. the 97 new regions identified in the Human Connectome Project. Importantly, this toolbox can help understand differences between psychiatric patients and controls, and this is demonstrated using schizophrenia and autism data, for which the functional and genetic annotations for the neuroimaging changes in patients are consistent with each other and help interpret the results. AVAILABILITY AND IMPLEMENTATION: BAT is implemented as a free and open-source MATLAB toolbox and is publicly available at http://123.56.224.61:1313/post/bat. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

A class-specific random forest (RF) model with cross-correlation constraints is developed for the spectral-spatial hyperspectral image (HSI) classification. The novelties of this letter are as follows: 1) normalization of the spectral feature vector by using cross correlation in the stochastic process and proposal of a spectral-spatial hybrid feature extraction based on the cross-correlation analysis; 2) establishment of an RF classifier model by using class-specific trees (CSTs); and 3) exploration of the performance of the proposed method by comparing with its several traditional classification methods on two real HSI data sets. Further research on the effects of parameter setup, such as the number of CSTs, spectral constraint scale, and size of the spatial neighbor, is discussed in terms of classification accuracy. Experimental results show that the performance of the proposed method is better than that of the traditional methods.

This letter explores a spectral-spatial tensor-based dimensionality reduction (DR) method to cope with hyperspectral image (HSI) feature extraction and classification. This method uses the Gabor filter banks as the bias spectral-spatial feature hybrider and further integrates the tensor-based alignment strategy for the discriminant locality with sparse factorization by extracting optimal spectral-spatial features and simultaneously maintaining structural relevance. Comparative experimental results with two real HSIs demonstrate that the proposed DR method has a considerable advantage over other traditional feature extraction methods.

In web mining, there are situations in which only few data is labeled which imposes difficulties on traditional web page classification algorithms. Active learning scheme is then proposed to sample the most representative unlabeled data, which are then annotated by external oracles. Most present active methods are based on series-mode query strategy, which deduces the process of active learning inefficient and unstable. In this paper, we propose a novel text oriented active semi-supervised classification model, which is so-called active SSC. Comparing with other active approaches, our model has the characteristic of comprehensibility, and thus it is easy to design a batch-mode query strategy. Experimental results on public text data showed our method is an effect and stable active approach.