In multi-view classification tasks (MVC), each view provides an unique perspective on the data, offering complementary information that can improve classification performance when properly integrated. However, traditional methods typically adopt a uniform processing strategy for all views before fusion, overlooking the fact that different views may require different treatments due to variations in their quality and informativeness. To address this limitation, we propose a novel framework called Uncertainty-Guided View-Strength-Aware Feature Utilization (UVF) for multi-view classification. Our approach introduces a view uncertainty estimation module to quantify the discriminative strength of each view. Based on this estimation, a Differentiated Feature Selector (DFS) adaptively selects features, retaining informative dimensions in weak views while preserving original features in strong views. Furthermore, we employ an uncertainty-guided fusion strategy that assigns dynamic weights to each view's contribution based on its uncertainty score, enhancing the robustness and reliability of the final decision. Experimental results on benchmark datasets demonstrate that our method significantly outperforms conventional approaches, achieving better classification accuracy and interpretability through strength-aware feature processing and fusion.

Determining the reasonable grasping order and reducing the interference to surrounding objects are critical for robotic grasping under mutually-stacked scenes. However, existing manipulation relationship reasoning methods generally predict object relationships from detection perspective, and require dense evaluation of all object pairs, leading to restricted reasoning accuracy caused by detection bias and relationship class imbalance. To solve this problem, we propose an EFficient Visual Manipulation Relationship Reasoning Network, called EF-VMR2N, with relationship attention and sparse graph. Specifically, relationship attention constructs two new attention mechanisms to holistically leverage the correlation characteristics between visual features and semantic information among object pairs by highlighting reasoning-specific features. Sparse graph infers the mutual relationships within the scene using fixed number of strongly-correlated triplet sets to improve the efficiency and accuracy of object pair evaluation. Extensive experiments on the VMRD and REGRAD datasets both show that the proposed EF-VMR2N achieves SOTA performance in terms of four evaluation metrics (mAP, OR, OP and IA), and compared with object detection, the improvements on manipulation reasoning are more remarkable. The stacked objects grasping under real-world scenarios further proves the effectiveness of the proposed method. Video: https://github.com/LiMing336/VideoDemonstration.

Most existing robotic grasp detection methods can achieve high accuracy in one single domain, but due to the differences in data distribution and object categories among different datasets, the accuracies of these methods decrease significantly in other domains. To solve this problem, we propose a novel domain-invariant grasp detection network with scene prototypes and structure priors to extract robust grasping features. First, a universal prototype guidance strategy, which models the hidden domain scenes and obtains universal scene prototypes based on their similarities, is designed to distinguish foreground objects from background in different domains. The structure perception enhancement method based on structure priors is then constructed to carry out fine-grained modeling of foreground regions and capture more detailed local information. In addition, we also design repulsive constraint regularization to assist the training process to alleviate the issue of similar prior distribution in vector space. Extensive comparative experiments are conducted on four public datasets and the results show that the proposed method achieves significant improvements of detection accuracy under cross-dataset scenarios. As a plug-in module, it could improve the generalization of existing grasp detection approaches by a large margin. Real-world robotic grasping experiments are also deployed to verify its effectiveness.

Microscopic 3D shape reconstruction using depth from focus (DFF) is crucial in precision manufacturing for 3D modeling and quality control. However, the absence of high-precision microscopic DFF datasets and the significant differences between existing DFF datasets and microscopic DFF data in optical design, imaging principles and scene characteristics hinder the performance of current DFF models in microscopic tasks. To address this, we introduce M3D, a novel microscopic DFF dataset, constructed using a self-developed microscopic device. It includes multi-focus image sequences of 1,952 scenes across five categories, with depth labels obtained through the 3D TFT algorithm applied to dense image sequences for initial depth estimation and calibration. All labels are then compared and analyzed against the design values, and those with large errors are eliminated. We also propose M3DNet, a frequency-aware end-to-end network, to tackle challenges like shallow depth-of-field (DoF) and weak textures. Results show that M3D compensates for the limitations of macroscopic DFF datasets and extends DFF applications to microscopic scenarios. M3DNet effectively captures rapid focus decay and improves performance on public DFF datasets by leveraging superior global feature extraction. Additionally, it exhibits strong robustness even in extreme conditions. Dataset and code are available at https://github.com/jiangfeng-Z/M3D.

Reliable 6D pose estimation is crucial for robotic tasks but presents significant challenges in environments with occlusion. Recent approaches tend to directly predict pose parameters of object with deep neural networks, lacking the modeling ability of non-adjacent and complex relationships of surface points in occluded scenarios. To solve this problem, we propose a novel occlusion-aware 6D pose estimation framework, which uses depth-guided graph neural network (GNN) to model potential relationships from RGBD input. Two semantic information, which are mask and binary code of object, are adaptively fused to extract 2D-3D correspondence related features in an effective manner. Both enhanced graph features and fused semantic information contribute to the performance improvement of pose estimation with occlusion. Extensive experiments indicate that our approach outperforms comparative methods by 1.2% and 1.9% on LMO and YCBV datasets (up to 30% for certain objects) and its validity is also verified under real-world pose estimation test.

In multi-view classification tasks, integrating information from multiple views effectively is crucial for improving model performance. However, most existing methods fail to fully leverage the complex relationships between views, often treating them independently or using static fusion strategies. In this paper, we propose a View-Association-Guided Dynamic Multi-View Classification method (AssoDMVC) to address these limitations. Our approach dynamically models and incorporates the relationships between different views during the classification process. Specifically, we introduce a view-relation-guided mechanism that captures the dependencies and interactions between views, allowing for more flexible and adaptive feature fusion. This dynamic fusion strategy ensures that each view contributes optimally based on its contextual relevance and the inter-view relationships. Extensive experiments on multiple benchmark datasets demonstrate that our method outperforms traditional multi-view classification techniques, offering a more robust and efficient solution for tasks involving complex multi-view data.

Deep learning-based speech enhancement (SE) methods predominantly draw upon two architectural frameworks: generative adversarial networks and diffusion models. In the realm of SE, capturing the local and global relations between signal frames is crucial for the success of these methods. These frameworks typically employ a UNet architecture as their foundational backbone, integrating Long Short-Term Memory (LSTM) networks or attention mechanisms within the UNet to effectively model both local and global signal relations. However, the coupled relation modeling way may not fully harness the potential of these relations. In this paper, we propose an innovative Association-based Fusion Speech Enhancement method (AFSE), a decoupled method. AFSE first constructs a graph that encapsulates the association between each time window of the speech signal, and then models the global relations between frames by fusing the features of these time windows in a manner akin to graph neural networks. Furthermore, AFSE leverages a UNet with dilated convolutions to model the local relations, enabling the network to maintain a high-resolution representation while benefiting from a wider receptive field. Experimental results demonstrate that the AFSE method significantly improves performance in speech enhancement tasks, validating the effectiveness and superiority of our approach. The code is available at https://github.com/jie019/AFSE_IJCAI2025.

In large aperture imaging, the shallow depth of field (DoF) phenomenon requires capturing multiple images at different focal levels, allowing us to infer depth information using depth from focus (DFF) techniques. However, most previous works design convolutional neural networks from a time domain perspective, often leading to blurred fine details in depth estimation. In this work, we propose a frequency-aware deep DFF network (FAD) that couples multi-scale spatial domain local features with frequency domain global structural features. Our main innovations include two key points: First, we introduce a frequency domain feature extraction module that uses the Fourier transform to transfer latent focus features into the frequency domain. This module adaptively captures essential frequency information for focus changes through element-wise multiplication, enhancing fine details in depth results while preserving global structural integrity. Second, the time-frequency joint module of FAD improves the consistency of depth information in sparse texture regions and the continuity in transition areas from both local and global complementary perspectives. Comprehensive experiments demonstrate that our model achieves compelling generalization and state-of-the-art depth prediction across various datasets. Additionally, it can be quickly adapted to real-world applications as a pre-trained model.

Segmentation is an important prerequisite for developing model healthcare systems, particularly for disease diagnosis and treatment planning. In the field of medical image segmentation, the U-shaped architecture, commonly referred to as U-Net, has emerged as the de facto standard and achieved remarkable success. However, due to the intrinsic locality of convolution operations, U-Net generally demonstrates limitations in explicitly modeling long-range dependency. Recent transformer-based models, designed for sequence-to-sequence prediction, have emerged as an alternative to traditional architectures, featuring innate global self-attention mechanisms. Unfortunately, they may sometimes suffer from limited localization abilities due to a lack of sufficient low-level details. To merit both Transformers and U-Net, in this paper, we propose a novel two-channel self-attention mechanism U-network, which performs feature extraction from two channels, CNN and Transformer, respectively. Compared to previous models, we propose two hierarchical feature fusion strategies from both spatial and channel dimensions. Moreover, to further promote the model performance, a loss function that can dynamically adjust the weights according to the output of each layer is constructed. Experimental results on five different datasets show that our method performs consistently outperforms state-of-the-art methods, and it also has an outstanding generalization ability to various medical image modalities.

Most available grasp detection methods tend to directly predict grasp configurations with deep neural networks, where all features are equally extracted and utilized, leading to the relative restriction of truly useful grasping features. Inspired by the observed three-section structure pattern revealed by human-labeled graspable rectangles, we first design a structure prior attention (SPA) module which uses two-dimensional encoding to enhance the local patterns and utilizes self-attention mechanism to reallocate distribution of grasping-specific features. Then, the proposed SPA module is integrated with fundamental feature extraction modules and residual connection to achieve the implicit and explicit feature fusion, which further serves as the building block of our proposed Unet-like grasp detection network. It takes RGBD images as input and outputs image-size feature maps, from which the grasp configurations can be determined. Extensive comparative experiments on the five public datasets prove our method’s superiority to other approaches in detection accuracy, achieving 99.2%, 96.1%, 98.0%, 86.7%, and 92.6% on the Cornell, Jacquard, Clutter, VMRD, and GraspNet datasets. With visual evaluation metrics and user study, the quality maps generated by our method possess more concentrative distribution of high-confidence grasps and clearer discrimination with backgrounds. In addition, its effectiveness is also verified by robotic grasping under real-world scenario, leading to higher success rate.