Recent deep model pruning methods predominantly focus on large-scale datasets and typically require finetuning before deployment. However, in real-world applications, pruning is often necessary for scenarios with fewer classification categories, where finetuning must be avoided to preserve the model’s generalization ability. To address these challenges, we introduce a novel pruning method called Cluster-based Redundancy Elimination (CRE). Specifically, CRE represents each convolutional kernel as a point in a high-dimensional space. A distance-based strategy is then used to compute a clustering radius for each convolutional layer. Based on these radii, core point filters are selected for pruning, as they represent redundant information that can be captured by neighboring filters in the high-dimensional space. This approach eliminates the need for finetuning, thus preserving the generalization of deep models. Extensive experiments on five benchmark datasets with limited classification categories, across multiple model architectures, demonstrate the effectiveness of our method and its superiority over several state-of-the-art pruning techniques.

Deep learning has achieved significant success in stereo matching, with its training process often supervised by LiDAR measurements. However, the sparsity of real-world LiDAR data limits the ability of deep models to extract effective features from stereo images. To address this issue, a novel deep learning-based framework called sparse LiDAR point cloud supervised stereo matching (SLSM-Net) is proposed. Specifically, dense reconstruction of sparse single-frame point clouds is first designed to avoid the error introduction with the mergence of multi-frame point clouds. To effectively densify point clouds of objects in local areas, stereo images are utilized as supervision information to train the deep models. Furthermore, a coarse-to-fine structure of the deep model is designed for stereo matching. A self-supervised learning strategy, which employs a photometric consistency constraint, is second proposed along with fully supervised learning to obtain dense and precise supervision information. This stage generates coarse disparity maps from stereo images. Finally, to fully leverage the complementary characteristics of LiDAR and stereo cameras, multi-scale feature fusion of point clouds and stereo images is performed by a residual block, where the feature maps of point clouds are derived from the densification reconstruction. This stage refines the results. Experimental results indicate that SLSM-Net outperforms current state-of-the-art methods, demonstrating superior performance in stereo matching.

Current deep model pruning methods mainly focus on large datasets and often involve finetuning before deployment. However, in real-world applications, pruning is typically needed for open scenarios with few classification categories, where fine-tuning should be avoided to preserve the generalization of deep models. To address these issues, we propose a novel deep model pruning method called Cluster-based Redundancy Elimination (CRE). Specifically, CRE first represents each convolutional kernel as a point in high-dimensional space. Second, a distance-based strategy is employed to compute a clustering radius for each convolutional layer. Finally, based on these clustering radii, core point filters are selected for pruning, as they extract redundant information that can be captured by neighboring filters in the high-dimensional space. Thus, finetuning deep models to recover their performance can be removed. Comprehensive experiments on five datasets with few categories validate the effectiveness of our approach and demonstrate its superiority over several state-of-the-art pruning methods.

High-dimensional omics data are often contaminated by sources of unwanted variations caused by platforms, batches, or other external factors. These interferences and noise can obscure critical signals related to cancer. Contaminated data are modeled as a combination of variables derived from the phenotype of interest (POI) and confounding factors. To identify these variables, a novel method called Decision Variable Analysis (DVA) is proposed. The novelty of DVA is to iteratively extract independent decisive variables for modeling the data. Specifically, a priori knowledge introduced as the definite variable linked with POI is removed from data through a residual operation. The number of variables is estimated from the residual matrix based on the zero gradient of singular values, rather than relying on random matrix theory or principal components analysis, which can produce unreliable results when the number of features exceeds the number of samples. Applications of DVA to both synthetic and real data demonstrate superior performance in identifying variables compared to conventional approaches. Improvements offered by DVA are illustrated across high-dimensional omics datasets, particularly those with smaller sample sizes relative to the number of features on different platforms. The results indicate that DVA is an effective method for dissecting sources of variation in high-dimensional data with disturbances.

The reconstruction of dense point clouds is an important foundation for downstream applications, such as object detection, semantic classification and surface reconstruction. Current methods focus on dense point cloud reconstruction for objects but neglect the whole scene. To address this issue, the reconstruction of dense point clouds supervised by inverse stereo matching (IS-Dense) is proposed. In detail, the Transformer model is first used to extract deep features form the point clouds. Second, point cloud features are expanded through the base upsampler. Ultimately, the point clouds would be coordinated following the feature expansion. Due to the uneven distribution of point clouds in the whole scene, some gaps and anomalies are presented in the data. Therefore, a point location refinement module supervised by inverse stereo matching is designed to solve this problem. For this module, the key is to utilize the reconstructed dense point clouds and the right image to estimate the left image. Supervised by real left images, the reconstructed dense point clouds are precise and even-distributed. The experimental results prove the superiority of the proposed method over current methods, especially for the whole scene.

The spectral super-resolution (SpeSR) from multispectral images (MSIs) to hyperspectral images (HSIs) can bring rich spectral information. The deep learning-based methods have demonstrated their powerful ability for the SpeSR task, which requires the paired HSI/MSI to train the model. However, HSIs and MSIs are always obtained at different times and under different imaging conditions, covering different areas. To address this issue, in this paper, a framework named BliEstGAN based on the generative adversarial network (GAN) is proposed to estimate the spectral resolution degradation between unpaired HSIs and MSIs that can be used for the blind SpeSR. Specifically, each MSI imaging sensor has its own unique spectral sampling process, which can be modeled as a spectral degradation from its paired HSI. Different spectral degradations can be discriminated by the deep model. Therefore, the generator of the GAN is used to estimate the spectral degradation from HSIs to MSIs, and the discriminator of the GAN is adopted to distinguish whether the estimated and real spectral degradation are similar. The large difference in spatial resolution between MSIs and HSIs makes them easy to discriminate against. Therefore, smooth hyperspectral and multispectral patches are extracted from HSIs and MSIs to eliminate this difference in spatial resolution. Furthermore, according to the imaging sensor mechanism, some special regularization terms are designed for the generator to guarantee its correct convergence. Finally, the estimated spectral resolution degradation can be adopted to generate HSI/MSI pairs for the supervised learning-based SpeSR methods. Experimental results demonstrate the effectiveness of the proposed method.

Understanding and elucidating human behavior across diverse scenarios represents a pivotal research challenge in pursuing seamless human-computer interaction. However, previous research on multi-participant dialogues has mostly relied on proprietary datasets, which are not standardized and openly accessible. To propel advancements in this domain, the MultiMediate'23 Challenge presents two sub-challenges: Eye contact detection and Next speaker prediction, aiming to foster a comprehensive understanding of multi-participant behavior. To tackle these challenges, we propose a multi-scale conformer fusion network (MSCFN) for enhancing the perception of multi-participant group behaviors. The conformer block combines the strengths of transformers and convolution networks to facilitate the establishment of global and local contextual relationships between sequences. Then the output features from all Conformer blocks are concatenated to fusion multi-scale representations. Our proposed method was evaluated using the officially provided dataset, and it achieves the best and second best performance in next speaker prediction and gaze detection tasks of MultiMediate'23, respectively.

The low resolution of remote sensing images often limits the land cover classification (LCC) performance. Super resolution (SR) can improve the image resolution, while greatly increasing the computational burden for the LCC due to the larger size of the input image. In this article, the SR-guided deep network (SRGDN) framework is proposed, which can generate meaningful structures from higher resolution images to improve the LCC performance without consuming more computational costs. In general, the SRGDN consists of two branches (i.e., SR branch and LCC branch) and a guidance module. The SR branch aims to increase the resolution of remote sensing images. Since high- and low-resolution image pairs cannot be directly provided by imaging sensors to train the SR branch, we introduce a self-supervised generative adversarial network (GAN) to estimate the downsampling kernel that can produce these image pairs. The LCC branch adopts the high-resolution network (HRNet) to retain as much resolution information with a few downsampling operations as possible. The guidance module teaches the LCC branch to learn the high-resolution information from the SR branch without the utilization of the higher-resolution images as the inputs. Furthermore, the guidance module introduces spatial pyramid pooling (SPP) to match the feature maps of different sizes in the two branches. In the testing stage, the guidance module and SR branch can be removed, and therefore do not create additional computational costs. Experimental results on three real datasets demonstrate the superiority of the proposed method over several well-known LCC approaches.

Convolutional neural network (CNN) has demonstrated to be a powerful tool for hyperspectral images (HSIs) classification. Previous CNN-based HSI classification methods only adopt the fixed-size patches to train the CNN model, and such single scale patches may not reflect the complex spatial structural information in the HSIs. In addition, although different layers of CNN can extract features of multiple scales, the traditional CNN model can only utilize features from the highest level for the classification task. These features, however, do not fully consider the strong complementary yet correlated information among different layers. To address these issues, in this paper, a multiscale densely-connected convolutional network (MS-DenseNet) framework is proposed to sufficiently exploit multiple scales information for the HSIs classification. Specifically, for each pixel, the MS-DenseNet, first, extracts its surrounding patches of multiple scales. These patches can separately constitute multiple scale training and testing samples. Within each specific scale sample, instead of using the forward convolutional layers, the MS-DenseNet adopts the dense blocks, which can connect each layer to other layers in a feed-forward fashion and thus can exploit the information among different layers for training and testing. Furthermore, since high correlations exist in patches of different scales, the MS-DenseNet introduces several dense blocks to fuse the multiscale information among different layers for the final HSI classification. Experimental results on several real HSIs demonstrate the superiority of the proposed MS-DenseNet over single scale-based CNN classification model and several well-known classification methods.

Sea land segmentation is an important technique in many remote sensing applications, such as coastline surveillance and near-shore ship detection. High resolution satellite optical imaging is able to capture the details in the coastal regions, introducing intra-class variance and interferences like waves, shadow and forestry regions. To overcome such variance and interferences, the high resolution optical satellite image is first over-segmented into super-pixels, i.e., homogeneous regions. Then the conditional random fields (CRFs) are adopted to model the relations of the super-pixels. The optimal labels of all the superpixels are determined by performing the loopy belief propagation on the CRFs. The final segmentation result is obtained by refining the pixelwise probability conditioned on the estimated superpixel labels with edge preserving filtering. Experimental results on Google Earth data of different coastal cities show the effectiveness of the proposed sea land segmentation method.

Fine-tuning of pretrained convolutional neural networks (CNNs) has been proven to be an effective strategy for remote sensing image scene classification, particularly when a limited number of labeled data sets are available for training purposes. However, such a fine-tuning process often needs that the input images are resized into a fixed size to generate input vectors of the size required by fully connected layers (FCLs) in the pretrained CNN model. Such a resizing process often discards key information in the scenes and thus deteriorates the classification performance. To address this issue, in this paper, we introduce a scale-free CNN (SF-CNN) for remote sensing scene classification. Specifically, the FCLs in the CNN model are first converted into convolutional layers, which not only allow the input images to be of arbitrary sizes but also retain the ability to extract discriminative features using a traditional sliding-window-based strategy. Then, a global average pooling (GAP) layer is added after the final convolutional layer so that input images of arbitrary size can be mapped to feature maps of uniform size. Finally, we utilize the resulting feature maps to create a new FCL that is fed to a softmax layer for final classification. Our experimental results conducted using several real data sets demonstrate the superiority of the proposed SF-CNN method over several well-known classification methods, including pretrained CNN-based ones.