ABSTRACT Weakly supervised semantic segmentation (WSSS) methods are extensively studied due to the availability of image‐level annotations. Relying on class activation maps (CAMs) derived from original classification networks often suffers from issues such as inaccurate object localization, incomplete object regions, and the inclusion of confusing background pixels. To address these issues, we propose a two‐stage method that enhances the foreground–background discriminative ability in a global context (FB‐DGC). Specifically, a cross‐domain feature calibration module (CFCM) is first proposed to calibrate foreground and background salient features using global spatial location information, thereby expanding foreground features while mitigating the impact of inaccurate localization in class activation regions. A class‐specific distance module (CSDM) is further adopted to facilitate the separation of foreground–background features, thereby enhancing the activation of target regions, which alleviates the over‐smoothing of features produced by the network and mitigates issues associated with confused features. In addition, an adaptive edge feature extraction (AEFE) strategy is proposed to identify target features in candidate boundary regions and capture missed features, compensating for drawbacks in recognising the co‐occurrence of multiple targets. The proposed method is extensively evaluated on the challenging PASCAL VOC 2012 and MS COCO 2014 datasets, demonstrating its feasibility and superiority.

Abstract Steel defect detection is important for industry production as it is tied to the product quality and production efficiency. However, previous steel defect detection methods based on deep convolutional neural networks heavily rely on large‐scale data for training and tend to have poor generalization ability for a novel defect category. In this paper, a novel few‐shot steel defect detection model based on multi‐scale semantic enhancement representation and mask category information mapping is introduced, where only a few annotated samples are acquired for the novel defect category. More concretely, three main components are built: an information‐guidance enhanced multi‐head detector is proposed to improve the representation of information in meta‐feature maps, a mask category representation module is designed to enhance the category feature representation of the mask region in the support set, and a novel multi‐scale category edge loss function is designed to assist the generation of category reweighting vector. Extensive experiments on the North‐east University few‐shot steel defect data set demonstrate that the proposed method significantly outperforms the state‐of‐the‐art methods and verify its effectiveness through ablation studies.

Facial expression recognition (FER) in the wild is challenging due to various unconstrained conditions, i.e., occlusions and head pose variations. Previous methods tend to improve the performance of facial expression recognition through resorting to holistic methods or coarse local-based methods, while ignoring the local fine-grained feature structure knowledge and the correlation between features. In this paper, we propose a Fine-Grained Association Graph Representation (FG-AGR) framework which can capture the local fine-grained facial expression representation. Firstly, an Adaptive Salient Region Induction (ASRI) is designed for adaptively highlighting the local saliency regions of facial expressions combined with spatial location information. Based on this, a Local Fine-grained Feature Extraction (LFFE) based on Visual Transformers is introduced to further extract fine but discriminative fine-grained features of saliency regions. Thirdly, an Adaptive Graph Association Reasoning (AGAR) based on Graph Convolutional Network is constructed to learn associated fine-grained feature combinations. Extensive experiments demonstrate that our FG-AGR achieves superior performance compared to the state-of-the-art methods with 90.81% on RAF-DB, 64.91% on AffectNet-7, 60.69% on AffectNet-8 and 91.09% on FERPlus.

Currently, Remote sensing object detection methods based on deep learning have been widely used in military investigation, ocean monitoring, urban planning and post-disaster relief. However, many difficulties, such as complex backgrounds, indistinguishable object appearances, large-scale variations, and non-uniform distributions, limit the accuracy of detectors. To deal with these problems, a key feature-enhanced network (KFENet) is proposed to improve the detection accuracy. Specifically, a multi-scale feature fusion (MSFF) module is given to obtain richer feature expression, which can explore inherent structural information of images at different scales. Next, an effective activation detection head (EAD-Head) is designed to adaptively capture key features required for classification and regression tasks, respectively. Finally, we implement our strategy within the framework of YOLOX. Experiments on DIOR and RSOD datasets show that the proposed algorithm outperforms state of the art methods.

Abstract Weakly supervised object detection (WSOD) is becoming increasingly important for computer vision tasks, as it alleviates the burden of manual annotation. Most WSOD techniques rely on multiple instance learning (MIL), which tends to localise the discriminative parts of salient objects instead of the whole object. In addition, network training is often supervised using simple image‐level annotations, without including object quantities or location information. However, this can lead to ambiguous differentiation of object instances, both in terms of location and semantics. To address these issues, propose an end‐to‐end triple critical feature capture network (TCFCNet) for WSOD is proposed. Specifically, a multi‐task branch, which can perform fully supervised classification and regression task, was integrated with a PCL in an end‐to‐end network for refining object locations in an online method. A cyclic parametric dropblock module (CPDM) was then designed to help the detector focus on the contextual information by using cyclic masking techniques to maximise the removal of the discriminative components of an object instance to alleviate the part domination problem. Finally, a feature decoupling module (FDM) is proposed to further reduce the ambiguous distinction of object instances by adaptively constructing robust critical features that adapt to multi‐task branch for classification and regression tasks, which contains a feature enhancement module and task‐specific polarisation functions. Comprehensive experiments are carried out on the challenging Pascal VOC 2007 and VOC 2012 datasets. The proposed method achieves a 54.6% mAP and a 44.3% mAP on the Pascal VOC 2007 and VOC 2012 datasets respectively, showed that our method outperformed existing mainstream techniques by a considerable margin.

Abstract Seed sorting based on deep neural networks is one of the important applications of seed variety identification and quality purification. However, DNNs is difficult to deploy on embedded devices since the consumption of computational and storage resource. To address these problems, this paper proposes a pipeline‐style neural network framework for real‐time seed sorting. First, we propose a novel algorithm, 2D information entropy, pruning redundant filters to realize structured pruning. Then, the pruning rate of each convolution layer is determined by visualizing the results of 2D entropy. Meanwhile, the pruned network is fine‐tuned to recover the performance. Finally, TensorRT is utilized to optimize and accelerate the pruned model for deployment in Jeston Nano. Experiments on two large‐scale seed‐sorting datasets demonstrate the significant improvement of the proposed method over existing model compression methods. Experimental results on Jeston Nano show that the pruned model 2EFP‐E achieves a single image inference speed of 107 FPS, with the best accuracy of 95.94% on the red kidney bean dataset.

Traditional fusion approaches and most deep learning-based methods usually generate the intermediate decision map, resulting in detail loss of source images or fusion results. In this work, to enhance the detailed features and structured information from source images, we propose a dual cascade attention network (DCAN) to obtain a more informative fusion image for PET and MRI images. In our approach, channel attention is employed to improve the ability of features representation and spatial attention can highlight informative regions in the proposed fusion network. Additionally, channel and spatial attention are sequential arrangement in channel-first. Moreover, to achieve good performance in the procedure of feature extraction and image reconstruction, two-stage training strategy is adopted to train our fusion model. Experimental results demonstrate that the proposed approach achieves remarkable performance for PET and MRI images fusion.

Facial Expression Recognition (FER) has achieved significant performance in recent years. However, learning discriminative expression features is still a pending issue, due to disturbances in the wild and high similarities across different expressions. To remedy this issue, we present an innovative region attention and refinement network (RARN) capable of learning discriminative and robust features for accurate FER in the wild. Specifically, RARN mainly learn discriminative features from two different aspects: Multi-head Local Attention Network (MLAN) and a Latent Feature Mining Network (LFMN). MLAN can accurately locate the regions of interest and extract robust features in the wild by employing two modules, in which the region generation module divides the midlevel feature maps into several subregions and local attention module highlights the local discriminative features of regions of interest through attention block. LFMN combines a series of latent expression features associated with their corresponding importance weights to further obtain the final discriminative expression features. Experimental results on three widely-used in-the-wild FER datasets demonstrate proposed algorithm achieves favorable performance against state-of-the-art methods. The source code will be found at https://github.com/lizoe/RARN.

Abstract With the performance increase of convolutional neural network (CNN), the disadvantages of CNN's high storage and high power consumption are followed. Among the methods mentioned in various literature, filter pruning is a crucial method for constructing lightweight networks. However, the current filter pruning method is still challenged by complicated processes and training inefficiency. This paper proposes an effective filter pruning method, which uses the saliency of the feature map (SFM), i.e. information entropy, as a theoretical guide for whether the filter is essential. The pruning principle use here is that the filter with a weak saliency feature map in the early stage will not significantly improve the final accuracy. Thus, one can efficiently prune the non‐salient feature map with a smaller information entropy and the corresponding filter. Besides, an over‐parameterized convolution method is employed to improve the pruned model's accuracy without increasing parameter at inference time. Experimental results show that without introducing any additional constraints, the effectiveness of this method in FLOPs and parameters reduction with similar accuracy has advanced the state‐of‐the‐art. For example, on CIFAR‐10, the pruned VGG‐16 achieves only a small loss of 0.39% in Top‐1 accuracy with a factor of 83.3% parameters, and 66.7% FLOPs reductions. On ImageNet‐100, the pruned ResNet‐50 achieves only a small accuracy degradation of 0.76% in Top‐1 accuracy with a factor of 61.19% parameters, and 62.98% FLOPs reductions.

In this paper, we propose a novel fusion framework based on a dense network with channel and spatial attention (CSpA-DN) for PET and MR images. In our approach, an encoder composed of the densely connected neural network is constructed to extract features from source images, and a decoder network is leveraged to yield the fused image from these features. Simultaneously, a self-attention mechanism is introduced in the encoder and decoder to further integrate local features along with their global dependencies adaptively. The extracted feature of each spatial position is synthesized by a weighted summation of those features at the same row and column with this position via a spatial attention module. Meanwhile, the interdependent relationship of all feature maps is integrated by a channel attention module. The summation of the outputs of these two attention modules is fed into the decoder and the fused image is generated. Experimental results illustrate the superiorities of our proposed CSpA-DN model compared with state-of-the-art methods in PET and MR images fusion according to both visual perception and objective assessment.

Conventional inverse synthetic aperture radar image is a 2-D range-Doppler projection of a target and does not provide 3-D information. Its formation also requires complex motion compensation when the target is uncooperative and maneuvering. On the other hand, multiple-input and multiple-output (MIMO) radar, in addition to having a wide virtual aperture and high cross-range resolution, could also obtain a target's 3-D image in one snapshot and thus have avoided the complex motion compensation needed. In this paper, we propose a 3-D imaging algorithm using three MIMO configurations. The signal model is derived based on a modified zero correlation zone code. A strong scatterer selection criterion is also proposed for the construction of the target profile.

Conventional envelope alignment method of ISAR imaging assumes that the target is a rigid body, and therefore, the correlation between one-dimensional range profiles at neighboring time slots gives a larger value compared to correlation between time separated range profiles. But this is not the case for targets such as helicopter which have rotating body parts. In this paper, an envelope alignment algorithm for helicopter target ISAR imaging is proposed. For every range profile, correlation with other range profiles are done, and the range profiles with larger correlation value are used to provide an initial estimate of the target motion parameters by polynomial fitting. Then the estimated parameters are averaged to get a better estimation. The position of the rotor signals are obtained by taking the difference of the range profiles in the slow time domain. The rotor signals are then suppressed by zero-force windowing technique. Simulation results have shown the effectiveness of this algorithm.