One of the primary challenges in Synthetic Aperture Radar (SAR) object detection lies in the pervasive influence of coherent noise. As a common practice, most existing methods, whether handcrafted approaches or deep learning-based methods, employ the analysis or enhancement of object spatial-domain characteristics to achieve implicit denoising. In this paper, we propose DenoDet V2, which explores a completely novel and different perspective to deconstruct and modulate the features in the transform domain via a carefully designed attention architecture. Compared to DenoDet V1, DenoDet V2 is a major advancement that exploits the complementary nature of amplitude and phase information through a band-wise mutual modulation mechanism, which enables a reciprocal enhancement between phase and amplitude spectra. Extensive experiments on various SAR datasets demonstrate the state-of-the-art performance of DenoDet V2. Notably, DenoDet V2 achieves a significant 0.8% improvement on SARDet-100K dataset compared to DenoDet V1, while reducing the model complexity by half.

Weakly supervised semantic segmentation (WSSS) aims to provide accurate pixel-level annotation based on only weak guidance, primarily derived from image-level labels. Recent WSSS methods exploit pseudo-labels generated from improved class activation maps (CAMs) to train a fine-grained classification model for semantic segmentation. However, these pseudo-labels are unreliable because they tend to either miss parts of the objects or include irrelevant regions due to weak guidance from individual images. In this paper, we propose a simple yet effective iterative mix-up strategy, Pseudo-Label-based Mix (PL-Mix), that refines pseudo-labels iteratively, thereby further enhancing WSSS performance. During each iteration, we migrate object regions from pseudo-labels produced in previous steps and render them with new contexts in a mix-up fashion. Due to model consistency enforcement across varied backgrounds and new combinations of multiple objects from enriched image samples, these pseudo-labels progressively become more accurate and reliable. Further enhanced by a masking strategy and a CAM-based earth mover’s distance loss, we achieve state-of-the-art performance on the PASCAL VOC2012 and MS COCO2014 benchmark datasets.

A community-imbalanced graph refers to a graph containing multiple communities with large differences in node and edge scales. Graph sampling is a widely used graph reduction technique to accelerate graph computations and simplify graph visualizations. However, existing graph sampling algorithms may encounter several problems, including the loss of small communities, disconnections between communities, and distortions of community scale distribution, on maintaining the community structures in a community-imbalanced graph. In this work, a new quality indicator is proposed to determine if a graph can be regarded as a community-imbalanced graph. A community-imbalanced graph sampling (CIGS) algorithm is proposed to address the community-imbalanced graph sampling problems. Three new evaluation metrics are proposed to assess the performance of community structure maintenance of graph sampling. An algorithm performance experiment and a user study are conducted to evaluate the effectiveness of the proposed CIGS.

Image Steganography without Embedding (SWE) is an emerging data hiding paradigm. Instead of embedding a secret message into a container image, SWE synthesises a novel image by using the secret message as a latent code. Current SWE methods have achieved high synthesis quality and strong resistance to steganalysis tools. However, it remains challenging to apply the SWE due to two reasons: (i) lack of synthesis diversity and (ii) recovery of secret messages under malicious image attacks. In this paper, we present a novel SWE framework with a disentanglement autoencoder to tackle the above challenges. Specifically, the autoencoder disentangles an image into a structure and texture representation. Then, we exploit the stability of the structure representation to improve secret message recovery reliability, while increasing synthesis diversity by randomising texture representations and employing a chaotic system for structure randomisation to enhance its security. To further achieve a robust message recovery under malicious attacks, an adversarial learning strategy is introduced into our framework, which guarantees high recovery accuracy. Our method outperforms other state-of-the-art SWE methods in terms of synthesis quality, synthesis diversity and secret message recovery accuracy under various image attacks. The source code is publicly available athttps://github.com/Lemok00/RDI-SWE.

Manual annotation for crowd counting remains labor-intensive and costly. Although existing semi-supervised methods partially alleviate this burden, they still face significant challenges regarding the quality of generated pseudo-labels and the utilization of unlabeled data. To address these issues, we propose a novel semi-supervised crowd counting framework, called Point-Adaptive Teacher (PAT). This framework integrates Adaptive Soft Threshold (AST) and contrastive learning to enhance pseudo-label quality and effectively leverage unlabeled data. Specifically, we employ the Swin Transformer as the backbone and develop Swin-P2PNet, which captures global contextual information through hierarchical window attention, improving the accuracy of pseudo-labels. Additionally, we design the AST that dynamically adjusts the sample loss weight by combining confidence and uncertainty predictions, thereby alleviating the effect of noise in pseudo-labels. Finally, we introduce a contrastive learning strategy requiring no extra parameters. This strategy enhances the model’s ability to learn latent representations from unlabeled data. Extensive experiments have been conducted on three public datasets, namely ShanghaiTech, JHU-Crowd++, and UCF-QNRF. The results demonstrate that our method achieves performance comparable to state-of-the-art methods.

In recent years, novel view synthesis from a monocular image has become a research hot-spot that attracts significant attention. Some recent work identifies latent vectors for high-quality view generation via iterative optimisation, which is a time-consuming process. In contrast, some others utilise an encoder learning a mapping function to approximately estimate optimal latent codes, which significantly reduces its processing time but sacrifices reconstruction quality. Consequently, how to balance synthesis quality and its generation efficiency still remains challenging. In this paper, we propose a residual-based encoder to incorporate with a 3D Generative Adversarial Networks (GAN), named ReE3D, for novel view synthesis. It applies an iterative prediction of latent codes to ensure much higher quality of novel view synthesis with an insignificant increase of processing time when compared to existing encoder-based 3D GAN inversion methods. Additionally, we enforce a novel geometric loss constraint on the encoder to predict view-invariant latent codes, thus effectively mitigating the trade-off between geometric and texture quality in 3D GAN inversion. Extensive experimental results demonstrate that our extended encoder-based method has achieved best trade-off performance in terms of novel view synthesis quality and its execution time. Our method has gained comparable synthesis quality with exponentially decreased processing time when compared to iterative optimisation methods, while improved synthesis performance of encoder-based methods significantly.

Adversarial training (AT) has shown impressive advantages in maintaining accuracy and enhancing robustness against adversarial examples. However, most existing AT techniques jointly optimize clean-example accuracy and adversarial-example robustness as dual objectives. This setting introduces an often overlooked issue; when optimizing hard samples near the decision boundary, the model may bolster robustness at the expense of accuracy or preserve accuracy to the detriment of robustness. To alleviate the accuracy-robustness sacrifices induced by hard samples, we propose mitigating hard sample interference (MHSI) from a sample-intervention perspective. MHSI aims to reduce the instability caused by hard samples during AT. Specifically, we introduce a weighted adaptive (WA) mechanism that strengthens the model's learning of clean samples, thereby reducing the negative impact of hard samples on accuracy. In addition, guided by an analysis of the gradient norm and the Hessian matrix, we design a dynamic calibration (DC) strategy that dynamically calibrates the probability outputs of hard samples to mitigate their damage to robustness. With these two modules, our approach significantly improves robustness without sacrificing accuracy. Extensive experiments on CIFAR-10, CIFAR-100, Tiny ImageNet, and SVHN demonstrate that MHSI effectively improves both accuracy and robustness and outperforms state-of-the-art methods under glass-box attacks. Notably, under an $l_{\infty }$ attack, MHSI yields up to a 6.22% robustness gain over the AT baseline. Our code is available at https://github.com/hubin111/MHSI.

Recent advances in AI-computing networks (ACN) provide a timely backdrop for assessing deep-learning (DL) research in healthcare. This systematic review synthesizes DL applications in multiple sclerosis (MS) and evaluates their readiness for ACN-enabled deployment. A Web of Science Core Collection search (2014-2024) retrieved 438 records; 264 met stringent inclusion criteria. Bibliometric and knowledge-mapping analyses reveal steady growth in publications and citations, with MRIbased UNet variants dominating lesion-segmentation and diseaseclassification tasks. Emerging themes include gait-sensor analytics, longitudinal progression modelling, quantitative susceptibility mapping, and the growing use of transfer and federated learning to overcome data scarcity and privacy barriers. These resourceaware strategies signal a shift toward distributed training and inference paradigms that align naturally with ACN architectures. Nevertheless, few studies report multi-institutional experiments or network-level performance metrics, underscoring the need for tighter integration between DL methods and ACN infrastructure. We highlight research gaps-particularly in cross-site model orchestration and low-latency, on-device inference-that AIcomputing networks are well positioned to address, enabling scalable and interoperable DL services for MS diagnosis and prognosis.

Incomplete multi-view deep clustering is an emerging research hot-pot to incorporate data information of multiple sources or modalities when parts of them are missing. Most of existing approaches encode the available data observations into multiple view-specific latent representations and subsequently integrate them for the next clustering task. However, they ignore that the latent representations are unique to a fixed set of data samples in all views. Meanwhile, the pair-wise similarities of missing data observations are also failed to utilize in latent representation learning sufficiently, leading to unsatisfactory clustering performance. To address these issues, we propose an incomplete multi-view deep clustering method with data imputation and alignment. Assuming that each data sample corresponds to a same latent representation among all views, it projects the latent representations into feature spaces with neural networks. As a result, not only the available data observations are reconstructed, but also the missing ones can be imputed accordingly. Moreover, a linear alignment measurement of linear complexity is defined to compute the pair-wise similarities of all data observations, especially including those of the missing. By executing the above two procedures iteratively, the discriminative latent representations can be learned and used to group the data into categories with off-the-shelf clustering algorithms. In experiment, the proposed method is validated on a set of benchmark datasets and achieves state-of-the-art performances.

Due to the limitation of the optical lens focal length and the resolution of the infrared detector, distant Closely-Spaced Infrared Small Target (CSIST) groups typically appear as mixing spots in the infrared image. In this paper, we propose a novel task, Sequential CSIST Unmixing, namely detecting all targets in the form of sub-pixel localization from a highly dense CSIST group. However, achieving such precise detection is an extremely difficult challenge. In addition, the lack of high-quality public datasets has also restricted the research progress. To this end, firstly, we contribute an open-source ecosystem, including SeqCSIST, a sequential benchmark dataset, and a toolkit that provides objective evaluation metrics for this special task, along with the implementation of 23 relevant methods. Furthermore, we propose the Deformable Refinement Network (DeRefNet), a model-driven deep learning framework that introduces a Temporal Deformable Feature Alignment (TDFA) module enabling adaptive inter-frame information aggregation. To the best of our knowledge, this work is the first endeavor to address the CSIST Unmixing task within a multi-frame paradigm. Experiments on the SeqCSIST dataset demonstrate that our method outperforms the state-of-the-art approaches with mean Average Precision (mAP) metric improved by 5.3%. Our dataset and toolkit are available from https://github.com/GrokCV/SeqCSIST.

In the context of transportation cyber-physical systems (T-CPS), backdoor attacks leveraging traffic images have emerged as a significant security threat. As T-CPS increasingly relies on visual information, such as real-time images captured by traffic cameras, for tasks like traffic sign recognition and autonomous driving, the risk of image-based backdoor attacks has grown substantially. Although various detection-based defense techniques have shown some success in identifying backdoored models, they often fail to fully eliminate backdoor effects, leaving residual security risks. To address this challenge, we propose a Frequency-Domain Hybrid Distillation (FDHD) method for backdoor defense, which effectively weakens the association between backdoor triggers and target labels by combining distillation mechanisms in both the frequency and pixel domains. Furthermore, we design a loss function that integrates feature reconstruction with adaptive alignment, enhancing the student network’s ability to mimic the teacher network and thereby bolstering the backdoor defense capability. Extensive experiments conducted by FDHD on multiple benchmark datasets against the five latest attacks demonstrate that our proposed defense method effectively reduces backdoor threats while maintaining high accuracy in predicting clean samples. This approach will protect against image-based backdoor attacks in T-CPS and lay the foundation for enhancing future traffic safety.

With the escalating demand for three-dimensional visual applications such as gaming, virtual reality, and autonomous driving, novel view synthesis has become a critical area of research. Current methods mainly depend on multiple views of the same subject to achieve satisfactory results, but there is often a significant lack of available data. Typically, only a single degraded image is available for reconstruction, which may be affected by occlusion, low resolution, or absence of color information. To overcome this limitation, we propose a two-stage feature matching approach designed specifically for single degraded images, leading to the synthesis of high-quality novel perspective images. This method involves the sequential use of an encoder for feature extraction followed by the fine-tuning of a generator for feature matching. Additionally, the integration of an information filtering module proposed by us during the GAN inversion process helps eliminate misleading information present in degraded images, thereby correcting the inversion direction. Extensive experimental results show that our method outperforms existing state-of-the-art single-view novel view synthesis techniques in handling challenges like occluded, grayscale, and low-resolution images. Moreover, the efficacy of our method remains unparalleled even when aforementioned method integrated with image restoration algorithms.

Guided depth super-resolution (GDSR) aims to enhance the level of detail in low-resolution depth images by utilizing the information present in the corresponding high-resolution RGB images. While existing methods utilize different approaches to guide the RGB image to the source image, they often ignore the texture similarity between these two images and usually suffer from unsatisfactory outline reconstruction of the depth map. In this article, we introduce an adaptive texture migration network (ATMNet) designed to mine rich feature information from RGB images and migrate them to the depth image. Specifically, we propose a multi-modal feature extractor (MMFE) to extract private and shared features between the depth map and RGB image. In addition, we present a texture migration module (TMM) to remap and fuse the features extracted from the raw image pairs. Last but not least, we develop a weighted adaptive loss to enhance the reconstruction of the edge areas in the depth map. Extensive experiments on public datasets such as Middlebury, NYUv2, and DIML demonstrate that our method outperforms the existing state-of-the-art GDSR methods and strikes a remarkable balance between performance and efficiency. The source code is available at https://github.com/MuggleTan/ATMNet .

Blind super-resolution (BSR) is entering a new era focused on diverse and complex applications, where the tradeoff between generalization and performance prevents models from performing as they should. Model performance decreases when trained on multiple degraded images due to the inter-class and intra-class imbalances in degradation prediction, which consists of degradation sampling and estimation. The inter-class imbalance in degradation estimation causes inaccurate estimates, leading to severe artifacts in images. The intra-class imbalance in degradation sampling causes a long-tail problem, leading to model collapse and satisfactory results only in specific applications. To tackle these challenges, we propose adaptive alignment contrastive learning (AACL), which includes adaptive degradation sampling (ADS) and \(\sigma\) -alignment. ADS utilizes non-linear sampling by weighting the parameters of the degradation process for training uniformly degraded images, avoiding the long-tail problem. \(\sigma\) -alignment controls the SD among positive samples; we identify a subset with small degraded distance, which aids contrastive learning in extracting representations more effectively. We extend AACL to several CNN-based and Transformer-based methods by coming up with a 6 \(\times\) 6 fair architecture with degradation representation fusion block (DRFB) and degradation representation fusion group (DRFG). DRFB and DRFG are designed for degradation representation fusion and image reconstruction, respectively. We evaluate on six types of degradation, and the improvement experiments on synthesized images show that our method balances performance and generalization and is applicable to networks with different architectures. The comparison experiments show that our improved methods achieve promising results compared to SOTA methods. Code is available at: https://github.com/para999/AACL .

Traffic sign recognition is a crucial aspect of autonomous vehicle research, and deep learning techniques have significantly contributed to its progress. Nevertheless, the distribution of traffic sign information in natural complex road conditions is long-tailed, and traffic sign identification in complex road conditions has become a significant barrier to autonomous vehicle applications. The imbalanced distribution of information on the dataset migrates to the feature space during training, resulting in imbalanced classifier prediction. In this paper, we propose the gradient rebalanced traffic sign recognition (GRTR) method to address this problem for the first time. GRTR first evaluates the prediction and classification bias of the classifier using the fitted deviation between the model’s output probability and the ground-truth distributions. Then, GRTR dynamically adjusts the correction and compensation factors following the classifier’s prediction and classification biases. GRTR rebalances the positive and negative sample gradients for each category based on the synergistic effect of the correction and compensation factors to prevent the transfer of distribution imbalance and to significantly enhance the performance of the traffic sign classifier under difficult road conditions. Experimental results demonstrate that our GRTR achieves state-of-the-art performance on long-tailed traffic sign and multilabel datasets.Note to Practitioners—Most traffic sign recognition algorithms are still designed based on the assumption of a balanced distribution of traffic signs in the dataset. Real-world autonomous vehicles require traffic sign recognition on datasets with severely imbalanced distributions. This paper proposes a general approach to solving the long-tailed traffic sign recognition problem.

Computer-aided diagnosis (CAD) has always been an important research topic for applying artificial intelligence in smart healthcare. Sufficient medical data are one of the most critical factors in CAD research. However, medical data are usually obtained in chronological order and cannot be collected all at once, which poses difficulties for the application of deep learning technology in the medical field. The traditional batch learning method consumes considerable time and space resources for real-time medical data, and the incremental learning method often leads to catastrophic forgetting. To solve these problems, we propose a real-time medical data processing method based on federated learning. We divide the process into the model stage and the exemplar stage. In the model stage, we use the federated learning method to fuse the old and new models to mitigate the catastrophic forgetting problem of the new model. In the exemplar stage, we use the most representative exemplars selected from the old data to help the new model review the old knowledge, which further mitigates the catastrophic forgetting problem of the new model. We use this method to conduct experiments on a simulated medical real-time data stream. The experimental results show that our method can learn a disease diagnosis model from a continuous medical real-time data stream. As the amount of data increases, the performance of the disease diagnosis model continues to improve, and the catastrophic forgetting problem has been effectively mitigated. Compared with the traditional batch learning method, our method can significantly save time and space resources.

Regularization is a crucial technology to improve the generalization of deep neural networks. However, traditional regularization method approaches are all scenario-specific, because they are generally with ingeniously designed feature representations from input layer, hidden layer, and output layer, which increase the difficulty of model development and interpretation. To this end, a novel practical and flexible regularization method is presented to obtain higher generalization and interpretability. Specifically, the feature maps are decoupled by global suppression and partial suppression from various scales and locate the salient feature with strong low-resolution semantic information. Moreover, the guided discarding specification for feature decoupling by measuring the feature contributions to network decisions, leads to the logics with better interpretability. Subsequently, the max values of the feature map are suppressed by discarding the corresponding salient features. Comprehensive experiments demonstrate that the proposed adaptive regularization outperforms the state-of-the-art performance in image classification accuracy, generalization, and interpretability on several widely used datasets. And adaptive regularization helps the network to mine the connection between salient features, nonsalient features, and ground truth, encouraging the network to construct multiple layers of feature associations.

Large-scale infectious diseases pose a tremendous risk to humans, with global outbreaks of COVID-19 causing millions of deaths and trillions of dollars in economic losses. To minimize the damage caused by large-scale infectious diseases, it is necessary to develop infectious disease prediction models to provide assistance for prevention. In this paper, we propose an XGBoost-LSTM mixed framework that predicts the spread of infectious diseases in multiple cities and regions. According to big traffic data, it was found that population flow is closely related to the spread of infectious diseases. Clustering and dividing cities according to population flow can significantly improve prediction accuracy. Meanwhile, an XGBoost is used to predict the transmission trend based on the key features of infection. An LSTM is used to predict the transmission fluctuation based on infection-related multiple time series features. The mixed model combines transmission trends and fluctuations to predict infections accurately. The proposed method is evaluated on a dataset of highly pathogenic infectious disease transmission published by Baidu and compared with other advanced methods. The results show that the model has an excellent predictive effect and practical value for large-scale infectious disease prediction.

Single domain generalization (single-DG) is a realistic yet challenging domain generalization scenario where a model trained on a single domain generalization scenario where a model trained on a single domain generalizes well to multiple unseen domains. Unlike typical single-DG methods that are essentially supervised data augmentation and focus mainly on the novelty of images, we propose a simple adversarial augmentation method, termed Progressive Diversity Generation (PDG), to synthesize novel and diverse images in a fully unsupervised manner. Specifically, PDG minimizes the uncertainty coefficient to ensure that synthesized images are novel. By modeling conditional probabilities with an auxiliary network, we transfer the adversarial process from semantics to images, thus eliminating dependency on labels. To enhance diversity, we propose the$f$-diversity, a collection of correlation or similarity measures, to allow our model to generate potential images from diverse perspectives. The proposed architecture combines a multi-attribute generator with a progressive generation framework to improve model performance. PDG is the unsupervised and easy-to-implement method that solves single-DG with only synthesized (source) images. Extensive experiments on multiple single-DG benchmarks show that PDG achieves remarkable results and outperforms existing supervised and unsupervised methods by a large margin in single domain generalization. Source code and data are available:https://github.com/Ruiding1/PDG.

Reference-based image super-resolution (RefSR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) images by introducing HR reference images. The key step of RefSR is to transfer reference features to LR features. However, existing methods still lack an efficient transfer mechanism, resulting in blurry details in the generated image. In this article, we propose a double-layer search module and an adaptive pooling fusion module group for reference-based image super-resolution, called DLASR. Based on the re-search strategy, the double-layer search module can produce an accurate index map and score map. These two maps are used to filter out accurate reference features, which greatly increases the efficiency of feature transfer in the later stage. Through two continuous feature-enhancement steps, the adaptive pooling fusion module group can transfer more valuable reference features to the corresponding LR features. In addition, a structure reconstruction module is proposed to recover the geometric information of the images, which further improves the visual quality of the generated image. We conduct comparative experiments on a variety of datasets, and the results prove that DLASR achieves significant improvements over other state-of-the-art methods, in terms of quantitative accuracy and qualitative visual effect. The code is available at https://github.com/clttyou/DLASR.

Scene text image super-resolution (STISR) in the wild has been shown to be beneficial to support improved vision-based text recognition from low-resolution imagery. An intuitive way to enhance STISR performance is to explore the well-structured and repetitive layout characteristics of text and exploit these as prior knowledge to guide model convergence. In this paper, we propose a novel gradient-based graph attention method to embed patch-wise text layout contexts into image feature representations for high-resolution text image reconstruction in an implicit and elegant manner. We introduce a non-local group-wise attention module to extract text features which are then enhanced by a cascaded channel attention module and a novel gradient-based graph attention module in order to obtain more effective representations by exploring correlations of regional and local patch-wise text layout properties. Extensive experiments on the benchmark TextZoom dataset convincingly demonstrate that our method supports excellent text recognition and outperforms the current state-of-the-art in STISR. The source code is available at https://github.com/xyzhu1/TSAN.

Micro-expressions (MEs) which are types of spontaneous facial movements, are difficult to recognize due to their short duration and low intensity. Recent ME recognition methods typically depend on spatio-temporal features around the eyebrow and mouth regions where MEs occur. And these features are often extracted from the onset and apex frames in which the intensities of MEs are considered to be zero and high respectively. In this paper, we improve the effectiveness of the spatio-temporal features by proposing a two-layer encoder of Transformer to model the features’ relations. In addition, the novel recognition scheme captures the more detailed motion dynamics of MEs by employing more frames rather than the onset and apex frames. The recognition scheme is further refined by developing a graph convolution network with a trainable adjacency matrix for Action Units (AUs). Extensive experiments on multiple public datasets demonstrate that our method has better or comparable performance to SOTA methods on multiple evaluation metrics.

Single domain generalization (SDG) is a realistic yet challenging domain generalization scenario that aims to generalize a model trained on a single domain to multiple unseen domains. Typical SDG methods are essentially supervised data augmentation strategies, which tend to enhance the novelty rather than the diversity of augmented samples. Insufficient diversity may jeopardize the model generalization ability. In this paper, we propose a novel adversarial method, termed Unsupervised Diversity Probe (UDP), to synthesize novel and diverse samples in fully unsupervised settings. More specifically, to ensure that samples are novel, we study SDG from an information-theoretic perspective that minimizes the uncertainty coefficients between synthesized and source samples. Considering that the variation in a single source domain is limited, we introduce a regularization imposed on the auxiliary module that synthesizes variable samples, incorporated with uncertainty coefficients in an adversarial manner to complement the diversity. Subsequently, an available region is utilized to guarantee the samples' safety. For the network architecture, we design a simple probe module that can synthesize samples in several different aspects. UDP is an unsupervised and easy-to-implement method that solves SDG using only synthetic (source) samples, thus reducing the dependence on task models. Extensive experiments on three benchmark datasets show that UDP achieves remarkable results and outperforms existing supervised and unsupervised methods by a large margin in single domain generalization.

Medical images are an important basis for doctors to diagnose diseases, but some medical images have low resolution due to hardware technology and cost constraints. Super-resolution technology can reconstruct low-resolution medical images into high-resolution images and enhance the quality of low-resolution images, thus assisting doctors in diagnosing diseases. However, traditional super-resolution methods mainly learn the mapping relationships among modal pixels from low resolution to high resolution, lacking the learning of high-level semantic features, resulting in a lack of understanding and utilization of semantic information, such as reconstructed objects, object attributes, and spatial relationships between two objects. In this paper, we propose a medical image super-resolution method based on semantic perception transfer learning. First, we propose a novel semantic perception super-resolution method that empowers super-resolution models to perceive high-level semantics by transferring features of the image description generation network in natural language processing. Second, we construct a semantic feature extraction network and an image description generation network and comprehensively utilized image and text modal data to learn transferable, high-level semantic features. Third, we train an end-to-end, semantic perception super-resolution model by fusing dynamic perceptual convolution, a semantic extraction network, and distillation polarization self-attention. Experiments show that semantic perception transfer learning can effectively improve the quality of super-resolution reconstruction.

Intelligent transportation infrastructure is essential to intelligent transportation system (ITS). With the continuous development of Internet of Things (IoT) technology, remote monitoring has become a critical part of ITS. However, due to the limitations of network transmission, production cost, and other factors, some video monitoring can obtain only low-resolution (LR) video. LR video features are seriously lost, thus affecting the performance of ITS. In this paper, based on the research of existing super-resolution algorithms, we focus on improving the reconstruction quality of video frame sequences by aiming at the insufficient utilization of inter-frame information and low reconstruction quality of existing video super-resolution algorithms. This paper proposes a video super-resolution algorithm based on inter-frame information utilization, which can effectively improve the performance of ITS. First, a novel U-shaped feature extractor is designed to fully extract the feature expression of video frame sequences. Second, a deformable inter-frame alignment module based on residual learning is constructed to make the inter-frame alignment more accurate and thus promote the mutual utilization of inter-frame information. Finally, an up and down sampling residual block is proposed to extract features that better match the upsampling reconstruction requirements. The experimental results show that the method has better reconstruction quality for monitoring video and is advanced and applicable compared to mainstream video oversampling methods.

Although deep learning methods have drastically improved the performance on visual recognition tasks in which large inter-class variances exist, similar-class recognition continues to pose significant challenges, mainly due to the close resemblance between similar classes. The challenge is further compounded in the case of few-shot learning because only a very small amount of training data is available; accordingly, a certain performance degradation has been observed when some few-shot methods are applied for classification tasks. To address the aforementioned issue, we propose a novel Relation Separation Network (RSNet) in this paper, aiming to boost few-shot learning by improving similar-class recognition performance. We assume that image features consist of common and private features, where the common features capture the basic attributes shared among similar classes and their private counterparts capture the unique attributes of each class. Our RSNet learns to decouple the common and private features of an image. As a result, the feature representation of an image is composed of two weakly associated but easily aligned components, and better classification performance is achieved by giving more attention to subtle features. Experimental results on the publicly available datasets miniImageNet, CUB, and CIFAR-FS show that the proposed model outperforms existing state-of-the-art methods. Specifically, compared to PT+MAP, RSNet improves the accuracy of classification on the CUB dataset by approximately 5% and that of similar-class classification by more than 10%.

With the development of intelligent medicine, lesion detection supported by Internet of Things (IoT), big data, and deep learning has become a hotspot. However, lesion detection technology based on deep learning requires huge amounts of high-quality medical image data, and the data from social IoT has the problems of uneven quality and lack of lesion labeling. Current studies usually ignore the unstable quality of IoT data and the interpretability of diagnostic results, resulting in deeper model layers, larger models, poor real-time performance, and lack of persuasion. To address the problems, this article first proposes a core data extraction method for multi-class lesion detection based on unlabeled medical image from social IoT. Then, we propose an ensemble algorithm based on lightweight models to improve the detection accuracy. Finally, we visualize pathological features to enhance the interpretability of core data and detection results. The experimental results show that our method can effectively extract the core data of multiple lesions from low-quality medical images and improve the accuracy of the lightweight lesion detection model as well as the interpretation of detection results.

We propose ComGAN, a simple unsupervised generative model, which simultaneously generates realistic images and high semantic masks under an adversarial loss and a binary regularization. In this paper, we first investigate two kinds of trivial solutions in the compositional generation process, and demonstrate their source is vanishing gradients on the mask. Then, we solve trivial solutions from the perspective of architecture. Furthermore, we redesign two fully unsupervised modules based on ComGAN (DS-ComGAN), where the disentanglement module associates the foreground, background and mask with three independent variables, and the segmentation module learns object segmentation. Experimental results show that (i) ComGAN's network architecture effectively avoids trivial solutions without any supervised information and regularization; (ii) DS-ComGAN achieves remarkable results and outperforms existing semi-supervised and weakly supervised methods by a large margin in both the image disentanglement and unsupervised segmentation tasks. It implies that the redesign of ComGAN is a possible direction for future unsupervised work.

Exploring the characterization laws of image data and improving the efficiency of image data characterization knowledge is essential to promote the development of the Internet of Things technology. Considering that images in the real world usually contain multiple objects, and the objects are closely dependent. For these reasons, it brings great challenges to the robust representation learning of multilabel images. In general, researchers model the relationship between objects based on a class activation map and use graph convolution to mine the dependencies between objects. However, graph structure data often contain noise, which means that the edges between nodes are sometimes not so reliable, and the relative importance of neighbors is also different. Based on this, our goal is to reduce noisy connections and false connections between objects, eliminate multilabel image representation bias, and learn robust representations. Therefore, we propose a robust representation learning method for multilabel images driven by graph attention network (RRL-GAT). Specifically, to reduce the accidental false connection of objects in the image, we propose the class attention graph convolution module (C-GAT) to mine the strong association structure between categories. Besides, for the dynamic correlation between objects in the image, we propose an adaptive graph attention convolution module (A-GAT) to capture the subtle dynamic dependencies in the image. The results on two authoritative data sets show that our method is significantly better than all current state-of-the-art methods. Besides, the visualization results show that RRL-GAT can capture the semantic relationship of a specific input image and has sufficient recognizability.

In recent years, with the rapid development of deep learning, super-resolution methods based on convolutional neural networks (CNNs) have made great progress. However, the parameters and the required consumption of computing resources of these methods are also increasing to the point that such methods are difficult to implement on devices with low computing power. To address this issue, we propose a lightweight single image super-resolution network with an expectation-maximization attention mechanism (EMASRN) for better balancing performance and applicability. Specifically, a progressive multi-scale feature extraction block (PMSFE) is proposed to extract feature maps of different sizes. Furthermore, we propose an HR-size expectation-maximization attention block (HREMAB) that directly captures the long-range dependencies of HR-size feature maps. We also utilize a feedback network to feed the high-level features of each generation into the next generation’s shallow network. Compared with the existing lightweight single image super-resolution (SISR) methods, our EMASRN reduces the number of parameters by almost one-third. The experimental results demonstrate the superiority of our EMASRN over state-of-the-art lightweight SISR methods in terms of both quantitative metrics and visual quality. The source code can be downloaded athttps://github.com/xyzhu1/EMASRN.

Stereo image super-resolution exploits additional features from cross view image pairs for high resolution (HR) image reconstruction. Recently, several new methods have been proposed to investigate cross view features along epipolar lines to enhance the visual perception of recovered HR images. Despite the impressive performance of these methods, global contextual features from cross view images are left unexplored. In this paper, we propose a cross view capture network (CVCnet) for stereo image super-resolution by using both global contextual and local features extracted from both views. Specifically, we design a cross view block to capture diverse feature embeddings from the views in stereo vision. In addition, a cascaded spatial perception module is proposed to redistribute each location in feature maps according to the weight it occupies to make the extraction of features more effective. Extensive experiments demonstrate that our proposed CVCnet outperforms the state-of-the-art image super-resolution methods to achieve the best performance for stereo image super-resolution tasks. The source code is available at https://github.com/xyzhu1/CVCnet.

Face images are typically a key component in the fields of security and criminal investigation. However, due to lighting and shooting angles, faces taken under low-light conditions are often difficult to recognize. Face super-resolution (FSR) technology can restore high-resolution faces based on low-resolution inputs. However, existing face super-resolution methods typically rely on prior knowledge of inaccurate faces estimated from low-resolution images. Faces restored by low-light inputs may suffer from problems such as low brightness and many missing details. In this article, we proposed an Illumination-Enhanced Face Super-Resolution (IEFSR) model that can progressively super-resolve low-light faces of 32 × 32 pixels by an upscaling factor of 8. While reconstructing the low-light low-resolution face into a clear and high-quality face, we introduce a coarse low-resolution (LR) restoration network to recover the LR face details hidden in the dark. In the generator, we use a series of style blocks with noise to make the generated faces appear to have a more realistic visual aesthetic. Additionally, we introduce spectrum normalization in the discriminator to improve training stability. Extensive experimental evaluations show that the proposed IEFSR yields visually and metrically more attractive results than existing state-of-the-art FSR methods.

Summary With the advent of the Internet of things era, cloud computing platforms will face the challenges of massive equipment requirements for access, massive data, insufficient bandwidth, and high power consumption. Edge computing, as a new technology, makes it possible to stream media over the edge network. However, many problems related to file sharing among edge nodes and to the files provided by cloud servers exist. We propose a novel replica placement strategy for mobile media streaming in edge computing (RPME) to address the aforementioned problem. First, we introduce a multilevel replica placement model in the RPME. In the RPME, we acquire the user information and the user‐item rating matrix when the user requests the media data. At the server, we cluster the users according the user information, update the user‐item rating matrix, and then generate a replica recommendation sequence. For when the server submits the replica recommendation sequence to the edge node considering the constraints of the edge node storage capacity and the limitations on the service capacity of the requested replica, we propose an effective replica placement mechanism in the RPME. To show the benefits of the RPME, we also present several experiments to prove its validity.

How to use artificial intelligence technology to mine human abnormal behavior from considerable video data generated by the Internet-of-Things system has been intensively studied for a long time. Existing deep learning anomaly detection algorithms deployed in the cloud typically perform supervised learning based on constant kinds of abnormal behavior data. However, this supervised learning model with preset abnormal behavior categories ignores the diversity and unpredictability of abnormal occurrences in open scenarios. Thus, we propose an abnormal behavior detection algorithm as an edge network service by combining the advantages of cloud computing and the efficiency of edge networks. This method combines the double verification of global behavior detection and local fine-grained action cycle alignment to detect whether a behavior is abnormal. Moreover, to enable abnormal behavior detection models to predict test samples whose categories do not appear during the training stage, we propose an active label learning algorithm based on cycle clustering, which not only improves the efficiency of data transmission between the edge and the cloud but also makes model updates in the cloud more efficient. Extensive and quantitative experimental results show that our method can not only accurately detect abnormal human behavior at the edge of limited resources but also has strong robustness under the interference of test samples of unknown categories.

Summary In recent years, Cyber‐Physical Systems (CPS) and Artificial Intelligence (AI) have made good progress in the medical field. The medical CPS (MCPS) based on AI can realize the efficient and reasonable utilization of medical resources and improve the quality of medical process. However, current MCPS are still facing several challenges, and the privacy protection of medical data is one of the most critical challenges. Since medical data is stored in different hospitals, most studies collect data from decentralized hospitals to train a disease diagnosis model, which is not conducive to the privacy protection of patients. And in some existing solutions, it is also difficult for doctors to select the optimal model from multiple models in clinical diagnosis. In this paper, we propose a novel scheme based on federated learning in MCPS for training disease diagnosis models from distributed medical image data. Our scheme is divided into three parts: the model provider, the server, and the consumer, and a detailed working process is designed for each part. This scheme can not only effectively solve the problem of privacy protection, but also solve the problem of model selection for doctors and save storage space. It can ensure that consumers automatically get a steadily improved disease diagnosis model. This scheme is performed on simulated distributed medical image datasets. The experimental results show the effectiveness and superiority of our scheme.

Artificial intelligence has achieved great success in the field of medical-assisted diagnosis, and a deep learning technology plays a very important role in medical image recognition. However, it usually takes medical institutions extra time, energy, and cost to obtain a credible and efficient deep learning model, which is not conducive to a wide range of applications, including medical image recognition and medical decision making. In this article, we propose a novel medical-assisted diagnosis model as a service (MDMaaS). Medical institutions can obtain and use the medical-assisted diagnosis models from the service providers directly; a model training and a model application in machine learning are assigned to a service provider and a consumer, respectively. We have designed a model acquisition method based on the conventional samples and small samples for MDMaaS providers, and we have also developed a trustworthy model-based recommendation method for MDMaaS consumers, which would help the medical institutions to obtain the reliable medical-assisted diagnosis models quickly and efficiently. Based on the MDMaaS, extensive experiments are performed to verify the effectiveness of the proposed method.

Mobile message push has become a ubiquitous technology in various applications such as online resource sharing, traffic surveillance, mobile health care and environmental monitoring. In mobile terminals, energy efficiency optimization is one of the most important issues due to battery power limitations, resource constraints and quality-of-service (QoS) requirements. Considering the timely delivery, network load and terminal diversity, this paper proposes an adaptive message push strategy (AMPS) for energy efficiency optimization in mobile terminals. In AMPS, running parameters including energy parameter, operating system (OS) version and connection/polling cost in mobile terminal are first acquired and sent to the server together with the requisition data, and then the dispatching module will automatically choose a message pushing mode between polling-based and connection-based ones. The AMPS was tested in real environments using mobile phones with different OSs. Experiment results show that AMPS can efficiently optimize energy exploitation with dynamic tradeoff between terminal using time and QoS performance in comparison with polling-based and connection-based message push strategies.