The sparsity of user–item interactions remains a fundamental obstacle in collaborative filtering, limiting the ability of Graph Neural Network (GNN)-based recommender systems to capture high-order user relationships without incurring over-smoothing and computational overhead. Existing social recommendation approaches mitigate this by incorporating social networks, yet most rely on explicit ties and fail to construct informative links in their absence. Meanwhile, contrastive learning (CL) has shown promise in improving representation quality, but current view generation strategies, augmentation-based for robustness and nonaugmentation-based for semantic fidelity, are seldom combined, leaving their complementary potential underexplored. We propose Social Generating with Multiview-guided Tuning (SGMT), a unified framework that addresses both challenges. First, an interest-aware social generation mechanism constructs synthetic user–user links from shared interaction patterns, theoretically shown to compress collaborative paths and uncover latent high-order relations. Second, we present two complementary CL modules, Noise-augmented View and Semantic-explored View, which we theoretically prove to preferentially enhance uniformity and alignment, respectively, two fundamental objectives in CL. Experiments on three real-world datasets show that SGMT outperforms state-of-the-art baselines, validating both the theoretical analysis and the practical efficacy of our model.

Beyond user-item modeling, item-to-item relationships are increasingly used to enhance recommendation. However, common methods largely rely on co-occurrence, making them prone to item popularity bias and user attributes, which degrades embedding quality and performance. Meanwhile, although diversity is acknowledged as a key aspect of recommendation quality, existing research offers limited attention to it, with a notable lack of causal perspectives and theoretical grounding. To address these challenges, we propose Cadence: Diversity Recommendation via Causal Deconfounding of Co-purchase Relations and Counterfactual Exposure—a plug-and-play framework built upon LightGCN as the backbone, primarily designed to enhance recommendation diversity while preserving accuracy. First, we compute the Unbiased Asymmetric Co-purchase Relationship (UACR) between items—excluding item popularity and user attributes—to construct a deconfounded directed item graph, with an aggregation mechanism to refine embeddings. Second, we leverage UACR to identify diverse categories of items that exhibit strong causal relevance to a user's interacted items but have not yet been engaged with. We then simulate their behavior under high-exposure scenarios, thereby significantly enhancing recommendation diversity while preserving relevance. Extensive experiments on real-world datasets demonstrate that our method consistently outperforms state-of-the-art diversity models in both diversity and accuracy, and further validates its effectiveness, transferability, and efficiency over baselines.

Satellite communication links leveraging multi-source physical layer features provide critical authentication technology for secure satellite networks. However, existing physical layer security schemes suffer two major limitations: i) most methods perform one-shot decisions without tracking trust evolution over changing link geometries; ii) fixed decision rules fail to accumulate evidence and degrade robustness when spoofing parameters adapt. We propose a multi-stage dynamic trust authentication framework. Firstly, we introduce a multi-feature statistical detection module that performs real-time preliminary authentication based on Doppler shifts and power residual analysis. Then we design a particle-filtering-based trust-state estimation mechanism that fuses historical credibility with new observations via adaptive resampling, enabling continuous trust evolution and swift anomaly response. Experiments on STK-simulated datasets demonstrate that the proposed method increases secure throughput by 5.5% and reduces false-alarm and missed-detection rates by 27% and 29%, respectively.

Federated Learning (FL) that extracts data knowledge while protecting the privacy of multiple clients has achieved remarkable results in distributed privacy-preserving IoT systems, including smart traffic flow monitoring, smart grid load balancing, and so on. Since most data collected from edge devices are unlabeled, unsupervised Federated Clustering (FC) is becoming increasingly popular for exploring pattern knowledge from complex distributed data. However, due to the lack of label guidance, the common Non-Independent and Identically Distributed (Non-IID) issue of clients have greatly challenged FC by posing the following problems: How to fuse pattern knowledge (i.e., cluster distribution) from Non-IID clients; How are the cluster distributions among clients related; and How does this relationship connect with the global knowledge fusion? In this paper, a more tricky but overlooked phenomenon in Non-IID is revealed, which bottlenecks the clustering performance of the existing FC approaches. That is, different clients could fragment a cluster, and accordingly, a more generalized Non-IID concept, i.e., Non-ICD (Non-Independent Completely Distributed), is derived. To tackle the above FC challenges, a new framework named GOLD (Global Oriented Local Distribution Learning) is proposed. GOLD first finely explores the potential incomplete local cluster distributions of clients, then uploads the distribution summarization to the server for global fusion, and finally performs local cluster enhancement under the guidance of the global distribution. Extensive experiments, including significance tests, ablation studies, scalability evaluations, qualitative results, etc., have been conducted to show the superiority of GOLD.

Cooperative path planning for UAVs using hierarchical optimal control is a critical technology for urban emergency communication networks. Existing methods typically decouple trajectory optimization from communication scheduling, which inherently leads to suboptimal performance. Attempting to solve the problem monolithically creates a large-scale Mixed-Integer Non-Linear Program that is computationally intractable for real-time deployment. Compounding this, current models often overlook the need for dynamic UAV role-switching, limiting the system’s functional flexibility and operational adaptability. We propose a hierarchical control framework that decomposes the problem into a dual-layer model predictive control architecture. Specifically, we linearize the original problem within a receding horizon control loop for long-term task and trajectory planning in the strategic layer. Guided by this, the lower trajectory layer employs non-linear model predictive control, solved via sequential convex programming, to generate real-time trajectories and perform adaptive role switching. Simulation results demonstrate that the proposed framework reduces mission makespan by 8.6% and increases the total data collected by 2.4%, validating its efficiency and robustness.

Graph-level clustering aims to partition a set of graphs into different clusters and has important applications in social networks, bioinformatics, etc. Although there have been some approaches to graph-level clustering such as various graph kernels and graph neural networks, it remains a huge challenge to select kernels and neural network architectures, since the task is unsupervised. Moreover, the clustering accuracy and model interpretability of these approaches are low and should be improved to satisfy practical needs. To address these issues, in this work, we propose a graph-level clustering method that uses Bayesian optimization to integrate various graph kernels (BOGK). BOGK aggregates the similarity matrices generated by different graph kernels and automatically learns the aggregation weights and a thresh olding parameter via maximizing internal cluster validity indices. Our BOGK is free of manual hyperparameter tuning via Bayesian optimization, while it enjoys considerable interpretability, as the weight for each similarity matrix represents the importance of different structural or pattern information in graphs. Experimental results show that our BOGK outperforms the state-of-the-art on ten graph benchmark datasets. Our code is available at https://anonymous.4open.science/r/BOGK-804E.

The rapid expansion of gaming industry requires advanced recommender systems tailored to its dynamic landscape. Existing Graph Neural Network (GNN)-based methods primarily prioritize accuracy over diversity, overlooking their inherent tradeoff. To address this, we previously proposed CPGRec, a balance-oriented gaming recommender system. However, CPGRec fails to account for critical disparities in player–game interactions, which carry varying significance in reflecting players’ personal preferences and may exacerbate over-smoothness issues inherent in GNN-based models. Moreover, existing approaches underutilize the reasoning capabilities and extensive knowledge of large language models (LLMs) in addressing these limitations. To bridge this gap, we propose two new modules. First, Preference-informed Edge Reweighting (PER) module assigns signed edge weights to qualitatively distinguish significant player interests and disinterests while then quantitatively measuring preference strength to mitigate over-smoothing in graph convolutions. Second, Preference-informed Representation Generation (PRG) module leverages LLMs to generate contextualized descriptions of games and players by reasoning personal preferences from comparing global and personal interests, thereby refining representations of players and games. Experiments on two Steam datasets demonstrate CPGRec+’s superior accuracy and diversity over state-of-the-art models. The code is accessible at https://github.com/HsipingLi/CPGRec-Plus .

Collaborative modeling in industrial processes with similar production procedures and complex industrial parameters is an effective approach to address issues about online soft measurement. However, it is a challenging task for collaborative modeling at the presence of non-independent and identically distributed (Non-IID) data, which is arising from variations in production materials and equipment. In this paper, we propose a cooperative reinforced resilient federated learning for clients with multiple working conditions to solve the Non-IID problem by implementing a personalized strategy centered on local data storage, which can accurately build personalized models suitable for distributed data on a single client. Inspired by the success of multi-agent reinforcement learning (MARL) in solving complex decision problems, we design a personalized aggregation mechanism guided by MARL and modeling process information. This mechanism optimizes model aggregation decisions based on data distribution and quality, thereby improving the effectiveness of the training process and enhancing the overall performance of personalized models. Meanwhile, we design an exploration and reward strategy for federated scenarios with distribution differences, which utilizes the collaborative modeling characteristics and advantages of federated learning to improve the exploration efficiency of reinforcement learning in solving personalized federated problems. Experimental results conducted on industry and benchmark datasets demonstrate that the proposed model surpasses existing methods in collaborative modeling under Non IID scenarios.

The garment structure serves as a crucial medium for expressing the designer's creative vision and showcasing the distinctive character of clothing items. Effective editing of garment structure in fashion images allows for an advanced preview of the design, accelerating the process of garment customization to meet individualized requirements. Although large-scale diffusion models have demonstrated impressive image generation and editing capabilities, no efforts have been made to exploit their potential in part-level editing of images. Unlike previous research, we define a clothing structure editing (CSE) task aimed at accurately editing the local structure of human-centered clothing images through simple instruction-based prompts while maintaining the consistency of clothing appearance. Specifically, this paper develops a new controllable triple-flow framework for structure editing named FashionTailor. An additional network called ClothingNet is proposed to extract the clothing details to address the rigid constraints of the original garment structure. Then, we propose a semantic-refined module to extract the semantic understanding of the source image and adaptively focus on the part to be edited. We also design a cross-blend attention mechanism to integrate fine-grained clothing features to guarantee precise alignment between appearance and target structure features. In addition, a garment structure dataset called StructureFashion has been collated, wherein each item of clothing is represented by multiple photos with diverse structure characteristics, containing over six million pairs. Finally, our method supports editing the structure of multiple parts on a garment simultaneously. Extensive experiments validate the effectiveness of our method for editing part-level human images in StructureFashion dataset and real-scenarios.

Existing face forgery detection methods achieve promising performance when training and testing forgery data are from identical manipulation types, while they fail to generalize well to unseen samples. In this paper, we experimentally investigate and find that the poor generalization of the methods mainly arises from their overfitting on the known fake patterns. Excessively focused on seen fakes, those detectors fail to effectively learn image-intrinsic information and the distributional disparity between real and fake images. Then, to address this issue, we redefine fake learning as real-fake distributional disparity learning. We propose a novel deepfake detection framework learning distributional disparity based on the differentiated reconstruction on real and fake images for improved generalization. Specifically, distributional disparity learning on differentiated reconstruction of the real and fake images, enforces the model to learn image-invariant intrinsic representations. The reconstruction on real and fake images forces the decoders to learn the distribution of real and fake images, respectively. Moreover, to avoid the influence from the specificalization of the known fake patterns, we further propose the information interaction learning on the encoded intrinsic information and the pixel disparity between the input image and its reconstruction to distinguish face forgeries that are even unknown. Extensive experiments on large-scale benchmark datasets demonstrated the effectiveness of addressing the overfitting issue of the classification network, and verified the superior performance of our method.

One-shot talking head synthesis aims to animate a source person’s portrait with driving video sequences. Recent facial keypoint-based methods have achieved remarkable animation performance and produced high-quality results. However, it remains challenging to perform cross-identity face reenactment by transferring subtle facial motions with correct geometry and appearance. To break the above limitations, in this paper, we propose a subtle motion compensation network to recover correct facial expressions by leveraging the decoupled 3D Morphable Model (3DMM) coefficient. In addition, to generate faithful animation results, a facial appearance feature memory bank is designed to learn accurate facial features and better recover the appearance. Experimental results have demonstrated that our proposed model can outperform state-of-the-art methods by generating faithful videos with correct subtle motion transfer and consistent identity preserving.

Category-level object pose estimation is a longstanding and fundamental task crucial for augmented reality and robotic manipulation applications. Existing RGB-based approaches struggle with multi-stage settings and heavily rely on off-the-shelf techniques, such as object detectors, depth estimators, non-differentiable NOCS shape alignment, etc. Extra dependencies lead to the accumulation of errors and complicate the whole pipeline, limiting the deployment of these approaches in practical applications. This paper streamlined an end-to-end framework unifying the single-frame and video-based category-level pose estimation. Specifically, instead of explicitly introducing extra dependencies, the DINOv2 encoder and depth decoder, as robust semantic and geometric prior extractors, are leveraged to produce intra-frame hierarchical semantic and geometric features. A spatial-temporal sparse query network is developed to model the implicit correspondence and inter-frame correlations between a set of implicit 3D query anchors and intra-frame features. Finally, a pose prediction head is employed using the bipartite matching algorithm. Experimental results demonstrate that our model achieves state-of-the-art performance compared with RGB-based categorical pose estimation methods on the REAL275 and CAMERA25 datasets. Our code is available at https://andrewchiyz.github.io/vision.3dv.seqpose/.

As a form of multimedia creation, visual novel (VN) conveys engaging narratives through the integrated presentation of text, images, and music, and has shown promise across various application domains. Recent advances in generative AI have fueled interest in automating VN creation using LLMs and other foundation models. However, fully end-to-end VN creation (i.e., from user description to executable VN) remains underexplored and presents several key challenges: 1) the hallucination and limited capacity of LLMs hinder the generation of long and coherent plots; 2) current models lack effective mechanisms for ensuring cross-modal consistency between plot, visual, and audio elements. To address these issues, we propose a hierarchical end-to-end framework for automatic VN generation and assembly, which employs an outline-guided autoregressive generation mechanism that transforms high-level user prompts into coherent plots, while a vision LLM-based self-correction mechanism ensures consistency between multimedia assets and plot content. Additionally, we introduce a script validation mechanism to ensure the executable of the final VN application. Experiments demonstrate that our framework generates high-quality VN applications with coherent storylines and consistent multimedia content.

The explosive growth of the video game industry has created an urgent need for recommendation systems that can scale with expanding catalogs and maintain user engagement. While prior work has explored accuracy and diversity in recommendations, existing models underutilize playtime, a rich behavioral signal unique to gaming platforms, and overlook the potential of multimodal information to enhance diversity. In this paper, we propose DP 2 Rec, a novel Dual-Phase Playtime-guided Recommendation model designed to jointly optimize accuracy and diversity. First, we introduce a playtime-guided interest intensity exploration module that separates strong and weak preferences via dual-beta modeling, enabling fine-grained user profiling and more accurate recommendations. Second, we present a playtime-guided multimodal random walks module that simulates player exploration using transitions guided by both playtime-derived interest similarity and multimodal semantic similarity. This mechanism preserves core preferences while promoting cross-category discovery through latent semantic associations and adaptive category balancing. Extensive experiments on a real-world game dataset show that DP 2 Rec outperforms existing methods in both recommendation accuracy and diversity. The dataset and source code are released at https://github.com/zqxwcevrtyui/DP2Rec

The evolution of individuals' living standards has transformed clothing preferences, elevating fashion beyond mere utility to a potent means of self-expression. However, the intricate task of outfit selection persists as a challenge, marked by traditional methods facing challenges such as the oversight of combined factors of scene and body shape, insufficient emphasis on detail-oriented matching, and overreliance on rigid hierarchical structures. To tackle these challenges, this article introduces a novel model, termedGlobal-Local matching network towardsOutfit recommendation for diverse bodyShapes andScenes (GLOSS). Specifically, we first introduce a newly compiled fashion dataset, StreetFashion, to capture the combined factors of body shapes and scene characteristics. Additionally, we develop innovative multi-level globality- and locality-aware matching methods to enhance the accuracy of outfit recommendations by comprehensively considering both global and local relationships among clothing items, outfits, users, and scenes. Furthermore, we develop a personalized outfit heterogeneous graph that incorporates historical interactions among fashion entities, enabling effective modeling of nonstrict hierarchical relationships. Evaluation conducted on both our collected dataset and an adapted existing dataset demonstrates the effectiveness of our proposed approach in outfit recommendation. Our codes are released at:https://github.com/ChenrMa/GLOSS.

Fine-grained deduplication (also known as delta compression) can achieve a better deduplication ratio compared to chunk-level deduplication. This technique removes not only identical chunks but also reduces redundancies between similar but non-identical chunks. Nevertheless, it introduces considerable I/O overhead in deduplication and restore processes, hindering the performance of these two processes and rendering fine-grained deduplication less popular than chunk-level deduplication to date. In this paper, we explore various issues that lead to additional I/O overhead and tackle them using several techniques. Moreover, we introduce MeGA, which attains fine-grained deduplication/restore speed nearly equivalent to chunk-level deduplication while maintaining the significant deduplication ratio benefit of fine-grained deduplication. Specifically, MeGA employs (1) a backup-workflow-oriented delta selector and cache-centric resemblance detection to mitigate poor spatial/temporal locality in the deduplication process, and (2) a delta-friendly data layout and “Always-Forward-Reference” traversal to address poor spatial/temporal locality in the restore workflow. Evaluations on four datasets show that MeGA achieves a better performance than other fine-grained deduplication approaches. Specifically, MeGA significantly outperforms the traditional greedy approach, providing 10–46 times better backup speed and 30–105 times more efficient restore speed, all while preserving a high deduplication ratio.

Recommender systems are essential in the ever-evolving landscape of e-commerce and social media platforms, delivering personalized recommendations by predicting user preferences. However, the growing need for explainable recommendation has arisen to enhance transparency and persuasiveness. In response, we present correlation-driven explainable recommendation with aspect and rating boosted representation learning (CER-ARRL), a unified joint-ranking framework that capitalizes on the robust capabilities of neural collaborative filtering to model the intricate dynamics among users, items, and explanations. By extracting information from explicit and implicit user emotional reviews, our framework enriches the representations of users and items. This integration yields simultaneous improvements in both item recommendation and explanation ranking tasks. In addition, CER-ARRL effectively exploits the structural correlation between phrases as well as the structural and semantic correlations between emojis to facilitate explanation ranking. This work represents the pioneering work to address the item-explanation joint recommendation task by integrating both interpretative phrases and illustrative emojis. Through extensive experiments on various datasets, including our collected dataset, we demonstrate the superiority of the proposed method over existing baselines.

As information technology has been enabling services, the modern service industry, a cornerstone of the global economy, has driven development of the era of the service economy. According to the World Bank, since the 21stcentury, the service sector's share of global GDP has consistently been between 61% and 65%. “Everything as a service” has become a phenomenon in human society. Education on service science and engineering has been developed to cultivate more young people for the emerging service industry and service society.

Automatic multi-organ segmentation of the abdominal region is a critical yet challenging task in computer-aided medical image analysis. Recent advances in CNN- and Transformer-based encoder-decoder models tend to implicitly learn context features by using enhanced effective receptive fields to capture local and global range dependencies. However, due to the complex anatomical structure, those models cannot recover the anatomical topology properly and result in broken organs with inaccurate semantic labels. Therefore, in this paper, by considering the anatomy priors of multi-organs, we propose a Context Prior Network, namely CPNet, which integrates the 3D context semantic relations and geometry priors as explicit anatomical constraints. Specifically, a Semantic Relation Prior Propagation (SRPP) module is designed to propagate the semantic relations between voxels progressively. Moreover, a Multiple Context Prior Prediction (MCPP) module is adopted to preserve the accurate shape and topology by recovering 3D contours and surface normal. Experimental results demonstrate our proposed model outperforms state-of-the-art models for multi-organ segmentation on Abdomen CT and MRI datasets, especially for recovering organs with correct semantic labels and anatomical structures.

Collocated clothing synthesis (CCS) has emerged as a pivotal topic in fashion technology, primarily concerned with the generation of a clothing item that harmoniously matches a given item. However, previous investigations have relied on using paired outfits, such as a pair of matching upper and lower clothing, to train a generative model for achieving this task. This reliance on the expertise of fashion professionals in the construction of such paired outfits has engendered a laborious and time-intensive process. In this paper, we introduce a new self-driven framework, named style- and texture-guided generative network (ST-Net), to synthesize collocated clothing without the necessity for paired outfits, leveraging self-supervised learning. ST-Net is designed to extrapolate fashion compatibility rules from the style and texture attributes of clothing, using a generative adversarial network. To facilitate the training and evaluation of our model, we have constructed a large-scale dataset specifically tailored for unsupervised CCS. Extensive experiments substantiate that our proposed method outperforms the state-of-the-art baselines in terms of both visual authenticity and fashion compatibility.

Text-guided image generative diffusion models achieve fast development on the generation and editing of high-quality images. To extend such success to video editing, some efforts combining image generation with video editing have been made, which however only achieve inferior performance. We attribute it to two challenges: 1) different from the static image generation, it is tricky for dynamic video information to ensure the temporal fidelity of motion consistency across different frames; 2) the randomness of the frame generation process makes it hard to continuously retain the similar spatial fidelity for the original detailed features. In this paper, we propose a new high-fidelity diffusion model-based zero-shot text-guided video editing network, called HiFiVEditor, which aims to conduct effective video editing with high fidelity of the original video's detailed and dynamic information. Specifically, we propose a Spatial-Temporal Fidelity Block (STFB) that enables the model to restore the spatial features by enlarging the spatial perceptual field to avoid loss of important information, and capture more dynamic information between different frames by using all frames for preserving temporal consistency to achieve better temporal fidelity. In addition, we introduce Null-Text Embedding to create a soft text embedding to optimize the noise learning process, so that the latent noise can be aligned with the prompt. Furthermore, to tune the video style and render it more realistic, we employ a Prior-Guided Perceptual Loss to constrain the prediction results to avoid deviating from the original video style. Extensive experiments demonstrate the superior video editing capability compared to existing works.

Performing an object detection task after the restoration of a hazy image, or rather detecting with the network backbone directly, will result in the inclusion of information mixed with dehazing, which tends to interfere with detection performance. To address these issues, we propose a novel framework for detecting objects in dehazed images via Dual Cross-Stage Partial Learning (DCSP). Specifically, we introduce a Cross-Stage Partial (CSP) module for extracting clean feature information after dehazing. Secondly, to enhance data integrity, we employ a skip-input strategy to supplement information related to object detection features that may be lost during the dehazing task, while avoiding the gradient vanishing problem. In addition, CSP is also introduced to facilitate comprehensive learning of multiple feature representations. Finally, to avoid the inclusion of irrelevant dehazing information in detection, we apply a Ground-Truth Flow at detection network (at dark3), for fine feature information calibration. Additionally, we created a synthetic fog dataset to expand the training data for DCSP. Experimental results on both synthetic and real-world datasets demonstrate the effectiveness and accuracy of the proposed method. The code is available at https://github.com/zhaojinbiao/DCSP.

In recent years, the video game industry has experienced substantial growth, presenting players with a vast array of game choices. This surge in options has spurred the need for a specialized recommender system tailored for video games. However, current video game recommendation approaches tend to prioritize accuracy over diversity, potentially leading to unvaried game suggestions. In addition, the existing game recommendation methods commonly lack the ability to establish strict connections between games to enhance accuracy. Furthermore, many existing diversity-focused methods fail to leverage crucial item information, such as item category and popularity during neighbor modeling and message propagation. To address these challenges, we introduce a novel framework, called CPGRec, comprising three modules, namely accuracy-driven, diversity-driven, and comprehensive modules. The first module extends the state-of-the-art accuracy-focused game recommendation method by connecting games in a more stringent manner to enhance recommendation accuracy. The second module connects neighbors with diverse categories within the proposed game graph and harnesses the advantages of popular game nodes to amplify the influence of long-tail games within the player-game bipartite graph, thereby enriching recommendation diversity. The third module combines the above two modules and employs a new negative-sample rating score reweighting method to balance accuracy and diversity. Experimental results on the Steam dataset demonstrate the effectiveness of our proposed method in improving game recommendations. The dataset and source codes are anonymously released at: https://github.com/CPGRec2024/CPGRec.git.

Collocated clothing synthesis using generative networks has become an emerging topic in the field of fashion intelligence, as it has significant potential economic value to increase revenue in the fashion industry. In previous studies, several works have attempted to synthesize visually-collocated clothing based on a given clothing item using generative adversarial networks (GANs) with promising results. These works, however, can only accomplish the synthesis of one collocated clothing item each time. Nevertheless, users may require different clothing items to meet their multiple choices due to their personal tastes and different dressing scenarios. To address this limitation, we introduce a novel batch clothing generation framework, named BC-GAN, which is able to synthesize multiple visually-collocated clothing images simultaneously. In particular, to further improve the fashion compatibility of synthetic results, BC-GAN proposes a new fashion compatibility discriminator in a contrastive learning perspective by fully exploiting the collocation relationship among all clothing items. Our model was examined in a large-scale dataset with compatible outfits constructed by ourselves. Extensive experiment results confirmed the effectiveness of our proposed BC-GAN in comparison to state-of-the-art methods in terms of diversity, visual authenticity, and fashion compatibility.

Fashion recommendation has become a prominent focus in the realm of online shopping, with various tasks being explored to enhance the customer experience. Recent research has particularly emphasized fashion recommendation based on body shapes, yet a critical aspect of incorporating multimodal data relevance has been overlooked. In this paper, we present the Contrastive Multimodal Cross-Attention Network, a novel approach specifically designed for fashion recommendation catering to diverse body shapes. By incorporating multimodal representation learning and leveraging contrastive learning techniques, our method effectively captures both inter- and intra-sample relationships, resulting in improved accuracy in fashion recommendations tailored to individual body types. Additionally, we propose a locality-aware cross-attention module to align and understand the local preferences between body shapes and clothing items, thus enhancing the matching process. Experimental results conducted on a diverse dataset demonstrate the state-of-the-art performance achieved by our approach, reinforcing its potential to significantly enhance the personalized online shopping experience for consumers with varying body shapes and preferences.

Today,fashion design can be readily performed by most people due to the rapid development of design tools. However, not everyone possesses the professional skills to produce an aesthetically pleasing design. In order to assist an inexperienced user during the design process, this research explored a new fashion-related disentanglement task, with the goal of creating novel fashion items with controllable attributes. The key idea is to develop a unified framework, called CTS-GAN, by disentangling the colors, textures, and shapes of fashion items simultaneously using a generative adversarial network (GAN). Specifically, we first introduced a fashion attribute encoder to decompose input fashion items into three latent spaces, i.e., color, texture, and shape. A fashion item pattern-making module (FIPM)-based generator was then proposed to control the corresponding parameters of color and texture in FIPMs independently and combine them with the shape features in order to accomplish the final generation of new fashion items. Furthermore, three independent pathways were introduced to extract the representations of color, texture, and shape in fashion items to optimize our CTS-GAN in an unsupervised manner. Extensive experimental results demonstrate the effectiveness of our CTS-GAN and suggest that it can generate diverse, novel fashion images by taking full advantage of the controllability of the colors, textures, and shapes of different fashion items.

In this article, we propose a new cross locality relation network (CLRNet) to generate high-quality crowd density maps for crowd counting in videos. Specifically, a cross locality relation module (CLRM) is proposed to enhance feature representations by modeling local dependencies of pixels between adjacent frames with an adapted local self-attention mechanism. First, different from the existing methods which measure similarity between pixels by dot product, a new adaptive cosine similarity is advanced to measure the relationship between two positions. Second, the traditional self-attention modules usually integrate the reconstructed features with the same weights for all the positions. However, crowd movement and background changes in a video sequence are uneven in real-life applications. As a consequence, it is inappropriate to treat all the positions in reconstructed features equally. To address this issue, a scene consistency attention map (SCAM) is developed to make CLRM pay more attention to the positions with strong correlations in adjacent frames. Furthermore, CLRM is incorporated into the network in a coarse-to-fine way to further enhance the representational capability of features. Experimental results demonstrate the effectiveness of our proposed CLRNet in comparison to the state-of-the-art methods on four public video datasets. The codes are available at: https://github.com/Amelie01/CLRNet.

The field of fashion compatibility learning has attracted great attention from both the academic and industrial communities in recent years. Many studies have been carried out for fashion compatibility prediction, collocated outfit recommendation, artificial intelligence (AI)-enabled compatible fashion design, and related topics. In particular, AI-enabled compatible fashion design can be used to synthesize compatible fashion items or outfits to improve the design experience for designers or the efficacy of recommendations for customers. However, previous generative models for collocated fashion synthesis have generally focused on the image-to-image translation between fashion items of upper and lower clothing. In this article, we propose a novel outfit generation framework, i.e., OutfitGAN, with the aim of synthesizing a set of complementary items to compose an entire outfit, given one extant fashion item and reference masks of target synthesized items. OutfitGAN includes a semantic alignment module (SAM), which is responsible for characterizing the mapping correspondence between the existing fashion items and the synthesized ones, to improve the quality of the synthesized images, and a collocation classification module (CCM), which is used to improve the compatibility of a synthesized outfit. To evaluate the performance of our proposed models, we built a large-scale dataset consisting of 20 000 fashion outfits. Extensive experimental results on this dataset show that our OutfitGAN can synthesize photo-realistic outfits and outperform the state-of-the-art methods in terms of similarity, authenticity, and compatibility measurements.

Synthesizing realistic images of fashion items which are compatible with given clothing images, as well as conditioning on multiple modalities, brings novel and exciting applications together with enormous economic potential. In this work, we propose a multi-modal collocation framework based on generative adversarial network (GAN) for synthesizing compatible clothing images. Given an input clothing item that consists of an image and a text description, our model works on synthesizing a clothing image which is compatible with the input clothing, as well as being guided by a given text description from the target domain. Specifically, a generator aims to synthesize realistic and collocated clothing images relying on image- and text-based latent representations learned from the source domain. An auxiliary text representation from the target domain is added for supervising the generation results. In addition, a multi-discriminator framework is carried out to determine compatibility between the generated clothing images and the input clothing images, as well as visual-semantic matching between the generated clothing images and the targeted textual information. Extensive quantitative and qualitative results demonstrate that our model substantially outperforms state-of-the-art methods in terms of authenticity, diversity, and visual-semantic similarity between image and text.

Video super-resolution (VSR) on mobile devices aims to restore high-resolution frames from their low-resolution counterparts, satisfying the requirements of performance, FLOPs and latency. On one hand, partial feature processing, as a classic and acknowledged strategy, is developed in current studies to reach an appropriate trade-off between FLOPs and accuracy. However, the splitting of partial feature processing strategy are usually performed in a blind manner, thereby reducing the computational efficiency and performance gains. On the other hand, current methods for mobile platforms primarily treat VSR as an extension of single-image super-resolution to reduce model calculation and inference latency. However, lacking inter-frame information interaction in current methods results in a suboptimal latency and accuracy trade-off. To this end, we propose a novel architecture, termed Feature Aggregating Network with Inter-frame Interaction (FANI), a lightweight yet considering frame-wise correlation VSR network, which could achieve real-time inference while maintaining superior performance. Our FANI accepts adjacent multi-frame low-resolution images as input and generally consists of several fully-connection-embedded modules, i.e., Multi-stage Partial Feature Distillation (MPFD) for capturing multi-level feature representations. Moreover, considering the importance of inter-frame alignment, we further employ a tiny Attention-based Frame Alignment (AFA) module to promote inter-frame information flow and aggregation efficiently. Extensive experiments on the well-known dataset and real-world mobile device demonstrate the superiority of our proposed FANI, which means that our FANI could be well adapted to mobile devices and produce visually pleasing results.

Recent advances in single image super-resolution (SISR) have achieved remarkable performance through deep learning. However, the high computational cost hinders the deployment of SISR models on edge devices. Instead of proposing new SISR models, a new trend is emerging to improve network efficiency by reducing parameters, FLOPs, and inference time through slight modifications to the original models. However, recent methods usually focus on reducing only one of three metrics, i.e., FLOPs, parameters and inference time, which inevitably increases the other two metrics. In this paper, we propose a novel Adaptive Student Inference Network (ASIN) on popular SISR models, which aims at reducing FLOPs and inference time while maintaining the number of parameters and restoring clearer high-resolution images. Specifically, our ASIN divides a SISR model into three components (head, body and tail) and adopts various strategies for each part. For head and tail parts, to ensure the restored images contain more detailed information, a novel auxiliary Enhanced Teacher Network (ETNet) is designed, which is trained with the ground-truth images to obtain more prior knowledge to guide student network to extract more accurate textures using a new knowledge distillation method. For the body part, owing to the varying difficulties of the reconstructions in different regions, we propose an Adaptive Depth Predicted Module (ADPM) to dynamically shorten average depth of network to reduce the computational cost of overall network. Extensive experiments on two datasets demonstrate the effectiveness and state-of-the-art performance of our ASIN compared to its counterparts.

Detecting insulators and their defects is a key task in real-time power grid surveillance with the rapid development of national smart grid. Traditional surveillance usually relies on maintenance personnel, leading to the issues of inefficiency and unsafety. Thus, with the prosperity of deep learning, we proposed a detection algorithm, named E-VarifocalNet, which is an enhanced version of the basic VarifocalNet method. The proposed E-VarifocalNet is specifically designed for detecting insulators and their defects. We developed a classification loss based on varifocal loss and the number of samples to solve the imbalance problem in object detection. Furthermore, a regression loss based on GIoU loss and Wasserstein distance is designed to gain higher flexibility in the representation of bounding boxes. Additionally, we applied a feature pyramid network based on dilated convolution and heatmap to build global and local semantic relations among pixels so as to enhance the detection accuracy on salient areas. Our dataset containing 2,100 images and 5,217 object instances was collected through real-time drones and an open data platform. Our E-VarifocalNet gets the highest mAP and a low model complexity on our dataset among state-of the-art object detectors, indicating the potential of our algorithm in real-time power grid surveillance applications.

Low-Light Stereo Image Enhancement (LLSIE) tackles the challenge of improving the illumination and restoring the details in stereo images. However, existing deep learning-based LLSIE methods trained on high-resolution low-light images often exhibit sub-optimal performance when interacting with information from the left and right views. We find that this is because of: (1) the high computational cost arising from quadratic complexity, which hinders the enhancement model's ability to process high-resolution images; and (2) the limitations of conventional fusion strategies in previous work, which inadequately capture cross-view cues, resulting in weak feature representation and compromised detail recovery. To address these limitations, we propose a novel Dynamic Grouped Interaction Network (DGI-Net) to enhance illumination and recover more details while reducing the computational cost. Specifically, DGI-Net employs the U-Net structure, which effectively mitigates noise during the low-light enhancement. Furthermore, we design a Grouped Stereo Interaction Module (GSIM) with a grouping strategy to efficiently discover cross-view cues while minimizing computations. To dynamically fuse stereo information and fully exploit cross-view correlations, we also introduce a Dynamic Embedding Module (DEM) to establish dynamic connections between inter-view cues and intra-view features, which performs dynamic weight processing on cross-view cues to eliminate noise during fusion. For intra-view processing, we present a Diversity Enhanced Block (DEB) to extract multi-scale features, thereby improving diversity and feature representation. This multi-scale feature extraction also addresses low image contrast in dark lighting conditions. Experimental results demonstrate that DGI-Net outperforms current state-of-the-art methods in low-light stereo image enhancement.

Traditional fashion design typically requires the expertise of designers, which limits the involvement of ordinary users during the design process. While it would be desirable for users to participate in the preliminary design phase, their lack of basic design knowledge may render them too inexperienced to produce satisfactory designs. To improve the design efficiency for common users, we present a novel interactive fashion design framework based on generative adversarial network (GAN). This framework can assist users in designing fashion items by drawing only rough scribbles and providing simple fashion styles. Specifically, we propose a new cross-domain feature fusion encoder network that maps design image features from different domains into a series of style vectors which are then fed into a generator. We demonstrate that the learned style vectors can decouple the representations of cross-domain design elements and control the design results through scribbles and style images. Furthermore, we propose a method for rewriting our model with scribbles and style images, to allow designers to train our model more easily. To examine the effectiveness of our proposed model, we constructed a large-scale dataset containing 90,000 pairs of fashion item images. Experimental results show that our proposed method outperforms state-of-the-art methods and can effectively control cross-domain image features, suggesting the potential of our model for providing users with an intelligence-driven interactive design tool.

Transformer-based approaches to image captioning have shown great success by utilizing long-term dependency for visual embedding. However, their coarse long-term dependency, using the multi-head self-attention mechanism to capture the contextual interactions between the visual tokens on the time step and (or) embedded dimension, fail to distinguish fine-grained features of local partition. In this case, some similar features are captured, which leads to feature redundancy that decreases the performance. To respond to this issue, this paper proposes a novel image captioner embedding visual cross-partition dependency, dubbed CropCap. Specifically, the visual sequence generated from the Swin Transformer-based pre-embedding network is fed into the proposed cross-partition dependency module to refinedly model the interaction between partial representations on both the time step and embedded dimension. Furthermore, we formulaically reason the proposed cross-partition dependency, and theoretically prove its correctness. Extensive comparisons on the benchmark MS-COCO dataset demonstrated the effectiveness addressing the information redundancy issue, and verified the superior performance of our method.

The process of fashion design involves creative expression through various methods, including sketch drawing, brush painting, and choices of textures and colors, all of which are employed to characterize the originality and uniqueness of the designed fashion items. Despite recent advances in intelligence-driven fashion design, the complexity of the diverse elements of a fashion item, such as its texture, color and shape, which are associated with the semantic information conveyed, continues to present challenges in terms of generating high-quality fashion images as well as achieving a controllable editing process. To address this issue, we propose a unified framework, FashionDiff, that leverages the diverse elements in fashion items to generate new items. Initially, we collected a large number of fashion images with multiple categories and created pairwise data in terms of sketch and additional data, such as brush areas, textures, or colors. To eliminate semantic discrepancies between these pairwise datasets, we introduce a feature modulation fusion (FMFusion) process, which enables interactive communication among different images, allowing them to be fused into latent spaces characterized by different resolutions. In order to produce high-quality editable fashion images, we develop a generator based on a state-of-the-art diffusion model called FD-ControlNet, which integrates latent spaces into different layers of the generator to generate ready-to-wear fashion items. Qualitative and quantitative experimental results demonstrate the effectiveness of our proposed method, and suggest that our model can offer flexible control over the generated images in terms of sketches, brush areas, textures, and colors.

Texture constitutes the color and fabric of fashion items. Its choice in fashion items can directly express the personality and emotional state of a wearer. Despite the rapid development of intelligence-driven fashion design, it remains challenging to achieve independent control over texture without affecting other attributes, due to the highly intertwined nature of texture space. To accomplish fine-grained texture disentanglement, we propose InspirNET, an unsupervised disentangled generative adversarial framework, that manipulates textures in a fine-grained latent space so as to produce new textures effectively, aiming to broaden the range of fashion options available to common users with distinct textures as well as boosting designers' potential for fashion innovation and inspiration. Specifically, we first introduce an auto-fashion attribute encoder to map the input fashion item into texture and structure spaces. To achieve unsupervised fine-grained texture disentanglement, our model proposes a K-textures disentanglement module that decomposes the texture space into several orthogonal vectors, each of which is empowered to control an independent texture element. In particular, by employing an orthogonal eigenvector to interpolate with another, a multitude of new textures can be generated easily. Qualitative and quantitative experiments demonstrate that our InspirNET can effectively utilize decomposed orthogonal vectors to generate a wide range of fashion items with diverse textures. Our model exhibits superior performance over state-of-the-art methods in terms of maintaining the authenticity of texture transfer.

Outfit generation is a challenging task in the field of fashion technology, in which the aim is to create a collocated set of fashion items that complement a given set of items. Previous studies in this area have been limited to generating a unique set of fashion items based on a given set of items, without providing additional options to users. This lack of a diverse range of choices necessitates the development of a more versatile framework. However, when the task of generating collocated and diversified outfits is approached with multimodal image-to-image translation methods, it poses a challenging problem in terms of non-aligned image translation, which is hard to address with existing methods. In this research, we present FCBoost-Net, a new framework for outfit generation that leverages the power of pre-trained generative models to produce multiple collocated and diversified outfits. Initially, FCBoost-Net randomly synthesizes multiple sets of fashion items, and the compatibility of the synthesized sets is then improved in several rounds using a novel fashion compatibility booster. This approach was inspired by boosting algorithms and allows the performance to be gradually improved in multiple steps. Empirical evidence indicates that the proposed strategy can improve the fashion compatibility of randomly synthesized fashion items as well as maintain their diversity. Extensive experiments confirm the effectiveness of our proposed framework with respect to visual authenticity, diversity, and fashion compatibility.

Automatically accomplishing intelligent fashion design with certain ‘inspiration’ images can greatly facilitate a designer’s design process, as well as allow users to interactively participate in the process. In this research, we propose a generative adversarial network with heatmap-guided semantic disentanglement (HSD-GAN) to perform an ‘intelligent’ design with ‘inspiration’ transfer. Our model aims to learn how to integrate the feature representations, from the styles of both source fashion items and target fashion items, in an unsupervised manner. Specifically, a semantic disentanglement attention-based encoder is proposed to capture the most discriminative regions of different input fashion items and disentangle the features into two key factors: attribute and texture. A generator is then developed to synthesize mixed-style fashion items by utilizing the two factors. In addition, a heatmap-based patch loss is introduced to evaluate the visual-semantic matching degree between the texture of the generated fashion items and the input texture information. Extensive experimental results show that our proposed HSD-GAN consistently achieves superior performance, compared to other state-of-the-art methods.

Due to the lack of faulty data on the target machine, intelligent networks often need to learn fault knowledge from other relevant machines. Unfortunately, data from different machines introduce individualized deviations, which may lead to overfitting of network learning and reduce its generalization ability. To address the problem, this article proposes a collaborative multimachine generalization method—causal consistency network (CCN), which mines the invariant causal information in individualized machines through a collaborative way to achieve knowledge generalization. In CCN, instead of emphasizing the domain invariance of features, causal consistency loss depicting the consistency of fault causality representations in deep latent variables is proposed. Moreover, to transform the individualized data of different machines into consistent representations, a collaborative training loss is proposed to describe the underlying invariant causal mechanism of the fault features. The generalization results among 6 machines containing 43 individual bearings and 20 operating conditions demonstrate the superiority of CCN.

The traditional fashion industry is heavily dependent on designers whose talent and vision have a significant impact on their innovative designs. Through taking advantage of recent advances in image-to-image translation by generative adversarial networks (GANs), marked improvement in designers’ efficiency is now possible. Considering both randomness and controllability in the design process, this article presents a novel artificial intelligence (AI)-based framework for fashion design. Under this framework, a sketch-generation module which is based on latent space is firstly introduced for designing various sketches. Secondly, a rendering-generation module is proposed to learn mapping between textures and sketches to complete the task of fashion design. In order to achieve effectiveness in synthesizing semantic-aware textures on sketches, a multi-conditional feature interaction module is developed in the rendering-generation model. Moreover, two different training schemes are introduced to optimize both the sketch-generation module and the rendering-generation module. In order to evaluate the performance of our proposed models, we built a large-scale dataset which consists of 115,584 pairs of fashion item images. Experimental results demonstrate the effectiveness of our proposed method, and indicate that our model can facilitate designers’ design process by taking full advantage of the controllability of different conditions (e.g., sketch and texture) and the randomness of latent space.

Current point-based trackers are usually implemented by the following two branches: a classification branch for predicting the target candidate locations and a regression branch for regressing the tracking box, which may lead to a spatial misalignment between the two tasks. Meanwhile, they ignore a meaningful exploration on how to define positive and negative samples during training and explicit border information for accurate box prediction. In this research, we investigate the key issues of point-based trackers and unlock their key limitations. First, we design a novel task-aligned component and a new loss function, named task-aligned loss, to learn the alignment of the classification and regression tasks. Second, we introduce a border alignment (BorderAlign) component in both the classification and regression branches to effectively exploit the border features of a tracking target. Third, we develop an adaptive training sample assignment (ATSA) to adaptively divide the positive and negative samples based on the statistical characteristics of the tracking object. Finally, a deformable transformer is developed to enhance the representations of search features and explore rich temporal contexts among video frames. Extensive experimental results demonstrate that the proposed tracker achieves state-of-the-art performance on six tracking benchmark datasets.

How to recommend outfits has gained considerable attention in both academia and industry in recent years. Many studies have been carried out regarding fashion compatibility learning, to determine whether the fashion items in an outfit are compatible or not. These methods mainly focus on evaluating the compatibility of existing outfits and rarely consider applying such knowledge to ‘design’ new fashion items. We propose the new task of generating complementary and compatible fashion items based on an arbitrary number of given fashion items. In particular, given some fashion items that can make up an outfit, the aim of this paper is to synthesize photo-realistic images of other, complementary, fashion items that are compatible with the given ones. To achieve this, we propose an outfit generation framework, referred to as COutfitGAN, which includes a pyramid style extractor, an outfit generator, a UNet-based real/fake discriminator, and a collocation discriminator. To train and evaluate this framework, we collected a large-scale fashion outfit dataset with over 200 K outfits and 800 K fashion items from the Internet. Extensive experiments show that COutfitGAN outperforms other baselines in terms of similarity, authenticity, and compatibility measurements.

In recent years, the identification of out-of-distribution faults has become a hot topic in the field of intelligent diagnosis. Existing researches usually adopt domain adaptation methods to complete the generalization of diagnostic knowledge with the aid of target domain data, but the acquisition of fault samples in real industries is extremely time-consuming and costly. Moreover, most researches focus on samples with fixed fault levels, ignoring the fact that system degradation is a continuous process. In response to the above intractable problems, this article proposed a causal disentanglement network (CDN) to realize cross-machine knowledge generalization and continuous degradation mode diagnosis. In CDN, multitask instance normalization and batch normalization structure was proposed to learn task-specific knowledge and enhance the informativeness of the extracted features. On this basis, a causal disentanglement loss was proposed, which minimized the mutual information of features between subtask structures and captured the causal invariant fault information for better generalization. The experimental results proved the superiority and generalization ability of CDN, and the visualization results proved the performance of CDN in causality mining.

Texture and shape in fashion, constituting essential elements of garments, characterize the body and surface of the fabric and outline the silhouette of clothing, respectively. The selection of texture and shape plays a critical role in the design process, as they largely determine the success of a new design for fashion items. In this research, we propose a texture and shape disentangled generative adversarial network (TSD-GAN) to perform “intelligent” design with the transformation of texture and shape in fashion items. Our TSD-GAN aims to learn how to disentangle the features of texture and shape of different fashion items in an unsupervised manner. Specifically, a fashion attribute encoder is developed to decompose the input fashion items into independent representations of texture and shape. Then, to learn the coarse or fine styles hidden in the features of texture and shape, a texture mapping network and a shape mapping network are proposed to disentangle the features into different hierarchical representations. The different hierarchical representations of texture and shape are then fed into a multi-factor-based generator to generate mixed-style fashion items. In addition, a multi-discriminator framework is developed to distinguish the authenticity and texture similarity between the generated images and the real images. Experimental results on different fashion categories demonstrate that our proposed TSD-GAN may be useful for assisting designers to accomplish the design process by transforming the texture and shape of fashion items.

Post-deduplication delta compression is a data reduction technique that calculates and stores the differences of very similar but non-duplicate chunks in storage systems, which is able to achieve a very high compression ratio. However, the low throughput of widely used resemblance detection approaches (e.g., N-Transform) usually becomes the bottleneck of delta compression systems due to introducing high computational overhead. Generally, this overhead mainly consists of two parts: ① calculating the rolling hash byte by byte across data chunks and ② applying multiple transforms on all of the calculated rolling hash values. In this article, we propose Odess, a fast and lightweight resemblance detection approach, that greatly reduces the computational overhead for resemblance detection while achieving high detection accuracy and a high compression ratio. Odess first utilizes a novel Subwindow-based Parallel Rolling (SWPR) hash method using Single Instruction Multiple Data [ 1 ] (SIMD) to accelerate calculation of rolling hashes (corresponding to the first part of the overhead). Odess then uses a novel Content-Defined Sampling method to generate a much smaller proxy hash set from the whole rolling hash set and quickly applies transforms on this small hash set for resemblance detection (corresponding to the second part of the overhead). Evaluation results show that during the stage of resemblance detection, the Odess approach is ∼31.4× and ∼7.9× faster than the state-of-the-art N-Transform and Finesse (a recent variant of N-Transform [ 39 ]), respectively. When considering an end-to-end data reduction storage system, the Odess-based system’s throughput is about 3.20× and 1.41× higher than the N-Transform- and Finesse-based systems’ throughput, respectively, while maintaining the high compression ratio of N-Transform and achieving ∼1.22× higher compression ratio over Finesse.

Due to the development of complex communication paradigms and the rise in the number of inter-connected digital devices, intrusion detection system (IDS) has become one basic and important security mechanism to identify cyber intrusions and protect computer networks. Currently, various deep learning algorithms have been studied in intrusion detection to achieve a high detection rate, whereas the detection performance may be still dependent on specific datasets. To maintain the detection performance, parameter optimization is believed as an effective solution. Motivated by this observation, in this work, we propose a concise but effective hyperparameter tuning process to enhance the artificial neural network (ANN) based IDS. In the evaluation, we consider three ANN variants and four datasets. The experimental results indicate that our approach can outperform similar studies and typical learning algorithms.

To obtain lower inference latency and less memory footprint of deep neural networks, model quantization has been widely employed in deep model deployment, by converting the floating points to low-precision integers. However, previous methods (such as quantization aware training and post training quantization) require original data for the fine-tuning or calibration of quantized model, which makes them inapplicable to the cases that original data are not accessed due to privacy or security. This gives birth to the data-free quantization method with synthetic data generation. While current data-free quantization methods still suffer from severe performance degradation when quantizing a model into lower bit, caused by the low inter-class separability of semantic features. To this end, we propose a new and effective data-free quantization method termed ClusterQ, which utilizes the feature distribution alignment for synthetic data generation. To obtain high inter-class separability of semantic features, we cluster and align the feature distribution statistics to imitate the distribution of real data, so that the performance degradation is alleviated. Moreover, we incorporate the diversity enhancement to solve class-wise mode collapse. We also employ the exponential moving average to update the centroid of each cluster for further feature distribution improvement. Extensive experiments based on different deep models (e.g., ResNet-18 and MobileNet-V2) over the ImageNet dataset demonstrate that our proposed ClusterQ model obtains state-of-the-art performance.

Convolutional Neural Networks (CNNs) are powerful for image representation, but the convolution operation may be influenced and degraded by the included noise, and the deep features may not be fully learned. In this paper, we propose a new encoder-decoder based image restoration network, termed Robust Low-Rank Convolution Network with Feature Denoising (LRCnet). LRCnet presents a novel low-rank convolution (LR-Conv) for image representation, and a residual dense connection (RDC) for feature fusion between encoding and decoding. Different from directly splitting convolution into ordinary convolution and mirror convolution as existing work, LR-Conv deploys a feature denoising module after the ordinary convolution to remove noise for mirror convolution. A low-rank embedding process is then used to project the convolutional features into a robust low-rank subspace, which can retain the local geometry of input signal to some extent and separate the signal and noise by finding low-rank structure of features to reduce the impact of noise on convolution. Besides, most networks increase the depth of network simply to obtain deep information and lack of effective connections to fuse the multilevel features, which may not fully discover the deep features in various layers. Thus, we design a residual dense connection with a channel attention to connect multilevel feature effectively to obtain more useful information to enhance the data representation. Extensive experiments on several datasets verified the effectiveness of LRCnet for image denoising.

Blind image deblurring is still a challenging problem due to the inherent ill-posed properties. To improve the deblurring performance, many supervised methods have been proposed. However, obtaining labeled samples from a specific distribution (or a domain) is usually expensive, and the data-driven training-based model also cannot be generalized to the blurry images in all domains. These challenges have given birth to certain unsupervised deblurring methods. However, there is a great chromatic aberration between the latent and original images, directly degrading the performance. In this paper, we therefore propose a novel unsupervised color retention network termed CRNet to perform blind motion deblurring. In addition, new concepts of blur offset estimation and adaptive blur correction are proposed to retain the color information when deblurring. As a result, unlike the previous studies, CRNet does not learn a mapping directly from the blurry image to the restored latent image, but from the blurry image to a motion offset. An adaptive blur correction operation is then performed on the blurry image to restore the latent image, thereby retaining the color information of the original image to the greatest extent. To further effectively retain the color information and extract the blur information, we also propose a new module called pyramid global blur feature perception (PGBFP). To quantitatively prove the effectiveness of our network in color retention, we propose a novel chromatic aberration quantization metrics in line with the human perception. Extensive quantitative and visualization experiments show that CRNet can obtain the state-of-the-art performance in unsupervised deblurring tasks.

Deep image inpainting methods have improved the inpainting performance greatly due to the powerful representation ability of deep learning. However, current deep inpainting networks still tend to produce unreasonable structures and blurry textures due to the ill-posed properties of the task, i.e., image inpainting is still a challenging topic. In this paper, we therefore propose a novel deep multi-resolution mutual learning (DMRML) strategy, which can fully explore the information from various resolutions. Specifically, we design a new image inpainting network, termed multi-resolution mutual network (MRM-Net), which takes the damaged images of different resolutions as input, then excavates and exploits the correlation among different resolutions to guide the image inpainting process. Technically, we designs two new modules called multi-resolution information interaction (MRII) and adaptive content enhancement (ACE). MRII aims at discovering the correlation of multiple resolutions and exchanging information, and ACE focuses on enhancing the contents using the interacted features. Note that we also present an memory preservation mechanism (MPM) to prevent from the information loss with the increasing layers. Extensive experiments on Paris Street View, Places2 and CelebA-HQ datasets demonstrate that our proposed MRM-Net can effectively recover the textures and structures, and performs favorably against other state-of-the-art methods.

The performance of spectral clustering heavily relies on the quality of affinity matrix. A variety of affinity-matrix-construction (AMC) methods have been proposed but they have hyperparameters to determine beforehand, which requires strong experience and leads to difficulty in real applications, especially when the inter-cluster similarity is high and/or the dataset is large. In addition, we often need to choose different AMC methods for different datasets, which still depends on experience. To solve these two challenging problems, in this paper, we present a simple yet effective method for automated spectral clustering. First, we propose to find the most reliable affinity matrix via grid search or Bayesian optimization among a set of candidates given by different AMC methods with different hyperparameters, where the reliability is quantified by the \textit{relative-eigen-gap} of graph Laplacian introduced in this paper. Second, we propose a fast and accurate AMC method based on least squares representation and thresholding and prove its effectiveness theoretically. Finally, we provide a large-scale extension for the automated spectral clustering method, of which the time complexity is linear with the number of data points. Extensive experiments of natural image clustering show that our method is more versatile, accurate, and efficient than baseline methods.

Multilabel learning focuses on assigning instances with different labels. In essence, the multilabel learning aims at learning a predictive function from feature space to a label space. The predictive function learning procedure can be regarded as a feature selection procedure and as a classifier construction procedure. For feature selection, we extract features for each label based on the learned positive and negative feature-label correlations. The positive and negative relationships can illustrate which labels can and cannot be well presented by the corresponding features, respectively, due to the semantic gap. For classifier construction, we perform sample-specific and label-specific classifications. The interlabel and interinstance correlations are combined in these two kinds of classifications. These two correlations are learned from both input features and output labels when the output labels are too sparse to reveal the informative correlation. However, there exists the semantic gap when combining input and output spaces to mine the labelwise relationship. The semantic gap can be bridged by the learned feature-label correlation. Finally, extensive experimental results on several benchmarks under four domains are presented to show the effectiveness of the proposed framework.

The current Siamese network based on region proposal network (RPN) has attracted great attention in visual tracking due to its excellent accuracy and high efficiency. However, the design of the RPN involves the selection of the number, scale, and aspect ratios of anchor boxes, which will affect the applicability and convenience of the model. Furthermore, these anchor boxes require complicated calculations, such as calculating their intersection-over-union (IoU) with ground truth bounding boxes. Due to the problems related to anchor boxes, we propose a simple yet effective anchor-free tracker (named Siamese corner networks, SiamCorners), which is end-to-end trained offline on large-scale image pairs. Specifically, we introduce a modified corner pooling layer to convert the bounding box estimate of the target into a pair of corner predictions (the bottom-right and the top-left corners). By tracking a target as a pair of corners, we avoid the need to design the anchor boxes. This will make the entire tracking algorithm more flexible and simple than anchor-based trackers. In our network design, we further introduce a layer-wise feature aggregation strategy that enables the corner pooling module to predict multiple corners for a tracking target in deep networks. We then introduce a new penalty term that is used to select an optimal tracking box in these candidate corners. Finally, SiamCorners achieves experimental results that are comparable to the state-of-art tracker while maintaining a high running speed. In particular, SiamCorners achieves a 53.7% AUC on NFS30 and a 61.4% AUC on UAV123, while still running at 42 frames per second (FPS).

In this article, a chain-structure echo state network (CESN) with stacked subnetwork modules is newly proposed as a new kind of deep recurrent neural network for multivariate time series prediction. Motivated by the philosophy of "divide and conquer," the related input vectors are first divided into clusters, and the final output results of CESN are then integrated by successively learning the predicted values of each clustered variable. Network structure, mathematical model, training mechanism, and stability analysis are, respectively, studied for the proposed CESN. In the training stage, least-squares regression is first used to pretrain the output weights in a module-by-module way, and stochastic local search (SLS) is developed to fine-tune network weights toward global optima. The loss function of CESN can be effectively reduced by SLS. To avoid overfitting, the optimization process is stopped when the validation error starts to increase. Finally, SLS-CESN is evaluated in chaos prediction benchmarks and real applications. Four different examples are given to verify the effectiveness and robustness of CESN and SLS-CESN.

Data deduplication is widely used to reduce the size of backup workloads, but it has the known disadvantage of causing poor data locality, also referred to as the fragmentation problem. This results from the gap between the hyper-dimensional structure of deduplicated data and the sequential nature of many storage devices, and this leads to poor restore and garbage collection (GC) performance. Current research has considered writing duplicates to maintain locality (e.g., rewriting) or caching data in memory or SSD, but fragmentation continues to lower restore and GC performance. Investigating the locality issue, we design a method to flatten the hyper-dimensional structured deduplicated data to a one-dimensional format, which is based on classification of each chunk’s lifecycle, and this creates our proposed data layout. Furthermore, we present a novel management-friendly deduplication framework, called MFDedup, that applies our data layout and maintains locality as much as possible. Specifically, we use two key techniques in MFDedup: Neighbor-duplicate-focus indexing (NDF) and Across-version-aware Reorganization scheme (AVAR). NDF performs duplicate detection against a previous backup, then AVAR rearranges chunks with an offline and iterative algorithm into a compact, sequential layout, which nearly eliminates random I/O during file restores after deduplication. Evaluation results with five backup datasets demonstrate that, compared with state-of-the-art techniques, MFDedup achieves deduplication ratios that are 1.12× to 2.19× higher and restore throughputs that are 1.92× to 10.02× faster due to the improved data layout. While the rearranging stage introduces overheads, it is more than offset by a nearly-zero overhead GC process. Moreover, the NDF index only requires indices for two backup versions, while the traditional index grows with the number of versions retained.

Multiple data reduction techniques have been investigated to lower storage costs for a wide variety of customers. In this work, we focus on similarity-based delta compression, which calculates and stores the difference of very similar, but non-duplicate, chunks in storage systems. Delta compression is often implemented along with deduplication and has been shown to achieve a much higher compression ratio. Currently, the N-Transform method is the most popular and widely-used approach to generate features for data content (e.g. chunks) to detect similar candidates (and then apply delta compression). For delta compression systems, though, the throughput of N-Transform is often the bottleneck. Finesse is a high throughput variant of N-Transform, but it suffers from lower detection accuracy and compression ratio. The computation overhead of N-Transform consists of two parts: calculating the rolling hash across data and applying time-consuming transforms on each hash. In this work, we propose Odess, a fast resemblance detection approach, that uses a novel Content-Defined Sampling method to generate a much smaller proxy hash set and then applies transforms on this small hash set. This reduces the calculations in the transform step from being the bottleneck. Meanwhile, Odess also leverages the faster Gear hash to generate rolling hashes. Thus, Odess greatly reduces the computational overhead for resemblance detection while achieving high detection accuracy and high compression ratio. Our evaluation results show that Odess is ~ 5.4× (Finesse) and ~ 26.9× (N-Transform) faster (on average) at generating features for resemblance detection. When considering an end-to-end data reduction storage system, Odess increases throughput by ~ 1.36× (Finesse) and ~ 2.76× (N-Transform) while maintaining the compression ratio of N-Transform and increasing the compression ratio ~ 1.22× over Finesse.

Deep neural network (DNN) compression can reduce the memory footprint of deep networks effectively, so that the deep model can be deployed on the portable devices. However, most of the existing model compression methods cost lots of time, e.g., vector quantization or pruning, which makes them inept to the real-world applications that need fast online computation. In this paper, we therefore explore how to accelerate the model compression process by reducing the computation cost. Then, we propose a new deep model compression method, termed Dictionary Pair-based Data-Free Fast DNN Compression, which aims at reducing the memory consumption of DNNs without extra training and can greatly improve the compression efficiency. Specifically, our proposed method performs tensor decomposition on the DNN model with a fast dictionary pair learning-based reconstruction approach, which can be deployed on different layers (e.g., convolution and fully-connection layers). Given a pre-trained DNN model, we first divide the parameters (i.e., weights) of each layer into a series of partitions for dictionary pair-based fast reconstruction, which can potentially discover more fine-grained information and provide the possibility for parallel model compression. Then, dictionaries of less memory occupation are learned to reconstruct the weights. Extensive experiments on popular DNNs (i.e., VGG-16, ResNet-18 and ResNet-50) showed that our proposed weight compression method can significantly reduce the memory footprint and speed up the compression process, with less performance loss.

Convolutional Neural Networks (CNNs)-guided deep models have obtained impressive performance for image representation, however the representation ability may still be restricted and usually needs more epochs to make the model converge in training, due to the useful information loss during the convolution and pooling operations. We therefore propose a general feature recovery layer, termed Low-rank Deep Feature Recovery (LDFR), to enhance the representation ability of the convolutional features by seamlessly integrating low-rank recovery into CNNs, which can be easily extended to all existing CNNs-based models. To be specific, to recover the lost information during the convolution operation, LDFR aims at learning the low-rank projections to embed the feature maps onto a low-rank subspace based on some selected informative convolutional feature maps. Such low-rank recovery operation can ensure all convolutional feature maps to be reconstructed easily to recover the underlying subspace with more useful and detailed information discovered, e.g., the strokes of characters or the texture information of clothes can be enhanced after LDFR. In addition, to make the learnt low-rank subspaces more powerful for feature recovery, we design a fusion strategy to obtain a generalized subspace, which averages over all learnt sub-spaces in each LDFR layer, so that the convolutional feature maps in test phase can be recovered effectively via low-rank embedding. Extensive results on several image datasets show that existing CNNs-based models equipped with our LDFR layer can obtain better performance.

Deep subspace clustering (DSC) with the auto-encoder and self-expression layer is of great concern due to encouraging performance. However, existing methods usually adopt a “single-task” strategy based on a single dataset, without considering other related tasks or data. As such, they cannot discover other useful information to improve the clustering task. Besides, the local structure preservation of the latent codes in mapping is usually ignored. In this paper, we therefore present an effective “multi-task” strategy via the self-supervised data augmentation, and propose a new end-to-end trainable Triplet Deep Subspace Clustering Network (TDSC-net). Specifically, TDSC-net firstly generates triplet data (i.e., anchor, positive and negative data) from input data by a spectral clustering module and a self-supervised data augmentation module. This can enable it to inherit the merits of self-supervised learning and multitask learning implicitly. After that, TDSC-net builds a triplet deep autoencoder network with a self-expression layer, which takes the triplet data as input, where they share the common network layers (i.e., autoencoder and self-expression layers) over the triple tasks for complementary learning and mutual supervision. A triplet loss is also included to retain the local information of deep latent codes, which will also benefit the self-expression. Furthermore, TDSC-net separates the self-expression layer from decoding process to improve the efficiency of reconstruction. Extensive results on several public datasets demonstrate the effectiveness of our triplet-task DSC strategy.

Although supervised single image deraining (SID) have obtained impressive results, they still cannot obtain satisfactory results on real images for the weak generalization of rain removal capacity. In this paper, we mainly discuss the semi-supervised SID and propose a new GAN-based deraining network called Semi-DerainGAN, which can use both synthetic and real data in a uniform network based on two supervised and unsupervised processes. For this task, a semi-supervised rain streak learner termed SSRML sharing the same parameters of both processes is derived, which makes the real images contribute more rain streak information, so that the resulted model has a strong generalization power to the real SID task. We also contribute a new real-world rain image dataset called Real200 to alleviate the difference between both synthetic and real image domains. Extensive results on public datasets show that our model can obtain competitive results, especially on the real rain images.

The rapid geographic spread of COVID-19, to which various factors may have contributed, has caused a global health crisis. Recently, the analysis and forecast of the COVID-19 pandemic have attracted worldwide attention. In this work, a large COVID-19 data set consisting of COVID-19 pandemic, COVID-19 testing capacity, economic level, demographic information, and geographic location data in 184 countries and 1241 areas from December 18, 2019, to September 30, 2020, were developed from public reports released by national health authorities and bureau of statistics. We proposed a machine learning model for COVID-19 prediction based on the broad learning system (BLS). Here, we leveraged random forest (RF) to screen out the key features. Then, we combine the bagging strategy and BLS to develop a random-forest-bagging BLS (RF-Bagging-BLS) approach to forecast the trend of the COVID-19 pandemic. In addition, we compared the forecasting results with linear regression (LR) model, [Formula: see text]-nearest neighbors (KNN), decision tree (DT), adaptive boosting (Ada), RF, gradient boosting DT (GBDT), support vector regression (SVR), extra trees (ETs) regressor, CatBoost (CAT), LightGBM (LGB), XGBoost (XGB), and BLS.The RF-Bagging BLS model showed better forecasting performance in terms of relative mean-square error (RMSE), coefficient of determination ([Formula: see text]), adjusted coefficient of determination ([Formula: see text]), median absolute error (MAD), and mean absolute percentage error (MAPE) than other models. Hence, the proposed model demonstrates superior predictive power over other benchmark models.

This paper is concerned with the fault reconstruction observer design problem with unknown nonlinearities, external disturbances and faults. First, a neural-network-based fault estimation approach is developed to generate the estimations of the actuator failures. In this design, the neural network strategy is utilized to approximate the totally unknown nonlinear functions. Then, an iterative adaptive observer is designed to offer the accurate estimations of the sensor faults, where the estimations in the previous iteration are applied in the current iteration to guarantee the convergence of sensor fault estimation errors. The developed neural network observer design approach can reconstruct the states, actuator and sensor faults for the unmanned surface vessel simultaneously. Finally, a practical example of the unmanned surface vessel is presented to illustrate the effectiveness and the potential of the proposed observer technique.

Multilabel classification deals with instances assigned with multiple labels simultaneously. It focuses on learning a mapping from feature space to label a space for out-of-sample extrapolation. The mapping can be seen as a feature selection process in the feature domain or as a classifier training process in the classifier domain. The existing methods do not effectively learn the mapping when combining these two domains together. In this article, we derive a mechanism to extract label-specific features in local and global levels. We also derive a mechanism to train label-specific classifiers in individual and joint levels. Extracting features globally and training classifiers jointly can be seen as a dual process of learning the mapping function on two domains in a coarse-tuned way, while extracting features locally and training classifiers individually can be seen as a dual process of learning the mapping function on two domains in a fine-tuned way. The two-level feature selection and the two-level classifier training are derived to make the entire mapping learning process robust. Finally, extensive experimental results on several benchmarks under four domains are presented to demonstrate the effectiveness of the proposed approach.

The projective dictionary pair learning (DPL) model jointly seeks a synthesis dictionary and an analysis dictionary by extracting the block-diagonal coefficients with an incoherence-constrained analysis dictionary. However, DPL fails to discover the underlying subspaces and salient features at the same time, and it cannot encode the neighborhood information of the embedded coding coefficients, especially adaptively. In addition, although the data can be well reconstructed via the minimization of the reconstruction error, useful distinguishing salient feature information may be lost and incorporated into the noise term. In this article, we propose a novel self-expressive adaptive locality-preserving framework: twin-incoherent self-expressive latent DPL (SLatDPL). To capture the salient features from the samples, SLatDPL minimizes a latent reconstruction error by integrating the coefficient learning and salient feature extraction into a unified model, which can also be used to simultaneously discover the underlying subspaces and salient features. To make the coefficients block diagonal and ensure that the salient features are discriminative, our SLatDPL regularizes them by imposing a twin-incoherence constraint. Moreover, SLatDPL utilizes a self-expressive adaptive weighting strategy that uses normalized block-diagonal coefficients to preserve the locality of the codes and salient features. SLatDPL can use the class-specific reconstruction residual to handle new data directly. Extensive simulations on several public databases demonstrate the satisfactory performance of our SLatDPL compared with related methods.

Recent works on video salient object detection have demonstrated that directly transferring the generalization ability of image-based models to video data without modeling spatial-temporal information remains nontrivial and challenging. Considering both intraframe accuracy and interframe consistency of saliency detection, this article presents a novel cross-attention based encoder-decoder model under the Siamese framework (CASNet) for video salient object detection. A baseline encoder-decoder model trained with Lovász softmax loss function is adopted as a backbone network to guarantee the accuracy of intraframe salient object detection. Self- and cross-attention modules are incorporated into our model in order to preserve the saliency correlation and improve intraframe salient detection consistency. Extensive experimental results obtained by ablation analysis and cross-data set validation demonstrate the effectiveness of our proposed method. Quantitative results indicate that our CASNet model outperforms 19 state-of-the-art image- and video-based methods on six benchmark data sets.

Low-rank coding-based representation learning is powerful for discovering and recovering the subspace structures in data, which has obtained an impressive performance; however, it still cannot obtain deep hidden information due to the essence of single-layer structures. In this article, we investigate the deep low-rank representation of images in a progressive way by presenting a novel strategy that can extend existing single-layer latent low-rank models into multiple layers. Technically, we propose a new progressive Deep Latent Low-Rank Fusion Network (DLRF-Net) to uncover deep features and the clustering structures embedded in latent subspaces. The basic idea of DLRF-Net is to progressively refine the principal and salient features in each layer from previous layers by fusing the clustering and projective subspaces, respectively, which can potentially learn more accurate features and subspaces. To obtain deep hidden information, DLRF-Net inputs shallow features from the last layer into subsequent layers. Then, it aims at recovering the hierarchical information and deeper features by respectively congregating the subspaces in each layer of the network. As such, one can also ensure the representation learning of deeper layers to remove the noise and discover the underlying clean subspaces, which will be verified by simulations. It is noteworthy that the framework of our DLRF-Net is general and is applicable to most existing latent low-rank representation models, i.e., existing latent low-rank models can be easily extended to the multilayer scenario using DLRF-Net. Extensive results on real databases show that our framework can deliver enhanced performance over other related techniques.

The Coronavirus Disease 2019 (COVID-19) began to outbreak since December 2019 and widely spread over the world. How to accurately predict the spread of COVID-19 is one of the essential issues for controlling the pandemic. This study establishes a general model that can predict the trend of COVID-19 in a country based on historical COVID-19 data in 184 countries. First, Savitzky-Golay (S-G) filter is utilized to detect multiple waves of COVID-19 in a country. Then, a PSO-SIR (particle swarm optimization susceptible-infected-recovery) model is provided for data augmentation. Finally, a novel PSO-BLS (particle swarm optimization broad learning system) is proposed for predicting the trend of COVID-19. Experimental results show that compared with the deep learning models (ANN, CNN, LSTM, and GRU), the PSO-BLS algorithm has higher accuracy and stability in predicting the number of active infected cases and removed cases.

With the development of the internet technology, a large amount of softwares have emerged to meet users' increasing needs. At the mean time, software systems have been faced with a problem that they must adapt to the dynamic network environment. It is obvious that a variety of software development models have been proposed in the past few decades. However, the majority of these methods are gradually unadaptable to new circumstances. In this paper, we proposed a new software development model integrating collective intelligence, service and ecosystem. On the one hand, we have introduced the model in detail. On the other hand, We took a practical example to demonstrate the effectiveness of the proposed model.

Recommender systems are currently utilized widely in e-commerce for product recommendations and within content delivery platforms. Previous studies usually use independent features to represent item content. As a result, the relationship hidden among the content features is overlooked. In fact, the reason that an item attracts a user may be attributed to only a few set of features. In addition, these features are often semantically coupled. In this paper, we present an optimization model for extracting the relationship hidden in content features by considering user preferences. The learned feature relationship matrix is then applied to address the cold-start recommendations and content-based recommendations. It could also easily be employed for the visualization of feature relation graphs. Our proposed method was examined on three public datasets: 1) hetrec-movielens-2k-v2; 2) book-crossing; and 3) Netflix. The experimental results demonstrated the effectiveness of our method in comparison to the state-of-the-art recommendation methods.

This paper presents a new framework for matching clothes by considering item in-between compatibility. In contrast to the use of visual features of clothing items, we only utilized their textual descriptions, i.e., title sentences, to constitute the basic features. Specifically, a longshort-term memory (LSTM) network was used for feature embeddings of title sentences. Given item pairs of queries and candidates, their feature embeddings achieved by Siamese LSTMs were integrated into style-compatible space characterized by a compatibility matrix. Our framework is examined on three large-scaled clothing item sets collected from Amazon, Taobao, and Polyvore, respectively. Experiments confirm the efficacy of our approach compared with several baseline methods.

Deep learning is a popular direction in computer vision and digital image processing. It is widely utilized in many fields, such as robot navigation, intelligent video surveillance, industrial inspection, and aerospace. With the extensive use of deep learning techniques, classification and object detection algorithms have been rapidly developed. In recent years, with the introduction of the concept of “unmanned retail,” object detection, and image classification play a central role in unmanned retail applications. However, open-source datasets of traditional classification and object detection have not yet been optimized for application scenarios of unmanned retail. Currently, classification and object detection datasets do not exist that focus on unmanned retail solely. Therefore, in order to promote unmanned retail applications by using deep learning-based classification and object detection, in this article we collected more than 30 000 images of unmanned retail containers using a refrigerator affixed with different cameras under both static and dynamic recognition environments. These images were categorized into ten kinds of beverages. After manual labeling, images in our constructed dataset contained 155 153 instances, each of which was annotated with a bounding box. We performed extensive experiments on this dataset using ten state-of-the-art deep learning-based models. Experimental results indicate great potential of using these deep learning-based models for real-world smart unmanned vending machines.

The choice of which clothes to wear affects how one is perceived, as well as constitutes an expression of one's personal style. Based on the recent advances in image-to-image translation by the conditional generative adversarial network (cGAN), we propose a new framework with a multidiscriminator by incorporating different types of conditional information into the discriminator of cGAN for clothing matches. In contrast with most extant frameworks under cGAN, with one generator and one discriminator, the proposed framework investigates the potential of utilizing conditional information delivered by multidiscriminators to guide the generator. Under this framework, we propose an Attribute-GAN with two discriminators and a category-attribute GAN (CA-GAN) with three discriminators. In order to evaluate the performance of our proposed models, we built a large-scale data set that consists of 19 081 pairs of collocation clothing images with 90 manually labeled attributes. Experimental results demonstrate that with supervision of the additional attribute discriminator or category discriminator, the quality of the generated clothing images by GANs is consistently improved in comparison with the state-of-the-art methods.

Scientific simulations in high-performance computing (HPC) environments generate vast volume of data, which may cause a severe I/O bottleneck at runtime and a huge burden on storage space for postanalysis. Unlike traditional data reduction schemes such as deduplication or lossless compression, not only can error-controlled lossy compression significantly reduce the data size but it also holds the promise to satisfy user demand on error control. Pointwise relative error bounds (i.e., compression errors depends on the data values) are widely used by many scientific applications with lossy compression since error control can adapt to the error bound in the dataset automatically. Pointwise relative-error-bounded compression is complicated and time consuming. We develop efficient precomputation-based mechanisms based on the SZ lossy compression framework. Our mechanisms can avoid costly logarithmic transformation and identify quantization factor values via a fast table lookup, greatly accelerating the relative-error-bounded compression with excellent compression ratios. In addition, we reduce traversing operations for Huffman decoding, significantly accelerating the decompression process in SZ. Experiments with eight well-known real-world scientific simulation datasets show that our solution can improve the compression and decompression rates (i.e., the speed) by about 40 and 80 p, respectively, in most of cases, making our designed lossy compression strategy the best-in-class solution in most cases.

Single image deraining task is still a very challenging task due to its ill-posed nature in reality. Recently, researchers have tried to fix this issue by training the CNN-based end-to-end models, but they still cannot extract the negative rain streaks from rainy images precisely, which usually leads to an over de-rained or under de-rained result. To handle this issue, this paper proposes a new coarse-to-fine single image deraining framework termed Multi-stream Hybrid Deraining Network (shortly, MH-DerainNet). To obtain the negative rain streaks during training process more accurately, we present a new module named dual path residual dense block, i.e., Residual path and Dense path. The Residual path is used to reuse com-mon features from the previous layers while the Dense path can explore new features. In addition, to concatenate different scaled features, we also apply the idea of multi-stream with shortcuts between cascaded dual path residual dense block based streams. To obtain more distinct derained images, we combine the SSIM loss and perceptual loss to preserve the per-pixel similarity as well as preserving the global structures so that the deraining result is more accurate. Extensive experi-ments on both synthetic and real rainy images demonstrate that our MH-DerainNet can deliver significant improvements over several recent state-of-the-art methods.

Road extraction from aerial images is a hot research topic. With the advancement of convolutional neural network (CNN), several CNN-based road detection methods have been developed. However, most of them do not make full use of the hierarchical features from the original aerial images. In this paper, we propose a novel residual dense U-Net (RDUN), a semantic segmentation network which combines the strengths of residual learning, DenseNet, and U-Net, to overcome the drawback. Our proposed RDUN can fully exploit the hierarchical features from all the convolutional layers, which utilizes the residual dense blocks (RDB) to build up a U-Net architecture. The benefits of our model are two-fold. First, by using the RDB abundant local features can be extracted and fused effectively. Second, based the local features, hierarchical features are constructed by shortcut connections between layers in RDB. Extensive experiments are carried out on a real-world road detection dataset and the results demonstrate the proposed RDUN outperforms state-of-the-art competitors.

Supervised and semi-supervised sequence labeling methods require large amounts of fully annotated training sequences or exact annotations of structured outputs. The problem of learning from partially annotated sequences arises in many applications, for example, Natural Language Processing and Computational Biology. In this paper, we propose Piecewise Convex Learning from Partial Labels (PW-CLPL) which is an effective discriminative structured learning method for sequence labeling as global training is intractable for partially annotated sequences. A small number of constraints is reformulated for the optimization, which improve the efficiency of parameters learning. Experimental results on the reconstructed dataset CoNLL-2000 show the effectiveness of the proposed model in the setting of partial annotations.

In recent years, deep learning techniques have been commonly used in the fields of image processing and computer vision. With the popularity of deep learning models, researchers have developed many effective object detection methods. Unmanned retail applications start to utilizing object detection algorithms for changing traditional retail modes. Until now, there is no public datasets for object detection in unmanned retail application environments. Moreover, state-of-the-art deep learning-based object detection models have not yet been examined in this application scenario. In this paper, we compiled a large-scale dataset which contains over 30,000 images captured in a refrigerator equipped with different cameras. 10 kinds of beverages were utilized for targeted objects. An empirical study on this dataset is performed by using several recent developed deep learning models. Results demonstrate the effectiveness of using deep learning techniques real-life unmanned retail environments.

Deep learning-based methods are continuously boosting the performance of detecting objects in natural images. On the contrary, detecting objects in Unmanned Aerial Vehicle (UAV) images remains to be a difficult task in the field of computer vision, due to the challenge of training a well-performed detection model working on UAV images which usually contain instances with varied orientations, scales, and contours, etc. Furthermore, only a few researchers have focused on this field, probably because of difficulties in UAV data acquisition and labelling. Inspired by this, we collected a large-scale dataset with multi-scale and high-resolution UAV images, named MOHR, which contains 10,631 images captured by a UAV affixed with three kinds of cameras. Since these images were captured in a suburban environment, we manually annotated five classes of objects, including car, truck, building, collapse and flood damage. An empirical study was then conducted by adopting six advanced object detection methods all of which are based on deep learning technologies. The results indicate the great potential of these evaluated object detection models, but also reveal that the research on such a challenging UAV dataset using current deep learning techniques is far reaching.

Temporal information services are critical in implementing emerging intelligent transportation systems. Nevertheless, it is challenging to realize timely temporal data update and dissemination due to an intermittent wireless connection and a limited communication bandwidth in dynamic vehicular networks. Some previous studies have considered the temporal data dissemination in vehicular networks, but they are limited to the service region, which is inside the coverage of roadside units. To enhance system scalability, it is imperative to exploit the synergic effect of vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) communications for providing efficient temporal information services in such an environment. With the above motivations, we propose a novel system architecture to enable efficient data scheduling in hybrid V2I/V2V communications by having the global knowledge of network resources of the system. On this basis, we formulate a temporal data upload and dissemination (TDUD) problem, aiming at optimizing two conflict objectives simultaneously, which are enhancing the data quality and improving the delivery ratio. Furthermore, we propose an evolutionary multi-objective algorithm calledMO-TDUD, which consists of a decomposition scheme for handling multiple objectives, a scalable chromosome representation forTDUDsolution encoding, and an evolutionary operator designed forTDUDsolution reproduction. The proposedMO-TDUDcan be adaptive to different requirements on data quality and delivery ratio by selecting the best solution from the derived Pareto solutions. Last but not least, we build the simulation model and implementMO-TDUDfor performance evaluation. The comprehensive simulation results demonstrate the superiority of the proposed solution.

This paper introduces a simple, yet effective, framework for clothes collocation by considering compatibility between items. In particular, we treat title sentences as the features of clothing items, instead of using clothing images. For feature transformation, the long-short term memory (LSTM) network is utilized for mapping title sentences into a low-dimensional space. Features of query and candidate items learned by the Siamese LSTMs are synthesized into a style space by a compatibility matrix. We evaluate our framework on two large-scale datasets compiled from Amazon and Taobao, respectively. Extensive experimental results show the effectiveness of our method in comparison to several state-of-the-art methods.

Books, as a representative of lengthy documents, convey rich semantics. Traditional document modeling methods, such as bag-of-words models, have difficulty capturing such rich semantics when only considering term-frequency features. In order to explore term spatial distributions over a book, a tree-structured book representation is investigated in this paper. Moreover, an efficient learning framework, Tree2Vector, is introduced for mapping tree-structured book data into vectorial space. In particular, we present two types of locality reconstruction (LR) models: Euclidean-type and cosine-type, during the transformation process of tree structures into vectorial representations. The LR is used for modeling the reconstruction process, in which each parent node in a tree is supposed to be reconstructed by its child nodes. The prominent advantage of this Tree2Vector framework is that it solely utilizes the local information within a single book tree. In addition, extensive experimental results demonstrate that Tree2Vector is able to deliver comparable or better performance in comparison to methods that consider the information of all trees in a database globally. Experimental results also suggest that cosine-type LR consistently performs better than Euclidean-type LR in applications of book and author recommendations.

The tree structure is one of the most powerful structures for data organization. An efficient learning framework for transforming tree-structured data into vectorial representations is presented. First, in attempting to uncover the global discriminative information of child nodes hidden at the same level of all of the trees, a clustering technique can be adopted for allocating children into different clusters, which are used to formulate the components of a vector. Moreover, a locality-sensitive reconstruction method is introduced to model a process, in which each parent node is assumed to be reconstructed by its children. The resulting reconstruction coefficients are reversely transformed into complementary coefficients, which are utilized for locally weighting the components of the vector. A new vector is formulated by concatenating the original parent node vector and the learned vector from its children. This new vector for each parent node is inputted into the learning process of formulating vectorial representation at the upper level of the tree. This recursive process concludes when a vectorial representation is achieved for the entire tree. Our method is examined in two applications: book author recommendations and content-based image retrieval. Extensive experimental results demonstrate the effectiveness of the proposed method for transforming tree-structured data into vectors.

In this paper, we propose a novel fault detection method for multivariate industrial processes. The method is based on AutoEncoder. AutoEncoder is a single-hidden-layer neural network that can learn low-dimensional nonlinear representations for high-dimensional data. In the proposed fault detection method, offline normal data are used to train an AutoEncoder, which is then used for online fault detection. The proposed method is compared with conventional methods on Tennessee Eastman process. The experimental results show that the proposed method is able to outperform other methods.

Dramatic growth of e-book sales revenue in recent years makes book recommendations essential to readers. Traditional bag-of-words models have difficulty of capturing the spatial information of terms over books. In this paper, a three-layer tree structure is used for representing each book. A framework, Tree2Vector, is designed for transforming tree-based book data into vectorial space. First, in order to characterize the global discriminative information of child nodes conveyed at the same level of all the trees, a clustering technique is used for assigning child nodes into different clusters, which are adopted for formulating the components of a vector. Furthermore, a locality reconstruction (LR) method is designed to model the reconstruction process, where each parent node is supposed to be reconstructed by its child nodes. The derived reconstruction coefficients are used for locally weighting the components of the vector. The process is repeated level-by-level until a vectorial representation is accomplished for a book tree. Our method is examined in content-based book recommendation. Experimental results exhibit the effectiveness of our framework.

In this paper, we present new models and algorithms for object-level video advertising. A framework that aims to embed content-relevant ads within a video stream is investigated in this context. First, a comprehensive optimization model is designed to minimize intrusiveness to viewers when ads are inserted in a video. For human clothing advertising, we design a deep convolutional neural network using face features to recognize human genders in a video stream. Human parts alignment is then implemented to extract human part features that are used for clothing retrieval. Second, we develop a heuristic algorithm to solve the proposed optimization problem. For comparison, we also employ the genetic algorithm to find solutions approaching the global optimum. Our novel framework is examined in various types of videos. Experimental results demonstrate the effectiveness of the proposed method for object-level video advertising.

This paper presents a character-level seque-nce-to-sequence learning method, RNNembed. This method allows the system to read raw characters, instead of words generated by preprocessing steps, into a pure single neural network model under an end-to-end framework. Specifically, we embed a recurrent neural network into an encoder–decoder framework and generate character-level sequence representation as input. The dimension of input feature space can be significantly reduced as well as avoiding the need to handle unknown or rare words in sequences. In the language model, we improve the basic structure of a gated recurrent unit by adding an output gate, which is used for filtering out unimportant information involved in the attention scheme of the alignment model. Our proposed method was examined in a large-scale dataset on an English-to-Chinese translation task. Experimental results demonstrate that the proposed approach achieves a translation performance comparable, or close, to conventional word-based and phrase-based systems.

In this paper, we present a new method for human object-level video advertising. A framework that aims to embed content-relevant ads within a video stream is investigated in this context. In particular, to support content-relevant advertising, we employ the discriminatively trained part based model to detect human objects in a video and then select the ads that are related to the detected human objects. For human clothing advertising, we design a deep Convolutional Neural Network (CNN) using face features to recognize human genders in a video stream. Human parts alignment is then implemented to extract human part features that are used for clothes retrieval. Our novel framework is examined in various types of videos. Experimental results demonstrate the effectiveness of the proposed method for human object-level video advertising.

Vehicle type recognition has many applications in video surveillance, urban traffic management and automatic driving. This paper presents a new vehicle type recognition method using feature encoding combined with Convolutional Neural Network (CNN). This method uses the CNN to learn the properties of the high-level image features. It is able to largely compensate the information loss if we use feature encoding solely. By contrast, to achieve satisfactory classification results, feature encoding algorithms do not need a large number of training samples. Thus, it can help CNN reduce the number of training samples. Therefore, we propose a hybrid algorithm by integrating method on vehicle type recognition in comparison to CNN, feature encoding algorithms and other competitive methods.

This paper presents a character-level sequence-to-sequence learning method, RNNembed. Specifically, we embed a Recurrent Neural Network (RNN) into an encoder-decoder framework and generate character-level sequence representation as input. The dimension of input feature space can be significantly reduced as well as avoiding the need to handle unknown or rare words in sequences. In the language model, we improve the basic structure of a Gated Recurrent Unit (GRU) by adding an output gate, which is used for filtering out unimportant information involved in the attention scheme of the alignment model. Our proposed method was examined in a large-scale dataset on a task of English-to-Chinese translation. Experimental results demonstrate that the proposed approach achieves a translation performance comparable, or close, to conventional word-based and phrase-based systems.

In this paper, we present a new framework for online video advertising by targeting clothes of principal actors. This framework aims to embed both character-relevant and content-relevant advertisements within a video stream. We identify the main actors automatically in a given video by using face clustering techniques. Specifically, we propose a new clustering algorithm based on face verification. Then we can find the main characters by counting the number of faces of each class. The video frames that include main characters are the best options to advertising. We segment the image patches of main characters from these video frames by object detection, and carry out different strategies for different sizes of images to match clothes ads. Experimental results demonstrate the effectiveness of our proposed method.

A new triple wing harmonium (TWH) model that integrates text metadata into a low-dimensional semantic space is proposed for the application of content-based movie recommendation. The text metadata considered here include movie synopsis, actor list, and user comments. We develop a new TWH model projecting these multiple textual features into low-dimensional latent topics with different probability distribution assumptions. A contrastive divergence (CD) algorithm is used for efficient learning and inference. Experimental results suggest that the proposed method performs better than the state-of-the-art algorithms for movie recommendation.

This paper introduces a new framework for the organization of electronic books (e-books) and their corresponding authors using a multilayer self-organizing map (MLSOM). An author is modeled by a rich tree-structured representation, and an MLSOM-based system is used as an efficient solution to the organizational problem of structured data. The tree-structured representation formulates author features in a hierarchy of author biography, books, pages, and paragraphs. To efficiently tackle the tree-structured representation, we used an MLSOM algorithm that serves as a clustering technique to handle e-books and their corresponding authors. A book and author recommender system is then implemented using the proposed framework. The effectiveness of our approach was examined in a large-scale data set containing 3868 authors along with the 10500 e-books that they wrote. We also provided visualization results of MLSOM for revealing the relevance patterns hidden from presented author clusters. The experimental results corroborate that the proposed method outperforms other content-based models (e.g., rate adapting poisson, latent Dirichlet allocation, probabilistic latent semantic indexing, and so on) and offers a promising solution to book recommendation, author recommendation, and visualization.

This paper introduces a new framework for author recommending using Multi-Layer Self-Organizing Map (ML-SOM). Concretely, an author is modeled by a tree-structured representation, and an MLSOM-based system is used as an efficient solution to the content-based author recommending problem. The tree-structured representation formulates author features in a hierarchy of author biography, written books and book comments. To efficiently tackle the tree-structured representation, we use an MLSOM algorithm that serves as a clustering technique to handle authors. The effectiveness of our approach was examined in a large-scale dataset containing 7426 authors, 205805 books they wrote, and 3027502 comments that readers have provided. The experimental results corroborate that the proposed approach outperforms current algorithms and can provide a promising solution to author recommendation.

A content-based movie recommender by using a Triple Wing Harmonium (TWH) model is proposed. TWH integrates text metadata into a low dimensional semantic space. movie synopsis, actor list and user comments are considered as the text metadata. A new TWH model is developed by projecting these multiple textual features into low dimensional latent topics. We have used a contrastive divergence algorithm for efficient learning and inference. Experimental results show that the proposed method performs better than the state-of-the-art content-based algorithms for movie recommendation.

In the Web2.0 era, user generated content has become the main source of information of many popular websites such as Flickr. In Flickr, each user can share his/her photos and browse others' easily. Tagging system is an important approach to the photo management in Flickr. Users can browse photos by clicking their attached tags. However, many photos have very few or even no tags, because only the up loader can mark tags for the photo. Meanwhile, when a user browses the photo he/she is interested in, he/she may have comments to express his/her independent viewpoint on the photo. Therefore, it is critical to recommend new tags or enrich the existing tag set based on user comments. Relying on Natural Language Processing (NLP) techniques, this paper introduces a word-based method in generating candidate tags extracted from user comments. In the phase of sorting and recommending tags, we propose an algorithm by jointly modeling the location information of candidate tags, statistical information and semantic similarity. Extensive experimental results demonstrate the effectiveness of our method.

Multilabel learning aims to predict labels of unseen instances by learning from training samples that are associated with a set of known labels. In this paper, we propose to use a hierarchical tree model for multilabel learning, and to develop the ML-Tree algorithm for finding the tree structure. ML-Tree considers a tree as a hierarchy of data and constructs the tree using the induction of one-against-all SVM classifiers at each node to recursively partition the data into child nodes. For each node, we define a predictive label vector to represent the predictive label transmission in the tree model for multilabel prediction and automatic discovery of the label relationships. If two labels co-occur frequently as predictive labels at leaf nodes, these labels are supposed to be relevant. The amount of predictive label co-occurrence provides an estimation of the label relationships. We examine the ML-Tree method on 11 real data sets of different domains and compare it with six well-established multilabel learning algorithms. The performances of these approaches are evaluated by 16 commonly used measures. We also conduct Friedman and Nemenyi tests to assess the statistical significance of the differences in performance. Experimental results demonstrate the effectiveness of our method.

This research investigates the potential of document semantic representation considering both term frequencies and term associations. In particular, we proposed a general framework of the use of term spectra to represent term spatial distributions and associations through a document. The term spectra we explored involved the use of three typical techniques: Discrete Cosine Transform (DCT), Discrete Fourier Transform (DFT), and Discrete Wavelet Transform (DWT). A term affinity graph was established to represent each document. We then employed a new document analysis method (recently developed by authors), named Multi-Dimensional Latent Semantic Analysis (MDLSA), which enables us to formulate an efficient semantic representation of a document based on the term affinity graph. Our algorithm was examined in the application of Web document classification. Experimental results demonstrate that the proposed technique not only gains much computational efficiency compared to Direct Graph Matching (DGM), but also outperforms the state-of-art algorithms such as VSM, PCA, RAP, and MLM.

Kmeans-type clustering aims at partitioning a data set into clusters such that the objects in a cluster are compact and the objects in different clusters are well separated. However, most kmeans-type clustering algorithms rely on only intracluster compactness while overlooking intercluster separation. In this paper, a series of new clustering algorithms by extending the existing kmeans-type algorithms is proposed by integrating both intracluster compactness and intercluster separation. First, a set of new objective functions for clustering is developed. Based on these objective functions, the corresponding updating rules for the algorithms are then derived analytically. The properties and performances of these algorithms are investigated on several synthetic and real-life data sets. Experimental studies demonstrate that our proposed algorithms outperform the state-of-the-art kmeans-type clustering algorithms with respect to four metrics: accuracy, RandIndex, Fscore, and normal mutual information.

In this paper, we consider the problem of in-depth document analysis. In particular, we propose a novel document analysis method, named multidimensional latent semantic analysis (MDLSA), which enables us to mine local information efficiently from a document with respect to term associations and spatial distributions. MDLSA works by first partitioning each document into paragraphs and building a term affinity graph, which represents the frequency of term cooccurrence in a paragraph. We then conduct a 2-D principal component analysis to achieve an optimal semantic mapping. This analysis involves finding the leading eigenvectors of the sample covariance matrix of a training set to characterize the lower dimensional semantic space. A hybrid document similarity measure is designed to further improve the performance of this framework. Our algorithm is examined in two document applications: retrieval and classification. Experimental results demonstrate that the proposed technique outperforms current algorithms with respect to accuracy and computational efficiency.

A novel framework using a Bayesian approach for content-based phishing web page detection is presented. Our model takes into account textual and visual contents to measure the similarity between the protected web page and suspicious web pages. A text classifier, an image classifier, and an algorithm fusing the results from classifiers are introduced. An outstanding feature of this paper is the exploration of a Bayesian model to estimate the matching threshold. This is required in the classifier for determining the class of the web page and identifying whether the web page is phishing or not. In the text classifier, the naive Bayes rule is used to calculate the probability that a web page is phishing. In the image classifier, the earth mover's distance is employed to measure the visual similarity, and our Bayesian model is designed to determine the threshold. In the data fusion algorithm, the Bayes theory is used to synthesize the classification results from textual and visual content. The effectiveness of our proposed approach was examined in a large-scale dataset collected from real phishing cases. Experimental results demonstrated that the text classifier and the image classifier we designed deliver promising results, the fusion algorithm outperforms either of the individual classifiers, and our model can be adapted to different phishing cases.