Detecting rumors on social media is challenging when posts are semantically underspecified and discussion threads are noisy or polarized, which can encourage detectors to exploit spurious correlations. We propose CLEAR (Contextual Potential Alignment Capture Network), an evidence-grounded framework that models hierarchical comment dynamics and incorporates auxiliary LLM-based veracity assessments for credibility-aware prediction. CLEAR couples prototype-conditioned flow purification with Dirichlet evidential learning to derive geometry-grounded evidence for calibrated inference. We further introduce an entropy-adaptive Hard-Shift reweighting strategy to suppress noise-driven shortcuts. Experiments on Weibo-19 (2927 samples) and PHEME (2018 samples) show that CLEAR achieves 93.16% and 91.56% accuracy, outperforming the average strong recent baselines by 3.2 and 5.5 percentage points, respectively. To stress-test generalization under distribution shift, we curate VRDD with 4020 posts (2348 non-rumors and 1672 rumors), a boundary-dense benchmark that emphasizes vague content. Results confirm CLEAR’s robustness to evolving rumor patterns and highlight the curriculum-dependent effect of reweighting.

Fake news detection is a critical challenge in the digital age, as the fabricated content rapidly undermines public trust across diverse domains. Existing methods rely on domain-agnostic feature extractors, which struggle to decouple heterogeneous features and suffer from semantic entanglement, reducing cross-domain generalization. Moreover, static architectures with fixed knowledge representations cannot adapt to evolving misinformation patterns during events such as public health crises and political upheavals. Conventional multi-domain frameworks also incur high computational costs because of redundant parameter scaling, hindering near real-time deployment. To address these issues, herein, we propose a D ual-layer H eterogeneous E xpert network and dynamic M emory augmentation for multi-domain F ake N ews D etection (DHEM-FND). First, a heterogeneous expert network disentangles domain-specific semantics through three pathways: (i) a multiscale convolutional neural network capturing local–global patterns, (ii) emotion-aware multilayer perceptron detecting sentiment–context discrepancies, (iii) and self-attention module with statistical priors identifying stylistic fingerprints. Orthogonal constraints minimize feature leakage across domains. Second, a dynamic memory augmentation mechanism tracks evolving fake news tactics by maintaining hierarchical domain prototypes, incrementally updated using attention-based alignment. Third, sparse gating and parameter reuse strategies optimize efficiency by activating only the top-k relevant experts during inference while freezing 75% of backbone parameters. We validated the DHEM model using four real-world datasets: Twitter15, Twitter16, Weibo21, and FineFake. We achieved an average detection accuracy of 92.57%, while reducing cross-domain interference by 41.2%, improving detection speed by 17.2%. Thus, the superior performance of the proposed method in dealing with evolving fake news and scalability is demonstrated.

No-reference image quality assessment (NR-IQA) aims to predict perceptual image quality without access to pristine references, which remains challenging due to diverse and complex distortions. Recent pseudo-reference-based methods attempt to mitigate this challenge but often rely on highfidelity pseudo-reference reconstruction. In contrast, this work shows that improving NR-IQA performance does not depend on reconstruction quality, but on effective representation learning, feature alignment, and deviation modeling between distorted images and pseudo references. To this end, we propose a novel NR-IQA framework that leverages low-quality pseudo references generated by a masked autoencoder with a lightweight decoder. Rather than pursuing detailed reconstruction, the pseudo reference is used to facilitate representation-level deviation modeling in a shared latent space via a cross-attention-based mechanism. Extensive experiments on multiple benchmark datasets demonstrate that the proposed method consistently outperforms state-of-the-art NR-IQA approaches while maintaining modest computational complexity. Our source code will be available at: https://github.com/jianjin008/L-IQA.

Image-text matching as a fundamental cross-modal understanding task presents unique challenges in weakly-aligned scenarios. Such data typically feature highly abstract textual captions with sparse entity references, creating a significant semantic gap with visual content. Current mainstream methods, primarily designed for strongly aligned data pairs, employ dynamic modeling or multi-dimensional similarity computation to achieve feature space mapping. However, they struggle with information asymmetry and modal heterogeneity in weakly aligned cases. To address this, we propose a Visual Perception Knowledge Enhancement (VPKE) framework. Unlike existing methods based on strong alignment assumptions, this framework mines latent image semantics through vision-language models and generates auxiliary captions, overcoming the information bottleneck of traditional text modalities. Its core innovation lies in an adaptive knowledge distillation mechanism that combines retrieval-augmented generation (RAG) with key entity extraction. This mechanism effectively filters noise when introducing external knowledge while optimizing cross-modal feature integration. The framework employs multi-level similarity evaluation to dynamically adjust fusion weights among original text, key entities, and auxiliary captions, enabling adaptive integration of diverse semantic features and significantly improving model flexibility. Additionally, multi-scale feature extraction further enhances cross-modal representation capabilities. Experimental results show that the proposed method performs excellently in image-text retrieval tasks on the MSCOCO and Flickr30K datasets, validating its effectiveness.

The Vision-language navigation task requires agents to efficiently interpret visual cues in the environment and accurately follow long-range instructions, posing significant challenges to their scene memory and spatial reasoning capabilities. Existing methods typically construct memory systems directly from raw visual observations. However, task-irrelevant cues commonly present in the environment can continuously introduce localization errors during navigation, severely limiting the agent’s performance in complex scenes. Meanwhile, due to the lack of transferable general knowledge priors, existing agents exhibit notable limitations in spatial perception, which undermines the reliability of their decision-making in unseen environments. To address these issues, this paper proposes the dynamic object filtering with Spatial Perception Enhancement for Vision-Language Navigation (SPENav), which aggregates open-vocabulary perception with multi-level information modeling. At the local level, the Hierarchical Semantic Prior Extractor and Room-Information-Guided Filtering construct task-oriented semantic priors to capture critical objects and suppress irrelevant features. At the global level, the Spatial-Instructional Guided Dual Attention module leverages spatial information and instruction guidance to enable the agent to develop selective memory that is goal- and task-oriented. On the unseen test split of R2R, SPENav achieves a 76% Success Rate (SR) and a 65% Success weighted by Path Length (SPL). These results demonstrate the effectiveness of task-oriented feature selection and multi-level semantic modeling in enhancing cross-modal understanding and adaptive navigation performance.

In large-scale Point-of-Interest (POI) recommendation, the conflict between accuracy and computational efficiency intensifies as POI catalogs grow. Traditional deep models struggle to balance quality with efficiency. To address this challenge, we propose a knowledge-distilled product quantization framework EffiPOI for efficient POI recommendation. EffiPOI jointly optimizes accuracy and efficiency by integrating product quantization with multi-modal knowledge distillation. Specifically, we first construct service-oriented multi-modal POI representations, which comprehensively capture each POI’s spatial coverage, temporal activity patterns, and semantic attributes. Based on these representations, we design a teacher-student distillation paradigm. The teacher model adopts a Mixture-of-Experts architecture to generate discriminative and semantically expressive POI representations, which serve as high-quality supervision signals for guiding the student model through knowledge distillation. The student model leverages product quantization to encode POIs into compact and computation-friendly representations, achieving a favorable tradeoff between representational compactness and predictive accuracy. To alleviate the performance degradation due to quantization, we develop a hybrid knowledge distillation strategy that transfers both response-aware and feature-aware knowledge from the teacher model to the student model. Experimental results on three real-world datasets show that the proposed method achieves 4.6%–12.1% improvements in accuracy and over 10× speedup in inference efficiency, outperforming existing POI recommendation models. Code is available at: https://github.com/pcm1217/EffiPOI .

Sequential recommendation systems are often misled by large-scale traffic data containing misinformation, which can trigger impulsive buying and reduce prediction accuracy. However, most existing methods either focus solely on long-term user preferences or assume a smooth evolution of user intentions, resulting in inadequate modeling and poor performance. This paper proposes an in-depth propaganda strategy by proposing an impulsive buying-aware model based on users’ long-term and short-term preference representation for sequential recommendation (INSPEQ), aiming to mitigate misinformation effects and enhance recommendation quality. INSPEQ first distinguishes long- and short-term preferences by extracting item attributes embedded in both intrinsic and extrinsic knowledge through relation paths. For long-term behavior modeling, a PATR-GRU network with apGRUs is used to learn stable user preferences by leveraging persistent item attributes and encoding temporal dependencies between adjacent and nonadjacent items along knowledge sequences, capturing both direct and indirect temporal influences. To capture short-term user preferences, a modified self-attention mechanism is introduced, enhanced with time-aware positional encoding, enabling the model to reflect recent shifts in user behavior more effectively within dynamic sessions. These dual representations are then adaptively fused via an MLP-based gated mechanism, assigning dynamic weights based on user impulsivity levels to flexibly balance stability and recency in decision-making. Extensive experiments on three real-world datasets demonstrate that INSPEQ consistently outperforms nineteen state-of-the-art methods. Specifically, it achieves up to +10.1 % in nDCG@5 and +11.4 % in HitRatio@5 over the strongest baselines, highlighting the effectiveness of jointly modeling preference dynamics while alleviating misinformation effects in practical recommendation environments.

The fast dissemination of online information has facilitated the evolution of multimodal fake news, thereby rendering the trustworthiness of its content difficult to identify. Some existing studies, although capturing the consistency and inconsistency features between different modalities, neglect to dynamically balance these two types of features based on their contributions during the features fusion. Thus, we propose a S emantic R elationship-based C onsistency and I nconsistency B alancing N etwork for multimodal fake news detection (SR-CIBN). Specifically, the global features are thoroughly investigated by hierarchically penetrating and interacting between multimodal features that are aligned by contrastive learning at both the intra- and inter-modal views. Then, the global consistency and inconsistency features are obtained through the interaction between the selected key image patches features and the global features. Additionally, the fusion intensity of the global consistency and inconsistency features is adjusted based on the image–text matching degree, resulting in the final optimized features. Under the joint learning framework we proposed, confusion between semantically similar real and fake news is effectively avoided by training with triplet loss based on the image–text semantic relationship. Our model surpasses comparable approaches, as shown by comprehensive experiments on the Twitter and Weibo datasets.

In real-world surveillance scenarios, person re-identification tasks are often seriously affected by occlusion problems, which requires the model to be able to not only extract powerful features, but also effectively recover features when they are occluded. Although existing methods disentangle visible human bodies by clustering semantic information, they often damage discriminative appearance due to the introduction of background noises. To solve this problem, we propose Heterogeneous Generative Tokens and Distance-aware Recovery (HGTDR) network, which aims to effectively extract discriminative appearance and recover the occluded body regions. HGTDR mainly contains two branches: a holistic stream and a part stream. The holistic stream utilizes ViT to capture the global context information and provide stable global features by establishing long-range relationships. In the part stream, we propose a Semantic Patch Generator (SPG), which combines the local attention mechanism to capture rich local semantics and further generate semantic patches. Further, considering the discrimination score and relevance score of semantic patches, we feed them into the proposed Adaptive Heterogeneous Semantic Token Generator (AHSTG) to gradually generate strong-response foreground and weak-response background features. In addition, to complete the features of occluded regions, the Distance-based Feature Recovery (DFR) module is designed. The module calculates the planar Euclidean distance of heterogeneous tokens and adaptively allocates the corresponding weights to dynamically recover the invisible bodies. Finally, we obtain discriminative and robust person descriptors. Extensive experiments on several challenging occluded, partial and holistic Re-ID datasets demonstrate that our proposed HGTDR network achieves superior performance and outperforms various state-of-the-art methods.

Due to its excellent query and storage efficiency to facilitate large-scale multimedia retrieval, multi-modal hashing (MMH) has garnered a lot of attention from researchers. Nevertheless, existing MMH methods still suffer from several challenges: 1) Existing MMH methods often rely on graphs to represent complex correlation, but are constrained by the quality of graph construction and the storage overhead. 2) Existing MMH methods only deal with complete multi-modal data where all modalities of each instance are available, but cannot work with incomplete multi-modal data which encounter the problem of missing modalities. 3) Existing MMH methods often ignore the inevitable weak-supervision issue. To address these challenges, this paper proposes an Incomplete Multi-modal wEakly-supervised Hashing with Consensus Bipartite Graph (IMEH-CBG) method, which learns consensus bipartite graph for incomplete multi-modal fusion and corrects weak labels for discriminant hash learning. As far as we know, this is the first MMH method to work with incomplete and weakly-supervised multi-modal data in an unified framework. IMEH-CBG selects unified anchor set and builds consensus bipartite graph jointly for incomplete multi-modal fusion to tackle the first and the second challenges. Then, the semantic labels are predicted and utilized to learn hash code in an asymmetric way to tackle the third challenge. Extensive experiments demonstrate the superiority of IMEH-CBG.

Deep hashing algorithms have demonstrated considerable success in recent years, particularly in cross-modal retrieval tasks. Although hash-based cross-modal retrieval methods have demonstrated considerable efficacy, the vulnerability of deep networks to adversarial examples represents a significant challenge for the hash retrieval. In the absence of target semantics, previous non-targeted attack methods attempt to attack depth models by adding disturbance to the input data, yielding some positive outcomes. Nevertheless, they still lack specific instance-level hash codes and fail to consider the diversity and semantic association of different modalities, which is insufficient to meet the attacker's expectations. In response, we present a novel Primary code Guided Targeted Attack (PGTA) against cross-modal hashing retrieval. Specifically, we integrate cross-modal instances and labels to obtain well-fused target semantics, thereby enhancing cross-modal interaction. Secondly, the primary code is designed to generate discriminable information with fine-grained semantics for target labels. Benign samples and target semantics collectively generate adversarial examples under the guidance of primary codes, thereby enhancing the efficacy of targeted attacks. Extensive experiments demonstrate that our PGTA outperforms the most advanced methods on three datasets, achieving State-of-the-Art targeted attack performance.

Conventional domain adaptation (DA) for person re-identification (ReID) aims to bridge the domain gap but often requires direct use of fully labeled source and target domains, raising significant data privacy concerns due to the inclusion of personal identity information (PII) in raw data. Source-free domain adaptation (SFDA) for person ReID effectively preserves PII within the authorized source model. Nevertheless, these methods are vulnerable to data privacy (e.g., portrait rights) of the target domain during retrieval, where attackers can exploit pedestrian images for malicious generation, leading to damage to an individual’s reputation. Beyond these limitations, we propose a novel framework called SecureDA to address privacy-preserving SFDA for person ReID, which can generate a privacy key to defend against potential attacks on PII. Technically, we introduce domain-specific adversarial attacks into DA, where the protected query and gallery images are encrypted to ensure secure image retrieval. Furthermore, we employ two simultaneous processes: 1) The global–local adversarial pathway (GLAP) leverages encrypted and original images as adversarial pairs, thereby fostering the development of robust ReID models; 2) The global–local collaborative pathway (GLCP) is mastered through positive pairs collected from the same domain, effectively mitigating the pernicious catastrophic forgetting phenomenon. Extensive experiments show that SecureDA achieves state-of-the-art performance on multiple DA benchmarks and even outperforms the conventional DA and SFDA methods, which inherently compromise data privacy.

The preservation and enhancement of texture information is crucial for the fusion of visible and infrared images. However, most current deep neural network (DNN)-based methods ignore the differences between texture and content, leading to unsatisfactory fusion results. To further enhance the quality of fused images, we propose a texture-content dual guided (TCDG-Net) network, which produces the fused image by the guidance inferred from source images. Specifically, a texture map is first estimated jointly by combining the gradient information of visible and infrared images. Then, the features learned by the shallow feature extraction (SFE) module are enhanced with the guidance of the texture map. To effectively model the texture information in the long-range dependencies, we design the texture-guided enhancement (TGE) module, in which the texture-guided attention mechanism is utilized to capture the global similarity of the texture regions in source images. Meanwhile, we employ the content-guided enhancement (CGE) module to refine the content regions in the fused result by utilizing the complement of the texture map. Finally, the fused image is generated by adaptively integrating the enhanced texture and content information. Extensive experiments on three benchmark datasets demonstrate the effectiveness of the proposed TCDG-Net in terms of qualitative and quantitative evaluations. Besides, the fused images generated by our proposed TCDG-Net also show better performance in downstream tasks, such as objection detection and semantic segmentation.

Providing sequential recommendations along with easily comprehensible natural language explanations can significantly enhance users’ trust in the recommender systems. However, this approach presents two key challenges: (1) The different objectives of the two tasks make it challenging to achieve joint optimization and mutual enhancement. (2) The simultaneous generation of accurate sequential recommendations and high-quality natural language explanations presents serious challenges to the model’s time and space efficiency. To address these challenges, we propose a general and efficient multi-task self-prompt framework for explainable sequential recommendation (ARTS), which improves collaboration performance and time and space efficiency of multi-task modules based on the generated personalized semantic prompts. Specifically, we propose a self-prompt generator that transfers the user’s global behavior features into the continuous prompt, achieving efficient information sharing among multi-task modules. Additionally, we design a personalized prompt-based short sequence inputs strategy under the pre-training and prompt-tuning paradigm, which achieves mutual enhancement among the multi-task modules and significantly improves the model’s time and space efficiency. Extensive experiments have verified that the proposed ARTS outperforms the state-of-the-art methods in both sequential recommendation and explanation generation tasks. The generality, efficiency and effectiveness of each module of the framework have also been validated through various experiments 1 .

Growth optimizer is a novel metaheuristic algorithm that has powerful numerical optimization capabilities. However, its parameters and search operators become crucial factors that significantly impact its optimization capability for engineering problems and benchmarks. Therefore, this paper proposes a quadruple parameter adaptation growth optimizer (QAGO) integrated with distribution, confrontation, and balance features. In QAGO, the quadruple parameter adaptation mechanism aims to reduce the algorithmic sensitivity for parameter setting and enhance the algorithmic adaptability. By employing parameter sampling that adheres to specific probability distributions, the parameter adaptation mechanism achieves dynamic tuning of the algorithm hyperparameters. Moreover, one-dimensional mapping and fitness difference methods are designed in the triple parameter self-adaptation mechanism based on the contradictory relationship to adjust the operator’s parameters. After that, ”spear” and ”shield” are balanced based on the Jensen–Shannon divergence in information theory. Furthermore, the topological structure of the operators is redesigned, and by combining the parameter adaptation mechanism, operator refinement is achieved. Refined operators can effectively utilize different evolutionary information to improve the quality of the solution. The experiment evaluates the performance of QAGO on distinct optimization problems on the CEC 2017 and CEC 2022 test suites. To demonstrate the capability of QAGO in solving real-world applications, it was applied to tackle two specific problems: multilevel threshold image segmentation and wireless sensor network node deployment. The results demonstrated that QAGO delivers highly promising optimization results compared to seventy-one high-performance competing algorithms, including the five IEEE CEC competition winners. The source code of the QAGO algorithm is publicly available at https://github.com/tsingke/QAGO .

The abundance of heavy data on social media enables users to share opinions freely, leading to the rapid spread of misleading content. However, existing fake news detection methods exaggerate the influence of public opinions, making it challenging to combat misinformation since its early spreading state. To tackle this issue, we propose a novel fake news detection framework through news semantic environment perception (NSEP) to identify fake news content. The NSEP framework consists of three major steps. First, NSEP divides the news semantic environment with time-constrained intervals into macro and micro semantic environments using an in-depth distinguisher module. Second, graph convolutional networks are applied to perceive the semantic inconsistencies between intrinsic news content and extrinsic post tokens in the macro semantic environment. Third, a micro semantic detection module guided by multihead attention and sparse attention is utilized to capture the semantic contradictions between news content and posts in the micro semantic environment, providing explicit evidence for determining the authenticity of fake news candidates. Empirical experiments conducted on real-world Chinese and English datasets show that the NSEP framework on Chinese datasets achieved as high as 86.8% accuracy, performing at most 14.1% higher accuracy than that of other state-of-the-art baseline methods and confirming that detecting news content through both micro and macro semantic environments is an effective methodology for alleviating early propagation of fake news. The findings also comprehensively indicate that both news items and posts are critical for the early debunking of fake news and in theories concerning information science.

Recently, deep neural network (DNN)-based methods have achieved good results in terms of the fusion of low spatial resolution hyperspectral (LR HS) and high spatial resolution multispectral (HR MS) images. However, the spectral band correlation (SBC) and the spatial nonlocal similarity (SNS) in hyperspectral (HS) images are not sufficiently exploited by them. To model the two priors efficiently, we propose a spectral-spatial dual graph unfolding network (SDGU-Net), which is derived from the optimization of graph regularized restoration models. Specifically, we introduce spectral and spatial graphs to regularize the reconstruction of the desired high spatial resolution hyperspectral (HR HS) image. To explore the SBC and SNS priors of HS images in feature space and utilize the powerful learning ability of DNNs simultaneously, the iterative optimization of the spectral and spatial graph regularized models is unfolded as a network, which is composed of spectral and spatial graph unfolding modules. The two kinds of modules are designed according to the solutions of the spectral and spatial graph regularized models. In these modules, we employ graph convolution networks (GCNs) to capture the SBC and SNS in the fused image. Then, the learned features are integrated by the corresponding feature fusion modules and fed into the feature condense module to generate the HR HS image. We conduct extensive experiments on three benchmark datasets and the results demonstrate the effectiveness of our proposed SDGU-Net.

Incomplete multi-view clustering has represented a significant role in grouping real images. In this study, a novel robust tensor subspace learning (RTSL) is proposed for incomplete multi-view clustering. Specifically, the missing samples within views are first recovered by matrix factorization. The recovered information is utilized for latent representations learning. And then, the obtained latent representations are organized from all views into a third-order tensor and the intrinsic sample relations are captured with tensor linear representation. Moreover, a low-rank sample coefficient tensor is sought to capture high-order connections among views by imposing the tensor nuclear norm. Compared with traditional learning paradigms in the vector space, the sample relations within each view as well as across views could be preserved with the aid of robust tensor subspace learning. As a result, our model can simultaneously handle the missing samples and exploit the intrinsic correlations, leading to enhanced representation capability and better quality of the recovered data. We design an efficient iterative optimization strategy to solve the proposed method. Experimental results on eight datasets show that our model outperforms other competing approaches.

With the advent of multi-modal data, multi-modal hashing has received increasing attention for it can configure complementary multi-modal fusion and support fast multimedia retrieval. Nevertheless, the “coarse-grained” modality weighting strategy widely used in existing methods always ignores the distinctive contributions of different features and is troubled by parameter adjustment. Besides, traditional supervised methods usually adopt “hard semantic” that reflects the logical relationship between data and labels, but fails to poring on the description degree of categories to data. To solve these problems, we propose amulti-Facet weIghting aSymmetric Multi-modal Hashing based on latent semantic distribution (FISMH)approach, which is divided into supervised paradigm SFISMH and unsupervised paradigm UFISMH. First, we design aMulti-facet Weighted Multi-modal Fusion modulethat utilizes both modality- and feature- wise weights to achieve multi-modal fusion, where the weight learning requires no additional parameter adjustment. Then, we design aLatent Semantic Distribution based Asymmetric Hash Learning module, which utilizes the pair- wise similarity and semantic distribution to guide hash learning, and avoids the challenging pair- wise factorization through asymmetric form. The semantic distribution is learned from the inherent information of feature space, which can further preserve the intra-class relationships. Finally, a discrete hash optimization is developed to reduce quantization and directly learn hash codes. The main difference between SFISMH and UFISMH is that the former utilizes category information while the latter explores the underlying data structure when constructing the pair- wise similarity. Extensive experiments demonstrate that both SFIMH and UFISMH outperform existing supervised and unsupervised multi-modal hashing methods, showcasing their exceptional performance.

Conventional domain adaptive (DA) methods for person re-identification (ReID) face knowledge transfer challenges when labeled data from the source domain cannot be accessed due to privacy constraints. Although the methods operating under source-absent DA settings attempt to address this challenge by using models pretrained on the source domain in their mutual teaching frameworks, failing to capture domain divergence in scenarios in which the source data are completely inaccessible can simultaneously introduce issues related to mutual convergence. In response, we introduce an adversarial attacks-driven mutual teaching (AAMT) framework as an innovative and applicable source-free DA person ReID scheme. Specifically, we first carefully develop a perturbation generator to generate source-style adversarial examples by leveraging a pretrained source model. Then, these diverse adversarial examples are employed to attack the mutual teaching model, implicitly measuring the domain divergence. Accordingly, we design a contrastive learning loss to enlarge the differences between the training pairs and further mitigate the mutual convergence issue. Extensive experiments demonstrate that AAMT outperforms the existing methods under both conventional and source-absent DA settings, achieving state-of-the-art performance.

Domain adaption (DA) for person re-identification (ReID) has attained considerable progress by transferring knowledge from a source domain with labels to a target domain without labels. Nonetheless, most of the existing methods require access to source data, which raises privacy concerns. Source-free DA has recently emerged as a response to these privacy challenges, yet its direct application to open-set pedestrian re-identification tasks is hindered by the reliance on a shared category space in existing methods. Current source-free DA approaches for person ReID still encounter several obstacles, particularly the divergence-agnostic problem and the notable domain divergence due to the absent source data. In this article, we introduce an Instance-level Adversarial Mutual Teaching (IAMT) framework, which utilizes adversarial views to tackle the challenges mentioned above. Technically, we first elaborately develop a variance-based division (VBD) module to segregate the target data into instance-level subsets based on their similarity and dissimilarity to the source using the source-trained model, implicitly tackling the divergence-agnostic problem. To mitigate domain divergence, we additionally introduce a dynamic adversarial alignment (DAA) strategy, aiming to enhance the consistence of feature distribution across domains by employing adversarial instances from the target data to confuse the discriminators. Experiments reveal the superiority of the IAMT over state-of-the-art methods for DA person ReID tasks, while preserving the privacy of the source data.

Most cross-modal retrieval methods assume the multi-modal training data is complete and has a one-to-one correspondence. However, in the real world, multi-modal data generally suffers from missing modality information due to the uncertainty of data collection and storage processes, which limits the practical application of existing cross-modal retrieval methods. Although some solutions have been proposed to generate the missing modality data using a single pseudo sample, this may lead to incomplete semantic restoration and sub-optimal retrieval results due to the limited semantic information it provides. To address this challenge, this article proposes an Incomplete Cross-Modal Retrieval with Deep Correlation Transfer (ICMR-DCT) method that can robustly model incomplete multi-modal data and dynamically capture the adjacency semantic correlation for cross-modal retrieval. Specifically, we construct intra-modal graph attention-based auto-encoder to learn modality-invariant representations by performing semantic reconstruction through intra-modality adjacency correlation mining. Then, we design dual cross-modal alignment constraints to project multi-modal representations into a common semantic space, thus bridging the heterogeneous modality gap and enhancing the discriminability of the common representation. We further introduce semantic preservation to enhance adjacency semantic information and achieve cross-modal semantic correlation. Moreover, we propose a nearest-neighbor weighting integration strategy with cross-modal correlation transfer to generate the missing modality data according to inter-modality mapping relations and adjacency correlations between each sample and its neighbors, which improves the robustness of our method against incomplete multi-modal training data. Extensive experiments on three widely tested benchmark datasets demonstrate the superior performance of our method in cross-modal retrieval tasks under both complete and incomplete retrieval scenarios. Our used datasets and source codes are available at https://github.com/shidan0122/DCT.git .

The occluded target person often lead to incorrect matching results, so eliminating the interference of background clutter is a key matter. To deal with the challenging occluded person re-identification (Re-ID) tasks, this paper proposes a new grouped pose estimation occlusion generation (GPEOG) network to fully use the person grouped keypoint information and the stable non-occluded features, and make the extracted features more robust and discriminative. The local branch uses the proposed Dynamic Adaptive module (DAM) to automatically adjust the patch size during training process and provides adaptive local features. The occlusion-suppression generation branch mainly includes the proposed effective occlusion suppression (EOS) module. It judges whether the person’s keypoint groups are occluded, and then generates the masks of the corresponding parts to suppress occlusion. Extensive experiments show that the proposed method has obvious advantages compared with several state-of-the-art methods.

Image-text matching is a hot topic in multi-modal analysis. The existing image-text matching algorithms focus on bridging the heterogeneity gap and mapping the feature into a common space under strong alignment assumption. However, these methods have unsatisfactory performance under the weak alignment scenario, which assumes that the text contains more abstract information, and the number of entities in the text is always fewer than objects in image. This is the first time, from our knowledge, to solve the image-text matching problem from the perspective of information difference with weak alignment. In order to both narrow the cross-modal heterogeneity gap and balance the information discrepancy, we proposed an imagination network to enrich the text modality based on pre-trained framework, which is helpful for image-text matching. The imagination network utilizes reinforcement learning to enhance the semantic information for text modality, and an action refinement strategy is designed to constrain the freedom and divergence of imagination. The experiment results show the superiority and generality of the proposed framework based on two pre-trained models, CLIP and BLIP on two most frequently-used datasets MSCOCO and Flickr30K.

Multimodal fake news detection methods based on semantic information have achieved great success. However, these methods only exploit the deep features of multimodal information, which leads to a large loss of valid information at the shallow level. To address this problem, we propose a progressive fusion network (MPFN) for multimodal disinformation detection, which captures the representational information of each modality at different levels and achieves fusion between modalities at the same level and at different levels by means of a mixer to establish a strong connection between the modalities. Specifically, we use a transformer structure, which is effective in computer vision tasks, as a visual feature extractor to gradually sample features at different levels and combine features obtained from a text feature extractor and image frequency domain information at different levels for fine-grained modeling. In addition, we design a feature fusion approach to better establish connections between modalities, which can further improve the performance and thus surpass other network structures in the literature. We conducted extensive experiments on two real datasets, Weibo and Twitter, where our method achieved 83.3% accuracy on the Twitter dataset, which has increased by at least 4.3% compared to other state-of-the-art methods. This demonstrates the effectiveness of MPFN for identifying fake news, and the method reaches a relatively advanced level by combining different levels of information from each modality and a powerful modality fusion method.

Recently, multi-view unsupervised feature selection has attracted much attention due to its efficiency and better interpretability in processing high-dimensional multi-view datasets. Most existing methods rely on the constructed similarity matrices to obtain reliable pseudo labels to guide the feature selection. However, the considerable adverse noise in the raw data inevitably impedes the exploration of true underlying similarity structures. Besides, the inter-view correlations are often ignored during the common representation learning , which limits the effective fusion of the essential information from multiple views. To solve these issues, we design a novel robust multi-view unsupervised feature selection framework. Specifically, our method seeks a set of noise-free view-specific similarity matrices by leveraging tensor robust principal component analysis , where the high-order connections among different views are well exploited through the constructed low-rank tensor. Meanwhile, a high-quality consensus similarity matrix is adaptively learned from the view-specific representations within the same unified framework to capture the shared local structures. To enhance the discriminative ability of the feature selection matrix, we further impose a rank constraint on the consensus similarity matrix to obtain reliable pseudo cluster indicators. We present an efficient optimization algorithm ground on the alternating direction method of multipliers to solve the proposed model. Experimental results on six multi-view datasets confirm the superiority of our method.

Many current video and text cross-modal retrieval research works focus on narrowing the semantic gap between video and text, but ignore the semantic difference between different sampled frames in the same video and the correlation of feature distribution of objects contained in different sampled frames in the same video, as a result, the features of the sampled frames in the final learned video cannot well represent the semantic features of the whole video. To overcome the shortcomings of existing studies, we first use a pre-trained video frame classification-aggregation network to make the object categories contained in different sampled frames in the same video be more close to the important object categories contained in the whole video, so as to promote the feature distribution of different sampled frames in the same video to be consistent, and increase the relevance of object features in different frames. Then we propose a video internal frame aggregation loss module to solve the problem of inconsistent feature distribution between different frame features encoded by video encoder in the same video and the aggregation feature of the sampled frame, thus enhancing the ability of video sampled frame aggregation feature representation. Experiments conducted on three common datasets MSVD, MSR-VTT and DiDeMo demonstrate the validity of the proposed approach.

Deep cross-modal hashing has achieved excellent retrieval performance with the powerful representation capability of deep neural networks. Regrettably, current methods are inevitably vulnerable to adversarial attacks, especially well-designed subtle perturbations that can easily fool deep cross-modal hashing models into returning irrelevant or the attacker’s specified results. Although adversarial attacks have attracted increasing attention, there are few studies on specialized attacks against deep cross-modal hashing. To solve these issues, we propose a targeted adversarial attack method against deep cross-modal hashing retrieval in this paper. To the best of our knowledge, this is the first work in this research field. Concretely, we first build a progressive fusion module to extract fine-grained target semantics through a progressive attention mechanism. Meanwhile, we design a semantic adaptation network to generate the target prototype code and reconstruct the category label, thus realizing the semantic interaction between the target semantics and the implicit semantics of the attacked model. To bridge modality gaps and preserve local example details, a semantic translator seamlessly translates the target semantics and then embeds them into benign examples in collaboration with a U-Net framework. Moreover, we construct a discriminator for adversarial training, which enhances the visual realism and category discrimination of adversarial examples, thus improving their targeted attack performance. Extensive experiments on widely tested cross-modal retrieval datasets demonstrate the superiority of our proposed method. Also, transferable attacks show that our generated adversarial examples have well generalization capability on targeted attacks. The source codes and datasets are available athttps://github.com/tswang0116/TA-DCH.

Various deep neural networks (DNNs) have been constructed to inject the spatial information of the panchromatic (PAN) image into the low spatial resolution multispectral (LR MS) image. However, most of them ignore the local dissimilarity (LD) prior between MS and PAN images, which has a negative influence on the fused image. Considering the above-mentioned issues, we propose a deep multiscale local dissimilarity network (DMLD-Net) to learn the LD prior at different scales and enhance the spatial and spectral information in the fused image better. Specifically, we first synthesize a downsampled PAN image from the original PAN image to match the scale of the LR MS image. Then, a LD metric is designed to calculate the dissimilarity map between the two images in feature space. According to the learned dissimilarity map, we utilize a LD-guided attention block (LDGAB) to suppress the impact of LD, which filters out the dissimilar information in the features of the PAN image. To learn the LD prior between MS and PAN images sufficiently, the multiscale architecture is considered and we infer the dissimilar maps hierarchically and inject filtered features into the LR MS image progressively. Finally, the fused image is generated by a reconstruction block. Through the LD learning at different scales, reasonable spatial information is extracted from the PAN image, by which the distortions in the fused image caused by LD can be reduced efficiently. Extensive experiments are conducted on GeoEye-1 and WorldView-2 datasets and the results demonstrate the effectiveness of the proposed DMLD-Net in terms of spatial and spectral preservation. The code is available at https://github.com/RSMagneto/DMLD-Net.

Hashing is an effective technique to improve the efficiency of large-scale recommender system by representing both users and items into binary codes. However, existing hashing-based recommendation methods still suffer from two important problems: 1) Cold-start. They employ the user-item interactions and single auxiliary information to learn the binary hash codes. But the full interaction history is not always available and the single auxiliary information may be missing. 2) Efficient optimization. They learn the hash codes with two-step relaxed optimization or one-step discrete hash optimization based on the cyclic coordinate descent, which results in significant quantization loss or still consumes considerable computation time. In this paper, we propose a Multi-modal Discrete Collaborative Filtering (MDCF) for efficient cold-start recommendation. We map the multi-modal features of users and items to a consensus Hamming space based on the matrix factorization framework. Specifically, a low-rank self-weighted multi-modal fusion module is designed to adaptively fuse the multi-modal features into binary hash codes. Additionally, to support large-scale recommendation, a fast discrete optimization method based on augmented Lagrangian multiplier is developed to directly compute the binary hash codes with simple operations. Experiments show the superior performance of the proposed method over state-of-the-art baselines.

Recently, various view synthesis distortion estimation models have been studied to better serve 3-D video coding. However, they can hardly model the relationship quantitatively among different levels of depth changes, texture degeneration, and view synthesis distortion (VSD), which is crucial for rate-distortion optimization and rate allocation. In this paper, an auto-weighted layer representation based view synthesis distortion estimation model is developed. Firstly, sub-VSD (S-VSD) is defined according to the level of depth changes and their associated texture degeneration. After that, a set of theoretical derivations demonstrate that the VSD can be approximately decomposed into the S-VSDs multiplied by their associated weights. To obtain the S-VSDs efficiently, a layer-based representation method is developed, where all the pixels with the same level of depth changes are represented with a layer. It enables the S-VSD calculation at the layer level. Meanwhile, a nonlinear mapping function is learnt to accurately represent the relationship between the VSD and S-VSDs, automatically providing weights for the S-VSDs during VSD estimation. To learn such a function, a dataset of the VSD and its associated S-VSDs are built, termed as VSDSet. Experimental results show that the VSD can be accurately estimated with the weights learnt by the nonlinear mapping function once its associated S-VSDs are available. The proposed method outperforms the relevant state-of-the-art methods in both accuracy and efficiency. The VSDSet and source code of the proposed method will be available athttps://github.com/jianjin008/.

Although there are lots of studies on scene text recognition, few of them focus on the recognition of the incomplete text. The recognition performance of existing text recognition algorithms on the incomplete text is far from the expected, and the recognition of the incomplete text is still challenging. In this paper, an end-to-end Two-Stage Inpainting Network for Incomplete Text (TSINIT) is proposed to reconstruct the incomplete text into the complete one even when the text is in various styles and with various backgrounds, and the reconstructed text can be recognized by the existing text recognition algorithms correctly. The proposed TSINIT is divided into text extraction module (TEM) and text reconstruction module (TRM) to make the inpainting only focus on the text. TEM separates the incomplete text from the background and character-like regions at the pixel level, which can reduce the ambiguity of text reconstruction caused by the background. TRM reconstructs the incomplete text towards the most possible text with the consideration of the abstract and semantic structures of the text. Furthermore, we build a synthetic incomplete text dataset (SITD), which contains contaminated and abraded text images. SITD is divided into 6 incomplete levels according to the number of pixels in the incomplete regions and the ratio of the incomplete characters to all characters. The experimental results show that the proposed method has better inpainting ability for the incomplete text compared with traditional image inpainting algorithms on the proposed SITD and real images. When using the same text recognition method, the recognition accuracy of the incomplete text on SITD can be improved much more with the help of the proposed TSINIT than with the traditional image inpainting methods.

Inspired by the powerful representation capability of deep neural networks, deep cross-modal hashing methods have recently drawn much attention and various deep cross-modal hashing methods have been developed. However, two key problems have not been solved well yet: 1) With advanced neural network models, how to seek the multi-modal alignment space which can effectively model the intrinsic multi-modal correlations and reduce the heterogeneous modality gaps. 2) How to effectively and efficiently preserve the modelled multi-modal semantic correlations into the binary hash codes under the deep learning paradigm. In this paper, we propose aHierarchical Message Aggregation Hashing(HMAH) method within an efficient teacher-student learning framework. Specifically, on the teacher end, we develop hierarchical message aggregation networks to construct a multi-modal complementary space by aggregating the semantic messages hierarchically across different modalities, which can better align the heterogeneous modalities and model the fine-grained multi-modal correlations. On the student end, we train a couple of student modules that learn hash functions to support cross-modal retrieval. We design a cross-modal correlation knowledge distillation strategy which seamlessly transfers the modelled fine-grained multi-modal semantic correlations from the teacher to the lightweight student modules. With the fine-grained knowledge supervision from teacher module, the semantic representation capability of hash functions can be enhanced. In addition, the whole learning framework avoids the time-consuming finetuning on the pre-trained deep models as existing methods and it is computationally efficient. Experimental results demonstrate the significant performance improvement of the proposed method on both retrieval accuracy and efficiency, compared with the state-of-the-art deep cross-modal hashing methods.

Multi-modal hashing technology can support large-scale multimedia retrieval well, because of its fast query speed and low storage consumption. Although many multi-modal hashing methods have been developed in the literature, they still suffer from three important problems: 1) Most multi-modal hashing methods assume that multi-modal data are complete. This ideal assumption limits their application to practical retrieval scenarios, where the modality-missing is common. 2) Existing partial multi-modal hashing methods directly model incomplete multi-modal data for hash learning, which may result in partial multi-modal semantics in the learned hash codes. 3) Most of the methods are based on the shallow learning framework, which inevitably suffers from limited representation capability. To solve the above problems, we propose a flexible deep partial multi-modal hash learning framework, named Neighbor-aware Completion Hashing (NCH). Our framework jointly performs the cross-modal completion learning for incomplete multi-modal data and the multi-modal hash learning. It can not only support model training with incomplete multi-modal data but also handle incomplete multi-modal queries. Besides, we design a neighbor-aware completion learning module to capture neighbor semantics and generate distribution-consistent completed features. Finally, we conduct extensive experiments to evaluate our method on both fully-paired and partial multi-modal retrieval scenarios. The experimental results verify the superiority of our proposed method over the state-of-the-art baselines. The experimental codes of our method are released at:https://github.com/FutureTwT/NCH.

The completeness of semantic expression plays an important role in cross-modal retrieval tasks, which contributes to align the cross-modal data and thus narrow the modality gap. But due to the abstractness of semantics, the same topic may have different aspects to be well described so it may be incomplete to express semantics with only one sample. In order to obtain semantic complementary information and strengthen similar information for samples with the same semantics, we utilize a graph convolutional network (GCN) to reconstruct the sample representation based on the adjacency relationship between the sample itself and its neighborhoods. We construct a local graph for each instance, and propose a novel Graph Feature Generator based on GCN and a fully-connected network to reconstruct node features based on local graph and map the features of two modalities into a common space. The Graph Feature Generator and Graph Feature Discriminator adopt a minimax game strategy to generate modality-invariant graph feature representations. Experiments on three benchmark datasets demonstrate the superiority of our proposed model compared with several state-of-the-art methods.

In order to carry out more accurate retrieval across image-text modalities, some scholars use fine-grained feature to align image and text. Most of them directly use attention mechanism to align image regions and words in the sentence, and ignore the fact that semantics related to an object is abstract and cannot be accurately expressed by object information alone. To overcome this weakness, we propose a hierarchical feature aggregation algorithm based on graph convolutional networks (GCN) to facilitate object semantic integrity by integrating attributes of an object and relations between objects hierarchically in both image and text modalities. In order to eliminate the semantic gap between modalities, we propose a cross-modal feature fusion method based on transformer to generate modal-specific feature representations by integrating both the object feature and global feature from the other modality. Then we map the fusion feature into a common space. Experiment results on the most frequently-used datasets MSCOCO and Flickr30K show the effectiveness of the proposed model compared with the latest methods.

Multi-modal hashing supports efficient multimedia retrieval well. However, existing methods still suffer from two problems: (1) Fixed multi-modal fusion. They collaborate the multi-modal features with fixed weights for hash learning, which cannot adaptively capture the variations of online streaming multimedia contents. (2) Binary optimization challenge. To generate binary hash codes, existing methods adopt either two-step relaxed optimization that causes significant quantization errors or direct discrete optimization that consumes considerable computation and storage cost. To address these problems, we first propose a Supervised Multi-modal Hashing with Online Query-adaption method. A self-weighted fusion strategy is designed to adaptively preserve the multi-modal features into hash codes by exploiting their complementarity. Besides, the hash codes are efficiently learned with the supervision of pair-wise semantic labels to enhance their discriminative capability while avoiding the challenging symmetric similarity matrix factorization. Further, we propose an efficient Unsupervised Multi-modal Hashing with Online Query-adaption method with an adaptive multi-modal quantization strategy. The hash codes are directly learned without the reliance on the specific objective formulations. Finally, in both methods, we design a parameter-free online hashing module to adaptively capture query variations at the online retrieval stage. Experiments validate the superiority of our proposed methods.

Identifying cancer subtypes is of key significance for the application of personalized medicine. It is devoted to using the unsupervised clustering methods to stratify cancer patients into diverse subgroups and provide specific treatments accordingly. Recently, the abundant multi-omics data have provided unprecedented opportunities to discover intrinsic subtypes at an integrative level. However, it still remains a challenging task to combine these heterogeneous data and reveal the underlying clustering patterns. In this paper, we propose a novel method to identify cancer subtypes based on multi-view subspace clustering with adaptive local structure learning. Specifically, our model can effectively explore the local similarity structure from each omics data and simultaneously discover the unified consensus similarity structure among all omics data. Moreover, we design an efficient Augmented Lagrangian Multiplier algorithm to optimize the proposed framework. The experimental results on both cancer datasets and image datasets confirmed the superior performance of the proposed method over existing alternatives. In addition, we conduct a case study on melanoma and identify a set of molecular functions specific to each subtype.

In recent years, 3D target detection and location methods in the field of autonomous driving have attracted increasing attention, but monocular 3D pedestrian localization research is still facing challenges. In this paper, a monocular pedestrian localization framework and a fine-grained location optimization method which is based on a bird's eye view are proposed. The monocular pedestrian localization framework is divided into three parts, which are coarse-grained localization, depth information reconstruction and fine-grained location optimization. In the stage of coarse-grained location, the human skeleton information is obtained from the original image by using the human skeleton point detection method, and then the pedestrian position is predicted through the method of a light-weight feed-forward neural network. In the stage of depth information reconstruction, the original image is used to reconstruct the corresponding bird's eye view with depth information through a parallel network. Finally, a fine-grained positioning optimization method makes it possible to get a more precise location with the help of the last two stages. The experimental results on the KITTI dataset show that our method has achieved better performance than the state-of-the-art methods.

Multi-modal hashing makes an important contribution to multimedia retrieval, where a key challenge is to encode heterogeneous modalities into compact hash codes. To solve this dilemma, graph-based multi-modal hashing methods generally define individual affinity matrix of each independent modality and apply linear algorithm for heterogeneous modalities fusion and compact hash learning. Several other methods construct graph Laplacian matrix based on semantic information to help learn discriminative hash code. However, these conventional methods roughly ignore the structural similarity of training set and the complex relations among multi-modal samples, which leads to unsatisfactory complementarity of fused hash codes. More notably, they are faced with two other important problems: huge computing and storage costs caused by graph construction and partial modality feature lost problem when incomplete query sample comes. In this paper, we propose a Flexible Graph Convolutional Multi-modal Hashing (FGCMH) method that adopts GCNs with linear complexity to preserve both the modality-individual and modality-fused structural similarity for discriminative hash learning. Necessarily, accurate multimedia retrieval can be performed on complete and incomplete datasets with our method. Specifically, multiple modality-individual GCNs under semantic guidance are proposed to act on each individual modality independently for intra-modality similarity preserving, then the output representations are fused into a fusion graph with adaptive weighting scheme. Hash GCN and semantic GCN, which share parameters in the first two layers, propagate fusion information and generate hash codes under high-level label space supervision. In the query stage, our method adaptively captures various multi-modal contents in a flexible and robust way, even if partial modality features are lost. Experimental results on three publicly datasets show the flexibility and effectiveness of our proposed method.

Most existing methods focus on improving the clarity and semantic consistency of the image with a given text, but do not pay attention to the multiple control of generated image content, such as the position of the object in generated image. In this paper, we introduce a novel position-based generative network (PBNet) which can generate fine-grained images with the object at the specified location. PBNet combines iterative structure with generative adversarial network (GAN). A location information embedding module (LIEM) is proposed to combine the location information extracted from the boundary block image with the semantic information extracted from the text. In addition, a silhouette generation module (SGM) is proposed to train the generator to generate object based on location information. The experimental results on CUB dataset demonstrate that PBNet effectively controls the location of the object in the generated image.

Abstract It can be a very challenging task when using level set method segmenting natural images with high intensity inhomogeneity and complex background scenes. A new synthesis level set method for robust image segmentation based on the combination of Retinex‐corrected saliency region information and edge information is proposed in this work. First, the Retinex theory is introduced to correct the saliency information extraction. Second, the Retinex‐corrected saliency information is embedded into the level set method due to its advantageous quality which makes a foreground object stand out relative to the backgrounds. Combined with the edge information, the boundary of segmentation will be more precise and smooth. Experiments indicate that the proposed segmentation algorithm is efficient, fast, reliable, and robust.

Recent studies show that hashing technology can achieve efficient similarity searching and many works have been done on supervised deep hash learning. However, under unsupervised scenarios, there are several issues to be solved when learning hashing codes based on visual features for image retrieval and clustering. In this article, we propose a simple but effective Unsupervised Deep K-means Hashing (UDKH) method to simultaneously alleviate the problems of image retrieval and clustering within a single learning framework. UDKH progressively improves the quality of cluster labels and binary hash codes by minimizing pair-wise supervision loss and optimizing the binary K-means to generate discriminative hash codes under the supervision of the learned cluster labels for effective image retrieval. Since the learned hash codes are discriminative, UDKH also improves the image clustering accuracy. Experiments on test datasets demonstrate its effectiveness for image retrieval and clustering.

Multi-modal hashing focuses on fusing different modalities and exploring the complementarity of heterogeneous multi-modal data for compact hash learning. However, existing multi-modal hashing methods still suffer from several problems, including: 1) Almost all existing methods generate unexplainable hash codes. They roughly assume that the contribution of each hash code bit to the retrieval results is the same, ignoring the discriminative information embedded in hash learning and semantic similarity in hash retrieval. Moreover, the length of hash code is empirically set, which will cause bit redundancy and affect retrieval accuracy. 2) Most existing methods exploit shallow models which fail to fully capture higher-level correlation of multi-modal data. 3) Most existing methods adopt online hashing strategy based on immutable direct projection, which generates query codes for new samples without considering the differences of semantic categories. In this paper, we propose a Semantic-driven Interpretable Deep Multi-modal Hashing (SIDMH) method to generate interpretable hash codes driven by semantic categories within a deep hashing architecture, which can solve all these three problems in an integrated model. The main contributions are: 1) A novel deep multi-modal hashing network is developed to progressively extract hidden representations of heterogeneous modality features and deeply exploit the complementarity of multi-modal data. 2) Learning interpretable hash codes, with discriminant information of different categories distinctively embedded into hash codes and their different impacts on hash retrieval intuitively explained. Besides, the code length depends on the number of categories in the dataset, which can reduce the bit redundancy and improve the retrieval accuracy. 3) The semantic-driven online hashing strategy encodes the significant branches and discards the negligible branches of each query sample according to the semantics contained in it, therefore it could capture different semantics in dynamic queries. Finally, we consider both the nearest neighbor similarity and semantic similarity of hash codes. Experiments on several public multimedia retrieval datasets validate the superiority of the proposed method.

The introduction of workflow in cloud computing has afforded a new and efficient way to tackle large-scale applications. As an NP-hard problem, how to schedule cloud workflows effectively and economically with deadline constraints and different kinds of tasks and resources is extraordinarily challenging. To solve this constrained problem, this paper intends to develop an intelligent scheduling system from the perspective of users to reduce expenditure of workflow, subject to the deadline and other execution constraints. A new estimation model of the task execution time is designed according to virtual machine settings in real public clouds and execution data from practical workflows. Based on the new model, an adaptive ant colony optimization algorithm is proposed to meet the quality of service and orchestrate tasks. The adaptiveness of the algorithm is embodied in two aspects. First, an adaptive solution construction method is designed that each solution is built with a dynamically changing resource pool, thus the search space of the algorithm is narrowed down and the execution time is decreased. Second, two heuristics with self-adaptive weight are introduced to adaptively meet different deadline settings. Simulating results on four types of workflows show that the proposed approach is effective and competitive.

With the rapid development of mobile devices, people are generating huge volumes of images data every day for sharing on social media, which draws much research attention to understanding the contents of images. Image clustering plays an important role in image understanding systems. Often, most of the existing image clustering algorithms flatten digital images that are originally represented by matrices into 1D vectors as the image representation for the subsequent learning. The drawbacks of vector-based algorithms include limited consideration of spatial relationship between pixels and computational complexity, both of which blame to the simple vectorized representation. To overcome the drawbacks, we propose a novel image clustering framework that can work directly on matrices of images instead of flattened vectors. Specifically, the proposed algorithm simultaneously learn the clustering results and preserve the original correlation information within the image matrix. To solve the challenging objective function, we propose a fast iterative solution. Extensive experiments have been conducted on various benchmark datasets. The experimental results confirm the superiority of the proposed algorithm.

Unknown Unknowns (UUs) are a kind of test data points on which predictive model confidence is high but the prediction incorrect. It is crucial to identify these instances for better understanding the limitation of predictive models and to avoid critical errors. However, existing methods utilize a fixed model to identify UUs in test data, resulting in limited performance and high cost. To address these limitations, we propose a regional candidate selection algorithm that seamlessly combines a deep neural network and humans to identify UUs in visual data. Specifically, we identify several candidate regions in the data space where UUs have high probability of being present. This is achieved by comparing labels learned by a deep network and predictions obtained with the original classification model. Moreover, inspired by active learning, diversity and training loss are utilized to obtain suitable query sequences. We evaluate our method using a publicly available image dataset. Experimental results conducted on this dataset demonstrate the improved performance of the proposed method over different baselines under different conditions and its robustness to noisy labels.

In this paper, the consensus achievement for first-order multi-agent systems is researched over undirected graph. Suppose that the system is unstable and exactly knows the communication delay that affects the actually transmitted information, consensus gain in the protocol is designed with the delay information. Then, conditions are obtained to guarantee consensus for any large yet bounded communication delay. At the end of the article, an example is given to demonstrate the validity of the conclusion.

The flourishing of social media platforms requires techniques for understanding the content of media on a large scale. However, state-of-the art video event understanding approaches remain very limited in terms of their ability to deal with data sparsity, semantically unrepresentative event names, and lack of coherence between visual and textual concepts. Accordingly, in this paper, we propose a method of grounding visual concepts for large-scale Multimedia Event Detection (MED) and Multimedia Event Captioning (MEC) in zero-shot setting. More specifically, our framework composes the following: (1) deriving the novel semantic representations of events from their textual descriptions, rather than event names; (2) aggregating the ranks of grounded concepts for MED tasks. A statistical mean-shift outlier rejection model is proposed to remove the outlying concepts which are incorrectly grounded; and (3) defining MEC tasks and augmenting the MEC training set by the videos detected in MED in a zero-shot setting. To the best of our knowledge, this work is the first time to define and solve the MEC task, which is a further step towards understanding video events. We conduct extensive experiments and achieve state-of-the-art performance on the TRECVID MEDTest dataset, as well as our newly proposed TRECVID-MEC dataset.

Most existing text-to-image generation methods focus on synthesizing images using only text descriptions, but this cannot meet the requirement of generating desired objects with given backgrounds. In this paper, we propose a Background-induced Generative Network (BGNet) that combines attention mechanisms, background synthesis, and multi-level discriminator to generate realistic images with given backgrounds according to text descriptions. BGNet takes a multi-stage generation as the basic framework to generate fine-grained images and introduces a hybrid attention mechanism to capture the local semantic correlation between texts and images. To adjust the impact of the given backgrounds on the synthesized images, synthesis blocks are added at each stage of image generation, which appropriately combines the foreground objects generated by the text descriptions with the given background images. Besides, a multi-level discriminator and its corresponding loss function are proposed to optimize the synthesized images. The experimental results on the CUB bird dataset demonstrate the superiority of our method and its ability to generate realistic images with given backgrounds.

Unsupervised multi-modal hashing has recently attracted broad attention in research area of large-scale multimedia retrieval for its low storage cost, high retrieval speed, and independence on semantic labels. However, the model learning process of existing methods still suffer from the problem of low efficiency: 1) Many existing methods measure the contributions of different modalities using fixed modality weights. In order to avoid over-fitting, they need an inefficient hyper-parameter adjustment process. 2) Most existing methods adopt inefficient optimization strategies to learn hash codes. In this letter, we propose an unsupervised Efficient Parameter-free Adaptive Multi-modal Hashing (EPAMH) model to adaptively capture the modality variations and preserve the discriminative semantics of multi-modal features into the binary hash codes. Moreover, we directly learn the binary codes with simple and efficient operations, which prevents the relaxing quantization errors and improves the model learning efficiency. Experiments prove the superior performance of EPAMH on three public multimedia retrieval datasets. Our source codes and testing datasets can be obtained at https://github.com/ChaoqunZheng/EPAMH.

Large-scale optimization with high dimensionality and high computational cost becomes ubiquitous nowadays. To tackle such challenging problems efficiently, devising distributed evolutionary computation algorithms is imperative. To this end, this paper proposes a distributed swarm optimizer based on a special master-slave model. Specifically, in this distributed optimizer, the master is mainly responsible for communication with slaves, while each slave iterates a swarm to traverse the solution space. An asynchronous and adaptive communication strategy based on the request-response mechanism is especially devised to let the slaves communicate with the master efficiently. Particularly, the communication between the master and each slave is adaptively triggered during the iteration. To aid the slaves to search the space efficiently, an elite-guided learning strategy is especially designed via utilizing elite particles in the current swarm and historically best solutions found by different slaves to guide the update of particles. Together, this distributed optimizer asynchronously iterates multiple swarms to collaboratively seek the optimum in parallel. Extensive experiments on a widely used large-scale benchmark set substantiate that the distributed optimizer could: 1) achieve competitive effectiveness in terms of solution quality as compared to the state-of-the-art large-scale methods; 2) accelerate the execution of the algorithm in comparison with the sequential one and obtain almost linear speedup as the number of cores increases; and 3) preserve a good scalability to solve higher dimensional problems.

Hashing could significantly accelerate large-scale image search by transforming the high-dimensional features into binary Hamming space, where efficient similarity search can be achieved with very fast Hamming distance computation and extremely low storage cost. As an important branch of hashing methods, multi-view hashing takes advantages of multiple features from different views for binary hash learning. However, existing multi-view hashing methods are either based on shallow models which fail to fully capture the intrinsic correlations of heterogeneous views, or unsupervised deep models which suffer from insufficient semantics and cannot effectively exploit the complementarity of view features. In this paper, we propose a novel Deep Collaborative Multi-view Hashing (DCMVH) method to deeply fuse multi-view features and learn multi-view hash codes collaboratively under a deep architecture. DCMVH is a new deep multi-view hash learning framework. It mainly consists of 1) multiple view-specific networks to extract hidden representations of different views, and 2) a fusion network to learn multi-view fused hash code. DCMVH associates different layers with instance-wise and pair-wise semantic labels respectively. In this way, the discriminative capability of representation layers can be progressively enhanced and meanwhile the complementarity of different view features can be exploited effectively. Finally, we develop a fast discrete hash optimization method based on augmented Lagrangian multiplier to efficiently solve the binary hash codes. Experiments on public multi-view image search datasets demonstrate our approach achieves substantial performance improvement over state-of-the-art methods.

Multi-modal hashing methods could support efficient multimedia retrieval by combining multi-modal features for binary hash learning at the both offline training and online query stages. However, existing multi-modal methods cannot binarize the queries, when only one or part of modalities are provided. In this article, we propose a novel Flexible Multi-modal Hashing (FMH) method to address this problem. FMH learns multiple modality-specific hash codes and multi-modal collaborative hash codes simultaneously within a single model. The hash codes are flexibly generated according to the newly coming queries, which provide any one or combination of modality features. Besides, the hashing learning procedure is efficiently supervised by the pair-wise semantic matrix to enhance the discriminative capability. It could successfully avoid the challenging symmetric semantic matrix factorization and O ( n 2 ) storage cost of semantic matrix. Finally, we design a fast discrete optimization to learn hash codes directly with simple operations. Experiments validate the superiority of the proposed approach.

Hashing has recently received substantial attention in large-scale multimedia search for its extremely low-cost storage cost and high retrieval efficiency. However, most existing hashing techniques focus on learning hash codes for single-view or cross-view retrieval. It is still an unsolved problem that how to efficiently learn discriminative binary codes for multi-view data that is common in real world multimedia search. In this paper, we propose an efficient Supervised Discrete Multi-view Hashing (SDMH) to solve the problem. SDMH first properly detects the shared binary hash codes, with an integrated multi-view feature mapping and latent hash coding, by exploiting the complementarity of different view-specific features and removing the involved inter-view redundancy. To further enhance the discriminative capability of hash codes, SDMH directly represses the explicit semantic labels of data samples with their corresponding binary codes. Different from most existing multi-view hashing methods that adopt “relaxing+rounding” hash optimization strategy or the discrete optimization method based on discrete cyclic coordinate descent, an efficient augmented Lagrangian multiplier (ALM) based discrete hash optimization method is developed in this paper to optimize the hash codes within a single step. Experimental results on four benchmark datasets demonstrate the superior performance of the proposed approach over state-of-the-art hashing techniques, in terms of both learning efficiency and retrieval accuracy.

via principal component analysis (PCA), the LDA algorithm can avoid the small sample size problem. Most existing supervised dimensionality reduction methods extract the principal component of data first, and then conduct LDA on it. However, "most variance" is very often the most important, but not always in PCA. Thus, this two-step strategy may not be able to obtain the most discriminant information for classification tasks. Different from traditional approaches which conduct PCA and LDA in sequence, we propose a novel method referred to as joint principal component and discriminant analysis (JPCDA) for dimensionality reduction. Using this method, we are able to not only avoid the small sample size problem but also extract discriminant information for classification tasks. An iterative optimization algorithm is proposed to solve the method. To validate the efficacy of the proposed method, we perform extensive experiments on several benchmark data sets in comparison with some state-of-the-art dimensionality reduction methods. A large number of experimental results illustrate that the proposed method has quite promising classification performance.

Object detection is important in real-world applications. Existing methods mainly focus on object detection with sufficient labelled training data or zero-shot object detection with only concept names. In this paper, we address the challenging problem of zero-shot object detection with natural language description, which aims to simultaneously detect and recognize novel concept instances with textual descriptions. We propose a novel deep learning framework to jointly learn visual units, visual-unit attention and word-level attention, which are combined to achieve word-proposal affinity by an element-wise multiplication. To the best of our knowledge, this is the first work on zero-shot object detection with textual descriptions. Since there is no directly related work in the literature, we investigate plausible solutions based on existing zero-shot object detection for a fair comparison. We conduct extensive experiments on three challenging benchmark datasets. The extensive experimental results confirm the superiority of the proposed model.

Deep hashing has recently received attention in cross-modal retrieval for its impressive advantages. However, existing hashing methods for cross-modal retrieval cannot fully capture the heterogeneous multi-modal correlation and exploit the semantic information. In this paper, we propose a novel Fusion-supervised Deep Cross-modal Hashing (FDCH) approach. Firstly, FDCH learns unified binary codes through a fusion hash network with paired samples as input, which effectively enhances the modeling of the correlation of heterogeneous multi-modal data. Then, these high-quality unified hash codes further supervise the training of the modality-specific hash networks for encoding out-of-sample queries. Meanwhile, both pair-wise similarity information and classification information are embedded in the hash networks under one stream framework, which simultaneously preserves cross-modal similarity and keeps semantic consistency. Experimental results on two benchmark datasets demonstrate the state-of-the-art performance of FDCH.

Multi-modal hashing fuses multi-modal features at both offline training and online query stage for compact binary hash learning. It has aroused extensive attention in research filed of efficient large-scale multimedia retrieval. However, existing methods adopt batch-based learning scheme or unsupervised learning paradigm. They cannot efficiently handle the very common online streaming multi-modal data (for batch-learning methods), or learn the hash codes suffering from limited discriminative capability and less flexibility for varied streaming data (for existing online multi-modal hashing methods). In this paper, we develop a supervised Flexible Online Multi-modal Hashing (FOMH) method to adaptively fuse heterogeneous modalities and flexibly learn the discriminative hash code for the newly coming data, even if part of the modalities is missing. Specifically, instead of adopting the fixed weights, the modalities weights in FOMH are automatically learned with the proposed flexible multi-modal binary projection to timely capture the variations of streaming samples. Further, we design an efficient asymmetric online supervised hashing strategy to enhance the discriminative capability of the hash codes, while avoiding the challenging symmetric semantic matrix decomposition and storage cost. Moreover, to support fast hash updating and avoid the propagation of binary quantization errors in online learning process, we propose to directly update the hash codes with an efficient discrete online optimization. Experiments on several public multimedia retrieval datasets validate the superiority of the proposed method from various aspects.

Multi-modal hashing is an effective technique to support large-scale multimedia retrieval, due to its capability of encoding heterogeneous multi-modal features into compact and similarity-preserving binary codes. Although great progress has been achieved so far, existing methods still suffer from several problems, including: 1) All existing methods simply adopt fixed modality combination weights in online hashing process to generate the query hash codes. This strategy cannot adaptively capture the variations of different queries. 2) They either suffer from insufficient semantics (for unsupervised methods) or require high computation and storage cost (for the supervised methods, which rely on pair-wise semantic matrix). 3) They solve the hash codes with relaxed optimization strategy or bit-by-bit discrete optimization, which results in significant quantization loss or consumes considerable computation time. To address the above limitations, in this paper, we propose an Online Multi-modal Hashing with Dynamic Query-adaption (OMH-DQ) method in a novel fashion. Specifically, a self-weighted fusion strategy is designed to adaptively preserve the multi-modal feature information into hash codes by exploiting their complementarity. The hash codes are learned with the supervision of pair-wise semantic labels to enhance their discriminative capability, while avoiding the challenging symmetric similarity matrix factorization. Under such learning framework, the binary hash codes can be directly obtained with efficient operations and without quantization errors. Accordingly, our method can benefit from the semantic labels, and simultaneously, avoid the high computation complexity. Moreover, to accurately capture the query variations, at the online retrieval stage, we design a parameter-free online hashing module which can adaptively learn the query hash codes according to the dynamic query contents. Extensive experiments demonstrate the state-of-the-art performance of the proposed approach from various aspects.

When utilising matrix factorisation to extract latent features for cross‐media retrieval, semantic information may be lost in the process of factorisation. In addition, many presented approaches directly mapped different modalities into an isomorphic semantic space to conduct the similarity measurement of different modalities, which also resulted in the loss of crucial information. To address these problems, a semantic convex matrix factorisation subspace learning approach is proposed for cross‐media retrieval between image and text. The proposed method can extract an intermediate‐level feature representation for the high dimensional image modality in order to weaken the loss of information, in the meantime, learn a semantic feature representation with semantic information for the lower dimension text modality to strengthen the discriminated capability. After that, the intermediate‐level feature representation of image is mapped into a latent semantic space by a projection matrix. Then the similarity of different modalities can be estimated in terms of uniform dimensional latent feature representations. Experimental results on three benchmark datasets demonstrate the superiority of the proposed approach over several state‐of‐the‐art approaches.

Semisupervised learning aims to leverage both labeled and unlabeled data to improve performance, where most of them are graph-based methods. However, the graph-based semisupervised methods are not capable for large-scale data since the computational consumption on the construction of graph Laplacian matrix is huge. On the other hand, the substantial unlabeled data in training stage of semisupervised learning could cause large uncertainties and potential threats. Therefore, it is crucial to enhance the robustness of semisupervised classification. In this paper, a novel large-scale robust semisupervised learning method is proposed in the framework of capped ℓ2,p-norm. This strategy is superior not only in computational cost because it makes the graph Laplacian matrix unnecessary, but also in robustness to outliers since the capped ℓ2,p-norm used for loss measurement. An efficient optimization algorithm is exploited to solve the nonconvex and nonsmooth challenging problem. The complexity of the proposed algorithm is analyzed and discussed in theory detailedly. Finally, extensive experiments are conducted over six benchmark data sets to demonstrate the effectiveness and superiority of the proposed method.

Through introducing the divide-and-conquer strategy, cooperative co-evolution (CC) has been successfully employed by many evolutionary algorithms (EAs) to solve large-scale optimization problems. In practice, it is common that different subcomponents of a large-scale problem have imbalanced contributions to the global fitness. Thus, how to utilize such imbalance and concentrate efforts on optimizing important subcomponents becomes an important issue for improving performance of cooperative co-EA, especially in distributed computing environment. In this paper, we propose a two-layer distributed CC (dCC) architecture with adaptive computing resource allocation for large-scale optimization. The first layer is the dCC model which takes charge of calculating the importance of subcomponents and accordingly allocating resources. An effective allocating algorithm is designed which can adaptively allocate computing resources based on a periodic contribution calculating method. The second layer is the pool model which takes charge of making fully utilization of imbalanced resource allocation. Within this layer, two different conformance policies are designed to help optimizers use the assigned computing resources efficiently. Empirical studies show that the two conformance policies and the computing resource allocation algorithm are effective, and the proposed distributed architecture possesses high scalability and efficiency.

Path planning is one of the most important problems in the development of autonomous underwater vehicles (AUVs). In some common AUV missions, e.g., wreckage search for rescue, an AUV is often required to traverse multiple targets in a complex environment with dense obstacles. In such case, the AUV path planning problem becomes even more challenging. In order to address the problem, this paper develops a two-layer algorithm, namely ACO-A*, by combining the ant colony optimization (ACO) with the A* search. Once a mission with a set of arbitrary targets is assigned, ACO is responsible to determine the traveling order of targets. But, prior to ACO, a cost graph indicating the necessary traveling costs among targets must be quickly established to facilitate traveling order evaluation. For this purpose, a coarse-grained modeling with a representative-based estimation (RBE) strategy is proposed. Following the order obtained by ACO, targets will be traversed one by one and the pairwise path planning to reach each target can be performed during vehicle driving. To deal with the dense obstacles, A* is adopted to plan paths based on a fine-grained modeling and an admissible heuristic function is designed for A* to guarantee its optimality. Experiments on both synthetic and realistic scenarios have been designed to validate the efficiency of the proposed ACO-A*, as well as the effectiveness of RBE and the necessity of A*.

When utilising non‐negative matrix factorisation (NMF) to decompose a data matrix into the product of two low‐rank matrices with non‐negative entries, the noisy components of data may be introduced into the matrix. Many approaches have been proposed to address the problem. Different from them, the authors consider the group sparsity and the geometric structure of data by introducing ‐norm and local structure preserving regularisation in the formulated objective function. A graph regularised sparse NMF de‐noising approach is proposed to learn discriminative representations for the original data. Since the non‐differentiability of ‐norm increases the computational cost, they propose an effective iterative multiplicative update algorithm to solve the objective function by using the Frobenius‐norm of transpose coefficient matrix. Experimental results on facial image datasets demonstrate the superiority of the proposed approach over several state‐of‐the‐art approaches.

Privacy issues have become a major concern in the web of resource sharing, and users often have difficulty managing their information disclosure in the context of high-quality experiences from social media and Internet of Things. Recent studies have shown that users’ disclosure decisions may be influenced by heuristics from the crowds, leading to inconsistency in the disclosure volumes and reduction of the prediction accuracy. Therefore, an analysis of why this influence occurs and how to optimize the user experience is highly important. We propose a novel heuristic model that defines the data structures of items and participants in social media, utilizes a modified decision-tree classifier that can predict participants’ disclosures, and puts forward a correlation analysis for detecting disclosure inconsistences. The heuristic model is applied to real-time dataset to evaluate the behavioral effects. Decision-tree classifier and correlation analysis indeed prove that some participants’ behaviors in information disclosures became decreasingly correlated during item requesting. Participants can be “persuaded” to change their disclosure behaviors, and the users’ answers to the mildly sensitive items tend to be more variable and less predictable. Using this approach, recommender systems in social media can thus know the users better and provide service with higher prediction accuracy.

This paper studies a specific class of multiobjective combinatorial optimization problems (MOCOPs), namely the permutation-based MOCOPs. Many commonly seen MOCOPs, e.g., multiobjective traveling salesman problem (MOTSP), multiobjective project scheduling problem (MOPSP), belong to this problem class and they can be very different. However, as the permutation-based MOCOPs share the inherent similarity that the structure of their search space is usually in the shape of a permutation tree, this paper proposes a generic multiobjective set-based particle swarm optimization methodology based on decomposition, termed MS-PSO/D. In order to coordinate with the property of permutation-based MOCOPs, MS-PSO/D utilizes an element-based representation and a constructive approach. Through this, feasible solutions under constraints can be generated step by step following the permutation-tree-shaped structure. And problem-related heuristic information is introduced in the constructive approach for efficiency. In order to address the multiobjective optimization issues, the decomposition strategy is employed, in which the problem is converted into multiple single-objective subproblems according to a set of weight vectors. Besides, a flexible mechanism for diversity control is provided in MS-PSO/D. Extensive experiments have been conducted to study MS-PSO/D on two permutation-based MOCOPs, namely the MOTSP and the MOPSP. Experimental results validate that the proposed methodology is promising.

Video-based pedestrian reidentification is an emerging task in video surveillance and is closely related to several real-world applications. Its goal is to match pedestrians across multiple nonoverlapping network cameras. Despite the recent effort, the performance of pedestrian reidentification needs further improvement. Hence, we propose a novel two-stream multirate recurrent neural network for video-based pedestrian reidentification with two inherent advantages: First, capturing the static spatial and temporal information; Second,Author: Figure II is not cited in the text. Please cite it at the appropriate place. dealing with motion speed variance. Given video sequences of pedestrians, we start with extracting spatial and motion features using two different deep neural networks. Then, we explore the feature correlation which results in a regularized fusion network integrating the two aforementioned networks. Considering that pedestrians, sometimes even the same pedestrian, move in different speeds across different camera views, we extend our approach by feeding the two networks into a multirate recurrent network to exploit the temporal correlations. Extensive experiments have been conducted on two real-world video-based pedestrian reidentification benchmarks: iLIDS-VID and PRID 2011 datasets. The experimental results confirm the efficacy of the proposed method. Our code will be released upon acceptance.

Spectral clustering (SC) has been proven to be effective in various applications. However, the learning scheme of SC is suboptimal in that it learns the cluster indicator from a fixed graph structure, which usually requires a rounding procedure to further partition the data. Also, the obtained cluster number cannot reflect the ground truth number of connected components in the graph. To alleviate these drawbacks, we propose a rank-constrained SC with flexible embedding framework. Specifically, an adaptive probabilistic neighborhood learning process is employed to recover the block-diagonal affinity matrix of an ideal graph. Meanwhile, a flexible embedding scheme is learned to unravel the intrinsic cluster structure in low-dimensional subspace, where the irrelevant information and noise in high-dimensional data have been effectively suppressed. The proposed method is superior to previous SC methods in that: 1) the block-diagonal affinity matrix learned simultaneously with the adaptive graph construction process, more explicitly induces the cluster membership without further discretization; 2) the number of clusters is guaranteed to converge to the ground truth via a rank constraint on the Laplacian matrix; and 3) the mismatch between the embedded feature and the projected feature allows more freedom for finding the proper cluster structure in the low-dimensional subspace as well as learning the corresponding projection matrix. Experimental results on both synthetic and real-world data sets demonstrate the promising performance of the proposed algorithm.

With the rapid development of e-commerce, logistics industry becomes a crucial component in the e-commercial ecological chain. Impelled by both economical and environmental benefit, logistics companies demand automated tools more urgently than ever. In this paper, a dynamic logistic dispatching system is proposed. The underlying model of the dispatching system is the dynamic vehicle routing problem which allows new orders being received as the working day progress. With this feature, the system becomes more practical than the systems with traditional static vehicle routing models, but is also more challenging as the vehicles must be scheduled in a dynamic way. The core of the system is a specially designed set-based particle swarm optimization algorithm. According to the characteristic of the problem, a new encoding scheme is defined by set and possibility, and a local refinement method is designed to accelerate the convergence speed of the algorithm. In addition, two more techniques: 1) region partition and 2) archive strategy are incorporated in the dispatching system to reduce the complexity of the problem and to facilitate the optimization process, helping the dispatcher control the vehicles in real time. The proposed system is tested on various benchmarks with different scales. Experimental results show that the proposed dispatching system is effective.

For several reasons, the cloud computing paradigm, e.g., mobile edge computing (MEC), is suffering from the problem of privacy issues. MEC servers provide personalization services to mobile users for better QoE qualities, but the ongoing migrated data from the source edge server to the destination edge server cause users to have privacy concerns and unwillingness of self‐disclosure, which further leads to a sparsity problem. As a result, personalization services ignore valuable user profiles across edges where users have accounts in and tend to predict users’ potential purchases with insufficient sources, thereby limiting further improvement of QoE through personalization of the contents. This paper proposes a novel model, called CEPTM, which (1) collects mobile user data across multiple MEC edge servers, (2) improves the users’ experience in personalization services by loading collected diverse data, and (3) lowers their privacy concern with the improved personalization. This model also reveals that famous topics in one edge server can migrate into several other edge servers with users’ favorite content tags and that the diverse types of items could increase the possibility of users accepting the personalization service. In the experiment section, we use exploratory factor analysis to mathematically evaluate the correlations among those factors that influence users’ information disclosure in the MEC network, and the results indicate that CEPTM (1) achieves a high rate of personalization acceptance due to the availability of more data as input and highly diverse personalization as output and (2) gains the users’ trust because it collects user data while respecting individual privacy concerns and providing better personalization. It outperforms a traditional personalization service that runs on a single‐edge server. This paper provides new insights into MEC diverse personalization services and privacy problems, and researchers and personalization providers can apply this model to merge popular users’ like trends throughout the MEC edge servers and generate better data management strategies.

The popular performance profiles and data profiles for benchmarking deterministic optimization algorithms are extended to benchmark stochastic algorithms for global optimization problems. A general confidence interval is employed to replace the significance test, which is popular in traditional benchmarking methods but suffering more and more criticisms. Through computing confidence bounds of the general confidence interval and visualizing them with performance profiles and (or) data profiles, our benchmarking method can be used to compare stochastic optimization algorithms by graphs. Compared with traditional benchmarking methods, our method is synthetic statistically and therefore is suitable for large sets of benchmark problems. Compared with some sample-mean-based benchmarking methods, e.g., the method adopted in black-box-optimization-benchmarking workshop/competition, our method considers not only sample means but also sample variances. The most important property of our method is that it is a distribution-free method, i.e., it does not depend on any distribution assumption of the population. This makes it a promising benchmarking method for stochastic optimization algorithms. Some examples are provided to illustrate how to use our method to compare stochastic optimization algorithms.

Large-scale optimization has become a significant yet challenging area in evolutionary computation. To solve this problem, this paper proposes a novel segment-based predominant learning swarm optimizer (SPLSO) swarm optimizer through letting several predominant particles guide the learning of a particle. First, a segment-based learning strategy is proposed to randomly divide the whole dimensions into segments. During update, variables in different segments are evolved by learning from different exemplars while the ones in the same segment are evolved by the same exemplar. Second, to accelerate search speed and enhance search diversity, a predominant learning strategy is also proposed, which lets several predominant particles guide the update of a particle with each predominant particle responsible for one segment of dimensions. By combining these two learning strategies together, SPLSO evolves all dimensions simultaneously and possesses competitive exploration and exploitation abilities. Extensive experiments are conducted on two large-scale benchmark function sets to investigate the influence of each algorithmic component and comparisons with several state-of-the-art meta-heuristic algorithms dealing with large-scale problems demonstrate the competitive efficiency and effectiveness of the proposed optimizer. Further the scalability of the optimizer to solve problems with dimensionality up to 2000 is also verified.

Detecting community structure has become one important technique for studying complex networks. Although many community detection algorithms have been proposed, most of them focus on separated communities, where each node can belong to only one community. However, in many real-world networks, communities are often overlapped with each other. Developing overlapping community detection algorithms thus becomes necessary. Along this avenue, this paper proposes a maximal clique based multiobjective evolutionary algorithm (MOEA) for overlapping community detection. In this algorithm, a new representation scheme based on the introduced maximal-clique graph is presented. Since the maximal-clique graph is defined by using a set of maximal cliques of original graph as nodes and two maximal cliques are allowed to share the same nodes of the original graph, overlap is an intrinsic property of the maximal-clique graph. Attributing to this property, the new representation scheme allows MOEAs to handle the overlapping community detection problem in a way similar to that of the separated community detection, such that the optimization problems are simplified. As a result, the proposed algorithm could detect overlapping community structure with higher partition accuracy and lower computational cost when compared with the existing ones. The experiments on both synthetic and real-world networks validate the effectiveness and efficiency of the proposed algorithm.

Seeking multiple optima simultaneously, which multimodal optimization aims at, has attracted increasing attention but remains challenging. Taking advantage of ant colony optimization (ACO) algorithms in preserving high diversity, this paper intends to extend ACO algorithms to deal with multimodal optimization. First, combined with current niching methods, an adaptive multimodal continuous ACO algorithm is introduced. In this algorithm, an adaptive parameter adjustment is developed, which takes the difference among niches into consideration. Second, to accelerate convergence, a differential evolution mutation operator is alternatively utilized to build base vectors for ants to construct new solutions. Then, to enhance the exploitation, a local search scheme based on Gaussian distribution is self-adaptively performed around the seeds of niches. Together, the proposed algorithm affords a good balance between exploration and exploitation. Extensive experiments on 20 widely used benchmark multimodal functions are conducted to investigate the influence of each algorithmic component and results are compared with several state-of-the-art multimodal algorithms and winners of competitions on multimodal optimization. These comparisons demonstrate the competitive efficiency and effectiveness of the proposed algorithm, especially in dealing with complex problems with high numbers of local optima.

Niching techniques have recently been incorporated into evolutionary algorithms (EAs) for multisolution optimization in multimodal landscape. However, existing niching techniques inevitably increase the time complexity of basic EAs due to the computation of the distance matrix of individuals. In this paper, we propose a fast niching technique. The technique avoids pairwise distance calculations by introducing the locality sensitive hashing, an efficient algorithm for approximately retrieving nearest neighbors. Individuals are projected to a number of buckets by hash functions. The similar individuals possess a higher probability of being hashed into the same bucket than the dissimilar ones. Then, interactions between individuals are limited to the candidates that fall in the same bucket to achieve local evolution. It is proved that the complexity of the proposed fast niching is linear to the population size. In addition, this mechanism induces stable niching behavior and it inherently keeps a balance between the exploration and exploitation of multiple optima. The theoretical analysis conducted in this paper suggests that the proposed technique is able to provide bounds for the exploration and exploitation probabilities. Experimental results show that the fast niching versions of the multimodal algorithms can exhibit similar or even better performance than their original ones. More importantly, the execution time of the algorithms is significantly reduced.

Existing differential evolution (DE) algorithms often face two challenges. The first is that the optimization performance is significantly affected by the ad hoc configurations of operators and parameters for different problems. The second is the long runtime for real-world problems whose fitness evaluations are often expensive. Aiming at solving these two problems, this paper develops a novel double-layered heterogeneous DE algorithm and realizes it in cloud computing distributed environment. In the first layer, different populations with various parameters and/or operators run concurrently and adaptively migrate to deliver robust solutions by making the best use of performance differences among multiple populations. In the second layer, a set of cloud virtual machines run in parallel to evaluate fitness of corresponding populations, reducing computational costs as offered by cloud. Experimental results on a set of benchmark problems with different search requirements and a case study with expensive design evaluations have shown that the proposed algorithm offers generally improved performance and reduced computational time, compared with not only conventional and a number of state-of-the-art DE variants, but also a number of other distributed DE and high-performing evolutionary algorithms. The speedup is significant especially on expensive problems, offering high potential in a broad range of real-world applications.

How to choose the optimal parameter is crucial for the kernel method, because kernel parameters perform significantly on the kernel method. In this paper, a novel approach is proposed to choose the kernel parameter for the kernel nearest-neighbor classifier (KNN). The values of the kernel parameter are computed through optimizing an object function designed for measuring the classification reliability of KNN. We test our approach on both artificial and real-word datasets, and the preliminary results demonstrate that our approach provides a practical solution.