Recent deepfake detection studies often treat unseen sample detection as a ``zero-shot" task, training on images generated by known models but generalizing to unknown ones. A key real-world challenge arises when a model performs poorly on unknown samples, yet these samples remain available for analysis. This highlights that it should be approached as a ``few-shot" task, where effectively utilizing a small number of samples can lead to significant improvement. Unlike typical few-shot tasks focused on semantic understanding, deepfake detection prioritizes image realism, which closely mirrors real-world distributions. In this work, we propose the Few-shot Training-free Network (FTNet) for real-world few-shot deepfake detection. Simple yet effective, FTNet differs from traditional methods that rely on large-scale known data for training. Instead, FTNet uses only one fake sample from an evaluation set, mimicking the scenario where new samples emerge in the real world and can be gathered for use, without any training or parameter updates. During evaluation, each test sample is compared to the known fake and real samples, and it is classified based on the category of the nearest sample. We conduct a comprehensive analysis of AI-generated images from 29 different generative models and achieve a new SoTA performance, with an average improvement of 8.7% compared to existing methods. This work introduces a fresh perspective on real-world deepfake detection: when the model struggles to generalize on a few-shot sample, leveraging the failed samples leads to better performance.

Deep cross-modal hashing has achieved remarkable success in cross-modal retrieval due to its fast retrieval speed and low storage cost, but it highly vulnerable to adversarial attacks. Mainstream defense methods rely on adversarial training, which often induces a robustness–standard performance trade-off, leading to the learning of only limited robust features and degraded standard performance. To address this issue, we propose a Spatial-Frequency Domain Complementary Learning (SFCL) framework to overcome these two challenges by: 1) exploiting the complementarity of spatial and frequency features to learn more comprehensive and adversarially robust features, addressing the limited robustness of existing defenses; 2) by supplementing frequency-domain information, it avoids the performance degradation commonly caused by adversarial training. Specifically, SFCL consists of two modules: a Spatial-Frequency Robust Gating (SFRG) module, which selects robust features and strengthens complementarity via a conditional mutual information-based loss; and a Robustness-Aware Feature Fusion (RAFF) module, which performs bidirectional feature interaction and fusion. Extensive experiments demonstrate significant robustness gains over existing state-of-the-art methods, along with improved standard performance.

Deep hashing is widely used in large-scale image retrieval systems due to its efficient retrieval performance. However, its susceptibility to adversarial attacks limits its security in practical applications. Adversarial training is the most effective method for improving robustness, but it often leads to a significant trade-off between robustness and retrieval accuracy. In this paper, we conduct spectral analysis and find that generating high-quality hash codes requires wide-frequency response models, whereas adversarial training forces the model into spectral collapse, degrading it to a low-frequency response model and weakening its discriminability. To address this issue, we propose a Spectral-Adaptive Adversarial Hashing (SAAH) framework, which selectively preserves discriminative and task-relevant frequency components while suppressing adversarially unstable ones, enabling robust hashing without sacrificing retrieval performance. Extensive experiments on benchmark datasets demonstrate that SAAH consistently achieves a superior balance between retrieval accuracy and adversarial robustness, achieving the best performance in both retrieval accuracy and robustness compared with existing robust hashing methods.

Human-Object Interaction (HOI) detection requires not only recognizingwhatthe interaction is but also understandingwhereit occurs. Although recent methods have achieved remarkable progress, they often lack effective joint modeling of spatial and semantic information, which is essential for accurate reasoning in complex scenes. In this paper, we propose a Semantic-Spatial Guided Reasoning (SSGR) framework that performs interaction reasoning by jointly modeling global semantic cues and fine-grained spatial priors. Specifically, SSGR constructs pair-specific spatial layouts to encode detailed spatial relationships and introduces a global semantic decoder to learn category-aware semantic representations. A semantic-spatial guided reasoning module further adaptively fuses these complementary cues, enabling unified reasoning and more discriminative interaction understanding. Extensive experiments on HICO-DET and V-COCO demonstrate that SSGR consistently outperforms prior methods under both standard and zero-shot settings, validating the effectiveness of our semantic-spatial reasoning paradigm.

Safety concerns remain one of the most significant obstacles to the large-scale deployment and continued advancement of autonomous driving (AD) systems. A major underlying cause of many safety-related incidents in AD systems is suboptimal or erroneous decision-making when the vehicle encounters corner cases (CCs)—rare, unexpected, or extreme situations that fall outside typical operating conditions. Although recent advances in artificial intelligence have driven substantial progress in both autonomous driving and corner-case research, the field still lacks a coherent conceptual foundation and a systematic, widely accepted categorization of CCs. In this survey, we address this gap by offering a structured review of the existing corner-case literature along three key dimensions: understanding, detection, and generation. The key contribution is a three-level classification of corner cases—spanning data-level, model-level, and semantic-level CCs—that clarifies and disambiguates competing definitions and perspectives on AD corner cases. Building on this framework, we examine simulator-based methods for corner-case selection and data generation, and we identify open challenges, promising directions, and potential solutions to more effectively handle corner cases in autonomous driving systems.

Image fusion aims to integrate multi-modal data, enhancing information completeness while reducing redundancy. However, image fusion faces the challenge of information imbalance. This imbalance is reflected in both intra-image variations, where different regions exhibit significant differences in information density and importance, and cross-modal inconsistencies, where corresponding regions in different modalities contribute unequally to the fused result. Most existing image fusion methods, such as models based on CNN and window attention mechanism, adopt a fixed approach that collects the same amount of neighboring information for each pixel. However, this fixed approach lacks adaptability to local variations in information density, ultimately limiting fusion performance. To this end, we propose IA2GNN, an imbalance-aware adaptive graph neural network designed for image fusion. To tackle intra- and cross-modal information imbalance, we leverage the flexibility of graph structures to dynamically adjust node connections during feature extraction and fusion. Specifically, we adjust the connectivity of nodes in the graph to allow regions with rich details to establish more connections, thereby enhancing the extraction and preservation of key information in these regions. For less informative regions, we reduce connections to prevent over-modeling. In this way, we optimize information distribution, achieving a balance between detail retention and information integration in the fusion results. Experimental results demonstrate that the proposed model outperforms baseline methods across multiple image fusion tasks, achieving up to +0.058 improvement in SSIM and +0.351 gain in MI.

Spatiotemporal Graph Learning (SGL) under Zero-Inflated Distribution (ZID) is crucial for urban risk management tasks, including crime prediction and traffic accident profiling. However, SGL models are vulnerable to adversarial attacks, compromising their practical utility. While adversarial training (AT) has been widely used to bolster model robustness, our study finds that traditional AT exacerbates performance disparities between majority and minority classes under ZID, potentially leading to irreparable losses due to underreporting critical risk events. In this paper, we first demonstrate the smaller top-k gradients and lower separability of minority class are key factors contributing to this disparity. To address these issues, we propose MinGRE, a framework for Minority Class Gradients and Representations Enhancement. MinGRE employs a multi-dimensional attention mechanism to reweight spatiotemporal gradients, minimizing the gradient distribution discrepancies across classes. Additionally, we introduce an uncertainty-guided contrastive loss to improve the inter-class separability and intra-class compactness of minority representations with higher uncertainty. Extensive experiments demonstrate that the MinGRE framework not only significantly reduces the performance disparity across classes but also achieves enhanced robustness compared to existing baselines. These findings underscore the potential of our method in fostering the development of more equitable and robust models.

Recent advancements in Multimodal Large Language Models (MLLMs) have shown remarkable capabilities across various multimodal contexts. However, their application in robotic scenarios, particularly for long-horizon manipulation tasks, reveals significant limitations. These limitations arise from the current MLLMs lacking three essential robotic brain capabilities: Planning Capability, which involves decomposing complex manipulation instructions into manageable sub-tasks; Affordance Perception, the ability to recognize and interpret the affordances of interactive objects; and Trajectory Prediction, the foresight to anticipate the complete manipulation trajectory necessary for successful execution. To enhance the robotic brain’s core capabilities from abstract to concrete, we introduce ShareRobot, a high-quality heterogeneous dataset that labels multi-dimensional information such as task planning, object affordance, and end-effector trajectory. ShareRobot’s diversity and accuracy have been meticulously refined by three human annotators. Building on this dataset, we developed RoboBrain, an MLLM-based model that combines robotic and general multi-modal data, utilizes a multi-stage training strategy, and incorporates long videos and high-resolution images to improve its robotic manipulation capabilities. Extensive experiments demonstrate that RoboBrain achieves state-of-the-art performance across various robotic tasks, highlighting its potential to advance robotic brain capabilities. Project website: RoboBrain.

Text generated by Large Language Models (LLMs) now rivals human writing, raising concerns about its misuse.However, mainstream AI-generated text detection (AGTD) methods primarily target document-level long texts and struggle to generalize effectively to sentencelevel short texts.And current sentence-level AGTD (S-AGTD) research faces two significant limitations: (1) lack of a comprehensive evaluation on complex human-AI hybrid content, where human-written text (HWT) and AI-generated text (AGT) alternate irregularly, and (2) failure to incorporate contextual information, which serves as a crucial supplementary feature for identifying the origin of the detected sentence.Therefore, in our work, we propose AutoFill-Refine, a high-quality synthesis strategy for human-AI hybrid texts, and then construct a dedicated S-AGTD benchmark dataset.Besides, we introduce SenDe-tEX, a novel framework for sentence-level AIgenerated text detection via style and context fusion.Extensive experiments demonstrate that SenDetEX significantly outperforms all baseline models in detection accuracy, while exhibiting remarkable transferability and robustness.

Deep Neural Networks (DNNs) play a crucial role in the smart Internet of Things (IoT), with widespread applications in inference tasks like interactive games, intelligent driving, and augmented reality. Along with these promising applications, various task-offloading methods were proposed to improve the utilization of system resources, given that DNN model inference typically requires substantial computational power. However, existing offloading methods focus primarily on a specific model, and research addressing heterogeneous DNN models (with different structures and layers) remains limited in practical IoT environments. Directly integrating these methods would require frequent re-initialization to adapt to changes in the search space during the offloading of mixed heterogeneous DNN inference tasks, resulting in insufficient flexibility and the waste of computational resources. Thus, we propose AutoCut, a global heterogeneous model offloading service based on a customized multi-objective differential evolution algorithm, to find low-latency and energyefficient offloading partitions. AutoCut utilizes a group-layer granularity partitioning that avoids frequent changes in the search space when continuously offloading heterogeneous DNN inference tasks, thereby improving search efficiency. Experiments with six popular models show that AutoCut significantly improves inference performance regarding latency and energy efficiency.

Vision-Language Models (VLMs) play a crucial role in the advancement of Artificial General Intelligence (AGI). As AGI rapidly evolves, addressing security concerns has emerged as one of the most significant challenges for VLMs. In this paper, we present extensive experiments that expose the vulnerabilities of conventional adaptation methods for VLMs, highlighting significant security risks. Moreover, as VLMs grow in size, the application of traditional adversarial adaptation techniques incurs substantial computational costs. To address these issues, we propose a parameter-efficient adversarial adaptation method called AdvLoRA based on Low-Rank Adaptation. We investigate and reveal the inherent low-rank properties involved in adversarial adaptation for VLMs. Different from LoRA, we enhance the efficiency and robustness of adversarial adaptation by introducing a novel reparameterization method that leverages parameter clustering and alignment. Additionally, we propose an adaptive parameter update strategy to further bolster robustness. These innovations enable our AdvLoRA to mitigate issues related to model security and resource wastage. Extensive experiments confirm the effectiveness and efficiency of AdvLoRA.

Accurate stock market prediction is crucial for investors seeking significant profits. With increased economic activity, various interrelations between listed companies have become important for accurate predictions. These relations can be represented as complex financial networks, aiding the development of effective graph neural network (GNN) prediction methods. However, current GNN-based methods for stock prediction typically rely on a single static network representation, which fails to capture the dynamic and multifaceted relationships inherent in financial markets. In this article, we propose the multilayer spatial–temporal graph neural network (MST-GNN) to model the complex and evolving interactions between stocks. The MST-GNN framework incorporates a novel spatial–temporal cross-layer high-order fusion mechanism, which includes two key components: spatial–temporal neighborhood aggregation and cross-layer high-order feature fusion. These components enable the model to effectively capture both the temporal evolution and cross-network feature interactions of stocks. Our extensive experiments on four stock networks from the China A-share market demonstrate that MST-GNN significantly outperforms existing GNN-based methods on stock price trend classification and return ranking tasks.

Lane detection is a critical technology for autonomous driving, but current deep learning-based methods face significant challenges due to the lack of diverse datasets, especially for nighttime conditions. Most datasets are predominantly composed of daytime images, making it difficult to develop models that perform reliably around the clock. Inspired by the parallel system theory, we explore a novel approach to generate comprehensive 24-hour datasets from daytime images alone. In this paper, we propose the Parallel Scene Information Collaboration (PSIC) framework, designed to enhance 24-hour lane detection using only daytime data. The PSIC framework consists of three key components: artificial scene generation, information collaboration, and lane line detection. First, we address the limitations of existing datasets by proposing two generators—one that transforms daytime images into realistic nighttime scenes, and another that refines nighttime images by adding daytime characteristics. Next, to mitigate noise in the generated scenes, we propose a Multi-Spatial Feature Fusion (MSFF) module that effectively integrates features from both real and artificial scenes through spatial collaboration. Finally, the combined information is used by an anchor-based detection head to accurately identify lane positions. Our experiments on the TuSimple, Night TuSimple, and CULane datasets demonstrate that our method achieves state-of-the-art performance in 24-hour lane line detection, significantly improving reliability and robustness across varying conditions.

3-D multiobject tracking (MOT) is an important task in numerous applications, including robotics and autonomous driving. Nevertheless, existing 3-D MOT solutions suffer from significant performance degradation under adverse weather conditions. Inspired by the fact that hard objects (e.g., missed detections or wrongly-associated objects) are more constructive to performance improvement, in this article, we leverage hard samples for robust 3-D MOT in adverse weather conditions. Specifically, we implement a cross-modality 3-D MOT framework to learn the 3-D region proposals from point clouds and RGB images, respectively. To minimize the risk of missed detection and wrong association, we introduce an adaptive hard sample mining scheme to align the 3-D region proposals provided by two modalities. We quantify the hard level by comparing the confidence values of the same object in the two branches and their distance in the embedding space. Meanwhile, we dynamically adjust the weights of hard samples during training to enhance the representation learning for robust 3-D MOT. Extensive experimental results showcase that our proposed solution effectively mitigates the missed detection and reduces wrong association with good generalization.

With the rise and prevalence of social bots, their negative impacts on society are gradually recognized, prompting research attention to effective detection and countermeasures. Recently, graph neural networks (GNNs) have flourished and have been applied to social bot detection research, improving the performance of detection methods effectively. However, existing GNN-based social bot detection methods often fail to account for the heterogeneous associations among users within social media contexts, especially the heterogeneous integration of social bots into human communities within the network. To address this challenge, we propose a heterogeneous compatibility perspective for social bot detection, in which we preserve more detailed information about the varying associations between neighbors in social media contexts. Subsequently, we develop a compatibility-aware graph neural network (CGNN) for social bot detection. CGNN consists of an efficient feature processing module, and a lightweight compatibility-aware GNN encoder, which enhances the model’s capacity to depict heterogeneous neighbor relations by emulating the heterogeneous compatibility function. Through extensive experiments, we showed that our CGNN outperforms the existing state-of-the-art (SOTA) method on three commonly used social bot detection benchmarks while utilizing only about 2% of the parameter size and 10% of the training time compared with the SOTA method. Finally, further experimental analysis indicates that CGNN can identify different edge categories to a significant extent. These findings, along with the ablation study, provide strong evidence supporting the enhancement of GNN’s capacity to depict heterogeneous neighbor associations on social media bot detection tasks.

Welcome to the second issue of IEEE Transactions on Computational Social Systems (TCSS) of 2024. This issue showcases an impressive array of 104 regular papers alongside our Special Issue on Big Data and Computational Social Intelligence for Guaranteed Financial Security, highlighting cutting-edge research aimed at harnessing big data and computational techniques to fortify financial security amidst the digital finance evolution. With a focus on addressing the intricate challenges of financial big data, enhancing the efficacy of artificial intelligence, and covering critical topics from data mining to digital currencies, this issue underscores the vital role of cross-disciplinary efforts in mitigating financial security risks.

The insurance industry plays a crucial role of the national financial system, and the operating efficiency of insurance companies has always been a significant subject of academic research. This study proposes a multiperiod DEA model to dynamically evaluate the operating efficiency of insurance companies, which not only overcomes the defect of the traditional DEA method ignoring the internal structure of decision-making units, but also extends the limitation of the leader–follower model in evaluating single-period efficiency. By analyzing the efficiency of seven listed insurance companies in China from 2009 to 2018, the following conclusions are drawn. The multiperiod DEA model demonstrates advantages over the leader–follower model. The profitability of insurance companies during the first five years is higher compared with the last five years. There is a significant correlation between premium financing efficiency and overall efficiency. Throughout both periods, the loss ratio, loss reserve ratio, and consumer price index (CPI) are always positively correlated with the efficiency of the insurer, while the gearing ratio is negatively related to the efficiency of the insurer. The correlation among gross domestic product (GDP), total insurance value, tradable financial assets, and corporate efficiency varies over time.

In the past few years, we have witnessed the rapid development and exponential growth of generative artificial intelligence (GAI) technologies including large language models (LLMs)-enabled ChatGPT and peripheral innovations. These technologies are designed to be humanlike intelligence and intuitive by providing direct access to systems using application programming interfaces (APIs). The GAI applications can fundamentally change economic and financial activities, through revolutionizing the ways that humans interact with machines and giving rise to new modes of production and behavior patterns. It is imperative to develop a new research paradigm that is more suitable than the currently dominating conventional research paradigms. This article presents the new paradigm for economic and financial research with GAI, covering the research objectives, scientific data, and models, and explores the underlying impact and perspective that bring to this field. We elaborate on the potential five scenarios including portfolio management, economic and financial prediction, extreme scenario analysis, policy analysis, and financial fraud detection. The new research paradigm with GAI proposed in this article can provide significant insights for a comprehensive understanding of innovation and transformation in this domain.

Video Visual Relation Detection (VidVRD) is a pivotal task in the field of video analysis. It involves detecting object trajectories in videos, predicting potential dynamic relation between these trajectories, and ultimately representing these relationships in the form oftriplets. Correct prediction of relation is vital for VidVRD. Existing methods mostly adopt the simple fusion of visual and language features of entity trajectories as the feature representation for relation predicates. However, these methods do not take into account the dependency information between the relation predication and the subject and object within the triplet. To address this issue, we propose the entity dependency learning network(EDLN), which can capture the dependency information between relation predicates and subjects, objects, and subject-object pairs. It adaptively integrates these dependency information into the feature representation of relation predicates. Additionally, to effectively model the features of the relation existing between various object entities pairs, in the context encoding phase for relation predicate features, we introduce a fully convolutional encoding approach as a substitute for the self-attention mechanism in the Transformer. Extensive experiments on two public datasets demonstrate the effectiveness of the proposed EDLN.

The robustness of machine learning (ML) models has gained much attention along with their wide application on various safety-required Industrial Internet of Things (IIoT) paradigms. Researchers found that some specific attacks added on sensor measurements can maliciously disturb IIoT monitors that are designed using ML architectures. The Traditional detection methods could judge whether the measurements are attacked to prevent the failure of monitors. Unfortunately, recent works argue that the commonly used detection methods could be circumvented through adaptive attacks that could acquire the mechanism of detectors; they could not truly enhance the robustness of ML models. Instead, general robust mechanisms should be performed to authentically enhance the robustness of models against any potential attacks with specific restrictions. On the basis of the above argument, we design a robust condition monitor for predicting the fault condition of IIoT systems using the adversarial training technique called robust temporal convolutional network (RTCN). The model is designed to be formally robust to attacks with restricted magnitude. The temporal convolutional network (TCN) is employed to design the base structure of the monitor. TCN can capture temporal information from sensors to enhance the feature extraction performance of models. We also present a novel false data injection (FDI) attack-generating method that utilizes the conception of adversarial perturbations to disturb well-trained monitors. The experimental results verify the efficiency of feature extraction performance of our model from IIoT systems. Furthermore, adversarial training mechanism through a min–max manner could effectively improve the reliability of ML-based IIoT monitors against strong FDI attacks.

By regarding the Chinese financial and economic sectors as a system, this article studies the stock volatility spillover in the system and explores its effects on the overall performance of the macroeconomy in China. The recent outbreak of COVID-19, U.S.–China trade friction, and three historical financial turbulences are involved to distinguish the changes in the spillover in these distinct crises, which has seldom been unveiled in the literature. By considering that the stock volatility spillover may vary over distinct timescales, the spillovers are disclosed through innovatively constructing the multi-scale spillover networks, followed by connectedness computation, based on variational mode decomposition (VMD) and generalized vector autoregression (GVAR) process. Our empirical analysis first demonstrates the different levels of increases in the total sectoral volatility spillover and changes in the roles of the sectors in the system under the aforementioned crises. Besides, the increases in the sectoral spillover in the long-term are verified to negatively impact the macroeconomy and can thereby act as warning signals.

The multimodality nature of web data has necessitated complex multimodal information retrieval for a wide range of web applications. Deep neural networks (DNNs) have been widely employed to extract semantic features from raw samples to improve retrieval accuracy. In addition, hashing is widely used to improve computational and storage efficiency. As such, deep hashing frameworks have been applied for multimodal retrieval tasks. However, there is still a great recognitive gap between primate brain structure-inspired DNNs and humans. On computer vision tasks, well-crafted DNN models can be easily defeated by invisible small attacks, and this phenomenon indicates a large recognition gap between DNN models and humans. Recently, adversarial defense methods have been shown to improve the human–machine recognition alignment in several classification tasks. However, the robustness problem on the retrieval tasks, especially on the deep hashing-based multimodal retrieval models, is still not well studied. Therefore, in this article, we present an adversarially robust training mechanism to improve model robustness for the purpose of human–machine recognition alignment on retrieval tasks. Through extensive experimental results on several social multimodal retrieval benchmarks, we show that the robust training hashing framework proposed can mitigate the recognition gap on retrieval tasks. Our study highlights the necessity of robustness enhancement on deep hashing models.

The wide application of deep neural networks (DNNs) has significantly improved the performance of hashing models on multimodal retrieval issues. DNN-based deep models can automatically learn semantic features from raw data to make human-level decisions. However, the superior generalization leads to potential privacy leakage risks. Strong DNN-based retrieval models enable malicious crawlers to search for nontag private information based on semantic similarity matching. Hence, executing effective privacy protection mechanisms against those retrieval software is essential for reliable social website construction. In this article, we propose a retrieval task-based adversarial perturbation generation method called Hashing Fake to meet this request. Specifically, DNNs are recently found to be vulnerable to a specific set of attacks called adversarial perturbations, which denote some magnitude-restricted signals added on objective samples to misguide well-crafted DNN models, and perturbations’ magnitudes are small enough that will not induce humans’ perception. Moreover, since existing adversarial perturbation generation methods are designed for supervised tasks, Hashing Fake constructs a differential approximation substitution for perturbation production on unsupervised retrieval tasks. Through extensive experiments on several deep retrieval benchmarks, we demonstrate that well-crafted perturbations using Hashing Fake can effectively misguide objective models’ recognitions to make false predictions. The added norm-restricted perturbations on objective samples will not alter humans’ perception; hence, Hashing Fake can be applied on real-world social websites to protect subscribers’ privacy against malicious retrieval software.

Investment decisions and risk management require understanding the time-varying dependencies between stocks. Graph-based learning systems have emerged as a promising approach for predicting stock prices by leveraging interfirm relationships. However, existing methods rely on a static stock graph predefined from finance domain knowledge and large-scale data engineering, which overlooks the dynamic dependencies between stocks. In this article, we present a novel framework called graph evolution recurrent unit (GERU), which uses a dynamic graph neural network to automatically learn the evolving dependencies from historical stock features, leading to better predictions. Our approach consists of three parts: first, we develop an adaptive dynamic graph learning (ADGL) module to learn latent dynamic dependencies from stock time series. Second, we propose a clustered ADGL (clu-ADGL) to handle large-scale time series by reducing time and memory complexity. Third, we combine the ADGL/clu-ADGL with a graph-gated recurrent unit to model the temporal evolutions of stock networks. Extensive experiments on real-world datasets show that our proposed methods outperform existing methods in predicting stock movements, capturing meaningful dynamic dependencies and temporal evolution patterns from the financial market, and achieving outstanding profitability in portfolio construction.

The aim of cross-modal retrieval is to search for flexible results across different types of multimedia data. However, the labeled data is usually limited and not well paired with different modalities in practical applications. These issues are not well addressed in the existing works, which cannot consider the semantic information about unlabeled and unpaired data, synchronously. Self-training is a well-known strategy to handle semi-supervised problems. Motivated by the self-training, this paper proposes a self-training-based cross-modal hashing framework (STCH) to tackle the semi-supervised and semi-paired challenges. In the framework, graph neural networks are used to capture potential intra-modality and inter-modality similarities to produce pseudo labels. Then the inconsistent pseudo labels of different modalities are refined with a heuristic filter to enhance the model robustness. To train STCH, we propose an alternating learning strategy to conduct the self-train by predicting pseudo labels during the training procedure, which can be seamlessly incorporated into semi-supervised and supervised learning. In this way, the proposed method can leverage sufficient semantic information to enhance the semi-supervised effect and address the semi-paired problem. Experiments on the real-world datasets demonstrate that our approach outperforms related methods on hash cross-modal retrieval.

Co-movement among individual firms’ stock prices can reflect complex interfirm relationships. This paper proposes a novel method to leverage such relationships for stock price predictions by adopting inductive graph representation learning on dynamic stock graphs constructed based on historical stock price co-movement. To learn node representations from such dynamic graphs for better stock predictions, we propose the hybrid-attention dynamic graph neural network, an inductive graph representation learning method. We also extended mini-batch gradient descent to inductive representation learning on dynamic stock graphs so that the model can update parameters over mini-batch stock graphs with higher training efficiency. Extensive experiments on stocks from different markets and trading simulations demonstrate that the proposed method significantly improves stock predictions. The proposed method can have important implications for the management of financial portfolios and investment risk. Summary of Contribution: Accurate predictions of stock prices have important implications for financial decisions. In today’s economy, individual firms are increasingly connected via different types of relationships. As a result, firms’ stock prices often feature synchronous co-movement patterns. This paper represents the first effort to leverage such phenomena to construct dynamic stock graphs for stock predictions. We develop hybrid-attention dynamic graph neural network (HAD-GNN), an inductive graph representation learning framework for dynamic stock graphs to incorporate temporal and graph attention mechanisms. To improve the learning efficiency of HAD-GNN, we also extend the mini-batch gradient descent to inductive representation learning on such dynamic graphs and adopt a t-batch training mechanism (t-BTM). We demonstrate the effectiveness of our new approach via experiments based on real-world data and simulations.

Detecting product adoption intentions on social media could yield significant value in a wide range of applications, such as personalized recommendations and targeted marketing. In the literature, no study has explored the detection of product adoption intentions on social media, and only a few relevant studies have focused on purchase intention detection for products in one or several categories. Focusing on a product category rather than a specific product is too coarse-grained for precise advertising. Additionally, existing studies primarily focus on using one type of text representation in target social media posts, ignoring the major yet unexplored potential of fusing different text representations. In this paper, we first formulate the problem of product adoption intention mining and demonstrate the necessity of studying this problem and its practical value. To detect a product adoption intention for an individual product, we propose a novel and general multiview deep learning model that simultaneously taps into the capability of multiview learning in leveraging different representations and deep learning in learning latent data representations using a flexible nonlinear transformation. Specifically, the proposed model leverages three different text representations from a multiview perspective and takes advantage of local and long-term word relations by integrating convolutional neural network (CNN) and long short-term memory (LSTM) modules. Extensive experiments on three Twitter datasets demonstrate the effectiveness of the proposed multiview deep learning model compared with the existing benchmark methods. This study also significantly contributes research insights to the literature about intention mining and provides business value to relevant stakeholders such as product providers.

In the present study, a two-stage data envelopment analysis (DEA) model and spatial econometric method were employed to evaluate and analyze the utilization efficiency of urban water resources and spatial–temporal differences in cities of China. The traditional DEA model was enhanced by adopting the Shannon entropy in the first stage. After selecting variables based on the previous step and the Bayes information criterion (BIC), redundant variables were removed. In the meanwhile, a comprehensive efficiency score (CES) was generated to rank the efficiency. Finally, spatial econometric analysis was applied to explore the spatial–temporal differences of urban water resource utilization efficiency. Results demonstrate that: 1) according to the calculations and analysis, communities should concentrate on increasing investment in equipment and technology that can help enhance water consumption efficiency, while overlooking some minor aspects, such as per capita gross domestic product (PCGDP); 2) most cities have poor water resource utilization efficiency (low CES). However, both input and output have much room for the improvement; 3) Lhasa, Beijing, Haikou, and Shanghai have high CES, indicating that the utilization efficiency of water resources is not entirely dependent on economic development; and 4) through performing the Lagrange multiplier (LM) test, the spatial error model (SEM) test is passed at the significant level of 5%. Moreover, the water resource utilization efficiency of a city may be enhanced with the economic development in neighboring cities.

In January 2021, the users of subreddit r/wallstreetbets (WSB) triggered an unprecedented short squeeze by driving up GameStop’s stock price to an unimaginable high point. During the event, a large number of users participated in the discussion about GameStop and coordinated trading behavior on r/WSB to push the stock price higher. In this article, we investigate the characteristics of the collective behaviors and social dynamics from the evolutions of topological structure, discussed topics, and user sentiment polarity (SP) by constructing dynamic interaction networks, modeling the topic, and analyzing the user sentiment. We find that the topological structure of the interaction network evolves toward a more efficient direction, the discussed topics change more centralized, and the user sentiment tends to be more positive and divergent. And we reveal that part of GameStop’s stock price is explained by the social media activity, popularity of the dominant topic, topic cohesiveness, SP of users, and sentiment divergence between interacted users on r/WSB. Our work quantitatively characterizes the interaction networks and user behavior during the GameStop short squeeze and provides an example to analyze the event which synchronously evolves in the physical space and cyberspace. It not only contributes to the analysis of social system behavior and structure but also provides valuable insights into the financial practice and policy decision-making.

As cybercrimes grow in scale with devastating economic costs, it is important to protect potential victims against diverse attacks. It is the uniform resource locators (URLs) that connect vulnerable users with potential attacks. Although numerous solutions (e.g., rule-based solutions and machine learning-based methods) are proposed for malicious URL detection, they can not provide robust performance due to the diversity of cybercrimes and can not cope with the explosive growth of malicious URLs with the evolution of obfuscation strategies. In this paper, we propose a deep learning-based system, dubbed as CyberLen, to detect malicious URLs robustly and effectively. Specifically, we use factorization machine (FM) to learn the latent interaction among lexical features. For the deep structural features, position embedding is introduced for token vectorization to reduce the ambiguity of URL tokens. Meanwhile, temporal convolution network (TCN) is utilized to learn the long-distance dependency among URL tokens. To fuse heterogeneous features, self-paced wide & deep learning strategy is proposed to train a robust model effectively. The proposed solution is evaluated on a large-scale URL dataset. Our experimental results show that position embedding is constructive to reducing the ambiguity of URL tokens, and the self-paced wide & deep learning strategy shows superior performance in terms of F1 score and convergence speed.

As an important national initiative, China’s public transport priority development strategy is conducive to the development of the public transport industry and urban economic construction. However, current public transport operation is inefficient due to the information asymmetry between the government and public transport enterprises, thereby inevitably generating losses. To address the issue of information asymmetry, this study designs a public transport subsidy allocation method from the perspective of incentives and discusses the allocation results using this method through a case study. The rationality of the proposed method and the scientificity and authenticity of the conclusions, are fully explained and demonstrated via the experimental simulation. Based on real published data of public transport enterprises, the proposed method can motivate public transport enterprises to improve their performance. In the case of reporting false data, the proposed method can motivate public transport enterprises to report accurate data.

The mass media has recorded major events around the world for a long time, which is very helpful in describing the dynamic changes in all aspects of human society, including the analysis of national influence using news data. Due to the publicity and significance of mass media, the results of influence analysis must be reliable. However, the current most influence analysis methods are mainly concentrated on social media networks and cannot simply be transferred to mass media. Due to the causality as the main driving factor of influence, we introduced the causal inference method convergent cross mapping, combined with the existing general influence analysis method, proposed a credible influence analysis method in mass media. This method can filter out non-causal influences, making the results more credible. We conducted experiments on the GDELT datasets, and the results proved the effectiveness and reliability of the proposed credible influence analysis in mass media.

The rapid development of Internet and online data explosions elicit strong aspirations for users to search for semantic relevant information based on available samples. While the data online are always released with different modalities like images, videos or texts, effective retrieval models should discover latent semantic information with different structures. Recently, the state-of-the-art deep cross modal retrieval frameworks have effectively enhanced the performance on commonly-used platforms using the deep neural networks (DNNs). Yet DNNs have been verified to be easily misguided by small perturbations, and there are already several attack generation methods proposed on DNN-based models for real-world tasks, but they are all focused on supervised tasks like classification or object recognition. To effectively evaluate the robustness of deep cross-modal retrieval frameworks, in this paper, we propose a retrieval-based adversarial perturbation generation method, and demonstrate that our perturbation could effectively attack the state-of-the-art deep cross-modal and single image retrieval hashing models.

In the past few years, with the rapid growth of digital technologies, Facebook, Twitter, and other social media platforms have become the digital oligarchies, which have the enormous capabilities to potentially control what is discussed in cyberspace. In the digital oligarchy era, social perception and social influence in different complex social systems have evolved quickly. In this article, we conducted large-scale empirical studies on social perception and social influence regarding the Trump phenomenon from personal perception, media, and public attention perspectives. We found that there exist obvious correlations between the posting behavior of Trump and the attention of news media. By constructing public attention networks using complex networks based on Google search information, we further reveal that digital platforms could affect social perception and social influence significantly. Especially, we obtained that the public attention can always be influenced by the political moments.

With the proliferation of Internet of Things (IoT) devices in the consumer market, the unprecedented sensing capability of IoT devices makes it possible to develop advanced sensing and complex inference tasks by leveraging heterogeneous sensors embedded in IoT devices. However, the limited power supply and the restricted computation capability make it challenging to conduct seamless sensing and continuous inference tasks on resource-constrained devices. How to conduct energy-efficient sensing and perform rich-sensor inference tasks on IoT devices is crucial for the success of IoT applications. Therefore, we propose a novel energy-efficient collaborative sensing framework to optimize the energy consumption of IoT devices. Specifically, we explore the latent correlations among heterogeneous sensors via an attention mechanism in temporal convolutional network to quantify the dependency among sensors, and characterize the heterogeneous sensors in terms of energy consumption to categorize them into low-power sensors and energy-intensive sensors . Finally, to decrease the sampling frequency of energy-intensive sensors , we propose a multi-task learning strategy to predict the statuses of energy-intensive sensors based on the low-power sensors . To evaluate the performance of the proposed collaborative sensing framework, we develop a mobile application to collect concurrent heterogeneous data streams from all sensors embedded in Huawei Mate 8. The experimental results show that latent correlation learning is greatly helpful to understand the latent correlations among heterogeneous streams, and it is feasible to predict the statuses of energy-intensive sensors by low-power sensors with high accuracy and fast convergence. In terms of energy consumption, the proposed collaborative sensing framework is able to preserve the energy consumption of IoT devices by nearly 50% for continuous data acquisition tasks.

Label imbalance and data missing are two major challenges in the problem of credit card fraud detection. However, existing matrix completion algorithms are generally difficult and cannot be easily applied to real-world credit card fraud detection since the scale of the normally used dataset is oversized. In this paper, we develop a spectral regularization algorithm to complete the large-scale sparse matrices, and further utilize an over-sampling algorithm to tackle the problem of the imbalance between positive and negative samples. Experimental results on a real-world dataset demonstrate that our model can outperform the state-of-the-art baseline methods. The proposed method could also be extended to other large-scale scenarios where data is missing or labels are imbalanced.

With the rampaging of Coronavirus disease 2019 (COVID-19) across the world, analyzing the dynamic characteristics and understanding the evolutionary patterns of clusters are becoming even more crucial for people and policymakers to make timely responses for avoiding injury caused by COVID-19. To solve the scarcity of the fine-grained spatiotemporal data, we construct a novel dataset about the spread of patients during the resurgent period of the COVID-19 epidemic at the Xinfadi Market in Beijing. Leveraging our self-build dataset, we analyze the evolutionary characteristics of the cluster of COVID-19 under anti-contagion policies and obtained some remarkable evolution patterns. These findings can provide significant insights for policymakers and researchers to understand the evolutionary characteristics regarding the cluster of COVID-19 and deploy effective anti-contagion policies.

How people divide their attention across their friends can help to understand key issues in the realm of geopolitics. Such attention exploration allows us to compare people who focus a large portion of their attention on a small set of close friends with those disperse their attention more widely. Using 2.5 million twitter data written by 130 thousands users, we find the balance of attention is a relatively stable property of people across different modalities of interaction. It displays subtle variation across people with different characteristics and different modalities of interaction. Specifically, people's attention is more focused in mention interactions, while those active in socialization tend to allocate higher portion of total attention to their close friends. Besides external interactions, people's inner interests also affect their attention allocation. People spreading multiple memes tend to be focused, and those with more even distribution of memes are focused on their intimate friends. Finally, people's relationships also plays an important role in their attention allocation. People are more likely to focus their attention on those most like them, and this similarity sequentially enhances the intimate relationship between them.

Geopolitics is a long-lasting key issue for governments and nations to assess the international political landscape. The great proliferation of social media in recently years have provided a new avenue to make such political actions in a data driven manner. As the information consumption ability of human is limited, there demands an automatic approach to effectively identify and trace the bursts continuously emerging on social media platforms. Existing studies focusing on named entities recognition or topic detection could provide useful insights for analyzing events that are already known, yet they are incapable of identifying timely emerging trending catchphrase or topics, or memes in general.To tackle with this issue, we elaborate a framework to identify online memes and trace their future dynamics. This framework identify memes based on their independency with regard to the context, and aggregate literal variants of a same meme together into a memeplex with a newly proposed MemeMesh algorithm. Evaluation results on a large scale Twitter dataset suggest that the framework could identify geopolitical memes effectively. Further exploration on meme popularity factors reveals that popularity memes tend to generate more variants during their diffusion, and establish their dominance by attracting a large volume of active users engaging in their diffusion. Causality analysis between meme diversity and user volume suggests that high diversity of meme variants can attract more users involved in spreading a meme at the initial, but these users seldom regenerate more variants in the later time.

Cyber crimes including computer virus/malwares, spam, illegal sales, and phishing websites are proliferated aggressively via the disguised Uniform Resource Locators (URL). Although numerous studies were conducted for the URL classification task, the traditional URL classification solutions retreated due to the hand-crafted feature engineering and the boom of newly generated URLs. In this paper, we study the representation learning of URLs, and explore the URL classification using deep learning. Specifically, we propose URL2vec to extract both the structural and lexical features of URLs, and apply temporal convolutional network (TCN) for the URL classification task. The experimental results show that URL2vec outperforms both word2vec and character-level embedding for URL representation, and TCN achieves the best performance than baselines with the precision up to 95.97%.

Detecting potential marketing pattern risks and preventing them can help enterprises lift operation efficiencies and reduce outlay costs. In this paper, we elaborate an ingenious method based on semi-supervised learning to identify latent marketing pattern risks for enterprises.

Online user behaviors are increasingly modulated by social media. Extant literature mainly focuses on investigating how network structures affect user behaviors. However, recent empirical results demonstrate that user behaviors and network structures usually coevolve dynamically, and topological patterns turn out to be inadequate for characterizing real-world user behaviors. In this paper, we present a dynamic model to deal with this challenge. This proposed model is mainly governed by two competing principles: homophily and homeostasis. Empirical evaluations of three online real-world datasets suggest that the proposed dynamic model can well predict long-range online user behaviors.

Public sentiments affecting health outcomes are increasingly modulated by social media. Existing literature mainly focus on investigating how network structure affects the contagion of health sentiments. However, most of these studies neglect that the interaction topology change in time. In fact, the change of inter-individual connections over time is associated with individual attributes. The mechanism through which individual attributes reshapes the connection topology is mainly governed by the competition between two principles, i.e., homophily (establishing or reinforcing social connections) and homeostasis (preserving the total strength of social connections to each individual). No existing approaches are yet able to accommodate these two competing effects at the same time. We thus propose a new statistical model (H2 model, Homophily and Homestasis model) to depict the evolution of temporal network, which is governed by the competition of homophily and homeostasis. In addition, we consider the mediation effect of external shock events, which enables us to separate exogenous confounding factors. Evaluation on Twitter data suggests that H2 model can capture long-range sentiment dynamics and external shock events. In sentiment prediction, H2 consistently outperforms existing methods in terms of error rate. Through the model's shock tensor, we successfully detect several typical events, and reveal that users in negative emotions are more influenced by external shock events than those with positive emotions. Our findings have practical significance for those who supervise and guide health sentiments in online communities.

Online emotion diffusion is a compound process that involves interactions with multiple modalities. For instance, different behaviors influence the velocity and scale of emotion diffusion in online communities. Depicting and predicting massive online emotions helps to guide the trend of emotion evolution, thus avoiding unprecedented damages in crises. However, most existing work tries to depict and predict online emotions based on models not considering related modalities. There still lacks an efficient modeling framework that promotes performance by leveraging multi-modality knowledge, and quantifies the interactions among different modalities. In this paper, we elaborate a computational model to jointly depict online emotions and behaviors. By introducing a common structure, we can quantify how user emotions interact with the corresponding behaviors. To scale up to large dataset, we propose a hierarchical optimization algorithm to accelerate the convergence of the model. Evaluation on Sina Weibo dataset suggests that prediction error rate is lowered by 69 percent with the proposed model. In addition, the proposed model helps to explain how user emotions influence consequent behaviors in extreme situations.

The proliferation of social media has increased the competition among different memes, which can be free texts, trending catchphrases, or micro media. As human attention is limited, these memes compete with each other, and go in and out of popularity at a rapid pace, sometimes even faster than we can recognize. Popular memes often shape the mindsets of online communities, and also shed light on their future tendencies. Considering the huge volume of memes generated and their continuous mutations, extracting and tracing online memes automatically is rather challenging. In this paper, we propose an automatic meme extraction algorithm. The proposed algorithm extracts massive memes based on phrases independency, and clusters phrase variants of a single meme efficiently. Evaluation on measles outbreak in the USA in 2015 indicates that the proposed algorithm could extract typical memes reflecting the fierce campaign between the pro-vaccination community and the anti-vaccination community. In both communities, memes are power-law distributed, and popular ones have many variants that appear more frequently. By tracing the evolution of online memes, we uncover that popular memes converge and generate peaks at times. Though the pro-vaccination community and the anti-vaccination community may focus on similar memes, they comprehend memes from totally different perspectives and deliver opposing opinions of measles vaccination.

The prevalence of social media has greatly catalyzed the dissemination and proliferation of online memes (e.g., ideas, topics, melodies, tags, etc.). However, this information abundance is exceeding the capability of online users to consume it. Ranking memes based on their popularities could promote online advertisement and content distribution. Despite such importance, few existing work can solve this problem well. They are either daunted by unpractical assumptions or incapability of characterizing dynamic information. As such, in this paper, we elaborate a model-free scheme to rank online memes in the context of social media. This scheme is capable to characterize the nonlinear interactions of online users, which mark the process of meme diffusion. Empirical studies on two large-scale, real-world datasets (one in English and one in Chinese) demonstrate the effectiveness and robustness of the proposed scheme. In addition, due to its fine-grained modeling of user dynamics, this ranking scheme can also be utilized to explain meme popularity through the lens of social influence.

Emotion entrainment accounts for the rhythmic convergence of human emotions through social interactions. This phenomenon abounds in various disciplines, i.e. effervescency in soccer games, anger proliferation in violence incidents, or anxiety diffusion in disasters. Although emotion entrainment is highly relevant to the quality of human daily life, the principles underpinning this phenomenon is still unclear. Previous dynamic models try to explain entrainment phenomenon by assuming symmetrical coupling among identical individuals. Yet this assumption clearly does not hold in real-world human interactions. As such, we propose an alternative model that captures asymmetric relationships. In depicting the coupling mechanism, the effect of social influence is also encoded. Experimental results on two emergent social events suggest that the proposed model characterizes emotion trends with high accuracy. Also, we explain the emotion dynamics by analyzing the reconstructed entrainment matrix. Our work may present practical implications for those who want to guide or regulate the emotion evolution in emergency events discussed online.

Revealing underlying social influence among users in social media is critical to understanding how users interact, on which a lot of security intelligence applications can be built. Existing methods fail to take into account the interaction relationships among memes. In this paper, we propose to simultaneously model social influence and meme interaction in information diffusion with novel multidimensional Hawkes processes. Experimental results on both synthetic and real world social media data show the efficacy of the proposed approach.

The sociological theory of entrainment accounts for the synchronization of human rhythmic modalities through social interactions: they coordinate in a variety of dimensions including linguistic styles, facial expressions, music pace, applause, and so on. Though highly relevant, emotion entrainment has received little attention to date. In addition, most previous studies on entrainment are done through small scale or controlled laboratory studies. In this paper, we investigate emotion entrainment in the context of online social media. To the best of our knowledge, this is the first time that emotion entrainment has been examined on a large scale, real world setting. For this purpose, we propose a framework that can model entrainment phenomenon and measure its effect. Our framework differentiates from previous research by its model-free essential and discerning in entrainment directions. These traits enable us to model entrainment dynamics under few assumptions, and distinguish emotion flow of entrainment. In our studies, we investigate entrainment patterns under different emotion states, i.e. positive, neutral and negative. We discover that entrainments under different emotions all follow a power law distribution. Besides, people are willing to entrain to others under positive emotion, and users with positive emotion are more likely to be entrained. By inspecting the interactions between entrainment and emotion, we reveal that entrainment has an effect of negotiating different emotion types toward an even distribution.

Discovering temporal patterns and changes in tobacco use has important practical implications in tobacco control. This paper presents one of the first comprehensive international studies of seasonal smoking patterns based on online searches performed. Using periodogram and cross-correlation, we find that smoking-related search behavior shows strong seasonality effect across countries. In addition, there are significant pairwise associations between such seasonality in different countries.

The rapid increasing popularity of micro-blogging has made it an important information seeking channel. By detecting recent popular topics from micro-blogging, we have opportunities to gain insights into internet hotspots. Generally, a topic's popularity is determined by two primary factors. One is how frequently a topic is discussed by users, and the other is how much influence those users have, since topics shown in the influential users' posts are more likely to attract others' attention. However, existing approaches interpret a topic's popularity with only the number of keywords related to it, which neglect the importance of the user influence to information diffusion in micro-blogging. In this paper, drawing upon the Cognitive Authority Theory and Social Network Theory, we propose a novel model that detects the most popular topics in micro-blogging with a user interest-based method. The proposed model first constructs a topic graph according to users' interests and their following relationship, and then calculates the topics' popularity with a link-based ranking algorithm. The popular topics detected by the method can reflect the relationship among users' interests, and the topics in the posts of influential users can be highlighted. Experimental results on the data of Twitter, a well-known and feature-rich micro-blogging service, show that the proposed method is effective in popular topic discovery.

A large portion of online news articles and postings are not originally created but reprinted or re-posted from other online news sources or portals. In this paper, we analyze the dynamics of online news propagation, using a large collection of Chinese online news activity data. We characterize prominent features of online news diffusion and compare them against the spreading patterns of the epidemic. Several critical factors influencing the news propagation process are identified, including the centrality and selectivity of source portals, and event variability.