Traditional short video recommendations primarily enhance user retention by reinforcing existing user preferences, potentially leading to information cocoons. Conversely, proactive recommendations aim to diversify user interests by exposing users to content beyond their historical preferences. However, current proactive approaches face three limitations: (1) homogeneous receptivity assumption, neglecting individual differences in users' openness to new interests; (2) short-term item exposure without interest anchoring, focusing on item-level shifts rather than interest evolution; and (3) static feedback utilization, failing to incorporate dynamic user feedback during the recommendation adequately. To address these challenges, we propose ProRec-Video, a proactive framework that guides hierarchical interest transitions through three innovations. First, User Receptivity Profiling assesses individual openness for new interests, ensuring personalized transition pacing. Second, Hierarchical Interest Transition Planning decomposes complex interest shifts into intermediate steps to generate smooth interest transition paths and semantically coherent video sequences, addressing overemphasis on item exposure. Third, Dynamic Feedback Adaptation integrates agent-based simulation and Reflexion mechanisms to refine interest transition paths and video sequences based on real-time user feedback, enhancing adaptability and satisfaction. Extensive experiments on two datasets demonstrate that ProRec-Video achieves a significant improvement in proactive recommendation performance, with an interest transition success rate of 85% and a user satisfaction rate of 78.3%.

In multi-hop question answering (MHQA) tasks, Chain of Thought (CoT) improves the quality of generation by guiding large language models (LLMs) through multi-step reasoning, and Knowledge Graphs (KGs) reduce hallucinations via semantic matching. Outcome Reward Models (ORMs) provide feedback after generating the final answers but fail to evaluate the process for multi-step reasoning. Traditional Process Reward Models (PRMs) evaluate the reasoning process but require costly human annotations or rollout generation. While implicit PRM is trained only with outcome signals and derives step rewards through reward parameterization without explicit annotations, it is more suitable for multi-step reasoning in MHQA tasks. However, existing implicit PRM has only been explored for plain text scenarios. When adapting to MHQA tasks, it cannot handle the graph structure constraints in KGs and capture the potential inconsistency between CoT and KG paths. To address these limitations, we propose the DPRM (Dual Implicit Process Reward Model). It trains two implicit PRMs for CoT and KG reasoning in MHQA tasks. Both PRMs, namely KG-PRM and CoT-PRM, derive step-level rewards from outcome signals via reward parameterization without additional explicit annotations. Among them, KG-PRM uses preference pairs to learn structural constraints from KGs. DPRM further introduces a consistency constraint between CoT and KG reasoning steps, making the two PRMs mutually verify and collaboratively optimize the reasoning paths. We also provide a theoretical demonstration of the derivation of process rewards. Experimental results show that our method outperforms 13 baselines on multiple datasets with up to 16.6% improvement on Hit@1.

Multimodal deepfake technologies have emerged rapidly in recent years, with wide application prospects in various fields. The conventional single-training paradigm with inherent limited generalization illustrates inadequate for addressing the continuous evolution of multimodal deepfakes. However, fine-tuning a model with new deepfake data faces past forgery patterns loss and the significant domain shift in diverse novel multimodal deepfake technologies. To address these issues, we propose a novel Quantum State Analytic Incremental Adaptability Learning method (Qsaint) for multimodal deepfake detection. To stabilize prior deepfake memory, Qsaint recursively learns detection-label mapping relations for the new deepfakes artifact with a closed-form solution, preserving the distribution memory from the historical deepfake domains without accessing previous videos. During incremental learning stages, we propose a deepfake quantum state adaptability module inspired by quantum information science. It adapts to the new forgery states and aligns them with the historical deepfake knowledge through cooling and evolution operations, eliminating deepfake domain shift issues. Comprehensive experiments demonstrate that Qsaint significantly mitigates the memory interference of historical deepfakes, effectively balancing the adaptability for new forgery tasks with the memorization of known deepfake patterns.

Instruction tuning (IT) improves large language models (LLMs) by aligning their outputs with human instructions, but its success depends critically on training data quality, and datasets such as Alpaca often contain noisy or suboptimal examples that undermine fine-tuning.Prior selection strategies score samples using general-purpose LLMs (e.g., GPT), leveraging their strong language understanding yet introducing inherent biases that misalign with the target model's behavior and yield unstable downstream performance.Influence-based methods address this by estimating each example's marginal contribution to overall performance, but they typically assume additive contributions and therefore overlook higher-order interactions among samples.To overcome these limitations, we propose JI 2 S, a novel framework that jointly models both marginal and combinatorial influences within sample groups.Applying JI 2 S to select the top 1,000 most influential examples from Alpaca, we fine-tune LLaMA2-7B, Mistral-7B, and LLaMA2-13B and evaluate them on Open LLM Benchmarks, MT-Bench, and GPT-4-judged pairwise comparisons.Our experiments show that JI 2 S consistently outperforms full-dataset training and strong baselines, highlighting the value of capturing joint influence for high-quality instruction fine-tuning.We provide our code in this GitHub repository.

Deepfake technology induces substantial societal challenges, establishing deepfake detection as an important area of research. However, existing research mainly relies on target deepfake datasets, which limits its generalizability across out-of-distribution tasks to some extent. Also, it often emphasizes visual modalities while neglecting the complementary information of the auditory data. Their autoregressive-based strategies also introduce long-term information interference, further constraining the detection performance. Consequently, the potential to exploit complementary relations between visual and auditory modalities and to leverage strongly correlated short-range information remains underexplored for the detection task. To address these challenges, this paper introduces Self-BiSterm, a novel self-supervised learning framework for deepfake detection. First, we propose a bidirectional synchronization distribution modeling mechanism, which calculates inconsistent distributions for video-to-audio and audio-to-video scenarios. This mechanism effectively measures audio-visual inconsistencies, improving the model's generalization performance in practical applications. Second, to mitigate the issue of long-term information distortion, we develop a short-term temporal dependency module to estimate the adjacent local receptive fields. This module facilitates the estimation of subsequent distributions by capturing short-term temporal dependencies with high precision. The effectiveness of the proposed Self-BiSterm framework is validated on various benchmarks, demonstrating superior performance compared to existing methods.

Multimodal deepfakes pose growing security threats across diverse domains, driven by rapid advancements in generative models. This demands effective Multimodal Deepfake Continual Detection (MDCD) methods capable of adapting to evolving and heterogeneous deepfake techniques. However, MDCD remains underexplored, facing two major challenges: (1) modality-specific feature disparities limit the effectiveness of simple feature fusion, exacerbating the forgetting of previous forgery-relevant knowledge; and (2) newly introduced deepfake videos initially exhibit limited scale that gradually expand, causing class imbalance dominated by forged samples, undermines authentic content understanding in comming tasks. To address these issues, we propose the Analytic Synaptic Dynamic Scaling Balancer (ADanser) that adapts to modality-specific biases and class imbalance while employing a closed-form update to preserve prior multimodal deepfake knowledge in an evolving data stream. Inspired by synaptic scaling in neuroscience, ADanser introduces a modality synaptic scaling mechanism that applies modality-aware attention to extract discriminative and complementary forgery patterns, improving cross-modal knowledge retention. Additionally, a class-wise contribution balancer dynamically reweights learning signals to reduce class bias and enhance authentic video representation. Extensive experiments on benchmark multimodal deepfake datasets demonstrate that ADanser significantly outperforms state-of-the-art continual learning methods, effectively coordinating adaptation and retention in imbalanced, cross-modal scenarios.

Abstract The typical aim of relational triple extraction is to identify entities along with their relations from unstructured text, which is a crucial task in information extraction. Recent methods achieve considerable performance by mining semantic associations within the input sentence but they hardly exploit the equally important semantic meaning of relations. Most methods simply represent relations as numeric labels and the rich semantic information is not fully utilized to enhance performance. To address the issue, we decompose the task into two sequential subtasks, entity pairing and relation matching, from a novel perspective and then propose a semantic-preserving model SPRel. Specifically, SPRel first extracts entity pairs associated with at least one relation, and then matches them with certain relations according to the descriptive text of relations. Comprehensive experiments on two widely used datasets demonstrate that SPRel outperforms previous methods, particularly in handling complex scenarios.

Multimodal Large Language Models (MLLMs), by expanding the model's capabilities to perceive and interact through multi-modalities, have significantly enhanced performance across various tasks. The perception of vision is an important modality developed into LLM, enabling research in vision-language to continuously lead the cutting-edge advancements in the MLLM community. However, the standard pre-training pipeline on image-text pairs results in a limited model understanding of relationships between multiple images and texts, as well as visual details. Additionally, the setting of fine-tuning with a frozen visual backbone hinders the enhancement of visual representations on new data. These two issues lead to suboptimal performance in models for demonstrative instruction following about multiple images. This work introduces a novel framework called MLoEM, which first converts long multimodal data into an interleaved image-instruction format, and then adopts a fully autoregressive architecture model, allowing for more robust and coherent learning from naturally occurring multimodal documents than pair-based pipeline. Additionally, we incorporate the Low-Rank Adaptation (LoRA) fine-tuning method, enhancing visual representations while maintaining the stability of previously learned knowledge. Finally, we utilize ensemble methods to enhance model performance on tasks. To alleviate the storage overhead issue of parallel ensembles with large models, we design an ensemble approach that shares the MLLM while only switching the LoRA matrices. In the experiments, the proposed MLoEM shows superior performance on the testing set.

Ultrasound tongue imaging is widely used in clinical linguistics and phonetics. Recently, deep neural networks, especially convolutional neural networks, have been widely used in the interpretation and analysis of ultrasound tongue images (UTI). Despite achieving satisfactory performance, deep models rely on a large amount of manually labeled data, which is often difficult to obtain in practical settings. To address this issue, this paper focuses on how to utilize a large amount of unlabeled UTI data to improve the performance of UTI classification task. Specifically, we explore self-supervised learning with masking modeling strategy. By predicting the masked part, our pre-trained model enables the neural network to infer contextual information. Then, we fine-tune the pre-trained model with a small amount of labeled data. Compared with the previous competing algorithms, our method can improve the classification accuracy by an average of 13.33% in four different scenarios.

User identity linkage aims to correlate multiple virtual identities of the same person across different social networks. It is widely used in person profile integration, potential friend recommendation, user behavior prediction, identity verification, etc. Since the data inconsistency and network heterogeneity among social networks, extracting features from profiles, contents, and network structures may lead to severe random noises. To this end, we propose PsyLink, a user identity linkage method via psychological characteristic modeling. To reduce the data noise, PsyLink extracts both internal personality characteristics and external interest characteristics to represent an individual. Furthermore, we design a neighborhood enhancement strategy to capture latent higher-order structural features through a graph contrastive learning technique. Under various parameter settings, the experimental results demonstrate that the macro-f1 of PsyLink improves 19.23% on average compared with state-of-the-art deep learning models. Combined with the psychological characteristics and graph contrastive learning, PsyLink is able to adequately learn the similarities and differences between user identities across social networks.

Both 5G Internet and COVID-19 pandemic have increasingly prompted thousands of companies and organizations to shift from a centralized office model to a distributed home model, which poses a new requirement: how to securely and rapidly share private data for coworkers on Internet. The covert communication systems are widely used to deliver private information because of the possibility of extending the system to Internet-scale size. However, most existing systems are inadequate to solve the requirement, since either the servers in centralized systems face the risk of being monitored and infiltrated, or the multi-hop routing schemes in decentralized systems lead to diverse attacks and high delivery latency. To this end, we proposed a scalable covert communication service for coworkers, called SC2. For security and hiddenness, we adopt the content slicing and multichannel routing to prevent adversary from monitoring and analyzing data. For scalability, we design a two-hop logic overlay to support low latency routing, and an adaptive channel selction technique to exploit the available bandwidth of the system. To evaluate the performance of SC2, we deploy the system in various IoT clouds and storage clouds. The experimental results demonstrate that SC2 is able to transmit both short messages and bulk content. Under various parameter settings, the delivery latency of SC2 linearly decreases with the number of channels, and SC2 takes full advantage of the available bandwidth with the growing number of users.

Meta-learning has emerged as an effective approach for few-shot text classification. However, current studies fail to realize the importance of the semantic interaction between sentence features and neglect to enhance the generalization ability of the model to new tasks. In this paper, we integrate an adversarial network architecture into the meta-learning system and leverage cost-effective modules to build a novel few-shot classification framework named SaAML. Significantly, our approach can exploit the temporal convolutional network to encourage more discriminative representation learning and explore the attention mechanism to promote more comprehensive feature expression, thus resulting in better adaptation for new classes. Through a series of experiments on four benchmark datasets, we demonstrate that our new framework acquires considerable superiority over state-of-the-art methods in all datasets, increasing the performance of 1-shot classification and 5-shot classification by 7.15% and 2.89%, respectively.

Aiming to challenge that current image deblurring methods fail to deal with complex scenes well and are computationally time-consuming, we present an attention-based lightweight network (MALNET) that restores sharp images in a pyramid learning pattern. The model introduces the lightweight chained residual unit as the core building block to construct the multi-path refinement network and embeds the simple yet efficient attention unit to obtain rich contextual information. Moreover, we explore multiple training strategies such as multi-scale joint loss and parameter sharing, making MALNET more flexible and compatible in various application scenarios. The experimental results sampled from the DOTA and GoPro datasets demonstrate that MALNET can enjoy 5 times faster speed and 7 times fewer bulk than those state-of-the-art approaches while still providing a competitive performance both quantitatively and qualitatively.

Meta-learning has achieved remarkable performance to address few-shot text classification. However, previous studies fail to attach importance to the label semantic completely and overlook the implicit interaction between lexical features of sentences. In this paper, we explore three enhancement components of the meta-learner aided by the label semantic and sentence-aware interaction, e.g., the label-augmented encoder, the interaction extractor, and the label semantic discriminator. Significantly, these modules are agnostic to the choice of few-shot text classification methods and can be easily incorporated into various existing meta-learning frameworks to improve the classification performance and adaptive ability of the model. We conduct extensive experiments on five benchmark datasets. The results demonstrate that meta-learning approaches upgraded by the above three enhancement components obtain considerable superiority over state-of-the-art models in all datasets. In particular, the average accuracy of 1-shot classification and 5-shot classification is increased by 3.60% and 2.28 %, respectively.

In this paper, we present our solution for ACM Multimedia 2022 Computational Paralinguistics Challenge. Our method employs the self-supervised learning paradigm, as it achieves promising results in computer vision and audio signal processing. Specifically, we firstly explore modifying the Swin Transformer architecture to learn general representation for the audio signals, accompanied with random masking on the log-mel spectrogram. The main goal of the pretext task is to predict the masked parts, by combining the advantages of the Swin-Transformer and masked modeling. For the downstream tasks, we utilize the labelled datasets to fine-tune the pre-trained model. Compared with the competitive baselines, our approach can provide significant performance improvements without ensembling.

Sarcasm is a sophisticated expression, commonly used on social media, e.g., Reddit and Twitter. The presence of sarcasm in social media text flips the polarity of sentiment, thus hindering the performance of works that require true sentiment, e.g., sentiment analysis and opinion mining. However, current works fail to exploit commonsense knowledge in the sarcasm detection task. In this paper, we revisit sarcasm detection from a novel perspective, which models commonsense knowledge as well as context semantics to reason with sarcasm. More specifically, we propose a commonsense-aware model with a heterogeneous graph attention network that leverages commonsense knowledge, enabling it to better understand implied sentiment behind the literal meaning. We conduct experiments on benchmark datasets from Reddit and Internet Argument Corpus. Experimental results show that our proposed approach yields superior performance with commonsense knowledge integrated.

The rapid development of social media makes it easy for people to acquire information while also providing a platform for publishing and spreading fake news. Fake news brings plenty of explicit and implicit risks to social stability, making fake news detection an issue that deserves attention. Recent methods based on graph neural networks (GNN) achieve impressive results in fake news detection, but their performance is still limited in practice due to the absence of high-order relations between nodes. In this paper, we propose a Sentiment-Aware Hypergraph Attention Network (SA-HyperGAT) for fake news detection. SA-HyperGAT can better leverage different kinds of information from news contents and user comments with hypergraphs, which can capture higher-order dependency between words and sentences compared with general graphs. Specifically, we first construct two hypergraphs with distinct types of nodes and hyperedges to utilize structural information of news contents and sentimental information of user comments. Then we adopt a hypergraph attention network with a dual attention mechanism to learn the composed representations of two hypergraphs for the final prediction. Our proposed SA-HyperGAT outperforms competitive baselines on two real-world datasets. Extensive experimental results prove the effectiveness of each component in SA-HyperGAT.

Biometric authentication is up-and-coming to replace the traditional identity authentication method (e.g., passwords, PIN, identification cards) for its convenience and intelligence. With more and more users using this method, the database becomes more extensive, and the functions are seriously challenged. Data outsourcing has advantages in terms of convenience and cost savings, so it has attracted much research effort. However, due to the biometric's immutability of the whole life, it is extremely sensitive, and disclosing it to a third party is undesirable. In this paper, we address the issue of securely outsourcing biometric database. We propose a novel boolean searchable symmetric encryption (SSE) to construct a secure interactive protocol when outsourcing. A new encrypted database construction method was proposed, using the more efficient boolean vectors. Based on this, We suggest three kinds of expressive SSE, supporting disjunctive query, boolean query, and lightweight settings. We prove the schemes' correctness and security theoretically. Our constructions use simple cryptographic tools, such as symmetric cryptography and pseudo-random functions. They are straightforward to understand and easy to implement. The experiments show that all our schemes are practical and more efficient than the existing methods.

In recent years, online social networks (OSNs) have undoubtedly become the most popular information diffusion channel. However, existing work has rarely focused on the end-to-end diffusion path, which means information diffusion between user pairs is still a problem unresolved. In this paper, an end-to-end independent cascade model (EICM) is proposed. Then, we present an efficient approach called end-to-end diffusion flow pruning (EDFP) that simplify large-scale social networks until main diffusion paths of end-to-end networks are identified.

The "Internetware" paradigm is fundamentally changing the traditional way of software development. More and more software projects are developed, maintained and shared on the Internet. However, a large quantity of heterogeneous software resources have not been organized in a reasonable and efficient way. Software feature is an ideal material to characterize software resources. The effectiveness of feature-related tasks will be greatly improved, if a multi-grained feature repository is available. In this paper, we propose a novel approach for organizing, analyzing and recommending software features. Firstly, we construct a Hierarchical rEpository of Software feAture (HESA). Then, we mine the hidden affinities among the features and recommend relevant and high-quality features to stakeholders based on HESA. Finally, we conduct a user study to evaluate our approach quantitatively. The results show that HESA can organize software features in a more reasonable way compared to the traditional and the state-of-the-art approaches. The result of feature recommendation is effective and interesting.

Virtual machine monitor (VMM)-based anti-malware systems have recently become a popular research topic in finding ways of overcoming the fundamental limitations of traditional host-based anti-malware systems, which are likely to be deceived and attacked by malicious codes. This paper analyzes existing VMM-based models of malware detection. "Out-of-the-box" detection, active defense model, or In-VM models have the same defects: (1) on top of the VMM, two virtual machines are used, one by the user (Guest OS) and the other as monitor (Host OS), and (2) users cannot directly view the detection results nor configure detection system in the Guest OS. A layered detection model is proposed to overcome these issues, the bottom layer is responsible for security for the layers above it. Detection results can be directly displayed in the Guest OS, and users can view and configure the detection system. Furthermore, the detection model can isolate malware attacks to the detection system in the Guest OS. Experiment results show the validity of the proposed detection model.

The extensive use of Online Social Networks (OSNs) has attracted such wide attention of academia that OSNs have become a hot topic. Based on the interaction simulation, we have designed and implemented a Facebook crawler, which can obtain the complete friend list of a Facebook user and overcome the drawback in [7] that the crawler can get at most 400 friends. We make an analysis and visualization of the crawled dataset, and find that 36.5% of Facebook users have changed the default privacy setting, compared to 26.6% and 16% in [7, 20] respectively, implying that the awareness of privacy protection of Facebook users has been greatly improved.

Many P2P software have the similar communication patterns with computer worms, thus they will bring in false positives for behaviour based worm detection. Up to now, little work is done on the research of the similarities between communication patterns of worm and P2P software as well as how to eliminate the worm-like P2P traffic. Based on the analysis of popular P2P software used nowadays and the host process information, this paper presents ASG, which is a novel host based algorithm to generate signatures for worm-like P2P communication patterns. The contribution of our work lies in three aspects: a) Analyzing communication pattern similarities between P2P traffic and worm traffic through examples. b) Designing one practical and simple signature format for worm-like P2P traffic based on the host process information, c) Presenting Automated Signature Generation (ASG) method to extract the signature of worm-like P2P traffic. Experiments with the popular used P2P software show that ASG can effectively extract the signature and reduce the false positives.