Long Chain-of-Thought (CoT) reasoning has shown great promise in complex reasoning tasks, but its application to medical decision-making presents unique challenges. Unlike structured tasks relying on static verification frameworks, medical decision-making requires dynamic validation through longitudinal clinical outcomes, exhibiting temporal-causal dependencies that complicate the verification of reasoning processes. Therefore, we introduce a novel data construction framework specifically designed for medical decision-making. First, the framework analyzes real-world clinical cases to construct a timeline of medical events and identify critical decision points, including examination, diagnosis, and treatment. Subsequently, it employs a clinical causality-aware strategy to generate decision-making questions at the identified points, along with reasoning traces and corresponding answers. Finally, information drawn from future nodes serves as clinical logic-constrained criteria to re-evaluate and refine the soundness of the generated reasoning and responses. Building on this, we present OncoCoT, an oncologic decision-making dataset derived from clinical records over the past four years across eight common cancer types. Furthermore, we distill a subset of OncoCoT into a dedicated benchmark, OncoEval, to facilitate systematic evaluation of clinical reasoning capabilities in LLMs. Evaluation results show that existing state-of-the-art reasoning models, such as Deepseek-r1 and GPT-o3, exhibit limited capability in addressing clinical problems in OncoEval, highlighting the need for further improvement.

Multimodal retrieval-augmented generation (RAG) systems enhance large vision-language models by integrating cross-modal knowledge, enabling their increasing adoption across real-world multimodal tasks. These knowledge databases may contain sensitive information that requires privacy protection. However, multimodal RAG systems inherently grant external users indirect access to such data, making them potentially vulnerable to privacy attacks, particularly membership inference attacks (MIAs). Existing MIA methods targeting RAG systems predominantly focus on the textual modality, while the visual modality remains relatively underexplored. To bridge this gap, we propose MrM, the first black-box MIA framework targeted at multimodal RAG systems. It utilizes a multi-object data perturbation framework constrained by counterfactual attacks, which can concurrently induce the RAG systems to retrieve the target data and generate information that leaks the membership information. Our method first employs an object-aware data perturbation method to constrain the perturbation to key semantics and ensure successful retrieval. Building on this, we design a counterfact-informed mask selection strategy to prioritize the most informative masked regions, aiming to eliminate the interference of model self-knowledge and amplify attack efficacy. Finally, we perform statistical membership inference by modeling query trials to extract features that reflect the reconstruction of masked semantics from response patterns. Experiments on two visual datasets and eight mainstream commercial visual-language models (e.g., GPT-4o, Gemini-2) demonstrate that MrM achieves consistently strong performance across both sample-level and set-level evaluations, and remains robust under adaptive defenses.

Open knowledge bases (e.g., websites) are widely adopted in Retrieval-Augmented Generation (RAG) systems to provide supplementary knowledge (e.g., latest information). However, such sources inevitably contain biased or harmful content, and incorporating these untrusted contents into the RAG process introduces significant safety risks, including the degradation of LLM performance and the potential generation of harmful outputs. Recent studies have shown that this vulnerability can be further amplified by adversarial poisoning attacks specifically targeting the knowledge sources. Most existing methods primarily emphasize improving the accuracy and efficiency of RAG systems, usually overlooking these critical safety concerns. In this paper, we propose a safety-aware retrieval framework (ShieldRAG) designed to augment language model generation by jointly optimizing for both relevance and safety in the retrieved knowledge content. The core idea of ShieldRAG is to transfer the safety knowledge implicitly encoded in powerful LLMs into the retriever model through an adversarial knowledge alignment mechanism. This can empower the retriever with the safety awareness, and adapt to the diverse and unknown distribution of unsafe content encountered in practical scenarios. We evaluate ShieldRAG on seven real-world datasets using five widely-used LLMs and two state-of-the-art poisoning attack strategies. Experimental results show that our method substantially improves the robustness of RAG systems against unsafe knowledge sources, while maintaining competitive performance in terms of generation accuracy and efficiency.

Peiru Yang, Haoran Zheng, Tong Ju, Shiting Wang, Wanchun Ni, Jiajun Liu, Shangguang Wang, Yongfeng Huang, Tao Qi. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Abstract Ensuring data consistency across multiple cloud service providers (CSPs) is essential for maintaining the reliability and security of multi-cloud storage systems. However, most existing related works rely on trusted third parties (TTPs) and are vulnerable to collusion, which undermines trust and efficiency. To overcome these challenges, we propose MDCV, a decentralized and collusion-resistant consistency verification scheme for multi-cloud storage. MDCV leverages smart contracts to autonomously coordinate consistency verification process without relying on any TTP, thereby ensuring transparency, immutability, and trust-free verification. Specifically, MDCV employs a collusion-resistant public verification protocol in which each CSP binds the proof generation process to its private key and generates non-interactive, publicly verifiable consistency proof using pseudo-random challenges derived from blockchain transaction IDs. This design not only prevents colluding CSPs from generating forged proofs but also supports public auditing. Formal security analysis demonstrates the soundness of the proposed scheme, while experimental results show that MDCV achieves efficient proof generation and verification with lower computational and communication overhead, validating its practicality in real-world multi-cloud backup auditing systems.

Text steganography helps protect citizens' freedom of speech and privacy in cyberspace by constructing innocent-looking texts to evade censorship and surveillance. Existing methods, especially generative ones, rely on delicate character-level manipulations, making them vulnerable to failure even under minor alterations. This paper introduces a novel framework that shifts from character-level to semantic space-based information hiding, which greatly improves improving robustness and reliability. The framework comprises three phases: preparation, where a stable semantic space is designed for hiding information; embedding, where a reversible codebook maps binary messages to semantemes via semantic encoding and source coding; and synthesis, where large language models act as multi-agent systems to produce stegotexts that preserve semantic consistency. Extensive experiments show that our method matches state-of-the-art techniques in hiding capacity and text quality, while significantly surpassing them in robustness-achieving at least an 81.2% higher correct message rate under three types of attacks. These findings highlight the strong potential of semantic-based text steganography.

Recently, on-device large models (e.g., 7B parameter LLMs and LVLMs) are increasingly deployed in real-world services such as mobile assistants, edge computing, and IoT systems, where low latency and resource efficiency are critical. Since large models trained on general-domain data are typically suboptimal for downstream services, In-Context Learning (ICL) technique is widely used to enhance their task-specific service performance without extra tuning. However, effective ICL methods generally necessitate a sufficient amount of supervised data to provide abundant service-related information, which usually leads to a long context (i.e., more texts or images input) for models to process. This long context issue can significantly degrade the real-time inference efficiency and also impact the information utilization effectiveness, particularly for on-device large models with limited long context modeling capacities. To tackle this challenge, we propose a Personalized Knowledge Refinement (PKR) framework to achieve efficient and effective ICL for on-device large models. Specifically, we first introduce a personalized knowledge extraction module, which analyzes the inference behaviors of the target model on small-scale supervised data and then convert these behavioral patterns into personalized service-specific knowledge as the instruction context. Furthermore, we propose an adaptive knowledge filtration mechanism to model the informativeness of the extracted knowledge and eliminate the redundant ones, further improving the knowledge encoding efficiency per unit of context length. Experiments based on 14 benchmarks spanning both textual and visual task services, and 4 large models, demonstrated that PKR consistently improves task accuracy while reducing context length and inference latency, making it a practical solution for real-world on-device services. Codes are released athttps://github.com/wanghl21/PKR.

Accurately recommending candidate news articles to users has always been the core challenge of news recommendation system. News recommendations often require modeling of user interest to match candidate news. Recent efforts have primarily focused on extracting local subgraph information in a global click graph constructed by the clicked news sequence of all users. However, the computational complexity of extracting global click graph information has hindered the ability to utilize far-reaching linkage which is hidden between two distant nodes in global click graph collaboratively among similar users. To overcome the problem above, we propose a Global-view Long Chain Interests Modeling for news recommendation (GLoCIM), which combines neighbor interest with long chain interest distilled from a global click graph, leveraging the collaboration among similar users to enhance news recommendation. We therefore design a long chain selection algorithm and long chain interest encoder to obtain global-view long chain interest from the global click graph. We design a gated network to integrate long chain interest with neighbor interest to achieve the collaborative interest among similar users. Subsequently we aggregate it with local news category-enhanced representation to generate final user representation. Then candidate news representation can be formed to match user representation to achieve news recommendation. Experimental results on real-world datasets validate the effectiveness of our method to improve the performance of news recommendation.

Linguistic steganography is a key information hiding technique but faces challenges like abrupt content shifts, detection risks, and high training resource demands. To address these, this paper introduces SCF-Stega, a controllable method based on Semantic Communications Framework. By using a knowledge graph to guide secret encoding and dynamically adjusting large language model outputs, SCF-Stega enhances text imperceptibility and semantic coherence. Experiments show improved text quality and strong resistance to steganalysis, without needing additional training data.

With the increasing prevalence of surveillance, safeguarding personal privacy has become a critical concern. To protect privacy, various linguistic steganography methods have been developed to conceal private information within seemingly innocuous text for covert communication. However, these methods are highly sensitive to alterations in the stego text, rendering the extraction of secret information impossible if any changes occur, thus limiting their practical application. In this paper, we introduce WinStega, an adaptive and robust linguistic steganography method that employs substring decoding to withstand edit attacks. WinStega embeds secret messages discontinuously using a sliding window approach, incorporating entropy-based constraints to enhance imperceptibility while preserving linguistic quality. This plug-and-play method does not require additional model training and can be implemented during the inference stage, enhancing the robustness of stego texts. Extensive evaluations using three language models demonstrate that WinStega produces high linguistic quality, imperceptible stegotexts and can partially recover secret messages even under adversarial attacks.

With the widespread application of large language models (LLMs) in the field of natural language processing (NLP), copyright protection issues are becoming increasingly important.As an effective means of copyright protection, watermarking technology can help developers and users prove the copyright ownership of the model.However, existing watermarking methods struggle to optimize both watermark effectiveness and model performance simultaneously.To overcome this challenge, in this paper, we propose a backdoor watermarking method named CRMark based on Chain-of-Thought (CoT) and reinforcement learning.This method embeds backdoor symbols into the prompt of the datasets and adds copyright information as the watermark into the model response.Reinforcement learning is further employed to alleviate the performance degradation of the watermark model in normal tasks.Experimental results show that the CRMark does not reduce the model's original task performance while effectively maintaining the effectiveness of backdoor watermarks, with the watermark success rate of up to 96.5%.

Linguistic steganalysis often encounters domain shift in practice, mainly due to differences in text sources and steganographic methods.These variations create distribution discrepancies between the training (source domain) and test (target domain) sets, reducing detection accuracy.Most existing domain adaptation methods for linguistic steganalysis rely on labeled source domain data to alleviate these issues, but due to data privacy or transmission costs, source domain data is often unavailable in many real-world scenarios.Without access to this data, models cannot directly compare feature distributions between the source and target domains, hindering the model's ability to learn the target domain's features and ultimately affecting detection performance for stego texts.In this paper, we propose a Clustering-driven Pseudo-labeling method for Source-free domain adaptation in Linguistic Steganalysis (CPSLS).During the adaptation phase, we leverage the clustering structure of the target domain data to generate pseudo-labels, helping the model identify stego features in the target domain.Additionally, we use a weighted classification loss function to reduce the impact of incorrect pseudo-labels.To prevent the model from overlooking the diversity between stego and cover texts during optimization, we introduce a prediction diversity loss, improving the model's ability to differentiate between the two.Experimental results show that CPSLS not only has stronger practical applicability but also outperforms existing domain adaptation linguistic steganalysis in terms of detection accuracy.

Large vision-language models (LVLMs) derive their capabilities from extensive training on vast corpora of visual and textual data. Empowered by large-scale parameters, these models often exhibit strong memorization of their training data, rendering them susceptible to membership inference attacks (MIAs). Existing MIA methods for LVLMs typically operate under white- or gray-box assumptions, by extracting likelihood-based features for the suspected data samples based on the target LVLMs. However, mainstream LVLMs generally only expose generated outputs while concealing internal computational features during inference, limiting the applicability of these methods. In this work, we propose the first black-box MIA framework for LVLMs, based on a prior knowledge-calibrated memory probing mechanism. The core idea is to assess the model memorization of the private semantic information embedded within the suspected image data, which is unlikely to be inferred from general world knowledge alone. We conducted extensive experiments across four LVLMs and three datasets. Empirical results demonstrate that our method effectively identifies training data of LVLMs in a purely black-box setting and even achieves performance comparable to gray-box and white-box methods. Further analysis reveals the robustness of our method against potential adversarial manipulations, and the effectiveness of the methodology designs. Our code and data are available at \url{https://github.com/spmede/KCMP}.

To circumvent the unbridled and ever-encroaching surveillance and censorship in cyberspace, steganography has garnered attention for its ability to hide private information in innocent-looking carriers. Current provably secure steganography approaches require a pair of encoder and decoder to hide and extract private messages, both of which must run the same model with the same input to obtain identical distributions. These requirements pose significant challenges to the practical implementation of steganography, including limited access to powerful hardware and the intolerance of any changes to the shared input. To relax the limitation of hardware and solve the challenge of vulnerable shared input, a novel and practically significant scenario with asymmetric resource should be considered, where only the encoder is high-resource and accessible to powerful models while the decoder can only read the stegano-graphic carriers without any other model's input. This paper proposes a novel provably robust and secure steganography framework for the asymmetric resource setting. Specifically, the encoder uses various permutations of distribution to hide secret bits, while the decoder relies on a sampling function to extract the hidden bits by guessing the permutation used. Further, the sampling function only takes the steganographic carrier as input, which makes the decoder independent of model's input and model itself. A comprehensive assessment of applying our framework to generative models substantiates its effectiveness. Our implementation demonstrates robustness when transmitting over binary symmetric channels with errors.

The primary aim of audio deepfake detection (ADD) is to thwart deception arising from forged audio generated through text-to-speech or voice conversion technologies. However, encoding speech signals using diverse compression algorithms introduces discrepancies that significantly impair the performance of existing countermeasure systems. To tackle these challenges, this letter proposes a robust audio deepfake detection method based on Compressed Domain Invariant Adversarial Representation Learning with Adaptive Token Pooling (DANet-ATP). This framework incorporates a Compression Codecs Discriminator (CCD) that, through adversarial learning in tandem with the backbone network, enhances the model's ability to extract more robust features across diverse compression codecs. Moreover, to efficiently prune redundant frame-level features while retaining vital spoofing cues, the letter designs a plug-and-play, parameter-free Adaptive Token Pooling module, significantly improving detection performance. Experimental results on the ASVspoof2021 DF dataset showcase the exceptional performance of the proposed model. Furthermore, a series of ablation experiments validate the validity and effectiveness of the proposed method.

Large language models (LLMs) demonstrate general intelligence across a variety of machine learning tasks, thereby enhancing the commercial value of their intellectual property (IP). To protect this IP, model owners typically allow user access only in a black-box manner, however, adversaries can still utilize model extraction attacks to steal the model intelligence encoded in model generation. Watermarking technology offers a promising solution for defending against such attacks by embedding unique identifiers into the model-generated content. However, existing watermarking methods often compromise the quality of generated content due to heuristic alterations and lack robust mechanisms to counteract adversarial strategies, thus limiting their practicality in real-world scenarios. In this paper, we introduce an adaptive and robust watermarking method (named ModelShield) to protect the IP of LLMs. Our method incorporates a self-watermarking mechanism that allows LLMs to autonomously insert watermarks into their generated content to avoid the degradation of model content. We also propose a robust watermark detection mechanism capable of effectively identifying watermark signals under the interference of varying adversarial strategies. Besides, ModelShield is a plug-and-play method that does not require additional model training, enhancing its applicability in LLM deployments. Extensive evaluations on two real-world datasets and three LLMs demonstrate that our method surpasses existing methods in terms of defense effectiveness and robustness while significantly reducing the degradation of watermarking on the model-generated content.

Recent advancements in deep learning have significantly improved linguistic steganalysis, but challenges persist when labeled samples are scarce in the target domain. Existing cross-domain linguistic steganalysis methods seek to improve model generalization by minimizing the domain discrepancy between the source and target domains. However, these steganalysis methods often struggle with incorrect alignment between stego and cover texts in both domains, which hampers the generalization of steganalysis models. Additionally, they struggle to capture domain-specific features of the target domain, reducing the effectiveness of steganalysis models in discriminating stego texts. To address these issues, we propose a novel Class-aware Adversarial unsupervised Domain Adaptation (CADA) method, which operates in two stages. In the first stage, Class-aware Adversarial Pre-Training (CAPT), we design the Weighted Class-Aware Domain Distance (WCADD) to leverage class information of stego and cover texts. This ensures accurate class-aware alignment across domains. In the CAPT stage, the steganalysis model is pre-trained with WCADD, Class-Aware Adversarial Training (CAAT), and Class-Aware Label Smoothing (CALS) to enhance its ability to extract domain-invariant features, thereby improving its generalization. In the second stage, Class-aware Fine-Tuning (CFT), we employ the pre-trained steganalysis model alongside the Class-Aware Progressive Strategy (CAPS) to generate pseudo-labels for the target domain. Fine-tuning the model with these pseudo-labels enhances its ability to recognize domain-specific features, thereby improving its performance in discriminating stego texts within the target domain. Extensive experiments demonstrate that our proposed method outperforms the existing baseline methods.

IoT device identification is vital for network asset and security management. However, existing methods use statistical features that can not identify IoT devices accurately in complex network environments.GraphIoTproposes using non-statistical features and building a heterogeneous graph neural network to identify IoT devices accurately. However, heterogeneous graph neural networks lack interpretability, which reduces trust in the model. Besides, it is difficult to deploy on resource-constrained devices, limiting the broad application of IoT device identification. To make IoT device identification interpretable, easy to deploy, and with high accuracy, we get the interpretation results ofGraphIoTthrough interpretability and further build the rule set based on the interpretation results. Considering there is no suitable interpreter forGraphIoTwith many nodes and edges, we proposeHGExplainer, which reduces the time complexity by splitting the interpretation target into important relation solving and edge solving and uses a novel solution method, ExpandTree. Then, we also designed a rule extractor, which can build rule sets based on the interpretation results. Experimental results on Yourthings and UNSW datasets show thatHGExplainercan build high fidelity, concise sample-level explanations in less than 3 seconds, and the established rule set can precisely identify IoT devices.

Recommender systems are widely employed to mitigate information overload by tailoring online content to individual preferences. Existing recommendation methods typically focus on optimizing the relevance between candidate item content and user historical behaviors. However, these methods often neglect the quality of recommended content, which can negatively affect user experience and hinder the long-term growth of platforms. In fact, addressing this issue is particularly challenging, as signal on content quality feedback is typically sparse in the user interaction data (e.g., clicks) commonly used for model training. In this article, we propose a human feedback alignment framework for recommender system (HFAR), which leverages well-aligned large language models to simulate human feedback on content quality to enhance recommendation. Specifically, we propose a multi-task learning-based knowledge transfer framework to infuse recommendation models with an awareness of fine-grained feedback on content quality from targeted perspectives. Furthermore, we develop a contrastive learning-based feedback integration mechanism to embed targeted human feedback into the ranking strategy to enable quality-aware recommendation decision-making. Besides, we propose a multi-objective joint training framework to optimize the model jointly under utility and quality objectives. Experiments show that HFAR achieves a maximum improvement of 84.78% in recommendation quality, while maintaining both recommendation accuracy and efficiency.

Recently, as one of the most popular covert communication technologies, generative steganography has received ever-increasing attention due to its promising performance against sophisticated steganalysis tools. However, it is quite difficult for the existing generative steganographic approaches to find a good tradeoff between hiding capacity and extraction accuracy, mainly due to the small capacity of their hiding spaces. To overcome this shortcoming, a Progressive Generative Steganography (PGS) network architecture is proposed to hide a secret message during the progressive image generation process to realize secure covert communication. Specifically, we first propose a robust Secret-to-Noise (S2N) mapping method to encode the secret message as a set of noise maps. Then, guided by these noise maps, a set of corresponding images ranging from low resolution to high resolution are progressively generated by the Single Generative Adversarial Networks (SINGAN). Consequently, a large-sized secret message can be hidden in the finally generated high-resolution image, since a set of high-capacity hiding spaces can be provided by the process of progressive image generation. Moreover, to improve the quality of image generation and the accuracy of secret message extraction, a Dense Secret-Feature Connection (DSFC) strategy is designed and integrated into the proposed PGS network architecture. Extensive experiments demonstrate that the proposed PGS outperforms the existing approaches in the aspects of both hiding capacity and message extraction, while maintaining promising anti-detectability and imperceptibility for covert communication.

Robustness and privacy protection are two important factors of trustworthy federated learning (FL). Existing FL works usually secure data privacy by perturbing local model gradients via the differential privacy (DP) technique, or defend against poisoning attacks by filtering the local gradients in the outlier of the gradient distribution before aggregation. However, these two issues are often addressed independently in existing works, and how to secure federated learning in both privacy and robustness still needs further exploration. In this paper, we unveil that although DP noisy perturbation can improve the learning robustness, DP-FL frameworks are not inherently robust and are vulnerable to a carefully-designed attack method. Furthermore, we reveal that it is challenging for existing robust FL methods to defend against attacks on DP-FL. This can be attributed to the fact that the local gradients of DP-FL are perturbed by random noise, and the selected central gradients inevitably incorporate a higher proportion of poisoned gradients compared to conventional FL. To address this problem, we further propose a new defense method for DP-FL (named Robust-DPFL), which can effectively distinguish poisoned and clean local gradients in DP-FL and robustly update the global model. Experiments on three benchmark datasets demonstrate that baseline methods cannot ensure task accuracy, data privacy, and robustness simultaneously, while Robust-DPFL can effectively enhance the privacy protection and robustness of federated learning meanwhile maintain the task performance.

In Visually-rich Document Understanding (VrDU), recent advances of incorporating layout and image features into the pre-training language models have achieved significant progress. Existing methods usually developed complicated dedicated architectures based on pre-trained models and fine-tuned them with costly high-quality data to eliminate the inconsistency of knowledge distribution between the pre-training task and specialized downstream tasks. However, due to their huge data demands, these methods are not suitable for few-shot settings, which are essential for quick applications with limited resources but few previous works are presented. To solve these problems, we propose a unified Knowledge-aware prompt-tuning framework for Visual-rich Document Understanding (KnowVrDU) to enable broad utilization for diverse concrete applications and reduce data requirements. To model heterogeneous VrDU structures without designing task-specific architectures, we propose to reformulate various VrDU tasks into a single question-answering format with task-specific prompts and train the pre-trained model with the parameter-efficient prompt tuning method. To bridge the knowledge gap between the pre-training task and specialized VrDU tasks without additional annotations, we propose a prompt knowledge integration mechanism to leverage external open-source knowledge bases. We conduct experiments on several benchmark datasets in few-shot settings and the results validate the effectiveness of our method.

Steganography, which hides messages within innocent-looking carriers, is an essential technique to protect data privacy. The rapid advancement of generative models makes AI-generated images a potential steganographic carrier. However, the distortion resulting from the embedding of messages makes it difficult for steganographic images to escape deep-learning based detection, especially since such distortion is more pronounced in generative images. To improve the imperceptibility of steganographic generative images, in this paper we propose a cover-source switching based steganographic scheme employing image retouching. Specifically, we utilize an image-adaptive LUTs (LookUp Tables) model to generate the LUT required to retouch the cover image. The generated LUT is then applied to the stego image, effectively obfuscating the steganographic behavior. To ensure accurate extraction, we introduce wet cost to mark ambiguous elements that are strictly prohibited from modification. Experimental results show that our scheme can significantly improve the imperceptibility of the steganographic generative images. Leveraging the reproducibility of generative images, we are able to embed secrets at the pixel level, resulting in higher payload and extraction accuracy compared to existing cover-source switching based steganographic methods.

Prompt-based Large Language Models(LLMs) are surprisingly powerful in generating natural language reasoning steps or Chains of Thoughts(CoT) for multi-hop question answering(QA). However, LLMs struggle when they lack access to necessary knowledge or when the knowledge within their parameters is outdated. Additionally, LLMs that rely solely on CoT tend to generate hallucinations during the reasoning process. To address these dilemmas, we propose the Chain of Question (CoQ) framework, a novel multi-hop QA approach. This approach decomposes a complex original question into multiple sub-questions according to a CoT to retrieve knowledge from an external knowledge base. It then answers the question process based on the retrieved knowledge in accordance with a CoT. We design that each point of thought generated during the reasoning process be supported by the knowledge retrieved in the external knowledge base. Experiments show that CoQ is effective in reducing model hallucinations, leading to higher factual accuracy than CoT. On average, it reduces factual errors by 31% over CoT, and even by 38% on the two most commonly used models today.

Generative Linguistic Steganography (GLS) is applied to protect privacy against excessive censorship by employing Language Models (LMs) to hide privacy messages in texts. To effectively circumvent censorship, GLS generates steganographic texts (stegos) that closely resemble normal human texts (covers) as possible. However, due to the inherent distribution difference between LM-generated text and human covers, existing methods that simply use LMs to generate stegos face challenges in achieving sufficient imperceptibility. To narrow the gap between stegos and covers, this paper proposes a distribution reformation method named ${\mathbf{Frequency}}$ ${\mathbf{REformed}}$ ${\mathbf{Softmax}}$ $\left( {{\mathbf{FREmax}}} \right)$. ${\mathbf{FREmax}}$ generates highly imperceptible stegos aligned with human text by reforming the softmax function in the generation stage of LMs. This reformation is based on the frequency distribution of tokens in the human corpus, ensuring that the distribution of LM-generated stegos closely resembles that of humans. Extensive experimental results show that ${\mathbf{FREmax}}$ improves the linguistic quality and imperceptibility of the generated stegos, providing a valuable remedy to existing GLS methods .1

Social media platforms, with their extensive and real-time text data, are important application environments for linguistic generative steganography. However, the fragmented and context-constrained nature of social media text leads to a notably low capacity for hiding messages, making linguistic steganography impractical in real social media platforms. More frustratingly, even for high-capacity linguistic generative steganography, the upper bound of capacity is limited to a low level when required to meet a slightly strict security level under Cachin's model. To overcome the low capacity challenge, we identified an indicator of capacity that is independent of any specific steganography method to analyze the origins of this challenge, then we proposed a novel linguistic steganography framework named Collaborative Steganography (Co-Stega). Co-Stega utilizes existing texts and contextual relevance between texts in social media, collaboratively embedding secret messages in an existing text and its contextually related text via efficient retrieval and generation respectively. Additionally, we proposed an innovative and simple technique called "Entropy Enhancement Strategy", which effectively increases the entropy of generated text, thereby enhancing capacity further. Our evaluation shows that Co-Stega significantly improves capacity and maintains text quality, making it a valuable extension for linguistic generative steganography in social media platforms.

Medical Knowledge Graph (KG) has shown great potential in various healthcare scenarios, such as drug recommendation and clinical decision support system. The factors that determine the role of a medical KG in practical applications are the scale, coverage, and quality of the medical knowledge it can provide. Most existing medical KGs are extracted from a single or a few information sources. However, medical knowledge extracted from insufficient information sources is usually highly incomplete or even biased, which results in a lack of data completeness and may lessen their effectiveness in real-world scenarios. Besides, the coverage of entity and relation types is inadequate in most previous works, which also might restrict their potential usage in future applications. In this paper, we build a unified system that can extract and manage medical knowledge from heterogeneous information sources. We first employ named entity recognition and relation extraction methods to extract knowledge triplets from medical texts. Then we propose a hierarchical entity alignment framework for further knowledge refinement. Based on our system, we construct a large-scale, high-quality, multi-source, and multi-lingual medical KG named LMKG, which includes 13 entity types and 17 relation types, and contains 403,784 entity and 1,225,097 relation instances. We conduct extensive experiments to evaluate the quality of LMKG. Experimental results show that LMKG can effectively enhance the performance of both upstream and downstream intelligent medicine applications. We have publicly released the KG resources and corresponding management service interface to facilitate research and applications in the medical field.

Linguistic steganography based on text generation is a hot topic in the field of text information hiding. Previous studies have managed to improve the syntactic quality of steganography texts using natural language processing techniques based on deep learning, but their steganography models still lack the ability to control the semantic and contextual characteristics in texts, which is caused by the shortage of relevant information they can obtain. This results in a great decline in the imperceptibility of steganographic texts. To address the problem, we propose a context-aware linguistic steganography method based on neural machine translation called NMT-Stega. The model generates translation containing secret messages based on the neural machine translation model with semantic fusion and language model reference units. In this way, the semantics and contexts of translation are controlled by the additional semantic and contextual features acquired from the text to be translated. Also, a new encoding that combines arithmetic coding with a waiting mechanism is proposed in our model. This method solves the low embedding capacity problem of waiting mechanism while ensuring the semantic and contextual characteristics of steganographic text are less modified. Experimental results show that our model outperforms the previous models and encoding methods in semantic correlation, embedding capacity and imperceptibility.

Variational autoencoder (VAE) is widely used in tasks of unsupervised text generation due to its potential of deriving meaningful latent spaces, which, however, often assumes that the distribution of texts follows a common yet poor-expressed isotropic Gaussian. In real-life scenarios, sentences with different semantics may not follow simple isotropic Gaussian. Instead, they are very likely to follow a more intricate and diverse distribution due to the inconformity of different topics in texts. Considering this, we propose a flow-enhanced VAE for topic-guided language modeling (FET-LM). The proposed FET-LM models topic and sequence latent separately, and it adopts a normalized flow composed of householder transformations for sequence posterior modeling, which can better approximate complex text distributions. FET-LM further leverages a neural latent topic component by considering learned sequence knowledge, which not only eases the burden of learning topic without supervision but also guides the sequence component to coalesce topic information during training. To make the generated texts more correlative to topics, we additionally assign the topic encoder to play the role of a discriminator. Encouraging results on abundant automatic metrics and three generation tasks demonstrate that the FET-LM not only learns interpretable sequence and topic representations but also is fully capable of generating high-quality paragraphs that are semantically consistent.

Yongfeng Huang, Yanyang Li, Yichong Xu, Lin Zhang, Ruyi Gan, Jiaxing Zhang, Liwei Wang. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Xinyu Zhu, Junjie Wang, Lin Zhang, Yuxiang Zhang, Yongfeng Huang, Ruyi Gan, Jiaxing Zhang, Yujiu Yang. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

In this work, we introduce the concept of complex text style transfer tasks, and constructed complex text datasets based on two widely applicable scenarios. Our dataset is the first large-scale data set of its kind, with 700 rephrased sentences and 1,000 sentences from the game Genshin Impact. While large language models (LLM) have shown promise in complex text style transfer, they have drawbacks such as data privacy concerns, network instability, and high deployment costs. To address these issues, we explore the effectiveness of small models (less than T5-3B) with implicit style pre-training through contrastive learning. We also propose a method for automated evaluation of text generation quality based on alignment with human evaluations using ChatGPT. Finally, we compare our approach with existing methods and show that our model achieves state-of-art performances of few-shot text style transfer models.

Linguistic steganalysis is the technology to distinguish whether looking-innocent texts hide covert (possibly hazardous) messages. Traditional methods, dominantly focusing on internal linguistic difference in texts, are seriously challenged by the recent linguistic steganography technology that can reduce the difference to near zero. However, even via the most advanced linguistic steganography methods, due to the random and uncontrollable message bits, steganographic texts may express content against common sense knowledge. To fully employ this defect of linguistic steganography, we propose LINK, a novel Linguistic steganalysis framework with the help of external Knowledge. We link texts to the external knowledge database, and employ Graph Neural Networks (GNNs) to translate linked knowledge into knowledge features, while linguistic features will be captured by the same modules from existing methods. Knowledge features and linguistic features will be combined to make final decisions. Extensive experimental results show that owing to additional external knowledge, the proposed framework can effectively compensate for the shortcomings of existing methods.1

Federated learning (FL) is an important technique for learning models from decentralized data in a privacy-preserving way. Existing FL methods usually uniformly sample clients for local model learning in each round. However, different clients may have significantly different data sizes, and the clients with more data cannot have more opportunities to contribute to model training, which may lead to inferior performance. In this paper, instead of client uniform sampling, we propose a novel data uniform sampling strategy for federated learning (FedSampling), which can effectively improve the performance of federated learning especially when client data size distribution is highly imbalanced across clients. In each federated learning round, local data on each client is randomly sampled for local model learning according to a probability based on the server desired sample size and the total sample size on all available clients. Since the data size on each client is privacy-sensitive, we propose a privacy-preserving way to estimate the total sample size with a differential privacy guarantee. Experiments on four benchmark datasets show that FedSampling can effectively improve the performance of federated learning.

With intelligent devices’ prevalence, mobile edge computing (MEC) technology can effectively process the massive data in the Internet of Things (IoT). The metadata produced in MEC plays a role in colossal data management. However, it also includes lots of user privacy information. Traditional privacy protection solutions sacrifice the metadata’s availability or are incompatible with the distributed environment. To overcome this dilemma, we propose a blockchain-based secure searching strategy for metadata (BSSMeta) in MEC. By leveraging a lightweight proxy re-encryption scheme and a decentralized key generation method on the blockchain, BSSMeta keeps the security of the metadata searching tasks. Meanwhile, a main/secondary smart contract method and a buffer uploading strategy are proposed for the efficiency of the searching and uploading processes. As a byproduct, BSSMeta also supports searching metadata in a multiuser environment and gives users access control to metadata. Finally, we provide a security analysis of BSSMeta and implement a prototype on the Hyperledger Fabric platform. The results of experiments on various workloads show that BSSMeta is feasible and flexible in security and efficiency.

Due to the widespread use of text in cyberspace, linguistic steganography, which hides secret information into normal texts, develops quickly in these years. While linguistic steganography protects users’ privacy, it also has the risk of being abused to endanger network security. Therefore, its corresponding detection technology, namely linguistic steganalysis, has attracted more and more researchers’ attention in the past several years. However, most of the current linguistic steganalysis methods rely heavily on a large number of labeled samples, which presents a significant gap from real-world scenarios where labeled steganographic samples are difficult to obtain. In this paper, we proposed the Pre-trained Language model with Self-training for Few-shot Linguistic Steganalysis (LSFLS) method which effectively copes with few-shot linguistic steganalysis through a small number of labeled samples and some auxiliary unlabeled samples. Numerous experiments have proved that the proposed method can achieve high detection accuracy of linguistic steganalysis when only a few labeled samples are provided (even less than 10), significantly improving the detection ability of existing methods in few-shot scenario. Furthermore, the experimental results demonstrate that the proposed method can maintain good detection capability in the case of data source mismatch and label unbalance. We believe that our work will greatly advance the practical application of linguistic steganalysis techniques.

With the rapid development of the internet and social media, linguistic steganography can be easily abused in social networks to make considerable damage to varied aspects like personal privacy, network virus and national defense. Currently, considerable linguistic steganalysis methods are proposed to detect harmful steganographic carriers. However, almost all the existing methods fail in real social networks, since they are only devoted to the linguistic features that are extreme insufficient owing to the extreme sparsity and extreme fragmentation challenges of real social networks. In this paper, we attempt to fill the long-standing gap that the datasets and effective methods are absent for hunting steganographic texts in social network scenarios. Concretely, we construct a dataset called Stego-Sandbox to simulate the real social network scenarios, which contains texts and their relation. And we propose an effective linguistic steganalysis framework integrating linguistic features contained in texts and context features represented by these connections. Extensive experimental results demonstrate owing to the captured context features, our proposed framework can effectively compensate for shortcomings of these existing methods and tremendously improve their detection ability in real social network scenarios.

Personalized news recommendation is important for users to find interesting news information and alleviate information overload. Although it has been extensively studied over decades and has achieved notable success in improving user experience, there are still many problems and challenges that need to be further studied. To help researchers master the advances in personalized news recommendation, in this article, we present a comprehensive overview of personalized news recommendation. Instead of following the conventional taxonomy of news recommendation methods, in this article, we propose a novel perspective to understand personalized news recommendation based on its core problems and the associated techniques and challenges. We first review the techniques for tackling each core problem in a personalized news recommender system and the challenges they face. Next, we introduce the public datasets and evaluation methods for personalized news recommendation. We then discuss the key points on improving the responsibility of personalized news recommender systems. Finally, we raise several research directions that are worth investigating in the future. This article can provide up-to-date and comprehensive views on personalized news recommendation. We hope this article can facilitate research on personalized news recommendation as well as related fields in natural language processing and data mining.

Relational triple extraction is a critical task for natural language processing.Existing methods mainly focused on capturing semantic information, but suffered from ignoring the syntactic structure of the sentence, which is proved in the relation classification task to contain rich relational information.This is due to the absence of entity locations, which is the prerequisite for pruning noisy edges from the dependency tree, when extracting relational triples.In this paper, we propose a unified framework to tackle this challenge and incorporate syntactic information for relational triple extraction.First, we propose to automatically contract the dependency tree into a core relational topology and eliminate redundant information with graph pooling operations.Then, we propose a symmetrical expanding path with graph unpooling operations to fuse the contracted core syntactic interactions with the original sentence context.We also propose a bipartite graph matching objective function to capture the reflections between the core topology and golden relational facts.Since our model shares similar contracting and expanding paths with encoder-decoder models like U-Net, we name our model as Relation U-Net (RelU-Net).We conduct experiments on several datasets and the results prove the effectiveness of our method.

InfoNCE is a widely used contrastive training loss. It aims to estimate the mutual information between a pair of variables by discriminating between each positive pair and its associated K negative pairs. It is proved that when the sample labels are clean, the lower bound of mutual information estimation is tighter when more negative samples are incorporated, which usually yields better model performance. However, in practice the labels often contain noise, and incorporating too many noisy negative samples into model training may be suboptimal. In this paper, we study how many negative samples are optimal for InfoNCE in different scenarios via a semi-quantitative theoretical framework. More specifically, we first propose a probabilistic model to analyze the influence of the negative sampling ratio K on training sample informativeness. Then, we design a training effectiveness function to measure the overall influence of training samples based on their informativeness. We estimate the optimal negative sampling ratio using the K value that maximizes the training effectiveness function. Based on our framework, we further propose an adaptive negative sampling method that can dynamically adjust the negative sampling ratio to improve InfoNCE-based model training. Extensive experiments in three different tasks show our framework can accurately predict the optimal negative sampling ratio, and various models can benefit from our adaptive negative sampling method.

Federated learning (FL) is a feasible technique to learn personalized recommendation models from decentralized user data. Unfortunately, federated recommender systems are vulnerable to poisoning attacks by malicious clients. Existing recommender system poisoning methods mainly focus on promoting the recommendation chances of target items due to financial incentives. In fact, in real-world scenarios, the attacker may also attempt to degrade the overall performance of recommender systems. However, existing general FL poisoning methods for degrading model performance are either ineffective or not concealed in poisoning federated recommender systems. In this paper, we propose a simple yet effective and covert poisoning attack method on federated recommendation, named FedAttack. Its core idea is using globally hardest samples to subvert model training. More specifically, the malicious clients first infer user embeddings based on local user profiles. Next, they choose the candidate items that are most relevant to the user embeddings as hardest negative samples, and find the candidates farthest from the user embeddings as hardest positive samples. The model gradients inferred from these poisoned samples are then uploaded for aggregation. Extensive experiments on two benchmark datasets show that FedAttack can effectively degrade the performance of various federated recommender systems, meanwhile cannot be effectively detected nor defended by many existing methods.

Vertical federated learning (VFL) is a privacy-preserving machine learning paradigm that can learn models from features distributed on different platforms in a privacy-preserving way. Since in real-world applications the data may contain bias on fairness-sensitive features (e.g., gender), VFL models may inherit bias from training data and become unfair for some user groups. However, existing fair machine learning methods usually rely on the centralized storage of fairness-sensitive features to achieve model fairness, which are usually inapplicable in federated scenarios. In this paper, we propose a fair vertical federated learning framework (FairVFL), which can improve the fairness of VFL models. The core idea of FairVFL is to learn unified and fair representations of samples based on the decentralized feature fields in a privacy-preserving way. Specifically, each platform with fairness-insensitive features first learns local data representations from local features. Then, these local representations are uploaded to a server and aggregated into a unified representation for the target task. In order to learn a fair unified representation, we send it to each platform storing fairness-sensitive features and apply adversarial learning to remove bias from the unified representation inherited from the biased data. Moreover, for protecting user privacy, we further propose a contrastive adversarial learning method to remove private information from the unified representation in server before sending it to the platforms keeping fairness-sensitive features. Experiments on three real-world datasets validate that our method can effectively improve model fairness with user privacy well-protected.

News recommendation aims to match news with personalized user interest. Existing methods for news recommendation usually model user interest from historical clicked news without the consideration of candidate news. However, each user usually has multiple interests, and it is difficult for these methods to accurately match a candidate news with a specific user interest. In this paper, we present a candidate-aware user modeling method for personalized news recommendation, which can incorporate candidate news into user modeling for better matching between candidate news and user interest. We propose a candidate-aware self-attention network that uses candidate news as clue to model candidate-aware global user interest. In addition, we propose a candidate-aware CNN network to incorporate candidate news into local behavior context modeling and learn candidate-aware short-term user interest. Besides, we use a candidate-aware attention network to aggregate previously clicked news weighted by their relevance with candidate news to build candidate-aware user representation. Experiments on real-world datasets show the effectiveness of our method in improving news recommendation performance.

User modeling is important for news recommendation. Existing methods usually first encode user's clicked news into news embeddings independently and then aggregate them into user embedding. However, the word-level interactions across different clicked news from the same user, which contain rich detailed clues to infer user interest, are ignored by these methods. In this paper, we propose a fine-grained and fast user modeling framework (FUM) to model user interest from fine-grained behavior interactions for news recommendation. The core idea of FUM is to concatenate the clicked news into a long document and transform user modeling into a document modeling task with both intra-news and inter-news word-level interactions. Since vanilla transformer cannot efficiently handle long document, we apply an efficient transformer named Fastformer to model fine-grained behavior interactions. Extensive experiments on two real-world datasets verify that FUM can effectively and efficiently model user interest for news recommendation.

News recommendation aims to help online news platform users find their preferred news articles. Existing news recommendation methods usually learn models from historical user behaviors on news. However, these behaviors are usually biased on news providers. Models trained on biased user data may capture and even amplify the biases on news providers, and are unfair for some minority news providers. In this paper, we propose a provider fairness-aware news recommendation framework (named ProFairRec), which can learn news recommendation models fair for different news providers from biased user data. The core idea of ProFairRec is to learn provider-fair news representations and provider-fair user representations to achieve provider fairness. To learn provider-fair representations from biased data, we employ provider-biased representations to inherit provider bias from data. Provider-fair and -biased news representations are learned from news content and provider IDs respectively, which are further aggregated to build fair and biased user representations based on user click history. All of these representations are used in model training while only fair representations are used for user-news matching to achieve fair news recommendation. Besides, we propose an adversarial learning task on news provider discrimination to prevent provider-fair news representation from encoding provider bias. We also propose an orthogonal regularization on provider-fair and -biased representations to better reduce provider bias in provider-fair representations. Moreover, ProFairRec is a general framework and can be applied to different news recommendation methods. Extensive experiments on a public dataset verify that our ProFairRec approach can effectively improve the provider fairness of many existing methods and meanwhile maintain their recommendation accuracy.

User modeling is critical for personalization. Existing methods usually train user models from task-specific labeled data, which may be insufficient. In fact, there are usually abundant unlabeled user behavior data that encode rich universal user information, and pre-training user models on them can empower user modeling in many downstream tasks. In this paper, we propose a user model pre-training method named UserBERT to learn universal user models on unlabeled user behavior data with two contrastive self-supervision tasks. The first one is masked behavior prediction and discrimination, aiming to model the contexts of user behaviors. The second one is behavior sequence matching, aiming to capture user interest stable in different periods. Besides, we propose a medium-hard negative sampling framework to select informative negative samples for better contrastive pre-training. Extensive experiments validate the effectiveness of UserBERT in user model pre-training.

News recommendation is often modeled as a sequential recommendation task, assuming there are rich short-term dependencies over historical clicked news. However, users usually have strong preferences on the temporal diversity of news information and may not tend to click similar news successively, which is very different from many sequential recommendation scenarios such as e-commerce recommendation. In this paper, we study whether news recommendation can be regarded as a standard sequential recommendation problem. Through extensive experiments on two real-world datasets, we find it suboptimal to model news recommendation as a conventional sequential recommendation problem. To handle this issue, we further propose a temporal diversity-aware sequential news recommendation method that can promote candidate news that are diverse from recently clicked news to help predict future clicks more accurately. Experiments show that our method can empower various news recommendation methods.

News representation is critical for news recommendation. Most existing methods learn news representations only from news texts while ignoring the visual information of news. In fact, users may click news not only due to the interest in news titles but also the attraction of news images. Thus, images are useful for representing news and predicting news clicks. Pretrained visiolinguistic models are powerful in multi-modal understanding, which can represent news from both textual and visual contents. In this paper, we propose a multimodal news recommendation method that can incorporate both textual and visual information of news to learn multimodal news representations. We first extract region-of-interests (ROIs) from news images via object detection. We then use a pre-trained visiolinguistic model to encode both news texts and image ROIs and model their inherent relatedness using co-attentional Transformers. In addition, we propose a crossmodal candidate-aware attention network to select relevant historical clicked news for the accurate modeling of user interest in candidate news. Experiments validate that incorporating multimodal news information can effectively improve the performance of news recommendation.

Accurate user interest modeling is important for news recommendation. Most existing methods for news recommendation rely on implicit feedbacks like click for inferring user interests and model training. However, click behaviors usually contain heavy noise, and cannot help infer complicated user interest such as dislike. Besides, the feed recommendation models trained solely on click behaviors cannot optimize other objectives such as user engagement. In this paper, we present a news feed recommendation method that can exploit various kinds of user feedbacks to enhance both user interest modeling and model training. We propose a unified user modeling framework to incorporate various explicit and implicit user feedbacks to infer both positive and negative user interests. In addition, we propose a strong-to-weak attention network that uses the representations of stronger feedbacks to distill positive and negative user interests from implicit weak feedbacks for accurate user interest modeling. Besides, we propose a multi-feedback model training framework to learn an engagement-aware feed recommendation model. Extensive experiments on a real-world dataset show that our approach can effectively improve the model performance in terms of both news clicks and user engagement.

With the rapid development of natural language processing technology and linguistic steganography, linguistic steganalysis gains considerable interest in recent years. Current advanced methods dominantly focus on statistical features in semantic view yet ignore syntax structure of text, which leads to limited performance to some newly statistically indistinguishable steganography algorithms. To fill this gap, in this paper, we propose a novel linguistic steganalysis framework named SeSy to integrate bothsemantic andsyntactic features. Specifically, we propose to employ transformer-architecture language model as semantics extractor and leverage a graph attention network to retain syntactic features. Extensive experimental results show that owing to additional syntactic information, the SeSy framework effectively brings about remarkable improvement to current advanced linguistic steganalysis methods.

Native ad is a popular type of online advertisement that has similar forms with the native content displayed on websites. Native ad click-through rate (CTR) prediction is useful for improving user experience and platform revenue. However, it is challenging due to the lack of explicit user intent, and user behaviors on the platform with native ads may be insufficient to infer users’ interest in ads. Fortunately, user behaviors exist on many online platforms that can provide complementary information for user-interest mining. Thus, leveraging multi-platform user behaviors is useful for native ad CTR prediction. However, user behaviors are highly privacy-sensitive, and the behavior data on different platforms cannot be directly aggregated due to user privacy concerns and data protection regulations. Existing CTR prediction methods usually require centralized storage of user behavior data for user modeling, which cannot be directly applied to the CTR prediction task with multi-platform user behaviors. In this article, we propose a federated native ad CTR prediction method named FedCTR, which can learn user-interest representations from cross-platform user behaviors in a privacy-preserving way. On each platform a local user model learns user embeddings from the local user behaviors on that platform. The local user embeddings from different platforms are uploaded to a server for aggregation, and the aggregated ones are sent to the ad platform for CTR prediction. Besides, we apply local differential privacy and differential privacy to the local and aggregated user embeddings, respectively, for better privacy protection. Moreover, we propose a federated framework for collaborative model training with distributed models and user behaviors. Extensive experiments on real-world dataset show that FedCTR can effectively leverage multi-platform user behaviors for native ad CTR prediction in a privacy-preserving manner.

The task of rationalization aims to extract pieces of input text as rationales to justify neural network predictions on text classification tasks. By definition, rationales represent key text pieces used for prediction and thus should have similar classification feature distribution compared to the original input text. However, previous methods mainly focused on maximizing the mutual information between rationales and labels while neglecting the relationship between rationales and input text. To address this issue, we propose a novel rationalization method that matches the distributions of rationales and input text in both the feature space and output space. Empirically, the proposed distribution matching approach consistently outperforms previous methods by a large margin. Our data and code are available.

News recommendation is important for online news services. Existing news recommendation models are usually learned from users' news click behaviors. Usually the behaviors of users with the same sensitive attributes (e.g., genders) have similar patterns and news recommendation models can easily capture these patterns. It may lead to some biases related to sensitive user attributes in the recommendation results, e.g., always recommending sports news to male users, which is unfair since users may not receive diverse news information. In this paper, we propose a fairness-aware news recommendation approach with decomposed adversarial learning and orthogonality regularization, which can alleviate unfairness in news recommendation brought by the biases of sensitive user attributes. In our approach, we propose to decompose the user interest model into two components. One component aims to learn a bias-aware user embedding that captures the bias information on sensitive user attributes, and the other aims to learn a bias-free user embedding that only encodes attribute-independent user interest information for fairness-aware news recommendation. In addition, we propose to apply an attribute prediction task to the bias-aware user embedding to enhance its ability on bias modeling, and we apply adversarial learning to the bias-free user embedding to remove the bias information from it. Moreover, we propose an orthogonality regularization method to encourage the bias-free user embeddings to be orthogonal to the bias-aware one to better distinguish the bias-free user embedding from the bias-aware one. For fairness-aware news ranking, we only use the bias-free user embedding. Extensive experiments on benchmark dataset show that our approach can effectively improve fairness in news recommendation with minor performance loss.

Tao Qi, Fangzhao Wu, Chuhan Wu, Yongfeng Huang. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Tao Qi, Fangzhao Wu, Chuhan Wu, Peiru Yang, Yang Yu, Xing Xie, Yongfeng Huang. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Heterogeneous parallel steganography (HPS) has become a new trend of current streaming media voice steganography, which hides secret information in the frames of streaming media with multiple kinds of orthogonal steganography. Because of the complexity and imperceptibility of HPS, detecting its existence is a challenge for previous steganalysis methods, especially in the case of short sliding window length and low embedding rate. In order to improve the situation, we design a fast and efficient detection method named the key feature extraction and fusion network (KFEF) based on attention mechanism. The proposed model is able to effectively extract the key characteristic of the exceptions due to steganography and fuse the extracted features for different steganographic algorithms used in HPS. Experimental results show that the proposed method significantly improves the classification accuracy in detecting both low embedding rate samples and short segment samples. In addition, the detection time consumption is shorter than other methods and meets real-time requirements. Finally, with the help of attention we can predict the approximate locations of secret information which may bring new ideas to further steganalysis.

Accurate user modeling is critical for news recommendation. Existing news recommendation methods usually model users' interest from their behaviors via sequential or attentive models. However, they cannot model the rich relatedness between user behaviors, which can provide useful contexts of these behaviors for user interest modeling. In this paper, we propose a novel user modeling approach for news recommendation, which models each user as a personalized heterogeneous graph built from user behaviors to better capture the fine-grained behavior relatedness. In addition, in order to learn user interest embedding from the personalized heterogeneous graph, we propose a novel heterogeneous graph pooling method, which can summarize both node features and graph topology, and be aware of the varied characteristics of different types of nodes. Experiments on large-scale benchmark dataset show the proposed methods can effectively improve the performance of user modeling for news recommendation.

Yubo Chen, Yunqi Zhang, Changran Hu, Yongfeng Huang. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Transformer has achieved great success in the NLP field by composing various advanced models like BERT and GPT.However, Transformer and its existing variants may not be optimal in capturing token distances because the position or distance embeddings used by these methods usually cannot keep the precise information of real distances, which may not be beneficial for modeling the orders and relations of contexts.In this paper, we propose DA-Transformer, which is a distanceaware Transformer that can exploit the real distance.We propose to incorporate the real distances between tokens to re-scale the raw self-attention weights, which are computed by the relevance between attention query and key.Concretely, in different self-attention heads the relative distance between each pair of tokens is weighted by different learnable parameters, which control the different preferences on long-or short-term information of these heads.Since the raw weighted real distances may not be optimal for adjusting selfattention weights, we propose a learnable sigmoid function to map them into re-scaled coefficients that have proper ranges.We first clip the raw self-attention weights via the ReLU function to keep non-negativity and introduce sparsity, and then multiply them with the rescaled coefficients to encode real distance information into self-attention.Extensive experiments on five benchmark datasets show that DA-Transformer can effectively improve the performance of many tasks and outperform the vanilla Transformer and its several variants.

The most important task in personalized news recommendation is accurate matching between candidate news and user interest. Most of existing news recommendation methods model candidate news from its textual content and user interest from their clicked news in an independent way. However, a news article may cover multiple aspects and entities, and a user usually has different kinds of interest. Independent modeling of candidate news and user interest may lead to inferior matching between news and users. In this paper, we propose a knowledge-aware interactive matching method for news recommendation. Our method interactively models candidate news and user interest to facilitate their accurate matching. We design a knowledge-aware news co-encoder to interactively learn representations for both clicked news and candidate news by capturing their relatedness in both semantic and entities with the help of knowledge graphs. We also design a user-news co-encoder to learn candidate news-aware user interest representation and user-aware candidate news representation for better interest matching. Experiments on two real-world datasets validate that our method can effectively improve the performance of news recommendation.

Personalized news recommendation is an essential technique for online news services. News articles usually contain rich textual content, and accurate news modeling is important for personalized news recommendation.

Abstract With increasing number of cloud data leakage accidents exposed, outsourced data control becomes a more and more serious concern of their owner. To relieve the concern of these cloud users, reliable logging schemes are widely used to generate proof for data confidentiality auditing. However, high frequency operation and fine operation granularity on cloud data both result in a considerably large volume of operation logs, which burdens communication and computation in log auditing. This paper proposes a multi-grained log auditing scheme to make logs volume smaller and log auditing more efficient. We design a logging mechanism to support multi-grained data access with Merkle Hash Tree structure. Based on multi-grained log, we present a log auditing approach to achieve data confidentiality auditing and leakage investigation by making an Access List. Experiments results indicate that our scheme obtains about 54% log volume and 60% auditing time of fine-grained log auditing scheme in our scenario.

In recent years, linguistic generative steganography has been greatly developed. The previous works are mainly to optimize the perceptual-imperceptibility and statistical-imperceptibility of the generated steganographic text, and the latest developments show that they have been able to generate steganographic texts that look authentic enough. However, we noticed that these works generally cannot control the semantic expression of the generated steganographic text, and we believe this will bring potential security risks. We named this kind of security challenges as cognitive-imperceptibility. We think this is a new challenge that the generative steganography models must strive to overcome in the future. In this letter, we conduct some preliminary attempts to solve this challenge. Experimental results show that the proposed methods can further constrain the semantic expression of the generated steganographic text on the basis of ensuring certain perceptual-imperceptibility and statistical-imperceptibility, so as to enhance its cognitive-imperceptibility.

Previous works about linguistic steganography such as synonym substitution and sampling-based methods usually manipulate observed symbols explicitly to conceal secret information, which may give rise to security risks. In this letter, in order to preclude straightforward operation on observed symbols, we explored generation-based linguistic steganography in latent space by means of encoding secret messages in the selection of implicit attributes (semanteme) of natural language. We proposed a novel framework of linguistic semantic steganography based on rejection sampling strategy. Concretely, we utilized controllable text generation model for embedding and semantic classifier for extraction. In experiments, a model based on CTRL and BERT is implemented for further quantitative assessment. Results reveal that our approach is able to achieve satisfactory efficiency as well as nearly perfect imperceptibility. Our code is available at https://github.com/YangzlTHU/Linguistic-Steganography-and-Steganalysis/tree/master/Steganography/Linguistic-Semantic-Steganography.

In recent years, linguistic steganography based on text auto-generation technology has been greatly developed, which is considered to be a very promising but also a very challenging research topic. Previous works mainly focus on optimizing the language model and conditional probability coding methods, aiming at generating steganographic sentences with better quality. In this paper, we first report some of our latest experimental findings, which seem to indicate that the quality of the generated steganographic text cannot fully guarantee its steganographic security, and even has a prominent perceptual-imperceptibility and statistical-imperceptibility conflict effect (Psic Effect). To further improve the imperceptibility and security of generated steganographic texts, in this paper, we propose a new linguistic steganography based on Variational Auto-Encoder (VAE), which can be called VAE-Stega. We use the encoder in VAE-Stega to learn the overall statistical distribution characteristics of a large number of normal texts, and then use the decoder in VAE-Stega to generate steganographic sentences which conform to both of the statistical language model as well as the overall statistical distribution of normal sentences, so as to guarantee both the perceptual-imperceptibility and statistical-imperceptibility of the generated steganographic texts at the same time. We design several experiments to test the proposed method. Experimental results show that the proposed model can greatly improve the imperceptibility of the generated steganographic sentences and thus achieves the state of the art performance.

Pooling is an important technique for learning text representations in many neural NLP models. In conventional pooling methods such as average, max and attentive pooling, text representations are weighted summations of the L1 or L∞ norm of input features. However, their pooling norms are always fixed and may not be optimal for learning accurate text representations in different tasks. In addition, in many popular pooling methods such as max and attentive pooling some features may be over-emphasized, while other useful ones are not fully exploited. In this paper, we propose an Attentive Pooling with Learnable Norms (APLN) approach for text representation. Different from existing pooling methods that use a fixed pooling norm, we propose to learn the norm in an end-to-end manner to automatically find the optimal ones for text representation in different tasks. In addition, we propose two methods to ensure the numerical stability of the model training. The first one is scale limiting, which re-scales the input to ensure non-negativity and alleviate the risk of exponential explosion. The second one is re-formulation, which decomposes the exponent operation to avoid computing the real-valued powers of the input and further accelerate the pooling operation. Experimental results on four benchmark datasets show that our approach can effectively improve the performance of attentive pooling.

Emotion-cause pair extraction, which aims at extracting both the emotion and its corresponding cause in text, is a significant and challenging task in emotion analysis. Previous work formulated the task in a two-step framework, i.e., emotion and cause extraction, and emotion-cause relation classification. However, different tasks may correlate with each other and the two-step framework does not fully exploit the interactions between tasks. In this paper, we propose a multi-task neural network to perform emotion-cause pair extraction in a unified model. The task of relation classification is learned together with emotion and cause extraction. To this end, we develop a method to obtain training samples for relation classification without the dependence on the result of emotion and cause extraction. To fully exploit the interactions between different tasks, our model shares useful features across tasks. Moreover, we propose a method to incorporate position-aware emotion information in cause extraction to further improve the performance. Experimental results show that our model outperforms the state-of-the-art model on emotion-cause pair extraction.

Extracting correlation features between codes-words with high computational efficiency is crucial to steganalysis of Voice over IP (VoIP) streams. In this paper, we utilized attention mechanisms, which have recently attracted enormous interests due to their highly parallelizable computation and flexibility in modeling correlation in sequence, to tackle steganalysis problem of Quantization Index Modulation (QIM) based steganography in compressed VoIP stream. We design a light-weight neural network named Fast Correlation Extract Model (FCEM) only based on a variant of attention called multi-head attention to extract correlation features from VoIP frames. Despite its simple form, FCEM outperforms complicated Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs) models on both prediction accuracy and time efficiency. It significantly improves the best result in detecting both low embedded rates and short samples recently. Besides, the proposed model accelerates the detection speed as twice as before when the sample length is as short as 0.1s, making it a excellent method for online services.

With the growing attention on multimedia security and rapid development of natural language processing technologies, various linguistic steganographic algorithms based on automatic text generation technology have been proposed increasingly, which brings great challenges in maintaining security of cyberspace. The prevailing linguistic steganalysis methods based on neural networks only conduct linguistic steganalysis with feature vectors from last layer of neural network, which may be insufficient for neural linguistic steganalysis. In this paper, we propose a neural linguistic steganalysis scheme based on densely connected Long short-term memory networks (LSTM) with feature pyramids which can incorporate more low level features to detect generative text steganographic algorithms. In the proposed framework, words in text are firstly mapped into semantic space with a hidden representation for better exploitation of the semantic features. Then, stacked bidirectional Long short-term memory networks are ultilized to extract different levels of semantic features. In order to incorporate more low level features from neural networks, we introduced two components: dense connections and feature pyramids to enhance the low level features in feature vectors. Finally, the semantic features from all levels are fused and we use a sigmoid layer to categorize the input text as cover or stego. Experiments showed that the proposed scheme can achieve the state-of-the-art results in detecting recently proposed linguistic steganographic algorithms.

Modeling user interest is critical for accurate news recommendation. Existing news recommendation methods usually infer user interest from click behaviors on news. However, users may click a news article because attracted by its title shown on the news website homepage, but may not be satisfied with its content after reading. In many cases users close the news page quickly after click. In this paper we propose to model user interest from both click behaviors on news titles and reading behaviors on news content for news recommendation. More specifically, we propose a personalized reading speed metric to measure users’ satisfaction with news content. We learn embeddings of users from the news content they have read and their satisfaction with these news to model their interest in news content. In addition, we also learn another user embedding from the news titles they have clicked to model their preference in news titles. We combine both kinds of user embeddings into a unified user representation for news recommendation. We train the user representation model using two supervised learning tasks built from user behaviors, i.e., news title based click prediction and news content based satisfaction prediction, to encourage our model to recommend the news articles which not only are likely to be clicked but also have the content satisfied by the user. Experiments on real-world dataset show our method can effectively boost the performance of user modeling for news recommendation.

With the explosion of online news, personalized news recommendation becomes increasingly important for online news platforms to help their users find interesting information. Existing news recommendation methods achieve personalization by building accurate news representations from news content and user representations from their direct interactions with news (e.g., click), while ignoring the high-order relatedness between users and news. Here we propose a news recommendation method which can enhance the representation learning of users and news by modeling their relatedness in a graph setting. In our method, users and news are both viewed as nodes in a bipartite graph constructed from historical user click behaviors. For news representations, a transformer architecture is first exploited to build news semantic representations. Then we combine it with the information from neighbor news in the graph via a graph attention network. For user representations, we not only represent users from their historically clicked news, but also attentively incorporate the representations of their neighbor users in the graph. Improved performances on a large-scale real-world dataset validate the effectiveness of our proposed method.

in this letter, we present a novel and extremely fast steganalysis method for voice over ip (voip) streams, driven by the need for a quick and accurate detection of possible steganography in VoIP streams. We firstly analyzed the correlations in carriers. To better exploit the correlations in code-words, we mapped vector quantization code-words into a semantic space. In order to achieve high detection efficiency, only one hidden layer was utilized to extract the correlations between these code-words. Finally, based on the extracted correlation features, we used the softmax classifier to categorize the input stream carriers. To boost the performance of this proposed model, we incorporate a simple knowledge distillation framework into the training process. Experimental results show that the proposed method achieves state-of-the-art performance both in detection accuracy and efficiency. In particular, the processing time of this method on average is only about 0.05% when sample length is as short as 0.1 s, attaching strong practical value to online serving of steganography monitor.

Chatbots such as Xiaoice have gained huge popularity in recent years. Users frequently mention their favorite works such as songs and movies in conversations with chatbots. Detecting these entities can help design better chat strategies and improve user experience. Existing named entity recognition methods are mainly designed for formal texts, and their performance on the informal chatbot conversation texts may not be optimal. In addition, these methods rely on massive manually annotated data for model training. In this article, we propose a neural approach to detect entities of works for Chinese chatbot. Our approach is based on a language model (LM) long-short term memory (LSTM) convolutional neural network (CNN) conditional random value (CRF), or LM-LSTM-CNN-CRF, framework, which contains a language model to generate context-aware character embeddings, a Bi-LSTM network to learn contextual character representations from global contexts, a CNN to learn character representations from local contexts, and a CRF layer to jointly decode the character label sequence. In addition, we propose an automatic text annotation method via quote marks to reduce the effort of manual annotation. Besides, we propose an iterative data purification method to improve the quality of the automatically constructed labeled data. Massive experiments on a real-world dataset validate that our approach can achieve good performance on entity detection for Chinese chatbots.

News recommendation can help users find interested news and alleviate information overload.The topic information of news is critical for learning accurate news and user representations for news recommendation.However, it is not considered in many existing news recommendation methods.In this paper, we propose a neural news recommendation approach with topic-aware news representations.The core of our approach is a topic-aware news encoder and a user encoder.In the news encoder we learn representations of news from their titles via CNN networks and apply attention networks to select important words.In addition, we propose to learn topic-aware news representations by jointly training the news encoder with an auxiliary topic classification task.In the user encoder we learn the representations of users from their browsed news and use attention networks to select informative news for user representation learning.Extensive experiments on a real-world dataset validate the effectiveness of our approach.

Estimating in-hospital costs from medical records is an important task with many applications such as accountable care. Existing methods for this task usually rely on manual feature engineering which needs massive domain knowledge, and do not exploit the textual information in medical records, e.g., diagnosis and operation texts. In this paper, we propose a neural in-hospital cost estimation (NICE) approach to estimate the in-hospital costs of patients from their admission records. Our approach can exploit the heterogeneous information in records, such as patient features, diagnosis/operation texts, and the diagnosis/operation IDs, via a multi-view learning framework. In addition, since different words, diagnoses and operations have different importance for cost estimation, we propose a hierarchical attention network to select important words, diagnoses and operations for learning informative record representations. Extensive experiments on a real-world medical dataset validate the effectiveness of our approach.

Sentiment classification is an important task in the sentiment analysis field. Many deep learning based sentiment classification methods have been proposed in recent years. However, these methods usually rely on massive labeled texts to train sentiment classifiers, which are expensive and time-consuming to annotate. Luckily, many high-quality sentiment lexicons have been constructed and can cover a large number of sentiment words. Since sentiment words are the basic units to convey sentiments in texts, these sentiment lexicons have the potential to improve the performance of neural sentiment classification. In this paper, we propose two approaches to exploit sentiment lexicons to enhance neural sentiment classification. In our first approach we use sentiment lexicons to learn sentiment-aware attentions. We propose a word sentiment classification task to classify the sentiments of words in a sentence based on their hidden representations in the attention network of neural sentiment classification models. We jointly train this task with neural sentiment classifier to facilitate the attention network to recognize and highlight sentiment-bearing words. In our second approach we use sentiment lexicons to learn sentiment-aware word embeddings. We design an auxiliary task to classify the sentiments of words in sentiment lexicons based on their word embeddings, and jointly train this task with neural sentiment classifier to encode sentiment information in sentiment lexicons to word embeddings. Extensive experiments on three benchmark datasets validate the effectiveness of our approach.

Review rating prediction is an important task in data mining and natural language processing fields, and has wide applications. Users usually express opinions towards many aspects in their reviews, and the overall review rating is a synthesis of these opinions. However, most existing review rating prediction methods ignore users' opinions on aspects, which is insufficient. In this paper, we propose a neural aspect-aware rating prediction approach for Chinese reviews. In our approach we propose a collaborative learning framework to jointly train review-level rating predictor and multiple aspect-level rating predictors. In our framework different rating predictors share the same review encoder model to exploit the inherent relatedness between them, but have different attention networks to focus on different informative texts for each task. The final review representation for rating prediction is a concatenation of the review representations from all predictors. Since word segmentation of Chinese reviews is usually inaccurate, we propose a multi-view learning model to learn review representations from both words and characters. Extensive experiments on real-world dataset validate the effectiveness of our approach.

Demographics of social media users such as gender are very important for personalized online services. However, the gender information of many users is usually not available. Luckily, the messages posted by social media users can provide rich clues for inferring their genders, since male and female users usually have differences in their message content. In addition, users with different genders often have different patterns in expressing emotions. In this paper, we propose a neural approach for gender prediction in social media based on both content and emotion of messages posted by users. The core of our approach is an emotion-aware hierarchical user representation model. Our model first learns message representations from words using message encoder and then learns user representations from messages using user encoder with hierarchical attention networks selecting important words and messages to learn informative user representations. In addition, we propose two methods to incorporate emotion information in messages into user representation learning. The first one is to incorporate emotion-aware message representations generated by a pre-trained emotion classifier into message representations. The second one is to train emotion-aware message encoders via jointly training our model with an auxiliary emotion classification task. Extensive experiments on two real-world datasets validate the effectiveness of our approach.

Neural network methods have achieved great success in sentiment classification. Recent studies have found that incorporating user and product information can effectively improve the performance of review sentiment classification. However, most of these studies only concentrate on the influence of users and products, ignoring the inherent correlation between users or products. This information is important for users or products since they can obtain more information from similar users or products. In this paper, we propose a novel framework for review rating prediction with user and product memory. First, besides the original user or product representations, we construct inferred representations from representative users or products which are stored in memory slots. These memory units can be viewed as refined knowledge representations of users or products learned from the data. Then, we employ two hierarchical networks with user attention and product attention using both the original and inferred representations. Experiments on benchmark datasets show that our method can achieve state-of-the-art performance. Besides, our approach performs much more better in cold-start scenarios where the training data is scarce.

Chuhan Wu, Fangzhao Wu, Mingxiao An, Tao Qi, Jianqiang Huang, Yongfeng Huang, Xing Xie. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Chuhan Wu, Fangzhao Wu, Suyu Ge, Tao Qi, Yongfeng Huang, Xing Xie. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Chuhan Wu, Fangzhao Wu, Tao Qi, Suyu Ge, Yongfeng Huang, Xing Xie. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Steganalysis of the Quantization Index Modulation (QIM) steganography in VoIP (Voice-over IP) stream is conducted in this research. VoIP is a popular media streaming and communication service on the Internet. QIM steganography makes it possible to hide secret information in VoIP streams. Detecting short and low embedding rates of QIM steganography samples remains an unsolved challenge. Recently, neural network models have been demonstrated to be capable of achieving remarkable performances and be successfully applied to many different tasks. The mainstream architectures of neural network include Convolution Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), which adopt totally different ways to understand various signals. In this paper, we first indicate a proper way to combine the strengths of these two architectures and then construct a novel and unified model called CNN-LSTM network to detect QIM-based steganography. In our model, Bidirectional Long Short-Term Memory Recurrent Neural Network (Bi-LSTM) is utilized to capture long time contextual information in carriers and CNN was used subsequently to capture both local features and global ones as well as temporal carrier features. Experiments showed that our model can achieve the state-of-art result in detecting QIM-based steganography in VoIP streams.

Personalized news recommendation is very important for online news platforms to help users find interested news and improve user experience. News and user representation learning is critical for news recommendation. Existing news recommendation methods usually learn these representations based on single news information, e.g., title, which may be insufficient. In this paper we propose a neural news recommendation approach which can learn informative representations of users and news by exploiting different kinds of news information. The core of our approach is a news encoder and a user encoder. In the news encoder we propose an attentive multi-view learning model to learn unified news representations from titles, bodies and topic categories by regarding them as different views of news. In addition, we apply both word-level and view-level attention mechanism to news encoder to select important words and views for learning informative news representations. In the user encoder we learn the representations of users based on their browsed news and apply attention mechanism to select informative news for user representation learning. Extensive experiments on a real-world dataset show our approach can effectively improve the performance of news recommendation.

News recommendation is very important to help users find interested news and alleviate information overload. Different users usually have different interests and the same user may have various interests. Thus, different users may click the same news article with attention on different aspects. In this paper, we propose a neural news recommendation model with personalized attention (NPA). The core of our approach is a news representation model and a user representation model. In the news representation model we use a CNN network to learn hidden representations of news articles based on their titles. In the user representation model we learn the representations of users based on the representations of their clicked news articles. Since different words and different news articles may have different informativeness for representing news and users, we propose to apply both word- and news-level attention mechanism to help our model attend to important words and news articles. In addition, the same news article and the same word may have different informativeness for different users. Thus, we propose a personalized attention network which exploits the embedding of user ID to generate the query vector for the word- and news-level attentions. Extensive experiments are conducted on a real-world news recommendation dataset collected from MSN news, and the results validate the effectiveness of our approach on news recommendation.

Demographics of online users such as age and gender play an important role in personalized web applications. However, it is difficult to directly obtain the demographic information of online users. Luckily, search queries can cover many online users and the search queries from users with different demographics usually have some difference in contents and writing styles. Thus, search queries can provide useful clues for demographic prediction. In this paper, we study predicting users' demographics based on their search queries, and propose a neural approach for this task. Since search queries can be very noisy and many of them are not useful, instead of combining all queries together for user representation, in our approach we propose a hierarchical user representation with attention (HURA) model to learn informative user representations from their search queries. Our HURA model first learns representations for search queries from words using a word encoder, which consists of a CNN network and a word-level attention network to select important words. Then we learn representations of users based on the representations of their search queries using a query encoder, which contains a CNN network to capture the local contexts of search queries and a query-level attention network to select informative search queries for demographic prediction. Experiments on two real-world datasets validate that our approach can effectively improve the performance of search query based age and gender prediction and consistently outperform many baseline methods.

Twitter is a popular social media platform for information sharing and dissemination. Many Twitter users post tweets to share their experiences about drugs and adverse drug reactions. Automatic detection of tweets mentioning drug names and adverse drug reactions at a large scale has important applications such as pharmacovigilance. However, detecting drug name and adverse drug reaction mentioning tweets is very challenging, because tweets are usually very noisy and informal, and there are massive misspellings and user-created abbreviations for these mentions. In addition, these mentions are usually context dependent. In this paper, we propose a neural approach with hierarchical tweet representation and multi-head self-attention mechanism to jointly detect tweets mentioning drug names and adverse drug reactions. In order to alleviate the influence of massive misspellings and user-created abbreviations in tweets, we propose to use a hierarchical tweet representation model to first learn word representations from characters and then learn tweet representations from words. In addition, we propose to use multi-head self-attention mechanism to capture the interactions between words to fully model the contexts of tweets. Besides, we use additive attention mechanism to select the informative words to learn more informative tweet representations. Experimental results validate the effectiveness of our approach.

Chinese word segmentation (CWS) is very important for Chinese text processing. Existing methods for CWS usually rely on a large number of labeled sentences to train word segmentation models, which are expensive and time-consuming to annotate. Luckily, the unlabeled data is usually easy to collect and many high-quality Chinese lexicons are off-the-shelf, both of which can provide useful information for CWS. In this paper, we propose a neural approach for Chinese word segmentation which can exploit both lexicon and unlabeled data. Our approach is based on a variant of posterior regularization algorithm, and the unlabeled data and lexicon are incorporated into model training as indirect supervision by regularizing the prediction space of CWS models. Extensive experiments on multiple benchmark datasets in both in-domain and cross-domain scenarios validate the effectiveness of our approach.

Chinese named entity recognition (CNER) is an important task in Chinese natural language processing field. However, CNER is very challenging since Chinese entity names are highly context-dependent. In addition, Chinese texts lack delimiters to separate words, making it difficult to identify the boundary of entities. Besides, the training data for CNER in many domains is usually insufficient, and annotating enough training data for CNER is very expensive and time-consuming. In this paper, we propose a neural approach for CNER. First, we introduce a CNN-LSTM-CRF neural architecture to capture both local and long-distance contexts for CNER. Second, we propose a unified framework to jointly train CNER and word segmentation models in order to enhance the ability of CNER model in identifying entity boundaries. Third, we introduce an automatic method to generate pseudo labeled samples from existing labeled data which can enrich the training data. Experiments on two benchmark datasets show that our approach can effectively improve the performance of Chinese named entity recognition, especially when training data is insufficient.

With the rapid development of natural language processing technology in the past few years, the steganography by text synthesis has been greatly developed. These methods can analyze the statistical feature distribution of a large number of training samples, and then generate steganographic texts that conform to such statistical distribution. For these steganography methods, previous steganalysis methods show unsatisfactory detection performance, which remains an unsolved problem and poses a great threat to the security of cyberspace. In this letter, we proposed a fast and efficient text steganalysis method to solve this problem. We first analyzed the correlations between words in these generated steganographic texts. Then, we map each word to a semantic space and used a hidden layer to extract the correlations between these words. Finally, based on the extracted correlation features, we used the softmax classifier to classify the input text. Experimental results show that the proposed model can achieve a high detection accuracy, which shows a state-of-the-art performance.

With the rapid development of natural language processing technologies, more and more text steganographic methods based on automatic text generation technology have appeared in recent years. These models use the powerful self-learning and feature extraction ability of the neural networks to learn the feature expression of massive normal texts. Then, they can automatically generate dense steganographic texts which conform to such statistical distribution based on the learned statistical patterns. In this letter, we observe that the conditional probability distribution of each word in the automatically generated steganographic texts will be distorted after embedded with hidden information. We use recurrent neural networks to extract these feature distribution differences and then classify those features into cover text and stego text categories. Experimental results show that the proposed model can achieve high detection accuracy. Besides, the proposed model can even make use of the subtle differences of the feature distribution of texts to estimate the amount of hidden information embedded in the generated steganographic text.

Linguistic steganography based on text carrier auto-generation technology is a current topic with great promise and challenges. Limited by the text automatic generation technology or the corresponding text coding methods, the quality of the steganographic text generated by previous methods is inferior, which makes its imperceptibility unsatisfactory. In this paper, we propose a linguistic steganography based on recurrent neural networks, which can automatically generate high-quality text covers on the basis of a secret bitstream that needs to be hidden. We trained our model with a large number of artificially generated samples and obtained a good estimate of the statistical language model. In the text generation process, we propose fixed-length coding and variable-length coding to encode words based on their conditional probability distribution. We designed several experiments to test the proposed model from the perspectives of information hiding efficiency, information imperceptibility, and information hidden capacity. The experimental results show that the proposed model outperforms all the previous related methods and achieves the state-of-the-art performance.

Sentence-level sentiment classification aims to mine fine-grained sentiment information from texts. Existing methods for this task are usually based on supervised learning and rely on massive labeled sentences for model training. However, annotating sufficient sentences is expensive and time-consuming. In this paper, we propose a neural sentence-level sentiment classification approach which can exploit heterogeneous sentiment supervision and reduce the dependence on labeled sentences. Besides the sentence-level supervision from labeled sentences, our approach can also incorporate the word-level supervision extracted from sentiment lexicons, document-level supervision extracted from labeled documents and sentiment relations between sentences extracted from unlabeled documents. A unified neural framework is proposed to fuse heterogeneous sentiment supervision to train sentence-level sentiment classification model. Experiments on benchmark datasets validate the effectiveness of our approach.

Quantization index modulation (QIM) steganography makes it possible to hide secret information in voice-over IP (VoIP) streams, which could be utilized by unauthorized entities to set up covert channels for malicious purposes. Detecting short QIM steganography samples, as is required by real circumstances, remains an unsolved challenge. In this paper, we propose an effective online steganalysis method to detect QIM steganography. We find four strong codeword correlation patterns in VoIP streams, which will be distorted after embedding with hidden data. To extract those correlation features, we propose the codeword correlation model, which is based on recurrent neural network (RNN). Furthermore, we propose the feature classification model to classify those correlation features into cover speech and stego speech categories. The whole RNN-based steganalysis model (RNN-SM) is trained in a supervised learning framework. Experiments show that on full embedding rate samples, RNN-SM is of high detection accuracy, which remains over 90% even when the sample is as short as 0.1 s, and is significantly higher than other state-of-the-art methods. For the challenging task of conducting steganalysis towards low embedding rate samples, RNN-SM also achieves a high accuracy. The average testing time for each sample is below 0.15% of sample length. These clues show that RNN-SM meets the short sample detection demand and is a state-of-the-art algorithm for online VoIP steganalysis.

This paper contributes a new large-scale dataset for weakly supervised cross-media retrieval, named Twitter100k. Current datasets, such as Wikipedia, NUS Wide, and Flickr30k, have two major limitations. First, these datasets are lacking in content diversity, i.e., only some predefined classes are covered. Second, texts in these datasets are written in well-organized language, leading to inconsistency with realistic applications. To overcome these drawbacks, the proposed Twitter100k dataset is characterized by two aspects: it has 100 000 image-text pairs randomly crawled from Twitter, and thus, has no constraint in the image categories; and text in Twitter100k is written in informal language by the users. Since strongly supervised methods leverage the class labels that may be missing in practice, this paper focuses on weakly supervised learning for cross-media retrieval, in which only text-image pairs are exploited during training. We extensively benchmark the performance of four subspace learning methods and three variants of the correspondence AutoEncoder, along with various text features on Wikipedia, Flickr30k, and Twitter100k. As a minor contribution, we also design a deep neural network to learn cross-modal embeddings for Twitter100k. Inspired by the characteristic of Twitter100k, we propose a method to integrate optical character recognition into cross-media retrieval. The experiment results show that the proposed method improves the baseline performance.

Domain adaptation is an important technology to handle domain dependence problem in sentiment analysis field.Existing methods usually rely on sentiment classifiers trained in source domains.However, their performance may heavily decline if the distributions of sentiment features in source and target domains have significant difference.In this paper, we propose an active sentiment domain adaptation approach to handle this problem.Instead of the source domain sentiment classifiers, our approach adapts the general-purpose sentiment lexicons to target domain with the help of a small number of labeled samples which are selected and annotated in an active learning mode, as well as the domain-specific sentiment similarities among words mined from unlabeled samples of target domain.A unified model is proposed to fuse different types of sentiment information and train sentiment classifier for target domain.Extensive experiments on benchmark datasets show that our approach can train accurate sentiment classifier with less labeled samples.

Data accountability can record and track data usage in cloud storage, in order to cope with users' fear of losing control of their own data or even data leakage. However, logs recording unauthorized data access could be omitted or falsified in untrusted cloud storage, which is the critical factor of unreliable data accountability. To address this problem, in this paper, we propose a novel Cloud Data Accountability Framework to ensure reliable logging for data accountability. In particular, we adopt programmable Java JAR file coupling with data to enclose access policy. This mechanism ensures that data access through JAR will trigger authentication and automated logging local to the JAR. To prevent data access without JAR and protect data from key abuse attack, we provide JAR-based Data Access Protocol. Extensive security and performance analysis makes comparison between our logging mechanism and the state-of-the-art. Results indicate that the proposed mechanism is more reliable and achieves space and time efficiency.

We propose a novel text steganography method using RNN Encoder-Decoder structure to generate quatrains, one genre of Chinese poetry. Compared to other text-generation based steganography methods which have either very low embedding rate or flaws in the naturalness of generated texts, our method has higher embedding rate and better text quality. In this paper, we use the LSTM Encoder-Decoder model to generate the first line of a quatrain with a keyword and then generate the following lines one by one. RNN has proved effective in generating poetry, but when applied to steganograpy, poetry quality decreases sharply, because of the redundancy we create to hide information. To overcome this problem, we propose a template-constrained generation method and develop a word-choosing approach using inner-word mutual information. Through a series of experiments, it is proven that our approach outperforms other poetry steganography methods in both embedding rate and poetry quality.

Sentence-level sentiment classification is important to understand users' fine-grained opinions. Existing methods for sentence-level sentiment classification are mainly based on supervised learning. However, it is difficult to obtain sentiment labels of sentences since manual annotation is expensive and time-consuming. In this paper, we propose an approach for sentence-level sentiment classification without the need of sentence labels. More specifically, we propose a unified framework to incorporate two types of weak supervision, i.e., document-level and word-level sentiment labels, to learn the sentence-level sentiment classifier. In addition, the contextual information of sentences and words extracted from unlabeled sentences is incorporated into our approach to enhance the learning of sentiment classifier. Experiments on benchmark datasets show that our approach can effectively improve the performance of sentence-level sentiment classification.

Recently, collaborative filtering algorithm is widely used in recommendation systems. Similarity calculation is an important step of this algorithm. Some of the existing similarity computing methods can not accurately measure the similarity, and the accuracy needs to be improved. In this paper, a new similarity calculation method is proposed, which is called JacRA. In order to verify the effectiveness of our proposed method, we compare several methods based on MovieLens data sets. Experimental results show that our method outperforms the contrast methods in terms of accuracy.

We propose a collaborative multi-domain sentiment classification approach to train sentiment classifiers for multiple domains simultaneously. In our approach, the sentiment information in different domains is shared to train more accurate and robust sentiment classifiers for each domain when labeled data is scarce. Specifically, we decompose the sentiment classifier of each domain into two components, a global one and a domain-specific one. The global model can capture the general sentiment knowledge and is shared by various domains. The domain-specific model can capture the specific sentiment expressions in each domain. In addition, we extract domain-specific sentiment knowledge from both labeled and unlabeled samples in each domain and use it to enhance the learning of domain-specific sentiment classifiers. Besides, we incorporate the similarities between domains into our approach as regularization over the domain-specific sentiment classifiers to encourage the sharing of sentiment information between similar domains. Two kinds of domain similarity measures are explored, one based on textual content and the other one based on sentiment expressions. Moreover, we introduce two efficient algorithms to solve the model of our approach. Experimental results on benchmark datasets show that our approach can effectively improve the performance of multi-domain sentiment classification and significantly outperform baseline methods.

Cloud storage is an increasingly popular application of cloud computing, which can provide on-demand outsourcing data services for both organizations and individuals. However, users may not fully trust the cloud service providers (CSPs) in that it is difficult to determine whether the CSPs meet their legal expectations for data security. Therefore, it is critical to develop efficient auditing techniques to strengthen data owners' trust and confidence in cloud storage. In this paper, we present a novel public auditing scheme for secure cloud storage based on dynamic hash table (DHT), which is a new two-dimensional data structure located at a third parity auditor (TPA) to record the data property information for dynamic auditing. Differing from the existing works, the proposed scheme migrates the authorized information from the CSP to the TPA, and thereby significantly reduces the computational cost and communication overhead. Meanwhile, exploiting the structural advantages of the DHT, our scheme can also achieve higher updating efficiency than the state-of-the-art schemes. In addition, we extend our scheme to support privacy preservation by combining the homomorphic authenticator based on the public key with the random masking generated by the TPA, and achieve batch auditing by employing the aggregate BLS signature technique. We formally prove the security of the proposed scheme, and evaluate the auditing performance by detailed experiments and comparisons with the existing ones. The results demonstrate that the proposed scheme can effectively achieve secure auditing for cloud storage, and outperforms the previous schemes in computation complexity, storage costs and communication overhead.

Multi-cloud storage is an emerging technique using a set of clouds to provide data storage service for clients cooperatively. Reliability reflects the risk of data on clouds to be damaged or tampered and is a basic consideration when choosing multi-cloud. However, there exists no work to help clients evaluate the reliability of cloud storage, which is an important consideration of choosing clouds. In this paper, we take the lead to propose the concept of reputation audit in multi-cloud storage, which makes such an evaluation a reality. To achieve a reputation audit scheme, we construct a framework and propose a concrete method based on integrity verification and data dynamics. The simulation results show that, by selecting and updating clouds based on the results of reputation audit, the reliability of multi-cloud storage service can be improved. Besides, we also show that the overhead of maintaining the reputation audit scheme is controllable.

Microblog sentiment classification is an interesting and important research topic with wide applications. Traditional microblog sentiment classification methods usually use a single model to classify the messages from different users and omit individuality. However, microblogging users frequently embed their personal character, opinion bias and language habits into their messages, and the same word may convey different sentiments in messages posted by different users. In this paper, we propose a personalized approach for microblog sentiment classification. In our approach, each user has a personalized sentiment classifier, which is decomposed into two components, a global one and a user-specific one. Our approach can capture the individual personality and at the same time leverage the common sentiment knowledge shared by all users. The personalized sentiment classifiers of massive users are trained in a collaborative way based on multi-task learning to handle the data sparseness problem. In addition, we incorporate users' social relations into our model to strengthen the learning of the personalized models. Moreover, we propose a distributed optimization algorithm to solve our model in parallel. Experiments on two real-world microblog sentiment datasets validate that our approach can improve microblog sentiment classification accuracy effectively and efficiently.

Domain adaptation is an important research topic in sentiment analysis area.Existing domain adaptation methods usually transfer sentiment knowledge from only one source domain to target domain.In this paper, we propose a new domain adaptation approach which can exploit sentiment knowledge from multiple source domains.We first extract both global and domain-specific sentiment knowledge from the data of multiple source domains using multi-task learning.Then we transfer them to target domain with the help of words' sentiment polarity relations extracted from the unlabeled target domain data.The similarities between target domain and different source domains are also incorporated into the adaptation process.Experimental results on benchmark dataset show the effectiveness of our approach in improving cross-domain sentiment classification performance.

Sentiment domain adaptation is widely studied to tackle the domain-dependence problem in sentiment analysis field. Existing domain adaptation methods usually train a sentiment classifier in a source domain and adapt it to the target domain using transfer learning techniques. However, when the sentiment feature distributions of the source and target domains are significantly different, the adaptation performance will heavily decline. In this paper, we propose a new sentiment domain adaptation approach by adapting the sentiment knowledge in general-purpose sentiment lexicons to a specific domain. Since the general sentiment words of general-purpose sentiment lexicons usually convey consistent sentiments in different domains, they have better generalization performance than the sentiment classifier trained in a source domain. In addition, we propose to extract various kinds of contextual sentiment knowledge from massive unlabeled samples in target domain and formulate them as sentiment relations among sentiment expressions. It can propagate the sentiment information in general sentiment words to massive domain-specific sentiment expressions. Besides, we propose a unified framework to incorporate these different kinds of sentiment knowledge and learn an accurate domain-specific sentiment classifier for target domain. Moreover, we propose an efficient optimization algorithm to solve the model of our approach. Extensive experiments on benchmark datasets validate the effectiveness and efficiency of our approach.

Low bit-rate speech codec offers so many advantages over other codecs that it has become increasingly popular in audio communications such as mobile and Voice over Internet Protocol (VoIP) communications, and thus, researching steganography in low bit-rate speech codec is of important significance. In this study, we proposed a universal VoIP steganography model for low bit-rate speech codec that uses the PESQ deterioration rate and the decoding error to automatically choose a data-embedding algorithm for each VoIP bitstream, which enables ones to achieve covert communications using a low bit-rate speech codec efficiently and securely. Because no or little attention has been paid to steganography in Internet Speech Audio Codec, it was chosen as the test codec to verify the effectiveness, security, and practicability of the proposed steganography model. The experimental results show that, with the proposed steganography model, it achieved the average PESQ deterioration rate of 4.04% (less than 5%, indicating strong imperceptibility) and a high data-hiding capacity up to 12 bits/frame (400 bits/s, three times larger than other methods), and the proposed steganography model could effectively resist the latest steganalysis. Copyright © 2015 John Wiley & Sons, Ltd.

Abstract Steganography in low bit‐rate speech streams is an important branch of Voice‐over‐Internet Protocol steganography. From the point of preventing cybercrimes, it is significant to design effective steganalysis methods. In this paper, we present a support‐vector‐machine‐based steganalysis of low bit‐rate speech exploiting statistic characteristics of pulse positions. Specifically, we utilize the probability distribution of pulse positions as a long‐time distribution feature, extract Markov transition probabilities of pulse positions according to the short‐time invariance characteristic of speech signals, and employ joint probability matrices to characterize the pulse‐to‐pulse correlation. We evaluate the performance of the proposed method with a large number of G.729a‐encoded speech samples and compare it with the state‐of‐the‐art methods. The experimental results demonstrate that our method significantly outperforms the previous ones on detection accuracy, false positive rate, and false negative rate at any given embedding rates or with any sample lengths. Particularly, this method can successfully detect steganography employing only one or a few of the potential cover bits, which is hard to be effectively detected by the existing methods. Copyright © 2016 John Wiley & Sons, Ltd.

Steganography in low bit-rare speech streams is an important branch of Voice-over-IP steganography. From the point of preventing cybercrimes, it is significant to design effective steganalysis methods. In this paper, we present a support-vector-machine based steganalysis of low bit-rate speech exploiting statistic characteristics of pulse positions. Specifically, we utilize the probability distribution of pulse positions as a long-time distribution feature, extract Markov transition probabilities of pulse positions according to the short-time invariance characteristic of speech signals, and employ joint probability matrices to characterize the pulse-to-pulse correlation. We evaluate the performance of the proposed method with a large number of G.729a encoded samples, and compare it with the state-of-the-art methods. The experimental results demonstrate that our method significantly outperforms the previous ones on detection accuracy at any given embedding rates or with any sample lengths. Particularly, this method can successfully detect steganography employing only one or a few of the potential cover bits, which is hard to be effectively detected by the existing methods.

Microblog sentiment classification is an important research topic which has wide applications in both academia and industry. Because microblog messages are short, noisy and contain masses of acronyms and informal words, microblog sentiment classification is a very challenging task. Fortunately, collectively the contextual information about these idiosyncratic words provide knowledge about their sentiment orientations. In this paper, we propose to use the microblogs' contextual knowledge mined from a large amount of unlabeled data to help improve microblog sentiment classification. We define two kinds of contextual knowledge: word-word association and word-sentiment association. The contextual knowledge is formulated as regularization terms in supervised learning algorithms. An efficient optimization procedure is proposed to learn the model. Experimental results on benchmark datasets show that our method can consistently and significantly outperform the state-of-the-art methods.

The popularity of microblogging platforms, such as Twitter, makes them important for information dissemination and sharing. However, they are also recognized as ideal places by spammers to conduct social spamming. Massive social spammers and spam messages heavily hurt the user experience and hinder the healthy development of microblogging systems. Thus, effectively detecting the social spammers and spam messages in microblogging is of great value. Existing studies mainly regard social spammer detection and spam message detection as two separate tasks. However, social spammers and spam messages have strong connections, since social spammers tend to post more spam messages and spam messages have high probabilities to be posted by social spammers. Combining social spammer detection with spam message detection has the potential to boost the performance of each task. In this paper, we propose a unified framework for social spammer and spam message co-detection in microblogging. Our framework utilizes the posting relations between users and messages to combine social spammer detection with spam message detection. In addition, we extract the social relations between users as well as the connections between messages, and incorporate them into our framework as regularization terms over the prediction results. Besides, we introduce an efficient optimization method to solve our framework. Extensive experiments on a real-world microblog dataset demonstrate that our framework can significantly and consistently improve the performance of both social spammer detection and spam message detection.

Sentiment classification is a hot research topic in both industrial and academic fields. The mainstream sentiment classification methods are based on machine learning and treat sentiment classification as a text classification problem. However, sentiment classification is widely recognized as a highly domain-dependent task. The sentiment classifier trained in one domain may not perform well in another domain. A simple solution to this problem is training a domain-specific sentiment classifier for each domain. However, it is difficult to label enough data for every domain since they are in a large quantity. In addition, this method omits the sentiment information in other domains. In this paper, we propose to train sentiment classifiers for multiple domains in a collaborative way based on multi-task learning. Specifically, we decompose the sentiment classifier in each domain into two components, a general one and a domain-specific one. The general sentiment classifier can capture the global sentiment information and is trained across various domains to obtain better generalization ability. The domain-specific sentiment classifier is trained using the labeled data in one domain to capture the domain-specific sentiment information. In addition, we explore two kinds of relations between domains, one based on textual content and the other one based on sentiment word distribution. We build a domain similarity graph using domain relations and encode it into our approach as regularization over the domain-specific sentiment classifiers. Besides, we incorporate the sentiment knowledge extracted from sentiment lexicons to help train the general sentiment classifier more accurately. Moreover, we introduce an accelerated optimization algorithm to train the sentiment classifiers efficiently. Experimental results on two benchmark sentiment datasets show that our method can outperform baseline methods significantly and consistently.

TCP incast problem has become a severe problem in datacenters due to the catastrophic collapse of goodput at the receiver side. Many solutions have been proposed, however, none of them solved it fundamentally. In this paper, we take a deep dive into the causes of incast problem, and find out that the root cause of TCP incast is the droptails induced by TCP congestion control algorithm based on packet losses. According to the root cause, we design a new TCP based on FAST TCP, a delay based congestion control algorithm, to radically solve TCP incast problem. The new TCP aims at maintaining a relatively small queue length that does not exceed the switch buffer and meanwhile fully occupy the bottleneck link. The simulation results demonstrate that our method cuts off most of the timeouts and attains high goodput under various conditions.

Correlated topical trend detection is very useful in analyzing public and social media influence. In this paper, we propose an algorithm that can both detect the correlation and discover the corresponding keywords that trigger the correlation. To detect the correlation, we use a projection vector to project two text streams onto the same space, and then use a least square cost function to regress one text stream over the other with different time lags. To extract the corresponding keywords, we impose the non-negative sparsity constraints over the projection parameters. In addition, we present an accelerated algorithm based on Nesterov's method to efficiently solve the optimization problem. In our experiments, we use both syntehtic and real data sets to demonstrate the advantages and capabilities of the proposed algorithm over CCA on the follower link prediction problem.

Heath information exchange (HIE) is the next step of health informatics development in China. Conformed to the administration feature, the design of HIE platform adopts a centralized architecture, which has to accommodate large amount of health information and support various types of query and data retrieval. Cloud computing is a suitable for the technology requirements of centralized design. In this paper, we proposed a medical information platform based on Hadoop, named after Medoop. The present Medoop utilizes HDFS to store the merged CDA documents efficiently, organize the feature information in CDA documents according to frequent business queries, and compute statistic distributedly in MapReduce paradigm. The objective of Medoop is to construct a comprehensive platform for health information storage and exchange, utilizing the components in Hadoop ecosystem.

Service composition-based network software customisation is currently a research hotspot in the field of software engineering. A key problem of the hotspot is how to efficiently discover services distributed over the Internet. In the service oriented architecture, service discovery suffers from the performance bottleneck of centralised universal description discovery and integration (UDDI), and inaccurate matching of service semantics. In this study, the authors describe a novel method for service labelling, registration and discovery, which is based on the role-goal-process-service meta-model. This approach enables ones to achieve accurate matching of service semantics by extending web service description language with RGP demand-information. The authors also suggest a peer-to-peer (P2P)-based architecture of service discovery to address the issues in the UDDI bottleneck and the complexity of semantic computation. By adopting the proposed approach, an experiment prototype system has been designed and implemented in Beijing municipal transportation system. The experimental results show the proposed approach is effective in addressing the aforementioned problems.

This paper presents a method to detect the quantization index modulation (QIM) steganography in G.723.1 bit stream. We show that the distribution of each quantization index (codeword) in the quantization index sequence has unbalanced and correlated characteristics. We present the designs of statistical models to extract the quantitative feature vectors of these characteristics. Combining the extracted vectors with the support vector machine, we build the classifier for detecting the QIM steganography in G.723.1 bit stream. The experiment shows that the method has far better performance than the existing blind detection method which extracts the feature vector in an uncompressed domain. The recall and precision of our method are all more than 90% even for a compressed bit stream duration as low as 3.6 s.

Data sparseness, the evident characteristic of short text, has always been regarded as the main cause of the low accuracy in the classification of short texts using statistical methods. Intensive research has been conducted in this area during the past decade. However, most researchers failed to notice that ignoring the semantic importance of certain feature terms might also contribute to low classification accuracy. In this paper we present a new method to tackle the problem by building a strong feature thesaurus (SFT) based on latent Dirichlet allocation (LDA) and information gain (IG) models. By giving larger weights to feature terms in SFT, the classification accuracy can be improved. Specifically, our method appeared to be more effective with more detailed classification. Experiments in two short text datasets demonstrate that our approach achieved improvement compared with the state-of-the-art methods including support vector machine (SVM) and Naïve Bayes Multinomial.

Low bit-rate speech codecs have been widely used in audio communications like VoIP and mobile communications, so that steganography in low bit-rate audio streams would have broad applications in practice. In this paper, the authors propose a new algorithm for steganography in low bit-rate VoIP audio streams by integrating information hiding into the process of speech encoding. The proposed algorithm performs data embedding while pitch period prediction is conducted during low bit-rate speech encoding, thus maintaining synchronization between information hiding and speech encoding. The steganography algorithm can achieve high quality of speech and prevent detection of steganalysis, but also has great compatibility with a standard low bit-rate speech codec without causing further delay by data embedding and extraction. Testing shows, with the proposed algorithm, the data embedding rate of the secret message can attain 4 bits/frame (133.3 bits/second).

An effective covert communication system can be achieved based on steganography in stream media such as VoIP (Voice over IP), and the LSB-based (Least-Significant-Bit) methods are the most popular strategies. However, LSB embedding in the flat regions of the speech could inevitably bring about the perceptible distortion leading to the degradation of speech quality. Besides, LSB methods are vulnerable for manifold corresponding steganalysis like RS detection. This paper presents an adaptive steganography scheme that selects lower embedding bit rate in the flat segments, while the sharp segments can camouflage more secret message. Further, synchronous overflow judgment and the tradeoff between hiding requirement and speech quality are also taken into account to construct a practical covert communication system on G.711 VoIP. Extensive experiments showed that the proposed adaptive scheme outperformed the LSB and could evade RS steganalysis. This proposed scheme is efficient, simple and can be used in real-time VoIP network with high hiding capacity.

A network covert channel is a passage along which information leaks across the network in violation of security policy in a completely undetectable manner. This study reveals our findings in analysing the principle of G.723.1 codec that there are ‘unused’ bits in G.723.1 encoded audio frames, which can be used to embed secret messages. A novel steganalysis method that employs the second detection and regression analysis is suggested in this study. The proposed method can not only detect the hidden message embedded in a compressed voice over Internet protocol (VoIP) speech, but also accurately estimate the embedded message length. The method is based on the second statistics, that is, doing a second steganography (embedding information in a sampled speech at an embedding rate followed by embedding another information at a different level of data embedding) in order to estimate the hidden message length. Experimental results have proven the effectiveness of the steganalysis method for detecting the covert channel in the compressed VoIP speech.

This paper describes a novel high-capacity steganography algorithm for embedding data in the inactive frames of low bit rate audio streams encoded by G.723.1 source codec, which is used extensively in Voice over Internet Protocol (VoIP). This study reveals that, contrary to existing thought, the inactive frames of VoIP streams are more suitable for data embedding than the active frames of the streams; that is, steganography in the inactive audio frames attains a larger data embedding capacity than that in the active audio frames under the same imperceptibility. By analyzing the concealment of steganography in the inactive frames of low bit rate audio streams encoded by G.723.1 codec with 6.3 kb/s, the authors propose a new algorithm for steganography in different speech parameters of the inactive frames. Performance evaluation shows embedding data in various speech parameters led to different levels of concealment. An improved voice activity detection algorithm is suggested for detecting inactive audio frames taking into packet loss account. Experimental results show our proposed steganography algorithm not only achieved perfect imperceptibility but also gained a high data embedding rate up to 101 bits/frame, indicating that the data embedding capacity of the proposed algorithm is very much larger than those of previously suggested algorithms.

This paper proposes a new approach based on the Histogram Shifting Algorithm (HSA) to reversible data hiding in bitmap image. The embedding is not carried out in the whole image but in small blocks in order to enhance the capacity. Firstly, the blocks are classified, and then, coding methods specific to each type of block are designed. Secondly, some blocks become unavailable for a second time embedding after the first one, while others are naturally unavailable. So as to separate these two kinds of blocks from each other, the status of blocks are defined, and a scheme of reserving blocks for taking identification information is proposed. Thirdly, the overall procedure of data embedding, extracting and image recovering are summarized. Test results on well-known images demonstrate the effectiveness of the new approach. The new method increases the total capacity of reversible data hiding to 1-4 times of the conventional method, and surprisingly, it also improves the PSNR value at the same time.

Differing from applying steganography on storage cover media, steganography on voice over IP (VoIP) must often delicately balance between providing adequate security and maintaining low latency for real-time services. This paper presents a novel real-time steganography model for VoIP that aims at providing good security for secret messages without sacrificing real-time performance. We achieve this goal by employing the well-known least-significant-bits (LSB) substitution approach to provide a reasonable tradeoff between the adequate information hiding requirement (good security and sufficient capacity) and the low latency requirement for VoIP. Further, we incorporate the M-sequence technique to eliminate the correlation among secret messages to resist the statistical detection based on the fact that the distribution of the LSBs in the stego-speech is not uniform and to provide a short-term security protection of secret messages. To accurately recover secret messages at the receiver side, we design a synchronization mechanism based on the RSA key agreement and the synchronized sequence transmission using techniques of the protocol steganography, which can effectively enhance the flexibility of the covert communication system and be extended to other steganography schemes based on real-time systems. We evaluate the effectiveness of our model with ITU-T G.729a as the codec of the cover speech in StegTalk, a covert communication system based on VoIP. The experimental results demonstrate that our technique provides good security and transparency for transmitting secret messages while adequately meeting the real-time requirement of VoIP.

This paper presents an adaptive steganography scheme for Voice over IP (VoIP). Differing from existing steganography techniques for VoIP, this scheme enhances the embedding transparency by taking into account the similarity between least significant bits (LSBs) and embedded messages. Moreover, we introduce the notion of partial similarity value (PSV). By properly setting the threshold PSV, we can adaptively balance the embedding transparency and capacity. We evaluate the effectiveness of this scheme with G.729a as the codec of the cover speech in StegTalk, a covert communication system based on VoIP. The experimental results demonstrate that our technique provides better performance than the traditional method.

In this paper we introduce a novel codebook partition algorithm for quantization index modulation (QIM), which is applied to information hiding in instant low bit-rate speech stream. The QIM method divides the codebook into two parts, each representing '0' and '1' respectively. Instead of randomly partitioning the codebook, the relationship between codewords is considered. The proposed algorithm - complementary neighbor vertices (CNV) guarantees that every codeword is in the opposite part to its nearest neighbor, and the distortion is limited by a bound. The feasibility of CNV is proved with graph theory. Moreover, in our work the secret message is embedded in the field of vector quantization index of LPC coefficients, getting the benefit that the distortion due to QIM is lightened adaptively by the rest of the encoding procedure. Experiments on iLBC and G.723.1 verify the effectiveness of the proposed method. Both objective and subjective assessments show the proposed method only slightly decreases the speech quality to an indistinguishable degree. The hiding capacity is no less than 100 bps. To the best of our knowledge, this is the first work adopting graph theory to improve the codebook partition while using QIM in low bit-rate streaming media.