Reliable behavior control is central to deploying Large Language Models (LLMs) on the web. Activation steering offers a tuning-free route to align attributes (e.g., truthfulness) that ensure trustworthy generation. Prevailing approaches rely on coarse heuristics and lack a principled account of where to steer and how strongly to intervene. To this end, we propose Position-wise Injection with eXact Estimated Levels (PIXEL), a position-wise activation steering framework that, in contrast to prior work, learns a property-aligned subspace from dual views (tail-averaged and end-token) and selects intervention strength via a constrained geometric objective with a closed-form solution, thereby adapting to token-level sensitivity without global hyperparameter tuning. PIXEL further performs sample-level orthogonal residual calibration to refine the global attribute direction and employs a lightweight position-scanning routine to identify receptive injection sites. We additionally provide representation-level guarantees for the minimal-intervention rule, supporting reliable alignment. Across diverse models and evaluation paradigms, PIXEL consistently improves attribute alignment while preserving model general capabilities, offering a practical and principled method for LLMs' controllable generation. Our code is available at https://anonymous.4open.science/r/PIXEL-Adaptive-Steering-95DC

• OOD data significantly impacts model performance, resulting in decreased prediction accuracy and reliability. • The continual tabular contrastive learning (C-TCL) method effectively manages OOD data while maintaining computational efficiency. • C-TCL achieves superior performance on CPU hardware, making it more accessible than GPU-based alternatives. • Experimental results from eight diverse datasets demonstrate C-TCL’s effectiveness, particularly in classification tasks. • The matrix augmentation technique using full data representation improves model efficiency compared to traditional slice-based methods. • Simplified contrastive loss calculations with dot product reduces computational overhead while maintaining performance.

Legal case retrieval (LCR) is an essential tool for not only assisting legal practitioners to efficiently retrieve precedents but also enabling ordinary individuals to find valuable legal case information without relying on expensive professional legal services. Our previous work CaseLink demonstrated the effectiveness of using case to case graph structures to improve retrieval accuracy. However, its high computational cost during inference on large-scale legal databases limits its practical use in real-world settings. The main inefficiency comes from constructing test time graphs and computing pairwise term frequency similarities of cases. This process has \(O(n^{2})\) complexity for \( n \) legal cases, making the runtime prohibitive as the number of candidates grows. For example, the retrieval time for one query on a database (COLIEE2022) with 1,563 candidate cases is more than 500 milliseconds , while the runtime would increase drastically to more than 3,500 seconds for a database (LeCaRDv2) with 55,192 candidate cases . To further enhance the retrieval performance while achieving a significant speed-up, in this extension paper, Cassette framework is proposed with a distillation strategy involving ranking objective and eigen-matching objective for an effective transfer of knowledge from a powerful and well-trained heavy teacher retriever to a lightweight and efficient hybrid student dual encoder. Specifically, the student query encoder is implemented as a multilayer perceptron model designed for fast online processing, whereas the student candidate encoder adopts a GNN architecture, suitable for an offline manner within the case database. Extensive experiments are conducted on three benchmark datasets, and the results verify the effectiveness of the ranking distillation while achieving high efficiency. The code has been released on https://github.com/yanran-tang/Cassette/ .

Abstract Legal case retrieval (LCR) is a specialised information retrieval task aimed at identifying relevant cases given a query case. LCR holds pivotal significance in facilitating legal practitioners to locate legal precedents. Existing LCR methods predominantly rely on traditional lexical models or language models; however, they typically overlook the domain-specific structural information embedded in legal documents. Our previous work CaseGNN (Tang et al., In: ECIR, 2024) successfully harnesses text-attributed graphs and graph neural networks to incorporate structural legal information. Nonetheless, three key challenges remain in enhancing the representational capacity of CaseGNN: (1) The under-utilisation of rich edge information in text-attributed case graph (TACG). (2) The insufficiency of training signals for graph contrastive learning. (3) The lack of contextualised legal information in node and edge features. In this paper, the LEXA model, an extension of CaseGNN, is proposed to overcome these limitations by jointly leveraging rich edge information, enhanced training signals, and contextualised embeddings derived from large language models (LLMs). Specifically, an edge-updated graph attention layer (EUGAT) is proposed to comprehensively update node and edge features during graph modelling, resulting in a full utilisation of structural information of legal cases. Moreover, LEXA incorporates a novel graph contrastive learning objective with graph augmentation to provide additional training signals, thereby strengthening the model’s legal comprehension capabilities. What’s more, given the remarkable contextualised understanding capabilities of LLMs for text encoding, LLMs are employed to generate node and edge features for the text-attributed case graph (TACG). Extensive experiments on two benchmark datasets from COLIEE 2022 and COLIEE 2023 demonstrate that LEXA not only significantly improves CaseGNN but also achieves supreme performance compared to state-of-the-art LCR methods. Code has been released on https://github.com/yanran-tang/CaseGNN .

Out-of-distribution (OOD) prediction remains a significant challenge in machine learning, particularly for tabular data where traditional methods often fail to generalize beyond their training distribution.This paper introduces Tabular Continual Contrastive Learning (TCCL), a novel framework designed to address OOD challenges in tabular data processing.TCCL integrates contrastive learning principles with continual learning mechanisms, featuring a three-component architecture: an Encoder for data transformation, a Decoder for representation learning, and a Learner Head.We evaluate TCCL against 14 baseline models, including state-of-the-art deep learning approaches and gradient-boosted decision trees (GBDT), across eight diverse tabular datasets.Our experimental results demonstrate that TCCL consistently outperforms existing methods in both classification and regression tasks on OOD data, with particular strength in handling distribution shifts.These findings suggest that TCCL represents a significant advancement in handling OOD scenarios for tabular data.

Despite recent advances, Text-to-video retrieval (TVR) is still hindered by multiple inherent uncertainties, such as ambiguous textual queries, indistinct text-video mappings, and low-quality video frames. Although interactive systems have emerged to address these challenges by refining user intent through clarifying questions, current methods typically rely on heuristic or ad-hoc strategies without explicitly quantifying these uncertainties, limiting their effectiveness. Motivated by this gap, we propose UMIVR, an Uncertainty-Minimizing Interactive Text-to-Video Retrieval framework that explicitly quantifies three critical uncertainties-text ambiguity, mapping uncertainty, and frame uncertainty-via principled, training-free metrics: semantic entropy-based Text Ambiguity Score (TAS), Jensen-Shannon divergence-based Mapping Uncertainty Score (MUS), and a Temporal Quality-based Frame Sampler (TQFS). By adaptively generating targeted clarifying questions guided by these uncertainty measures, UMIVR iteratively refines user queries, significantly reducing retrieval ambiguity. Extensive experiments on multiple benchmarks validate UMIVR's effectiveness, achieving notable gains in Recall@1 (69.2\% after 10 interactive rounds) on the MSR-VTT-1k dataset, thereby establishing an uncertainty-minimizing foundation for interactive TVR.

Existing Moment retrieval (MR) methods focus on Single-Moment Retrieval (SMR). However, one query can correspond to multiple relevant moments in real-world applications. This makes the existing datasets and methods insufficient for video temporal grounding. By revisiting the gap between current MR tasks and real-world applications, we introduce a high-quality datasets called QVHighlights Multi-Moment Dataset (QV-M$^2$), along with new evaluation metrics tailored for multi-moment retrieval (MMR). QV-M$^2$ consists of 2,212 annotations covering 6,384 video segments. Building on existing efforts in MMR, we propose a framework called FlashMMR. Specifically, we propose a Multi-moment Post-verification module to refine the moment boundaries. We introduce constrained temporal adjustment and subsequently leverage a verification module to re-evaluate the candidate segments. Through this sophisticated filtering pipeline, low-confidence proposals are pruned, and robust multi-moment alignment is achieved. We retrain and evaluate 6 existing MR methods on QV-M$^2$ and QVHighlights under both SMR and MMR settings. Results show that QV-M$^2$ serves as an effective benchmark for training and evaluating MMR models, while FlashMMR provides a strong baseline. Specifically, on QV-M$^2$, it achieves improvements over prior SOTA method by 3.00% on G-mAP, 2.70% on mAP@3+tgt, and 2.56% on mR@3. The proposed benchmark and method establish a foundation for advancing research in more realistic and challenging video temporal grounding scenarios. Code is released at https://github.com/Zhuo-Cao/QV-M2.

Text-guided Video Temporal Grounding (VTG) aims to localize relevant segments in untrimmed videos based on textual descriptions, encompassing two subtasks: Moment Retrieval (MR) and Highlight Detection (HD). Although previous typical methods have achieved commendable results, it is still challenging to retrieve short video moments. This is primarily due to the reliance on sparse and limited decoder queries, which significantly constrain the accuracy of predictions. Furthermore, suboptimal outcomes often arise because previous methods rank predictions based on isolated predictions, neglecting the broader video context. To tackle these issues, we introduce FlashVTG, a framework featuring a Temporal Feature Layering (TFL) module and an Adaptive Score Refinement (ASR) module. The TFL module replaces the traditional decoder structure to capture nuanced video content variations across multiple temporal scales, while the ASR module improves prediction ranking by integrating context from adjacent moments and multi-temporal-scale features. Extensive experiments demonstrate that FlashVTG achieves state-of-the-art performance on four widely adopted datasets in both MR and HD. Specifically, on the QVHighlights dataset, it boosts mAP by 5.8% for MR and 3.3% for HD. For short-moment retrieval, FlashVTG increases mAP to 125% of previous SOTA performance. All these improvements are made without adding training burdens, underscoring its effectiveness. Our code is available at https://github.com/Zhuo-Cao/FlashVTG.

Text-Video Retrieval (TVR) methods typically match query-candidate pairs by aligning text and video features in coarse-grained, fine-grained, or combined (coarse-to-fine) manners. However, these frameworks predominantly employ a one(query)-to-one(candidate) alignment paradigm, which struggles to discern nuanced differences among candidates, leading to frequent mismatches. Inspired by Comparative Judgement in human cognitive science, where decisions are made by directly comparing items rather than evaluating them independently, we propose TokenBinder. This innovative two-stage TVR framework introduces a novel one-to-many coarse-to-fine alignment paradigm, imitating the human cognitive process of identifying specific items within a large collection. Our method employs a Focused-view Fusion Network with a sophisticated cross-attention mechanism, dynamically aligning and comparing features across multiple videos to capture finer nuances and contextual variations. Extensive experiments on six benchmark datasets confirm that TokenBinder substantially outperforms existing state-of-the-art methods. These results demonstrate its robustness and the effectiveness of its fine-grained alignment in bridging intra- and inter-modality information gaps in TVR tasks. Code is avaliable at https://github.com/bingqingzhang/TokenBinder.

Temporal entity alignment (TEA), which identifies equivalent entities across temporal knowledge graphs (TKGs), plays a vital role in integrating multiple TKGs.Simply adapting traditional EA models to TKGs cannot achieve satisfactory results, driving the need for dedicated studies in TEA. However, existing TEA models often fail to effectively capture the importance of temporal features and the richness of temporal context during embedding learning. Moreover, the challenge of temporal heterogeneity, which is prevalent in real-world TKGs, has not been adequately studied. In this work, we propose a HTEA framework to address these limitations. Specifically, we introduce a frequency-based temporal embedding module that incorporates the importance of temporal features for each entity, along with a temporal attention mechanism that prioritizes more informative context based on temporal richness. We further design an iterative module to detect temporal heterogeneity and refine the related facts accordingly. In this way, entity embeddings can be improved progressively, yielding more accurate and consistent alignment outcomes.Extensive experiments showcase the efficacy of our HTEA model, especially under the existence of temporal heterogeneity in real-world TKGs.

As a building block of knowledge acquisition, knowledge graph completion (KGC) aims at inferring missing facts in knowledge graphs (KGs) automatically. Previous studies mainly focus on graph convolutional network (GCN)-based KG embedding (KGE) to determine the representations of entities and relations, accordingly predicting missing triplets. However, most existing KGE methods suffer from limitations in predicting tail entities that are far away or even unreachable in KGs. This limitation can be attributed to the related high-order information being largely ignored. In this work, we focus on learning the information from the related high-order neighbors in KGs to improve the performance of prediction. Specifically, we first introduce a set of new nodes called pedalnodes to augment the KGs for facilitating message passing between related high-order entities, effectively injecting the information of high-order neighbors into entity representation. Additionally, we propose strength-guided graph neural networks to aggregate neighboring entity representations. To address the issue of transmitting irrelevant higher order information to entities through pedal nodes, which can potentially hurt entity representation, we further propose to dynamically integrate the aggregated representation of each node with its corresponding self-representation. Extensive experiments have been conducted on three benchmark datasets and the results demonstrate the superiority of our method compared to strong baseline models.

In the field of biomedicine, Document-level relation extraction (DocRE) aims to reason about complex relational facts among entities by reading, inferring, and aggregating among entities over multiple sentences in a document. Existing studies construct document-level graphs to enrich interactions between entities. However, these methods pay more attention to the entity nodes and their connections, regardless of the rich knowledge entailed in the original corpus. In this paper, we propose a contrastive learning enhanced document-level graph relation representation(CGDRE) which mines the semantic knowledge from the original corpus and improve the ability of DocRE. Firstly, we use a coreference contrastive learning module to capture the potential semantic knowledge. Secondly, we construct a heterogeneous graph to enhance the graph structure information according to the original document and semantic knowledge. Lastly, CGDRE infers relations on the aggregated graph and uses focal loss to train the model. Remarkably, it is amazing that CGDRE can effectively alleviate the long-tailed distribution problem in the DocRE. Experiments on the public datasets, CDR, GDA and DocRED, show that CGDRE can significantly outperform other baselines, achieving a significant performance improvement. Extensive analyses demonstrate that the performance of our CGDRE is contributed by the capture of the semantic knowledge enhanced graph relation representation.

Document-level relation extraction is an important task in natural language processing that involves identifying and classifying relations between entities mentioned in a document. Traditional approaches often focus on individual sentences or local context, overlooking the broader context of the entire document. In this paper, we propose an Evidence Sentence Augmented Sequence-to-Sequence method for Document-level Relation Extraction(called ESASS-DRE). Our method introduces evidence sentences into the sequence-to-sequence framework to improve document-level relation extraction. The approach consists of two main steps: evidence sentence selection and relation extraction. Firstly, we identify a set of evidence sentences that contain crucial information relevant to the target relation. These sentences are selected based on their importance and contextual relevance. Secondly, the selected evidence sentences are combined with the original document and used as input to the sequence-to-sequence model. These generated sequences are decoded into relation labels, indicating the type of relationship between the entities. By incorporating evidence sentences into the model, we provide additional context and relevant information, enabling the model to make more informed predictions. Experiments conducted on benchmark datasets demonstrate the effectiveness of our method. Compared to traditional approaches, our method achieves higher accuracy and robustness in document-level relation extraction tasks. The incorporation of evidence sentences allows the model to capture the broader context of the document, leading to improved performance. (e.g., by 2.96/3.64 Ign F1/F1 on DocRED).

Knowledge graph completion (KGC) aims at inferring missing links between two entities. Most previous models focus on learning representations for entities and relations via graph neural networks. In this formalism, representations heavily rely on structural information. However, it is common for Knowledge graphs (KGs) to be unconnected due to their inherent incompleteness, resulting in a significant loss of vital structural information. To overcome this issue, this paper proposes to increase the connectedness of KGs to facilitate message passing between entities. Specifically, we first augment KGs with a series of auxiliary triplets derived from outer nodes. Subsequently, a dynamically weighted graph convolutional layer is employed to unequally aggregate the representations of neighboring entities and dynamically combine this aggregated information with information from themselves. Finally, ConvE is utilized to calculate scores for all triplets. Extensive experiments on two benchmark datasets demonstrate the superiority of our method compared to strong baseline models.

Fenghua Cheng, Xue Li, Zi Huang, Jinxiang Wang, Sen Wang. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

In case law, the precedents are the relevant cases that are used to support the decisions made by the judges and the opinions of lawyers towards a given case. This relevance is referred to as the case-to-case reference relation. To efficiently find relevant cases from a large case pool, retrieval tools are widely used by legal practitioners. Existing legal case retrieval models mainly work by comparing the text representations of individual cases. Although they obtain a decent retrieval accuracy, the intrinsic case connectivity relationships among cases have not been well exploited for case encoding, therefore limiting the further improvement of retrieval performance. In a case pool, there are three types of case connectivity relationships: the case reference relationship, the case semantic relationship, and the case legal charge relationship. Due to the inductive manner in the task of legal case retrieval, using case reference as input is not applicable for testing. Thus, in this paper, a CaseLink model based on inductive graph learning is proposed to utilise the intrinsic case connectivity for legal case retrieval, a novel Global Case Graph is incorporated to represent both the case semantic relationship and the case legal charge relationship. A novel contrastive objective with a regularisation on the degree of case nodes is proposed to leverage the information carried by the case reference relationship to optimise the model. Extensive experiments have been conducted on two benchmark datasets, which demonstrate the state-of-the-art performance of CaseLink. The code has been released on https://github.com/yanran-tang/CaseLink.

The energy industry is undergoing significant transformations as it strives to achieve net-zero emissions and future-proof its infrastructure, where every participant in the power grid has the potential to both consume and produce energy resources. Federated learning -- which enables multiple participants to collaboratively train a model without aggregating the training data -- becomes a viable technology. However, the global model parameters that have to be shared for optimization are still susceptible to training data leakage. In this work, we propose confined gradient descent (CGD) that enhances the privacy of federated learning by eliminating the sharing of global model parameters. CGD exploits the fact that a gradient descent optimization can start with a set of discrete points and converges to another set in the neighborhood of the global minimum of the objective function. As such, each participant can independently initiate its own private global model~(referred to as the confined model ), and collaboratively learn it towards the optimum. The updates to their own models are worked out in a secure collaborative way during the training process.In such a manner, CGD retains the ability of learning from distributed data but greatly diminishes information sharing. Such a strategy also allows the proprietary confined models to adapt to the heterogeneity in federated learning, providing inherent benefits of fairness. We theoretically and empirically demonstrate that decentralized CGD øne provides a stronger differential privacy (DP) protection; \two is robust against the state-of-the-art poisoning privacy attacks; þree results in bounded fairness guarantee among participants; and \four provides high test accuracy (comparable with centralized learning) with a bounded convergence rate over four real-world datasets.

Relation extraction is one of the most essential tasks of knowledge construction, but it depends on a large amount of annotated data corpus. Few-shot relation extraction is proposed as a new paradigm, which is designed to learn new relationships between entities with merely a small number of annotated instances, effectively mitigating the cost of large-scale annotation and long-tail problems. To generalize to novel classes not included in the training set, existing approaches mainly focus on tuning pre-trained language models with relation instructions and developing class prototypes based on metric learning to extract relations. However, the learned representations are extremely sensitive to discrepancies in intra-class and inter-class relationships and hard to adaptively classify the relations due to biased class features and spurious correlations, such as similar relation classes having closer inter-class prototype representation. In this paper, we introduce an adaptive class augmented prototype network with instance-level and representation-level augmented mechanisms to strengthen the representation space. Specifically, we design the adaptive class augmentation mechanism to expand the representation of classes in instance-level augmentation, and class augmented representation learning with Bernoulli perturbation context attention to enhance the representation of class features in representation-level augmentation and explore adaptive debiased contrastive learning to train the model. Experimental results have been demonstrated on FewRel and NYT-25 under various few-shot settings, and the proposed model has improved accuracy and generalization, especially for cross-domain and different hard tasks.

The new generation of organic light emitting diode display is designed to enable the high dynamic range (HDR), going beyond the standard dynamic range (SDR) supported by the traditional display devices. However, a large quantity of videos are still of SDR format. Further, most pre-existing videos are compressed at varying degrees for minimizing the storage and traffic flow demands. To enable movie-going experience on new generation devices, converting the compressed SDR videos to the HDR format (i.e., compressed-SDR to HDR conversion) is in great demands. The key challenge with this new problem is how to solve the intrinsic many-to-many mapping issue. However, without constraining the solution space or simply imitating the inverse camera imaging pipeline in stages, existing SDR-to-HDR methods can not formulate the HDR video generation process explicitly. Besides, they ignore the fact that videos are often compressed. To address these challenges, in this work we propose a novel imaging knowledge-inspired parallel networks (termed as KPNet) for compressed-SDR to HDR (CSDR-to-HDR) video reconstruction. KPNet has two key designs: Knowledge-Inspired Block (KIB) and Information Fusion Module (IFM). Concretely, mathematically formulated using some priors with compressed videos, our conversion from a CSDR-to-HDR video reconstruction is conceptually divided into four synergistic parts: reducing compression artifacts, recovering missing details, adjusting imaging parameters, and reducing image noise. We approximate this process by a compact KIB. To capture richer details, we learn HDR representations with a set of KIBs connected in parallel and fused with the IFM. Extensive evaluations show that our KPNet achieves superior performance over the state-of-the-art methods.

Recent advances in Tiny Machine Learning (TinyML) empower low-footprint embedded devices for real-time on-device Machine Learning (ML). While many acknowledge the potential benefits of TinyML, its practical implementation presents unique challenges. This study aims to bridge the gap between prototyping single TinyML models and developing reliable TinyML systems in production: (1) Embedded devices operate in dynamically changing conditions. Existing TinyML solutions primarily focus on inference, with models trained offline on powerful machines and deployed as static objects. However, static models may underperform in the real world due to evolving input data distributions. We propose online learning to enable training on constrained devices, adapting local models toward the latest field conditions. (2) Nevertheless, current on-device learning methods struggle with heterogeneous deployment conditions and the scarcity of labeled data when applied across numerous devices. We introduce federated meta-learning incorporating online learning to enhance model generalization, facilitating rapid learning. This approach ensures optimal performance among distributed devices by knowledge sharing. (3) Moreover, TinyML’s pivotal advantage is widespread adoption. Embedded devices and TinyML models prioritize extreme efficiency, leading to diverse characteristics ranging from memory and sensors to model architectures. Given their diversity and non-standardized representations, managing these resources becomes challenging as TinyML systems scale up. We present semantic management for the joint management of models and devices at scale. We demonstrate our methods through a basic regression example and then assess them in three real-world TinyML applications: handwritten character image classification, keyword audio classification, and smart building presence detection. The results confirm the effectiveness of our approaches from various perspectives, such as accuracy improvement, resource savings, and engineering effort reduction.

Biomedical Entity Linking(BEL) is the task of linking biomedical mentions in natural language corpora such as diseases and drugs to standard entities in a given knowledge base. The same biomedical entity can have multiple mentions, including synonyms, morphological variations and names with different word order. Thus, making predictions for each mention with an insufficient context in a single biomedical document is challenging, especially when mentions or linked entities are unseen during training. This paper proposes an inference method for biomedical entity linking based on the heterogeneous graph constructed on multiple documents. We first design a joint representation learning method and compute mention-mention and mention-entity similarity in a unified semantic space. Based on them, we utilize the mutual nearest neighbors relationship between mentions and the relationship between mention and entity to construct a heterogeneous graph. Finally, we apply a clustering-based method to make linking predictions, which associates each mention in the documents with a unique entity. Extensive experiments on three biomedical entity linking benchmarks (MedMentions, BC5CDR and NCBI) demonstrate that our method outperforms other state-of-the-art entity linking models.

Document-level relation extraction (DocRE) is an important task in natural language processing, with applications in knowledge graph construction, question answering, and biomedical text analysis. However, existing approaches to DocRE have limitations in predicting relations between entities using fixed entity representations, which can lead to inaccurate results. In this paper, we propose a novel DocRE model that addresses these limitations by using a relation-specific entity representation method and evidence sentence augmentation. Our model uses evidence sentence augmentation to identify top-k evidence sentences for each relation and a relation-specific entity representation method that aggregates the importance of entity mentions using an attention mechanism. These two components work together to capture the context of each entity mention in relation to the specific relation being predicted and select evidence sentences that support accurate relation identification. Finally, we re-predicts entity relations based on the evidence sentences, called relationship reordering module. This module re-predicts entity relationships based on the predicted set of evidence sentences to form k sets of relationship predictions, and then averages these k+1 sets of results to obtain the final relationship predictions. Experimental results on the DocRED dataset demonstrate that our proposed model achieves an F1 score of 62.84% and an lgn F1 score of 60.79%, outperforming state-of-the-art methods.

Document-level relation extraction (DocRE) is a more challenging task for which multi-label and multi-entity problems need to be resolved effectively than its sentence-level counterpart. It aims at extracting relationships between two entities at once while taking into account significant cross-sentence features and long-distance semantic representation. In this paper, we propose a self-supervised commonsense-enhanced DocRE model, called CommDRE, without external knowledge. First, we introduce self-supervised learning to represent the commonsense knowledge of each entity in an entity pair. Second, we convert the cross-sentence entity pairs into anonymous entity pairs with a coreference commonsense alternative. Finally, we perform semantic relation representation learning on the anonymous entity pairs and automatically convert them into target entity pairs. Experimental results show that it performs significantly better than strong baselines by 2.76% F1, and commonsense knowledge has an important contribution to the DocRE through the ablation study.

Current state-of-the-art semantic segmentation methods usually require high computational resources for accurate segmentation. Knowledge distillation has been one promising way to achieve a good trade-off between accuracy and efficiency. However, current distillation methods focus on transferring the spatial relations and ignore the multi-scale context interaction. This paper proposes one novel pixel- region relation distillation (PPRD) to transfer the multi-scale pixel-region relation (PRR) from the teacher to the student. We get the multi-scale regions with pyramid pooling and characterize the multi-scale PRR between the feature and the multi-scale regions. Transferring such PRR from the teacher to the student is beneficial for the student to mimic the teacher better in terms of multi-scale context interaction. Experimental results on two challenging datasets, Cityscapes and Pascal VOC 2012, show that the proposed approach outperforms state-of-the-art distillation methods.

Current video object detection (VOD) models often encounter issues with over-aggregation due to redundant aggregation strategies, which perform feature aggregation on every frame. This results in suboptimal performance and increased computational complexity. In this work, we propose an image-level Object Detection Difficulty (ODD) metric to quantify the difficulty of detecting objects in a given image. The derived ODD scores can be used in the VOD process to mitigate over-aggregation. Specifically, we train an ODD predictor as an auxiliary head of a still-image object detector to compute the ODD score for each image based on the discrepancies between detection results and ground-truth bounding boxes. The ODD score enhances the VOD system in two ways: 1) it enables the VOD system to select superior global reference frames, thereby improving overall accuracy; and 2) it serves as an indicator in the newly designed ODD Scheduler to eliminate the aggregation of frames that are easy to detect, thus accelerating the VOD process. Comprehensive experiments demonstrate that, when utilized for selecting global reference frames, ODD-VOD consistently enhances the accuracy of Global-frame-based VOD models. When employed for acceleration, ODD-VOD consistently improves the frames per second (FPS) by an average of 73.3% across 8 different VOD models without sacrificing accuracy. When combined, ODD-VOD attains state-of-the-art performance when competing with many VOD methods in both accuracy and speed. Our work represents a significant advancement towards making VOD more practical for real-world applications. The code will be released at https://github.com/bingqingzhang/odd-vod.

Knowledge distillation has been widely applied in semantic segmentation to reduce the model size and computational complexity. The prior knowledge distillation methods for semantic segmentation mainly focus on transferring the spatial relation knowledge, neglecting to transfer the channel correlation knowledge in the feature space, which is vital for semantic segmentation. We propose a novel Channel Correlation Distillation (CCD) method for semantic segmentation to solve this issue. The correlation between channels tells how likely these channels belong to the same categories. We force the student to mimic the teacher by minimizing the distance between the channel correlation maps of the student and the teacher. Furthermore, we propose the multi-scale discriminators to sufficiently distinguish the multi-scale differences between the teacher and student segmentation outputs. Extensive experiments on three popular datasets: Cityscapes, CamVid, and Pascal VOC 2012 validate the superiority of our CCD. Experimental results show that our CCD could consistently improve the state-of-the-art methods with various network structures for semantic segmentation.

In recent years, the performance of multi-frame quality enhancement algorithms for compressed videos has been greatly improved compared with single-frame based algorithms. However, the existing methods mainly focus on mining the temporal information of multiple frames. The large number of reference frames reduces the exploration of spatial information, although the existing single-frame based algorithms for enhancement, denoising, and super-resolution demonstrate the significance of the spatial information. To address this problem, we propose a plug-and-play module called Spatio-temporal Information Balance (STIB) to adaptively balance the spatial and temporal information. In our method, we use a feature extractor to exploit richer spatial information, and use a refinement module to refine the aligned temporal information, to be more conducive to the fusion of spatio-temporal information. Finally, we use the deformable convolution based re-alignment module to do alignment and fusion in feature space for balancing the spatio-temporal information. Experiments show that our module can significantly improve the performance of the existing multi-frame based enhancement algorithms.

Genome-wide analysis has demonstrated both health and social benefits. However, large scale sharing of such data may reveal sensitive information about individuals. One of the emerging challenges is identity tracing attack that exploits correlations among genomic data to reveal the identity of DNA samples. In this paper, we first demonstrate that the adversary can narrow down the sample's identity by detecting his/her genetic relatives and quantify such privacy threat by employing a Shannon entropy-based measurement. For example, we exemplify that when the dataset size reaches 30% of the population, for any target from that population, the uncertainty of the target's identity is reduced to merely 2.3 bits of entropy (i.e., the identity is pinned down within 5 people). Direct application of existing approaches such as differential privacy (DP), secure multiparty computation (MPC) and homomorphic encryption (HE) may not be applicable to this challenge in genome-wide analysis because of the compromise on utility (i.e., accuracy or efficiency). Towards addressing this challenge, this paper proposes a framework named$\upsilon$Fragto facilitate privacy-preserving data sharing and computation in genome-wide analysis.$\upsilon$Fragmitigates privacy risks by using a vertical fragmentation to disrupt the genetic architecture on which the adversary relies for identity tracing without sacrificing the capability of genome-wide analysis. We theoretically prove that it preserves the correctness of the primitive functionalities and algorithms ranging from basic summary statistics to advanced neural networks. Our experiments demonstrate that$\upsilon$Fragoutperforms secure multiparty computation (MPC) and homomorphic encryption (HE) protocols, with a speedup of more than 221x for training neural networks, and also traditional non-private algorithms and a state-of-the-art noise-based differential privacy (DP) solution in most settings.

The past few years have witnessed the great success of multi-frame quality enhancement for compressed video. Although the existing methods based on deformable alignment have achieved the state-of-the-art performance, they do not pay enough attention to the recovery of detail information. In this work, we propose a Spatio-Temporal Detail Retrieval (STDR) method to promote the recovery of detail information. To alleviate the problem of inaccurate deformable offsets caused by the fixed receptive field, motivated by multi-task learning, we design a plug-and-play Multi-path Deformable Alignment (MDA) module to generate more accurate offsets by integrating the alignment features of different receptive fields, so that the temporal detail information can be better recovered. For the spatial detail information restoration, several residual dense blocks with channel attention layer are utilized in the reconstruction module to explore valuable high-frequency spatial information from the fused multi-path alignment features. Meanwhile, a complementary loss function based on the Pearson correlation coefficient is developed to ameliorate the over-smoothing shortcoming caused by pixel-wise mean square or absolute value loss. Experimental results demonstrate that the proposed STDR network achieves superior performance compared with the state-of-the-art methods in both quantitative and qualitative evaluations.

Recent progress on parse tree encoder for sentence representation learning is notable. However, these works mainly en-code tree structures recursively, which is not conducive to parallelization. On the other hand, these works rarely take into account the labels of arcs in dependency trees. To address both issues, we propose Dependency-Transformer, which applies a relation-attention mechanism that works in concert with the self-attention mechanism. This mechanism aims to encode the dependency and the spatial positional relations between nodes in the dependency tree of sentences. By a score-based method, we successfully inject the syntax information without affecting Transformer’s parallelizability. Our model outperforms or is comparable to the state-of-the-art methods on four tasks for sentence representation and has obvious advantages in computational efficiency.

In recent years, with the rise of various online learning plat-forms and game live broadcasting industry, screen content video is explosively increasing. There is an urgent demand to reduce the inevitable compression artifacts produced by traditional lossy compression method. However, there do not exist any research on compressed Screen Content Video (SCV) quality enhancement. Since SCV frames always con-sist of two main types of contents with different characteris-tics, i.e., text and graphic, a Content Adaptive model based on Two branches (CAT) is proposed in this paper. For en-hancing graphic, we utilize temporal information by motion compensation, while for enhancing text, we explore spatial relevance in horizontal and vertical directions to recover sharp edges. Furthermore, a content adaptive block is used to select content related features for the collaborative enhancement of both contents. We build a large SCV dataset compressed by H.266/VVC Test Model (VTM 12.1). On this dataset, experi-mental results demonstrate that the proposed method achieves the state-of-the-art performance on 10 different kinds of SCV test sequences.

Recently, with the rise of high dynamic range (HDR) display devices, there is a great demand to transfer traditional low dynamic range (LDR) images into HDR versions. The key to success is how to solve the many-to-many mapping problem. However, the existing approaches either do not consider constraining solution space or just simply imitate the inverse camera imaging pipeline in stages, without directly formulating the HDR image generation process. In this work, we address this problem by integrating LDR-to-HDR imaging knowledge into an UNet architecture, dubbed as Knowledge-inspired UNet (KUNet). The conversion from LDR-to-HDR image is mathematically formulated, and can be conceptually divided into recovering missing details, adjusting imaging parameters and reducing imaging noise. Accordingly, we develop a basic knowledge-inspired block (KIB) including three subnetworks corresponding to the three procedures in this HDR imaging process. The KIB blocks are cascaded in the similar way to the UNet to construct HDR image with rich global information. In addition, we also propose a knowledge inspired jump-connect structure to fit a dynamic range gap between HDR and LDR images. Experimental results demonstrate that the proposed KUNet achieves superior performance compared with the state-of-the-art methods. The code, dataset and appendix materials are available at https://github.com/wanghu178/KUNet.git.

Document-level relation extraction (Doc-level RE) is a more practical and challenging task, which provides a new perspective on obtaining factual knowledge from the more complex cross-sentence text. Recent Doc-level RE, based on pre-trained language models, uses graph neural networks to implicitly model relation reasoning in a document. However, it is not perfect that the model neglects explicit reasoning clues, leading to a weak ability and a lack of capability to model long-distance relationships. In this paper, we propose to explicitly model the heterogeneous affinity graph, HAG, including a mention graph (MG) and a coreference graph (CG). We first construct CG to cluster the expressions together as a coreference array. Then, MG and CG are incorporated to capture the reasoning clues from the adjacent affinity matrix. Moreover, HAG is aggregated into an isomorphic entity graph according to the noise suppression mechanism and RGCN. Finally, the classification is established on the normalized graph to infer the relations of entity pairs. Experimental results significantly outperform baselines by nearly 1.7% ∼ 2.0% in F1 on three public datasets, DocRED, DialogRE, and MPDD. We further conduct ablation experiments to demonstrate the effectiveness of the proposed approach.

Recently, black-box domain adaptation attracts a lot of attention, which is a new concept to realize domain adaptation with only a cloud API service instead of the source data or well-trained source model, reflecting the focus on development of cloud services and concerns about data security. However, the existing black-box domain adaptation methods always only use high-confidence samples which limits their performance. We propose a self-alignment approach based on statistic moment matching to realize black-box domain adaptation. We construct a model for target domain in the initial stage of our work. Then, we put target data into source model API to obtain the pseudo-labels and divide the target data into high-confidence and low-confidence parts according to their pseudo-labels confidence. By matching the data distributions between these two parts and self-supervised learning on high-confidence part, the performance on both parts samples can be boosted respectively. Information maximization is also applied to the target data to further improve their classification performance. Experiment results confirm that our method achieves state-of-the-art performance.

With the successful application of deformable convolution in aligning different video frames, it has also been used in video compression artifact reduction. The existing methods based on deformable convolution only apply 2D convolutional layers to generate the features for predicting alignment offsets, which is inaccurate due to limited receptive field. In this letter, we propose a new end-to-end network called Coarse-to-Fine Spatio-Temporal Information Fusion (CF-STIF) for compressed video quality enhancement by predicting better offsets with a larger receptive field. Specifically, several 3D convolutional layers are first to roughly fuse the spatio-temporal information in the video sequence, and then a Multi-level Residual Fusion Module (MLRF) is developed to generate global and local fused fine features from different levels for predicting deformable offsets. Thanks to the inherent advantages of 3D convolution and multi-scale strategy, the receptive field is greatly increased in both spatial and temporal dimensions, so that information from neighboring frames can be efficiently aggregated. In the end, the enhanced frame is derived by the proposed reconstruction module (REModule). Both qualitative and quantitative experimental results show that the proposed CF-STIF performs better than the state-of-the-art approaches.

With the prevalence of online social platforms, social recommendation has emerged as a promising direction that leverages the social network among users to enhance recommendation performance. However, the available social relations among users are usually extremely sparse and noisy, which may lead to inferior recommendation performance. To alleviate this problem, this paper novelly exploits the implicit higher-order social influence and dependencies among users to enhance social recommendation. In this paper, we propose a novel embedding method for general bipartite graphs, which defines inter-class message passing between explicit relations and intra-class message passing between implicit higher-order relations via a novel sequential modelling paradigm. Inspired by recent advances in self-attention-based sequential modelling, the proposed model features a self-attentive representation learning mechanism for implicit user-user relations. Moreover, this paper also explores the inductive embedding learning for social recommendation problems to improve the recommendation performance in cold-start settings. The proposed inductive learning paradigm for social recommendation enables embedding inference for those cold-start users and items (unseen during training) as long as they are linked to existing nodes in the original network. Extensive experiments on real-world datasets demonstrate the superiority of our method and suggest that higher-order implicit relationship among users is beneficial to improving social recommendation.

Medication recommendation aiming at accurate prescription is a significant clinical application that assists caregivers in professional practice of medicine, and obtaining informative patient representations plays an important role in building effective recommendation models. Meanwhile, conducting attentive multi-hop reading on Memory Neural Network (MemNN) that stores knowledge from previous admissions is widely applied to derive contextual patterns for accurate patient representations. However, regular attentive reading may repeatedly attend to the same slots of MemNN. Although the coverage mechanism is proposed to tackle the problem, it is based on the assumption that there is one-to-one alignment between source information and target outputs, which medical records do not follow. In pursuit of a valuable model for medication recommendation, we propose the Multi-hop Reading with Selective Coverage (MRSC). MRSC firstly conducts information selection on MemNN based on the coverage of each slot. Then the method involves coverage into the attention calculation during the multi-hop reading on MemNN, making sure that all important historical records is fully utilized by balancing attention within selected information. Experiments on real-world clinical dataset demonstrate that MRSC successfully derives informative patient representations for the recommendation by conducting selection on MemNN and limiting attention adjustment within selected information.

Unsupervised cross-domain object detection transfers a detection model trained on a source domain to the target domain that has a different data distribution from the source domain. Conventional domain adaptation detection protocols need source domain data during adaptation. However, due to some reasons such as data security, privacy and storage, we cannot access the source data in many practical applications. In this paper, we focus on source-data free domain adaptive object detection, which uses the pre-trained source model instead of the source data for cross-domain adaptation. Due to the lack of source data, we cannot directly align domain distribution between domains. To challenge this, we propose the Source style transferred Mean Teacher (SMT) for source-data free Object Detection. The batch normalization layers in the pre-trained model contain the style information and the data distribution of the non-observed source data. Thus we use the batch normalization information from the pre-trained source model to transfer the target domain feature to the source-like style feature to make full use of the knowledge from the pre-trained source model. Meanwhile, we use the consistent regularization of the Mean Teacher to further distill the knowledge from the source domain to the target domain. Furthermore, we found that by adding perturbations associated with the target domain distribution, the model can increase the robustness of domain-specific information, thus making the learned model generalized to the target domain. Experiments on multiple domain adaptation object detection benchmarks verify that our method is able to achieve state-of-the-art performance.

The past few years have witnessed the great success of using multi-frame information to enhance the quality of compressed video. Most existing methods do frame or feature alignments to bring similar information from neighborhood frames much closer to enhance frame quality. However, inaccurate motion estimation will bring new artifacts. In this paper, we propose a new approach without alignment, which takes each non-Peak Quality Frame (non-PQF) and its two adjacent Peak Quality Frames (PQFs) as input. A Pre-processing module based on multi-scale feature extraction strategy is used to broaden receptive field of the network. Then, an enhancement module uses a two-stream feature extraction architecture to combine deep architecture and attention mechanism to further gather similar information. In this module, the lost high-frequency and similar information can be further retrieved from the adjacent PQFs. The proposed network is trained in an end-to-end manner. Compared with the alignment based methods, the competitive results can be obtained. A large number of qualitative and quantitative experimental results demonstrate the robustness and effectiveness of the proposed method.

The current unsupervised cross-domain detection methods need source domain data to retrain the detection model in target domain. However, the source domain data may be unavailable due to privacy, decentralization, or computation resource restrictions. A natural idea is to optimize the parameters of the source domain model by self-supervised learning based on pseudo labels. We propose another approach from the viewpoint of noise perturbation without pseudo-labeling. It can be assumed that the source and target domains are actually derived from a domain invariant space through domain-specific perturbations, respectively. A super target domain can be constructed by augmenting more target domain perturbations to the target domain images. The optimal direction of the target domain to the domain invariant space can be approximated as the alignment direction from the super target domain to the target domain. Based on this idea, we propose a novel method called SOAP (SOurce data-free domain Adaptation through domain Perturbation) which can remove domain perturbation from the target domain. The image-level, instance-level, and category consistency regularizations based on Mean Teacher structure are proposed to learn the correct alignment direction. Specifically, the category consistency can also further improve the classification accuracy. Extensive experiments on multiple domain adaptation scenarios demonstrate that SOAP achieves better performance surpassing the baseline (Faster R-CNN) and multiple state-of-the-art domain adaptation methods which need to access source domain data.

Suicide is a critical issue in modern society. Early detection and prevention of suicide attempts should be addressed to save people's life. Current suicidal ideation detection (SID) methods include clinical methods based on the interaction between social workers or experts and the targeted individuals and machine learning techniques with feature engineering or deep learning for automatic detection based on online social contents. This article is the first survey that comprehensively introduces and discusses the methods from these categories. Domain-specific applications of SID are reviewed according to their data sources, i.e., questionnaires, electronic health records, suicide notes, and online user content. Several specific tasks and data sets are introduced and summarized to facilitate further research. Finally, we summarize the limitations of current work and provide an outlook of further research directions.

With the rapid development of mobile devices, people are generating huge volumes of images data every day for sharing on social media, which draws much research attention to understanding the contents of images. Image clustering plays an important role in image understanding systems. Often, most of the existing image clustering algorithms flatten digital images that are originally represented by matrices into 1D vectors as the image representation for the subsequent learning. The drawbacks of vector-based algorithms include limited consideration of spatial relationship between pixels and computational complexity, both of which blame to the simple vectorized representation. To overcome the drawbacks, we propose a novel image clustering framework that can work directly on matrices of images instead of flattened vectors. Specifically, the proposed algorithm simultaneously learn the clustering results and preserve the original correlation information within the image matrix. To solve the challenging objective function, we propose a fast iterative solution. Extensive experiments have been conducted on various benchmark datasets. The experimental results confirm the superiority of the proposed algorithm.

In various web applications like targeted advertising and recommender systems, the available categorical features (e.g., product type) are often of great importance but sparse. As a widely adopted solution, models based on Factorization Machines (FMs) are capable of modelling high-order interactions among features for effective sparse predictive analytics. As the volume of web-scale data grows exponentially over time, sparse predictive analytics inevitably involves dynamic and sequential features. However, existing FM-based models assume no temporal orders in the data, and are unable to capture the sequential dependencies or patterns within the dynamic features, impeding the performance and adaptivity of these methods. Hence, in this paper, we propose a novel Sequence-Aware Factorization Machine (SeqFM) for temporal predictive analytics, which models feature interactions by fully investigating the effect of sequential dependencies. As static features (e.g., user gender) and dynamic features (e.g., user interacted items) express different semantics, we innovatively devise a multi-view self-attention scheme that separately models the effect of static features, dynamic features and the mutual interactions between static and dynamic features in three different views. In SeqFM, we further map the learned representations of feature interactions to the desired output with a shared residual network. To showcase the versatility and generalizability of SeqFM, we test SeqFM in three popular application scenarios for FM-based models, namely ranking, classification and regression tasks. Extensive experimental results on six large-scale datasets demonstrate the superior effectiveness and efficiency of SeqFM.

In recommender systems, the user-item interaction data is usually sparse and not sufficient for learning comprehensive user/item representations for recommendation. To address this problem, we propose a novel dual-bridging recommendation model (DBRec). DBRec performs latent user/item group discovery simultaneously with collaborative filtering, and interacts group information with users/items for bridging similar users/items. Therefore, a user's preference over an unobserved item, in DBRec, can be bridged by the users within the same group who have rated the item, or the user-rated items that share the same group with the unobserved item. In addition, we propose to jointly learn user-user group (item-item group) hierarchies, so that we can effectively discover latent groups and learn compact user/item representations. We jointly integrate collaborative filtering, latent group discovering and hierarchical modelling into a unified framework, so that all the model parameters can be learned toward the optimization of the objective function. We validate the effectiveness of the proposed model with two real datasets, and demonstrate its advantage over the state-of-the-art recommendation models with extensive experiments.

Network embedding has been proven effective to learn low-dimensional vector representations for network vertices, and recently received a tremendous amount of research attention. However, most of existing methods for network embedding merely focus on preserving the first and second order proximities between nodes, and the important properties of node centrality are neglected. Various centrality measures such as Degree, Closeness, Betweenness, Eigenvector and PageRank centralities have been designed to measure the importance of individual nodes. In this paper, we focus on a novel yet unsolved problem that aims to learn low-dimensional continuous nodes representations that not only preserve the network structure, but also keep the centrality information. We propose a generalizable model, namely GraphCSC, that utilizes both linkage information and centrality information to learn low-dimensional vector representations for network vertices. The learned embeddings by GraphCSC are able to preserve different centrality information of nodes. In addition, we further propose GraphCSC-M, a more comprehensive model that can preserve different centrality information simultaneously through learning multiple centrality-specific embeddings, and a novel attentive multi-view learning approach is developed to compress multiple embeddings of one node into a compact vector representation. Extensive experiments have been conducted to demonstrate that our model is able to preserve different centrality information of nodes, and achieves better performance on several benchmark tasks compared with recent state-of-the-art network embedding methods.

The capability of extracting sequential patterns from the user-item interaction data is now becoming a key feature of recommender systems. Though it is important to capture the sequential effect, existing methods only focus on modelling the sparse item-wise sequential effect in user preference and only consider the homogeneous user interaction behaviors (i.e., a single type of user behavior). As a result, the data sparsity issue inevitably arises and makes the learned sequential patterns fragile and unreliable, impeding the sequential recommendation performance of existing methods. Hence, in this paper, we propose AIR, namely attentional intention-aware recommender systems to predict category-wise future user intention and collectively exploit the rich heterogeneous user interaction behaviors (i.e., multiple types of user behaviors). In AIR, we propose to represent user intention as an action-category tuple to discover category-wise sequential patterns and to capture varied effect of different types of actions for recommendation. A novel attentional recurrent neural network (ARNN) is proposed to model the intention migration effect and infer users' future intention. Besides, an intention-aware factorization machine (ITFM) is developed to perform intention-aware sequential recommendation. Experiments on two real-life datasets demonstrate the superiority and practicality of AIR in sequential top-k recommendation tasks.

Mobile keyboard suggestion is typically regarded as a word-level language modeling problem. Centralized machine learning techniques require the collection of massive user data for training purposes, which may raise privacy concerns in relation to users' sensitive data. Federated learning (FL) provides a promising approach to learning private language modeling for intelligent personalized keyboard suggestions by training models on distributed clients rather than training them on a central server. To obtain a global model for prediction, existing FL algorithms simply average the client models and ignore the importance of each client during model aggregation. Furthermore, there is no optimization for learning a well-generalized global model on the central server. To solve these problems, we propose a novel model aggregation with an attention mechanism considering the contribution of client models to the global model, together with an optimization technique during server aggregation. Our proposed attentive aggregation method minimizes the weighted distance between the server model and client models by iteratively updating parameters while attending to the distance between the server model and client models. Experiments on two popular language modeling datasets and a social media dataset show that our proposed method outperforms its counterparts in terms of perplexity and communication cost in most settings of comparison.

The human genome can reveal sensitive information and is potentially re-identifiable, which raises privacy and security concerns about sharing such data on wide scales. In this work, we propose a preventive approach for privacy-preserving sharing of genomic data in decentralized networks for Genome-wide association studies (GWASs), which have been widely used in discovering the association between genotypes and phenotypes. The key components of this work are: a decentralized secure network, with a privacy- preserving sharing protocol, and a gene fragmentation framework that is trainable in an end-to-end manner. Our experiments on real datasets show the effectiveness of our privacy-preserving approaches as well as significant improvements in efficiency when compared with recent, related algorithms.

Unlike traditional video recommendations, micro-video inherits the characteristics of social platforms, such as social relation. A large amount of micro-videos showing explosive growth is badly affecting the user’s choice. In this paper, we propose a multi-source multi-net micro-video recommendation model that recommends micro-videos fitting users’ best interests. Different from existing works, as micro-video inherits the characteristics of social platforms, we simultaneously incorporate multi-source content data of items and multi-networks of users to learn user and item representations for recommendation. This information can be complementary to each other in a way that multi-modality data can bridge the semantic gap among items, while multi-type user networks, such as following and reposting, are able to propagate the preferences among users. Furthermore, to discover the hidden categories of micro-videos that properly match users’ interests, we interactively learn the user–item representations and perform the hidden item category clustering. The resulted categorical representations are interacted with user representations to model user preferences at different levels of hierarchies. Finally, multi-source content item data, multi-type user networks and hidden item categories are jointly modelled in a unified recommender, and the parameters of the model are collaboratively learned to boost the recommendation performance. Experiments on a real dataset demonstrate the effectiveness of the proposed model and its advantage over the state-of-the-art baselines.

Fine-grained visual recognition is an important problem in pattern recognition applications. However, it is a challenging task due to the subtle interclass difference and large intraclass variation. Recent visual attention models are able to automatically locate critical object parts and represent them against appearance variations. However, without consideration of spatial dependencies in discriminative feature learning, these methods are underperformed in classifying fine-grained objects. In this paper, we present a deep attention-based spatially recursive model that can learn to attend to critical object parts and encode them into spatially expressive representations. Our network is technically premised on bilinear pooling, enabling local pairwise feature interactions between outputs from two different convolutional neural networks (CNNs) that correspond to distinct region detection and relevant feature extraction. Then, spatial long-short term memory (LSTMs) units are introduced to generate spatially meaningful hidden representations via the long-range dependency on all features in two dimensions. The attention model is leveraged between bilinear outcomes and spatial LSTMs for dynamic selection on varied inputs. Our model, which is composed of two-stream CNN layers, bilinear pooling, and spatial recursive encoding with attention, is end-to-end trainable to serve as the part detector and feature extractor whereby relevant features are localized, extracted, and encoded spatially for recognition purpose. We demonstrate the superiority of our method over two typical fine-grained recognition tasks: fine-grained image classification and person re-identification.

Video-based person re-identification (re-id) is a central application in surveillance systems with a significant concern in security. Matching persons across disjoint camera views in their video fragments are inherently challenging due to the large visual variations and uncontrolled frame rates. There are two steps crucial to person re-id, namely, discriminative feature learning and metric learning. However, existing approaches consider the two steps independently, and they do not make full use of the temporal and spatial information in the videos. In this paper, we propose a Siamese attention architecture that jointly learns spatiotemporal video representations and their similarity metrics. The network extracts local convolutional features from regions of each frame and enhances their discriminative capability by focusing on distinct regions when measuring the similarity with another pedestrian video. The attention mechanism is embedded into spatial gated recurrent units to selectively propagate relevant features and memorize their spatial dependencies through the network. The model essentially learns which parts (where) from which frames (when) are relevant and distinctive for matching persons and attaches higher importance therein. The proposed Siamese model is end-to-end trainable to jointly learn comparable hidden representations for paired pedestrian videos and their similarity value. Extensive experiments on three benchmark datasets show the effectiveness of each component of the proposed deep network while outperforming state-of-the-art methods.

Most of the existing analytics on ICU data mainly focus on mortality risk prediction and phenotyping analysis. However, they have limitations in providing sufficient evidence for decision making in a dynamically changing clinical environment. In this paper, we propose a novel approach that simultaneously analyses different organ systems to predict the illness severity of patients in an ICU, which can intuitively reflect the condition of the patients in a timely fashion. Specifically, we develop a novel deep learning model, namely MTRNN-ATT, which is based on multi-task recurrent neural networks. The physiological features of each organ system in time-series representations are learned by a single long short-term memory unit as a specific task. To utilize the relationships between organ systems, we use a shared LSTM unit to exploit the correlations between different tasks for further performance improvement. Also, we apply an attention mechanism in our deep model to learn the selective features at each stage to achieve better prediction results. We conduct extensive experiments on a real-world clinical dataset (MIMIC-III) to compare our method with many state-of-the-art methods. The experiment results demonstrate that the proposed approach performs better on the prediction tasks of illness severity scores.

As a common strategy in sales-supply chains, the prediction of sales volume offers precious information for companies to achieve a healthy balance between supply and demand. In practice, the sales prediction task is formulated as a time series prediction problem which aims to predict the future sales volume for different products with the observation of various influential factors (e.g., brand, season, discount, etc.) and corresponding historical sales records. However, with the development of contemporary commercial markets, the dynamic interaction between influential factors with different semantic meanings becomes more subtle, causing challenges in fully capturing dependencies among these variables. Besides, though seeking similar trends from the history benefits the accuracy for the prediction of upcoming sales, existing methods hardly suit sales prediction tasks because the trends in sales time series are more irregular and complex. Hence, we gain insights from the encoder-decoder recurrent neural network (RNN) structure, and propose a novel framework named TADA to carry out trend alignment with dualattention, multi-task RNNs for sales prediction. In TADA, we innovatively divide the influential factors into internal feature and external feature, which are jointly modelled by a multi-task RNN encoder. In the decoding stage, TADA utilizes two attention mechanisms to compensate for the unknown states of influential factors in the future and adaptively align the upcoming trend with relevant historical trends to ensure precise sales prediction. Experimental results on two real-world datasets comprehensively show the superiority of TADA in sales prediction tasks against other state-of-the-art competitors.

Heterogenous information network embedding aims to embed heterogenous information networks (HINs) into low dimensional spaces, in which each vertex is represented as a low-dimensional vector, and both global and local network structures in the original space are preserved. However, most of existing heterogenous information network embedding models adopt the dot product to measure the proximity in the low dimensional space, and thus they can only preserve the first-order proximity and are insufficient to capture the global structure. Compared with homogenous information networks, there are multiple types of links (i.e., multiple relations) in HINs, and the link distribution w.r.t relations is highly skewed. To address the above challenging issues, we propose a novel heterogenous information network embedding model PME based on the metric learning to capture both first-order and second-order proximities in a unified way. To alleviate the potential geometrical inflexibility of existing metric learning approaches, we propose to build object and relation embeddings in separate object space and relation spaces rather than in a common space. Afterwards, we learn embeddings by firstly projecting vertices from object space to corresponding relation space and then calculate the proximity between projected vertices. To overcome the heavy skewness of the link distribution w.r.t relations and avoid "over-sampling'' or "under-sampling'' for each relation, we propose a novel loss-aware adaptive sampling approach for the model optimization. Extensive experiments have been conducted on a large-scale HIN dataset, and the experimental results show superiority of our proposed PME model in terms of prediction accuracy and scalability.

Recognition of human intention based on Electroencephalography (EEG) signals attracts strong research interest in pattern recognition because of its promising applications that enable non-muscular communications and controls. Over the past few years, most EEG-based recognition works make significant efforts to learn extracted features to explore specific patterns between a segment of EEG signals and the corresponding activities. Unfortunately, vectorization-based feature representations, either vector-like or matrix-like ones, suffer from massive signal noise and difficulties of exploiting signal correlations between adjacent sensors of EEG signals. Most importantly, EEG signals are represented by one unique frequency and then fed into the subsequent learning model. Neglecting different frequencies of EEG signals can be detrimental to activity recognition because a particular frequency of EEG signals is more helpful to recognize some activities. Inspired by this idea, we propose to extract EEG signals with different frequencies and introduce a novel Multi-task deep learning model to learn the human intentions. We have conducted extensive experiments on a publicly available EEG benchmark dataset and compared our method with many state-of-the-art algorithms. The experimental results demonstrate that the proposed Multi-task deep recurrent neural network outperforms all the compared methods in a multi-class scenario.

Learning features from multiple views has attracted much research attention in different machine learning tasks, such as multiclass and multilabel classification problems. In this paper, we propose a multiclass multilabel multiview learning framework with a linear computational cost where an example is associated with at least one label and represented by multiple information sources. We simultaneously analyze all features by learning an integrated projection matrix. We can also automatically select more important views for subsequent classifier to predict each class. As the proposed objective function is nonsmooth and difficult to solve, we apply a novel optimization method that converts the multiview learning problem to a set of linear single-view learning problems by bridging our problem to an easily solvable approach. Compared to the conventional methods which learn the entire projection matrix, our algorithm independently optimizes each column of the projection matrix for each class, which can be easily parallelized. In each column optimization, the most computationally intensive step is pure and simple matrix-by-vector multiplication. As a result, our algorithm is much more applicable to large-scale problems than the multiview learning methods with a nonlinear computational cost. Moreover, rigorous convergence proof of the proposed algorithm is also provided. To evaluate the effectiveness of the proposed approach, experimental comparisons are made with state-of-the-art algorithms in multiclass and multilabel classification tasks on many multiview benchmarks. We also report the efficiency comparison results on different numbers of data samples. The experimental results demonstrate that our algorithm can achieve superior performance to all the compared algorithms.

Understanding and recognizing human activities is a fundamental research topic for a wide range of important applications such as fall detection and remote health monitoring and intervention. Despite active research in human activity recognition over the past years, existing approaches based on computer vision or wearable sensor technologies present several significant issues such as privacy (e.g., using video camera to monitor the elderly at home) and practicality (e.g., not possible for an older person with dementia to remember wearing devices). In this paper, we present a low-cost, unobtrusive, and robust system that supports independent living of older people. The system interprets what a person is doing by deciphering signal fluctuations using radio-frequency identification (RFID) technology and machine learning algorithms. To deal with noisy, streaming, and unstable RFID signals, we develop a compressive sensing, dictionary-based approach that can learn a set of compact and informative dictionaries of activities using an unsupervised subspace decomposition. In particular, we devise a number of approaches to explore the properties of sparse coefficients of the learned dictionaries for fully utilizing the embodied discriminative information on the activity recognition task. Our approach achieves efficient and robust activity recognition via a more compact and robust representation of activities. Extensive experiments conducted in a real-life residential environment demonstrate that our proposed system offers a good overall performance and shows the promising practical potential to underpin the applications for the independent living of the elderly.

With recent advances in radio-frequency identification (RFID), wireless sensor networks, and Web services, physical things are becoming an integral part of the emerging ubiquitous Web. Finding correlations among ubiquitous things is a crucial prerequisite for many important applications such as things search, discovery, classification, recommendation, and composition. This article presents DisCor-T , a novel graph-based approach for discovering underlying connections of things via mining the rich content embodied in the human-thing interactions in terms of user, temporal, and spatial information. We model this various information using two graphs, namely a spatio-temporal graph and a social graph. Then, random walk with restart (RWR) is applied to find proximities among things, and a relational graph of things (RGT) indicating implicit correlations of things is learned. The correlation analysis lays a solid foundation contributing to improved effectiveness in things management and analytics. To demonstrate the utility of the proposed approach, we develop a flexible feature-based classification framework on top of RGT and perform a systematic case study. Our evaluation exhibits the strength and feasibility of the proposed approach.

In the era of big data, a mechanism that can automatically annotate disease codes to patients’ records in the medical information system is in demand. The purpose of this work is to propose a framework that automatically annotates the disease labels of multi-source patient data in Intensive Care Units (ICUs). We extract features from two main sources, medical charts and notes. The Bag-of-Words model is used to encode the features. Unlike most of the existing multi-label learning algorithms that globally consider correlations between diseases, our model learns disease correlation locally in the patient data. To achieve this, we derive a local disease correlation representation to enrich the discriminant power of each patient data. This representation is embedded into a unified multi-label learning framework. We develop an alternating algorithm to iteratively optimize the objective function. Extensive experiments have been conducted on a real-world ICU database. We have compared our algorithm with representative multi-label learning algorithms. Evaluation results have shown that our proposed method has state-of-the-art performance in the annotation of multiple diagnostic codes for ICU patients. This study suggests that problems in the automated diagnosis code annotation can be reliably addressed by using a multi-label learning model that exploits disease correlation. The findings of this study will greatly benefit health care and management in ICU considering that the automated diagnosis code annotation can significantly improve the quality and management of health care for both patients and caregivers.

How to accurately forecast seasonal time series is very important for many business area such as marketing decision, planning production and profit estimation. In this paper, we propose a weighted gradient Radial Basis Function Network based AutoRegressive (WGRBF-AR) model for modeling and predicting the nonlinear and non-stationary seasonal time series. This WGRBF-AR model is a synthesis of the weighted gradient RBF network and the functional-coefficient autoregressive (FAR) model through using the WGRBF networks to approximate varying coefficients of FAR model. It not only takes the advantages of the FAR model in nonlinear dynamics description but also inherits the capability of the WGRBF network to deal with non-stationarity. We test our model using ten-years retail sales data on five different commodity in US. The results demonstrate that the proposed WGRBF-AR model can achieve competitive prediction accuracy compared with the state-of-the-art.

In the era of the Internet of Things, enormous number of sensors have been deployed in different locations, generating massive time-series sensory data with geo-tags. However, such sensory readings are easily missing due to various reasons such as the hardware malfunction, connection errors, and data corruption. This paper focuses on this challenge--how to accurately yet efficiently recover the missing values for corrupted time-series sensor data with geo-stamps. In this paper, we formulate the time-series sensor data as a 3-order tensor that naturally preserves sensors' temporal and spatial dependencies. Then we exploit its low-rank and sparse-noise structures by drawing upon recent advances in Robust Principal Component Analysis (RPCA) and tensor completion theory. The main novelty of this paper lies in that, we design a highly efficient optimization method that combines the alternating direction method of multipliers and accelerated proximal gradient to recover the data tensor. Besides testing our method using the synthetic data, we also design a real-world testbed by passive RFID (RadioFrequency IDentification) sensors. The results demonstrate the effectiveness and accuracy of our approach.

Data veracity is a grand challenge for various tasks on the Web. Since the web data sources are inherently unreliable and may provide conflicting information about the same real-world entities, truth discovery is emerging as a countermeasure of resolving the conflicts by discovering the truth, which conforms to the reality, from the multi-source data. A major challenge related to truth discovery is that different data items may have varying numbers of true values (or multi-truth), which counters the assumption of existing truth discovery methods that each data item should have exactly one true value. In this paper, we address this challenge by exploiting and leveraging the implications from multi-source data. In particular, we exploit three types of implications, namely the implicit negative claims, the distribution of positive/negative claims, and the co-occurrence of values in sources' claims, to facilitate multi-truth discovery. We propose a probabilistic approach with improvement measures that incorporate the three implications in all stages of truth discovery process. In particular, incorporating the negative claims enables multi-truth discovery, considering the distribution of positive/negative claims relieves truth discovery from the impact of sources' behavioral features in the specific datasets, and considering values' co-occurrence relationship compensates the information lost from evaluating each value in the same claims individually. Experimental results on three real-world datasets demonstrate the effectiveness of our approach.

Truth discovery is the problem of detecting true values from the conflicting data provided by multiple sources on the same data items. Since sources' reliability is unknown a priori, a truth discovery method usually estimates sources' reliability along with the truth discovery process. A major limitation of existing truth discovery methods is that they commonly assume exactly one true value on each data item and therefore cannot deal with the more general case that a data item may have multiple true values (or multi-truth). Since the number of true values may vary from data item to data item, this requires truth discovery methods being able to detect varying numbers of truth values from the multi-source data. In this paper, we propose a multi-truth discovery approach, which addresses the above challenges by providing a generic framework for enhancing existing truth discovery methods. In particular, we redeem the numbers of true values as an important clue for facilitating multi-truth discovery. We present the procedure and components of our approach, and propose three models, namely the byproduct model, the joint model, and the synthesis model to implement our approach. We further propose two extensions to enhance our approach, by leveraging the implications of similar numerical values and values' co-occurrence information in sources' claims to improve the truth discovery accuracy. Experimental studies on real-world datasets demonstrate the effectiveness of our approach.

Despite the active research into, and the development of, human activity recognition over the decades, existing techniques still have several limitations, in particular, poor performance due to insufficient ground-truth data and little support of intra-class variability of activities (i.e., the same activity may be performed in different ways by different individuals, or even by the same individuals with different time frames). Aiming to tackle these two issues, in this paper, we present a robust activity recognition approach by extracting the intrinsic shared structures from activities to handle intra-class variability, and the approach is embedded into a semi-supervised learning framework by utilizing the learned correlations from both labeled and easily-obtained unlabeled data simultaneously. We use l2,1 minimization on both loss function and regularizations to effectively resist outliers in noisy sensor data and improve recognition accuracy by discerning underlying commonalities from activities. Extensive experimental evaluations on four community-contributed public datasets indicate that with little training samples, our proposed approach outperforms a set of classical supervised learning methods as well as those recently proposed semi-supervised approaches.

General health examination is an integral part of healthcare in many countries. Identifying the participants at risk is important for early warning and preventive intervention. The fundamental challenge of learning a classification model for risk prediction lies in the unlabeled data that constitutes the majority of the collected dataset. Particularly, the unlabeled data describes the participants in health examinations whose health conditions can vary greatly from healthy to very-ill. There is no ground truth for differentiating their states of health. In this paper, we propose a graph-based, semi-supervised learning algorithm called SHG-Health (Semi-supervised Heterogeneous Graph on Health) for risk predictions to classify a progressively developing situation with the majority of the data unlabeled. An efficient iterative algorithm is designed and the proof of convergence is given. Extensive experiments based on both real health examination datasets and synthetic datasets are performed to show the effectiveness and efficiency of our method.

With the latest developments in database technologies, it becomes easier to store the medical records of hospital patients from their first day of admission than was previously possible. In Intensive Care Units (ICU), modern medical information systems can record patient events in relational databases every second. Knowledge mining from these huge volumes of medical data is beneficial to both caregivers and patients. Given a set of electronic patient records, a system that effectively assigns the disease labels can facilitate medical database management and also benefit other researchers, e.g., pathologists. In this paper, we have proposed a framework to achieve that goal. Medical chart and note data of a patient are used to extract distinctive features. To encode patient features, we apply a Bag-of-Words encoding method for both chart and note data. We also propose a model that takes into account both global information and local correlations between diseases. Correlated diseases are characterized by a graph structure that is embedded in our sparsity-based framework. Our algorithm captures the disease relevance when labeling disease codes rather than making individual decision with respect to a specific disease. At the same time, the global optimal values are guaranteed by our proposed convex objective function. Extensive experiments have been conducted on a real-world large-scale ICU database. The evaluation results demonstrate that our method improves multi-label classification results by successfully incorporating disease correlations.

The emerging Internet of Things (IoT) bridges the gap between the physical and the digital worlds, which enables a deeper understanding of user preferences and behaviors. The rich interactions and relations between users and things call for effective and efficient recommendation approaches to better meet users’ interests and needs. In this article, we focus on the problem of things recommendation in IoT, which is important for many applications such as e-Commerce and health care. We discuss the new properties of recommending things of interest in IoT, and propose a unified probabilistic factor based framework by fusing relations across heterogeneous entities of IoT, for example, user-thing relations, user-user relations, and thing-thing relations, to make more accurate recommendations. Specifically, we develop a hypergraph to model things’ spatiotemporal correlations, on top of which implicit things correlations can be generated. We have built an IoT testbed to validate our approach and the experimental results demonstrate its feasibility and effectiveness.

Many real-world applications rely on multiple data sources to provide information on their interested items. Due to the noises and uncertainty in data, given a specific item, the information from different sources may conflict. To make reliable decisions based on these data, it is important to identify the trustworthy information by resolving these conflicts, i.e., the truth discovery problem. Current solutions to this problem detect the veracity of each value jointly with the reliability of each source for each data item. In this way, the efficiency of truth discovery is strictly confined by the problem scale, which in turn limits truth discovery algorithms from being applicable on a large scale. To address this issue, we propose an approximate truth discovery approach, which divides sources and values into groups according to a user-specified approximation criterion. The groups are then used for efficient inter-value influence computation to improve the accuracy. Our approach is applicable to most existing truth discovery algorithms. Experiments on real-world datasets show that our approach improves the efficiency compared to existing algorithms while achieving similar or even better accuracy. The scalability is further demonstrated by experiments on large synthetic datasets.

Truth-finding is the fundamental technique for corroborating reports from multiple sources in both data integration and collective intelligent applications. Traditional truth-finding methods assume a single true value for each data item and therefore cannot deal will multiple true values (i.e., the multi-truth-finding problem). So far, the existing approaches handle the multi-truth-finding problem in the same way as the single-truth-finding problems. Unfortunately, the multi-truth-finding problem has its unique features, such as the involvement of sets of values in claims, different implications of inter-value mutual exclusion, and larger source profiles. Considering these features could provide new opportunities for obtaining more accurate truth-finding results. Based on this insight, we propose an integrated Bayesian approach to the multi-truth-finding problem, by taking these features into account. To improve the truth-finding efficiency, we reformulate the multi-truth-finding problem model based on the mappings between sources and (sets of) values. New mutual exclusive relations are defined to reflect the possible co-existence of multiple true values. A finer-grained copy detection method is also proposed to deal with sources with large profiles. The experimental results on three real-world datasets show the effectiveness of our approach.

Understanding and recognizing the activities performed by people is a fundamental research topic for a wide range of important applications such as fall detection of elderly people. In this paper, we present the technical details behind Freedom, a low-cost, unobtrusive system that supports independent livingof the older people. The Freedom system interprets what aperson is doing by leveraging machine learning algorithmsand radio-frequency identification (RFID) technology. To dealwith noisy, streaming, unstable RFID signals, we particularlydevelop a dictionary-based approach that can learn dictionariesfor activities using an unsupervised sparse coding algorithm. Our approach achieves efficient and robust activity recognitionvia a more compact representation of the activities. Extensiveexperiments conducted in a real-life residential environmentdemonstrate that our proposed system offers a good overallperformance (e.g., achieving over 96% accuracy in recognizing23 activities) and has the potential to be further developed tosupport the independent living of elderly people.

Activity recognition is a core component of ubiquitous computing applications (e.g., fall detection of elder people) since many of such applications require an intelligent environment to infer what a person is doing or attempting to do. Unfortunately, the success of existing approaches on activity recognition relies heavily on people's involvement such as wearing battery-powered sensors, which might not be practical in real-world situations (e.g., people may forget to wear sensors). In this paper, we propose a device-free, real-time posture recognition technique using an array of pure passive RFID tags. In particular, posture recognition is treated as a machine learning problem where a series of probabilistic model is built via learning how the Received Signal Strength Indicator (RSSI) from the tag array is distributed when a person performs different postures. We also design a segmentation algorithm to divide the continuous, multidimensional RSSI data stream into a set of individual segments by analyzing the shape of the RSSI data. Our approach for posture recognition eliminates the need for the monitored subjects to wear any devices. To the best of our knowledge, this work is the first on device-free posture recognition using low cost, unobtrusive RFID technology. Our experimental studies demonstrate the feasibility of the proposed approach for posture recognition.

Falls are among the leading causes of hospitalization for the elderly and illness individuals. Considering that the elderly often live alone and receive only irregular visits, it is essential to develop such a system that can effectively detect a fall or abnormal activities. However, previous fall detection systems either require to wear sensors or are able to detect a fall but fail to provide fine-grained contextual information (e.g., what is the person doing before falling, falling directions). In this paper, we propose a device-free, fine-grained fall detection system based on pure passive UHF RFID tags, which not only is capable of sensing regular actions and fall events simultaneously, but also provide caregivers the contexts of fall orientations. We first augment the Angle-based Outlier Detection Method (ABOD) to classify normal actions (e.g., standing, sitting, lying and walking) and detect a fall event. Once a fall event is detected, we first segment a fix-length RSSI data stream generated by the fall and then utilize Dynamic Time Warping (DTW) based kNN to distinguish the falling direction. The experimental results demonstrate that our proposed approach can distinguish the living status before fall happening, as well as the fall orientations with a high accuracy. The experiments also show that our device-free, fine-grained fall detection system offers a good overall performance and has the potential to better support the assisted living of older people.

RFID-based localization and tracking has some promising potentials. By combining localization with its identification capability, existing applications can be enhanced and new applications can be developed. In this paper, we investigate a tag-free indoor localizing and tracking problem (e.g., people tracking) without requiring subjects to carry any tags or devices in a pure passive environment. We formulate localization as a classification task. In particular, we model the received signal strength indicator (RSSI) of passive tags using multivariate Gaussian Mixture Model (GMM), and use the Expectation Maximization (EM) to learn the maximum likelihood estimates of the model parameters. Several other learning-based probabilistic approaches are also explored in the localization problem. To track a moving subject, we propose GMM based Hidden Markov Model (HMM) and k Nearest Neighbor (kNN) based HMM approaches. We conduct extensive experiments in a testbed formed by passive RFID tags, and the experimental results demonstrate the effectiveness and accuracy of our approach.

People take regular medical examinations mostly not for discovering diseases but for having a peace of mind regarding their health status. Therefore, it is important to give them an overall feedback with respect to all the health indicators that have been ranked against the whole population. In this paper, we propose a framework of mining Personal Health Index (PHI) from a large and comprehensive geriatric medical examination (GME) dataset. We define PHI as an overall score of personal health status based on a complement probability of health risks. The health risks are calculated using the information from the cause of death (COD) dataset that is linked to the GME dataset. Especially, the highest health risk is revealed in the cases of people who had been taking GME for some years and then passed away for medical reasons. The proposed framework consists of methods in data pre-processing, feature extraction and selection, and model selection. The effectiveness of the proposed framework is validated by a set of comprehensive experiments based on the records of 102,258 participants. As the first of this kind, our work provides a baseline for further research.

Background learning is a pre-processing of motion detection which is a basis step of video analysis. For the static background, many previous works have already achieved good performance. However, the results on learning dynamic background are still much to be improved. To address this challenge, in this paper, a novel and practical method is proposed based on deep auto-encoder networks. Firstly, dynamic background images are extracted through a deep auto-encoder network (called Background Extraction Network) from video frames containing motion objects. Then, a dynamic background model is learned by another deep auto-encoder network (called Background Learning Network) using the extracted background images as the input. To be more flexible, our background model can be updated on-line to absorb more training samples. Our main contributions are 1) a cascade of two deep auto-encoder networks which can deal with the separation of dynamic background and foregrounds very efficiently; 2) a method of online learning is adopted to accelerate the training of Background Extraction Network. Compared with previous algorithms, our approach obtains the best performance over six benchmark data sets. Especially, the experiments show that our algorithm can handle large variation background very well.

Device-free passive localization aims to localize or track targets without requiring them to carry any devices or to be actively involved with the localization process. This technique has received much attention recently in a wide range of applications including elderly people surveillance, intrud

With recent advances in radio-frequency identification (RFID), wireless sensor networks, and Web-based services, physical things are becoming an integral part of the emerging ubiquitous Web. In this paper, we focus on the things recommendation problem in Internet of Things (IoT). In particular, we propose a unified probabilistic based framework by fusing information across relationships between users (i.e., users'social network) and things (i.e., things correlations) to make more accurate recommendations. The proposed approach not only inherits the advantages of the matrix factorization, but also exploits the merits of social relationships and thing-thing correlations. We validate our approach based on an Internet of Things platform and the experimental results demonstrate its feasibility and effectiveness.

Privacy preservation is becoming a critical issue to data‐mining processes. In practice, a data transformation process is often needed to preserve privacy. However, data transformation would introduce a data quality issue. In this case, the impact on data quality due to the data transformation should be estimated and made clear to the user of the data transformation process. In this article, we consider the problem of k ‐anonymization transformation in associative classification. The privacy preservation and data quality issues are considered in twofold. First, we propose a frequency‐based data quality metric to represent the data quality for associative classification. Second, a novel heuristic algorithm, namely minimum classification correction rate transformation, is proposed. The algorithm is guided by the classification correction rate of the given datasets. We validate our proposed metric and algorithm with University of California–Irvine repository datasets. The experiment results have shown that our proposed metric can effectively demonstrate the data quality for associative classification. The results also show that the proposed algorithm is not only efficient but also highly effective.

This paper presents a semi-supervised method for categorizing human actions using multiple visual features. The proposed algorithm simultaneously learns multiple features from a small number of labeled videos, and automatically utilizes data distributions between labeled and unlabeled data to boost the recognition performance. Shared structural analysis is applied in our approach to discover a common subspace shared by each type of feature. In the subspace, the proposed algorithm is able to characterize more discriminative information of each feature type. Additionally, data distribution information of each type of feature has been preserved. The aforementioned attributes make our algorithm robust for action recognition, especially when only limited labeled training samples are provided. Extensive experiments have been conducted on both the choreographed and the realistic video datasets, including KTH, Youtube action and UCF50. Experimental results show that our method outperforms several state-of-the-art algorithms. Most notably, much better performances have been achieved when there are only a few labeled training samples.

Online human gesture recognition has a wide range of applications in computer vision, especially in human-computer interaction applications. The recent introduction of cost-effective depth cameras brings a new trend of research on body-movement gesture recognition. However, there are two major challenges: (i) how to continuously detect gestures from unsegmented streams, and (ii) how to differentiate different styles of the same gesture from other types of gestures. In this article, we solve these two problems with a new effective and efficient feature extraction method—Structured Streaming Skeleton (SSS)—which uses a dynamic matching approach to construct a feature vector for each frame. Our comprehensive experiments on MSRC-12 Kinect Gesture, Huawei/3DLife-2013, and MSR-Action3D datasets have demonstrated superior performances than the state-of-the-art approaches. We also demonstrate model selection based on the proposed SSS feature, where the classifier of squared loss regression with l 2,1 norm regularization is a recommended classifier for best performance.

State-of-the-art studies on cyberbullying detection, using text classification, predominantly take it for granted that streaming text can be completely labelled. However, the rapid growth of unlabelled data generated in real time from online content

With recent advances in radio-frequency identification (RFID), wireless sensor networks, and Web services, physical things are becoming an integral part of the emerging ubiquitous Web. Correlation discovery for ubiquitous things is critical for many important applications such as things search, recommendation, annotation, classification, clustering, composition, and management. In this paper, we propose a novel approach for discovering things correlation based on user, temporal, and spatial information captured from usage events of things. In particular, we use a spatio-temporal graph and a social graph to model things usage contextual information and user-thing relationships respectively. Then, we apply random walks with restart on these graphs to compute correlations among things. This correlation analysis lays a solid foundation and contributes to improved effectiveness in things management. To demonstrate the utility of our approach, we perform a systematic case study and comprehensive experiments on things annotation.

Online human gesture recognition has a wide range of applications in computer vision, especially in human-computer interaction applications. Recent introduction of cost-effective depth cameras brings on a new trend of research on body-movement gesture recognition. However, there are two major challenges: i) how to continuously recognize gestures from unsegmented streams, and ii) how to differentiate different styles of a same gesture from other types of gestures. In this paper, we solve these two problems with a new effective and efficient feature extraction method that uses a dynamic matching approach to construct a feature vector for each frame and improves sensitivity to the features of different gestures and decreases sensitivity to the features of gestures within the same class. Our comprehensive experiments on MSRC-12 Kinect Gesture and MSR-Action3D datasets have demonstrated a superior performance than the stat-of-the-art approaches.

The emerging of the internet-based services poses a privacy threat to the individuals. Data transformation to meet a privacy standard becomes a requirement for typical data processing for the services. (k, e)-anonymisation is one of the most promising data transformation approaches, since it can provide high-accuracy aggregate query results. Though, the computational cost of the algorithm providing optimal solutions for such approach is not very high, i.e., O(n2). In certain environments, the data to be processed can be appended at any time. In this paper, we address an efficiency issue of the incremental privacy preservation using (k, e)-anonymisation approach. The impact of the increment is observed theoretically. We propose an incremental algorithm based on such observation. The algorithm can replace the quadratic-complexity processing by a linear function on some part of the dataset, while the optimal results are guaranteed. Additionally, a few indexes are proposed to further improve the efficiency of the proposed algorithm. The experiments have been conducted to validate our work. From the results, it can be seen that the proposed work is highly efficient comparing with the non-incremental algorithm and an approximation algorithm.

Human action recognition in videos draws strong research interest in computer vision because of its promising applications for video surveillance, video annotation, interactive gaming, etc. However, the amount of video data containing human actions is increasing exponentially, which makes the management of these resources a challenging task. Given a database with huge volumes of unlabeled videos, it is prohibitive to manually assign specific action types to these videos. Considering that it is much easier to obtain a small number of labeled videos, a practical solution for organizing them is to build a mechanism which is able to conduct action annotation automatically by leveraging the limited labeled videos. Motivated by this intuition, we propose an automatic video annotation algorithm by integrating semi-supervised learning and shared structure analysis into a joint framework for human action recognition. We apply our algorithm on both synthetic and realistic video datasets, including KTH [20], CareMedia dataset [1], Youtube action [12] and its extended version, UCF50 [2]. Extensive experiments demonstrate that the proposed algorithm outperforms the compared algorithms for action recognition. Most notably, our method has a very distinct advantage over other compared algorithms when we have only a few labeled samples.

Both similarity-based and popularity-based document ranking functions have been successfully applied to information retrieval (IR) in general. However, the dimension of semantic granularity also should be considered for effective retrieval. In this article, we propose a semantic granularity-based IR model that takes into account the three dimensions, namely similarity, popularity, and semantic granularity, to improve domain-specific search. In particular, a concept-based computational model is developed to estimate the semantic granularity of documents with reference to a domain ontology. Semantic granularity refers to the levels of semantic detail carried by an information item. The results of our benchmark experiments confirm that the proposed semantic granularity based IR model performs significantly better than the similarity-based baseline in both a bio-medical and an agricultural domain. In addition, a series of user-oriented studies reveal that the proposed document ranking functions resemble the implicit ranking functions exercised by humans. The perceived relevance of the documents delivered by the granularity-based IR system is significantly higher than that produced by a popular search engine for a number of domain-specific search tasks. To the best of our knowledge, this is the first study regarding the application of semantic granularity to enhance domain-specific IR.

The sharing of experience in eLearning has recently attracted the attention of researchers to improve efficacy of eLearning in higher learning especially in the absence of face-to-face contact with educators, lecturers, facilitators and tutors. We propose the recommendation of learning experiences (LEs), a type of tacit knowledge, from a learner to other learners by capturing LEs in the form of events of interactions which are similar to sequence of events. The sequence of events is treated as basket data and analyzed using sequential data mining to determine the patterns of learning. Collaborative filtering recommendation is introduced to recommend appropriate LEs to peer learners for the improvement of learning.

Existing algorithms for positive unlabeled learning (PU learning) only work with certain data. However, data uncertainty is prevalent in many real-world applications such as sensor network, market analysis and medical diagnosis. In this paper, based on positive naive Bayes (PNB), which is a PU learning algorithm for certain data, we propose an algorithm to handle uncertain data. However, it requires the prior probability of positive class and in real-life applications it is generally difficult for the users to provide this parameter, which is a drawback inherited from traditional PNB algorithm. We improve it by selecting the value of the prior probability of positive class automatically that can make the obtained classifier achieved optimal performance on the validation set. The conducted experiments show that the proposed algorithm yields good performance without user-specified the prior probability of positive class and has satisfactory performance even on highly uncertain data.

Trust management in an open RFID system environment is a nontrivial problem, where different organizations have different business workflows and operate on different standards and protocols. Open RFID systems can only be effective if the systems can trust each other and be collaborative. The open system environment is also constantly evolving. So the trust and the collaborations need to be constantly maintained to cope with changes. RFID is becoming a ubiquitous computing technology imposing security and privacy threats. Counterfeiting in supply chain management is an attack with cloned and fraud RFID tags in order to gain illegal benefits. In this paper we will extend our previous work on a trust framework and construct a computational model for the trust management. The trust evaluation is built into the process of transactions of the data exchange and authorization in order to facilitate a better data sharing and access control. An example of wine counterfeiting will be presented and we will show how our computational trust model helps in reducing fraud brand of wines in supply chain management (SCM).

A new nonslicing floorplan representation, the moving block sequence (MBS), is proposed in this paper. Our idea of the MBS originates from the observation that placing blocks on a chip has some similarities to playing the game, Tetrisreg. Since no extra constraints are exerted on solution spaces, the MBS is not only useful for evolutionary algorithms, but also for dealing with rectangular, convex rectilinear, and concave rectilinear blocks, similarly and simultaneously, without partitioning rectilinear blocks into subblocks. This is owed to a special structure designed for recording the information of both convex and concave rectilinear blocks in a uniform form. Theoretical analyses show that the computational cost of transforming an MBS to a floorplan with rectangular blocks, in terms of the number of blocks, is between linear and quadratic. Furthermore, as a follow-up of our previous works, a new organizational evolutionary algorithm (OEA) based on the MBS (MBS-OEA) is proposed. With the intrinsic properties of the MBS in mind, three new evolutionary operators are designed in the MBS-OEA. To test the performance of the MBS-OEA, benchmarks with hard rectangular, soft rectangular, and hard rectilinear blocks are used. The number of blocks in these benchmarks varies from 9 to 300. Also, the MBS-OEA and several well-designed existing algorithms are compared. The results show that the MBS-OEA can find high quality solutions for various problems. Additionally, the MBS-OEA shows a good performance in solving the problems with 300 hard rectangular blocks, 100 soft rectangular blocks, and 100 hybrid blocks, including both soft rectangular and hard rectilinear blocks. This illustrates that the MBS-OEA is not only suitable for solving a wide range of problems, but also competent for solving large-scale problems. Finally, a set of specific experiments is designed to identify the key component that is mainly responsible for the good performance of the MBS-OEA.

Domain specific information retrieval has become in demand. Not only domain experts, but also average non-expert users are interested in searching domain specific (e.g., medical and health) information from online resources. However, a typical problem to average users is that the search results are always a mixture of documents with different levels of readability. Non-expert users may want to see documents with higher readability on the top of the list. Consequently the search results need to be re-ranked in a descending order of readability. It is often not practical for domain experts to manually label the readability of documents for large databases. Computational models of readability needs to be investigated. However, traditional readability formulas are designed for general purpose text and insufficient to deal with technical materials for domain specific information retrieval. More advanced algorithms such as textual coherence model are computationally expensive for re-ranking a large number of retrieved documents. In this paper, we propose an effective and computationally tractable concept-based model of text readability. In addition to textual genres of a document, our model also takes into account domain specific knowledge, i.e., how the domain-specific concepts contained in the document affect the document's readability. Three major readability formulas are proposed and applied to health and medical information retrieval. Experimental results show that our proposed readability formulas lead to remarkable improvements in terms of correlation with users' readability ratings over four traditional readability measures.

This paper presents a neural network based technique for the classification of segments of road images into cracks and normal images. The density and histogram features are extracted. The features are passed to a neural network for the classification of images into images with and without cracks. Once images are classified into cracks and non-cracks, they are passed to another neural network for the classification of a crack type after segmentation. Some experiments were conducted and promising results were obtained. The selected results and a comparative analysis are included in this paper.

In this letter, we propose a class of self-stabilizing learning algorithms for minor component analysis (MCA), which includes a few well-known MCA learning algorithms. Self-stabilizing means that the sign of the weight vector length change is independent of the presented input vector. For these algorithms, rigorous global convergence proof is given and the convergence rate is also discussed. By combining the positive properties of these algorithms, a new learning algorithm is proposed which can improve the performance. Simulations are employed to confirm our theoretical results.

Collaborative filtering is regarded as one of the most promising recommendation algorithms. The item-based approaches for collaborative filtering identify the similarity between two items by comparing users' ratings on them. In these approaches, ratings produced at different times are weighted equally. That is to say, changes in user purchase interest are not taken into consideration. For example, an item that was rated recently by a user should have a bigger impact on the prediction of future user behaviour than an item that was rated a long time ago. In this paper, we present a novel algorithm to compute the time weights for different items in a manner that will assign a decreasing weight to old data. More specifically, the users' purchase habits vary. Even the same user has quite different attitudes towards different items. Our proposed algorithm uses clustering to discriminate between different kinds of items. To each item cluster, we trace each user's purchase interest change and introduce a personalized decay factor according to the user own purchase behaviour. Empirical studies have shown that our new algorithm substantially improves the precision of item-based collaborative filtering without introducing higher order computational complexity.

A video fingerprint is a kind of digital watermark used in digital video for tracking pirate copies in a multi-user environment. Different users receive the same video with different watermarks designed for uniquely identifying designated users. As a value-added business service, fingerprinting is independent from video compression. In current fingerprinting schemes, a compressed video has to be decoded once for every user in order to add on individual fingerprints. Then the video is encoded again before the dispatch. However, the multiple decode/re-encode operations can result in poor system performance. In this paper, we propose a new integrated fingerprint algorithm, which can be applied directly to the compressed video without decoding/reencoding. Our experiments show that the performance of fingerprinting improved with no compromise to the robustness.

User requirements of multimedia authentication are various. In some cases, the user requires an authentication system to monitor a set of specific areas with respective sensitivity while neglecting other modification. Most current existing fragile watermarking schemes are mixed systems, which can not satisfy accurate user requirements. Therefore, in this paper we designed a sensor-based multimedia authentication architecture. This system consists of sensor combinations and a fuzzy response logic system. A sensor is designed to strictly respond to given area tampering of a certain type. With this scheme, any complicated authentication requirement can be satisfied, and many problems such as error tolerant tamper method detection will be easily resolved. We also provided experiments to demonstrate the implementation of the sensor-based system

A potential security problem in frequency domain video encryption is that some trivial information such as the distribution of DCT coefficients may leak out secret. To illuminate this problem, we performed a successful attack on video using the distribution information of DCT coefficients. Then, according to the weak points discovered, a novel video encryption algorithm, working on run-length coded data, is proposed. It has amended identified security problems, while preserving high efficiency and the adaptability to cooperate with compression schemes.

Streaming video application requires high security as well as high computational performance. In video encryption, traditional selective algorithms have been used to partially encrypt the relatively important data in order to satisfy the streaming performance requirement. Most video selective encryption algorithms are inherited from still image encryption algorithms, the encryption on motion vector data is not considered. The assumption is that motion vector data are not as important as pixel image data. Unfortunately, in some cases, motion vector itself may be sufficient enough to leak out useful video information. Normally motion vector data consume over half of the whole video stream bandwidth, neglecting their security may be unwise. In this paper, we target this security problem and illustrate attacks at two different levels that can restore useful video information using motion vectors only. Further, an information analysis is made and a motion vector information model is built. Based on this model, we describe a new motion vector encryption algorithm called MVEA. We show the experimental results of MVEA. The security strength and performance of the algorithm are also evaluated.

Pattern discovery in temporal event sequences is of great importance in many application domains, such as telecommunication network fault analysis. In reality, not every type of event has an accurate timestamp. Some of them, defined as inaccurate events may only have an interval as possible time of occurrence. The existence of inaccurate events may cause uncertainty in event ordering. The traditional support model cannot deal with this uncertainty, which would cause some interesting patterns to be missing. A new concept, precise support, is introduced to evaluate the probability of a pattern contained in a sequence. Based on this new metric, we define the uncertainty model and present an algorithm to discover interesting patterns in the sequence database that has one type of inaccurate event. In our model, the number of types of inaccurate events can be extended to k readily, however, at a cost of increasing computational complexity.

This paper proposes a novel application of fuzzy logic to web data mining for two basic problems of a website: popularity and satisfaction. Popularity means that people will visit the website while satisfaction refers to the usefulness of the site. We will illustrate that the popularity of a website is a fuzzy logic problem. It is an important characteristic of a website in order to survive in Internet commerce. The satisfaction of a website is also a fuzzy logic problem that represents the degree of success in the application of information technology to the business. We propose a framework of fuzzy logic for the representation of these two problems based on web data mining techniques to fuzzify the attributes of a website.