Ensuring web accessibility is crucial for advancing social welfare, justice, and equality in digital spaces, yet the vast majority of website user interfaces remain non-compliant, due in part to the resource-intensive and unscalable nature of current auditing practices. While WCAG-EM offers a structured methodology for site-wise conformance evaluation, it involves great human efforts and lacks practical support for execution at scale. In this work, we present an auditing framework, AAA, which operationalizes WCAG-EM through a human-AI partnership model. AAA is anchored by two key innovations: GRASP, a graph-based multimodal sampling method that ensures representative page coverage via learned embeddings of visual, textual, and relational cues; and MaC, a multimodal large language model-based copilot strategy that supports auditors through cross-modal reasoning and intelligent assistance in high-effort tasks. Together, these components enable scalable, end-to-end web accessibility auditing, empowering human auditors with AI-enhanced assistance for real-world impact. We further contribute four novel datasets designed for benchmarking core stages of the audit pipeline. Extensive experiments demonstrate the effectiveness of our methods, providing insights that small-scale language models can serve as capable experts when fine-tuned.

With the increasing scale and complexity of graph data, node attributes are also becoming richer and more complex, particularly in the form of informative text. Classic GNNs equipped with shallow attribute encoders are no longer sufficient to handle such data independently, making model collaboration across heterogeneous architectures an inevitable trend. Recently, the integration of Large Language Models (LLMs) and GNNs has attracted significant attention, yet the inherent disparity between these models remains a key challenge. Promising solutions have considered fine-tuning Small Language Models (SLMs) to bridge the gap between GNNs and frozen LLMs. However, this introduces another problem: these heterogeneous models bring complementary knowledge, but how to effectively integrate them and allow mutual refinement becomes a significant research gap. To address these challenges, we introduce COLA, a collaborative large–small model framework that enables seamless cooperation among semantic LLMs, task-specific fine-tuned SLMs, and structure-aware GNNs. COLA features a unique Consensus–Complement Coordination Mechanism (C3M), wherein its Mixture-of-Coordinators (MoC) architecturally aligns the LLM and SLM. Built upon this, a flexible graph-knowledge infusion strategy encourages the joint alignment and graph knowledge learning of textual representations. Extensive evaluations across nine diverse datasets show that COLA consistently achieves state-of-the-art performance, validating the effectiveness and generality of our collaborative paradigm.

Junhao Jia, Hanwen Zheng, Yueyi Wu, Huangwei Chen, Haishuai Wang, Jiajun Bu, Lei Wu. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Fine-grained hand interaction with Inertial Measurement Unit (IMU) and machine learning offers a low-cost and effective solution. However, the robustness and generalizability of machine learning models are highly dataset-dependent. Existing datasets for interaction design are typically constructed through extensive real user data collection, which limits interaction diversity and personalization. To address these challenges, we propose Rob2HanD, a novel data-generation tool which utilizes large language models (LLMs) to regulate the motion processes of the robotic arm and rapidly constructs IMU datasets. Rob2HanD demonstrates the capability to generate large and usable IMU interaction datasets under few-shot or zero-shot conditions, thereby enhancing the potential for diverse and personalized fine-grained hand interactions. Using a real human dataset, we evaluate machine learning models trained on Rob2HanD-generated data and validate the usability of Rob2HanD. In real-world applications, models trained on Rob2HanD-generated datasets demonstrate strong performance across a variety of customized interaction tasks.

Graph data mining techniques in real-world scenarios often encounter significant computational challenges, especially when the graph contains a large number of nodes and edges. Recently, Graph Condensation (GC) has emerged to offer data-centric solutions that address the challenge of graph volume, enhancing the efficiency of graph data mining and storage. Current methods in GC rely solely on optimizing heuristic metrics of one-way maintenance of key information in the condensed graph. However, the maintenance of key information may be insufficient due to the significant condensation ratio, yet these methods lack an effective mechanism to verify and compensate for that. To this end, this paper aims to enhance the maintenance of key information through a reconstruction based alignment mechanism. More specifically, inspired by the Kolmogorov Complexity, we revisit the theoretical foundations of GC and propose a way-back mechanism that introduces a feedback loop of learning to reconstruct the original graph from the condensed graph, with the objective of key information align ment, namely the WbGC. We modify several GC methods with our mechanism, and the experiments show that our approach provides an enhanced solution for GC. Code is available at https://anonymous.4open.science/r/WbGC-1842.

With the proliferation of mobile sensing technology, substantial time series data have been generated and accumulated in various distributed domains, providing the basis for practical applications. Federated Learning (FL) has emerged as an essential framework for machine learning on decentralized data, especially with its potential for privacy-preserving. However, deployed with distributed and various edge devices, existing FL frameworks struggle to address the statistical heterogeneity of spatial feature distribution shifts from heterogeneous sensors. Due to the complex temporal dynamics of real-world time series data, temporal feature heterogeneity may result in suboptimal model adaptation performance. To this end, we propose a novelFederated learning approach withOrthogonal regularization forSpatial-Temporal heterogeneity (FedOST) on time series classification. It is featured in three aspects: (1) In the local training phase, we utilize an orthogonal projection to disentangle and align the shared and personalized features, as well as complementary information from different views of the time series data to formulate a robust multi-view training. (2) In the global aggregation phase, we adopt trainable global prototypes to improve feature space separation through orthogonal constraint, to serve as refined global knowledge in the local training. (3) In the testing phase, we leverage an uncertainty-aware test-time adaptation scheme to tackle the temporal feature shifts of unlabeled test data. We conduct extensive evaluations on real-world datasets, where FedOST outperforms existing state-of-the-art baselines with significant advantages.

Graph anomaly detection (GAD) refers to identifying abnormal graph nodes or edges that heavily deviate from normal observations. Existing approaches inevitably suffer from the influence of imbalanced data and privacy protection. This shortcoming poses challenges in optimizing node embeddings and detecting multitype anomalies simultaneously, resulting in decreased accuracy of existing GAD models. To address this shortcoming, we introduce a new federated learning model for graph anomaly detection (FedGAD). FedGAD enables collaborative unsupervised learning among decentralized data centers without requiring direct access to the distributed subgraphs. Specifically, FedGAD masks and reconstructs the neighborhood features to enhance the knowledge of node representations. Considering the data diversity across distributed clients, we also design a cross-clients' node representation module that enables nodes to reconstruct neighbors by leveraging information from other clients. Furthermore, we use a multiscale contrastive learning function, which includes both structure-level and contextual-level learning functions, to detect graph anomalies in the condition that subgraphs located at different clients show imbalanced data distributions. Experimental results on seven benchmark datasets demonstrate the superior performance of FedGAD compared with baseline methods, verifying its capability of improving GAD performance.

DIS ’25, Funchal, Portugal

Neural Representations for Videos (NeRV) has emerged as a promising implicit neural representation (INR) approach for video analysis, which represents videos as neural networks with frame indexes as inputs. However, NeRV-based methods are time-consuming when adapting to a large number of diverse videos, as each video requires a separate NeRV model to be trained from scratch. In addition, NeRV-based methods spatially require generating a high-dimension signal (i.e., an entire image) from the input of a low-dimension timestamp, and a video typically consists of tens of frames temporally that have a minor change between adjacent frames. To improve the efficiency of video representation, we propose Meta Neural Representations for Videos, named MetaNeRV, a novel framework for fast NeRV representation for unseen videos. MetaNeRV leverages a meta-learning framework to learn an optimal parameter initialization, which serves as a good starting point for adapting to new videos. To address the unique spatial and temporal characteristics of video modality, we further introduce spatial-temporal guidance to improve the representation capabilities of MetaNeRV. Specifically, the spatial guidance with a multi-resolution loss aims to capture the information from different resolution stages, and the temporal guidance with an effective progressive learning strategy could gradually refine the number of fitted frames during the meta-learning process. Extensive experiments conducted on multiple datasets demonstrate the superiority of MetaNeRV for video representations and video compression.

Recently, large language models (LLMs) and multimodal large language models (MLLMs) have demonstrated promising results on document visual question answering (VQA) task, particularly after training on document instruction datasets. An effective evaluation method for document instruction data is crucial in constructing instruction data with high efficacy, which, in turn, facilitates the training of LLMs and MLLMs for document VQA. However, most existing evaluation methods for instruction data are limited to the textual content of the instructions themselves, thereby hindering the effective assessment of document instruction datasets and constraining their construction. In this paper, we propose ProcTag, a data-oriented method that assesses the efficacy of document instruction data. ProcTag innovatively performs tagging on the execution process of instructions rather than the instruction text itself. By leveraging the diversity and complexity of these tags to assess the efficacy of the given dataset, ProcTag enables selective sampling or filtering of document instructions. Furthermore, DocLayPrompt, a novel semi-structured layout-aware document prompting strategy, is proposed for effectively representing documents. Experiments demonstrate that sampling existing open-sourced and generated document VQA/instruction datasets with ProcTag significantly outperforms current methods for evaluating instruction data. Impressively, with ProcTag-based sampling in the generated document datasets, only 30.5 percent of the document instructions are required to achieve 100 percent efficacy compared to the complete dataset.

Graph Neural Networks (GNNs) have shown remarkable performance in modeling graph data. However, Labeling graph data typically relies on unreliable information, leading to noisy node labels. Existing approaches for GNNs under Label Noise (GLN) employ supervision signals beyond noisy labels for robust learning. While empirically effective, they tend to over-reliance on supervision signals built upon external assumptions, leading to restricted applicability. In this work, we shift the focus to exploring how to extract useful information and learn from the graph itself, thus achieving robust graph learning. From an information theory perspective, we theoretically and empirically demonstrate that the graph itself contains reliable information for graph learning under label noise. Based on these insights, we propose the Topological Feature Reconstruction (TFR) method. Specifically, TFR leverages the fact that the pattern of clean labels can more accurately reconstruct graph features through topology, while noisy labels cannot. TFR is a simple and theoretically guaranteed model for robust graph learning under label noise. We conduct extensive experiments across datasets with varying properties. The results demonstrate the robustness and broad applicability of our proposed TFR compared to state-of-the-art baselines. Codes are available at https://github.com/eaglelab-zju/TFR.

The exploration of bundle recommendation has garnered significant attention for its potential to enhance user experience and augment business sales. Previous research in this domain has primarily focused on modeling user-item and user-bundle interactions, utilizing multi-view collaboration to bolster the accuracy of bundle recommendations. Nevertheless, existing methodologies exhibit limitations, notably in the inadequate modeling of multi-view information and the absence of multi-grained details. Consequently, addressing the intricate correlation among users, items, and bundles necessitates a sophisticated approach capable of capturing both global and local nuances. We present a novel framework named Large Language Model-based Multi-View and Multi-Grained Learning for Bundle Recommendation (LLMCBR). We introduce an LLM-based semantic refinement module to summarize and encode bundle-level knowledge. To bridge the gap between semantic representation and collaborative signals, we design an adaptation strategy. Furthermore, LLMCBR leverages multi-view and multi-granular modeling to unify collaborative signals. Specifically, LLMCBR integrates item preferences within both bundle-view and item-view, thereby augmenting the comprehensiveness of multi-view data. Following this integration, each view undergoes stratification into multiple granularities to facilitate the acquisition of multi-grained details. We introduce a multiple contrastive instance mechanism to regulate the influence of different granularities and views. This mechanism empowers the model to comprehend complex consumer behaviors across various dimensions. LLMCBR is extensively evaluated over three real-world datasets, and the experimental results demonstrate its superiority.

Recent advances in medical AI have shown a clear trend towards large models in healthcare. However, developing large models for multi-modal medical diagnosis remains challenging due to a lack of sufficient modal-complete medical data. Most existing multi-modal diagnostic models are relatively small and struggle with limited feature extraction capabilities. To bridge this gap, we propose AdaCoMed, an adaptive collaborative-learning framework that synergistically integrates the off-the-shelf medical single-modal large models with multi-modal small models. Our framework first employs a mixture-of-modality-experts (MoME) architecture to combine features extracted from multiple single-modal medical large models, and then introduces a novel adaptive co-learning mechanism to collaborate with a multi-modal small model. This co-learning mechanism, guided by an adaptive weighting strategy, dynamically balances the complementary strengths between the MoMEfused large model features and the cross-modal reasoning capabilities of the small model. Extensive experiments on two representative multi-modal medical datasets (MIMICIV-MM and MMIST ccRCC) across six modalities and four diagnostic tasks demonstrate consistent improvements over state-of-the-art baselines, making it a promising solution for real-world medical diagnosis applications. The code is available at https://github.com/Zoew420/AdaCoMed.

Graphical user interface (GUI) has become integral to modern society, making it crucial to be understood for human-centric systems. However, unlike natural images or documents, GUIs comprise artificially designed graphical elements arranged to convey specific semantic meanings. Current multi-modal large language models (MLLMs) already proficient in processing graphical and textual components suffer from hurdles in GUI understanding due to the lack of explicit spatial structure modeling. Moreover, obtaining high-quality spatial structure data is challenging due to privacy issues and noisy environments. To address these challenges, we present MP-GUI, a specially designed MLLM for GUI understanding. MP-GUI features three precisely specialized perceivers to extract graphical, textual, and spatial modalities from the screen as GUI-tailored visual clues, with spatial structure refinement strategy and adaptively combined via a fusion gate to meet the specific preferences of different GUI understanding tasks. To cope with the scarcity of training data, we also introduce a pipeline for automatically data collecting. Extensive experiments demonstrate that MP-GUI achieves impressive results on various GUI understanding tasks with limited data. Our codes and datasets are publicly available at https://github.com/BigTaige/MP-GUI.

Tianyuan Huang, Zepeng Zhu, Hangdi Xing, Zirui Shao, Zhi Yu, Chaoxiong Yang, Jiaxian He, Xiaozhong Liu, Jiajun Bu. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Multimodal large language models (MLLMs) have shown impressive capabilities in document understanding, a rapidly growing research area with significant industrial demand.As a multimodal task, document understanding requires models to possess both perceptual and cognitive abilities.However, due to different types of annotation noise in training, current MLLMs often face conflicts between perception and cognition.Taking a document VQA task (cognition) as an example, an MLLM might generate answers that do not match the corresponding visual content identified by its OCR (perception).This conflict suggests that the MLLM might struggle to establish an intrinsic connection between the information it "sees" and what it "understands".Such conflicts challenge the intuitive notion that cognition is consistent with perception, hindering the performance and explainability of MLLMs.In this paper, we define the conflicts between cognition and perception as Cognition and Perception (C&P) knowledge conflicts, a form of multimodal knowledge conflicts, and systematically assess them with a focus on document understanding.Our analysis reveals that even GPT-4o, a leading MLLM, achieves only 75.26% C&P consistency.To mitigate the C&P knowledge conflicts, we propose a novel method called Multimodal Knowledge Consistency Fine-tuning.Our method reduces C&P knowledge conflicts across all tested MLLMs and enhances their performance in both cognitive and perceptual tasks.

Detecting out-of-distribution (OOD) samples poses a significant safety challenge when deploying models in open-world scenarios. Advanced works assume that OOD and in-distributional (ID) samples exhibit a distribution discrepancy, showing an encouraging direction in estimating the uncertainty with embedding features or predicting outputs. In this work, we propose a data structure-aware approach to mitigate the sensitivity of distances to the “curse of dimensionality”, where high-dimensional features are mapped to the manifold of ID samples, leveraging the well-known manifold assumption. Specifically, we present a novel distance termed as tangent distance, which tackles the issue of generalizing the meaningfulness of distances on testing samples to detect OOD inputs. Extensive experiments show that the tangent distance performs competitively with other post hoc OOD detection baselines on common and large-scale benchmarks.

Time series analysis is crucial across various fields like energy, environment, transportation, finance and health. Deep learning has significantly advanced this field, particularly, the Time Series Foundation Model (TSFM) excels in multiple domains due to extensive pre-training. In this work, we focus on TSFM’s challenges in medical practice: limited computing resources and medical data privacy. TSFM variants include fine-tuned models and those pre-trained for rapid deployment on diverse data. There may not be enough computing resources to train physiological signals locally in hospitals, and generalized TSFM is still inferior to task-specific methods on private, imbalanced local data. To address this, we propose PhysioPFM, a framework for efficiently personalizing TSFM. Our approach involves low-rank pre-training on public datasets, generator training by trained LoRA weights, and efficient weight generation via local data. Experimental results demonstrate that integrating generated models with TSFM enhances performance, and transferability, and reduces the need for additional sensitive data training.

Unsupervised Anomaly Detection (UAD) plays a crucial role in identifying abnormal patterns within data without labeled examples, holding significant practical implications across various domains. Although the individual contributions of representation learning and clustering to anomaly detection are well-established, their interdependencies remain under-explored due to the absence of a unified theoretical framework. Consequently, their collective potential to enhance anomaly detection performance remains largely untapped. To bridge this gap, in this paper, we propose a novel probabilistic mixture model for anomaly detection to establish a theoretical connection among representation learning, clustering, and anomaly detection. By maximizing a novel anomaly-aware data likelihood, representation learning and clustering can effectively reduce the adverse impact of anomalous data and collaboratively benefit anomaly detection. Meanwhile, a theoretically substantiated anomaly score is naturally derived from this framework. Lastly, drawing inspiration from gravitational analysis in physics, we have devised an improved anomaly score that more effectively harnesses the combined power of representation learning and clustering. Extensive experiments, involving 17 baseline methods across 30 diverse datasets, validate the effectiveness and generalization capability of the proposed method, surpassing state-of-the-art methods.

Healthcare data frequently contain a substantial proportion of missing values, necessitating effective time series imputation to support downstream disease diagnosis tasks. However, existing imputation methods focus on discrete data points and are unable to effectively model sparse data, resulting in particularly poor performance for imputing substantial missing values. In this paper, we propose a novel approach, ImputeINR, for time series imputation by employing implicit neural representations (INR) to learn continuous functions for time series. ImputeINR leverages the merits of INR in that the continuous functions are not coupled to sampling frequency and have infinite sampling frequency, allowing ImputeINR to generate fine-grained imputations even on extremely sparse observed values. Extensive experiments conducted on eight datasets with five ratios of masked values show the superior imputation performance of ImputeINR, especially for high missing ratios in time series data. We also validate that applying ImputeINR to impute missing values in healthcare data enhances the performance of downstream disease diagnosis tasks.

Integrating multi-omics datasets through data-driven analysis offers a comprehensive understanding of the complex biological processes underlying various diseases, particularly cancer. Graph Neural Networks (GNNs) have recently demonstrated remarkable ability to exploit relational structures in biological data, enabling advances in multi-omics integration for cancer subtype classification. Existing approaches often neglect the intricate coupling between heterogeneous omics, limiting their capacity to resolve subtle cancer subtype heterogeneity critical for precision oncology. To address these limitations, we propose a framework named Graph Transformer for Multi-omics Cancer Subtype Classification (GTMancer). This framework builds upon the GNN optimization problem and extends its application to complex multi-omics data. Specifically, our method leverages contrastive learning to embed multi-omics data into a unified semantic space. We unroll the multiplex graph optimization problem in that unified space and introduce dual sets of attention coefficients to capture structural graph priors both within and among multi-omics data. This approach enables global omics information to guide the refining of the representations of individual omics. Empirical experiments on seven real-world cancer datasets demonstrate that GTMancer outperforms existing state-of-the-art algorithms.

The detection and segmentation of thyroid nodules in ultrasound images pose considerable challenges due to significant variability in the shape and size of nodules, as well as the influence of artifacts and background noise. To mitigate these challenges, we introduce Bidirectional Attention YOLO (BDA-YOLO), an advanced YOLOv11 model that integrates two innovative feature fusion modules: the Dynamic Self-Attention Module (DSAM) and the Bidirectional Fusion Module (BDFM). The DSAM processes input feature maps through convolution and flattening prior to the application of self-attention mechanisms, facilitating a more precise integration of multi-level feature information. Meanwhile, BDFM enhances feature interactions across different layers, thereby improving the network’s capacity to capture intricate characteristics of nodules. Experimental evaluations conducted on the TN3K and DDTI datasets indicate that our proposed method exceeds existing state-of-the-art models, including RT-DETRv1 and RT-DETRv2, achieving superior Box Average Precision (AP) and Mask AP metrics, alongside notable enhancements in precision and recall. These findings highlight the efficacy of our feature fusion approach in improving both the detection and segmentation outcomes for the localization of thyroid nodules on ultrasound imaging.

Accurate segmentation of brain tumors, especially gliomas, is essential for medical diagnosis, treatment planning, and monitoring of disease progression. Traditional manual segmentation methods are time-consuming and prone to variability, while deep learning-based approaches, such as convolutional neural networks (CNNs), have significantly improved performance. However, CNNs often struggle to capture global context due to their limited receptive fields. To address these challenges, we propose SwinFusion-XL, a novel segmentation model based on the Swin UNETR architecture that integrates local and global features for enhanced brain tumor segmentation. SwinFusion-XL leverages a Superficial Mamba Fusion Module (SMFM) for the effective integration of multi-scale features, combining both channel and spatial attention mechanisms to enhance feature representations. Additionally, a Cross xLSTM Module (CxLSTM) is introduced to model cross-feature dependencies and capture long-range spatial, channel, and volumetric relationships. Extensive experiments on the BraTS2020 and BraTS2021 brain tumor datasets demonstrate that our model consistently outperforms state-of-the-art methods, achieving significant improvements in segmentation performance.

Time series anomaly detection aims to identify unusual patterns in data or deviations from systems' expected behavior. The reconstruction-based methods are the mainstream in this task, which learn point-wise representation via unsupervised learning. However, the unlabeled anomaly points in training data may cause these reconstruction-based methods to learn and reconstruct anomalous data, resulting in the challenge of capturing normal patterns. In this paper, we propose a time series anomaly detection method based on implicit neural representation (INR) reconstruction, named TSINR, to address this challenge. Due to the property of spectral bias, TSINR enables prioritizing low-frequency signals and exhibiting poorer performance on high-frequency abnormal data. Specifically, we adopt INR to parameterize time series data as a continuous function and employ a transformer-based architecture to predict the INR of given data. As a result, the proposed TSINR method achieves the advantage of capturing the temporal continuity and thus is more sensitive to discontinuous anomaly data. In addition, we further design a novel form of INR continuous function to learn inter- and intra-channel information, and leverage a pre-trained large language model to amplify the intense fluctuations in anomalies. Extensive experiments demonstrate that TSINR achieves superior overall performance on both univariate and multivariate time series anomaly detection benchmarks compared to other state-of-the-art reconstruction-based methods. Our codes are available here.

Multi-label node classification is an important yet under-explored domain in graph mining as many real-world nodes belong to multiple categories rather than just a single one. Although a few efforts have been made by utilizing Graph Convolution Networks (GCNs) to learn node representations and model correlations between multiple labels in the embedding space, they still suffer from the ambiguous feature and ambiguous topology induced by multiple labels, which reduces the credibility of the messages delivered in graphs and overlooks the label correlations on graph data. Therefore, it is crucial to reduce the ambiguity and empower the GCNs for accurate classification. However, this is quite challenging due to the requirement of retaining the distinctiveness of each label while fully harnessing the correlation between labels simultaneously. To address these issues, in this paper, we propose a Correlation-aware Graph Convolutional Network (CorGCN) for multi-label node classification. By introducing a novel Correlation-Aware Graph Decomposition module, CorGCN can learn a graph that contains rich label-correlated information for each label. It then employs a Correlation-Enhanced Graph Convolution to model the relationships between labels during message passing to further bolster the classification process. Extensive experiments on five datasets demonstrate the effectiveness of our proposed CorGCN.

Dual-energy X-ray absorptiometry (DXA) enables accurate bone mineral density but requires specialized equipment and protocols. X-ray-based BMD screening offers opportunistic early detection, though prior methods struggle with X-ray intensity variations and demand large datasets. We introduce a cross-modal knowledge distillation BMD prediction framework (CMKD-BMD) fusing X-ray/CT data to enhance X-ray-only BMD prediction. Each single-modal student network employs a multi-scale visual extractor for hierarchical features, an unsupervised graph-based structural learner for anatomical relationships, and an adaptive fusion module to generate a unified representation. The teacher network integrates single-modal representations from students and transfers multimodal knowledge from the teacher to students. Our model outperforms existing methods on both the collected dataset (comprising 1620 X-rays and 280 CT cases) and the publicly available VerSe2019 dataset, demonstrating superior BMD estimation performance. Furthermore, we developed OrthoSim, an orthopedic surgical simulation platform with CMKD-BMD, which has shown promising clinical effectiveness in trial evaluations. Our code is available at https://github.com/KeyueShi/CMKD-BMD.

Graphical User Interface (GUI) agents possess significant commercial and social value, and GUI agents powered by advanced multimodal large language models (MLLMs) have demonstrated remarkable potential. Currently, existing GUI agents usually utilize sequential episodes of multi-step operations across pages as the prior GUI knowledge, which fails to capture the complex transition relationship between pages, making it challenging for the agents to deeply perceive the GUI environment and generalize to new scenarios. Therefore, we design an automated pipeline to transform the sequential episodes into page graphs, which explicitly model the graph structure of the pages that are naturally connected by actions. To fully utilize the page graphs, we further introduce Retrieval-Augmented Generation (RAG) technology to effectively retrieve reliable perception guidelines of GUI from them, and a tailored multi-agent framework PG-Agent with task decomposition strategy is proposed to be injected with the guidelines so that it can generalize to unseen scenarios. Extensive experiments on various benchmarks demonstrate the effectiveness of PG-Agent, even with limited episodes for page graph construction. Our codes will be publicly available at https://github.com/chenwz-123/PG-Agent.

In real-world cooking scenarios, users often need to create personalized recipes based on limited ingredients, dietary goals, and various restrictions, such as the availability of equipment, flavor preferences, and health conditions. Existing recipe generation methods lack the flexibility and adaptability to meet these individual needs. We present LetMeCook, an end-to-end interactive system for personalized recipe generation that leverages multimodal perception, hybrid retrieval, and content generation. Given a photo of the user's refrigerator and a dietary profile, LetMeCook detects available ingredients, retrieves relevant recipe candidates, and generatively refines them based on user requirements, such as ingredient substitution and flavor adjustment. The system provides both textual and visual previews of the adapted recipes, offering a highly interactive and user-centric experience.

In recent years, graph-based multi-view learning has received widespread attention for its ability to utilize data dependencies to capture more comprehensive information from ubiquitous multi-view data. However, with the increase in data size and complexity, Euclidean space struggles to capture the hierarchical and exponentially expanding relationships of multi-view data in limited dimensions, leading to embedding distortion and insufficient cross-view alignment and interaction. To this end, we propose a hyperbolic multi-view heat diffusion method. Firstly, we utilize the negative curvature advantage of the hyperbolic space to construct a graph representation for each view separately, so that each view can still retain its hierarchical structure in relatively low dimensions. Then we construct a graph heat diffusion process on hyperbolic manifolds to ensure that each view is locally smoothed and globally aggregated while achieving semantic consistency through virtual views. We show that the method can be interpreted as a Riemannian gradient descent process for collaborative learning on hyperbolic manifolds, which not only effectively fuses multimodal information, but also significantly enhances the interaction and unified representation among different views. Experimental results show that the proposed framework achieves excellent performance in a variety of multi-view scenarios.

The past decade has witnessed rapid advancements in cross-modal retrieval, with significant progress made in accurately measuring the similarity between cross-modal pairs. However, the persistent hubness problem, a phenomenon where a small number of targets frequently appear as nearest neighbors to numerous queries, continues to hinder the precision of similarity measurements. Despite several proposed methods to reduce hubness, their underlying mechanisms remain poorly understood. To bridge this gap, we analyze the widely-adopted Inverted Softmax approach and demonstrate its effectiveness in balancing target probabilities during retrieval. Building on these insights, we propose a probability-balancing framework for more effective hubness reduction. We contend that balancing target probabilities alone is inadequate and, therefore, extend the framework to balance both query and target probabilities by introducing Sinkhorn Normalization (SN). Notably, we extend SN to scenarios where the true query distribution is unknown, showing that current methods, which rely solely on a query bank to estimate target hubness, produce suboptimal results due to a significant distributional gap between the query bank and targets. To mitigate this issue, we introduce Dual Bank Sinkhorn Normalization (DBSN), incorporating a corresponding target bank alongside the query bank to narrow this distributional gap. Our comprehensive evaluation across various cross-modal retrieval tasks, including image-text retrieval, video-text retrieval, and audio-text retrieval, demonstrates consistent performance improvements, validating the effectiveness of both SN and DBSN. All codes are publicly available at https://github.com/ppanzx/DBSN.

Osteoporosis has emerged as a significant global public health challenge, affecting more than 200 million people worldwide. Precise screening through bone mineral density (BMD) assessment is essential for timely intervention. Although dual-energy X-ray absorptiometry (DXA) serves as the gold standard for BMD evaluation, its widespread application is hampered by high costs and limited accessibility. In recent years, deep learning has demonstrated great potential in osteoporosis classification and predicting BMD from X-ray and CT scans. However, existing public datasets have notable limitations, particularly the absence of paired multimodal imaging datasets for cross-modal modeling and the lack of accurate BMD annotations required for osteoporosis-specific research and clinical model optimization. To address these issues, we introduce the Lumbar Multimodal Osteoporosis Screening dataset (LUMOS), the first multimodal dataset specifically designed for lumbar osteoporosis screening. LUMOS integrates clinical data from 803 patients, including 1,620 anteroposterior/lateral lumbar X-rays with BMD values and T-scores, comprehensive demographic information, and 280 lumbar CT scans. The advent of LUMOS is expected to propel forward research on automated osteoporosis classification, BMD prediction, and other related tasks. Its standardized and multimodal nature fills critical gaps in lumbar osteoporosis data, providing high-quality data support for the development and validation of medical AI algorithms in the early detection of osteoporosis. The dataset is available at https://keyueshi.github.io/LUMOS/.

Graph Neural Networks (GNNs) have achieved great success but are often considered to be challenged by varying levels of homophily in graphs. Recent empirical studies have surprisingly shown that homophilic GNNs can perform well across datasets of different homophily levels with proper hyperparameter tuning, but the underlying theory and effective architectures remain unclear. To advance GNN universality across varying homophily, we theoretically revisit GNN message passing and uncover a novel \textit{smoothness-generalization dilemma}, where increasing hops inevitably enhances smoothness at the cost of generalization. This dilemma hinders learning in high-order homophilic neighborhoods and all heterophilic ones, where generalization is critical due to complex neighborhood class distributions that are sensitive to shifts induced by noise or sparsity. To address this, we introduce the Inceptive Graph Neural Network (IGNN) built on three simple yet effective design principles, which alleviate the dilemma by enabling distinct hop-wise generalization alongside improved overall generalization with adaptive smoothness. Benchmarking against 30 baselines demonstrates IGNN's superiority and reveals notable universality in certain homophilic GNN variants. Our code and datasets are available at \href{https://github.com/galogm/IGNN}{https://github.com/galogm/IGNN}.

Graph Neural Networks (GNNs) excel in graph mining tasks thanks to their message-passing mechanism, which aligns with the homophily assumption. However, connected nodes can also exhibit inconsistent behaviors, termed heterophilic patterns, sparking interest in heterophilic GNNs (HTGNNs). Although the message-passing mechanism seems unsuitable for heterophilic graphs owing to the propagation of dissimilar messages, it is still popular in HTGNNs and consistently achieves notable success. Some efforts have investigated such an interesting phenomenon, but are limited in the data perspective. The model-perspective understanding remains largely unexplored, which is conducive to guiding the designs of HTGNNs. To fill this gap, we build the connection between node discriminability and the compatibility matrix (CM). We reveal that the effectiveness of the message passing in HTGNNs may be credited to increasing the proposed Compatibility Matrix Discriminability (CMD). However, the issues of sparsity and noise pose great challenges to leveraging CM. Thus, we propose CMGNN, a novel approach to alleviate these issues while enhancing the CM and node embeddings explicitly. A thorough evaluation involving 13 datasets and comparison against 20 well-established baselines highlights the superiority of CMGNN.

Individuals with upper-body motor impairments, but having full lower-body functionality, can use their feet to interact with mobile devices, such as smartphones and tablets. However, they face significant challenges controlling mobile devices and entering text via touchscreen with their feet. Moreover, such interactions are typically limited to specific scenarios, such as sitting on a bed. To address this, we developed a wearable, pressure-sensor-based foot interaction system featuring a novel two-key text entry method. Two use case studies, with a total of 18 participants, demonstrated that our system enables these individuals to control devices and achieve acceptable text entry speeds in diverse scenarios.

The rapid growth of mobile web technologies has revolutionized how people manage daily activities, emphasizing the critical need for accessible mobile user interfaces (UIs) that accommodate users with disabilities and situational impairments. Current AI-driven UI understanding methods show promise but primarily target general UI modeling, neglecting nuanced, user-centric accessibility requirements. To bridge this gap, we first conducted a formative study with 12 visually impaired participants. Our study uncovers selective-accessible issues, a new class of accessibility challenges requiring finer granularity and selective focus on UI components, which existing methods largely overlook. Our findings also reveal that the severity of issues varies across interaction stages, with earlier stages posing a more significant impact. Building on these insights, we propose a comprehensive framework of three accessibility stages: focusability, information, and functionality (FIF), encompassing 12 sub-tasks under 3 overarching tasks. Identifying UI element focusability prediction (UFP) as a pivotal yet underexplored task within FIF, hindered by the absence of dedicated datasets, we introduce a new dataset (NOS) with 117,480 annotated components addressing accessibility issues comprehensively. To further enhance UFP, we introduce Graph-based UI Focusability Prediction (GIFT), a method leveraging graph neural networks to model UFP-targeted UI relationships. User studies validate the dataset's quality, while experiments show GIFT's effectiveness in improving UFP outcomes. Our code and datasets are publicly available to support further web inclusivity advancements at https://github.com/eaglelab-zju/NOS.

With the rapid advancement of deep learning, computer-aided diagnosis and treatment have become crucial in medicine. UNet is a widely used architecture for medical image segmentation, and various methods for improving UNet have been extensively explored. One popular approach is incorporating transformers, though their quadratic computational complexity poses challenges. Recently, State-Space Models (SSMs), exemplified by Mamba, have gained significant attention as a promising alternative due to their linear computational complexity. Another approach, neural memory Ordinary Differential Equations (nmODEs), exhibits similar principles and achieves good results. In this paper, we explore the respective strengths and weaknesses of nmODEs and SSMs and propose a novel architecture, the nmSSM decoder, which combines the advantages of both approaches. This architecture possesses powerful nonlinear representation capabilities while retaining the ability to preserve input and process global information. We construct nmSSM-UNet using the nmSSM decoder and conduct comprehensive experiments on the PH2, ISIC2018, and BU-COCO datasets to validate its effectiveness in medical image segmentation. The results demonstrate the promising application value of nmSSM-UNet. Additionally, we conducted ablation experiments to verify the effectiveness of our proposed improvements on SSMs and nmODEs.

Discrete Ordinary Differential Equations (ODEs) have been employed to develop lightweight deep neural networks in recent years. In this letter, we introduce a novel lightweight UNet variant called the neural memory Self-Supervised State Space Model (nmS4M-UNet) for medical image segmentation, where discrete ODEs serve as the decoder. The proposed nmS4M block has learnable gates and performs multi-head computation to enhance memory updates. Additionally, the nmS4M-UNet incorporates a self-supervised learning branch to improve feature extraction capabilities. The intermediate features are reused as partial input to the decoder, helping to mitigate network overfitting. The nmS4M-UNet reduces the number of parameters by 29.70% compared to the standard UNet. Experimental results on the PH2, ISIC2018, and BU-COCO datasets demonstrate that the proposed nmS4M-UNet achieves performance comparable to state-of-the-art models.

Graph data, prevalent across domains like social networks, biological systems, and recommendation systems, presents significant challenges due to its large scale and complex structure. The advent of Graph Neural Networks (GNNs) has revolutionized graph data mining by effectively capturing node dependencies and neighborhood information. However, the computational complexity of processing large-scale graphs remains a major hurdle, as real-world graphs often consist of millions or even billions of nodes and edges. Efficient techniques like message passing and sampling have helped mitigate this issue, but memory and processing constraints persist. A promising approach to addressing these challenges is learning to reduce the size of large-scale graphs while retaining essential information, thus facilitating faster and more efficient graph data mining tasks, such as graph condensation, reduction, coarsening, summarization, and so on. Despite the differences in terminology, approaches under these topics share the same motivation: to generate smaller yet informative graphs that can replace the original large-scale datasets. In this article, we unify these approaches under the concept of Graph Scaling (GS), highlighting the shared motivation across multiple topics. Alongside this definition, to clarify the question of what principles should be followed when scaling a graph and how a scaled graph was formulated, we propose a taxonomy to methodically categorize and understand existing methods. Moreover, by organizing the dataset and evaluation metrics, we aim to provide a more comprehensive understanding of the GS methods from a practical perspective. Moving forward, we delve into the limitations and challenges of GS methods, identifying the shortcomings and potential in the literature. Finally, we conclude this article by outlining future directions and offering concise guidelines to inspire future research in this field. A full paper list and online resources about GS are available at https://github.com/Frostland12138/Awesome-Graph-Scaling .

Detecting out-of-distribution (OOD) samples poses a significant safety challenge when deploying models in open-world scenarios. Advanced works assume that OOD and in-distributional (ID) samples exhibit a distribution discrepancy, showing an encouraging direction in estimating the uncertainty with embedding features or predicting outputs. Besides incorporating auxiliary outlier as decision boundary, quantifying a “meaningful distance” in embedding space as uncertainty measurement is a promising strategy. However, these distances-based approaches overlook the data structure and heavily rely on the high-dimension features learned by deep neural networks, causing unreliable distances due to the “curse of dimensionality”. In this work, we propose a data structure-aware approach to mitigate the sensitivity of distances to the “curse of dimensionality”, where high-dimensional features are mapped to the manifold of ID samples, leveraging the well-known manifold assumption. Specifically, we present a novel distance termed astangent distance, which tackles the issue of generalizing the meaningfulness of distances on testing samples to detect OOD inputs. Inspired by manifold learning for adversarial examples, where adversarial region probability density is close to the orthogonal direction of the manifold, and both OOD and adversarial samples have common characteristic$-$imperceptible perturbations with shift distribution, we propose that OOD samples are relatively far away from the ID manifold, wheretangent distancedirectly computes the Euclidean distance between samples and the nearest submanifold space$-$instantiated as the linear approximation of local region on the manifold. We provide empirical and theoretical insights to demonstrate the effectiveness of OOD uncertainty measurements on the low-dimensional subspace. Extensive experiments show that thetangent distanceperforms competitively with other post hoc OOD detection baselines on common and large-scale benchmarks, and the theoretical analysis supports our claim that ID samples are likely to reside in high-density regions, explaining the effectiveness of internal connections among ID data.

With the rapid development of mobile network technology and wearable mobile devices, user-scenario interactions generate a large amount of user behavioral data in the form of multivariate event sequences. Due to data isolation, these multi-scenario events need to be jointly trained to achieve better prediction results. However, traditional federated learning methods face significant challenges when handling distributed event sequences. And the effectiveness of existing modeling approaches for event sequences in federated contexts has not been thoroughly explored. To this end, we propose Federated Multivariate Temporal Point Processes (FedMTPP), which enables learning from distributed event sequences within a novel federated learning framework and leverages efficient event modeling technology, MTPP, to forecast future events. Specifically, FedMTPP restores the temporal structure of the original event sequence by rearranging event embeddings and redesigns the autoregressive-based hidden representation computation in traditional MTPP, making it more suitable for federated prediction tasks. Additionally, FedMTPP incorporates advanced encryption techniques to effectively safeguard user privacy and security. Experimental results on both synthetic and real datasets demonstrate that FedMTPP substantially improves the performance of local models and achieves results comparable to state-of-the-art centralized MTPP methods.

Personalized federated learning (PFL) enhances the performance of customized client models through collaborative training without compromising data privacy and ownership. Some previous PFL methods rely on rich prior knowledge about the types of data heterogeneity (such as class imbalance or feature skew), which greatly limits their application ranges. In this paper, we study theUniversal Personalization in Federated Learning (UniPFL), the problem that has no prior knowledge about the types of data heterogeneity. In real-world PFL scenarios, UniPFL is potential because the data distributions of clients are usually heterogeneous and unknown to the server, where quantity imbalance, class imbalance, feature skew, or hybrid heterogeneity are possible contingencies. To address UniPFL, we proposeFedFD, a novel framework with local data augmentation and global concept fusion, which is based on the recent advances inthe foundation generative models(e.g., diffusion models, BLIP-2). On the client side, FedFD utilizes a diffusion model to assist local training by generating augmented data samples, and is then efficiently fine-tuned to be personalized. On the server side, we customize the aggregation strategies based on model similarities to learn both personalized models and diverse feature concepts. Extensive experiments show that FedFD reaches the state-of-the-art on (1) CIFAR-10 and CIFAR-100 for class imbalance; (2) DomainNet and Office-10 for feature skew, and (3) hybrid heterogeneity with both class and feature shifts.

Semantic foggy scene understanding (SFSU) emerges a challenging task under out-of-domain distribution (OD) due to uncertain cognition caused by degraded visibility. With the strong assumption of data centralization, unsupervised domain adaptation (UDA) reduces vulnerability under OD scenario. Whereas, enlarged domain gap and growing privacy concern heavily challenge conventional UDA. Motivated by gap decomposition and data decentralization, we establish a decentralized domain adaptation (DDA) framework calledTranslate thEnAdapt (abbr.TEA) for privacy preservation. Our highlights lie in. (1) Regarding federated hallucination translation, aDisentanglement andContrastive-learning basedGenerativeAdversarialNetwork (abbr.DisCoGAN) is proposed to impose contrastive prior and disentangle latent space in cycle-consistent translation. To yield domain hallucination, client minimizes cross-entropy of local classifier but maximizes entropy of global model to train translator. (2) Regarding source-free regularization adaptation, aPrototypical-knowledge basedRegularizationAdaptation (abbr.ProRA) is presented to align joint distribution in output space. Soft adversarial learning relaxes binary label to rectify inter-domain discrepancy and inner-domain divergence. Structure clustering and entropy minimization drive intra-class features closer and inter-class features apart. Extensive experiments exhibit efficacy of our TEA which achieves 55.26% or 46.25% mIoU in adaptation from GTA5 to Foggy Cityscapes or Foggy Zurich, outperforming other DDA methods for SFSU.

Unsupervised graph anomaly detection aims at identifying rare patterns that deviate from the majority in a graph without the aid of labels, which is important for a variety of real-world applications. Recent advances have utilized graph neural networks (GNNs) to learn effective node representations by aggregating information from neighborhoods. This is motivated by the hypothesis that nodes in the graph tend to exhibit consistent behaviors with their neighborhoods. However, such consistency can be disrupted by graph anomalies in multiple ways. Most existing methods directly employ GNNs to learn representations, disregarding the negative impact of graph anomalies on GNNs, resulting in suboptimal node representations and anomaly detection performance. While a few recent approaches have redesigned GNNs for graph anomaly detection under semi-supervised label guidance, how to address the adverse effects of graph anomalies on GNNs in unsupervised scenarios and learn effective representations for anomaly detection are still underexplored. To bridge this gap, in this article, we propose a simple, yet effective framework for guarding GNNs for unsupervised graph anomaly detection (G3AD). Specifically, G3AD first introduces two auxiliary networks along with correlation constraints to guard the GNNs against inconsistent information encoding. Furthermore, G3AD introduces an adaptive caching (AC) module to guard the GNNs from directly reconstructing the observed graph data that contains anomalies. Extensive experiments demonstrate that our G3AD can outperform 20 state-of-the-art methods on both synthetic and real-world graph anomaly datasets, with flexible generalization ability in different GNN backbones.

Unsupervised Graph Domain Adaptation (UGDA) aims to transfer knowledge from a labelled source graph to an unlabelled target graph in order to address the distribution shifts between graph domains. Previous works have primarily focused on aligning data from the source and target graph in the representation space learned by graph neural networks (GNNs). However, the inherent generalization capability of GNNs has been largely overlooked. Motivated by our empirical analysis, we reevaluate the role of GNNs in graph domain adaptation and uncover the pivotal role of the propagation process in GNNs for adapting to different graph domains. We provide a comprehensive theoretical analysis of UGDA and derive a generalization bound for multi-layer GNNs. By formulating GNN Lipschitz for k-layer GNNs, we show that the target risk bound can be tighter by removing propagation layers in source graph and stacking multiple propagation layers in target graph. Based on the empirical and theoretical analysis mentioned above, we propose a simple yet effective approach called A2GNN for graph domain adaptation. Through extensive experiments on real-world datasets, we demonstrate the effectiveness of our proposed A2GNN framework.

Audio Description (AD) aims to generate narrations of information that is not accessible through unimodal hearing in movies to aid the visually impaired in following film narratives. Current solutions rely heavily on manual work, resulting in high costs and limited scalability. While automatic methods have been introduced, they often yield descriptions that are sparse and omit key details. ddressing these challenges, we propose a novel automated pipeline, the Multi-modal Movie Audio Description (MMAD). MMAD harnesses the capabilities of three key modules as well as the power of Llama2 to augment the depth and breadth of the generated descriptions. Specifically, first, we propose an Audio-aware Feature Enhancing Module to provide the model with multi-modal perception capabilities, enriching the background descriptions with a more comprehensive understanding of the environmental features. Second, we propose an Actor-tracking-aware Story Linking Module to aid in the generation of contextual and character-centric descriptions, thereby enhancing the richness of character depictions. Third, we incorporate a Subtitled Movie Clip Contextual Alignment Module, supplying semantic information about various time periods throughout the movie, which facilitates the consideration of the full movie narrative context when describing silent segments, thereby enhancing the richness of the descriptions. Experiments on widely used datasets convincingly demonstrates that MMAD significantly surpasses existing strong baselines in performance, establishing a new state-of-the-art in the field. Our code will be released at https://github.com/Daria8976/MMAD.

Parsing documents from pixels, such as pictures and scanned PDFs, into hierarchical structures is extensively demanded in the daily routines of data storage, retrieval and understanding.However, previously the research on this topic has been largely hindered since most existing datasets are small-scale, or contain documents of only a single type, which are characterized by a lack of document diversity.Moreover, there is a significant discrepancy in the annotation standards across datasets.In this paper, we introduce a large and diverse document hierarchy parsing (DHP) dataset to compensate for the data scarcity and inconsistency problem.We aim to set a new standard as a more practical, long-standing benchmark.Meanwhile, we present a new DHP framework designed to grasp both fine-grained text content and coarsegrained pattern at layout element level, enhancing the capacity of pre-trained text-layout models in handling the multi-page and multi-level challenges in DHP.Through exhaustive experiments, we validate the effectiveness of our proposed dataset and method 1 .

Natural language inference (NLI) aims to infer the relationship between two texts: premise and hypothesis. However, many existing methods overlook the problem of overestimation of model performance due to superficial correlation biases in NLI datasets. We study this problem and find that most current models have taken NLI as one of the text-matching tasks, which ignores the asymmetry of the premise and hypothesis of NLI. Therefore, we propose a simple and effective augmentation method, Inversive-Reasoning Augmentation (IRA), to remove the superficial correlation bias. After training the different NLI models with our IRA-augmented data based on two widely-used NLI datasets, we observe more fair evaluation results of the performance and robustness of the various NLI models.

Dynamic graph data mining has gained popularity in recent years due to the rich information contained in dynamic graphs and their widespread use in the real world. Despite the advances in dynamic graph neural networks (DGNNs), the rich information and diverse downstream tasks have posed significant difficulties for the practical application of DGNNs in industrial scenarios. To this end, in this paper, we propose to address them by pre-training and present the Contrastive Pre-Training Method for Dynamic Graph Neural Networks (CPDG). CPDG tackles the challenges of pre-training for DGNNs, including generalization capability and long-short term modeling capability, through a flexible structural-temporal subgraph sampler along with structural-temporal contrastive pre-training schemes. Extensive experiments conducted on both large-scale research and industrial dynamic graph datasets show that CPDG outperforms existing methods in dynamic graph pre-training for various downstream tasks under three transfer settings.

Learning item representation is crucial for a myriad of on-line e-commerce applications. The nucleus of retail item representation learning is how to properly fuse the semantics within a single item, and the interactions across different items generated by user behaviors (e.g., co-click or co-view). Product semantics depict the intrinsic characteristics of the item, while the interactions describe the relationships between items from the perspective of human perception. Existing approaches either solely rely on a single type of information or loosely couple them together, leading to hindered representations. In this work, we propose a novel model named TESPA to reinforce semantic modeling and interaction modeling mutually. Specifically, collaborative filtering signals in the interaction graph are encoded into the language models through fine-grained topological pre-training, and the interaction graph is further enriched based on semantic similarities. After that, a novel multi-channel co-training paradigm is proposed to deeply fuse the semantics and interactions under a unified framework. In a nutshell, TESPA is capable of enjoying the merits of both sides to facilitate item representation learning. Experimental results of on-line and off-line evaluations demonstrate the superiority of our proposal.

Federated learning (FL) is undergoing significant traction due to its ability to perform privacy-preserving training on decentralized data. In this work, we focus on sensitive time series data collected by distributed sensors in real-world applications. However, time series data introduce the challenge of dual spatial-temporal feature skew due to their dynamic changes across domains and time, differing from computer vision. This key challenge includes inter-client spatial feature skew caused by heterogeneous sensor collection and intra-client temporal feature skew caused by dynamics in time series distribution. We follow the framework of Personalized Federated Learning (pFL) to handle dual feature drifts to enhance the capabilities of customized local models. Therefore, in this paper, we propose a method FedST to solve key challenges through orthogonal feature decoupling and regularization in both training and testing stages. During training, we collaborate time view and frequency view of time series data to enrich the mutual information and adopt orthogonal projection to disentangle and align the shared and personalized features between views, and between clients. During testing, we apply prototype-based predictions and model-based predictions to achieve model consistency based on shared features. Extensive experiments on multiple real-world classification datasets and multimodal time series datasets show our method consistently outperforms state-of-the-art baselines with clear advantages.

Unsupervised Graph Domain Adaptation (UGDA) involves the transfer of knowledge from a label-rich source graph to an unlabeled target graph under domain discrepancies. Despite the proliferation of methods designed for this emerging task, the lack of standard experimental settings and fair performance comparisons makes it challenging to understand which and when models perform well across different scenarios. To fill this gap, we present the first comprehensive benchmark for unsupervised graph domain adaptation named GDABench, which encompasses 16 algorithms across diverse adaptation tasks. Through extensive experiments, we observe that the performance of current UGDA models varies significantly across different datasets and adaptation scenarios. Specifically, we recognize that when the source and target graphs face significant distribution shifts, it is imperative to formulate strategies to effectively address and mitigate graph structural shifts. We also find that with appropriate neighbourhood aggregation mechanisms, simple GNN variants can even surpass state-of-the-art UGDA baselines. To facilitate reproducibility, we have developed an easy-to-use library PyGDA for training and evaluating existing UGDA methods, providing a standardized platform in this community. Our source codes and datasets can be found at https://github.com/pygda-team/pygda.

Graph Neural Networks (GNNs) exhibit strong potential in node classification task through a message-passing mechanism. However, their performance often hinges on high-quality node labels, which are challenging to obtain in real-world scenarios due to unreliable sources or adversarial attacks. Consequently, label noise is common in real-world graph data, negatively impacting GNNs by propagating incorrect information during training. To address this issue, the study of Graph Neural Networks under Label Noise (GLN) has recently gained traction. However, due to variations in dataset selection, data splitting, and preprocessing techniques, the community currently lacks a comprehensive benchmark, which impedes deeper understanding and further development of GLN. To fill this gap, we introduce NoisyGL in this paper, the first comprehensive benchmark for graph neural networks under label noise. NoisyGL enables fair comparisons and detailed analyses of GLN methods on noisy labeled graph data across various datasets, with unified experimental settings and interface. Our benchmark has uncovered several important insights that were missed in previous research, and we believe these findings will be highly beneficial for future studies. We hope our open-source benchmark library will foster further advancements in this field. The code of the benchmark can be found in https://github.com/eaglelab-zju/NoisyGL.

Recent neural information retrieval models using dense text representations generated by pre-trained models commonly face two issues. First, a pre-trained model (e.g., BERT) usually truncates a long document before giving its representation, which may cause the loss of some important semantic information. Second, although pre-training models like BERT have been widely used in generating sentence embeddings, a substantial body of literature has shown that the pre-training models often represent sentence embeddings in a homogeneous and narrow space, known as the problem of representation anisotropy, which hurts the quality of dense vector retrieval. In this paper, we split the query and the document in information retrieval into two sets of natural sentences and generate their sentence embeddings with BERT, the most popular pre-trained model. Before aggregating the sentence embeddings to get the entire embedding representations of the input query and document, to alleviate the usual representation degeneration problem of sentence embeddings from BERT, we sample the variational auto-encoder's latent space distribution to obtain isotropic sentence embeddings and utilize supervised contrastive learning to uniform the distribution of these sentence embeddings in the representation space. Our proposed model undergoes training optimization for both the query and the document in the abovementioned aspects. Our model performs well in evaluating three extensively researched neural information retrieval datasets.

Unsupervised Graph Domain Adaptation (UGDA) has emerged as a practical solution to transfer knowledge from a label-rich source graph to a completely unlabelled target graph. However, most methods require a labelled source graph to provide supervision signals, which might not be accessible in the real-world settings due to regulations and privacy concerns. In this paper, we explore the scenario of source-free unsupervised graph domain adaptation, which tries to address the domain adaptation problem without accessing the labelled source graph. Specifically, we present a novel paradigm called GraphCTA, which performs model adaptation and graph adaptation collaboratively through a series of procedures: (1) conduct model adaptation based on node's neighborhood predictions in target graph considering both local and global information; (2) perform graph adaptation by updating graph structure and node attributes via neighborhood contrastive learning; and (3) the updated graph serves as an input to facilitate the subsequent iteration of model adaptation, thereby establishing a collaborative loop between model adaptation and graph adaptation. Comprehensive experiments are conducted on various public datasets. The experimental results demonstrate that our proposed model outperforms recent source-free baselines by large margins.

Abstract Marker‐based optical motion capture (MoCap) aims to localize 3D human motions from a sequence of input raw markers. It is widely used to produce physical movements for virtual characters in various games such as the role‐playing game, the fighting game, and the action‐adventure game. However, the conventional MoCap cleaning and solving process is extremely labor‐intensive, time‐consuming, and usually the most costly part of game animation production. Thus, there is a high demand for automated algorithms to replace costly manual operations and achieve accurate MoCap cleaning and solving in the game industry. In this article, we design a divide‐and‐conquer‐based MoCap solving network, dubbed MarkerNet, to estimate human skeleton motions from sequential raw markers effectively. In a nutshell, our key idea is to decompose the task of direct solving of global motion from all markers into first modeling sub‐motions of local parts from the corresponding marker subsets and then aggregating sub‐motions into a global one. In this manner, our model can effectively capture local motion patterns w.r.t. different marker subsets, thus producing more accurate results compared to the existing methods. Extensive experiments on both real and synthetic data verify the effectiveness of the proposed method.

Next destination recommendation is a crucial research area for understanding human travel behavior. However, existing studies often overlook the problem of underfitting, which arises due to the limited regularity in users' travel patterns. To tackle this issue, we leverage diverse co-occurrence patterns (CoPs) to discover potential user preferences. These patterns capture intersections with similar spatial and temporal characteristics in users' travels. However, traditional graph neural network (GNN)-based approaches struggle to effectively handle complex spatial-temporal CoPs. To overcome these challenges, we propose a novel framework called DHIN (Dynamic Heterogeneous Information Network). Firstly, to address the problem of underfitting, DHIN constructs intricate CoPs by leveraging abundant features and connection relationships. Additionally, to solve the needs of cold-start users, DHIN generates potential connections by capturing dynamic urban hotspots based on global users' travel trajectories. Moreover, to model dynamic heterogeneous information, DHIN utilizes a hierarchical attention mechanism and integrates a dynamic encoder. The mechanism integrates multi-level attention to learn informative embeddings from heterogeneous attributes and structures, while the dynamic encoder processes dynamic temporal information for updating node representations. Finally, extensive experiments conducted on real-world trajectory data demonstrate the effectiveness of the proposed DHIN model.

Table structure recognition (TSR) aims at extracting tables in images into machine-understandable formats. Recent methods solve this problem by predicting the adjacency relations of detected cell boxes, or learning to generate the corresponding markup sequences from the table images. However, they either count on additional heuristic rules to recover the table structures, or require a huge amount of training data and time-consuming sequential decoders. In this paper, we propose an alternative paradigm. We model TSR as a logical location regression problem and propose a new TSR framework called LORE, standing for LOgical location REgression network, which for the first time combines logical location regression together with spatial location regression of table cells. Our proposed LORE is conceptually simpler, easier to train and more accurate than previous TSR models of other paradigms. Experiments on standard benchmarks demonstrate that LORE consistently outperforms prior arts. Code is available at https:// github.com/AlibabaResearch/AdvancedLiterateMachinery/tree/main/DocumentUnderstanding/LORE-TSR.

Graph clustering is a fundamental task in graph analysis, and recent advances in utilizing graph neural networks (GNNs) have shown impressive results. Despite the success of existing GNN-based graph clustering methods, they often overlook the quality of graph structure, which is inherent in real-world graphs due to their sparse and multifarious nature, leading to subpar performance. Graph structure learning allows refining the input graph by adding missing links and removing spurious connections. However, previous endeavors in graph structure learning have predominantly centered around supervised settings, and cannot be directly applied to our specific clustering tasks due to the absence of ground-truth labels. To bridge the gap, we propose a novel method called homophily-enhanced structure learning for graph clustering (HoLe). Our motivation stems from the observation that subtly enhancing the degree of homophily within the graph structure can significantly improve GNNs and clustering outcomes. To realize this objective, we develop two clustering-oriented structure learning modules, i.e., hierarchical correlation estimation and cluster-aware sparsification. The former module enables a more accurate estimation of pairwise node relationships by leveraging guidance from latent and clustering spaces, while the latter one generates a sparsified structure based on the similarity matrix and clustering assignments. Additionally, we devise a joint optimization approach alternating between training the homophily-enhanced structure learning and GNN-based clustering, thereby enforcing their reciprocal effects. Extensive experiments on seven benchmark datasets of various types and scales, across a range of clustering metrics, demonstrate the superiority of HoLe against state-of-the-art baselines.

Inexhaustible web content carries abundant perceptible information beyond text.Unfortunately, most prior efforts in pre-trained Language Models (LMs) ignore such cyberrichness, while few of them only employ plain HTMLs, and crucial information in the rendered web, such as visual, layout, and style, are excluded.Intuitively, those perceptible web information can provide essential intelligence to facilitate content understanding tasks.This study presents an innovative Gestalt Enhanced Markup (GEM) Language Model inspired by Gestalt psychological theory for hosting heterogeneous visual information from the render tree into the language model without requiring additional visual input.Comprehensive experiments on multiple downstream tasks, i.e., web question answering and web information extraction, validate GEM superiority.

Simultaneous Speech Translation (SimulST) is a task focused on ensuring high-quality translation of speech in low-latency situations.Despite this, the modality gap (e.g., unknown word boundaries) between audio and text presents a challenge.This gap hinders the effective application of policies from simultaneous text translation (SimulMT) and compromises the performance of offline speech translation.To address this issue, we first leverage the Montreal Forced Aligner (MFA) and utilize audio transcription pairs in pre-training the acoustic encoder, and introduce a token-level crossmodal alignment that allows the wait-k policy from SimulMT to better adapt to SimulST.This token-level boundary alignment simplifies the decision-making process for predicting read/write actions, as if the decoder were directly processing text tokens.Subsequently, to optimize the SimulST task, we propose a robust and random wait-k-tokens strategy.This strategy allows a single model to meet various latency requirements and minimizes error accumulation of boundary alignment during inference.Our experiments on the MuST-C dataset show that our method achieves a better tradeoff between translation quality and latency.

Implicit neural representations (INRs) aim to learn a continuous function (i.e., a neural network) to represent an image, where the input and output of the function are pixel coordinates and RGB/Gray values, respectively. However, images tend to consist of many objects whose colors are not perfectly consistent, resulting in the challenge that image is actually a discontinuous piecewise function and cannot be well estimated by a continuous function. In this paper, we empirically investigate that if a neural network is enforced to fit a discontinuous piecewise function to reach a fixed small error, the time costs will increase exponentially with respect to the boundaries in the spatial domain of the target signal. We name this phenomenon the exponential-increase hypothesis. Under the exponential-increase hypothesis, learning INRs for images with many objects will converge very slowly. To address this issue, we first prove that partitioning a complex signal into several sub-regions and utilizing piecewise INRs to fit that signal can significantly speed up the convergence. Based on this fact, we introduce a simple partition mechanism to boost the performance of two INR methods for image reconstruction: one for learning INRs, and the other for learning-to-learn INRs. In both cases, we partition an image into different sub-regions and dedicate smaller networks for each part. In addition, we further propose two partition rules based on regular grids and semantic segmentation maps, respectively. Extensive experiments validate the effectiveness of the proposed partitioning methods in terms of learning INR for a single image (ordinary learning framework) and the learning-to-learn framework. Code is released here.

Unsupervised graph anomaly detection is crucial for various practical applications as it aims to identify anomalies in a graph that exhibit rare patterns deviating significantly from the majority of nodes. Recent advancements have utilized Graph Neural Networks (GNNs) to learn high-quality node representations for anomaly detection by aggregating information from neighborhoods. However, the presence of anomalies may render the observed neighborhood unreliable and result in misleading information aggregation for node representation learning. Selecting the proper neighborhood is critical for graph anomaly detection but also challenging due to the absence of anomaly-oriented guidance and the interdependence with representation learning. To address these issues, we utilize the advantages of reinforcement learning in adaptively learning in complex environments and propose a novel method that incorporates Reinforcement neighborhood selection for unsupervised graph ANomaly Detection (RAND). RAND begins by enriching the candidate neighbor pool of the given central node with multiple types of indirect neighbors. Next, RAND designs a tailored reinforcement anomaly evaluation module to assess the reliability and reward of considering the given neighbor. Finally, RAND selects the most reliable subset of neighbors based on these rewards and introduces an anomaly-aware aggregator to amplify messages from reliable neighbors while diminishing messages from unreliable ones. Extensive experiments on both three synthetic and two real-world datasets demonstrate that RAND outperforms the state-of-the-art methods.

Surface subdivision is crucial in 3D mesh presentation and storage. Existing classical subdivision methods adopt manually-defined rules and fail to precisely restore a high-resolution mesh. Recently, a data-driven subdivision method delivers neural networks to learn complex non-linear subdivision schemes, but it suffers from time-consuming training data preparation and limited subdivision levels. In this paper, we propose a new data-driven surface subdivision framework, namely Implicit Neural Distance Optimization Neural Subdivision (INDONS), aiming to subdivide coarse meshes to restore high-resolution meshes. Our method subdivides coarse meshes once in each forward process while can be recursively applied to obtain multilevel subdivision results, which is the same as the classical subdivision mechanism. Moreover, we introduce a new loss function, named implicit neural distance, to provide precise supervised signals for training the subdivision network. The implicit neural distance is defined as the distance between a mesh and an implicit neural field. Extensive experiments demonstrate that our INDONS framework is flexible to be deployed, and INDONS outperforms classical subdivision methods and existing data-driven subdivision method in restoring high-resolution meshes.

Empowered by labeled datasets, supervised pre-training based transfer learning (SPTL) has made significant advances for image classification applications. However, due to privacy-preserving protocol and unaccessible annotation, it emerges as a novel problem in federated learning scenarios whether unsupervised pre-training based transfer learning (UPTL) is available for semantic segmentation. In this work, we define federated transfer learning (FedTL) in the absence of source domain label, and track research progress on pavement crack benchmark. The main challenges of FedTL include: i) a privacy-protecting distributed training framework that extends UPTL to the constraints of federated settings, and ii) a self-learning semantic segmentation approach that develops self-supervised learning paradigm to simultaneously learn category and shape representations. Motivated by that, we propose a FedCrack model to absorb feature disentanglement and prototype clustering into vision Transformer, which obtains the pre-trained encoder on source domain without accessing annotation. Thereafter, a fine-tuning stage is presented to learn decoder with scaling attention on target domain for fine-grained crack segmentation. The effectiveness of proposed FedCrack can be demonstrated with superior performance of 82.14% on mIoU and 9.85 FPS on speed in extensive experiments. To the best of our knowledge, it is the first work in FedTL to gain weights of unsupervised pre-training representations on source domain locally, gradients of which are then aggregated to a federated central model that also fine-tunes the transferable parameters by target domain.

Graph Neural Networks (GNNs), which generalize deep neural networks to graph-structured data, have drawn considerable attention and achieved state-of-the-art performance in numerous graph related tasks. However, existing GNN models mainly focus on designing graph convolution operations. The graph pooling (or downsampling) operations, that play an important role in learning hierarchical representations, are usually overlooked. In this paper, we proposed a novel multi-view graph pooling operator dubbed as MVPool, which ranks nodes across different views with different contextual graph information. Meanwhile, attention mechanism is utilized to promote the collaboration of different views for generating robust node rankings. Then the pooling operation adaptively selects a subset of nodes to form an induced subgraph based on the ranking list. To preserve the underlying graph topological information, we further introduce a structure learning mechanism to learn a refined graph structure for the pooled graph at each layer. The proposed MVPool operator is a general strategy that can be integrated into various graph neural network architectures. By combining MVPool operator with graph neural networks, we perform hierarchical representation learning for both node and graph level classification as well as clustering tasks. Experimental results on nine widely used benchmarks demonstrate the effectiveness of our proposed model.

3D convolutional neural networks have revealed superior performance in processing volumetric data such as video and medical imaging. However, the competitive performance by leveraging 3D networks results in huge computational costs, which are far beyond that of 2D networks. In this paper, we propose a novel Hilbert curve-based cross-dimensionality distillation approach that facilitates the knowledge of 3D networks to improve the performance of 2D networks. The proposed Hilbert Distillation (HD) method preserves the structural information via the Hilbert curve, which maps high-dimensional (>=2) representations to one-dimensional continuous space-filling curves. Since the distilled 2D networks are supervised by the curves converted from dimensionally heterogeneous 3D features, the 2D networks are given an informative view in terms of learning structural information embedded in well-trained high-dimensional representations. We further propose a Variable-length Hilbert Distillation (VHD) method to dynamically shorten the walking stride of the Hilbert curve in activation feature areas and lengthen the stride in context feature areas, forcing the 2D networks to pay more attention to learning from activation features. The proposed algorithm outperforms the current state-of-the-art distillation techniques adapted to cross-dimensionality distillation on two classification tasks. Moreover, the distilled 2D networks by the proposed method achieve competitive performance with the original 3D networks, indicating the lightweight distilled 2D networks could potentially be the substitution of cumbersome 3D networks in the real-world scenario.

User recommendation aims at recommending users with potential interests in the social network. Previous works have mainly focused on the undirected social networks with symmetric relationship such as friendship, whereas recent advances have been made on the asymmetric relationship such as the following and followed by relationship. Among the few existing direction-aware user recommendation methods, the random walk strategy has been widely adopted to extract the asymmetric proximity between users. However, according to our analysis on real-world directed social networks, we argue that the asymmetric proximity captured by existing random walk based methods are insufficient due to the inbalance in-degree and out-degree of nodes. To tackle this challenge, we propose InfoWalk, a novel informative walk strategy to efficiently capture the asymmetric proximity solely based on random walks. By transferring the direction information into the weights of each step, InfoWalk is able to overcome the limitation of edges while simultaneously maintain both the direction and proximity. Based on the asymmetric proximity captured by InfoWalk, we further propose the qualitative (DNE-L) and quantitative (DNE-T) directed network embedding methods, capable of preserving the two properties in the embedding space. Extensive experiments conducted on six real-world benchmark datasets demonstrate the superiority of the proposed DNE model over several state-of-the-art approaches in various tasks.

On-demand food delivery (OFD) platforms have greatly impacted the food service industry, where OFD recommendation systems play a central role in enhancing user experience and raising revenues. OFD recommendation, compared with existing online e-commerce recommendation systems, needs to put more emphasis on fulfillment time related variables, because the order fulfillment cycle time (OFCT) which refers to the time elapsed between a user placing a food order and receiving the food significantly influences a user's choice from the recommended items. In this paper, we investigate the OFCT related information and propose a Fulfillment-Time-Aware Personalized Ranking (FTAPR) method for recommendation. FTAPR mainly consists of three components. First, Transformers are used to estimate OFCT based on a large amount of user order sequences. Then, the predicted OFCT and other OFCT related features are fused and encoded by a deep & cross network to learn fulfillment time related feature representation. At the last step, the time bias representation from the deep & cross network is integrated into the ranking system to deliver final search results. Extensive offline and online experiments on real-world datasets collected from one of China's largest OFD platforms Ele.me show the superiority of our model, e.g., an online A/B testing shows that FTAPR brings 1.3% and 2.5% gains in CTR and CVR compared with baselines.

Knowledge Distillation (KD) aims at transferring knowledge from a larger well-optimized teacher network to a smaller learnable student network. Existing KD methods have mainly considered two types of knowledge, namely the individual knowledge and the relational knowledge. However, these two types of knowledge are usually modeled independently while the inherent correlations between them are largely ignored. It is critical for sufficient student network learning to integrate both individual knowledge and relational knowledge while reserving their inherent correlation. In this paper, we propose to distill the novel holistic knowledge based on an attributed graph constructed among instances. The holistic knowledge is represented as a unified graph-based embedding by aggregating individual knowledge from relational neighborhood samples with graph neural networks, the student network is learned by distilling the holistic knowledge in a contrastive manner. Extensive experiments and ablation studies are conducted on benchmark datasets, the results demonstrate the effectiveness of the proposed method. The code has been published in https://github.com/wyc-ruiker/HKD

The e-commerce era is witnessing rising new arrivals of items on e-commerce platforms every day. Identifying potential popular items accurately is of great importance in creating commercial value. Click-Through Rate (CTR) is a general indicator to evaluate item popularity. However, existing methods fail in new arrivals prediction because of sparse item features, missing item statistics and high time complexity of computing for all pairs of users and items. To tackle these challenges, we propose a novel Adversarial Two-tower Neural Network (ATNN) model for new arrivals CTR predictions by introducing an adversarial network to a two-tower network. We design a generator and a discriminator to better learn an item vector based on item profiles without item statistics. We also develop a strategy with an O(1) time complexity for a new item's popularity prediction by constructing a user group and utilizing its mean user vector in a time-efficient manner. We implement ATNN on a largescale real-world dataset from one of the world's largest ecommerce platforms, “Tmall.com”. Empirical results show that ATNN is strongly capable of learning item vectors from item profiles for e-commerce. Furthermore, by introducing multi-task learning technology, we extend ATNN to food delivery service. Experimental results on one popular food delivery platform, “Ele.me”, demonstrate that ATNN can recognize attractive and welcoming new restaurants that have higher Value per Page View (VpPV) and generate more Gross Merchandise Volume (GMV).

Live-streaming platforms have recently gained significant popularity by attracting an increasing number of young users and have become a very promising form of online shopping. Similar to the traditional online shopping platforms such as Taobao, live-streaming platforms also suffer from online malicious fraudulent behaviors where many transactions are not genuine. The existing anti-fraud models proposed to recognize fraudulent transactions on traditional online shopping platforms are inapplicable on live-streaming platforms. This is mainly because live-streaming platforms are characterized by a unique type of heterogeneous live-streaming networks where multiple heterogeneous types of nodes such as users, live-streamers, and products are connected with multiple different types of edges associated with edge features. In this paper, we propose a new approach based on a heterogeneous graph neural network for LIve-streaming Fraud dEtection (called LIFE). LIFE designs an innovative heterogeneous graph learning model that fully utilizes various heterogeneous information of shopping transactions, users, streamers, and items from a given live-streaming platform. Moreover, a label propagation algorithm is employed within our LIFE framework to handle the limited number of labeled fraudulent transactions for model training. Extensive experimental results on a large-scale Taobao live-streaming platform demonstrate that the proposed method is superior to the baseline models in terms of fraud detection effectiveness on live-streaming platforms. Furthermore, we conduct a case study to show that the proposed method is able to effectively detect fraud communities for live-streaming e-commerce platforms.

Graph similarity learning, which measures the similarities between a pair of graph-structured objects, lies at the core of various machine learning tasks such as graph classification, similarity search, etc. In this paper, we devise a novel graph neural network based framework to address this challenging problem, motivated by its great success in graph representation learning. As the vast majority of existing graph neural network models mainly concentrate on learning effective node or graph level representations of a single graph, little effort has been made to jointly reason over a pair of graph-structured inputs for graph similarity learning. To this end, we propose Hierarchical Hypergraph Matching Networks (H2sup>MN) to calculate the similarities between graph pairs with arbitrary structure. Specifically, our proposed H2MN learns graph representation from the perspective of hypergraph, and takes each hyperedge as a subgraph to perform subgraph matching, which could capture the rich substructure similarities across the graph. To enable hierarchical graph representation and fast similarity computation, we further propose a hyperedge pooling operator to transform each graph into a coarse graph of reduced size. Then, a multi-perspective cross-graph matching layer is employed on the coarsened graph pairs to extract the inter-graph similarity. Comprehensive experiments on five public datasets empirically demonstrate that our proposed model can outperform state-of-the-art baselines with different gains for graph-graph classification and regression tasks.

Image retrieval with text feedback is an emerging research topic with the objective of integrating inputs from multiple modalities as queries. In this paper, queries contain a reference image plus text feedback that describes modifications between this image and the desired image. The existing work for this task mainly focuses on designing a new fusion network to compose the image and text. Still, little research pays attention to the modality gap caused by the inconsistent distribution of features from different modalities, which dramatically influences the feature fusion and similarity learning between queries and the desired image. We propose a Distribution-Aligned Text-based Image Retrieval (DATIR) model, which consists of attention mutual information maximization and hierarchical mutual information maximization, to bridge this gap by increasing non-linear statistic dependencies between representations of different modalities. More specifically, attention mutual information maximization narrows the modality gap between different input modalities by maximizing mutual information between the text representation and its semantically consistent representation captured from the reference image and the desired image by the difference transformer. For hierarchical mutual information maximization, it aligns distributions of features from the image modality and the fusion modality by estimating mutual information between a single-layer representation in the fusion network and the multi-level representations in the desired image encoder. Extensive experiments on three large-scale benchmark datasets demonstrate that we can bridge the modality gap between different modalities and achieve state-of-the-art retrieval performance.

Recent advances have been made in applying convolutional neural networks to achieve more precise prediction results for medical image segmentation problems. However, the success of existing methods has highly relied on huge computational complexity and massive storage, which is impractical in the real-world scenario. To deal with this problem, we propose an efficient architecture by distilling knowledge from well-trained medical image segmentation networks to train another lightweight network. This architecture empowers the lightweight network to get a significant improvement on segmentation capability while retaining its runtime efficiency. We further devise a novel distillation module tailored for medical image segmentation to transfer semantic region information from teacher to student network. It forces the student network to mimic the extent of difference of representations calculated from different tissue regions. This module avoids the ambiguous boundary problem encountered when dealing with medical imaging but instead encodes the internal information of each semantic region for transferring. Benefited from our module, the lightweight network could receive an improvement of up to 32.6% in our experiment while maintaining its portability in the inference phase. The entire structure has been verified on two widely accepted public CT datasets LiTS17 and KiTS19. We demonstrate that a lightweight network distilled by our method has non-negligible value in the scenario which requires relatively high operating speed and low storage usage.

Many IoT applications have the requirements of conducting complex IoT events processing (e.g., speech recognition) that are hardly supported by low-end IoT devices due to limited resources. Most existing approaches enable complex IoT event processing on low-end IoT devices by statically allocating tasks to the edge or the cloud. In this article, we present Queec, a QoE-aware edge computing system for complex IoT event processing under dynamic workloads. With Queec, the complex IoT event processing tasks that are relatively computation-intensive for low-end IoT devices can be transparently offloaded to nearby edge nodes at runtime. We formulate the problem of scheduling multi-user tasks to multiple edge nodes as an optimization problem, which minimizes the overall offloading latency of all tasks while avoiding the overloading problem. We implement Queec on low-end IoT devices, edge nodes, and the cloud. We conduct extensive evaluations, and the results show that Queec reduces 56.98% of the offloading latency on average compared with the state-of-the-art under dynamic workloads, while incurring acceptable overhead.

Network embedding plays a crucial role in network analysis to provide effective representations for a variety of learning tasks. Existing attributed network embedding methods mainly focus on preserving the observed node attributes and network topology in the latent embedding space, with the assumption that nodes connected through edges will share similar attributes. However, our empirical analysis of real-world datasets shows that there exist both commonality and individuality between node attributes and network topology. On the one hand, similar nodes are expected to share similar attributes and have edges connecting them (commonality). On the other hand, each information source may maintain individual differences as well (individuality). Simultaneously capturing commonality and individuality is very challenging due to their exclusive nature and existing work fail to do so. In this paper, we propose a deep generative embedding (DGE) framework which simultaneously captures commonality and individuality between network topology and node attributes in a generative process. Stochastic gradient variational Bayesian (SGVB) optimization is employed to infer model parameters as well as the node embeddings. Extensive experiments on four real-world datasets show the superiority of our proposed DGE framework in various tasks including node classification and link prediction.

Graph classification aims to extract accurate information from graph-structured data for classification and is becoming more and more important in the graph learning community. Although Graph Neural Networks (GNNs) have been successfully applied to graph classification tasks, most of them overlook the scarcity of labeled graph data in many applications. For example, in bioinformatics, obtaining protein graph labels usually needs laborious experiments. Recently, few-shot learning has been explored to alleviate this problem with only a few labeled graph samples of test classes. The shared sub-structures between training classes and test classes are essential in the few-shot graph classification. Existing methods assume that the test classes belong to the same set of super-classes clustered from training classes. However, according to our observations, the label spaces of training classes and test classes usually do not overlap in a real-world scenario. As a result, the existing methods don't well capture the local structures of unseen test classes. To overcome the limitation, in this paper, we propose a direct method to capture the sub-structures with a well initialized meta-learner within a few adaptation steps. More specifically, (1) we propose a novel framework consisting of a graph meta-learner, which uses GNNs based modules for fast adaptation on graph data, and a step controller for the robustness and generalization of meta-learner; (2) we provide quantitative analysis for the framework and give a graph-dependent upper bound of the generalization error based on our framework; (3) the extensive experiments on real-world datasets demonstrate that our framework gets state-of-the-art results on several few-shot graph classification tasks compared to baselines.

The e-commerce appeals to a multitude of online shoppers by providing personalized experiences and becomes indispensable in our daily life. Accurately predicting user preference and making a recommendation of favorable items plays a crucial role in improving several key tasks such as Click Through Rate (CTR) and Conversion Rate (CVR) in order to increase commercial value. Some state-of-the-art collaborative filtering methods exploiting non-linear interactions on a user-item bipartite graph are able to learn better user and item representations with Graph Neural Networks (GNNs), which do not learn hierarchical representations of graphs because they are inherently flat. Hierarchical representation is reportedly favorable in making more personalized item recommendations in terms of behaviorally similar users in the same community and a context of topic-driven taxonomy. However, some advanced approaches, in this regard, are either only considering linear interactions, or adopting single-level community, or computationally expensive. To address these problems, we propose a novel method with Hierarchical bipartite Graph Neural Network (HiGNN) to handle large-scale e-commerce tasks. By stacking multiple GNN modules and using a deterministic clustering algorithm alternately, HiGNN is able to efficiently obtain hierarchical user and item embeddings simultaneously, and effectively predict user preferences on a larger scale. Extensive experiments on some real-world e-commerce datasets demonstrate that HiGNN achieves a significant improvement compared to several popular methods. Moreover, we deploy HiGNN in Taobao, one of the largest e-commerces with hundreds of million users and items, for a series of large-scale prediction tasks of item recommendations. The results also illustrate that HiGNN is arguably promising and scalable in real-world applications.

The recent years have witnessed the rapid growth of IoT (Internet of Things) applications. A typical IoT application usually consists of three essential parts: the device side, the cloud side, and the client side. The development of a complete IoT application is very difficult for non-expert developers because it involves drastically different technologies and complex interactions between different sides. Unlike traditional IoT development platforms which use separate approaches for these three sides, we present TinyLink 2.0, an integrated IoT development approach with a single coherent language. It achieves high expressiveness for diverse IoT applications by an enhanced IFTTT rule design and a virtual sensor mechanism which helps developers express application logic with machine learning. Moreover, TinyLink 2.0 optimizes the IoT application performance by using both static and dynamic optimizers, especially for resource-constrained IoT devices. We implement TinyLink 2.0 and evaluate it with eight case studies, a user study, and a detailed evaluation of the proposed programming language as well as the performance optimizers. Results show that TinyLink 2.0 can speed up IoT development significantly compared with existing approaches from both industry and academia, while still achieving high expressiveness.

Graph embedding has become the research focus in both academic and industrial communities due to its powerful capabilities. The majority of existing work overwhelmingly learn node embeddings in the context of static, plain or attributed, homogeneous graphs. However, many real-world applications frequently involve bipartite graphs with temporal and attributed interaction edges, named temporal interaction graphs. The temporal interactions usually imply different facets of interest and might even evolve over time, thus putting forward huge challenges in learning effective node representations. In this paper, we propose a novel framework named TigeCMN to learn node representations from a sequence of temporal interactions. Specifically, we devise two coupled memory networks to store and update node embeddings in external matrices explicitly and dynamically, which forms deep matrix representations and could enhance the expressiveness of the node embeddings. We conduct experiments on two real-world datasets and the experimental results empirically demonstrate that TigeCMN can outperform the state-of-the-arts with different gains.

The e-commerce era is witnessing a rapid development of various annual online promotions, such as Black Friday, Cyber Monday, and Alibaba's 11.11, etc. S ales P redictions for O nline P romotions (SPOP) are a set of sales related forecasts for the promotion day, including gross merchandise volume, sales volume, best selling products, etc. SPOP is highly important for e-commerce platforms to efficiently organize merchandise and maximize business values. However, sales patterns during the promotions are varied according to different scenarios, each model of which is designed with different features, static or dynamic, for one task in particular. Therefore, several models are proposed with part of features that are possibly beneficial to other tasks, which indicates the universal representation for the items needs to be learned across different promotion scenarios. To address this problem, this paper proposes a D eep I tem N etwork for O nline P romotions (DINOP). In DINOP, we design a novel T arget U sers C ontrolled G ated R ecurrent U nit (TUC-GRU) structure for dynamic features, and provide a new attention mechanism introducing static users profiles. In contrast to traditional prediction models, the network we proposed can effectively and efficiently learn universal item representation by incorporating users' properties as controllers. Furthermore, it can successfully discover the static and dynamic features guided by the multi-task learning, and is easily extended to other sales related prediction problems without retraining. Empirical results show that performance of DINOP in the real data set of Alibaba's Global Shopping Festival is superior to other state-of-the-arts practical methodologies in terms of the convergence rate and prediction accuracy.

Distance metric learning has been shown to be an effective and efficient method which can lead to significant improvements in classification, clustering, and retrieval. Conventional metric learning methods which apply a global metric to capture the input features and their correlations are unsuitable for heterogeneous data, whose features vary according to different clusters. Thus, a number of local metric learning methods have been proposed to overcome this limitation. Unfortunately, most of them suffer from over-fitting or need the data set to be under a strong assumption, such as Gaussian distribution. To avoid over-fitting caused by learning local metric matrices separately in the previous researches, we propose a novel local metric learning method in which we learn a metric matrix for each pair of instances as linear combinations of basis metric matrices defined on different clusters of the input space without any assumption for data set. We get the parameters of the linear combinations by using the relationship between direct distance and anchor~distance between a pair of data points. Additionally, we design a novel expectation-maximization based optimization method to learn the linear combinations and basis metric matrices. Experimental results show that our method outperforms state-of-the-art methods on various data sets.

While numerous wireless applications are emerging to simplify our daily life, updating price tags in large-scale markets is still largely performed manually. Considering the potential adoption of electronic price (e-price) tags, we propose MoRa, a LoRa-based system for timely and energy-efficient price update in Markets. By leveraging the category information in the market, we design a hierarchical address scheme. Based on the scheme, we can perform efficient multicast with nearly zero overhead in group joining and activity scheduling. Furthermore, considering LoRa's characteristics and the asymmetry between the gateway and nodes, we let the gateway make fast and repetitive transmissions before the uncovered nodes send light weight NAKs. Extensive testbed experiments and simulation evaluations are conducted. Results demonstrate that MoRa can efficiently improve the performance in terms of update delay and energy consumption.

Feature selection has attracted a great deal of interest over the past decades. By selecting meaningful feature subsets, the performance of learning algorithms can be effectively improved. Because label information is expensive to obtain, unsupervised feature selection methods are more widely used than the supervised ones. The key to unsupervised feature selection is to find features that effectively reflect the underlying data distribution. However, due to the inevitable redundancies and noise in a dataset, the intrinsic data distribution is not best revealed when using all features. To address this issue, we propose a novel unsupervised feature selection algorithm via joint local learning and group sparse regression (JLLGSR). JLLGSR incorporates local learning based clustering with group sparsity regularized regression in a single formulation, and seeks features that respect both the manifold structure and group sparse structure in the data space. An iterative optimization method is developed in which the weights finally converge on the important features and the selected features are able to improve the clustering results. Experiments on multiple real-world datasets (images, voices, and web pages) demonstrate the effectiveness of JLLGSR.

With over 34 billion IoT devices to be installed by 2020, the Internet of Things (IoT) is fundamentally changing our lives. One of the greatest benefits of the IoT is the powerful automations achieved by applying rules to IoT devices. For instance, a rule named "Make me a cup of coffee when I wake up'' automatically turns on the coffee machine when the sensor in the bedroom detects motion in the morning. With large numbers of possible rules out there, a recommendation system is of great necessity to help users find rules they need. However, little effort has been made to design a model tailored for the IoT rule recommendation, which comes with lots of new challenges compared with traditional recommendation tasks. We not only need to re-define "users'' and "items'' in the recommendation task, but also have to consider a new type of entities, devices, and the extra information and constraints brought by them. To handle these challenges, we propose a novel efficient recommendation algorithm, which not only considers the implicit feedback of users on rules, but also takes user-rule-device interactions and the match between rule device requirements and user device possessions into account. In collaboration with Samsung, one of the leading companies in this field, we have designed an IoT rule recommendation framework and evaluated our algorithm on a real-life industry dataset. Experiments show the effectiveness and efficiency of our method.

Attributed network embedding focuses on learning low-dimensional latent representations of nodes which can well preserve the original topological and node attributed proximity at the same time. Existing works usually assume that nodes with similar topology or similar attributes should also be close in the embedding space. This assumption ignores the phenomenon of partial correlation between network topological and node attributed similarities i.e. nodes with similar topology may be dissimilar in their attributes and vice versa. Partial correlation between the two information sources should be considered especially when there exist fraudulent edges (i.e., information from one source is vague) or unbalanced data distributions (i.e, topology structure similarity and node attribute similarity have different distributions). However, it is very challenging to consider the partial correlation between topology and attributes due to the heterogeneity of these two information sources. In this paper, we take partial correlation between topology and attributes into account and propose the Personalized Relation Ranking Embedding (PRRE) method for attributed networks which is capable of exploiting the partial correlation between node topology and attributes. The proposed PRRE model utilizes two thresholds to define different node relations and employs the Expectation-Maximization (EM) algorithm to learn these thresholds as well as other embedding parameters. Extensive experiments results on multiple real-world datasets show that the proposed PRRE model significantly outperforms the state-of-the-art methods in terms of various evaluation metrics.

Network representation learning (RL) aims to transform the nodes in a network into low-dimensional vector spaces while preserving the inherent properties of the network. Though network RL has been intensively studied, most existing works focus on either network structure or node attribute information. In this paper, we propose a novel framework, named ANRL, to incorporate both the network structure and node attribute information in a principled way. Specifically, we propose a neighbor enhancement autoencoder to model the node attribute information, which reconstructs its target neighbors instead of itself. To capture the network structure, attribute-aware skip-gram model is designed based on the attribute encoder to formulate the correlations between each node and its direct or indirect neighbors. We conduct extensive experiments on six real-world networks, including two social networks, two citation networks and two user behavior networks. The results empirically show that ANRL can achieve relatively significant gains in node classification and link prediction tasks.

Wireless network simulation is a fundamental service aiming at providing controlled and repeatable environment for protocol design, performance testing, etc. The existing simulators focus on reproducing the packet behaviors on individual links. However, as observed in some recent works, individual link behaviors alone are not enough to characterize the protocol performance. As a result, while the existing works can mimic the link behaviors very closely, they often fail to simulate protocol level performance. In this paper, we propose a novel performance-aware simulation approach which can preserve not only the link-level behaviors but also the performance-level behaviors. We first devise an accurate performance model by combining link quality and the spatial-temporal link correlation. Based on the performance modeling, we then propose a Performance Aware Hidden Markov Model (PA-HMM), where the protocol performance is directly fed into the Markov state transitions. PA-HMM is able to simulate both link-level behaviors and high-level protocol performance. We conduct extensive testbed and simulation experiments with broadcast and anycast protocols. The results show that compared to the state-of-the-art work, 1) the performance model is able to accurately characterize wireless communication performance and 2) the protocol performance is closely simulated as compared to the empirical results.

The e-commerce era is witnessing a rapid increase of mobile Internet users. Major e-commerce companies nowadays see billions of mobile accesses every day. Hidden in these records are valuable user behavioral characteristics such as their shopping preferences and browsing patterns. And, to extract these knowledge from the huge dataset, we need to first link records to the corresponding mobile devices. This Mobile Access Records Resolution (MARR) problem is confronted with two major challenges: (1) device identifiers and other attributes in access records might be missing or unreliable; (2) the dataset contains billions of access records from millions of devices. To the best of our knowledge, as a novel challenge industrial problem of mobile Internet, no existing method has been developed to resolve entities using mobile device identifiers in such a massive scale. To address these issues, we propose a SParse Identifier-linkage Graph (SPI-Graph) accompanied with the abundant mobile device profiling data to accurately match mobile access records to devices. Furthermore, two versions (unsupervised and semi-supervised) of Parallel Graph-based Record Resolution (PGRR) algorithm are developed to effectively exploit the advantages of the large-scale server clusters comprising of more than 1,000 computing nodes. We empirically show superior performances of PGRR algorithms in a very challenging and sparse real data set containing 5.28 million nodes and 31.06 million edges from 2.15 billion access records compared to other state-of-the-arts methodologies.

Web accessibility evaluation examines how well websites comply with accessibility guidelines which help people with disabilities to perceive, navigate and contribute to the Web. This demanding task usually requires manual assessment by experts with many years of training and experience. However, not enough experts are available to carry out the increasing number of evaluation projects while non-experts often have different opinions about the presence of accessibility barriers. Addressing these issues, we introduce a crowdsourcing system with a novel truth inference algorithm to derive reliable and accurate assessments from conflicting opinions of evaluators. Extensive evaluation on 23,901 complex tasks assessed by 50 people with and without disabilities shows that our approach outperforms state of the art approaches. In addition, we conducted surveys to identify frequent barriers that people with disabilities are facing in their daily lives and the difficulty to access Web pages when they encounter these barriers. The frequencies and severities of barriers correlate with their derived importance in our evaluation project.

For mobile apps, user-perceived delays are critical for user satisfaction. According to our measurement, long delays are commonly caused by network and storage I/O operations while short delays are mainly caused by UI rendering. Short delays are not uncommon, which account for 55.3% in our measurement cases. Previous app performance studies have largely focused on I/O operations but the understanding of UI rendering impact is limited. In this work, we propose DRAW, a system that performs two UI rendering analyses to help app developers pinpoint rendering problems and resolve short delays. The first analysis outlines the wasted rendering time on invisible or covered UI components, namely the overdraw problem. The second analysis is to identify the responsible UI components and rendering operations that cause overall low rendering efficiency. We implement DRAW on Android and apply it to study 1,158 real-world Android apps. Results show that DRAW is helpful as it can pinpoint the responsible UI components and specific rendering operations. Four concrete case studies of real-world apps are further presented to show how DRAW can help developers improve the UI rendering performance of their apps.

Existing works for extracting navigation objects from webpages focus on navigation menus, so as to reveal the information architecture of the site. However, web 2.0 sites such as social networks, e-commerce portals etc. are making the understanding of the content structure in a web site increasingly difficult. Dynamic and personalized elements such as top stories, recommended list in a webpage are vital to the understanding of the dynamic nature of web 2.0 sites. To better understand the content structure in web 2.0 sites, in this paper we propose a new extraction method for navigation objects in a webpage. Our method will extract not only the static navigation menus, but also the dynamic and personalized page-specific navigation lists. Since the navigation objects in a webpage naturally come in blocks, we first cluster hyperlinks into different blocks by exploiting spatial locations of hyperlinks, the hierarchical structure of the DOM-tree and the hyperlink density. Then we identify navigation objects from those blocks using the SVM classifier with novel features such as anchor text lengths etc. Experiments on real-world data sets with webpages from various domains and styles verified the effectiveness of our method.

Recent years have seen a surge of research on social recommendation techniques for improving recommender systems due to the growing influence of social networks to our daily life. The intuition of social recommendation is that users tend to show affinities with items favored by their social ties due to social influence. Despite the extensive studies, no existing work has attempted to distinguish and learn the personalized preferences between strong and weak ties, two important terms widely used in social sciences, for each individual in social recommendation. In this paper, we first highlight the importance of different types of ties in social relations originated from social sciences, and then propose anovel social recommendation method based on a new Probabilistic Matrix Factorization model that incorporates the distinction of strong and weak ties for improving recommendation performance. The proposed method is capable of simultaneously classifying different types of social ties in a social network w.r.t. optimal recommendation accuracy, and learning a personalized tie type preference for each user in addition to other parameters. We conduct extensive experiments on four real-world datasets by comparing our method with state-of-the-art approaches, and find encouraging results that validate the efficacy of the proposed method in exploiting the personalized preferences of strong and weak ties for social recommendation.

We consider the problem of finding map regions that best match query keywords. This region search problem can be applied in many practical scenarios such as shopping recommendation, searching for tourist attractions, and collision region detection for wireless sensor networks. While conventional map search retrieves isolate locations in a map, users frequently attempt to find regions of interest instead, e.g., detecting regions having too many wireless sensors to avoid collision, or finding shopping areas featuring various merchandise or tourist attractions of different styles. Finding regions of interest in a map is a non-trivial problem and retrieving regions of arbitrary shapes poses particular challenges. In this paper, we present a novel region search algorithm, dense region search (DRS), and its extensions, to find regions of interest by estimating the density of locations containing the query keywords in the region. Experiments on both synthetic and real-world datasets demonstrate the effectiveness of our algorithm.

Color and tone stylization in images and videos strives to enhance unique themes with artistic color and tone adjustments. It has a broad range of applications from professional image postprocessing to photo sharing over social networks. Mainstream photo enhancement softwares, such as Adobe Lightroom and Instagram, provide users with predefined styles, which are often hand-crafted through a trial-and-error process. Such photo adjustment tools lack a semantic understanding of image contents and the resulting global color transform limits the range of artistic styles it can represent. On the other hand, stylistic enhancement needs to apply distinct adjustments to various semantic regions. Such an ability enables a broader range of visual styles. In this paper, we first propose a novel deep learning architecture for exemplar-based image stylization, which learns local enhancement styles from image pairs. Our deep learning architecture consists of fully convolutional networks for automatic semantics-aware feature extraction and fully connected neural layers for adjustment prediction. Image stylization can be efficiently accomplished with a single forward pass through our deep network. To extend our deep network from image stylization to video stylization, we exploit temporal superpixels to facilitate the transfer of artistic styles from image exemplars to videos. Experiments on a number of data sets for image stylization as well as a diverse set of video clips demonstrate the effectiveness of our deep learning architecture.

The past decade has witnessed a tremendous growth in the variety and complexity of mobile applications (apps). Although considerable amount of efforts have been spent to improve app performance, smartphones nowadays still face many performance challenges. We discover that the resource contention of multiple running apps, caused by resource bottleneck(s), is a key factor that affects the smartphone performance. In this paper, we present APB, an Automatic tool that detects Performance issues caused by resource Bottleneck(s) on commodity Android smartphones. APB employs an innovative bottleneck-hypersurface model to quantify performance issues given a specific system state. Then, based on the model, APB identifies a list of apps that contribute most to the resource contention, which can well inform the end user to take action such as killing background apps to resolve the performance issue. We implement APB on commodity Android platforms and widely evaluate its effectiveness with real user studies. Results show that APB outperforms three baseline approaches and helps users to improve smartphone performance by 10 to 67 percent, with less than 1 percent runtime overhead.

The ZigBee communication can be easily and severely interfered by Wi-Fi traffic. Error recovery, as an important means for ZigBee to survive Wi-Fi interference, has been extensively studied in recent years. The existing works add upfront redundancy to in-packet blocks for recovering a certain number of random corruptions. Therefore, the bursty nature of ZigBee in-packet corruptions under Wi-Fi interference is often considered harmful, since some blocks are full of errors which cannot be recovered and some blocks have no errors but are still requiring redundancy. As a result, they often use interleaving to reshape the bursty errors, before applying complex FEC codes to recover the re-shaped random distributed errors. In this paper, we take a different view that burstiness may be helpful. With burstiness, the in-packet corruptions are often consecutive and the requirement for error recovery is reduced as “recovering any k consecutive errors” instead of “recovering any random k errors”. This lowered requirement allows us to design far more efficient code than the existing FEC codes. Motivated by this implication, we exploit the corruption burstiness to design a simple yet effective error recovery code using XOR operations (called ZiXOR). ZiXOR uses XOR code and the delay is significantly reduced. More, ZiXOR uses RSSI-hinted approach to detect in packet corruptions without CRC, incurring almost no extra transmission overhead. The testbed evaluation results show that ZiXOR outperforms the state-of-the-art works in terms of the throughput (by 47 percent) and latency (by 22 percent).

Inferring fine-grained link metrics by using aggregated path measurements, known as network tomography, is an effective and efficient way to facilitate various network operations, such as network monitoring, load balancing, and failure diagnosis. Given the network topology and a set of interesting links, we study the problem of calculating the link metrics of these links by end-to-end cycle-free path measurements among selected monitors, i.e., preferential link tomography. Since assigning nodes as monitors usually requires non-negligible operational cost, we focus on assigning a minimum number of monitors to identify these interesting links. We propose an optimal monitor assignment (OMA) algorithm for preferential link tomography in communication networks. OMA first partitions the graph representing the network topology into multiple graph components. Then, OMA carefully assigns monitors inside each graph component and at the boundaries of multiple graph components. We theoretically prove the optimality of OMA by proving: 1) the monitors assigned by OMA are able to identify all interesting links and 2) the number of monitors assigned by OMA is minimal. We also implement OMA and evaluate it through extensive simulations based on both real topologies and synthetic topologies. Compared with two baseline approaches, OMA reduces the number of monitors assigned significantly in various network settings.

In this paper, we study the problem of decomposing the end-to-end delay into the per-hop delay for each packet, in multi-hop wireless ad hoc networks. Knowledge on the per-hop per-packet delay can greatly improve the network visibility and facilitate network measurement and management. We propose Domo, a passive, lightweight, and accurate delay tomography approach to decomposing the packet end-to-end delay into each hop. We first formulate the per packet delay tomography problem into a set of optimization problems by carefully considering the constraints among various timing quantities. At the network side, Domo attaches a small overhead to each packet for constructing constraints of the optimization problems. By solving these optimization problems by semi-definite relaxation at the PC side, Domo is able to estimate the per-hop delays with high accuracy as well as give a upper bound and lower bound for each unknown per-hop delay. We implement Domo and evaluate its performance extensively using both trace-driven studies and large-scale simulations. Results show that Domo significantly outperforms two existing methods, nearly tripling the accuracy of the state-of-the-art.

Data dissemination is a fundamental service offered by low-power wireless networks. Sender selection is the key to the dissemination performance and has been extensively studied. Sender impact metric plays a significant role in sender selection, since it determines which senders are selected for transmission. Recent studies have shown that spatial link diversity has a significant impact on the efficiency of broadcast. However, the existing metrics overlook such impact. Besides, they consider only gains but ignore the costs of sender candidates. As a result, existing works cannot achieve accurate estimation of the sender impact. Moreover, they cannot well support data dissemination with network coding, which is commonly used for lossy environments. In this paper, we first propose a novel sender impact metric, namely, γ, which jointly exploits link quality and spatial link diversity to calculate the gain/cost ratio of the sender candidates. Then, we develop a generic sender selection scheme based on the γ metric (called γ-component) that can generally support both types of dissemination using native packets and network coding. Extensive evaluations are conducted through real testbed experiments and large-scale simulations. The performance results and analysis show that γ achieves far more accurate impact estimation than the existing works. In addition, the dissemination protocols based on γ-component outperform the existing protocols in terms of completion time and transmissions (by 20.5% and 23.1%, respectively).

Social networks often provide group features to help users with similar interests associate and consume content together. Recommending groups to users poses challenges due to their complex relationship: user-group affinity is typically measured implicitly and varies with time; similarly, group characteristics change as users join and leave. To tackle these challenges, we adapt existing matrix factorization techniques to learn user-group affinity based on two different implicit engagement metrics: (i) which group-provided content users consume; and (ii) which content users provide to groups. To capture the temporally extended nature of group engagement we implement a time-varying factorization. We test the assertion that latent preferences for groups and users are sparse in investigating elastic-net regularization. Experiments using data from DeviantArt indicate that the time-varying implicit engagement-based model provides the best top-K group recommendations, illustrating the benefit of the added model complexity.

Rapid prototyping of IoT platforms is essential for developers to obtain first-mover advantage, validate feasibility of innovative ideas and the key technologies. In this paper, we present TinyLink, an approach for rapid and cost-effective prototyping of IoT platforms. With TinyLink, developers can specify the key platform functionalities and let TinyLink deal with the details of hardware components. Then TinyLink creates user constraints from the functionalities, as well as the inherent hardware constraints. The high level goal is to automatically select the hardware components so that they can satisfy the user requirements with the lowest cost. TinyLink solves the optimization problem and outputs a list of hardware components. We implement TinyLink and evaluate it using real-world IoT platform requirements. Results show that TinyLink achieves a lower cost compared with an existing IoT application, without affecting the functionalities of the platform.

Air quality monitoring has attracted a lot of attention from governments, academia and industry, especially for PM2.5due to its significant impact on our respiratory systems. In this paper, we present the design, implementation, and evaluation of Mosaic, a low cost urban PM2.5 monitoring system based on mobile sensing. In Mosaic, a small number of air quality monitoring nodes are deployed on city buses to measure air quality. Current low-cost particle sensors based on light-scattering, however, are vulnerable to airflow disturbance on moving vehicles. In order to address this problem, we build our air quality monitoring nodes, Mosaic-Nodes, with a novel constructive airflow-disturbance design based on a carefully tuned airflow structure and a GPS-assisted filtering method. Further, the buses used for system deployment are selected by a novel algorithm which achieves both high coverage and low computation overhead. The collected sensor data is also used to calculate the PM2.5of locations without direct measurements by an existing inference model. We apply the Mosaic system in a testing urban area which includes more than 70 point-of-interests. Results show that the Mosaic system can accurately obtain the urban air quality with high coverage and low cost.

With email overload becoming a billion-level drag on the economy, personalized email prioritization is of urgent need to help predict the importance level of an email. Despite lots of previous effort on the topic, broadcast email, an important type of emails with its unique challenges and intriguing opportunities, has been overlooked. The most salient opportunity lies in that effective collaborative filtering can be exploited due to thousands of receivers of a typical broadcast email. However, every broadcast email is completely cold and it is very costly to obtain users' preference feedback. Fortunately, there exist up to million-level broadcast mailing lists in a real life email system. Similar mailing lists can provide useful extra information for broadcast email prioritization in a target mailing list. How to mine such useful extra information is a challenging problem that has never been touched. In this work, we propose the first broadcast email prioritization framework considering large numbers of mailing lists by formulating this problem as a cross domain recommendation problem. An optimization framework is proposed to select the optimal set of source domains considering multiple criteria including overlap of users, feedback pattern similarity and coverage of users. Our method is thoroughly evaluated on a real world industrial dataset from Samsung Electronics and is proved highly effective and outperforms all the baselines.

Email is one of the most important communication tools today, but email overload resulting from the large number of unimportant or irrelevant emails is causing trillion-level economy loss every year. Thus personalized email prioritization algorithms are of urgent need. Despite lots of previous effort on this topic, broadcast email, an important type of email, is overlooked in previous literature. Broadcast emails are significantly different from normal emails, introducing both new challenges and opportunities. On one hand, lack of real senders and limited user interactions invalidate the key features exploited by traditional email prioritization algorithms; on the other hand, thousands of receivers for one broadcast email bring us the opportunity to predict importance through collaborative filtering. However, broadcast emails face a severe cold-start problem which hinders the direct application of collaborative filtering. In this paper, we propose the first framework for broadcast email prioritization by designing a novel active learning model that considers the collaborative filtering, implicit feedback and time sensitive responsiveness features of broadcast emails. Our method is thoroughly evaluated on a large scale real world industrial dataset from Samsung Electronics. Our method is proved highly effective and outperforms state-of-the-art personalized email prioritization methods.

In wireless sensor networks, sensor nodes are usually self-organized, delivering data to a central sink in a multi-hop manner. Reconstructing the per-packet routing path enables fine-grained diagnostic analysis and performance optimizations of the network. The performances of existing path reconstruction approaches, however, degrade rapidly in large scale networks with lossy links. This paper presents Pathfinder, a robust path reconstruction method against packet losses as well as routing dynamics. At the node side, Pathfinder exploits temporal correlation between a set of packet paths and efficiently compresses the path information using path difference. At the sink side, Pathfinder infers packet paths from the compressed information and employs intelligent path speculation to reconstruct the packet paths with high reconstruction ratio. We propose a novel analytical model to analyze the performance of Pathfinder. We further evaluate Pathfinder compared with two most related approaches using traces from a large scale deployment and extensive simulations. Results show that Pathfinder outperforms existing approaches, achieving both high reconstruction ratio and low transmission cost.

Collaborative filtering (CF) is out of question the most widely adopted and successful recommendation approach. A typical CF-based recommender system associates a user with a group of like-minded users based on their individual preferences over all the items, either explicit or implicit, and then recommends to the user some unobserved items enjoyed by the group. However, we find that two users with similar tastes on one item subset may have totally different tastes on another set. In other words, there exist many user-item subgroups each consisting of a subset of items and a group of like-minded users on these items. It is more reasonable to predict preferences through one user's correlated subgroups, but not the entire user-item matrix. In this paper, to find meaningful subgroups, we formulate a new Multiclass Co-Clustering (MCoC) model, which captures relations of user-to-item, user-to-user, and item-to-item simultaneously. Then, we combine traditional CF algorithms with subgroups for improving their top-$N$recommendation performance. Our approach can be seen as a new extension of traditional clustering CF models. Systematic experiments on several real data sets have demonstrated the effectiveness of our proposed approach.

The notion of global time is of great importance for many sensor network applications. Time reconstruction methods aim to reconstruct the global time with respect to a reference clock. To achieve microsecond accuracy, MAC-layer timestamping is required for recording packet transmission and reception times. The timestamps, however, can be invalid due to multiple reasons, such as imperfect system designs, wireless corruptions, or timing attacks, etc. In this paper, we propose ART, an accurate and robust time reconstruction approach to detecting invalid timestamps and recovering the needed information. ART is much more accurate and robust than threshold-based approach, especially in dynamic networks with inherently varying propagation delays. We evaluate our approach in both testbed and a real-world deployment. Results show that: 1) ART achieves a high detection accuracy with low false-positive rate and low false-negative rate; 2) ART achieves a high recovery accuracy of less than 2 ms on average, much more accurate than previously reported results.

Recent wireless sensor networks (WSNs) are becoming increasingly complex with the growing network scale and the dynamic nature of wireless communications. Many measurement and diagnostic approaches depend on per-packet routing paths for accurate and fine-grained analysis of the complex network behaviors. In this paper, we propose iPath, a novel path inference approach to reconstructing the per-packet routing paths in dynamic and large-scale networks. The basic idea of iPath is to exploit high path similarity to iteratively infer long paths from short ones. iPath starts with an initial known set of paths and performs path inference iteratively. iPath includes a novel design of a lightweight hash function for verification of the inferred paths. In order to further improve the inference capability as well as the execution efficiency, iPath includes a fast bootstrapping algorithm to reconstruct the initial set of paths. We also implement iPath and evaluate its performance using traces from large-scale WSN deployments as well as extensive simulations. Results show that iPath achieves much higher reconstruction ratios under different network settings compared to other state-of-the-art approaches.

Inferring per-link metrics through aggregated path measurements, known as network tomography, is an effective way to facilitate various network operations, such as network monitoring, load balancing, and fault diagnosis. We study the problem of identifying additive link metrics of a set of interesting links from end-to-end cycle-free path measurements among selected monitors, i.e., preferential link tomography. Since assigning a node as a monitor usually requires non-negligible operational cost, we focus on assigning the minimum number of monitors (i.e., optimal monitor assignment) to identify all interesting links. By modeling the network as a connected graph, we propose Scalpel, a scalable preferential link tomography approach. Scalpel trims the original graph by a two-stage graph trimming algorithm and reuses an existing method to assign monitors in the trimmed graph. We theoretically prove Scalpel has several key properties: 1) the graph trimming algorithm in Scalpel is minimal in the sense that further trimming the graph does not reduce the number of monitors; 2) the obtained assignment is able to identify all interesting links in the original graph; and 3) an optimal monitor assignment in the graph after trimming is also an optimal monitor assignment in the original graph. We implement Scalpel and evaluate it based on both synthetic topologies and real network topologies. Compared with state-of-the-art, Scalpel reduces the number of monitors by 39.0% to 98.6% when 50% to 1% of all links are interesting links.

Detecting and diagnosing anomalies in networked embedded systems like sensor networks is a very difficult task, due to the variable workloads and severe resource constraints. In this paper, we focus on how to aid bug diagnosis after the system has been deployed. We notice that most node-level debugging tools can provide detailed program information inside the node but fail to detect when and where a problem occurs in the network. On the other hand, most network-level diagnosis tools can effectively detect a problem from the network but fail to narrow down the problem within the node because they lack detailed program information. To close the gap, we propose D2, a new method for post-deployment anomaly detection and diagnosis in networked embedded systems by combining program profiling and symptom mining. D2 employs binary instrumentation to perform lightweight function count profiling. Based on the statistics, D2 uses PCA (Principal Component Analysis) based approach for automatically detecting network anomalies. Compared with previous methods, D2 is able to point programmers closer to the most likely causes by a novel approach combining statistical tests and program call graph analysis. We implement our method based on TinyOS 2.1.1 and evaluate its effectiveness by case studies in the development of a working sensor network. Results show that our method can aid programmers to diagnose problems quickly in real-world sensor network systems, and at the same time, incurs an acceptable overhead to the running system.

Wireless Sensor Networks (WSNs) have been successfully applied in many application areas. Understanding the wireless link performance is very helpful for both protocol designers and network managers. Loss tomography is a popular approach to inferring the per-link loss ratios from end-to-end delivery ratios. Previous studies, however, are usually targeted for networks with static or slowly changing routing paths. In this work, we propose Dophy, a Dynamic loss tomography approach specifically designed for dynamic WSNs where each node dynamically selects the forwarding nodes towards the sink. The key idea of Dophy is based on an observation that most existing protocols use retransmissions to achieve high data delivery ratio. Dophy employs arithmetic encoding to compactly encode the number of retransmissions along the paths. Dophy incorporates two mechanisms to optimize its performance. First, Dophy intelligently reduces the size of symbol set by aggregating the number of retransmissions, reducing the encoding overhead significantly. Second, Dophy periodically updates the probability model to minimize the overall transmission overhead. We implement Dophy on the Tiny OS platform and evaluate its performance extensively using large-scale simulations. Results show that Dophy achieves both high encoding efficiency and high estimation accuracy. Comparative studies show that Dophy significantly outperforms traditional loss tomography approaches in terms of accuracy.

Wireless link correlation can greatly affect the performance of wireless protocols such as flooding, and opportunistic routing. Researchers have proposed a variety of approaches to optimize existing protocols exploiting link correlation. Most existing works directly measure link correlation using packet-level transmissions and receptions. Measurement alone is insufficient because it lacks predictive power and scalability. In this paper, we present CorModel, a model for predicting link correlation in low-power wireless networks. Based on the underlying causes of link correlation, we explore four easily measurable parameters for our modeling. Besides PHY-layer parameters that previous studies have explored, we find that network-layer parameters can also have significant impact on link correlation. We validate our model and illustrate its usefulness by integrating it into existing protocols for more accurate correlation estimation. Experimental results show that our model can significantly increase the accuracy of wireless link estimation, resulting in better protocol performance.

When evaluating the accessibility of a large website, we rely on sampling methods to reduce the cost of evaluation. This may lead to a biased evaluation when the distribution of checkpoint violations in a website is skewed and the selected samples do not provide a good representation of the entire website. To improve sampling quality, stratified sampling methods first cluster web pages in a site and then draw samples from each cluster. In existing stratified sampling methods, however, all the pages in a website need to be analyzed for clustering, causing huge I/O and computation costs. To address this issue, we propose a novel page sampling method based on URL clustering for web accessibility evaluation, namely URLSamp. Using only the URL information for stratified page sampling, URLSamp can efficiently scale to large websites. Meanwhile, by exploiting similarities in URL patterns, URLSamp cluster pages by their generating scripts and can thus effectively detect accessibility problems from web page templates. We use a data set of 45 web sites to validate our method. Experimental results show that our URLSamp method is both effective and efficient for web accessibility evaluation.

Energy efficiency is one of the most important factors that affect the applicability of wireless sensor networks (WSNs) in many practical scenarios. Many low-power media access control (MAC) protocols have been proposed in the past decade to improve the energy efficiency of sensor nodes. In these low-power MAC protocols, preambles are widely used to wake up the receivers asynchronizely. However, the data delivery potential of these preambles has not been exploited. In this paper, we propose a coded preamble (COPE), which exploits the data delivery potential of preambles by encoding the preambles by network coding. COPE has two salient features. First, a passive receiver set selection scheme enables nodes to decide whether to receive the overheard preamble packets, without introducing extra communication overhead. Second, COPE supports multiple routing primitives, such as unicast and broadcast, making it be a versatile 2.5 layer between the low-power link layer (layer 2) and the network layer (layer 3). We analyze COPE by a novel analytical model. Results show that COPE is able to improve the energy efficiency of both unicast and broadcast significantly. We also implement COPE in TinyOS/TelosB platform and evaluate its energy efficiency. Results show that COPE significantly reduces the radio-on-time in practical network settings.

Recent advances in object detection have led to the development of segmentation by detection approaches that integrate top-down geometric priors for multiclass object segmentation. A key yet under-addressed issue in utilizing top-down cues for the problem of multiclass object segmentation by detection is efficiently generating robust and accurate geometric priors. In this paper, we propose a random geometric prior forest scheme to obtain object-adaptive geometric priors efficiently and robustly. In the scheme, a testing object first searches for training neighbors with similar geometries using the random geometric prior forest, and then the geometry of the testing object is reconstructed by linearly combining the geometries of its neighbors. Our scheme enjoys several favorable properties when compared with conventional methods. First, it is robust and very fast because its inference does not suffer from bad initializations, poor local minimums or complex optimization. Second, the figure/ground geometries of training samples are utilized in a multitask manner. Third, our scheme is object-adaptive but does not require the labeling of parts or poselets, and thus, it is quite easy to implement. To demonstrate the effectiveness of the proposed scheme, we integrate the obtained top-down geometric priors with conventional bottom-up color cues in the frame of graph cut. The proposed random geometric prior forest achieves the best segmentation results of all of the methods tested on VOC2010/2012 and is 90 times faster than the current state-of-the-art method.

Random forest (RF) is a very important classifier with applications in various machine learning tasks, but its promising performance heavily relies on the size of labeled training data. In this paper, we investigate constructing of RFs with a small size of labeled data and find that the performance bottleneck is located in the node splitting procedures; hence, existing solutions fail to properly partition the feature space if there are insufficient training data. To achieve robust node splitting with insufficient data, we present semisupervised splitting to overcome this limitation by splitting nodes with the guidance of both labeled and abundant unlabeled data. In particular, an accurate quality measure of node splitting is obtained by carrying out the kernel-based density estimation, whereby a multiclass version of asymptotic mean integrated squared error criterion is proposed to adaptively select the optimal bandwidth of the kernel. To avoid the curse of dimensionality, we project the data points from the original high-dimensional feature space onto a low-dimensional subspace before estimation. A unified optimization framework is proposed to select a coupled pair of subspace and separating hyperplane such that the smoothness of the subspace and the quality of the splitting are guaranteed simultaneously. Our algorithm efficiently avoids overfitting caused by bad initialization and local maxima when compared with conventional margin maximization-based semisupervised methods. We demonstrate the effectiveness of the proposed algorithm by comparing it with state-of-the-art supervised and semisupervised algorithms for typical computer vision applications, such as object categorization, face recognition, and image segmentation, on publicly available data sets.

People often take photographs at tourist sites and these pictures usually have two main elements: a person in the foreground and scenery in the background. This type of “souvenir photo” is one of the most common photos clicked by tourists. Although algorithms that aid a user-photographer in taking a well-composed picture of a scene exist [Ni et al. 2013], few studies have addressed the issue of properly positioning human subjects in photographs. In photography, the common guidelines of composing portrait images exist. However, these rules usually do not consider the background scene. Therefore, in this article, we investigate human-scenery positional relationships and construct a photographic assistance system to optimize the position of human subjects in a given background scene, thereby assisting the user in capturing high-quality souvenir photos. We collect thousands of well-composed portrait photographs to learn human-scenery aesthetic composition rules. In addition, we define a set of negative rules to exclude undesirable compositions. Recommendation results are achieved by combining the first learned positive rule with our proposed negative rules. We implement the proposed system on an Android platform in a smartphone. The system demonstrates its efficacy by producing well-composed souvenir photos.

Graph-based ranking models have been widely applied in information retrieval area. In this paper, we focus on a well known graph-based model - the Ranking on Data Manifold model, or Manifold Ranking (MR). Particularly, it has been successfully applied to content-based image retrieval, because of its outstanding ability to discover underlying geometrical structure of the given image database. However, manifold ranking is computationally very expensive, which significantly limits its applicability to large databases especially for the cases that the queries are out of the database (new samples). We propose a novel scalable graph-based ranking model called Efficient Manifold Ranking (EMR), trying to address the shortcomings of MR from two main perspectives: scalable graph construction and efficient ranking computation. Specifically, we build an anchor graph on the database instead of a traditional$k$-nearest neighbor graph, and design a new form of adjacency matrix utilized to speed up the ranking. An approximate method is adopted for efficient out-of-sample retrieval. Experimental results on some large scale image databases demonstrate that EMR is a promising method for real world retrieval applications.

Monocular multiple-object tracking is a fundamental yet under-addressed computer vision problem. In this paper, we propose a novel learning framework for tracking multiple objects by detection. First, instead of heuristically defining a tracking algorithm, we learn that a discriminative structure prediction model from labeled video data captures the interdependence of multiple influence factors. Given the joint targets state from the last time step and the observation at the current frame, the joint targets state at the current time step can then be inferred by maximizing the joint probability score. Second, our detection results benefit from tracking cues. The traditional detection algorithms need a nonmaximal suppression postprocessing to select a subset from the total detection responses as the final output and a large number of selection mistakes are induced, especially under a congested circumstance. Our method integrates both detection and tracking cues. This integration helps to decrease the postprocessing mistake risk and to improve performance in tracking. Finally, we formulate the entire model training into a convex optimization problem and estimate its parameters using the cutting plane optimization. Experiments show that our method performs effectively in a large variety of scenarios, including pedestrian tracking in crowd scenes and vehicle tracking in congested traffic.

Nowadays many people are members of multiple online social networks simultaneously, such as Facebook, Twitter and some other instant messaging circles. But these networks are usually isolated from each other. Mapping common users across these social networks will benefit many applications. Methods based on username comparison perform well on parts of users, however they can not work in the following situations: (a) users choose different usernames in different networks; (b) a unique username corresponds to different individuals. In this paper, we propose to utilize social structures to improve the mapping performance. Specifically, a novel subspace learning algorithm, Manifold Alignment on Hypergraph (MAH), is proposed. Different from traditional semi-supervised manifold alignment methods, we use hypergraph to model high-order relations here. For a target user in one network, the proposed algorithm ranks all users in the other network by their possibilities of being the corresponding user. Moreover, methods based on username comparison can be incorporated into our algorithm easily to further boost the mapping accuracy. Experimental results have demonstrated the effectiveness of our proposed algorithm in mapping users across networks.

Social explanation, the statement with the form of "A and B also like the item", is widely used in almost all the major recommender systems in the web and effectively improves the persuasiveness of the recommendation results by convincing more users to try. This paper presents the first algorithm to generate the most persuasive social explanation by recommending the optimal set of users to be put in the explanation. New challenges like modeling persuasiveness of multiple users, different types of users in social network, sparsity of likes, are discussed in depth and solved in our algorithm. The extensive evaluation demonstrates the advantage of our proposed algorithm compared with traditional methods.

The desirability of being able to search for specific persons in surveillance videos captured by different cameras has increasingly motivated interest in the problem of person re-identification, which is a critical yet under-addressed challenge in multi-camera tracking systems. The main difficulty of person re-identification arises from the variations in human appearances from different camera views. In this paper, to bridge the human appearance variations across cameras, two coupled dictionaries that relate to the gallery and probe cameras are jointly learned in the training phase from both labeled and unlabeled images. The labeled training images carry the relationship between features from different cameras, and the abundant unlabeled training images are introduced to exploit the geometry of the marginal distribution for obtaining robust sparse representation. In the testing phase, the feature of each target image from the probe camera is first encoded by the sparse representation and then recovered in the feature space spanned by the images from the gallery camera. The features of the same person from different cameras are similar following the above transformation. Experimental results on publicly available datasets demonstrate the superiority of our method.

The desire of enabling computers to learn semantic concepts from large quantities of Internet videos has motivated increasing interests on semantic video understanding, while video segmentation is important yet challenging for understanding videos. The main difficulty of video segmentation arises from the burden of labeling training samples, making the problem largely unsolved. In this paper, we present a novel nearest neighbor-based label transfer scheme for weakly supervised video segmentation. Whereas previous weakly supervised video segmentation methods have been limited to the two-class case, our proposed scheme focuses on more challenging multiclass video segmentation, which finds a semantically meaningful label for every pixel in a video. Our scheme enjoys several favorable properties when compared with conventional methods. First, a weakly supervised hashing procedure is carried out to handle both metric and semantic similarity. Second, the proposed nearest neighbor-based label transfer algorithm effectively avoids overfitting caused by weakly supervised data. Third, a multi-video graph model is built to encourage smoothness between regions that are spatiotemporally adjacent and similar in appearance. We demonstrate the effectiveness of the proposed scheme by comparing it with several other state-of-the-art weakly supervised segmentation methods on one new Wild8 dataset and two other publicly available datasets.

In multi-hop wireless ad-hoc networks, packet delivery delay is one of the most important performance metrics. While a lot of research efforts have been spent on measuring and optimizing the end-to-end delay performance, there usually lack accurate and lightweight methods for decomposing the end-to-end delay into the per-hop delay for each packet. Knowledge on the per-hop per-packet delay can greatly improve the network visibility and facilitate network measurement and management. In this paper, we propose Domo, a passive, lightweight and accurate delay tomography approach to decomposing the packet end-to-end delay into each hop. The basic idea is to formulate the problem into a set of optimization problems by carefully considering the constraints among various timing quantities. At the network side, Domo attaches a small overhead to each packet for constructing constraints of the optimization problems. At the PC side, Domo employs semi-definite relaxation and several other methods to efficiently solve the optimization problems. We implement Domo and evaluate its performance extensively using large-scale simulations. Results show that Domo significantly outperforms two existing methods, nearly tripling the accuracy of the state-of-the-art.

With the development of imaging techniques and the Internet, it is hard to effectively and efficiently manage, index and store large amounts of videos. Video summarization appears to this need by obtaining useful information in videos. Recently, the reconstruction concept has been introduced into video summarization. However, most of these existing reconstruction methods where a dictionary of key frames is selected to reconstruct the original video best might come up with redundant information. In this paper, we propose a novel framework named Video Summarization based on Nonnegative Linear Reconstruction (VSNLR) which allows only additive, not subtractive, linear combinations. Our approach consists of two core steps: (1) we detect the shot boundary and choose a representative frame for every shot, then all the representative frames form the candidate set; (2) for every frame in the candidate set, VSNLR selects related frames to reconstruct the given frame by using the nonnegative linear reconstruction function. The key frames are selected by minimizing the sum of reconstruction errors. Experiments on a dataset and comparison to the state-of-art method demonstrate our advantage.

We study the problem of identifying additive and static link metrics of a set of interesting links in a communication network, by using end-to-end cycle-free path measurements among selected monitors. To uniquely identify the metrics of these interesting links, three questions should be addressed: monitor assignment (which nodes should serve as monitors), paths selection (which cycle-free paths connecting each pair of monitors will be used), and link metric calculation. Since assigning a node as a monitor usually requires non-negligible operational cost, we focus on assigning the minimum number of monitors (i.e., Optimal monitor assignment) to identify all interesting links. By modeling the network as a connected graph, we propose Scalpel, an efficient preferential link tomography approach. Scalpel trims the original graph by a two-stage graph trimming algorithm and reuses existing method to assign monitors in the trimmed graph. We theoretically prove Scalpel has several key properties: 1) the graph trimming algorithm in Scalpel is minimal in the sense that further trimming the graph cannot reduce the number of monitors, 2) the obtained assignment is able to identify all interesting links in the original graph, and 3) an optimal monitor assignment in the graph after trimming is also an optimal monitor assignment in the original graph. Extensive simulations based on both synthetic topologies and real network topologies show the effectiveness of Scalpel. Compared with state-of-the-art, our approach reduces the number of monitors by 39.0%~98.6% when 50%~1% of all links are interesting links.

We present Dylog, a dynamic logging facility for networked embedded systems. Dylog employs several techniques to enable lightweight and interactive logging. First, Dylog uses binary instrumentation for dynamically inserting or removing logging statements, enabling interactive debugging at the runtime. Second, Dylog incorporates an efficient storage system and log collection protocol for recording and transferring the logging messages. In particular, Dylog significantly reduces the communication cost by storing string identifiers and restoring them back to corresponding strings at the PC. Third, Dylog employs MAC layer timestamping and a linear clock model for reconstructing the synchronized time of the logging messages with a very high precision. We implement and evaluate Dylog on TinyOS 2.1.1/TelosB. Results show that Dylog incurs a reasonable overhead. Dylog can help gain great visibility into the system behaviors, and diagnose performance issues at the source code level.

We propose Plena, a packet length adaptation scheme exploiting recently proposed Error Estimating Codes for low power sensor networks. We carefully consider two low power MAC protocols, i.e., X-MAC and LPP, in modeling the energy consumptions of different packet lengths. Based on the modeling and analysis, we give closed form solutions for the optimal packet length in terms of energy efficiency. We employ the recently proposed error estimating codes (EEC) for estimating the bit error rate (BER). EEC allows for instant estimation of the BER in a partially correct packet, making it possible for predicting packet reception ratios (PRRs) at different packet lengths without the need of direct measurement in an iterative process. We evaluate Plena's computation overhead based on the commonly used TelosB nodes. Results show that the computation overhead of Plena is acceptably small. We also perform extensive simulations to study the performance of Plena under different configurations. Results show that Plena achieves better energy efficiency than existing works.

Bulk data dissemination is a basic building block for enabling software update and reprogramming in wireless sensor networks. The recent structure based approach looks promising for efficient dissemination since it facilitates transmission and sleep scheduling. However, a number of limitations exist in existing structured protocols. In this paper, we propose a correlated core based solution for efficient bulk data dissemination in wireless sensor networks. We propose an efficient backbone node selection algorithm to construct the core structure by exploiting link correlation. We also design a novel negotiation mechanism which greatly reduces the control message overhead as compared to the existing structured protocols. We conduct both simulation and testbed experiments, and the results show that our proposed solution outperforms the state-of-the-art in terms of both the number of transmissions and the completion time.

The notion of global time is of great importance for many sensor network applications. To achieve microsecond accuracy, MAC-level timestamping is required for recording packet transmission and reception times. The MAC-level timestamps, however, are known to be error-prone, especially with low power listening techniques. In this paper, we propose ART, an accurate and robust time reconstruction approach to detecting invalid timestamps and recovering the needed information. We evaluate our approach in both testbed and a real-world deployment. Results show ART is accurate and robust for deployed sensor networks.

We address the problem of removing video color tone jitter that is common in amateur videos recorded with hand-held devices. To achieve this, we introduce color state to represent the exposure and white balance state of a frame. The color state of each frame can be computed by accumulating the color transformations of neighboring frame pairs. Then, the tonal changes of the video can be represented by a time-varying trajectory in color state space. To remove the tone jitter, we smooth the original color state trajectory by solving an L1 optimization problem with PCA dimensionality reduction. In addition, we propose a novel selective strategy to remove small tone jitter while retaining extreme exposure and white balance changes to avoid serious artifacts. Quantitative evaluation and visual comparison with previous work demonstrate the effectiveness of our tonal stabilization method. This system can also be used as a preprocessing tool for other video editing methods.

Millions of users share their opinions on Twitter, making it a valuable platform for tracking and analyzing public sentiment. Such tracking and analysis can provide critical information for decision making in various domains. Therefore it has attracted attention in both academia and industry. Previous research mainly focused on modeling and tracking public sentiment. In this work, we move one step further to interpret sentiment variations. We observed that emerging topics (named foreground topics) within the sentiment variation periods are highly related to the genuine reasons behind the variations. Based on this observation, we propose a Latent Dirichlet Allocation (LDA) based model, Foreground and Background LDA (FB-LDA), to distill foreground topics and filter out longstanding background topics. These foreground topics can give potential interpretations of the sentiment variations. To further enhance the readability of the mined reasons, we select the most representative tweets for foreground topics and develop another generative model called Reason Candidate and Background LDA (RCB-LDA) to rank them with respect to their “popularity” within the variation period. Experimental results show that our methods can effectively find foreground topics and rank reason candidates. The proposed models can also be applied to other tasks such as finding topic differences between two sets of documents.

Recent advances in Microelectronic Mechanical Systems (MEMS) and wireless communication technologies have fostered the rapid development of networked embedded systems like wireless sensor networks. System software for these self-organizing systems often needs to be updated for a variety of reasons. We present a holistic software update (i.e., reprogramming) system called R3 for networked embedded systems. R3 has two salient features. First, the binary differencing algorithm within R3 (R3diff) ensures an optimal result in terms of the delta size under a configurable cost measure. Second, the similarity preserving method within R3 (R3sim) optimizes the binary code format for achieving a large similarity with a small metadata overhead. Overall, R3 achieves the smallest delta size compared with other software update approaches such as Stream, Rsync, RMTD, Zephyr, Hermes, and R2 (e.g., 50%--99% reduction compared to Stream and about 20%--40% reduction compared to R2). R3’s implementation on TelosB/TinyOS is lightweight and efficient. We release our code at http://code.google.com/p/r3-dongw.

Wireless reprogramming is a crucial technique for software deployment in wireless sensor networks (WSNs). Code dissemination is a basic building block to enable wireless reprogramming. We present ECD, an Efficient Code Dissemination protocol leveraging 1-hop link quality information based on the TinyOS platform. Compared to prior works, ECD has three salient features. First, it supports dynamically configurable packet sizes. By increasing the packet size for high PHY rate radios, it significantly improves the transmission efficiency. Second, it employs an accurate sender selection algorithm to mitigate transmission collisions and transmissions over poor links. Third, it employs a simple impact-based backoff timer design to shorten the time spent in coordinating multiple eligible senders so that the largest impact sender is most likely to transmit. We implement ECD based on TinyOS and evaluate its performance extensively via testbed experiments and simulations. Results show that ECD outperforms state-of-the-art protocols, Deluge and MNP, in terms of completion time and data traffic (e.g., about 20 percent less traffic and 20-30 percent shorter completion time compared to Deluge).

Previous packet length optimizations for sensor networks often employ a fixed optimal length scheme, while in this study we present DPLC, a Dynamic Packet Length Control scheme. To make DPLC more efficient in terms of channel utilization, we incorporate a lightweight and accurate link estimation method. We further provide two easy-to-use services, i.e., small message aggregation and large message fragmentation, to facilitate upper-layer application programming. The implementation of DPLC based on TinyOS 2.1 is lightweight, with respect to computation, memory, and header overhead. Our experiments using a real indoor testbed running CTP show that DPLC achieves the best performance compared with previous works.

Node splitting is an important issue in Random Forest but robust splitting requires a large number of training samples. Existing solutions fail to properly partition the feature space if there are insufficient training data. In this paper, we present semi-supervised splitting to overcome this limitation by splitting nodes with the guidance of both labeled and unlabeled data. In particular, we derive a nonparametric algorithm to obtain an accurate quality measure of splitting by incorporating abundant unlabeled data. To avoid the curse of dimensionality, we project the data points from the original high-dimensional feature space onto a low-dimensional subspace before estimation. A unified optimization framework is proposed to select a coupled pair of subspace and separating hyper plane such that the smoothness of the subspace and the quality of the splitting are guaranteed simultaneously. The proposed algorithm is compared with state-of-the-art supervised and semi-supervised algorithms for typical computer vision applications such as object categorization and image segmentation. Experimental results on publicly available datasets demonstrate the superiority of our method.

Weakly supervised image segmentation is a challenging problem in computer vision field. In this paper, we present a new weakly supervised image segmentation algorithm by learning the distribution of spatially structured super pixel sets from image-level labels. Specifically, we first extract graph lets from each image where a graph let is a small-sized graph consisting of super pixels as its nodes and it encapsulates the spatial structure of those super pixels. Then, a manifold embedding algorithm is proposed to transform graph lets of different sizes into equal-length feature vectors. Thereafter, we use GMM to learn the distribution of the post-embedding graph lets. Finally, we propose a novel image segmentation algorithm, called graph let cut, that leverages the learned graph let distribution in measuring the homogeneity of a set of spatially structured super pixels. Experimental results show that the proposed approach outperforms state-of-the-art weakly supervised image segmentation methods, and its performance is comparable to those of the fully supervised segmentation models.

With the rapidly increasing number of personal image collections on the web, it is of great importance to annotate these user-uploaded images in personalized manner. But personalized image annotation is largely ignored by the mainstream of image annotation research. In this paper, we focus on personalizing the automatic image annotation by proposing a general framework which jointly exploits the generic content-based image annotation, personal image tagging history and the content of personal history images. In our framework, two sets of candidate annotations are extracted for each image based on content-based annotation and personal image tagging history. Considering that the user's interest may not stay the same, when exploiting the personal image tagging history, we also take the content of personal history images into account to avoid the noise. To get the final annotations, we propose an unsupervised algorithm based on reinforcement learning to combine the above two candidate annotation sets. Encouraging results show that the proposed framework is effective and promising for personalizing automatic image annotation.

In wireless sensor networks, sensor nodes are usually self-organized, delivering data to a central sink in a multi-hop manner. Reconstructing the per-packet routing path enables fine-grained diagnostic analysis and performance optimizations of the network. The performances of existing path reconstruction approaches, however, degrade rapidly in large scale networks with lossy links. In this paper, we propose Pathfinder, a robust path reconstruction method against packet losses as well as routing dynamics. At the node side, Pathfinder exploits temporal correlation between a set of packet paths and efficiently compresses the path information using path difference. At the PC side, Pathfinder infers packet paths from the compressed information and employs intelligent path speculation to reconstruct the packet paths with high reconstruction ratio. We evaluate several variations of Pathfinder as well as two most related approaches using traces from a large scale deployment and extensive simulations. Results show that Pathfinder outperforms existing approaches, achieving both high reconstruction ratio and low transmission overhead.

Reprogramming over the air is important for maintaining a wireless sensor network. Traditional reprogramming approaches assume always-on link layers. Given the energy limitation of sensor nodes, always-on link layers are often not desired for most sensor network applications. In this paper, we propose ROLP, a novel reprogramming protocol built on a widely used low power link layer in wireless sensor networks. ROLP employs an efficient control packets self-suppression scheme for reliable data transmission. ROLP also employs an adaptively falling asleep scheme based on the neighbor information to reduce the energy consumption. We implement ROLP based on TinyOS and evaluate its performance in two different indoor networks. Compared with the standard reprogramming protocol in TinyOS, ROLP is able to reduce the radio-on time by 57.6% and 39.0% in the two networks. Since radio operations cost most of the energy, these reductions save significant amount of energy and prolong the lifetime of a wireless sensor network.

Detecting and diagnosing anomalies in networked embedded systems like sensor networks is a very difficult task, due to the variable workloads and severe resource constraints. We notice that most node-level debugging tools can provide detailed program information inside the node but fail to detect when and where a problem occurs in the network. On the other hand, most network-level diagnosis tools can effectively detect a problem from the network but fail to narrow down the problem within the node because they lack detailed program information. To close the gap, we propose D2, a new anomaly detection and diagnosis method by combining program profiling and symptom mining. D2 employs binary instrumentation to perform lightweight function count profiling. Based on the statistics, D2 uses PCA (Principal Component Analysis) based approach for automatically detecting network anomalies. Compared to previous methods, D2 is able to point programmers closer to the most likely causes by a novel approach combining statistical tests and program call graph analysis. We implement our method based on TinyOS 2.1.1 and evaluate its effectiveness by case studies in the development of a working sensor network. Results show that our method is effective for detecting and diagnosing problems in real-world sensor network systems, and at the same time, incurs an acceptable overhead.

Nowadays, micro-blogging systems like Twitter have become one of the most important ways for information sharing. In Twitter, a user posts a message (tweet) and the others can forward the message (retweet). Mention is a new feature in micro-blogging systems. By mentioning users in a tweet, they will receive notifications and their possible retweets may help to initiate large cascade diffusion of the tweet. To enhance a tweet's diffusion by finding the right persons to mention, we propose in this paper a novel recommendation scheme named as whom-to-mention. Specifically, we present an in-depth study of mention mechanism and propose a recommendation scheme to solve the essential question of whom to mention in a tweet. In this paper, whom-to-mention is formulated as a ranking problem and we try to address several new challenges which are not well studied in the traditional information retrieval tasks. By adopting features including user interest match, content-dependent user relationship and user influence, a machine learned ranking function is trained based on newly defined information diffusion based relevance. The extensive evaluation using data gathered from real users demonstrates the advantage of our proposed algorithm compared with the traditional recommendation methods.

We present R2, an incremental reprogramming approach using relocatable code, to improve program similarity for efficient incremental reprogramming in networked embedded systems. R2 achieves a higher degree of similarity than existing approaches by mitigating effects of both function shifts and data shifts. R2 adopts a content-aware differencing algorithm to generate small delta files for efficient dissemination. Besides, it makes efficient use of memory and does not degrade program quality. We implement R2 based on TinyOS 2.1 and demonstrate its advantages through detailed analysis of TinyOS examples. We also present case studies on the software programs of a large-scale sensor system GreenOrbs. Results show that R2 reduces the dissemination cost by approximately 65 percent compared to state-of-the-art network reprogramming approach Deluge, and reduces the dissemination cost by approximately 20 percent compared to Zephyr and Hermes the latest works on incremental reprogramming.

Existing mechanisms for handover authentication mainly focus on designing a secure authentication module, little attention has been paid to protect users' privacy when they are authenticated by the access points for data access. Further, most existing approaches do not support user revocation. In this paper, we present a secure and efficient authentication protocol named Handauth. Similar to the mechanisms of this field, Handauth provides user authentication and session key establishment. However, compared to other well-known approaches, Handauth not only enjoys both computation and communication efficiency, but also achieves strong user anonymity and untraceablility, forward secure user revocation, conditional privacy-preservation, AAA server anonymity, access service expiration management, access point authentication, easily scheduled revocation, dynamic user revocation and attack resistance. Experimental results show that the proposed approach is feasible for real applications.

Coclustering targets on grouping the samples (e.g.,documents and users) and the features (e.g., words and ratings) simultaneously. It employs the dual relation and the bilateral information between the samples and features. In many real-world applications, data usually reside on a submanifold of the ambient Euclidean space, but it is nontrivial to estimate the intrinsic manifold of the data space in a principled way. In this paper, we focus on improving the coclustering performance via manifold ensemble learning, which is able to maximally approximate the intrinsic manifolds of both the sample and feature spaces. To achieve this, we develop a novel coclustering algorithm called relational multimanifold coclustering based on symmetric nonnegative matrix trifactorization, which decomposes the relational data matrix into three submatrices. This method considers the intertype relationship revealed by the relational data matrix and also the intratype information reflected by the affinity matrices encoded on the sample and feature data distributions. Specifically, we assume that the intrinsic manifold of the sample or feature space lies in a convex hull of some predefined candidate manifolds. We want to learn a convex combination of them to maximally approach the desired intrinsic manifold. To optimize the objective function, the multiplicative rules are utilized to update the submatrices alternatively. In addition, both the entropic mirror descent algorithm and the coordinate descent algorithm are exploited to learn the manifold coefficient vector. Extensive experiments on documents, images, and gene expression data sets have demonstrated the superiority of the proposed algorithm compared with other well-established methods.

3-D facial expression generation, including synthesis and retargeting, has received intensive attentions in recent years, because it is important to produce realistic 3-D faces with specific expressions in modern film production and computer games. In this paper, we present joint sparse learning (JSL) to learn mapping functions and their respective inverses to model the relationship between the high-dimensional 3-D faces (of different expressions and identities) and their corresponding low-dimensional representations. Based on JSL, we can effectively and efficiently generate various expressions of a 3-D face by either synthesizing or retargeting. Furthermore, JSL is able to restore 3-D faces with holes by learning a mapping function between incomplete and intact data. Experimental results on a wide range of 3-D faces demonstrate the effectiveness of the proposed approach by comparing with representative ones in terms of quality, time cost, and robustness.

As one of the most basic photo manipulation processes, photo cropping is widely used in the printing, graphic design, and photography industries. In this paper, we introduce graphlets (i.e., small connected subgraphs) to represent a photo's aesthetic features, and propose a probabilistic model to transfer aesthetic features from the training photo onto the cropped photo. In particular, by segmenting each photo into a set of regions, we construct a region adjacency graph (RAG) to represent the global aesthetic feature of each photo. Graphlets are then extracted from the RAGs, and these graphlets capture the local aesthetic features of the photos. Finally, we cast photo cropping as a candidate-searching procedure on the basis of a probabilistic model, and infer the parameters of the cropped photos using Gibbs sampling. The proposed method is fully automatic. Subjective evaluations have shown that it is preferred over a number of existing approaches.

A body sensor network (BSN) is a wireless network of biosensors and a local processing unit, which is commonly referred to as the personal wireless hub (PWH). Personal health information (PHI) is collected by biosensors and delivered to the PWH before it is forwarded to the remote healthcare center for further processing. In a BSN, it is critical to only admit eligible biosensors and PWH into the network. Also, securing the transmission from each biosensor to PWH is essential not only for ensuring safety of PHI delivery, but also for preserving the privacy of PHI. In this paper, we present the design, implementation, and evaluation of a secure network admission and transmission subsystem based on a polynomial-based authentication scheme. The procedures in this subsystem to establish keys for each biosensor are communication efficient and energy efficient. Moreover, based on the observation that an adversary eavesdropping in a BSN faces inevitable channel errors, we propose to exploit the adversary's uncertainty regarding the PHI transmission to update the individual key dynamically and improve key secrecy. In addition to the theoretical analysis that demonstrates the security properties of our system, this paper also reports the experimental results of the proposed protocol on resource-limited sensor platforms, which show the efficiency of our system in practice.

In this paper, we study the effective semi-supervised hashing method under the framework of regularized learning-based hashing. A nonlinear hash function is introduced to capture the underlying relationship among data points. Thus, the dimensionality of the matrix for computation is not only independent from the dimensionality of the original data space but also much smaller than the one using linear hash function. To effectively deal with the error accumulated during converting the real-value embeddings into the binary code after relaxation, we propose a semi-supervised nonlinear hashing algorithm using bootstrap sequential projection learning which effectively corrects the errors by taking into account of all the previous learned bits holistically without incurring the extra computational overhead. Experimental results on the six benchmark data sets demonstrate that the presented method outperforms the state-of-the-art hashing algorithms at a large margin.

Wireless sensor networks (WSNs) can be successfully applied in a wide range of applications. Efficient data dissemination is a fundamental service which enables many useful high-level functions such as parameter reconfiguration, network reprogramming, etc. Many current data dissemination protocols employ network coding techniques to deal with packet losses. The coding overhead, however, becomes a bottleneck in terms of dissemination delay. We exploit the concurrency potential of sensor nodes and propose MT-Deluge, a multithreaded design of a coding-based data dissemination protocol. By separating the coding and radio operations into two threads and carefully scheduling their executions, MT-Deluge shortens the dissemination delay effectively. An incremental decoding algorithm is employed to further improve MT-Deluge's performance. Experiments with 24 TelosB motes on four representative topologies show that MT-Deluge shortens the dissemination delay by 25.5-48.6 percent compared to a typical data dissemination protocol while keeping the merits of loss resilience.

Document summarization is of great value to many real world applications, such as snippets generation for search results and news headlines generation. Traditionally, document summarization is implemented by extracting sentences that cover the main topics of a document with a minimum redundancy. In this paper, we take a different perspective from data reconstruction and propose a novel framework named Document Summarization based on Data Reconstruction (DSDR). Specifically, our approach generates a summary which consist of those sentences that can best reconstruct the original document. To model the relationship among sentences, we introduce two objective functions: (1) linear reconstruction, which approximates the document by linear combinations of the selected sentences; (2) nonnegative linear reconstruction, which allows only additive, not subtractive, linear combinations. In this framework, the reconstruction error becomes a natural criterion for measuring the quality of the summary. For each objective function, we develop an efficient algorithm to solve the corresponding optimization problem. Extensive experiments on summarization benchmark data sets DUC 2006 and DUC 2007 demonstrate the effectiveness of our proposed approach.

Recently, graph-based ranking algorithms have received considerable interests in machine learning, computer vision and information retrieval communities. Ranking on data manifold (or manifold ranking, MR) is one of the representative approaches. One of the limitations of manifold ranking is its high computational complexity (O(n3), where n is the number of samples in database). In this paper, we cast the manifold ranking into a Bregman divergence optimization framework under which we transform the original MR to an equivalent optimal kernel matrix learning problem.With this new formulation, two effective and efficient extensions are proposed to enhance the ranking performance. Extensive experimental results on two real world image databases show the effectiveness of the proposed approach.

In this paper, we study the problem of large-scale Kernel Logistic Regression (KLR). A straightforward approach is to apply stochastic approximation to KLR. We refer to this approach as non-conservative online learning algorithm because it updates the kernel classifier after every received training example, leading to a dense classifier. To improve the sparsity of the KLR classifier, we propose two conservative online learning algorithms that update the classifier in a stochastic manner and generate sparse solutions. With appropriately designed updating strategies, our analysis shows that the two conservative algorithms enjoy similar theoretical guarantee as that of the non-conservative algorithm. Empirical studies on several benchmark data sets demonstrate that compared to batch-mode algorithms for KLR, the proposed conservative online learning algorithms are able to produce sparse KLR classifiers, and achieve similar classification accuracy but with significantly shorter training time. Furthermore, both the sparsity and classification accuracy of our methods are comparable to those of the online kernel SVM.

Co-clustering targets on grouping the samples and features simultaneously. It takes advantage of the duality between the samples and features. In many real-world applications, the data points or features usually reside on a submanifold of the ambient Euclidean space, but it is nontrivial to estimate the intrinsic manifolds in a principled way. In this study, we focus on improving the co-clustering performance via manifold ensemble learning, which aims to maximally approximate the intrinsic manifolds of both the sample and feature spaces. To achieve this, we develop a novel co-clustering algorithm called Relational Multi-manifold Co-clustering (RMC) based on symmetric nonnegative matrix tri-factorization, which decomposes the relational data matrix into three matrices. This method considers the inter-type relationship revealed by the relational data matrix and the intra-type information reflected by the affinity matrices. Specifically, we assume the intrinsic manifold of the sample or feature space lies in a convex hull of a group of pre-defined candidate manifolds. We hope to learn an appropriate convex combination of them to approach the desired intrinsic manifold. To optimize the objective, the multiplicative rules are utilized to update the factorized matrices and the entropic mirror descent algorithm is exploited to automatically learn the manifold coefficients. Experimental results demonstrate the superiority of the proposed algorithm.

Previous video stabilization methods often employ homographies to model transitions between consecutive frames, or require robust long feature tracks. However, the homography model is invalid for scenes with significant depth variations, and feature point tracking is fragile in videos with textureless objects, severe occlusion or camera rotation. To address these challenging cases, we propose to solve video stabilization with an additional depth sensor such as the Kinect camera. Though the depth image is noisy, incomplete and low resolution, it facilitates both camera motion estimation and frame warping, which make the video stabilization a much well posed problem. The experiments demonstrate the effectiveness of our algorithm.

Wireless reprogramming is a crucial technique for managing large-scale wireless sensor networks (WSNs). It is, however, energy intensive to disseminate the code to enable reprogramming. Incremental reprogramming is a promising approach to reduce the dissemination cost. In incremental reprogramming, only the delta between the new code and the old code needs to be disseminated, resulting much less energy consumption. The differencing algorithm plays a key role in incremental reprogramming. It takes inputs of two successive versions of codes and generates a small delta script for dissemination. Existing incremental algorithms have several limitations. First, they do not ensure the smallest delta size for dissemination. Second, some of them may incur a large overhead in terms of execution time and memory consumption. To address these issues, we propose DASA, an efficient differencing algorithm based on suffix array. DASA performs byte-level comparison and ensure the optimal result in terms of the delta size. Moreover, DASA has a low execution overhead. The time complexity and space complexity of DASA are O(n log n) and O(n), respectively. To the best of our knowledge, DASA is the optimal algorithm with the lowest time and space complexity for reprogramming WSNs.

Recently, the task of unsupervised face-name association has received a considerable interests in multimedia and information retrieval communities. It is quite different with the generic facial image annotation problem because of its unsupervised and ambiguous assignment properties. Specifically, the task of face-name association should obey the following three constraints: (1) a face can only be assigned to a name appearing in its associated caption or to null; (2) a name can be assigned to at most one face; and (3) a face can be assigned to at most one name. Many conventional methods have been proposed to tackle this task while suffering from some common problems, eg, many of them are computational expensive and hard to make the null assignment decision. In this paper, we design a novel framework named face-name association via commute distance (FACD), which judges face-name and face-null assignments under a unified framework via commute distance (CD) algorithm. Then, to further speed up the on-line processing, we propose a novel anchor-based commute distance (ACD) algorithm whose main idea is using the anchor point representation structure to accelerate the eigen-decomposition of the adjacency matrix of a graph. Systematic experiment results on a large scale and real world image-caption database with a total of 194,046 detected faces and 244,725 names show that our proposed approach outperforms many state-of-the-art methods in performance. Our framework is appropriate for a large scale and real-time system.

Collaborative filtering (CF) is one of the most successful recommendation approaches. It typically associates a user with a group of like-minded users based on their preferences over all the items, and recommends to the user those items enjoyed by others in the group. However we find that two users with similar tastes on one item subset may have totally different tastes on another set. In other words, there exist many user-item subgroups each consisting of a subset of items and a group of like-minded users on these items. It is more natural to make preference predictions for a user via the correlated subgroups than the entire user-item matrix. In this paper, to find meaningful subgroups, we formulate the Multiclass Co-Clustering (MCoC) problem and propose an effective solution to it. Then we propose an unified framework to extend the traditional CF algorithms by utilizing the subgroups information for improving their top-N recommendation performance. Our approach can be seen as an extension of traditional clustering CF models. Systematic experiments on three real world data sets have demonstrated the effectiveness of our proposed approach.

ABSTRACT Dynamic program update protocols provide a convenient way to reprogram sensor nodes after deployment. However, designing a secure program update protocol for wireless sensor networks is a difficult task because wireless networks are susceptible to attacks and nodes have limited resources. Recently, two secure program update protocols using orthogonality principle have been found to be vulnerable to two impersonation attacks, although these attacks are rather restrictive. This paper reports one new attack that is more general and makes the program update protocols even more vulnerable. With this attack, an attacker can easily impersonate the base station to install his/her preferred program on sensor nodes and then obtain control over the network. As a remedy, two simple countermeasures are suggested to defend against all these attacks. Finally, the security properties of the two proposed solutions are formally validated by a model checking tool. Copyright © 2011 John Wiley & Sons, Ltd.

ABSTRACT To allow many users to hold a secure teleconference in mobile networks, a secure conference scheme with dynamic participation is necessary. However, designing a secure and efficient conference scheme is a difficult task because wireless networks are susceptible to attacks and wireless devices have limited resources. Recently, a lightweight and secure conference scheme has been suggested. Later, it has been found that this solution has security weaknesses and a modified version to overcome them has been presented. Compared with other conference schemes, these two schemes have many advantages. In this short paper, security study of these conference schemes in mobile networks has been performed with the following findings: (1) both the original scheme and the modified version are still vulnerable to our proposed impersonation attack; (2) they lack a mechanism to confirm the delivery of relevant messages, leading to protocol disruption. Therefore, these two schemes cannot be deployed for the real world applications without further development. Then, some efficient countermeasures are given for enhancing the security of both schemes. Further, the security properties of the improved protocol are formally validated by a model checking tool called AVISPA. Finally, several basic principles are suggested for the design of a secure conference scheme. Copyright © 2011 John Wiley & Sons, Ltd.

The luminance of a natural scene is often of high dynamic range (HDR). In this paper, we propose a new scheme to handle HDR scenes by integrating locally adaptive scene detail capture and suppressing gradient reversals introduced by the local adaptation. The proposed scheme is novel for capturing an HDR scene by using a standard dynamic range (SDR) device and synthesizing an image suitable for SDR displays. In particular, we use an SDR capture device to record scene details (i.e., the visible contrasts and the scene gradients) in a series of SDR images with different exposure levels. Each SDR image responds to a fraction of the HDR and partially records scene details. With the captured SDR image series, we first calculate the image luminance levels, which maximize the visible contrasts, and then the scene gradients embedded in these images. Next, we synthesize an SDR image by using a probabilistic model that preserves the calculated image luminance levels and suppresses reversals in the image luminance gradients. The synthesized SDR image contains much more scene details than any of the captured SDR image. Moreover, the proposed scheme also functions as the tone mapping of an HDR image to the SDR image, and it is superior to both global and local tone mapping operators. This is because global operators fail to preserve visual details when the contrast ratio of a scene is large, whereas local operators often produce halos in the synthesized SDR image. The proposed scheme does not require any human interaction or parameter tuning for different scenes. Subjective evaluations have shown that it is preferred over a number of existing approaches.

Reconstruction of a 3-D face model from a single 2-D face image is fundamentally important for face recognition and animation because the 3-D face model is invariant to changes of viewpoint, illumination, background clutter, and occlusions. Given a coupled training set that contains pairs of 2-D faces and the corresponding 3-D faces, we train a novel coupled radial basis function network (C-RBF) to recover the 3-D face model from a single 2-D face image. The C-RBF network explores: 1) the intrinsic representations of 3-D face models and those of 2-D face images; 2) mappings between a 3-D face model and its intrinsic representation; and 3) mappings between a 2-D face image and its intrinsic representation. Since a particular face can be reconstructed by its nearest neighbors, we can assume that the linear combination coefficients for a particular 2-D face image reconstruction are identical to those for the corresponding 3-D face model reconstruction. Therefore, we can reconstruct a 3-D face model by using a single 2-D face image based on the C-RBF network. Extensive experimental results on the BU3D database indicate the effectiveness of the proposed C-RBF network for recovering the 3-D face model from a single 2-D face image.

The goal of feature selection is to identify the most informative features for compact representation, whereas the goal of active learning is to select the most informative instances for prediction. Previous studies separately address these two problems, despite of the fact that selecting features and instances are dual operations over a data matrix. In this paper, we consider the novel problem of simultaneously selecting the most informative features and instances and develop a solution from the perspective of optimum experimental design. That is, by using the selected features as the new representation and the selected instances as training data, the variance of the parameter estimate of a learning function can be minimized. Specifically, we propose a novel approach, which is called Unified criterion for Feature and Instance selection (UFI), to simultaneously identify the most informative features and instances that minimize the trace of the parameter covariance matrix. A greedy algorithm is introduced to efficiently solve the optimization problem. Experimental results on two benchmark data sets demonstrate the effectiveness of our proposed method.

Wireless medical sensor networks (MSNs) enable ubiquitous health monitoring of users during their everyday lives, at health sites, without restricting their freedom. Establishing trust among distributed network entities has been recognized as a powerful tool to improve the security and performance of distributed networks such as mobile ad hoc networks and sensor networks. However, most existing trust systems are not well suited for MSNs due to the unique operational and security requirements of MSNs. Moreover, similar to most security schemes, trust management methods themselves can be vulnerable to attacks. Unfortunately, this issue is often ignored in existing trust systems. In this paper, we identify the security and performance challenges facing a sensor network for wireless medical monitoring and suggest it should follow a two-tier architecture. Based on such an architecture, we develop an attack-resistant and lightweight trust management scheme named ReTrust. This paper also reports the experimental results of the Collection Tree Protocol using our proposed system in a network of TelosB motes, which show that ReTrust not only can efficiently detect malicious/faulty behaviors, but can also significantly improve the network performance in practice.

The development of medical sensor networks (MSNs) is imperative for e-healthcare, but security remains a formidable challenge yet to be resolved. Traditional cryptographic mechanisms do not suffice given the unique characteristics of MSNs, and the fact that MSNs are susceptible to a variety of node misbehaviors. In such situations, the security and performance of MSNs depend on the cooperative and trust nature of the distributed nodes, and it is important for each node to evaluate the trustworthiness of other nodes. In this paper, we identify the unique features of MSNs and introduce relevant node behaviors, such as transmission rate and leaving time, into trust evaluation to detect malicious nodes. We then propose an applicationindependent and distributed trust evaluation model for MSNs. The trust management is carried out through the use of simple cryptographic techniques. Simulation results demonstrate that the proposed model can be used to effectively identify malicious behaviors and thereby exclude malicious nodes. This paper also reports the experimental results of the Collection Tree Protocol with the addition of our proposed model in a network of TelosB motes, which show that the network performance can be significantly improved in practice. Further, some suggestions are given on how to employ such a trust evaluation model in some application scenarios.

Different from traditional one-sided clustering techniques, coclustering makes use of the duality between samples and features to partition them simultaneously. Most of the existing co-clustering algorithms focus on modeling the relationship between samples and features, whereas the intersample and interfeature relationships are ignored. In this paper, we propose a novel coclustering algorithm named Locally Discriminative Coclustering (LDCC) to explore the relationship between samples and features as well as the intersample and interfeature relationships. Specifically, the sample-feature relationship is modeled by a bipartite graph between samples and features. And we apply local linear regression to discovering the intrinsic discriminative structures of both sample space and feature space. For each local patch in the sample and feature spaces, a local linear function is estimated to predict the labels of the points in this patch. The intersample and interfeature relationships are thus captured by minimizing the fitting errors of all the local linear functions. In this way, LDCC groups strongly associated samples and features together, while respecting the local structures of both sample and feature spaces. Our experimental results on several benchmark data sets have demonstrated the effectiveness of the proposed method.

Seamless handover over multiple access points is highly desirable to mobile nodes, but ensuring security and efficiency of this process is challenging. This paper shows that prior handover authentication schemes incur high communication and computation costs, and are subject to a few security attacks. Further, a novel handover authentication protocol named PairHand is proposed. PairHand uses pairing-based cryptography to secure handover process and to achieve high efficiency. Also, an efficient batch signature verification scheme is incorporated into PairHand. Experiments using our implementation on laptop PCs show that PairHand is feasible in real applications.

Code dissemination in a wireless sensor network (WSN) is the process of propagating a new program image or relevant commands to sensor nodes. As a WSN is usually deployed in hostile environments, secure code dissemination is and will continue to be a major concern. Most code dissemination protocols are based on the centralized approach in which only the base station has the authority to initiate code dissemination. However, it is desirable and sometimes necessary to disseminate code images in a distributed manner which allows multiple authorized network users to simultaneously and directly update code images on different nodes without involving the base station. Motivated by this consideration, we develop a secure and distributed code dissemination protocol named DiCode. A salient feature of DiCode is its ability to resist denial-of-service attacks which have severe consequences on network availability. Further, the security properties of our protocol are demonstrated by theoretical analysis. To verify the efficiency of the proposed approach in practice, we also implement the proposed mechanism in a network of resource-constrained sensor nodes.

The most critical challenge for the recommendation system is to achieve the high prediction quality on the large scale sparse data contributed by the users. In this paper, we present a novel approach to the social recommendation problem, which takes the advantage of the graph Laplacian regularization to capture the underlying social relationship among the users. Differently from the previous approaches, that are based on the conventional gradient descent optimization, we formulate the presented graph Laplacian regularized social recommendation problem into a low-rank semidefinite program, which is able to be efficiently solved by the quasi-Newton algorithm. We have conducted the empirical evaluation on a large scale dataset of high sparsity, the promising experimental results show that our method is very effective and efficient for the social recommendation task.

Wireless reprogramming is a crucial technique for software deployment in wireless sensor networks (WSNs). Code dissemination is a basic building block to enable wireless repro-gramming. We present ECD, an Efficient Code Dissemination protocol leveraging 1-hop link quality information. Compared to prior works, ECD has three salient features. First, it supports dynamically configurable packet sizes. By increasing the packet size for high PHY rate radios, it significantly improves the transmission efficiency. Second, it employs an accurate sender selection algorithm to mitigate transmission collisions and transmissions over poor links. Third, it employs a simple impact-based backoff timer design to shorten the time spent in coordinating multiple eligible senders so that the largest impact sender is most likely to transmit. We implement ECD based on TinyOS and evaluate its performance extensively. Testbed experiments show that ECD outperforms state-of-the-art protocols, Deluge and MNP, in terms of completion time and data traffic. (e.g., about 20% less traffic and 20-30% shorter completion time compared to Deluge).

We present ℛ2, an incremental ℛeprogramming approach using Relocatable code, to improve program similarity for efficient incremental reprogramming in networked embedded systems. ℛ2achieves a higher degree of similarity than existing approaches by mitigating the effects of both function shifts and data shifts. ℛ2makes efficient use of memory and does not degrade program quality. It adopts an optimized differencing algorithm to generate small delta files for efficient dissemination. We demonstrate ℛ2's advantages through detailed analysis of TinyOS examples. We also present case studies on the software programs of a large-scale and long-term sensor system—GreenOrbs. Results show that ℛ2reduces the dissemination cost by approximately 65% compared to Deluge—state-of-the-art network reprogramming approach, and reduces the dissemination cost by approximately 20% compared to Zephyr and Hermes—the latest works on incremental reprogramming.

A distributed access control module in wireless sensor networks (WSNs) allows the network to authorize and grant user access privileges for in-network data access. Prior research mainly focuses on designing such access control modules for WSNs, but little attention has been paid to protect user's identity privacy when a user is verified by the network for data accesses. Often, a user does not want the WSN to associate his identity to the data he requests, particularly in a single-owner multi-user WSN. In this paper, we present the design, implementation, and evaluation of a novel approach, Priccess, to ensure privacy-preserving access control. In addition to the theoretical analysis that demonstrates the security properties of Priccess, this paper also reports the experimental results of Priccess in a network of Imote2 motes, which show the efficiency of Priccess in practice.

Manifold Ranking (MR), a graph-based ranking algorithm, has been widely applied in information retrieval and shown to have excellent performance and feasibility on a variety of data types. Particularly, it has been successfully applied to content-based image retrieval, because of its outstanding ability to discover underlying geometrical structure of the given image database. However, manifold ranking is computationally very expensive, both in graph construction and ranking computation stages, which significantly limits its applicability to very large data sets. In this paper, we extend the original manifold ranking algorithm and propose a new framework named Efficient Manifold Ranking (EMR). We aim to address the shortcomings of MR from two perspectives: scalable graph construction and efficient computation. Specifically, we build an anchor graph on the data set instead of the traditional k-nearest neighbor graph, and design a new form of adjacency matrix utilized to speed up the ranking computation. The experimental results on a real world image database demonstrate the effectiveness and efficiency of our proposed method. With a comparable performance to the original manifold ranking, our method significantly reduces the computational time, makes it a promising method to large scale real world retrieval problems.

Anomaly detection is an essential functionality of Wireless Sensor Networks (WSNs) due to their complex behaviors and the wireless dynamics. In dynamic WSNs, many characteristics such as network topology, locations of sensor nodes, change frequently over time. We observe that indirect correlations among multiple attributes of a sensor node can be utilized to capture and model the historical behaviors. Prior studies overlooked indirect correlations while in this study we exploit it for detecting anomaly efficiently and accurately. Therefore, we propose ICAD, an indirect correlation based anomaly detection approach. By applying the Markov chain, the state transition probability matrix is calculated and it is subsequently used to detect anomalies. Compared to prior approaches, ICAD can detect different types of anomalies simultaneously. Furthermore, ICAD is implemented based on TinyOS and evaluated in a test-bed with 17 TelosB motes. Evaluation results show that ICAD has high detection accuracy with acceptable overhead.

Analysis of opinions, known as opinion mining or sentiment analysis, has attracted a great deal of attention recently due to many practical applications and challenging research problems. In this article, we study two important problems, namely, opinion lexicon expansion and opinion target extraction. Opinion targets (targets, for short) are entities and their attributes on which opinions have been expressed. To perform the tasks, we found that there are several syntactic relations that link opinion words and targets. These relations can be identified using a dependency parser and then utilized to expand the initial opinion lexicon and to extract targets. This proposed method is based on bootstrapping. We call it double propagation as it propagates information between opinion words and targets. A key advantage of the proposed method is that it only needs an initial opinion lexicon to start the bootstrapping process. Thus, the method is semi-supervised due to the use of opinion word seeds. In evaluation, we compare the proposed method with several state-of-the-art methods using a standard product review test collection. The results show that our approach outperforms these existing methods significantly.

We consider the active learning problem, which aims to select the most representative points. Out of many existing active learning techniques, optimum experimental design (OED) has received considerable attention recently. The typical OED criteria minimize the variance of the parameter estimates or predicted value. However, these methods see only global euclidean structure, while the local manifold structure is ignored. For example, I-optimal design selects those data points such that other data points can be best approximated by linear combinations of all the selected points. In this paper, we propose a novel active learning algorithm which takes into account the local structure of the data space. That is, each data point should be approximated by the linear combination of only its neighbors. Given the local reconstruction coefficients for every data point and the coordinates of the selected points, a transductive learning algorithm called Locally Linear Reconstruction (LLR) is proposed to reconstruct every other point. The most representative points are thus defined as those whose coordinates can be used to best reconstruct the whole data set. The sequential and convex optimization schemes are also introduced to solve the optimization problem. The experimental results have demonstrated the effectiveness of our proposed method.

Sparse coding has received an increasing amount of interest in recent years. It is an unsupervised learning algorithm, which finds a basis set capturing high-level semantics in the data and learns sparse coordinates in terms of the basis set. Originally applied to modeling the human visual cortex, sparse coding has been shown useful for many applications. However, most of the existing approaches to sparse coding fail to consider the geometrical structure of the data space. In many real applications, the data is more likely to reside on a low-dimensional submanifold embedded in the high-dimensional ambient space. It has been shown that the geometrical information of the data is important for discrimination. In this paper, we propose a graph based algorithm, called graph regularized sparse coding, to learn the sparse representations that explicitly take into account the local manifold structure of the data. By using graph Laplacian as a smooth operator, the obtained sparse representations vary smoothly along the geodesics of the data manifold. The extensive experimental results on image classification and clustering have demonstrated the effectiveness of our proposed algorithm.

We propose ReXOR, a lightweight and density-aware reprogramming protocol for wireless sensor networks using XOR. It employs XOR encoding in the retransmission phase to reduce the communication cost. In sparse and lossy networks, it delivers much better performance than Deluge, a typical reprogramming protocol for sensor networks. Compared to prior coding-based reprogramming protocols, it has two salient features. First, it is computationally much more lightweight than previous coding-based reprogramming protocols using Random Linear Codes or Fountain Codes. Second, it is density-aware by adapting its interpage waiting time. Hence, it achieves good performance in both dense and sparse networks. We have implemented ReXOR based on TinyOS and evaluate its performance extensively. Results show that ReXOR is indeed lightweight compared with previous coding-based reprogramming protocols in terms of computation overhead. The results also show that ReXOR achieves good network-level performance in both dense and sparse networks, compared with Deluge and a typical coding-based reprogramming protocol, Rateless Deluge.

There are various kinds of social media information, including different types of objects and relations among these objects, in music social communities such as Last.fm and Pandora. This information is valuable for music recommendation. However, there are two main challenges to exploit this rich social media information: (a) There are many different types of objects and relations in music social communities, which makes it difficult to develop a unified framework taking into account all objects and relations. (b) In these communities, some relations are much more sophisticated than pairwise relation, and thus cannot be simply modeled by a graph. We propose a novel music recommendation algorithm by using both multiple kinds of social media information and music acoustic-based content. Instead of graph, we use hypergraph to model the various objects and relations, and consider music recommendation as a ranking problem on this hypergraph. While an edge of an ordinary graph connects only two objects, a hyperedge represents a set of objects. In this way, hypergraph can be naturally used to model high-order relations.

Seamless roaming over wireless networks is highly desirable to mobile users, and security such as authentication of mobile users is challenging. In this paper, we propose a privacy-preserving universal authentication protocol, called Priauth, which provides strong user anonymity against both eavesdroppers and foreign servers, session key establishment, and achieves efficiency. Most importantly, Priauth provides an efficient approach to tackle the problem of user revocation while supporting strong user untraceability.

A distributed access control module in wireless sensor networks (WSNs) allows the network to authorize and grant user access privileges for in-network data access. Prior research mainly focuses on designing such access control modules for WSNs, but little attention has been paid to protect user's identity privacy when a user is verified by the network for data accesses. Often, a user does not want the WSN to associate his identity to the data he requests. In this paper, we present the design, implementation, and evaluation of a novel approach, Priccess, to ensure distributed privacy-preserving access control. In Priccess, users who have similar access privileges are organized into the same group by the network owner. A network user signs a query command on behalf of his group and then sends the signed query to the sensor nodes of his interest. The signature can be verified by its recipient as coming from someone authorized without exposing the actual signer. In addition to the theoretical analysis that demonstrates the security properties of Priccess, this paper also reports the experimental results of Priccess in a network of Imote2 motes, which show the efficiency of Priccess in practice.

In many real world applications, labeled data are usually expensive to get, while there may be a large amount of unlabeled data. To reduce the labeling cost, active learning attempts to discover the most informative data points for labeling. Recently, Optimal Experimental Design (OED) techniques have attracted an increasing amount of attention. OED is concerned with the design of experiments that minimizes variances of a parameterized model. Typical design criteria include D-, A-, and E-optimality. However, all these criteria are based on an ordinary linear regression model which aims to minimize the empirical error whereas the geometrical structure of the data space is not well respected. In this paper, we propose a novel optimal experimental design approach for active learning, called Laplacian G-Optimal Design (LapGOD), which considers both discriminating and geometrical structures. By using Laplacian Regularized Least Squares which incorporates manifold regularization into linear regression, our proposed algorithm selects those data points that minimizes the maximum variance of the predicted values on the data manifold. We also extend our algorithm to nonlinear case by using kernel trick. The experimental results on various image databases have shown that our proposed LapGOD active learning algorithm can significantly enhance the classification accuracy if the selected data points are used as training data.

In the form of topic discussions, users interact with each other to share knowledge and exchange information in online forums. Modeling the evolution of topic discussion reveals how information propagates on Internet and can thus help understand sociological phenomena and improve the performance of applications such as recommendation systems. In this paper, we argue that a user’s participation in topic discussions is motivated by either her friends or her own preferences. Inspired by the theory of information flow, we propose dynamic topic discussion models by mining influential relationships between users and individual preferences. Reply relations of users are exploited to construct the fundamental influential social network. The property of discussed topics and time lapse factor are also considered in our modeling. Furthermore, we propose a novel measure called ParticipationRank to rank users according to how important they are in the social network and to what extent they prefer to participate in the discussion of a certain topic. The experiments show our model can simulate the evolution of topic discussions well and predict the tendency of user’s participation accurately.

The phenomenon of propagation is universal in our daily life. For example, infectious diseases can be transmitted from one person to another, hot news is disseminated widely on the Internet, and classic passages written by the popular users can be shared by many other users in online social network sites. With the emerging of online social network sites, such as Facebook, YouTube, and Flickr, many literatures try to analyse patterns of information propagation and design effective virtual marketing strategies in these sites. However, few metrics have been designed to measure the characteristic of information propagation. In this paper, we propose a novel metric for measuring members’ contribution to information propagation in online social network sites. As a case, we analyse large-scale traces of members’ contribution to photo dissemination in Flickr and find that the distribution of members’ contribution follows a power law distribution, which reveals that most of the information is created and propagated by a few members.

People are more and more willing to participate in online forums to share their knowledge and experience. However, it may not be easy for them to find their desired threads in online forums due to the information overload problem. Traditional recommendation approaches can not be directly applied to online forums due to two reasons. First, unlike traditional movie or music recommendation problem, there is no rating information in online forums. Second, the sparsity problem is more severe since the users may only read threads but take no actions. To address these limitations, in this paper we propose to make use of the reply relationships among users, as well as thread contents. A learning algorithm is introduced to infer a user-thread alignment manifold in which both users and thread contents can be well represented. Thus, the relatedness between users and threads can be measured on this alignment manifold, and the closest threads which can best meet the corresponding user's information needs are recommended. Experiments on a dataset crawled from digg.com have demonstrated the superiority of our algorithm over traditional recommendation algorithms.

Based on the theory of Markov Random Fields, a Bayesian regularization model for diffusion tensor images (DTI) is proposed in this paper. The low-degree parameterization of diffusion tensors in our model makes it less computationally intensive to obtain a maximum a posteriori (MAP) estimation. An approximate solution to the problem is achieved efficiently using hierarchical Markov Chain Monte Carlo (HMCMC), and a loopy belief propagation algorithm is applied to a coarse grid to obtain a good initial solution for hierarchical MCMC. Experiments on synthetic and real data demonstrate the effectiveness of our methods.

Previous packet length optimizations for sensor networks often employ a fixed optimal length scheme, while in this study we present DPLC, a Dynamic Packet Length Control scheme. To make DPLC more efficient in terms of channel utilization, we incorporate a lightweight and accurate link estimation method that captures both physical channel conditions and interferences. We further provide two easy-to-use services, i.e., small message aggregation and large message fragmentation, to facilitate upper-layer application programming. The implementation of DPLC based on TinyOS 2.1 is lightweight, with respect to computation, memory, and header overhead. Our experiments using a real indoor testbed running CTP show that DPLC results in a 13% reduction in transmission overhead and a 41.8% reduction in energy consumption compared with the original protocol, and a 21% reduction in transmission overhead and a 15.1% reduction in energy consumption compared with simple aggregation schemes.

Acoustic-based music recommender systems have received increasing interest in recent years. Due to the semantic gap between low level acoustic features and high level music concepts, many researchers have explored collaborative filtering techniques in music recommender systems. Traditional collaborative filtering music recommendation methods only focus on user rating information. However, there are various kinds of social media information, including different types of objects and relations among these objects, in music social communities such as Last.fm and Pandora. This information is valuable for music recommendation. However, there are two challenges to exploit this rich social media information: (a) There are many different types of objects and relations in music social communities, which makes it difficult to develop a unified framework taking into account all objects and relations. (b) In these communities, some relations are much more sophisticated than pairwise relation, and thus cannot be simply modeled by a graph. In this paper, we propose a novel music recommendation algorithm by using both multiple kinds of social media information and music acoustic-based content. Instead of graph, we use hypergraph to model the various objects and relations, and consider music recommendation as a ranking problem on this hypergraph. While an edge of an ordinary graph connects only two objects, a hyperedge represents a set of objects. In this way, hypergraph can be naturally used to model high-order relations. Experiments on a data set collected from the music social community Last.fm have demonstrated the effectiveness of our proposed algorithm.

Bag of features (BoF) representation has attracted an increasing amount of attention in large scale image processing systems. BoF representation treats images as loose collections of local invariant descriptors extracted from them. The visual codebook is generally constructed by using an unsupervised algorithm such as K-means to quantize the local descriptors into clusters. Images are then represented by the frequency histograms of the codewords contained in them. To build a compact and discriminative codebook, codeword selection has become an indispensable tool. However, most of the existing codeword selection algorithms are supervised and the human labeling may be very expensive. In this paper, we consider the problem of unsupervised codeword selection, and propose a novel algorithm called Discriminative Codeword Selection (DCS). Motivated from recent studies on discriminative clustering, the central idea of our proposed algorithm is to select those codewords so that the cluster structure of the image database can be best respected. Specifically, a multi-output linear function is fitted to model the relationship between the data matrix after codeword selection and the indicator matrix. The most discriminative codewords are thus defined as those leading to minimal fitting error. Experiments on image retrieval and clustering have demonstrated the effectiveness of the proposed method.

Social tagging services allow users to annotate various on-line resources with freely chosen keywords (tags). They not only facilitate the users in finding and organizing online re-sources, but also provide meaningful collaborative semantic data which can potentially be exploited by recommender systems. Traditional studies on recommender systems fo-cused on user rating data, while recently social tagging data is becoming more and more prevalent. How to perform re-source recommendation based on tagging data is an emerg-ing research topic. In this paper we consider the problem of document (e.g. Web pages, research papers) recommen-dation using purely tagging data. That is, we only have data containing users, tags, documents and the relation-ships among them. We propose a novel graph-based rep-resentation learning algorithm for this purpose. The users, tags and documents are represented in the same semantic space in which two related objects are close to each other. For a given user, we recommend those documents that are sufficiently close to him/her. Experimental results on two data sets crawled from Del.icio.us and CiteULike show that our algorithm can generate promising recommendations and outperforms traditional recommendation algorithms.

Online monitoring of temporally-sequenced news streams for interesting patterns and trends has gained popularity in the last decade. In this paper, we study a particular news stream monitoring task: timely detection of bursty events which have happened recently and discovery of their evolutionary patterns along the timeline. Here, a news stream is represented as feature streams of tens of thousands of features (i.e., keyword. Each news story consists of a set of keywords.). A bursty event therefore is composed of a group of bursty features, which show bursty rises in frequency as the related event emerges. In this paper, we give a formal definition to the above problem and present a solution with the following steps: (1) applying an online multi-resolution burst detection method to identify bursty features with different bursty durations within a recent time period; (2) clustering bursty features to form bursty events and associating each event with a power value which reflects its bursty level; (3) applying an information retrieval method based on cosine similarity to discover the event’s evolution (i.e., highly related bursty events in history) along the timeline. We extensively evaluate the proposed methods on the Reuters Corpus Volume 1. Experimental results show that our methods can detect bursty events in a timely way and effectively discover their evolution. The power values used in our model not only measure event’s bursty level or relative importance well at a certain time point but also show relative strengths of events along the same evolution.

We propose FIT, a flexible, lightweight, and real-time scheduling system for wireless sensor platforms. There are three salient features of FIT. First, its two-tier hierarchical framework supports customizable application-specific scheduling policies, hence, FIT is very flexible. Second, FIT is lightweight in terms of minimizing the thread number to reduce preemptions and memory consumption while at the same time ensuring system schedulability. We propose a novel Minimum Thread Scheduling Policy (MTSP) exploration algorithm within FIT to achieve this goal. Finally, FIT provides a detailed real-time schedulability analysis method to help check if application's temporal requirements can be met. We implemented FIT on MicaZ motes and carried out extensive evaluations. Results demonstrate that FIT is indeed flexible and lightweight for implementing real-time applications, at the same time, the schedulability analysis provided can predict the real-time behavior. FIT is a promising scheduling system for implementing complex real-time applications in sensor networks.

Online advertising has now turned to be one of the major revenue sources for today's Internet companies. Among the different channels of advertising, contextual advertising takes the great part. There are already lots of studies done for the keyword extraction problem in contextual advertising for English, however, little has been conducted for Chinese, which is mainly different from English linguistically. In this paper, we focus on the problem of Chinese advertising keywords extraction and propose a novel approach based on the idea of classification. We adopt C4.5 as the classifier model and select appropriate features with Chinese linguistic characteristic taken into consideration. The experimental results indicate that our approach is promising.

The tradeoff between revenue and market share triggers the emergence of targeted advertising. However, all previous related work only focuses on the topical relevance of ads and does not concern the attitudes of consumers. In our paper, we propose a novel advertising strategy DASA (dissatisfaction-oriented advertising based on sentiment analysis) which takes the attitudes of consumers into consideration and promotes ads according to what consumers are unsatisfied with. Our work is most suitable for user generated content which contains plenty of consumerspsila opinionated information. The experiments show encouraging results.

Rapid advances in the enabling technologies for mobile and ubiquitous computing help portable devices to join collaborative graphics design conveniently. The limitation of display size and computational power of these embedded devices makes it hard for mobile users to browse large pattern that renewed in real time efficiently. We present a novel user interest region prediction algorithm and related system that use state duration based segmental probability model (SDSPM) to forecast user's intention in the near future. Related experiment was carried out to test the effectiveness of the algorithm. Based on the prediction, only sub-patterns that might be interested to user are issued to the embedded sites. The proposed user interest prediction system is effective and the feasibility of the collaborative graphics design system in ubiquitous environment is enhanced.

This paper proposes a temporal error concealment algorithm based on the edge directions of the pixels surrounding the lost macroblock (MB). On one hand, for each pixel on the external boundary of the restored MB, the spatial boundary distortion is calculated along the edge direction, which preserves edge alignment as well as ensures spatial continuity. On the other hand, the edge directions are utilized to identify whether the lost MB contains multiple objects, and then determine the restore mode of the four 8x8 blocks in the lost MB: simultaneously or independently. The experimental results for several test video sequences demonstrate that higher performance is achieved in both objective and subjective quality of the reconstructed frames compared with conventional error concealment algorithms.

In this paper, we describe a block-based threshold modulation error diffusion algorithm to parallelize the halftone process without generating both block-boundary and diagonal artifacts. A novel scan path is used to pass quantization error effectively between blocks, and the input-dependent threshold modulation is applied during the inner-block processing. To obtain suitable parameter set for the error weights and threshold modulation strength, a cost function is designed. Experimental results show that the proposed algorithm generates high quality halftone image which is visually closed to those generated by serial error diffusion algorithms. It takes the advantage of both quality and parallelism compared to other halftoning approaches.

Automatic news extraction from news pages is important in many Web applications such as news aggregation. However, the existing news extraction methods based on template-level wrapper induction have three serious limitations. First, the existing methods cannot correctly extract pages belonging to an unseen template. Second, it is costly to maintain up-to-date wrappers for a large amount of news websites, because any change of a template may invalidate the corresponding wrapper. Last, the existing methods can merely extract unformatted plain texts, and thus are not user friendly. In this paper, we tackle the problem of template-independent Web news extraction in a user-friendly way. We formalize Web news extraction as a machine learning problem and learn a template-independent wrapper using a very small number of labeled news pages from a single site. Novel features dedicated to news titles and bodies are developed. Correlations between news titles and news bodies are exploited. Our template-independent wrapper can extract news pages from different sites regardless of templates. Moreover, our approach can extract not only texts, but also images and animates within the news bodies and the extracted news articles are in the same visual style as in the original pages. In our experiments, a wrapper learned from 40 pages from a single news site achieved an accuracy of 98.1% on 3,973 news pages from 12 news sites.

We present a holistic dynamic linking and loading mechanism in networked embedded systems. Our design and implementation are guided by four requirements, which are to provide (i) minimal code size (ii) efficient execution and loading speed (iii) portable design (iv) isolated kernel/application development. First, we develop a tool to minimize the standard ELF format via many techniques in order to reduce the code dissemination cost. Second, we employ the techniques of pre-relocating and pre-linking (to kernel functions) to reduce the run-time linking overhead, thus improving the loading speed. Third, based on relocatable ELF and the modular design of the dynamic linker and loader, our approach can be easily ported to different platforms. Fourth, by maintaining a kernel jump table, we provide a clean isolation between kernel and application development. We have implemented the dynamic linking and loading mechanism on SenSpire OS, a micro sensor node operating system. The evaluation results show that our design and implementation meet our design goals: the code size of our SELF format is only 15%–30% of that of standard ELF, 38%–83% of that of CELF, a compact ELF format for the Contiki operating system; the loading speed improvement varies from 40%–50% compared to the standard mechanism; our design is portable to both MicaZ and TelosB motes, and we allow updating both application modules and kernel services in isolation without prior knowledge about the whole system information.

As a crucial step of the visual cognition and perception, visual attention analysis shows great importance in many research or application areas. In this paper, we treat this problem from a new angle, inspired by the classic gravitational field theory. By defining "mass" of each pixel, we compute "force" between them to obtain a so-called pseudo gravitational field over an image. Then, we propose an iteration algorithm to simulate the movement of the fixation points affected by this field. Finally, stable visual attention points or areas are obtained when those fixation points finally aggregate around some special pixels or areas. The main contributions are threefold: (1) by introducing classic gravitational field theory into visual attention analysis, a new point of view is proposed; (2) a competition scheme is constructed and the meaning of attraction can be applied into visual attention analysis intuitively; (3) by using pixels into computation through down sampling, a faster analysis method is achieved. The experimental result shows that our method is effective and consistent with the generally accepted definition of visual attention.

Content Based Image Retrieval (CBIR) has become one of the most active research areas in computer science. Relevance feedback is often used in CBIR systems to bridge the semantic gap. Typically, users are asked to make relevance judgements on some query results, and the feedback information is then used to re-rank the images in the database. An effective relevance feedback algorithm must provide the users with the most informative images with respect to the ranking function. In this paper, we propose a novel active learning algorithm, called Convex Laplacian Regularized I-optimal Design (CLapRID), for relevance feedback image retrieval. Our algorithm is based on a regression model which minimizes the least square error on the labeled images and simultaneously preserves the intrinsic geometrical structure of the image space. It selects the most informative images which minimize the average predictive variance. The optimization problem of CLapRID can be cast as a semidefinite programming (SDP) problem, and solved via interior-point methods. Experimental results on COREL database have demonstrate the effectiveness of the proposed algorithm for relevance feedback image retrieval. Categories and Subject Descriptors H.3.3 [Information storage and retrieval]: Information search and retrieval—Relevance feedback; G.3 [Mathematics of Computing]: Probability and Statistics—Experimental design

In speech based emotion recognition, both acoustic features extraction and features classification are usually time consuming,which obstruct the system to be real time. In this paper, we proposea novel feature selection (FSalgorithm to filter out the low efficiency features towards fast speech emotion recognition.Firstly, each acoustic feature's discriminative ability, time consumption and redundancy are calculated. Then, we map the original feature space into a nonlinear one to select nonlinear features,which can exploit the underlying relationship among the original features. Thirdly, high discriminative nonlinear feature with low time consumption is initially preserved. Finally, a further selection is followed to obtain low redundant features based on these preserved features. The final selected nonlinear features are used in features' extraction and features' classification in our approach, we call them qualified features. The experimental results demonstrate that recognition time consumption can be dramatically reduced in not only the extraction phase but also the classification phase. Moreover, a competitive of recognition accuracy has been observed in the speech emotion recognition.

Social tagging is becoming increasingly popular in many Web 2.0 applications where users can annotate resources (e.g. Web pages) with arbitrary keywords (i.e. tags). A tag recommendation module can assist users in tagging process by suggesting relevant tags to them. It can also be directly used to expand the set of tags annotating a resource. The benefits are twofold: improving user experience and enriching the index of resources. However, the former one is not emphasized in previous studies, though a lot of work has reported that different users may describe the same concept in different ways. We address the problem of personalized tag recommendation for text documents. In particular, we model personalized tag recommendation as a "query and ranking" problem and propose a novel graph-based ranking algorithm for interrelated multi-type objects. When a user issues a tagging request, both the document and the user are treated as a part of the query. Tags are then ranked by our graph-based ranking algorithm which takes into consideration both relevance to the document and preference of the user. Finally, the top ranked tags are presented to the user as suggestions. Experiments on a large-scale tagging data set collected from Del.icio.us have demonstrated that our proposed algorithm significantly outperforms algorithms which fail to consider the diversity of different users' interests.

We present here a prototype of an algorithm for vehicle speed estimation. Different from previous approaches, our algorithm requires no road markers and fewer manual calibrations. Based on specific projection rules, we find a relation between the in-camera coordinate and the real world coordinate. A non-linear regression is employed to estimate the model parameters. This model enables us to estimate the real world position of the vehicles directly from a video sequence taken by a surveillance camera. The algorithm shows its ability to produce accurate estimations in our experiments.

User-Interactive Question Answering (QA) communities such as Yahoo! Answers are growing in popularity. However, as these QA sites always have thousands of new questions posted daily, it is difficult for users to find the questions that are of interest to them. Consequently, this may delay the answering of the new questions. This gives rise to question recommendation techniques that help users locate interesting questions. In this paper, we adopt the Probabilistic Latent Semantic Analysis (PLSA) model for question recommendation and propose a novel metric to evaluate the performance of our approach. The experimental results show our recommendation approach is effective.

We consider the problem of template-independent news extraction. The state-of-the-art news extraction method is based on template-level wrapper induction, which has two serious limitations. 1) It cannot correctly extract pages belonging to an unseen template until the wrapper for that template has been generated. 2) It is costly to maintain up-to-date wrappers for hundreds of websites, because any change of a template may lead to the invalidation of the corresponding wrapper. In this paper we formalize news extraction as a machine learning problem and learn a template-independent wrapper using a very small number of labeled news pages from a single site. Novel features dedicated to news titles and bodies are developed respectively. Correlations between the news title and the news body are exploited. Our template-independent wrapper can extract news pages from different sites regardless of templates. In experiments, a wrapper is learned from 40 pages from a single news site. It achieved 98.1% accuracy over 3,973 news pages from 12 news sites.

This paper proposes an efficient relevance feedback based interactive model for keyword generation in sponsored search advertising. We formulate the ranking of relevant terms as a supervised learning problem and suggest new terms for the seed by leveraging user relevance feedback information. Active learning is employed to select the most informative samples from a set of candidate terms for user labeling. Experiments show our approach improves the relevance of generated terms significantly with little user effort required.

Social annotations on a Web document are highly generalized description of topics contained in that page. Their tagged frequency indicates the user attentions with various degrees. This makes annotations a good resource for summarizing multiple topics in a Web page. In this paper, we present a tag-oriented Web document summarization approach by using both document content and the tags annotated on that document. To improve summarization performance, a new tag ranking algorithm named EigenTag is proposed in this paper to reduce noise in tags. Meanwhile, association mining technique is employed to expand tag set to tackle the sparsity problem. Experimental results show our tag-oriented summarization has a significant improvement over those not using tags.

Collaborative graphics design systems allow users, spread across different locations, to work together designing common patterns. One key element that, in most cases, is poorly supported in such systems is semantic preservation. To make the real-time collaborative graphics design system more efficient, it is essential that the semantic violation problem be well solved. In this paper we present the semantic maintenance architecture and the classification of semantic preservation. A new approach and related algorithm that can resolve semantic conflict problem efficiently is also proposed. Semantic consistency maintenance helps to promote the collaborative awareness abilities. The architecture and algorithm has been tested in the CoDesign prototype system.

Coordination Mechanism is a critical issue in the cooperative work. However, there are no universal and efficient coordination mechanisms that are fit for the collaborative graphic designing system accounts for it’s special characteristic and properties. In this paper we propose a novel intelligent coordination mechanism, which enable members to concentrate on the most of controversial part of the cooperative work by utilizing human awareness information. Related algorithm that includes two key concepts: expectation and convergence is proposed. The mechanism and algorithm have been realized in our prototype system and are proved significantly promote the coordination efficiency.

Background modeling is an important component of many computer vision systems. The numerous approaches to this problem differ in the statistical models used to describe the temporal behavior of single pixels. Without proper use of spatial coherence between pixel values, these models suffer greatly from memory consumption. In order to reduce spatial redundancy in the data, we propose a novel block- based background model which clusters pixel values within each small block of frames, and build weighted indexes for each pixel to track color values temporally. Compared with traditional models, the proposed model greatly reduces average number of bytes needed to model a pixel, and can be used in real-time video surveillance systems.

Cartoon-like stylization for video suffers the problems of low speed and unpleasant visual effect of edges in contiguous frames. In this paper, we present a novel approach of cartoon-like stylization for video which is fit for the high real-time requirement of some applications. In this approach, the edge detection algorithm bases on the Laplacian of Gaussian (LOG). It is faster and has better visual effect of edges in contiguous frames. In addition to that, we use an implementation based on querying the tables of filter coefficients and quantization values in the steps of extended nonlinear diffusion and color quantization to further improve the speed of our method. The real-time method brings amusing effect to the entertaining systems such as video chat or gaming. The results of our experiments show that our method performs well on the human faces and scenery.

This paper proposes an error concealment method for whole-frame losses based on motion stability estimation. Motions of 4×4 blocks in the lost frame are estimated as stable or unstable after a bidirectional motion vector (MV) projection. Then a two-step decoding is employed for both the lost frame and its succeeding frame. Blocks in the lost frame with stable motion are first concealed directly with the projected MV, which provides adequate information for concealing unstable motion blocks in the next step. Then a list of MV candidates is tested for each block in the lost frame with unstable motion, which minimizes the distortion in both the lost frame and its succeeding frame. Experimental results demonstrate that the proposed method outperforms previous methods in both objective and subjective performance.

Focused crawling is a critical technique for topical resource discovery on the Web. We propose a new frontier prioritizing algorithm, namely, the OTIE (On-line Topical Importance Estimation) algorithm, which efficiently and effectively combines link-based and content-based analysis to evaluate the priority of an uncrawled URL in the frontier. We then demonstrate OTIE's advantages over traditional prioritizing algorithms by real crawling experiments.

With the booming development of the Web, popular Chinese forums enable people to find experienced customers' reviews for products. In order to get an all-around opinion about one product, users need to go through plenty of web pages, which is time-consuming and inefficient. Consequently, automatic review mining and summarization has become a hot research topic recently. However, previous approaches are not applicable for mining Chinese customer reviews. In this paper, we introduce SOPING, a Chinese customer review mining system that mines reviews from forums. Specifically, we propose a novel search-based approach to extract product features and a feature-oriented sentence orientation determination method. Our experimental results show that our proposed techniques are highly effective.

Blogs have been expanded at an incredible speed in recent years. Plentiful personal information makes blogs a popular way mining user profiles. In this paper, we propose a novel bloggers' interests modeling approach based on forgetting mechanism. A new forgetting function is introduced to track interest drift. Based on that, the Short Term Interest Models (STIM) and Long Term Interest Models (LTIM) are constructed to describe bloggers' short-term and long-term interests. The experiments show that both models can identify bloggers' preferences well respectively.

BACKGROUND: With the rapid emergence of RNA databases and newly identified non-coding RNAs, an efficient compression algorithm for RNA sequence and structural information is needed for the storage and analysis of such data. Although several algorithms for compressing DNA sequences have been proposed, none of them are suitable for the compression of RNA sequences with their secondary structures simultaneously. This kind of compression not only facilitates the maintenance of RNA data, but also supplies a novel way to measure the informational complexity of RNA structural data, raising the possibility of studying the relationship between the functional activities of RNA structures and their complexities, as well as various structural properties of RNA based on compression. RESULTS: RNACompress employs an efficient grammar-based model to compress RNA sequences and their secondary structures. The main goals of this algorithm are two fold: (1) present a robust and effective way for RNA structural data compression; (2) design a suitable model to represent RNA secondary structure as well as derive the informational complexity of the structural data based on compression. Our extensive tests have shown that RNACompress achieves a universally better compression ratio compared with other sequence-specific or common text-specific compression algorithms, such as Gencompress, winrar and gzip. Moreover, a test of the activities of distinct GTP-binding RNAs (aptamers) compared with their structural complexity shows that our defined informational complexity can be used to describe how complexity varies with activity. These results lead to an objective means of comparing the functional properties of heteropolymers from the information perspective. CONCLUSION: A universal algorithm for the compression of RNA secondary structure as well as the evaluation of its informational complexity is discussed in this paper. We have developed RNACompress, as a useful tool for academic users. Extensive tests have shown that RNACompress is a universally efficient algorithm for the compression of RNA sequences with their secondary structures. RNACompress also serves as a good measurement of the informational complexity of RNA secondary structure, which can be used to study the functional activities of RNA molecules.

In this paper, a parallel algorithm for RBF-based surface reconstruction from contours is presented. The reconstruction process is divided into local reconstruction and global reconstruction phases. In the local reconstruction phase, surface patches are reconstructed from a set of adjacent slices of contours, these tasks are scheduled to be executed in parallel. In the global reconstruction phase, weighted function and local function values are evaluated in parallel. To balance the overload among CPUs or threads and get best parallel computing performance, the total sample points involved in task groups are distributed evenly. The experiment results show that the proposed schedule accelerate the reconstruction process greatly.

Video-based channel that can transmit people's facial expression information is designed to support real-time interaction between collaborators. Yet exchanging facial video in ubiquitous environment is confined by the low bandwidth and limited computational resources on mobile embedded devices. In this paper, we present a novel two dimensional (2D) facial animation communication system which enables the embedded sites to gain awareness of PC users' facial expression via facial animation. The system was examined by experiments and survey. Results show that 2D facial animation provides participants' emotion awareness and is a novel visual communication pattern.

Collaborative graphics editing systems in ubiquitous environment support users to join and leave the collaboration dynamically at any time. The shared workspace state of these users will be inconsistent for the modification of the other session members. It is therefore necessary to initialize the application instance of the latecomer with the current state. In this paper, we propose a rejoining scheme that is based on the user interest prediction algorithm we proposed. The scheme is efficient for latecomers to rejoin the collaborative design session. The scheme is realized in our prototype system of collaborative graphics editing system that can work well in ubiquitous environments.

In this paper, we propose a novel error concealment algorithm for the whole-frame loss based on multiframe. In concealing procedure, besides improve the quality of the concealed lost frames, the proposed algorithm also guarantees the quality of the succeeding frames that reference the lost frame in order to control the propagation of errors. Macroblocks in the lost frame are concealed in an appropriate order which is determined by exploiting the information from partly decoded succeeding frames. Experimental results demonstrate that the proposed algorithm not only outperforms previous algorithms in both PSNR evaluation and visual quality of the concealed lost frames, but also minimizes the error propagation.

Wireless video transmission is expected to experience whole frame losses since data packets always carry a whole frame in low-bitrate coding scenario. In this paper, a novel whole frame loss recovery scheme is proposed. A vector map Vmap for each frame is generated and encoded into the bitstream of the next frame. The Vmap contains useful information for decoder to recover the motion vector of the previous lost frame, and the content of Vmap is based on concealment modes, which are dynamically decided while encoding by estimating the motion variations through statistical analysis. Experimental results demonstrate that the proposed scheme is very effective in dealing with the whole frame loss occurring in wireless video transmission.

In previous systems of speech emotion recognition, supervised learning are frequently employed to train classifiers on lots of labeled examples. However, the labeling of abundant data requires much time and many human efforts. This paper presents an enhanced co-training algorithm to utilize a large amount of unlabeled speech utterances for building a semi-supervised learning system. It uses two conditionally independent attribute views(i.e. temporal features and statistic features) of unlabeled examples to augment a much smaller set of labeled examples. Our experimental results demonstrate that compared with the method based on the supervised training, the proposed system makes 9.0% absolute improvement on female model and 7.4% on male model in terms of average accuracy. Moreover, the enhanced co-training algorithm achieves comparable performance to the co-training prototype, while it can reduce the classification noise which is produced by error labeling in the process of semi-supervised learning.

This poster focuses on the study of term context dependence in the application of sentence retrieval. Based on Markov Random Field (MRF), three forms of dependence among query terms are considered. Under different assumptions of term dependence relationship, three feature functions are defined, with the purpose to utilize association features between query terms in sentence to evaluate the relevance of sentence. Experimental results have proven the efficiency of the proposed retrieval models in improving the performance of sentence retrieval.

No abstract available.

Query ambiguity prevents existing retrieval systems from returning reasonable results for every query. As there is already lots of work done on resolving ambiguity, vague queries could be handled using corresponding approaches separately if they can be identified in advance. Quantification of the degree of (lack of) ambiguity laysthe groundwork for the identification. In this poster, we propose such a measure using query topics based on the topic structure selected from the Open Directory Project (ODP) taxonomy. We introduce clarity score to quantify the lack of ambiguity with respect to data sets constructed from the TREC collections and the rank correlation test results demonstrate a strong positive association between the clarity scores and retrieval precisions for queries.

This paper makes an intensive investigation of the application of Bayesian network in sentence retrieval and introduces three Bayesian network based sentence retrieval models with or without consideration of term relationships. Term relationships in this paper are considered from two perspectives: relationships between pairs of terms and relationships between terms and term sets. Experiments have proven the efficiency of Bayesian network in the application of sentence retrieval. Particularly, retrieval result with consideration of the second kind of term relationship performs better in improving retrieval precision.

A number of existing information retrieval systems propose the notion of query context to combine the knowledge of query and user into retrieval to reveal the most exact description of user's information needs. In this paper we interpret query context as a document consisting of sentences related to the current query. This kind of query context is used to re-estimate the relevance probabilities of top-ranked documents and then re-rank top-ranked documents. The experiments show that the proposed context-based approach for information retrieval can greatly improved relevance of search results.

With heterogeneous devices proliferating, there is a need for collaborative graphics editing systems that are tailored to the capabilities of their current computing and communications environment. Mobile network and device capabilities constrain the application implementation, which implies that the storage space and computing power is limited and network is unreliable. This paper presents a semi-replicated architecture and an on demand consistency maintenance scheme for mobile sites to cooperate with other desktops efficiently. To maintain the consistency of the replicas on mobile embedded devices, either data consistency or semantic similarity is maintained based on the prediction of the regions of interest. The scheme is realized in our prototype system of collaborative graphics editing system that can work well in heterogeneous environments

Tracing the movement of telepointer is one of the most efficient ways to aware remote collaborators' drawing actions in real-time Internet based collaborative graphics editing systems. However, delay jitter may occur, which will make the presence of telepointer jumpy and lead to semantic misunderstanding or operation conflicting in collaboration. This paper presents a novel algorithm that predicts the moving track of telepointer based on historical movement cases. The algorithm is tested in our prototype system and is proved that it can limit the negative impact of jitter and provide collaborators with smooth and accurate telepointer motion awareness information

One of the new features adopted in the latest H.264/AVC video coding standard is the mode decision (MD), which greatly improves the rate-distortion performance but it also takes up a significant encoding time to determine the best mode. This paper presents a predictive block-size selection algorithm to extraordinarily improve the encoding efficiency in H.264. The proposed algorithm mainly takes advantage of two efficient predictive methods: one is predictive skipping checking the sub-macroblock-level modes according to the sum of absolute difference (SAD) of each macroblock and the other is filtering some sub-macroblock-level modes with an adaptive threshold obtained from the mode information in the previous determined macroblock. Experimental results and comparative analysis are given to verify that our proposed algorithm can achieve a speed-up factor with 30% compared with the current Fast Full Search algorithm, with negligible average PSNR loss of 0.071dB and bit rate increase of 2.78%.

To-H.264 transcoding, due to its high practical values for a wide range of realtime video applications, has become an active research topic. However, most of the previous works do not address a very important new feature of H.264, multiple-reference-frame motion estimation. In this paper, employing forward domain vector selection (FDVS) and top-down splitting algorithms, a novel fast multiple-reference-frame based motion estimation & mode decision algorithm for H.263-to-H.264 transcoder is proposed. Potential block-sizes reduction and early-stop strategies are also adopted to accelerate the processing. Moreover, a novel adaptive search range selection algorithm is proposed to further speed up the motion search while the coding efficiency is not compromised. Proposed algorithm can also be extended to support transcoding other format videos to H.264, such as MPEG2-to-H.264. Experimental results show that the computational complexity can be reduced to about 11% of reference transcoder with negligible PSNR drop and a little bit-rate increase.

Facial expression means not only the feature motions but also the subtle appearance changes in the face. The details are important to facial expression synthesis. In this paper, a novel method based on vector field decomposition is presented to model subtle facial expressions. The intensity change of facial expression image is represented by Helmholtz-Hodge vector field decomposition. Based on this method, subtle facial expression mapping is carried out much faster and better. Experiment results show that our method is robust and convincible to model the subtle facial expression.

This paper presents an emotion recognition system from clean and noisy speech. Geodesic distance was adopted to preserve the intrinsic geometry of emotional speech. Based on the geodesic distance estimation, an enhanced Lipschitz embedding was developed to embed the 64-dimensional acoustic features into a six-dimensional space. In order to avoid the problems brought by noise reduction, emotion recognition from noisy speech was performed directly. Linear discriminant analysis (LDA), principal component analysis (PCA) and feature selection by sequential forward selection (SFS) with support vector machine (SVM) were also included to compress acoustic features before classifying the emotional states of clean and noisy speech. Experimental results demonstrate that compared with other methods, the proposed system makes approximately 10% improvement. The performance of our system is also robust when speech data is corrupted by increasing noise

Both perceptually and statistically, compressed video with large or disordered motion is sensitive to errors. In this paper, we propose a novel fast error-resilient video coding scheme, which is based on significant macroblock (MB) determination and protection. The scheme takes three impact factors (inter-block mode, motion vector difference and SAD value) to build a statistical model. The model takes error concealment (EC) into consideration in advance and generates several parameters for further significant degree (SD) evaluation for MBs. During encoding, we build an SD table for each frame based on the parameters and pick up those MBs with the largest SD values as significant MBs (SMBs). Few additional computations are induced into SMB determination, thus make our scheme practical in real time video coding scenarios. Simulations show that the scheme has an acceptable SMB determination accuracy and the corresponding protection method can prevent errors effectively

Surface reconstruction from particle system is a difficult problem. In this paper, we introduce a method to reconstruct the surface from particle-based simulation of ocean waves with complex shape and splashes. Each 2D slice of particle volume is projected onto an image, and then the image is segmented for recognition of water body and splash fragments. Outlines of the water body are extracted directionally. The mesh of surface is reconstructed by tiling the triangular strip from the outlines of the slices. The experiments show that the efficiency of this method is very competitive.

Locking is a common technique in distributed editing system used to prevent conflicting, maintain user intention and result consistency. Collaborative pattern design system is a special class, thus has special locking scheme. A novel scheme system forecast lock is proposed such that some regions are picked up by system where user wants to operate in the following steps, and these regions are locked then. System will settle conflict whenever they occurred. By using the scheme proposed, system locks regions to forward users' operations with great fluency, and prevent operating conflict.

The emerging H.264 advanced video coding (H.264/AVC) standard achieves significant improvement in coding efficiency compared with previous standards such as MPEG-2 and H.263. One major contribution of its gain profits from variable-block-size motion estimation, which also leads to higher computational complexity. In this paper, we propose a fast predictive variable-block-size motion estimation algorithm for H.264/AVC. The algorithm takes advantage of three effective predictive schemes-stationary block prediction, predictive search for non-stationary blocks and predictive multi-pattern refinement search in merging process. Experimental results and comparative analysis have shown that our proposed algorithm can reduce the computational complexity up to about 2% of fast full search motion estimation algorithm, with negligible average PSNR loss of 0.065 dB and bit rate increase of 2.43%.

This paper proposes a reversible color transform for 16-bit-color (hicolor) picture coding. The work is motivated by the increasing needs of multimedia applications on low-end devices such as mobile phones and PDAs. They have limited resources and up to 16-bit displays. Current image/video coding systems can hardly manage this case effectively. To enhance coding efficiency on this condition, a reversible color transform customized for hicolor systems is derived from Y'CrCb and JPEG2000 Reversible Component Transformation (RCT). The transform proves simple but highly-decorrelating, and able to reduce the computation time of decoding. Comparison experiment demonstrates the effectiveness of this transform with equal or even higher coding efficiency on low-end devices with 16-bit display mode.

With the increasing number of people who can afford to make videos to record their lives, home videos play a more and more important role in multimedia. Video abstraction is an efficient way to help review such a huge amount of home videos. A home video abstraction technique combining audio and video features is presented. The audio contents are firstly classified as silence, pure speech, non-pure speech, music and background sound using support vector machines (SVMs). Then, non-pure speech is further classified into song and other non-pure speech using SVM, and background sound is classified into laughter, applause, scream and others using hidden Markov models (HMMs). For video contents, motion level and blur degree are acquired. Finally, video segments containing special features, such as speech, laughter, song, applause, scream, and specified motion level and blur degree, are extracted as the main parts of the abstract. The remaining parts of the abstract are generated using key frame information. Experimental results show that the proposed algorithm can extract the desired parts of a home video to generate satisfactory video abstracts.

Grouping photos into semantically meaningful categories is an important issue in many applications that use low-level features to deal with consumer photographs. However, low-level features such as color and texture did not contain the local and spatial properties of images. And high accuracy cannot be obtained for general semantic classification problems. An approach based on Bayesian framework and one-step relevance feedback was proposed. Knowledge from low-level features and spatial properties was integrated into Bayesian framework. Furthermore, a one-step relevance feedback method was implemented to specify the optimal division strategy of images. The system provides the ability to utilize the local and spatial properties to classify new images. Experimental results show that high accuracy can be obtained for general semantic classification problems.

Classification of digital photos using low-level features is an important but very difficult issue in many applications that deal with consumer photographs. Previous methods for image classification problems always aimed at some specific classification problems such as city vs. landscape. These methods were always based on some specific ground truths and they were not suitable for general classification problems. A system based on Bayesian framework and learning mechanism was proposed. The most effective and efficient classification strategy could be automatically obtained. The system was implemented and performance test was conducted using a database of real consumer photos. Experimental results show that high accuracy can be obtained for general classification problems.

Blurring is inevitable in many image acquisition systems. Blind image restoration is one of deblur methods, and because it does not need a priori point spread function (PSF), it becomes more and more popular. In this paper, a novel method of blind image restoration is presented which can be applied to various kinds of images. The method has no limitation on blurred images, and does not need any extra database supports. Gaussian Blur is assumed to be able to present every other blur, and by successfully dealing with Gaussian Blur, our method can be suitable to deal with other blur methods. In order to reduce the noise, a pre-processing step is introduced. With edge detection, point spread function can be reconstructed automatically. Then an iterative algorithm is used to restore the image. Experimental results show the good performance of this method.

The reality and controllability of the facial animation are enhanced in our work by modeling mouth in a parametric approach described by 7 parameters: superior and anterior bend for upper and lower lip, width of the mouth, weight of radium of lips. These parameters were calculated by tracking only 4 points around the mouth from video, and the cost of computation is low. The coarticulation model of Cohen and Massaro was adopted to generate the natural key frames and the transitional frames. Considering the difference between Chinese and English, an algorithm was introduced to calculate the coefficients of dominance functions. Based on the muscle-based facial model and the parametric mouth model, the 3D head can talk vividly.

We develop a system to generate individual facial expression with minimal interactions, and expression cloning can be created for different 3D mesh model. In our system, expressions can be created with the muscle model, and muscular vectors adhere to the vertex in 3D mesh face model. Expressions can be generated after deforming the muscular vectors and rotation angle of jaw parameter and rotation angle of eyes. New approach for expression cloning for different 3D models is presented, radical based function (RBF) is employed to match the vertices of 3D source model and destination model, the cloning expression is obtained by deforming the proportioned muscular vector model. Cloned expression animations preserve the relative motions, dynamics, and character of the original facial animations. If manual tuning or computational costs are high in creating animations for one model, creating similar animations for new models will take similar efforts. Experimental results show the high vividness of our system.

Real-time automatic face tracking is a great challenge in computer vision and computer graphics. We develop a system to automatically track the face by integrating auto-generation of features of first frame, feature correspondence and Kalman filter based on face attribution and motion estimation. First, facial features of the first frame are acquired automatically by face detachment and Plessey corner detector according to FDP in MPEG-4 based on anatomical knowledge and general 3D model. Feature correspondence is obtained by maximizing the cross-correlation and Kalman filter. Automatic face detection makes full use of the knowledge of 3D general model and face attribute, and face tracking is more efficient by using motion estimation. Experimental results show the high prospect of this algorithm.

To create 3D realistic talking face has been a challenge for a long time. Previous works emphasize text or speech driven talking face respectively while the animation result is not very realistic or natural-looking. In the proposed approach, text and speech are considered to drive the 3D talkingface coordinately. The text is translated into a sequence of visemes' transcription. And time vector of the sequence is extracted from the speech corresponding to the text after it is segmented into phonetic sequence. A muscle based viseme vector is defined for static viseme. And then, with the time vector and the static visemes's sequence, dynamic visemes are generated through time-related dominance function. Finally, according to the frame rate to be rendered, intermediate frames are interpolated between key frames to make the animation result looks more natural and realistic than those obtained based on the text or speech-driven only.

The ability to undo operations is an indispensable feature of single user editing systems, but supporting group undo in real-time collaborative editing systems is still a difficult problem. In this paper, we propose an undo framework and algorithms to achieve group undo in image-based collaborative graphics editing systems. The basic idea is to interpret an undo command as a concurrent inverse operation by means of image operation transformation algorithm, so that an operation is always undoable under its current context. Through exploiting relations among operations, space cost for operation preservation is greatly reduced. The global undo, local undo and selective undo mode are supported in our solution. The undo algorithms are also applicable in single-user applications. The algorithms are implemented in CoDesign-a multi-level collaborative graphics designing system, which aims at supporting both object-based and image-based collaborative pattern design.

Bitmap-based collaborative graphics editing systems are a special class of real-time collaborative editing systems. Undo is an important and difficult problem in these systems. Existing solutions show low efficiency because additional space cost should be added to achieve the function of undo. In this paper, we propose a new solution to resolve the undo problem. The basic idea is to reduce space cost through exploring relations among operations. The algorithm given in the paper can undo any operation at any time, and can greatly reduce the space cost while increasing a few time cost. The issues about undo scope and undo mode are also discussed in the paper.

A real-time collaborative graphic design system is a special class of computer-supported collaborative work (CSCW) systems. It allows a group of users to simultaneously view and design some graphics documents from geographically dispersed sites connected by computer networks. Conflict resolution and operation consistency remains one of the core issues in this type of system. This paper discusses the problem of preserving consistency operations in face of conflict, and proposes a multi-version approach to resolve it. In the approach, first operations are divided into two categories, and then a new structure is adopted to represent these operations. It keeps the consistency of operations at very low cost by increasing multiple object versions in face of conflicts. The algorithms presented in this paper have been used in the CoDesign system. The result proves this approach can improve the system performance in the real-time collaborative graphic designing system.

A new method is proposed for real-time video object segmentation from a static background under shadows and varying lighting. The method first learns a statistical model and a type model for the background scene in HSV (hue, saturation and value) space. The latter one represents the reflection characteristic of the object surface in the background and helps decide whether the hue or saturation component should be used to detect changes in the statistical model. At the change detection stage, a multi-criteria decision is taken in order to deal with various light conditions such as shadows. Lastly, a speedup morphologic filter and a connected components labeling algorithm refine the segmentation results. The method makes full and better use of color information in images. The capability of accurate segmentation and real-time execution is demonstrated for several test and practical video sequences.

The MPEG-4 standard consists of object based coding schemes for multimedia and enables content-based manipulation. An approach based on information fusion, which can be used for the detection and location of video objects (people) is proposed in this paper. The method efficiently integrates color, edge and motion information of video streams. The HSV space provides a better use of the color information than that in the traditional RGB space. Using edge extracted from the motion region for contour adaptation provides an original method to eliminate the contour error. The approach integrating various characteristic features of video streams, when applied in the context of video surveillance and background replacement, performs real-time video object detection and location on a PC.

For effective collaborative working between the parties in a pattern design project team, it is essential that a highly-efficient and feasible cooperative platform is available. This paper presents an agent-based infrastructure for the automated collaborative design of textile industrial patterns. Cooperative awareness among distributed participants is one of the most important issues associated with collaborative pattern design. Several techniques based on awareness intensity management are proposed. In addition, the Pattern Knowledge Library enables intelligent design on the World Wide Web and thus greatly enhances the functionality of computer-supported cooperative design (CSCD). This prototype is intended to serve as a useful cooperative system for designers and should allow faster, better and more economic collaborative design of patterns.