Logs serve as a primary source of information for engineers to diagnose failures in large-scale online service systems. Log parsing, which extracts structured events from massive unstructured log data, is a critical first step for downstream tasks like anomaly detection and failure diagnosis. With advances in large language models (LLMs), leveraging their strong text understanding capabilities has proven effective for accurate log parsing. However, existing LLM-based log parsers all focus on the constant part of logs, ignoring the potential contribution of the variable part to log parsing. This constant-centric strategy brings four key problems. First, inefficient log grouping and sampling with only constant information. Second, a relatively large number of LLM invocations due to constant-based cache, leading to low log parsing accuracy and efficiency. Third, a relatively large number of consumed constant tokens in prompts leads to high LLM invocation costs. At last, these methods only retain placeholders in the results, losing the system visibility brought by variable information in logs.

Root cause Localization methods play a crucial role in ensuring the stability of large-scale microservice systems. However, existing methods either rely on unsupervised approaches with limited localization accuracy or on supervised learning that requires large volumes of historical anomaly data. Such data is often unavailable in newly deployed systems. To address this limitation, we attempt to use the labeled data from mature systems to help new systems build root cause localization models. Specifically, we propose UDA-RCL, an unsupervised domain adaptation root cause localization method using multimodal data (log, metric and trace data). UDA-RCL first incorporates an aggregation based event extraction module to standardize the format of multimodal data from different systems. Then, it utilizes a multimodal event encoder and multimodal domain adversarial adaptation module to narrow the feature distribution gap between different systems. Furthermore, taking into account the situation of sparse anomaly samples, existing methods' classifiers are either hard to transfer to other systems or struggle to capture the process of anomaly propagation, we propose a PageRank classifier module. This module employs a neural network embedded with anomaly propagation rules to output the final root cause ranking results, alleviating the issue of sparse anomaly samples. Extensive experiments have proven that our method achieves the best results in both supervised and transfer learning scenarios.

As Large-scale Language Models (LLMs) advance, the development of engaging Role-Playing Conversational Agents (RPCAs) has gained prominence. Despite this progress, there is a notable absence of benchmarks designed around dialogues, rather than question-answering formats, to assess the effectiveness of RPCA interactions. This paper introduces the RAIDEN benchmark, containing a comprehensive dataset specifically developed for RPCA evaluation, comprising over 40,000 multi-turn utterances across 135 characters. The benchmark focuses on assessing particular dimensions at different stages of a conversation, facilitated through interactions conducted by annotators. This approach allows the evaluation phase to concentrate on specific response dimensions, and thus subjectivity in dialogue evaluation is reduced. To further enhance objectivity, evaluators compare responses from two different models rather than assessing a single response in isolation. Besides, we introduce RPCAJudger, a specialized judging LLM tailored for automatic RPCA evaluation. The evaluations conducted by RPCAJudger closely mirror human judgments, and its API-free methodology serves to prevent potential data leakage. All the models and all non-private leaderboard data will be made publicly available.

Logs record essential information about system operations and serve as a critical source for anomaly detection, which has generated growing research interest. Utilizing large language models (LLMs) within a retrieval-augmented generation (RAG) framework for log-based anomaly detection is an effective approach due to its strong generalization capabilities and efficient few-shot performance. However, the effectiveness of this method hinges on the quality of the knowledge source, which can be impacted by noise and changes within the software systems. Facing these problems, in this paper, we propose a novel log-based anomaly detection method named EagerLog, employing active learning to choose the logs for humans to label, thereby adding them to the knowledge source, thus enhancing the knowledge source and maintaining its quality. Our experiments on three open datasets (BGL, Thunderbird, Zookeeper) and one industrial dataset demonstrate that EagerLog can achieve 93.65% F1 score with approximately 10 labeled log sequences, surpassing existing methods by 15.32%.

Deep-learning-based face recognition (FR) systems are susceptible to adversarial examples in both digital and physical domains. Physical attacks present a greater threat to deployed systems as adversaries can easily access the input channel, allowing them to provide malicious inputs to impersonate a victim. This paper addresses the limitations of existing projector-camera-based adversarial light attacks in practical FR setups. By incorporating device-aware adaptations into the digital attack algorithm, such as resolution-aware and color-aware adjustments, we mitigate the degradation from digital to physical domains. Experimental validation showcases the efficacy of our proposed algorithm against real and spoof adversaries, achieving high physical similarity scores in FR models and state-of-the-art commercial systems. On average, there is only a 14% reduction in scores from digital to physical attacks, with high attack success rate in both white- and black-box scenarios.

As deepfake and adversarial attacks evolve, facial recognition systems are encountering increasingly diverse threats. Most existing face liveness detection algorithms focus on single tasks, like spoofing or deepfake attack detection. The corresponding datasets have limited coverage of attack methods, with original data mostly sourced from the internet or laboratory environments. Moreover, existing datasets lack textual annotations, particularly for attack clues, limiting algorithms’ ability to utilize semantic assistance from text. To address these issues, we propose a large-scale unified attack dataset, which includes newly collected facial videos from 5,000 individuals, along with generated videos corresponding to 52 face attack methods. The dataset contains 795k videos and 60k images across four different quality levels. Through semi-automated annotation, we provide detailed textual descriptions. This is the first face attack dataset with textual descriptions. Additionally, we propose a text-guided face attack detection method, demonstrating significant improvements in accuracy using fine-grained textual descriptions. Our dataset will be released at https://ms-ufad.github.io.

As Industrial Internet of Things (IIoT) software systems become increasingly complex, precise failure diagnosis has become both essential and challenging. Current log-based failure diagnosis methods lack scalability for different failure types. In IIoT software systems, the number of failure types is constantly growing, and retraining the model each time a new failure type is introduced is highly resource-intensive. Additionally, traditional log-based failure diagnosis models often require log parsing as a preliminary step, which can also be resource-consuming. To address these challenges, we propose a scalable log-based failure diagnosis method named ScalaLog. ScalaLog builds on RAG by utilizing LLM-based summarization to extract key log information, applying sample augmentation to increase the number of samples, and using CoT prompts to guide the LLM in failure diagnosis. Experiments on various public and real-world datasets demonstrate that ScalaLog significantly enhances failure diagnosis accuracy without the need for training or log parsing.

Anomaly detection in multivariate time series (MTS) plays a crucial role in various domains, particularly in multimedia. While significant progress has been made in modeling normal data patterns and detecting anomalies based on deviations, existing methods face challenges in capturing complex inter-variable relationships, particularly in the presence of asynchronous dependencies. To address the challenges of detecting anomalies in MTS with complex inter-variable relationships, we propose an asynchronous inter-variable relationship-aware anomaly detection method. This approach simultaneously extracts both synchronous and asynchronous feature pairs between variables and leverages attention mechanisms to automatically learn unified inter-variable relationships across these dependencies. Additionally, we utilize a memory network to store and dynamically update the normal patterns of inter-variable relationships, computing anomaly scores based on deviations in these relationships. Extensive experiments demonstrate that our method outperforms existing baseline approaches, achieving an average F1 score of 97.05% across five benchmark datasets. Ablation studies further validate the effectiveness of each component of our method.

Accurate job runtime prediction is critical for efficient scheduling in high-performance computing (HPC) platforms.For instance, precise predictions enable techniques such as backfilling, where small jobs are executed ahead of schedule to maximize resource utilization and enhance computational efficiency.However, existing runtime prediction methods primarily rely on job metadata (e.g., submission time, requested runtime, and required memory) while ignoring the content of job scripts, which limits their accuracy.To address this issue, we propose an Online Retrieval-Augmented Language Model (ORA) for job runtime prediction.ORA encodes both metadata and script information from historical jobs into feature vectors to form a database, enabling similarity-based retrieval to assist in predicting the runtime of new jobs.To address distribution shifts, ORA incrementally updates the database without requiring model retraining.Additionally, personalized retrieval mechanisms are employed to mitigate the * Co-corresponding author.

Accurately diagnosing the fault that causes the failure is crucial for maintaining the reliability of a microservice system after a failure occurs. Mainstream fault diagnosis approaches are data-driven and mainly rely on three modalities of runtime data: traces, logs, and metrics. Diagnosing faults with multiple modalities of data in microservice systems has been a clear trend in recent years because different types of faults and corresponding failures tend to manifest in data of various modalities. Accurately diagnosing faults by fully leveraging multiple modalities of data is confronted with two challenges: 1) how to minimize information loss when extracting features for data of each modality; 2) how to correctly capture and utilize the relationships among data of different modalities. To address these challenges, we propose FAMOS, a Fault diagnosis Approach for MicrOservice Systems through effective multi-modal data fusion. On the one hand, FAMOS employs independent feature extractors to preserve the intrinsic features for each modality. On the other hand, FAMOS introduces a new Gaussian-attention mechanism to accurately correlate data of different modalities and then captures the inter-modality relationship with a crossattention mechanism. We evaluated FAMOS on two datasets constructed by injecting comprehensive and abundant faults into an open-source microservice system and a real-world industrial microservice system. Experimental results demonstrate the FAMOS's effectiveness in fault diagnosis, achieving significant improvements in F1 scores compared to state-of-the-art (SOTA) methods, with an increase of 20.33 %.

Log-based anomaly detection is essential for maintaining software availability. However, existing log-based anomaly detection approaches heavily rely on fine-grained exact labels of log entries which are very hard to obtain in real-world systems. This brings a key problem that anomaly detection models require supervision signals while labeled log entries are unavailable. Facing this problem, we propose a new labeling strategy called inexact labeling that instead of labeling an log entry, system experts can label a bag of log entries in a time span. Furthermore, we propose MIDLog, a weakly supervised log-based anomaly detection approach with inexact labels. We leverage the multiinstance learning paradigm to achieve explicit separation of anomalous log entries from the inexact labeled anomalous log set so as to deduce exact anomalous log labels from inexact labeled log sets. Extensive evaluation on three public datasets shows that our approach achieves an F1 score of over 85% with inexact labels.

Commit messages concisely describe code changes in natural language and are important for software maintenance. Several approaches have been proposed to automatically generate commit messages, but they still suffer from critical limitations, such as time-consuming training and poor generalization ability. To tackle these limitations, we propose to borrow the weapon of large language models (LLMs) and in-context learning (ICL). Our intuition is based on the fact that the training corpora of LLMs contain extensive code changes and their pairwise commit messages, which makes LLMs capture the knowledge about commits, while ICL can exploit the knowledge hidden in the LLMs and enable them to perform downstream tasks without model tuning. However, it remains unclear how well LLMs perform on commit message generation via ICL. In this paper, we conduct an empirical study to investigate the capability of LLMs to generate commit messages via ICL. Specifically, we first explore the impact of different settings on the performance of ICL-based commit message generation. We then compare ICL-based commit message generation with state-of-the-art approaches on a popular multilingual dataset and a new dataset we created to mitigate potential data leakage. The results show that ICL-based commit message generation significantly outperforms state-of-the-art approaches on subjective evaluation and achieves better generalization ability. We further analyze the root causes for LLM's underperformance and propose several implications, which shed light on future research directions for using LLMs to generate commit messages.

Despite the remarkable performance of Deep Neural Networks (DNNs) in many domains, their vulnerability to adversarial attacks limits their application in real-world scenarios. Attackers can craft adversarial examples by adding imperceptible perturbations to the original samples, causing the DNN to misclassify. To better understand the characteristics of adversarial attacks, numerous black-box adversarial attack methods have been proposed, particularly decision-based methods. These methods typically start with an initial adversarial sample, pulling it toward the original sample (attack direction), while preserving the adversarial sample’s class (fitting direction). However, existing methods struggle to escape local optima and face conflicts between the fitting and attack directions, which limits their query efficiency and attack success rate (ASR). To address these issues, we propose a black-box adversarial attack method based on orthogonal distribution evolution. Using a covariance matrix evolution strategy, the method fits the distribution of adversarial samples in the orthogonal direction to the attack direction, mitigating the conflict between the fitting and attack directions. Additionally, we design an adaptive search step adjustment strategy that probabilistically increases the step size in the orthogonal direction when better adversarial samples are hard to find, thus helping to escape local optima. Experimental results show that our method significantly outperforms existing baseline methods, achieving up to a 13% improvement in average ASR.

Log parsing is a prerequisite for log analysis. Recently, large language models (LLMs) have demonstrated high accuracy in log parsing. However, their frequent invocations incur substantial costs. To address this issue, some methods have turned to small language models (SLMs), which offer improved efficiency but suffer from reduced accuracy due to limited model capacity. To achieve both high accuracy and efficiency, we propose CSLParser, a collaborative log parsing framework using SLMs and LLMs. CSLParser delegates most log parsing tasks to SLMs and selectively invokes LLMs to correct parsing results generated by SLMs, thereby effectively reducing the invocation cost of LLMs while maintaining high accuracy. Specifically, to enhance the accuracy of SLMs, we propose a diversified sampling strategy to select diverse samples for training, enabling SLMs to effectively handle diverse log patterns. To efficiently invoke LLMs, we design a rule-based selection strategy to identify hard cases that are challenging for SLMs to correctly parse, which are subsequently corrected by LLMs. Additionally, we propose a dynamic template updating mechanism that merges similar templates based on structural and semantic information to further enhance parsing accuracy. Extensive experiments on public large-scale log datasets show that CSLParser outperforms state-of-the-art baselines in both accuracy and efficiency.

Log-based anomaly detection is an important task in ensuring the stability and reliability of software systems. One of the key problems in this task is the lack of labeled logs. Existing works usually leverage large-scale labeled logs from mature systems to train an anomaly detection model of a target system based on the idea of transfer learning. However, these works still require a certain number of labeled logs from the target system. In this paper, we take a step forward and study a valuable yet underexplored setting: zero-label cross-system log-based anomaly detection, that is, no labeled logs are available in the target system. Specifically, we propose ZeroLog, a system-agnostic representation meta-learning method that enables cross-system log-based anomaly detection under zero-label conditions. To achieve this, we leverage unsupervised domain adaptation to perform adversarial training between the source and target domains, aiming to learn system-agnostic general feature representations. By employing meta-learning, the learned representations are further generalized to the target system without any target labels. Experimental results on three public log datasets from different systems show that ZeroLog reaches over $\mathbf{8 0 \%}$ F1-score without labels, comparable to state-of-the-art cross-system methods trained with labeled logs, and outperforms existing methods under zero-label conditions.

Log parsing transforms log messages into structured formats, serving as a crucial step for log analysis. Despite a variety of log parsers that have been proposed, their performance on evolving log data remains unsatisfactory due to reliance on human-crafted rules or learning-based models with limited training data. The recent emergence of large language models (LLMs) has demonstrated strong abilities in understanding natural language and code, making it promising to apply LLMs for log parsing. Consequently, several studies have proposed LLM-based log parsers. However, LLMs may produce inaccurate templates, and existing LLM-based log parsers directly use the template generated by the LLM as the parsing result, hindering the accuracy of log parsing. Furthermore, these log parsers depend heavily on historical log data as demonstrations, which poses challenges in maintaining accuracy when dealing with scarce historical log data or evolving log data. To address these challenges, we propose AdaParser, an effective and adaptive log parsing framework using LLMs with self-generated in-context learning (SG-ICL) and self-correction. To facilitate accurate log parsing, AdaParser incorporates a novel component, a template corrector, which utilizes the LLM to correct potential parsing errors in the templates it generates. In addition, AdaParser maintains a dynamic candidate set composed of previously generated templates as demonstrations to adapt evolving log data. Extensive experiments on public large-scale datasets indicate that AdaParser outperforms state-of-the-art methods across all metrics, even in zero-shot scenarios. Moreover, when integrated with different LLMs, AdaParser consistently enhances the performance of the utilized LLMs by a large margin.

Log-based anomaly detection is a essential task for ensuring the reliability and performance of software systems. However, the performance of existing anomaly detection methods heavily relies on labeling, while labeling a large volume of logs is highly challenging. To address this issue, many approaches based on transfer learning and active learning have been proposed. Nevertheless, their effectiveness is hindered by issues such as the gap between source and target system data distributions and cold-start problems. In this paper, we propose LogAction, a novel log-based anomaly detection model based on active domain adaptation. LogAction integrates transfer learning and active learning techniques. On one hand, it uses labeled data from a mature system to train a base model, mitigating the cold-start issue in active learning. On the other hand, LogAction utilize free energy-based sampling and uncertainty-based sampling to select logs located at the distribution boundaries for manual labeling, thus addresses the data distribution gap in transfer learning with minimal human labeling efforts. Experimental results on six different combinations of datasets demonstrate that LogAction achieves an average 93.01% F1 score with only 2% of manual labels, outperforming some state-of-the-art methods by 26.28%. Website: https://logaction.github.io

Log-based fault diagnosis is essential for maintaining software system availability. However, existing fault diagnosis methods are built using a task-independent manner, which fails to bridge the gap between anomaly detection and root cause localization in terms of data form and diagnostic objectives, resulting in three major issues: 1) Diagnostic bias accumulates in the system; 2) System deployment relies on expensive monitoring data; 3) The collaborative relationship between diagnostic tasks is overlooked. Facing this problems, we propose a novel end-to-end log-based fault diagnosis method, Chimera, whose key idea is to achieve end-to-end fault diagnosis through bidirectional interaction and knowledge transfer between anomaly detection and root cause localization. Chimera is based on interactive multitask learning, carefully designing interaction strategies between anomaly detection and root cause localization at the data, feature, and diagnostic result levels, thereby achieving both sub-tasks interactively within a unified end-to-end framework. Evaluation on two public datasets and one industrial dataset shows that Chimera outperforms existing methods in both anomaly detection and root cause localization, achieving improvements of over 2.92%~5.00% and 19.01% ~ 37.09%, respectively. It has been successfully deployed in production, serving an industrial cloud platform.

Log-based software reliability maintenance systems are crucial for sustaining stable customer experience. However, existing deep learning-based methods represent a black box for service providers, making it impossible for providers to understand how these methods detect anomalies, thereby hindering trust and deployment in real production environments. To address this issue, this paper defines a trustworthiness metric—diagnostic faithfulness—for models to gain service providers’ trust, based on surveys of SREs at a major cloud provider. We design two evaluation tasks: attention-based root cause localization and event perturbation. Empirical studies demonstrate that existing methods perform poorly in diagnostic faithfulness. Consequently, we propose FaithLog, a faithful log-based anomaly detection system, which achieves faithfulness through a carefully designed causality-guided attention mechanism and adversarial consistency learning. Evaluation results on two public datasets and one industrial dataset demonstrate that the proposed method achieves state-of-the-art performance in diagnostic faithfulness.

Frequent software updates lead to log evolution, posing generalization challenges for current log anomaly detection. Traditional log anomaly detection research focuses on using small deep learning models (SMs), but these models inherently lack generalization due to their closed-world assumption. Large language models (LLMs) exhibit strong semantic understanding and generalization capabilities, making them promising for log anomaly detection. However, they suffer from computational inefficiencies. To balance efficiency and generalization, we propose a collaborative log anomaly detection scheme (CoorLog) that uses an adaptive coordinator to integrate SM and LLM. The coordinator determines if incoming logs have evolved. Non-evolved logs are routed to the SM, while evolved logs are directed to the LLM for detailed inference using the constructed Evol-CoT. To gradually adapt to evolution, we introduce the adaptive evolution mechanism (AEM), which updates the coordinator to redirect evolved logs identified by the LLM to the SM. Simultaneously, the SM is fine-tuned to inherit the LLM’s judgment on these logs. Extensive experiments on real-world datasets demonstrate that CoorLog achieves superior F1-scores in both intra-version and inter-version anomaly detection. Additionally, CoorLog reduces processing time by 91.63% and token consumption by 85.59% compared to using an LLM alone.

The development of effective diagnostic methodolo-gies for software system failures is of paramount importance. Traditional methods, which rely on specialized terminology and intricate reasoning, require users to have a technical background, resulting in reduced flexibility and decreased user-friendliness. With the rise of generative large language models, optimizing human-computer interaction has become a critical area of focus. Additionally, the inherent intelligence and extensive knowledge of large language models make them both easy and effective to employ for fault diagnosis assistance. We introduce IFKG, an advanced tool for diagnosing software system failures. IFKG integrates generative large language models with knowledge graphs, employing natural language interactions to implement fault detection and deliver solutions. IFKG enables users to upload descriptive problems, retrieve pertinent information from the knowledge graph, and present diagnostic results in natural language. Our accuracy assessments across diverse software system failures indicate that the IFKG provides targeted and actionable recommendations, effectively assisting users in ad-dressing a range of software system issues. The tool is available on GitHub at https://github.com/mako-xxlIFKG, and the demo video can be found on YouTube: https://youtu.belDie2vgZm2hk.

ABSTRACT Nowadays, more and more IT services are being hosted on cloud systems, which render cloud systems to grow into a huge complex with millions of physical servers, multi‐layer software stacks and the processing of cloud service requests across many servers and software layers. It is highly demanded for cloud service providers to have the capability of getting the knowledge on cloud service behaviour directly from the service execution instead of from people's expertise. This paper studies the problem of tracing cloud service's processing of requests across components in cloud environments and proposes cloud tracing mechanisms for this purpose. We also developed model‐based studies of our proposed mechanisms for analysing certain designs of the mechanisms. The implementation of the proposed cloud tracing is deployed onto the environments of OpenStack, Kubernetes and Hadoop, and the experiments on these environments demonstrate that our mechanisms effectively trace cloud service behaviour and generate a single complete request execution path, while without our mechanisms the cloud tracing either fails to work or results in thousands of path segments. Our mechanisms have a low performance overhead (2.3%) in the experiments.

Code generation models based on the pre-training and fine-tuning paradigm have been increasingly attempted by both academia and industry, resulting in well-known industrial models such as Codex, CodeGen, and PanGu-Coder. After being pre-trained on a large-scale corpus of code, a model is further fine-tuned with datasets specifically for the target downstream task, e.g., generating code from natural language description. The target code being generated can be classified into two types: a standalone function, i.e., a function that invokes or accesses only built-in functions and standard libraries, and a non-standalone function, i.e., a function that invokes or accesses user-defined functions or third-party libraries. To effectively generate code especially non-standalone functions (largely ignored by existing work), in this article, we present Wenwang, an approach to improving the capability of a pre-trained model on generating code beyond standalone functions. Wenwang consists of two components: a fine-tuning dataset named WenwangData and a fine-tuned model named WenwangCoder. Compared with existing fine-tuning datasets, WenwangData additionally covers non-standalone functions. Besides the docstring and code snippet for a function, WenwangData also includes its contextual information collected via program analysis. Based on PanGu-Coder, we produce WenwangCoder by fine-tuning PanGu-Coder on WenwangData with our context-aware fine-tuning technique so that the contextual information can be fully leveraged during code generation. On CoderEval and HumanEval, WenwangCoder outperforms three state-of-the-art models with similar parameter sizes (at the scale of around 300M), namely CodeGen, PanGu-Coder, and PanGu-FT. Although WenwangCoder does not outperform ChatGPT on HumanEval, WenwangCoder with smaller model parameter sizes can achieve similar effects to ChatGPT on CoderEval. Our experimental results also shed light on a number of promising optimization directions based on existing pre-trained models.

Log-based AIOps is a widely researched topic aiming at reducing the developer burden in system maintenance. Since industrial developers prefer lightweight supervised solutions for log-based AIOps, the strong dependence of these solutions on labeled data creates significant challenges for teams new to building log-based AIOps capabilities, such as high labeling costs, inconsistent annotations, and manual management issues. Log-labeling faces challenges in integrating existing artifacts to reduce labeling costs and manage labels effectively. To the best of our knowledge, no prior research addresses of assisting log-labeling problem. In this article, we propose a new approach called LogLabeler to assist developers in annotating and managing log labels. LogLabeler leverages existing artifacts for initial-label-acquisition, minimizes labeling costs by automatically generating all log labels, and shields developers from manual label management through a human-in-the-loop refinement approach. Evaluations on real-world datasets from Alibaba and open-source datasets show that LogLabeler can effectively supplement log labels, achieving comparable accuracy to existing baselines while operating more efficiently. Furthermore, we demonstrate LogLabeler's practical effectiveness at Alibaba through a case study, highlighting its benefits to developers.

Distributed storage systems are fundamental infrastructures of today’s large-scale software systems such as cloud systems. Diagnosing anomalies in distributed storage systems is essential for maintaining software availability. Existing anomaly diagnosis approaches mainly rely on the run-time data including monitoring data and application logs. However, collecting and analyzing the run-time data requires huge computing, storage, and management costs. Typically, more fine-grained run-time data can reveal more symptoms of anomalies, but on the contrary, requires more computing, storage, and management costs. As a result, solving the anomaly diagnosis problem is a balancing between the quality of run-time data and system overhead or cost. In this paper, we take into account both data quality and system overhead or cost by introducing a new type of run-time data-Raft logs. Raft logs are naturally produced by distributed storage systems and collecting raft logs will not bring any extra system overhead. To verify the ability of Raft logs in reflecting anomalies, we conduct a comprehensive study on the interconnection between the anomalies and Raft logs. Based on the study, we propose an effectiveRaft-BasedAnomalyDiagnosis approach namedRBAD. For evaluation, we expose the first open-sourced comprehensive dataset with multiple runtime data containing both Raft logs, application logs and monitoring data. Experiments based on this dataset demonstrate RBAD’s superiority, outperforming monitoring-based methods by 15.38% and log-based methods by 53.10%.

Database Management Systems (DBMS) form the backbone of modern large-scale software systems, where reliable anomaly detection and diagnosis are essential for ensuring system availability. However, existing log-based methods often impose significant performance overhead by collecting large volumes of logs, which is impractical for DBMS requiring high read/write throughput. This paper addresses a critical yet underexplored challenge: how to balance logging granularity with runtime efficiency for effective anomaly management in databases. We presentE-Log, a novel fine-grained elastic log-based framework for anomaly detection and diagnosis. E-Log intelligently adjusts the amount and detail of logging based on system state—maintaining lightweight logging during normal operation for efficient anomaly detection, and triggering rich, informative logging only upon anomaly suspicion for accurate diagnosis. This adaptive strategy significantly reduces runtime overhead while preserving diagnostic precision. We implement E-Log on Apache IoTDB and evaluate it using benchmarks including TSBS, TPCx-IoT, and IoT-Bench. Experimental results show that E-Log improves anomaly detection accuracy by 3.15% and diagnosis performance by 9.32% compared to state-of-the-art methods. Moreover, it reduces log storage size by 43.53% and increases average write throughput by 26.22%. These results highlight E-Log's potential to enable efficient, accurate, and scalable anomaly management in high-performance database systems.

As software systems grow increasingly intricate, the precise detection of anomalies have become both essential and challenging. Current log-based anomaly detection methods depend heavily on vast amounts of log data leading to inefficient inference and potential misguidance by noise logs. However, the quantitative effects of log reduction on the effectiveness of anomaly detection remain unexplored. Therefore, we first conduct a comprehensive study on six distinct models spanning three datasets. Through the study, the impact of log quantity and their effectiveness in representing anomalies is qualifies, uncovering three distinctive log event types that differently influence model performance. Drawing from these insights, we propose LogCleaner: an efficient methodology for the automatic reduction of log events in the context of anomaly detection. Serving as middleware between software systems and models, LogCleaner continuously updates and filters anti-events and duplicative-events in the raw generated logs. Experimental outcomes highlight LogCleaner’s capability to reduce over 70% of log events in anomaly detection, accelerating the model’s inference speed by approximately 300%, and universally improving the performance of models for anomaly detection.

Code generation models based on the pre-training and fine-tuning paradigm have been increasingly attempted by both academia and industry, resulting in well-known industrial models such as Codex, CodeGen, and PanGu-Coder. To evaluate the effectiveness of these models, multiple existing benchmarks (e.g., HumanEval and AiXBench) are proposed, including only cases of generating a standalone function, i.e., a function that may invoke or access only built-in functions and standard libraries. However, non-standalone functions, which typically are not included in the existing benchmarks, constitute more than 70% of the functions in popular open-source projects, and evaluating models' effectiveness on standalone functions cannot reflect these models' effectiveness on pragmatic code generation scenarios (i.e., code generation for real settings of open source or proprietary code).

Log-based anomaly detection plays a crucial role in ensuring the stability of software. However, current approaches for log-based anomaly detection heavily depend on a vast amount of labeled historical data, which is often unavailable in many real-world systems. To mitigate this problem, we leverage the features of the abundant historical labeled logs of mature systems to help construct anomaly detection models of new systems with very few labels, that is, to generalize the model ability trained from labeled logs of mature systems to achieve anomaly detection on new systems with insufficient data labels. Specifically, we propose MetaLog, a generalizable cross-system anomaly detection approach. MetaLog first incorporates a globally consistent semantic embedding module to obtain log event semantic embedding vectors in a shared global space. Then it leverages the meta-learning paradigm to improve the model's generalization ability. We evaluate MetaLog's performance on four public log datasets (HDFS, BGL, OpenStack, and Thunderbird) from four different systems. Results show that MetaLog reaches over 80% F1-score when using only 1% labeled logs of the target system, showing similar performance with state-of-the-art supervised anomaly detection models trained with 100% labeled data. Besides, it outperforms state-of-art transfer-learning-based cross-system anomaly detection models by 20% in the same settings of 1% labeled training logs of the target system.

Log-based anomaly detection is an essential task in maintaining software reliability. Existing log-based anomaly detection approaches often consist of three key phases: log parsing, event embedding, and model construction. Event embedding efficiently extracts semantic information from log events and produces vector representations of log events. However, existing event embedding methods suffer from two key problems. First, semantic noises are buried in log events leading to inevitable gaps between the obtained semantics from log events and their essential meanings. Second, there exists a gap between general semantic embedding and the specific embedding requirement of anomaly detection tasks. To mitigate these problems and improve the quality of representations of log events, we propose a novel anomaly detection approach named LLMeLog. It leverages the capabilities of large language models (LLMs) to enrich the contents of log events with in-context learning techniques. Then it utilizes the enriched log events to fine-tune a pre-trained BERT model. At last, it trains a transformer-based anomaly detection model with the event representations produced by the pre-trained BERT model. Evaluation results on three public log datasets show that LLMeLog achieves the best performance across all datasets, boasting F1-scores exceeding 99%. Besides, when using only 10% of labeled data as training data, our approach can still achieve over 90% F1-scores.

Log-based anomaly detection plays a crucial role in maintaining the reliability of software systems. Unsupervised models are more suitable for real-world usage because they do not rely on huge data labeling efforts. However, their effectiveness is limited because of the lack of supervision of data labels. To balance model effectiveness and labeling efforts, existing approaches enhance model capabilities by incorporating relatively few but key human labels as a golden signal, thereby improving the model ability with acceptable labeling efforts. However, these methods still face limitations of complex human labels and insufficient utilization of human knowledge. In this paper, we introduce LogCAE, a two-stage log anomaly detection approach based on active learning and contrastive learning. It utilizes an unsupervised model to learn from unlabeled log data without human labels and incorporates human knowledge through active learning during online optimization. We employ contrastive learning to optimize the representation of log samples in feature space for more efficient usage of human labels. We conducted experiments on three distinct public log datasets (Thunderbird, BGL, and Zookeeper). The results show that our method improves 12.93% F1-score on average with 6.06% labeled data samples. Besides, our approach is more effective in utilizing human labels than state-of-the-art approaches.

Parser-based log compressors have been widely explored in recent years because the explosive growth of log volumes makes the compression performance of general-purpose compressors unsatisfactory. These parser-based compressors preprocess logs by grouping the logs based on the parsing result and then feed the preprocessed files into a general-purpose compressor. However, parser-based compressors have their limitations. First, the goals of parsing and compression are misaligned, so the inherent characteristics of logs were not fully utilized. In addition, the performance of parser-based compressors depends on the sample logs and thus it is very unstable. Moreover, parser-based compressors often incur a long processing time. To address these limitations, we propose Denum, a simple, general log compressor with high compression ratio and speed. The core insight is that a majority of the tokens in logs are numeric tokens (i.e. pure numbers, tokens with only numbers and special characters, and numeric variables) and effective compression of them is critical for log compression. Specifically, Denum contains a Numeric Token Parsing module, which extracts all numeric tokens and applies tailored processing methods (e.g. store the differences of incremental numbers like timestamps), and a String Processing module, which processes the remaining log content without numbers. The processed files of the two modules are then fed as input to a general-purpose compressor and it outputs the final compression results. Denum has been evaluated on 16 log datasets and it achieves an 8.7% -- 434.7% higher average compression ratio and 2.6× -- 37.7× faster average compression speed (i.e. 26.2 MB/S) compared to the baselines. Moreover, integrating Denum's Numeric Token Parsing module into existing log compressors can provide a 11.8% improvement in their average compression ratio and achieve 37% faster average compression speed.

Distributed databases are fundamental infrastructures of today's large-scale software systems such as cloud systems. Detecting anomalies in distributed databases is essential for maintaining software availability. Existing approaches, predominantly developed using Loghub-a comprehensive collection of log datasets from various systems-lack datasets specifically tailored to distributed databases, which exhibit unique anomalies. Additionally, there's a notable absence of datasets encompassing multi-anomaly, multi-node logs. Consequently, models built upon these datasets, primarily designed for standalone systems, are inadequate for distributed databases, and the prevalent method of deeming an entire cluster anomalous based on irregularities in a single node leads to a high false-positive rate. This paper addresses the unique anomalies and multivariate nature of logs in distributed databases. We expose the first open-sourced, comprehensive dataset with multivariate logs from distributed databases. Utilizing this dataset, we conduct an extensive study to identify multiple database anomalies and to assess the effectiveness of state-of-the-art anomaly detection using multivariate log data. Our findings reveal that relying solely on logs from a single node is insufficient for accurate anomaly detection on distributed database. Leveraging these insights, we propose MultiLog, an innovative multivariate log-based anomaly detection approach tailored for distributed databases. Our experiments, based on this novel dataset, demonstrate MultiLog's superiority, outperforming existing state-of-the-art methods by approximately 12%.

Microservices architecture has garnered extensive attention for its stability and scalability. However, in the complex and dynamic landscape of microservices systems, a incident in one service can propagate to others, resulting in significant economic losses and degraded user experiences. Therefore, the effective and precise localization of incidents in microservices systems becomes a critical concern. Previous research has leveraged runtime data (logs, metrics, call traces) and historical incident data to assist in root cause localization. However, due to the scarcity of business incidents (those causing severe impacts on business operations) and the fact that many incidents are reported by users, relevant run-time data and sufficient historical data are often unavailable, rendering previous methods impractical. In response to this challenge, we propose an online contrastive learning-based method for root cause localization of business incidents(OCRCL). We fully exploit incident tickets and the static dependency graph of services, integrating both textual semantic information and structural information from the dependency graph to discover root causes. Furthermore, we suggest that online contrastive learning can exhibit excellent performance with limited data and enable real-time model updates, making it better suited for industrial scenarios. Our approach demonstrates significant improvements over baseline methods across three real-world industrial datasets, highlighting its effectiveness in root cause localization.

Changes are inevitable and frequent in large-scale online service systems, which has been one of the leading causes that induce incidents. Change risk detection (CRD) aims to help engineers detect high-risk changes so that proactive actions can be taken to avoid incidents, which is vital for the availability and reliability of online service systems. Though some efforts have been dedicated to CRD, their performances are still far from satisfactory in practice. To better understand the practical challenges of CRD, we conducted the first empirical study on a large-scale online service system in Ant Group. Through this study, we identified four critical challenges, including poor interpretability, adaptation to diverse change types, indirect anomaly factors, and expected but false alarm anomalies. To address these challenges, we propose an effective and eXplainable Change Risk Detection framework named XCRD. XCRD can detect change-induced unexpected anomalies using multi-source data and provide explainable alerts for engineers to facilitate anomaly diagnosis and mitigation. We have successfully deployed XCRD in Ant Group for the past 14 months, demonstrating a significant performance improvement in CRD. We also discuss some successful cases and lessons learned during our study. To our knowledge, we are the first to deeply investigate CRD in industrial scenarios. We believe that our work can provide valuable insights for engineers and researchers to understand and improve CRD in practice.

Vertical federated learning (VFL) enables collaborative machine learning on vertically partitioned data with privacy-preservation. Most VFL methods face three daunting challenges in real-world applications. First, most existing VFL methods assume that at least one party holds the complete set of labels of all data samples. However, this assumption often violates the nature of many practical scenarios, where the parties only have partial labels. Second, the heterogeneity and dynamic of computational and communication resources in participated parties may cause the straggler problem and slow down training convergence. Third, the confidential label information could be exposed through malicious parties during VFL. To address these challenges, we propose a novel VFL algorithm named Cascade Vertical Federated Learning (CVFL), in which partitioned labels can be fully utilized to train neural networks with privacy-preservation. To mitigate the straggler problem, we design a novel optimization objective to increase straggler's contribution to the trained models. To mitigate the label privacy risks, we design a novel defense approach to protect the label privacy of CVFL. We conduct comprehensive experiments and the results demonstrate the effectiveness and efficiency of CVFL. Further, the proposed defense approach can achieve a better tradeoff between label privacy and model utility than two widely-used defense approaches.

To ensure the stability and reliability of microservice systems, timely and accurate anomaly detection is of utmost importance. Recently, considering the lack of labels in real-world scenarios and the collaborative and complementary relationships of multi-modal data in reflecting system anomalies, unsupervised multi-modal anomaly methods have been proposed. However, existing methods face challenges in effectively distinguishing normal hard samples (they are normal but hard to classify correctly) from anomalies. This is mainly caused by two aspects. First, the hard sample patterns are complex. Second, the convergence speed is inconsistent between hard and simple samples. To overcome these issues, we propose an unsupervised adversarial contrastive multi-modal anomaly detection method (UAC-AD). We utilize contrastive learning to help learn the complex patterns of hard samples and enlarge the distance between hard and anomaly samples. Meanwhile, the adversarial framework automatically identifies hard samples and fine-grained adjusts the training weights to each modality part of these hard samples. In this case, The hard sample problems of two aspects can be alleviated. We extensively evaluate UAC-AD on two open-source simulated datasets and a real industrial dataset from a large communication company. Extensive experimental results demonstrate the effectiveness of our approach in anomaly detection. We also release the code and dataset for replication and future research.

This paper proposes a Semi-Attention Partition (SAP) method to learn well-aligned part features for occluded person re-identification (re-ID). Currently, the mainstream methods employ either external semantic partition or attention-based partition, and the latter manner is usually better than the former one. Under this background, this paper explores a potential that the weak semantic partition can be a good teacher for the strong attention-based partition. In other words, the attention-based student can substantially surpass its noisy semantic-based teacher, contradicting the common sense that the student usually achieves inferior (or comparable) accuracy. A key to this effect is: the proposed SAP encourages the attention-based partition of the (transformer) student to be partially consistent with the semantic-based teacher partition through knowledge distillation, yielding the so-called semi-attention. Such partial consistency allows the student to have both consistency and reasonable conflict with the noisy teacher. More specifically, on the one hand, the attention is guided by the semantic partition from the teacher. On the other hand, the attention mechanism itself still has some degree of freedom to comply with the inherent similarity between different patches, thus gaining resistance against noisy supervision. Moreover, we integrate a battery of well-engineered designs into SAP to reinforce their cooperation (e.g., multiple forms of teacher-student consistency), as well as to promote reasonable conflict (e.g., mutual absorbing partition refinement and a supervision signal dropout strategy). Experimental results confirm that the transformer student achieves substantial improvement after this semi-attention learning scheme, and produces new state-of-the-art accuracy on several standard re-ID benchmarks.

Log-based anomaly detection is an essential aspect of maintaining software reliability, particularly in the context of microservice systems. However, existing log-based anomaly detection approaches rely on historical anomalous labeled data or require huge labeling efforts. This makes existing log-based anomaly detection approaches inefficient. In this paper, we propose AcLog, a novel anomaly detection approach that incorporates human knowledge to enhance model ability based on the framework of deep active learning. It incorporates an unsupervised model to learn from normal log data rather than historical anomalous labeled data and leverages active learning to incorporate human knowledge as a golden signal to augment the quality of training log data. Experiment results on three open log datasets and one log dataset collected from a real-world microservice system show that our approach improves over 7% F1-score with 5% labeled training data on average.

Log-based anomaly detection is becoming more and more important for maintaining the availability of modern microservice systems. Existing supervised/semi-supervised log anomaly detection models require a large amount of human-labeled logs for training which are hard to collect in real-world systems. Unsupervised models often perform poorly without explicit anomaly labels. To improve the performance of unsupervised models, in this paper, we first make an empirical study of existing unsupervised models to tackle the reason why they often produce unsatisfied results. We find that anomaly detection results produced by existing unsupervised models are significantly affected by two key problems including Not-Cover (NC) problem and Suspicious-Noise (SN) problem. To solve these problems, we propose a novel augmentation framework called AFALog. AFALog leverages the idea of active learning to incorporate human knowledge so as to augment data quality. It can support almost all existing unsupervised models and improve their performance. Our experiments on two open datasets and one dataset collected from a real-world microservice system demonstrate that DALog improves the F1-score by an average of 6.61%, with only 5.9% labeled training data.

In online service systems, a majority of incidents are caused by changes, which can influence user experience and cause huge economic loss. Experiences with a real-world, large-scale online service system show that more than half of the change-induced incidents are reported by users. Identifying root-cause changes for these incidents is challenging due to the inherent gap between user-perceived functional-level incident information and component-level change details. Inadequate causal knowledge also brings challenges. In this paper, we propose a novel causal knowledge mining based approach aiming at root-cause change identification for user-reported incidents named Raccoon. To bridge the gap between incidents and changes, it utilizes the fault tree and software product line to represent incidents and changes at the user-perceived functional level. They are also used as the backbone of causal knowledge. To overcome the lack of causal knowledge, Raccoon adopts efficient knowledge extraction and inference methods. Moreover, Raccoon provides recommendations at the software product line and change granularity to meet diverse demands of incident triage and root-cause change identification scenarios in incident management. We evaluate Raccoon on a real-world dataset collected in a large-scale online service system. The result shows that Raccoon significantly outperforms the state-of-the-art baseline approaches, which proves its effectiveness.

In online service systems, software changes cause a majority of incidents (i.e., unplanned interruptions and outages). Managing change-induced incidents efficiently is crucial for ensuring the reliability and availability of online service systems. Understanding the incidents can help improve change-induced incident management. The task is challenging because the life cycle of change-induced incidents is complicated due to diverse change deployment and incident resolution procedures. Detailed records of the incidents and changes, together with a comprehensive analysis, are needed to gain an in-depth understanding. In this paper, we conduct a qualitative and quantitative study on 231 change-induced incidents in a real-world, large-scale online service system. Detailed change tickets and incident timeline in the post-mortems provides extensive information about the incident life cycle, enabling us to understand each incident in depth. Based on the data, we give a generic model of the complicated life cycle of change-induced incidents. Following the model, we systematically study the whole life cycle of the incident, including the introduction and resolution stages, and answer what affects the efficiency of resolution. We obtain 9 major findings from our study. Based on the findings, we discuss existing techniques and promising future directions for improving change-induced incident management.

Software defect prediction can automatically locate defective code modules to focus testing resources better. Traditional defect prediction methods mainly focus on manually designing features, which are input into machine learning classifiers to identify defective code. However, there are mainly two problems in prior works. First manually designing features is time consuming and unable to capture the semantic information of programs, which is an important capability for accurate defect prediction. Second the labeled data is limited along with severe class imbalance, affecting the performance of defect prediction.In response to the above problems, we first propose a new unsupervised domain adaptation method using pseudo labels for defect prediction(UDA-DP). Compared to manually designed features, it can automatically extract defective features from source programs to save time and contain more semantic information of programs. Moreover, unsupervised domain adaptation using pseudo labels is a kind of transfer learning, which is effective in leveraging rich information of limited data, alleviating the problem of insufficient data.Experiments with 10 open source projects from the PROMISE data set show that our proposed UDA-DP method outperforms the state-of-the-art methods for both within-project and cross-project defect predictions. Our code and data are available at https://github.com/xsarvin/UDA-DP.

With the increasing scale and complexity of cloud platforms and big-data analytics platforms, it is becoming more and more challenging to understand and diagnose the processing of a service request across multi-layer software stacks of such platforms. One way that helps to deal with this problem is to accurately capture the complete end-to-end execution path of service requests among all involved components. This paper presents REPTrace, a generic methodology for capturing such execution paths in a transparent fashion. Moreover, this paper demonstrates the effectiveness of REPTrace by presenting how REPTrace can be leveraged for knowledge extraction and anomaly detection on the platforms’ request processing. Our experimental results show that, REPTrace enables capturing a holistic view of the request processing across multiple layers of the platforms (which is missing in official documentation) and discovering important undocumented features of the platforms. Fault injection experiments show execution anomalies are detected with 93% precision and 96% recall with aid of REPTrace.

Chinese spam text detection is essential for social media since these texts affect the user experience of Chinese speakers and pollute the community. The underlying text classification method is employed to explore the unique combinations of characters that represent clues of spam information from annotated or further augmented data. However, based on the diversity of Chinese characters in glyphs, the spammers frequently wrap the spam content in another visually close text to fool the model but make sure people understand. This paper proposes to adopt the essence of human cognition of these adversarial texts into spam text detection models, by designing a pre-trained model to learn the morphology semantics of Chinese characters and represent their contextual meanings from scratch. The model pre-trains on self-supervised Chinese corpus and fine-tunes on spam-annotated community texts. Besides, cooperating with the pre-trained model that can capture the morphological features of Chinese, a new data perturbation method is introduced to guide the optimization towards the direction of recognizing the actual meaning of a text after spammers tamper with partial characters by visually close ones. The experimental results have shown that our proposed methodology can notably improve the performance of spam text detection as well as maintain robustness against adversarial samples.

In knowledge-grounded dialogue generation, pre-trained language models (PLMs) can be expected to deepen the fusing of dialogue context and knowledge because of their superior ability of semantic understanding. Unlike adopting the plain text knowledge, it is thorny to leverage the structural commonsense knowledge when using PLMs because most PLMs can only operate plain texts. Thus, linearizing commonsense knowledge facts into plan text is a compulsory trick. However, a dialogue is always aligned to a lot of retrieved fact candidates; as a result, the linearized text is always lengthy and then significantly increases the burden of using PLMs. To address this issue, we propose a novel two-stage framework SAKDP. In the first pre-screening stage, we use a ranking network PriorRanking to estimate the relevance of a retrieved knowledge fact. Thus, facts can be clustered into three sections of different priorities. As priority decreases, the relevance decreases, and the number of included facts increases. In the next dialogue generation stage, we use section-aware strategies to encode the linearized knowledge. The powerful but expensive PLM is only used for a few facts in the higher priority sections, reaching the performance-efficiency balance. Both the automatic and human evaluation demonstrate the superior performance of this work.

Vertical federated learning (VFL) enables collaborative machine learning on vertically partitioned data with privacy-preservation, attracting widespread attentions from academia and industry. Most existing VFL methods face two daunting challenges in real-world applications. First, most VFL methods assume at least one party holds the complete set of labels of all data samples. However, this assumption often violates the nature of many scenarios, where the parties only have partial labels. Second, the limitation of computational and communication resources in participated parties may cause the straggler problem and slow down training convergence. To address these challenges, we propose a novel VFL algorithm named Cascade Vertical Federated Learning (CVFL), in which partitioned labels can be fully utilized to train neural networks. To mitigate the straggler problem, we design a novel optimization objective to increase straggler's contribution to the trained models. We conduct comprehensive experiments and the results demonstrate the effectiveness and efficiency of CVFL.

Understanding the microarchitectural resource characteristics of datacenter jobs has become increasingly critical to guarantee the performance of jobs while improving resource utilization. Prior work studied the resource characteristics of datacenter jobs at the OS level, little reveals the deep and detailed characteristics at the microarchitecture level due to the lack of related open traces. In this paper, we provide a new open trace, AMTrace (Alibaba Microarchitecture Trace) 1, which is profiled from 8,577 high-end physical hosts from Alibaba’s datacenter by a hardware/software co-design monitoring method. AMTrace provides the microarchitectural metrics of 9.8 × 105 Linux containers with ”Per-Container-Per-Logic CPU” granularity. Different from existing open traces, AMTrace provides a new perspective to analyze the microarchitectural resource characteristics of datacenter jobs. Based on AMTrace, we first reveal the uneven resource usage of jobs among multiple logic CPUs. Then, we analyze the impact of resource contention of CPU and memory bandwidth on job performance. Finally, we analyze the job performance under different CPU provisioning modes from microarchitecture perspective. These analyses lead to constructive insights for datacenter resource management and optimization. Furthermore, we discuss possible research opportunities on AMTrace and we believe that AMTrace will inspire more exciting research on microarchitecture and resource management.

Unit testing could be used to validate the correctness of basic units of the software system under test. To reduce manual efforts in conducting unit testing, the research community has contributed with tools that automatically generate unit test cases, including test inputs and test oracles (e.g., assertions). Recently, ATLAS, a deep learning (DL) based approach, was proposed to generate assertions for a unit test based on other already written unit tests. Despite promising, the effectiveness of ATLAS is still limited. To improve the effectiveness, in this work, we make the first attempt to leverage Information Retrieval (IR) in assertion generation and propose an IR-based approach, including the technique of IR-based assertion retrieval and the technique of retrieved-assertion adaptation. In addition, we propose an integration approach to combine our IR-based approach with a DL-based approach (e.g., ATLAS) to further improve the effectiveness. Our experimental results show that our IR-based approach outperforms the state-of-the-art DL-based approach, and integrating our IR-based approach with the DL-based approach can further achieve higher accuracy. Our results convey an important message that information retrieval could be competitive and worthwhile to pursue for software engineering tasks such as assertion generation, and should be seriously considered by the research community given that in recent years deep learning solutions have been over-popularly adopted by the research community for software engineering tasks.

With the increasing complexity of modern software systems, it is essential yet hard to detect anomalies and diagnose problems precisely. Existing log-based anomaly detection approaches rely on a few key assumptions on system logs and perform well in some experimental systems. However, real-world industrial systems are often with poor logging quality, in which system logs are noisy and often violate the assumptions of existing approaches. This makes these approaches inefficient. This paper first conducts a comprehensive study on the system logs of three large-scale industrial software systems. Through the study, we identify four typical anti-patterns that affect the detection results the most. Based on these patterns, we propose HiLog, an effective human-in-the-loop log-based anomaly detection approach that integrates human knowledge to augment anomaly detection models. With little human labeling effort, our approach can significantly improve the effectiveness of existing models. Experiment results on three large-scale industrial software systems show that our method improves over 50% precision rate on average.

Cloud computing is growingly popular for hosting IT services, and is also growing into a huge complex with millions of physical servers, multi-layer software stacks and the processing of cloud service requests across many servers and software layers. It is highly demanded for cloud service providers to have the capability of getting the knowledge on cloud service behavior directly from the service execution instead of from people's expertise. This paper studies the problem of tracing cloud services' processing of requests across components in cloud environments, and proposes cloud tracing mechanisms for this purpose. The implementation of the proposed cloud tracing is deployed onto an OpenStack cloud environment, and the experiments performed on the cloud environment shows that our mechanisms effectively trace cloud service behavior and generate a single complete request execution path, while without our mechanisms the cloud tracing either could not work or results in thousands of path segments. Our mechanisms' performance overhead is low (2.3%).

Traditional conversational systems can only access the given query during the response generation, leading to meaningless responses. To this end, researchers proposed to enhance dialogue generation by integrating external knowledge. Although such methods have achieved remarkable gains, the use of only single-source knowledge often makes existing knowledge-enhanced methods degenerate into traditional models in real scenarios because of the insufficient knowledge coverage of single-source knowledge. To improve the applicability of knowledge-enhanced methods, we propose two novel frameworks to use heterogeneous knowledge from multiple sources. We first propose an MHKD-Seq2Seq framework, which can use different heterogeneous knowledge by identifying abstract-level knowledge behaviors; meanwhile, a Diffuse-Aggregate scheme is used to process multiple knowledge simultaneously and produce a unified result. The next framework MHKD-ARPLM can leverage the advantages of pretrained language models with Knowledge Linearization techniques. In experiments, we collected dialogues from previously open-released datasets and built a multi-source knowledge-aligned dataset TriKE-Weibo, which involves three knowledge sources: commonsense, texts, and infobox tables. Extensive evaluations demonstrate the performance leadership of our approaches against competitive baseline models.

Social recommendation, which leverages social connections to construct Recommender Systems (RS), plays an important role in alleviating information overload. Recently, Graph Neural Networks (GNNs) have received increasing attention due to their great capacity for graph data. Since data in RS is essentially in the structure of graphs, GNN-based RS is flourishing. However, existing works lack in-depth thinking of social recommendations. These methods contain implicit assumptions that are not well analyzed in practice. To tackle these problems, we conduct statistical analyses on widely used social recommendation datasets. We design metrics to evaluate the social information, which can provide guidance about whether and how we should use this information in the RS task. Based on these analyses, we propose a Distillation Enhanced SocIal Graph Network (DESIGN). We train a model that integrates information from the user-item interaction graph and the user-user social graph and train two auxiliary models that only use one of the above graphs respectively. These models are trained simultaneously, where the knowledge distillation technique restricts the training process and makes them learn from each other. Our extensive experiments show that our model significantly and consistently outperforms the state-of-the-art competitors on real-world datasets.

Pass selection and phase ordering are two critical compiler auto-tuning problems. Traditional heuristic methods cannot effectively address these NP-hard problems especially given the increasing number of compiler passes and diverse hardware architectures. Recent research efforts have attempted to address these problems through machine learning. However, the large search space of candidate pass sequences, the large numbers of redundant and irrelevant features, and the lack of training program instances make it difficult to learn models well. Several methods have tried to use expert knowledge to simplify the problems, such as using only the compiler passes or subsequences in the standard levels (e.g., -O1, -O2, and -O3) provided by compiler designers. However, these methods ignore other useful compiler passes that are not contained in the standard levels. Principal component analysis (PCA) and exploratory factor analysis (EFA) have been utilized to reduce the redundancy of feature data. However, these unsupervised methods retain all the information irrelevant to the performance of compilation optimization, which may mislead the subsequent model learning. To solve these problems, we propose a compiler pass selection and phase ordering approach, called Iterative Compilation based on Metric learning and Collaborative filtering (ICMC) . First, we propose a data-driven method to construct pass subsequences according to the observed collaborative interactions and dependency among passes on a given program set. Therefore, we can make use of all available compiler passes and prune the search space. Then, a supervised metric learning method is utilized to retain useful feature information for compilation optimization while removing both the irrelevant and the redundant information. Based on the learned similarity metric, a neighborhood-based collaborative filtering method is employed to iteratively recommend a few superior compiler passes for each target program. Last, an iterative data enhancement method is designed to alleviate the problem of lacking training program instances and to enhance the performance of iterative pass recommendations. The experimental results using the LLVM compiler on all 32 cBench programs show the following: (1) ICMC significantly outperforms several state-of-the-art compiler phase ordering methods, (2) it performs the same or better than the standard level -O3 on all the test programs, and (3) it can reach an average performance speedup of 1.20 (up to 1.46) compared with the standard level -O3.

Dialogues systems endow machines with the ability to converse with humans using natural language. Nonetheless, previous Seq2Seq-based generative dialogue systems often generate safe but meaningless responses, such as ‘I don't know' or ‘I think so'. To this end, researchers proposed to infuse external knowledge into dialogue generation, and such knowledge-enhanced methods have achieved remarkable improvements in the open-domain dialogue systems. External knowledge is an exogenous input, where the estrangement inevitably exists between knowledge and dialogue context. Although previous knowledge-enhanced works can already use commonsense knowledge to generate informative responses, they always use knowledge in a single-channel paradigm, which is hard to accurately handle different data-flows and then tends to generate irrational dialogue responses. Thus, they tend to be confused and generate strange responses when infusing the knowledge into dialogue generation, such as ‘I just ate a basketball’, dramatically degrading the user experience. To address this problem, this paper proposes a novelChannel-Aware Knowledge FusingNetwork (CAKF). Rather than following the traditional single-channel paradigm, CAKF employs three unique channels to handle different data-flows more clearly and rationally: abasechannel serves like a vanilla Seq2Seq decoder; acontextchannel to utilize the contextual information, and aknowledgechannel to infuse commonsense knowledge into the dialogue generation. Above such three channels, aSequential Manageris built to maintain the global sequential decision state, aggregate the local data-flows, and make the final prediction. Experiments on two open-released datasets (a Chinese Weibo and an English Reddit) demonstrated the superior performance of this work against various state-of-the-art approaches.

In recent years, applying deep learning to detect semantic code clones has received substantial attention from the research community. Accordingly, various evaluation benchmark datasets, with the most popular one as BigCloneBench, are constructed and selected as benchmarks to assess and compare different deep learning models for detecting semantic clones. However, there is no study to investigate whether an evaluation benchmark dataset such as BigCloneBench is properly used to evaluate models for detecting semantic code clones. In this article, we present an experimental study to show that BigCloneBench typically includes semantic clone pairs that use the same identifier names, which however are not used in non-semantic-clone pairs. Subsequently, we propose an undesirable-by-design Linear-Model that considers only which identifiers appear in a code fragment; this model can achieve high effectiveness for detecting semantic clones when evaluated on BigCloneBench, even comparable to state-of-the-art deep learning models recently proposed for detecting semantic clones. To alleviate these issues, we abstract a subset of the identifier names (including type, variable, and method names) in BigCloneBench to result in AbsBigCloneBench and use AbsBigCloneBench to better assess the effectiveness of deep learning models on the task of detecting semantic clones.

Despite achieving remarkable performance, previous knowledge-enhanced works usually only use a single-source homogeneous knowledge base of limited knowledge coverage.Thus, they often degenerate into traditional methods because not all dialogues can be linked with knowledge entries.This paper proposes a novel dialogue generation model, MSKE-Dialog, to solve this issue with three unique advantages: ( 1) Rather than only one, MSKE-Dialog can simultaneously leverage multiple heterogeneous knowledge sources (it includes but is not limited to commonsense knowledge facts, text knowledge, infobox knowledge) to improve the knowledge coverage; (2) To avoid the topic conflict among the context and different knowledge sources, we propose a Multi-Reference Selection to better select context/knowledge; (3) We propose a Multi-Reference Generation to generate informative responses by referring to multiple generation references at the same time.Extensive evaluations on a Chinese dataset show the superior performance of this work against various state-of-the-art approaches.To our best knowledge, this work is the first to use the multi-source heterogeneous knowledge in the open-domain knowledge-enhanced dialogue generation.

The availability of enormous event data with timestamps has aroused the study of Dynamic Knowledge Graphs (KGs). In dynamic KGs, temporal link prediction is an important task, which predicts future interactions between entities. Compared with conventional statistic link prediction tasks, temporal link prediction has three main challenges: i) How to deal with new entities that we have not observed before. ii) How to model the temporal evolutionary patterns. iii) How to adapt to the changes in KGs without re-training the model. To deal with these challenges, we present a novel reinforcement learning approach with an update mechanism to integrate temporal information. To predict future events, we train a time-aware agent to navigate the graph conditioned on the input query to find predictive paths. The experimental results indicate a clear improvement over the state-of-the-art methods.

Nowadays, web data is often used to train a dialogue system. However, noises in web data can disturb the training process, as well as can impact the performance. Consequently, dialogue models tend to be brittle when receiving noisy inputs during the inference. This paper proposes a novel framework, Multi-Path Training (MPT), for training a robust dialogue response generation system. MPT improves the robustness to the noisy training data and the noisy inference queries using three paths. Experimental results show MPT can outperform baselines using the same backbone model, and also prove MPT can improve the robustness to the noise in both the training and inference stage.

Graph Neural Networks (GNNs) have become the predominant technique for learning over graphs. However, there is little understanding of why GNNs are successful in practice. In this paper, we revisit GNNs for node classification tasks and find that GNNs mainly gain from iterative message aggregation, performing embedding transformation during aggregation leads to over-fitting and poor performance. Furthermore, the over-fitting problem is more severe in heterogeneous GNNs because of more complex operations. Driven by these findings, we propose a highway heterogeneous graph attention network that decouples message aggregation and embedding transformation to alleviate the over-fitting problem. To model heterogeneous structures, we design a hierarchical attention mechanism and a node-type dependent gate mechanism. Extensive experimental results show that our model has superior performance over the state-of-the-art methods.

Due to limited knowledge carried by queries, traditional dialogue systems often face the dilemma of generating boring responses, leading to poor user experience. To alleviate this issue, this paper proposes a novel infobox knowledge-aware dialogue generation approach, HITA-Graph, with three unique features. First, open-domain infobox tables that describe entities with relevant attributes are adopted as the knowledge source. An order-irrelevance Hierarchical Infobox Table Encoder is proposed to represent an infobox table at three levels of granularity. In addition, an Infobox-Dialogue Interaction Graph Network is built to effectively integrate the infobox context and the dialogue context into a unified infobox representation. Second, a Hierarchical Infobox Attribute Attention mechanism is developed to access the encoded infobox knowledge at different levels of granularity. Last but not least, a Dynamic Mode Fusion strategy is designed to allow the Decoder to select a vocabulary word or copy a word from the given infobox/query. We extract infobox tables from Chinese Wikipedia and construct an infobox knowledge base. Extensive evaluation on an open-released Chinese corpus demonstrates the superior performance of our approach against several representative methods.

Today, software systems are getting increasingly large and complex and a short failure time may cause huge loss. Therefore, it is important to detect and diagnose anomalies accurately and timely. System logs are a straightforward and important source of information for anomaly detection and diagnosis. However, existing log-based approaches have three key limitations. First, they are not designed for processing real-time log streams. Second, they require restrictions on training log data. Third, they lack the adaptiveness to system update. To break through these limitations, we propose LogFlash, a real-time streaming anomaly detection and diagnosis approach that enables both training and detection in a real-time streaming processing manner. By assigning a dynamic pairwise transition rate to each template pair and model the transition possibility as typical power-law distribution, our approach achieves real-time model construction and updates. Experiment results show that it reduces over 5 times of training and detection time compared with the state-of-art works while maintaining the capability of accurate anomaly diagnosis.

Generative dialogue systems tend to produce generic responses, which often leads to boring conversations.For alleviating this issue, Recent studies proposed to retrieve and introduce knowledge facts from knowledge graphs.While this paradigm works to a certain extent, it usually retrieves knowledge facts only based on the entity word itself, without considering the specific dialogue context.Thus, the introduction of the context-irrelevant knowledge facts can impact the quality of generations.To this end, this paper proposes a novel commonsense knowledge-aware dialogue generation model, ConKADI.We design a Felicitous Fact mechanism to help the model focus on the knowledge facts that are highly relevant to the context; furthermore, two techniques, Context-Knowledge Fusion and Flexible Mode Fusion are proposed to facilitate the integration of the knowledge in the ConKADI.We collect and build a large-scale Chinese dataset aligned with the commonsense knowledge for dialogue generation.Extensive evaluations over both an open-released English dataset and our Chinese dataset demonstrate that our approach ConKADI outperforms the state-of-the-art approach CCM, in most experiments.

Insufficient semantic understanding of dialogue always leads to the appearance of generic responses, in generative dialogue systems. Recently, high-quality knowledge bases have been introduced to enhance dialogue understanding, as well as to reduce the prevalence of boring responses. Although such knowledge-aware approaches have shown tremendous potential, they always utilize the knowledge in a black-box fashion. As a result, the generation process is somewhat uncontrollable, and it is also not interpretable. In this paper, we introduce a topic fact-based commonsense knowledge-aware approach, TopicKA. Different from previous works, TopicKA generates responses conditioned not only on the query message but also on a topic fact with an explicit semantic meaning, which also controls the direction of generation. Topic facts are recommended by a recommendation network trained under the Teacher-Student framework. To integrate the recommendation network and the generation network, this paper designs four schemes, which include two non-sampling schemes and two sampling methods. We collected and constructed a large-scale Chinese commonsense knowledge graph. Experimental results on an open Chinese benchmark dataset indicate that our model outperforms baselines in terms of both the objective and the subjective metrics.

Anomaly detection in distributed systems has been a fertile research area, and a range of anomaly detectors have been proposed for distributed systems. Unfortunately, there is no systematic quantitative study of the efficacy of different anomaly detectors, which is of great importance to reveal the deficiencies of existing anomaly detectors and shed light on future research directions. In this paper, we investigate how various anomaly detectors behave on anomalies of different types and the reasons for the same, by extensively injecting software faults into three widely-used distributed systems. We use a statementlevel fault injection method to observe the anomalies, characterize these anomalies, and analyze the detection results from anomaly detectors of three categories. We find that: (1) the distributed systems' own error reporting mechanisms are able to report most of the anomalies (from 82.1% to 92.8%) but they incur a high false alarm rate of 26.6%. (2) State-of-the-art anomaly detectors are able to detect the existence of anomalies with 99.08% precision and 90.60% recall, but there is still a long way to go to pinpoint the accurate location of the detected anomalies, and (3) Log-based anomaly detection techniques outperform other anomaly detection techniques, but not for all anomaly types.

Image hashing is critical for large-scale image analytic-based applications, such as image retrieval. Although there have been dozens of hashing approaches, few of them take the hierarchical structure of the image categories into consideration. In this paper, we propose to incorporate the category taxonomy information in a deep reinforcement learning (DRL) model for image hashing. In particular, we learn an agent to predict the hashing codes sequentially under the DRL theme. Each coordinate of the hashing function can take the errors incurred by previous ones into consideration and hence more reliable hashing codes can be obtained than learning them independently. Besides, we design a novel level-specific reward function to gradually refine the hashing function according to the taxonomy information. Extensive experiments on two popular datasets demonstrate effectiveness of the proposed method.

Bugs severely damage the reliability of open source software. In order to improve the reliability of open source software, bug tracking system is built to collect and manage bugs reported from users all over the world. When system failures occur, users investigate whether failures are induced by software bugs and then report bugs. However, it is usually difficult and time consuming to identify bugs from system failures. To accelerate bug reporting and reduce the time users spend on identifying bugs, we present BugIdentifier, an automatic bug identifying approach based on log mining. BugIdentifier combines Doc2Vec with Deep Neural Network (DNN) and treats bug identifying as a binary classification problem. Doc2Vec is adopted to train a log sequence embedding model that transforms log sequences into feature vectors, and then DNN is used to identify whether the log sequence is bug-induced or not. The results of our empirical evaluation show that our approach can automatically identify real-world bugs of Hadoop and OpenStack with the F1-score higher than 75%, specifically, old-version bugs of OpenStack can be identified with 97% F1-score, as a result, bug reporting can be accelerated correspondingly.

The goal of this paper is to automatically generate synthetic data to enable data analyzers to cope with the problem of insufficient data. Taking the most typical machine learning tasks, classification and regression, as an example, limited and insufficient samples cause low generalization of machine learning models, which cannot provide reasonable predictions. Data are insufficient either because of sample rarity or because data are impeded to be accessed for privacy concerns or confidential protection. To overcome this, we present a Synthetic Data Approach for Classification and Regression, adopting probability distribution and k-nearest neighbor model to generate synthetic data. We first estimate the probability distribution of each feature and construct a k-nearest neighbor model for all original data samples. Then we generate random samples based on probability distributions, adopt the k-nearest neighbor model to validate these random samples, and output the synthetic samples. We use proposed synthetic approaches to generate synthetic data of five publicly available datasets for classification and regression, respectively, and evaluate the performance of machine learning models to evaluate the resemblance between synthetic data and original data. The experimental results show that the synthetic data can resemble the original data, which indicates it is an effective approach for data analyzer to overcome the problem of insufficient data.

Recent years have witnessed a surge of interest on response generation for neural conversation systems. Most existing models are implemented by following the Encoder-Decoder framework and operate sentences of conversations at word-level. The word-level model is suffering from the Unknown Words Issue and the Preference Issue, which seriously impact the quality of generated responses, for example, generated responses may become irrelevant or too general (i.e. safe responses). To address these issues, this paper proposes a hybrid-level Encoder-Decoder model (HL-EncDec), which not only utilizes the word-level features but also character-level features. We conduct several experiments to evaluate HL-EncDec on a Chinese corpus, experimental results show our model significantly outperforms other non-word-level models in automatic metrics and human annotations and is able to generate more informative responses. We also conduct experiments with a small-scale English dataset to show the generalization ability.

Recent years have witnessed a surge of interest in building conversation systems such as smart agents or chatbots. The most existing generation-based neural responding conversation systems are implemented by RNN Encoder-Decoder framework relying on word-level modeling with explicit segmentation. A word-level model typically maintains a fixed vocabulary, which correspondingly encounters the unknown words and segmentation issues. In this paper, we proposed a fully character-level Encoder-Decoder model DLCEncDec without explicit segmentation for neural responding conversation. DLCEncDec utilizes both of fine-grained character embedding features and coarse-grained n-gram features. Coarse-grained n-gram features are captured by constructing a convolutional layer and a four-layer highway network on the top of the character embeddings. The appearance of out-of-vocabulary words (i.e. unknown words) can be addressed due to the fully character-level operating. We evaluate the DLCEncDec on a Chinese corpus consisting of 4.44 million message-response pairs from Sina Weibo. Experimental results show that our fully character-level model DLCEncDec significantly outperforms baseline models in terms of BLEU and ROUGE.

For massive data processing platforms such as Spark, configuration tuning is a necessary step since it is closely related to task parallelism, resource allocation and fault tolerance, which has a great influence on performance. However, to tune more than 190 interrelated configuration parameters of Spark for performance optimization is a challenging job. In this paper, a neural network based configuration tuning approach is proposed. In this approach, a neural network model is trained to predict the increase or decrease of configurations which determines the next search space. And a performance model based on random forest is used to improve search efficiency by predicting running time of jobs instead of running jobs actually. We evaluated the approach with four typical Spark applications. Experiment results show that compared to the default configuration, on average 42.8% of execution time of Spark applications are reduced by our approach. Moreover, the proposed approach outperforms over related approaches with optimal configuration and less search time.

It is desired (especially for young people) to shop for the same or similar products shown in the multimedia contents (such as online TV programs). This indicates an urgent demand for improving the experience of TV-to-Online (T2O). In this paper, a transfer learning approach as well as a prototype system for effortless T2O experience is developed. In the system, a key component is high-precision product search, which is to fulfill exact matching between a query item and the database ones. The matching performance primarily relies on distance estimation, but the data characteristics cannot be well modeled and exploited by a simple Euclidean distance. This motivates us to introduce distance metric learning (DML) for improving the distance estimation. However, in traditional DML methods, the side information (such as the similar/dissimilar constraints or relevance/irrelevance judgements) in the target domain is leveraged. These methods may fail due to limited side information. Fortunately, this issue can be alleviated by utilizing transfer metric learning (TML) to exploit information from other related domains. In this paper, a novel manifold regularized heterogeneous multitask metric learning framework is proposed, in which each domain is treated equally. The proposed approach allows us to simultaneously exploit the information from other domains and the unlabeled information. Furthermore, the ranking-based loss is adopted to make our model more appropriate for search. Experiments on two challenging real-world datasets demonstrate the effectiveness of the proposed method. This TML approach is expected to impact the transformation of the emerging T2O trend in both TV and online video domains.

Detecting execution anomalies is very important to monitoring and maintenance of cloud systems. People often use execution logs for troubleshooting and problem diagnosis, which is time consuming and error-prone. There is great demand for automatic anomaly detection based on logs. In this paper, we mine a time-weighted control flow graph (TCFG) that captures healthy execution flows of each component in cloud, and automatically raise anomaly alerts on observing deviations from TCFG. We outlined three challenges that are solved in this paper, including how to deal with the interleaving of multiple threads in logs, how to identify operational logs that do not contain any transactional information, and how to split the border of each transaction flow in the TCFG. We evaluate the effectiveness of our approach by leveraging logs from an IBM public cloud production platform and two simulated systems in the lab environment. The evaluation results show that our TCFG mining and anomaly diagnosis both perform over 80% precision and recall on average.

As the complex workload scheduling and resource allocating mechanism in big data system, programmers' configuration error is one of the most typical root causes of unsuccessful termination of jobs, which can result in performance deterioration, availability degradation, resource inefficiency and user unsatisfactory. In this paper, we propose an approach called SD-Predictor, to predict misconfiguration-induced unsuccessful executions of jobs combining static job configurations and dynamic runtime system state before scheduling and execution, so as to save computing resource and scheduling overheads in big data system. We implement and incorporate SD-Predictor with a popular scheduling framework YARN to optimize job scheduling so as to avoid negative impacts by misconfigured jobs. Moreover, we explore correlations between configurations and termination status of jobs and provide some recommendations for configuration optimization. The experiment results show that our approach performs at 78% of precision, 52% of recall and 2% of false positive rate in unsuccessful job prediction, with significantly better recall and false positive rate than related works.

Detecting runtime anomalies is very important to monitoring and maintenance of distributed services. People often use execution logs for troubleshooting and problem diagnosis manually, which is time consuming and error-prone. In this paper, we propose an approach for automatic anomaly detection based on logs. We first mine a hybrid graph model that captures normal execution flows inter and intra services, and then raise anomaly alerts on observing deviations from the hybrid model. We evaluate the effectiveness of our approach by leveraging logs from an IBM public cloud production platform and two simulated systems in the lab environment. Evaluation results show that our hybrid graph model mining performs over 80% precision and 70% recall and anomaly detection performs nearly 90% precision and 80% recall on average.

Software bugs have been one of the dominantcauses of system failures, especially in cloud systems basedon open source platforms. One big challenge fortroubleshooting these cloud systems is to pinpoint thesoftware bug-induced failure in large and complex log fileswhich is a nightmare for administrators. So far, there hasbeen little study on how to identity bug-induced failuresbased on log analysis. In this paper, we analyze and describefeatures of bug-induced failure logs from bug repository and Q&A websites, and then propose a general automaticapproach to pinpoint logs of bug-induced failure from logfiles of open cloud platform. In the approach, two algorithmscalled MPIN and SPIN are presented for log classification. We evaluate our approach by applying logs collected frombug repositories of OpenStack and Hadoop, and five Q&A websites. The experimental result shows that the proposedapproach can identify logs of bug-induced failure inOpenStack logs with 83.9% precision, and for Hadoop logswith 82.52% precision.

One of the major advantages of the cloud storage system is that it simplifies the time-consuming processes of hardware and software provisioning, deployment and distribution for users. Currently there is a tremendous increase in the scale of data generated as well as being consumed by applications on cloud storage systems, but besides availability metrics, few of cloud storage services provide data privacy guarantees in their Service Level Agreements (SLAs), which can be a major concern for the adoption of cloud storage services. The paper firstly proposes an analysis approach for the data privacy, then introduces privacy into the service quality, and defines a service quality evaluation model including privacy, availability, throughput, transfer time and operating cost. At last, a novel data distribution strategy called SLA-DO is presented. The comparison experiment results confirmed that SLA-DO strategy shows better performance in SLAs compliance, resource utilization, security guarantees and multiple cloud environmental adaptability than the data distribution policy adopted in HDFS and OpenStack.

Motivated by frequent failures in cloud computing systems, we analyze failure frequency and failure continuity of tasks from the Google cloud cluster, and find what we call killer tasks that suffer from frequent failures and repeated rescheduling. Killer tasks cause unnecessary resource wasting and significant increase of scheduling workloads, which can be a big concern in cloud systems. We aim to recognize killer tasks at the very early stage of their occurrence so that they can be addressed proactively instead of being rescheduled repeatedly, so as to promote reliability and save resources. To recognize killer tasks from a large amount of tasks in real time is really challenging. In this paper, we first investigate characteristics and behavior patterns of killer tasks and then develop two machine learning based methods, K-HUNTER and C-HUNTER, for online recognition of killer tasks. The empirical results show that our approach performs at 97% of precision in recognizing killer tasks with an 89% timing advance and 88% of resource saving for the cloud system on average.

Availability is one of the most essential attributes of qualities of cloud services. Most popular public cloud services claim availability commitments with corresponding penalties in their SLAs. To gain the maximal profits, cloud providers should choose an optimal penalty strategy in the competitive cloud market. In this paper, we firstly survey the penalty calculation methods of cloud providers. Based on the survey, we propose a competitive penalty model and a corresponding penalty based profit maximization algorithm for cloud providers. According to the model, each cloud provider would choose the best fit penalty strategy to gain the maximal expected profit during the game procedure. The proposed model is evaluated with real data of popular cloud providers with sensitive analysis, and is valuable for cloud providers to define their penalty strategy.

Availability is the most essential attribute of qualities of cloud services. Most popular public cloud services claim availability commitments with corresponding penalties in their SLAs. However, lack of clarity in availability commitment and penalty make it hard for consumers to understand the SLAs well and furthermore, to compare different cloud providers under different contexts, which would even become an obstacle for enterprise consumers to embrace public clouds. We present a cloud SLA availability commitment framework including availability calculation and penalty calculation services, and compare SLAs of well-known public IaaS cloud providers with investigation of their merits and defects. We also present a business model for cloud providers to find the optimal penalty degree for their SLAs, which will help in defining availability based SLA of cloud services.

This paper proposes a comparative study of cloud dependability between two methods -- bug analysis and fault injection for assessing the impact of component failure on cloud service availability. We focus on the IaaS cloud with open source platform Open Stack. The actual bug data are analyzed to show numerical examples of dependability assessment. A fault injection tool has also been developed to create failures of components and then observe their effects on services. The comparison analysis between two methods shows that bug analysis method has richer features for analyzing but not as precise as fault injection.

Hadoop Distributed File System (HDFS) has been widely adopted to support Internet applications because of its reliable, scalable and low-cost storage capability. Blue Sky, one of the most popular e-Learning resource sharing systems in China, is utilizing HDFS to store massive courseware. However, due to the inefficient access mechanism of HDFS, access latency of reading files from HDFS significantly impacts the performance of processing user requests. This paper introduces a two-level correlation based file prefetching approach, taking the characteristics of HDFS into consideration, to improve performance by reducing access latency. Four placement patterns to store prefetched data are presented, with policies to achieve trade-off between performance and efficiency of HDFS prefetching. Moreover, a dynamic replica selection algorithm is investigated to improve the efficiency of HDFS prefetching. The proposed prefetching approach has been implemented in Blue Sky, and experimental results prove that correlation based file prefetching can significantly reduce access latency therefore improve performance of Hadoop-based Internet applications.

Services computing paradigm together with Web services have significantly promoted the automation of business process in enterprise. Prevalent service composition technologies, such as WS-BPEL and WSCI, provide promising means to deal with machine-to-machine communication. Traditionally, in the phase of business process modeling, there usually require some human-involved tasks. Recent new technologies such as BPEL4People and Human Task begin to consider involving human interaction in business process. However, such approaches still have some limitations. On one hand, they exactly require some extensions of current BPEL standards. As a result, the existing business processes have to be rewritten and redeployed. On the other hand, they yet lack of the development and deployment supports of flexible and reusable user interfaces in business process. In this paper, we address these issues by enabling human interaction in business process with rich web applications. Our approach models human tasks as services, and can be seamlessly integrated to current BPEL without any modifications to existing engine and processes. We further support building human task presentations from service-oriented rich user interfaces. During the process execution, the corresponding task stakeholders can select, configure and compose these reusable and rich UI components according to their own application context.

Under the consolidation scenario in Clouds, the network dimension should be considered as important as the computing power of machines. Traditional consolidation procedure is usually made according to the experience, which mainly focused on the hardware capability of the target system, like CPU, Memory and etc. Along with the consolidation of the computing power, the network communication among machines is also consolidated. The consolidation procedure needs to cover this change and avoid network problems after moving the applications into target virtualized system. This paper presents a novel approach to provide optimization taking network traffic into account during consolidation.

Proposal of cloud computing is tightly coupled with low cost. Reduction of cost is considered as an important advantage of cloud. However, there are no available tools proper for cost calculation and analysis in Cloudenvironment. This paper presents our efforts towards filling in the gap. We format suits of metrics and formulas for the calculation of Cloud Total Cost of Ownership(TCO) and Utilization Cost, considering the elastic feature of Cloud infrastructure and widely adopted virtualization technology in Cloud. This provides afoundation for evaluating economic efficiency of Cloud and provides indications for cost optimization of Cloud.We have developed our calculation and analysis approach into a web tool which is used in the internal Cloud environment and demonstrate initially its analysis capability on the cost distribution and utilization imbalance factor.

Cloud platforms offer resource utilization as on demand service, which lays the foundation for applications to scale during runtime. However, just-in-time scalability is not achieved by simply deploying applications to cloud platforms. Existing approaches require developers to rewrite their applications to leverage the on-demand resource utilization, thus bind applications to specific cloud infrastructure. In this paper, profiles are used to capture expertspsila knowledge of scaling different types of applications. The profile-based approach automates the deployment and scaling of applications in cloud. Just-in-time scalability is achieved without binding to specific cloud infrastructure. A real case is used to demonstrate the process and feasibility of this profile-based approach.

The openness, heterogeneity, dynamism and autonomy characteristics of Internetware bring challenges to the management. In this paper, we introduce a web-delivered management service framework to manage Internetware systems. Through this framework, the Internetware systems can be managed centrally, and management functions can be easily added and applied by publishing and subscribing management services. The key innovations of this framework are: using an abstract managed object model to describe the units to be managed and their relationships, and every management function is manipulated as a service which can be further composed into processes. The benefits are management services and processes can be published and subscribed by administrators and domain experts, and the services can be delivered remotely. Both the management service and the process can be reused thus the total management cost can be reduced.

The fundamental characteristics of SOA, loose coupling and on-demand integration, enable organizations to seek more flexibility and responsiveness from their business IT systems. However, this brings challenges to assure QoS, especially availability, which should be considered in an integrated way in an SOA environment. Traditionally, availability is measured for each IT resource, but within SOA environments, rather than being considered individually, availability should be analyzed from an end-to-end view from both business and IT perspectives. In this paper, to address the availability problem of SOA, we propose a methodology that analyzes availability weak points in SOA deployment frameworks, leveraging workflow definitions that specify availability requirements at business level. This methodology includes an effective way to calculate high availability enhancement recommendations for a given SOA deployment topology with near-minimum cost, while meeting the business-level availability requirements. A prototype has been implemented as an extension to IBM's SOA deployment framework. Its efficiency and performance are analyzed here.

As Web services proliferate, the discovery and integration of appropriate services will become a challenging task particularly with services that share common similarities and functionalities. To assist requesters to find Web services of interest easily, we need to enhance the present service discovery mechanisms with ranking method. Enlightened by disciplines of Web 2.0, the anchor semantics enabled ranking method is proposed. In this method, the service requestor, as a consumer, can also become the producer to contribute not only the request description but also the usage reputation as part of service description. Based on case study and analogical analysis, we argue that this method improves recall and precise metrics at the same time.

A generalized declarative constraint framework is proposed in this paper to address challenges of applying constraints to model-driven SOA deployment and configuration. By representing deployment metamodel and domain knowledge as patterns, constraints can be easily described, automatically validated and analyzed. This framework reduces the human-intensiveness and error-proneness in the activity of applying constraints to a SOA deployment topology. We implemented this framework over a model-driven SOA deployment platform leveraging OCL as the declarative constraint language, and verified it in industry deployment scenarios.

Augmenting web services with explicit semantics forms the foundation of Service Oriented Architectures (SOAs) automation. As more and more Semantic Web Services (SWSs) are deployed, similar SWSs could have quite different quality-of-service (QoS) levels. The QoS-aware discovery becomes an important challenge. While some efforts try to solve it via Constraint Programming (CP), they suffer from the purely syntactic matchmaking method. Furthermore, the construction of constraints and the selection of services are completely dependent on the literal translation from QoS descriptions, which increase obstacles to actually apply CP. In this paper, we propose a semantic QoS-aware framework for SWSs discovery by combining the semantic matchmaking and CP. Initially, a QoS ontology is presented to define QoS data into service descriptions. Then the ontology reasoning is adopted to change previous syntactic matchmaking into a semantic way. Through confirming the compatibility of concepts, complex QoS conditions are solved as constraints and a selection algorithm is proposed to obtain the optimal offer. Finally, the prototype implementation of our framework is discussed and a SWSs discovery case is used to illustrate the comprehensive discovery process.

The proper operation of Service-Oriented Architecture (SOA) depends on underlying system services of operating systems, so efficient and effective migration of Service Hosting Environment is critical to cope with intrinsic-changed nature of SOA. However, due to the large amount of configuration items, complicated mapping and complex dependency relationship among system services, migrating into a new Service Hosting Environment satisfying the operation requirement of SOA becomes an error-prone and time-consuming task. The SCM project in IBM develops a novel approach to migrate Service Hosting Environment shaped in Unix-like systems. Firstly, this approach builds a set of configuration models to describe various system services. Then based on models, it presents knowledge based mapping to translate system service configurations between Service Hosting Environments. Finally, it designs a dependency hierarchy deducting algorithm to compute the dependency relationship among system services for migration traceability and error determination. A SCM prototype has performed well on largely reducing time, labor and errors in real migration cases.

Efficient problem detection methods play an important role in system management. In this paper, a formal method is described for problem detection in large scale and distributed enterprise IT environment. Events from distributed system components are collected, filtered and correlated. Leveraging these correlated events, the behavior of a distributed system is presented as a problem detection state machine (PDSM). PDSM is built up automatically from system logs without any specification of the target system. This approach combines logs from multi-sources and does not require any human involved or experimental instructions. It is generally applicable to a large class of distributed systems. Experimental results show that the implementation of PDSM performs problem detection efficiently in typical distributed enterprise systems.

Availability is one of the important factors to be considered for business-driven IT service management. This paper addresses the issue of analyzing what we call availability weak-points in an SOA deployment framework, leveraging workflow definitions to specify the high availability requirement at the business process level. In our weak-point analysis framework, we present an effective analysis methodology to calculate the optimal high availability solution with minimum cost, while meeting the business level availability requirements. We evaluate the weakpoint analysis methodology, and show that our methodology can identify a near-optimal solution for availability enhancement over the SOA deployment framework.

XML plays an important role in building enterprise applications. However, most of the XML-based applications, particularly the emerging Web services, suffer from low performance caused by XML processing and thus bring negative user experience in terms of response time. We argue that by reducing the considerable overhead in garbage collection the XML processing performance can be improved. We begin by conducting a set of experiments to understand the XML parser's memory characteristics, such as heap composition, object size and type distributions, object lifetime, and so on. Then, we get the valuable findings for improving performance that XML processing, which violates the weak generational hypothesis, is a memory allocation intensive workload in which most objects are small and long-lived. The findings can benefit the design of XML parsing specific GC and related tools designed to improve XML processing performance for Web services

The configuration of a service-based system has a significant impact on the nonfunctional requirements of the system as a whole. However, finding the best configuration is very challenging and sometimes impossible for administrators because so many factors have to be considered. More importantly, a service based system has to be frequently reconfigured to adapt to rapid and continuous changes in user requirements and runtime environments. In this paper we propose an autonomic computing approach to the problem of reconfiguration, that is, enabling the service based system to configure itself by means of a loop of monitoring, analyzing, planning and executing actions. We begin by formalizing the definition of configuration and reconfiguration. Then, we describe how we implemented the autonomic computing mechanisms for reconfiguring service-based systems to satisfy service level agreements with minimal resource consumption. The approach is demonstrated on a resilient service provisioning environment. Finally, the preliminary experiments are evaluated to determine the effectiveness of proposed approach.

Accurately and effectively detecting the locations where search queries are truly about has huge potential impact on increasing search relevance. In this paper, we define a search query's dominant location (QDL) and propose a solution to correctly detect it. QDL is geographical location(s) associated with a query in collective human knowledge, i.e., one or few prominent locations agreed by majority of people who know the answer to the query. QDL is a subjective and collective attribute of search queries and we are able to detect QDLs from both queries containing geographical location names and queries not containing them. The key challenges to QDL detection include false positive suppression (not all contained location names in queries mean geographical locations), and detecting implied locations by the context of the query. In our solution, a query is recursively broken into atomic tokens according to its most popular web usage for reducing false positives. If we do not find a dominant location in this step, we mine the top search results and/or query logs (with different approaches discussed in this paper) to discover implicit query locations. Our large-scale experiments on recent MSN Search queries show that our query location detection solution has consistent high accuracy for all query frequency ranges.

The dynamic nature of the Grid environment brings challenges to the services provisioning technique. A major aspect of Grid is to match demands for services with resources. Since resources and demands are fluctuating, Grid services provisioning need to be adjusted accordingly in responding to changes. In this paper, we propose a novel approach, Grid services migration, to enabling services provisioning on demand. The major motivation is to achieve the desired Quality of Service (QoS) such as performance, reliability, and etc. of Grid services on resources assembled dynamically responding to dynamic demand. In this paper, the technical characteristics of Grid services migration and its taxonomy are introduced. The Grid service migration algorithms are also presented. The Grid services migration framework with MAPE (Monitor, Analyze, Plan, and Execute) loop is proposed based on Open Grid Services Architecture (OGSA) and the prototype named Services Ecosystem, a resilient infrastructure for services provisioning on demand in Grid is developed. Demonstrations and experiments are conducted and results are presented to illustrate the capabilities of Grid services migration.

In this paper, we propose the concepts of virtual partner and inspector into the Web services composition. Virtual partner, as an IT level concept, is a Web service (pseudo Web service) using the same interface with the actual partner but different binding message. A virtual partner can be invoked directly by a business process described by BPEL, so that the BPEL programmer can test both application's functionality and non functionality performance early in the development cycle to avoid any problems in the final runtime, or test the selection of their partners in business level design. The IT virtual partners provide developers with a range of the techniques which let them explore every aspect of their program. Inspector is proposed when using the third-party process engine. An inspector itself is also a Web service. The programmer can register any required output information in it. The IT virtual partner and the inspector concepts have been integrated in our WSCE, a flexible Web Services Composition Environment for a business process. WSCE is a prototype of autonomic modeling and simulation environment. With the help of a third-party BPEL engine, it provides programmer with concepts and tools to facilitate business process programming.

Data warehouse and Online Analytical Processing(OLAP) play a key role in business intelligent systems. With the increasing amount of spatial data stored in business database, how to utilize these spatial information to get insight into business data from the geo-spatial point of view is becoming an important issue of data warehouse and OLAP. However, traditional data warehouse and OLAP tools can not fully exploit spatial data in coordinates because multi-dimensional spatial data does not have implicit or explicit concept hierarchy to compute pre-aggregation and materialization in data warehouse. In this paper we extend the traditional set-grouping hierarchy into multi-dimensional data space and propose to use spatial index tree as the hierarchy on spatial dimension. With spatial hierarchy, spatial data warehouse can be built accordingly. Our approach preserve the star schema in data warehouse while building the hierarchy on spatial dimension, and can be easily integrated into existing data warehouse and OLAP systems. To process spatial OLAP query in spatial data warehouse, we propose an OLAP-favored search method which can utilize the pre-aggregation result in spatial data warehouse to improve the performance of spatial OLAP queries. For generality, the algorithm is developed based on Generalized Index Searching Tree(GiST). To improve the performance of OLAP-favored search, we further introduce a heuristic search method which can provide an approximate answer to spatial OLAP query. Experiment result shows the efficiency of our method.