Despite the recent progress of audio-driven video generation, existing methods mostly focus on driving facial movements, leading to non-coherent head and body dynamics. Moving forward, it is desirable yet challenging to generate holistic human videos with both accurate lip-sync and delicate co-speech gestures w.r.t. given audio. In this work, we propose AudCast, a generalized audio-driven human video generation framework adopting a cascade Diffusion-Transformers (DiTs) paradigm, which synthesizes holistic human videos based on a reference image and a given audio. 1) Firstly, an audio-conditioned Holistic Human DiT architecture is proposed to directly drive the movements of any human body with vivid gesture dynamics. 2) Then to enhance hand and face details that are well-knownly difficult to handle, a Regional Refinement DiT leverages regional 3D fitting as the bridge to reform the signals, producing the final results. Extensive experiments demonstrate that our framework generates high-fidelity audio-driven holistic human videos with temporal coherence and fine facial and hand details. Resources can be found at https://guanjz20.github.io/projects/AudCast.

The non-consensual exploitation of facial manipulation has emerged as a pressing societal concern. In tandem with the identification of such fake content, recent research endeavors have advocated countering manipulation techniques through proactive interventions, specifically the incorporation of adversarial noise to impede the manipulation in advance. Nevertheless, with insufficient consideration of robustness, we show that current methods falter in providing protection after simple perturbations, e.g., blur. In addition, traditional optimization-based methods face limitations in scalability as they struggle to accommodate the substantial expansion of data volume, a consequence of the time-intensive iterative pipeline. To solve these challenges, we propose a learning-based model, Adversarial Robust Safeguard (ARS), to generate desirable protection noise in a single forward process, concurrently exhibiting a heightened resistance against prevalent perturbations. Specifically, our method involves a two-way protection design, characterized by a basic protection component responsible for generating efficacious noise features, coupled with robust protection for further enhancement. In robust protection, we first fuse image features with spatially duplicated noise embedding, thereby accounting for inherent information redundancy. Subsequently, a combination comprising a differentiable perturbation module and an adversarial network is devised to simulate potential information degradation during the training process. To evaluate it, we conduct experiments on four manipulation methods and compare recent works comprehensively. The results of our method exhibit good visual effects with pronounced robustness against varied perturbations at different levels.

Significant progress in video question answering (VideoQA) have been made thanks to thriving large image-language pretraining frameworks. Although image-language models can efficiently represent both video and language branches, they typically employ goal-free vision perception and do not interact vision with language well during the answer generation, thus omitting crucial visual cues. In this paper, we are inspired by the human recognition and learning pattern and propose VideoDistill, a framework with language-aware (i.e., goal-driven) behavior in both vision perception and answer generation. VideoDistill generates answers only from question-related visual embeddings and follows a thinking-observing-answering approach that closely resembles human behavior, distinguishing it from previous research. Specifically, we develop a language-aware gating mechanism to replace the standard cross-attention, avoiding language's direct fusion into visual representations. We incorporate this mechanism into two key components of the entire framework. The first component is a differentiable sparse sampling module, which selects frames containing the necessary dynamics and semantics relevant to the questions. The second component is a vision refinement module that merges existing spatial-temporal attention layers to ensure extracting multi-grained visual semantics associated with the questions. We conduct evaluations on various challenging video question-answering benchmarks, and VideoDistill achieves state-of-the-art performance in both general and long-form VideoQA datasets. In Addition, we verify that VideoDistill can effectively alleviate the utilization of language shortcut solutions in the EgoTaskQA dataset.

Teeth localization, segmentation, and labeling in 2D images have great potential in modern dentistry to enhance dental diagnostics, treatment planning, and population-based studies on oral health. However, general instance segmentation frameworks are incompetent due to 1) the sub-tle differences between some teeth’ shapes (e.g., maxillary first premolar and second premolar), 2) the teeth's position and shape variation across subjects, and 3) the presence of abnormalities in the dentition (e.g., caries and edentulism). To address these problems, we propose a ViT-based frame-work named TeethSEG, which consists of stacked Multi-Scale Aggregation (MSA) blocks and an Anthropic Prior Knowledge (APK) layer. Specifically, to compose the two modules, we design a unique permutation-based upscaler to ensure high efficiency while establishing clear segmentation boundaries with multi-head selflcross-gating layers to emphasize particular semantics meanwhile maintaining the divergence between token embeddings. Besides, we collect the first open-sourced intraoral image dataset IO150K, which comprises over 150k intraoral photos, and all photos are annotated by orthodontists using a human-machine hybrid algorithm. Experiments on IO150K demonstrate that our TeethSEG outperforms the state-of-the-art segmentation models on dental image segmentation.

Existing methods to fine-tune LLMs, like Adapter, Prefix-tuning, and LoRA, which introduce extra modules or ad-ditional input sequences to inject new skills or knowledge, may compromise the innate abilities of LLMs. In this paper, we propose LLaMA-Excitor, a lightweight method that stimulates the LLMs' potential to better follow instructions by gradually paying more attention to worthwhile information. Specifically, LLaMA-Excitor does not directly change the intermediate hidden state during the self-attention calculation. We designed the Excitor block as a bypass module that reconstructs Keys and changes the importance of Values in self-attention using learnable prompts. LLaMA-Excitor ensures a self-adaptive allocation of additional attention to input instructions, thus effectively preserving LLMs' pre-trained knowledge when fine-tuning LLMs on low-quality instruction-following datasets. Furthermore, we unify the modeling of multi-modal and language-only tuning, extending LLaMA-Excitor to a powerful visual instruction follower without the need for complex multi-modal alignment. Our approach is evaluated in language-only and multi-modal scenarios. Compared with the original LLaMA-7B, LLaMA-Excitor is the only PEFT method that maintains basic capabilities and achieves +3.12% relative improvement on the MMLU benchmark. In the visual instruction tuning, we achieve a new state-of-the-art image captioning performance on MSCOCO (157.5 CIDEr), and a comparable performance on ScienceQA (88.39%) to cutting-edge models with more parameters and extensive vision-language pertaining. The code will be available at https://zoubo9034.github.io/Excitor/.

Network traffic classification is a crucial research area aiming to enhance service quality, streamline network management, and bolster cybersecurity. To address the growing complexity of transmission encryption techniques, various machine learning and deep learning methods have been proposed. However, existing approaches face two main challenges. Firstly, they struggle with model inefficiency due to the quadratic complexity of the widely used Transformer architecture. Secondly, they suffer from inadequate traffic representation because of discarding important byte information while retaining unwanted biases. To address these challenges, we propose NetMamba, an efficient linear-time state space model equipped with a comprehensive traffic representation scheme. We adopt a specially selected and improved unidirectional Mamba architecture for the networking field, instead of the Transformer, to address efficiency issues. In addition, we design a traffic representation scheme to extract valid information from massive traffic data while removing biased information. Evaluation experiments on six public datasets encompassing three main classification tasks showcase NetMamba's superior classification performance compared to state-of-the-art baselines. It achieves an accuracy rate of nearly$99 \%$(some over$99 \%$) in all tasks. Additionally, NetMamba demonstrates excellent efficiency, improving inference speed by up to 60 times while maintaining comparably low memory usage. Furthermore, NetMamba exhibits superior few-shot learning abilities, achieving better classification performance with fewer labeled data. To the best of our knowledge, NetMamba is the first model to tailor the Mamba architecture for networking.

Artificial Intelligence of Things (AIoT) applications have advanced rapidly. However, most of them are inherently vulnerable to security threats and may be the source of spoofing attacks, and meanwhile, in AIoT systems, the transfer of real-time data from terminals and the cloud strains network bandwidth. To defend against attacks, save network forwarding resources, and relieve authentication pressure on the receiver end, it is essential to verify the source identity of AIoT terminals on the forwarding path. In this article, we propose PDIV, a probabilistic and distributed identity validation solution for AIoT applications. In the framework of PDIV, honest forwarders can verify the authenticity of the source identity of packets with a certain probability and filter spoofed packets to prevent them from spreading, reduce end-to-end network latency, and increase throughput as much as possible. Additionally, PDIV, a blockchain-based system that uses Merkle Patricia Trie (MPT) on the blockchain, makes it practical and efficient to realize distributed storage and verification of identity information. Moreover, we theorize about the tradeoff between PDIV network performance and detection effectiveness, as well as how PDIV enables defenses against different attacks, such as spoofing and Distributed Denial-of-Service attacks. Furthermore, we implement PDIV on network simulator version 3 (NS3) and evaluate its performance. The simulation results demonstrate that PDIV can prevent the spread of spoofed packets effectively and PDIV works better than currently available blockchain-based public-key infrastructure (PKI) approaches in terms of network latency.

Internet of Things (IoT) devices have achieved rapid development but most of them are vulnerable to spoofing attacks and spoofing-related attacks. It is crucial to verify source identity at the near-source end to defend against attacks, save network forwarding resources, and relieve the authentication pressure on the receiver end. In this paper, we propose Smart-PKI, a blockchain-based distributed identity validation scheme for IoT Devices. In the architecture of Smart-PKI, near-source forwarders can verify the authenticity of the source identity of packets and can filter spoofed packets. Besides, we apply Merkle Patricia Trie (MPT) to the Smart-PKI blockchain to enable lightweight blockchain copy storage and efficient retrieval and verification of identity information on forwarders. Meanwhile, Smart-PKI proposes an identity restoration mechanism and enables solutions for the attacks caused by public and private key compromise. Furthermore, we implement Smart-PKI on Network Simulator Version 3 (NS3) and evaluate its performance against reflection denial-of-service (DDoS) attacks. The simulation results demonstrate the effectiveness and efficiency of Smart-PKI and it outperforms existing blockchain-based PKI solutions for IoT devices in terms of network latency for verifying certificates.

Wireless networks are vulnerable to many attacks due to its dynamic environments. Unrestricted forwarding of packets from untrusted sources in wireless networks has caused many serious security threats and a great waste of network resources. This paper presents RepuFilter, a probabilistic packet filtering scheme based on trust evaluation, which enables a shared data plane to provide security services for wireless networks. RepuFilter proposes a dynamic trust evaluation model based on the transitivity of trust evaluation between network users, and applies the model to packet filtering. In the framework of RepuFilter, forwarders verify packets with a certain probability, and discard packets from untrusted sources, to prevent the packets from being spread, and reduce inspection expenses as much as possible. We implement RepuFilter and evaluate its performance based on Network Simulator Version 3 (NS3). Simulation results prove that RepuFilter can resist the spread of untrusted packets more effectively than the classic and latest trust models and RepuFilter can meet current network performance requirements,

Abuses of forgery techniques have created a considerable problem of misinformation on social media. Although scholars devote many efforts to face forgery detection (a.k.a DeepFake detection) and achieve some results, two issues still hinder the practical application. 1) Most detectors do not generalize well to unseen datasets. 2) In a supervised manner, most previous works require a considerable amount of manually labeled data. To address these problems, we propose a simple contrastive pertaining framework for DeepFake detection (DFCP), which works in a finetuning-after-pretraining manner, and requires only a few labels (5%). Specifically, we design a two-stream framework to simultaneously learn high-frequency texture features and high-level semantics information during pretraining. In addition, a video-based frame sampling strategy is proposed to mitigate potential noise data in the instance-discriminative contrastive learning to achieve better performance. Experimental results on several downstream datasets show the state-of-the-art performance of the proposed DFCP, which works at frame-level (w/o temporal reasoning) with high efficiency but outperforms video-level methods.

The emergence of photo-realistic deepfakes on a large scale has become a significant societal concern, which has garnered considerable attention from the research community. Several recent studies have identified the critical issue of “temporal inconsistency” resulting from the frame reassembling process of deepfake generation techniques. However, due to the lack of task-specific design, the spatio-temporal modeling of current methods remains insufficient in three critical aspects: 1) inapparent temporal changes are prone to be undermined compared to abundant spatial cues; 2) minor inconsistent regions are often concealed by motions with greater amplitude during downsampling; 3) capturing both transient inconsistencies and persistent motions simultaneously remains a significant challenge. In this paper, we propose a novel Dual-Modality Co-Learning framework tailored for these characteristics, which achieves more effectual deepfake detection with complementary information from RGB and optical flow modalities. In particular, we designed a Multi-Scale Motion Regularization module to encourage the network to equally prioritize both the significant spatial cues and the subtle temporal facial motion cues. Additionally, we developed a Multi-Span Cross-Attention module to effectively integrate the information from both RGB and optical flow modalities and improve the detection accuracy with multi-span predictions. Extensive experiments validate the effectiveness our ideas and demonstrate the superior performance of our approach.

The tremendous progress of vision-to-language retrieval over these years is fueled by contrastive vision-language pretraining (VLP), such as CLIP. Although, contrastive methods do not exhibit the same level of performance on other downstream tasks (e.g., video question answering and natural language grounding). One possible reason is they ignore the misalignment between vision and language, especially the absence of spatial information in language. To mitigate this issue, We start from a new perspective and propose a contrastive VLP framework with spatial reconstruction on text (SpaceCLIP). Specifically, we introduce a unique reconstruction method to assign text representations into the same spatial structure with images or videos and a pretraining objective, SpatialNCE, to reduce the computational overhead and ensure performance on downstream tasks. Empirically, we show SpaceCLIP outperforms other methods with performance gains ranging from 2.1% up to 9.0% on MSRVTT and EgoCLIP multiple-choice questions answering, 2.5% up to 11.0% on EPIC-KITCHENS-100 and MSRVTT multi-instance retrieval, and 0.31% up to 7.2% on Ego4D natural language query benchmark.

With current advancements in deep learning technology, it is becoming easier to create high-quality face forgery videos, causing concerns about the misuse of deepfake technology. In recent years, research on deepfake detection has become a popular topic. Many detection methods have been proposed, most of which focus on exploiting image artifacts or frequency domain features for detection. In this work, we propose using real images of the same identity as a reference to improve detection performance. Specifically, a real image of the same identity is used as a reference image and input into the model together with the image to be tested to learn the distinguishable identity representation, which is achieved by contrastive learning. Our method achieves superior performance on both FaceForensics++ and Celeb-DF with relatively little training data, and also achieves very competitive results on cross-manipulation and cross-dataset evaluations, demonstrating the effectiveness of our solution.

Recent advances in face forgery techniques produce nearly visually untraceable deepfake videos, which could be leveraged with malicious intentions. As a result, researchers have been devoted to deepfake detection. Previous studies have identified the importance of local low-level cues and temporal information in pursuit to generalize well across deepfake methods, however, they still suffer from robustness problem against post-processings. In this work, we propose the Local- & Temporal-aware Transformer-based Deepfake Detection (LTTD) framework, which adopts a local-to-global learning protocol with a particular focus on the valuable temporal information within local sequences. Specifically, we propose a Local Sequence Transformer (LST), which models the temporal consistency on sequences of restricted spatial regions, where low-level information is hierarchically enhanced with shallow layers of learned 3D filters. Based on the local temporal embeddings, we then achieve the final classification in a global contrastive way. Extensive experiments on popular datasets validate that our approach effectively spots local forgery cues and achieves state-of-the-art performance.

The massive amounts of monitoring data in network applications bring an urgent need for intelligent operation in large distributed systems. The key problem is precisely detecting anomalies in multivariate time series (MTS) monitoring metrics with the awareness of different application scenarios. Unsupervised MTS anomaly detection methods aim at detecting data anomalies from historical MTS without considering the out-of-band information (including user feedback and background information like code deployment status), which leads to poor performance in practice. To take advantage of the out-of-band information, we propose ACVAE, an MTS anomaly detection algorithm through active learning and contrast VAE-based detection models, which simultaneously learns MTS data’s normal and anomalous patterns for anomaly detection. We also use a learnable prior to capture system status from the background information. Moreover, we propose a query model for VAE-based methods, which can learn to query labels of the most useful instances to train the detection model. We evaluate our algorithm on three different monitoring situations in eBay’s search back-end systems.ACVAEachieves a range F1 score of 0.68~0.96 with only 3% labels, significantly outperforming the best competing methods by 0.18~0.50, and even better than a supervised ensemble method designed by domain experts in eBay.

Today's large datacenters house a massive number of machines, each of which is being closely monitored with multivariate time series (e.g., CPU idle, memory utilization) to ensure service quality. Detecting outlier machine instances with multivariate time series is crucial for service management. However, it is a challenging task due to the multiple classes and various shapes, high dimensionality, and lack of labels of multivariate time series. In this article, we propose DOMI, a novel unsupervised model that combines Gaussian mixture VAE with 1D-CNN, todetectoutliermachineinstances. Its core idea is to capture the normal patterns of machine instances by learning their latent representations that consider the shape characteristics, reconstruct input data by the learned representations, and apply reconstruction probabilities to determine outliers. Moreover, DOMI interprets the detected outlier instance based on the reconstruction probability changes of univariate time series. Extensive experiments have been conducted on the dataset collected from 1821 machines with a 1.5-month-period, which are deployed in ByteDance, a top global content service provider. DOMI achieves the best F1-Score of 0.94 and AUC score of 0.99, significantly outperforming the best performing baseline method by 0.08 and 0.03, respectively. Moreover, its interpretation accuracy is up to 0.93.

Anomaly detection is a crucial task for monitoring various status (i.e., metrics) of entities (e.g., manufacturing systems and Internet services), which are often characterized by multivariate time series (MTS). In practice, it's important to precisely detect the anomalies, as well as to interpret the detected anomalies through localizing a group of most anomalous metrics, to further assist the failure troubleshooting. In this paper, we propose InterFusion, an unsupervised method that simultaneously models the inter-metric and temporal dependency for MTS. Its core idea is to model the normal patterns inside MTS data through hierarchical Variational AutoEncoder with two stochastic latent variables, each of which learns low-dimensional inter-metric or temporal embeddings. Furthermore, we propose an MCMC-based method to obtain reasonable embeddings and reconstructions at anomalous parts for MTS anomaly interpretation. Our evaluation experiments are conducted on four real-world datasets from different industrial domains (three existing and one newly published dataset collected through our pilot deployment of InterFusion). InterFusion achieves an average anomaly detection F1-Score higher than 0.94 and anomaly interpretation performance of 0.87, significantly outperforming recent state-of-the-art MTS anomaly detection methods.

Nowadays the privacy issue arising in data mining applications has attracted much attention. In the context of distributed data mining, a major concern of the participant is that its privacy may be disclosed to other participants or a third party. To protect privacy, one can apply a differential privacy approach to perturb the data before sharing them with others, which generally causes a negative effect on the mining result. Thus there is a trade-off between privacy and the mining result. In this paper, we study a distributed classification scenario where a mediator builds a classifier based on the perturbed query results returned by a number of users. We propose a game theoretical approach to analyze how users choose their privacy budgets. Specifically, interactions among users are modeled as a game in satisfaction form. And an algorithm is proposed for users to learn the satisfaction equilibrium (SE) of the game. Experimental results demonstrate that, when the differences among users' expectations are not significant, the proposed learning algorithm can converge to an SE, at which every user achieves a balance between the accuracy of the classifier and the preserved privacy.

Many important cloud services require replicating massive data from one datacenter (DC) to multiple DCs. While the performance of pair-wise inter-DC data transfers has been much improved, prior solutions are insufficient to optimize bulk-data multicast, as they fail to explore the rich inter-DC overlay paths that exist in geo-distributed DCs, as well as the remaining bandwidth reserved for online traffic under fixed bandwidth separation scheme. To take advantage of these opportunities, we present BDS+, a near-optimal network system for large-scale inter-DC data replication. BDS+ is an application-level multicast overlay network with a fully centralized architecture, allowing a central controller to maintain an up-to-date global view of data delivery status of intermediate servers, in order to fully utilize the available overlay paths. Furthermore, in each overlay path, it leverages dynamic bandwidth separation to make use of the remaining available bandwidth reserved for online traffic. By constantly estimating online traffic demand and rescheduling bulk-data transfers accordingly, BDS+ can further speed up the massive data multicast. Through a pilot deployment in one of the largest online service providers and large-scale real-trace simulations, we show that BDS+ can achieve 3- 5× speedup over the provider's existing system and several well-known overlay routing baselines of static bandwidth separation. Moreover, dynamic bandwidth separation can further reduce the completion time of bulk data transfers by 1.2 to 1.3 times.

Microarchitectural timing channels, e.g., timing-based side channels or covert channels, endanger victims' data confidentiality by accurately measuring the time difference of accessing shared microarchitectural resources (e.g., cache and DRAM). Lowering the accuracy of such timers is a feasible and widely discussed direction for mitigating such channels. However, few solutions have paid attentions to the special dedicated thread timer, which measures the time by utilizing a dedicated thread to increment a time counter in an endless loop.In this paper, we present a novel approach, DegradeTimer, to degrade the dedicated thread timer. It first eliminates local cache sharing between the attacker thread and the timer thread in case the CPU's hyper-threading feature is enabled, by modifying Linux kernel's thread scheduling policy to enforce that any two threads sharing writable memory are dispatched to different physical cores. Then, it applies two novel cache coherence protocols, M-MESI and M-MOSEI, to randomly delay responses to remote cache requests. In this way, the attacker thread and the timer thread are dispatched to different cores and provided with fuzzy cache time transmission. We have implemented a prototype of DegradeTimer for both x86 and ARM architecture using the Linux kernel and the gem5 full system simulator. The evaluation results show that DegradeTimer provides a strong security guarantee against microarchitectural timing channels. Furthermore, the performance overhead introduced by DegradeTimer is less than 6%.

Clustering has long been an important research topic in machine learning, and is highly valuable in many application tasks. In recent years, many methods have achieved high clustering performance by applying deep generative models. In this paper, we point out that directly using q(z|y, x) instead of resorting to the mean-field approximation (as is adopted in previous works) in Gaussian Mixture Variational Auto-Encoder can benefit the unsupervised clustering task. We improve the performance of Gaussian Mixture VAE, by optimizing it with a Monte Carlo objective (including the q(z|y, x) term), with non-reparameterized Variational Inference for Monte Carlo Objectives (VIMCO) method. In addition, we propose std annealing to stabilize the training process and empirically show its effects on forming well-separated embeddings with different variational inference methods. Experimental results on five benchmark datasets show that our proposed algorithm NVISA outperforms several baseline algorithms as well as the previous clustering methods based on Gaussian Mixture VAE.

Shared resources facilitate stealthy communication channels, including side channels and covert channels, which greatly endanger the information security, even in cloud environments. As a commonly shared resource, DRAM memory also serves as a source of stealthy channels. Existing solutions rely on two common features of DRAM-based channels, i.e., high cache miss and high bank locality, to detect the existence of such channels. However, such solutions could be defeated. In this paper, we point out the weakness of existing detection solutions by demonstrating a new advanced DRAM-based channel, which utilizes the hardware Intel SGX to conceal cache miss and bank locality. Further, we propose a novel neural network based solution DRAMD to detect such advanced stealthy channels. DRAMD uses hardware performance counters to track not only cache miss events that are used by existing solutions, but also counts of branches and instructions executed, as well as branch misses. Then DRAMD utilizes neural networks to model the access patterns of different applications and therefore detects potential stealthy communication channels. Our evaluation shows that DRAMD achieves up to 99% precision with 100% recall. Furthermore, DRAMD introduces less than 5% performance overheads and negligible impacts on legacy applications.

Network management database (NID) is essential in modern large-scale networks. Operators rely on NID to provide accurate and up-to-date data, however, NID-like any other databases-can suffers from latent issues such as inconsistent, incorrect, and missing data. In this work, we first reveal latent data issues in NIDs using real traces from a large cloud provider, Tencent. Then we design and implement a diagnostic system, NAuditor, for unsupervised identification of latent issues in NIDs. In the process, we design a compact and graph-based data structure to efficiently encode the complete NID as a Knowledge Graph, and model the diagnostic problems as unsupervised Knowledge Graph Refinement problems. We show that the new encoding achieves superior performance than alternatives, and can facilitate adoption of state-of-the-art KGR algorithms. We also have used NAuditor in a production NID, and found 71 real latent issues, which all have been confirmed by operators.

Low Earth Orbit (LEO) satellite network is experiencing renewed interest due to its potential to revolutionize wide-area communications. However, it may suffer different failure modes than many traditional networks given its location in the complex space environment. In this paper, we address this problem by focusing on the survivability of LEO satellite network under link failures. Specifically, we investigate two kinds of guaranteed based approaches to solve this problem and combine them in a uniform framework using optimization method, e.g., linear programming: 1) protects against any combination of up to k concurrent link failures, for a configurable value k; 2) enables guarantees such as "user i is guaranteed binetwork bandwidth at least β% of the time". Simulation on up-to-date mega-constellation show that our proposed framework can protect delay-sensitive traffic from suffering long delay or data loss. In addition, they can support more traffic demands for the specified level of availability (e.g., 90%) under link failures.

Nowadays, masquerade attack caused by identity misuse is a kind of severe insider threat and a common security problem for most organizations. Although the anomaly detection based on machine learning has been applied as a common method for this problem, most existing approaches for detecting masquerade attack usually use host-based data to profile user behavior and detect anomaly. However, host-based data are too sensitive to collect, which results in poor universality and makes it difficult for enterprises to deploy. In this paper, we propose a cheap and general masquerade detection method with high accuracy to profile user network behavior and detect anomaly, using network packet sketches (IP, port, protocol, etc.). As network packet headers with standardized format are non-sensitive, our method is suitable for most enterprises and much cheaper to deploy. To validate the method, we collected more than 10 TB normal user data and 5 GB simulated masquerader data in real enterprise environment. The experimental results show that our method achieves good performance with average AUC up to 0.988 and the approach of modeling user network behavior by network packet sketches results in good time efficiency.

Internet-based service companies monitor a large number of KPIs (Key Performance Indicators) to ensure their service quality and reliability. Correlating KPIs by fluctuations reveals interactions between KPIs under anomalous situations and can be extremely useful for service troubleshooting. However, such a KPI flux-correlation has been little studied so far in the domain of Internet service operations management. A major challenge is how to automatically and accurately separate fluctuations from normal variations in KPIs with different structural characteristics (such as seasonal, trend and stationary) for a large number of KPIs. In this paper, we propose CoFlux, an unsupervised approach, to automatically (without manual selection of algorithm fitting and parameter tuning) determine whether two KPIs are correlated by fluctuations, in what temporal order they fluctuate, and whether they fluctuate in the same direction. CoFlux's robust feature engineering and robust correlation score computation enable it to work well against the diverse KPI characteristics. Our extensive experiments have demonstrated that CoFlux achieves the best F1-Scores of 0.84 (0.90), 0.92 (0.95), 0.95 (0.99), in answering these three questions, in the two real datasets from a top global Internet company, respectively. Moreover, we showed that CoFlux is effective in assisting service troubleshooting through the applications of alert compression, recommending Top N causes, and constructing fluctuation propagation chains.

Industry devices (i.e., entities) such as server machines, spacecrafts, engines, etc., are typically monitored with multivariate time series, whose anomaly detection is critical for an entity's service quality management. However, due to the complex temporal dependence and stochasticity of multivariate time series, their anomaly detection remains a big challenge. This paper proposes OmniAnomaly, a stochastic recurrent neural network for multivariate time series anomaly detection that works well robustly for various devices. Its core idea is to capture the normal patterns of multivariate time series by learning their robust representations with key techniques such as stochastic variable connection and planar normalizing flow, reconstruct input data by the representations, and use the reconstruction probabilities to determine anomalies. Moreover, for a detected entity anomaly, OmniAnomaly can provide interpretations based on the reconstruction probabilities of its constituent univariate time series. The evaluation experiments are conducted on two public datasets from aerospace and a new server machine dataset (collected and released by us) from an Internet company. OmniAnomaly achieves an overall F1-Score of 0.86 in three real-world datasets, signicantly outperforming the best performing baseline method by 0.09. The interpretation accuracy for OmniAnomaly is up to 0.89.

Despite many years of improvements to it, TCP still suffers from an unsatisfactory performance. For services dominated by short flows (e.g., web search and e-commerce), TCP suffers from the flow startup problem and cannot fully utilize the available bandwidth in the modern Internet: TCP starts from a conservative and static initial window (IW, 2-4 or 10), while most of the web flows are too short to converge to the best sending rate before the session ends. For services dominated by long flows (e.g., video streaming and file downloading), the congestion control (CC) scheme manually and statically configured might not offer the best performance for the latest network conditions. To address these two challenges, we propose TCP-RL, which uses reinforcement learning (RL) techniques to dynamically configure IW and CC in order to improve the performance of TCP flow transmission. Basing on the latest network conditions observed at the server side of a web service, TCP-RL dynamically configures a suitable IW for short flows through group-based RL, and dynamically configures a suitable CC scheme for long flows through deep RL. Our extensive experiments show that for short flows, TCP-RL can reduce the average transmission time by about 23%; and for long flows, compared with the performance of 14 CC schemes, TCP-RL's performance ranks top 5 for about 85% of the 288 given static network conditions, whereas for about 90% of conditions, its performance drops by less than 12% compared with that of the best-performing CC schemes for the same network conditions.

For large Internet companies, it is very important to monitor a large number of KPIs (Key Performance Indicators) and detect anomalies to ensure the service quality and reliability. However, large-scale anomaly detection on millions of KPIs is very challenging due to the large overhead of model selection, parameter tuning, model training, or labeling. In this paper we argue that KPI clustering can help: we can cluster millions of KPIs into a small number of clusters and then select and train model on a per-cluster basis. However, KPI clustering faces new challenges that are not present in classic time series clustering: KPIs are typically much longer than other time series, and noises, anomalies, phase shifts and amplitude differences often change the shape of KPIs and mislead the clustering algorithm. To tackle the above challenges, in this paper we propose a robust and rapid KPI clustering algorithm, ROCKA. It consists of four steps: preprocessing, baseline extraction, clustering and assignment. These techniques help group KPIs according to their underlying shapes with high accuracy and efficiency. Our evaluation using real-world KPIs shows that ROCKA gets F-score higher than 0.85, and reduces model training time of a state-of-the-art anomaly detection algorithm by 90%, with only 15% performance loss.

Latency, which directly affects the user experience and revenue of web services, is far from ideal in reality, due to the well-known TCP flow startup problem. Specifically, since TCP starts from a conservative and static initial window (IW, 2~4 or 10), most of the web flows are too short to have enough time to find its best congestion window before the session ends. As a result, TCP cannot fully utilize the available bandwidth in the modern Internet. In this paper, we propose to use group-based reinforcement learning (RL) to enable a web server, through trial-and-error, to dynamically set a suitable IW for a web flow before its transmission starts. Our proposed system, SmartIW, collects TCP flow performance metrics (e.g., transmission time, loss rate, RTT) in real-time without any client assistance. Then these metrics are aggregated into groups with similar features (subnet, ISP, province, etc.) to satisfy RL's requirement. SmartIW has been deployed in one of the top global search engines for more than a year. Our online and testbed experiments show that, compared to the common practice of IW=10, SmartIW can reduce the average transmission time by 23% to 29%.

To ensure undisrupted business, large Internet companies need to closely monitor various KPIs (e.g., Page Views, number of online users, and number of orders) of its Web applications, to accurately detect anomalies and trigger timely troubleshooting/mitigation. However, anomaly detection for these seasonal KPIs with various patterns and data quality has been a great challenge, especially without labels. In this paper, we proposed Donut, an unsupervised anomaly detection algorithm based on VAE. Thanks to a few of our key techniques, Donut greatly outperforms a state-of-arts supervised ensemble approach and a baseline VAE approach, and its best F-scores range from 0.75 to 0.9 for the studied KPIs from a top global Internet company. We come up with a novel KDE interpretation of reconstruction for Donut, making it the first VAE-based anomaly detection algorithm with solid theoretical explanation.

The privacy issue arising in data mining applications has attracted much attention in recent years. In the context of distributed data mining, the participant can employ data perturbation techniques to protect its privacy. Data perturbation generally causes a negative effect on the mining result, which means there is a trade-off between privacy and the mining result. In this paper, we study a distributed classification scenario where a number of users provide data to a mediator to train a classifier. Interactions among users are modeled as a game in satisfaction form. And an algorithm is proposed for users to learn the satisfaction equilibrium (SE) of the game. The basis idea is that the user gradually reduces the perturbation in data until it is satisfied with the classification accuracy. Experimental results based on real data demonstrate that, when the differences among users' expectations are not significant, the proposed learning algorithm can converge to an SE, at which every user achieves a balance between the classification accuracy and the preserved privacy.

Previous studies have shown the WiFi, as the dominant last hop access to Internet, has become the weakest link in the round-trip network delay. Therefore it is critical to understand and minimize the WiFi interference in order to reduce the WiFi hop delay. For the first time in the literature, this paper defines an intuitive and accurate metric to quantify the impact of interference on each actual packet. For each packet traveling through the access point, it measures the percentage of MAC layer delay wasted due to neighbor APs' interference. This metric is defined based on a packet's various (measured or inferred) timestamps and can be measured with a small kernel modification on a commodity AP with little overhead. Our 29-AP two- month measurement results in the wild show that this metric is a strong indicator of interference's impact on WiFi hop delay. Using this metric as input, distributed channel selection on individual APs reduces the median WiFi hop delay by up to 5X. Collaborative optimization on multiple APs reduces the overall WiFi hop delay by 5X compared to the default channel.

Today's WiFi networks deliver a large fraction of traffic. However, the performance and quality of WiFi networks are still far from satisfactory. Among many popular quality metrics (throughput, latency), the probability of successfully connecting to WiFi APs and the time cost of the WiFi connection set-up process are the two of the most critical metrics that affect WiFi users' experience. To understand the WiFi connection set-up process in real-world settings, we carry out measurement studies on 5 million mobile users from 4 representative cities associating with 7 million APs in 0.4 billion WiFi sessions, collected from a mobile “WiFi Manager” App that tops the Android/iOS App market. To the best of our knowledge, we are the first to do such large scale study on: how large the WiFi connection set-up time cost is, what factors affect the WiFi connection set-up process, and what can be done to reduce the WiFi connection set-up time cost. Based on our data-driven measurement and analysis, we reveal the insights as follows: (1) Connection set-up failure and large connection set-up time cost are common in today's WiFi use. As large as 45% of the users suffer connection set-up failures, and 15% (5%) of them have large connection set-up time costs over 5 seconds (10 seconds). (2) Contrary to the state-of-the-art work, scan, one of the subphase of four phases in the connection set-up process, contributes the most (47%) to the overall connection set-up time cost. (3) Mobile device model and AP model can greatly help us to predict the connection set-up time cost if we can make good use of the hidden information. Based on the measurement analysis, we develop a machine learning based AP selection strategy that can significantly improve WiFi connection set-up performance, against the conventional strategy purely based on signal strength, by reducing the connection set-up failures from 33% to 3.6% and reducing 80% time costs of the connection set-up processes by more than 10 times.

Current TCP is very inefficient for web services. Web transactions are often very short-lived. TCP flow starts with a conservative initial congestion window (IW), which causes multiple round-trip times to finish the transmission even if the end-to-end bandwidth is sufficient for the transaction to be finished in one round-trip time. Previous research efforts have been focusing on finding the overall best IW for the entire Internet or a service company. However, we observe that one-IW-fits-all is suboptimal after one year of online measurement in Baidu, one of the top global search engine companies. To reduce the TCP latency, we propose TCP WISE, which dynamically assigns suitable IWs for different user cluster at different times on the server side. The values of users' IWs are proactively learned based on the historical experience on the server-side. Our testbed experiments show that our learning algorithm can handle the network changes and converge to the best IW. We have deployed TCP WISE in one of Baidu's production data center, and results shows that the 80thpercentile latency of the HTTP responses has been reduced by 10.4% compared with current TCP with a fixed IW of 10.

WiFi has become the primary method to access the Internet. However, the WiFi-hop latency, particularly in dense-WiFi environments, is far from satisfactory [1], to support delay-sensitive applications such as Web browsing and VoIP. The WiFi latency mainly comes from two kinds of queues: the host queue and the distributed queue, which is caused by CSMA/CA mechanism when multiple nodes contend for the channel. While the host queue can be easily bypassed using priority scheduling at end-host, the distributed queue is not. Previously, IEEE 802.11e tries to provide priorities in this distributed queue by adjusting the MAC layer parameters, but it does not scale when there are increasing number of delay-sensitive flows. In this paper, we propose and design QAir, a practical solution to reduce WiFi latency of delay-sensitive flows in dense WiFi networks. QAir takes a different approach to transfer this distributed queue to host queue. Consequently, the delay-sensitive flows can bypass the entire queue and their latency can be greatly reduced. QAir works in a distributed manner with no centralized scheduler. We have implemented QAir on commodity WiFi devices. Experimental results show that, compared to the 802.11 DCF baseline, QAir can reduce the average WiFi-hop latency of delay-sensitive flows by 50-75%.

Recently, the conflict between exploiting the value of personal data and protecting individuals' privacy has attracted much attention. Personal data market provides a promising solution to this conflict, while determining the price of privacy is a tough issue. In this paper, we study the pricing problem in a setting where a data collector sequentially buys data from multiple data owners whose valuations of privacy are randomly drawn from an unknown distribution. To maximize the total payoff, the collector needs to dynamically adjust the prices offered to owners. We model the sequential decision-making problem of the collector as a multi-armed bandit problem with each arm representing a candidate price. Specifically, the privacy protection technique adopted by the collector is taken into account. Protecting privacy generally causes a negative effect on the value of data, and this effect is embodied by the time-variant distributions of the rewards associated with arms. Based on the classic upper confidence bound policy, we propose two learning policies for the bandit problem. The first policy estimates the expected reward of a price by counting how many times the price has been accepted by data owners. The second policy treats the time-variant data value as a context and uses ridge regression to estimate the rewards in different contexts. Simulation results on real-world data demonstrate that by applying the proposed policies, the collector can get a payoff which is close to that he can get by setting a fixed price, which is the best in hindsight, for all data owners.

Failures are not uncommon in production data center networks (DCNs) nowadays. It takes long time for the DCN routing to recover from a failure and find new forwarding paths, significantly impacting realtime and interactive applications at the upper layer. In this paper, we present a fault-tolerant DCN solution, called F2Tree, which is readily deployed in existing DNCs. F2Tree can significantly improve the failure recovery time only through a small amount of link rewiring and switch configuration changes. Through testbed and emulation experiments, we show that F2Tree can greatly reduce the routing recovery time after failure (by 78%) and improve the performance of upper layer applications when routing failure happens (96% less deadline-missing requests).

This paper presents a new method for Computer Hardware Experiment teaching. Previous hardware experiments of computer is built on a computer connected to a hardware equipment locally, and then student operate the hardware devices on the switches, buttons, etc. to carry out experiments and verify the developed hardware code is correct. Such experimental methods have lots of significant limitations. To solve the problems, we have developed a cloud-based real hardware experiment system platform. Students can use the experiment board anywhere internet available at any time. They can land at any place where they can access the cloud platform server, that will help them to get a hardware device, and then they can operate, and get results the same as local. We set up a cloud, and many sub-clouds in different cities. Each sub-clouds have a certain number of the special hardware experiment devices. The hardware device is assigned automatically, especially it can solve the problem of insufficient experiment hardware within a university. The platform has been set up with an experimental database, to analysis what is the difficult points for students and make a big data to help for improving computer hardware experiment educational methods. It provided a good experimental support for the online hardware experiment teaching content of MOOC.

As a novel Internet architecture, Named Data Networking (NDN) shifts the communication model from address-centric to content-centric. An NDN router caches the data in its content store, greatly reducing network traffic. NDN adopts the hierarchical naming schema, which allows the name aggregation and enables high scalability. However, in a richly connected topology, the nearest data replica are often not on the path dictated by NDN's tree-like data fetching model. This might result in a lower data delivery efficiency compared with the flat self-certifying naming schema in other Information-Centric Networking (ICN) architectures. To address the low efficiency problem, we propose a CDN-like enhancement to the NDN design, called Fetching the Nearest Replica (FNR). In FNR, when a consumer sends an interest for a popular data, the data is fetched from the nearest replica in the network, regardless of whether it is on the best path from the producer to the consumer. We present the design details and theoretical overhead analysis for FNR. Our evaluation results using ndnSIM simulator show that on average FNR reduces the total (inter-domain, intra-domain) traffic by 25.6% (53.0%, 18.2%) on average, compared to the default NDN approach. In addition, the average latency is reduced by 37% and the average cost is reduced by 51.4%. To the best of our knowledge, this paper is the first NDN enhancement in the literature to support nearest replica fetching in NDN.

Response time plays a key role in Web services, as it significantly impacts user engagement, and consequently the Web providers' revenue. Using a large search engine as a case study, we propose a machine learning based analysis framework, called FOCUS, as the first step to automatically debug high search response time (HSRT) in search logs. The output of FOCUS offers a promising starting point for operators' further investigation. FOCUS has been deployed in one of the largest search engines for 2.5 months and analyzed about one billion search logs. Compared with a previous approach, FOCUS generates 90% less items for investigation and achieves both higher recall and higher precision. The results of FOCUS enable us to make several interesting observations. For example, we find that popular queries are more image-intensive (e.g., TV series and shopping), but they have relatively low SRT because they are cached well by servers. Additionally, as suggested by the first-month analysis results of FOCUS, we conduct an optimization on image transmission time. A one-month real-world deployment shows that we successfully reduce the 80th percentile of search response time by 253ms, and reduce the fraction of HSRT by one third.

As mobile Internet is now indispensable in our daily lives, WiFi's latency performance has become critical to mobile applications' quality of experience. Unfortunately, WiFi hop latency in the wild remains largely unknown. In this paper, we first propose an effective approach to break down the round trip network latency. Then we provide the first systematic study on WiFi hop latency in the wild based on the latency and WiFi factors collected from 47 APs on T university campus for two months. We observe that WiFi hop can be the weakest link in the round trip network latency: more than 50% (10%) of TCP packets suffer from WiFi hop latency larger than 20ms (100ms), and WiFi hop latency occupies more than 60% in more than half of the round trip network latency. To help understand, troubleshoot, and optimize WiFi hop latency for WiFi APs in general, we train a decision tree model. Based on the model's output, we are able to reduce the median latency by 80% from 50ms to 10ms in one real case, and reduce the maximum latency from 250ms to 50ms in another real case.

It is crucial for Internet company to provide highly reliable web-based services. The web-based services always have many components running in the large-scale infrastructure with complex interactions. As an indispensable part of high reliability, the diagnosis remains to be a thorny problem. With the growth of system scale and complexity, it becomes even more difficult. In this paper, we propose an automatic diagnosis system based on causality graph to help system operators find the root causes. The causality graph is mainly extracted from the historical data of the monitoring system, and the method consists of four steps. 1) It utilizes a data mining method to extract the initial causality graph. 2) Once a failure happens, it lists top-k suspects with a ranking algorithm based on the causality graph. 3) Then system operators check the suspects and label them either right or wrong. 4) A supervised learning algorithm takes the labels as the input to tune the causality graph, in order to improve the diagnosis accuracy on step 2 iteratively. This method requires neither knowledge about the design and implementation details of the web-based service, nor instrumenting the services' source code. Our controlled experiments show that the root causes can be ranked in top 3 with 100% accuracy after countable learning iterations.

The enterprise Wi-Fi networks enable the collection of large-scale users' mobility information at an indoor level. The collected trajectory data is very valuable for both research and commercial purposes, but the use of the trajectory data also raises serious privacy concerns. A large body of work tries to achieve k-anonymity (hiding each user in an anonymity set no smaller than k) as the first step to solve the privacy problem. Yet it has been qualitatively recognized that k-anonymity is still risky when the diversity of the sensitive information in the k-anonymity set is low. There, however, still lacks a study that provides a quantitative understanding of that risk in the trajectory dataset. In this work, we present a large-scale measurement based analysis of the low-diversity risk over four weeks of trajectory data collected from Tsinghua, a campus that covers an area of 4 km2, on which 2,670 access points are deployed in 111 buildings. Using this dataset, we highlight the high risk of the low diversity. For example, we find that even when 5-anonymity is satisfied, the sensitive attributes of 25% of individuals can be easily guessed. We also find that although a larger k increases the size of anonymity sets, the corresponding improvement on the diversity of anonymity sets is very limited (decayed exponentially). These results suggest that diversity-oriented solutions are necessary.

802.11 (WiFi) networks have become increasingly important for our daily lives. However, previous work has shown that enterprise WiFi performance is often unsatisfactory and that over-utilization and interference from rogue APs are the two primary reasons. To address the above problem, this paper proposes to improve the capacity of WiFi infrastructures by increasing the enterprise AP deployment density, as well as disabling the wired Internet access in buildings to eliminate rogue APs and their interference. We deployed several WiFi networks with different AP density and vendors on Tsinghua campus. Based on the measurement results from our real-world deployments, we made three main observations: 1) in general, higher AP density improves WiFi performance; 2) over-dense deployment with unnecessarily high transmission power can worsen WiFi performance. 3) choice of AP vendors also has an impact on WiFi performance.

WiFi latency is a key factor impacting the user experience of modern mobile applications, but it has not been well studied at large scale. In this paper, we design and deploy WiFiSeer, a framework to measure and characterize WiFi latency at large scale. WiFiSeer comprises a systematic methodology for modeling the complex relationships between WiFi latency and a diverse set of WiFi performance metrics, device characteristics, and environmental factors. WiFiSeer was deployed on Tsinghua campus to conduct a WiFi latency measurement study of unprecedented scale with more than 47,000 unique user devices. We observe that WiFi latency follows a long tail distribution and the 90th (99th) percentile is around 20 ms (250 ms). Furthermore, our measurement results quantitatively confirm some anecdotal perceptions about impacting factors and disapprove others. We deploy three practical solutions for improving WiFi latency in Tsinghua, and the results show significantly improved WiFi latencies. In particular, over 1,000 devices use our AP selection service based on a predictive WiFi latency model for 2.5 months, and 72% of their latencies are reduced by over half after they re-associate to the suggested APs.

Failures are not uncommon in production data center networks (DCNs) nowadays, and it takes long time for the network to recover from a failure and find new forwarding paths, significantly impacting real time and interactive applications at the upper layer. The slow failure recovery is due to two primary reasons. First, there lacks immediate backup paths for downward links in DCN with multi-rooted tree topology. Second, distributed routing protocols in DCN take time to converge after failures. In this paper, we present a fault-tolerant DCN solution, called F2Tree, that can significantly improve the failure recovery time in current DCNs, only through a small amount of link rewiring and switch configuration changes. Because F2Tree does not change any existing software or hardware, it is readily deployed in production DCNs, where other existing proposals fail to achieve. Through testbed and emulation experiments, we show that F2Tree can greatly reduce the time of failure recovery by 78%. Our experimental results also show that, for partition-aggregate applications (popular in DCN) under various failure conditions, F2Tree reduces the ratio of deadline-missing requests by more than 96% compared to current DCNs.

Closely monitoring service performance and detecting anomalies are critical for Internet-based services. However, even though dozens of anomaly detectors have been proposed over the years, deploying them to a given service remains a great challenge, requiring manually and iteratively tuning detector parameters and thresholds. This paper tackles this challenge through a novel approach based on supervised machine learning. With our proposed system, Opprentice (Operators' apprentice), operators' only manual work is to periodically label the anomalies in the performance data with a convenient tool. Multiple existing detectors are applied to the performance data in parallel to extract anomaly features. Then the features and the labels are used to train a random forest classifier to automatically select the appropriate detector-parameter combinations and the thresholds. For three different service KPIs in a top global search engine, Opprentice can automatically satisfy or approximate a reasonable accuracy preference (recall >= 0.66 and precision>= 0.66). More importantly, Opprentice allows operators to label data in only tens of minutes, while operators traditionally have to spend more than ten days selecting and tuning detectors, which may still turn out not to work in the end.

Enterprise 802.11 Network (EWLAN) is an important infrastructure to the Mobile Internet, but its performance is being significantly impacted by the ever-increasing Rogue access points (RAPs). For example, in the university EWLAN we studied, the number of RAPs is more than seven times that of the enterprise APs. In this paper, we propose a generic methodology to measure RAP's carrier sense interference and hidden terminal interference, and it only uses readily available SNMP metrics, without any additional measurement hardware. Our results show that, on average, the carrier sense interference due to RAPs causes only 5% access delay increase at the MAC layer, because of careful engineering and software optimization. However, hidden terminal interference due to RAPs causes (a much more severe) up to 30% MAC layer loss rate increase on average, because no existing approach has explicitly dealt with the hidden terminal impact from rogue APs. Overall, the RAP interference would increase the IP layer delay at the WiFi hop by up to 50%.

When using search engines, users often care about search response time (SRT) in addition to result accuracy. It is thus the operators' responsibility to closely monitor and improving SRT. The first critical step of improving SRT is to pinpoint the root causes of slow SRT. However, this task is very challenging because SRT can be impacted by many factors, e.g., networks, data centers, browsers, and the page content. In this paper, we propose FOCUS, a systematic framework to narrow down the debugging space of slow SRT by identifying the bottleneck of slow SRT regarding various factors. The bottleneck provides operators more specific direction for further investigation. We deployed FOCUS in a global top search engine. Based on the output of FOCUS, operators successfully identified four potential causes which would not have been easy to find without FOCUS. Our what-if simulation analysis shows that, the proposed solutions, focusing on these bottlenecks, can improve SRT significantly, and they are more effective than some ad hoc solutions.

Page Views (PVs) are very crucial for search engines due to their close relationship to the revenue. When PVs change significantly, operators must be informed so that they can diagnose and fix the problem quickly, and prevent further loss. In reality, PVs can be counted in many ways (e.g., PVs originated from different ISPs), and different PVs are of different interest to operators (e.g., the PVs of a larger ISP is more important). As a result, different PVs often require different detection standards, or thresholds. However, attempts to tune a number of thresholds have been hampered by the cost of the manual effort involved. To address the above problem, we propose a practical framework, called PTL (practical threshold learning). Operators only need to provide a few simple labels about the detection results, then PTL will automatically tune the thresholds for different PVs. Using 4-month PVs from a global top search engine, our evaluation demonstrates that PTL can improve the accuracy of detection dramatically. More importantly, it introduces very little labeling overhead for operators. For example, when detecting the PVs of 103 ISPs, PTL can reduce the overall false negative rate from 96% to 9% using only 29 labels per week on average.

Modern data centers need to satisfy stringent low-latency for real-time interactive applications (e.g. search, web retail). However, short delay-sensitive flows often have to wait a long time for memory and link resource occupied by a few of long bandwidth-greedy flows because they share the same switch output queue (OQ). To address the above flow interference problem, this paper advocates more fine-grained flow separation in the switches than traditional OQ. We propose CQRD, a simple and cost-effective queue management scheme for data center switches, through only minor changes to the buffering and scheduling scheme. No change to the transport layer or coordination among switches is required. Simulation results show that CQRD can reduce the FCT of short flows by 20–44% in a single switch and 8–30% in a multi-stage data center switch network, only at the cost of a minor goodput decrease of large of flows.

We formalize the construction of fault blocks by a state transition model based on finite state automata. Based on the model, a boundary diffusion method is presented for the rectilinear-monotone orthogonal convex fault model such as the rectangular fault model and minimal-connected-component (MCC) faulty model, whereby an adaptive fault-tolerant routing algorithm, called X-Y boundary routing algorithm (X-YBRA), is presented for deadlock-free fault-tolerant adaptive routing outside the fault blocks. To improve the network resources utilization, we put forward a routing diffusion method in the fault block, which completely solves the routing problem in the fault block. The experiment result shows that the diffusion overhead of our method is far lower than that of the traditional routing algorithms such as distance vector and link state routing algorithms with the light loss in convergence time. For the occurrence and recovery of random faults, the expansion and shrinkage of the fault block are also discussed. Accordingly, the dynamic boundary and routing updating methods are put forward to respond to these cases. Based on these methods, we develop low earth orbit satellite networks into an adaptive fault-tolerant system in routing. Our works can be also applied to other 2D mesh networks such as the interconnect multiprocessor computer systems.

Traditional DFA based DPI (Deep Packet Inspection) string matching architectures either suffer from throughput bottleneck or unfeasible memory requirement, or both. Bloom Filter based schemes, on the other hand, only provide indefinite and unprecise match results. In this paper, we propose a novel string matching data structure called Overlapped Substring Classifier(OSC), which tries to compromise between these two ends. Instead of using incoming byte flow directly, we use OSC to extract the characteristic digest of the incoming string, which we demonstrate would be sufficient for locating a very small set of possible match, using DFA techniques. This type of match ambiguity and false-positive inaccuracy can be tuned to be negligible. The scheme is perfectly suitable for efficient and parallel hardware implementation, which makes ultra high performance and low memory usage simultaneously possible. A hardware architecture is also designed supporting singlethreaded scanning rate of 10Gbp, with only moderate memory requirement and clock rate assumption.

As scalable routers being a promising way to scale to higher capacity, scalable switch fabric as its key component has received a great deal of attention. However, the reliability calculation method for general switch fabric does not suit scalable switch fabrics with special features. In this paper, we study the features of scalable switch fabrics, and propose a novel reliability measure called Failure Influence which focuses on the robustness of scalable switch fabric. Using our improved heuristic algorithms, we analyze the Failure Influence properties of some classical topologies and compare them with Plus 2^i (P2i) of our prior work. Experimental results show that P2i has better Failure Influence properties. Since the Peak Point Problem (PPP) of P2i, Add Edge P2i (AE-P2i) and Failure Influence Optimal P2i (FO-P2i) are also proposed to solve this problem.

A packet scheduler is a primary component of the improved quality of service (QoS) model for today's Internet. Although many fair packet schedulers have been proposed through theoretical consideration, practical high-speed packet schedulers remain elementary. The disparity arises because existent schedulers either lack of necessary QoS guarantee or have an unacceptable cost of computation and storage. In this paper, we propose a simple and efficient packet scheduler called hardware optimized bit reversal permutation (HOBRP) based scheduler. Besides some common merits including low time- and space-complexity, bounded end-to-end delay guarantee and constant fairness index that many well-known schedulers have already owned, our HOBRP still possesses two additional features: One is that the end-to-end delay bound of HOBRP is tunable, which makes itself flexible enough to provide different levels of delay bounds for diverse types of application flows. The other is that all the operations and structures used by HOBRP are very simple and easy to be pipelined and paralleled, which benefits an intuitive high-speed hardware design scheme.

Load-balanced architectures appear to be a promising way to scale Internet to extra high capacity. However, architectures based on mesh topology have a node-degree of N, which prevents these architectures from large node numbers. This consideration motivates us to study the properties of node degree and its impact on the corresponding load-balanced architectures. In this paper we first show the asymptotically minimal node degree for any topology to achieve a constant ideal throughput under uniform traffic pattern when the channel bandwidth is fixed. We further introduce a unidirectional direct interconnection topology, named Plus 2^i (P2i), with this minimal node degree and prove that it has an ideal throughput of no less than twice the channel bandwidth under uniform traffic pattern. Based on the property, we provide the P2i load-balanced (PLB) architecture. Using this architecture, we show that scalability, 100% throughput and packet ordering can be all achieved and the scheduling algorithm is easy to implement. To the best of our knowledge, this is the first load-balanced architecture constructed on multi-hop direct interconnection topologies without packet reordering problem.

A rate-based switch fabric called smoothed buffer crossbar has been proposed in our recent work. It can provide 100% rate-guaranteed service with only a two-cell buffer at each crosspoint, and can also support best-effort service with an additional bandwidth regulator. However, the widely used max- flow model for bandwidth regulation is neither scalable nor of 100% throughput. In order to evaluate the performance of bandwidth regulator, we first introduce the 100% ideal throughput measurement in this paper. Then we prove that the total arrival average (TAA) algorithm has got this property, which does not happen in the max-flow allocation. Then an O(N) algorithm called proportion scaling allocation (PSA) is presented. Simulations reveal that PSA can deliver nearly 100% throughput even in a frequently changing traffic pattern. As a comparison, both theoretical and experimental evidences are given to show the inefficiency of the max-flow model.

Input queued switching architectures have become predominant in high speed switches and routers. In this paper, we change the point of view from weight-based matching to weight-based service, and propose a distributed load adaptive scheduling (DLAS) algorithm. In DLAS, the round robin arbiters are used to find a matching between the input ports and output ports. Once the matching between an input-output pair is established, the scheduler will keep it for a certain period, which is a function of the number of cells queued in the corresponding VOQ. Simulation results show that our scheme achieves high throughput and low delay under admissible traffic. For uniform Bernoulli i.i.d. traffic, it achieves 100% throughput, and for nonuniform traffic, its throughput is almost 100%.

QoS (quality-of-service) control is one of the most important mechanisms in the next-generation Internet, where QoS routing (QoSR) is a promising solution. We propose a multi-constrained intradomain QoS routing protocol SQOSPF. The advantages of this protocol include easy implementation, multi-constrained QoS support, high-speed convergence and multiple QoSR algorithms support. Stochastic Petri net is employed to model SQOSPF and analyze impacts of update threshold and routing holding time upon the load of networks and routers. Extensive simulations show that choosing appropriate update threshold and routing holding time can excessively reduce the extra load and keep routing performance at the same time.

Multi-constrained quality-of-service routing (QoSR) is to find a feasible path that satisfies multiple constraints simultaneously, as an NPC problem, which is also a big challenge for the upcoming next-generation networks. In this paper, we propose SA/spl I.bar/MCP, a novel heuristic algorithm, by applying simulated annealing to Dijkstra's algorithm. This algorithm first uses a nonlinear energy function to translate multiple QoS weights into a single metric and then seeks to find a feasible path by simulated annealing. The paper outlines simulated annealing algorithm and analyzes the problems met when we apply it to QoSR. Extensive simulations demonstrate that SA/spl I.bar/MCP has good scalability regarding both network size and the number of QoS constraints with high performance. Furthermore, when most QoS requests are feasible, the running time of SA/spl I.bar/MCP is about O(k(m+nlogn)), which is only k times that of the traditional Dijkstra's algorithm, where k is the number of QoS constraints.