Given the high cost associated with directly measuring the Traffic Matrix (TM), researchers have devoted efforts to devising methods for estimating the complete TM from low-cost link loads by solving a set of heavily ill-posed linear equations. Today’s increasingly intricate networks present an even greater challenge: as adaptive and dynamically changing routing strategies are gradually replacing traditional fixed routing schemes, the routing matrix within these equations can no longer be deemed reliable. In our previous work, we pioneered a flow-based generative model, FlowTM, which estimated the TM by establishing an invertible correlation between the TM and link loads without relying on the routing matrix. We demonstrated that the missing information in the ill-posed equations can be decoupled from the TM and learned jointly with the invertible mapping. Considering that acquiring a complete training set for FlowTM is often impractical in many real-world networks, we further propose an enhanced model, FlowTM+, in this extended work. It incorporates anInspectormodule to mine deeper latent structures from the partially observed TM data and link load measurements. This new technique effectively compensates for unobservable information in the training data. Extensive experiments demonstrate that FlowTM improves the performance of the best baseline by 38%–58% when the actual routing matrix is absent. Remarkably, with only 2% of the training data, FlowTM+ achieves an estimation accuracy comparable to that of state-of-the-art baselines trained with full routing knowledge and complete training data.

Real-time safety assessment (RTSA) of dynamic systems holds substantial implications across diverse fields, including industrial and electronic applications. However, the complexity and rapid flow nature of data streams, coupled with the expensive label cost and pose significant challenges. To address these issues, a novel confusion-based learning framework, termed confusion-and-detection method plus (CADM+), is proposed in this article. When drift occurs, the model is updated with uncertain samples, which may cause confusion between existing and new concepts, resulting in performance differences. The cosine similarity is used to measure the degree of such conceptual confusion in the model. Furthermore, the change of standard deviation within a fixed-size cosine similarity window is introduced as an indicator for drift detection. Theoretical demonstrations show the asymptotic increase of cosine similarity. In addition, the approximate independence of the change in standard deviation with the number of trained samples is indicated. Finally, the extreme value theory (EVT) is applied to determine the threshold of judging drifts. Several experiments are conducted to verify its effectiveness. Experimental results prove that the proposed framework is more suitable for RTSA tasks compared with state-of-the-art algorithms. The source code is available at https://github.com/THUFDD/CADM-plus.

Due to network operation and maintenance relying heavily on network traffic monitoring, traffic matrix analysis has been one of the most crucial issues for network management related tasks. However, it is challenging to reliably obtain the precise measurement in computer networks because of the high measurement cost, and the unavoidable transmission loss. Although some methods proposed in recent years allowed estimating network traffic from partial flow-level or link-level measurements, they often perform poorly for traffic matrix estimation nowadays. Despite strong assumptions like low-rank structure and the prior distribution, existing techniques are usually task-specific and tend to be significantly worse as modern network communication is extremely complicated and dynamic. To address the dilemma, this paper proposed a diffusion-based traffic matrix analysis framework named Diffusion-TM, which leverages problem-agnostic diffusion to notably elevate the estimation performance in both traffic distribution and accuracy. The novel framework not only takes advantage of the powerful generative ability of diffusion models to produce realistic network traffic, but also leverages the denoising process to unbiasedly estimate all end-to-end traffic in a plug-and-play manner under theoretical guarantee. Moreover, taking into account that compiling an intact traffic dataset is usually infeasible, we also propose a two-stage training scheme to make our framework be insensitive to missing values in the dataset. With extensive experiments with real-world datasets, we illustrate the effectiveness of Diffusion-TM on several tasks. Moreover, the results also demonstrate that our method can obtain promising results even with 5% known values left in the datasets.

The open environment presents a challenging issue for the online safety assessment of dynamic systems, which means that unknown scenarios may arise unexpectedly. These unknown scenarios can be task-related and result in the within-class distribution mismatch. Addressing this challenge in the semi-supervised safety assessment task, where unlabeled training data contain task-related unknown scenarios, has not been explored. This article investigates this new semi-supervised safety assessment problem. A novel active-labeling-aided semi-supervised learning scheme is proposed to tackle the within-class distribution mismatch between labeled and unlabeled training data. The proposed scheme begins by detecting out-of-distribution unlabeled data through the construction of a deep support vector data description network for each class. Subsequently, an active labeling approach along with its kernel extension is introduced, taking into account both distribution mismatch degree and sample representativeness. The proposed active labeling approach can be seamlessly integrated with any semi-supervised learning algorithm to enhance its performance in handling task-related unknown scenarios. The effectiveness and applicability of the proposed method are demonstrated through case studies based on a bearing dataset and operation data from an actual deep-sea manned submersible.

A pitch enhancement filter is designed with the objective to approach the optimal rate-distortion trade-off. The filter shows significant perceptual benefits, restating that information-theoretical and perceptual criteria are usually consistent. The filter is easy to implement and can be used as a complement to existing audio codecs. Our experiments show that it can improve the reconstruction quality of the AMR-WB standard.

The coding of audio-visual signals is generally based on different paradigms for high and low rates. At high rates the signal is approximated directly and at low rates only signal features are transmitted. The recently introduced distribution preserving quantization (DPQ) paradigm provides a seamless transition between these two regimes. In this paper we present a simplified scheme that preserves the power spectral density (PSD) rather than the probability distribution. In a practical system the PSD must be estimated. We show that both forward adaptive and backward adaptive PSD estimation are possible. Our experimental results confirm that preservation of PSD at finite precision leads to a unified coding paradigm that provides effective coding at both high and low rates. An audio coding application shows the perceptual benefits of PSD preserving quantization.

Modern networks are highly variable and, as a result, source coders are commonly used under conditions that they were not designed for. We address this problem with a source-coding philosophy that aims at the instantaneous re-optimization of a source coder to match a wide range of constraints on rate or quality and a wide range of packet-loss rates. We present a number of technologies that can be reconfigured by solving analytic relations that use the current conditions and a statistical description of the source as input. The technologies include distribution-preserving quantizers, flexible multiple-description quantizers, and a rate distribution scheme. Based on the generic technologies, we created a complete audio coder. Formal listening tests show that the resulting audio coding scheme with full flexibility provides a quality that is on-par with the best standardized codecs for any particular rate.

A new quantization scheme that preserves the probability distribution of the source signal is presented. The distribution preserving quantization (DPQ) achieves the optimal trade-off between mean square error and bit rate asymptotically. It provides a continuum ranging from rate-distortion optimal signal quantization to parametric coding. The method can be used as a core component for scalable coding. Its efficacy is illustrated by applying the scheme to audio coding.