Multi-Hop Question Answering (MHQA) requires step-by-step reasoning across multiple pieces of information to answer complex questions. The cache-aided Retrieval-Augmented Generation (RAG) can accelerate the process of external knowledge retrieval at each reasoning step for MHQA. However, existing methods focus on the internal structure and ignore the misalignment between the queries’ arrival order and cache hit order. To tackle this, we propose Mnemosyne, a cache hit order fitting method designed to accelerate the RAG progress for MHQA. Specifically, our cache-aware order fitting strategy adjusts the order of queries arrival via graph reordering to better align with the cache hit order, thereby reducing the likelihood of failed or unproductive retrieval attempts. The multi-granularity caching storage mechanism is designed to loosen the strict hit condition to multiple similar semantic matching modes, facilitating that relevant documents can still be retrieved. Experiments conducted on four multi-hop QA datasets demonstrate that Mnemosyne effectively reduces retrieval latency while enhancing task answer F1 score, achieving a superior trade-off between efficiency and effectiveness.

Federated Graph Learning (FGL) has emerged as an efficient paradigm to address the pronounced data heterogeneity common in real-world decentralized graph datasets, attracting significant interest from both academia and industry. Existing FGL methods often degrade the global model's performance due to a fundamental mismatch between the model's fixed-geometry embedding space and the diverse geometric structures of client data. To address this issue, we propose a novel framework for Personalized Federated Riemannian Graph Learning, namely FedRGL. FedRGL introduces a personalized mixed-curvature product space for each heterogeneous client, mapping local graph data into a tailored geometric space composed of Euclidean, hyperbolic, and spherical manifolds. Furthermore, it leverages a Ricci-Gated Graph Convolutional Network to dynamically adapt its message-passing mechanism to the local topology of each graph. Extensive experiments across diverse geometric heterogeneity settings demonstrate that FedRGL significantly outperforms state-of-the-art FGL methods in terms of model accuracy and generalization.

The goal of Personalized Federated Learning (PFL) achieves model personalization on statistical heterogeneous scenarios. Existing PFL approaches roughly treat the entire model parameters as a whole unit for generating personalized weights, and neglect the latent correlations among parameters of different client models, which is crucial for model personalization by the same learning tasks on statistical heterogeneity. To solve this issue, we propose a graph-guided aggregation framework for layer-wised PFL, namely FedHigh. The framework leverages layered knowledge sharing of client models to generate latent correlation graphs in the hyperbolic space. The latent correlation graphs can efficiently guide the personalized aggregation. Experimental results demonstrate that FedHigh achieves significant accuracy improvements up to 42.6%, 33.1%, and 7.5% on graph, computer vision (CV), and natural language processing (NLP) datasets, respectively. Meanwhile, FedHigh accelerates convergence speed up to 52.6% compared to state-of-the-art baselines on CV datasets. Additionally, the scalability of FedHigh also attractively outperforms other baselines with different world sizes.

With the widespread application of large language models (LLMs), their advantages in context analysis and semantic inferencing are becoming increasingly prominent, and we are attempting to introduce them into the realm of DNS repair. As is well known, DNS, as the core of Internet infrastructure, has complex policies and a fragile system, where even a small misconfiguration can lead to catastrophic service failures. Especially in large-scale networks, analyzing detected errors and generating repair solutions often requires operators to invest a significant amount of time and effort. This paper proposes a scalable framework, DNSHolmes, designed to leverage LLMs for generating DNS configuration repair solutions. Specifically, this method first addresses the numerous potential root causes in large-scale errors by abstracting them into a small number of State Equivalence Classes (SECs). It then adopts a deterministic finite automaton (DFA) to compute a Critical Path Graph (CPG) for each class, precisely isolating the minimal set of records responsible for the failure. Crucially, the CPG serves as a focused, verifiable context for a Large Language Model (LLM), guiding it to generate accurate patches while overcoming the fundamental context-length limitations that help LLM with effective repair reasoning. Our evaluation on large-scale public datasets and a real-world campus network demonstrates that DNSHolmes can reduce operational effort by 78.4% and achieve efficient repair in large-scale DNS configuration.

Heterogeneous Federated Learning (HtFL) enables collaborative learning across clients with diverse model architectures and non-IID data distributions, which are prevalent in real-world edge computing applications. Existing HtFL approaches typically employ proxy datasets to facilitate knowledge sharing or implement coarse-grained model-level knowledge transfer. However, such approaches not only elevate risks of user privacy leakage but also lead to the loss of fine-grained model-specific knowledge, ultimately creating barriers to effective knowledge sharing. To address these challenges, we propose FedFree, a novel data-free and model-free HtFL framework featuring two key innovations. First, FedFree introduces a reverse layer-wise knowledge transfer mechanism that aggregates heterogeneous client models into a global model solely using Gaussian-based pseudo data, eliminating reliance on proxy datasets. Second, it leverages Knowledge Gain Entropy (KGE) to guide targeted layer-wise knowledge alignment, ensuring that each client receives the most relevant global updates tailored to its specific architecture. We provide rigorous theoretical convergence guarantees for FedFree and conduct extensive experiments on CIFAR-10 and CIFAR-100. Results demonstrate that FedFree achieves substantial performance gains, with relative accuracy improving up to 46.3% over state-of-the-art baselines. The framework consistently excels under highly heterogeneous model/data distributions and in large scale settings.

There has been a growing interest in applying machine learning to real-world tasks. However, due to the black-box nature of machine learning models, it is crucial to 1) verify important properties of a model and 2) understand the reasons behind a model’s prediction before deploying them in a production environment. Existing approaches typically handle them as two separate and sometimes orthogonal topics. In this paper, we show that the verification and interpretability of machine learning models are tightly related and can be unified by satisfiability modulo theories (SMT). Our key insight is: not only a wide range of properties of machine learning models can be formulated as SMT problems and verified accordingly, but many commonly studied interpretability questions can also be answered by iteratively checking the satisfiability and related properties of multiple SMT problems. Leveraging this insight, we design UINT, a general verification and interpretability framework for learning-based networking systems. UINT 1) allows operators to specify verification and interpretability problems as SMT formulas, 2) encodes the target machine learning models into SMT constraints, and 3) automatically simplifies and solves the corresponding verification and interpretability problems using commodity SMT solvers. We implement a prototype of UINT and evaluate it on real-world learning-based networking systems. Results demonstrate the efficiency and efficacy of UINT in verifying and interpreting key questions for these systems.

DNS misconfiguration can result in severe social and financial consequences. Existing DNS configuration verification tools employ a centralized architecture, where all zone files are collected for verification. This architecture faces significant scalability issues (e.g., the verifier becoming the performance bottleneck and not supporting incremental verification). Inspired by the recent proposal of distributed data plane verification and the resemblance between the network data plane and DNS configuration, we propose to rearchitect DNS configuration verification with a distributed design. Our key insight is that by analyzing the query processing behavior of each DNS zone file in parallel and stitching the results in a symbolic way, we can substantially scale up the verification of DNS configuration. Evaluation shows that an up to 9.51× speed up on a dataset with over 410,000 resource records while having small overhead.

Network performance estimation is a key enabler in achieving efficient network operation. It can assist in important network tasks such as capacity planning, topology design, and parameter tuning. The traditional methods often rely on discrete event simulation and modeling by queuing theory. However, they consider some ideal scenarios with assumptions at a packet level and hardly generalize with ideal assumptions to the real world. In this paper, we propose FlowSeer, a novel framework for generalized network performance estimation at a flow level. FlowSeer employs a graph neural network to catch topology features and a pioneering transformer to learn and model flow features. Furthermore, the FlowSeer framework adopts a two-stage feature extraction mechanism, which can improve accuracy in large-scale networks. Compared with the state-of-the-art models, our extensive experiments demonstrate FlowSeer can improve the MAPE by up to 65.18% in diverse scenarios.

While Distributed Machine Learning (DML) has been widely used to achieve decent performance, it is still challenging to take full advantage of data and devices distributed at multiple vantage points to adapt and learn, especially it is non-trivial to address dynamic and divergence challenges based on the linear aggregation framework as follows: (1) heterogeneous learning data at different devices (i.e., non-IID data) resulting in model divergence and (2) in the case of time-varying communication links, the limited ability for devices to reconcile model divergence. In this paper, we contribute a non-linear class aggregation framework HyperPrism that leverages distributed mirror descent with averaging done in the mirror descent dual space and adapts the degree of Weighted Power Mean (WPM) used in each round. Moreover, HyperPrism could adaptively choose different mapping for different layers of the local model with a dedicated hypernetwork per device, achieving automatic optimization of DML in high divergence settings. We perform rigorous analysis and experimental evaluations to demonstrate the effectiveness of adaptive, mirror-mapping DML. In particular, we extend the generalizability of existing related works and position them as special cases within HyperPrism. Our experimental results show that HyperPrism can improve the convergence speed up to 98.63% and scale well to more devices compared with the state-of-the-art, all with little additional computation overhead compared to traditional linear aggregation.

Federated learning (FL) is a key solution for datadriven the Artificial Intelligence of Things (AIoT). Although much progress has been made, scalability remains a core challenge for real-world FL deployments. Existing solutions either suffer from accuracy loss or do not fully address the connectivity dynamicity of FL systems. In this article, we tackle the scalability issue with a novel, adaptive FL framework called FedSwarm, which improves system scalability for AIoT by deploying multiple collaborative edge servers. FedSwarm has two novel features: 1) adaptiveness on the number of local updates and 2) dynamicity of the synchronization between edge devices and edge servers. We formulate FedSwarm as a local update adaptation and perdevice dynamic server selection problem and prove FedSwarm‘s convergence bound. We further design a control mechanism consisting of a learning-based algorithm for collaboratively providing local update adaptation on the servers’ side and a bonus-based strategy for spurring dynamic per-device server selection on the devices’ side. Our extensive evaluation shows that FedSwarm significantly outperforms other studies with better scalability, lower energy consumption, and higher model accuracy.

All existing coflow scheduling algorithms compute dynamic-rate schedules that change the rates of flows during transmission. In this paper, we make a crucial finding: although dynamically adjusting the rates of flows could lead to a better coflow completion time (CCT) in theory, it would introduce additional pressures on the congestion control mechanism in the underlying network, which result in poor CCT in practice. This difference between theoretical CCT and practical CCT is further exacerbated in wide-area networks, where the topology does not provide any bisection guarantee as data center networks do. To this end, we designed a fixed-rate coflow scheduling policy called FSCO. Although in theory, the best fixed-rate schedule is usually suboptimal, it keeps the in-flight traffic relatively steady, reducing the risk of triggering congestion control. The core of FSCO is an efficient scheduling algorithm based on the classic network utilization maximization (NUM) framework. We implement a prototype of FSCO and evaluate its performance extensively using real-world topologies and coflow traces. Experimental results show that the total CCT reduces up to 30% compared to baselines while yielding up to 12× speedups compared to the solver.

There has been a growing interest in applying machine learning to real-world tasks. However, due to the blackbox nature of machine learning models, it is crucial to (1) verify important properties of a model and (2) understand the reasons behind a model's prediction before deploying them in a production environment. Existing approaches typically handle them as two separate and sometimes orthogonal topics. In this paper, we show that the verification and interpretability of machine learning models are tightly related and can be unified by satisfiability modulo theories (SMT). Our key insight is: not only a wide range of properties of machine learning models can be formulated as SMT problems and verified accordingly, but many commonly studied interpretability questions can also be answered by iteratively checking the satisfiability and related properties of multiple SMT problems. Leveraging this insight, we design UINT, a general verification and interpretability framework for learning-based networking systems. UINT (1) allows operators to specify verification and interpretability problems as SMT formulas, (2) encodes the target machine learning models into SMT constraints, and (3) automatically solves the corresponding verification and interpretability problems using commodity SMT solvers. We implement a prototype of UINT and evaluate it on real-world learning-based networking systems. Results demonstrate the efficiency and efficacy of UINT in verifying and interpreting key questions for these systems.

Aspect-based sentiment classification aims to automatically predict the sentiment polarity of the specific aspect in a text. However, it is challenging to confirm the mapping between the aspect and the core context since a number of existing methods concentrate on building the global relations of the full context rather than the partial connections based on the aspects. Motivated by the fundamental insights of reinforcement learning, we propose a novelHeterogeneousReinforcementLearningNetwork for aspect-based sentiment analysis (HRLN) to alleviate these issues, which contains two primary components, a heterogeneous network module, and a knowledge graph-based reinforcement learning module consistent with common-sense knowledge and emotional knowledge. To evaluate the effectiveness of HRLN, we conduct extensive experiments on five benchmark datasets, which indicate that HRLN achieves competitive performance and yields state-of-the-art results on all datasets. Additionally, we present an intuitive comprehension of why our HRLN model is more robust for aspect-based sentiment classification via case studies.

The synchronized Local-SGD(Stochastic gradient descent) strategy becomes a more popular in distributed deep learning (DML) since it can effectively reduce the frequency of model communication and ensure global model convergence. However, it works not well and leads to excessive training time in heterogeneous environments due to the difference in workers' performance. Especially, in some data unbalanced scenarios, these differences between workers may aggravate low utilization of resources and eventually lead to stragglers, which seriously hurt the whole training procedure. Existing solutions either suffer from a heterogeneity of computing resources or do not fully address the environment dynamics.

Story generation, which aims to generate a long and coherent story automatically based on the title or an input sentence, is an important research area in the field of natural language generation. There is relatively little work on story generation with appointed emotions. Most existing works focus on using only one specific emotion to control the generation of a whole story and ignore the emotional changes in the characters in the course of the story. In our work, we aim to design an emotional line for each character that considers multiple emotions common in psychological theories, with the goal of generating stories with richer emotional changes in the characters. To the best of our knowledge, this work is first to focuses on characters' emotional lines in story generation. We present a novel model-based attention mechanism that we call SoCP (Storytelling of multi-Character Psychology). We show that the proposed model can generate stories considering the changes in the psychological state of different characters. To take into account the particularity of the model, in addition to commonly used evaluation indicators(BLEU, ROUGE, etc.), we introduce the accuracy rate of psychological state control as a novel evaluation metric. The new indicator reflects the effect of the model on the psychological state control of story characters. Experiments show that with SoCP, the generated stories follow the psychological state for each character according to both automatic and human evaluations.

The majority of large-scale data intensive applications executed by data centers are based on Map-Reduce or its open-source implementation, Spark. However, the native Spark's data locality has significant side effects. Spark prioritizes high-priority tasks. But tasks at the next level of data locality are never started because Spark's delay scheduling mechanism. Additionally, the current scheduling approach on Spark only considers allocating reasonably computing resources for each task, but ignores the balance between computing resources and network resources. Network-intensive jobs will generate a large number of cross-rack tasks on heterogeneous clusters. More extensively, geographically distributed big data analytics clusters will produce greater network latency due to network transfer, which will decrease the performance of the whole cluster. Therefore, we propose OctopusKing a scheduling optimization approach based on the task completion time awareness. OctopusKing employs a deep learning algorithm to predict task completion time(TCT), which comprehensively considers four influence factors including the data size of each task, data nodes heterogeneity, the computation complexity of each task and network transfers. Based on the prediction of task completion time, our approach balances non-local task scheduling and network transfer time, which decreases the overall computation time of a job and improves the utilization of cluster resources effectively. Finally, we have implemented OctopusKing on Spark cluster with a 10 nodes. We show the convincing evidence that OctopusKing reduces average job completion time by 21% in comparison to Spark's default scheduler with little overhead.

This paper tackles a new problem in outlier detection: how to promptly detect the local outlier of a large-scale mixed attribute data in the big data era. This poses significant challenges due to a lack of access to the entire mixed attribute dataset at any individual compute machine. Proposed appr oaches firstly form a mechanism that deletes the massive clear non-noise and extracts cluster-based pre-noise set. Furthermore, we analyze pre-noise set using multi-step distributed LOF computing method on the Spark platform. Finally, the ordered LOF list is the output result. Comprehensive experiments are implemented by large-scale Benchmark datasets and the Spark platform. Extensive results show that the performance of our approaches are superior to the previous ones (4X faster than baseline LOF/2X faster than DLOF) when compared to state-of-the-art techniques, and therefore is believed to be able to give better guidance to local outlier detection of mixed attribute data.