Multivariate time series (MTS) analysis prevails in real-world applications such as finance, climate science and healthcare. The various self-attention mechanisms, the backbone of the state-of-the-art Transformer-based models, efficiently discover the temporal dependencies, yet cannot well capture the intricate cross-correlation between different features of MTS data, which inherently stems from complex dynamical systems in practice. To this end, we propose a novel correlated attention mechanism, which not only efficiently captures feature-wise dependencies, but can also be seamlessly integrated within the encoder blocks of existing well-known Transformers to gain efficiency improvement. In particular, correlated attention operates across feature channels to compute cross-covariance matrices between queries and keys with different lag values, and selectively aggregate representations at the sub-series level. This architecture automates representation learning of not only instantaneous but also lagged cross-correlations, while inherently capturing time series auto-correlation. When combined with prevalent Transformer baselines, correlated attention mechanism constitutes a better alternative for encoder-only architectures and achieves state-of-the-art results in imputation and classification.

Dynamic Time Warping (DTW) is a widely used metric for time series matching. However, when applied to few-shot action recognition (FSAR), DTW often encounters the "identical matching" issue, where multiple frames from one video are matched to a single frame from another. To address this, we introduce FTP-FSAR, a novel metric-based FSAR approach designed to mitigate this challenge. FTP-FSAR proposes an innovative alignment metric that incorporates temporal priors, guiding the matching process by encouraging the alignment of frames with similar temporal progression, thus improving the accuracy of frame matching. Additionally, FTP-FSAR integrates a dual framework, combining a foundation model with transductive learning to optimize feature extraction. Extensive experiments across multiple datasets demonstrate that FTP-FSAR outperforms existing methods, achieving the best results in 3 out of 4 benchmarks across 1-shot, 3-shot, and 5-shot settings, with performance improvements of up to 4.5%.

We propose a framework for resource provisioning with QoS guarantees in shared infrastructure networks. Our novel framework provides tunable probabilistic service guarantees for throughput and delay. Key to our approach is a Modified Dirft-plus-Penalty (MDP) policy that ensures long-term stability while capturing short-term probabilistic service guarantees using linearized upper-confidence bounds. We characterize the feasible region of service guarantees and show that our MDP procedure achieves mean rate stability and an optimality gap that vanishes with the frame size over which service guarantees are provided. Finally empirical simulations validate our theory and demonstrate the favorable performance of our algorithm in handling QoS in multi-infrastructure networks.

We propose Distributed Universal Max-Weight (DUMW) as a novel optimal control framework for distributed wireless SDN. DUMW is theoretically throughput-optimal and practically congruent with SDN system idiosyncrasies. Our algorithmic development non-trivially extends the throughput-optimal Universal Max-Weight (UMW) policy to permit distributed control and optimal inter-domain scheduling under the setting of heterogeneously delayed network state information. Furthermore, we design controller synchronization strategies that resolve the problem of multi-domain flow installation and are tailored to DUMW for maintaining throughput-optimality with negligible communication overhead. Extensive experiments validate our theoretical finding and demonstrate the favorable performance of DUMW. Under the setting of reliable links with wireless interference, DUMW achieves the same throughput as that of an optimal centralized controller and exhibits superiorscalability.

This paper studies the Partial Optimal Transport (POT) problem between two unbalanced measures with at most n supports and its applications in various AI tasks such as color transfer or domain adaptation. There is hence a need for fast approximations of POT with increasingly large problem sizes in arising applications. We first theoretically and experimentally investigate the infeasibility of the state-of-the-art Sinkhorn algorithm for POT, which consequently degrades its qualitative performance in real world applications like point-cloud registration. To this end, we propose a novel rounding algorithm for POT, and then provide a feasible Sinkhorn procedure with a revised computation complexity of O(n^2/epsilon^4). Our rounding algorithm also permits the development of two first-order methods to approximate the POT problem. The first algorithm, Adaptive Primal-Dual Accelerated Gradient Descent (APDAGD), finds an epsilon-approximate solution to the POT problem in O(n^2.5/epsilon). The second method, Dual Extrapolation, achieves the computation complexity of O(n^2/epsilon), thereby being the best in the literature. We further demonstrate the flexibility of POT compared to standard OT as well as the practicality of our algorithms on real applications where two marginal distributions are unbalanced.

Stochastic network models with all components being observable and controllable have been the focus of classic network optimization theory for decades. However, in modern network systems, it is common that the network controller can only observe and operate on some nodes (i.e., overlay nodes), and the other nodes (i.e., underlay nodes) are neither observable nor controllable. Moreover, the dynamics can be non-stochastic or even adversarial. In this paper, we focus on the network utility maximization (NUM) problem for networks with overlay-underlay structures. The network dynamics, such as packet admissions, external arrivals and control actions of underlay nodes, can be stochastic, non-stochastic or even adversarial. We propose the Tracking Drift-plus-Penalty (TDP*) algorithm that only operates on the overlay nodes and does not require direct observations of the underlay nodes, and analyze the tradeoffs between the average utility and queue backlog. We show that as long as the peak queue backlog of the network is sublinear in time horizon, TDP* can solve the NUM problem, i.e., reaching the maximum utility while preserving stability.

Federated learning (FL) is the distributed machine learning framework that enables collaborative training across multiple parties while ensuring data privacy. Practical adaptation of XGBoost, the state-of-the-art tree boosting framework, to FL remains nascent due to high cost incurred by conventional privacy-preserving methods. Such limitations can be attributed to the lack of formal analytical model to enable new privacy methods well customized to federated XGBoost. To this end, we propose a novel formulation, termed splitting matrix, in the context of federated XGBoost that mathematically characterizes the role of passive party (PP) having been neglected in the literature. This new formulation facilitates our novel adoption of secure matrix multiplication protocol into federated XGBoost to propose FedXGBoost as a framework for secure XGBoost in federated setting with lossless accuracy and negligible overhead. Extensive experiments on both synthetic and real datasets exhibit our algorithm’s empirical outperformance over known methods in the literature.

Inverse text normalization (ITN) is the task that transforms text in spoken-form into written-form. While automatic speech recognition (ASR) produces text in spoken-form, human and natural language understanding systems prefer to consume text in written-form. ITN generally deals with semiotic phrases (e.g., numbers, date, time). However, lack of studies to deal with phonetization phrases, which is ASR’s output when it handles unseen data (e.g., foreign-named entities, domain names), although these exist in the same form in the spoken-form text. The reason is that phonetization phrases are infinite patterns and language-dependent. In this study, we introduce a novel end2end model that can handle both semiotic phrases (SEP) and phonetization phrases (PHP), named AdapITN. We call it "Adap" because it allows for handling unseen PHP. The model performs only when necessary by providing a mechanism to narrow normalized regions and external query knowledge, reducing the runtime significantly.

The emergence of large-scale wireless networks with partially-observable and time-varying dynamics has imposed new challenges on the design of optimal control policies. This paper studies efficient scheduling algorithms for wireless networks subject to generalized interference constraint, where mean arrival and mean service rates are unknown and non-stationary. This model exemplifies realistic edge devices' characteristics of wireless communication in modern networks. We propose a novel algorithm termed MW-UCB for generalized wireless network scheduling, which is based on the Max-Weight policy and leverages the Sliding-Window Upper-Confidence Bound to learn the channels' statistics under non-stationarity. MW-UCB is provably throughput-optimal under mild assumptions on the variability of mean service rates. Specifically, as long as the total variation in mean service rates over any time period grows sub-linearly in time, we show that MW-UCB can achieve the stability region arbitrarily close to the stability region of the class of policies with full knowledge of the channel statistics. Extensive simulations validate our theoretical results and demonstrate the favorable performance of MW-UCB.

We study the Unbalanced Optimal Transport (UOT) between two measures of possibly different masses with at most $n$ components, where the marginal constraints of standard Optimal Transport (OT) are relaxed via Kullback-Leibler divergence with regularization factor $\tau$. Although only Sinkhorn-based UOT solvers have been analyzed in the literature with the iteration complexity of ${O}\big(\tfrac{\tau \log(n)}{\varepsilon} \log\big(\tfrac{\log(n)}{{\varepsilon}}\big)\big)$ and per-iteration cost of $O(n^2)$ for achieving the desired error $\varepsilon$, their positively dense output transportation plans strongly hinder the practicality. On the other hand, while being vastly used as heuristics for computing UOT in modern deep learning applications and having shown success in sparse OT problem, gradient methods applied to UOT have not been formally studied. In this paper, we propose a novel algorithm based on Gradient Extrapolation Method (GEM-UOT) to find an $\varepsilon$-approximate solution to the UOT problem in $O\big( \kappa \log\big(\frac{\tau n}{\varepsilon}\big) \big)$ iterations with $\widetilde{O}(n^2)$ per-iteration cost, where $\kappa$ is the condition number depending on only the two input measures. Our proof technique is based on a novel dual formulation of the squared $\ell_2$-norm UOT objective, which fills the lack of sparse UOT literature and also leads to a new characterization of approximation error between UOT and OT. To this end, we further present a novel approach of OT retrieval from UOT, which is based on GEM-UOT with fine tuned $\tau$ and a post-process projection step. Extensive experiments on synthetic and real datasets validate our theories and demonstrate the favorable performance of our methods in practice. We showcase GEM-UOT on the task of color transfer in terms of both the quality of the transfer image and the sparsity of the transportation plan.

We consider robust variants of the standard optimal transport, named robust optimal transport, where marginal constraints are relaxed via Kullback-Leibler divergence. We show that Sinkhorn-based algorithms can approximate the optimal cost of robust optimal transport in $\widetilde{\mathcal{O}}(\frac{n^2}{\varepsilon})$ time, in which $n$ is the number of supports of the probability distributions and $\varepsilon$ is the desired error. Furthermore, we investigate a fixed-support robust barycenter problem between $m$ discrete probability distributions with at most $n$ number of supports and develop an approximating algorithm based on iterative Bregman projections (IBP). For the specific case $m = 2$, we show that this algorithm can approximate the optimal barycenter value in $\widetilde{\mathcal{O}}(\frac{mn^2}{\varepsilon})$ time, thus being better than the previous complexity $\widetilde{\mathcal{O}}(\frac{mn^2}{\varepsilon^2})$ of the IBP algorithm for approximating the Wasserstein barycenter.

Studies on the Named Entity Recognition (NER) task have shown outstanding results that reach human parity on input texts with correct text formattings, such as with proper punctuation and capitalization. However, such conditions are not available in applications where the input is speech, because the text is generated from a speech recognition system (ASR), and that the system does not consider the text formatting. In this paper, we (1) presented the first Vietnamese speech dataset for NER task, and (2) the first pre-trained public large-scale monolingual language model for Vietnamese that achieved the new state-of-the-art for the Vietnamese NER task by 1.3% absolute F1 score comparing to the latest study. And finally, (3) we proposed a new pipeline for NER task from speech that overcomes the text formatting problem by introducing a text capitalization and punctuation recovery model (CaPu) into the pipeline. The model takes input text from an ASR system and performs two tasks at the same time, producing proper text formatting that helps to improve NER performance. Experimental results indicated that the CaPu model helps to improve by nearly 4% of F1-score.

QR decomposition of a matrix is one of the essential operations that is used for solving linear equations and finding least-squares solutions. We propose a coded computing strategy for parallel QR decomposition with applications to solving a full-rank square system of linear equations in a high-performance computing system. Our strategy is applied to the parallel Gram-Schmidt algorithm, which is one of the three commonly used algorithms for QR decomposition. Conventional coding strategies cannot preserve the orthogonality of Q. We prove a condition for a checksum-generator matrix to restore the degraded orthogonality of the decoded Q through low-cost post-processing, and construct a checksum-generator matrix for single-node failures. We obtain the minimal number of checksums required for singlenode failures under the "in-node checksum storage setting", where checksums are stored in original nodes, and further adapt the coded QR decomposition to this setting.

We introduce MobiTOP (mobile tagging of objects and people), a map-based application which allows users to contribute and share geo-referenced multimedia annotations via mobile devices. An important feature of MobiTOP is that annotations are hierarchical, allowing annotations to be annotated to an arbitrary depth. MobiTOPpsilas interface was designed using a participatory design methodology to ensure that the user interface meets the needs of potential users. In an evaluation, a group of student-teachers involved in a geographical field study were tasked to collaboratively identify rock formations using the MobiTOP system. The students who were in the field were guided by their lab counterparts on the tasks required to identify the rock formations. Results suggest the potential of the MobiTOP system for information sharing.