Online change detection (OCD) aims to rapidly identify change points in streaming data and is critical in applications such as power system monitoring, wireless network sensing, and financial anomaly detection. Existing OCD methods typically assume precise system knowledge, which is unrealistic due to estimation errors and environmental variations. Moreover, existing OCD methods often struggle with efficiency in large-scale systems. To overcome these challenges, we propose RoS-Guard, a robust and optimal OCD algorithm tailored for linear systems with uncertainty. Through a tight relaxation and reformulation of the OCD optimization problem, RoS-Guard employs neural unrolling to enable efficient parallel computation via GPU acceleration. The algorithm provides theoretical guarantees on performance, including expected false alarm rate and worst-case average detection delay. Extensive experiments validate the effectiveness of RoS-Guard and demonstrate significant computational speedup in large-scale system scenarios.

Distributionally Robust Optimization (DRO), which aims to find an optimal decision that minimizes the worst case cost over the ambiguity set of probability distribution, has been widely applied in diverse applications,e.g., network behavior analysis, risk management,etc.Nevertheless, prevailing DRO techniques encounter three primary challenges in distributed environments: 1) addressing asynchronous updating efficiently; 2) leveraging the prior distribution effectively; 3) appropriately adjusting the degree of robustness based on varying scenarios. To this end, we propose an asynchronous distributed algorithm, namedAsynchronousSingle-looPalternatIve gRadient projEction (ASPIRE) algorithm with the itErativeActiveSEt method (EASE) to tackle the federated distributionally robust optimization (FDRO) problem. In addition, a new uncertainty set,i.e., constrained$D$-norm uncertainty set, is developed to effectively leverage the prior distribution and flexibly control the degree of robustness. We further enhance the proposed framework by integrating various uncertainty sets and conducting a comprehensive theoretical analysis of the computational complexity associated with each uncertainty set. To expedite convergence speed, we also introduce ASPIRE-ADP, a method that can dynamically adjust the number of active workers. Finally, our theoretical analysis elucidates that the proposed algorithm is guaranteed to converge and the iteration complexity and communication complexity are also analyzed. Extensive empirical studies on real-world datasets validate that the proposed method excels not only in achieving fast convergence and robustness against data heterogeneity and malicious attacks but also in effectively managing the trade-off between robustness and performance.

Trilevel learning (TLL) with zeroth order constraints is a fundamental problem in machine learning, arising in scenarios where gradient information is inaccessible due to data privacy or model opacity, such as in federated learning, healthcare, and financial systems. These problems are notoriously difficult to solve due to their inherent complexity and the lack of first order information. Moreover, in many practical scenarios, data may be distributed across various nodes, necessitating strategies to address trilevel learning problems without centralizing data on servers to uphold data privacy. To this end, an effective distributed trilevel zeroth order learning framework DTZO is proposed in this work to address the trilevel learning problems with level-wise zeroth order constraints in a distributed manner. The proposed DTZO is versatile and can be adapted to a wide range of (grey-box) trilevel learning problems with partial zeroth order constraints. In DTZO, the cascaded polynomial approximation can be constructed without relying on gradients or sub-gradients, leveraging a novel cut, i.e., zeroth order cut. Furthermore, we theoretically carry out the non-asymptotic convergence rate analysis for the proposed DTZO in achieving the $\epsilon$-stationary point. Extensive experiments have been conducted to demonstrate and validate the superior performance of the proposed DTZO.

Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of applications, e.g., medical question-answering, mathematical sciences, and code generation. However, they also exhibit inherent limitations, such as outdated knowledge and susceptibility to hallucinations. Retrieval-Augmented Generation (RAG) has emerged as a promising paradigm to address these issues, but it also introduces new vulnerabilities. Recent efforts have focused on the security of RAG-based LLMs, yet existing attack methods face three critical challenges: (1) their effectiveness declines sharply when only a limited number of poisoned texts can be injected into the knowledge database (2) they lack sufficient stealth, as the attacks are often detectable by anomaly detection systems, which compromises their effectiveness, and (3) they rely on heuristic approaches to generate poisoned texts, lacking formal optimization frameworks and theoretic guarantees, which limits their effectiveness and applicability. To address these issues, we propose coordinated Prompt-RAG attack (PR-attack), a novel optimization-driven attack that introduces a small number of poisoned texts into the knowledge database while embedding a backdoor trigger within the prompt. When activated, the trigger causes the LLM to generate pre-designed responses to targeted queries, while maintaining normal behavior in other contexts. This ensures both high effectiveness and stealth. We formulate the attack generation process as a bilevel optimization problem leveraging a principled optimization framework to develop optimal poisoned texts and triggers. Extensive experiments across diverse LLMs and datasets demonstrate the effectiveness of PR-Attack, achieving a high attack success rate even with a limited number of poisoned texts and significantly improved stealth compared to existing methods. These results highlight the potential risks posed by PR-Attack and emphasize the importance of securing RAG-based LLMs against such threats.

Time series analysis has achieved great success in cyber security such as intrusion detection and device identification. Learning similarities among multiple time series is a crucial problem since it serves as the foundation for downstream analysis. Due to the complex temporal dynamics of the event-triggered time series, it often remains unclear which similarity metric is appropriate for security-related tasks, such as anomaly detection and clustering. The overarching goal of this paper is to develop an unsupervised learning framework that is capable of learning similarities among a set of event-triggered time series. From the machine learning vantage point, the proposed framework harnesses the power of both hierarchical multi-resolution sequential autoencoders and the Gaussian Mixture Model (GMM) to effectively learn the low-dimensional representations from the time series. Finally, the obtained similarity measure can be easily visualized for the explanation. The proposed framework aspires to offer a stepping stone that gives rise to a systematic approach to model and learn similarities among a multitude of event-triggered time series. Through extensive qualitative and quantitative experiments, it is revealed that the proposed method outperforms state-of-the-art methods considerably.

Machine learning (ML) model trading, known for its role in protecting data privacy, faces a major challenge: information asymmetry. This issue can lead to model deception, a problem that current literature has not fully solved, where the seller misrepresents model performance to earn more. We propose a game-theoretic approach, adding a verification step in the ML model market that lets buyers check model quality before buying. However, this method can be expensive and offers imperfect information, making it harder for buyers to decide. Our analysis reveals that a seller might probabilistically conduct model deception considering the chance of model verification. This deception probability decreases with the verification accuracy and increases with the verification cost. To maximize seller payoff, we further design optimal pricing schemes accounting for heterogeneous buyers’ strategic behaviors. Interestingly, we find that reducing information asymmetry benefits both the seller and buyer. Meanwhile, protecting buyer order information doesn’t improve the payoff for the buyer or the seller. These findings highlight the importance of reducing information asymmetry in ML model trading and open new directions for future research.

Trilevel learning, also called trilevel optimization (TLO), has been recognized as a powerful modelling tool for hierarchical decision process and widely applied in many machine learning applications, such as robust neural architecture search, hyperparameter optimization, and domain adaptation. Tackling TLO problems has presented a great challenge due to their nested decision-making structure. In addition, existing works on TLO face the following key challenges: 1) they all focus on the non-distributed setting, which may lead to privacy breach; 2) they do not offer any non-asymptotic convergence analysis which characterizes how fast an algorithm converges. To address the aforementioned challenges, this paper proposes an asynchronous federated trilevel optimization method to solve TLO problems. The proposed method utilizes u-cuts to construct a hyper-polyhedral approximation for the TLO problem and solve it in an asynchronous manner. We demonstrate that the proposed u-cuts are applicable to not only convex functions but also a wide range of non-convex functions that meet the u-weakly convex assumption. Furthermore, we theoretically analyze the non-asymptotic convergence rate for the proposed method by showing its iteration complexity to obtain ϵ-stationary point is upper bounded by O(1/ϵ²). Extensive experiments on real-world datasets have been conducted to elucidate the superiority of the proposed method, e.g., it has a faster convergence rate with a maximum acceleration of approximately 80%.

The primary goal of online change detection (OCD) is to promptly identify changes in the data stream. OCD problem find a wide variety of applications in diverse areas, e.g., security detection in smart grids and intrusion detection in communication networks. Prior research usually assumes precise knowledge of the system parameters. Nevertheless, this presumption often proves unattainable in practical scenarios due to factors such as estimation errors, system updates, etc. This paper aims to take the first attempt to develop a triadic-OCD framework with certifiable robustness, provable optimality, and guaranteed convergence. In addition, the proposed triadic-OCD algorithm can be realized in a fully asynchronous distributed manner, easing the necessity of transmitting the data to a single server. This asynchronous mechanism could also mitigate the straggler issue that faced by traditional synchronous algorithm. Moreover, the non-asymptotic convergence property of Triadic-OCD is theoretically analyzed, and its iteration complexity to achieve an $\epsilon$-optimal point is derived. Extensive experiments have been conducted to elucidate the effectiveness of the proposed method.

Out-of-Distribution (OOD) generalization in machine learning is a burgeoning area of study. Its primary goal is to enhance the adaptability and resilience of machine learning models when faced with new, unseen, and potentially adversarial data that significantly diverges from their original training datasets. In this paper, we investigate time series OOD generalization via pre-trained Large Language Models (LLMs). We first propose a novel \textbf{T}ri-level learning framework for \textbf{T}ime \textbf{S}eries \textbf{O}OD generalization, termed TTSO, which considers both sample-level and group-level uncertainties. This formula offers a fresh theoretic perspective for formulating and analyzing OOD generalization problem. In addition, we provide a theoretical analysis to justify this method is well motivated. We then develop a stratified localization algorithm tailored for this tri-level optimization problem, theoretically demonstrating the guaranteed convergence of the proposed algorithm. Our analysis also reveals that the iteration complexity to obtain an $\epsilon$-stationary point is bounded by O($\frac{1}{\epsilon^{2}}$). Extensive experiments on real-world datasets have been conducted to elucidate the effectiveness of the proposed method.

The exponential growth of scale and complexity in distributed systems necessitates significant maintenance efforts. Logs play an indispensable role in system operation and maintenance since they record crucial runtime information. However, recent studies on log anomaly detection have primarily focused on deep learning methods, which entail high computational complexity for learning temporal and semantic features from logs. Moreover, most deep learning-based approaches for log anomaly detection require supervised training, which is labor intensive. To address these challenges, this article proposes a framework calledGAE-Log.GAE-Logleverages event graphs and knowledge graphs to model logs comprehensively. By integrating temporal dynamics through event graphs and incorporating contextual information from knowledge graphs,GAE-Logenhances the understanding of the system's status. Moreover,GAE-Logemploys adversarial training of autoencoders for anomaly detection on logs. The effectiveness ofGAE-Logis evaluated through an ablation study and comprehensive comparisons using both public and synthetic log datasets. The results demonstrate thatGAE-Logoutperforms state-of-the-art methods in log anomaly detection, achieving significant performance improvements.

Machine learning (ML) model trading prevents data breaches in privacy-sensitive data-driven applications. Departing from commonly assumed complete information scenarios, we consider the more practical trading scenario where model deception may emerge under information asymmetry. More specifically, the model seller may provide false information on model quality to maximize her payoff. This paper takes the first step in tackling information asymmetry through the lens of model verification. We propose an ML model market that allows buyers to verify model quality before purchasing. Such verification can be costly and often imperfect, which makes the buyer's decision highly nontrivial. We first formulate the ML model trading process as a three-stage sequential game with imperfect information, where the seller determines the model delivery strategy after observing the buyer's order decision. Our analysis reveals that at the equilibrium, the seller will probabilistically conduct model deception, considering the possibility of model verification. The equilibrium deception probability increases with the buyer's verification cost and decreases with verification accuracy. Interestingly, we also show that reducing information asymmetry through verification benefits both the buyer and seller. We further consider a second market model with buyer order information protection, where the buyer's order information is unobservable before the seller makes the delivery strategy. Our analysis shows a surprising result under this market model: protecting buyer's order information will not increase the payoff of either the buyer or seller.

This paper studies the secrecy wireless information and power transfer problem in ultra-dense cloud radio access network (UD-CRAN) with wireless fronthaul, which is a promising framework for future Internet of Things (IoT). The transmission schemes of wireless fronthaul and access links are jointly designed, while addressing the characteristics of ultra-dense network such as base station diversity and high probability of line-of-sight transmission. Specifically, we employ the idea of block diagonalization to deal with the fronthaul interference, which support multi-stream fronthaul transmission for each remote radio head (RRH). We then jointly optimize the power allocation in the fronthaul and the resource allocation in the access link which includes beamforming for information and energy transmission, on/off of RRHs, and user-RRH association. In order to solve the formulated mixed integer non-convex optimization problem, we leverage the sparsity of beamforming vectors brought by the ultra-dense RRHs. We then solve the reformulated problem by employing the successive convex approximation approach. Finally, numerical results are presented to demonstrate the effectiveness of the proposed scheme.

Internet of Things (IoT) time-series analysis has found numerous applications in a wide variety of areas, ranging from health informatics to network security. Nevertheless, the complex spatial–temporal dynamics and high dimensionality of IoT time series make the analysis increasingly challenging. In recent years, the powerful feature extraction and representation learning capabilities of deep learning (DL) have provided an effective means for IoT time-series analysis. However, few existing surveys on time series have systematically discussed unsupervised DL-based methods. To fill this void, we investigate unsupervised DL for IoT time series, i.e., unsupervised anomaly detection and clustering, under a unified framework. We also discuss the application scenarios, public data sets, existing challenges, and future research directions in this area.

Distributionally Robust Optimization (DRO), which aims to find an optimal decision that minimizes the worst case cost over the ambiguity set of probability distribution, has been applied in diverse applications, e.g., network behavior analysis, risk management, etc. However, existing DRO techniques face three key challenges: 1) how to deal with the asynchronous updating in a distributed environment; 2) how to leverage the prior distribution effectively; 3) how to properly adjust the degree of robustness according to difference scenarios. To this end, we propose an asynchronous distributed algorithm, named Asynchronous Single-looP alternatIve gRadient projEction (ASPIRE) algorithm with the itErative Active SEt method (EASE) to tackle the distributed distributionally robust optimization (DDRO) problem. Furthermore, a new uncertainty set, i.e., constrained $D$-norm uncertainty set, is developed to effectively leverage the prior distribution and flexibly control the degree of robustness. Finally, our theoretical analysis elucidates that the proposed algorithm is guaranteed to converge and the iteration complexity is also analyzed. Extensive empirical studies on real-world datasets demonstrate that the proposed method can not only achieve fast convergence, remain robust against data heterogeneity and malicious attacks, but also tradeoff robustness with performance.

Hadoop provides a powerful platform that allows reliable, scalable, and distributed processing of massive data sets across a cluster of computers. Log data record events taken place in the Hadoop system that helps to understand system activities and diagnose problems. However, system upgrades and updates often change the syntax and patterns of logs, rendering the machine-learning models that were designed for the legacy system ineffective. Retraining the machine-learning models with new data sets from scratch might improve the accuracy of the machine-learning model. Nevertheless, annotating new data sets is often time consuming and labor intensive. In this article, we propose a domain adaptive log anomaly prediction framework called LogAT to effectively transfer learned knowledge from the existing labeled data set (source domain) to the new unlabeled data set (target domain) by adopting an unsupervised domain adaption method. Furthermore, a hierarchical anomaly knowledge graph has been constructed to represent the domain knowledge that facilitates the subsequent detection and diagnosis of system faults. Extensive experiments have been conducted on public and real-world data sets to validate the effectiveness of the proposed framework as well as each module. Our results show that LogAT achieves superior performance over the state-of-the-art methods for predicting log anomalies and acquiring considerable performance improvement in terms of AUC-ROC score on different Hadoop application data sets.

Interferences due to non-orthogonality of signals usually exist in wireless networks when the number of users is larger than the sequence length, such as non-orthogonality of the pilots in multi-cell systems and non-orthogonality of the signature sequences in overloaded code-division-multiple-access (CDMA) systems. We address this effect from the perspective of non-orthogonal sequence design in a two-cell multiple-antenna network. Specifically, we aim at designing pilot sequences to minimize the sum mean-squared-error (MSE) of channel estimation with a given sequence length$\tau $where$\tau \in [K,2K]$and$K$is the number of users per cell. Considering the strength disparity between channels originating from the home cell and the neighbor cell, this problem boils down to minimizing the sum of squares of weighted correlations among sequences, whose lower bound is obtained inclosed formand can be regarded as a generalization of the well-known Welch bound (Welch, 1974). We prove this extended Welch bound is achievable, and design an algorithm based on the Davies-Higham method to generate the interference-minimizing sequences. Three fundamental properties of the proposed sequences are presented. Finally, we derive closed-form expressions of the average signal-to-interference-plus-noise-ratio (SINR) and rate for data transmission, based on which the optimal training duration can be found.

In this article, a wideband Direction-of-Arrival (DOA) estimation method for underdetermined scenarios is proposed, which effectively solves the basis mismatch problem. Based on the fractional Fourier transform (FRFT), the wideband received signal model with a coprime array is first derived by exploiting the aggregation characteristic of wideband linear frequency modulated (LFM) signals in the fractional Fourier (FRF) domain. Then, in order to increase the degree of freedom, an extended uniform linear array is built, and the covariance matrix of the signal is reconstructed by employing the penalized atomic norm minimization with the consecutive spatial dictionary. Meanwhile, without the knowledge of the noise level, the noise variance is estimated from the noisy incomplete data, which is utilized to improve the covariance matrix reconstruction performance. Additionally, for the unconditional model, the Cramér-Rao bound for the wideband DOA estimation based on a coprime array is derived. Different from the existing methods, the proposed method not only can estimate more DOAs of wideband signals than the number of physical sensors in the presence of unknown amplitude distortion but also can obtain more accurate DOA estimation performance without basis mismatch. The effectiveness of the proposed method is verified by our numerical results.

Time synchronization is crucial for the safe and reliable operation of the fifth generation (5G) network, especially for applications requiring ultra-reliable low-latency data transmissions. The time synchronization problem, however, becomes increasingly challenging in high-mobility scenarios because the channel conditions, e.g., the multipath delay spread may vary rapidly. While there exist numerous works on the design of efficient channel decoding algorithms, decoding linear codes such as polar codes in the presence of synchronization errors is a less-explored topic. In this paper, we aim to fill this void and develop a systemic approach to decode general binary linear codes over binary symmetric channels with synchronization errors in which the lack of synchronization is modeled as the deletion channel model. The maximum likelihood (ML) decoding problem for binary linear codes over deletion channels is first formulated as a nonlinear optimization problem, in which a set of linear constraints are employed to characterize the input-output relationship of a deletion channel. It turns out that both the objective function and the constraints of this optimization problem are nonlinear, which poses significant challenges against the design of efficient decoding algorithms. As a remedy, we first replace the nonlinear objective function of this optimization problem via a lower bound. And we prove this lower bound is a linear function in the special case that the input is binary. We then apply the linear programming (LP) relaxation approach to obtain an approximate solution to the proposed nonlinear optimization problem. An adaptive branch-and-cut decoding algorithm has also been developed by making use of the ML-certificate property of the LP decoder for deletion channel. It is seen through simulation studies that the proposed decoding algorithm can achieve close-to-optimal bit error rate (BER) decoding performance at moderate computational complexity.

Broadband connectivity and mobile technology have been widely applied in the world. With these advanced technologies, the proliferation of smart devices and their applications by accessing mobile internet have come up with a giant leap forward, leading to the ever‐increasing scale and complexity of cellular networks. This presents imminent challenges to anomaly detection in cellular networks. In this study, the authors discuss challenges and current literature of anomaly detection for cellular networks to embrace the ‘big data’ era. First, they review the state‐of‐the‐art techniques in the area of anomaly detection in cellular networks. Then, the challenges are pinpointed for anomaly detection due to the cellular network big data. Finally, they introduce a big data analytic‐based anomaly detection method for cellular networks.

Proactive anomaly detection and diagnosis play an essential role in ensuring the security and stability of a large-scale information technology (IT) system with thousands or even millions of components that are interacting with each other. Collective contextual anomalies (CCAs) carry the characteristics of both collective and contextual anomalies. This type of anomalies is common in IT system monitoring, often manifested as security risks to be ameliorated, service outages to be eliminated, or stragglers to be mitigated. However, most existing studies emphasize primarily on the detection of point anomalies while the prediction or early detection of CCA has been an underexplored topic. In this paper, we propose a framework for discovering and studying CCAs in multiple time series based on a combination of semi-supervised deep learning, time series modeling, and graph analysis. A primary advantage of the proposed framework is that it can effectively predict CCAs with no human intervention. In addition, the performance of the proposed method can be further enhanced via learning from a small amount of anomalous training data, if it is available. Finally, the proposed framework is of low computational complexity and is thus particularly suitable for large-scale data streams. Simulation studies are carried out to reveal the superior performance of the proposed method and underscore the significant benefits of combining deep neural networks with time series analysis and graph models for the prediction and analysis of CCAs.

As we enter the Internet of Things (IoT) era in which the communication network is becoming increasingly dynamic, heterogeneous, and complex, it is desirable to have cognitive communication systems and networks that possess multiple interacting capabilities for situation assessment, resource management, online/distributed learning, big-data processing, and intelligent decision making. AI techniques, such as deep learning, probabilistic graph model, and reinforcement learning, aided with big data and IoT, provide a wide variety of tools and solutions to many new problems encountered in the design, operation, and optimization of cognitive communication systems and networking, including resource management, situation assessment, channel identification, anomaly detection, root cause analysis, and online/distributed learning.

This paper studies the multigroup multicast precoding problem in frame-based multigateway multibeam satellite communications with feeder link interference. We formulate a sum rate maximization problem that incorporates the minimum signal-to-interference-and-noise-ratio requirement for each user, the sum power constraint at each gateway, as well as the per feed power constraints at the satellite. Both transparent payload and payload with on-board processing are considered. In the former case, we propose a centralized algorithm by employing the successive convex approximation (SCA) approach, which iteratively approximates the original nonconvex problem to a second-order cone program. Moreover, in order, for each gateway, to compute its precoding vector locally with local channel state information, we devise a decentralized algorithm by incorporating consensus alternating direction method of multipliers (ADMM) into the SCA framework. For the latter case, we devise a two-stage precoding scheme where, in the first stage, a leakage-based minimum mean-square-error scheme is employed to control the feeder link interference efficiently. In the subsequent second stage, we use the SCA-ADMM approach to deal with the user link interference while maximizing the sum rate. Finally, numerical results are presented to demonstrate the performance of the proposed schemes.

A cognitive robot usually needs to perform multiple tasks in practice and needs to locate the desired area for each task. Since deep learning has achieved substantial progress in image recognition, to solve this area detection problem, it is straightforward to label a functional area (affordance) image dataset and apply a well-trained deep-model-based classifier on all the potential image regions. However, annotating the functional area is time consuming and the requirement of large amount of training data limits the application scope. We observe that the functional area are usually related to the surrounding object context. In this work, we propose to use the existing object detection dataset and employ the object context as effective prior to improve the performance without additional annotated data. In particular, we formulate a two-stream network that fuses the object-related and functionality-related feature for functional area detection. The whole system is formulated in an end-to-end manner and easy to implement with current object detection framework. Experiments demonstrate that the proposed network outperforms current method by almost 20% in terms of precision and recall.

This paper considers the design of network coding schemes for reliable wireless broadcast and multicast transmissions, in which the same packet is broadcast to a group of receivers. Network coding across multiple broadcasted packets is employed to generate redundant packets for the broadcast retransmissions so that the lost packets can be recovered. It is assumed that optimal decoders are employed at the receivers and the focus is on the design of short block codes with small numbers of redundant bits. To this end, use if first made of the residual graph representation to calculate the error probability of the optimal decoder. Then two code design schemes are proposed to minimize the error probability, including a low-complexity deterministic greedy code design algorithm as well as a stochastic code construction algorithm inspired by the simulated annealing technique. Extensive simulation studies have been carried out to assess the performance of the proposed schemes. It is seen that for a given number of retransmissions, the proposed network coding schemes can considerably increase the average number of recovered packages per user at the receivers and thereby improve the spectral efficiency over traditional coding methods.

This paper considers a joint transceiver design for the weighed sum energy efficiency maximization problem for downlink transmissions in a multi-cell multi-user multiple-input multiple-output (MIMO) system. To make the formulated problem be more valuable in practice, a more practical power consumption model is adopted in which part of the processing power is dependent on data rate. With per-user rate requirements and per-base station (BS) power constraints, the resulting optimization problem is non-convex. A centralized solution is firstly proposed, which alternatively performs transmit beamforming optimization and receive beamforming optimization with the help of successive convex approximation (SCA) based iteration. Based on the centralized design that requires global channel state information (CSI), a more meaningful decentralized algorithm is further designed, which solves the problem with the requirement of only exchanging quite a small amount of information among BSs. Specifically, the key methodology used in the decentralized algorithm is to reformulate transmit beamforming optimization as an equivalent global consensus problem when receive beamforming is fixed as minimum mean square error (MMSE) receiver, which is solved effectively by exploiting the alternative direction method of multipliers (ADMM) technique. Numerical results demonstrate the effectiveness and superiority of our proposed algorithms, and also illustrate the impact of the rate-dependent power consumption on the energy efficiency.

In this paper, we present deep network analyzer (DNA), a big data analytics platform for anomaly detection (AD) and root cause analysis (RCA) in mobile wireless networks. DNA is motivated by the growing scale and complexity of cellular networks along with the lack of advanced big data analytics tools for effective network management. It abstracts the RCA process into two modules, namely rule (fingerprint) learning and the module of AD and fingerprint matching. We first develop a rare association rule mining method to learn the symptoms of network anomalies and to build a fingerprint knowledge database from the historic data. Then a statistical machine learning approach is employed to identify the anomalies within the incoming dataset collected via various probes in the network and map the fingerprints of the detected anomalies to the rules in the knowledge database. The DNA platform has been tested using the real production data from the field and has been shown to be a highly effective platform for AD and RCA for large-scale cellular systems serving tens of millions of mobile users.

Millimeter wave (mmWave) communications have recently attracted large research interest, since the huge available bandwidth can potentially lead to the rates of multiple gigabit per second per user. Though mmWave can be readily used in stationary scenarios, such as indoor hotspots or backhaul, it is challenging to use mmWave in mobile networks, where the transmitting/receiving nodes may be moving, channels may have a complicated structure, and the coordination among multiple nodes is difficult. To fully exploit the high potential rates of mmWave in mobile networks, lots of technical problems must be addressed. This paper presents a comprehensive survey of mmWave communications for future mobile networks (5G and beyond). We first summarize the recent channel measurement campaigns and modeling results. Then, we discuss in detail recent progresses in multiple input multiple output transceiver design for mmWave communications. After that, we provide an overview of the solution for multiple access and backhauling, followed by the analysis of coverage and connectivity. Finally, the progresses in the standardization and deployment of mmWave for mobile networks are discussed.

For the potential of providing rates of multiple Giga-bps in a single channel, millimeter wave (mmWave) communications have recently attracted substantial research interest. While mmWave technology is already being used in stationary scenarios such as indoor hotspots or backhaul, it is challenging to use mmWave frequencies in mobile networks, where transmitting/receiving nodes may be moving, channels may have a complicated structure, and the coordination among multiple nodes is difficult. To fully exploit the high potential rates of mmWave in mobile networks, many significant technical challenges must be tackled. The main objective of this IEEE JSAC Special Issue on “Millimeter wave communications for future mobile networks” is to collect the most recent technical advances in mmWave for future mobile networks. The response from the community to the call has been overwhelming. We received 96 submissions with a call period short than 4 months. Many of the submissions are from the most well known research groups in the field. After a strict review process, we decided to accept 38 papers, which will be published in two issues. The papers were selected based on the technical relevance and merits. Unfortunately, due to space limitations, a number of interesting papers were not selected, despite the merits that they had. We sincerely hope those papers can find other publishing venues.

Nowadays, the quality of wireless network becomes critical for the network service providers (NSP). A poor performed network may lead to customer complaints, even loss of revenue from user churns. To ensure the quality of service, the key quality indicators (KQI) which reflect the quality of specific use cases have been collected alongside the network performance counters (NPC) for network performance analysis. To start, the NSP mines the network performance data to discover KQI anomalies which may cause poor user experience. If there is any KQI anomaly has been detected, the NSP investigates the associated NPCs to identify the possible root causes. Since the number of use cases increases dramatically and the volume of collected network performance data grows tremendously everyday, the wireless network anomaly root cause analysis becomes extremely challenging. How to efficiently discover the relationship between NPCs and KQI outliers becomes the key to identify the root causes of anomalies in the wireless network. To solve this problem, in this paper, we propose an efficient rare association rule mining algorithm called Spark-based Rare Association Rule Mining (SRAM) which leverages not only the efficiency of FP-growth algorithm but also the powerful big data processing mechanism of Spark platform. We have implemented our algorithm on the Spark platform and tested with various of data sets. The result shows our method can efficiently mine rare association rules from big volume of data.

Wi-Fi fingerprint-based localization has attracted significant research interest recently. Previous works in this area mainly focus on locating an individual user, whereas the additional assistance from peer-to-peer interactions has not been fully exploited. In this paper, we propose a cooperative localization method which not only utilizes the initial results by the fingerprint-based algorithm but also takes into account the physical constraint of pairwise distances to refine the localization estimates for multiple users simultaneously. The experimental results demonstrate that our algorithm is robust against the ranging error and the outdated fingerprint database. With the proposed peer selection scheme, it considerably improves localization accuracy. We further extend our framework to single-user motion tracking and localization based only on access-point-connectivity data.

This paper presents an application of the firefly algorithm (FA) to Gaussian process (GP)-based localization. Partial radio-frequency (RF) signature map is first collected and used to train the GP model. The hyperparameters of the GP prior model are searched by the FA. GP regression is then used to generate an estimation of the RF signature map for the entire area to be localized. This is in contrast to traditional fingerprinting-based localization where a database of RF signature has to be collected for the entire area of interest. Using the estimated signature map, the position of the device is estimated using a combined likelihood function from multiple access points (APs). The proposed scheme relies on only existing infrastructures and can be used both indoor and outdoor. Experiments using indoor WiFi APs show that median error is around 3 m.

We develop robust power control strategies for cognitive radios in the presence of sensing delay and model parameter uncertainty. We use a discrete-time Markov chain (DTMC) to characterize the primary users' (PU) dynamics as well as the fading channel. The power control problem is formulated as a Markov decision process (MDP) problem, which can be optimally solved by dynamic programming. However, due to the time-varying nature of the wireless channel and the spectrum sensing overhead, typically only the delayed sensing results are available at any time. The delay in spectrum sensing, if not properly accounted for, could significantly deteriorate the power control performance. Furthermore, the false sensing data and limited feedback cause noisy estimate of the transition probability matrix, leading to further performance degradation of the power control and channel outage. We first propose power control schemes based on a delayed MDP formulation, that account for all possible current channel state based on the delayed channel state. In addition, we propose an outage constraint to protect PU transmissions and properly manage channel outage. We then propose a robust power control framework that optimizes the worst-case system performance. Extensive simulation results are provided to demonstrate the effectiveness of the proposed power control algorithms.

Power control plays a key role in realizing reliable and spectrum-efficient communications in a cognitive radio network. In this paper we study secure power control schemes for cognitive radios via a robust Markov decision process (MDP) approach. We first use the discrete time Markov chain (DTMC) model to characterize the primary user's (PU) activities as well as the dynamics of the fading channel. The resulting power control problem can be optimally solved by a dynamic programming (DP) approach. The presence of malicious users, however, necessitates a cooperative spectrum sensing approach that requires extra signal processing and information exchange efforts to identify false spectrum sensing reports. Such a cooperative approach incurs considerable delay in spectrum sensing that may significantly deteriorate the performance of the DP strategy. Furthermore, the false sensing data generated by malicious users may give rise to erroneous estimation of the transition probabilities. Consequently, the solution obtained based on the estimated transition matrices may exhibit poor performance. To cope with these challenges, we propose a framework of power control schemes based on the robust MDP approach that is capable of achieving reasonably good performance when both spectrum sensing delay and estimation errors are present. The tradeoffs between the robustness, the achievable throughput, and the sensing delay are also discussed. Extensive simulation results are presented to demonstrate the performance of the proposed power control strategies.

The revolutionary success of commercial broadband wireless network has spurred significant interest in employing related technologies such as 3GPP long term evolution (LTE) technologies to build a nationwide mobile broadband network for public safety entities. The successful deployment and operation of a public safety network, however, are more challenging than the traditional commercial wireless networks due to the high reliability and security requirements, unpredictable traffic patterns, and fast time-varying user population density. In this paper, we propose a framework of coordinated dual-layer beamforming schemes to address these challenges. The distinct features of the proposed systems are detailed and the architecture and realization are discussed. While a smart antenna is capable of forming multiple beams, it is unlikely to activate all of them simultaneously. We thus develop efficient beam scheduling algorithms that could adapt to the time-varying channel conditions and traffic loads by adaptively adjusting the beam switching sequence. We also reveal that the coordinated dual-layer beamforming technologie not only has the potential of significantly enhancing the system capacity, but also can turn a NP-complete beam scheduling problem into a problem that can be easily solved if network MIMO technologies are employed. It is seen through simulation studies that the proposed schemes could offer multi-fold capacity improvements over more traditional systems equipped with broader antenna patterns (e.g., omnidirectional). Also, while we focus on public safety network in this paper, the proposed coordinated dual-layer beamforming framework can be equally applied to provide mobile access to any hotspot area with dense mobile users, e.g., a conference room or a stadium.

Distributed space-time block coding with orthogonal frequency division multiplexing (DSTBC-OFDM) and distributed space-frequency block coding with orthogonal frequency division multiplexing (DSFBC-OFDM) are introduced in this paper. Unlike the original DSTBC scheme which was designed for networks with a fixed number of relays operating in frequency-nonselective channels with insignificant Doppler spread, the proposed schemes are designed specifically for networks with an arbitrary number of relays and wireless channels with significant delay spread and Doppler spread. It is shown that geographically distributed antennas employing the new transmission schemes in a frequency-selective and severe Doppler spread environment can achieve the same performance (in terms of diversity gain and coding gain) as the previously proposed scheme.

In this paper, we propose jointly-optimized beamforming algorithms for cognitive networks to maximize the achievable rates, where primary and cognitive users share the same spectrum and are equipped with multiple antennas. We consider the transmission of a single information stream in both primary and secondary links. No coordination is required between the primary and cognitive users and the interference cancellation is done at the cognitive user. Specifically, the beamforming vectors of the cognitive link are designed to maximize the achievable rate under the condition that the interference both at the primary and cognitive receivers is completely nullified. Furthermore, it is proved that the achievable rate of a general N_t^{C}x 2 (N_t^{C} transmit antennas at the cognitive transmitter and 2 receive antennas at the cognitive receiver) cognitive multiple-input multiple-output (MIMO) link employing the optimal proposed beamformers (which completely nullifies the interference to and from the primary link while maximizing its own achievable rate) is the same as the rate of an interference-free (N_t^{C}-1)x 1 multiple-input single-output (MISO) link employing an optimal maximal ratio transmission beamforming vector. The sum rate performance of the proposed algorithms is evaluated by Monte Carlo simulations. The impact of the number of transmit and receive antennas on the proposed algorithm is also discussed.

The capacity of a wireless network could be considerably improved by employing directional antennas that are capable of illuminating multiple beams toward different directions. However, more beams from the same BS may lead to stronger inter-cell interference. In this paper, we consider coordinated beam scheduling schemes to mitigate the inter-cell interference. We first formulate this problem as a combinatorial optimization problem. We then reveal that the complexity of this problem hinges upon a single scalar termed as degree of constraint (DoC), which is related to the degree of conflict a beam is subject to. If the DoC is at least three, the general beam scheduling problem is NP-hard. If DoC is smaller than three, which corresponds to a relevant subclass of the beam scheduling problem arising in practice, this problem can be solved in polynomial time. We propose an optimal beam scheduling algorithm based on the auction method to this particular subclass of problem. This algorithm is of low complexity and is well suited for distributed implementations. We then extend the auction algorithm to solve the general multi-cell beam scheduling problem. The performance of the proposed algorithms is finally assessed through extensive simulation studies.

In this paper, we consider the problem of resource allocation in the DFT-Spread-OFDMA (DFT-S-OFDMA) uplink. We show that the resource allocation problem can be formulated as a set packing problem, which in general is NP-hard. We propose polynomial-time message-passing based algorithms, one of which is guaranteed to yield a solution that is within a constant fraction of the optimal solution and is also asymptotically optimal in the limit as the number of subcarriers in the system goes to infinity. The message-passing based algorithm is also extended to solve the resource allocation problem over a multi-cell uplink in a distributed fashion. Our algorithms account for finite input alphabets and non-ideal practical outer codes. Extensive simulations are performed to assess the performance of the proposed algorithms and it is shown that they yield near-optimal solutions at a low complexity and with a low memory requirement.

We analyze the performance of message-passing decoding of finite-length concatenated codes. We first show that the message-passing decoder is closely related to a dual optimization decoder. The connections between these two decoders are further elucidated by proving that both of them attain the same objective function value of a generalized linear programming decoder in the limit as the signal-to-noise ratio (SNR) goes to infinity. Consequently, the framework of pseudo-weight analysis, which was originally proposed for analyzing the linear programming decoder, can be extended to analyze the performance of the message-passing decoder for finite-length codes. We then derive lower bounds to the pseudo-weights of general concatenated codes by utilizing the special structure of their parity-check matrices. We finally present a method to increase the max-fractional weight by adding redundant parity-check constraints and thereby improving the decoding performance. Simulation studies are carried out to assess the performance of the proposed algorithms and substantiate the theoretic claims.

We propose resource allocation algorithms based on the auction method for uplink OFDMA cellular networks. We consider cellular systems that employ the traditional static frequency reuse as well as the next-generation systems that aim to achieve a universal frequency reuse via base-station coordination. Our algorithms are designed for finite input alphabets and also account for non-ideal practical outer codes, and they can be implemented in a distributed manner, when applied for multi-cell resource allocation. The proposed algorithms have a complexity of O(N) per user per iteration, where N denotes the number of subcarriers in the system, and are also well suited for parallel implementations. We also address power and bandwidth constraints that are motivated by practical concerns. The proposed algorithms exhibit very low complexity and simulation results demonstrate that they offer near-optimal performance.

Robustness of optimization models for networking problems has been an under-explored area. Yet most existing algorithms for solving robust optimization problems are centralized, thus not suitable for many communication networking problems that demand distributed solutions. This paper represents the first step towards building a framework for designing distributed robust optimization algorithms. We first discuss several models for describing parameter uncertainty sets that can lead to decomposable problem structures. These models include general polyhedron, D-norm, and ellipsoid. We then apply these models to solve robust power control in wireless networks and robust rate control in wireline networks. In both applications, we propose distributed algorithms that converge to the optimal robust solution. Various tradeoffs among performance, robustness, and distributiveness are illustrated both analytically and through simulations.

We propose a general network planning framework for multi-radio multi-channel wireless networks. Under this framework, data routing, resource allocation, and scheduling are jointly designed to maximize a network utility function. We first treat such a cross-layer design problem with fixed radio distributions across the nodes and formulate it as a large-scale convex optimization problem. A primal-dual method together with the column-generation technique is proposed to efficiently solve this problem. We then consider the radio allocation problem, i.e., the optimal placement of radios within the network to maximize the network utility function. This problem is formulated as a large- scale combinatorial optimization problem. We derive the necessary conditions that the optimal solution should satisfy, and then develop a sequential optimization scheme to solve this problem. Simulation studies are carried out to assess the performance of the proposed cross-layer network planning framework. It is seen that the proposed approach can significantly enhance the overall network performance.

In this paper, we propose a new linear programming formulation for the decoding of general linear block codes. Different from the original formulation given by Feldman, the number of total variables to characterize a parity-check constraint in our formulation is less than twice the degree of the corresponding check node. The equivalence between our new formulation and the original formulation is proven. The new formulation facilitates to characterize the structure of linear block codes, and leads to new decoding algorithms. In particular, we show that any fundamental polytope is simply the intersection of a group of the so-called minimum polytopes, and this simplified formulation allows us to formulate the problem of calculating the minimum Hamming distance of any linear block code as a simple linear integer programming problem with much less auxiliary variables. We then propose a branch-and-bound method to compute a lower bound to the minimum distance of any linear code by solving a corresponding linear integer programming problem. In addition, we prove that, for the family of single parity-check (SPC) product codes, the fractional distance and the pseudodistance are both equal to the minimum distance. Finally, we propose an efficient algorithm for decoding SPC product codes with low complexity and maximum-likelihood (ML) decoding performance.

We treat the problem of designing the optimal transmission scheme that is adapted to the battery state, the channel and buffer conditions, and the incoming traffic rate. We assume that the battery states can be tracked at every time slot, so that the problem is formulated as a large-scale Markov decision process (MDP). An efficient sparse sampling method is employed to obtain a solution. Simulation results are provided to demonstrate that the proposed schemes can considerably increase the lifetime of the battery-powered wireless systems while satisfying QoS (quality of Service) constraints.

We consider the maximum-likelihood decoding of single parity-check (SPC) product code. We first prove that, for the family of SPC product code, the fractional distance and the pseudo-distance are both equal to the minimum Hamming distance. We then develop an efficient algorithm for decoding SPC product codes with low complexity and near maximum likelihood decoding performance at practical SNRs.

We propose a new linear programming formulation for the decoding of general linear block codes. Different from the original formulation given in [1], the number of total variables to characterize a parity-check constraint in our formulation is less than twice the degree of the corresponding check node. The equivalence between our new formulation and the original formulation is proven. Moreover, we show that any fundamental polytope is simply the intersection of a group of so-called minimum polytopes. Based on this, we propose a branch-and-bound method to compute a non-trivial lower bound to the minimum distance of a linear block code with affordable complexity.

Abstract We propose a 4G system for transmission of video from a server at the base station to numerous wireless clients. We employ the latest technology in scalable video compression (3‐D wavelet video coding) and in channel coding (punctured turbo codes); for the physical layer, we resort to the multicarrier chip‐interleaved system with two‐layer interleaving, which achieves high spectral efficiency and is very suitable for downlink applications. We develop fast algorithms for a cross‐layer resource allocation that minimize the expected distortion of the reconstructed video averaged over all clients. The algorithms find a near‐optimal power, bandwidth, and subcarrier allocation at the physical layer and a source‐channel symbol allocation at the application layer. Our experimental results demonstrate that such a cross‐layer optimization framework leads to higher quality performance of the overall system. Copyright © 2007 John Wiley & Sons, Ltd.

We consider the radio allocation problem for multi-radio multi-channel cognitive wireless networks, i.e., the optimal placement of radios within the network to maximize the network utility function. This problem is formulated as a large-scale combinatorial optimization problem. We derive the necessary conditions that the optimal solution should satisfy, and then develop a sequential optimization scheme to solve this problem. Simulation studies are carried out to assess the performance of the proposed radio allocation framework. It is seen that the proposed approach can significantly enhance the overall network performance.

We consider the decoding problem for low-density parity-check codes, and apply nonlinear programming methods. This extends previous work using linear programming (LP) to decode linear block codes. First, a multistage LP decoder based on the branch-and-bound method is proposed. This decoder makes use of the maximum-likelihood-certificate property of the LP decoder to refine the results when an error is reported. Second, we transform the original LP decoding formulation into a box-constrained quadratic programming form. Efficient linear-time parallel and serial decoding algorithms are proposed and their convergence properties are investigated. Extensive simulation studies are performed to assess the performance of the proposed decoders. It is seen that the proposed multistage LP decoder outperforms the conventional sum-product (SP) decoder considerably for low-density parity-check (LDPC) codes with short to medium block length. The proposed box-constrained quadratic programming decoder has less complexity than the SP decoder and yields much better performance for LDPC codes with regular structure

We propose a multicarrier interleave-division multiple-access scheme for ultra-wideband wireless communications. For the uplink, a chip-level interleaving method at the transmitter with a simple turbo receiver is proposed to effectively suppress the frequency-selective fading and multiple-access interference. Both hard and soft frequency notching methods are suggested to suppress the narrowband interference. For the downlink, a two-layer interleaving scheme is proposed, in which data from different users are separated in the frequency-domain while the different data streams for the same user are separated in the interleaver-domain. A joint power and subcarrier allocation algorithm is developed to exploit the multiuser diversity and thereby improve the system performance. The performance of the proposed system is evaluated by both theoretic analysis and simulations. It is seen that within a few iterations, the proposed system exhibits single-user performance even under over-loaded conditions

This paper proposes a unified two-dimensional maximum-likelihood multipath channel estimator for both multicarrier and single-carrier CDMA systems with cyclic prefix. The conditions for the optimal training sequence and the lower bound for the performance of this estimator are obtained by solving an optimisation problem. Optimal and sub-optimal training sequences are designed from several classes of sequences with ideal autocorrelation and cross correlation properties. Furthermore, the complexity of this channel estimator is reduced considerably by applying the optimal training sequence. The simulation results demonstrate that this channel estimator combined with optimal or sub-optimal sequence can give near single-user channel estimation performance even for fully loaded systems.

This paper shows how a unified model can he used to describe the signal received in frequency domain over a multiple-antenna, frequency-selective, multipath fading channel for multicarrier CDMA and single carrier CDMA systems with cyclic prefix. This realization validates the abstraction of these CDMA problems into a mathematical model which can be used for practical problems such as channel estimation and data detection. Moreover, the optimal multiuser detection problem based on the maximal-likelihood criteria for this unified model is addressed as a combinatorial optimization problem. A sequential quadratic programming approach is proposed to approximate this problem, which corresponds to a near-optimal software-defined decision-feedback multiuser receiver for both single-carrier and multicarrier CDMA systems. An efficient gradient projection algorithm is applied for data detection in the proposed receiver. The simulation results show that this scheme can reach near single-user performance even for fully-loaded systems and outperform conventional soft-interference cancellation methods.

In this paper we suggest a new single carrier cyclic prefix assisted multiple access scheme for the asynchronous broadband CDMA system in uplink. Frequency domain equalization is applied instead of conventional RAKE combining or time domain equalization. A new serial type multistage interference cancellation in frequency domain is proposed to cancel the multiple access interference while keeping low computational complexity. We also investigated the effects of interference rejection weight control for this multistage interference cancellation. Another hybrid system combining the successive interference cancellation and parallel interference cancellation is also proposed, which can reach the similar performance of the serial type while reducing the processing delay greatly.