We consider the problem of clustering data points coming from sub-Gaussian mixtures. Existing methods that provably achieve the optimal mislabeling error, such as the Lloyd algorithm, are usually vulnerable to outliers. In contrast, clustering methods seemingly robust to adversarial perturbations are not known to satisfy the optimal statistical guarantees. We propose a simple robust algorithm based on the coordinatewise median that obtains the optimal mislabeling rate even when we allow adversarial outliers to be present. Our algorithm achieves the optimal error rate in constant iterations when a weak initialization condition is satisfied. In the absence of outliers, in fixed dimensions, our theoretical guarantees are similar to that of the Lloyd algorithm. Extensive experiments on various simulated and public datasets are conducted to support the theoretical guarantees of our method.

Clustering is a fundamental tool in statistical machine learning in the presence of heterogeneous data. Most recent results focus primarily on optimal mislabeling guarantees when data are distributed around centroids with sub-Gaussian errors. Yet, the restrictive sub-Gaussian model is often invalid in practice, since various real-world applications exhibit heavy-tail distributions around the centroids or suffer from possible adversarial attacks that call for robust clustering with a robust data-driven initialization. In this paper, we present initialization and subsequent clustering methods that provably guarantee near-optimal mislabeling for general mixture models when the number of clusters and data dimensions are finite. We first introduce a hybrid clustering technique with a novel multivariate trimmed mean type centroid estimate to produce mislabeling guarantees under a weak initialization condition for general error distributions around the centroids. A matching lower bound is derived, up to factors depending on the number of clusters. In addition, our approach also produces similar mislabeling guarantees even in the presence of adversarial outliers. Our results reduce to the sub-Gaussian case in finite dimensions when errors follow sub-Gaussian distributions. To solve the problem thoroughly, we also present novel data-driven robust initialization techniques and show that, with probabilities approaching one, these initial centroid estimates are sufficiently good for the subsequent clustering algorithm to achieve the optimal mislabeling rates. Furthermore, we demonstrate that Lloyd’s algorithm is suboptimal for more than two clusters even when errors are Gaussian and for two clusters when error distributions have heavy tails. Both simulated data and real data examples further support our robust initialization procedure and clustering algorithm.

Decentralized finance (DeFi) borrowing and lending platforms are crucial to the decentralized economy, involving two main participants: lenders who provide assets for interest and borrowers who offer collateral exceeding their debt and pay interest. Collateral volatility necessitates over-collateralization to protect lenders and ensure competitive returns. Traditional DeFi platforms use a fixed interest rate curve based on the utilization rate (the fraction of available assets borrowed) and determine over-collateralization offline through simulations to manage risk. This method doesn't adapt well to dynamic market changes, such as price fluctuations and evolving user needs, often resulting in losses for lenders or borrowers. In this paper, we introduce an adaptive, data-driven protocol for DeFi borrowing and lending. Our approach includes a high-frequency controller that dynamically adjusts interest rates to maintain market stability and competitiveness with external markets. Unlike traditional protocols, which rely on user reactions and often adjust slowly, our controller uses a learning-based algorithm to quickly find optimal interest rates, reducing the opportunity cost for users during periods of misalignment with external rates. Additionally, we use a low-frequency planner that analyzes user behavior to set an optimal over-collateralization ratio, balancing risk reduction with profit maximization over the long term. This dual approach is essential for adaptive markets: the short-term component maintains market stability, preventing exploitation, while the long-term planner optimizes market parameters to enhance profitability and reduce risks. We provide theoretical guarantees on the convergence rates and adversarial robustness of the short-term component and the long-term effectiveness of our protocol. Empirical validation confirms our protocol's theoretical benefits.

Automated Market Makers (AMMs) are essential in Decentralized Finance (DeFi) as they match liquidity supply with demand. They function through liquidity providers (LPs) who deposit assets into liquidity pools. However, the asset trading prices in these pools often trail behind those in more dynamic, centralized exchanges, leading to potential arbitrage losses for LPs. This issue is tackled by adapting market maker bonding curves to trader behavior, based on the classical market microstructure model of Glosten and Milgrom. Our approach ensures a zero-profit condition for the market maker's prices. We derive the differential equation that an optimal adaptive curve should follow to minimize arbitrage losses while remaining competitive. Solutions to this optimality equation are obtained for standard Gaussian and Lognormal price models using Kalman filtering. A key feature of our method is its ability to estimate the external market price without relying on price or loss oracles. We also provide an equivalent differential equation for the implied dynamics of canonical static bonding curves and establish conditions for their optimality. Our algorithms demonstrate robustness to changing market conditions and adversarial perturbations, and we offer an on-chain implementation using Uniswap v4 alongside off-chain AI co-processors.

Motivated by applications in large-scale and multi-agent reinforcement learning, we study the non-asymptotic performance of stochastic approximation (SA) schemes with delayed updates under Markovian sampling. While the effect of delays has been extensively studied for optimization, the manner in which they interact with the underlying Markov process to shape the finite-time performance of SA remains poorly understood. In this context, our first main contribution is to show that under time-varying bounded delays, the delayed SA update rule guarantees exponentially fast convergence of the \emph{last iterate} to a ball around the SA operator’s fixed point. Notably, our bound is \emph{tight} in its dependence on both the maximum delay $\tau_{max}$, and the mixing time $\tau_{mix}$. To achieve this tight bound, we develop a novel inductive proof technique that, unlike various existing delayed-optimization analyses, relies on establishing uniform boundedness of the iterates. As such, our proof may be of independent interest. Next, to mitigate the impact of the maximum delay on the convergence rate, we provide the first finite-time analysis of a delay-adaptive SA scheme under Markovian sampling. In particular, we show that the exponent of convergence of this scheme gets scaled down by $\tau_{avg}$, as opposed to $\tau_{max}$ for the vanilla delayed SA rule; here, $\tau_{avg}$ denotes the average delay across all iterations. Moreover, the adaptive scheme requires no prior knowledge of the delay sequence for step-size tuning. Our theoretical findings shed light on the finite-time effects of delays for a broad class of algorithms, including TD learning, Q-learning, and stochastic gradient descent under Markovian sampling.

The increased adoption of smart contracts in many industries has made them an attractive target for cybercriminals, leading to millions of dollars in losses. Thus, continuously fixing newly found vulnerabilities of smart contracts becomes a routine software maintenance task for running smart contracts. However, fixing the vulnerabilities that are specific to the smart contract domain requires security knowledge that many developers lack. Without effective tool support, this task can be very costly in terms of manual labor. To fill this critical need, in this paper, we propose VFIX, which automatically generates security patches for vulnerable smart contracts. In particular, VFIX provides a novel program analysis framework that can incorporate different fix patterns for fixing various types of vulnerabilities. To address the unique challenges in accurately fixing smart contract vulnerabilities, VFIX innovatively combines template-based repair with a set of static program analysis techniques specially designed for smart contracts. Specifically, given an input smart contract, VFIX conducts ensemble identification based on multiple static verification tools to identify vulnerabilities for an automatic fix. Then, VFIX generates patches using template-based fix patterns, and conducts static program analysis (e.g., program dependency computation, pointer analysis) for smart contracts to accurately infer and populate the parameter values for the fix templates. Finally, VFIX performs static verification to ensure that the patched contract is free of vulnerabilities. Our evaluations on 144 real smart contracts containing different types of vulnerabilities show that VFIX can successfully fix 94% of the vulnerabilities and preserve the expected normal behaviors of the smart contracts.

We study federated learning (FL), where power-limited wireless devices utilize their local datasets to collaboratively train a global model with the help of a remote parameter server (PS). The PS has access to the global model and shares it with the devices for local training using their datasets, and the devices return the result of their local updates to the PS to update the global model. The algorithm continues until the convergence of the global model. This framework requires downlink transmission from the PS to the devices and uplink transmission from the devices to the PS. The goal of this study is to investigate the impact of the bandwidth-limited shared wireless medium on the performance of FL with a focus on the downlink. To this end, the downlink and uplink channels are modeled as fading broadcast and multiple access channels, respectively, both with limited bandwidth. For downlink transmission, we first introduce a digital approach, where a quantization technique is employed at the PS followed by a capacity-achieving channel code to transmit the global model update over the wireless broadcast channel at a common rate such that all the devices can decode it. Next, we propose analog downlink transmission, where the global model is broadcast by the PS in an uncoded manner. We consider analog transmission over the uplink in both cases, since its superiority over digital transmission for uplink has been well studied in the literature. We further analyze the convergence behavior of the proposed analog transmission approach over the downlink assuming that the uplink transmission is error-free. Numerical experiments show that the analog downlink approach provides significant improvement over the digital one with a more notable improvement when the data distribution across the devices is not independent and identically distributed. The experimental results corroborate the convergence analysis, and show that a smaller number of local iterations should be used when the data distribution is more biased, and also when the devices have a better estimate of the global model in the analog downlink approach.

Log-based cyber threat hunting has emerged as an important solution to counter sophisticated cyber attacks. However, existing approaches require non-trivial efforts of manual query construction and have overlooked the rich external knowledge about threat behaviors provided by open-source Cyber Threat Intelligence (OSCTI). To bridge the gap, we build ThreatRaptor, a system that facilitates cyber threat hunting in computer systems using OSCTI. Built upon mature system auditing frameworks, ThreatRaptor provides (1) an unsupervised, light-weight, and accurate NLP pipeline that extracts structured threat behaviors from unstructured OSCTI text, (2) a concise and expressive domain-specific query language, TBQL, to hunt for malicious system activities, (3) a query synthesis mechanism that automatically synthesizes a TBQL query from the extracted threat behaviors, and (4) an efficient query execution engine to search the big system audit logging data.

Log-based cyber threat hunting has emerged as an important solution to counter sophisticated attacks. However, existing approaches require non-trivial efforts of manual query construction and have overlooked the rich external threat knowledge provided by open-source Cyber Threat Intelligence (OSCTI). To bridge the gap, we propose ThreatRaptor, a system that facilitates threat hunting in computer systems using OSCTI. Built upon system auditing frameworks, ThreatRaptor provides (1) an unsupervised, light-weight, and accurate NLP pipeline that extracts structured threat behaviors from unstructured OSCTI text, (2) a concise and expressive domain-specific query language, TBQL, to hunt for malicious system activities, (3) a query synthesis mechanism that automatically synthesizes a TBQL query for hunting, and (4) an efficient query execution engine to search the big audit logging data. Evaluations on a broad set of attack cases demonstrate the accuracy and efficiency of ThreatRaptor in practical threat hunting.

We study federated edge learning (FEEL), where wireless edge devices, each with its own dataset, learn a global model collaboratively with the help of a wireless access point acting as the parameter server (PS). At each iteration, wireless devices perform local updates using their local data and the most recent global model received from the PS, and send their local updates to the PS over a wireless fading multiple access channel (MAC). The PS then updates the global model according to the signal received over the wireless MAC, and shares it with the devices. Motivated by the additive nature of the wireless MAC, we propose an analog `over-the-air' aggregation scheme, in which the devices transmit their local updates in an uncoded fashion. However, unlike recent literature on over-the-air FEEL, here we assume that the devices do not have channel state information (CSI), while the PS has imperfect CSI. On the other hand, the PS is equipped with multiple antennas to alleviate the destructive effect of the channel, exacerbated due to the lack of perfect CSI. We design a receive beamforming scheme at the PS, and show that it can compensate for the lack of perfect CSI when the PS has a sufficient number of antennas. We also derive the convergence rate of the proposed algorithm highlighting the impact of the lack of perfect CSI, as well as the number of PS antennas. Both the experimental results and the convergence analysis illustrate the performance improvement of the proposed algorithm with the number of PS antennas, where the wireless fading MAC becomes deterministic despite the lack of perfect CSI when the PS has a sufficiently large number of antennas.

We study federated learning (FL) at the wireless edge, where power-limited devices with local datasets collaboratively train a joint model with the help of a remote parameter server (PS). We assume that the devices are connected to the PS through a bandwidth-limited shared wireless channel. At each iteration of FL, a subset of the devices are scheduled to transmit their local model updates to the PS over orthogonal channel resources, while each participating device must compress its model update to accommodate to its link capacity. We design novel scheduling and resource allocation policies that decide on the subset of the devices to transmit at each round, and how the resources should be allocated among the participating devices, not only based on their channel conditions, but also on the significance of their local model updates. We then establish convergence of a wireless FL algorithm with device scheduling, where devices have limited capacity to convey their messages. The results of numerical experiments show that the proposed scheduling policy, based on both the channel conditions and the significance of the local model updates, provides a better long-term performance than scheduling policies based only on either of the two metrics individually. Furthermore, we observe that when the data is independent and identically distributed (i.i.d.) across devices, selecting a single device at each round provides the best performance, while when the data distribution is non-i.i.d., scheduling multiple devices at each round improves the performance. This observation is verified by the convergence result, which shows that the number of scheduled devices should increase for a less diverse and more biased data distribution.

The need for countering Advanced Persistent Threat (APT) attacks has led to the solutions that ubiquitously monitor system activities in each enterprise host, and perform timely abnormal system behavior detection over the stream of monitoring data. However, existing stream-based solutions lack explicit language constructs for expressing anomaly models that capture abnormal system behaviors, thus facing challenges in incorporating expert knowledge to perform timely anomaly detection over the large-scale monitoring data. To address these limitations, we build SAQL, a novel stream-based query system that takes as input, a real-time event feed aggregated from multiple hosts in an enterprise, and provides an anomaly query engine that queries the event feed to identify abnormal behaviors based on the specified anomaly models. SAQL provides a domain-specific query language, Stream-based Anomaly Query Language ( SAQL), that uniquely integrates critical primitives for expressing major types of anomaly models. In the demo, we aim to show the complete usage scenario of SAQL by (1) performing an APT attack in a controlled environment, and (2) using SAQL to detect the abnormal behaviors in real time by querying the collected stream of system monitoring data that contains the attack traces. The audience will have the option to interact with the system and detect the attack footprints in real time via issuing queries and checking the query results through a command-line UI.

We study federated learning (FL) at the wireless edge, where power-limited devices with local datasets train a joint model with the help of a remote parameter server (PS). We assume that the devices are connected to the PS through a bandwidth-limited shared wireless channel. At each iteration of FL, a subset of the devices are scheduled to transmit their local model updates to the PS over orthogonal channel resources. We design novel scheduling policies, that decide on the subset of devices to transmit at each round not only based on their channel conditions, but also on the significance of their local model updates. Numerical results show that the proposed scheduling policy provides a better long-term performance than scheduling policies based only on either of the two metrics individually. We also observe that when the data is independent and identically distributed (i.i.d.) across devices, selecting a single device at each round provides the best performance, while when the data distribution is non-i.i.d., more devices should be scheduled.

This paper develops deterministic upper and lower bounds on the influence measure in a network, more precisely, the expected number of nodes that a seed set can influence in the independent cascade model. In particular, our bounds exploit r-nonbacktracking walks and Fortuin-Kasteleyn-Ginibre (FKG) type inequalities, and are computed by message passing algorithms. Further, we provide parameterized versions of the bounds that control the trade-off between efficiency and accuracy. Finally, the tightness of the bounds is illustrated on various network models.

The need for countering Advanced Persistent Threat (APT) attacks has led to the solutions that ubiquitously monitor system activities in each enterprise host, and perform timely attack investigation over the monitoring data for uncovering the attack sequence. However, existing general-purpose query systems lack explicit language constructs for expressing key properties of major attack behaviors, and their semantics-agnostic design often produces inefficient execution plans for queries. To address these limitations, we build Aiql, a novel query system that is designed with novel types of domain-specific optimizations to enable efficient attack investigation. Aiql provides (1) a domain-specific data model and storage for storing the massive system monitoring data, (2) a domain-specific query language, Attack Investigation Query Language (Aiql) that integrates critical primitives for expressing major attack behaviors, and (3) an optimized query engine based on the characteristics of the data and the semantics of the query to efficiently schedule the execution. We have deployed Aiql in NEC Labs America comprising 150 hosts. In our demo, we aim to show the complete usage scenario of Aiql by (1) performing an APT attack in a controlled environment, and (2) using Aiql to investigate such attack by querying the collected system monitoring data that contains the attack traces. The audience will have the option to perform the APT attack themselves under our guidance, and interact with the system and investigate the attack via issuing queries and checking the query results through our web UI.

This paper develops upper and lower bounds on the influence measure in a network, more precisely, the expected number of nodes that a seed set can influence in the independent cascade model. In particular, our bounds exploit nonbacktracking walks, Fortuin-Kasteleyn-Ginibre type inequalities, and are computed by message passing algorithms. Nonbacktracking walks have recently allowed for headways in community detection, and this paper shows that their use can also impact the influence computation. Further, we provide parameterized versions of the bounds that control the trade-off between the efficiency and the accuracy. Finally, the tightness of the bounds is illustrated with simulations on various network models.

Attack detection problems in the smart grid are posed as statistical learning problems for different attack scenarios in which the measurements are observed in batch or online settings. In this approach, machine learning algorithms are used to classify measurements as being either secure or attacked. An attack detection framework is provided to exploit any available prior knowledge about the system and surmount constraints arising from the sparse structure of the problem in the proposed approach. Well-known batch and online learning algorithms (supervised and semisupervised) are employed with decision- and feature-level fusion to model the attack detection problem. The relationships between statistical and geometric properties of attack vectors employed in the attack scenarios and learning algorithms are analyzed to detect unobservable attacks using statistical learning methods. The proposed algorithms are examined on various IEEE test systems. Experimental analyses show that machine learning algorithms can detect attacks with performances higher than attack detection algorithms that employ state vector estimation methods in the proposed attack detection framework.

This paper studies the statistical convergence and consistency of regularized boosting methods, where the samples need not be independent and identically distributed but can come from stationary weakly dependent sequences. Consistency is proven for the composite classifiers that result from a regularization achieved by restricting the 1-norm of the base classifiers' weights. The less restrictive nature of sampling considered here is manifested in the consistency result through a generalized condition on the growth of the regularization parameter. The weaker the sample dependence, the faster the regularization parameter is allowed to grow with increasing sample size. A consistency result is also provided for data-dependent choices of the regularization parameter.

Recently, Cyberlocker services have gained great popularity in the file-sharing market. Driven by tremendous benefits a large number of files such as popular movies are uploaded to Cyberlockers. We explore the profit chain of file-sharing networks based on Cyberlockers and find that an important issue is how to collect the download URLs of popular files stored at different Cyberlockers and share them with public users. In this paper, we focus on these sites collecting and sharing the Cyberlocker URLs of movies, called Cyberlocker URL-sharing sites. First, we extract 1,587 URL-sharing sites based on 31,525 valid pages returned by Google search and demonstrate that the quality distribution of these sites follows a power-law. Second, we analyze the link citations among URL-sharing sites and build the directed link citation graph. By characterizing basic metrics of the graph, such as cited strength and in/out-degree, we understand the structure of URL-sharing sites in depth. Furthermore, we discover that Cyberlocker URLs can be disseminated dynamically through crawler mechanisms among different sites, and highlight the implications of such metrics in this context. Additionally, we study the security risks of 1,587 URL-sharing sites. The results show that security risks do exist when surfing 155 suspicious URL-sharing sites such as myrls.me and rapid4me.com although the majority sites (90.23%) are safe. Finally, some preliminary suggestions are discussed from the industry point of view for how to improve the effectiveness of searching, collecting and disseminating Cyberlocker URLs. To the best of our knowledge, this is the first work on the measurement and understanding of Cyberlocker URL-sharing sites.

A new segmentation fusion method is proposed that ensembles the output of several segmentation algorithms applied on a remotely sensed image. The candidate segmentation sets are processed to achieve a consensus segmentation using a stochastic optimization algorithm based on the Filtered Stochastic BOEM (Best One Element Move) method. For this purpose, Filtered Stochastic BOEM is reformulated as a segmentation fusion problem by designing a new distance learning approach. The proposed algorithm also embeds the computation of the optimum number of clusters into the segmentation fusion problem.

We analyze a class of distributed quantized consensus algorithms for arbitrary networks. In the initial setting, each node in the network has an integer value. Nodes exchange their current estimate of the mean value in the network, and then update their estimate by communicating with their neighbors in a limited capacity channel in an asynchronous clock setting. Eventually, all nodes reach consensus with quantized precision. We start the analysis with a special case of a distributed binary voting algorithm, then proceed to the expected convergence time for the general quantized consensus algorithm proposed by Kashyap et al. We use the theory of electric networks, random walks, and couplings of Markov chains to derive an O(N3log N) upper bound for the expected convergence time on an arbitrary graph of size N, improving on the state of art bound of O(N4log N) for binary consensus and O(N5) for quantized consensus algorithms. Our result is not dependent on the graph topology. Simulations are performed to validate the analysis.

With the rapid development of smart devices and wireless communication, especially with the pre-launch of Google Glass, augmented reality (AR) has received enormous attention recently. AR adds virtual objects into a user's real-world environment enabling live interaction in three dimensions. Limited by the small display of AR devices, content selection is one of the key issues to improve user experience. In this paper, we present an aggregated random walk algorithm incorporating personal preferences, location information, and temporal information in a layered graph. By adaptively changing the graph edge weight and computing the rank score, the proposed AR recommender system predicts users' preferences and provides the most relevant recommendations with aggregated information.

New methods that exploit sparse structures arising in smart grid networks are proposed for the state estimation problem when data injection attacks are present. First, construction strategies for unobservable sparse data injection attacks on power grids are proposed for an attacker with access to all network information and nodes. Specifically, novel formulations for the optimization problem that provide a flexible design of the trade-off between performance and false alarm are proposed. In addition, the centralized case is extended to a distributed framework for both the estimation and attack problems. Different distributed scenarios are proposed depending on assumptions that lead to the spreading of the resources, network nodes and players. Consequently, for each of the presented frameworks a corresponding optimization problem is introduced jointly with an algorithm to solve it. The validity of the presented procedures in real settings is studied through extensive simulations in the IEEE test systems.

In this work, a variational Bayesian framework for efficient training of echo state networks (ESNs) with automatic regularization and delay&sum (D&S) readout adaptation is proposed. The algorithm uses a classical batch learning of ESNs. By treating the network echo states as fixed basis functions parameterized with delay parameters, we propose a variational Bayesian ESN training scheme. The variational approach allows for a seamless combination of sparse Bayesian learning ideas and a variational Bayesian space-alternating generalized expectation-maximization (VB-SAGE) algorithm for estimating parameters of superimposed signals. While the former method realizes automatic regularization of ESNs, which also determines which echo states and input signals are relevant for "explaining" the desired signal, the latter method provides a basis for joint estimation of D&S readout parameters. The proposed training algorithm can naturally be extended to ESNs with fixed filter neurons. It also generalizes the recently proposed expectation-maximization-based D&S readout adaptation method. The proposed algorithm was tested on synthetic data prediction tasks as well as on dynamic handwritten character recognition.

The information-theoretic notion of energy efficiency is studied in the context of various joint source-channel coding problems. The minimum transmission energyE(D) required to communicate a source over a noisy channel so that it can be reconstructed within a target distortionDis analyzed. Unlike the traditional joint source-channel coding formalisms, no restrictions are imposed on the number of channel uses per source sample. For single-source memoryless point-to-point channels,E(D) is shown to be equal to the product of the minimum energy per bitEbminof the channel and the rate-distortion functionR(D) of the source, regardless of whether channel output feedback is available at the transmitter. The primary focus is on Gaussian sources and channels affected by additive white Gaussian noise under quadratic distortion criteria, with or without perfect channel output feedback. In particular, for two correlated Gaussian sources communicated over a Gaussian multiple-access channel, inner and outer bounds on the energy-distortion region are obtained, which coincide in special cases. For symmetric channels, the difference between the upper and lower bounds on energy is shown to be at most a constant even when the lower bound goes to infinity asD→ 0. It is also shown that simple uncoded transmission schemes perform better than the separation-based schemes in many different regimes, both with and without feedback.

In this work a new adaptive fast variational sparse Bayesian learning (V-SBL) algorithm is proposed that is a variational counterpart of the fast marginal likelihood maximization approach to SBL. It al lows one to adaptively construct a sparse regression or classification function as a linear combination of a few basis functions by minimizing the variational free energy. In the case of non-informative hyperpriors, also referred to as automatic relevance determination, the minimization of the free energy can be efficiently realized by computing the fixed points of the update expressions for the variational distribution of the sparsity parameters. The criteria that establish convergence to these fixed points, termed pruning conditions, allow an efficient addition or removal of basis functions; they also have a simple and intuitive interpretation in terms of a component's signal-to-noise ratio. It has been demonstrated that this interpretation allows a simple empirical adjustment of the pruning conditions, which in turn improves sparsity of SBL and drastically accelerates the convergence rate of the algorithm. The experimental evidence collected with synthetic data demonstrates the effectiveness of the proposed learning scheme.

Precoding has been extensively studied for point-to-point communications, including the problems of constructing the precoding codebook and selecting the best precoder. This paper investigates precoding for a multicast channel in which a base station is sending the same information to all users and each user sends back the index of its best precoding matrix. It is assumed that users do not collaborate and that no channel state information is known at the base station. Optimization problems are formulated to reduce the packet drop rate. A set of probabilistic algorithms that effectively reduce the average package drop rate are presented. It is shown numerically that these new schemes lead to significant improvements.

Recommendation systems have received considerable attention recently. However, most research has been focused on improving the performance of collaborative filtering (CF) techniques. Social networks, indispensably, provide us extra information on people's preferences, and should be considered and deployed to improve the quality of recommendations. In this paper, we propose two recommendation models, for individuals and for groups respectively, based on social contagion and social influence network theory. In the recommendation model for individuals, we improve the result of collaborative filtering prediction with social contagion outcome, which simulates the result of information cascade in the decision-making process. In the recommendation model for groups, we apply social influence network theory to take interpersonal influence into account to form a settled pattern of disagreement, and then aggregate opinions of group members. By introducing the concept of susceptibility and interpersonal influence, the settled rating results are flexible, and inclined to members whose ratings are "essential".

In this paper, we consider scaling laws for maximal energy efficiency of communicating a message to all the nodes in a wireless network, as the number of nodes in the network becomes large. Two cases of large wireless networks are studied-dense random networks and constant density (extended) random networks. In addition, we also study finite size regular networks in order to understand how regularity in node placement affects energy consumption. We first establish an information-theoretic lower bound on the minimum energy per bit for multicasting in arbitrary wireless networks when the channel state information is not available at the transmitters. Upper bounds are obtained by constructing a simple flooding scheme that requires no information at the receivers about the channel states or the locations and identities of the nodes. The gap between the upper and lower bounds is only a constant factor for dense random networks and regular networks, and differs by a poly-logarithmic factor for extended random networks. Furthermore, we show that the proposed upper and lower bounds for random networks hold almost surely in the node locations as the number of nodes approaches infinity.

In this paper, we study the minimum energy per bit required for communicating a message to all the destination nodes in a wireless network. The physical layer is modeled as an additive white Gaussian noise (AWGN) channel affected by circularly symmetric fading. The fading coefficients are known at neither transmitters nor receivers. We provide an information-theoretic lower bound on the energy requirement of general multicasting in arbitrary networks as the solution of a linear program, when no restrictions are placed on the bandwidth or the delay. We study the performance of decode-and-forward operating in the noncoherent wideband scenario, and compare it with the lower bound, for a variety of network classes where all nonsource nodes are destinations. For three-terminal networks with one source and two cooperative destination nodes, the energy expenditure of decode-and-forward is shown to be at most twice the lower bound and optimal in many cases. We also show that for arbitrary networks withknodes, the energy requirement of decode-and-forward is at mostk-1 times that of the lower bound regardless of the magnitude of channel gains. In networks that can be represented as directed acyclic graphs (DAGs), we establish the minimum energy per bit, also achieved by decode-and-forward. In addition, we also study regular networks where the energy consumption of decode-and-forward is shown to be almost order optimal in many situations of interest.

We address the problem of estimating the probability of an observed string that is drawn i.i.d. from an unknown distribution. Motivated by models of natural language, we consider the regime in which the length of the observed string and the size of the underlying alphabet are comparably large. In this regime, the maximum likelihood distribution tends to overestimate the probability of the observed letters, so the Good–Turing probability estimator is typically used instead. We show that when used to estimate the sequence probability, the Good–Turing estimator is not consistent in this regime. We then introduce a novel sequence probability estimator that is consistent. This estimator also yields consistent estimators for other quantities of interest and a consistent universal classifier.

The energy-distortion function (E(D)) for a network is defined as the minimum total energy required to achieve a target distortion D at the receiver without putting any restrictions on the number of channel uses per source sample. E(D) is studied for a sensor network in which multiple sensors transmit their noisy observations of a Gaussian source to the destination over a Gaussian multiple access channel with perfect channel output feedback. While the optimality of separate source and channel coding is proved for the case of a single sensor, this optimality is shown to fail when there are multiple sensors in the network. A network with two sensors is studied in detail. First a lower bound on E(D) is given. Then, two achievability schemes are proposed: a separation based digital scheme and a Schalkwijk-Kailath (SK) type uncoded scheme. The gap between the lower bound and the upper bound based on separation is shown to be a constant even as the total energy requirement goes to infinity in the low distortion regime. On the other hand, as the distortion requirement is relaxed, the SK based scheme is shown to outperform separation in certain cases, proving that the optimality of source-channel separation does not hold in the multi-sensor setting.

This paper introduces a modeling framework for multivariate regression with agents observing attribute-distributed data, coordinated by a fusion center. Under this model, a prototype algorithm resembling the L2boosting algorithm can effectively minimize the training error, yet it suffers from over-training and slow convergence. A thorough comparison among the agents can speed up convergence of training error and eliminate irrelevant variables, yet it imposes a high demand for data transmission. In this paper, an intelligent agent selection algorithm (based on heuristic functions) is proposed to speed up convergence at low cost of data transmission. The new algorithm can achieve an ensemble estimator of better generalization error with less communication, which is verified by simulation on artificial and real data sets.

We study the minimum energy per bit required for communicating a message to all the destination nodes in a wireless network. The physical layer is modeled as an additive white Gaussian noise channel affected by circularly symmetric fading. The fading coefficients are known at neither transmitters nor receivers. We provide an information-theoretic lower bound on the energy requirement of multicasting in arbitrary wireless networks as the solution of a linear program. We study the broadcast performance of decode-and-forward operating in the non-coherent wideband scenario, and compare it with the lower bounds. For arbitrary networks with k nodes, the energy requirement of decode-and-forward is within a factor of (k-1) of the lower bound regardless of the magnitude of channel gains. We also show that decode-and-forward achieves the minimum energy per bit in networks that can be represented as directed acyclic graphs, thus establishing the exact minimum energy per bit for this class of networks. We also study regular networks where the area is divided into cells, each cell containing at least k and at most k¿ nodes placed arbitrarily within the cell. A path loss model (with path loss exponent ¿ > 2) dictates the channel gains between the nodes. It is shown that the ratio between the upper bound using decode-and-forward based flooding, and the lower bound is at most a constant times (k¿¿+2/k).

We present a novel mechanism for consensus building in sensor networks. The proposed algorithm has three main properties that make it suitable for sensor network learning. First, the proposed algorithm is based on robust nonparametric statistics and thereby needs little prior knowledge about the network and the function that needs to be estimated. Second, the algorithm uses only local information about the network and it communicates only with nearby sensors. Third, the algorithm is completely asynchronous and robust. It does not need to coordinate the sensors to estimate the underlying function and it is not affected if other sensors in the network stop working. Therefore, the proposed algorithm is an ideal candidate for sensor networks deployed in remote and inaccessible areas, which might need to change their objective once they have been set up.

We consider scaling laws for maximal energy efficiency of communicating a message to all the nodes in a random wireless network, as the number of nodes in the network becomes large. Two cases of large wireless networks are studied — dense random networks and constant density (extended) random networks. We first establish an information-theoretic lower bound on the minimum energy per bit for multicasting that holds for arbitrary wireless networks when the channel state information is not available at the transmitters. These lower bounds are then evaluated for two cases of random networks. Upper bounds are also obtained by constructing a simple flooding scheme that requires no information at the receivers about the channel states or the locations and identities of the nodes. The gap between the upper and lower bounds is only a constant factor for dense random networks and differs by a poly-logarithmic factor for extended random networks. Furthermore, the proposed upper and lower bounds hold almost surely in the node locations as the number of nodes approaches infinity.

We present a novel mechanism for consensus building in sensor networks. The proposed algorithm has three main properties that make it suitable for general sensor-network learning. First, the proposed algorithm is based on robust nonparametric statistics and thereby needs little prior knowledge about the network and the function that needs to be estimated. Second, the algorithm uses only local information about the network and it communicates only with nearby sensors. Third, the algorithm is completely asynchronous and robust. It does not need to coordinate the sensors to estimate the underlying function and it is not affected if other sensors in the network stop working. Therefore, the proposed algorithm is an ideal candidate for sensor networks deployed in remote and inaccessible areas, which might need to change their objective once they have been set up.

In this paper, an algorithm is developed for collaboratively training networks of kernel-linear least-squares regression estimators. The algorithm is shown to distributively solve a relaxation of the classical centralized least-squares regression problem. A statistical analysis shows that the generalization error afforded agents by the collaborative training algorithm can be bounded in terms of the relationship between the network topology and the representational capacity of the relevant reproducing kernel Hilbert space. Numerical experiments suggest that the algorithm is effective at reducing noise. The algorithm is relevant to the problem of distributed learning in wireless sensor networks by virtue of its exploitation of local communication. Several new questions for statistical learning theory are proposed.

This paper provides self-contained proof of a theorem relating probabilistic coherence of forecasts to their non-domination by rival forecasts with respect to any proper scoring rule. The theorem recapitulates insights achieved by other investigators, and clarifies the connection of coherence and proper scoring rules to Bregman divergence.

It is proven in this work that it is NP-complete to exhaustively enumerate small error-prone substructures in arbitrary, finite-length low-density parity-check (LDPC) codes. Two error-prone patterns of interest include stopping sets for binary erasure channels (BECs) and trapping sets for general memoryless symmetric channels. Despite the provable hardness of the problem, this work provides an exhaustive enumeration algorithm that is computationally affordable when applied to codes of practical short lengthsnap 500. By exploiting the sparse connectivity of LDPC codes, the stopping sets of sizeles13and the trapping sets of sizeles11can be exhaustively enumerated. The central theorem behind the proposed algorithm is a new provably tightupperboundon the error rates of iterative decoding over BECs. Based on a tree-pruning technique, this upper bound can be iteratively sharpened until its asymptotic order equals that of the error floor. This feature distinguishes the proposed algorithm from existing non-exhaustive ones that correspond to findinglowerboundsof the error floor. The upper bound also provides a worst case performance guarantee that is crucial to optimizing LDPC codes when the target error rate is beyond the reach of Monte Carlo simulation. Numerical experiments on both randomly and algebraically constructed LDPC codes demonstrate the efficiency of the search algorithm and its significant value for finite-length code optimization.

A new universal estimator of divergence is presented for multidimensional continuous densities based on$k$-nearest-neighbor ($k$-NN) distances. Assuming independent and identically distributed (i.i.d.) samples, the new estimator is proved to be asymptotically unbiased and mean-square consistent. In experiments with high-dimensional data, the$k$-NN approach generally exhibits faster convergence than previous algorithms. It is also shown that the speed of convergence of the$k$-NN method can be further improved by an adaptive choice of$k$.

We analyze the delay performance of RTS/CTS-based (Request-To-Send/Clear-To-Send) multi-channel MAC (Medium Access Control) schemes for wireless networks. These schemes usually employ multiple data subchannels for data transmission and one control subchannel to send the RTS/CTS dialogue for channel reservation. Through theoretical analysis and simulations, we show that, in fully-connected networks, such multi-channel MAC schemes suffer longer delays than the corresponding single channel MAC scheme, that puts the RTS/CTS dialogue on the same channel as data packet transmissions. This conclusion holds even when data packets have different priorities and higher priority traffic is sent ahead of lower priority traffic.

We study the problem of target tracking in a sensor network environment. In particular, we consider a target that moves according to a Markov chain, and a tracker that queries sets of sensors to obtain tracking information. We are interested in finding the minimum number of queries per time step such that a target is trackable under three different requirements. First we investigate the case where the tracker is required to know the exact location of the target at each time step. We then relax this requirement and explore the case where the tracker may lose track of the target at a given time step, but it is able to ";catch-up"; at a later time, regaining up-to-date information about the target's track. Finally, we consider the case where tracking information is only known after a delay of d time steps. We provide necessary and sufficient conditions on the number of queries per time step to track in the above three cases. These conditions are stated in terms of the entropy rate of the target's Markov chain.

We consider the problem of estimating the probability of an observed string drawn i.i.d. from an unknown distribution. The key feature of our study is that the length of the observed string is assumed to be of the same order as the size of the underlying alphabet. In this setting, many letters are unseen and the empirical distribution tends to overestimate the probability of the observed letters. To overcome this problem, the traditional approach to probability estimation is to use the classical Good-Turing estimator. We introduce a natural scaling model and use it to show that the Good-Turing sequence probability estimator is not consistent. We then introduce a novel sequence probability estimator that is indeed consistent under the natural scaling model.

This paper focuses on finite-dimensional upper and lower bounds on decodable thresholds of Zopfmand binary low-density parity-check (LDPC) codes, assuming belief propagation decoding on memoryless channels. A concrete framework is presented, admitting systematic searches for new bounds. Two noise measures are considered: the Bhattacharyya noise parameter and the soft bit value for a maximum a posteriori probability (MAP) decoder on the uncoded channel. For ZopfmLDPC codes, an iterative m-dimensional bound is derived for m-ary-input/symmetric-output channels, which gives a sufficient stability condition for ZopfmLDPC codes and is complemented by a matched necessary stability condition introduced herein. Applications to coded modulation and to codes with nonequiprobably distributed codewords are also discussed. For binary codes, two new lower bounds are provided for symmetric channels, including a two-dimensional iterative bound and a one-dimensional noniterative bound, the latter of which is the best known bound that is tight for binary-symmetric channels (BSCs), and is a strict improvement over the existing bound derived by the channel degradation argument. By adopting the reverse channel perspective, upper and lower bounds on the decodable Bhattacharyya noise parameter are derived for nonsymmetric channels, which coincides with the existing bound for symmetric channels

We study throughput scaling in an ad-hoc wireless network where the communication domain is divided into overlapping neighborhoods and n mobile nodes are restricted to move within their assigned neighborhood. In our model, when a node is located in a region not shared with any other neighborhood, it transmits to nodes of its own neighborhood only; when it is in an area that overlaps with another neighborhood, it transmits to nodes of the overlapping neighborhood. Communication between source-destination pairs is subject to interference from other nodes. By adopting a deterministic approach, we obtain an achievable throughput which is a function of properties of the node locations and neighborhood dimensions. As special cases of our neighborhood model, the results of P. Gupta and P.R. Kumar (2000) and M. Grossglauser and D. Tse (2002) can be recovered. We then study the case of random placement of nodes with nalphaneighborhoods, where 0 les alpha les 1, and achieve a throughput of Omega (n1-alpha/2). Hence our model captures every order of growth for the throughput, encompassing the results from both P. Gupta and P.R. Kumar (2000) and M. Grossglauser and D. Tse (2002) as extreme situations

In this paper, computational aspects of the panel aggregation problem are addressed. Motivated primarily by applications of risk assessment, an algorithm is developed for fusing large corpora of internally incoherent probability assessments. The algorithm is characterized by a provable performance guarantee, and is demonstrated to be orders of magnitude faster than existing tools when tested on several real-world data-sets. In addition, unexpected connections between research in risk assessment and wireless sensor networks are exposed, as several key ideas are illustrated to be useful in both fields

We study the convergence and consistency of boosting algorithms for classification. The standard method, as the sample size increases say from m to m+1, is to re-initialize the boosting algorithm with an arbitrary prediction rule. In contrast to this "batch" approach, we propose a boosting procedure that is recursive in the sense that for sample size m+1, the algorithm is re-started with the composite classifier that was obtained for sample size m at a specific point, the linking point. We adopt the regularization technique of early stopping, which consists in stopping the procedure based on the 1-norm of the composite classifier. We prove that such recursive boosting methods achieve consistency provided certain stopping and linking points criteria are met. We show that these conditions can be satisfied for widely used loss functions

We consider the problem of estimating the total probability of all symbols that appear with a given frequency in a string of i.i.d. random variables with unknown distribution. We focus on the regime in which the block length is large yet no symbol appears frequently in the string. This is accomplished by allowing the distribution to change with the block length. Under a natural convergence assumption on the sequence of underlying distributions, we show that the total probabilities converge to a deterministic limit, which we characterize. We then show that the good-turing total probability estimator is strongly consistent

Assuming iterative decoding for binary erasure channels (BECs), a novel tree-based technique for upper bounding the bit error rates (BERs) of arbitrary, finite low-density parity-check (LDPC) codes is provided and the resulting bound can be evaluated for all operating erasure probabilities, including both the waterfall and the error floor regions. This upper bound can also be viewed as a narrowing search of stopping sets, which is an approach different from the stopping set enumeration used for lower bounding the error floor. When combined with optimal leaf-finding modules, this upper bound is guaranteed to be tight in terms of the asymptotic order. The Boolean framework proposed herein further admits a composite search for even tighter results. For comparison, a refinement of the algorithm is capable of exhausting all stopping sets of size les 13 for irregular LDPC codes of length n ap 500, which requires (13500) ap 1.67 times 1025trials if a brute force approach is taken. These experiments indicate that this upper bound can be used both as an analytical tool and as a deterministic worst-performance (error floor) guarantee, the latter of which is crucial to optimizing LDPC codes for extremely low BER applications, e.g., optical/satellite communications

A method for divergence estimation between multidimensional distributions based on nearest neighbor distances is proposed. Given i.i.d. samples, both the bias and the variance of this estimator are proven to vanish as sample sizes go to infinity. In experiments on high-dimensional data, the nearest neighbor approach generally exhibits faster convergence compared to previous algorithms based on partitioning

This paper addresses the problem of distributed learning under communication constraints, motivated by distributed signal processing in wireless sensor networks and data mining with distributed databases. After formalizing a general model for distributed learning, an algorithm for collaboratively training regularized kernel least-squares regression estimators is derived. Noting that the algorithm can be viewed as an application of successive orthogonal projection algorithms, its convergence properties are investigated and the statistical behavior of the estimator is discussed in a simplified theoretical setting.

This paper studies universal estimation of divergence from the realizations of two unknown finite-alphabet sources. Two algorithms that borrow techniques from data compression are presented. The first divergence estimator applies the Burrows–Wheeler block sorting transform to the concatenation of the two realizations; consistency of this estimator is shown for all finite-memory sources. The second divergence estimator is based on the Context Tree Weighting method; consistency is shown for all sources whose memory length does not exceed a known bound. Experimental results show that both algorithms perform similarly and outperform string-matching and plug-in methods.

This paper proposes a new algorithm based on the Context-Tree Weighting (CTW) method for universal compression of a finite-alphabet sequence x1nwith side information y1navailable to both the encoder and decoder. We prove that with probability one the compression ratio converges to the conditional entropy rate for jointly stationary ergodic sources. Experimental results with Markov chains and English texts show the effectiveness of the algorithm

Motivated by sensor networks and other distributed settings, several models for distributed learning are presented. The models differ from classical works in statistical pattern recognition by allocating observations of an independent and identically distributed (i.i.d.) sampling process among members of a network of simple learning agents. The agents are limited in their ability to communicate to a central fusion center and thus, the amount of information available for use in classification or regression is constrained. For several basic communication models in both the binary classification and regression frameworks, we question the existence of agent decision rules and fusion rules that result in a universally consistent ensemble; the answers to this question present new issues to consider with regard to universal consistency. This paper addresses the issue of whether or not the guarantees provided by Stone's theorem in centralized environments hold in distributed settings.

The distributed nature of wireless sensor networks illustrates well classical engineering tradeoffs: how to minimize communication (and possibly computation) cost, and thus energy dissipation, while maintaining acceptable performance levels in estimation and inference applications. We study a simple sensor network under dependent Gaussian noise and develop strategies for parameter estimation in a variety of communication scenarios. From an energy point of view, sending all data to a fusion center is the most costly, but leads to optimum performance results. Processing data at each sensor and sending parameter estimates and associated quality measures is a reasonable communication saving procedure and yet, in some cases, may lead to performance equivalent to sending all data to the fusion center. A sequential procedure is most parsimonious in terms of communication cost and especially effective in large wireless sensor networks. We explore those conditions for which little, or no loss in performance is encountered with this sequential procedure. Specifically, we provide analytical expressions for the maximum likelihood estimator under "geometric" dependent noise. We show, by means of analysis and simulations, that the performance is only marginally degraded when the noise is assumed to be independent.

This paper proposes a new algorithm based on the context-tree weighting method for universal compression of a finite-alphabet sequence x/sub 1//sup n/ with side information y/sub 1//sup n/ available to both the encoder and decoder. We prove that with probability one the compression ratio converges to the conditional entropy rate for jointly stationary ergodic sources. Experimental results with Markov chains and English texts show the effectiveness of the algorithm.

We study throughput scaling of an ad-hoc network where the nodes are restricted to move on vertical or horizontal lines on a square. A constant throughput is asymptotically achievable by the proposed scheduling algorithm, using in certain cases more than two hops strategies, without requiring each node to become the nearest neighbor of every other node. Hence the throughput result for one-dimensional mobility obtained by Diggavi et al. in Proc. IEEE ISIT, 2002 still holds under stricter mobility constraints

We consider the broadcast-relay channel: a broadcast channel where receivers are permitted to assist in the distribution of data to other receivers by relaying. We extend the previous results and demonstrate that effective transmission strategies can be derived by combining well-known techniques for broadcast and relay channels. Additionally, we derive new outer bounds to capacity regions

We present a universal estimator of the divergence D(PparQ) for two arbitrary continuous distributions P and Q satisfying certain regularity conditions. This algorithm, which observes i.i.d. samples from both P and Q, is based on the estimation of the Radon-Nikodym derivative dP/dQ via a data-dependent partition of the observation space. Strong convergence of this estimator is proved with an empirically equivalent segmentation of the space. This basic estimator is further improved by adaptive partitioning schemes and by bias correction. In the simulations, we compare our estimators with the plug-in estimator and estimators based on other partitioning approaches. Experimental results show that our methods achieve the best convergence performance in most of the tested cases

We study the statistical convergence and consistency of regularized Boosting methods, where the samples are not independent and identi- cally distributed (i.i.d.) but come from empirical processes of stationary β-mixing sequences. Utilizing a technique that constructs a sequence of independent blocks close in distribution to the original samples, we prove the consistency of the composite classifiers resulting from a regulariza- tion achieved by restricting the 1-norm of the base classifiers’ weights. When compared to the i.i.d. case, the nature of sampling manifests in the consistency result only through generalization of the original condition on the growth of the regularization parameter.

Density evolution (DE) is one of the most powerful analytical tools for low-density parity-check (LDPC) codes and graph codes with message passing decoding algorithms. With channel symmetry as one of its fundamental assumptions, density evolution has been widely and successfully applied to different channels, including binary erasure channels (BECs), binary symmetric channels (BSCs), binary additive white Gaussian noise (BiAWGN) channels, etc. This paper generalizes density evolution for asymmetric memoryless channels, which in turn broadens the applications to general memoryless channels, e.g., z-channels, composite white Gaussian noise channels, etc. The central theorem underpinning this generalization is the convergence to perfect projection for any fixed-size supporting tree. A new iterative formula of the same complexity is then presented and the necessary theorems for the performance concentration theorems are developed. Several properties of the new density evolution method are explored, including stability results for general asymmetric memoryless channels. Simulations, code optimizations, and possible new applications suggested by this new density evolution method are also provided. This result is also used to prove the typicality of linear LDPC codes among the coset code ensemble when the minimum check node degree is sufficiently large. It is shown that the convergence to perfect projection is essential to the belief propagation (BP) algorithm even when only symmetric channels are considered. Hence, the proof of the convergence to perfect projection serves also as a completion of the theory of classical density evolution for symmetric memoryless channels.

We present a universal estimator of the divergence D(P/spl par/Q) for two arbitrary continuous distributions P and Q satisfying certain regularity conditions. This algorithm, which observes independent and identically distributed (i.i.d.) samples from both P and Q, is based on the estimation of the Radon-Nikodym derivative dP/dQ via a data-dependent partition of the observation space. Strong convergence of this estimator is proved with an empirically equivalent segmentation of the space. This basic estimator is further improved by adaptive partitioning schemes and by bias correction. The application of the algorithms to data with memory is also investigated. In the simulations, we compare our estimators with the direct plug-in estimator and estimators based on other partitioning approaches. Experimental results show that our methods achieve the best convergence performance in most of the tested cases.

In this paper, an alternative model of restricted mobility by considering nodes confined to overlapping neighborhoods is presented. The throughput per source-destination (S-D) pair can be kept constant as the number of nodes increases. For arbitrary assignment of nodes to neighborhoods, achievable orders of throughput per S-D pair as a function of properties of the node locations and neighborhood dimensions is obtained. The succession of neighborhoods that a packet crosses from sender to destination is predetermined by a routing algorithm resulting from establishing a correspondence between the traffic pattern through the neighborhoods in the wireless network and a 2-D mesh network of processing units.

Motivated by sensor networks and traditional methods of statistical pattern recognition, a model for distributed learning is formulated. The model is in line with learning models considered in the context of Stone-type classifiers, but differs in the dependency structure of the sampling process; questions of universal consistency are addressed.

In this paper, we analyze the effect of scaling laws in random heterogeneous networks. This paper describes a square grid with shortcuts and a scaling law with wired shortcuts

In this correspondence, we present a new universal entropy estimator for stationary ergodic sources, prove almost sure convergence, and establish an upper bound on the convergence rate for finite-alphabet finite memory sources. The algorithm is motivated by data compression using the Burrows-Wheeler block sorting transform (BWT). By exploiting the property that the BWT output sequence is close to a piecewise stationary memoryless source, we can segment the output sequence and estimate probabilities in each segment. Experimental results show that our algorithm outperforms Lempel-Ziv (LZ) string-matching-based algorithms.

We derive upper bounds on the transport capacity of wireless networks. The bounds obtained are solely dependent on the geographic locations and power constraints of the nodes. As a result of this derivation, we are able to conclude the optimality, in the sense of scaling of transport capacity with the number of nodes, of a multihop communication strategy for a class of network topologies.

We address the problem of how throughput in a wireless network scales as the number of users grows. Following the model of Gupta and Kumar, we consider n identical nodes placed in a fixed area. Pairs of transmitters and receivers wish to communicate but are subject to interference from other nodes. Throughput is measured in bit-meters per second. We provide a very elementary deterministic approach that gives achievability results in terms of three key properties of the node locations. As a special case, we obtain /spl Omega/(/spl radic/n) throughput for a general class of network configurations in a fixed area. Results for random node locations in a fixed area can also be derived as special cases of the general result by verifying the growth rate of three parameters. For example, as a simple corollary of our result we obtain a stronger (almost sure) version of the /spl radic/n//spl radic/(logn) throughput for random node locations in a fixed area obtained by Gupta and Kumar. Results for some other interesting non-independent and identically distributed (i.i.d.) node distributions are also provided.

We determine the capacity region of a degraded Gaussian relay channel with multiple relay stages. This is done by building an inductive argument based on the single-relay capacity theorem of Cover and El Gamal. For an arbitrary distribution of noise powers, we derive the optimal power distribution strategy among the transmitter and the relays and the best possible improvement in signal-to-noise ratio (SNR) that can be achieved from using a given number of relays. The time-division multiplexing operation of the relay channel in the wideband regime is analyzed and it is shown that time division does not achieve minimum energy per bit.

Given a set of synchronized video sequences of a dynamic scene taken by different cameras, we address the problem of creating a virtual video of the scene from a novel viewpoint. A key aspect of our algorithm is a method for recursively propagating dense and physically accurate correspondences between the two video sources. By exploiting temporal continuity and suitably constraining the correspondences, we provide an efficient framework for synthesizing realistic virtual video. The stability of the propagation algorithm is analyzed, and experimental results are presented.

The Burrows Wheeler transform (1994) is a reversible sequence transformation used in a variety of practical lossless source-coding algorithms. In each, the BWT is followed by a lossless source code that attempts to exploit the natural ordering of the BWT coefficients. BWT-based compression schemes are widely touted as low-complexity algorithms giving lossless coding rates better than those of the Ziv-Lempel codes (commonly known as LZ'77 and LZ'78) and almost as good as those achieved by prediction by partial matching (PPM) algorithms. To date, the coding performance claims have been made primarily on the basis of experimental results. This work gives a theoretical evaluation of BWT-based coding. The main results of this theoretical evaluation include: (1) statistical characterizations of the BWT output on both finite strings and sequences of length n /spl rarr/ /spl infin/, (2) a variety of very simple new techniques for BWT-based lossless source coding, and (3) proofs of the universality and bounds on the rates of convergence of both new and existing BWT-based codes for finite-memory and stationary ergodic sources. The end result is a theoretical justification and validation of the experimentally derived conclusions: BWT-based lossless source codes achieve universal lossless coding performance that converges to the optimal coding performance more quickly than the rate of convergence observed in Ziv-Lempel style codes and, for some BWT-based codes, within a constant factor of the optimal rate of convergence for finite-memory sources.

Let X/sub 1/, X/sub 2/,... be an arbitrary random process taking values in a totally bounded subset of a separable metric space. Associated with X/sub i/ we observe Y/sub i/ drawn from an unknown conditional distribution F(y|X/sub i/=x) with continuous regression function m(x)=E[Y|X=x]. The problem of interest is to estimate Y/sub n/ based on X/sub n/ and the data {(X/sub i/, Y/sub i/)}/sub i=1//sup n-1/. We construct appropriate data-dependent nearest neighbor and kernel estimators and show, with a very elementary proof, that these are consistent for every process X/sub 1/, X/sub 2/,.

Principal curves, like principal components, are a tool used in multivariate analysis for ends like feature extraction. Defined in their original form, principal curves need not exist for general distributions. The existence of principal curves with bounded length for any distribution that satisfies some minimal regularity conditions has been shown. We define principal curves with bounded turn, show that they exist, and present a learning algorithm for them. Principal components are a special case of such curves when the turn is zero.

We demonstrate that in some situations, perceptual quality can be maintained using an approach based on synthesizing "virtual" images of a scene that match frames from a source video clip. We use this algorithm for interpolation of video frames in the time domain, using a small amount of information to construct an approximation of the original video. Our algorithm is well-suited for the limitations in bandwidth and complexity characteristic of wireless multimedia channels. Since the approach is based on estimating functions of the underlying camera motion parameters, it can capture relationships between image correspondences that extend across many (perhaps hundreds) of video frames. Each interpolated image can be rendered using only a few tens of bytes of side information, and the rendering process itself has low computational requirements. We present experimental results to demonstrate that for certain types of video, our algorithm can give significant perceptual improvement over MPEG-4 coded video at the same low bit rate.

A universal variable-to-fixed length algorithm for binary memoryless sources which converges to the entropy of the source at the optimal rate is known. We study the problem of universal variable-to-fixed length coding for the class of Markov sources with finite alphabets. We give an upper bound on the performance of the code for large dictionary sizes and show that the code is optimal in the sense that no codes exist that have better asymptotic performance. The optimal redundancy is shown to be H log log M/log M where H is the entropy rate of the source and M is the code size. This result is analogous to Rissanen's (1984) result for fixed-to-variable length codes. We investigate the performance of a variable-to-fixed coding method which does not need to store the dictionaries, either at the coder or the decoder. We also consider the performance of both these source codes on individual sequences. For individual sequences we bound the performance in terms of the best code length achievable by a class of coders. All the codes that we consider are prefix-free and complete.

Describes MW (Master-Worker) - a software framework that allows users to quickly and easily parallelize scientific computations using the master-worker paradigm on the Computational Grid. MW provides both a "top-level" interface to application software and a "bottom-level" interface to existing Grid computing toolkits. Both interfaces are briefly described. We conclude with a case study, where the necessary Grid services are provided by the Condor high-throughput computing system, and the MW-enabled application code is used to solve a combinatorial optimization problem of unprecedented complexity.

This paper is concerned with the efficient temporal propagation of correspondences between frames of two video sequences, an integral component of many video processing tasks. The main contribution is a framework for the recursive propagation of these correspondences. The propagation consists of a time update step and a measurement update step. The time update depends only on the dynamics of the rotating source cameras, while the measurement update can be tailored to any member of a general class of image correspondence algorithms. Using these results, the correspondence between points of each frame pair can be propagated and updated in a fraction of the time required to estimate correspondences anew at every frame. We discuss an application of the recursive correspondence propagation framework to the creation of virtual video. Previous virtual view algorithms have been used to generate synthetic video of a static scene, in which objects seem frozen in time. In contrast, the algorithms described here allow the creation of "true" virtual video, in the sense that the synthetic video evolves dynamically along with the scene. While virtual video is our motivating application, the recursive correspondence propagation framework applies to any two-camera video application in which correspondence is difficult and prohibitively time-consuming to estimate by processing frame pairs independently.

As digital video becomes more pervasive, efficient ways of searching and annotating video according to content will be increasingly important. Such tasks arise, for example, in the management of digital video libraries for content-based retrieval and browsing. We develop tools based on camera motion for analyzing and annotating a class of structured video using the low-level information available directly from MPEG-compressed video. In particular, we show that in certain structured settings, it is possible to obtain reliable estimates of camera motion by directly processing data easily obtained from the MPEG format. Working directly with the compressed video greatly reduces the processing time and enhances storage efficiency. As an illustration of this idea, we have developed a simple basketball annotation system which combines the low-level information extracted from an MPEG stream with the prior knowledge of basketball structure to provide high-level content analysis, annotation, and browsing for events such as wide-angle and close-up views, fast breaks, probable shots at the basket, etc. The methods used in this example should also be useful in the analysis of high-level content of structured video in other domains.

We investigate the performance of the Lempel-Ziv (1978) incremental parsing scheme on nonstationary sources. We show that it achieves the best rate achievable by a finite-state block coder for the nonstationary source. We also show a similar result for a lossy coding scheme given by Yang and Kieffer (see ibid., vol.42, p.239-45, 1996) which uses a Lempel-Ziv scheme to perform lossy coding.

We consider the problem of determining when a given source can be used to approximate the output due to any input to a given channel. We provide achievability and converse results for a general source and channel. For the special case of a full-rank discrete memoryless channel we give a stronger converse result than we can give for a general channel.

The usefulness of a video database relies on whether the video of interest can be easily located. To allow exploring, browsing, and retrieving videos according to their visual content, efficient techniques for evaluating the visual similarity between different video clips are necessary. We present a framework for measuring video similarity across different resolutions-both spatial and temporal. In particular, the video clips to be compared can be properly aligned through the use of suitable weighting functions and alignment constraints. Dynamic programming techniques are employed to obtain the video similarity measure with a reasonable computational cost. An application to searching MPEG compressed video by example is presented to demonstrate the potential use of the proposed video similarity measure.

We develop deterministic necessary and sufficient conditions on individual noise sequences of a stochastic approximation algorithm for the error of the iterates to converge at a given rate. Specifically, suppose {/spl rho//sub n/} is a given positive sequence converging monotonically to zero. Consider a stochastic approximation algorithm x/sub n+1/=x/sub n/-a/sub n/(A/sub n/x/sub n/-b/sub n/)+a/sub n/e/sub n/, where {x/sub n/} is the iterate sequence, {a/sub n/} is the step size sequence, {e/sub n/} is the noise sequence, and x* is the desired zero of the function f(x)=Ax-b. Then, under appropriate assumptions, we show that x/sub n/-x*=o(/spl rho//sub n/) if and only if the sequence {e/sub n/} satisfies one of five equivalent conditions. These conditions are based on well-known formulas for noise sequences: Kushner and Clark's (1978) condition, Chen's (see Proc. IFAC World Congr., p.375-80, 1996) condition, Kulkarni and Horn's (see IEEE Trails Automat. Contr., vol.41, p.419-24, 1996) condition, a decomposition condition, and a weighted averaging condition. Our necessary and sufficient condition on {e/sub n/} to achieve a convergence rate of {/spl rho//sub n/} is basically that the sequence {e/sub n///spl rho//sub n/} satisfies any one of the above five well-known conditions. We provide examples to illustrate our result. In particular, we easily recover the familiar result that if a/sub n/=a/n and {e/sub n/} is a martingale difference process with bounded variance, then x/sub n/-x*=o(n/sup -1/2/(log(n))/sup /spl beta//) for any /spl beta/>1/2.

Reliable object recognition is an essential part of most visual systems. Model-based approaches to object recognition use a database (a library) of modeled objects; for a given set of sensed data, the problem of model-based recognition is to identify and locate the objects from the library that are present in the data. We show that the complexity of model-based recognition depends very heavily on the number of object models in the library even if each object is modeled by a small number of discrete features. Specifically, deciding whether a discrete set of sensed data can be interpreted as transformed object models from a given library is NP-complete if the transformation is any combination of translation, rotation, scaling, and perspective projection. This suggests that efficient algorithms for model-based recognition must use additional structure to avoid the inherent computational difficulties. © 1998 John Wiley & Sons, Inc.

Classical and recent results in statistical pattern recognition and learning theory are reviewed in a two-class pattern classification setting. This basic model best illustrates intuition and analysis techniques while still containing the essential features and serving as a prototype for many applications. Topics discussed include nearest neighbor, kernel, and histogram methods, Vapnik-Chervonenkis theory, and neural networks. The presentation and the large (though nonexhaustive) list of references is geared to provide a useful overview of this field for both specialists and nonspecialists.

This paper considers estimation of a univariate density from an individual numerical sequence. It is assumed that (1) the limiting relative frequencies of the numerical sequence are governed by an unknown density, and (2) there is a known upper bound for the variation of the density on an increasing sequence of intervals. A simple estimation scheme is proposed, and is shown to be L/sub 1/ consistent when (1) and (2) apply. In addition, it is shown that there is no consistent estimation scheme for the set of individual sequences satisfying only condition (1).

A random number generator generates fair coin flips by processing deterministically an arbitrary source of nonideal randomness. An optimal random number generator generates asymptotically fair coin flips from a stationary ergodic source at a rate of bits per source symbol equal to the entropy rate of the source. Since optimal noiseless data compression codes produce incompressible outputs, it is natural to investigate their capabilities as optimal random number generators. We show under general conditions that optimal variable-length source codes asymptotically achieve optimal variable-length random bit generation in a rather strong sense. In particular, we show in what sense the Lempel-Ziv (1978) algorithm can be considered an optimal universal random bit generator from arbitrary stationary ergodic random sources with unknown distributions.

We find tight upper and lower bounds on the growth rate for the covering numbers of functions of bounded variation in the /spl Lscr//sub 1/ metric in terms of all the relevant constants. We also find upper and lower bounds on covering numbers for general function classes over the family of /spl Lscr//sub 1/(dP) metrics in terms of a scale-sensitive combinatorial dimension of the function class.

In this paper, uniformly consistent estimation (learnability) of decision rules for pattern classification under a family of probability measures is investigated. In particular, it is shown that uniform boundedness of the metric entropy of the class of decision rules is both necessary and sufficient for learnability under each of two conditions: (i) the family of probability measures is totally bounded, with respect to the total variation metric, and (ii) the family of probability measures contains an interior point, when equipped with the same metric. In particular, this shows that insofar as uniform consistency is concerned, when the family of distributions contains a total variation neighborhood, nothing is gained by this knowledge about the distribution. Then two sufficient conditions for learnability are presented. Specifically, it is shown that learnability with respect to each of a finite collection of families of probability measures implies learnability with respect to their union; also, learnability with respect to each of a finite number of measures implies learnability with respect to the convex hull of the corresponding families of uniformly absolutely continuous probability measures.

This paper examines learning problems in which the target function is allowed to change. The learner sees a sequence of random examples, labelled according to a sequence of functions, and must provide an accurate estimate of the target function sequence. We consider a variety of restrictions on how the target function is allowed to change, including infrequent but arbitrary changes, sequences that correspond to slow walks on a graph whose nodes are functions, and changes that are small on average, as measured by the probability of disagreements between consecutive functions. We first study estimation, in which the learner sees a batch of examples and is then required to give an accurate estimate of the function sequence. Our results provide bounds on the sample complexity and allowable drift rate for these problems. We also study prediction, in which the learner must produce online a hypothesis after each labelled example and the average misclassification probability over this hypothes...

Selecting image features whose correspondences can be accurately established between images is a key step in many image processing problems, such as camera and object motion estimation, 3D structure reconstruction, and image registration. In this paper, we present a new method of selecting good features for estimating motion from images. Our approach is different from other existing approaches in that we formulate feature tracking as a signal parameter estimation problem, give a quantitative measure of feature quality in terms of how accurately the feature can be tracked, and can adaptively select features with different shapes and sizes which depend on the local variations of the images. Through the analysis of this feature quality measure, we can characterize the basic properties that allow a feature to be well tracked. Some experimental results are shown to demonstrate the advantages and robustness of the proposed method.

We consider tomographic image reconstruction from a limited number of noisy projections. An efficient algorithm based on maximum likelihood estimation (MLE) is developed to reconstruct images of multiple discs with unknown locations and radii. The algorithm is successfully applied to images with signal-to-noise ratio (SNR) as low as 0 dB, using as few as 16 projections, and containing as many as twelve discs with widely varying radii. Experimental results show that our approach significantly outperforms conventional convolution back projection. The algorithm is successfully extended to the multiple ellipse case.

Synchronizing codewords (SCs) have been previously studied as a means to stop error propagation in variable-length codes. However, SCs retain one disadvantage: the symbols after the SC may be put in the wrong positions since the number of decoded symbols before the SC can be different from the original number due to channel errors. Thus we propose the idea of extended synchronizing codewords (ESCs) which can overcome the drawback of SCs. After the decoder receives an ESC, the decoder correctly knows it is in synchronization, regardless of the preceding slippage. We derive some of the essential properties of ESCs and provide several upper bounds on the amount of overhead needed in designing a code with an ESC.

Two algorithms for two-dimensional convex shape reconstruction from noisy ray probe measurements are developed and compared. Given a coordinate system located within the object, the data consists of a finite set of angles together with the corresponding radial distances to the boundary corrupted by additive noise. We first characterize when such data is consistent with some convex shape. The algorithms estimate the target shape by finding the consistent set of probe measurements that is closest to the original noisy data. A direct formulation leads to a quadratic minimization problem with nonlinear constraints. By applying a simple transformation, an alternative algorithm is developed that trades off performance for computational simplicity as it requires quadratic minimization with linear constraints. Both algorithms are successfully applied to a variety of shapes with substantial noise.

We derive a six parameter system to estimate and compensate the effects of camera motion-zoom, pan, tilt and swing. As compared to other existing methods, this model describes more precisely the effect of different kinds of camera motions. A recursive least-squares estimator has been used to solve for the motion parameters. Experiments suggest that our algorithm converges to satisfactory results when about 10 pairs of corresponding pairs between two image frames are available.

Rates of convergence for nearest neighbor estimation are established in a general framework in terms of metric covering numbers of the underlying space. The first result is to find explicit finite sample upper bounds for the classical independent and identically distributed (i.i.d.) random sampling problem in a separable metric space setting. The convergence rate is a function of the covering numbers of the support of the distribution. For example, for bounded subsets of R/sup r/, the convergence rate is O(1/n/sup 2/r/). The main result is to extend the problem to allow samples drawn from a completely arbitrary random process in a separable metric space and to examine the performance in terms of the individual sample sequences. The authors show that for every sequence of samples the asymptotic time-average of nearest neighbor risks equals twice the time-average of the conditional Bayes risks of the sequence. Finite sample upper bounds under arbitrary sampling are again obtained in terms of the covering numbers of the underlying space. In particular, for bounded subsets of R/sup r/ the convergence rate of the time-averaged risk is O(1/n/sup 2/r/). The authors then establish a consistency result for k/sub n/-nearest neighbor estimation under arbitrary sampling and prove a convergence rate matching established rates for i.i.d. sampling. Finally, they show how their arbitrary sampling results lead to some classical i.i.d. sampling results and in fact extend them to stationary sampling. The framework and results are quite general while the proof techniques are surprisingly elementary.>

We present a new algorithm to find motion vectors of B frames (bidirectional frames) in the MPEG encoding scheme which significantly reduces the computations required compared with the conventional method. We interpolate the motion vectors for each macro block in the B frame using the motion vector of the next P frame (predictive frame) and do a small search to final a better matching. The number of computations is reduced by factor of 5/spl sim/20, but the matching stays comparable with that of exhaustive search.>

A multiresolution chain coding scheme for contours is developed based on 4-connected chain codes at progressively more refined grid sizes. By taking advantage of a specific set of possible paths a contour can travel, the algorithm presented generates a multiresolution representation of a contour, with the final refinement representing the original contour with the same accuracy as conventional 4-connected chain coding. The technique described requires only a small overhead in comparison to 4-connected chain coding, so has the advantage of providing a hierarchical representation of a contour at little cost. The generation of a multi-scale representation allows the algorithm to perform well in the presence of storage or transmission limitations, as only a fraction of the data is required to obtain a detailed representation of the entire contour.>

Considers the problem of tomographic image reconstruction from a limited number of noisy projections. Maximum likelihood estimation is used to reconstruct an image of multiple discs with unknown locations and radii. An efficient algorithm is developed to tackle the computation. The algorithm is successfully applied to images containing as many as twelve discs with widely varying radii and with signal-to-noise ratio as low as 0 dB, and the number of projections used is as few as 16. Experimental results show that the approach significantly outperforms the conventional convolution back-projection method.>

Considers the problem of computing the length of a curve from digitized versions of the curve using parallel computation. The authors' aim is to study the inherent parallel computational complexity of this problem as a function of the digitization level. Precise formulations for the digitization, the parallel computation, and notions of local and nonlocal computations are given. It is shown that length cannot be computed locally from digitizations on rectangular tessellations. However, for a random tessellation and appropriate deterministic ones, the authors show that the length of straight line segments can be computed locally. Implications of the authors' results for a method for image segmentation and a number of open problems are discussed.>

The authors prove by means of a counterexample that it is not sufficient, for probably approximately correct (PAC) learning under a class of distributions, to have a uniform bound on the metric entropy of the class of concepts to be learned. This settles a conjecture of Benedek and Itai (1991).>

The following problem arises in many applications involving classification, identification, and inference. There is a set of objects X, and a particular x /spl isin/ X is chosen (unknown to us). Based on information obtained about x in a sequential manner, one wishes to decide whether x belongs to one class of objects A/sub 0/ or a different class of objects A/sub 1/. The authors study a general paradigm applicable to a broad range of problems of this type, which they refer to as problems of class identification or discernibility. They consider various types of information sequences, and various success criteria including discernibility in the limit, discernibility with a stopping criterion, uniform discernibility, and discernibility in the Cesaro sense. They consider decision rules both with and without memory. Necessary and sufficient conditions for discernibility are provided for each case in terms of separability conditions on the sets A/sub 0/ and A/sub 1/. They then show that for any sets A/sub 0/ and A/sub 1/, various types of separability can be achieved by allowing failure on appropriate sets of small measure. Applications to problems in language identification, system identification, and discrete geometry are discussed.>

We consider the problem of classifying an unknown concept into one of two subclasses of concepts. Specifically, if C is a concept class and Co and Cl are two disjoint subsets of C, given an unknown c E Co U Cl we wish to decide whether c c Co or c E Cl based on a set of random examples. We consider both uniform and non-uniform probably correct classification for which the number of samples is or is not required to be independent of c, respectively. For both cases, we obtain necessary and sufficient conditions on Co and Cl that allow probably correct classification. The conditions obtained are in terms of separability and/or coverability conditions on the classes Co and Cl. Furthermore, in the non-uniform case we show that this is equivalent to classification in the limit. Several examples of the applicability of our results are also provided.

We consider the problem of learning of an arbitrary functionselected from the non-smooth class of functions that are of bounded variation.Bounds on the prediction errors resulting from sequential type algorithms are achieved for various scenarios.It is shown that for any algorithm there exists a sequence of samples and a function that results in a unit error at each step.If the samples are i.i.d.uniform then there exists an algorithm such that for any function of bounded variation the expected error is less than O(log n/n).Furthermore, for any algorithm there exists a function such that the expected error is greater than 0(1/n).We then introduce a mixed sampling setting in which an adversary can choose points but the actual samples are drawn uniformly from a 6 neighborhood of his selection.We show that there exists an algorithm such that for any function of bounded variation and for any sequence of adversarychosen points the expected cumulative error is less than 0(+ log n).Finally, results are derived for the uniform sampling case with noisy observations.We show that the order of growth of the expected error with noise sequences in Z1 remains unaltered.However, for i.i.d.zero mean and finite variance noise there exists an algorithm with expected error less than 0(1/n*), and for i.i.d.Gaussian noise any algorithm has expected error of at least o(l/@).

An extension of the standard probably approximately correct (PAC) learning model that allows the use of generalized samples is introduced. A generalized sample is viewed as a pair consisting of a functional on the concept class together with the value obtained by the functional operating on the unknown concept. It appears that this model can be applied to a number of problems in signal processing and geometric reconstruction to provide sample size bounds under a PAC criterion. A specific application of the generalized model to a problem of curve reconstruction is considered, and some connections with a result from stochastic geometry are discussed.>

In this paper, we introduce an extension of the standard PAC learning model which allows the use of generalized samples. We view a generalized sample as a pair consisting of a functional on the concept class together with the value obtained by the functional operating on the unknown concept. It appears that this model can be applied to a number of problems in signal processing and geometric reconstruction to provide sample size bounds under a PAC criterion. We consider a specific application of the model to a problem of curve reconstruction, and discuss some connections with a result from stochastic geometry.