For many inverse problems, the data on which the solution is based is acquired sequentially. We present an approach to the solution of such inverse problems where a sensor can be directed (or otherwise reconfigured on the fly) to acquire a particular measurement. An example problem is magnetic resonance image reconstruction. We use an estimate of mutual information derived from an empirical conditional distribution provided by a generative model to guide our measurement acquisition given measurements acquired so far. The conditionally generated data is a set of samples which are representative of the plausible solutions that satisfy the acquired measurements. We present experiments on toy and real world data sets. We focus on image data but we demonstrate that the method is applicable to a broader class of problems. We also show how a learned model such as a deep neural network can be leveraged to allow generalisation to unseen data. Our informed adaptive sensing method outperforms random sampling, variance based sampling, sparsity based methods, and compressed sensing.

Network data constantly evolves with new network applications and protocols. There is a need for robust techniques to detect anomalous behaviour. Offline models trained with static data lose validity when new variants of traffic emerge. They require retraining but the need for ground truth and lengthy training times make this task challenging. Meanwhile, online models which detect outliers in streaming data are susceptible to the curse of dimensionality and natural variability. Today’s anomalies may be tomorrow’s new traffic and existing methods do not provide a way to differentiate between them. We propose a framework that makes the most of both approaches: an offline deep learning model extracts features of normal traffic and provides a bias for an online outlier detection model to select data for training. The online model retains its previously learnt knowledge and retrains itself with new data. Online thresholds are updated in a drifting manner and the Mann-Whitney U test is incorporated to prevent inaccurate updates. We perform analysis on the scores, develop heuristics to detect new traffic and evaluate using three deep learning models and four outlier detection methods on the UNSW-NB15 and CTU-13 datasets. The framework improves upon any individual offline or online models in isolation.

The challenge of anomaly detection is to obtain an accurate understanding of expected behaviour which is intensified when the data are distributed heterogeneously. Transmitting raw data to a central site incurs high communication overhead and raises privacy issues. The concept of Edge AI allows computation to be performed at the edge site allowing for quick decision making in mission critical scenarios such as self-driving cars. A model is learnt locally and its parameters are transmitted and aggregated. However, existing methods of aggregation do not account for variance and heterogeneous distribution of data. They also do not consider edge constraints such as limited computational, memory and communication capabilities of edge devices. In this work, a fully Bayesian approach is employed by means of a Bayesian Random Vector Functional Link AutoEncoder being incorporated with Expectation Propagation for distributed training. Our anomaly detection system operates without any transmission of raw data, is robust under inhomogeneous network densities and under uneven and biased data distributions. It allows for asynchronous updates to converge in a few iterations and is a relatively simple neural network addressing edge constraints without compromising on performance as compared to existing more complex models.

With the advancement in technology, normal network traffic is becoming more heterogeneous. In this scenario, the problem of detecting anomalies is intensified. In the literature, offline methods see more data and can be optimised to achieve lower false positive rates. However, they cannot readily adapt to changing network conditions or capture concept-drift. This necessitates an incremental online learning model. On the other hand, online training is easily affected by noise. In this paper, we propose a hybrid Online Offline system in which the Offline model retains general characteristics of network traffic while the Online model continuously learns. The Offline model acts as a bias for the Online model to select new data to learn from. The Online model retains its knowledge and adapts to the changing ground truth. They are put to work together to detect anomalies. We implement this idea with an Online Support Vector Machine (SVM) which retains its support vectors and shifts its decision boundary guided by an Offline Radius Nearest Neighbor (Rad-NN). The method is evaluated on the NSL-KDD 2009 dataset. This relatively simple model achieves over 95% accuracy on known anomalies and over 60% detection rate on most of the unknown anomalies.

A long-standing obstacle to progress in deep learning is the problem of vanishing and exploding gradients. Although, the problem has largely been overcome via carefully constructed initializations and batch normalization, architectures incorporating skip-connections such as highway and resnets perform much better than standard feedforward architectures despite well-chosen initialization and batch normalization. In this paper, we identify the shattered gradients problem. Specifically, we show that the correlation between gradients in standard feedforward networks decays exponentially with depth resulting in gradients that resemble white noise whereas, in contrast, the gradients in architectures with skip-connections are far more resistant to shattering, decaying sublinearly. Detailed empirical evidence is presented in support of the analysis, on both fully-connected networks and convnets. Finally, we present a new “looks linear” (LL) initialization that prevents shattering, with preliminary experiments showing the new initialization allows to train very deep networks without the addition of skip-connections.

Content replication and placement is an effective technique to improve data availability and enhancing network performance. This area received attention in Content Delivery Networks (CDNs), but is less explored by the research community in Wireless Mesh Networks (WMNs). A number of replica placement algorithms were specifically designed for CDNs, but they do not consider the special features of wireless networks such as insufficient bandwidth, low server capacity, contention to access the wireless medium, etc. In this paper, we propose a fully-fledged object replication and placement scheme for WMNs. In our model, each mesh router acts as a replica server with limited storage capacity. The challenge is to minimize the demand-weighted access cost by replicating popular objects and placing them at replica servers (mesh routers) as close as possible to the requesting mesh clients, while minimizing the number of replicas subject to the storage capacity constraint. Our scheme is lightweight, distributed, scalable and adaptable. Using simulation tests, we demonstrate the scalability and performance gain of our scheme over its counterparts with respect to access latency, network load and server load.

Recently, Wireless Mesh Networks (WMNs) have attracted much of interest from both academia and industry, due to their potential to provide an alternative broadband wireless Internet connectivity. However, due to different reasons such as multi-hop forwarding and the dynamic wireless link characteristics, the performance of current WMNs is rather low when clients are soliciting Web contents. Due to the evolution of advanced mobile computing devices; it is anticipated that the demand for bandwidth-onerous popular content (especially multimedia content) in WMNs will dramatically increase in the coming future. Content replication is a popular approach for outsourcing content on behalf of the origin content provider. This area has been well explored in the context of the wired Internet, but has received comparatively less attention from the research community when it comes to WMNs. There are a number of replica placement algorithms that are specifically designed for the Internet. But they do not consider the special features of wireless networks such as insufficient bandwidth, low server capacity, contention to access the wireless medium, etc. In this paper, we propose a distributed heuristic solution for the replica placement problem in WMNs. This problem is known to be NP-complete. Our heuristic considers the local popularity of an object replica. The local popularity can be defined as the relative demand for an object within a partition of the network compared to the whole network. The heuristic collects popularity information periodically to compute the number of replicas for future period requests. Using extensive simulation tests, we demonstrate the performance gain of our heuristic compared to other relevant ones.

In this paper, we propose a distributed heuristic solution for the replica placement problem in Wireless Mesh Networks (WMNs). This problem is known to be NP-complete. Our heuristic considers the local popularity of an object replica. The local popularity can be defined as the relative demand for an object within a partition of the network compared to the whole network. The heuristic collects popularity information periodically to compute the number of replicas for future period requests. We perform simulation experiments to investigate the performance of our heuristic.

The reweighing process of particle filtering can result in very large variance of particle weights; a problem known as degeneracy. The usual solution to this is an intermediary resampling step in which particles with lower weights are replaced by copies of those with large weights. This resampling inevitably results in loss of the information contained in those particles of low weights. Most of the existing stochastic and deterministic resampling schemes cause further loss of information because all the resampled particles have equal weights. These current techniques are what we would call, “hard resamplers,” and they impede the accumulation of information over many successive observations, which affects the detection of very covert targets. This paper presents two variants of a soft and deterministic resampling procedure that retains information contained in the weights over multiple observations. We demonstrate their effectiveness using a track-before-detect application.

Wireless mesh networks (WMNs) were proposed to provide low-cost, easy deployment and robust access to the Internet. One of the objectives in WMNs is the improvement of client throughput, which can be achieved using content replication. Many replication schemes are specifically designed for the Internet. However, they do not account for the different characteristics of wireless networks, such as insufficient and fluctuating bandwidth, packet loss, contention to access the wireless medium, etc. In this paper, we study the problem of object replication and placement in WMNs, where mesh nodes act as replica servers in a P2P model that improves Quality of Experience by replicating content as close as possible to the requesting mesh clients. Furthermore, we aim to optimize the number of replicas per object to better utilize the storage capacity per node. In WMNs, the wireless link-quality is paramount in the placement decision and the measurement of the object access cost. Therefore, we propose a link-quality aware, distributed and scalable scheme for object replication. The proposed scheme exploits the long-term link-quality routing metrics to augment the replica placement decision and the instantaneous link-quality metrics for replica server selection. The simulation results show that our proposed scheme has better performance compared to other replication schemes.

Content replication has gained much popularity in recent years both in the wired and wireless infrastructures. A key challenge faced by Wireless Mesh Networks (WMNs) is to determine the number and locations of content replicas (e.g. video clip) such that the mesh clients access cost is minimized. Furthermore, the dynamic nature of the wireless environment favors a distributed and adaptive solution to this problem. In this paper, we present an efficient, lightweight and scalable object replication and placement scheme for WMNs. Since the placement problem is NP-Complete, the scheme decomposes the problem into smaller sub-problems to facilitate the distributed approach in a P2P fashion. Moreover, it exploits the long-term link-quality routing metrics to augment the replica placement decision and the instantaneous link-quality metrics for replica server selection. The effectiveness of our scheme is evaluated through extensive simulation studies.

We propose Gaussian process dynamical models (GPDMs) as a new, nonparametric paradigm in acoustic models of speech. These use multidimensional, continuous state-spaces to overcome familiar issues with discrete-state, HMM-based speech models. The added dimensions allow the state to represent and describe more than just temporal structure as systematic differences in mean, rather than as mere correlations in a residual (which dynamic features or AR-HMMs do). Being based on Gaussian processes, the models avoid restrictive parametric or linearity assumptions on signal structure. We outline GPDM theory, and describe model setup and initialization schemes relevant to speech applications. Experiments demonstrate subjectively better quality of synthesized speech than from comparable HMMs. In addition, there is evidence for unsupervised discovery of salient speech structure.

Content replication and placement is an area that has been well explored in the scope of Content Delivery Networks (CDNs), but has received relatively less attention from the research community when it comes to Wireless Mesh Networks (WMNs). There are a number of Replica Placement Algorithms (RPAs) that are specifically designed for CDNs. But they do not consider the special features of wireless networks. In this paper, we propose a new heuristic called MP-DNA (Multiple Partitions per Delegate Node Assignment). We study the problem of optimal content replication and placement in WMNs. In our model, each mesh router acts as a replica server with limited storage capacity. The challenge is to replicate content as close as possible to the requesting clients and thus reduce the access latency per object, while minimizing the number of replicas. We formulate this problem in terms of combinatorial optimization and propose a novel, distributed, scalable heuristic for content replication. Using simulation tests, we demonstrate that MP-DNA is scalable, performing well with respect to the number of replica servers and the number of objects. Furthermore, MP-DNA can achieve a performance close to the Greedy-Global heuristic, with a significant reduction in running time, mean throughput and storage space required.

Wireless Mesh Networks (WMNs) extend Internet access in areas where the wired infrastructure is not available. A problem that arises is the congestion around gateways, delayed access latency and low throughput. Therefore, object replication and placement is essential for multi-hop wireless networks. Many replication schemes are proposed for the Internet, but they are designed for CDNs that have both high bandwidth and high server capacity, which makes them unsuitable for the wireless environment. Object replication has received comparatively less attention from the research community when it comes to WMNs. In this paper, we propose an object replication and placement scheme for WMNs. In our scheme, each mesh router acts as a replica server in a peer-to-peer fashion. Our scheme exploits graph partitioning to build a hierarchy from fine-grained to coarse-grained partitions. The challenge is to replicate content as close as possible to the requesting clients and thus reduce the access latency per object, while minimizing the number of replicas. Using simulation tests, we demonstrate that our scheme is scalable, performing well with respect to the number of replica servers and the number of objects. The proposed scheme can give improved performance in terms of convergence time, throughput, hop count and hit ratio.

Memetic algorithms and cooperative coevolution are emerging fields in evolutionary computation which have shown to be powerful tools for real-world application problems and for training neural networks. Cooperative coevolution decomposes a problem into subcomponents that evolve independently. Memetic algorithms provides further enhancement to evolutionary algorithms with local refinement. The use of crossover-based local refinement has gained attention in memetic computing. This paper employs a cooperative coevolutionary framework that utilises the strength of local refinement via crossover. The framework is evaluated by training recurrent neural networks on grammatical inference problems. The results show that the proposed approach can achieve better performance than the standard cooperative coevolution framework.

In this paper, an adaptive modularity cooperative coevolutionary framework is presented for training feedforward neural networks. The modularity adaptation framework is composed of different neural network encoding schemes which transform from one level to another based on the network error. The proposed framework is compared with canonical cooperative coevolutionary methods. The results show that the proposal outperforms its counterparts in terms of training time, success rate and scalability.

Large amounts of Java software have been written since the language's escape into unsuspecting software ecology more than ten years ago. Surprisingly little is known about the structure of Java programs in the wild: about the way methods are grouped into classes and then into packages, the way packages relate to each other, or the way inheritance and composition are used to put these programs together. We present the results of the first in-depth study of the structure of Java programs. We have collected a number of Java programs and measured their key structural attributes. We have found evidence that some relationships follow power-laws, while others do not. We have also observed variations that seem related to some characteristic of the application itself. This study provides important information for researchers who can investigate how and why the structural relationships we find may have originated, what they portend, and how they can be managed.

Evolutionary algorithms perform optimization using a population of sample solution points. An interesting development has been to view population-based optimization as the process of evolving an explicit, probabilistic model of the search space. This paper investigates a formal basis for continuous, population-based optimization in terms of a stochastic gradient descent on the Kullback-Leibler divergence between the model probability density and the objective function, represented as an unknown density of assumed form. This leads to an update rule that is related and compared with previous theoretical work, a continuous version of the population-based incremental learning algorithm, and the generalized mean shift clustering framework. Experimental results are presented that demonstrate the dynamics of the new algorithm on a set of simple test problems.

Gaussian processes are usually parameterised in terms of their covari- ance functions. However, this makes it difficult to deal with multiple outputs, because ensuring that the covariance matrix is positive definite is problematic. An alternative formulation is to treat Gaussian processes as white noise sources convolved with smoothing kernels, and to param- eterise the kernel instead. Using this, we extend Gaussian processes to handle multiple, coupled outputs.

The prisoner's dilemma (PD) involves contests between two players and may naturally be played on a spatial grid using voter model rules. In the model of spatial PD discussed here, the sites of a two-dimensional lattice are occupied by strategies. At each time step, a site is chosen to play a PD game with one of its neighbors. The strategy of the chosen site then invades its neighbor with a probability that is proportional to the payoff from the game. Using results from the analysis of voter models, it is shown that with simple linear strategies, this scenario results in the long-term survival of only one strategy. If three nonlinear strategies have a cyclic dominance relation between one another, then it is possible for relatively cooperative strategies to persist indefinitely. With the voter model dynamics, however, the average level of cooperation decreases with time if mutation of the strategies is included. Spatial effects are not in themselves sufficient to lead to the maintenance of cooperation.

The thermal perceptron is a simple extension to Rosenblatt's perceptron learning rule for training individual linear threshold units. It finds stable weights for nonseparable problems as well as separable ones. Experiments indicate that if a good initial setting for a temperature parameter, T0, has been found, then the thermal perceptron outperforms the Pocket algorithm and methods based on gradient descent. The learning rule stabilizes the weights (learns) over a fixed training period. For separable problems it finds separating weights much more quickly than the usual rules.

A general method for building and training multilayer perceptrons composed of linear threshold units is proposed. A simple recursive rule is used to build the structure of the network by adding units as they are needed, while a modified perceptron algorithm is used to learn the connection strengths. Convergence to zero errors is guaranteed for any boolean classification on patterns of binary variables. Simulations suggest that this method is efficient in terms of the numbers of units constructed, and the networks it builds can generalize over patterns not in the training set.