Estimating the latent dynamics underlying biological processes is a central problem in computational biology. State-space models with Gaussian statistics are widely used for estimation of such latent dynamics and have been successfully utilized in the analysis of biological data. Gaussian statistics, however, fail to capture several key features of the dynamics of biological processes (e.g., brain dynamics) such as abrupt state changes and exogenous processes that affect the states in a structured fashion. Although Gaussian mixture process noise models have been considered as an alternative to capture such effects, data-driven inference of their parameters is not well-established in the literature. The objective of this paper is to develop efficient algorithms for inferring the parameters of a general class of Gaussian mixture process noise models from noisy and limited observations, and to utilize them in extracting the neural dynamics that underlie auditory processing from magnetoencephalography (MEG) data in a cocktail party setting. We develop an algorithm based on Expectation-Maximization to estimate the process noise parameters from state-space observations. We apply our algorithm to simulated and experimentally-recorded MEG data from auditory experiments in the cocktail party paradigm to estimate the underlying dynamic Temporal Response Functions (TRFs). Our simulation results show that the richer representation of the process noise as a Gaussian mixture significantly improves state estimation and capturing the heterogeneity of the TRF dynamics. Application to MEG data reveals improvements over existing TRF estimation techniques, and provides a reliable alternative to current approaches for probing neural dynamics in a cocktail party scenario, as well as attention decoding in emerging applications such as smart hearing aids. Our proposed methodology provides a framework for efficient inference of Gaussian mixture process noise models, with application to a wide range of biological data with underlying heterogeneous and latent dynamics.

Motivated by models of human decision making proposed to explain commonly observed deviations from conventional expected value preferences, we formulate two stochastic multi-armed bandit problems with distorted probabilities on the cost distributions: the classic K-armed bandit and the linearly parameterized bandit. In both settings, we propose algorithms that are inspired by Upper Confidence Bound (UCB) algorithms, incorporate cost distortions, and exhibit sublinear regret assuming Holder continuous weight distortion functions. For the K-armed setting, we show that the algorithm, called W-UCB, achieves problem-dependent regret O(L2 M2 log n / Δ(2/α – 1), where n is the number of plays, Δ is the gap in distorted expected value between the best and next best arm, L and alpha are the Holder constants for the distortion function, and M is an upper bound on costs, and a problem-independent regret bound of O((KL2M2)(α/2) n(2 – α)/2)). We also present a matching lower bound on the regret, showing that the regret of W-UCB is essentially unimprovable over the class of Holder-continuous weight distortions. For the linearly parameterized setting, we develop a new algorithm, a variant of the Optimism in the Face of Uncertainty Linear bandit (OFUL) algorithm called WOFUL (Weight-distorted OFUL), and show that it has regret O(d√n polylog(n)) with high probability, for sub-Gaussian cost distributions.

Cumulative prospect theory (CPT) is known to model human decisions well, with substantial empirical evidence supporting this claim. CPT works by distorting probabilities and is more general than the classic expected utility and coherent risk measures. We bring this idea to a risk-sensitive reinforcement learning (RL) setting and design algorithms for both estimation and control. The RL setting presents two particular challenges when CPT is applied: estimating the CPT objective requires estimations of the entire distribution of the value function and finding a randomized optimal policy. The estimation scheme that we propose uses the empirical distribution to estimate the CPT-value of a random variable. We then use this scheme in the inner loop of a CPT-value optimization procedure that is based on the well-known simulation optimization idea of simultaneous perturbation stochastic approximation (SPSA). We provide theoretical convergence guarantees for all the proposed algorithms and also empirically demonstrate the usefulness of our algorithms.

Global optimization problems with limited structure (e.g., convexity or differentiability of the objective function) can arise in many fields. One approach to solving these problems is by modeling the evolution of a probability density function over the solution space, similar to the Fokker–Planck equation for diffusions, such that at each time instant, additional weight is given to better solutions. We propose an addition to the class of model-based methods, cumulative weighting optimization (CWO), whose general version can be proven convergent to an optimal solution and stable under disturbances (e.g., floating point inaccuracy). These properties encourage us to design a class of CWO algorithms for solving global optimization problems. Beyond the general convergence and stability analysis, we prove that with some additional assumptions the Monte Carlo version of the CWO algorithm is also convergent and stable. Interestingly, the well known cross-entropy method is a CWO algorithm.

We study the problem of assigning wavelengths to a given set of multicast traffic requests with the objective of minimizing the number of wavelengths used per fiber. We assume that the underlying optical network is a tree and that all-optical networking paradigm is employed. First, we prove that the problem is NP hard even when the underlying network is a simple star or path. Since assigning wavelengths to a given set of unicast traffic requests on star and path networks is easy, this shows that the the multicast wavelength assignment problem is fundamentally harder than the unicast scenario. Next we present GREEDY and SUBTREE-BASED: two simple deterministic algorithms for assigning wavelengths to a given set of multicast traffic requests when the underlying network is a tree with maximum node degree 3 and 4 respectively. Of the two wavelength assignment schemes, GREEDY is a 5/2-approximation algorithm, and SUBTREE-BASED is an approximation algorithm with approximation ratio 10/3, 3 and 2 for the cases when the underlying network is a tree with degree 4, 3 and 2, respectively.

This paper presents a new randomized search method called evolutionary random policy search (ERPS) for solving infinite-horizon discounted-cost Markov-decision-process (MDP) problems. The algorithm is particularly targeted at problems with large or uncountable action spaces. ERPS approaches a given MDP by iteratively dividing it into a sequence of smaller, random, sub-MDP problems based on information obtained from random sampling of the entire action space and local search. Each sub-MDP is then solved approximately by using a variant of the standard policy-improvement technique, where an elite policy is obtained. We show that the sequence of elite policies converges to an optimal policy with probability one. Some numerical studies are carried out to illustrate the algorithm and compare it with existing procedures.

In this paper we study the problem of efficient grooming of given non-uniform multicast traffic demands on a unidirectional SONET/WDM ring. The goal is to try to minimize the network cost as given by (i) the number of wavelengths required per fiber and (ii) the number of electronic add-drop multiplexers (ADMs) required on the ring. The problem is NP hard for both the cost functions. We observe that the problem with cost function (i) can be reduced to a corresponding traffic grooming problem for unicast traffic which can then be modelled as a standard circular-arc graph coloring problem. For cost function (ii), we construct a graph based heuristic and compare it against the multicast extension of the best known unicast traffic grooming heuristic (Zhang, 2000). We observe that our heuristic requires fewer ADMs than required by the multicast extension of the unicast heuristic given in (Zhang, 2000). We also develop a lower bound and compare it against some upper bounds to study the maximum penalty of not employing intelligent wavelength assignment and/or traffic grooming under the unidirectional SONET/WDM ring scenario.

rdquoWe present a novel group-based mechanism to reconfigure the virtual topology of survivable MPLS/WDM networks by using the existing shared protection backup resource. With this mechanism, lightpaths are divided into groups and those in the same group can be reconfigured simultaneously at one step. Ideally, this mechanism won't incur any service disruption during the reconfiguration process. The optimal reconfiguration policy is obtained through solving following two problems: the Grouping problem which minimizes the reconfiguration steps, and the Sequencing problem which minimizes the network resource used during the reconfiguration process. We prove these two problems to be NP-hard and present efficient heuristic algorithms. A general mathematical method of rollout is applied to the heuristics to improve the solution quality. Numerical results are presented to show the optimal tradeoff between the reconfiguration duration and the required redundant capacity.

We consider the downlink rate control problem in a wireless channel. A dynamic programming optimization method is introduced to obtain the optimal bit-rate/delay control policy in the downlink for packet transmission in wireless networks with fading channels. We assume that the base station is capable of transmitting data packets in the downlink with different bit rates, R/sub 0/<R/sub 1/</spl middot//spl middot//spl middot/

Optimal structured feedback control policies for rate-based flow control of available bit rate service in asynchronous transfer mode networks are obtained in the presence of information and propagation delays, using a numerically efficient two-timescale simultaneous perturbation stochastic approximation (SPSA) algorithm. Models comprising both a single bottleneck node and a network with multiple bottleneck nodes are considered. A convergence analysis of the algorithm is presented. Numerical experiments demonstrate fast convergence even in the presence of significant delays. We also illustrate performance comparisons with the well-known explicit rate indication for congestion avoidance (ERICA) algorithm and describe another algorithm (based on ERICA) that does not require estimating available bandwidth (as in ERICA).

For an object being rigidly grasped and manipulated by multiple robotic mechanisms, the internal loading characteristics at a common coordinate set within the object are considered. It is demonstrated that representation of internal forces and moments in these common coordinates give insight into force and load distribution schemes developed previously. In particular, it is shown how internal loads may be created in some end-effector force distribution schemes even when no component in the null-space of the grasp matrix is included. It is further shown that a particular pseudoinverse of the grasp matrix, which can be shown to be consistent with the kinematic constraints, may be used to eliminate this situation. The case of a two-arm system is used to illustrate the concepts introduced.>

The issue of distributing the task among the multiple robot arms while considering the manipulator dynamics is considered. The forces and moments required to move an object are distributed in such a way that extra degrees of freedom within the system may be used to satisfy or optimize criteria related to the manipulator dynamics. A method to perform such subtasks is introduced, and examples of possible criteria noted. It is expected that such techniques will produce trajectories which will be more desirable for the individual arms, dynamically, since the dynamics are considered in the task distribution.>

The problem of selecting joint space trajectories for redundant manipulators is considered. Solutions which allow secondary tasks to be performed by the arm simultaneously with end-effector motions may be selected in a number of ways. An algorithm to accomplish this, by means of conditions on a scalar function of the joint variables, is introduced and analyzed. Problems inherent in schemes involving constraints in configuration space are considered.>

A new architecture for local area networks, based on the carry-lookahead circuit has been described in recent papers. It features a cost-effective hardware round-robin priority circuit, a bus and a shift register ring network that uses the same hardware. This paper shows that such an architecture can be used to minimize Σ Ui Wi where Ui is a real number urgency and Wi is the waiting time of message i, where Ui is a constant, or a time varying function which is a sequence of constant steps. Such a capability is useful in distributed real-time control systems. This paper demonstrates that a network that implements priority, bus and shift register ring-hardware can create new capabilities for local area networks.

The modeling and analysis of nonlinear systems described by differential equations driven by point process noise are considered. The stochastic calculus of McShane is generalized to include such differential equations, and a more general canonical extension is defined. It is proved that this canonical extension possesses the same desirable properties for point process noise that it does for the noise processes, such as Brownian motion, considered by McShane. In addition, a new stochastic integral with respect to a point process is defined; this alternative integral obeys the rules of ordinary calculus. As a special case of the analysis of such systems, linear systems with multiplicative point process noise are investigated. The consistency of the canonical extension is studied by means of the product integral. Finally, moment equations and criteria for the stochastic stability of linear systems with multiplicative Poisson noise are derived.

Methods of Fourier analysis and an assumed folded normal density approximation are applied to a nonlinear estimation problem in which the observation consists of a doubly stochastic Poisson process. A particular application of optical phase tracking is discussed. The performance, as computed by Monte Carlo simulation, of the resulting suboptimal estimator compares favorably to that of the suboptimal estimators of Snyder.