Standard deep reinforcement learning (DRL) aims to maximize expected reward, considering collected experiences equally in formulating a policy. This differs from human decision-making, where gains and losses are valued differently and outlying outcomes are given increased consideration. It also fails to capitalize on opportunities to improve safety and/or performance through the incorporation of distributional context. Several approaches to distributional DRL have been investigated, with one popular strategy being to evaluate the projected distribution of returns for possible actions. We propose a more direct approach whereby risk-sensitive objectives, specified in terms of the cumulative distribution function (CDF) of the distribution of full-episode rewards, are optimized. This approach allows for outcomes to be weighed based on relative quality, can be used for both continuous and discrete action spaces, and may naturally be applied in both constrained and unconstrained settings. We show how to compute an asymptotically consistent estimate of the policy gradient for a broad class of risk-sensitive objectives via sampling, subsequently incorporating variance reduction and regularization measures to facilitate effective on-policy learning. We then demonstrate that the use of moderately "pessimistic" risk profiles, which emphasize scenarios where the agent performs poorly, leads to enhanced exploration and a continual focus on addressing deficiencies. We test the approach using different risk profiles in six OpenAI Safety Gym environments, comparing to state of the art on-policy methods. Without cost constraints, we find that pessimistic risk profiles can be used to reduce cost while improving total reward accumulation. With cost constraints, they are seen to provide higher positive rewards than risk-neutral approaches at the prescribed allowable cost.

In contrast to perturbation-based attacks, patch-based attacks are physically realizable, and are therefore increasingly studied. However, prior work neglects the possibility of adaptive attacks optimized for 3D pose. For the first time, to our knowledge, we consider the challenge of designing and evaluating attacks on image sequences using 3D optimization along entire 3D kinematic trajectories. In this context, we study a type of dynamic attack, referred to as "adaptive active attacks" (AAA), that takes into consideration the pose of the observer being targeted. To better address the threat and risk posed by AAA attacks, we develop several novel risk-based and trajectory-based metrics. These are designed to capture the risk of attack success for attacking earlier in the trajectory to derail autonomous driving systems as well as tradeoffs that may arise given the possibility of additional detection. We evaluate performance of white-box targeted attacks using a subset of ImageNet classes, and demonstrate, in aggregate, that AAA attacks can pose threats beyond static attacks in kinematic settings in situations of predominantly looming motion (i. e., a prevalent use case in automated vehicular navigation). Results demonstrate that AAA attacks can exhibit targeted attack success exceeding 10% in aggregate, and for some specific classes, up to 15% over their static counterparts. However, taking into consideration the probability of detection by the defender shows a more nuanced risk pattern. These new insights are important for guiding future adversarial machine learning studies and suggest researchers should consider defense against novel threats posed by dynamic attacks for full trajectories and videos.

Human-aware robot navigation promises a range of applications in which mobile robots bring versatile assistance to people in common human environments. While prior research has mostly focused on modeling pedestrians as independent, intentional individuals, people move in groups; consequently, it is imperative for mobile robots to respect human groups when navigating around people. This paper explores learning group-aware navigation policies based on dynamic group formation using deep reinforcement learning. Through simulation experiments, we show that group-aware policies, compared to baseline policies that neglect human groups, achieve greater robot navigation performance (e.g., fewer collisions), minimize violation of social norms and discomfort, and reduce the robot's movement impact on pedestrians. Our results contribute to the development of social navigation and the integration of mobile robots into human environments.

Accurate state and uncertainty estimation is imperative for mobile robots and self driving vehicles to achieve safe navigation in pedestrian rich environments. A critical component of state and uncertainty estimation for robot navigation is to perform robustly under out-of-distribution noise. Traditional methods of state estimation decouple perception and state estimation making it difficult to operate on noisy, high dimensional data. Here, we describe an approach that combines the expressiveness of deep neural networks with principled approaches to uncertainty estimation found in recursive filters. We particularly focus on techniques that provide better robustness to out-of-distribution noise and demonstrate applicability of our approach on two scenarios: a simple noisy pendulum state estimation problem and real world pedestrian localization using the nuScenes dataset [1]. We show that our approach improves state and uncertainty estimation compared to baselines while achieving approximately 3× improvement in computational efficiency.

Our work focuses on unsupervised and generative methods that address the following goals: (1) learning unsupervised generative representations that discover latent factors controlling image semantic attributes, (2) studying how this ability to control attributes formally relates to the issue of latent factor disentanglement, clarifying related but dissimilar concepts that had been confounded in the past, and (3) developing anomaly detection methods that leverage representations learned in the first goal. For goal 1, we propose a network architecture that exploits the combination of multiscale generative models with mutual information (MI) maximization. For goal 2, we derive an analytical result, lemma 1, that brings clarity to two related but distinct concepts: the ability of generative networks to control semantic attributes of images they generate, resulting from MI maximization, and the ability to disentangle latent space representations, obtained via total correlation minimization. More specifically, we demonstrate that maximizing semantic attribute control encourages disentanglement of latent factors. Using lemma 1 and adopting MI in our loss function, we then show empirically that for image generation tasks, the proposed approach exhibits superior performance as measured in the quality and disentanglement of the generated images when compared to other state-of-the-art methods, with quality assessed via the Fréchet inception distance (FID) and disentanglement via mutual information gap. For goal 3, we design several systems for anomaly detection exploiting representations learned in goal 1 and demonstrate their performance benefits when compared to state-of-the-art generative and discriminative algorithms. Our contributions in representation learning have potential applications in addressing other important problems in computer vision, such as bias and privacy in AI.

We develop a framework for novelty detection (ND) methods relying on deep embeddings, either discriminative or generative, and also propose a novel framework for assessing their performance. While much progress was made recently in these approaches, it has been accompanied by certain limitations: most methods were tested on relatively simple problems (low resolution images / small number of classes) or involved non-public data; comparative performance has often proven inconclusive because of lacking statistical significance; and evaluation has generally been done on non-canonical problem sets of differing complexity, making apples-to-apples comparative performance evaluation difficult. This has led to a relative confusing state of affairs. We address these challenges via the following contributions: We make a proposal for a novel framework to measure the performance of novelty detection methods using a trade-space demonstrating performance (measured by ROCAUC) as a function of problem complexity. We also make several proposals to formally characterize problem complexity. We conduct experiments with problems of higher complexity (higher image resolution / number of classes). To this end we design several canonical datasets built from CIFAR-10 and ImageNet (IN-125) which we make available to perform future benchmarks for novelty detection as well as other related tasks including semantic zero/adaptive shot and unsupervised learning. Finally, we demonstrate, as one of the methods in our ND framework, a generative novelty detection method whose performance exceeds that of all recent best-in-class generative ND methods.

This paper considers attacks against machine learning algorithms used in remote sensing applications. The remote sensing domain presents a suite of challenges that are not fully addressed by current research focused on natural image data. In this paper we present a new study of adversarial examples in the context of satellite image classification problems. Using a recently curated data set and associated classifier, we provide a preliminary analysis of adversarial examples in settings where the targeted classifier is permitted multiple observations of the same location over time. While our experiments to date are purely digital, our problem setup incorporates a number of practical considerations that an attacker would need to take into account when mounting physical attacks.

We study the problem of classifying images when no training exemplars are available for some image classes, and therefore direct classification is not possible. We use instead semantic attributes: if attributes of yet unseen classes can be determined, then class labels may themselves be decided based on prior knowledge of class to attributes relationships. We present several methods for determining attributes, including (A) an approach based on attribute classifiers, and approaches using (B) MAP and (C) MMSE attribute estimators using image classifiers for known classes. Preliminary tests obtained using a dataset comprised of ImageNet images and Human218 attributes yield encouraging performance.

This paper considers online classification problems where each object to be classified consists of a sequence of measurements, termed here a track. We present an approach that combines ideas from sequential hypothesis testing with those from conformal prediction to address track level outliers - entire measurement sequences that are novel relative to the statistical model. We show with analysis and empirical results that this approach preserves the optimal performance of the underlying sequential hypothesis testing when outliers are absent and provides an error rate guarantee in the presence of contamination by novel tracks.

Most target tracking applications developed for wireless sensor networks thus far employ passive sensors which detect the target's emissions. Their performance is thereby dictated by the magnitude of the target's signal and the receivers' sensitivity. Instead, we propose using a network of low-power Doppler radars that actively measure the target's radial velocity. Nodes combine their measurements to solve a system of nonlinear equations that estimate the target's position and velocity. Because these equations have no closed-form solutions we solve them using numerical methods. These methods however can lead to local minima or even diverge in the presence of even small measurement noise. For this reason we couple the proposed numerical method with an Extended Kalman filter that models the target's movement along a straight line. Clearly the target does not always follow a linear path and so we augment this simple Kalman filter with a method that detects the target's turns and updates the filter's dynamical model accordingly. The combination of the two approaches improve tracking accuracy compared to the numerical solution alone. Results from simulations and a prototype implementation suggest that the combined solution can effectively track a mobile target with average localization error as low as 25 cm in a 10×10 m outdoor field.

Our work addresses pose estimation in a distributed camera framework. We examine how processing cameras can best reach a consensus about the pose of an object when they are each given a model of the object, defined by a set of point coordinates in the object frame of reference. The cameras can only see a subset of the object feature points in the midst of background clutter points, not knowing which image points match with which object points, nor which points are object points or background points. The cameras individually recover a prediction of the object's pose using their knowledge of the model, and then exchange information with their neighbors, performing consensus updates locally to obtain a single estimate consistent across all cameras, without requiring a common centralized processor. Our main contributions are: 1) we present a novel algorithm performing consensus updates in 3-D world coordinates penalized by a 3-D model, and 2) we perform a thorough comparison of our method with other current consensus methods. Our method is consistently the most accurate, and we confirm that the existing consensus method based upon calculating the Karcher mean of rotations is also reliable and fast. Experiments on simulated and real imagery are reported.

In this paper, we consider a distributed medium access control (MAC) for mobile ad-hoc networks (MANET). We propose an adaptive threshold-based medium contention scheme to exploit diversity gain in multiple dimensions, in particular, the time and user diversity. In our scheme, each node performs a c

In this paper we discuss the role of a broker in a market-oriented resource allocation model for large-scale heterogeneous systems. The simplified model is based upon a three party system, provider-broker-consumer. The allocation of resources is determined by their price, their utility to the consumer, and by the satisfaction of the consumer. The role of the broker is to add societal objectives to resource allocation algorithms and to mediate between greedy consumers and selfish providers. A simulation experiment was conducted to study the transient and the steady-state behavior of several performance measures, including the average consumer satisfaction, the average utility, and the hourly revenue.

The execution of a complex task in any environment requires planning. Planning is the process of constructing an activity graph given by the current state of the system, a goal state, and a set of activities. If we wish to execute a complex computing task in a heterogeneous computing environment with autonomous resource providers, we should be able to adapt to changes in the environment. A possible solution is to construct a family of activity graphs beforehand and investigate the means of switching from one member of the family to another when the execution of one activity graph fails. In this paper, we study the conditions when plan switching is feasible. Then we introduce an approach for plan switching and report the simulation results of this approach

A landslide occurs when the balance between a hill's weight and the countering resistance forces is tipped in favor of gravity. While the physics governing the interplay between these competing forces is fairly well understood, prediction of landslides has been hindered thus far by the lack of field measurements over large temporal and spatial scales necessary to capture the inherent heterogeneity in a landslide.We propose a network of sensor columns deployed at hills with landslide potential with the purpose of detecting the early signals preceding a catastrophic event. Detection is performed through a three-stage algorithm: First, sensors collectively detect small movements consistent with the formation of a slip surface separating the sliding part of hill from the static one. Once the sensors agree on the presence of such a surface, they conduct a distributed votingalgorithm to separate the subset of sensors that moved from the static ones. In the second phase, moved sensors self-localize through a trilateration mechanism and their displacements are calculated. Finally, the direction of the displacements as well as the locations of the moved nodes are used to estimate the position of the slip surface. This information along with collected soil measurements e.g. soil pore pressures) are subsequently passed to a Finite Element Model that predicts whether and when a landslide will occur.Our initial results from simulated landslides indicate that we can achieve accuracy in the order of cm in the localization as well as the slip surface estimation steps of our algorithm. This accuracy persists as the density and the size of the sensor network decreases as well as when considerable noise is present in the ranging estimates. As for our next step, we plan to evaluate the performance of our system in controlled environments under a variety of hill configurations.

A novel procedure for learning a probabilistic model from mass spectrometry data that accounts for domain specific noise and mitigates the complexity of Bayesian structure learning is presented. We evaluate the algorithm by applying the learned probabilistic model to microorganism detection from mass spectrometry data.

In this paper we outline a sensor network for landslide detection. Network sensors deployed on the surface and underground the hill under observation, use distributed signal processing techniques to self-localize and detect any changes in their relative locations. When such movements occur, changes in location as well as soil parameters are passed to a central location and used as input to a Finite Element Model that predicts whether and when a landslide will occur.

With the proliferation of inexpensive cameras, availability of large-capacity disk storage, and the ubiquitous presence of high-speed, broad-band communication networks, it is now economically and technically feasible to construct and deploy multi-camera video surveillance systems. In line with this is the need of intelligent, robust, (semi-)automated video analysis paradigms to assist the operators in scene analysis and event classification. In this paper, we summarize our current research work toward realizing such a multi-camera video surveillance system. Interested readers can visit our web sites http://www.ece.ucsb.edu//sup /spl sim//echang and http://www.cs.ucsb.edu//sup /spl sim//yfwang for more information.

A class of synchronization protocols for dense, large-scale sensor networks is presented. The protocols build on the recent work of Hong, Cheow, and Scaglione [5, 6] in which the synchronization update rules are modeled by a system of pulse-coupled oscillators. In the present work, we define a class of models that converge to a synchronized state based on the local communication topology of the sensor network only, thereby lifting the all-to-all communication requirement implicit in [5, 6]. Under some rather mild assumptions of the connectivity of the network over time, these protocols still converge to a synchronized state when the communication topology is time varying.

We describe a general technique for estimating the intractable quantities that occur in a wide variety of large-scale probabilistic models. The technique transforms intractable sums into integrals which are subsequently approximated via saddle point methods. When applied to sigmoid and noisy-OR networks, the technique yields a generic mean-field approximation as well as a second order Gaussian approximation that accounts for the pairwise correlations between random variables in the network. In two example models, we observe that our lowest order approximation is identical to expressions obtained using Plefka's approach for deriving the TAP equations.

The ability to discriminate a reentry vehicle (RV) from booster parts and other debris is critical to theater ballistic missile defense (TBMD). As it travels along its trajectory, a threat missile separates into a reentry vehicle (RV) and clutter. The latter consists of several tanks, separation debris and fragments of hot fuel. Interception of the RV requires discrimination of the RV from the clutter. The required discrimination must be performed no later than 30 seconds before intercept. A time-delay neural network (TDNN) is proposed for discrimination of the RV from other missile parts debris. The rate of change of the IR signature over several seconds is used as a discriminant. The performances of two different approaches are compared: 1) A TDNN that employs backpropagation weight updates is used to calculate activation levels for output nodes. The RV is selected via winner-take-all. A TDNN that updates weights using a cross-entropy error function with a softmax activation function is used to estimate assignment probabilities. The RV is subsequently selected via a probabilistic assignment algorithm that imposes the constraint that there can only be a single RV. We found that the TDNN employing backpropagated softmax learning performed better than the TDNN employing backpropagated least mean square learning.

We develop deterministic necessary and sufficient conditions on individual noise sequences of a stochastic approximation algorithm for the error of the iterates to converge at a given rate. Specifically, suppose {/spl rho//sub n/} is a given positive sequence converging monotonically to zero. Consider a stochastic approximation algorithm x/sub n+1/=x/sub n/-a/sub n/(A/sub n/x/sub n/-b/sub n/)+a/sub n/e/sub n/, where {x/sub n/} is the iterate sequence, {a/sub n/} is the step size sequence, {e/sub n/} is the noise sequence, and x* is the desired zero of the function f(x)=Ax-b. Then, under appropriate assumptions, we show that x/sub n/-x*=o(/spl rho//sub n/) if and only if the sequence {e/sub n/} satisfies one of five equivalent conditions. These conditions are based on well-known formulas for noise sequences: Kushner and Clark's (1978) condition, Chen's (see Proc. IFAC World Congr., p.375-80, 1996) condition, Kulkarni and Horn's (see IEEE Trails Automat. Contr., vol.41, p.419-24, 1996) condition, a decomposition condition, and a weighted averaging condition. Our necessary and sufficient condition on {e/sub n/} to achieve a convergence rate of {/spl rho//sub n/} is basically that the sequence {e/sub n///spl rho//sub n/} satisfies any one of the above five well-known conditions. We provide examples to illustrate our result. In particular, we easily recover the familiar result that if a/sub n/=a/n and {e/sub n/} is a martingale difference process with bounded variance, then x/sub n/-x*=o(n/sup -1/2/(log(n))/sup /spl beta//) for any /spl beta/>1/2.