Redundancy in deep neural network (DNN) models has always been one of their most intriguing and important properties. DNNs have been shown to overparameterize, or extract a lot of redundant features. In this work, we explore the impact of size (both width and depth), activation function, and weight initialization on the susceptibility of deep neural network models to extract redundant features. To estimate the number of redundant features in each layer, all the features of a given layer are hierarchically clustered according to their relative cosine distances in feature space and a set threshold. It is shown that both network size and activation function are the two most important components that foster the tendency of DNNs to extract redundant features. The concept is illustrated using deep multilayer perceptron and convolutional neural networks on MNIST digits recognition and CIFAR-10 dataset, respectively.

This paper proposes a new and efficient technique to regularize the neural network in the context of deep learning using correlations among features. Previous studies have shown that oversized deep neural network models tend to produce a lot of redundant features that are either the shifted version of one another or are very similar and show little or no variations, thus resulting in redundant filtering. We propose a way to address this problem and show that such redundancy can be avoided using regularization and adaptive feature dropout mechanism. We show that regularizing both negative and positive correlated features according to their differentiation and based on their relative cosine distances yields network extracting dissimilar features with less overfitting and better generalization. This concept is illustrated with deep multilayer perceptron, convolutional neural network, sparse autoencoder, gated recurrent unit, and long short-term memory on MNIST digits recognition, CIFAR-10, ImageNet, and Stanford Natural Language Inference data sets.

The survey outlines and compares popular computational techniques for quantitative description of shapes of major structural parts of the human brain, including medial axis and skeletal analysis, geodesic distances, Procrustes analysis, deformable models, spherical harmonics, and deformation morphometry, as well as other less widely used techniques. Their advantages, drawbacks, and emerging trends, as well as results of applications, in particular, for computer-aided diagnostics, are discussed.

To estimate the number of unimodal components in a mixture model of a marginal probability distribution of signals while learning the model with a conventional Expectation-Maximization (EM) algorithm, a modification of the well-known Akaike information criterion (AIC) called the modified AIC (mAIC), is proposed. Embedding the mAIC into the EM algorithm allows us to exclude sequentially, one-by-one, the least informative components from their initially excessive, or over-fitting set. Experiments on modeling empirical marginal signal distributions with mixtures of continuous or discrete Gaussians in order to describe the visual appearance of synthetic phantoms and real medical 3D images (lung CT and brain MRI) demonstrate a marked and monotone increase of the mAIC towards its maximum at the proper number that is known for the synthetic phantom or practically justified for the real image. These results confirm the accuracy and robustness of the proposed automated mAIC-EM based learning.

NASA is supposed to implement a sustainable and affordable human and robotic program to explore the solar system and beyond as it is the first goal of The Presidents Vision for U.S. Space Exploration. The robotic exploration across the solar system consists of exploring Jupiters moons, asteroids and other bodies to search for evidence of life, and to understand the history of the solar system. Trajectory generation for a robotic vehicle is an essential part of the total mission planning. To save energy by exploiting possible situation such as wind will assist a robotic explorer extend its life span and perform tasks more reliably. In this paper, we propose to utilize Nonlinear Trajectory Generation (NTG) methodology to generate 3D opportunistic trajectories for an Aerobot by exploiting wind. The Aerobot is dynamically controlled by three propellers which are respectively parallel to the local three Cartesian axes. Constraints for the Aerobot control are derived from Euler-Lagrange equations since the Aerobot satisfies with the Lagrange-D'Alembert principle. The new proposed Aerobot model takes the aerodynamics into account. The results show that NTG can take the advantage of wind profiles to save significant energy for the defined goal.

The 40th anniversary of Apollo 11 project with man landing on the moon reminds the world again by what science and engineering can do if the man is determined to do. However, a huge step can only be achieved step by step which may be relatively small at the beginning. Robotic exploration can provide necessary information needed to do the further step safely, with less cost, more conveniently. Trajectory generation for a robotic vehicle is an essential part of the total mission planning. To save energy by exploiting possible resources such as wind will assist a robotic explorer extend its life span and perform tasks more reliably. In this paper, we propose to utilize Nonlinear Trajectory Generation (NTG) methodology to generate energy efficient trajectores for the JPL Aerobot by exploiting wind. The Aerobot model is decoupled into longitudinal and lateral dynamics with control inputs as elevator deflection δe, thrust demand δT, vectoring angle δvfor the longitudinal motion, aileron deflection δa, rudder deflection δrfor the lateral motion. The outputs are the velocities and orientation of the Aerobot. The Aerobot state space model parameters are obtained from experimental identification on AURORA Airship since the actual JPL Aerobot is similar to the AURORA Airship. In this paper, the results show that with the state-space model, the proposed trajectory generation method can guide the Aerobot to take advantage of previously known wind profile to generate an energy-efficient trajectory.

This paper presents a detailed procedure to apply newly-proposed DMOC (Discrete Mechanics and Optimal Control) methodology to solve optimal control problems. DMOC is based on a direct discretization of Lagrange-d'Alembert principle for a system. First, this tutorial explains the principle of DMOC, and how to formulate the problem in DMOC. Next the steps are shown about how to install and configure nonlinear programming solver IPOPT, and how to use the modeling language AMPL. In particular, the user-defined function is involved with AMPL to solve a more complicated problem. Furthermore, a glider example is provided in this tutorial to solve optimal control problem with the user-defined 2D time-varying B-spline ocean current model. The ocean current original data was collected by HF-Radar stations located around Monterey Bay, CA in August 2000. Practically, this tutorial is shown how to use DMOC to solve optimal control problems with IPOPT and AMPL as the components. The possible users are robotic researchers, control system engineers, operations management researchers, and so on.

Discrete Mechanics and Optimal Control (DMOC) is newly developed methodology to solve the optimal control problems of mechanical systems. It is based on a direct discretization of the Lagrange-d'Alembert principle. The constraints are modeled as Euler-Langrange equations for discrete states. With the application of DMOC, a new real-time trajectory generation approach is proposed. The formulated problem is solved in the open source free nonlinear programming solver IPOPT (Interior Point OPTimizer). Superior performance of DMOC is verified through a standard forced VanDerPol Oscillator example. It is shown DMOC is much faster than NTG (Nonlinear Trajectory Generation) in computation time, it is 1.31% of the CPU time. Finally, for real-time trajectory generation, a dynamical glider example is shown as one of its possible application fields.

The problem of showing that Lagrangian Coherent Structures (LCS) are useful in determining near optimal trajectories for autonomous underwater vehicles (AUVs) known as gliders is investigated. This paper extends our preliminary results in couple ways. First, the ocean current flows are modeled by 3D B-spline functions in which the input variables are latitude, longitude and time, and the output variable is the ocean current velocity in 2D. The 3D (2D and time-varying) B-spline model of the ocean current is utilized in the Nonlinear Trajectory Generation (NTG) algorithm to find the optimal trajectory of the glider. The trajectories found using the 2D and time-varying B-spline ocean flows model are compared with the trajectories from 2D B-spline model in which the time is assumed to be constant in the ocean current model. In the second part of the paper, the dynamical glider model is established and controlled by gyroscopic forces rather than the simplified kinematic glider model as in the previous work. Finally, numerical solutions of several scenarios and animations of glider trajectories are presented. The results show that the 2D and time-varying B-spline ocean model not only can make the whole trajectory generating process much easier, but also the glider can reach the same destination in a comparable time and with much less energy than it does with the previous 2D ocean current model for both the kinematic and dynamic glider models. Both of the kinematic and dynamic glider optimal trajectories, successfully generated with 2D and time varying ocean current B-spline models, are shown to correspond to LCS.