Parkinson's disease (PD) is the second most prevalent neurodegenerative disorder in the world. PD is known to lead to marked alterations in cortico-thalamo-basal ganglia activity and subsequent movements, which may provide a biomarker for PD diagnosis. Delay differential analysis (DDA) is a time domain analysis framework based on embedding theory in nonlinear dynamics. An embedding reveals the nonlinear invariant properties of an unknown dynamical system (here the brain) from a single time series electroencephalography (EEG) or behavioral signals. The DDA models serve as a low-dimensional nonlinear functional basis onto which the data are mapped. The combination of behavioral and neurological observations gives rise to a multimodal analysis framework that could improve our understanding and classification of PD using time series data from physical systems. We demonstrate how 750 ms of multimodal data can be used to improve DDA classification performance of PD, over clean EEG or behavioral time series data on their own, in two distinct virtual reach to grasp tasks in an uncertain and dynamic virtual reality environment. Thus, multimodal DDA may provide a tool for aiding the clinician in the diagnosis of PD and bolster classification performance through the combination of a wide array of neural or behavioral signals.

We propose a time-domain approach to detect frequencies, frequency couplings, and phases using nonlinear correlation functions. For frequency analysis, this approach is a multivariate extension of discrete Fourier transform, and for higher-order spectra, it is a linear and multivariate alternative to multidimensional fast Fourier transform of multidimensional correlations. This method can be applied to short and sparse time series and can be extended to cross-trial and cross-channel spectra (CTS) for electroencephalography data where multiple short data segments from multiple trials of the same experiment are available. There are two versions of CTS. The first one assumes some phase coherency across the trials, while the second one is independent of phase coherency. We demonstrate that the phase-dependent version is more consistent with event-related spectral perturbation analysis and traditional Morlet wavelet analysis. We show that CTS can be applied to short data windows and yields higher temporal resolution than traditional Morlet wavelet analysis. Furthermore, the CTS can be used to reconstruct the event-related potential using all linear components of the CTS.

The future is uncertain because some forthcoming events are unpredictable and also because our ability to foresee the myriad consequences of our own actions is limited. Here we studied how humans select actions under such extrinsic and intrinsic uncertainty, in view of an exponentially expanding number of prospects on a branching multivalued visual stimulus. A triangular grid of disks of different sizes scrolled down a touchscreen at a variable speed. The larger disks represented larger rewards. The task was to maximize the cumulative reward by touching one disk at a time in a rapid sequence, forming an upward path across the grid, while every step along the path constrained the part of the grid accessible in the future. This task captured some of the complexity of natural behavior in the risky and dynamic world, where ongoing decisions alter the landscape of future rewards. By comparing human behavior with behavior of ideal actors, we identified the strategies used by humans in terms of how far into the future they looked (their "depth of computation") and how often they attempted to incorporate new information about the future rewards (their "recalculation period"). We found that, for a given task difficulty, humans traded off their depth of computation for the recalculation period. The form of this tradeoff was consistent with a complete, brute-force exploration of all possible paths up to a resource-limited finite depth. A step-by-step analysis of the human behavior revealed that participants took into account very fine distinctions between the future rewards and that they abstained from some simple heuristics in assessment of the alternative paths, such as seeking only the largest disks or avoiding the smaller disks. The participants preferred to reduce their depth of computation or increase the recalculation period rather than sacrifice the precision of computation.

The relationship between movement kinematics and human brain activity is an important and fundamental question for the development of neural prosthesis. The peak velocity and the peak acceleration could best reflect the feedforward-type movement; thus, it is worthwhile to investigate them further. Most related studies focused on the correlation between kinematics and brain activity during the movement execution or imagery. However, human movement is the result of the motor planning phase as well as the execution phase and researchers have demonstrated that statistical correlations exist between EEG activity during the motor planning and the peak velocity and the peak acceleration using grand-average analysis. In this paper, we examined whether the correlations were concealed in trial-to-trial decoding from the low signal-to-noise ratio of EEG activity. The alpha and beta powers from the movement planning phase were combined with the alpha and beta powers from the movement execution phase to predict the peak tangential speed and acceleration. The results showed that EEG activity from the motor planning phase could also predict the peak speed and the peak acceleration with a reasonable accuracy. Furthermore, the decoding accuracy of the peak speed and the peak acceleration could both be improved by combining band powers from the motor planning phase with the band powers from the movement execution.

Human performance approaches that of an ideal observer and optimal actor in some perceptual and motor tasks. These optimal abilities depend on the capacity of the cerebral cortex to store an immense amount of information and to flexibly make rapid decisions. However, behavior only approaches these limits after a long period of learning while the cerebral cortex interacts with the basal ganglia, an ancient part of the vertebrate brain that is responsible for learning sequences of actions directed toward achieving goals. Progress has been made in understanding the algorithms used by the brain during reinforcement learning, which is an online approximation of dynamic programming. Humans also make plans that depend on past experience by simulating different scenarios, which is called prospective optimization. The same brain structures in the cortex and basal ganglia that are active online during optimal behavior are also active offline during prospective optimization. The emergence of general principles and algorithms for goal-directed behavior has consequences for the development of autonomous devices in engineering applications.

A PC based desktop Virtual Reality system was developed for rehabilitating hand function in stroke patients. The system uses two hand input devices, a CyberGlove and a RMII force feedback glove, to allow the user to interact with one of four rehabilitation exercises. Each of which is designed to exercise one specific parameter of hand movement, namely range, speed, fractionation or strength. The therapy program is semi-automated and personalized to each user through the use of performance-based target levels. These are adapted between sessions in order to induce the user to improve. Feedback is provided to each user throughout the exercise sessions. To further motivate the user to continue the exercise program, screen displays are designed as interactive games. The system is described and sample data is presented from preliminary studies performed on control subjects.

The essential grammatical information of American Sign Language (ASL) is conveyed through changes in the movement and spatial contouring of the hands and arms. An interactive computer graphic system is described for the analysis and modeling of sign language movement. This system consists of four components. The first component reconstructs actual movements in three dimensions and allows the user to interactively segment and transform the data for later analysis. The second component allows a user to interactively create synthetic signs by specifying angle functions in a jointed model. The third component provides a novel technique for manipulating movement quality independently of spatial path. The fourth component allows the building of complex stimuli and real-time stimulus sequencing for psycholinguistic experiments. The emphasis is on interactive techniques and data structures that allow analysis and modeling of the complex hand and arm movements of American Sign Language.