We present a general model for tracking smooth trajectories of multiple targets in complex data sets, where tracks potentially cross each other many times. As the number of overlapping trajectories grows, exploiting smoothness becomes increasingly important to disambiguate the association of successive points. However, in many important problems an effective parametric model for the trajectories does not exist. Hence we propose modeling trajectories as independent realizations of Gaussian processes with kernel functions which allow for arbitrary smooth motion. Our generative statistical model accounts for the data as coming from an unknown number of such processes, together with expectations for noise points and the probability that points are missing. For inference we compare two methods: A modified version of the Markov chain Monte Carlo data association (MCMCDA) method, and a Gibbs sampling method which is much simpler and faster, and gives better results by being able to search the solution space more efficiently. In both cases, we compare our results against the smoothing provided by linear dynamical systems (LDS). We test our approach on videos of birds and fish, and on 82 image sequences of pollen tubes growing in a petri dish, each with up to 60 tubes with multiple crossings. We achieve 93% accuracy on image sequences with up to ten trajectories (35 sequences) and 88% accuracy when there are more than ten (42 sequences). This performance surpasses that of using an LDS motion model, and far exceeds a simple heuristic tracker.

We present a method for automatically aligning words to image regions that integrates specific object classifiers (e.g., "car" detectors) with weak models based on appearance features. Previous strategies have largely focused on the latter, and thus have not exploited progress on object category recognition. Hence, we augment region labeling with object detection, which simplifies the problem by reliably identifying a subset of the labels, and thereby reducing correspondence ambiguity overall. Comprehensive testing on the SAIAPR TC dataset shows that principled integration of object detection improves the region labeling task.

A methodology and algorithm are presented that generate motions imitating the way humans complete a lifting task under various loading conditions. The path taken depends on "natural" parameters: the figure geometry, the given load, the final destination, and, especially, the strength model of the agent. Additional user controllable parameters of the motion are the comfort of the action and the perceived exertion of the agent. The algorithm uses this information to incrementally compute a motion path of the end-effector moving the load. It is therefore instantaneously adaptable to changing force, loading, and strength conditions. Various strategies are used to model human behavior (such as reducing moment, pull back, add additional joints, and jerk) that compute the driving torques as the situation changes. The strength model dictates acceptable kinematic postures. The resulting algorithm offers torque control without the tedious user expression of driving forces under a dynamics model. The algorithm runs in near-realtime and offers an agent-dependent toolkit for fast path prediction. Examples are presented for various lifting tasks, including one-and two-handed lifts, and raising the body from a seated posture.