Conventional scalable compressive video sampling cannot sufficiently exploit the structured sparsity within video sequences. This paper proposes a novel quality scalable structured compressive video sampling (SS-CVS) framework with hierarchical subspace learning to support video transmission over heterogeneous network conditions. The proposed framework incorporates the union of data-driven subspaces (UoDS) model to introduce structured sparsity into sensing matrix for progressive acquisition of measurements. Hierarchical subspace learning is developed to generate the subspaces with their bases based on adaptive subspace clustering in a progressive fashion. Consequently, two hierarchical structures are derived to enable progressive mapping of subspaces and bases on the structured sensing matrices to support quality scalability. To guarantee the convergence of hierarchical subspace learning, the training set is updated with the adaptive group set from the reconstructed reference frames. It is demonstrated that SS-CVS can guarantee stable recovery for each quality layer under the constraints of block restricted isometric property (RIP). Furthermore, the proposed SS-CVS model is generalized to tensor subspaces for scalable compressive sampling of high-dimensional signals. Experimental results demonstrate that the proposed algorithm can achieve quality scalabilities with an improved reconstructed performance in comparison to the state-of-the-art scalable compressive video sampling methods.

Due to the popularity of 3D videos, 2D-to-3D video conversion has become a hot research topic for the past few years. The most critical issue in 3D video synthesis is the estimation of depth maps for the video frames. Numerous efforts have been devoted in fully automatic and semi-automatic depth estimation approaches, although the discontinuity of depth field and the ambiguity of motion boundary are still the main challenges in depth estimation. This paper proposes a semi-automatic structure-aware sparse-to-dense depth estimation method, which leverages the tensor voting at two different levels to propagate depth across frames. In the first level, a 4D tensor voting is performed to remove outliers caused by inaccurate motion estimation. Noticing that the 4D tensors of correctly matched points should lie on the smooth layer in the manifold, we utilize the variety saliency defined by the eigen-system of the tensor for outlier removal. In the second level, a high-dimensional tensor voting algorithm, incorporating spatial location, motion, and color into the tensor representation, is devised to propagate the depth from the sparse points to the entire image domain. By projecting the input feature into the tangent space, the relation between the location, motion, color, and the depth can be established by voting process. Extensive experiments on public data set validate the effectiveness of the proposed method in comparison with state-of-the-art depth estimation approaches.

Transform sensing is proposed to sense the space through certain beam patterns designed for a transformation basis. The received signals will be the result of transformation instead of the original raw data. In doing so, the process of generating an arbitrary beam pattern on the phased array and multiple-input multiple-output is described. We further propose a transform-sensing mechanism using wavelets, and compare it with traditional sensing methods. Simulations and experiments demonstrate how to generate the transmission patterns and sense through the transform-sensing mechanism. The results show that transform sensing obtains high-resolution data on the target area, while spending less time on nontarget ones that need low resolutions. In this way, the new sensing mechanism generates a multiresolution result, which balances resolution and sensing efficiency.

This paper presents a transductive multicomponent video segmentation algorithm, which is capable of segmenting the predefined object of interest in the frames of a video sequence. To ensure temporal consistency, a temporal coherent parametric min-cut algorithm is developed to generate segmentation hypotheses based on visual cues and motion cues. Furthermore, each hypothesis is evaluated by an energy function from foreground resemblance, foreground/background divergence, boundary strength, and visual saliency. In particular, the state-of-the-art R-convolutional neural network descriptor is leveraged to encode the visual appearance of the foreground object. Finally, the optimal segmentation of the frame can be attained by assembling the segmentation hypotheses through the Monte Carlo approximation. In particular, multiple foreground components are built to capture the variances of the foreground object in shapes and poses. To group the frames into different components, a tree-structured graphical model named temporal tree is designed, where visually similar and temporally coherent frames are arranged in branches. The temporal tree can be constructed by iteratively adding frames to the active nodes by probabilistic clustering. In addition, each component, consisting of frames in the same branch, is characterized by a support vector machine classifier, which is learned in a transductive fashion by jointly maximizing the margin over the labeled frames and the unlabeled frames. As the frames from the same video sequence follow the same distribution, the transductive classifiers achieve stronger generalization capability than inductive ones. Experimental results on the public benchmarks demonstrate the effectiveness of the proposed method in comparison with other state-of-the-art supervised and unsupervised video segmentation methods.

Massive visual traffic data have become available recently. Though it opens the realm of intelligent traffic analysis, processing the data in a timely manner is difficult yet critical to time sensitive decisions, which are typical to traffic related management. In this paper, we study time-bounded aggregation analytics on large visual traffic data including traffic images and videos. We first find that current MapReduce framework can not work well due to two challenges: first, significant dual diversities exist on data distributions and processing time; second, apriori knowledge on these distributions and time costs are not always available. However, we also observe spatial and temporal locality on data values and processing time. Based on the examination, we design Traffic At-a-Glance (TaG), an augmented MapReduce framework for time-bounded traffic analytics jobs. Particularly, we propose a novel sampling algorithm that exploits traffic data localities and stratifies samples based on data distributions and processing time. It runs in an iterative, adaptive manner without apriori knowledge. Moreover, we propose a heuristic scheduling algorithm with considerations of batch processing overhead. Further, we refine the load balancing mechanism based on data processing time locality to respect job time bounds. In addition, we extend TaG to well handle traffic videos by sampling video data based on motion information encoded in the videos. We implement TaG on Hadoop and conduct extensive experiments on a large visual traffic dataset. The evaluations on different data sizes show TaG is able to achieve high accuracy within time bounds.

Many mobile robots are expected to work for or interact with humans indoors in applications such as guided shopping, policing, and senior care. Mobile robots' sensors alone are insufficient in order to realize these applications, infrastructural support is needed. Existing support for mobile robots requires heavy or expensive infrastructures with limited scalability or deployment of unsightly lines or magnetic strips. This paper presents S-Mirror, a novel approach that "reflects" various ambient signals towards mobile robots, greatly extending their sensing abilities. S-Mirror forms a network of S-Mirror nodes that mainly reflect visual signals (as well as electronic and acoustic signals) to assist mobile robots. To illustrate the advantages of S-Mirror, we develop a localization approach for mobile robots that integrates S-Mirror and robots' on-board motion sensors. We implement S-Mirror and a mobile robot prototype on commercial off-the-shelf hardware. Our real-world experimental validation shows that S-Mirror achieves accurate timely localization with low network bandwidth consumption as well as robustness and scalability to many mobile robots.

In the past few decades, we have witnessed the success of bag-of-features (BoF) models in scene classification, object detection, and image segmentation. Whereas it is also well acknowledged that the limitation of BoF-based methods lies in the low-level feature encoding and coarse feature pooling. This paper proposes a novel scene classification method, which leverages several semantic codebooks learned in a multitask fashion for robust feature encoding, and designs a context-aware image representation for efficient feature pooling. Apart from conventional universal codebook learning approaches, the proposed method encodes each class of local features with a unique semantic codebook, which captures the distinct distribution of different semantic classes more effectively. Instead of learning each semantic codebook separately, we learn a compact global codebook, of which each semantic codebook is a sparse subset, with a two-stage iterative multitask learning algorithm. While minimizing the clustering divergence, the semantic codeword assignment is solved by submodular optimization simultaneously. Built upon the global and semantic codebooks, a context-aware image representation is further developed to encode both global and semantic features in image representation via contextual quantization, semantic response computation, and semantic pooling. Extensive experiments have been conducted to validate the effectiveness of the proposed method on various public benchmarks with several popular local features.

The rapid growth of web images presents new challenges as well as opportunities to the task of image understanding. Conventional approaches rely heavily on fine-grained annotations, such as bounding boxes and semantic segmentations, which are not available for web-scale images. In general, images over the Internet are accompanied with descriptive texts, which are relevant to their contents. To bridge the gap between textual and visual analysis for image understanding, this paper presents an algorithm to learn the relations between scenes, objects, and texts with the help of image-level annotations. In particular, the relation between the texts and objects is modeled as the matching probability between the nouns and the object classes, which can be solved via a constrained bipartite matching problem. On the other hand, the relations between the scenes and objects/texts are modeled as the conditional distributions of their co-occurrence. Built upon the learned cross-domain relations, an integrated model brings together scenes, objects, and texts for joint image understanding, including scene classification, object classification and localization, and the prediction of object cardinalities. The proposed cross-domain learning algorithm and the integrated model elevate the performance of image understanding for web images in the context of textual descriptions. Experimental results show that the proposed algorithm significantly outperforms conventional methods in various computer vision tasks.

Following a person is a fundamental requirement for human-robot interaction. In this paper we propose a novel tracking approach for robust human feet tracking which integrates human locomotion into tracking algorithms. The vertical displacement between the two feet is analyzed and we observe that this displacement during the walking cycle is close to a modulated cosine waveform. Based on this, we propose an adaptive model for the human walking pattern. We divide the motion of the human feet into local motion and global motion. The local motion is modeled by a modified cosine wave that updates along time. Global motion is estimated by the continuity between successive frames. This model is combined with particle filtering to guide the searching of the feet. A 2D Gaussian mask is generated according to the predicted position estimated by the motion model and used to modify the weight of the particles. Experiments are implemented in several human walking videos and the algorithm is evaluated against the generic particle filtering method. Results show that the feet can be tracked successfully with significant improvements compared to the generic particle filtering method.

This paper presents a method for the generation of dynamic walking gaits with a 3D Dual-SLIP model and its application to a simulated Hubo+ based humanoid. Previous approaches with the Dual-SLIP model have only focused on the planar case, wherein self-stable gaits can be found. When extended to 3D here, this model has not been found to exhibit self-stable gaits, requiring new methods for gait optimization and control. By taking advantage of a newly discovered symmetry condition for the Dual-SLIP model, this work proposes a quarter period (half step) optimization process to find periodic walking gaits in 3D. An LQR controller is developed to regulate the state of the model at leg midstance (MS) based on its return map dynamics. The Dual-SLIP model is extended by introducing a bio-inspired leg length actuation scheme in order to describe high-speed walking gaits (up to 2 m/s for human-compatible parameters). Finally, the CoM trajectory and footstep positions from the 3D Dual-SLIP are used as a reference in a task-space controller with a Hubo+ based humanoid model. By tracking these references, the methods successfully produce human-like dynamic walking gaits in simulation which are robust to disturbances. The whole-body control system for walking can handle uneven terrain with variation up to 10% of its leg length. This represents the first humanoid dynamic walking approach based on a 3D Dual-SLIP model.

Human tracking in video has many practical applications such as visual guided navigation, assisted living, etc. In such applications, it is necessary to accurately track multiple humans across multiple cameras, subject to real-time constraints. Despite recent advances in visual tracking research, the tracking systems purely relying on visual information fail to meet the accuracy and real-time requirements at the same time. In this paper, we present a novel accurate and real-time human tracking system called VM-Tracking. The system aggregates the information of motion (M) sensor on human, and integrates it with visual (V) data based on physical locations. The system has two key features, i.e. location-based VM fusion and appearance-free tracking, which significantly distinguish itself from other existing human tracking systems. We have implemented the VM-Tracking system and conducted comprehensive experiments on challenging scenarios.

The Dual-SLIP model has been proposed as a walking template that inherently encodes a rich set of human-like features. Previous work has used the 3D Dual-SLIP with bio-inspired leg actuation to generate a human-like dynamic walking gait over a wide range of speeds. The work presented in this paper extends the 3D Dual-SLIP walking strategy to uneven terrain. With nonlinear optimization based on a multiple-shooting formulation, actuated Dual-SLIP walking gaits over uneven terrain are identified that handle 1-step elevation changes up to ±10 cm. Moreover, this Dual-SLIP actuation strategy enables a constant center of mass (CoM) forward speed at leg midstance to be maintained. The resultant gaits have revealed a leg lengthening/shortening strategy that is similar to that adopted by a human when walking over prepared, uneven terrain. Results demonstrate that the CoM trajectories and ground reaction force patterns found with the approach are comparable to the human data found in the biomechanics literature. The trajectories generated by the Dual-SLIP model are also demonstrated to orchestrate a dynamic walking motion with an anthropomorphic humanoid model in simulation over uneven terrain.

This paper presents a novel video-based face recognition algorithm by using a sequential sampling and updating scheme, named sequential sample consensus. The proposed algorithm aims at providing a sequential scheme that can be applied to streaming video data. Different from existing approaches, the training video sequences serve as the sample space, and the person's identity in the testing sequence is characterized using an identity probability mass function (PMF) that is sequentially updated. For each testing frame, samples are randomly drawn from the sample space, and the numbers of samples for each identity are determined by the identity PMF. The testing frame is evaluated against the drawn samples to calculate the weights, and the sample weights are used for updating the identity PMF. Benefiting from the sampling procedure, the change in both the numbers and the weights of the samples for each individual leads to quick reaction of the algorithm. The proposed algorithm is robust against misclassification caused by pose variations, and sensitive to identity switching during recognition. The algorithm is evaluated using both public and self-made datasets, and shows better performance than other video-based face recognition approaches.

In this paper, a new radar waveform based on wavelet packets is proposed. This waveform shows significant advantages over conventional linear frequency modulated waveforms for range and velocity detections of moving targets. The proposed waveform can decouple its envelope and carrier for range and velocity estimations, respectively, because of which the Doppler spread term can be totally removed when estimating velocity and the Doppler effect ignored when estimating range. This way, the proposed waveform produces higher resolution in range or velocity detection. The proposed waveform can be applied to the synthetic aperture radar for improved performance.

This paper introduces a new approach for humanoid robot quasi-static walking for improving stability performance, which is called ski-type walking. By adding two canes held by hands, the supporting region and stability margin are enlarged in comparison with biped walking. We first study the mechanism of cane-assisted walking by human beings. Based on the study, we develop two ski-type gaits, Crawl_1 and Craw-2, respectively for the humanoid robot Hubo. The stability performance for the two configurations is compared, which leads to the adoption of the Crawl_2 gait. Furthermore the length of the canes is selected to support a feasible while stable Crawl_2 gait. Finally simulation and experiments are performed to verify the new ski-type gait.

Electronic channel sounding plays a vital role in developing wireless communication systems. It is critical for transceivers' equalization and filtering operations. However, current pure electronic channel sounding techniques are not well-suited for emerging scenarios such as opportunistic spectrum access, channel impulse response (CIR) based wireless positioning, and wireless security applications, which demand rapid, high-resolution, and spectrum agile channel measurements on commercial off-the-shelf (COTS) devices. To address these critical issues, this paper proposes EV-Sounding, a novel methodology for visual assisted electronic channel sounding. Based on frequency domain channel sounding, EV-Sounding leverages cameras for visual estimation of sparsity locations to reduce the number of frequency samples, thus speeding up the sounding process. EV-Sounding achieves both high-resolution CIR measurements and spectrum agility. We prototype an EV-Sounding system on COTS devices. Our real-world experimental results and extensive simulations validate EV-Sounding's performance.

Inherent statistical correlation for context-based prediction and structural interdependencies for local coherence is not fully exploited in existing lossless image coding schemes. This paper proposes a novel prediction model where the optimal correlated prediction for a set of pixels is obtained in the sense of the least code length. It not only exploits the spatial statistical correlations for the optimal prediction directly based on 2D contexts, but also formulates the data-driven structural interdependencies to make the prediction error coherent with the underlying probability distribution for coding. Under the joint constraints for local coherence, max-margin Markov networks are incorporated to combine support vector machines structurally to make max-margin estimation for a correlated region. Specifically, it aims to produce multiple predictions in the blocks with the model parameters learned in such a way that the distinction between the actual pixel and all possible estimations is maximized. It is proved that, with the growth of sample size, the prediction error is asymptotically upper bounded by the training error under the decomposable loss function. Incorporated into the lossless image coding framework, the proposed model outperforms most prediction schemes reported.

Detecting generic object categories in images and videos are a fundamental issue in computer vision. However, it faces the challenges from inter and intraclass diversity, as well as distortions caused by viewpoints, poses, deformations, and so on. To solve object variations, this paper constructs a structure kernel and proposes a multiscale part-based model incorporating the discriminative power of kernels. The structure kernel would measure the resemblance of part-based objects in three aspects: 1) the global similarity term to measure the resemblance of the global visual appearance of relevant objects; 2) the part similarity term to measure the resemblance of the visual appearance of distinctive parts; and 3) the spatial similarity term to measure the resemblance of the spatial layout of parts. In essence, the deformation of parts in the structure kernel is penalized in a multiscale space with respect to horizontal displacement, vertical displacement, and scale difference. Part similarities are combined with different weights, which are optimized efficiently to maximize the intraclass similarities and minimize the interclass similarities by the normalized stochastic gradient ascent algorithm. In addition, the parameters of the structure kernel are learned during the training process with regard to the distribution of the data in a more discriminative way. With flexible part sizes on scale and displacement, it can be more robust to the intraclass variations, poses, and viewpoints. Theoretical analysis and experimental evaluations demonstrate that the proposed multiscale part-based representation model with structure kernel exhibits accurate and robust performance, and outperforms state-of-the-art object classification approaches.

Nowadays, more and more objects can be represented with electronic identifiers, e.g., people can be recognized from their laptops' MACs, and products can be identified by their RFID numbers. Localizing electronic identifiers is more and more important for a fully digitalized life. However, traditional wireless localization techniques are not satisfactory in performance to determine these electronic identifiers' positions. Some of them require costly hardware to achieve high accuracy and, hence, are not practical. The others are inaccurate and not robust against environmental noises, e.g., RSSI-based localization. Therefore, an accurate and practical approach for localizing electronic identifiers is needed. In this paper, we propose a new localization technique called EV-Loc. In EV-Loc, we make use of visual signals to help improve the accuracy of wireless localization. Our technique fully takes advantage of the high accuracy of visual signals and pervasiveness of electronic signals. To effectively couple these two signals together, we have designed an E-V match engine to find the correspondence between an object's electronic identifier and its visual appearance. We have implemented our technique on mobile devices and evaluated it in the real world. The localization error is less than 1 m. We have also evaluated our approach using large-scale simulations. The results show that our approach is accurate and robust.

This paper presents a novel video-based face recognition algorithm using a sequential sampling and updating scheme, named sequential sample consensus (SSC). Different from the existing approaches, the training video sequences serve as the sample space, and the person's identity in the testing sequence is characterized by an identity probability mass function (PMF) that is sequentially updated. For each testing frame, samples are randomly drawn from the sample space with the numbers of samples for each identity determined by the identity PMF. The testing frame is evaluated against the drawn samples to calculate the weights, and the sample weights are utilized for updating the identity PMF. The proposed algorithm is robust against misclassification caused by pose variations, and sensitive to identity switching during recognition. The algorithm is evaluated using both public and self-made databases, and shows better performance than other video-based face recognition approaches.

The method of human-like handwriting and drawing is addressed with a three-link arm. Three strategies of trajectory planning are considered: the basic stroke method, the Bezier curve method, and the non-gradient numerical optimization method. Planar patterns of handwriting or drawing are converted into the vector form of joint angles in which sequence and speed of the movement can be designed to imitate human handwriting and drawing. A nonlinear three-link three-dimensional arm, similar to a human arm, is developed to track the planned trajectories. The feasibility of these methods is demonstrated by simulation.

Human localization is an enabling technology for many mobile applications. As more and more people carry mobile phones with them, we can now localize a person by localizing his mobile phone. However, it is observed that presence of human bodies introduces heavy interference to mobile phone signals. This has been one of the major causes of inaccurate wireless localization for humans. In this paper, we propose using video cameras to help estimate human body's interference on mobile device's signals. We combine human orientation detection and human/phone/AP relative position inference estimation to better measure how a human blocks or reflects wireless signals. We have also developed a signal distortion compensation model. Based on these technologies, we have implemented a human localization system called EV-Human. Real world experiments show that our EV-system can accurately and robustly localize humans.

This paper provides a novel side-view face authentication method based on discrete wavelet transform and random forest. A subset selection method that increases the number of training samples and allows subsets to preserve the global information is presented. The authentication method can be summarized to have the following steps: profile extraction, wavelet decomposition, subset splitting and random forest verification. The new method takes the advantage of wavelet's localization property in both frequency and spatial domains, while maintaining the generalized properties of random forest. The implementation of the proposed method is computationally feasible and the experimental results show that the performance is satisfactory. Future improvements are discussed in the paper.

Detecting multiple targets and obtaining a record of trajectories of identical targets that interact mutually infer countless applications in a large number of fields. However, it presents a significant challenge to the technology of object tracking. This paper describes a novel structured learning-based graph matching approach to track a variable number of interacting objects in complicated environments. Different from previous approaches, the proposed method takes full advantage of neighboring relationships as the edge feature in a structured graph, which performs better than using the node feature only. Therefore, a structured graph matching model is established, and the problem is regarded as structured node and edge matching between graphs generated from successive frames. In essence, it is formulated as the maximum weighted bipartite matching problem to be solved using the dynamic Hungarian algorithm, which is applicable to optimally solving the assignment problem in situations with changing edge costs or weights. In the proposed graph matching model, the parameters of the structured graph matching model are determined in a stochastic learning process. In order to improve the tracking performance, bilateral tracking is also used. Finally, extensive experimental results on Dynamic Cell, Football, and Car sequences demonstrate that the new approach effectively deals with complicated target interactions.

Modeling complicated dynamics of humanoids is considered in this paper. It is proposed that the three-link module is a pivotal element in formulation of high-dimensional humanoids. The equations of motion of the module are first derived by projection of the free body equations of motion onto the space of the translation of an arbitrary point and Bryant angles and angular velocities. It is then shown that the formulation can be used as a construction module for systems with a larger number of segments. Holonomic and nonholonomic constraints are included to show the feasibility and versatility for representing other parts of the humanoid system. Digital computer simulations are presented to test the module's formulation and demonstrate its behavior in two maneuvers, jumping and sway.

Video cameras have been deployed at almost every critical location, and they keep generating huge volumes of video data. The current visual processing technologies are not efficient in handling all these data for surveillance purposes, and a large amount of human power is needed to process them. In this paper, we propose the E-V system, which uses electronic footprints to help sort through this swamp of data. Electronic footprints are wireless signals emitted by mobile devices carried by people. They are ubiquitous and amenable to collection and indexing. We study how to use electronic footprints to help quickly and accurately identify object's appearance model from large volumes of video data. We have formulated the problem and provided efficient algorithms to achieve the identification on large data sets. Real world experiments and large-scale simulations have been done, which confirms the feasibility and efficiency of the proposed algorithms.

Nowadays, an increasing number of objects can be represented by their wireless electronic identifiers. For example, people can be recognized by their phone numbers or their phones' WiFi' MAC addresses and products can be identified by their RFID numbers. Localizing objects with electronic identifiers is increasingly important as our lives become increasingly "digitalized". However, traditional wireless localization techniques cannot meet the fast growing needs of accurate and cost efficient localization. Some of these techniques require expensive hardware to achieve high accuracy, which is impractical for massive deployment. Others, such as WiFi RSSI based localization, are inaccurate and not robust to environmental noise. In this paper, we propose a new localization technique called EV-Loc. In EV-Loc, we use visual signals to help improve the accuracy of wireless localization. Our technique fully leverages visual signals' high accuracy and electronic signals' pervasiveness. To effectively couple these two signals, we design an E-V match engine to find the correspondence between an object's electronic identifier and its visual appearance. We implement our technique on mobile devices and evaluate it in real-world scenarios. The localization error is less than 1 m. We also evaluate our approach using large scale simulations. The results show that our approach is accurate and robust.

We present a novel visual tracking method for measuring the speed of a moving vehicle within a structured environment using stationary stereo cameras. In the proposed method, visual stereo tracking and motion estimation in 3-D are integrated within the framework of particle filtering. The visual tracking processes in the two views are coupled with each other since they are dependent upon the same 3-D motion and correlated in the observations. Considering that the vehicle's motion is physically constrained by the environment, we further utilize the path constraint reconstructed from stereo views to reduce the uncertainty about the vehicle's motion and improve the accuracy for both tracking and speed measuring. The proposed method overcomes the challenges arising from the limitation of depth accuracy in a long-range stereo, and the experiments on the synthetic and real-world sequences have demonstrated its effectiveness and accuracy in both the tracking performance and the speed measurement.

Neurofilaments are long flexible cytoplasmic protein polymers that are transported rapidly but intermittently along the axonal processes of nerve cells. Current methods for studying this movement involve manual tracking of fluorescently tagged neurofilament polymers in videos acquired by time-lapse fluorescence microscopy. Here, we describe an automated tracking method that uses particle filtering to implement a recursive Bayesian estimation of the filament location in successive frames of video sequences. To increase the efficiency of this approach, we take advantage of the fact that neurofilament movement is confined within the boundaries of the axon. We use piecewise cubic spline interpolation to model the path of the axon and then we use this model to limit both the orientation and location of the neurofilament in the particle tracking algorithm. Based on these two spatial constraints, we develop a prior dynamic state model that generates significantly fewer particles than generic particle filtering, and we select an adequate observation model to produce a robust tracking method. We demonstrate the efficacy and efficiency of our method by performing tracking experiments on real time-lapse image sequences of neurofilament movement, and we show that the method performs well compared to manual tracking by an experienced user. This spatially constrained particle filtering approach should also be applicable to the movement of other axonally transported cargoes.

Summary form only given. In this paper, we propose an optimized in painting-based macro block(MB) prediction mode (IP-mode) in the state-of-the-art H.264/AVC video compression engine, and belief propagation (BP) is applied to achieve the global spatio-temporal consistency between the predicted content and the co-located known region. To decrease the computing complexity of the iterative BP algorithm, we explore structure and motion features by tensor votes projected from the decoded regions, to assign the priority of message scheduling and prune the intolerable labels. No side information is need to be coded into the bit stream, while the structure and motion information is estimated from the decoded region at decoder side. Compared with the existing prediction modes in H.264/AVC, the proposed IP-mode only encode the macro block header and residual data, where the residual is lighter in homogeneous texture regions by the optimized BP algorithm with label pruning. Experiments validate that the proposed video compression scheme can achieve a better R-D performance, and the computing complexity is largely reduced through the inference of structure and motion features.

To correctly detect dynamic targets and obtain a record of the trajectories of identical targets in appearance over time, has become significantly more challenging and infers countless applications in biomedicine. In this paper, we propose a novel structured learning-based graph matching algorithm to track a variable number of interacting objects in dynamic environments. Different from previous approaches, the proposed method takes full advantage of neighboring relationships as edge feature in the structured graph. The target problem is regarded as structured node and edge matching between graphs generated from successive frames. In essence, it is formulated as the maximum weighted bipartite matching problem which is solved by dynamic Hungarian algorithm. The parameters of the structured graph matching model can be acquired in a stochastic graduated learning step in different dynamic environments. The extensive experiments on dynamic cell and football sequences demonstrate that the resulting approach deals effectively with complicated target interactions.

Stereo vision provides us the ability to perceive objects in 3D. By continuously tracking an object in stereo videos, the information about object motion can be acquired. In this paper, we propose a novel method which unifies the tracking and speed measuring within the Bayesian filtering framework and solves this problem using particle filtering approach. Vehicle speed measuring is studied as a specific example. Environment constraints are integrated into the tracking for reducing the uncertainty to achieve more robust and accurate results. Experiment results on both synthetic and real traffic sequences demonstrate the accuracy and effectiveness of our proposed method.

With the ever increasing concern of vision-based video analysis and coding over resource-limited systems, this paper proposes a novel video coding scheme that exploits low- quality video data and formulates as an inverse learning based video reconstruction from online training by diverse stochastic processes. Given a sparsely sampled incomplete data, the intrinsic nonlocal and spatio-temporal geometric regularity related to online training examples in the key frames are considered as a state-dependent uncertainty estimation problem using Gaussian Process (GP) regression. Unlike non-parametric or exemplar- based sampling methods, we consider non-parametric system models for sequential state estimation by using the Unscented Kalman Filter (UKF) as the state estimator. It inherits the unscented transform for linearization to the transition function and the observation function. Once an approximate motion and observation model is available, it can naturally be incorporated to make a further performance improvement.

This paper presents a new video compression framework over H.264/AVC scheme that integrates our proposed structured priority belief propagation (BP)-based inpainting prediction (IP) to exploit the intrinsic nonlocal and geometric regularity in video samples. Unlike the existing edge-based inpainting adopted in lossy image coding, the optimal predictor could maintain the pixel-wise fidelity and the robust error resilience without any assistant information. Beyond the local prediction limitation of traditional intra and inter-modes, the priority BP with regularized structure priors of a spatio-temporal Markov random field is imposed on the predictor in an adaptive and more convergent sense. Specifically, the structured sparsity of the predicted macroblock region is inferred by tensor voting projected from the co-located decoded regions. In turn, the priority and visiting order of nodes are assigned according to the sets of updated beliefs as the propagation of messages. Through relatively few iterations of forward and backward process, the sparse inference of priority BP would ensure a stable marginal belief distribution on the structure and texture through updating local messages and beliefs. Within the optimal mode selection on rate-distortion optimization (RDO), the IP-mode with structured priority BP outperforms the existing vision-based approaches, and specially achieves a better objective rate-distortion performance besides visual quality. The IP-mode with structured priority BP can be applied to both I and P frames to generate low entropy residue, e.g., homogeneous visual patterns, and the computation complexity is also competitive with one iteration of sparse inference. Moreover, it behaves more resilient with an intrinsic probabilistic inference than the intra and inter-modes.

Although wavelet-based scalable video coding becomes the state-of-the-art video compression engine for its adaptability to heterogeneous networks and clients, a large number of attempts have been made to integrate local directionality onto discrete wavelet transform to explore the intrinsic geometrical structures. Taking into consideration that the contours and textures scattered in different scales change their directional resolutions as their curvatures change, we investigate adaptive directional resolutions along scales to achieve the dual (scale and orientation) multiresolution transform. This paper proposes nonuniform directional frequency decompositions for video representation and approximation, and exploits the nonuniformity of orientation multiresolution distribution and designs nonuniform directional filter banks to make the geometrical transform more sparse and efficient. The nonuniform directional frequency decomposition under arbitrary scales is fulfilled by a non-symmetric binary tree (NSBT) topology structure with nonuniform directional filterbank design. In turn, the proposed scalable video coding framework, called DMSVC, is enriched with the dual multiresolution transform. Each temporal subband through motion compensated temporal filtering is further decomposed into multiscale subbands, and the highpass wavelet subspaces are divided into an arbitrary number of directional subspaces in alignment with the orientation distribution via phase congruency to establish NSBT. The paraunitary perfect reconstruction condition is provided through a polyphase identical form of filter bank. Comparing with the isolated wavelet basis, our transform provides a greater correlated set of localized and anisotropic basis functions. The spatio-temporal subband coefficients are coded by a 3-D ESCOT entropy coding algorithm which is adopted to match the structure of NSBT. Experimental results show that the reconstructed video frames DMSVC in the proposed DMSVC scheme have better visual quality than existing scalable video coding schemes. It could produce higher compression ratio on video sequences full of directional edges and textures.

This paper develops a complete framework for perceptual video coding with anisotropic diffusion-based abstraction and completion under the spatio-temporal variation regularity. A soft clustering method is applied to retrieve transferable semantic implications of sampled frames in a sparser way. The restoration inference process as a learning equivalent optimization problem from a given set of sparse data, serves as non-parametric or exemplar-based sampling method by taking 3-D spatio-temporal similarity into consideration. Aside from pixel-wise color-based matching, two patch-based similarity metrics on motion-based pixel-wise similarity and semantic coherence are introduced for the exemplar-based reconstruction under complex situations. For each pixel in the abstracted frames, a set of matched patches will provide inference and its color is predicted with multi-hypothesis via weighted averaging. A pixel-wise confidence map based on spatio-temporal feature is also provided for important point selection so as to reduce the computation cost to an acceptable level. We validate both compression efficiency and restoration performance from coding gain, SSIM, optical flow consistency, and just-noticeable distortion (JND) on a variety of sources.

We present a novel tracking method for effectively tracking objects in structured environments. The tracking method finds applications in security surveillance, traffic monitoring, etc. In these applications, the movements of objects are constrained by structured environments. Therefore, the relationship between objects and environments can be exploited as additional information for improving the performance of tracking. We use the environment state to model the relationship between the objects and environments, and integrate it into the framework of Bayesian tracking. In this paper, distance transform is used to model the environment state, and particle filtering is employed as the paradigm for solving the Bayesian tracking problem. The adaptive dynamics model and environment prior are devised for the particle filter to fully utilize the environment information in the tracking process. Experiments on some video surveillance sequences demonstrate the effectiveness and robustness of our approach for tracking object motions in structured environments.

Image registration is an essential step in many image processing applications that need visual information from multiple images for comparison, integration or analysis. Recently researchers have introduced image registration techniques using the log-polar transform (LPT) for its rotation and scale invariant properties. However, the accuracy of the approach limits by the number of samples used in the mapping process, which affects directly the computational cost. Motivated by the success of LPT based approach and its limitation, we propose a novel projective polar transform (PPT) based image registration method that is robust to translation, scale, and rotation and yields high accuracy while requires low computational cost. Unlike LPT that 2D interpolation is needed in the mapping process, our method uses one-to-one mapping mechanism that directly arranges image from Cartesian to polar coordinate according to pre-computed PPT map. An innovative projection mechanism is proposed to reduce the image from two to one dimensional vectors. With the proposed approach, the mapping procedure is accelerated and the matching process can be performed in 1D. This results in great reduction of the computational cost.

This paper presents a generic video coding framework with the texture abstraction and completion, inspired by a strong grouping bias of local elements in Gestalt psychology. Abstracting imagery by grouping perceptual salience from anisotropic diffusion, it decomposes video images into two layers composing of semantic components and residual detail. The similarity between textures of abstraction layer is motivated to infer the restoration of missing detail, under the spatio-temporal variation regularity. Through a motion and spatial context of moton, hence, a group of pictures (GOP) is divided into key frames and abstracted frames to form the final compressed data. An abstraction refinement is tuned to improve matching of detail restoration based on bilateral filtering. The proposed approach is more generic without incurring any specific side information, and achieves up to 20% bit saving versus standard H.264 at similar visual quality levels.

Multi-category classification is an ongoing research topic with numerous applications. In this paper, a novel approach called margin and domain integrated classifier (MDIC) is addressed. It handles multi-class problems as a combination of several target classes plus outliers. The basic idea behind the proposed approach is that target classes possess structured characteristics while outliers scatter around in the feature space. In our approach the domain description and large-margin discrimination are adjustable and therefore higher classification accuracy leads to better performance. The properties of MDIC are analyzed and the performance comparisons using synthetic and real data are presented.

Under the probabilistic framework, we consider the problem of tracking a group of highly correlated targets and propose to embed the correlation into the sampling procedure, where the correlation serves as both a prior information to improve the efficiency and a constraint to prevent trackers from confusion or drifting. Experiments under different settings demonstrate promising results in robustness with linear complexity.

A novel probabilistic tracking system is presented, which includes a sequential particle sampler and a fragment-based measurement model. Rather than generating particles independently in a generic particle filter, the correlation between particles is used to improve sampling efficiency, especially when the target moves in an unexpected and abrupt fashion. We propose to update the proposal distribution by dynamically incorporating the most recent measurements and generating particles sequentially, where the contextual confidence of the user on the measurement model is also involved. Besides, the matching template is divided into non-overlapping fragments, and by learning the background information only a subset of the most discriminative target regions are dynamically selected to measure each particle, where the model update is easily embedded to handle fast appearance changes. The two parts are dynamically fused together such that the system is able to capture abrupt motions and produce a better localization of the moving target in an efficient way. With the improved discriminative power, the new algorithm also succeeds in handling partial occlusions and clutter background. Experiments on both synthetic and real-world data verify the effectiveness of the new algorithm and demonstrate its superiority over existing methods.

Image registration is an essential step in many image processing applications that need visual information from multiple images for comparison, integration, or analysis. Recently, researchers have introduced image registration techniques using the log-polar transform (LPT) for its rotation and scale invariant properties. However, it suffers from nonuniform sampling which makes it not suitable for applications in which the registered images are altered or occluded. Inspired by LPT, this paper presents a new registration algorithm that addresses the problems of the conventional LPT while maintaining the robustness to scale and rotation. We introduce a novel adaptive polar transform (APT) technique that evenly and effectively samples the image in the Cartesian coordinates. Combining APT with an innovative projection transform along with a matching mechanism, the proposed method yields less computational load and more accurate registration than that of the conventional LPT. Translation between the registered images is recovered with the new search scheme using Gabor feature extraction to accelerate the localization procedure. Moreover an image comparison scheme is proposed for locating the area where the image pairs differ. Experiments on real images demonstrate the effectiveness and robustness of the proposed approach for registering images that are subjected to occlusion and alteration in addition to scale, rotation, and translation.

A novel variant of particle filters is presented, where new particles are generated sequentially by adapting the proposal density dynamically according to the likelihood of the current particle which are just generated. The new algorithm is able to capture more nonlinear motion and produce a better localization of the moving target in an efficient way. Experiments on both synthetic and real-world data verify its effectiveness and demonstrate its superiority over the generic particle filter.

We present a new scheme for efficient and robust video object tracking in constrained environments. It finds its application in security surveillance, traffic monitoring, etc. In these applications movements of objects are restricted by the environments; therefore, environment constraints can be exploited as heuristic information for improving the performance of tracking. In this paper we use the distance field to represent environment constraints and integrate it into the framework of particle filtering. Experiments on some video surveillance sequences demonstrate the effectiveness of our approach.

Image registration is an essential task in many image processing applications. Log-polar transform (LPT) is a well known tool for image processing for its rotation and scale invariant properties. However it suffers from nonuniform sampling which makes it not suitable for the environment when registered images are altered or occluded. Inspired by LPT, this paper presents a new algorithm that addresses the problems of the conventional LPT while maintaining the invariant properties. We introduce adaptive polar transform (APT) technique and an innovative matching mechanism that serve as image processing tool for recovering scale and rotation change of the registered image. Unlike the conventional LPT that focuses only at the fovea area of the registered image, our proposed APT gives even consideration to the entire image. Translation of the registered image is recovered with the new search scheme by using Gabor feature extraction to accelerate the localization procedure. The experimental results indicate that our proposed method outperforms conventional LPT in registering images that are altered or occluded.

This paper proposes a rotation and scale invariant object recognition method which combines image feature extraction in Cartesian coordinate with the log-polar mapping and similarity measure techniques for classification. The method yields robustness for fast computation without any rescale needed in the target image. The method also does not use motion estimation. Thus, the method is also robust in recognizing objects without prior information of the objectpsilas motion or previous location. Experiments with real video sequences are provided to verify the effectiveness of the proposed approach in practice.

A quality constrained compression algorithm based on discrete wavelet transform (DWT) is proposed. The spatial-frequency decomposition property of DWT provides possibility for not only the new compression algorithm but also a frequency-domain quality assessment method. For facilitating the new algorithm, a new quality metric in the wavelet domain called WNMSE is suggested, which assesses the quality of an image with the weighted sum of normalized mean square errors of the wavelet coefficients. The metric is consistent with the human judgment of visual quality as well as able to estimate the quality during the compression process. Based on the relationship between the statistic features, quantization steps, and the weighted normalized mean square error value of the image, we develop a quality constrained quantization algorithm which can determine the quantization step-sizes for all the wavelet subbands for compressing the image to a desired visual quality accurately.

We consider the joint application of Human Vision System and rate allocation theory in logo watermarking. A new algorithm is proposed under this framework, in which the logo is multi-level decomposed, compressed and embedded into the decomposed host with the energy of the compressed logo optimally distributed across different subbands of the host. The scheme is verified to be robust against general signal attacks, and experimental comparison shows its superiority to other approaches. Moreover it is quite suitable for hardware implementations by using reversible wavelet transform.

A high-throughput robotic system has been developed to enable the automatic handling of nanoliter volumes of highly viscous biomaterials for crystallizing membrane proteins using lipidic mesophases. The in meso method introduced a few years ago has produced crystal structures of a number of important membrane proteins. The bottleneck to achieve high throughput automation of this, so-called in meso method, is the handling of nanoliter volumes of the highly viscous cubic phase as part of the crystallization process. The cubic phase sticks to everything that it has contact with and has a tendency to disable dispensing robotic tools. In this paper, we discuss the factors that influence the successful and automatic delivery of nanoliter volumes of the cubic phase. A creative cubic-phase-based coordinate measuring mechanism is presented for controlling the dispensing distance of the cubic phase which is critically important for the successful performance of the system. A mathematical model describing the cubic-phase delivery is proposed and verified. We also present the optimization of liquid handling parameters for the successful and automatic delivery of different precipitant solutions. The performance characteristics of the robotic system in terms of accuracy and reproducibility of delivering nano volumes of highly viscous biomaterials and micro volumes of different precipitant solutions are reported. Note to Practitioners-Automatic handling of nanoliter volumes of highly viscous biomaterials is a practically challenging yet important task in research-and-development activities of biology and drug industry, including protein crystallization. The latter, however, is a key step for determining the structure of the proteins. Only when the structure of the proteins is revealed, can one understand the function of the proteins and/or develop drugs to cure various kinds of diseases. This paper uses a practical project, crystallizing membrane proteins using lipidic mesophases, as an example to develop approaches for automatically handling highly viscous biomaterials. This paper presents the factors that influence the successful and automatic delivery of nanoliter volumes of viscous biomaterials and introduces a new coordinate measuring mechanism using the viscous biomaterials for controlling the dispensing distance of the biomaterials, which is critical for the successful delivery of viscous materials. A mathematical model describing the delivery of viscous material is for guiding the design of practical delivering systems. This paper also presents a number of parameters for successful and automatic delivery of different precipitant solutions which are involved in the biomaterial screening activities in drug development and biological research. A simple and effective computer vision approach for measuring the volume of bio-solutions and biomaterials in the nanoliter scale is presented which is verified by the fluorescence intensity measurement of the delivered samples. The performance of the automatic system developed in terms of accuracy and reproducibility of delivering nanovolumes of highly viscous biomaterials is reported which proves that the proposed approach is not only practically useful but also feasible for many applications

As a requisite of content-based multimedia technologies, video object (VO) extraction is of great importance. In recent years, approaches have been proposed to handle VO extraction directly as a classification problem. This type of methods calls for state-of-the-art classifiers because the extraction performance is directly related to the accuracy of classification. Promising results have been reported for single object extraction using support vector machines (SVM) and its extensions such as psi-learning. Multiple object extraction, on the other hand, still imposes great difficulty as multi-category classification is an ongoing research topic in machine learning. This paper introduces the newly developed multi-category psi-learning as the multiclass classifier for multiple VO extraction, and demonstrates its effectiveness and advantages by experiments

In this paper, a new image denoising algorithm using simple thresholding operations and wavelet coefficient magnitude based Wiener filtering is proposed. A hard thresholding operation is initially performed on the noisy wavelet coefficients. The initial significance map is then refined by use of multi-resolution coefficient support map which considers the local spatial features of the image. As a final denoising step, optimal Wiener filtering is performed on the thresholded wavelet coefficients using only magnitude information. The performance of proposed algorithm is evaluated on standard test images and found to perform competitively to the state-of-art image denoising algorithms in the literature.

Binary support vector machines (SVM) have proven effective in classification. However, problems remain with respect to feature selection in multi-class classification. This article proposes a novel multi-class SVM, which performs classification and feature selection simultaneously via L1-norm penalized sparse representations. The proposed methodology, together with our developed regularization solution path, permits feature selection within the framework of classification. The operational characteristics of the proposed methodology is examined via both simulated and benchmark examples, and is compared to some competitors in terms of the accuracy of prediction and feature selection. The numerical results suggest that the proposed methodology is highly competitive

A combined spatial- and temporal-domain wavelet shrinkage algorithm for video denoising is presented in this paper. The spatial-domain denoising technique is a selective wavelet shrinkage method which uses a two-threshold criteria to exploit the geometry of the wavelet subbands of each video frame, and each frame of the image sequence is spatially denoised independently of one another. The temporal-domain denoising technique is a selective wavelet shrinkage method which estimates the level of noise corruption as well as the amount of motion in the image sequence. The amount of noise is estimated to determine how much filtering is needed in the temporal-domain, and the amount of motion is taken into consideration to determine the degree of similarity between consecutive frames. The similarity affects how much noise removal is possible using temporal-domain processing. Using motion and noise level estimates, a video denoising technique is established which is robust to various levels of noise corruption and various levels of motion.

In video streaming, time-varying packet loss rate leads both degradation and inconsistency to video quality. In this paper, we propose a novel content-based retransmission scheme to address the above two problems in 3-D wavelet video streaming. The fundamental idea is to consider the contents of the packets and the traffic of the Internet simultaneously to optimize the quality of service (QoS). In the encoding stage, the content information is generated and stored, based on which an intelligent hybrid decision system is designed to perform a two-step optimization in transmission. First, the scheme for retransmitting the lost packets is designed to minimize the distortion of each group of video frames (GOF). Secondly, the bandwidth used for transmitting each GOF is dynamically allocated according to the changing traffic conditions such that the quality of video is consistent. The goal is to enable the streaming system to deliver the best possible visual quality of video under the same channel conditions. Simulation results show that our proposed approach can improve the video quality (more than 2 dB) as well as reduce the quality variation significantly

Support vector machines (SVM) are state-of-the-art learning machines and have found a great deal of success in a wide range of applications. In the framework of SVM, each sample belongs to either one class or the other. This requirement, however, makes it difficult to apply SVM to the applications where the data exhibit partial or unclear class memberships. To address this problem, this paper reformulates the standard SVM to be a new learning machine that is capable of dealing with binary (or hard) as well as real-valued (or soft) class memberships. The new machine, which is named soft SVM (S-SVM), has been integrated into a classification-based video object extraction approach, and the experimental results demonstrate the effectiveness of the new approach.

For very low bit-rate video compression, wavelet based approaches suffer the ghost effect caused by larger temporal distortions. We propose an innovative temporal-domain wavelet filtering scheme called sub-grouping transformation (SGT), which divides wavelet coefficients in the temporal direction into sub-groups and applies the temporal wavelet transform to individual sub-groups. By splitting high motion coefficient arrays into smoother sub-groups, the proposed algorithm can achieve high compression ratio and reduce the ghost effect at the same time.

Support vector machines (SVM) is originally designed for binary classification. To extend it to multi-class scenario, a typical conventional way is to decompose an M-class problem into a series of two-class problems, for which one-against-all is the earliest and one of the most widely used implementations. However, certain theoretical analysis reveals a drawback, i.e., the competence of each classifier is totally neglected when the results of classification from the multiple classifiers are combined for the final decision. To overcome this limitation, this paper introduces reliability measures into the multi-class framework. Two measures are designed: static reliability measure (SRM) and dynamic reliability measure (DRM). SRM works on a collective basis and yields a constant value regardless of the location of the test sample. DRM, on the other hand, accounts for the spatial variation of the classifier's performance. Based on these two reliability measures, a new decision strategy for the one-against-all method is proposed, which is tested on benchmark data sets and demonstrates its effectiveness.

As a requisite of the emerging content-based multimedia technologies, video object (VO) extraction is of great importance. This paper presents a novel semiautomatic segmentation and tracking method for single VO extraction. Unlike traditional approaches, the proposed method formulates the separation of the VO from the background as a classification problem. Each frame is divided into small blocks of uniform size, which are called object blocks if the centering pixels belong to the object, or background blocks otherwise. After a manual segmentation of the first frame, the blocks of this frame are used as the training samples for the object-background classifier. A newly developed learning tool called /spl psi/-learning is employed to train the classifier which outperforms the conventional Support Vector Machines in linearly nonseparable cases. To deal with large and complex objects, a multilayer approach constructing a so-called hyperplane tree is proposed. Each node of the tree represents a hyperplane, responsible for classifying only a subset of the training samples. Multiple hyperplanes are thus needed to classify the entire set. Through the combination of the multilayer scheme and /spl psi/-learning, one can avoid the complexity of nonlinear mapping as well as achieve high classification accuracy. During the tracking phase, the pixel in the center of every block in a successive frame is classified by a sequence of hyperplanes from the root to a leaf node of the hyperplane tree, and the class of the block is identified accordingly. All the object blocks thus form the object of interest, whose boundary unfortunately is stair-like due to the block effect. In order to obtain the pixel-wise boundary in a cost efficient way, a pyramid boundary refining algorithm is designed, which iteratively selects a few informative pixels for class label checking, and reduces uncertainty about the actual boundary of the object. The proposed method has been applied on video sequences with various spatial and temporal characteristics, and experimental results demonstrate it to be effective, efficient, and robust.

A selective wavelet shrinkage algorithm for digital image denoising is presented. The performance of this method is an improvement upon other methods proposed in the literature and is algorithmically simple for large computational savings. The improved performance and computational speed of the proposed wavelet shrinkage algorithm is presented and experimentally compared with established methods. The denoising method incorporated in the proposed algorithm involves a two-threshold validation process for real-time selection of wavelet coefficients. The two-threshold criteria selects wavelet coefficients based on their absolute value, spatial regularity, and regularity across multiresolution scales. The proposed algorithm takes image features into consideration in the selection process. Statistically, most images have regular features resulting in connected subband coefficients. Therefore, the resulting subbands of wavelet transformed images in large part do not contain isolated coefficients. In the proposed algorithm, coefficients are selected due to their magnitude, and only a subset of those selected coefficients which exhibit a spatially regular behavior remain for image reconstruction. Therefore, two thresholds are used in the coefficient selection process. The first threshold is used to distinguish coefficients of large magnitude and the second is used to distinguish coefficients of spatial regularity. The performance of the proposed wavelet denoising technique is an improvement upon several other established wavelet denoising techniques, as well as being computationally efficient to facilitate real-time image-processing applications.

This paper presents a novel feature selection method which is named filtered and supported sequential forward search (FS/spl I.bar/SFS) in the context of support vector machines (SVM). In comparison with conventional wrapper methods employing the sequential forward search (SFS) strategy, it has two important properties that reduce the computation time of SVM training during the feature selection process. First, instead of utilizing all the training samples to obtain the classifier, FS/spl I.bar/SFS, by taking advantage of the existence of support vectors in SVM, dynamically maintains an active data set for each SVM to be trained on. In this way, the computational demand of a single SVM training decreases. Secondly, a new criterion, in which the discriminant ability of individual features and the correlation between them are both taken into consideration, is proposed to effectively filter out non-essential features before every SFS iteration begins. As a result, the total amount of training is significantly reduced. The proposed approach is tested on both synthetic and real data to demonstrate its effectiveness and efficiency.

A completely new focus in the area of robotics research has been proposed to enable the automatic handling of nanoliter volumes of highly viscous biomaterials. The robot in question is of immediate need in the proteomics research community following two important recent announcements: the completion of the Human Genome Project (HGP) and the development of the novel in meso method for membrane protein crystallization. The completion of the HGP has sparked off intense activity in the area of protein structure determination. The in meso method introduced a few years ago has helped us get crystals of membrane proteins, such as bacteriorhodopsin, and provides a novel methodology for membrane protein crystallization. The bottleneck to achieve high throughput automation of this, so called in meso method is the handling of nanoliter volumes of the highly viscous cubic phase as part of the crystallization process. The cubic phase sticks to everything that it contacts and has a tendency to disable dispensing robotic tools. In this paper, we discuss the factors that influence the successful delivery of nanoliter volumes of the cubic phase in a controlled and reproducible fashion. An innovative technique of coordinate measurement using the cubic phase as a transducer is proposed and verified. We also describe a robotic system designed and built to address this issue and present the results of crystallization using our in meso robot.

A robotic theory is presented to explain the process of protein crystallization. The theory employs the approach of path planning for robot team formation, but imposes two constraints-simplicity and locality because proteins do not have the intelligence to plan complex paths in a global manner. Based on the two constraints, we develop three rules which govern the development of a successful path. The path defines the direction and speed of the motion for all the proteins in forming the crystal. Physical analysis at the molecular level is conducted to verify that the hypothesized motion for crystallization is achievable. Computer simulation and experimental results are presented to show the promise of the new approach in the study of protein crystallization.

A mechanism for automating the delivery of viscous biomaterials is presented. This is essential for automating the crystallization of membrane proteins with in meso method. The bottleneck of the in meso method is to dispense highly viscous biomaterials which sticks to everything it contacts. It is especially challenging to dispense accurately small amounts (nano-liter) of biomaterial. In this paper, mathematical modeling of the shape and volume of the biomaterial delivered by the syringe is first performed. Optimal distance between the dispensing tool and the base of the well is then derived to guarantee the successful delivery of an accurate amount of the biomaterials. An effective path for robot to move the syringe is presented which has to be positioned at the optimal distance efficiently. Experimental results are presented to verify the theoretical results.

A novel approach for navigation of cleaning robots in an unknown workspace is presented which includes a new map representation method and a complete coverage navigation mechanism. First, we discuss a triangular cell map representation which makes the cleaning robot navigate with a shorter path and increased flexibility than a rectangular cell map representation. Then, we propose the complete coverage navigation and map construction methods which enable the cleaning robot to navigate the complete workspace with no complete information about the environment. Finally, we evaluate the performance of our proposed triangular cell map via the existing distance-transform-based path planning method comparing to that of the rectangular cell map.

An optimal three-dimensional (3-D) coefficient tree structure for 3-D zerotree wavelet video coding is considered in this paper. The 3-D zerotree wavelet video coding is inspired by the success of the two-dimensional (2-D) zerotree wavelet image coding. Existing 3-D zerotree wavelet video codecs use the either symmetric or symmetric-alike 3-D tree structure, which is a straightforward extension of the symmetric 2-D tree structure in the zerotree wavelet image coding. In this paper, we show that the 3-D zerotree coding does not need to be applied symmetrically along all the directions, as the 2-D zerotree image coding does. We find that an asymmetric 3-D tree structure working with a more flexible asymmetric 3-D wavelet transform can produce a higher compression ratio than traditional symmetric approaches. The new 3-D tree structure can be used to improve the rate-distortion performance of many existing 3-D zerotree wavelet video codecs without sacrificing other features such as scalability and computational efficiency. The new tree structure is applied to the 3-D set partitioning in hierarchical trees method and receives convincing peak signal-to-noise ratio improvements in experiments.

The paper presents an object tracking method for object-based video processing which uses a two-dimensional (2D) Gabor wavelet transform (GWT) and a 2D golden section algorithm. An object in the current frame is modeled by local features from a number of the selected feature points, and the global placement of these feature points. The feature points are stochastically selected based on the energy of their GWT coefficients. Points with higher energy have a higher probability of being selected since they are visually more important. The amplitudes of the GWT coefficients of a feature point are then used as the local feature. The global placement of the feature points is determined by a 2D mesh whose feature is the area of the triangles formed by the feature points. In this way, a local feature is represented by a GWT coefficient amplitude vector, and a global feature is represented by a triangle area vector. One advantage of the 2D mesh is that the direction of its triangle area vector is invariant to affine transform. Consequently, the similarity between two local features or two global features can be defined as a function of the angle and the length ratio between two vectors, and the overall similarity between two objects is a weighted sum of the local and global similarities. In order to find the corresponding object in the next frame, the 2D golden section algorithm is employed. Our results show that the method is robust to object deformation and supports object tracking in noisy video sequences.

This paper presents a new method for tracking an object in a video sequence which uses a 2D Gabor wavelet transform (GWT), a 2D mesh, and a 2D golden section algorithm. An object is modeled by local features from a number of the selected feature points, and the global placement of these feature points. The feature points are stochastically selected based on the energy of their GWT coefficients. Points with higher energy have a higher probability of being selected. The amplitudes of the GWT coefficients of a feature point are then used as the local feature. The global placement of the feature points is determined by a 2D mesh whose feature is the area of the triangles formed by the feature points. The overall similarity between two objects is a weighted sum of the local and global similarities. In order to find the corresponding object in the video sequence, the 2D golden section algorithm is employed, and this can be shown to be the fastest algorithm to find the maximum of a unimodal function. Our results show that the method is robust to object deformation and supports object tracking in noisy video sequences.

A small power biped robot is designed by the load sharing method of walking gait. Walking can be classified into one-foot and two-foot standing during the common speed smooth walking. In two-foot standing, the weight of robot can be distributed to all joints. However, in one-foot standing, it is burdened to one foot, especially to ankle joint. So bigger actuator or even the passive joint is considered at the ankle joint; but bigger actuator increases the weight of robot and passive joint does not control ankle. When the small power actuator is used at all joints, ankle joint is sometimes overloaded during walking and this makes walking impossible. This problem is solved using the appropriate walking gait, that is, it depresses or distributes the driving power of the ankle joint when the load is applied to the ankle, and drives the ankle joint before the load is applied. By this method, the small power biped robot can be implemented without overloading the ankle joint. Experiments are conducted to prove the proposed method.

Real-time motion planning is one of the most challenging problems for successful employment of personal robots. A motion planning strategy via human commands is proposed. The human commands produce primitive motions which can form complex trajectories in real-time. Proper modifications to the primitive motions can generate trajectories which avoid obstacles. An automatic modification mechanism also can assist the robot to reach a target point quickly. Experimental results are presented to verify the effectiveness of the proposed scheme.

A new method for speeding up the integer wavelet transforms constructed by the lifting scheme is proposed. The proposed method packs multiple pixels (wavelet coefficients) in a single word; therefore, it can make use of the 32-bit or 64-bit computational capability of modern computers to accomplish multiple addition/subtraction operations in one instruction cycle. As a result, our method can save the decomposition/reconstruction time by up to 37% on 32-bit machines and require much less working memory in comparison with the original wavelet transform algorithms.

A new method for speeding up the integer reversible wavelet transforms constructed by the lifting scheme is proposed. The proposed method packs multiple pixels (wavelet coefficients) in a single word; therefore, it can make use of the 32-bit or 64-bit computational capability of modern computers to accomplish multiple addition/subtraction operations in one instruction cycle. As a result, the proposed method can save the decomposition/reconstruction time by up to 37 percent on 32-bit machines in comparison with the original wavelet transform algorithms. Furthermore, the packed integer wavelet transform requires much less working memory.

The coordination problem of a human arm and a robot manipulator is explored using compliant motion and predictive control. The problem arises when a human arm and a robot manipulator coordinate for the execution of a task in unscheduled task environment. In such scenarios the arm, by virtue of its intelligence, is assumed to lead the task while the manipulator is required to comply with the motion of the arm and support the object load. Such a scheme is superior to the better known multiple manipulator coordination problem, which normally assumes known trajectories and a structured task environment. By coordinating manipulator with the arm of its operator the uncertainty due to the environment can be reduced while load sharing can help relieve the arm of the physical strain. This paper addresses the problem in the framework of model-based predictive control. The transfer function from the manipulator position command to the wrist sensor force output is defined. The desired set point for the manipulator force is set to equal the gravitational force. A predictive control scheme is then used to design a two-degree of freedom controller for the problem. The simulation results indicate that the manipulator effectively takes over the object load and the arm force stays close to zero. Moreover, the manipulator is seen to be highly responsive to the arm movement and relatively small arm force can effectively initiate the manipulation task.

We present the design of an experimental prototype cable-driven suspended platform for robotic applications, and the feasibility of using this platform in certain industrial applications is considered. In order to study and improve dynamic stability, the point-to-point motion of the platform is observed and analyzed in a laboratory environment. To focus the study, the domain of experiments is restricted to only two-dimensional space. At the starting point, or departure, the platform displays a lag and at the destination, or terminal point, the platform has a significant overshoot which causes a large settling time. Certain methods are suggested and implemented for damping the oscillation and decreasing the settling time. In the three-dimensional domain, two experiments are performed: one experiment is devoted to drilling holes in a piece of wood and another to loosening a nut.

Personal robotics is a new and attractive use of robotic technologies. We study one of its important topics-real-time motion planning. We propose to use primitive motions and their combination to make this possible. A reflexive motion control scheme is proposed to activate appropriate primitive motions for a desired complex motion. Experimental results are presented to show the effectiveness of the proposed scheme.

Summary form only given. As a reversible integer wavelet transform, the TS transform gains much attention in both lossless and lossy compression. It is a good approximation to the (2, 6) wavelet, which is one of the best biorthogonal wavelets for image compression, and it can be implemented by only using integer addition/subtraction and shift. Most gray-scale images are in 8-bit form; the TS transform coefficients of them can be represented by 16-bit words, while in most modern computers, 32-bit arithmetic and 16-bit arithmetic have the same speed. We propose a method to speed up the TS transform for image compression. The new algorithm is called the packed-TS transform. The proposed method packs two adjacent pixels in one double-word; therefore, it can make use of the 32-bit computational capability of modern computers to accomplish two additions/subtractions in one instruction cycle. The packed-TS transform is also a reversible transform. In the packed-TS transform, the two adjacent pixels or coefficients are stored in the high-word and the low-word of a double-word, respectively. Then the decomposition/reconstruction is performed on this double-word. We compare the performance of the original TS transform and the proposed packed-TS transform on five images: Girl, Lena, Peppers, Couple and Man, respectively . The experiment shows that the packed-TS transform is faster than the original TS transform by about 30 percent, with a comparable performance in the quality of the reconstructed images.

The human-robot coordination with rotational motion is discussed and experimented. In order to deal with the rotational motion, a new force sensor is proposed to measure the force provided by the human operator in addition to the force sensor previously proposed for the compliant motion of the robot. The force sensor on the human-hand side interprets the desired motion of the human operator while the force sensor of the robot side defines the actual motion of the end-effector. The two motions then serve as the reference input and actual output of the feedback control system respectively. With the new force sensor, load distribution becomes a simple issue. It can be done by simply specifying a distribution factor which determines the ratio between the forces provided by the robot and the arm respectively. Experiment shows that the coordination scheme is effective for both the rotational and translational motions in the human-robot coordination.

The role of a compliant motion scheme is to control a robot manipulator in contact with its environment. By accommodating with the interaction force, the manipulator can be used to accomplish tasks that involve constrained motions. This paper presents a new work on the compliant motion control for position-controlled manipulator. A new method for achieving efficient interaction between the manipulator and its environment is developed. The interaction force is reduced using a quadratic cost function. In this method, the compliance is formulated as function of the interaction force. An adaptive element is used to update the compliance. The stability of the system is investigated using the Lyapunov approach. Experiments are conducted in our laboratory to demonstrate the use of such scheme.

Coordination of a human arm and a manipulator to share a load is a new problem. In this paper, we propose a variable compliance control as a coordination mechanism for the arm and the manipulator. The compliance will be updated according to the force exerted by the manipulator. This coordination mechanism is proposed to accommodate with the uncertainties associated with the impedance of the human arm and the weight of the object. Furthermore, the stability of the system and the manipulation speed will be improved. The stability problem is studied using the small gain theorem. Experiments were conducted in our laboratory to prove the effectiveness of the proposed scheme.

Arm-manipulator coordination is a new and attractive use of robotic manipulators in unstructured environments. In this paper, we propose a reflexive motion control as a coordination mechanism for the manipulator to share the load with the arm. Reflexive motion is an inspiration of biological systems. The motion is formulated in a manner which assists the manipulator to comprehend the intention of the human arm. It reduces the strain on the arm which is due to the weight of the object. In addition, the speed of manipulation on the object is improved significantly. The effects of the reflexive behavior on stability and load sharing of the system are investigated. Experimental results are presented to prove the effectiveness of the proposed scheme.

Surface digitization is an approach to obtain three-dimensional measurement of an unknown sculptured surface. The quality and the efficiency of surface digitization are two main concerns which have drawn much attention in the manufacturing industries. A cost-effective digitizing approach can not only reduce the measuring time, but also improve the accuracy of measurement. In the past, the digitizing methods were performed in such a way that the digitizing points were chosen at intervals with a constant distance. However, many local areas of a sculptured surface can be well-represented by fewer points. Therefore dense digitizing of conventional digitizing schemes is not efficient. In this paper, a new irregular digitizing scheme based on the wavelet transform is proposed. First an initial depth map of sculptured surface is obtained by using computer vision. From the initial depth map, the wavelet transform is used to select the necessary digitizing points for further digitization by highly accurate measuring equipment. Once the digitizing points are re-measured, the surface can be reconstructed by wavelet synthesis technique.

In this study, a coupled pattern generator is used to achieve rhythmic gait movements by a planar five-link musculoskeletal biped model. In particular, five coupled oscillators are designed which provide the biped with the desired joint angles while a controller tracks the specified trajectories. The system is adaptive in that a high-level parameter controller can adjust the parameters of the pattern generator to change the frequency of the desired joint trajectories. Simulations show the biped walking while changing frequency of the gait as its joint angles track the pattern generator.

We present an adaptive compression and resampling technique using wavelet packet shrinkage. The motivating application is surface digitization for computer-aided manufacturing. The objective is to improve estimation accuracy using a sparse and irregular, but more accurate, resampling of the data. We provide modified thresholds for wavepacket shrinkage, heuristics for choosing the resampling grid, and a mean estimator using measured data. Computed results illustrate application of the proposed technique.

Arm-manipulator coordination is a new coordination problem in robotics. Human arm-manipulator coordination is superior to both arm-arm coordination and manipulator-manipulator coordination. Likewise, it is expected to have powerful impacts on many applications in the manufacturing, service, and construction industries. This paper introduces this problem and investigates the coordination using a compliant control method. There are several problems in this scheme. First, the stability problem is studied using the Routh-Hurwitz criterion. Then, a very important aspect of the arm-manipulator coordination problem, load sharing, is investigated. Experimental analysis proves the effectiveness of the proposed schemes.

This paper presents a new work on compliant motion control for position-controlled manipulators. Two new methods for achieving efficient interaction between the manipulator and its environment are developed. In both methods, the compliance is formulated as a function of the interaction force. The first method uses an adaptive element to update the compliance, while the second one adopts a scheme that is believed to be used in human beings to control the stiffness of muscles. The stability of the system is investigated using the small gain theorem. Experiments are conducted in our lab to demonstrate the use of such schemes.

Surface digitization is a method for obtaining three-dimensional measurement of an unknown surface. The performance and quality of digitization are dependent upon the choice of digitizing devices and approaches. An efficient digitizing method not only reduces the elapsed time of the digitizing process, but also improves the precision of measurement. This paper presents a new nonuniform technique for digitizing of sculptured surfaces based on the short time Fourier transform (STFT). The digitizing distances in the x and y directions for each pre-divided block of the surface are adjusted according to the highest spatial frequencies contained in the block which is in contrast to the uniform technique in which the digitizing distance is determined according to the highest spatial frequencies of the entire surface. The speed of the digitizing process is thus improved and meanwhile the accuracy of the surface representation is maintained. Simulations and experimental results are provided to verify the proposed method.

Power reduction in the ankle joints of a biped robot is considered in this paper. The ankle of human being has small torque and is very flexible within a certain range (very stiff near and beyond this range). This characteristic makes foot landing soft and gives a good contact between its sole and the ground. This feature can be implemented in a biped robot by using a small actuator for the ankle joints. A small actuator consumes less energy and makes the robot leg light. With less power in the ankle joints, walking becomes more difficult to control for the robot. This problem can be solved by providing the feedback control presented in this paper. The authors demonstrate two locomotion examples, standing and walking respectively, to show the validity of the proposed control scheme. In standing, the control input is the displacement of the ankle joint of the supporting leg. The control mechanism decides the bending angle of the body and the position of the swinging leg. For walking, only the bending angle of the body is used to avoid the discontinuity of the control input. Experimental results are presented to show the effectiveness of the new mechanism.

Coordinating two robot manipulators to handle flexible materials has a wide range of applications in the manufacturing industry. However, this problem has not been seriously addressed until recently. The two robot manipulators have to follow complicated trajectories to maintain a minimum interaction force with the flexible beam. These trajectories are very complicated and not suitable for real time systems. In this paper, three approximation methods of the optimal trajectories are introduced. The first one uses a piecewise linear approximation of the optimal trajectories, while the second one uses adaptive piecewise linear approximation. The third method utilizes the behavior of the optimal trajectories and applies a continuous approximation of the optimal trajectories using an ellipsoid. The interaction moment under these approximations is investigated. The authors found that the first method produces a peak in the interaction moment which can damage the object. The second method controls the peaks in the moment to a prescribed limit. In the third method, the moment is very small and the force is very close to the optimal force.

Coordinating two robot manipulators to handle flexible materials has a wide range of applications in the manufacturing industry. However, this problem has not been seriously addressed until recently. The two robot manipulators have to follow complicated trajectories to maintain a minimum interaction force with the flexible beam. These trajectories are very complicated and not suitable for real time systems. A compliant motion scheme is proposed to reduce the interaction forces and moments. The stability of the proposed system is investigated. Experimental results encourage the proposed scheme.

In a previous paper (1993), the authors proposed an optimal trajectory for two manipulators to bend a flexible beam. The criterion was to minimize the interaction forces and moments between the beam and the end-effectors. It turned out that computation for specifying such a trajectory was complicated since an elliptic integral was involved in the computation. In this study, a circular arc is used as the motion trajectory of the two end-effectors. Since a circular arc is easy to specify, the computation time is greatly reduced. However, the interaction forces and moments become non-minimal. To overcome this problem, a neural network mechanism is proposed to adjust the trajectory in real-time such that the interaction forces and moments are reduced. The residual forces and moments are further minimized by a force feedback control mechanism. Simulation results are presented to verify the proposed method.>

In order for biped robots to handle a variety of tasks, the robot must be able to traverse different terrains. Different terrains require different walking gaits, and if all these gaits must be programmed by human operators then this programming can be a very large and time consuming process. If, however, the robot has the capability to automatically generate different gaits when placed on unfamiliar terrain, then the need to program many different gaits is eliminated. This paper looks at a method to generate gaits based on distal supervised learning. This method incorporates a forward model of the robot dynamics and uses it to convert stability information into information on how to adjust the robot's joints so as to regain stability. The method is tested with a simulation of the SD-2 biped robot.>

Coordinating two robot manipulators to handle flexible materials has a wide range of applications in the manufacturing industry. However, this problem has not been seriously addressed until recently. The two robot manipulators have to follow complicated trajectories to maintain a minimum interaction force with the flexible beam. These trajectories are very complicated and not suitable for real time systems. In this paper, three approximation methods of the optimal trajectories are introduced. The first one uses a piece-wise linear approximation of the optimal trajectories, while the second one uses adaptive piece-wise linear approximation. The third method utilizes the behavior of the optimal trajectories and applies a continuous approximation of the optimal trajectories using an ellipsoid. The interaction force under these approximations is investigated. The authors found that the first method produces a peak in the interaction force which can damage the object. The second method controls the peaks in the force to a prescribed limit. In the third method, the force is very close to the optimal force. Experimentation was used to verify the approximation methods.>

Using a local area network (LAN) to coordinate multiple industrial robots is considered in this paper. The LAN uses the standard Manufacturing Automation Protocol (MAP) to control the access of the communication bus. Since the MAP based LAN was not designed for real-time applications, performance evaluation of the LAN becomes an important issue. A new evaluation method is proposed in this paper. The method evaluates the efficiency of the LAN by a parameter called effective token passing ratio. This parameter is affected by the speed and the access protocol of the LAN as well as the task decomposition-and-allocation scheme for computing the coordination algorithms. Previous methods, however, only evaluate the speed and the access protocol of the LAN. As a result, the new method provides more options for improving the LAN based system. To guarantee real-time coordination of multiple robots, the lower bound of the effective token passing ratio is determined and the probability of successful coordination when the transmission errors are possible is also considered.>

The inertias of the parallel and serial connections of two robots in the operational space are studied in this paper. First, it is reviewed that the symmetry between the velocity and force in a robotic chain results in velocity-force duality of the parallel and serial connections. The possible duality in the operational inertia matrix is then investigated. From the dynamic equations of the two connections, it is found that there does not exist a real duality in the operational inertia matrix. However, the real duality becomes possible when the coupling effects between the two robots are negligible. This conditional duality is called "quasi-duality" in this paper. As a result of this study, the duality behaviors of the two connections in velocity, force and inertia are completely obtained.>

Two serially connected robots in which the base of one robot is held by the end-effector of the other are more capable than a single robot. Serial connection of two robots can enlarge work space and generate large Cartesian force and Cartesian velocities. Two coordination methods are proposed for the robots to generate large Cartesian velocities while avoiding degeneracy. The first method, which is based on a least-energy consumption criterion, decomposes a desired Cartesian velocity into two velocities, one for each robot. Robot joint velocities are then adjusted using a null space approach to avoid degeneracy. The second method includes a degeneracy index in the cost function. As a result, joint velocities are obtained without any further adjustment. For real time applications, computationally simple methods are presented. The results are verified by case studies.>

A neural network mechanism is proposed to modify the rhythmic motion (gait) of a two-legged robot when walking on sloping surfaces using a sensory input. The robot starts walking on a terrain with no previous knowledge, but accumulates walking experience during walking, thus constantly improving its walking gait. The proposed network consists of 20 reciprocally inhibited and excited neurons. An unsupervised learning rule was implemented using reinforcement signals. Two learning algorithms are introduced. The primary concern in the first algorithm was the speed of gait modification, whereas the second algorithm provided a solution with minimum energy consumption. A static learning approach where learning takes place only at prespecified moments is proposed.>

Using a local area network (LAN) to coordinate multiple industrial robots in a multirobot system (workcell) is studied. The properties of the multiple robot coordination task are first discussed. Then, a method is proposed to evaluate a pre-defined task decomposition and allocation scheme for the coordination task. In this method, a parameter called effective token passing ratio (ETPR) is defined to measure how efficient the scheme is when the token bus LAN is used as the communication medium. A lower bound of the ETPR is analytically determined based on time constraints imposed on the task. By using this method, one does not need to determine the communication delay and computation time of individual tasks (or subtasks) decomposed from the coordination task. Instead, one can compare the ETPR of the network with the lower bound. From the comparison, it can be determined whether the task decomposition and allocation scheme is appropriate. The probability that real-time coordination is successful is also computed.>

The application of robotic manipulators to complex tasks such as assembly, or insertion often requires position/force control of the end-effector, and this has been widely studied. A related task is robot compliant motion over unknown objects. The goal is to move the effector, while maintaining contact, about the object. For this application a dynamic hybrid velocity/force controller is studied. The constraints are characterized in the manner proposed by M. Mason (1981). A nominal velocity trajectory is computed, and nominal hybrid joint commands are explicitly given in terms of the sensed joint coordinates and sensed local contact information. For robustness, servoing is added, and an example design is given. Finally, the step response of the controller is simulated for the case of rolling the effector about an unknown object.>

A mechanism for the control of the heel-off to toe-off motion of a dynamic biped gait is presented. It is argued that the reaction force between the ground and the biped foot is an important factor in a satisfactory implementation of dynamic gait. This force must be properly controlled while the foot is in the heel-off to toe-off phase. Since the foot rolls over the ground surface in this phase and the global description of the surface is unknown, the foot has to execute a nonholonomic constraint motion. The formulations of nonholonomic constraints and system dynamics under the constraints are derived for the roll-over motion. A feedback control mechanism is designed based on the formulations. The formulation and the feedback control mechanism are applied to a four-link biped model. Successful simulation results of the heel-off to toe-off motion prove that the formulation and control mechanism are valid for the nonholonomically constrained motion of the biped.>

The use of vision sensors to study object deformation is discussed. The vision sensor is used to identify the deformation characteristic and estimate the deformation of flexible beams. First, a nonlinear equation describing the deformation of the beam is given. Then, cubic spline functions are used to approximate the contour of the deformed beam after the beam is detected by the vision sensor. Using the cubic spline approximation and Maclaurin's series expansion, a procedure for determining the deformation characteristic is formulated. To estimate the deformation of the beam, a numerical differential method for solving the nonlinear equation is used. The error of this numerical method can be controlled with the aid of the vision sensor. Experimental results are given to verify that the proposed methods are effective.>

Unknown load distribution between two coordinated industrial robots is studied using two different methods. In the first method, called the load estimation method, the parameters associated with the load are estimated using the information from two wrist force sensors. As a result, the load becomes known, and conventional methods can then be applied to distribute the force. In the second method, the force compensation method, one of the robots, called the leader, takes the major role of carrying the load. The load is compensated for by another robot called the follower until the leader can carry the load to follow a satisfactory trajectory. Simulation results which demonstrate the validity of the force compensation method are presented. >

Strategies for automatic assembly of deformable objects are presented. A flexible beam mating with a rigid hole is taken as the target of investigation. The deflection behavior of a deformable beam is studied. The geometric and force conditions for successful assembly of rigid parts are reviewed. Based on these conditions, strategies for assembling deformable beams are proposed. The strategies are defined by the motion trajectory of the tool which handles the beam. It is found that when the clearance between the beam and the hole is loose, the desired trajectory should be the same as the curve formed by the deformed beam. When the clearance is tight, the position and orientation of the tool should be adjusted, following the involute of the deformed beam curve.>

Robot manipulators installed on legged mobile platforms are suggested for enlarging robot workspace. To plan the motion of such a system, the arm-platform motion coordination problem is raised, and a task decomposition is proposed to solve the problem. A given task described by the destination position and orientation of the end effector is decomposed into subtasks for arm manipulation and for platform configuration, respectively. The former is defined as the end-effector position and orientation with respect to the platform, and the latter as the platform position and orientation in the base coordinates. Three approaches are proposed for the task decomposition. The approaches are also evaluated in terms of the displacements, from which an optimal approach can be selected. >

The authors study the impact effect on a robotic system with a force-sensing device installed between the end-effector and the end of the robot. It is shown that physical contact of a robot with environments causes abrupt changes of velocity and force to the end-effector and the force sensor, respectively. The dynamic behavior of the force sensor following the impact is studied. Theoretical analysis and experiments show that the sensor generates a low-to-high transition signal corresponding to an impact, which can be used to identify the contact. To utilize this signal, an electronic switch is designed. The signal triggers the switch, which in turn interrupts the control computer. The control computer can then stop the robot motion immediately after an impact occurs. As a result, damage to, or malfunction of, the robotic system following the impact can be avoided. Experimental results show that the proposed switching mechanism works effectively.>

Multilimbed industrial robots that have at least one arm and two or more legs are suggested for enlarging robot workspace in industrial automation. To plan the motion of a multilimbed robot, the arm-leg motion-coordination problem is raised and task decomposition is proposed to solve the problem; that is, a given task described by the destination position and orientation of the end-effector is decomposed into subtasks for arm manipulation and for leg locomotion, respectively. The former is defined as the end-effector position and orientation with respect to the legged main body, and the latter as the main-body position and orientation in the world coordinates. Three approaches are proposed for the task decomposition. The approaches are further evaluated in terms of energy consumption, from which an optimal approach can be selected.>

The coordination of the arm and main body of multilimbed robots to generate large Cartesian forces is discussed. Two methods are developed. In the first method, motions for the arm and main body are separately considered. For a given Cartesian force, an optimal configuration is determined for the arm. Based on the arm configuration, the main body is adjusted to satisfy the constraints of the position and orientation of the end-effector. To reduce the energy consumption, the nonlinear programming techniques could be further applied to the main body to obtain an optimal configuration. The second method treats the multilimbed robot as a single system and directly applies the nonlinear programming technique to the arm and main body with the arm torque limitation as an added set of constraints. Since the multilimbed robot is considered as a single system, the method had the least sense of coordination and is computationally more complicated than the first method.>

When two robots execute coordinated motion in industrial application such as assembly, not only is the specification of the desired motion of each robot's end effector required, but also that of the desired contact force exerted on the end effectors. The involvement of the forces is the motivation for a consideration of compliance. The compliant coordination control scheme of two moving robots is presented. One robot is equipped with a force/torque sensing wrist, which provides force feedback signals and permits the fine coordinated motion control of complex assembly tasks. A control strategy is derived so that the measured force/torque can be used to modify the trajectory of one of the robots. This control scheme can be realized in real time by using the proposed computational structure. The system stability is related to the force-displacement conversion gain in the forward path. An experiment that screws a moving nut onto a moving bolt has been carried out as an illustration.>

A scheme to enable the SD-2 biped robot to climb sloping surfaces is proposed. By means of sensing devices, namely position sensors on the joints and force sensors underneath the heel and toe, the robot is able to detect the transition of the supporting terrain from a flat floor to a sloping surface. An algorithm is developed for the biped robot control system to evaluate the inclination of the supporting foot and the unknown gradient, and a compliant motion scheme is then used to enable the robot to transfer from level walking to climbing the slope. While the robot walks on the slope, the gait synthesis is a simple modification to the one used for level walking. Experiments with the SD-2 biped robot show that the overall scheme, while simple to implement, is powerful and reliable enough to permit walking from level to slope or vice versa. Finally, it is argued that the proposed mechanism can be extended to quasi-dynamic and dynamic gaits.>

Two coordination methods to enable a multilimbed robot to generate large Cartesian forces are presented. The first method selects an optimal configuration for the arm in terms of the force generation, regardless of the position where the force needs to be applied. The position and orientation of the main body are then adjusted to make the end-effector reach the required position. The second method treats the multilimbed robot as a single robotic system and applies the nonlinear programming technique to the system. As a result, optimal configurations are obtained for both the arm and mainbody at the same time.>

The dynamics and kinematic behavior of two coordinating robots in assembly is mathematically modeled. A typical assembly task can be decomposed into three phases: independent-motion phase, contact phase, and constraint-motion phase. Mathematical modeling is conducted in these phases as well. In the first phase, dynamic equations for robots holding an object are derived from individual dynamic equations of the robot and object. In the second phase, collision effects on the robot motions are studied and modeled. In the third phase, the emphasis is on the modeling of the kinematic constraints, from which a unified set of dynamic equations is obtained for the two coordinating robots.>

The load distribution problem for two coordinating industrial robots handling a single object is studied. When two industrial robots grasp a single object, the total number of degrees of freedom is usually greater than six. Thus, the joint torques of two robots for a required motion of the object is not unique. The redundant degrees of freedom may be used to optimize certain kind of performance. The least energy consumption is selected as the optimization criterion. Optimal algorithms without and with a bound on the joint torques are investigated. The results show that the algorithms are computationally complicated and not suitable for real-time applications. Alternatively, optimal algorithms are proposed for load distribution with minimum exerted forces on the object. These algorithms require less computational time, which makes them attractive for real-time applications.>

A scheme to enable a biped robot to climb sloping surfaces is proposed. By means of sensing devices, namely, position sensors on the joints and force sensors underneath the heel and toe, a biped robot called SD-2 is able to detect the transition of the supporting terrain from a flat floor to a sloping surface and to walk on the slope. Since in the frontal plane almost no difference exists for biped locomotion on level or on slope, the study concentrates on the sagittal plane.>

The design of a distributed computer system for coordinated-motion control of two industrial robots is discussed. The control mechanism for the coordinated-motion control has a hierarchical nature. For hierarchically related computational tasks, a tree-structured multicomputer system is selected as the architecture of the distributed computer system (DCS). The emphasis of the work is on the design of a distributed operating system (DOS) called REKCOR, which stands for REal-time Kernel for Coordinating Robots. It provides transparent operations to the users, and maintains the flexibility and modularity of the system. The successful design of REKCOR is verified by a practical experiment, in which the distributed computer system controls two industrial robots to successfully execute a coordinated-motion.

When two robot arms execute coordinated motions in assembly, collision between the two end-effectors cannot be avoided. The collision effects on the motions of two coordinating robots are studied in this paper. At the moment of a collision, an impulse exerted on the two end-effectors is generated. This impulse will cause abrupt velocity-changes in robot motions. The direction of the impulse depends on the relative orientation of the end-effectors when the collision takes place. The magnitude of the impulse, on the other hand, depends on the relative velocity between the two end-effectors immediately before the collision, the joint positions, and the inertia matrices of the two robots. The minimization of the abrupt velocity-changes is discussed. The study reveals that to reduce the abrupt velocity-changes, the two coordinating robots must be symmetrical in all aspects, and the direction vector of the impulse must be an eigenvector of the Jacobian inertia matrix for the robot arms. Since the two conditions impose constraints on the robot trajectories, the approach of minimization is valid only when the two robot arms follow pre-planned paths.

Tasks for two coordinated industrial robots always bring the robots in contact with a same object. Physically the three form a closed kinematic chain mechanism. When the chain is in motion, the positions and orientations of the two robots must satisfy a set of holonomic equality constraints for every time instant. To eliminate motion errors between them, one of them is assigned to carry the major part of the task. Its motion is planned accordingly. The motion of the second robot is to follow that of the first robot as specified by the relations of the joint velocities derived from the constraint conditions. Thus if any modification of the motion is needed in real time, only the motion of the first robot is modified. The modification for the second robot is done implicitly through the constraint conditions. Specifically, when the joint displacements, velocities, accelerations and torques/forces of the first robot are known for the planned or modified motion, the corresponding variables for the second robot can be determined through the constrained relations.

When two robot arms execute coordinated motions in assembly, collision between the two end-effectors cannot be avoided. The collision effects on the robot motions are studied in this paper. Firstly, the joint velocities of the two coordinating arms have abrubt changes at the instant of collision. Secondly, an impulsive force exerted on the end-effector is generated. The magnitude and the direction of the impulsive force depend on the relative position and orientation between two collideing end-effectors. Therefore the impulsive force can be used to detect the position and orientation errors between two coordinating arms. A wrist force sensor is suggested to be installed at the wrist of each robot arm. The information produced by the wrist force sensors can then be used to adjust the relative position and orientation of the two robot arms in assembly.

The computation of applied torques in real time in the dynamic control of a strongly coupled multibody linkage system is complicated and time consuming. The Newton-Euler state-space formulation is used for computing the dynamics. By using this formulation, the backward recursion for calculating angular velocities and angular accelerations is eliminated. The calculation of linear acceleration is simplified, and it involves only two steps. This reduction in the height of the evaluation tree in calculating the applied torques makes parallel processing more effective. A multiprocessor system composed of a central CPU and a group of satellite CPU's is suggested for implementing the computations. The task of each satellite CPU is to take care of one link of the system by calculating all its related data. The central CPU's task is to compute the applied torques. Because of this arrangement, the required software system is the same for all satellite CPU's. This modularity reduces the burden of software design greatly. The proposed multiprocessing scheme results in a flexible and modular system which is adaptive for a variety of dynamic configurations. Computer simulation results of this strategy are presented to show that the suggested multiprocessing scheme achieves a good speedup factor over the uniprocessing system.

A loosely coupled multiprocessor system dedicated to dynamic control of multilink system is designed. The multiprocessor system includes one central processor and a group of satellite processors. For the simplicity and flexibility of the design, both the system hardware and software are designed in a modular fashion. How to use this multiprocessor to control a multilink system is also presented in this paper. Newton-Euler state space formulation is used to derive equations of motion and a nonlinear control scheme is used for control of a multilink system. For the control scheme selected, task scheduling of parallel computation among the processors is studied. Finally, a two-link robotic system is used as an example to demonstrate the effectiveness of the multiprocessor system and the task scheduling scheme.

In some comercially available industrial robots such as the Cincinnati T3, and the Bendix MA-510, the actuator of the third joint, which is one of the larger actuators, is mounted on the same platform as that of the second joint in order to reduce the load on the second joint. With this arrangement, the torque that moves the third moving link is transmitted through a four-bar or five-bar linkage mechanism, which is a closed-chain structure having a planar motion. Although a variety of computational schemes for the input joint torques/forces of industrial robots having open-chain mechanisms can be found in the literature, an efficient method of computation for robots with closed kinematic chain mechanisms is not available. A computationally efficient scheme is presented for industrial robots having three-dimensional closed-chain linkages. First the closed-chain is virtually cut open, and the kinematics of the virtual open-chain mechanism are analyzed. The holonomic constraints are applied to the virtually cut joint. As a result, the spatial closed-chain linkage can be considered as a tree-structured open-chain mechanism with kinematic constraints. Based on the known recursive Newton-Euler formulation, a computational scheme is developed for industrial robots having three-dimensional closed kinematic chain mechanisms. Two examples are given illustrating the approach.

Impact effects of biped contact with the environment are modeled and studied. A biped system is subjected to an instant velocity change at the moment of impact with the environment. This instant velocity change is derived as a function of the biped state and the contact speed. The effects of the impact on the state as well as on the constraints are studied in biped landing on heels and toes simultaneously or on toes first. Large angular velocities may be developed by a biped immediately after the impact. The control strategy that is called for in this case is zero final velocities and a somewhat arbitrary final position. Rate feedback and nonlinear position feedback are employed for stability. This type of feedback control is proved effective by a digital computer simulation of the biped system. The action of plantar fascia during toe landing is represented by a spring and dashpot pair. The arch of the foot is prevented by this action from collapsing. Digital computer simulations of toe landing are presented.