This paper introduces a novel deep network for estimating depth maps from a light field image. For utilizing the views more effectively and reducing redundancy within views, we propose a view selection module that generates an attention map indicating the importance of each view and its potential for contributing to accurate depth estimation. By exploring the symmetric property of light field views, we enforce symmetry in the attention map and further improve accuracy. With the attention map, our architecture utilizes all views more effectively and efficiently. Experiments show that the proposed method achieves state-of-the-art performance in terms of accuracy and ranks the first on a popular benchmark for disparity estimation for light field images.

This paper proposes a low-cost and portable polycamera system and accompanying methods for capturing and synthesizing stereoscopic 360° panoramas. The polycamera consists of only four cameras with fisheye lenses. Synthesizing panoramas from only four views is challenging because the cameras view very differently and the captured images have significant distortions and color degradation including vignetting, contrast loss, and blurriness. For coping with these challenges, this paper proposes methods for rectifying the polyview images, estimating depth of the scene, and synthesizing stereoscopic panoramas. The proposed camera is compact in size, light in weight, and inexpensive. The proposed methods allow the synthesis of visually pleasing stereoscopic 360° panoramas using the images captured with the proposed polycamera. We have built a prototype of the polycamera and tested it on a set of scenes with different characteristics of depth ranges and depth variations. The experiments show that the proposed camera and methods are effective in generating stereoscopic 360° panoramas that can be viewed on popular virtual reality displays.

We propose CatAR, a novel stereoscopic augmented reality (AR) cataract surgery training system. It provides dexterous instrument tracking ability using a specially designed infrared optical system with 2 cameras and 1 reflective marker. The tracking accuracy on the instrument tip is 20 µm, much higher than previous simulators. Moreover, our system allows trainees to use and to see real surgical instruments while practicing. Five training modules with 31 parameters were designed and 28 participants were enrolled to conduct efficacy and validity tests. The results revealed significant differences between novice and experienced surgeons. Improvements in surgical skills after practicing with CatAR were also significant.

Heterogeneous CPU-GPU systems have recently emerged as an energy-efficient computing platform. A robust integrated CPU-GPU simulator is essential to facilitate researches in this direction. While few integrated CPU-GPU simulators are available, similar tools that support OpenCL 2.0, a widely used new standard with promising heterogeneous computing features, are currently missing. In this paper, we extend the existing integrated CPU-GPU simulator, gem5-gpu, to support OpenCL 2.0. In addition, we conduct experiments on the extended simulator to see the impact of new features introduced by OpenCL 2.0. Our OpenCL 2.0 compatible simulator is successfully validated against a state-of-the-art commercial product, and is expected to help boost future studies in heterogeneous CPU-GPU systems.

In this article, we propose a novel personalized ranking system for amateur photographs. The proposed framework treats the photograph assessment as a ranking problem and we introduce the idea of personalized ranking , which ranks photographs considering both their aesthetic qualities and personal preferences. Photographs are described using three types of features: photo composition , color and intensity distribution , and personalized features . An aesthetic prediction model is learned from labeled photographs by using the proposed image features and RBF-ListNet learning algorithm. The experimental results show that the proposed framework outperforms in the ranking performance: a Kendall's tau value of 0.432 is significantly higher than those obtained by the features proposed in one of the state-of-the-art approaches (0.365) and by learning based on support vector regression (0.384). To realize personalization in ranking, three approaches are proposed: the feature-based approach allows users to select photographs with specific rules, the example-based approach takes the positive feedback from users to rerank the photograph, and the list-based approach takes both positive and negative feedback from users into consideration. User studies indicate that all three approaches are effective in both aesthetic and personalized ranking.

The purpose of this paper is to optimize the stereoscopic 3D experience when users watch those contents. We propose two factors in stereoscopic experience: visual fatigue and depth perception. In order to optimize stereoscopic experience, we propose five principles to reduce visual fatigue and enhance depth perception and implement them by cropping and warping. Using our methods, time-consuming problems of existing view interpolation methods such as camera calibration, accurate dense depth map, and inpainting are avoided. We also design a GUI that enables users to efficiently edit the stereoscopic video, optimize the stereoscopic experience, and preview the stereoscopic result. The user study shows that our method is successful in optimizing stereoscopic experience. One example is shown in Figure 1.

In this work, we focus on clustering faces in home photos by face recognition technologies. We propose two methods to improve the approach based on a well-known algorithm, local binary patterns. The adoption of the partial matching metric improves the recognition accuracy under face pose variations, while the adoption of the Gabor filter improves the accuracy under noises and various illuminations. We evaluate our methods on two home photo sets. In both evaluations, the results show that our methods improve the performance in accuracy. Compared to baseline LBP methods, in both evaluations, the results show that our methods improve the performance in accuracy from 90.4% to 99.5% and from 94.7% to 99.6% in two home-photo data sets respectively. Even compared to Google Picasa, the number of clusters where there is only one photo (thus can not be merged with other clusters, also means not good), our methods show that the number of "single" clusters reduced by half can be achieved. Our experience also shows that GPU speedup for Gabor filter can reach 140 times, and the overall system plus clustering can thus have 10 times speedup for face recognition.

In this paper, we propose a novel personalized ranking system for amateur photographs. Although some of the features used in our system are similar to previous work, new features, such as texture, RGB color, portrait (through face detection), and black-and-white, are included for individual preferences. Our goal of automatically ranking photographs is not intended for award-wining professional photographs but for photographs taken by amateurs, especially when individual preference is taken into account.

This paper proposes a new approach for video stabilization. Most existing video stabilization methods adopt a framework of three steps, motion estimation, motion compensation and image composition. Camera motion is often estimated based on pairwise registration between frames. Thus, these methods often assume static scenes or distant backgrounds. Furthermore, for scenes with moving objects, robust methods are required for finding the dominant motion. Such assumptions and judgements could lead to errors in motion parameters. Errors are compounded by motion compensation which smoothes motion parameters. This paper proposes a method to directly stabilize a video without explicitly estimating camera motion, thus assuming neither motion models nor dominant motion. The method first extracts robust feature trajectories from the input video. Optimization is then performed to find a set of transformations to smooth out these trajectories and stabilize the video. In addition, the optimization also considers quality of the stabilized video and selects a video with not only smooth camera motion but also less unfilled area after stabilization. Experiments show that our method can deal with complicated videos containing near, large and multiple moving objects.

With the current widespread use of digital cameras, the process of selecting and maintaining personal photos is becoming an onerous task. To our knowledge, there has been little research on photo evaluation based on computational esthetics. Photographers around the world have established some general rules for taking good photos. Building upon artistic theories and human visual perception is difficult since the results tend to be subjective. Although automatically ranking award-wining professional photos may not be a sensible pursuit, such an approach may be reasonable for photos taken by amateurs. In the next section, we introduce rules for such a system.

Abstract This paper investigates a new approach for color transfer. Rather than transferring color from one image to another globally, we propose a system with a stroke‐based user interface to provide a direct indication mechanism. We further present a multiple local color transfer method. Through our system the user can easily enhance a defect (source) photo by referring to some other good quality (target) images by simply drawing some strokes. Then, the system will perform the multiple local color transfer automatically. The system consists of two major steps. First, the user draws some strokes on the source and target images to indicate corresponding regions and also the regions he or she wants to preserve. The regions to be preserved which will be masked out based on an improved graph cuts algorithm. Second, a multiple local color transfer method is presented to transfer the color from the target image(s) to the source image through gradient‐guided pixel‐wise color transfer functions. Finally, the defect (source) image can be enhanced seamlessly by multiple local color transfer based on some good quality (target) examples through an interactive and intuitive stroke‐based user interface.

We present a novel method for acquisition, modeling, compression, and synthesis of realistic facial deformations using polynomial displacement maps. Our method consists of an analysis phase where the relationship between motion capture markers and detailed facial geometry is inferred, and a synthesis phase where novel detailed animated facial geometry is driven solely by a sparse set of motion capture markers. For analysis, we record the actor wearing facial markers while performing a set of training expression clips. We capture real-time high-resolution facial deformations, including dynamic wrinkle and pore detail, using interleaved structured light 3D scanning and photometric stereo. Next, we compute displacements between a neutral mesh driven by the motion capture markers and the high-resolution captured expressions. These geometric displacements are stored in a polynomial displacement map which is parameterized according to the local deformations of the motion capture dots. For synthesis, we drive the polynomial displacement map with new motion capture data. This allows the recreation of large-scale muscle deformation, medium and fine wrinkles, and dynamic skin pore detail. Applications include the compression of existing performance data and the synthesis of new performances. Our technique is independent of the underlying geometry capture system and can be used to automatically generate high-frequency wrinkle and pore details on top of many existing facial animation systems.

In this paper, we present an interactive system to transfer and concatenate motions in different articulated figures, such as human and dog. This system first constructs a union skeleton that contains both bone structures with bone correspondence manually assigned by users, and then aligns the initial poses of these two skeletons. With the initial poses aligned, motion data can be transferred between them. In addition, by blending similar poses of different skeletons, we can produce a seamless transition sequence which can be used to drive a meta-mesh and generate an animated morphing result.

This paper presents a new technique for rendering bidirectional texture functions (BTFs) at different levels of detail (LODs). Our method first decomposes each BTF image into multiple subbands with a Laplacian pyramid. Each vector of Laplacian coefficients of a texel at the same level is regarded as a Laplacian bidirectional reflectance distribution function (BRDF). These vectors are then further compressed by applying principal components analysis (PCA). At the rendering stage, the LOD parameter for each pixel is calculated according to the distance from the viewpoint to the surface. Our rendering algorithm uses this parameter to determine how many levels of BTF Laplacian pyramid are required for rendering. Under the same sampling resolution, a BTF gradually transits to a BRDF as the camera moves away from the surface. Our method precomputes this transition and uses it for multiresolution BTF rendering. Our Laplacian pyramid representation allows real-time anti-aliased rendering of BTFs using graphics hardware. In addition to provide visually satisfactory multiresolution rendering for BTFs, our method has a comparable compression rate to the available single-resolution BTF compression techniques.

Summary: A web-based three-dimensional (3D) protein retrieval system is available for protein structure data including all PDB and FSSP dataset. In this system, we use a visual-based matching method to compare the protein structure from multiple viewpoints. It takes less than three seconds for each query with 90% accuracy on an average. Availability: The web-based query interface and downloadable files can be accessed via http://3d.csie.ntu.edu.tw/ProteinRetrieval/ Contact: [email protected] Supplementary information: Further details of the proposed method are available at http://graphics.csie.ntu.edu.tw/~jsyeh/3Dprotein/

In this paper, we present a new method for surface extraction from volume data which preserves sharp features, maintains consistent topology and generates surface adaptively without crack patching. Our approach is based on the marching cubes algorithm, a popular method to convert volumetric data to polygonal meshes. The original marching cubes algorithm suffers from problems of topological inconsistency, cracks in adaptive resolution and inability to preserve sharp features. Most of marching cubes variants only focus on one or some of these problems. Although these techniques could be combined to solve these problems altogether, such a combination might not be straightforward. Moreover, some feature-preserving variants introduce an additional problem, inter-cell dependency. Our method provides a relatively simple and easy-to-implement solution to all these problems by converting 3D marching cubes into 2D cubical marching squares, resolving topology ambiguity with sharp features and eliminating inter-cell dependency by sampling face sharp features. We compare our algorithm with other marching cubes variants and demonstrate its effectiveness on various applications.

Conceptual Farm is a virtual reality platform for generating and observing the behaviors of different autonomous characters. By providing (1) descriptions for characters' behaviors and (2) 3D animations and sound, life-like characters in a realistic habitat can be created, modified, and interact with both users and other characters in real time. The flexible, manageable and scalable nature of Conceptual Farm leads to its desirability in zoological research, general education, game and film production, and even decorative arts.

Digital sculpting is a new trend for creating 3D models, but its application in the manipulation of volumetric data raises several issues that need to be addressed. With the extended marching cubes algorithm (EMC), sharp features of 3D models are well preserved. Additionally, the dynamic nature of modifying models in real time needs to be dealt with in sculpting applications: since the sampling of sharp features is implicit, direct modification on cell data will cause problems. A feature refinement strategy is proposed to preserve the dynamically modified model correct, and efficiently. Overall, the proposed methods provide an adaptive resolution and feature-preserved sculpting system that handles dynamic behavior in real-time performance.

In this paper, we propose an appearance representation for general complex materials which can be applied in real-time rendering framework. By combining a single parametric shading function (such as the Phong model) and the proposed spatial-varying residual function (SRF), this representation can recover the appearance of complex materials with little loss of visual fidelity. The difference between the real data and the parametric shading is directly fitted by a specific function for easy reconstruction. It is simple, flexible and easy to be implemented on programmable graphics hardware. Experiments show that the mean square error (MSE) between the reconstructed appearance and real photographs is less than 5%.

A method is proposed in this paper to automatically generate the animation skeleton of a model such that the model can be manipulated according to the skeleton. With our method, users can construct the skeleton in a short time, and bring a static model both dynamic and alive. The primary steps of our method are finding skeleton joints, connecting the joints to form an animation skeleton, and binding skin vertices to the skeleton. Initially, a repulsive force field is constructed inside a given model, and a set of points with local minimal force magnitude are found based on the force field. Then, a modified thinning algorithm is applied to generate an initial skeleton, which is further refined to become the final result. When the skeleton construction completes, skin vertices are anchored to the skeleton joints according to the distances between the vertices and joints. In order to build the repulsive force field, hundreds of rays are shot radially from positions inside the model, and it leads to that the force field computation takes most of the execution time. Therefore, an octree structure is used to accelerate this process. Currently, the skeleton generated from a typical 3D model with 1000 to 10000 polygons takes less than 2 minutes on a Intel Pentium 4 2.4 GHz PC.

A skeleton is a lower dimensional shape description of an object. The requirements of a skeleton differ with applications. For example, object recognition requires skeletons with primitive shape features to make similarity comparison. On the other hand, surface reconstruction needs skeletons, which contain detailed geometry information to reduce the approximation error in the reconstruction process. Whereas many previous works are concerned about skeleton extraction, most of these methods are sensitive to noise, time consuming, or restricted to specific 3D models. A practical approach for extracting skeletons from general 3D models using radial basis functions (RBFs) is proposed. A skeleton generated with this approach conforms more to the human perception. Given a 3D polygonal model, the vertices are regarded as centers for RBF level set construction. Next, a gradient descent algorithm is applied to each vertex to locate the local maxima in the RBF; the gradient is calculated directly from the partial derivatives of the RBF. Finally, with the inherited connectivity from the original model, local maximum pairs are connected with links driven by the active contour model. The skeletonization process is completed when the potential energy of these links is minimized.

Abstract A large number of 3D models are created and available on the Web, since more and more 3D modelling anddigitizing tools are developed for ever increasing applications. The techniques for content‐based 3D model retrievalthen become necessary. In this paper, a visual similarity‐based 3D model retrieval system is proposed.This approach measures the similarity among 3D models by visual similarity, and the main idea is that if two 3Dmodels are similar, they also look similar from all viewing angles. Therefore, one hundred orthogonal projectionsof an object, excluding symmetry, are encoded both by Zernike moments and Fourier descriptors as features forlater retrieval. The visual similarity‐based approach is robust against similarity transformation, noise, model degeneracyetc., and provides 42%, 94% and 25% better performance (precision‐recall evaluation diagram) thanthree other competing approaches: (1) the spherical harmonics approach developed by Funkhouser et al., (2) theMPEG‐7 Shape 3D descriptors, and (3) the MPEG‐7 Multiple View Descriptor. The proposed system is on the Webfor practical trial use ( http://3d.csie.ntu.edu.tw ), and the database contains more than 10,000 publicly available3D models collected from WWW pages. Furthermore, a user friendly interface is provided to retrieve 3D modelsby drawing 2D shapes. The retrieval is fast enough on a server with Pentium IV 2.4 GHz CPU, and it takes about2 seconds and 0.1 seconds for querying directly by a 3D model and by hand drawn 2D shapes, respectively. Categories and Subject Descriptors (according to ACM CCS): H.3.1 [Information Storage and Retrieval]: Indexing Methods

We develop an interactive haptic system, which can be a further aid for digital Chinese painting. When an artist is holding our force feedback device, one feels like holding a real painting brush with all the contact and bending forces, since the viscosity, friction, and the bending force of a brush touching the paper are simulated. First we derive a physical dynamics model as bending springs for bristles to construct a 3D brush. Then we simulate the ink-water transfer system for ink spreading and color blending. Our system is a real-time system and users can interact with it holding a digital brush supported by either a Phantom force feedback device or a "WACOM pressure sensing pen" on a tablet. A pilot experiment was conducted, and the results show that brush writing with haptic feedback is better than that of the same visual display but without haptic feedback.

A robust, accurate and inexpensive approach to estimate 3D facial motion from multi-view video is proposed, where two mirrors located near one's cheeks can reflect the side views of markers on one face. Nice properties of mirrored images are utilized to simplify the proposed tracking algorithm significantly, while a Kalman filter is employed to reduce the noise and to predict the occluded marker positions. More than 50 markers on one face are continuously tracked at 30 frames per second. The estimated 3D facial motion data has been practically applied to our facial animation system. In addition, the dataset of facial motion can also be applied to the analysis of co-articulation effects, facial expressions, and audio-visual hybrid recognition system.

In this paper, a lifelike talking head system is proposed. The talking head, which is driven by speaker independent speech recognition, requires only one single face image to synthesize lifelike facial expression. The proposed system uses speech recognition engines to get utterances and corresponding time stamps in the speech data. Associated facial expressions can be fetched from an expression pool and the synthetic facial expression can then be synchronized with speech. When applied to Internet, our web-enabled talking head system can be a vivid merchandise narrator, and only requires 50 K bytes/minute with an additional face image (about 40 Kbytes in CIF format, 24 bit-color, JPEG compression). The system can synthesize facial animation more than 30 frames/sec on a Pentium II 266 MHz PC.

The molecular binding problem, one of the most important problems in structure based drug design, can be formulated as a global energy optimization problem by using molecular mechanics. A novel computational algorithm is proposed to address the molecular binding problem. The algorithm is derived from genetic algorithms (GA) plus simulated annealing (SA) hybrid techniques, namely population based annealing genetic algorithms (PAG). We have applied the algorithm to find binding structures for three drug protein molecular pairs. One of the three drugs is an anti cancer drug methotrexate (MTX) and the other two are analogues of the antibacterial drug trimethoprim. Moreover, we have also studied two other well resolved ligand receptor molecular complex which are obtained from the Protein Data Bank (PDB): Thermolysin-HONH-benzylmalonyl-L-Ala-Gly-p-nitroanilide complex (5tln) and HIV-1 protease-Hydroxyethylene isostere inhibitor complex. Hydroxyethylene isostere inhibitor is one of new potential HIV-1 protease inhibitors synthesized. Through our experiments, all of the binding results not only keep the energy at low levels, but also have a promising binding geometrical structure in terms of number of hydrogen bonds formed.

A simple procedure that uses only three feature points to infer 3D head motion from consecutive video frames is presented. In this procedure, a feature triangle formed using the three feature points is automatically calibrated, and an iterative method that simulates steepest descent method is applied to estimate one's head motion. A prediction algorithm is adopted in occasional cases where the estimated result is not acceptable. This procedure has been applied to live video with an update rate of 7 frames/sec (250 frames/sec without feature extraction) on a Pentium-II 233 MHz PC, independent of cameras and users.

Image Talk uses a single image to automatically create talking sequences in real time. The image can be acquired from a photograph, video clip, or hand drawn characters. This interactive system accepts Chinese text and talks back in Mandarin Chinese, generating facial expression in real time. Image Talk analyzes Chinese text by converting it to a standard Pinyin system used in Taiwan and fetches the associated facial expressions from an expression pool. The expressions are synchronized with the sound and played back in a talking sequence. Image Talk also incorporates eye blinking, small scale head rotation and translation perturbations, to make the resulting sequence more natural. It is also easy to switch to any other face images. The result is quite entertaining, and can easily be used as a new human machine interface, as well as for lip sync in computer animated characters.

This paper proposes three new ideas and one revision concerning image metrics and their applications. Of most importance is the multiresolution shape metric, which measures distances according to image content shape information. Accompanied with it is a feature/non-feature image characterization philosophy. The second effort is the modification of an existing color metric, whose distance computation is dominated by color distribution of images. The third idea concerns the image querying behavior of the general public, so as to propose and design a versatile power metric, which holds the properties of both the shape metric and the color metric, for dealing with various kinds of hand-drawn queries. Finally, after applying the shape metric to video shot boundary detection, a multiresolution scene change detection algorithm is proposed. For now, we have implemented these metrics on a personal computer for two applications. One is for image database management with content-based indexing and the other is for video shot boundary detection. Experiments show that the speed and the accuracy of both image retrieval and scene change detection are quite good.

Article Free Access Share on Head motion and latency compensation on localization of 3D sound in virtual reality Authors: Jiann-Rong Wu Department of Psychology, National Taiwan University, Taiwan, R.O.C. Department of Psychology, National Taiwan University, Taiwan, R.O.C.View Profile , Cha-Dong Duh View Profile , Ming Ouhyoung View Profile , Jei-Tun Wu View Profile Authors Info & Claims VRST '97: Proceedings of the ACM symposium on Virtual reality software and technologySeptember 1997 Pages 15–20https://doi.org/10.1145/261135.261140Published:01 September 1997Publication History 8citation1,405DownloadsMetricsTotal Citations8Total Downloads1,405Last 12 Months44Last 6 weeks5 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Hand gesture is one of the most natural and expressive ways for the hearing impaired. However, because of the complexity of dynamic gestures, most researches are focused either on static gestures, postures, or a small set of dynamic gestures. As real-time recognition of a large set of dynamic gestures is considered, some efficient algorithms and models are needed. To solve this problem in Taiwanese Sign Language, a statistics based context sensitive model is presented and both gestures and postures can be successfully recognized. A gesture is decomposed as a sequence of postures and the postures can be quickly recognized using hidden Markov model. With the probability resulted from hidden Markov model and the probability of each gesture in a lexicon, a gesture can be easily recognized in a linguistic way in real-time.

Abstract Because of the view independence and photo realistic image generation in a diffuse environment, radiosity is suitable for an interactive walk through system. The drawback of radiosity is that it is time‐consuming in form factor estimation, and furthermore, inserting, deleting or moving an object makes the whole costly rendering process repeat itself. To solve this problem, we encapsulate necessary information for form factor calculation and visibility estimation in each object, which is called a reusable radiosity object. An object is defined as a cluster or clusters of triangles. Whenever a scene updates, the radiosity algorithm looks up the prestored information in each object, thus speeding itself up by two orders of magnitude. Besides, solution time based on cluster representatives is linear to the number of objects since each object is reusable, encapsulated with preprocessed data in every level of hierarchy. We also analyze the unregarded error on visibility estimation and propose a statistically optimal adaptive algorithm to maintain the same error for each link.

Article Free Access Share on A 3D tracking experiment on latency and its compensation methods in virtual environments Authors: Jiann-Rong Wu Communication & Multimedia Lab, Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan, R.O.C. Communication & Multimedia Lab, Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan, R.O.C.View Profile , Ming Ouhyoung Communication & Multimedia Lab, Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan, R.O.C. Communication & Multimedia Lab, Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan, R.O.C.View Profile Authors Info & Claims UIST '95: Proceedings of the 8th annual ACM symposium on User interface and software technologyDecember 1995 Pages 41–49https://doi.org/10.1145/215585.215650Published:01 December 1995Publication History 15citation863DownloadsMetricsTotal Citations15Total Downloads863Last 12 Months26Last 6 weeks5 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Abstract Many ways of communications are used between human and computer, while using gesture is considered to be one of the most natural way in a virtual reality system. Because of its intuitiveness and its capability of helping the hearing impaired or speaking impaired, we develop a gesture recognition system. Considering the world‐wide use of ASL (American Sign Language), this system focuses on the recognition of a continuous flow of alphabets in ASL to spell a word followed by the speech synthesis, and adopts a simple and efficient windowed template matching recognition strategy to achieve the goal of a real‐time and continuous recognition. In addition to the abduction and the flex information in a gesture, we introduce a concept of contact‐point into our system to solve the intrinsic ambiguities of some gestures in ASL. Five tact switches, served as contact‐points and sensed by an analogue to digital board, are sewn on a glove cover to enhance the functions of a traditional data glove.

Molecular binding, important in drug design, explores the accurate binding structures between molecules. This exploration can be formulated as a global optimization problem. However, the problem in molecular binding is that the search space is very large and the computational cost increases tremendously with the growth of the degrees of freedom. In this paper, we utilize a new algorithm called the annealing genetic algorithm to solve the global optimization problem in molecular binding. Using a protein with three anti-cancer drugs in our model, our algorithm can find a binding structure with a complicated energy computation within a couple of hours and the experimental results indicate that the solutions are reasonable.>

Abstract In this paper we propose a novel prediction method for the head motion using Grey System theory, where a 6D tracker is attached to an HMD on a user's head in virtual reality applications. Our prediction method using Grey System Model can greatly reduce the latency by at least one half and reduce image jittering. A system latency below 100 ms or even 50 ins can be achieved, even though without prediction the latency is around 200 ms. Using 6 points in prediction with Grey System Model is currently the best in tracker prediction as we tried from 2 points to 10 points. In order to measure the latency, we also propose a way to measure it in an HMD system precisely and conveniently. During the process, we have implemented four different prototypes respectively on a PC486, a SUN SparcStation10, an SGI IndigoR4000, and a high performance computer image generator. The computation complexity of our prediction method is relatively low and therefore real time requirement is easily met.

This paper is about using the sense of touch, the hapti c system, as part of our everyday interface wit h computationally created worlds .In Part I, we discuss a particular system, called Sandpaper, designed for experimenting with feeling texture .In part II, we discuss how control analysis helps us understand th e behavior of the types of hardware and software we use t o implement force display .We will not comprehensively review the literature of forc e display devices and their applications; descriptions can b e found in e .g .[Sutherland651, [No11721, [Batter721 , [Atkinson77l, [Brooks88j, [Ouh-young881, [Smith881 , Part IForce display technology works by using mechanica l actuators to apply forces to the user, By simulating th e physics of the user's virtual world, we compute these forces in real-time, then send them to the actuators so that the user feels them, The force display technology we use in th e Sandpaper system is a motor-driven two-degree of freedo m joystick (built by Max Behensky and Doug Milliken) .Th e joystick position is reported to the software, whic h computes the appropriate forces for the joystick's motors .Why create texture?Force display is especially useful for communicating surfac e texture and bulk properties of objects and environments a s well as dynamics of objects .In this way force display technologies augment the strengths of computer-generate d graphics and sound in creating convincingly realistic environments (figures 1, 4) .In the Sandpaper system, we use a novel technique to allo w the user to feel textures .We create very small virtua l

We began in 1967 a project to develop a haptic+display for 6-D force fields of interacting protein molecules. We approached it in four stages: a 2-D system, a 3-D system tested with a simple task, a 6-D system tested with a simple task, and a full 6-D molecular docking system, our initial goal. This paper summarizes the entire project---the four systems, the evaluation experiments, the results, and our observations. The molecular docking system results are new.Our principal conclusions are:• Haptic display as an augmentation to visual display can improve perception and understanding both of force fields and of world models populated with impenetrable objects.• Whereas man-machine systems can outperform computer-only systems by orders of magnitude on some problems, haptic-augmented interactive systems seem to give about a two-fold performance improvement over purely graphical interactive systems. Better technology may give somewhat more, but a ten-fold improvement does not seem to be in the cards.• Chemists using GROPE-III can readily reproduce the true docking positions for drugs whose docking is known (but not to them) and can find very good docks for drugs whose true docks are unknown. The present tool promises to yield new chemistry research results; it is being actively used by research chemists.• The most valuable result from using GROPE-III for drug docking is probably the radically improved situation awareness that serious users report. Chemists say they have a new understanding of the details of the receptor site and its force fields, and of why a particular drug docks well or poorly.• We see various scientific/education applications for haptic displays but believe entertainment, not scientific visualization, will drive and pace the technology.• The hardware-software system technology we have used is barely adequate, and our experience sets priorities for future development.• Some unexpected perceptual phenomena were observed. All of these worked for us, not against us.

A simplified docking problem is studied. The user attempts to find the potential-energy minimum in a 6-D space defined by six Hooke's-law springs attached to a manipulable object. This space has no other local minima. A pilot-controlled experiment with seven subjects, twelve trials each, showed that performance with a force display is better (p>

A real-time molecular docking system is developed that uses an electrically coupled remote manipulator as a force display. The system, which uses integrates interactive computer graphics and high-speed calculation of the interaction forces between a drug and a receptor site in a molecule, is designed to be a tool for molecular scientists. The manipulator is used to generate the forces and torques exerted on the drug molecule when it is aligned with the receptor site by the user's hand. The manipulator serves both as an input device for 6-D manipulation and as an output device for generating forces. Preliminary testing indicates that the system might enhance the biochemist's understanding and performance.>

An unrestricted Chinese text-to-speech system has been successfully constructed. The text input which is written in Mandarin Phonetic Symbols II (MPS II) instead of traditional Chinese orthography is analyzed by a simple parser. Then each syllable in the text is assigned proper duration and pitch, while its gain is modified, and its reflection coefficients interpolated or decimated. The stress, intonation and inter-syllable pause are also considered. In other words, a model of concatenating syllables of Mandarin Chinese based on simplified phonetic rules and some preliminary analysis of natural speech data is proposed. We believe that our model is sufficient compared with some more complicated approaches [1]. After editing the syllables according to our proposed synthesis rules, a continuous speech signal is synthesized, and then transmitted to speech interface card for speech output. Samples of synthesized speech by our text-to-speech system have been tested on subjects with satisfactory results. Evaluation of the system in intelligibility, comprehensibility and fluency has been conducted and an average of 96%, 95.9% and 75% respectively was reached.