System provenance forensic analysis has been studied by a large body of research work. This area needs fine granularity data such as system calls along with event fields to track the dependencies of events. While prior work on security datasets has been proposed, we found a useful dataset of realistic attacks and details that can be used for provenance tracking is lacking. We created a new dataset of eleven vulnerable cases for system forensic analysis. It includes the full details of system calls including syscall parameters. Realistic attack scenarios with real software vulnerabilities and exploits are used. Also, we created two sets of benign and adversary scenarios which are manually labeled for supervised machine-learning analysis. We demonstrate the details of the dataset events and dependency analysis.

Programming courses in colleges often involve a myriad of coding assignments, which brings heavy grading workloads for instructors. To alleviate this problem, automatic programming evaluation tools are becoming more of a requirement than an option. However, after considering the actual requirements in our teaching practice, we have noticed that the current solutions still suffer from shortcomings and limitations. In the process of addressing the challenges, we propose and implement a brand new code assessment application based on PowerShell, which shows both extendibility and configurability. In particular, we integrate both black-box testing and the lexical analysis into the system, thus achieving a customized solution to meet specific requirements. This paper presents the architecture and design of our automatic code assessment application. Furthermore, we conduct empirical evaluations on the proposed system following the Technology Acceptance Model, and also investigate the drawbacks of manual assessment of coding assignments in terms of reliability and fairness. Finally, the evaluations demonstrate the effectiveness of our proposed auto-grader in facilitating the code assessment targeting college-level programming courses.

Deep Metric Learning (DML) models often require strong local and global representations, however, effective integration of local and global features in DML model training is a challenge. DML models are often trained with specific loss functions, including pairwise-based and proxy-based losses. The pairwise-based loss functions leverage rich semantic relations among data points, however, they often suffer from slow convergence during DML model training. On the other hand, the proxy-based loss functions often lead to significant speedups in convergence during training, while the rich relations among data points are often not fully explored by the proxy-based losses. In this paper, we propose a novel DML approach to address these challenges. The proposed DML approach makes use of a hybrid loss by integrating the pairwise-based and the proxy-based loss functions to leverage rich data-to-data relations as well as fast convergence. Furthermore, the proposed DML approach utilizes both global and local features to obtain rich representations in DML model training. Finally, we also use the second-order attention for feature enhancement to improve accurate and efficient retrieval. In our experiments, we extensively evaluated the proposed DML approach on four public benchmarks, and the experimental results demonstrate that the proposed method achieved state-of-the-art performance on all benchmarks.

In this paper, we propose a simple and effective method to properly assign weights to the query samples and compute aggregated matching scores using these weights in multi-query object matching. Multi-query object matching commonly exists in many real-life problems such as finding suspicious objects in surveillance videos. In this problem, a query object is represented by multiple samples and the matching candidates in a database are ranked according to their similarities to these query samples. In this context, query samples are not equally effective to find the target object in the database, thus one of the key challenges is how to measure the effectiveness of each query to find the correct matching object. So far, however, very little attention has been paid to address this issue. Therefore, we propose a simple but effective way, Inverse Model Frequency (IMF), to measure of matching effectiveness of query samples. Furthermore, we introduce a new score aggregation method to boost the object matching performance given multiple queries. We tested the proposed method for vehicle re-identification and image retrieval tasks. Our proposed approach achieves state-of-the-art matching accuracy on two vehicle re-identification datasets (VehicleID/VeRi-776) and two image retrieval datasets (the original & revisited Oxford/Paris). The proposed approach can seamlessly plug into many existing multi-query object matching approaches to further boost their performance with minimal effort.

Modern FPGAs often contain heterogeneous architectures and clocking resources which must be considered to achieve desired solutions. As the design complexity keeps growing, placement has become critical for FPGA timing closure. In this paper, we present an analytical placement algorithm for heterogeneous FPGAs to optimize its worst slack and clock constraints simultaneously. First, a heterogeneity-aware and memory-friendly delay model is developed to accurately and rapidly assess each connection delay. Then, a two-stage clock region refinement method is presented to effectively resolve the clock and resource violations. Finally, we develop a novel timing-based co-optimization method to generate optimized placement without any clocking violations. Compared with the state-of-the-art placer based on the advanced commercial tool Xilinx Vivado 2019.1 with the Xilinx 7 Series FPGA architecture, our algorithm achieves the best worst slack and routed wirelength while satisfying all clock constraints.

As the feature sizes keep shrinking, interconnect delays have become a major limiting factor for FPGA timing closure. Traditional placement algorithms that address wirelength alone are no longer sufficient to close timing, especially for the large-scale heterogeneous FPGAs. In this paper, we resolve the crucial FPGA placement problem by optimizing wirelength and timing simultaneously. First, a smoothed routing-architecture-aware timing model is proposed to accurately estimate each interconnect delay. Then, a timing-driven delay look-up table is constructed to further speed up delay access. Finally, we present an effective wirelength and timing co-optimization strategy to produce high-quality placements without timing violations. Compared with Vivado 2019.1 on Xilinx benchmark suites for xc7k325t device, experimental results show that our algorithm achieves not only a 6.6% improvement in worst slack but also a 3.2% reduction for routed wirelength.

Prediction of protein subcellular location has currently become a hot topic because it has been proven to be useful for understanding both the disease mechanisms and novel drug design. With the rapid development of automated microscopic imaging technology in recent years, classification methods of bioimage-based protein subcellular location have attracted considerable attention for images can describe the protein distribution intuitively and in detail. In the current study, a prediction method of protein subcellular location was proposed based on multi-view image features that are extracted from three different views, including the four texture features of the original image, the global and local features of the protein extracted from the protein channel images after color segmentation, and the global features of DNA extracted from the DNA channel image. Finally, the extracted features were combined together to improve the performance of subcellular localization prediction. From the performance comparison of different combination features under the same classifier, the best ensemble features could be obtained. In this work, a classifier based on Stacked Auto-encoders and the random forest was also put forward. To improve the prediction results, the deep network was combined with the traditional statistical classification methods. Stringent cross-validation and independent validation tests on the benchmark dataset demonstrated the efficacy of the proposed method.

In recent years, we have witnessed an increasing demand to process big data in numerous applications. It is observed that there often exist substantial amounts of repetitive data in different portions of a big data repository/dataset for applications such as genome sequence analyses. In this paper, we present a novel method, called the VA-Store, to reduce the large space requirement for repetitive data in prevailing genome sequence analysis tasks using k-mers (i.e., subsequences of length k) with multiple k values. The VA-Store maintains a physical store for one portion of the input dataset (i.e., k0-mers) and supports multiple virtual stores for other portions of the dataset (i.e., k-mers with k ≠ k0). Utilizing important relationships among repetitive data, the VA-Store transforms a given query on a virtual store into one or more queries on the physical store for execution. Both precise and approximate transformations are considered. Accuracy estimation models for approximate solutions are derived. Query optimization strategies are suggested to improve query performance. Our experiments using real and synthetic datasets demonstrate that the VA-Store is quite promising in providing effective storage and efficient query processing for solving a kernel database problem on repetitive big data for genome sequence analysis applications.

The evaluation models for long time historical data is important in many applications. In this study, based on Age measure defined by Yager, we propose the definitions of Age Sequence and Age Series. Then, we provide a Generalized Recursive Smoothing method. Some classical smoothing models in evaluation problems can be seen as special cases of Generalized Recursive Smoothing method. In order to obtain more reasonable and effective aggregation results of the historical data, we propose some different Age Sequences, e.g., the Generalized Harmonic Age Sequence and p Age Sequence, which theoretically can provide infinite more recursive smoothing methods satisfying different preferences of decision makers.

In multicriteria group decision-making problems, we need to determine the relative importances among criteria as well as among experts. When doing so, however, we often face the situations where consensuses within experts over different criteria need to be considered, and where uncertainties arise when experts do evaluations. Therefore, we need some special and reasonable methods to generate weights in such situations. In this study, three elaborately devised methods suggest ways to generate relative importance among experts, criteria, and the combination of them, respectively. The first one elicits the consensus extents within experts over different criteria, by which it can generate suitable weights among different criteria. The second one fully considers the uncertain nature when experts do evaluations, and proposes a fuzzy model which can generate weighting vector among experts according to the certainty degrees of valuations given by all the experts. In the last method, when relative importances among both experts and criteria are predetermined in the form of two capacities with dimensionnandm, respectively, we find an interesting mechanism to successfully melt them into onenm-dimensional capacity which is based on given cognitive strength and on the proposed concept of compromised active/passive consensus.

Understanding a power wheelchair users mobility characteristics is critical because mobility is an important factor for social participation and quality of life of an individual. Although power wheelchairs can improve the mobility for people with disabilities, research has shown that power wheelchair users tend to live an inactive lifestyle. A sedentary lifestyle exposes wheelchair users to a greater risk of secondary health issues, such as cardiovascular diseases, obesity, diabetes, etc. Therefore, it is critical to assess wheelchair users mobility to ensure that they maintain an active lifestyle. However, existing health tracking applications are not suitable for power wheelchair users. They either require sensors to be installed on the wheels of a wheelchair (hence bringing installation and maintenance burdens) or are designed for able individuals by detecting the users steps, whose characteristics are significantly different from the dynamics of a power wheelchair. Furthermore, data captured by the inertial sensors (e.g., accelerometer or gyroscope) demonstrates a wide variety of patterns owing to different terrains on which the wheelchair travels. In this study, we propose to use the accelerometer in a smartphone for data collection, and employ mathematics and physics techniques to process and transform the raw data so that patterns intrinsic to wheelchair maneuvers are revealed. Based on the processed data, we developed a learning-based approach to analyze wheelchair users mobility by leveraging such patterns. We have conducted a sequence of experiments to evaluate the proposed approach. Experimental results showed that our approach correctly recognized all the bouts (segments of continuous movement), and achieved accurate measurements on bout maneuvering time and maximum period of continuous movement, which are critical indicators of a wheelchair users mobility.

We study the properties of OWA multiplication monoid. By introducing and-accumulation vectors of OWA operators, we consider the set of all n-dimensional OWA operators as a lattice. Then, we analyze some monotonicity properties of OWA operators based on an ordering induced by and-accumulation vectors. We also show that the lattice-theoretical operations are a kind of counterpart of the OWA multiplication monoid (and its dual, additive monoid). An example of using the OWA multiplication monoid and the lattice-theoretical structure in decision making problem is provided.

We propose the concept of ordered weighted averaging (OWA) generation function with some properties, illustrative examples, and usages. A number of new properties of OWA operators and relations between some well-known OWA operators are proposed and proved using OWA Generation Functions. We discuss some orness/andness adjustment methods for a predetermined OWA operator. A practical application about investment prediction problem is given with detailed illustration and analysis, which can show the advantages of the adjustment methods proposed in this paper. In particular, the concept consistent adjustment matrix is proposed and the interests and advantages of it can be shown by its special properties.

In this study, we propose the concept of piled ordered weighted averaging (OWA) operators, which generalize the centered OWA operators and also connect the step OWA operators with the Hurwicz OWA operators with given the orness degree. We propose a controllable algorithm to generate the family of piled OWA operators depending on their predefined three parameters: orness degree, step-like or Hurwicz-like degree, and the numbers of “supporting” vectors. By these preferences, we can generate infinite more piled OWA operators with miscellaneous forms, and each of them is similar to the well-known binomial OWA operator, which is very useful but only has one form corresponding to one given orness degree.

This paper presents a multimodal gesture recognition framework using video and floor pressure data. The key contribution of this research is to show that using additional floor pressure data significantly improves the recognition of visually ambiguous gestures. To effectively combine gesture recognition results from both the visual and pressure sensing modalities, we have adopted a two-stage cascaded sequential information integration scheme. In Stage-1 of the scheme, an unknown movement segment is first classified into a gesture group based on the visual features, and then in Stage-2, the input movement is further recognized as a gesture within the gesture group according to the pressure features. In the proposed framework, the hidden Markov models (HMMs) are used to model and recognize gestures using features from video and pressure data. The experimental results obtained on an in-house video and floor pressure gesture dataset demonstrate the efficacy of the proposed multimodal gesture recognition framework.

Combining outputs of a pool of individual classifiers appropriately, as a hot research topic of pattern classification, can generate statistically significant increase in classification performances. During the last decades, several fusion algorithms were presented, but few of those focus on two-class classification which possesses wide application area such as sentiment classification, cancer differentiation and so on. Thus the main purpose of this paper is to develop a highly effective fusion algorithm, i.e. intuitionistic fuzzy reasoning fusion algorithm, to increase the performance of a multiple two-class classifiers system. The outputs of component classifiers are represented by a set of intuitionistic fuzzy values at first and the fusion process is interpreted as aggregation of intuitionistic fuzzy information. The proposed algorithm includes three versions using the intuitionistic fuzzy arithmetic average operator, intuitionistic fuzzy weighted average operator and induced intuitionistic fuzzy ordered weighted average aggregation operator, respectively. The proposed fusion algorithm can combine both evidences of the hypothesis that a test pattern belongs to a class and evidences of the hypothesis that the test pattern does not belong to the class at the same time. We compare the versions of our algorithm with four other fusion techniques on five open-access datasets. The proposed algorithm exhibits good predictive abilities, compared to the best individual classifiers and other comparable fusion techniques. Further, the experiments show some interesting results about measuring and weighting component classifiers.

This paper presents a robust framework for online full-body gesture spotting from visual hull data. Using view-invariant pose features as observations, hidden Markov models (HMMs) are trained for gesture spotting from continuous movement data streams. Two major contributions of this paper are 1) view-invariant pose feature extraction from visual hulls, and 2) a systematic approach to automatically detecting and modeling specific nongesture movement patterns and using their HMMs for outlier rejection in gesture spotting. The experimental results have shown the view-invariance property of the proposed pose features for both training poses and new poses unseen in training, as well as the efficacy of using specific nongesture models for outlier rejection. Using the IXMAS gesture data set, the proposed framework has been extensively tested and the gesture spotting results are superior to those reported on the same data set obtained using existing state-of-the-art gesture spotting methods.

Contemporary database applications often perform queries in hybrid data spaces (HDS) where vectors can have a mix of continuous valued and non-ordered discrete valued dimensions. To support efficient query processing for an HDS, a robust indexing method is required. Existing indexing techniques to process queries efficiently either apply to continuous data spaces (e.g., the R-tree) or non-ordered discrete data spaces (e.g., the ND-tree). No techniques directly indexing vectors in HDSs have been reported in the literature. In this paper, we propose a new multidimensional indexing technique, called the C-ND tree, to directly index vectors in an HDS. To build such an index, we first introduce some essential geometric concepts (e.g., hybrid bounding rectangle) in HDSs. The C-ND tree structure and the relevant tree building and query processing algorithms based on these geometric concepts in HDSs are then presented. Strategies have been suggested to make the values in continuous dimensions and non-ordered discrete dimensions comparable and controllable. Novel node splitting heuristics which exploit characteristics of both continuous and discrete dimensions are proposed. Performance of the C-ND tree is compared with that of linear scan, R*-tree and ND-tree using range queries on hybrid data. Experimental results demonstrate that the C-ND tree is quite promising in supporting range queries in HDSs.

This paper presents a multi-view articulated human motion tracking framework using particle filter with manifold learning through Gaussian process latent variable model. The dimensionality of the input image observation and joint angles are reduced using Gaussian process models to improve the tracking efficiency. The forward and backward mappings between the two low dimensional spaces are then obtained using relevance vector machine and Batesian mixture of experts (BME). Improved sampling schemes and auto-initialization are obtained using BME.Without using a 3D body model, effective likelihood evaluation is obtained through RVM using images from multiple views. Tracking results obtained using real videos with complex dance movement show the efficacy of the proposed approach.

This paper presents a multi-view approach to the tracking of people location and orientation. To achieve efficient and accurate likelihood evaluation, a novel likelihood computation method is proposed. Mixtures of Gaussian (MoG) are used to represent the color models of subjects. The scaled unscented transformation is used to project the MoG color models onto the image plane to predict the color distribution for a motion sample. The efficacy of the proposed approach is demonstrated by experiment results obtained using real videos.

In this paper, we present a monocular 3D arm movement tracking system using adaptive particle filter. The effective sample size (ESS) is analyzed in the adaptive particle filter to tackle the abrupt dynamic changes of the arm movement. Sample-efficiency-optimized auxiliary particle filter (SEO-APF) is invoked when low ESS is detected. In SEO-APF, the auxiliary variable weights are computed to minimize the true importance weight variance, so the tracking results and the efficiency of the particle filters are improved. Experimental results have demonstrated the efficacy of this approach for 3D arm movement tracking.

This paper presents an approach to clustering, tracking and recognizing footprints of a single subject from pressure data obtained using a pressure sensing floor. The proposed method clusters active footprint areas on the floor and recognizes and tracks the footprints according to their 2D shapes and geometrical relationships among foot clusters. Experimental results show the efficacy of the proposed approach.

This is a commentary paper on ldquoMulti-view access monitoring and singularization in interlocksrdquo. Stefano et al. (2008) presents a multi-view approach to access monitoring and classification of single or multiple occupants (singularization detection) in access restricted areas (interlocks). Two main contributions are made by Stefano et al. (2008). One is a robust background subtraction algorithm using multiple background models in various illuminations. This algorithm is used for floor area and floor border segmentation. The second contribution is to extract features from multiple views (two views in the experiments) for singularization detection.

Schema mapping plays a fundamental role in modern information systems. Mapping composition is an operator that combines a chain of successive schema mappings into a single schema mapping. By pre-computing the composed schema mapping, the system can achieve significant performance benefits. However, when a change occurs on any mapping in the chain, the composed schema mapping has to be maintained correspondingly. In this paper we consider a restricted form of the problem in the XML setting and propose an incremental maintenance approach. Specifically, given a chain of successive mappings, we transform intermediately them into trees that consist of atomic rules and then divide the composition into sub-compositions of the atomic rules. The dividing composition approach provides a fine-grained perspective of the composition relationships between the mappings. We depict such information through an auxiliary data structure called composition relationship graph (CRG). When changes occur on any mapping in the chain, the corresponding maintenance algorithms are developed based on the dividing approach and the CRG, which compute the changes on the composed mapping and then repair it into the new version, such that the computation involves only the atomic rules that are relevant with the maintenance. We evaluate our maintenance approach and report the first experiments results, which show that it is efficient.

We examine in detail some properties of gesture recognition models which utilize a reduced number of parameters and lower algorithmic complexity compared to traditional hidden Markov models. We show that the reduced parameter models are comparable to standard HMM-based gesture recognition models in their ability to effectively model gestures, and in some cases superior when training data is limited. We also show that in order to effectively differentiate similar gestures, a gesture recognition model must utilize a large number of states, a scenario which can only be adequately handled by reducer parameter methods to maintain real-time speeds.

In this paper, we propose a video-based full-body gesture recognition system independent of the view angle of the cameras. We performed multilinear analysis on the silhouette images of the static poses making up the gestures by tensor decomposition and projection. Each pair of silhouette images is projected to a view-invariant low dimensional pose coefficient vector space. These pose vectors are then used as input vectors in hidden Markov model (HMM) for gesture recognition. This system worked effectively in our experiments using real videos.

In this paper, we tackle robust human pose recognition using unlabelled markers obtained from an optical marker-based motion capture system. A coarse-to-fine fast pose matching algorithm is presented with the following three steps. Given a query pose, firstly, the majority of the non-matching poses are rejected according to marker distributions along the radius and height dimensions. Secondly, relative rotation angles between the query pose and the remaining candidate poses are estimated using a fast histogram matching method based on circular convolution implemented using the fast Fourier transform. Finally, rotation angle estimates are refined using nonlinear least square minimization through the Levenberg-Marquardt minimization. In the presence of multiple solutions, false poses can be effectively removed by thresholding the minimized matching scores. The proposed framework can handle missing markers caused by occlusion. Experimental results using real motion capture data show the efficacy of the proposed approach.

This paper deals with the issue of fast and accurate physical environment restoration in a real time biofeedback system for stroke patient rehabilitation. The biofeedback system provides an interactive multimodal environment for patients to practice functional therapeutic reaching and grasping tasks, while receiving different types of feedback which indicate measures of performances and information for subsequent movements. The development of such a system will reduce rehabilitation time, promote more extensive recovery and alleviate rehabilitation monotony.

In this paper, we introduce the semantic network model (SNM), a generalization of the hidden Markov model (HMM) that uses factorization of state transition probabilities to reduce training requirements, increase the efficiency of gesture recognition and on-line learning, and allow more precision in gesture modeling. We demonstrate the advantages both formally and experimentally, using examples such as full-body multimodal gesture recognition via optical motion capture and a pressure sensitive floor, as well as mouse/pen gesture recognition. Our results show that our algorithm performs much better than the traditional approach in situations where training samples are limited and/or the precision of the gesture model is high.

This paper presents an approach to robust 3D people tracking using multiple synchronized and calibrated cameras. The goal is to improve people tracking accuracy when the subjects being tracked partially occlude each other in some of the camera views. To achieve this goal, Monte Carlo fine-tuning is deployed to rectify 3D people locations obtained from partially occluded image observations. In our approach, Gaussian mixture models and axis-parallel ellipsoids are used to represent the appearance and the 3D body structures of the subjects, respectively. Related parameters are learned off-line. Experimental results obtained using real videos illustrate that the proposed approach is capable of accurate and robust 3D people tracking under partial or complete occlusions.

This paper introduces a framework to track 3D human movement using Gaussian process dynamic model (GPDM) and particle filter. The framework combines the particle filter and discriminative learning approaches so that the 3D human model is not needed and optimal proposal distribution can be used. The structure of the joint motion and appearance are modelled using GPDM in a low dimensional space. Relevance vector machine (RVM) is used to construct the regression mapping between image latent space and joint angle latent space using the small training data set. Backward mapping from appearance to motion latent space makes the samples better drawn according to the most recent observation. Forward mapping from joint angle to silhouettes makes computation fast without generating synthetic images in tracking for particle weight evaluation. The experimental results show that our approach can track 3D people movement accurately given noisy image and different subjects' movements.

In this paper, a Bayesian mixture expert (BME) framework for the estimation of 3D human poses from two uncalibrated wide-baseline cameras is presented. The two cameras will reduce the ambiguities of the pose estimation greatly and is easy to implement. BME is learnt to conduct multimodal pose estimation regression. K-means algorithm considering Euclidean distance and maximum-value distance for the joint angle vector is used for the initial clustering in BME learning. This will give the better cluster results to separate the ambiguous poses into different experts. Also a weighted PCA is implemented in an expectation-maximization (EM) framework to learn the parameters of the BME. This can reduce the dimension of the training data more effectively compared with global PCA. The system is trained with synthesized silhouettes from motion capture data. The experimental results on synthesized and real images illustrate that our approach does not need precise camera calibration and can estimate the poses effectively

In this paper, a robust approach to contour line extraction in cluttered images using context is presented. Object contours are often used as key features in model-based 2D and 3D tracking systems. In many applications, object contours can be approximated using line segments, e.g. in human limb tracking. In cluttered images, undesired edges other than the true contour lines often present severe disturbance for reliable object tracking. In our approach, we reduce the effect of unwanted edges by exploring context information, including edge orientation and previous contour line position and orientation. Experimental results have shown that by using context, the number of false contour lines can be reduced significantly, which will greatly improve the tracking performance.

Movement-based interactive dance has recently attracted great interest in the performing arts. While utilizing motion capture technology, the goal of this project was to design the necessary real-time motion analysis engine, staging, and communication systems for the completion of a movement-based interactive multimedia dance performance. The movement analysis engine measured the correlation of dance movement between three people wearing similar sets of retro-reflective markers in a motion capture volume. This analysis provided the framework for the creation of an interactive dance piece, Lucidity, which will be described in detail. Staging such a work also presented additional challenges. These challenges and our proposed solutions will be discussed. We conclude with a description of the final work and a summary of our future research objectives.

Similarity searches in multidimensional Non-ordered Discrete Data Spaces (NDDS) are becoming increasingly important for application areas such as bioinformatics, biometrics, data mining and E-commerce. Efficient similarity searches require robust indexing techniques. Unfortunately, existing indexing methods developed for multidimensional (ordered) Continuous Data Spaces (CDS) such as the R-tree cannot be directly applied to an NDDS. This is because some essential geometric concepts/properties such as the minimum bounding region and the area of a region in a CDS are no longer valid in an NDDS. Other indexing methods based on metric spaces such as the M-tree and the Slim-trees are too general to effectively utilize the special characteristics of NDDSs, resulting in nonoptimized performance. In this article, we propose a new dynamic data-partitioning-based indexing technique, called the ND-tree, to support efficient similarity searches in an NDDS. The key idea is to extend the relevant geometric concepts as well as some indexing strategies used in CDSs to NDDSs. Efficient algorithms for ND-tree construction and techniques to solve relevant issues such as handling dimensions with different alphabets in an NDDS are presented. Our experimental results on synthetic data and real genome sequence data demonstrate that the ND-tree outperforms the linear scan, the M-tree and the Slim-trees for similarity searches in multidimensional NDDSs. A theoretical model is also developed to predict the performance of the ND-tree for random data.

There is an increasing demand for similarity searches in a multidimensional non-ordered discrete data space (NDDS) from application areas such as bioinformatics and data mining. The non-ordered and discrete nature of an NDDS raises new challenges for developing efficient indexing methods for similarity searches. In this article, we propose a new indexing technique, called the NSP-tree , to support efficient similarity searches in an NDDS. As we know, overlap causes a performance degradation for indexing methods (e.g., the R-tree) for a continuous data space. In an NDDS, this problem is even worse due to the limited number of elements available on each dimension of an NDDS. The key idea of the NSP-tree is to use a novel discrete space-partitioning (SP) scheme to ensure no overlap at each level in the tree. A number of heuristics and strategies are incorporated into the tree construction algorithms to deal with the challenges for developing an SP-based index tree for an NDDS. Our experiments demonstrate that the NSP-tree is quite promising in supporting efficient similarity searches in NDDSs. We have compared the NSP-tree with the ND-tree, a data-partitioning-based indexing technique for NDDSs that was proposed recently, and the linear scan using different NDDSs. It was found that the search performance of the NSP-tree was better than those of both methods.

In this paper, we present a robust pause detection algorithm for an interactive dance system using 3D motion capture data. The algorithm uses joint angle and shape context as two types of feature vectors obtained from both labeled and unlabeled data streams to detect pauses of the mover in the space. By looking at variances of the feature vectors over a time window, the probability of pause is computed. Satisfactory interactive dance performances have been successfully created and presented using the reported system.

We present a robust framework for real-time processing of 3D motion capture data. It autonomously analyzes the input data to build a model of the observed subjects, and is expected to perform in situations where the observed features are unstable or frequently occluded. We have implemented the method and presently rely on a marker based motion capture system to provide unlabeled 3D coordinates of non-occluded markers. However, we hope this to be a step towards using regular video camera systems with feature detection to perform autonomous markerless motion capture.

Visualization and knowledge of detailed pressure information can play a vital role in multimodal sensing of human movement. We have designed a high-resolution pressure sensing floor prototype with a sensor density of one sensor per square centimeter that can provide real-time information about the location of the performer on the floor as well as the amount of pressure being exerted on the floor. Hardware and software have been developed for detecting, collecting, transmitting and rendering a graphical representation of the pressure data gathered from the sensors. This prototype can be easily reconfigured to cover large floor areas, and integrates closely with video, audio and motion-based sensing technologies to facilitate robust multimodal sensing. In our demonstrations, we show that the system accurately captures and transmits pressure information, and we illustrate how this forms a basis for a variety of applications including use in rehabilitation, virtual reality, entertainment, and children activity centers.

In this paper, we present a dance scoring system developed using marker-based motion capture. The dance score is outputted in form of Labanotation. Promising results have been obtained using the proposed dance scoring system

In this paper, the problem of simultaneous structure from motion estimation for multiple independently moving objects from a monocular image sequence is addressed. Two Bayesian algorithms are presented for solving this problem using the sequential importance sampling (SIS) technique. The empirical posterior distribution of object motion and feature separation parameters is approximated by weighted samples. The first algorithm addresses the problem when only two moving objects are present. A singular value decomposition (SVD)-based sample clustering algorithm is shown to be capable of separating samples related to different objects. A pair of SIS procedures is used to track the posterior distribution of the motion parameters. In the second algorithm, a balancing step is added into the SIS procedure to preserve samples of low weights so that all objects have enough samples to propagate empirical motion distributions. By using the proposed algorithms, the relative motions of all the moving objects with respect to the camera can be simultaneously estimated. Both algorithms have been tested on synthetic and real-image sequences. Improved results have been achieved.

Multimodal sensing has attracted much attention in solving a wide range of problems, including target detection, tracking, classification, activity understanding, speech recognition, etc. In surveillance applications, different types of sensors, such as video and acoustic sensors, provide distinct observations of ongoing activities. We present a fusion framework using both video and acoustic sensors for vehicle detection and tracking. In the detection phase, a rough estimate of target direction-of-arrival (DOA) is first obtained using acoustic data through beam-forming techniques. This initial DOA estimate designates the approximate target location in video. Given the initial target position, the DOA is refined by moving target detection using the video data. Markov chain Monte Carlo techniques are then used for joint audio-visual tracking. A novel fusion approach has been proposed for tracking, based on different characteristics of audio and visual trackers. Experimental results using both synthetic and real data are presented. Improved tracking performance has been observed by fusing the empirical posterior probability density functions obtained using both types of sensors.

In this paper, we propose an algorithm for robust 3D motion estimation of wide baseline cameras from noisy feature correspondences. The posterior probability density function of the camera motion parameters is represented by weighted samples. The algorithm employs a hierarchy coarse-to-fine strategy. First, a coarse prior distribution of camera motion parameters is estimated using the random sample consensus scheme (RANSAC). Based on this estimate, a refined posterior distribution of camera motion parameters can then be obtained through importance sampling. Experimental results using both synthetic and real image sequences indicate the efficacy of the proposed algorithm.

In this paper, we report a real-time gesture driven interactive system with multimodal feedback for performing arts, especially dance. The system consists of two major parts., a gesture recognition engine and a multimodal feedback engine. The gesture recognition engine provides real-time recognition of the performer's gesture based on the 3D marker coordinates from a marker-based motion capture system. According to the recognition results, the multimodal feedback engine produces associated visual and audio feedback to the performer. This interactive system is simple to implement and robust to errors in 3D marker data. Satisfactory interactive dance performances have been successfully created and presented using the reported system

This paper deals with phrase structure detection in contemporary western dance. Phrases are a sequence of movements that exist at a higher semantic abstraction than gestures. The problem is important since phrasal structure in dance, plays a key role in communicating meaning. We detect two fundamental dance structures - ABA and the Rondo, as they form the basis for more complex movement sequences. There are two key ideas in our work - (a) the use of a topological framework for deterministic structure detection and (b) novel phrasal distance metrics. The topological graph formulation succinctly captures the domain knowledge about the structure. We show how an objective function can be constructed given the topology. The minimization of this function yields the phrasal structure and phrase boundaries. The distance incorporates both movement and hierarchical body structure. The results are excellent with low median error of 7% (ABA) and 15% (Rondo).

A novel approach to Bayesian structure from motion (SfM) using inertial information and sequential importance sampling (SIS) is presented. The inertial information is obtained from camera-mounted inertial sensors and is used in the Bayesian SfM approach as prior knowledge of the camera motion in the sampling algorithm. Experimental results using both synthetic and real images show that, when inertial information is used, more accurate results can be obtained or the same estimation accuracy can be obtained at a lower cost.

The Euclidean distance measure has been used in comparing feature vectors of images, while the cosine angle distance measure is used in document retrieval. We theoretically analyze these two distance measures based on feature vectors normalized by image size and experiment with them in the context of a color image database. We find that the cosine angle distance, in general, works equally well for image databases. We show, for a given query vector, the characteristics of feature vectors that will be favored by one measure but not by the other. We compute k-nearest neighbors for query images using both Euclidean and cosine angle distance for a small image database. The experimental data corroborate our theoretical results.

We have analyzed the properties of the HSV (hue, saturation and value) color space with emphasis on the visual perception of the variation in hue, saturation and intensity values of an image pixel. We extract pixel features by either choosing the hue or the intensity as the dominant property based on the saturation value of a pixel. The feature extraction method has been applied for both image segmentation as well as histogram generation applications - two distinct approaches to content based image retrieval (CBIR). Segmentation using this method shows better identification of objects in an image. The histogram retains a uniform color transition that enables us to do a window-based smoothing during retrieval. The results have been compared with those generated using the RGB color space.

We present a new technique for detecting moving targets from image sequences captured by moving sensors. Feature points are detected and tracked through the image sequences. A validity vector is used to describe the consistency of the feature trajectories with sensor motion. By using the sequential importance sampling method, an approximation to the posterior distribution of the sensor motion and the validity vector is derived and the feature points belonging to the moving target are then segmented out. Real image examples are included.

In this paper the structure from motion (SfM) problem is addressed using sequential Monte Carlo methods. A new SfM algorithm based on random sampling is derived to estimate the posterior distributions of camera camera motion and scene structure for the perspective projection camera model. Experimental results show that challenging issues in solving the structure from motion problem including errors in feature tracking, feature occlusion, motion/structure ambiguity, processing mixed-domain sequences and handling mismatched features can be well modeled and effectively addressed using the proposed method.

In this paper a robust structure from motion (SfM) algorithm using a discrete H/sub /spl infin// filter is presented. Existing SfM algorithms do not work well when large motion or measurements modeling uncertainties are present. By using a H/sub /spl infin// filter these modeling uncertainties can be successfully handled. This algorithm has been tested on synthetic image sequences and results show the superiority of the H/sub /spl infin// filtering approach.

The reduction of inherent ambiguities in structure from motion (SfM) using inertial data is addressed. First, we show that the translation-rotation ambiguity in SfM from a noisy flow field computed from two frames can be completely eliminated by using noise free inertial rate data. Secondly, we show that the admissible solution space for SfM from noisy feature correspondences can be reduced by using inertial data.

Researchers in image processing and computer vision fields have agonized over the last twenty-five years, to come up with robust methods for the structure from motion (SFM) problem. Two dominant approaches, based on flow and feature correspondences have been pursued. Despite tremendous efforts, only limited success in special cases can be claimed. We feel that with the availability of newer, general, robust methods for computing optical flow, fast methods for estimating the focus of expansion (FOE), inexpensive cameras, and inertial information, robust, real-time solutions to this problem may be possible in the near future. Some of our recent work in these areas is presented.