Geo-tagged photo-based tourist attraction recommendation can discover users’ travel preferences from their taken photos, so as to recommend suitable tourist attractions to them. However, existing visual content-based methods cannot fully exploit the user and tourist attraction information of photos to extract visual features, and do not differentiate the significance of different photos. In this article, we propose multi-level visual similarity-based personalized tourist attraction recommendation using geo-tagged photos (MEAL). MEAL utilizes the visual contents of photos and interaction behavior data to obtain the final embeddings of users and tourist attractions, which are then used to predict the visit probabilities. Specifically, by crossing the user and tourist attraction information of photos, we define four visual similarity levels and introduce a corresponding quintuplet loss to embed the visual contents of photos. In addition, to capture the significance of different photos, we exploit the self-attention mechanism to obtain the visual representations of users and tourist attractions. We conducted experiments on two datasets crawled from Flickr, and the experimental results proved the advantage of this method.

With the development of technology, mobile and wearable devices can obtain many different kinds of real-time activities from users. In social domains, the sharing of real-time activities is proved to help users to understand the current state of others and promote social relationships. However, most studies focused on real-time simple activities sharing, and the willingness of sharing real-time daily activities has not been fully studied. In this paper, we conduct an in-situ study to explore the willingness of sharing real-time daily activities among friends and study the factors that could influence sharing willingness. We find that daily activity types, friend types, location types, time periods, and participants’ privacy attitude classifications could influence sharing willingness. Our findings provide a basis for real-time communication systems to support the sharing of different types of daily activities. Furthermore, we present some design implications to better support real-time daily activities sharing.

Topic models have achieved big success in recent years. To detect topics in a text stream, various online topic models have been proposed in the literature. The limitations of these works include that: 1) most of them run with fixed topic numbers and 2) the overlaps between the topics may enlarge in the evolving process. Hierarchical topic model is a candidate solution to these problems since it can reveal many useful relationships between the topics. These relationships can help to find high quality topics and reduce topic overlaps. In this paper, a knowledge-based semisupervised hierarchical online topic detection framework is proposed. The proposed framework can detect topics in an online hierarchical way. In addition, it has been proven that introducing external knowledge can improve the performance of text mining. Therefore, the knowledge from external knowledge sources and human experts are also integrated in the proposed framework. Experiments are conducted to evaluate the proposed framework with different metrics. The results show that compared with the baseline methods, our framework can achieve better performance with competitive time efficiency.

Complex activity recognition is a valuable issue in mobile and wearable computing. Since complex activities are strongly relevant to users' locations, location data can be used in complex activity recognition. Current activity recognition methods utilizing location data either do not make the most of location data or have a “cold-start” problem. In this paper, we propose CAROLINA (complex activity recognition using acceleration, vital sign, and location data) to recognize complex activities from wearable sensors. CAROLINA divides a map into grid cells and builds a grid cell-activity matrix by integrating check-in and ATUS (American Time Use Survey) datasets. Since some grid cells do not have check-in records, there are some null entries in the matrix, and all the entries are reconstructed by matrix factorization. Meanwhile, a POI (point of interest) category dataset is exploited to build a grid cell-grid cell similarity matrix, which is used to reduce the loss of matrix factorization. The results of experiments show that CAROLINA can effectively utilize location data to improve complex activity recognition performance.

Rare category detection aims to find interesting and statistically significant anomalies and incorporates ideas from active learning and semisupervised learning. The challenge of rare category detection is to find the rare classes of the anomalies in a data set where the data distribution is skewed. Most existing rare category detection methods suppose that the user knows the specific number of all classes in advance, which cannot be satisfied in most real scenarios. In this paper, we propose a new rare category detection framework composed of active learning and semisupervised hierarchical density-based clustering. The advantage of our method is that it is prior free and can benefit the rare category detecting process with the labeled data. In addition, the proposed framework can handle tasks with nonlinear mappings, which increases the ability to find rare classes when the class boundary is sophisticated. Compared to existing methods, better results are achieved by our method on both real and synthetic data sets in the experiment.

Semantic human activity (SHA) refers to users' activities performed in their daily lives (e.g., having dinner, shopping, etc.). SHA recognition is a promising issue in wearable and mobile computing. Most existing methods represent a SHA based on a single view, e.g., representing a SHA as a combination of human body actions, representing a SHA as a distribution of latent semantics. Since SHAs are complicated in nature, single views lack the ability of comprehensively profiling SHAs. In this paper, we propose a bi-view semi-supervised learning based method for recognizing SHAs using accelerometers. First, we represent a SHA based on two different views. One view represents a SHA as a distribution of latent activities in an unsupervised manner, and the other view represents a SHA as a set of human crafted features extracted in a hierarchical way. Second, we use a semi-supervised learning framework, which exploits the complementary information provided by the two views, to improve the classification accuracy based on both labeled and unlabeled data. Extensive experiments show that representing SHAs based on bi-views is more effective than representing SHAs based on single views, and our method is able to yield a competitive SHA recognition performance.

With the advent of multi-touch tabletops, software developers are creating innovative interfaces for applications such as interactive map systems. In this article, we summarized three representative interfaces from prevalent electronic map applications and literatures—gesture based, widget based, and hybrid, which supported both gestures and widgets—and then compared the effectiveness and the efficiency of these three interfaces by experiments. The results show that the hybrid interface had a better performance, requiring less task-completion time and operations, and causing less arm fatigue. The widget-based interface had the least distance traversed by fingers, but it required many tapping operations. The gesture-based interface was easier to learn compared to the widget-based interface, but the distance traversed during finger movements was the longest. Based on the findings from our experiments, we discuss the implications for the design of map exploration interfaces on multi-touch tabletops.

Mobile and wearable devices based activity recognition systems utilize built-in sensors to identify the activities performed by users pervasively. However, most of these systems do not explicitly present the sensing process to users and are prone to uncertainty. The presence of uncertainty makes users feel confused about the behaviors of activity recognition systems, which may affect the confidence of users. Uncertainty visualization has become an interesting research topic purporting to help users better understand systems. In this paper, we present an uncertainty visualization to reveal the process of mobile and wearable devices based activity recognition systems. We conducted an experiment to evaluate the uncertainty visualization by using a particular simulated mobile and wearable devices based activity recognition application. The results showed that the uncertainty visualization was effective in helping users understand and trust the judgments and inferences of the activity recognition application. Based on the advice of participants, we concluded a few directions to improve the uncertainty visualization.

The vast volumes of community-contributed geotagged photos (CCGPs) available on the Web can be utilized to make travel location recommendations. The sparsity of user location interactions makes it difficult to learn travel preferences, because a user usually visits only a limited number of travel locations. Static topic models can be used to solve the sparsity problem by considering user travel topics. However, all travel histories of a user are regarded as one document drawn from a set of static topics, ignoring the evolving of topics and travel preferences. In this paper, we propose a dynamic topic model (DTM) and matrix factorization (MF)-based travel recommendation method. A DTM is used to obtain the temporally fine-grained topic distributions (i.e., implicit topic information) of users and locations. In addition, a large amount of explicit information is extracted from the metadata and visual contents of CCGPs, check-ins, and point of interest categories datasets. The information is used to obtain user-user and location-location similarity information, which is imposed as two regularization terms to constraint MF. The proposed method is evaluated on a publicly available Flickr dataset. Experimental results demonstrate that the proposed method can generate significantly superior recommendations compared to other state-of-the-art travel location recommendation studies.

Air pollution has adverse effects on humans and ecosystem, and spatially fine-grained air quality information (i.e., the air quality information of every fine-grained area) can help people to avoid unhealthy outdoor activities. However, the number of air quality monitoring stations is usually limited, and thus spatially fine-grained air quality estimation is a challenging task. This paper proposes a method for inferring spatially fine-grained air quality information throughout a city. On one hand, since air quality is affected by multiple factors (e.g., factory waste gases and automobile exhaust fumes), this method employs various data sources, including traffic, road network, point of interests (POIs), and check-ins from social network services, which are related to air quality, to conduct the estimation. On the other hand, since the labeled data are highly limited due to the sparseness of monitoring stations, this method uses an improved ensemble semi-supervised learning (Semi-EP) to establish the relationship between the various data sources and urban air quality. Semi-EP firstly generates multiple classifiers from the original labeled data set and these classifiers are retrained in the iterative co-training process. Then, ensemble pruning technique is used to select the most-diverse subset from these multiple classifiers. This method is evaluated on the real-world dataset of Hangzhou city, China, and the experimental results have demonstrated its advantages over state-of-the-art methods.

Effectively utilizing multimodal information (e.g., heart rate and acceleration) is a promising way to achieve wearable sensor based human activity recognition (HAR). In this paper, an activity recognition approach MARCEL (Multimodal Activity Recognition with Classifier Ensemble) is proposed, which exploits the diversity of base classifiers to construct a good ensemble for multimodal HAR, and the diversity measure is obtained from both labeled and unlabeled data. MARCEL uses neural network (NN) as base classifiers to construct the HAR model, and the diversity of classifier ensemble is embedded in the error function of the model. In each iteration, the error of the model is decomposed and back-propagated to base classifiers. To ensure the overall accuracy of the model, the weights of base classifiers are learnt in the classifier fusion process with sparse group lasso. Extensive experiments show that MARCEL is able to yield a competitive HAR performance, and has its superiority on exploiting multimodal signals.

This article presents nonspeech audio (i.e., English-based spearcons and Chinese-based spearcons) to represent distance, and forward-direction for pedestrian navigation in an eyes-free environment. Experiment in the field is carried out with the involvement of 10 participants (i.e., native Chinese) using within-subject design to evaluate English-based spearcons, Chinese-based spearcons, and Chinese text-to-speech (TTS). Results from the experiment suggest that Chinese-based spearcons are efficient in task completion compared to Chinese TTS. Moreover, Chinese-based spearcons are more effective in conveying distance and forward-direction compared to English-based spearcons in pedestrian navigation. Overall, participants have shown their satisfaction with Chinese-based spearcons as an auditory feedback in pedestrian navigation.

With the prevalence of mobile social network services, people can post location-based questions on their social networks to satisfy their needs anytime anywhere. In this article, the authors study location-based questions that people post on microblogs, which is a popular form of social network service. The authors collected posts with geo-tags from Sina Weibo and conducted the study based on about a thousand location-based questions. Their results reveal unique characteristics of location-based questions by analyzing what people ask, how they ask, why they ask, and the context when asking. Location-based questions are closely related to people’s offline activities. Spatial restriction, subjectivity, interactivity, and propagation are the main characteristics that people value for choosing social networks to ask location-based questions. People also apply different phrasing skills to different types of questions. The questions people ask in different contexts also have different focuses. Based on their findings, the authors discuss practical design implications for social networks, location-based Q&A systems, and other applications with location-based features.

A personally semantic place is a space that is frequently visited by an individual user and carries important semantic meanings (e.g. home, work, etc.) to the user. Many location-aware applications could be greatly enhanced by the ability of automatic discovery of personally semantic places. The discovery of a user's personally semantic places involves obtaining the physical locations and semantic meanings of these places. In this paper, we propose approaches to address both of the problems. For the physical place extraction problem, a hierarchical clustering algorithm is proposed to firstly extract visit points from the GPS trajectories, and then clusters these visit points to form physical places. For the semantic place recognition problem, the temporal, spatial and sequential features in which the places have been visited are explored to categorize them into pre-defined types. An extensive set of experiments conducted based on a dataset of real-world GPS trajectories has demonstrated the effectiveness of the proposed approaches.

Knowledge workers frequently change activities, either by choice or through interruptions. With an increasing number of activities and activity switches, it is becoming more and more difficult for knowledge workers to keep track of their desktop activities. This article presents our efforts to achieve activity awareness through automatic classification of user's everyday desktop activities. For getting a deeper understanding, we investigate performance of various classifiers with respect to discriminative power of time-, interaction-, and content-based feature sets for different work scenarios and users. Specifically, by viewing an activity as a sequence of desktop interactions we present (1) a methodology for translating a user's desktop interactions into activities, (2) evaluation of the discriminative power of different activity features and feature types, and (3) analysis of supervised classification models for classifying desktop activity under two different scenarios, i.e., an activity-centric scenario and a user-centric scenario. The experiments are carried out on a real-world dataset, and the results show satisfactory accuracy using relatively few and simple types of features.

Road congestion has been one of the major issues in most metropolises, and thus, it is crucial to detect road congestions effectively and efficiently. Traditional solutions require the deployment of dedicated sensors on the roadside or on the vehicles, which suffer from high installation and maintenance costs and limited coverage. In this paper, we propose an alternative solution by exploiting the sensing ability of mobile phones. However, it is challenging to detect road congestions in a daily-living environment using undedicated mobile phones while guaranteeing energy efficiency. The proposed system only depends on the accelerometer and cellular signal, which have been proven to be energy efficient as compared with other built-in sensors (e.g., GPS). It consists of three interactive modules: (a) an accelerometer-based vehicular movement detection module for detecting the periods when the mobile phone user is traveling by vehicle; (b) a map-matching module relying on the cellular signal for determining the traveled road segments; and (c) a road congestion estimation module for inferring the congestion degree of the traveled road segments. We evaluated the proposed system based on real-world datasets, with promising results.

This article presents a new design of using nonspeech audio (i.e., earcons, spearcons, and short pulses) to represent distance and forward-direction for pedestrian navigation in eyes-free environment. Experiment in the field is carried out with the involvement of 15 participants using within-subject design to evaluate the newly developed earcons, spearcons, and short pulses for distance and forward-direction in pedestrian navigation. Results from the experiment suggest that spearcons are efficient in tasks completion, and it conveys distance and forward-direction information to participants more accurately compared with earcons and short pulses. Overall, participants have shown their satisfaction with spearcons as an audio feedback in pedestrian navigation.

In this paper, we propose an approach to build a multi-way concept hierarchy from a text corpus, which is based on WordNet and multi-way hierarchical clustering. In addition, a new evaluation metric is presented, and our approach is compared with 4 kinds of existing methods on the Amazon Customer Review data set.

The proliferation of digital cameras and the growing practice of online photo sharing using social media sites such as Flickr have resulted in huge volumes of geotagged photos available on the Web. Based on users' traveling preferences elicited from their travel experiences exposed on social media sites by sharing geotagged photos, we propose a new method for recommending tourist locations that are relevant to users (i.e., personalization) in the given context (i.e., context awareness). We obtain user-specific travel preferences from his/her travel history in one city and use these to recommend tourist locations in another city. Our technique is illustrated on a sample of publicly available Flickr dataset containing photos taken in various cities of China. Results show that our context-aware personalized method is able to predict tourists' preferences in a new or unknown city more precisely and generate better recommendations compared to other state-of-the-art landmark recommendation methods.

With the development of pervasive computing, location-aware pervasive games have proliferated quickly. To counter the inefficiency of information adaptation based on a location and a radius, this article presents the design and implementation of Your Way Your Missions (YWYM), which is a location-aware pervasive game exploiting the routes of players. YWYM provides a Google Maps–based tool for players to predefine routes, and utilizes self-reporting method to obtain the planned routes of players. By such a design, missions are returned to players in terms of the location properties of missions and the planned routes of players. A field study was conducted to investigate the usage of YWYM, and the results indicate that route predefining and self-reporting methods are an effective approach to obtain the planned routes of players, and information adaptation based on the planned routes of players could help them find missions at any place and any time.

This article presents the design of iReminder, an intuitive location-based reminder that delivers reminding messages based on users' future routes. iReminder is implemented on mobile phones, and it can predict users' future routes by collecting their daily trajectory data. Then it delivers a reminding message via the mobile phone when it senses that the user is going to the task location. A field study was conducted on how iReminder extends its potential to help users perform everyday tasks and compared the method adopted by iReminder with the method used by traditional location-based reminders. The experimental results show that iReminder outperforms traditional location-based reminder because it delivers reminding messages more appropriately. A detailed discussion is also given to investigate the ideal message delivery point, and the discussion results show that a location-based reminding message is more useful and more likely to be accepted by the user if it is triggered by considering his or her future route.

Geotagged photos of users on social media site, i.e., Flickr provide plentiful location-based data, which has been exploited for location-based services, such as mapping geotags to places and recommendation of personalized landmarks. As users’ preferences to visit a location or multiple locations in a certain sequence could be affected by their current temporal, and weather context. This paper considers the problem of mining context-aware significant semantic travel sequences from geotagged photos.

In this paper we present a novel concept of hybrid auditory feedback in mobility assistance for people with visual disabilities in indoor environment. Hybrid auditory feedback is a gradual con-version of sound from speech-only to non-speech (i.e., spearcons) based on the sound repetitiveness and the users' frequency of the travelled route. Using a within-subject design, eight participants carried out a task using a mobility assistant application and followed a same route for few days. Preliminary results suggest that hybrid sounds in auditory feedback are more effective than non-speech and are pleasant compared to speech-only.

A place is a locale that is frequently visited by an individual user and carries important semantic meanings (e.g. home, work, etc.). Many location-aware applications will be greatly enhanced with the ability of the automatic discovery of personally semantic places. The discovery of a user's personally semantic places involves obtaining the physical locations and semantic meanings of these places. In this paper, we propose approaches to address both of the problems. For the physical place extraction problem, a hierarchical clustering algorithm is proposed to firstly extract visit points from the GPS trajectories, and then these visit points can be clustered to form physical places. For the semantic place recognition problem, Bayesian networks (encoding the temporal patterns in which the places are visited) are used in combination with a customized POI (i.e. place of interest) database (containing the spatial features of the places) to categorize the extracted physical places into pre-defined types. An extensive set of experiments have been conducted to demonstrate the effectiveness of the proposed approaches based on a dataset of real-world GPS trajectories.

An average white-collar worker deals with enormous amount of digital information on daily basis. Recently, there has been a growing interest to support their work. However, in order to be really supportive there is a need to know the current activity of the user at all times. In this paper we present a new technique that takes advantage of temporal aspects of user activity behavior to infer when it is most likely that an activity switch is occurring. We then describe "Activity Switch Detector" an interactive switch notification system embodying these ideas, and an extensive user study by ten participants to test the validity of the approach and present its results.

We study the question of hand posture recognition by developing a new class of Bag-of-Features (BoF), namely Hierarchical BoF. The Hierarchical BoF captures spatial information by dividing whole hand area into several sub-regions and projecting local features onto horizontal and vertical directions. A similarity measurement based on Histogram Intersection Kernel (HIK) is proposed to classify hand postures. According to experimental result in Section 4, Hierarchical BoF works well in hand posture recognition task owing to the following properties: 1) the model captures spatial information of each component of hand postures; 2) the model has good adaptability for various complex background conditions.

Hand posture is a natural and effective interaction between human and robot. In this paper, we use monocular camera as input device, and an improved Bag-of-Words (BoW) method is proposed to detect and recognize hand posture based on a new descriptor ARPD (Appearance and Relative Position Descriptor) and spectral embedding clustering algorithm. To track hand motion rapidly and accurately, we have designed a new framework based on improved BoW and CAMSHIFT algorithm. The thorough evaluation of our algorithm is presented to show its usefulness.

Awareness of the user's current context, that is, the task the user resides in, can allow for various ways of supporting their work. In this article we present a novel approach for mining user's regular tasks on the basis of temporal proximity of users’ desktop actions without user intervention in a completely automatic, unobtrusive, and unsupervised manner. The proposed method is based on time-based clustering of a user's desktop actions and, in contrast to previous approaches, it does not enforce fixed time constraints on the mined actions, and typical transition times, when they exist, emerge from the logged activity data for that specific user. The performance of our technique was evaluated on a large data set of 724 days of desktop work by five knowledge workers. The results showed that our approach performed very well and was able to cluster those temporally proximate user actions that commonly used fix time window techniques fail to deal with. Notably, the main improvement is in the recall of the tasks, where our approach recalled 10.18% more actions than the predefined time window technique.

This paper presents a system where the personal route of a user is predicted using a probabilistic model built from the historical trajectory data . Route patterns are extracted from personal trajectory data using a novel mining algorithm, Continuous Route Pattern Mining (CRPM), which can tolerate different kinds of disturbance in trajectory data. Furthermore, a client–server architecture is employed which has the dual purpose of guaranteeing the privacy of personal data and greatly reducing the computational load on mobile devices. An evaluation using a corpus of trajectory data from 17 people demonstrates that CRPM can extract longer route patterns than current methods. Moreover, the average correct rate of one step prediction of our system is greater than 71%, and the average Levenshtein distance of continuous route prediction of our system is about 30% shorter than that of the Markov model based method.

Invariant feature descriptors such as SIFT and GLOH have been demonstrated to be very robust for image matching and object recognition. However, such descriptors are typically of high dimensionality, e.g. 128-dimension in the case of SIFT. This limits the performance of feature matching techniques in terms of speed and scalability. A new compact feature descriptor, called Kernel Projection Based SIFT (KPB-SIFT), is presented in this paper. Like SIFT, our descriptor encodes the salient aspects of image information in the feature point's neighborhood. However, instead of using SIFT's smoothed weighted histograms, we apply kernel projection techniques to orientation gradient patches. The produced KPB-SIFT descriptor is more compact as compared to the state-of-the-art, does not require pre-training step needed by PCA based descriptors, and shows superior advantages in terms of distinctiveness, invariance to scale, and tolerance of geometric distortions. We extensively evaluated the effectiveness of KPB-SIFT with datasets acquired under varying circumstances.

In this paper, we identify and solve a new type of spatial queries, called continuous visible nearest neighbor (CVNN) search. Given a data set P, an obstacle set O, and a query line segment q, a CVNN query returns a set of (p, R) tuples such that p e P is the nearest neighbor (NN) to every point r along the interval R e q as well as p is visible to r. Note that p may be NULL, meaning that all points in P are invisible to all points in R, due to the obstruction of some obstacles in O. In this paper, we formulate the problem and propose efficient algorithms for CVNN query processing, assuming that both P and O are indexed by R-trees. In addition, we extend our techniques to several variations of the CVNN query. Extensive experiments verify the efficiency and effectiveness of our proposed algorithms using both real and synthetic datasets.

Reverse nearest neighbor (RNN) queries have a broad application base such as decision support, profile-based marketing, resource allocation, data mining, etc. Previous work on RNN search does not take obstacles into consideration. In the real world, however, there are many physical obstacles (e.g., buildings, blindages, etc.), and their presence may affect the visibility/distance between two objects. In this paper, we introduce a novel variant of RNN queries, namely visible reverse nearest neighbor (VRNN) search, which considers the obstacle influence on the visibility of objects. Given a data set P, an obstacle set O, and a query point q, a VRNN query retrieves the points in P that have q as their nearest neighbor and are visible to q. We propose an efficient algorithm for VRNN query processing, assuming that both P and O are indexed by R-trees. Our method does not require any pre-processing, and employs half-plane property and visibility check to prune the search space.

This paper presents a novel local image descriptor for dense wide-baseline matching purposes, coined SULD (Speeded-Up Local Descriptor). SULD approximates or even outperforms than previously proposed schemes such as SURF and DAISY, and can be computed and compared much faster. This is achieved by summing up the Haar wavelet responses rather than the gradient, by computing convolutions recursively and by using low dimensions descriptor. The proposed approach was tested with ground truth laser scanned depth maps as well as on image pairs of different resolutions and the results show that good reconstruction is achieved even with only two small images.

This paper introduces and solves a novel type of spatial queries, namely, Optimal-Location-Selection (OLS) search, which has many applications in real life. Given a data object set D_A, a target object set D_B, a spatial region R, and a critical distance d_c in a multidimensional space, an OLS query retrieves those target objects in D_B that are outside R but have maximal optimality. Here, the optimality of a target object b \in D_B located outside R is defined as the number of the data objects from D_A that are inside R and meanwhile have their distances to b not exceeding d_c. When there is a tie, the accumulated distance from the data objects to b serves as the tie breaker, and the one with smaller distance has the better optimality. In this paper, we present the optimality metric, formalize the OLS query, and propose several algorithms for processing OLS queries efficiently. A comprehensive experimental evaluation has been conducted using both real and synthetic data sets to demonstrate the efficiency and effectiveness of the proposed algorithms.

Reverse nearest neighbor (RNN) queries have a broad application base such as decision support, profile-based marketing, resource allocation, etc. Previous work on RNN search does not take obstacles into consideration. In the real world, however, there are many physical obstacles (e.g., buildings) and their presence may affect the visibility between objects. In this paper, we introduce a novel variant of RNN queries, namely, visible reverse nearest neighbor (VRNN) search, which considers the impact of obstacles on the visibility of objects. Given a data set P, an obstacle set O, and a query point q in a 2D space, a VRNN query retrieves the points in P that have q as their visible nearest neighbor. We propose an efficient algorithm for VRNN query processing, assuming that P and O are indexed by R-trees. Our techniques do not require any preprocessing and employ half-plane property and visibility check to prune the search space. In addition, we extend our solution to several variations of VRNN queries, including: 1) visible reverse k-nearest neighbor (VRkNN) search, which finds the points in P that have q as one of their k visible nearest neighbors; 2) \delta-VRkNN search, which handles VRkNN retrieval with the maximum visible distance \delta constraint; and 3) constrained VRkNN (CVRkNN) search, which tackles the VRkNN query with region constraint. Extensive experiments on both real and synthetic data sets have been conducted to demonstrate the efficiency and effectiveness of our proposed algorithms under various experimental settings.

Given two sets DAand DBof multidimensional objects, a spatial region R, and a critical distance dc, an optimal-nearest- neighbor (ONN) query retrieves outside R, the object in DBwith maximum optimality. Let CAR (Sp,p) be the cardinality of the subset Sp of objects in DAwhich locate within R and are enclosed by the vicinity circle centered at p with radius dc. Then, an objectois said to be better than another one o' if (i) CAR (So,o) = CAR (So,o'), or (ii) when CAR (So,o) = CAR (So',o') the sum of the weighted distance from each object in Sotoois smaller than the sum of the weighted distance between every object in So' and o'. This type of queries is quite useful in many decision making applications. In this paper, we formalize the ONN query, develop the optimality metric, and propose several algorithms for finding optimal nearest neighbors efficiently. Our techniques assume that both DAand DBare indexed by R-trees. Extensive experiments demonstrate the efficiency and scalability of our proposed algorithms using both real and synthetic datasets.

Given a set of trajectories D, a query object (point or trajectory) q, and a query interval T, a mutual (i.e., symmetric) nearest neighbor (MNN) query over trajectories finds from D within T, the set of trajectories that are among the k1nearest neighbors (NNs) of q, and meanwhile, have q as one of their k2NNs. This type of queries considers proximity of q to the trajectories and the proximity of the trajectories to q, which is useful in many applications (e.g., decision making, data mining, pattern recognition, etc.). In this paper, we first formalize MNN query and identify some problem characteristics, and then develop two algorithms to process MNN queries efficiently. In particular, we thoroughly investigate two classes of queries, viz. MNNPand MNNTqueries, which are defined w.r.t. stationary query points and moving query trajectories, respectively. Our techniques utilize the advantages of batch processing and reusing technology to reduce the I/O (i.e., number of node/page accesses) and CPU costs significantly. Extensive experiments demonstrate the efficiency and scalability of our proposed algorithms using both real and synthetic datasets.

The development of multimedia annotation technique provides the possibility to redesign the interfaces of widely used media players, and EmoPlayer is such a media player that can be used to play video clips with affective annotations. A user can select a character in a video clip and view the distribution of his/her emotions along the video timeline through a colour bar based interface. Two experiments were conducted to evaluate the efficiency of affective annotation. The results of these experiments indicate that affective annotation is effective in both improving the speed of locating a specific scene within a video clip and helping comprehend a video clip in a limited viewing time period. Based on the analysis of recorded operations of participants, the strategies employed by participants and the factors that might influence the utilization of affective annotation are also highlighted.

Although there exists dozens of vision based 3D head tracking methods, none of them considers the problem of large motion, especially the movement along the Z axis. In this paper we propose a novel tracking method to handle this problem by using Scale Invariant Feature Transform (SIFT) based registration algorithm. Salient SIFT features are first detected and tracked between two images, and then the 3D points corresponding to these features are obtained from a stereo camera. With these 3D points, a registration algorithm in a RANSAC framework is employed to detect the outliers and estimate the head pose. Performance evaluation shows an accurate pose recovery (3° RMS) when the head has large motion, even with movement along the Z axis was about 150 cm.

Local-lag mechanism can maintain consistency for replicated continuous applications, but with a tradeoff of adding delay to local operations. To relieve the negative effects of the delay, this paper proposes an HCI method named echo. With the help of the echo method users can immediately perceive the results of their operations and how large the lag is. In order to evaluate the proposed method, a desktop collaborative virtual environment (CVE) system and a virtual object control task were employed to study the effects of the echo method on human performance (including task completion time, error count, and interaction quality). Experimental results indicate that when the lag exceeds 100 ms the echo method can improve human performance with the effects becoming more evident when a larger lag is used.

In this paper, an adaptive-filtering policy for distributed virtual environments (DVEs) was brought forward. It took advantages of two traditional filtering policies (static-filtering policy and equidistance-transmission policy) and could adaptively select the fittest filtering policy corresponding to current motion characteristic of the virtual object in DVEs. An experiment was designed to compare this filtering policy with the two traditional filtering policies on task performance. Objective (task completion time, state updates transmitted and state updates per second) and subjective (filtering quality) evaluations were recorded in the experiment. The experiment results show that if mean state updates per second is equal or greater than 20 in the task, SUT and SUPS can be decreased by adaptive-filtering policy without task performance affected obviously. However, if mean state updates per second is around 10 in the task, accompanied with SUT and SUPS decreased by adaptive-filtering policy, task performance also deteriorated badly

Nowadays, handheld wireless devices are being widely used by people to collaborate with one another using same handheld device in field or using desktop PC in office. Therefore, to study the effects of platform heterogeneity on task performance in collaborative virtual environments (CVEs) is helpful to improve future collaboration efficiency. In this paper, an experiment with highly interactive task was designed to study these effects. The task performance was evaluated under different combinations of three platforms: PC with a sensor, PDA with a stylus and PDA with direction buttons. The experiment results indicate that for simple 3D collaboration task, if right type of visualization and input device was chosen, we would be satisfied with performance both on homogeneous platforms and heterogeneous platforms

The "priority-based directed minimum spanning tree" (PST) is designed for distributed interactive applications (DIAs) in which the data are delivered to the receiver node according to their priority; however PST cannot balance the bandwidth usage among the nodes and the system will be unstable when its scale increases. This paper proposes a novel overlay multicast protocol named priority based overlay multicast (POM), which uses the priority, delay and available bandwidth to construct the multicast tree, besides a filtering mechanism is proposed to rebuild the tree when the available bandwidth shortage appears. The effectiveness of our protocol is demonstrated in the simulation results, showing that POM uses the available bandwidth efficiently, besides it keeps the system stable when its scale increases

Distributed Interactive Applications (DIAs) is different with other network-based applications, since the data should be delivered to the receiver node according to its priority. Among the existing overlay multicast protocols the "priority-based directed minimum spanning tree" (PST) is designed for DIAs. But in PST the available bandwidth (avail-bw) is used inefficiently and the system becomes unstable when its scale increases. This paper proposed a novel overlay multicast protocol named Priority Based Overlay Multicast (POM), which uses the entities’ priority, links’ delay and node’s avail-bw to build multicast tree. By using POM the nodes with high priority receive the data in short delay, while the nodes with low priority tolerate a long delay in order to balance the available bandwidth usage among the nodes. The simulation results show that POM uses the avail-bw efficiently and keeps the system stable when its scale increases. Keywords: DIAs; Priority; Available bandwidth.

An efficient DDM (data distribution management) approach-fuzzy grouping mechanism was presented. In order to reduce the matching times and increase the data filtering rate, this new mechanism adopted two techniques based on the fuzzy correlation space approach: fuzzy consistent relation based weight distribution and grid based preprocessing. Simulation results indicate that: compared with existing approaches, fuzzy grouping mechanism not only keeps high and stable data filtering rates in different situations but also greatly reduces matching times.

Local-lag mechanism can eliminate short-term inconsistencies for collaborative virtual environments (CVEs). However, local-lag mechanism increases the response time and deteriorates the task performance of the system when lag is large. This paper first analyzes the effects of local-lag mechanism on task performance in a pilot experiment. We find that the key reason for the task performance deterioration is that subjects do not know the current position of the virtual object controlled by him/her and how long the lag is. Based on this observation, this paper proposes a method named echo, which gives some cues to subjects and makes them perceive these information. Experiment results indicate that the method can improve the task performance significantly. Especially when lag exceeds 600 ms, almost half of subjects cannot finish the task without the help of echo method, but with the help of the echo method all subjects can finish the task fluently.

In a collaborative virtual environment (CVE), users need to do some collaborative work, which is highly synchronous. To achieve effective collaboration, the level of simulation accuracy (LOA) of a user's behavior in CVE must be high. However, if we adopt a static high LOA, a system's scalability is greatly decreased. Thus a scalability request must compromise with a collaborative request. The paper analyses a CVE system's scalability request and collaborative request, and presents a classification of collaborative work and a definition of collaborative knowledge base (CKB). Based on CKB, a knowledge-based adaptive message filtering technique, which improves the CVE system's scalability, is put forward.