Frustration in games can have negative consequences for both players and those around them. Despite these effects, methods for supporting frustration management in hectic gameplay are scarce. In this paper, we present Frustration Buddy, a novel tangible intervention system aimed at managing frustration during play. Frustration Buddy leverages distraction and soothing techniques to help players control their emotions. Our paper provides an artifact contribution, complemented by exploratory findings from semi-structured interviews with nine participants on frustration in games, how they manage it, and their opinions on Frustration Buddy’s design and applicability in their gaming environments. The results suggest participants were open to the idea of a device supporting frustration management and that the use of a tangible device like a stress ball was well-received. We provide insights on how to design such a device based on our design decisions and the preferences expressed by the gamers in our interviews.

Gesture recognition is a tool to enable novel interactions with different techniques and applications, like Mixed Reality and Virtual Reality environments. With all the recent advancements in gesture recognition from skeletal data, it is still unclear how well state-of-the-art techniques perform in a scenario using precise motions with two hands. This paper presents the results of the SHREC 2024 contest organized to evaluate methods for their recognition of highly similar hand motions using the skeletal spatial coordinate data of both hands. The task is the recognition of 7 motion classes given their spatial coordinates in a frame-by-frame motion. The skeletal data has been captured using a Vicon system and pre-processed into a coordinate system using Blender and Vicon Shogun Post. We created a small, novel dataset with a high variety of durations in frames. This paper shows the results of the contest, showing the techniques created by the 5 research groups on this challenging task and comparing them to our baseline method.

Toxic behavior is known to cause harm in online games. Players regularly experience negative, hateful, or inappropriate behavior. Interventions, such as banning players or chat message filtering, can help combat toxicity but are not widely available or even comprehensively studied regarding their approaches and evaluations. We conducted a systematic literature review that provides insights into the current state of interventions literature, outlining their strengths and shortcomings. We identified 36 interventions and qualitatively analyzed their approaches. We describe the types of toxicity being addressed, the entities through which they act, the methods used by intervention systems, and how they are evaluated. Our results provide guidance for future interventions, outlining a design space based on known systems. Furthermore, our findings highlight gaps in the literature, e.g., a sparsity of empirical evaluations, and underexplored areas in the design space, enabling researchers to explore novel directions for future interventions.

We present the first experiments from a pilot study with Ruby’s Mission, an applied gaming intervention for reducing feelings of loneliness in children with a chronic illness. Based on a previously performed literature study, Ruby’s Mission sets out to train specific socioemotional skills, through four children sharing emotional experiences. The present study contributes insights in (1) how asymmetrical player roles in a multiplayer game affect in-game social interactions, and (2) if socioemotional skills affect the in-game social interactions in a multiplayer video game. Experiments were conducted during a public event for introducing children to science. A baseline measure of socioemotional skills was made using the emotional awareness questionnaire (EAQ). Video and audio recordings were made during gameplay, and were annotated afterwards on social interaction. As per expectations, results suggest that asymmetrical player roles encourage different levels of social interaction. Analysing the effect of socioemotional skills on an individual player’s in-game social interactions, surprisingly revealed a slight negative effect, where social interactions decreased as socioemotional skills improved. Interestingly, when investigating this effect on per gameplay group level, no interaction was found. Results also reveal that external factors such as parent interference, play environment, and peer relations might have an effect on social interactions as well.

For egocentric vision tasks such as action recognition, there is a relative scarcity of labeled data. This increases the risk of overfitting during training. In this paper, we address this issue by introducing a multitask learning scheme that employs related tasks as well as related datasets in the training process. Related tasks are indicative of the performed action, such as the presence of objects and the position of the hands. By including related tasks as additional outputs to be optimized, action recognition performance typically increases because the network focuses on relevant aspects in the video. Still, the training data is limited to a single dataset because the set of action labels usually differs across datasets. To mitigate this issue, we extend the multitask paradigm to include datasets with different label sets. During training, we effectively mix batches with samples from multiple datasets. Our experiments on egocentric action recognition in the EPIC-Kitchens, EGTEA Gaze+, ADL and Charades-EGO datasets demonstrate the improvements of our approach over single-dataset baselines. On EGTEA we surpass the current state-of-the-art by 2.47 percent. We further illustrate the cross-dataset task correlations that emerge automatically with our novel training scheme.

Recognition of pain in animals is essential for their welfare. However, since there is no verbal communication, this assessment depends solely on the ability of the observer to locate visible or audible signs of pain. The use of grimace scales is proven to be efficient in detecting the pain visually, but the assessment quality depends on the level of training of the assessor and the validity is not easily ensured. There is a clear need for automating the pain assessment process. This work provides a system for pain prediction in horses, based on grimace scales. The pipeline automatically determines the quantitative pose of the equine head and finds facial landmarks before classification, proposing a novel scale-normalisation approach for equine heads. The pain estimation is achieved for each facial region of interest separately, following the clinical pain estimation procedure. We introduce a database of horse images, annotated by professional veterinarians for training and assessment. We also propose a data augmentation method to alleviate the data scarcity issues, which relies on generating realistic 3D equine face models based on 2D annotated images. We show that the data augmentation method improves the performance of both quantitative pose estimation and landmark detection. Our results establish a strong baseline for automatic equine pain estimation.

Children with a chronic disease, such as cystic fibrosis or juvenile arthritis, often face obstacles that can have a negative impact on children’s physical, social-emotional and cognitive development, beyond the actual illness itself. Children with chronic conditions are, on average, lonelier than their peers without such conditions. Feelings of loneliness in children and adolescents have been associated with a wide range of negative outcomes, including school drop-out, depressive symptoms, social anxiety, suicide ideation, low self-esteem, eating disorders, and sleep problems. As such, the present investigation sets out to reduce these feelings of loneliness for children with chronic conditions, and aims to do so by the structured design of an applied gaming intervention. Specifically, the present paper contributes (1) a literature-based understanding on training socioemotional skills as a novel means to reduce feelings of loneliness in chronically ill children, (2) intervention objectives that are aligned to this goal, and (3) a structured proposal for design guidelines that implement the intervention objectives into ‘Ruby’s Mission’; an applied gaming intervention for reducing loneliness of children with chronic illness.

Recognition of pain in equines (such as horses and donkeys) is essential for their welfare. However, this assessment depends solely on the ability of the observer to locate visible signs of pain since there is no verbal communication. The use of Grimace scales is proven to be efficient in detecting pain but is time-consuming and also dependent on the level of training of the annotators and, therefore, validity is not easily ensured. There is a need for automation of this process to help training. This work provides a system for pain prediction in horses, based on Grimace scales. The pipeline automatically finds landmarks on horse faces before classification. Our experiments show that using different classifiers for different poses of the horse is necessary, and fusion of different features improves results. We furthermore investigate the transfer of horse-based models for donkeys and illustrate the loss of accuracy in automatic landmark detection and subsequent pain prediction.

The 6D object pose is widely applied in robotic grasping, virtual reality and visual navigation. However, heavy occlusion, changing light conditions and cluttered scenes make this problem challenging. To address these issues, we propose a novel approach that effectively extracts color and depth features from RGB-D images considering the local and global geometric relationships. After that, we apply a graph attention mechanism to fully exploit representations between these features and then fuse them together to predict the 6D pose of a given object. The evaluation results indicate that our method significantly improves the accuracy of the estimated 6D pose and achieves the state-of-the-art performance on LineMOD, YCB-Video, and a new dataset. Ablation studies demonstrate the effect of our network modules.

Scene understanding of large-scale 3D point clouds of an outer space is still a challenging task. Compared with simulated 3D point clouds, the raw data from LiDAR scanners consist of tremendous points returned from all possible reflective objects and they are usually non-uniformly distributed. Therefore, its cost-effective to develop a solution for learning from raw large-scale 3D point clouds. In this track, we provide large-scale 3D point clouds of street scenes for the semantic segmentation task. The data set consists of 80 samples with 60 for training and 20 for testing. Each sample with over 2 million points represents a street scene and includes a couple of objects. There are five meaningful classes: building, car, ground, pole and vegetation. We aim at localizing and segmenting semantic objects from these large-scale 3D point clouds. Four groups contributed their results with different methods. The results show that learning-based methods are the trend and one of them achieves the best performance on both Overall Accuracy and mean Intersection over Union. Next to the learning-based methods, the combination of hand-crafted detectors are also reliable and rank second among comparison algorithms.

Deep learning approaches have been established as the main methodology for video classification and recognition. Recently, 3-dimensional convolutions have been used to achieve state-of-the-art performance in many challenging video datasets. Because of the high level of complexity of these methods, as the convolution operations are also extended to an additional dimension in order to extract features from it as well, providing a visualization for the signals that the network interpret as informative, is a challenging task. An effective notion of understanding the network's innerworkings would be to isolate the spatio-temporal regions on the video that the network finds most informative. We propose a method called Saliency Tubes which demonstrate the foremost points and regions in both frame level and over time that are found to be the main focus points of the network. We demonstrate our findings on widely used datasets for thirdperson and egocentric action classification and enhance the set of methods and visualizations that improve 3D Convolutional Neural Networks (CNNs) intelligibility. Our code1and a demo video2are also available.

Presence and the sense of embodiment are essential concepts for the experience of our self and virtual bodies, but there is little quantitative evidence for a relation between these, and this relation becomes more complicated when there are real and virtual bodies in augmented reality (AR). We investigate the experience of body ownership, agency, self-location and self-presence in AR where users can see their real body and a virtual body from behind. Active arm movement congruency and virtual anthropomorphism are varied. We found significant effects of movement congruency but not anthropomorphism, a strong correlation between self-presence and body ownership, and a moderate correlation between self-presence and agency and self-location.

Manual annotation of rodent behaviors in video is time-consuming. By learning a classifier, we can automate the labeling process. Still, this strategy requires a sufficient number of labeled examples. Moreover, we need to train new classifiers when there is a change in the set of behaviors that we consider or in the manifestation of these behaviors in video. Consequently, there is a need for an efficient way to annotate rodent behaviors. In this paper we introduce a framework for interactive behavior annotation in video based on active learning. By putting a human in the loop, we alternate between learning and labeling. We apply the framework to three rodent behavior datasets and show that we can train accurate behavior classifiers with a strongly reduced number of labeled samples. We confirm the efficacy of the tool in a user study demonstrating that interactive annotation facilitates efficient, high-quality behavior measurements in practice.

The classic rubber hand illusion (RHI) experiment studies the sense of embodiment over a fake limb. Distinguished subcomponents of embodiment are ownership (sense of self-attribution of a body), agency (sense of having motor control), and self-location (the spatial experience of being inside a body), and are typically evoked in either reality or virtual reality. In augmented reality (AR), however, visually present real limbs can be augmented with (multiple) fake virtual limbs, which results in a variation of the RHI, the augmented reality supernumerary hand illusion (ARSHI). Such conditions occur, for example, in first-person AR games and in AR-interfaces for tele-robotics. In this article, we examined to what extent humans can experience the sense of embodiment over a supernumerary virtual arm in addition to one or two real arms. We also examine how embodiment is affected by the perceptual visual-tactile synchronicity of the virtual and real limbs, and by the synchronicity of active movement of the virtual and real hand. Embodiment was measured subjectively by questionnaire and objectively by skin conductance responses (SCRs). Questionnaire responses show that ownership, agency, and self-location can be evoked over the virtual arm in the presence of a real arm, and that they are significantly stronger for synchronous conditions than for asynchronous conditions. The perceptual and motorical synchronous condition with three visible hands led to an experience of owning the virtual hand. These responses further show that agency was also strongly experienced over the supernumerary virtual arm, and responses regarding self-location suggest a shift in sensed location when one real arm was in view and an additional location when both real arms where in view. SCRs show no significant effect of condition, but do show a significant habituation effect as a function of the number of conditions performed by participants. When analyzing the relations at the individual participant level between the questionnaire data and skin conductance, we found two clusters of participants: (1) participants with low questionnaire responses and low-medium SCRs and (2) participants with high questionnaire responses and low-high SCRs. Finally, we discuss how virtual hand appearance/realism and willingness to accept virtual limbs could play an important role in the ARSHI, and provide insights on intricacies involved with measuring and evaluating RHIs.

The H.265/MPEG-H High Efficiency Video Coding compliant encoding process faces the challenge of high computational complexity. Particularly, in the case of inter-picture prediction, most of the computational resources are allocated for the motion estimation (ME) process. In turn, ME and motion compensation enable improving coding efficiency by addressing the blocks of video frames as corresponding displacements from one or more reference blocks. These displacements do not necessarily have to be limited to integer sample positions, but may have an accuracy of half sample or quarter sample positions, which are identified during fractional sample refinement. In this paper, a context-based scheme for fractional sample refinement is proposed. The scheme takes the advantage of already obtained information in prior ME steps and provides significant flexibility in terms of parameterization. In this way, it adaptively achieves a desired tradeoff between computational complexity and coding efficiency. According to the experimental results obtained for an example algorithm utilizing the proposed framework, a significant decrease in the number of search points can be achieved. For instance, considering only 6 instead of 16 fractional sample positions results in a tradeoff of only 0.4% Bjøntegaard Delta-rate loss for high-definition video sequences compared with the conventional interpolation-and-search method.

In social settings, people interact in close proximity. When analyzing such encounters from video, we are typically interested in distinguishing between a large number of different interactions. Here, we address training deformable part models (DPMs) for the detection of such interactions from video, in both space and time. When we consider a large number of interaction classes, we face two challenges. First, we need to distinguish between interactions that are visually more similar. Second, it becomes more difficult to obtain sufficient specific training examples for each interaction class. In this paper, we address both challenges and focus on the latter. Specifically, we introduce a method to train body part detectors from nonspecific images with pose information. Such resources are widely available. We introduce a training scheme and an adapted DPM formulation to allow for the inclusion of this auxiliary data. We perform cross-dataset experiments to evaluate the generalization performance of our method. We demonstrate that our method can still achieve decent performance, from as few as five training examples.

In this work, we aim to segment and detect water in videos. Water detection is beneficial for appllications such as video search, outdoor surveillance, and systems such as unmanned ground vehicles and unmanned aerial vehicles . The specific problem, however, is less discussed compared to general texture recognition. Here, we analyze several motion properties of water. First, we describe a video pre-processing step, to increase invariance against water reflections and water colours. Second, we investigate the temporal and spatial properties of water and derive corresponding local descriptors . The descriptors are used to locally classify the presence of water and a binary water detection mask is generated through spatio-temporal Markov Random Field regularization of the local classifications. Third, we introduce the Video Water Database, containing several hours of water and non-water videos, to validate our algorithm. Experimental evaluation on the Video Water Database and the DynTex database indicates the effectiveness of the proposed algorithm, outperforming multiple algorithms for dynamic texture recognition and material recognition.

Augmented reality applications using stereo head-mounted displays are not capable of perfectly blending real and virtual objects. For example, depth in the real world is perceived through cues such as accommodation and vergence. However, in stereo head-mounted displays these cues are disconnected since the virtual is generally projected at a static distance, while vergence changes with depth. This conflict can result in biased depth estimation of virtual objects in a real environment. In this research, we examined whether redundant tactile feedback can reduce the bias in perceived depth in a reaching task. In particular, our experiments proved that a tactile mapping of distance to vibration intensity or vibration position on the skin can be used to determine a virtual object's depth. Depth estimation when using only tactile feedback was more accurate than when using only visual feedback, and when using visuotactile feedback it was more precise and occurred faster than when using unimodal feedback. Our work demonstrates the value of multimodal feedback in augmented reality applications that require correct depth perception, and provides insights on various possible visuotactile implementations.

We model dyadic (two-person) interactions by discriminatively training a spatio-temporal deformable part model of fine-grained human interactions. All interactions involve at most two persons. Our models are capable of localizing human interactions in unsegmented videos, marking the interactions of interest in space and time. Our contributions are as follows: First, we create a model that localizes human interactions in space and time. Second, our models use multiple pose and motion features per part. Third, we experiment with different ways of training our models discriminatively. When testing on the target class our models achieve a mean average precision score of 0.86. Cross dataset tests show that our models generalize well to different environments.

One of the major reasons why people find music so enjoyable is its emotional impact. Creating emotion-based playlists is a natural way of organizing music. The usability of online music streaming services could be greatly improved by developing emotion-based access methods, and automatic music emotion recognition (MER) is the most quick and feasible way of achieving it. When resorting to music for emotional regulation purposes, users are interested in the MER method to predict their induced, or felt emotion. The progress of MER in this area is impeded by the absence of publicly accessible ground-truth data on musically induced emotion. Also, there is no consensus on the question which emotional model best fits the demands of the users and can provide an unambiguous linguistic framework to describe musical emotions. In this paper we address these problems by creating a sizeable publicly available dataset of 400 musical excerpts from four genres annotated with induced emotion. We collected the data using an online “game with a purpose” Emotify, which attracted a big and varied sample of participants. We employed a nine item domain-specific emotional model GEMS (Geneva Emotional Music Scale). In this paper we analyze the collected data and report agreement of participants on different categories of GEMS. We also analyze influence of extra-musical factors on induced emotion (gender, mood, music preferences). We suggest that modifications in GEMS model are necessary.

The cold start problem for new users or items is a great challenge for recommender systems. New items can be positioned within the existing items using a similarity metric to estimate their ratings. However, the calculation of similarity varies by domain and available resources. In this paper, we propose a content-based music recommender system which is based on a set of attributes derived from psychological studies of music preference. These five attributes, namely, Mellow, Unpretentious, Sophisticated, Intense and Contemporary (MUSIC), better describe the underlying factors of music preference compared to music genre. Using 249 songs and hundreds of ratings and attribute scores, we first develop an acoustic content-based attribute detection using auditory modulation features and a regression by sparse representation. We then use the estimated attributes in a cold start recommendation scenario. The proposed content-based recommendation significantly outperforms genre-based and user-based recommendation based on the root-mean-square error. The results demonstrate the effectiveness of these attributes in music preference estimation. Such methods will increase the chance of less popular but interesting songs in the long tail to be listened to.

Skinning using point handles has experimentally shown its effectiveness for stretching, twisting, and supple deformations [Jacobson et al. 2011] which are difficult to achieve using rigid bones. However, point handles are much less effective for limbs bending since their influence weights vary over the skin mesh. This poster presents an efficient scheme, which expands the space of deformations possible to the point handles by supporting rigid bending using a rigidity energy minimization framework. Thus, 3D points that are much easier to design while provide larger space of deformations than a skeleton can be used as alternative handles in character skinning.

Music as a form of art is intentionally composed to be emotionally expressive. The emotional features of music are invaluable for music indexing and recommendation. In this paper we present a cross-comparison of automatic emotional analysis of music. We created a public dataset of Creative Commons licensed songs. Using valence and arousal model, the songs were annotated both in terms of the emotions that were expressed by the whole excerpt and dynamically with 1 Hz temporal resolution. Each song received 10 annotations on Amazon Mechanical Turk and the annotations were averaged to form a ground truth. Four different systems from three teams and the organizers were employed to tackle this problem in an open challenge. We compare their performances and discuss the best practices. While the effect of a larger feature set was not very apparent in the static emotion estimation, the combination of a comprehensive feature set and a recurrent neural network that models temporal dependencies has largely outperformed the other proposed methods for dynamic music emotion estimation.

Abstract The demand for large geometric models is increasing, especially of urban environments. This has resulted in production of massive point cloud data from images or LiDAR. Visualization and further processing generally require a detailed, yet concise representation of the scene's surfaces. Related work generally either approximates the data with the risk of over‐smoothing, or interpolates the data with excessive detail. Many surfaces in urban scenes can be modeled more concisely by planar approximations. We present a method that combines these polygons into a watertight model. The polygon‐based shape is closed with free‐form meshes based on visibility information. To achieve this, we divide 3‐space into inside and outside volumes by combining a constrained Delaunay tetrahedralization with a graph‐cut. We compare our method with related work on several large urban LiDAR data sets. We construct similar shapes with a third fewer triangles to model the scenes. Additionally, our results are more visually pleasing and closer to a human modeler's description of urban scenes using simple boxes.

Multiple people tracking from multiple cameras can suffer from various problems, particularly from inter-person occlusions. This paper attempts to solve the problems by analyzing the view visibility and ranking the reliability of the cues from 2D views. It combines the visibility with the smoothness constraints into a probability framework, which offers a more flexible and robust estimation. Moreover, it introduces 3D reference lines to estimate the 2D position of every individual in the input images. These lines can estimate more accurate and robust 2D positions. The experimental results and quantitative evaluations on the standard data set show the effectiveness of the method.

Abstract Detailed geometric models of the real world are in increasing demand. LiDAR data is appropriate to reconstruct urban models. In urban scenes, the individual surfaces can be reconstructed and connected to form the scene geometry. There are various methods for reconstructing the free‐form shape of a point sample on a single surface. However, these methods do not take the context of the surface into account. We present the guided α‐shape: an extension of the well known α‐shape that uses lines (guides) to indicate preferred locations for the boundary of the shape. The guided α‐shape uses (parts of) these lines as boundary where the points suggest that this is appropriate. We prove that the guided α‐shape can be constructed in O((n + m) log (n + m)) time, from an input of n points and m guides. We apply guided α‐shapes to urban reconstruction from LiDAR, where neighboring surfaces can be connected conveniently along their intersection lines into adjacent surfaces of a 3D model. We analyze guided α‐shapes of both synthetic and real data and show they are consistently better than α‐shapes for this application.

Indexing has become a key element in the pipeline of a multimedia retrieval system, due to continuous increases in database size, data complexity, and complexity of similarity measures. The primary goal of any indexing algorithm is to overcome high computational costs involved with comparing the query to every object in the database. This is achieved by efficient pruning in order to select only a small set of candidate matches. Vantage indexing is an indexing technique that belongs to the category of embedding or mapping approaches, because it maps a dissimilarity space onto a vector space such that traditional access methods can be used for querying. Each object is represented by a vector of dissimilarities to a small set of m reference objects, called vantage objects. Querying takes place within this vector space. The retrieval performance of a system based on this technique can be improved significantly through a proper choice of vantage objects. We propose a new technique for selecting vantage objects that addresses the retrieval performance directly, and present extensive experimental results based on three data sets of different size and modality, including a comparison with other selection strategies. The results clearly demonstrate both the efficacy and scalability of the proposed approach.

Object recognition systems need effective image descriptors to obtain good performance levels. Currently, the most widely used image descriptor is the SIFT descriptor that computes histograms of orientation gradients around points in an image. A possible problem of this approach is that the number of features becomes very large when a dense grid is used where the histograms are computed and combined for many different points. The current dominating solution to this problem is to use a clustering method to create a visual codebook that is exploited by an appearance based descriptor to create a histogram of visual keywords present in an image. In this paper we introduce several novel bag of visual keywords methods and compare them with the currently dominating hard bag-of-features (HBOF) approach that uses a hard assignment scheme to compute cluster frequencies. Furthermore, we combine all descriptors with a spatial pyramid and two ensemble classifiers. Experimental results on 10 and 101 classes of the Caltech-101 object database show that our novel methods significantly outperform the traditional HBOF approach and that our ensemble methods obtain state-of-the-art performance levels.

Most of the development of pose recognition focused on a single person. However, many applications of computer vision essentially require the estimation of multiple people. Hence, in this paper, we address the problems of estimating poses of multiple persons using volumes estimated from multiple cameras. One of the main issues that causes the multiple person from multiple cameras to be problematic is the present of `ghost' volumes. This problem arises when the projections of two different silhouettes of two different persons onto the 3D world overlap in a place where in fact there is no person in it. To solve this problem, we first introduce a novel principal axis-based framework to estimate the 3D ground plane positions of multiple people, and then use the position cues to label the multi-person volumes (voxels), while considering the voxel connectivity. Having labeled the voxels, we fit the volume of each person with a body model, and determine the pose of the person based on the model. The results on real videos demonstrate the accuracy and efficiency of our approach.

3D media has emerged rapidly as a new type of content within the multimedia domain. The recent acceleration of 3D content production, witnessed across all fields up to user-generated content, is causing a huge amount of traffic and data stored and transmitted using Internet technologies. Recent advances in 3D acquisition and 3D graphics rendering technologies boosted the creation of 3D model archives for several application domains. These include archaeology and cultural heritage, computer-assisted design (CAD), medicine and bioinformatics, 3D face recognition and security, entertainment and serious gaming, spatial data and 3D city management.

Recent research in image recognition has shown that combining multiple descriptors is a very useful way to improve classification performance. Furthermore, the use of spatial pyramids that compute descriptors at multiple spatial resolution levels generally increases the discriminative power of the descriptors. In this paper we focus on combination methods that combine multiple descriptors at multiple spatial resolution levels. A possible problem of the naive solution to create one large input vector for a machine learning classifier such as a support vector machine, is that the input vector becomes of very large dimensionality, which can increase problems of overfitting and hinder generalization performance. Therefore we propose the use of stacking support vector machines where at the first layer each support vector machine receives the input constructed by each single descriptor and is trained to compute the right output class. A second layer support vector machine is then used to combine the class probabilities of all trained first layer support vector models to learn the right output class given these reduced input vectors. We have performed experiments on 20 classes from the Caltech object database with 10 different single descriptors at 3 different resolutions. The results show that our 2-layer stacking approach outperforms the naive approach that combines all descriptors directly in a very large single input vector.

Three-Dimensional face recognition is a challenging task with a large number of proposed solutions [1, 2]. With variations in pose and expression the identification of a face scan based on 3D geome-try is difficult. To improve on this task and to evaluate existing face

Many 3D face matching techniques have been developed to perform face recognition. Among these techniques are variants of 3D facial curve matching, which are techniques that reduce the amount of face data to one or a few 3D curves. The face’s central profile, for instance, proved to work well. However, the selection of the optimal set of 3D curves and the best way to match them is still under-exposed. We propose a 3D face matching framework that allows profile and contour based face matching. Using this framework we evaluate profile and contour types including those described in literature, and select subsets of facial curves for effective and efficient face matching. Results on the 3D face retrieval track of SHREC’07 (the 3D SHape Retrieval Contest) shows the highest mean average precision achieved so far, using only three facial curves of 45 samples each.

In this work we compute the similarity of 3D faces using a set of eight contour curves. These contours were selected and matched using our 3D face matching framework. In previous work, we performed extensive research to the selection of distinctive facial curves for 3D face matching. To relate the performance of several of these curves to other face matching methods, we participated the Shape Retrieval Contest (SHREC) of 3D face scans. Within this contest we have used a set of eight C-contours and tested their face retrieval performance using two different distance measures. In an attempt to increase the expression invariance of these curves, we employed our 3D face matching framework to match either 100% of the selected features or the subset of the best 60% of the selected features. Results show that the selected distance measure can have a great influence on the distinctiveness of facial curves. In case of large variations in facia) expressiveness, the subset of the best 60% of the features increases the overall performance. With a recognition rate of 91.1% and a mean average precision of 0.693 our method performs reasonably well compared to other methods.

Since 2006 the 3D shape retrieval contest SHREC has been organized. The general objective is to evaluate the effectiveness of 3D shape retrieval algorithms. 3D media retrieval is overlooked in most commercial search engines, while at the same time it is expected to represent a huge amount of traffic and data stored in the Internet. Recent advances in technology have made available cost-effective scanning devices that could not even be imagined a decade ago. It is now possible to acquire 3D data of a physical object in a few seconds and produce a digital model of its geometry that can be easily shared on the Internet. On the other hand, most PCs connected to the Internet are nowadays equipped with high-performance 3D graphics hardware, that support rendering, interaction and processing capabilities from home environments to enterprise scenarios.

The steep rise in music downloading over CD sales has created a major shift in the music industry away from physical media formats and towards online products and services. Music is one of the most popular types of online information and there are now hundreds of music streaming and download services operating on the World-Wide Web. Some of the music collections available are approaching the scale of ten million tracks and this has posed a major challenge for searching, retrieving, and organizing music content. Research efforts in music information retrieval have involved experts from music perception, cognition, musicology, engineering, and computer science engaged in truly interdisciplinary activity that has resulted in many proposed algorithmic and methodological solutions to music search using content-based methods. This paper outlines the problems of content-based music information retrieval and explores the state-of-the-art methods using audio cues (e.g., query by humming, audio fingerprinting, content-based music retrieval) and other cues (e.g., music notation and symbolic representation), and identifies some of the major challenges for the coming years.

This paper presents a new multiview alignment algorithm that performs both the coarse and fine alignment of unordered sets of range scans. Our algorithm selects quadruples of range scans, which have feature points of their 2D projections in common. These quadruples are then verified using an Iterative Closest Point (ICP) algorithm. The accepted quadruples form incomplete models of an object that can be aligned using isometries of the Principal Component Analysis (PCA) in combination with an ICP algorithm. The output of our method is a set of finely aligned meshes. Our method was applied to range scans of different clusters of objects varying in the type of acquisition system, the number of range scans, the scan resolution, and scan accuracy. Results show that our method is both effective and efficient for the alignment of meshes: it is capable of aligning object meshes with various properties, aligns the majority of meshes without a priori knowledge, and doesn't require a multiview ICP algorithm to improve the final alignment.

For the RISM A/II collection of musical incipit (short extracts of scores, taken from the beginning), we have established a ground truth based on the opinions of human experts. It contains correctly ranked matches for a set of given queries. These ranked lists contain groups of documents whose ranks were not significantly different. In other words, they are only partially ordered. To make use of the available information for measuring the quality of retrieval results, we introduce the "average dynamic recall" (ADR) that averages the recall among a dynamic set of relevant documents, taking into account the fact that the ground truth reliably orders groups of matches, but not always individual matches. Dynamic recall measures how many of the documents that should have appeared before or at a given position in the result list actually have appeared. ADR at a given position averages this measure up to the given position. Our measure was first used at the MIREX 2005 Symbolic Melodic Similarity contest

We introduce a relevance feedback mechanism for part-based shape retrieval. The database polygons are decomposed but the query polygon is not. In an initial search in the database, the database polygon parts are matched against the query polygon. The best matches are shown to the user, who has to decide which are relevant to its query. In successive iterations, the system infers which parts of the query are of interest, and makes a search with those parts only.

This paper introduces a measure for computing the dissimilarity between multiple polylines and a polygon based on the turning function, and describes a part-based retrieval system using that dissimilarity measure. This dissimilarity can be efficiently computed in time O(kmn log mn), where m denotes the number of vertices in the polygon, and n is the total number of vertices in the k polylines that are matched against the polygon. This dissimilarity measure identifies similarities even when a significant portion of one shape is different from the other, for example because the shape is articulated, or because of occlusion or distortion. The effectiveness of the dissimilarity measure in demonstrated in a part-based shape retrieval system. Quantitative experimental verification is performed with a known ground-truth, the MPEG-7 Core Experiment test set, in a comparison with the Curvature Scale Space method, and a global turning angle function method.

We present a method for searching databases of symbolically represented polyphonic music that exploits advantages of transportation distances such as continuity and partial matching in the pitch dimension. By segmenting queries and database documents, we also gain partial matching in the time dimension. Thus, we can find short queries in long database documents, and have a method more robust against pitch and tempo fluctuations in the queries or database documents than we would with transportation distances alone.

Recent developments in techniques for modeling, digitizing and visualizing 3D shapes has led to an explosion in the number of available 3D models on the Internet and in domain-specific databases. This has led to the development of 3D shape retrieval systems that, given a query object, retrieve similar 3D objects. For visualization, 3D shapes are often represented as a surface, in particular polygonal meshes, for example in VRML format. Often these models contain holes, intersecting polygons, are not manifold, and do not enclose a volume unambiguously. On the contrary, 3D volume models, such as solid models produced by CAD systems, or voxels models, enclose a volume properly. This paper surveys the literature on methods for content based 3D retrieval, taking into account the applicability to surface models as well as to volume models. The methods are evaluated with respect to several requirements of content based 3D shape retrieval, such as: (1) shape representation requirements, (2) properties of dissimilarity measures, (3) efficiency, (4) discrimination abilities, (5) ability to perform partial matching, (6) robustness, and (7) necessity of pose normalization. Finally, the advantages and limits of the several approaches in content based 3D shape retrieval are discussed.

We propose a novel type of decomposition for polygonal shapes. It is thought that, for the task of object recognition, the human visual system uses a part-based representation. Decompositions based on skeletons have been previously proposed in computer vision. Our method is the first one, however, based on the straight line skeleton. Compared to the medial axis, the straight line skeleton has a few advantages: it contains only straight segments and has a lower combinatorial complexity. The skeletal nodes and the way they are generated are the basis for our decomposition, which has two stages that result in a decomposition into (possibly overlapping) parts. First, a number of visually striking parts are identified, then their boundaries are successively simplified, by locally removing detail. Our method runs in time ((n+r1+r22)log2 n), after the skeleton construction, where n is the number of vertices in the polygon, r1 the number of split events, and r2 the number of reflex edge annihilations. The decomposition is invariant to rigid motions and uniform scalings. We present results indicating that it provides plausible decompositions for a variety of shapes. This makes it attractive for partial shape matching in content-based image retrieval.

Due to the recent improvements in laser scanning technology, 3D visualization and modeling, there is an increasing need for tools supporting the automatic search for 3D objects in archives. In this paper we describe a new geometric approach to 3D shape comparison and retrieval for arbitrary objects described by 3D polyhedral models that may contain gaps. In contrast with existing approaches, our approach takes the overall relative spatial location into account by representing the 3D shape as a weighted point set. To compare two objects geometrically, we generate for each object a weighted point set, which represents for each non-empty grid cell a salient point. We compare three methods to obtain in each grid cell a salient point and a weight: (1) choose the vertex in the cell with the highest Gaussian curvature, and choose as weight a measure for that curvature, (2) choose the area-weighted mean of the vertices in the cell, and choose as weight a measure denoting the normal variation of the facets in the cell and (3) choose the center of mass of all vertices in the cell, and choose as weight one. Finally, we compute the similarity between two shapes by comparing their weighted point sets using a new shape similarity measure based on weight transportation that is a variation of the Earth Mover's Distance. Unlike the Earth Mover's Distance, the new shape similarity measure satisfies the triangle inequality. This property makes it suitable for use in indexing schemes that depend on the triangle inequality, such as the one we introduce, based on so-called vantage objects. The strength of our approach is proven through experimental results using a database consisting of 133 models such as mugs, cars and boats, and a database consisting of 512 models, mostly air planes, classified into conventional air planes, delta-jets, multi-fuselages, biplanes, helicopters and other models. The results show that the retrieval performance is better than related shape matching methods.

In this paper we show that in sorting-based applications of parametric search, Quicksort can replace the parallel sorting algorithms that are usually advocated, and we argue that Cole's optimization of certain parametric-search algorithms may be unnecessary under realistic assumptions about the input. Furthermore, we present a generic, flexible, and easy-to-use framework that greatly simplifies the implementation of algorithms based on parametric search. We use our framework to implement an algorithm that solves the Fréchet-distance problem. The implementation based on parametric search is faster than the binary-search approach that is often suggested as a practical replacement for the parametric-search technique.

Shape matching is an important ingredient in shape retrieval, recognition and classification, alignment and registration, and approximation and simplification. This paper treats various aspects that are needed to solve shape matching problems: choosing the precise problem, selecting the properties of the similarity measure that are needed for the problem, choosing the specific similarity measure, and constructing the algorithm to compute the similarity. The focus is on methods that lie close to the field of computational geometry.

This paper describes a number of design issues and programming paradigms that affect the development of Cgal, the computational geometry algorithms library. Genericity of the library is achieved by concepts such as templates, iterators, and traits classes. This allows the application programmer to plug in own types of containers and point types, for example. The paper gives an explanation of these concepts and examples of how they are used.

Affine invariant pattern metrics are useful for shape recognition. It is important that such a metric is robust for various defects. We formalize these types of robustness using four axioms. Then, we present the reflection metric. This is an affine invariant metric defined for the large family of "boundary patterns". A boundary pattern is a finite union of n-1 dimensional algebraic surface patches in ℝn. Such a pattern may have multiple connected components. We prove that the reflection metric satisfies the four robustness axioms.

Article A general method for partial point set matching Share on Authors: Michiel Hagedoorn Utrecht University, Department of Computing Science, Padualaan 14, 3584 CH Utrecht, The Netherlands Utrecht University, Department of Computing Science, Padualaan 14, 3584 CH Utrecht, The NetherlandsView Profile , Remco Veltkamp Utrecht University, Department of Computing Science, Padualaan 14, 3584 CH Utrecht, The Netherlands Utrecht University, Department of Computing Science, Padualaan 14, 3584 CH Utrecht, The NetherlandsView Profile Authors Info & Claims SCG '97: Proceedings of the thirteenth annual symposium on Computational geometryAugust 1997 Pages 406–408https://doi.org/10.1145/262839.263030Published:01 August 1997 5citation510DownloadsMetricsTotal Citations5Total Downloads510Last 12 Months2Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Abstract Modeling a curve through minimizing its energy yields an overall smooth curve. A common way to model shape features is to perform the minimization subject to a number of interpolation constraints. This way of modeling is attractive because the designer is not bothered with the precise representation of the curve (e.g. control points). However, local shape specification by means of interpolation constraints is very limited. On the other hand, local deformation by repositioning control points is powerful but very laborious, and destroys the minimal energy property. In this paper, deform operators are introduced for 3D curve modeling that have built‐in energy terms that have an intuitive effect. These operators allow local shape modification and do justice to the energy minimization way of modeling.

Abstract Computational Morphology is the analysis of form by computational means. This discipline typically uses techniques from Computational Geometry and Computer Aided Geometric Design. The present paper is more specifically about the construction and manipulation of closed object boundaries through a set of scattered points in 3D. Original results are developed in three stages of computational morphology: impose a geometrical structure on the set of points; construct a polyhedral boundary surface from this geometrical structure; build a hierarchy of polyhedral approximations together with localization information; The economic advantage of this approach is that there is no dependency on any specific data source. It can be used for various types of data sources or when the source is unknown.

Abstract This survey presents an overview to various types of continuity of curves and surfaces, in particular parametric (Cn), visual or geometric (Cn, Gn), Frenet frame (Fn), and tangent surface continuity (Tn), and discusses the relation with curve and surface modeling, visibility of (dis) continuities, and graphics rendering algorithms. It is the purpose of this paper to provide an overview of types of continuity, and to put many terms and definitions on a common footing in order to give an understanding of the subject. 1991 Computing Reviews Classification: I.3.5 [Computer Graphics] Computational geometry and object modeling. I.3.7 [Computer Graphics] Three‐Dimensional Graphics and Realism.