The use of attention mechanisms in deep learning solutions has become popular within natural language processing tasks. The use of these mechanisms allows managing the relevance of the elements of a sequence in accordance with their context, however, this relevance has been observed independently between the pairs of elements of a sequence (self-attention) or between the application domain of a sequence (contextual attention), leading to the loss of relevant information and limiting the representation of the sequences. To tackle these particular issues, we propose a dual attention mechanism, which trades off the previous limitations, by considering the internal and contextual relationships between the elements of the sequence. Additionally, we propose the extension of the dual attention mechanism into a multi-layer perspective, through the weighted fusion of the different encoding layers of deep architectures. As the interpretation of abusive language is highly context-dependent, its identification is an ideal task to evaluate the proposed attention mechanism. Accordingly, we considered six standard collections for the abusive language identification task. The obtained results are encouraging; the proposed hierarchical attention mechanism outperformed the current self-attention and contextual attention mechanisms coupled with recurrent neural networks and Transformers, as well as, state-of-the-art approaches in detecting abusive language.

We address the challenge of detecting questionable content in online media, specifically the subcategory of comic mischief. This type of content combines elements such as violence, adult content, or sarcasm with humor, making it difficult to detect. Employing a multimodal approach is vital to capture the subtle details inherent in comic mischief content. To tackle this problem, we propose a novel end-to-end multimodal system for the task of comic mischief detection. As part of this contribution, we release a novel dataset for the targeted task consisting of three modalities: video, text (video captions and subtitles), and audio. We also design a HIerarchical Cross-attention model with CAPtions (HICCAP) to capture the intricate relationships among these modalities. The results show that the proposed approach makes a significant improvement over robust baselines and state-of-the-art models for comic mischief detection and its type classification. This emphasizes the potential of our system to empower users, to make informed decisions about the online content they choose to see.

CodaLab Competitions is an open source web platform designed to help data scientists and research teams to crowd-source the resolution of machine learning problems through the organization of competitions, also called challenges or contests. CodaLab Competitions provides useful features such as multiple phases, results and code submissions, multi-score leaderboards, and jobs running inside Docker containers. The platform is very flexible and can handle large scale experiments, by allowing organizers to upload large datasets and provide their own CPU or GPU compute workers.

The present work shows the development of a data collection platform that permits to collect new video and voice data sets and allows to apply a standardized personality test to analyze the effectiveness of the automatic personality recognizers with respect to the results of a standardized personality test of the same participant and thus, have elements to improve the evaluated models. These models can then be integrated into intelligent learning environments to personalize and adapt the content presented to students based on their dominant personality traits.

As a safety-critical application, automated driving aims to provide autonomy and safe navigation. Although recent advances in perception algorithms based on deep learning have boosted the progress in such field, state of the art depends on availability of large datasets. Thus, most visual analysis in this context is related to the detection and classification of previously known classes. However, dynamic environments with complex interactions among traffic elements can lead to unpredictable anomalous situations that are not registered previously. On the other hand, most of prior work on visual novelty detection points out novelty instances but does not provide additional useful information that can be used in the next levels for decision-making. In this paper we propose an approach to detect global visual novelties and their corresponding type in real driving scenarios. Based on pixel-wise and perceptual information, we use a generative adversarial network to detect novelties and a support vector machine to identify their categories. Its performance is experimentally evaluated using the Ford self-driving dataset. Experimental results show that the average area under the curve (AUC) surpasses 0.97 for novelty detection, and novelty type identification reaches an average of 92.7% of accuracy requiring only a small number of samples for training.

Explainability and interpretability are two critical aspects of decision support systems. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of apparent personality recognition. To the best of our knowledge, this is the first effort in this direction. We describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, evaluation protocol, proposed solutions and summarize the results of the challenge. We investigate the issue of bias in detail. Finally, derived from our study, we outline research opportunities that we foresee will be relevant in this area in the near future.

Personality analysis has been widely studied in psychology, neuropsychology, and signal processing fields, among others. From the past few years, it also became an attractive research area in visual computing. From the computational point of view, by far speech and text have been the most considered cues of information for analyzing personality. However, recently there has been an increasing interest from the computer vision community in analyzing personality from visual data. Recent computer vision approaches are able to accurately analyze human faces, body postures and behaviors, and use these information to inferapparentpersonality traits. Because of the overwhelming research interest in this topic, and of the potential impact that this sort of methods could have in society, we present in this paper an up-to-date review of existing vision-based approaches for apparent personality trait recognition. We describe seminal and cutting edge works on the subject, discussing and comparing their distinctive features and limitations. Future venues of research in the field are identified and discussed. Furthermore, aspects on the subjectivity in data labeling/evaluation, as well as current datasets and challenges organized to push the research on the field are reviewed.

This special section is formed by 15 articles of outstanding quality that together comprise a snapshot of cutting edge automated machine learning (AutoML) research.

The papers in this special issue comprise all aspects of computer vision and pattern recognition devoted to image and video inpainting, including related tasks like denoising, debluring, sampling, super-resolutkon enhancement, restoration, hallucination, etc. The special issue was associated to the 2018 Chalearn Looking at People Satellite ECCV Workshop1 and the 2018 ChaLearn Challenges on Image and Video Inpainting.

Adrian Pastor López-Monroy, Fabio A. González, Manuel Montes, Hugo Jair Escalante, Thamar Solorio. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

The papers in this special section examine the concept of automated face analysis (AFA). AFA has received special attention from the computer vision and pattern recognition communities. Research progress often gives the impression that problems such as face recognition and face detection are solved, at least for some scenarios. Several aspects of face analysis remain open problems, including the implementation of large scale face recognition/detection methods for in the wild images, emotion recognition, micro-expression analysis, and others. The community keeps making rapid progress on these topics, with continual improvement of current methods and creation of new ones that push the state-of-the-art. Applications are countless, including security and video surveillance, human computer/robot interaction, communication, entertainment, and commerce, while having an important social impact in assistive technologies for education and health. The importance of face analysis, together with the vast amount of work on the subject and the latest developments in the field, motivated us to organize a special section on this theme. The scope of the compilation comprises all aspects of face analysis from a computer vision perspective. Including, but not limited to: recognition, detection, alignment, reconstruction of faces, pose estimation of faces, gaze analysis, age, emotion, gender, and facial attributes estimation, and applications among others.

The papers in this special section focus on personality analysis. Automatic analysis of videos to characterize human behavior has become an active area of research with applications in affective computing, human-machine interfaces, gaming, security, marketing, and health, just to mention a few. Research advances in multimedia information processing, computer vision and pattern recognition have lead to established methodologies that are able to successfully recognize consciously executed actions, or intended movements (e.g., gestures, actions, interactions with objects and other people). However, recently there has been much progress in terms of computational approaches to characterize sub-conscious behaviors, which may be revealing aptitudes or competence, hidden intentions, and personality traits. Much remains to be done still, but it is essential nowadays to have a compilation of cutting edge work in this direction to identify potential opportunities and challenges involved. In this line, we edited a special issue on automatic methods for apparent personality analysis. Personality refers to individual differences in characteristic patterns of thinking, feeling and behaving.

People form first impressions about the personalities of unfamiliar individuals even after very brief interactions with them. In this study we present and evaluate several models that mimic this automatic social behavior. Specifically, we present several models trained on a large dataset of short YouTube video blog posts for predicting apparent Big Five personality traits of people and whether they seem suitable to be recommended to a job interview. Along with presenting our audiovisual approach and results that won the third place in the ChaLearn First Impressions Challenge, we investigate modeling in different modalities including audio only, visual only, language only, audiovisual, and combination of audiovisual and language. Our results demonstrate that the best performance could be obtained using a fusion of all data modalities. Finally, in order to promote explainability in machine learning and to provide an example for the upcoming ChaLearn challenges, we present a simple approach for explaining the predictions for job interview recommendations.

A wide range of new possibilities in the area of intelligent transportation systems (ITS) emerged when sensors, such as accelerometers, were introduced in practically every smartphone. A clear example is using a driver's smartphone to detect the vertical movement experienced by the vehicle when passing over a pothole or bump; in other words, sensing the quality of the road. To this end, several approaches have been proposed in the literature, most of them based on thresholds applied to accelerometer readings. Nonetheless, no fair comparison of these approaches had been done until now, mainly because of the lack of public datasets. In this paper, we propose a platform to create road data sets that could be used by the community to create their own roads with their own requirements. Using this platform, we assembled a data set of 30 roads plagued with potholes and bumps, which we used to evaluate the most popular heuristics previously reported. From our study, a heuristic, called STDEV(Z), based on standard deviation analysis proposed by Mednis et al. obtained the best results among the considered reference methodologies. This finding suggests that measures of dispersion, specifically standard deviation, are among the best indicators to identify disruptions on accelerometer readings. From this point, we fused features used by all these heuristics within our own feature vector, which we used with a support vector machine. We show that the proposed methodology clearly outperforms all other evaluated methods. To support these conclusions, results were statistically validated. We expect to lay the first steps to homogenize future comparisons as well as to provide stronger baselines to be considered in subsequent works.

The interest in action and gesture recognition has grown considerably in the last years. In this paper, we present a survey on current deep learning methodologies for action and gesture recognition in image sequences. We introduce a taxonomy that summarizes important aspects of deep learning for approaching both tasks. We review the details of the proposed architectures, fusion strategies, main datasets, and competitions. We summarize and discuss the main works proposed so far with particular interest on how they treat the temporal dimension of data, discussing their main features and identify opportunities and challenges for future research.

This paper reviews and discusses research advances on “explainable machine learning” in computer vision. We focus on a particular area of the “Looking at People” (LAP) thematic domain: first impressions and personality analysis. Our aim is to make the computational intelligence and computer vision communities aware of the importance of developing explanatory mechanisms for computer-assisted decision making applications, such as automating recruitment. Judgments based on personality traits are being made routinely by human resource departments to evaluate the candidates' capacity of social insertion and their potential of career growth. However, inferring personality traits and, in general, the process by which we humans form a first impression of people, is highly subjective and may be biased. Previous studies have demonstrated that learning machines can learn to mimic human decisions. In this paper, we go one step further and formulate the problem of explaining the decisions of the models as a means of identifying what visual aspects are important, understanding how they relate to decisions suggested, and possibly gaining insight into undesirable negative biases. We design a new challenge on explainability of learning machines for first impressions analysis. We describe the setting, scenario, evaluation metrics and preliminary outcomes of the competition. To the best of our knowledge this is the first effort in terms of challenges for explainability in computer vision. In addition our challenge design comprises several other quantitative and qualitative elements of novelty, including a “coopetition” setting, which combines competition and collaboration.

This paper reviews the historic of ChaLearn Looking at People (LAP) events. We started in 2011 (with the release of the first Kinect device) to run challenges related to human action/activity and gesture recognition. Since then we have regularly organized events in a series of competitions covering all aspects of visual analysis of humans. So far we have organized more than 10 international challenges and events in this field. This paper reviews associated events, and introduces the ChaLearn LAP platform where public resources (including code, data and preprints of papers) related to the organized events are available. We also provide a discussion on our main findings and perspectives of ChaLearn LAP activities.

Face detection is the first step for automatic face recognition systems. However, detecting faces is not an easy task due to variations in factors such as pose, illumination, scale among others. Efficient face detection algorithms like the one proposed by Viola-Jones allows one to detect faces in r eal-time with high accuracy rates. However, this algorithm involves several stages that consume huge computation, particularly for the training process. Also, increasing input image’s size for face detection and using large training data sets for face recognition demand additional computing resources to achieve real-time processing. In this paper we present a parallel approach to perform three stages of the Viola-Jones face detection algorithm, particularly for the integral image computation, Haar-like features estimation and the evaluation of these features. Our experimental results show that our proposed approach obtains better performance than the OpenCV library implementations.

Accurately classifying roadway surface disruptions (RSDs) plays a crucial role to enhance quality transportation and road safety. To this end, smartphones are becoming an ad hoc tool to collect road data, while the user is at the steering wheel. In this paper, for the first time, sensed data are represented with a novel technique inspired in the bag of words representation. New results suggest that segments of accelerometer readings play a key role to characterize different classes of events, boosting classification performance. A novel data collection process was conducted in real-life environments, where the smartphones were freely placed at five user-surveyed locations, within a fleet of cars and trucks. To the best of our knowledge, this is the largest and most heterogenous data set for RSDs, and we make it publicly available. We approach the problem of identifying RSDs as one of supervised learning, where we contrast representative classifiers, most of them not previously reported. We exhaustively evaluated the performance of all classifiers in six data sets, most of them resembling actual data sets used in similar projects. We found that in all cases, the best classifier outperforms the best results reported so far. The proposed methodology was extensively evaluated through a sensitivity analysis to determine the relevance of the parameters. Experimental results reveal that the representation technique boosts considerably the classification performance when compared with the state of the art solutions, reducing in one order of magnitude the false-positives/negatives rate and surpassing the classification accuracy for about 10% in a multiclass data set.

This paper provides an overview of the Joint Contest on Multimedia Challenges Beyond Visual Analysis. We organized an academic competition that focused on four problems that require effective processing of multimodal information in order to be solved. Two tracks were devoted to gesture spotting and recognition from RGB-D video, two fundamental problems for human computer interaction. Another track was devoted to a second round of the first impressions challenge of which the goal was to develop methods to recognize personality traits from short video clips. For this second round we adopted a novel collaborative-competitive (i.e., coopetition) setting. The fourth track was dedicated to the problem of video recommendation for improving user experience. The challenge was open for about 45 days, and received outstanding participation: almost 200 participants registered to the contest, and 20 teams sent predictions in the final stage. The main goals of the challenge were fulfilled: the state of the art was advanced considerably in the four tracks, with novel solutions to the proposed problems (mostly relying on deep learning). However, further research is still required. The data of the four tracks will be available to allow researchers to keep making progress in the four tracks.

k-NN is one of the most popular and effective classifiers nowadays. However, it has some limitations that overcome its applicability in large scale scenarios: basically, it requires storing the whole training set, and it computes distances of a test sample with the training data set. These limitati ons have been traditionally alleviated with data reduction techniques. This paper introduces a multi-objective evolutionary approach for data reduction. Our method simultaneously generates prototypes and selects features for k-NN classifiers. Contrary to most of the existing approaches, our method treats the problem with multi-objective evolutionary optimizers. We show the effectiveness of our proposal in benchmark data and compare its performance with state of the art techniques.

This paper introduces a framework for gesture and action recognition based on the \nevolution of temporal gesture primitives, or subgestures. Our work is inspired on the \nprinciple of producing genetic variations within a population of gesture subsequences, \nwith the goal of obtaining a set of gesture units that enhance the generalization capability of standard gesture recognition approaches. In our context, gesture primitives are evolved over time using dynamic programming and generative models in order to recognize complex actions. In few generations, the proposed subgesture-based representation \nof actions and gestures outperforms the state of the art results on the MSRDaily3D and \nMSRAction3D datasets.

Cellular or fine grained Genetic Algorithms (GAs) are a massively parallel algorithmic approach to GAs. Decentralizing their population allows alternative ways to explore and to exploit the solutions landscape. Individuals interact locally through nearby neighbors while the entire population is globally exploring the search space throughout a predefined population's topology. Having a decentralized population requires the definition of other algorithmic configuration parameters; such as shape and number of individuals within the local neighborhood, population's topology shape and dimension, local instead of global selection criteria, among others. In this article, attention is paid to the population's topology dimension in cGAs. Several benchmark problems are assessed for 1, 2, and 3 dimensions while combining a local selection criterion that significantly affect overall selective pressure. On the other hand, currently available high performance processing platforms such as Field Programmable Gate Arrays (FPGAs) and Graphics Processing Units (GPUs) offer massively parallel fabrics. Therefore, having a strong empirical base to understand structural properties in cellular GAs would allow to combine physical properties of these platforms when designing hardware architectures to tackle difficult optimization problems where timing constraints are mandatory.

The bag of visual words is a well established representation in diverse computer vision problems. Taking inspiration from the fields of text mining and retrieval, this representation has proved to be very effective in a large number of domains. In most cases, a standard term-frequency weighting scheme is considered for representing images and videos in computer vision. This is somewhat surprising, as there are many alternative ways of generating bag of words representations within the text processing community. This paper explores the use of alternative weighting schemes for landmark tasks in computer vision: image categorization and gesture recognition. We study the suitability of using well-known supervised and unsupervised weighting schemes for such tasks. More importantly, we devise a genetic program that learns new ways of representing images and videos under the bag of visual words representation. The proposed method learns to combine term-weighting primitives trying to maximize the classification performance. Experimental results are reported in standard image and video data sets showing the effectiveness of the proposed evolutionary algorithm.

Following previous series on Looking at People (LAP) challenges [1], [2], [3], in 2015 ChaLearn runs two new competitions within the field of Looking at People: age and cultural event recognition in still images. We propose the first crowd-sourcing application to collect and label data about apparent age of people instead of the real age. In terms of cultural event recognition, tens of categories have to be recognized. This involves scene understanding and human analysis. This paper summarizes both challenges and data, providing some initial baselines. The results of the first round of the competition were presented at ChaLearn LAP 2015 IJCNN special session on computer vision and robotics http://www.dtic.ua.es/~jgarcia/IJCNN2015. Details of the ChaLearn LAP competitions can be found at http://gesture.chalearn.org/.

ChaLearn is organizing the Automatic Machine Learning (AutoML) contest for IJCNN 2015, which challenges participants to solve classification and regression problems without any human intervention. Participants' code is automatically run on the contest servers to train and test learning machines. However, there is no obligation to submit code; half of the prizes can be won by submitting prediction results only. Datasets of progressively increasing difficulty are introduced throughout the six rounds of the challenge. (Participants can enter the competition in any round.) The rounds alternate phases in which learners are tested on datasets participants have not seen, and phases in which participants have limited time to tweak their algorithms on those datasets to improve performance. This challenge will push the state of the art in fully automatic machine learning on a wide range of real-world problems. The platform will remain available beyond the termination of the challenge.

k-NN is one of the most popular and effective models for pattern classification. However, it has two main drawbacks that hinder the application of this method for large data sets: (1) the whole training set has to be stored in memory, and (2) for classifying a test pattern it has to be compared to all other training instances. In order to overcome these shortcomings, prototype generation (PG) methods aim to reduce the size of the training set while maintaining or increasing the classification performance of k-NN. Accordingly, most PG methods aim to generate instances that try to maximize classification performance. Nevertheless, in most cases, the reduction objective is only implicitly optimized. This paper introduces EMOPG, a novel approach to PG based on multi-objective optimization that explicitly optimizes both objectives: accuracy and reduction. Under EMOPG, prototypes are initialized with a subset of training instances selected through a tournament, according to a weighting term. A multi-objective evolutionary algorithm, PAES (Pareto Archived Evolution Strategy), is implemented to adjust the position of the initial prototypes. The optimization process aims to simultaneously maximize the classification performance of prototypes while reducing the number of instances with respect to the training set. A strategy for selecting a single solution from the set of non-dominated solutions is proposed. We evaluate the performance of EMOPG using a suite of benchmark data sets and compare the performance of our proposal with respect to the one obtained by alternative techniques. Experimental results show that our proposed method offers a better trade-off between accuracy and reduction than other methods.

Surrogate-based methods aim at reducing the evaluation of expensive fitness functions in optimization processes. Several surrogate-based methods for evolutionary optimization have been proposed so far, including those based on granular computing / clustering. Granular computing provides granules as an assemblage of entities arranged together by their similarity, functional or physical adjacency, indistinguishability, coherency, or the like. Techniques like this avoid multiple and unnecessary evaluations of individuals repeatedly. In this paper, with the aim of granular computing as a method of grouping data, such information is exploited to obtain knowledge of the structure and parameters of individuals and then, design a Neuro-Fuzzy network that adapts granules' parameters, providing convergence to acceptable solutions with a reduced number of evaluations of the fitness function. We implement this adaptive surrogate in a genetic algorithm and show its performance using benchmark functions.

Nearest-neighbor (NN) methods are highly effective and widely used pattern classification techniques. There are, however, some issues that hinder their application for large scale and noisy data sets; including, its high storage requirements, its sensitivity to noisy instances, and the fact that test cases must be compared to all of the training instances. Prototype (PG) and feature generation (FG) techniques aim at alleviating these issues to some extent; where, traditionally, both techniques have been implemented separately. This paper introduces a genetic programming approach to tackle the simultaneous generation of prototypes and features to be used for classification with a NN classifier. The proposed method learns to combine instances and attributes to produce a set of prototypes and a new feature space for each class of the classification problem via genetic programming. An heterogeneous representation is proposed together with ad-hoc genetic operators. The proposed approach overcomes some limitations of NN without degradation in its classification performance. Experimental results are reported and compared with several other techniques. The empirical assessment provides evidence of the effectiveness of the proposed approach in terms of classification accuracy and instance/feature reduction.

This paper introduces a novel approach for building heterogeneous ensembles based on genetic programming (GP). Ensemble learning is a paradigm that aims at combining individual classifier's outputs to improve their performance. Commonly, classifiers outputs are combined by a weighted sum or a voting strategy. However, linear fusion functions may not effectively exploit individual models' redundancy and diversity. In this research, a GP-based approach to learn fusion functions that combine classifiers outputs is proposed. Heterogeneous ensembles are aimed in this study, these models use individual classifiers which are based on different principles (e.g. decision trees and similarity-based techniques). A detailed empirical assessment is carried out to validate the effectiveness of the proposed approach. Results show that the proposed method is successful at building very effective classification models, outperforming alternative ensemble methodologies. The proposed ensemble technique is also applied to fuse homogeneous models' outputs with results also showing its effectiveness. Therefore, an in-depth analysis from different perspectives of the proposed strategy to build ensembles is presented with a strong experimental support.

In this paper, we propose a novel approach called class-specific maximization of mutual information (CSMMI) using a submodular method, which aims at learning a compact and discriminative dictionary for each class. Unlike traditional dictionary-based algorithms, which typically learn a shared dictionary for all of the classes, we unify the intraclass and interclass mutual information (MI) into an single objective function to optimize class-specific dictionary. The objective function has two aims: 1) maximizing the MI between dictionary items within a specific class (intrinsic structure) and 2) minimizing the MI between the dictionary items in a given class and those of the other classes (extrinsic structure). We significantly reduce the computational complexity of CSMMI by introducing an novel submodular method, which is one of the important contributions of this paper. This paper also contributes a state-of-the-art end-to-end system for action and gesture recognition incorporating CSMMI, with feature extraction, learning initial dictionary per each class by sparse coding, CSMMI via submodularity, and classification based on reconstruction errors. We performed extensive experiments on synthetic data and eight benchmark data sets. Our experimental results show that CSMMI outperforms shared dictionary methods and that our end-to-end system is competitive with other state-of-the-art approaches.

Evolutionary algorithms have gained popularity as an alternative for dealing with multi-objective optimization problems. However, these algorithms require to perform a relatively high number of fitness function evaluations in order to generate a reasonably good approximation of the Pareto front. This can be a shortcoming when fitness evaluations are computationally expensive. In this paper, we propose an approach that combines an evolutionary algorithm with an ensemble of surrogate models based on support vector machines (SVM), which are used to approximate the fitness functions of a problem. The proposed approach performs a model selection process for determining the appropriate hyperparameters values for each SVM in the ensemble. The ensemble is constructed in an incremental fashion, such that the models are updated with the knowledge gained during the evolutionary process, but the information from previous evaluated regions is also preserved. A criterion based on surrogate fidelity is also proposed for determining when should the surrogates be updated. We evaluate the performance of our proposal using a benchmark of test problems widely used in the literature and we compare our results with respect to those obtained by the NSGA-II. Our proposed approach is able to significantly reduce the number of fitness function evaluations performed, while producing solutions which are close to the true Pareto front.

We organized a Grand Challenge and Workshop on Multi-Modal Gesture Recognition.

The recognition of continuous natural gestures is a complex and challenging problem due to the multi-modal nature of involved visual cues (e.g. fingers and lips movements, subtle facial expressions, body pose, etc.), as well as technical limitations such as spatial and temporal resolution and unreliable depth cues. In order to promote the research advance on this field, we organized a challenge on multi-modal gesture recognition. We made available a large video database of 13,858 gestures from a lexicon of 20 Italian gesture categories recorded with a Kinect™ camera, providing the audio, skeletal model, user mask, RGB and depth images. The focus of the challenge was on user independent multiple gesture learning. There are no resting positions and the gestures are performed in continuous sequences lasting 1-2 minutes, containing between 8 and 20 gesture instances in each sequence. As a result, the dataset contains around 1.720.800 frames. In addition to the 20 main gesture categories, "distracter" gestures are included, meaning that additional audio and gestures out of the vocabulary are included. The final evaluation of the challenge was defined in terms of the Levenshtein edit distance, where the goal was to indicate the real order of gestures within the sequence. 54 international teams participated in the challenge, and outstanding results were obtained by the first ranked participants.

This paper elaborates on the benefits of using particle swarm model selection (PSMS) for building effective ensemble classification models. PSMS searches in a toolbox for the best combination of methods for preprocessing, feature selection and classification for generic binary classification tasks. Throughout the search process PSMS evaluates a wide variety of models, from which a single solution (i.e. the best classification model) is selected. Satisfactory results have been reported with the latter formulation in several domains. However, many models that are potentially useful for classification are disregarded for the final model. In this paper we propose to re-use such candidate models for building effective ensemble classifiers. We explore three simple formulations for building ensembles from intermediate PSMS solutions that do not require of further computation than that of the traditional PSMS implementation. We report experimental results on benchmark data as well as on a data set from object recognition. Our results show that better models can be obtained with the ensemble version of PSMS, motivating further research on the combination of candidate PSMS models. Additionally, we analyze the diversity of the classification models, which is known to be an important factor for the construction of ensembles.

In this paper a novel approach for improving automatic image annotation methods is proposed. The approach is based on the fact that accuracy of current image annotation methods is low if we look at the most confident label only. Instead, accuracy is improved if we look for the correct label within the set of the top−k candidate labels. We take advantage of this fact and propose a Markov random field (MRF) based on word co-occurrence information for the improvement of annotation systems. Through the MRF structure we take into account spatial dependencies between connected regions. As a result, we are considering semantic relationships between labels. We performed experiments with iterated conditional modes and simulated annealing as optimization strategies in a subset of the Corel benchmark collection. Experimental results of the proposed method together with a k−nearest neighbors classifier as our annotation method show important error reductions. 1

Artificial neural networks have been proven to be effective learning algorithms since their introduction. These methods have been widely used in many domains, including scientific, medical, and commercial applications with great success. However, selecting the optimal combination of preprocessing methods and hyperparameters for a given data set is still a challenge. Recently a method for supervised learning model selection has been proposed: Particle Swarm Model Selection (PSMS). PSMS is a reliable method for the selection of optimal learning algorithms together with preprocessing methods, as well as for hyperparameter optimization. In this paper we applied PSMS for the selection of the (pseudo) optimal combination of preprocessing methods and hyperparameters for a fixed neural network on benchmark data sets from a challenging competition: the (IJCNN 2007) agnostic vs prior knowledge challenge. A forum for the evaluation of methods for model selection and data representation discovery. In this paper we further show that the use of PSMS is useful for model selection when we have no knowledge about the domain we are dealing with. With PSMS we obtained competitive models that are ranked high in the official results of the challenge.