This study proposes to subtype patients with Parkinson's disease (PD) based on speech characteristics using unsupervised learning. The speech of 88 individuals was analyzed with two sets of features: (1) traditional acoustic metrics via Parselmouth (Praat) and (2) embeddings from the self-supervised wav2vec model. Each was clustered with Gaussian Mixture Models, producing two subtypes. Embedding-based clusters showed significant clinical differences in gait, postural stability, freezing, and tremor, with milder cases linked to education, suggesting cognitive reserve effects. Acoustic-based clusters showed limited distinctions. The results indicate that speech embeddings better capture the severity of motor symptoms, supporting speech as an accessible and non-invasive biomarker for stratification, early diagnosis, and personalized treatment in PD.

Parkinson's Disease (PD) is a neurodegenerative disorder marked by diverse motor and non-motor symptoms, driven largely by dopaminergic neuronal loss in the substantia nigra. Traditional approaches to PD treatment have been impeded by the disease's clinical heterogeneity, highlighting the need for more specific subtyping methods. Recent advancements propose utilizing clinical and biological data to uncover biologically relevant PD subtypes, with neuroimaging being a suitable candidate given the disease's impact on the brain. Previous neuroimaging studies on PD subtyping have utilized small datasets and identified age-related subtypes of PD. We address these limitations by leveraging the Parkinson's Progression Markers Initiative dataset and utilizing Gaussian Mixture Models (GMMs) to cluster neuroimaging data, while preserving clinical data solely for validation purposes. The analysis included 747 patients, each characterized by 12 brain volume features obtained through quality-controlled neuroimaging processing pipelines. GMM clustering revealed three distinct PD subtypes with varying severity of motor symptoms but similar non-motor symptoms and cognitive scores. The subtypes identified - mild, intermediate, and severe motor severity - demonstrated significant differences in motor symptom severity, mainly in terms of bradykinesia, rigidity, and tremor. Our approach provides a stratification that could enhance targeted treatment strategies. Future research should investigate the integration of multimodal and longitudinal data to elucidate further the potential of neuroimaging-based subtyping in predicting disease progression and therapeutic outcomes.

Software effort estimation predicts resources needed for a project, including person-hours and costs, and is vital for effective planning and budgeting. This paper compares two grammar-based evolutionary algorithms: grammar-based genetic programming (GGP) and grammatical evolution (GE). Both algorithms are tested on public project datasets and compared with machine learning models such as support vector machines, artificial neural networks, and least-squares linear regression. Results demonstrate that GGP and GE outperform alternative methods across two evaluation metrics, highlighting their effectiveness in estimating software effort.

Sign language is a visual and gestural communication system used by deaf and hearing-impaired people. Despite numerous deep learning methods proposed for automatic interpretation, a gap persists in developing applications that effectively utilize these models for assisting sign language studies and inclusion. We introduce LERMO (https://lermo.app/), a web game merging machine learning and gamification to enhance sign language fingerspelling. Inspired by Wordle™, LERMO offers an interactive word-guessing game where users can play using a video camera. We create a new dataset of labeled landmark fingerspelling and design our model to ensure optimal speed and efficiency to run on a web browser. We survey approximately 40 users, which find LERMO user-friendly and innovative. From those, 95% believe LERMO could be used to enhance fingerspelling skills.

The widespread use of machine learning (ML) in software engineering (SE) encounters a notable challenge: the need for various domain-specific parameters in algorithms. The issue arises when attempting to reuse these parameters across different applications, resulting in sub-optimal outcomes. This hindrance significantly contributes to the limited migration of ML solutions from research labs to industrial settings. This paper underscores the pressing need for novel research to tackle the overarching problem of generic algorithm customisation. To address this, we propose leveraging Automated Machine Learning (AutoML) approaches. These techniques automatically select intelligible models and their corresponding hyper-parameters for forecasting software defect-proneness. More specifically, this paper adapts an AutoML approach to the field of software defect prediction, namely a hyper-heuristic evolutionary algorithm for automatically designing decision tree algorithms (HEAD-DT), originally proposed to address a generic optimisation problem. We benchmark against the popular general software defect-proneness prediction framework (GSDP) and some standard classifiers. Experimental results reveal that the proposed HEAD-DT implementation surpasses other algorithms across three distinct evaluation measures.

Recently, machine learning has dominated software engineering research, much of which can be attributed to the success of large language models in handling source code-related tasks. Yet, despite the significant advancements in pre-trained language models for such area, the exploration of their potential in a multi-task learning (MTL) environment remains largely unaddressed. For such, this paper offers a comparative analysis between task-specific and multi-task approaches, focusing on two main tasks: Natural Language Code Search and Unit Test Case Generation. We propose a methodology based on prompt MTL and perform an extensive evaluation, which contrasts the performance of MTL models against their single-task counterparts making use of three pre-trained models across seven datasets. Delving deeper, we conduct an additional exploratory analysis to uncover the underlying causes that explain the observed results, investigating the specificities that govern the application of MTL in this context. Our empirical results sketch a nuanced landscape. MTL does not improve the results when compared to its single-task counterparts. Nevertheless, there are some scenarios for particular models, or when data is scarce, that make MTL and STL achieve quite similar results. The most important finding, however, is that the nature of the pre-training tasks significantly affects the fine-tuning capabilities of MTL, opening space for more guided research on how to pre-train and fine-tune those models.

This study evaluates the potential of the “Segment Anything Model” (SAM) as a robust alternative for medical imaging segmentation in a zero-shot learning context. We evaluate SAM's performance across six diverse medical imaging datasets spanning four different imaging modalities. By employing eight unique prompting strategies we reveal comprehensive insights into SAM's adaptability. The Bounding Box strategy, with its variations, matched or even outperformed existing benchmarks. On the Breast Ultrasound Images dataset, SAM notably outperformed SOTA models, attesting to its capability as a robust zero-shot segmentation tool. Conversely, challenges arose with datasets having indistinct boundaries and inconsistent annotations, as in skin lesion images. The study also establishes a practical set of guidelines aimed at optimizing SAM's clinical usage. The findings underscore SAM's potential as a powerful, versatile tool for medical imaging segmentation, alleviating the burden of manual segmentation and potentially improving ground truth masks for labeling new datasets. With its minimal resource requirements and promising results, SAM represents an exciting advancement in medical imaging analysis.

The COVID-19 pandemic has overwhelmed healthcare systems worldwide. Computed tomography imaging has emerged as an essential tool in diagnosing and monitoring cases, allowing for the detection of pulmonary changes even in the early stages of the disease. This study introduces a radiomics-based predictive model aimed at predicting the need for oxygen support in COVID-19 patients. Utilizing a private dataset collected from a local hospital with 81 patients that underwent two longitudinal chest CT scans, we employed two machine learning algorithms, Random Forest and XGBoost, to analyze radiomic features extracted from lung segmentation. We also explore incorporating clinical features in addition to the radiomic ones and using feature selection techniques to handle the high-dimensionality of the dataset. Our best model achieves an AUC of 0.81 in the test set. Our results indicate that only using radiomic features from the last time point reach a higher performance than with additional data. Therefore, it should be practical to implement a framework to predict the need of oxygen support in a clinical setting, as all the information required by the model comes from a single CT scan.

Behavioural cloning is an imitation learning technique that teaches an agent how to behave via expert demonstrations. Recent approaches use self-supervision of fully-observable unlabelled snapshots of the states to decode state pairs into actions. However, the iterative learning scheme employed by these techniques is prone to get trapped into bad local minima. Previous work uses goal-aware strategies to solve this issue. However, this requires manual intervention to verify whether an agent has reached its goal. We address this limitation by incorporating a discriminator into the original framework, offering two key advantages and directly solving a learning problem previous work had. First, it disposes of the manual intervention requirement. Second, it helps in learning by guiding function approximation based on the state transition of the expert's trajectories. Third, the discriminator solves a learning issue commonly present in the policy model, which is to sometimes perform a ‘no action’ within the environment until the agent finally halts.

Benchmark frameworks and datasets allow us to analyze and understand the knowledge that NLP models capture about the world they were trained on. Several Transformer models have recently been adapted to code-related tasks such as code search, in which the goal is to find the most semantically relevant code given a query written in natural language. To achieve satisfactory performance, the retrieval models heavily rely on the quality of the query. In this paper, we introduce the Natural Language Code Search Robustness Benchmark (COBE), which provides a more holistic evaluation of the state-of-the-art models considering several aspects of the retrieval models, such as: (i) retrieval capabilities measured in multiple ranking metrics; (ii) robustness to a plethora of input perturbations; (iii) efficiency in terms of training and retrieval times; and (iv) stability across fine-tuning runs. We shed a light over important questions showing that simply computing performance-based retrieval metrics does not suffice to evaluate this kind of model. The proposed benchmark introduces novel metrics and measurement strategies that allow a rigorous quantitative analysis of input-query robustness while providing an understanding of model generalization behavior. We perform an extensive set of experiments using state-of-the-art models such as CodeBert, GraphCodeBert, and CodeT5. Those models are fine-tuned over many different scenarios in six programming languages. Several models trained in this study outperform their state-of-the-art counterparts, which provides evidence that the standard fine-tuning approach used in code search related work is sub-optimal. The proposed benchmark is a powerful tool to evaluate code search models, providing insights on how they behave during fine-tuning and how they are interpreting the input queries.

Despite the success of neural architectures for Visual Question Answering (VQA), several recent studies have shown that VQA models are mostly driven by superficial correlations that are learned by exploiting undesired priors within training datasets. They often lack sufficient image grounding or tend to overly-rely on textual information, failing to capture knowledge from the images. This affects their generalization to test sets with slight changes in the distribution of facts. To address such an issue, some bias mitigation methods have relied on new training procedures that are capable of synthesizing counterfactual samples by masking critical objects within the images, and words within the questions, while also changing the corresponding ground truth. We propose a novel model-agnostic counterfactual training procedure, namely Efficient Counterfactual Debiasing (ECD), in which we introduce a new negative answer-assignment mechanism that exploits the probability distribution of the answers based on their frequencies, as well as an improved counterfactual sample synthesizer. Our experiments demonstrate that ECD is a simple, computationally-efficient counterfactual sample-synthesizer training procedure that establishes itself as the new state of the art for unbiased VQA.

Compressed neural-network models have a growing relevance in the Deep Learning literature, since they allow the deployment of AI on devices with computational constraints for many automation purposes. Despite the amount and diversity of work for this purpose, there is no standard benchmark in the literature, and it is often difficult to choose the proper approach due to the large difference between models, datasets, and training details that are tested. Therefore, this paper proposes a standard experimental benchmark for different model compression approaches for the object detection task, using a fixed model (the well-known YOLOv3) and training scheme. Between Pruning, Knowledge Distillation, and Neural Architecture Search, our experiments reveal that the best trade-off is by using pruning, which enables the creation of a model with 80.67% mAP of the original model but removing 98.8% of the parameters, 96.53% of the Multiply-Accumulate Operations, and reducing the storage size from 235.44MB to 11.61MB.

In this paper, we introduce a novel approach for training image-text alignment models, namely ADAPT.Image-text alignment methods are often used for cross-modal retrieval, i.e., to retrieve an image given a query text, or captions that successfully label an image.ADAPT is designed to adjust an intermediate representation of instances from a modality a using an embedding vector of an instance from modality b.Such an adaptation is designed to filter and enhance important information across internal features, allowing for guided vector representations -which resembles the working of attention modules, though far more computationally efficient.Experimental results on two large-scale Image-Text alignment datasets show that ADAPT-models outperform all the baseline approaches by large margins.Particularly, for Image Retrieval, ADAPT, with a single model, outperforms the state-of-the-art approach by a relative improvement of R@1 ≈ 24% and for Image Annotation, R@1 ≈ 8% on Flickr30k dataset.On MS COCO it provides an improvement of R@1 ≈ 12% for Image Retrieval, and ≈ 7% R@1 for Image Annotation.Code is available at https://github.com/jwehrmann/retrieval.pytorch.

Radiological chest examinations like chest X-ray play a fundamental role in the fight against the outbreak of COVID-19 pneumonia, caused by the coronavirus strain SARSCov-2. This study aims to investigate classification models to differentiate chest X-ray images of COVID-19-based and typical pneumonia using hand-crafted radiomic features, understanding the distinctive radiographic features of COVID19. A total of 136 segmented chest X-rays from two public databases were used to train and evaluate the classification methods. The PyRadiomics library was used to extract first and second-order statistical texture features in the right (R), left (L), and in superior, middle and bottom lung zones for each lung side. For performing feature selection, data was split in training (80%) and test (20%) sets. Stratified K-folds (K=5) was used within the training dataset for cross-validation. The most relevant radiomic features were selected after measuring validation accuracy and relative feature importance. Support vector machines (SVM), random forest (RF), AdaBoost (AB), and logistic regression (LR) were analyzed as potential classifiers. The AB model was the best discriminant method between features related to COVID-19-based when compared to typical pneumonia, using a model of lung segmentation in six distinct lung zones (AUC =0.98). Our study shows a predominance of radiomic feature selection in the right lung, with a tendency to the upper lung zone.

Recent research advances in Computer Vision and Natural Language Processing have introduced novel tasks that are paving the way for solving AI-complete problems. One of those tasks is called Visual Question Answering (VQA). This system takes an image and a free-form, open-ended natural-language question about the image, and produce a natural language answer as the output. Such a task has drawn great attention from the scientific community, which generated a plethora of approaches that aim to improve the VQA predictive accuracy. Most of them comprise three major components: (i) independent representation learning of images and questions; (ii) feature fusion so the model can use information from both sources to answer visual questions; and (iii) the generation of the correct answer in natural language. With so many approaches being recently introduced, it became unclear the real contribution of each component for the ultimate performance of the model. The main goal of this paper is to provide a comprehensive analysis regarding the impact of each component in VQA models. Our extensive set of experiments cover both visual and textual elements, as well as the combination of these representations in form of fusion and attention mechanisms. Our major contribution is to identify core components for training VQA models so as to maximize their predictive performance.

Imitation from observation is a computational technique that teaches an agent on how to mimic the behavior of an expert by observing only the sequence of states from the expert demonstrations. Recent approaches learn the inverse dynamics of the environment and an imitation policy by interleaving epochs of both models while changing the demonstration data. However, such approaches often get stuck into sub-optimal solutions that are distant from the expert, limiting their imitation effectiveness. We address this problem with a novel approach that overcomes the problem of reaching bad local minima by exploring: (i) a self-attention mechanism that better captures global features of the states; and (ii) a sampling strategy that regulates the observations that are used for learning. We show empirically that our approach outperforms the state-of-the-art approaches in four different environments by a large margin.

Recently, learning-based approaches for 3D reconstruction from 2D images have gained popularity due to its modern applications, e.g., 3D printers, autonomous robots, self-driving cars, virtual reality, and augmented reality. The computer vision community has applied a great effort in developing functions to reconstruct the full 3D geometry of objects and scenes. However, to extract image features, they rely on convolutional neural networks, which are ineffective in capturing long-range dependencies. In this paper, we propose to substantially improve Occupancy Networks, a state-of-the-art method for 3D object reconstruction. For such we apply the concept of self-attention within the network's encoder in order to leverage complementary input features rather than those based on local regions, helping the encoder to extract global information. With our approach, we were capable of improving the original work in 5.05% of mesh IoU, 0.83% of Normal Consistency, and more than 10× the Chamfer-L1 distance. We also perform a qualitative study that shows that our approach was able to generate much more consistent meshes, confirming its increased generalization power over the current state-of-the-art.

This paper proposes a framework for training language-invariant cross-modal retrieval models. We also introduce a novel character-based word-embedding approach, allowing the model to project similar words across languages into the same word-embedding space. In addition, by performing cross-modal retrieval at the character level, the storage requirements for a text encoder decrease substantially, allowing for lighter and more scalable retrieval architectures. The proposed language-invariant textual encoder based on characters is virtually unaffected in terms of storage requirements when novel languages are added to the system. Our contributions include new methods for building character-level-based word-embeddings, an improved loss function, and a novel cross-language alignment module that not only makes the architecture language-invariant, but also presents better predictive performance. We show that our models outperform the current state-of-the-art in both single and multi-language scenarios. This work can be seen as the basis of a new path on retrieval research, now allowing for the effective use of captions in multiple-language scenarios. Code is available at https://github.com/jwehrmann/lavse.

The amount of digital pornographic content over the Internet grows daily and accessing such a content has become increasingly easier. Hence, there is a real need for mechanisms that can protect particularly-vulnerable audiences (e.g., children) from browsing the web. Recently, object detection methods based on deep neural networks such as CNNs have improved the effectiveness and efficiency of identifying and blocking pornographic content. Even though improvements in detecting intimate parts have been significant, the occlusion of the content is still primarily done by either blurring or removing regions of the image in an intrusive fashion. A recent study has addressed the problem of censoring the pornographic content in a non-intrusive way by generating the so-called seamless censorship via cycle-consistent generative adversarial networks. Such an approach has managed to automatically add bikinis to naked women without explicit supervision or paired training data. In this paper, we extend that method by designing a novel cycle-consistency framework that leverages sensitive information from an attention-based multi-label convolutional neural network. We evaluate the quality of our novel generative model by conducting a web survey with over 1000 opinions regarding the resulting images from our method and from baseline approaches. Results of the survey show that our method considerably improves the state-of-the-art on the seamless censorship task.

Current state-of-the-art approaches for Natural Language Processing tasks such as text classification are either based on Recurrent or Convolutional Neural Networks. Notwithstanding, those approaches often require a long time to train, or large amounts of memory to store the entire trained models. In this paper, we introduce a novel neural network architecture for ultra-fast, memory-efficient text classification. The proposed architecture is based on word embeddings trained directly over the class space, which allows for fast, efficient, and effective text classification. We divide the proposed architecture into four main variations that present distinct capabilities for learning temporal relations. We perform several experiments across four widely-used datasets, in which we achieve results comparable to the state-of-the-art while being much faster and lighter in terms of memory usage. We also present a thorough ablation study to demonstrate the importance of each component within each proposed model. Finally, we show that our model predictions can be visualized and thus easily explained.

Ensemble learning is the machine learning paradigm that aims at integrating several base learners into a single system under the assumption that the collective consensus outperforms a single strong learner, be it regarding effectiveness, efficiency, or any other problem-specific metric. Ensemble learning comprises three main phases: generation, selection, and integration, and there are several possible (deterministic or stochastic) strategies for executing one or more of those phases. In this paper, our focus is on improving the predictive accuracy of the well-known AdaBoost algorithm. By using its former voting weights as starting point in a global search carried by an Estimation of Distribution Algorithm, we are capable of improving AdaBoost up to ≈ 11 % regarding predictive accuracy in a thorough experimental analysis with multiple public datasets.

This paper provides a very simple yet effective character-level architecture for learning bidirectional retrieval models. Aligning multimodal content is particularly challenging considering the difficulty in finding semantic correspondence between images and descriptions. We introduce an efficient character-level inception module, designed to learn textual semantic embeddings by convolving raw characters in distinct granularity levels. Our approach is capable of explicitly encoding hierarchical information from distinct base-level representations (e.g., characters, words, and sentences) into a shared multimodal space, where it maps the semantic correspondence between images and descriptions via a contrastive pairwise loss function that minimizes order-violations. Models generated by our approach are far more robust to input noise than state-of-the-art strategies based on word-embeddings. Despite being conceptually much simpler and requiring fewer parameters, our models outperform the state-of-the-art approaches by 4.8% in the task of description retrieval and 2.7% (absolute R@1 values) in the task of image retrieval in the popular MS COCO retrieval dataset. We also show that our models present solid performance for text classification, specially in multilingual and noisy domains.

One of the most challenging machine learning problems is a particular case of data classification in which classes are hierarchically structured and objects can be assigned to multiple paths of the class hierarchy at the same time. This task is known as hierarchical multi-label classification (HMC), with applications in text classification, image annotation, and in bioinformatics problems such as protein function prediction. In this paper, we propose novel neural network architectures for HMC called HMCN, capable of simultaneously optimizing local and global loss functions for discovering local hierarchical class-relationships and global information from the entire class hierarchy while penalizing hierarchical violations. We evaluate its performance in 21 datasets from four distinct domains, and we compare it against the current HMC state-of-the-art approaches. Results show that HMCN substantially outperforms all baselines with statistical significance, arising as the novel state-of-the-art for HMC.

Nowadays there are more than 250 million visually-impaired people worldwide and mobility autonomy in outdoor environments is perhaps the greatest challenge they have to face. More specifically, crossing the street with no human aid is an open problem, since the majority of the pedestrian traffic lights in underdeveloped countries do not provide sound aids. There are very few studies addressing the detection of pedestrian traffic lights based on images acquired by mobile devices, and to the best of our knowledge there is a clear gap in the literature regarding the use of recent state-of-the-art computer vision approaches such as deep neural networks for addressing such a problem. In this paper, we investigate the current state-of-the-art in localization/detection and classification based on deep neural networks, and we present a solution for the detection of pedestrian traffic lights together with their current state for helping visually-impaired people to cross the streets with the aid of their mobile devices. For such, we provide a novel public dataset with 4,399 labeled images of pedestrian traffic lights and we present a detailed comparison among the state-of-the-art methods for classification and localization/detection. We show empirical evidence regarding the feasibility of embedding our approach in mobile devices so it can be used by people with visual impairment.

Action recognition is the computer vision task of identifying what action is happening in a given sequence of frames. Traditional approaches rely on handcrafted features and domain-specific image processing algorithms, often resulting in limited accuracy. The substantial advances in deep learning and the availability of larger datasets have allowed techniques that yield much better performance without domain-specific knowledge to recognize actions being performed based on the raw information from video sequences, a strategy called representation learning. However, deep neural networks usually require very large labeled datasets for training and properly generalizing, and due to their high capacity, they often overfit small data, hence providing suboptimal results. This work aims to check the real feasibility of employing deep learning in the context of smallsized action recognition datasets. Our goal is to verify whether deep learning approaches can provide improved performance in cases in which labeled data is not abundant. In order to do so, we perform a thorough empirical analysis in which we investigate distinct network architectures with hyperparameter optimization, as well as different data pre-processing techniques and fusion methods.

The easy access and widespread of the Internet makes it easier than ever to reach content of any kind at any moment, and while that poses several advantages, there is also the fact that sensitive audiences may be inadvertently exposed to nudity content they did not ask for. Virtually every work on nudity and pornography censorship focus solely on performing binary classification, where the result is used to decide whether to completely ignore the accessed content or not. Such an approach may compromise user experience because the entire content, either images or frames of a video, has to be removed/blocked. In this paper, we propose a paradigmatic shift in the literature of adult censorship: instead of detecting and excluding the identified content, we propose to automatically filter out only the sensitive regions of an image. For that, we have developed an image-to-image translation approach based on adversarial training that implicitly locates sensitive regions in images and covers them whilst preserving its semantics, i.e., putting appropriate clothing. We test this novel approach on images of nude women, in which we are capable of automatically generating bikinis that cover the sensitive parts without the additional effort of previously annotating body parts. Our results are visually impressive, proving that it is possible to perform seamless nudity censorship with small effort of data collection and annotation.

Multimodal bidirectional retrieval is a very challenging task that consists of semantically aligning two distinct modalities such as images and textual descriptions, allowing the retrieval of content from one of the modalities given the other. The goal is to find a common semantic space for both modalities in order to discover the correspondences between them. This paper provides a fast and effective attention-based architecture for learning representations for multimodal retrieval, namely SEAM. It is based on a self-attention module that is designed to enhance relevant textual information from word-embeddings while suppressing irrelevant data. We design three incarnations of SEAM, so we can properly assess the performance of the self-attention module when operating over distinct representations, namely: (i) the word-embeddings themselves; (ii) features learned from convolutional layers in distinct granularities; and (iii) features learned from gated recurrent units (GRUs). The output of the self-attention module is projected over a shared multimodal space, so we can map the semantic correspondence between images and descriptions via a contrastive pairwise loss function that minimizes order-violations. We analyze several architectural choices for our approach, and we compare our best models with the current state-of-the-art approaches in the largest and most well-known multimodal retrieval dataset, namely Microsoft COCO. Results show that SEAM outperforms the current state-of-the-art in most cases while being a much faster approach for the task of multimodal retrieval.

With the novel and fast advances in the area of deep neural networks, several challenging image-based tasks have been recently approached by researchers in pattern recognition and computer vision. In this paper, we address one of these tasks, which is to match image content with natural language descriptions, sometimes referred as multimodal content retrieval. Such a task is particularly challenging considering that we must find a semantic correspondence between captions and the respective image, a challenge for both computer vision and natural language processing areas. For such, we propose a novel multimodal approach based solely on convolutional neural networks for aligning images with their captions by directly convolving raw characters. Our proposed character-based textual embeddings allow the replacement of both word-embeddings and recurrent neural networks for text understanding, saving processing time and requiring fewer learnable parameters. Our method is based on the idea of projecting both visual and textual information into a common embedding space. For training such embeddings we optimize a contrastive loss function that is computed to minimize order-violations between images and their respective descriptions. We achieve state-of-the-art performance in the largest and most well-known image-text alignment dataset, namely Microsoft COCO, with a method that is conceptually much simpler and that possesses considerably fewer parameters than current approaches.

Decision trees are one of the most widely employed classification models, mainly due to their capability of being properly interpreted and understood by the domain specialist. However, decision-tree induction algorithms have limitations due to the typical recursive top-down greedy search they implement. Such local search may often lead to quality loss while the partitioning process occurs, generating statistically insignificant rules. In order to avoid the typical greedy strategy and to prevent convergence to local optima, we present a novel Estimation of Distribution Algorithm (EDA) for decision-tree induction, namely Ardennes. For evaluating the proposed approach, we present results of an empirical analysis in 10 real-world classification datasets. We compare Ardennes with both a well-known traditional greedy algorithm for decision-tree induction and also with a more recent global population-based approach. Results show the feasibility of using EDAs as a means to avoid the previously-described problems. We report gains when using Ardennes in terms of accuracy and - equally important - tree comprehensibility.

The movie domain is one of the most common scenarios to test and evaluate recommender systems. These systems are often implemented through a collaborative filtering model, which relies exclusively on the user's feedback on items, ignoring content features. Content-based filtering models are nevertheless a potentially good strategy for recommendation, even though identifying relevant semantic representation of items is not a trivial task. Several techniques have been employed to continuously improve the content representation of items in content-based recommender systems, including low-level and high-level features, text analysis, and social tags. Recent advances on deep learning, particularly on convolutional neural networks, are paving the way for better representations to be extracted from unstructured data. In this work, our main goal is to understand whether these networks can extract sufficient semantic representation from items so we can better recommend movies in content-based recommender systems. For that, we propose DeepRecVis, a novel approach that represents items through features extracted from keyframes of the movie trailers, leveraging these features in a content-based recommender system. Experiments shows that our proposed approach outperforms systems that are based on low-level feature representations.

Activity recognition applications is growing in importance due to two key factors: first there is increased need for more human assistance and surveillance; and second, increased availability of datasets and improved image recognition algorithms have allowed effective recognition of more sophisticated activities. In this paper we develop an activity recognition approach to support visually impaired people that leverages these advances. Specifically, our approach uses a dataset of videos taken from the point of view of a guide-dog to train a convolutional neural-network to recognize the activities taking place around the camera and provide feedback to a visually impaired human user. Our experiments show that our trained models surpass the current state-of-the-art for identifying activities in the doc-centric activity dataset.

With the growth of video content produced by mobile cameras and surveillance systems, an increasing amount of data is becoming available and can be used for a variety of applications such as video surveillance, smart homes, smart cities, and in-home elder monitoring. Such applications focus in recognizing human activities in order to perform different tasks allowing the opportunity to support people in their different scenarios. In this paper we propose a deep neural architecture for kitchen human action recognition. This architecture contains an ensemble of convolutional neural networks connected through different fusion methods to predict the label of each action. Experiments show that our architecture achieves the novel state-of-the-art for identifying cooking actions in a well-known kitchen dataset.

Most work on tweet sentiment analysis is mono-lingual and the models that are generated by machine learning strategies do not generalize across multiple languages. Cross-language sentiment analysis is usually performed through machine translation approaches that translate a given source language into the target language of choice. Machine translation is expensive and the results that are provided by theses strategies are limited by the quality of the translation that is performed. In this paper, we propose a language-agnostic translation-free method for Twitter sentiment analysis, which makes use of deep convolutional neural networks with character-level embeddings for pointing to the proper polarity of tweets that may be written in distinct (or multiple) languages. The proposed method is more accurate than several other deep neural architectures while requiring substantially less learnable parameters. The resulting model is capable of learning latent features from all languages that are employed during the training process in a straightforward fashion and it does not require any translation process to be performed whatsoever. We empirically evaluate the efficiency and effectiveness of the proposed approach in tweet corpora based on tweets from four different languages, showing that our approach comfortably outperforms the baselines. Moreover, we visualize the knowledge that is learned by our method to qualitatively validate its effectiveness for tweet sentiment classification.

Data clustering is an unsupervised learning task that can be regarded as the combinatorial optimisation NP-hard problem of assigning N objects to one (or more) among k clusters. Most data clustering algorithms require the user to set a number of pre-defined parameters that have decisive impact in the formation of clusters, such as the number of clusters (initial or final), cluster radius, minimum number of objects, and similar parameters. In addition, several clustering algorithms are limited with regard to the shape of clusters that can be found, a limitation usually resulting from the optimisation process performed over a given distance metric. In this work, we propose a novel clustering algorithm that addresses the two aforementioned problems regarding the amount of parameters and cluster shape. Our approach makes use of the theory of Estimation of Distribution Algorithms in order to probabilistic sample a set of must-link/cannot link constraints in order to generate a data partition with the proper number of clusters. We name our method PASCAL, and we empirically show that it is capable of not only detecting the right number of clusters but also of properly assigning objects to the correct cluster in a variety of artificial and real problems whose solutions are known in advance.

Data set preprocessing is a critical step for the successful application of machine learning algorithms in classification tasks. Even though we rely on learning algorithms to pinpoint the optimal decision boundaries in the feature space by properly detecting latent relationships among the input features, their performance is often bounded by the discriminative power of the available features. Therefore, much effort has been devoted to developing preprocessing methods that are capable of transforming the input data with the final goal of aiding the machine learning algorithm in building high-quality classification models. One such a method is feature construction, which is a flexible preprocessing procedure that exploits linear and nonlinear transformations of the original feature space in an attempt to capture useful information that is not explicit in the original data. Since the task of feature construction can be modelled as a heuristic search in the space of novel latent features, this paper investigates an evolutionary approach for performing such a task, namely grammatical evolution (GE). In our proposed approach, GE is employed for building an extra novel feature from the available input data in order to maximize the predictive performance of the learning algorithm in training data. Results show that many interesting implicit relationships are indeed found by the evolutionary approach, improving the performance of two well-known decision-tree induction algorithms.

Automatic pattern recognition from videos is a high-complexity task, and well-established Machine Learning algorithms have difficulties in handling it in an efficient and effective fashion. Convolutional Neural Networks are the state-of-the-art method for supervised image classification, borrowing concepts from image processing in order to ensure some degree of scale and position invariance. They are capable of detecting primary features, which are then combined by subsequent layers of the CNN architecture, resulting in the detection of higher-order complex and relevant novel features. Considering that a video is a set of ordered images in time, we propose in this paper to explore CNNs in the context of movie trailers genre classification. Our contributions are twofold. First, we have developed a novel movie trailers dataset with more than 3500 trailers whose genres are known, and we make it publicly available for the interested reader. Second, we detail a novel classification method that encapsulates a CNN architecture to perform movie trailer genre classification, namely CNN-MoTion, and we compare it with state-of-the-art feature extraction techniques for movie classification such as Gist, CENTRIST, w-CENTRIST, and low-level feature extraction. Results show that our novel method significantly outperforms the current state-of-the-art approaches.

BACKGROUND: Hierarchical Multi-Label Classification is a classification task where the classes to be predicted are hierarchically organized. Each instance can be assigned to classes belonging to more than one path in the hierarchy. This scenario is typically found in protein function prediction, considering that each protein may perform many functions, which can be further specialized into sub-functions. We present a new hierarchical multi-label classification method based on multiple neural networks for the task of protein function prediction. A set of neural networks are incrementally training, each being responsible for the prediction of the classes belonging to a given level. RESULTS: The method proposed here is an extension of our previous work. Here we use the neural network output of a level to complement the feature vectors used as input to train the neural network in the next level. We experimentally compare this novel method with several other reduction strategies, showing that it obtains the best predictive performance. Empirical results also show that the proposed method achieves better or comparable predictive performance when compared with state-of-the-art methods for hierarchical multi-label classification in the context of protein function prediction. CONCLUSIONS: The experiments showed that using the output in one level as input to the next level contributed to better classification results. We believe the method was able to learn the relationships between the protein functions during training, and this information was useful for classification. We also identified in which functional classes our method performed better.

Molecular Dynamics simulations of protein receptors are an emergent tool in rational drug discovery. Nevertheless, employing Molecular Dynamics trajectories in virtual screening of large repositories is a very costly procedure, which ultimately may become unfeasible. Data clustering have been applied in this context with the goal of reducing the overall computational cost in order to make this task feasible. In this paper, we develop a novel estimation of distribution algorithm called Clus-EDA for clustering entire trajectories using structural features from the substrate-binding cavity of the protein receptor. This novel approach is capable of reducing the original trajectory to about 4% of its original size whilst keeping all relevant information for the analysis of receptor-ligand binding. The resulting partition generated by the estimation of distribution algorithm is further validated by analyzing the interactions between 20 ligands and a Fully-Flexible Receptor model containing a 20 ns Molecular Dynamics simulation trajectory. Results show that Clus-EDA is capable of outperforming traditional clustering algorithms such as k-means and hierarchical clustering by providing the smallest variance of the free energy of binding within the conformations in each cluster.

Multi-population genetic algorithms have been used with success for several multi-objective optimization problems. In this paper, we present a new general multi-population genetic algorithm for evolving decision trees. It was designed to improve the possibility of evolving balanced decision trees, simultaneously optimized for the predictions of each class. Single-population genetic algorithms namely tend to construct decision trees with great variance in single class accuracies. The proposed approach is tested over 10 UCI datasets, and it is compared with a single-population genetic algorithm as well as with traditional decision-tree induction algorithms. Results show that the designed multi-population approach provides classification results comparable to C4.5 and CART in terms of accuracy and tree size, while outperforming them regarding balanced solutions (in terms of average class accuracy and range of single-class accuracies).

Hierarchical Multi-label Classification (HMC) is a classification task where classes are organized in a hierarchical taxonomy, and instances can be simultaneously classified in more than one class. This paper investigates the HMC problem of classifying proteins in functions organized according to the Gene Ontology hierarchical taxonomy. This is a complex task, since the Gene Ontology hierarchy is organized as a Directed Acyclic Graph with thousands of classes hierarchically represented. We propose a neural network-based method to incorporate label-dependency during learning. The experimental results show that the proposed method achieves competitive results when compared to the state-of-the-art methods from the literature.

Top-down induction of decision trees (TDIDT) is a powerful method for data classification. A major issue in TDIDT is the decision on which attribute should be selected for dividing the nodes in subsets, creating the tree. For performing such a task, decision trees make use of a split criterion, which is usually an information-theory based measure. Apparently, there is no free-lunch regarding decision-tree split criteria, as is the case of most things in machine learning. Each application may benefit from a distinct split criterion, and the problem we pose here is how to identify the suitable split criterion for each possible application that may emerge. We propose in this paper a grammatical evolution algorithm for automatically generating split criteria through a context-free grammar. We name our new approach ESC-GE (Evolutionary Split Criteria with Grammatical Evolution). It is empirically evaluated on public gene expression datasets, and we compare its performance with state-of-the-art split criteria, namely the information gain and gain ratio. Results show that ESC-GE outperforms the baseline criteria in the domain of gene expression data, indicating its effectiveness for automatically designing tailor-made split criteria.

Decision-tree induction algorithms are widely used in machine learning applications in which the goal is to extract knowledge from data and present it in a graphically intuitive way. The most successful strategy for inducing decision trees is the greedy top-down recursive approach, which has been continuously improved by researchers over the past 40 years. In this paper, we propose a paradigm shift in the research of decision trees: instead of proposing a new manually designed method for inducing decision trees, we propose automatically designing decision-tree induction algorithms tailored to a specific type of classification data set (or application domain). Following recent breakthroughs in the automatic design of machine learning algorithms, we propose a hyper-heuristic evolutionary algorithm called hyper-heuristic evolutionary algorithm for designing decision-tree algorithms (HEAD-DT) that evolves design components of top-down decision-tree induction algorithms. By the end of the evolution, we expect HEAD-DT to generate a new and possibly better decision-tree algorithm for a given application domain. We perform extensive experiments in 35 real-world microarray gene expression data sets to assess the performance of HEAD-DT, and compare it with very well known decision-tree algorithms such as C4.5, CART, and REPTree. Results show that HEAD-DT is capable of generating algorithms that significantly outperform the baseline manually designed decision-tree algorithms regarding predictive accuracy and F-measure.

Hierarchical Multi-Label Classification (HMC) is a challenging task in data mining and machine learning. Each instance in HMC can be classified into two or more classes simultaneously. These classes are structured in a hierarchy, in the form of either a tree or a directed acyclic graph. Therefore, an instance can be assigned to two or more paths from the hierarchical structure, resulting in a complex classification problem with hundreds or thousands of classes. Several methods have been proposed to deal with such problems, including several algorithms based on well-known bio-inspired techniques, such as neural networks, ant colony optimization, and genetic algorithms. In this work, we propose a novel global method called GEHM, which makes use of grammatical evolution for generating HMC rules. In this approach, the grammatical evolution algorithm evolves the antecedents of classification rules, in order to assign instances from a HMC dataset to a probabilistic class vector. Our method is compared to bio-inspired HMC algorithms in protein function prediction datasets. The empirical analysis conducted in this work shows that GEHM outperforms the bio-inspired algorithms with statistical significance, which suggests that grammatical evolution is a promising alternative to deal with hierarchical multi-label classification of biological data.

Software effort estimation is an important task within software engineering. It is widely used for planning and monitoring software project development as a means to deliver the product on time and within budget. Several approaches for generating predictive models from collected metrics have been proposed throughout the years. Machine learning algorithms, in particular, have been widely-employed to this task, bearing in mind their capability of providing accurate predictive models for the analysis of project stakeholders. In this paper, we propose a grammatical evolution approach for software metrics estimation. Our novel algorithm, namely SEEGE, is empirically evaluated on public project data sets, and we compare its performance with state-of-the-art machine learning algorithms such as support vector machines for regression and artificial neural networks, and also to popular linear regression. Results show that SEEGE outperforms the other algorithms considering three different evaluation measures, clearly indicating its effectiveness for the effort estimation task.

This study reports the empirical analysis of a hyper-heuristic evolutionary algorithm that is capable of automatically designing top-down decision-tree induction algorithms. Top-down decision-tree algorithms are of great importance, considering their ability to provide an intuitive and accurate knowledge representation for classification problems. The automatic design of these algorithms seems timely, given the large literature accumulated over more than 40 years of research in the manual design of decision-tree induction algorithms. The proposed hyper-heuristic evolutionary algorithm, HEAD-DT, is extensively tested using 20 public UCI datasets and 10 microarray gene expression datasets. The algorithms automatically designed by HEAD-DT are compared with traditional decision-tree induction algorithms, such as C4.5 and CART. Experimental results show that HEAD-DT is capable of generating algorithms which are significantly more accurate than C4.5 and CART.

Decision tree induction is one of the most employed methods to extract knowledge from data, since the representation of knowledge is very intuitive and easily understandable by humans. The most successful strategy for inducing decision trees, the greedy top-down approach, has been continuously improved by researchers over the years. This work, following recent breakthroughs in the automatic design of machine learning algorithms, proposes a hyper-heuristic evolutionary algorithm for automatically generating decision-tree induction algorithms, named HEAD-DT. We perform extensive experiments in 20 public data sets to assess the performance of HEAD-DT, and we compare it to traditional decision-tree algorithms such as C4.5 and CART. Results show that HEAD-DT can generate algorithms that significantly outperform C4.5 and CART regarding predictive accuracy and F-Measure.

BACKGROUND: This paper addresses the prediction of the free energy of binding of a drug candidate with enzyme InhA associated with Mycobacterium tuberculosis. This problem is found within rational drug design, where interactions between drug candidates and target proteins are verified through molecular docking simulations. In this application, it is important not only to correctly predict the free energy of binding, but also to provide a comprehensible model that could be validated by a domain specialist. Decision-tree induction algorithms have been successfully used in drug-design related applications, specially considering that decision trees are simple to understand, interpret, and validate. There are several decision-tree induction algorithms available for general-use, but each one has a bias that makes it more suitable for a particular data distribution. In this article, we propose and investigate the automatic design of decision-tree induction algorithms tailored to particular drug-enzyme binding data sets. We investigate the performance of our new method for evaluating binding conformations of different drug candidates to InhA, and we analyze our findings with respect to decision tree accuracy, comprehensibility, and biological relevance. RESULTS: The empirical analysis indicates that our method is capable of automatically generating decision-tree induction algorithms that significantly outperform the traditional C4.5 algorithm with respect to both accuracy and comprehensibility. In addition, we provide the biological interpretation of the rules generated by our approach, reinforcing the importance of comprehensible predictive models in this particular bioinformatics application. CONCLUSIONS: We conclude that automatically designing a decision-tree algorithm tailored to molecular docking data is a promising alternative for the prediction of the free energy from the binding of a drug candidate with a flexible-receptor.

This paper presents a survey of evolutionary algorithms that are designed for decision-tree induction. In this context, most of the paper focuses on approaches that evolve decision trees as an alternate heuristics to the traditional top-down divide-and-conquer approach. Additionally, we present some alternative methods that make use of evolutionary algorithms to improve particular components of decision-tree classifiers. The paper's original contributions are the following. First, it provides an up-to-date overview that is fully focused on evolutionary algorithms and decision trees and does not concentrate on any specific evolutionary approach. Second, it provides a taxonomy, which addresses works that evolve decision trees and works that design decision-tree components by the use of evolutionary algorithms. Finally, a number of references are provided that describe applications of evolutionary algorithms for decision-tree induction in different domains. At the end of this paper, we address some important issues and open questions that can be the subject of future research.

Decision tree induction algorithms are well known techniques for assigning objects to predefined categories in a transparent fashion. Most decision tree induction algorithms rely on a greedy top-down recursive strategy for growing the tree, and pruning techniques to avoid overfitting. Even though such a strategy has been quite successful in many problems, it falls short in several others. For instance, there are cases in which the hyper-rectangular surfaces generated by these algorithms can only map the problem description after several sub-sequential partitions, which results in a large and incomprehensible tree. Hence, we propose a new decision tree induction algorithm based on clustering which seeks to provide more accurate models and/or shorter descriptions more comprehensible for the end-user. We do not base our performance analysis solely on the straightforward comparison of our proposed algorithm to baseline methods. Instead, we propose a data-dependent analysis in order to look for evidences which may explain in which situations our algorithm outperforms a well-known decision tree induction algorithm.

Decision tree induction algorithms are widely used in knowledge discovery and data mining, specially in scenarios where model comprehensibility is desired. A variation of the traditional univariate approach is the so-called oblique decision tree, which allows multivariate tests in its non-terminal nodes. Oblique decision trees can model decision boundaries that are oblique to the attribute axes, whereas univariate trees can only perform axis-parallel splits. The majority of the oblique and univariate decision tree induction algorithms perform a top-down strategy for growing the tree, relying on an impurity-based measure for splitting nodes. In this paper, we propose a novel bottom-up algorithm for inducing oblique trees named BUTIA. It does not require an impurity-measure for dividing nodes, since we know a priori the data resulting from each split. For generating the splitting hyperplanes, our algorithm implements a support vector machine solution, and a clustering algorithm is used for generating the initial leaves. We compare BUTIA to traditional univariate and oblique decision tree algorithms, C4.5, CART, OC1 and FT, as well as to a standard SVM implementation, using real gene expression benchmark data. Experimental results show the effectiveness of the proposed approach in several cases.

In Hierarchical Multi-Label Classification problems, each instance can be classified into two or more classes simultaneously, differently from conventional classification. Additionally, the classes are structured in a hierarchy, in the form of either a tree or a directed acyclic graph. Hence, an instance can be assigned to two or more paths from the hierarchical structure, resulting in a complex classification problem with possibly hundreds of classes. Many methods have been proposed to deal with such problems, some of them employing a single classifier to deal with all classes simultaneously (global methods), and others employing many classifiers to decompose the original problem into a set of subproblems (local methods). In this work, we propose a novel local method named HMC-LMLP, which uses one Multi-Layer Perceptron per hierarchical level. The predictions in one level are used as inputs to the network responsible for the predictions in the next level. We make use of two distinct Multi-Layer Perceptron algorithms: Back-propagation and Resilient Back-propagation. In addition, we make use of an error measure specially tailored to multi-label problems for training the networks. Our method is compared to state-of-the-art hierarchical multi-label classification algorithms, in protein function prediction datasets. The experimental results show that our approach presents competitive predictive accuracy, suggesting that artificial neural networks constitute a promising alternative to deal with hierarchical multi-label classification of biological data.

Software engineering metrics prediction has been a challenge for researchers throughout the years. Several approaches for deriving satisfactory predictive models from empirical data have been proposed, although none has been massively accepted due to the difficulty of building a generic solution applicable to a considerable number of different software projects. The most common strategy on estimating software metrics is the linear regression statistical technique, for its ease of use and availability in several statistical packages. Linear regression has numerous shortcomings though, which motivated the exploration of many techniques, such as data mining and other machine learning approaches. This paper reports different strategies on software metrics estimation, presenting a case study executed within a large worldwide IT company. Our contributions are the lessons learned during the preparation and execution of the experiments, in order to aid the state of the art on prediction models of software development projects.