Abstract The incorporation of machine learning algorithms into high‐risk decision‐making tasks has raised some alarms in the scientific community. Research shows that machine learning‐based technologies can contain biases that cause unfair decisions for certain population groups. The fundamental danger of ignoring this problem is that machine learning methods can not only reflect the biases present in our society but could also amplify them. This article presents the design and validation of a technology to assist the fair automation of classification problems. In essence, the proposal is based on taking advantage of the intermediate solutions generated during the resolution of classification problems through using Auto‐ML tools, in particular, AutoGOAL, to create unbiased/fair classifiers. The technology employs a multi‐objective optimization search to find the collection of models with the best trade‐offs between performance and fairness. To solve the optimization problem, we introduce a combination of Probabilistic Grammatical Evolution Search and NSGA‐II. The technology was evaluated using the Adult dataset from the UCI repository, a common benchmark in related research. Results were compared with other published results in scenarios with single and multiple fairness definitions. Our experiments demonstrate the technology's ability to automate classification tasks while incorporating fairness constraints. Additionally, our method achieves competitive results against other bias mitigation techniques. A notable advantage of our approach is its minimal requirement for machine learning expertise, thanks to its Auto‐ML foundation. This makes the technology accessible and valuable for advancing fairness in machine learning applications. The source code is available online for the research community.

This paper presents AutoGOAL, a system for automatic machine learning (AutoML) that uses heterogeneous techniques.In contrast with existing AutoML approaches, our contribution can automatically build machine learning pipelines that combine techniques and algorithms from different frameworks, including shallow classifiers, natural language processing tools, and neural networks.We define the heterogeneous AutoML optimization problem as the search for the best sequence of algorithms that transforms specific input data into the desired output.This provides a novel theoretical and practical approach to AutoML.Our proposal is experimentally evaluated in diverse machine learning problems and compared with alternative approaches, showing that it is competitive with other AutoML alternatives in standard benchmarks.Furthermore, it can be applied to novel scenarios, such as several NLP tasks, where existing alternatives cannot be directly deployed.The system is freely available and includes in-built compatibility with a large number of popular machine learning frameworks, which makes our approach useful for solving practical problems with relative ease and effort.

The massive amount of biomedical information published online requires the development of automatic knowledge discovery technologies to effectively make use of this available content. To foster and support this, the research community creates linguistic resources, such as annotated corpora, and designs shared evaluation campaigns and academic competitive challenges. This work describes an ecosystem that facilitates research and development in knowledge discovery in the biomedical domain, specifically in Spanish language. To this end, several resources are developed and shared with the research community, including a novel semantic annotation model, an annotated corpus of 1045 sentences, and computational resources to build and evaluate automatic knowledge discovery techniques. Furthermore, a research task is defined with objective evaluation criteria, and an online evaluation environment is setup and maintained, enabling researchers interested in this task to obtain immediate feedback and compare their results with the state-of-the-art. As a case study, we analyze the results of a competitive challenge based on these resources and provide guidelines for future research. The constructed ecosystem provides an effective learning and evaluation environment to encourage research in knowledge discovery in Spanish biomedical documents.

The process of extracting knowledge from natural language text poses a complex problem that requires both a combination of machine learning techniques and proper feature selection.Recent advances in Automatic Machine Learning (AutoML) provide effective tools to explore large sets of algorithms, hyperparameters and features to find out the most suitable combination of them.This paper proposes a novel AutoML strategy based on probabilistic grammatical evolution, which is evaluated on the health domain by facing the knowledge discovery challenge in Spanish text documents.Our approach achieves state-ofthe-art results and provides interesting insights into the best combination of parameters and algorithms to use when dealing with this challenge.Source code is provided for the research community.