Although Vision Language Models (VLMs) have shown generalization in medical imaging, pathology presents unique challenges due to ultra-high resolution, complex tissue structures, and nuanced semantics. These factors make pathology VLMs prone to hallucinations, i.e., generating outputs inconsistent with visual evidence, which undermines clinical trust. Existing RAG approaches in this domain largely depend on text-based knowledge bases, limiting their ability to leverage diagnostic visual cues. To address this, we propose Patho-AgenticRAG, a multimodal RAG framework with a database built on page-level embeddings from authoritative pathology textbooks. Unlike traditional text-only retrieval systems, it supports joint text–image search, enabling retrieval of textbook pages that contain both the queried text and relevant visual cues, thus avoiding the loss of critical image-based information. Patho-AgenticRAG also supports reasoning, task decomposition, and multi-turn search interactions, improving accuracy in complex diagnostic scenarios. Experiments show that Patho-AgenticRAG significantly outperforms existing multimodal models in complex pathology tasks like multiple-choice diagnosis and visual question answering.

Genetic programming-based feature construction has achieved significant success in recent years as an automated machine learning technique to enhance learning performance. However, overfitting remains a challenge that limits its broader applicability. To improve generalization, we prove that vicinal risk, estimated through noise perturbation or mixup-based data augmentation, is bounded by the sum of empirical risk and a regularization termb–either finite difference or the vicinal Jensen gap. Leveraging this decomposition, we propose an evolutionary feature construction framework that jointly optimizes empirical risk and the vicinal Jensen gap to control overfitting. Since datasets may vary in noise levels, we develop a noise estimation strategy to dynamically adjust regularization strength. Furthermore, to mitigate manifold intrusionb–where data augmentation may generate unrealistic samples that fall outside the data manifoldb–we propose a manifold intrusion detection mechanism. Experimental results on 58 datasets demonstrate the effectiveness of Jensen gap minimization compared to other complexity measures. Comparisons with 15 machine learning algorithms further indicate that genetic programming with the proposed overfitting control strategy achieves superior performance.

Genetic programming-based symbolic regression is a widely used machine learning technique, but its effectiveness can be limited as the number of input features increases. In genetic programming, two-stage feature selection has been extensively applied to enhance performance when dealing with a large number of input features. Existing two-stage feature selection methods typically require reinitializing new GP trees based on the selected features after feature selection, which disrupts the building blocks accumulated during evolution. In this paper, we propose a crossover operator that is aware of the selected features to leverage the feature selection results, thereby bypassing the need for reinitialization. This operator guides the crossover process to prioritize selected features, gradually eliminating unimportant features while preserving evolved building blocks. Experimental results validate the proposed method across three different feature-selection mechanisms on 98 datasets, demonstrating its effectiveness and broad applicability across various feature-selection strategies.

Symbolic regression is a key task in machine learning, aiming to discover mathematical expressions that best describe a dataset. While deep learning has increased interest in using neural networks for symbolic regression, many existing approaches rely on pre-trained models. These models require significant computational resources and struggle with regression tasks involving unseen functions and variables. A pre-training-free paradigm is needed to better integrate with search-based symbolic regression algorithms. To address these limitations, we propose a novel framework for symbolic regression that integrates evolutionary feature construction with a neural network, without the need for pre-training. Our approach adaptively generates symbolic trees that align with the desired semantics in real-time using a language model trained via online supervised learning, providing effective building blocks for feature construction. To mitigate hallucinations from the language model, we design a retrieval-augmented generation mechanism that explicitly leverages searched symbolic expressions. Additionally, we introduce a scale-invariant data augmentation technique that further improves the robustness and generalization of the model. Experimental results demonstrate that our framework achieves state-of-the-art accuracy across 25 regression algorithms and 120 regression tasks.

Symbolic regression searches for analytic expressions that accurately describe studied phenomena. The main promise of this approach is that it may return an interpretable model that can be insightful to users, while maintaining high accuracy. The current standard for benchmarking these algorithms is SRBench, which evaluates methods on hundreds of datasets that are a mix of real-world and simulated processes spanning multiple domains. At present, the ability of SRBench to evaluate interpretability is limited to measuring the size of expressions on real-world data, and the exactness of model forms on synthetic data. In practice, model size is only one of many factors used by subject experts to determine how interpretable a model truly is. Furthermore, SRBench does not characterize algorithm performance on specific, challenging sub-tasks of regression such as feature selection and evasion of local minima. In this work, we propose and evaluate an approach to benchmarking SR algorithms that addresses these limitations of SRBench by 1) incorporating expert evaluations of interpretability on a domain-specific task, and 2) evaluating algorithms over distinct properties of data science tasks. We evaluate 12 modern symbolic regression algorithms on these benchmarks and present an in-depth analysis of the results, discuss current challenges of symbolic regression algorithms and highlight possible improvements for the benchmark itself.

Evolutionary feature construction is a technique that has been widely studied in the domain of automated machine learning. A key challenge that needs to be addressed in feature construction is its tendency to overfit the training data. Instead of the traditional approach to control overfitting by reducing model complexity, this paper proposes to control overfitting based on bias-variance decomposition. Specifically, this paper proposes reducing the variance of a model, i.e., reducing the variance of predictions when exposed to data with injected noise, to improve its generalization performance within a multi-objective optimization framework. Experiments conducted on 42 datasets demonstrate that the proposed method effectively controls overfitting and outperforms six model complexity measures for overfitting control. Moreover, further analysis reveals that controlling overfitting adhering to bias-variance decomposition outperforms several plausible variants, highlighting the importance of controlling overfitting based on solid machine learning theory.

Evolutionary feature construction is a key technique in evolutionary machine learning, with the aim of constructing high-level features that enhance performance of a learning algorithm. In real-world applications, engineers typically construct complex features based on a combination of basic features, re-using those features as modules. However, modularity in evolutionary feature construction is still an open research topic. This paper tries to fill that gap by proposing a modular and hierarchical multitree genetic programming (GP) algorithm that allows trees to use the output values of other trees, thereby representing expressive features in a compact form. Based on this new representation, we propose a macro parent-repair strategy to reduce redundant and irrelevant features, a macro crossover operator to preserve interactive features, and an adaptive control strategy for crossover and mutation rates to dynamically balance the trade-off between exploration and exploitation. A comparison with seven bloat control methods on 98 regression datasets shows that the proposed modular representation achieves significantly better results in terms of test performance and smaller model size. Experimental results on the state-of-the-art symbolic regression benchmark demonstrate that the proposed symbolic regression method outperforms 22 existing symbolic regression and machine learning algorithms, providing empirical evidence for the superiority of the modularized evolutionary feature construction method.

In recent years, genetic programming has achieved impressive results on evolutionary feature construction tasks. To increase search effectiveness, researchers have developed many semantic-based crossover and mutation operators to guide genetic programming searches toward the target semantics. However, semantics has not yet been explored for the hoist mutation operator, which is an operator designed for controlling the bloat effect. Although the hoist mutation operator can significantly reduce model sizes, the most informative subtree may be disrupted by the randomness in mutation. To address this issue, we develop a semantic-based hoist mutation operator in this paper to preserve the most informative subtree that has the largest cosine similarity between its semantics and the target semantics. Experimental results on 98 regression datasets from the Penn Machine Learning Benchmark show that using this operator not only significantly reduces model size, but also improves the test accuracy of features constructed by genetic programming. A comparison with seven bloat control methods shows that the proposed operator achieves the best trade-off between accuracy and model size. Moreover, an experiment on the state-of-the-art symbolic regression benchmark shows that genetic programming with the semantic-based hoist mutation operator achieves the best test accuracy and competitive model sizes compared with 22 symbolic regression and machine learning algorithms.

Ensemble learning methods have been widely used in machine learning in recent years due to their high predictive performance. With the development of genetic programming-based symbolic regression methods, many papers begin to choose a popular ensemble learning method, random forests, as the baseline competitor. Instead of considering them as competitors, an alternative idea might be to consider symbolic regression as an enhancement technique for random forest. Genetic programming-based symbolic regression methods which fit a smooth function are complementary to the piecewise nature of decision trees, as the smooth variation is common in regression problems. In this article, we propose to form an ensemble model with symbolic regression-based decision trees to address this issue. Furthermore, we design a guided mutation operator to speed up the search on high-dimensional problems, a multi-fidelity evaluation strategy to reduce the computational cost and an ensemble selection mechanism to improve predictive performance. Finally, experimental results on a regression benchmark with 120 datasets show that the proposed ensemble model outperforms 25 existing symbolic regression and ensemble learning methods. Moreover, the proposed method can provide notable insights on an XGBoost hyperparameter performance prediction task, which is an important application area of ensemble learning methods.

Evolutionary feature construction is an important technique in the machine learning domain for enhancing learning performance. However, traditional genetic programming-based feature construction methods often suffer from bloat, which means the sizes of constructed features increase excessively without improved performance. To address this issue, this paper proposes a double-stage lexicase selection operator to control bloat while not damaging search effectiveness. This new operator contains a two-stage selection process, where the first stage selects individuals based on fitness values and the second stage selects individuals based on tree sizes. Therefore, the proposed operator can control bloat meanwhile leveraging the advantage of the lexicase selection operator. Experimental results on 98 regression datasets show that compared to the traditional bloat control method of having a depth limit, the proposed selection operator not only significantly reduces the sizes of constructed features on all datasets but also keeps a similar level of predictive performance. A comparative experiment with seven bloat control methods shows that the double lexicase selection operator achieves the best trade-off between the model performance and the model size.

Random forest (RF) is a type of ensemble-based machine learning method that has been applied to a variety of machine learning tasks in recent years. This article proposes an evolutionary approach to generate an oblique RF for regression problems. More specifically, our method induces an oblique RF by transforming the original feature space to a new feature space through the evolutionary feature construction method. To speed up the searching process, the proposed method evaluates each set of features based on a decision tree (DT) rather than an RF. In order to obtain an RF, we archive top-performing features and corresponding trees during the search. In this way, both the features and the forest can be constructed simultaneously in a single run. The proposed evolutionary forest is applied to 117 benchmark problems with different characteristics and compared with some state-of-the-art regression methods, including several variants of the RF and gradient boosted DTs (GBDTs). The experimental results suggest that the proposed method outperforms the existing RF and GBDT methods.

Symbolic regression has become a hot topic in recent years due to the surging demand for interpretable machine learning methods. Traditionally, symbolic regression problems are mainly solved by genetic algorithms. Nonetheless, with the development of deep learning, reinforcement learning based symbolic regression methods have received attention gradually. Unfortunately, hardly any of those reinforcement learning based methods have been proven effectively to solve real world regression problems as genetic algorithm based methods. In this paper, we find a general reinforcement learning based symbolic regression method is difficult to solve real world problems since it is hard to balance between exploration and exploitation. To deal with this problem, we propose a hybrid method to use both genetic algorithm and reinforcement learning for solving symbolic regression problems. By doing so, we can combine the advantages of reinforcement learning and genetic algorithm and achieve better performance than using them alone. To validate the effectiveness of the proposed method, we apply the proposed method to ten benchmark datasets. The experimental results show that the proposed method achieves competitive performance compared with several well-known symbolic regression methods on those datasets.

Evaluation metrics play an important role in accessing the performance of a regression method. In practice, these multiple evaluation metrics can be used in two ways. The first way defines a loss function by aggregating multiple metrics, while the second way defines a multiobjective loss function by considering each metric as an objective function. In this paper, we propose a new way to use multiple evaluation metrics, which is different from the aggregating method and the mutliobjective method. Our method is based on genetic programming. The idea is to randomly use one metric in each iteration of the selection operator. Therefore, multiple metrics can be used alternatively in the running process. To validate the effectiveness of our new approach, we conduct experiments on ten benchmark datasets. The experimental results show that the new approach can improve the population diversity, and can achieve the performance better than or similar to that of the traditional symbolic regression algorithms.