Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data.

Adam Prügel-Bennett

dblp:p/AdamPrugelBennett · DBLP profile ↗
← Back
57ranked-venue papers
7as first author
6since 2021 · last 2026
0000-0002-1329-5077ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Artificial intelligence and machine learning · 45 · 6 first-author · 6 since 2021Applied, interdisciplinary, general and emerging computing · 5Graphics, computer vision, multimedia, augmented reality and games · 4 · 1 since 2021Databases, data management, data science and information retrieval · 3Theory of computation · 3 · 1 first-authorComputer networks · 1

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Artificial intelligence
11 papers
Image recognition and object detection · 19% Trustworthy machine learning · 19% Vision and language · 16%
Theoretical computer science
2 papers
Mathematical optimization · 100%

Topics — the 30 heaviest of 35, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Machine learning › Trustworthy machine learning › AI-generated content detection
AI-generated image detection
1.012026
Penny-Wise and Pound-Foolish in AI-Generated Image Detection · IEEE Trans. Pattern Anal. Mach. Intell. 2026
Machine learning › Learning theory
generalization
1.012026
Penny-Wise and Pound-Foolish in AI-Generated Image Detection · IEEE Trans. Pattern Anal. Mach. Intell. 2026
Computer vision › Vision and language › vision-language model
prompt learning
1.012026
Penny-Wise and Pound-Foolish in AI-Generated Image Detection · IEEE Trans. Pattern Anal. Mach. Intell. 2026
Computer vision › Vision and language
vision-language model
1.012026
Penny-Wise and Pound-Foolish in AI-Generated Image Detection · IEEE Trans. Pattern Anal. Mach. Intell. 2026
Machine learning › Representation and self-supervised learning › representation learning › structured representation learning
set representation learning
0.822020
FSPool: Learning Set Representations with Featurewise Sort Pooling · ICLR 2020
Learning Representations of Sets through Optimized Permutations · ICLR (Poster) 2019
Machine learning › Deep learning architectures and training
recurrent neural network
0.812024
Rethinking Deep Thinking: Stable Learning of Algorithms using Lipschitz Constraints · NeurIPS 2024
Mathematical optimization
combinatorial optimization
0.812024
Rethinking Deep Thinking: Stable Learning of Algorithms using Lipschitz Constraints · NeurIPS 2024
Mathematical optimization › combinatorial optimization › vehicle routing
traveling salesman problem
0.812024
Rethinking Deep Thinking: Stable Learning of Algorithms using Lipschitz Constraints · NeurIPS 2024
Machine learning › Efficient and distributed learning › active learning
annotation cost reduction
0.712023
Guiding Labelling Effort for Efficient Learning With Georeferenced Images · IEEE Trans. Pattern Anal. Mach. Intell. 2023
Machine learning › Learning paradigms
semi-supervised learning
0.712023
Guiding Labelling Effort for Efficient Learning With Georeferenced Images · IEEE Trans. Pattern Anal. Mach. Intell. 2023
Machine learning › Trustworthy machine learning
interpretability
0.612022
On the Effects of Artificial Data Modification · ICML 2022
Machine learning › Trustworthy machine learning › robustness › corruption robustness
occlusion robustness
0.612022
On the Effects of Artificial Data Modification · ICML 2022
Machine learning › Trustworthy machine learning
robustness evaluation
0.612022
On the Effects of Artificial Data Modification · ICML 2022
Computer vision › Image recognition and object detection
shape-texture bias
0.612022
On the Effects of Artificial Data Modification · ICML 2022
Machine learning › Representation and self-supervised learning › representation learning
disentangled representation learning
0.412020
Linear Disentangled Representations and Unsupervised Action Estimation · NeurIPS 2020
Machine learning › Deep learning architectures and training › neural network layer design
pooling
0.412020
FSPool: Learning Set Representations with Featurewise Sort Pooling · ICLR 2020
Machine learning › Representation and self-supervised learning › symmetry learning
symmetry-aware representation learning
0.412020
Linear Disentangled Representations and Unsupervised Action Estimation · NeurIPS 2020
Computer vision › Image recognition and object detection
object detection
0.412019
Deep Set Prediction Networks · NeurIPS 2019
Computer vision › 3D vision › geometric deep learning › set learning
set prediction
0.412019
Deep Set Prediction Networks · NeurIPS 2019
Mathematical optimization › combinatorial optimization › permutation problems
permutation optimization
0.412019
Learning Representations of Sets through Optimized Permutations · ICLR (Poster) 2019
Computer vision › Image recognition and object detection
object counting
0.312018
Learning to Count Objects in Natural Images for Visual Question Answering · ICLR (Poster) 2018
Computer vision › Vision and language
visual question answering
0.312018
Learning to Count Objects in Natural Images for Visual Question Answering · ICLR (Poster) 2018
Computer vision › Image recognition and object detection › image classification
object classification
0.312026
Penny-Wise and Pound-Foolish in AI-Generated Image Detection · IEEE Trans. Pattern Anal. Mach. Intell. 2026
Machine learning › Generative modeling
variational autoencoder
0.112020
Linear Disentangled Representations and Unsupervised Action Estimation · NeurIPS 2020
Machine learning › Probabilistic and Bayesian machine learning › statistical inference
bayesian inference
0.112006
A Bayesian Framework for Extracting Human Gait Using Strong Prior Knowledge · IEEE Trans. Pattern Anal. Mach. Intell. 2006
Computer vision › Face, body and person analysis
human pose estimation
0.112006
A Bayesian Framework for Extracting Human Gait Using Strong Prior Knowledge · IEEE Trans. Pattern Anal. Mach. Intell. 2006
Bioinformatics and computational biology
sequence analysis
0.012004
Training HMM structure with genetic algorithm for biological sequence analysis · Bioinform. 2004
Computer vision › Video understanding and tracking
motion tracking
0.012006
A Bayesian Framework for Extracting Human Gait Using Strong Prior Knowledge · IEEE Trans. Pattern Anal. Mach. Intell. 2006
Machine learning › Representation and self-supervised learning
hebbian learning
0.011993
Non-Linear Statistical Analysis and Self-Organizing Hebbian Networks · NIPS 1993
Machine learning › Representation and self-supervised learning › representation learning › unsupervised representation learning
self-organizing networks
0.011993
Non-Linear Statistical Analysis and Self-Organizing Hebbian Networks · NIPS 1993

Methods — techniques the papers use, named apart from their topics

recurrent computation · 1.5lipschitz constraint · 1.5learnable prompt design · 1.0fine-tuning · 1.0balanced objective · 1.0convolutional neural network · 0.7autoencoder · 0.7active learning · 0.7mixup · 0.6cutmix · 0.6permutation learning · 0.4genetic algorithm · 0.0baum-welch training · 0.0
YearPublicationVenuePosition
2026 Penny-Wise and Pound-Foolish in AI-Generated Image Detection
abstract
The rise of AI-generated images has sparked serious concerns about their potential misuse across various domains, prompting the urgent need for robust detection methods. Despite advancements, many current approaches prioritize short-term gains at the expense of long-term effectiveness. This paper critiques the overly specialized approach of fine-tuning pre-trained models for short-term gains on a single AI image dataset, while disregarding the long-term imperative of achieving generalization and knowledge retention. To address this trade-off issue, we propose a novel learning framework (PoundNet) for the generalization of AI-generated image detection on a pre-trained vision-language model. PoundNet incorporates a learnable prompt design and a balanced objective to preserve broad knowledge from upstream tasks (object classification) while enhancing generalization for downstream tasks (AI-generated image detection). We train PoundNet on a single standard AI image dataset, following common practice in the literature. We then evaluate its performance across 10 large-scale public AI-generated image detection datasets with 5 main evaluation metrics, forming the largest benchmark test set for assessing the generalization ability of AI-generated image detection models, to our knowledge. The comprehensive benchmark evaluation demonstrates that PoundNet successfully balances generalization with knowledge retention, achieving a remarkable relative improvement of 19% in AI-generated image detection performance compared to state-of-the-art methods, while maintaining a strong performance of 63% on object classification tasks.
Yabin Wang 0001, Zhiwu Huang, Zhou Su 0001, Adam Prügel-Bennett, Xiaopeng Hong
IEEE Trans. Pattern Anal. Mach. Intell.4
2025 A variational autoencoder for probabilistic non-negative matrix factorisation
abstract
Abstract We introduce and demonstrate the variational autoencoder (VAE) for probabilistic non-negative matrix factorisation (PAE-NMF). We design a network which can perform non-negative matrix factorisation (NMF) and add in aspects of a VAE to make the coefficients of the latent space probabilistic. By restricting the weights in the final layer of the network to be non-negative and using the non-negative Weibull distribution we produce a probabilistic form of NMF which allows us to generate new data and find a probability distribution that effectively links the latent and input variables. Our approach uses a minimum description length methodology to provide a method for achieving automatic regularisation; as it is designed using neural networks it can leverage deep learning frameworks for automatic differentiation, fast gradient descent algorithms and GPU support. We demonstrate the effectiveness of PAE-NMF on three heterogeneous datasets: images, financial time series and genomic.
Steven Squires, Adam Prügel-Bennett, Mahesan Niranjan
Pattern Anal. Appl.2
2024 Detect Closer Surfaces That Can be Seen: New Modeling and Evaluation in Cross-Domain 3D Object Detection
abstract
The performance of domain adaptation technologies has not yet reached an ideal level in the current 3D object detection field for autonomous driving, which is mainly due to significant differences in the size of vehicles, as well as the environments they operate in when applied across domains. These factors together hinder the effective transfer and application of knowledge learned from specific datasets. Since the existing evaluation metrics are initially designed for evaluation on a single domain by calculating the 2D or 3D overlap between the prediction and ground-truth bounding boxes, they often suffer from the overfitting problem caused by the size differences among datasets. This raises a fundamental question related to the evaluation of the 3D object detection models’ cross-domain performance: Do we really need models to maintain excellent performance in their original 3D bounding boxes after being applied across domains? From a practical application perspective, one of our main focuses is actually on preventing collisions between vehicles and other obstacles, especially in cross-domain scenarios where correctly predicting the size of vehicles is much more difficult. In other words, as long as a model can accurately identify the closest surfaces to the ego vehicle, it is sufficient to effectively avoid obstacles. In this paper, we propose two metrics to measure 3D object detection models’ ability of detecting the closer surfaces to the sensor on the ego vehicle, which can be used to evaluate their cross-domain performance more comprehensively and reasonably. Furthermore, we propose a refinement head, named EdgeHead, to guide models to focus more on the learnable closer surfaces, which can greatly improve the cross-domain performance of existing models not only under our new metrics, but even also under the original BEV/3D metrics. Our code is available at https://github.com/Galaxy-ZRX/EdgeHead.
Ruixiao Zhang 0001, Yihong Wu 0004, Juheon Lee, Xiaohao Cai, Adam Prügel-Bennett
ECAI5
2024 Rethinking Deep Thinking: Stable Learning of Algorithms using Lipschitz Constraints
abstract
Iterative algorithms solve problems by taking steps until a solution is reached. Models in the form of Deep Thinking (DT) networks have been demonstrated to learn iterative algorithms in a way that can scale to different sized problems at inference time using recurrent computation and convolutions. However, they are often unstable during training, and have no guarantees of convergence/termination at the solution. This paper addresses the problem of instability by analyzing the growth in intermediate representations, allowing us to build models (referred to as Deep Thinking with Lipschitz Constraints (DT-L)) with many fewer parameters and providing more reliable solutions. Additionally our DT-L formulation provides guarantees of convergence of the learned iterative procedure to a unique solution at inference time. We demonstrate DT-L is capable of robustly learning algorithms which extrapolate to harder problems than in the training set. We benchmark on the traveling salesperson problem to evaluate the capabilities of the modified system in an NP-hard problem where DT fails to learn.
Jay Bear, Adam Prügel-Bennett, Jonathon Hare
NeurIPS2
2023 Guiding Labelling Effort for Efficient Learning With Georeferenced Images
abstract
We describe a novel semi-supervised learning method that reduces the labelling effort needed to train convolutional neural networks (CNNs) when processing georeferenced imagery. This allows deep learning CNNs to be trained on a per-dataset basis, which is useful in domains where there is limited learning transferability across datasets. The method identifies representative subsets of images from an unlabelled dataset based on the latent representation of a location guided autoencoder. We assess the method's sensitivities to design options using four different ground-truthed datasets of georeferenced environmental monitoring images, where these include various scenes in aerial and seafloor imagery. Efficiency gains are achieved for all the aerial and seafloor image datasets analysed in our experiments, demonstrating the benefit of the method across application domains. Compared to CNNs of the same architecture trained using conventional transfer and active learning, the method achieves equivalent accuracy with an order of magnitude fewer annotations, and 85 % of the accuracy of CNNs trained conventionally with approximately 10,000 human annotations using just 40 prioritised annotations. The biggest gains in efficiency are seen in datasets with unbalanced class distributions and rare classes that have a relatively small number of observations.
Takaki Yamada, Miquel Massot-Campos, Adam Prügel-Bennett, Oscar Pizarro, Stefan B. Williams, Blair Thornton
IEEE Trans. Pattern Anal. Mach. Intell.3
2022 On the Effects of Artificial Data Modification
abstract
Data distortion is commonly applied in vision models during both training (e.g methods like MixUp and CutMix) and evaluation (e.g. shape-texture bias and robustness). This data modification can introduce artificial information. It is often assumed that the resulting artefacts are detrimental to training, whilst being negligible when analysing models. We investigate these assumptions and conclude that in some cases they are unfounded and lead to incorrect results. Specifically, we show current shape bias identification methods and occlusion robustness measures are biased and propose a fairer alternative for the latter. Subsequently, through a series of experiments we seek to correct and strengthen the community’s perception of how augmenting affects learning of vision models. Based on our empirical results we argue that the impact of the artefacts must be understood and exploited rather than eliminated.
Antonia Marcu, Adam Prügel-Bennett
ICML2
2020 FSPool: Learning Set Representations with Featurewise Sort Pooling
Yan Zhang 0067, Jonathon S. Hare, Adam Prügel-Bennett
ICLR3
2020 Linear Disentangled Representations and Unsupervised Action Estimation
abstract
Disentangled representation learning has seen a surge in interest over recent times, generally focusing on new models which optimise one of many disparate disentanglement metrics. Symmetry Based Disentangled Representation learning introduced a robust mathematical framework that defined precisely what is meant by a ``linear disentangled representation''. This framework determined that such representations would depend on a particular decomposition of the symmetry group acting on the data, showing that actions would manifest through irreducible group representations acting on independent representational subspaces. \citet{forwardvae} subsequently proposed the first model to induce and demonstrate a linear disentangled representation in a VAE model. In this work we empirically show that linear disentangled representations are not present in standard VAE models and that they instead require altering the loss landscape to induce them. We proceed to show that such representations are a desirable property with regard to classical disentanglement metrics. Finally we propose a method to induce irreducible representations which forgoes the need for labelled action sequences, as was required by prior work. We explore a number of properties of this method, including the ability to learn from action sequences without knowledge of intermediate states and robustness under visual noise. We also demonstrate that it can successfully learn 4 independent symmetries directly from pixels.
Matthew Painter, Adam Prügel-Bennett, Jonathon S. Hare
NeurIPS2
2019 Saliency Map on Cnns for Protein Secondary Structure Prediction
abstract
Deep learning, a powerful methodology for data-driven modelling, has been shown to be useful in tackling several problems in the biomedical domain. However, deep neural architectures lack interpretability of how predictions from them are made on any test input. While several approaches to "opening the black box" are being developed, their application to biological and medical data is very much as its infancy. Here, we consider the specific problem of protein secondary structure prediction using the techniques of saliency maps to explain decisions of a deep neural network. The analysis leads to two important observations: (a) one-hot-encoded amino-acids are irrelevant in the presence of PSSM values as extra features; and (b) in predicting α-helices at any position, amino-acids to the right are far more important than those to the left. The latter observation may have a biological basis relating to the synthesis of proteins by ribosome movement from left to right, sequentially adding amino-acids.
Guillermo Romero Moreno, Mahesan Niranjan, Adam Prügel-Bennett
ICASSP3
2019 Learning Representations of Sets through Optimized Permutations
Yan Zhang 0067, Jonathon S. Hare, Adam Prügel-Bennett
ICLR (Poster)3
2019 Deep Set Prediction Networks
abstract
Current approaches for predicting sets from feature vectors ignore the unordered nature of sets and suffer from discontinuity issues as a result. We propose a general model for predicting sets that properly respects the structure of sets and avoids this problem. With a single feature vector as input, we show that our model is able to auto-encode point sets, predict the set of bounding boxes of objects in an image, and predict the set of attributes of these objects.
Yan Zhang 0067, Jonathon S. Hare, Adam Prügel-Bennett
NeurIPS3
2018 Learning to Count Objects in Natural Images for Visual Question Answering
Yan Zhang 0067, Jonathon S. Hare, Adam Prügel-Bennett
ICLR (Poster)3
2017 A Method of Integrating Spatial Proteomics and Protein-Protein Interaction Network Data
Steven Squires, Rob M. Ewing, Adam Prügel-Bennett, Mahesan Niranjan
ICONIP (5)3
2017 Non-Negative Matrix Factorization with Exogenous Inputs for Modeling Financial Data
Steven Squires, Luis Montesdeoca, Adam Prügel-Bennett, Mahesan Niranjan
ICONIP (2)3
2017 Rank Selection in Nonnegative Matrix Factorization using Minimum Description Length
abstract
Nonnegative matrix factorization (NMF) is primarily a linear dimensionality reduction technique that factorizes a nonnegative data matrix into two smaller nonnegative matrices: one that represents the basis of the new subspace and the second that holds the coefficients of all the data points in that new space. In principle, the nonnegativity constraint forces the representation to be sparse and parts based. Instead of extracting holistic features from the data, real parts are extracted that should be significantly easier to interpret and analyze. The size of the new subspace selects how many features will be extracted from the data. An effective choice should minimize the noise while extracting the key features. We propose a mechanism for selecting the subspace size by using a minimum description length technique. We demonstrate that our technique provides plausible estimates for real data as well as accurately predicting the known size of synthetic data. We provide an implementation of our code in a Matlab format.
Steven Squires, Adam Prügel-Bennett, Mahesan Niranjan
Neural Comput.2
2016 Improving the performance of evolutionary engine calibration algorithms with principal component analysis
abstract
By studying the fitness landscape properties of engine calibration problem we propose a new Principal Component Analysis (PCA) based optimisation algorithm for the problem. The engine calibration problem in this paper is to minimise the fuel consumption, gas emission and particle emission of a Jaguar car engine. To evaluate the fuel consumption and emissions of the engine, a model of the engine that was developed in University of Birmingham was used. A strength Pareto method is used to convert the three objectives into one fitness value. Then a local search algorithm is used to find local optima. We then study these local optima to find the properties of good solutions in the landscape. Our studies on the good solutions show that the best solutions in the landscape show some patterns. We perform Principal Component Analysis (PCA) on the good solutions and show that these components present certain properties, which can be exploited to develop new exploration operators for evolutionary algorithms. We use the newly proposed operator on some well-known algorithms and show that the performance of the algorithms can be improved significantly.
Mohammad-Hassan Tayarani-Najaran, Adam Prügel-Bennett, Hongming Xu 0001, Xin Yao 0001
CEC2
2016 An Analysis of the Fitness Landscape of Travelling Salesman Problem
abstract
The fitness landscape of the travelling salesman problem is investigated for 11 different types of the problem. The types differ in how the distances between cities are generated. Many different properties of the landscape are studied. The properties chosen are all potentially relevant to choosing an appropriate search algorithm. The analysis includes a scaling study of the time to reach a local optimum, the number of local optima, the expected probability of reaching a local optimum as a function of its fitness, the expected fitness found by local search and the best fitness, the probability of reaching a global optimum, the distance between the local optima and the global optimum, the expected fitness as a function of the distance from an optimum, their basins of attraction and a principal component analysis of the local optima. The principal component analysis shows the correlation of the local optima in the component space. We show how the properties of the principal components of the local optima change from one problem type to another.
Mohammad-Hassan Tayarani-Najaran, Adam Prügel-Bennett
Evol. Comput.2
2016 A Low Dimensional Approximation For Competence In Bacillus Subtilis
abstract
The behaviour of a high dimensional stochastic system described by a chemical master equation (CME) depends on many parameters, rendering explicit simulation an inefficient method for exploring the properties of such models. Capturing their behaviour by low-dimensional models makes analysis of system behaviour tractable. In this paper, we present low dimensional models for the noise-induced excitable dynamics in Bacillus subtilis, whereby a key protein ComK, which drives a complex chain of reactions leading to bacterial competence, gets expressed rapidly in large quantities (competent state) before subsiding to low levels of expression (vegetative state). These rapid reactions suggest the application of an adiabatic approximation of the dynamics of the regulatory model that, however, lead to competence durations that are incorrect by a factor of 2. We apply a modified version of an iterative functional procedure that faithfully approximates the time-course of the trajectories in terms of a two-dimensional model involving proteins ComK and ComS. Furthermore, in order to describe the bimodal bivariate marginal probability distribution obtained from the Gillespie simulations of the CME, we introduce a tunable multiplicative noise term in a two-dimensional Langevin model whose stationary state is described by the time-independent solution of the corresponding Fokker-Planck equation.
An Nguyen 0003, Adam Prügel-Bennett, Srinandan Dasmahapatra
IEEE ACM Trans. Comput. Biol. Bioinform.2
2015 Run-Time Analysis of Population-Based Evolutionary Algorithm in Noisy Environments
abstract
This paper analyses a generational evolutionary algorithm using only selection and uniform crossover. With a probability arbitrarily close to one the evolutionary algorithm is shown to solve onemax in O(n log2(n)) function evaluations using a population of size c,n, log(n). We then show that this algorithm can solve onemax with noise variance n again in O(n log2(n)) function evaluations.
Adam Prügel-Bennett, Jonathan E. Rowe, Jonathan L. Shapiro
FOGA1
2015 Ising Bandits with Side Information
Shaona Ghosh, Adam Prügel-Bennett
ECML/PKDD (1)2
2015 Novel centroid selection approaches for KMeans-clustering based recommender systems
Sobia Zahra, Mustansar Ali Ghazanfar, Asra Khalid, Muhammad Awais Azam, Usman Naeem, Adam Prügel-Bennett
Inf. Sci.6
2014 Constructing smart portfolios from data driven quantitative investment models
abstract
In this paper we present a smart portfolio management methodology, which advances existing portfolio management techniques at two distinct levels. First, we develop a set of investment models that target regimes found in the data over different time horizons. We then build a meta-model which uses the Kelly criterion to determine an optimal allocation over these investment strategies, thus simultaneously capturing regimes operating in the data over different time horizons. Finally, in order to detect changes in the relevant data regime, and hence investment allocations, we use a forecasting algorithm which relies on a Kalman filter. We call our combined method, that uses both the Kelly criterion and the Kalman filter, the K2 algorithm. Using a large-scale historical dataset of both stocks and indices, we show that our K2 algorithm gives better risk adjusted returns in terms of the Sharpe ratio, better average gain to average loss ratio and higher probability of success compared to existing benchmarks, when measured in out-of-sample tests.
Chetan Saran Mehra, Adam Prügel-Bennett, Enrico H. Gerding, Valentin Robu
CIFEr2
2014 Leveraging clustering approaches to solve the gray-sheep users problem in recommender systems
Mustansar Ali Ghazanfar, Adam Prügel-Bennett
Expert Syst. Appl.2
2014 On the Landscape of Combinatorial Optimization Problems
abstract
This paper carries out a comparison of the fitness landscape for four classic optimization problems: Max-Sat, graph-coloring, traveling salesman, and quadratic assignment. We have focused on two types of properties, local average properties of the landscape, and properties of the local optima. For the local optima we give a fairly comprehensive description of the properties, including the expected time to reach a local optimum, the number of local optima at different cost levels, the distance between optima, and the expected probability of reaching the optima. Principle component analysis is used to understand the correlations between the local optima. Most of the properties that we examine have not been studied previously, particularly those concerned with properties of the local optima. We compare and contrast the behavior of the four different problems. Although the problems are very different at the low level, many of the long-range properties exhibit a remarkable degree of similarity.
Mohammad-Hassan Tayarani-Najaran, Adam Prügel-Bennett
IEEE Trans. Evol. Comput.2
2013 Evolving Fisher Kernels for Biological Sequence Classification
abstract
Fisher kernels have been successfully applied to many problems in bioinformatics. However, their success depends on the quality of the generative model upon which they are built. For Fisher kernel techniques to be used on novel problems, a mechanism for creating accurate generative models is required. A novel framework is presented for automatically creating domain-specific generative models that can be used to produce Fisher kernels for support vector machines (SVMs) and other kernel methods. The framework enables the capture of prior knowledge and addresses the issue of domain-specific kernels, both of which are current areas that are lacking in many kernel-based methods. To obtain the generative model, genetic algorithms are used to evolve the structure of hidden Markov models (HMMs). A Fisher kernel is subsequently created from the HMM, and used in conjunction with an SVM, to improve the discriminative power. This paper investigates the effectiveness of the proposed method, named GA-SVM. We show that its performance is comparable if not better than other state of the art methods in classifying secretory protein sequences of malaria. More interestingly, it showed better results than the sequence-similarity-based approach, without the need for additional homologous sequence information in protein enzyme family classification. The experiments clearly demonstrate that the GA-SVM is a novel way to find features with good performance from biological sequences, that does not require extensive tuning of a complex model.
Kyoung-Jae Won, Craig Saunders, Adam Prügel-Bennett
Evol. Comput.3
2012 Kernel-Mapping Recommender system algorithms
Mustansar Ali Ghazanfar, Adam Prügel-Bennett, Sándor Szedmák
Inf. Sci.2
2012 Maximum Satisfiability: Anatomy of the Fitness Landscape for a Hard Combinatorial Optimization Problem
abstract
The fitness landscape of MAX-3-SAT is investigated for random instances above the satisfiability phase transition. This paper includes a scaling analysis of the time to reach a local optimum, the number of local optima, the expected probability of reaching a local optimum as a function of its fitness, the expected fitness found by local search and the best fitness, the probability of reaching a global optimum, the size and relative positions of the global optima, the mean distance between the local and global optima, the expected fitness as a function of the Hamming distance from an optimum and their basins of attraction. These analyses show why the problem becomes hard for local search algorithms as the system size increases. The paper also shows how a recently proposed algorithm can exploit long-range correlations in the fitness landscape to improve on the state-of-the-art heuristic algorithms.
Adam Prügel-Bennett, Mohammad-Hassan Tayarani-Najaran
IEEE Trans. Evol. Comput.1
2011 Incremental Kernel Mapping Algorithms for Scalable Recommender Systems
abstract
Recommender systems apply machine learning techniques for filtering unseen information and can predict whether a user would like a given item. Kernel Mapping Recommender (KMR) system algorithms have been proposed, which offer state-of-the-art performance. One potential drawback of the KMR algorithms is that the training is done in one step and hence they cannot accommodate the incremental update with the arrival of new data making them unsuitable for the dynamic environments. From this line of research, we propose a new heuristic, which can build the model incrementally without retraining the whole model from scratch when new data (item or user) are added to the recommender system dataset. Furthermore, we proposed a novel perceptron-type algorithm, which is a fast incremental algorithm for building the model that maintains a good level of accuracy and scales well with the data. We show empirically over two datasets that the proposed algorithms give quite accurate results while providing significant computation savings.
Mustansar Ali Ghazanfar, Sándor Szedmák, Adam Prügel-Bennett
ICTAI3
2010 Benefits of a Population: Five Mechanisms That Advantage Population-Based Algorithms
abstract
This paper identifies five distinct mechanisms by which a population-based algorithm might have an advantage over a solo-search algorithm in classical optimization. These mechanisms are illustrated through a number of toy problems. Simulations are presented comparing different search algorithms on these problems. The plausibility of these mechanisms occurring in classical optimization problems is discussed. The first mechanism we consider relies on putting together building blocks from different solutions. This is extended to include problems containing critical variables. The second mechanism is the result of focusing of the search caused by crossover. Also discussed in this context is strong focusing produced by averaging many solutions. The next mechanism to be examined is the ability of a population to act as a low-pass filter of the landscape, ignoring local distractions. The fourth mechanism is a population's ability to search different parts of the fitness landscape, thus hedging against bad luck in the initial position or the decisions it makes. The final mechanism is the opportunity of learning useful parameter values to balance exploration against exploitation.
Adam Prügel-Bennett
IEEE Trans. Evol. Comput.1
2010 Learning the Large-Scale Structure of the MAX-SAT Landscape Using Populations
abstract
A new algorithm for solving maximum satisfiability (MAX-SAT) problems is introduced which clusters good solutions, and restarts the search from the closest feasible solution to the centroid of each cluster. This is shown to be highly efficient for finding good solutions of large MAX-SAT problems. We argue that this success is due to the population learning the large-scale structure of the fitness landscape. Systematic studies of the landscape are presented to support this hypothesis. In addition, a number of other strategies are tested to rule out other possible explanations of the success. Preliminary results are shown, indicating that extensions of the proposed algorithm can give similar improvements on other hard optimization problems.
Mohamed Qasem, Adam Prügel-Bennett
IEEE Trans. Evol. Comput.2
2009 Improving Performance in Combinatorial Optimisation Using Averaging and Clustering
Mohamed Qasem, Adam Prügel-Bennett
EvoCOP2
2008 Complexity of Max-SAT using stochastic algorithms
abstract
Hill-climbing has been shown to be more effective than exhaustive search in solving satisfiability problems. Also, it has been used either by itself or in combination with other methods to solve the most difficult region of SAT, the phase transition. We show that hill-climbing also finds SAT problems difficult around the phase transition. It too follows an easy-hard-eays transition.
Mohamed Qasem, Adam Prügel-Bennett
GECCO2
2007 Finding critical backbone structures with genetic algorithms
abstract
This paper introduces the concept of a critical backbone as a minimal set of variable or part of the solution necessary to be within the basin of attraction of the global optimum. The concept is illustrated with a new class of test problems Backbone in which the critical backbone structure is completely transparent. The performance of a number of standard heuristic search methods is measure for this problem. It is shown that a hybrid genetic algorithm that incorporates a descent algorithm solves this problem extremely efficiently. Although no rigorous analysis is given the problem is sufficiently transparent that this result is easy to understand. The paper concludes with a discussion of how the emergence of a critical backbone may be the salient feature in a phase transition from typically easy to typically hard problems.
Adam Prügel-Bennett
GECCO1
2007 An evolutionary method for learning HMM structure: prediction of protein secondary structure
abstract
BACKGROUND: The prediction of the secondary structure of proteins is one of the most studied problems in bioinformatics. Despite their success in many problems of biological sequence analysis, Hidden Markov Models (HMMs) have not been used much for this problem, as the complexity of the task makes manual design of HMMs difficult. Therefore, we have developed a method for evolving the structure of HMMs automatically, using Genetic Algorithms (GAs). RESULTS: In the GA procedure, populations of HMMs are assembled from biologically meaningful building blocks. Mutation and crossover operators were designed to explore the space of such Block-HMMs. After each step of the GA, the standard HMM estimation algorithm (the Baum-Welch algorithm) was used to update model parameters. The final HMM captures several features of protein sequence and structure, with its own HMM grammar. In contrast to neural network based predictors, the evolved HMM also calculates the probabilities associated with the predictions. We carefully examined the performance of the HMM based predictor, both under the multiple- and single-sequence condition. CONCLUSION: We have shown that the proposed evolutionary method can automatically design the topology of HMMs. The method reads the grammar of protein sequences and converts it into the grammar of an HMM. It improved previously suggested evolutionary methods and increased the prediction quality. Especially, it shows good performance under the single-sequence condition and provides probabilistic information on the prediction result. The protein secondary structure predictor using HMMs (P.S.HMM) is on-line available http://www.binf.ku.dk/~won/pshmm.htm. It runs under the single-sequence condition.
Kyoung-Jae Won, Thomas Hamelryck, Adam Prügel-Bennett, Anders Krogh
BMC Bioinform.3
2007 Optimal parameters for search using a barrier tree Markov model
William Benfold, Jonathan Hallam, Adam Prügel-Bennett
Theor. Comput. Sci.3
2006 A Bayesian Framework for Extracting Human Gait Using Strong Prior Knowledge
abstract
Extracting full-body motion of walking people from monocular video sequences in complex, real-world environments is an important and difficult problem, going beyond simple tracking, whose satisfactory solution demands an appropriate balance between use of prior knowledge and learning from data. We propose a consistent Bayesian framework for introducing strong prior knowledge into a system for extracting human gait. In this work, the strong prior is built from a simple articulated model having both time-invariant (static) and time-variant (dynamic) parameters. The model is easily modified to cater to situations such as walkers wearing clothing that obscures the limbs. The statistics of the parameters are learned from high-quality (indoor laboratory) data and the Bayesian framework then allows us to "bootstrap" to accurate gait extraction on the noisy images typical of cluttered, outdoor scenes. To achieve automatic fitting, we use a hidden Markov model to detect the phases of images in a walking cycle. We demonstrate our approach on silhouettes extracted from fronto-parallel ("sideways on") sequences of walkers under both high-quality indoor and noisy outdoor conditions. As well as high-quality data with synthetic noise and occlusions added, we also test walkers with rucksacks, skirts, and trench coats. Results are quantified in terms of chamfer distance and average pixel error between automatically extracted body points and corresponding hand-labeled points. No one part of the system is novel in itself, but the overall framework makes it feasible to extract gait from very much poorer quality image sequences than hitherto. This is confirmed by comparing person identification by gait using our method and a well-established baseline recognition algorithm.
Adam Prügel-Bennett, Robert I. Damper
IEEE Trans. Pattern Anal. Mach. Intell.2
2006 Phase transitions and symmetry breaking in genetic algorithms with crossover
Alex Rogers, Adam Prügel-Bennett, Nicholas R. Jennings
Theor. Comput. Sci.2
2006 Evolving the structure of hidden Markov models
abstract
A genetic algorithm (GA) is proposed for finding the structure of hidden Markov Models (HMMs) used for biological sequence analysis. The GA is designed to preserve biologically meaningful building blocks. The search through the space of HMM structures is combined with optimization of the emission and transition probabilities using the classic Baum-Welch algorithm. The system is tested on the problem of finding the promoter and coding region of C. jejuni. The resulting HMM has a superior discrimination ability to a handcrafted model that has been published in the literature.
Kyoung-Jae Won, Adam Prügel-Bennett, Anders Krogh
IEEE Trans. Evol. Comput.2
2006 A heuristic bidding strategy for buying multiple goods in multiple english auctions
abstract
This paper presents the design, implementation, and evaluation of a novel bidding algorithm that a software agent can use to obtain multiple goods from multiple overlapping English auctions. Specifically, an Earliest Closest First heuristic algorithm is proposed that uses neurofuzzy techniques to predict the expected closing prices of the auctions and to adapt the agent's bidding strategy to reflect the type of environment in which it is situated. This algorithm first identifies the set of auctions that are most likely to give the agent the best return and then, according to its attitude to risk, it bids in some other auctions that have approximately similar expected returns, but which finish earlier than those in the best return set. We show through empirical evaluation against a number of methods proposed in the multiple auction literature that our bidding strategy performs effectively and robustly in a wide range of scenarios.
Minghua He, Nicholas R. Jennings, Adam Prügel-Bennett
ACM Trans. Internet Techn.3
2005 Optimal simulated annealing schedules for larger problems
abstract
We present a method for optimizing parameters for a search algorithm (we choose simulated annealing as a specific example) on a finite search space. The search is described as a Markov process, giving the average cost on a specific problem as a function of the search parameters. A minimization is then performed over the parameter space to provide an optimal parameter set. We demonstrate this technique on a toy problem; we then use a 'barrier tree' model to reduce 20-variable Max-SAT problems from over a million search points to more manageable 30-40 states. The annealing schedules produced do not perform as well as predicted, but there is some evidence that a single schedule optimized over a problem set may produce better results.
William Benfold, Jonathan Hallam, Adam Prügel-Bennett
Congress on Evolutionary Computation3
2005 Barrier-based models of hard problems and crossover
abstract
Model problems are useful in the study of combinatorial optimisation algorithms as they allow results to be calculated that are difficult or impossible with real problems. However, these 'toy' problems are often contrived to show a particular feature, and it is difficult to know how they compare to real problems. We present a framework for creating models of hard optimisation problems that captures large landscape features such as local optima and their basins. The framework aggregates configurations by partitioning the search space based on the structure of basins and barriers in the landscape. This results in a model problem with a massively reduced number of states. For the model problem it is readily feasible to study simple optimisation algorithms using a Markov chain analysis. Genetic algorithms are one type of problem that is hard to model in this framework. We demonstrate the difficulty in modelling just one aspect of these algorithms, crossover, by trying a simple technique to model it.
Jonathan Hallam, Adam Prügel-Bennett
Congress on Evolutionary Computation2
2005 Evolving hidden Markov models for protein secondary structure prediction
abstract
New results are presented for the prediction of secondary structure information for protein sequences using hidden Markov models (HMMs) evolved using a genetic algorithm (GA). We achieved a Q/sub 3/ measure of 75% using one of the most stringent data set ever used for protein secondary structure prediction. Our results beat the best hand-designed HMM currently available and are comparable to the best known techniques for this problem. A hybrid GA incorporating the Baum-Welch algorithm was used. The topology of the HMM was restricted to biologically meaningful building blocks. Mutation and crossover operators were designed to explore this space of topologies.
Kyoung-Jae Won, Thomas Hamelryck, Adam Prügel-Bennett, Anders Krogh
Congress on Evolutionary Computation3
2005 A Distributed Approach to Musical Composition
Michael O. Jewell, Lee Middleton, Mark S. Nixon, Adam Prügel-Bennett, Sylvia C. Wong
KES (3)4
2005 Large Barrier Trees for Studying Search
abstract
Barrier trees are a method for representing the landscape structure of high-dimensional discrete spaces such as those that occur in the cost function of combinatorial optimization problems. The leaves of the tree represent local optima and a vertex where subtrees join represents the lowest cost saddle-point between the local optima in the subtrees. This paper introduces an extension to existing Barrier tree methods that make them more useful for studying heuristic optimization algorithms. It is shown that every configuration in the search space can be mapped onto a vertex in the Barrier tree. This provides additional information about the landscape, such as the number of configurations in a local optimum. It also allows the computation of additional statistics such as the correlation between configurations in different parts of the Barrier tree. Furthermore, the mappings allow the dynamic behavior of a heuristic search algorithms to be visualized. This extension is illustrated using an instance of the MAX-3-SAT problem.
Jonathan Hallam, Adam Prügel-Bennett
IEEE Trans. Evol. Comput.2
2004 An adaptive bidding agent for multiple English auctions: a neuro-fuzzy approach
abstract
This work presents the design, implementation and evaluation of a novel bidding strategy for obtaining goods in multiple overlapping English auctions. The strategy uses fuzzy sets to express trade-offs between multi-attribute goods and exploits neuro-fuzzy techniques to predict the expected closing prices of the auctions and to adapt the agent's bidding strategy to reflect the type of environment in which it is situated. We show, through empirical evaluation against a number of methods proposed in the multiple auction literature, that our strategy performs effectively and robustly in a wide range of scenarios.
Minghua He, Nicholas R. Jennings, Adam Prügel-Bennett
FUZZ-IEEE3
2004 The Block Hidden Markov Model for Biological Sequence Analysis
Kyoung-Jae Won, Adam Prügel-Bennett, Anders Krogh
KES2
2004 A Data Clustering and Streamline Reduction Method for 3D MR Flow Vector Field Simplification
Bernardo Silva Carmo, Yin-Heung Pauline Ng, Adam Prügel-Bennett, Guang-Zhong Yang
MICCAI (1)3
2004 Training HMM structure with genetic algorithm for biological sequence analysis
abstract
SUMMARY: Hidden Markov models (HMMs) are widely used for biological sequence analysis because of their ability to incorporate biological information in their structure. An automatic means of optimizing the structure of HMMs would be highly desirable. However, this raises two important issues; first, the new HMMs should be biologically interpretable, and second, we need to control the complexity of the HMM so that it has good generalization performance on unseen sequences. In this paper, we explore the possibility of using a genetic algorithm (GA) for optimizing the HMM structure. GAs are sufficiently flexible to allow incorporation of other techniques such as Baum-Welch training within their evolutionary cycle. Furthermore, operators that alter the structure of HMMs can be designed to favour interpretable and simple structures. In this paper, a training strategy using GAs is proposed, and it is tested on finding HMM structures for the promoter and coding region of the bacterium Campylobacter jejuni. The proposed GA for hidden Markov models (GA-HMM) allows, HMMs with different numbers of states to evolve. To prevent over-fitting, a separate dataset is used for comparing the performance of the HMMs to that used for the Baum-Welch training. The GA-HMM was capable of finding an HMM comparable to a hand-coded HMM designed for the same task, which has been published previously.
Kyoung-Jae Won, Adam Prügel-Bennett, Anders Krogh
Bioinform.2
2004 When a genetic algorithm outperforms hill-climbing
Adam Prügel-Bennett
Theor. Comput. Sci.1
2004 Symmetry breaking in population-based optimization
abstract
Argues that the performance of evolutionary algorithms working on hard optimization problems depends strongly on how the population breaks the "symmetry" of the search space. The splitting of the search space into widely separate regions containing local optima is a generic property of a large class of hard optimization problem. This phenomenon is discussed by reference to two well studied examples, the Ising perceptron and the satisfiability problem (K-SAT). A finite population will quickly concentrate on one region of the search space. The cost of crossover between solutions in different regions of search space can accelerate this symmetry breaking. This, in turn, can dramatically reduce the amount of exploration, leading to suboptimal solutions being found. An analysis of symmetry breaking using diffusion model techniques borrowed from classical population genetics is presented. This shows how symmetry breaking depends on parameters such as the population size and selection rate.
Adam Prügel-Bennett
IEEE Trans. Evol. Comput.1
2003 Barrier Trees For Search Analysis
Jonathan Hallam, Adam Prügel-Bennett
GECCO2
2003 Automatic gait recognition using area-based metric
Jeff P. Foster, Mark S. Nixon, Adam Prügel-Bennett
Pattern Recognit. Lett.3
2001 New Area Based Metrics for Automatic Gait Recognition
abstract
Gait is a new biometric aimed to recognise a subject by the manner in which they walk. Gait has several advantages over other biometrics, most notably that it is non-invasive and perceivable at a distance when other biometrics are obscured. We present a new area based metric, called gait masks, which provides statistical data intimately related to the gait of the subject and motivated by medical studies. This provides the first statistical approach that can expose the dynamics of the change in area of a subject. Early results show promising results with a recognition rate of 90% on a standard database. Further, there appear to be performance advantages with respect to handling of noise associated with this new approach, together with capability for extension and generalisation. Future research will capitalise on the advantages of this new approach, together with analysis on a larger database. 1.
Jeff P. Foster, Mark S. Nixon, Adam Prügel-Bennett
BMVC3
2001 Modeling crossover-induced linkage in genetic algorithms
abstract
The dynamics of a genetic algorithm undergoing ranking selection, mutation, and two-point crossover for the ones-counting problem is studied using a statistical mechanics approach. This approach has been used previously to study this problem, but with uniform crossover. Two-point crossover induces additional linkage between nearby loci, which changes the dynamics significantly. To account for this linkage, the evolution of the autocorrelation function is incorporated into a model of the dynamics. This complicates the analysis and requires several additional approximations to be made. However, the model we derive is shown to capture the main features of the dynamics and is in good agreement with simulations.
Adam Prügel-Bennett
IEEE Trans. Evol. Comput.1
1999 Genetic drift in genetic algorithm selection schemes
abstract
A method for calculating genetic drift in terms of changing population fitness variance is presented. The method allows for an easy comparison of different selection schemes and exact analytical results are derived for traditional generational selection, steady-state selection with varying generation gap, a simple model of Eshelman's CHC algorithm (1991), and (/spl mu/+/spl lambda/) evolution strategies. The effects of changing genetic drift on the convergence of a GA are demonstrated empirically.
Alex Rogers, Adam Prügel-Bennett
IEEE Trans. Evol. Comput.2
1996 Learning Synfire Chains: Turning Noise into Signal
John Hertz, Adam Prügel-Bennett
Int. J. Neural Syst.2
1993 Non-Linear Statistical Analysis and Self-Organizing Hebbian Networks
Jonathan L. Shapiro, Adam Prügel-Bennett
NIPS2