Abstract Unsupervised learning has gained prominence in the big data era, offering a means to extract valuable insights from unlabeled datasets. Deep clustering has emerged as an important unsupervised category, aiming to exploit the non-linear mapping capabilities of neural networks in order to enhance clustering performance. The majority of deep clustering literature focuses on minimizing the inner-cluster variability in some embedded space while keeping the learned representation consistent with the original high-dimensional dataset. In this work, we propose soft silhouette , a probabilistic formulation of the silhouette coefficient. Soft silhouette rewards compact and distinctly separated clustering solutions such as the conventional silhouette coefficient. When optimized within a deep clustering framework, soft silhouette guides the learned representations towards forming compact and well-separated clusters. In addition, we introduce an autoencoder-based deep learning architecture that is suitable for optimizing the soft silhouette objective function. The proposed deep clustering method has been tested and compared with several well-studied deep clustering methods on various benchmark datasets, yielding very satisfactory clustering results.

Estimating the number of clusters k while clustering the data is a challenging task. An incorrect cluster assumption indicates that the number of clusters k gets wrongly estimated. Consequently, the model fitting becomes less important. In this work, we focus on the concept of unimodality and propose a flexible cluster definition called locally unimodal cluster . A locally unimodal cluster extends for as long as unimodality is locally preserved across pairs of subclusters of the data. Then, we propose the UniForCE method for locally unimodal clustering. The method starts with an initial overclustering of the data and relies on the unimodality graph that connects subclusters forming unimodal pairs. Such pairs are identified using an appropriate statistical test. UniForCE identifies maximal locally unimodal clusters that are statistically significant by computing a spanning forest in the unimodality graph. Experimental results on both real and synthetic datasets illustrate that the proposed methodology is particularly flexible and robust in discovering regular and highly complex cluster shapes. Most importantly, it automatically provides an adequate estimation of the number of clusters.

Counterfactuals have been recognized as an effective approach to explain classifier decisions. In this work, we focus on the use of counterfactuals to explain clustering solutions. First, we present a general definition for counterfactuals for model-based clustering that includes plausibility and feasibility constraints. Then we consider the counterfactual generation problem for$k$-means and Gaussian clustering assuming Euclidean distance. Our approach takes as input the factual, the target cluster, a binary mask indicating actionable or immutable features and a plausibility factor specifying how far from the cluster boundary the counterfactual should be placed. In the$k$-means clustering case, analytical mathematical formulas are presented for computing the optimal solution, while in the Gaussian clustering case (assuming full, diagonal, or spherical covariances) our method requires the numerical solution of a nonlinear equation with a single parameter only. We demonstrate the advantages of our approach through illustrative examples and quantitative experimental comparisons.

Unimodality, pivotal in statistical analysis, offers insights into dataset structures and drives sophisticated analytical procedures. While unimodality’s confirmation is straightforward for one-dimensional data using methods like Silverman’s approach and Hartigans’ dip statistic, its generalization to higher dimensions remains challenging. By extrapolating one-dimensional unimodality principles to multi-dimensional spaces through linear random projections and leveraging point-to-point distancing, our method, rooted in $\alpha$-unimodality assumptions, presents a novel multivariate unimodality test named $\textit{mud-pod}$. Both theoretical and empirical studies confirm the efficacy of our method in unimodality assessment of multidimensional datasets as well as in estimating the number of clusters.

Abstract Kernel k -means extends the k -means algorithm to identify non-linearly separable clusters but is inherently sensitive to cluster initialization. To address this challenge, we first formulate the kernel k-means ++ method, which conveys the efficient center initialization strategy of k -means++ from Euclidean to kernel space. Building on this, we propose global kernel k-means ++ ( $$\text {GK}k\text {M}$$ ++), a novel clustering algorithm designed to balance clustering error minimization with reduced computational cost. $$\text {GK}k\text {M}$$ ++ extends the well-established global kernel k -means algorithm by incorporating the stochastic initialization strategy of kernel k -means++. This approach significantly reduces computational complexity while preserving superior clustering error minimization capabilities akin to traditional global kernel k -means. The experimental results on synthetic, real, and graph datasets indicate that $$\text {GK}k\text {M}$$ ++ consistently outperforms both kernel k -means with random initialization and kernel k -means++, while achieving solutions comparable to those provided by the exhaustive and computational intensive global kernel k -means method.

Distance from unimodality (DFU) has been found to correlate well with human judgment for the assessment of polarized opinions.However, its un-normalized nature makes it less intuitive and somewhat difficult to exploit in machine learning (e.g., as a supervised signal).In this work a normalized version of this measure, called nDFU, is proposed that leads to better assessment of the degree of polarization.Then, we propose a methodology for K-class text classification, based on nDFU, that exploits polarized texts in the dataset.Such polarized instances are assigned to a separate K+1 class, so that a K+1-class classifier is trained.An empirical analysis on three datasets for abusive language detection, shows that nDFU can be used to model polarized annotations and prevent them from harming the classification performance.Finally, we further exploit nDFU to specify conditions that could explain polarization given a dimension and present text examples that polarized the annotators when the dimension was gender and race.Our code is available at https://github.com/ipavlopoulos/ndfu.

Abstract Greek literary papyri, which are unique witnesses of antique literature, do not usually bear a date. They are thus currently dated based on palaeographical methods, with broad approximations which often span more than a century. We created a dataset of 242 images of papyri written in “bookhand” scripts whose date can be securely assigned, and we used it to train algorithms for the task of dating, showing its challenging nature. To address data scarcity, we extended our dataset by segmenting each image into its respective text lines. By using the line-based version of our dataset, we trained a Convolutional Neural Network, equipped with a fragmentation-based augmentation strategy, and we achieved a mean absolute error of 54 years. The results improve further when the task is cast as a multi-class classification problem, predicting the century. Using our network, we computed precise date estimations for papyri whose date is disputed or vaguely defined, employing explainability to understand dating-driving features.

Abstract Clustering is one of the most fundamental unsupervised learning tasks with numerous applications in various fields. Clustering methods based on neural networks, called deep clustering methods, leverage the representational power of neural networks to enhance clustering performance. ClusterGan constitutes a generative deep clustering method that exploits generative adversarial networks (GANs) to perform clustering. However, it inherits some deficiencies of GANs, such as mode collapse, vanishing gradients and training instability. In order to tackle those deficiencies, the generative approach of implicit maximum likelihood estimation (IMLE) has been recently proposed. In this paper, we present a clustering method based on generative neural networks, called neural implicit maximum likelihood clustering, which adopts ideas from both ClusterGAN and IMLE. The proposed method has been compared with ClusterGAN and other neural clustering methods on both synthetic and real datasets, demonstrating promising results.

Dynamic replication is a wide-spread multi-copy routing approach for efficiently coping with the intermittent connectivity in mobile opportunistic networks. According to it, a node forwards a message replica to an encountered node based on a utility value that captures the latter's fitness for delivering the message to the destination. The popularity of the approach stems from its flexibility to effectively operate in networks with diverse characteristics without requiring special customization. Nonetheless, its drawback is the tendency to produce a high number of replicas that consume limited resources such as energy and storage. To tackle the problem we make the observation that network nodes can be grouped, based on their utility values, into clusters that portray different delivery capabilities. We exploit this finding to transform the basic forwarding strategy, which is to move a packet using nodes of increasing utility, and actually forward it through clusters of increasing delivery capability. The new strategy works in synergy with the basic dynamic replication algorithms and is fully configurable, in the sense that it can be used with virtually any utility function. We also extend our approach to work with two utility functions at the same time, a feature that is especially efficient in mobile networks that exhibit social characteristics. By conducting experiments in a wide set of real-life networks, we empirically show that our method is robust in reducing the overall number of replicas in networks with diverse connectivity characteristics without at the same time hindering delivery efficiency.

One of the most interesting problems in network analysis is community detection, i.e. the partitioning of nodes into communities, with many edges connecting nodes of the same community and comparatively few edges connecting nodes of different communities. We introduce a new quality measure to evaluate a partitioning of an undirected and unweighted graph into communities that is called inclusion. This quality measure evaluates how well each node is included in its community by considering both its existent and its non-existent edges. We have implemented a strategy that maximizes the inclusion criterion by moving each time a single node to another community. We also considered inclusion as a criterion for evaluating partitions provided by spectral clustering. In our experimental study, the inclusion criterion is compared to the widely used modularity criterion providing improved community detection results without requiring the a priori specification of the number of communities.

In this work, we propose a novel, multi-threshold method for lip contour extraction from high-resolution static lip images acquired in an uncontrolled environment and emphasizing on the contour details. The introduced method is a “Divide and conquer” approach. We broke the problem of lip contour extraction into two base sub-problems of locating the upper and lower lip contours using a novel threshold selection algorithm and combining them into the solution for the original lip contour. The method was evaluated on a set of lip images taken from healthy subjects as well as on a set acquired from elder people with degraded lip shapes, and diagnosed with solar cheilosis. Our approach is a low complexity algorithm for robust and accurate lip contour extraction revealing shape details, which might be of value in diverse applied fields such as automated lip biometric and monitoring patients with affected lip contour.

Video slide shows constitute a very frequent video type. Typically, in order to produce such a video, a series of still images are used and processed. We present a method to solve the video slide show inversion problem where we are given a video slide show and we wish to extract the sequence of still images that have been used to produce this video. Our approach relies on a fast and efficient key-frame extraction method that partitions the video into content homogeneous segments and extracts a representative key-frame for each video segment. An important characteristic of this method is that the number of key-frames is determined automatically. Next, the set of key-frames is further processed to solve various problems such as mixing of images, fade-in/fade-out and zooming effects that produce keyframes not belonging to the original image sequence, thus they must be detected and discarded. We provide illustrative examples from the application of the method on several video slide shows with different characteristics.

Mutli-copy routing is an efficient and wide-spread approach for coping with the intermittent connectivity of mobile opportunistic networks. One popular multi-copy approach is known as “dynamic” replication; each node creates replicas on a contact basis using a utility that determines a node's fitness for delivering a message to its destination. This scheme is highly flexible, configurable with different utility functions and able to operate efficiently in networks with diverse characteristics. Nonetheless, its drawback is the tendency to produce a high number of replicas that consume limited resources such as energy and storage. Our approach to tackle this problem relies on the observation that, based on their utility values, the network nodes can be grouped into clusters that portray different delivery capabilities. Then, to avoid unnecessary replication, we exploit this finding to replicate a packet to nodes that belong to clusters with increasing delivery capability instead of replicating it to nodes with increasing utility. The new method works in synergy with the basic dynamic replication algorithms and is fully configurable, in the sense that it can be used with virtually any utility function. By conducting experiments in a diverse set of real-life networks, we empirically show that the method effectively reduces the overall number of replicas without hindering delivery efficiency.

Most of the existing approaches for camera motion detection are based on optical flow analysis and the use of the affine motion model. However, these methods are computationally expensive due to the cost of optical flow estimation and may be inefficient in the presence of moving objects whose motion is independent of the camera motion. We present an effective approach to detect camera motions by considering four trapezoidal regions in each frame and computing the horizontal and vertical translations of those regions. Then, simple decision rules based on the translations of the regions are employed in order to decide for the existence and the type of camera motion in each frame. In this way, three signals are constructed (pan, tilt, zoom) which are subsequently filtered to improve the robustness of the method. Comparative experiments on a variety of videos indicate that our method efficiently detects any type of camera motion (pan, tilt, zoom), even in the case where moving objects exist in the video sequence.

MOTIVATION: Transient S-sulfenylation of cysteine thiols mediated by reactive oxygen species plays a critical role in pathology, physiology and cell signaling. Therefore, discovery of new S-sulfenylated sites in proteins is of great importance towards understanding how protein function is regulated upon redox conditions. RESULTS: We developed PRESS (PRotEin S-Sulfenylation) web server, a server which can effectively predict the cysteine thiols of a protein that could undergo S-sulfenylation under redox conditions. We envisage that this server will boost and facilitate the discovery of new and currently unknown functions of proteins triggered upon redox conditions, signal regulation and transduction, thus uncovering the role of S-sulfenylation in human health and disease. AVAILABILITY AND IMPLEMENTATION: The PRESS web server is freely available at http://press-sulfenylation.cse.uoi.gr/ CONTACTS: [email protected] or [email protected] SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

A method for modeling and removing outliers from 2-D sets of scattered points is presented. The method relies on a principle due to Helmholtz stating that every large deviation from uniform noise should be perceptible, provided that the deviation is generated by an a contrario model of geometric structures. By assuming local linearity, we first employ a robust algorithm to model the local manifold of the corrupted data by local line segments. Our rationale is that long line segments should not be expected in a noisy set of points. This assumption leads to the modeling of the lengths of the line segments by a Pareto distribution, which is the adopted a contrario model for the observations. The model is successfully evaluated on two problems in computer vision: shape recovery and linear regression.

An algorithm for sampling image edge points is presented. At first, the image edges are summarized by line segments, which represent the long axis of highly eccentric ellipses. Then, each line segment is partitioned into a number of bins and the point which is closer to the center of the bin is selected. Experiments on widely used databases demonstrate that the proposed method is accurate and provides samples that preserve the coherence of the initial information of the edge map, which is of importance in image retrieval applications.

An efficient shot summarization method is presented based on agglomerative clustering of the shot frames. Unlike other agglomerative methods, our approach relies on a cluster merging criterion that computes the content homogeneity of a merged cluster. An important feature of the proposed approach is the automatic estimation of the number of a shot's most representative frames, called keyframes. The method starts by splitting each video sequence into small, equal sized clusters (segments). Then, agglomerative clustering is performed, where from the current set of clusters, a pair of clusters is selected and merged to form a larger unimodal (homogeneous) cluster. The algorithm proceeds until no further cluster merging is possible. At the end, the medoid of each of the final clusters is selected as keyframe and the set of keyframes constitutes the summary of the shot. Numerical experiments demonstrate that our method reasonable estimates the number of ground-truth keyframes, while extracting non-repetitive keyframes that efficiently summarize the content of each shot.

Reliable video summarization is one of the most important problems in digital video processing and analysis. The most common approach used for shot representation is the extraction of a set of key-frames sufficiently representing the total content of the shot. In such way, the whole video content can be represented using only a few, cautiously picked, non redundant key-frames maintaining at the same time a great percentage of information. A typical approach is to extract key frames using clustering. However, using a single image descriptor to extract key-frames is not sufficient due to large variations in the visual content of videos. In our approach, a weighted multi-view clustering algorithm is employed to combine two different image descriptors into a single similarity matrix, that serves as an input to a spectral clustering algorithm. Each image descriptor (view) does not contribute equally to the similarity matrix, but the weighted multi-view clustering algorithm associates a weight with each view and learns these weights automatically. Numerical experiments using a variety of videos demonstrate that our method is capable of efficiently summarizing video shots regardless of the characteristics of the visual content of the video.

Chromosome analysis is an important and difficult task for clinical diagnosis and biological research. A color imaging technique, multiplex fluorescent in situ hybridization (M-FISH), has been developed to ease the analysis of the process. Using an M-FISH technique each chromosome class (1,2, …,22,X,Y) is stained with a unique color. However, significant variations between images are observed due to a number of factors such as uneven hybridization and spectral overlap among channels. These types of variations influence the pixel classification accuracy of image classification methods which are supervised and require a set of annotated images for training. In this paper, we present a fully unsupervised M-FISH chromosome image classification methodology. Our main contributions are 1) the assumption that the intensity of a chromosome pixel is sampled from multiple Gaussian components [Gaussian mixture model (GMM)] such that each component corresponds to one chromosome class, and 2) the initialization of the GMM model using the emission information of each chromosome class. This is feasible since prior to the M-FISH image acquirement, we already know which chromosome class is emitting to each of the five M-FISH image channels. The method has been tested on a large number of M-FISH images and an overall accuracy of 89.85% is reported. Our method is unsupervised and presents higher classification accuracy even when it is compared with common supervised based methods. Since the developed classification method does not require training data, it is highly convenient when ground truth does not exist.

Noisy optimization problems arise very often in real-life applications. A common practice to tackle problems characterized by uncertainties, is the re-evaluation of the objective function at every point of interest for a fixed number of replications. The obtained objective values are then averaged and their mean is considered as the approximation of the actual objective value. However, this approach can prove inefficient, allocating replications to unpromising candidate solutions. We propose a hybrid approach that integrates the established Particle Swarm Optimization algorithm with the Reinforcement Learning approach to efficiently tackle noisy problems by intelligently allocating the available computational budget. Two variants of the proposed approach, based on different selection schemes, are assessed and compared against the typical alternative of equal sampling. The results are reported and analyzed, offering significant evidence regarding the potential of the proposed approach.

We present a new regression mixture model where each mixture component is a multi-kernel version of the Relevance Vector Machine (RVM). In the proposed model, we exploit the enhanced modeling capability of RVMs due to their embedded sparsity enforcing properties. %The main contribution of this %work is the employment of RVM models as components of a mixture %model and their application to the time series clustering problem. Moreover, robustness is achieved with respect to the kernel parameters, by employing a weighted multi-kernel scheme. The mixture model is trained using the maximum a posteriori (MAP) approach, where the Expectation Maximization (EM) algorithm is applied offering closed form update equations for the model parameters. An incremental learning methodology is also presented to tackle the parameter initialization problem of the EM algorithm. The efficiency of the proposed mixture model is empirically demonstrated on the time series clustering problem using various artificial and real benchmark datasets and by performing comparisons with other regression mixture models.

Exploiting multiple representations, or views, for the same set of instances within a clustering framework is a popular practice for boosting clustering accuracy. However, some of the available sources may be misleading (due to noise, errors in measurement etc.) in revealing the true structure of the data, thus, their inclusion in the clustering process may have negative influence. This aspect seems to be overlooked in the multi-view literature where all representations are equally considered. In this work, views are expressed in terms of given kernel matrices and a weighted combination of the kernels is learned in parallel to the partitioning. Weights assigned to kernels are indicative of the quality of the corresponding views' information. Additionally, the combination scheme incorporates a parameter that controls the admissible sparsity of the weights to avoid extremes and tailor them to the data. Two efficient iterative algorithms are proposed that alternate between updating the view weights and recomputing the clusters to optimize the intra-cluster variance from different perspectives. The conducted experiments reveal the effectiveness of our methodology compared to other multi-view methods.

A probabilistic real time tracking algorithm is proposed. The distribution of the target is represented by a Gaussian mixture model (GMM) and the weighted likelihood of the target is maximized in order to localize it in an image sequence. The role of the weight is important as it allows gradient based optimization to be performed, which would not be feasible in a context of standard likelihood representations. The algorithm models both the object to be tracked and local background elements and handles scale changes in target's appearance. It is experimentally demonstrated that the algorithm runs in real time, and it is at least at the same performance level with the mean shift algorithm while it provides more accurate target localization in non trivial scenarios (e.g. shadows).

Learning the number of clusters is a key problem in data clustering. We present dip-means, a novel robust incremental method to learn the number of data clusters that may be used as a wrapper around any iterative clustering algorithm of the k-means family. In contrast to many popular methods which make assumptions about the underlying cluster distributions, dip-means only assumes a fundamental cluster property: each cluster to admit a unimodal distribution. The proposed algorithm considers each cluster member as a ''viewer'' and applies a univariate statistic hypothesis test for unimodality (dip-test) on the distribution of the distances between the viewer and the cluster members. Two important advantages are: i) the unimodality test is applied on univariate distance vectors, ii) it can be directly applied with kernel-based methods, since only the pairwise distances are involved in the computations. Experimental results on artificial and real datasets indicate the effectiveness of our method and its superiority over analogous approaches.

A new Bayesian model is proposed for image segmentation based upon Gaussian mixture models (GMM) with spatial smoothness constraints. This model exploits the Dirichlet compound multinomial (DCM) probability density to model the mixing proportions (i.e., the probabilities of class labels) and a Gauss-Markov random field (MRF) on the Dirichlet parameters to impose smoothness. The main advantages of this model are two. First, it explicitly models the mixing proportions as probability vectors and simultaneously imposes spatial smoothness. Second, it results in closed form parameter updates using a maximum a posteriori (MAP) expectation-maximization (EM) algorithm. Previous efforts on this problem used models that did not model the mixing proportions explicitly as probability vectors or could not be solved exactly requiring either time consuming Markov Chain Monte Carlo (MCMC) or inexact variational approximation methods. Numerical experiments are presented that demonstrate the superiority of the proposed model for image segmentation compared to other GMM-based approaches. The model is also successfully compared to state of the art image segmentation methods in clustering both natural images and images degraded by noise.

A new relevance feedback (RF) approach for content-based image retrieval is presented. This approach uses Gaussian mixture (GM) models of the image features and a query that is updated in a probabilistic manner. This update reflects the preferences of the user and is based on the models of both the positive and negative feedback images. The retrieval is based on a recently proposed distance measure between probability density functions, which can be computed in closed form for GM models. The proposed approach takes advantage of the form of this distance measure and updates it very efficiently based on the models of the user-specified relevant and irrelevant images. It is also shown that this RF framework is fairly general and can be applied in case other image models or distance measures are used instead of those proposed in this work. Finally, comparative numerical experiments are provided, which that demonstrate the merits of the proposed RF methodology and the use of the distance measure, and also the advantages of using GMs for image modelling.

In this paper, we present a new Bayesian model for the blind image deconvolution (BID) problem. The main novelty of this model is the use of a sparse kernel-based model for the point spread function (PSF) that allows estimation of both PSF shape and support. In the herein proposed approach, a robust model of the BID errors and an image prior that preserves edges of the reconstructed image are also used. Sparseness, robustness, and preservation of edges are achieved by using priors that are based on the Student's-t probability density function (PDF). This pdf, in addition to having heavy tails, is closely related to the Gaussian and, thus, yields tractable inference algorithms. The approximate variational inference methodology is used to solve the corresponding Bayesian model. Numerical experiments are presented that compare this BID methodology to previous ones using both simulated and real data.

Video indexing requires the efficient segmentation of video into scenes. The video is first segmented into shots and a set of key-frames is extracted for each shot. Typical scene detection algorithms incorporate time distance in a shot similarity metric. In the method we propose, to overcome the difficulty of having prior knowledge of the scene duration, the shots are clustered into groups based only on their visual similarity and a label is assigned to each shot according to the group that it belongs to. Then, a sequence alignment algorithm is applied to detect when the pattern of shot labels changes, providing the final scene segmentation result. In this way shot similarity is computed based only on visual features, while ordering of shots is taken into account during sequence alignment. To cluster the shots into groups we propose an improved spectral clustering method that both estimates the number of clusters and employs the fast global k-means algorithm in the clustering stage after the eigenvector computation of the similarity matrix. The same spectral clustering method is applied to extract the key-frames of each shot and numerical experiments indicate that the content of each shot is efficiently summarized using the method we propose herein. Experiments on TV-series and movies also indicate that the proposed scene detection method accurately detects most of the scene boundaries while preserving a good tradeoff between recall and precision.

Sparse kernel methods are very efficient in solving regression and classification problems. The sparsity and performance of these methods depend on selecting an appropriate kernel function, which is typically achieved using a cross-validation procedure. In this paper, we propose an incremental method for supervised learning, which is similar to the relevance vector machine (RVM) but also learns the parameters of the kernels during model training. Specifically, we learn different parameter values for each kernel, resulting in a very flexible model. In order to avoid overfitting, we use a sparsity enforcing prior that controls the effective number of parameters of the model. We present experimental results on artificial data to demonstrate the advantages of the proposed method and we provide a comparison with the typical RVM on several commonly used regression and classification data sets.

Kernel k-means is an extension of the standard k -means clustering algorithm that identifies nonlinearly separable clusters. In order to overcome the cluster initialization problem associated with this method, we propose the global kernel k-means algorithm, a deterministic and incremental approach to kernel-based clustering. Our method adds one cluster at each stage, through a global search procedure consisting of several executions of kernel k-means from suitable initializations. This algorithm does not depend on cluster initialization, identifies nonlinearly separable clusters, and, due to its incremental nature and search procedure, locates near-optimal solutions avoiding poor local minima. Furthermore, two modifications are developed to reduce the computational cost that do not significantly affect the solution quality. The proposed methods are extended to handle weighted data points, which enables their application to graph partitioning. We experiment with several data sets and the proposed approach compares favorably to kernel k -means with random restarts.

In this paper a relevance feedback (RF) approach for content based image retrieval (CBIR) is described and evaluated. The approach uses Gaussian mixture (GM) models of the image features and a query that is updated in a probabilistic manner. This update reflects the preferences of the user and is based on the models of both positive and negative feedback images. Retrieval is based on a recently proposed distance measure between probability density functions (pdfs), which can be computed in closed form for GM models. The proposed approach takes advantage of the form of this distance measure and updates it very efficiently based on the models of the user specified relevant and irrelevant images. For evaluation purposes, comparative experimental results are presented that demonstrate the merits of the proposed methodology.

Kernel k-means is an extension of the standard k-means clustering algorithm that identifies nonlinearly separable clusters. In order to overcome the cluster initialization problem associated with this method, in this work we propose the global kernel k-means algorithm, a deterministic and incremental approach to kernel-based clustering. Our method adds one cluster at each stage through a global search procedure consisting of several executions of kernel k-means from suitable initializations. This algorithm does not depend on cluster initialization, identifies nonlinearly separable clusters and, due to its incremental nature and search procedure, locates near optimal solutions avoiding poor local minima. Furthermore a modification is proposed to reduce the computational cost that does not significantly affect the solution quality. We test the proposed methods on artificial data and also for the first time we employ kernel k-means for MRI segmentation along with a novel kernel. The proposed methods compare favorably to kernel k-means with random restarts.

Image priors based on products have been recognized to offer many advantages because they allow simultaneous enforcement of multiple constraints. However, they are inconvenient for Bayesian inference because it is hard to find their normalization constant in closed form. In this paper, a new Bayesian algorithm is proposed for the image restoration problem that bypasses this difficulty. An image prior is defined by imposing Student-t densities on the outputs of local convolutional filters. A variational methodology, with a constrained expectation step, is used to infer the restored image. Numerical experiments are shown that compare this methodology to previous ones and demonstrate its advantages.

We propose a pixel similarity-based algorithm enabling accurate rigid registration between single and multimodal images presenting gross dissimilarities due to noise, missing data or outlying measures. The method relies on the partitioning of a reference image by a Student's t-mixture model (SMM). This partition is then projected onto the image to be registered. The main idea is that a t-component in the reference image corresponds to a t-component in the image to be registered. If the images are correctly registered the weighted sum of distances between the corresponding components is minimized. The use of SMM components is justified by the property that they have heavier tails than standard Gaussians, thus providing robustness to outliers. Experimental results indicate that, even in the case of images presenting low SNR or important amount of dissimilarities due to temporal changes, the proposed algorithm compares favorably to the histogram-based mutual information method that is widely used in a variety of applications.

In this paper we present a new Bayesian model for the blind image deconvolution (BID) problem. The main novelties of this model are three. The first one is the use of a sparse kernel-based model for the point spread function (PSF) that allows estimation of both PSF shape and support. The second one is a robust distribution of the BID model errors and the third novelty is an image prior that preserves edges of the reconstructed image. Sparseness, robustness and preservation of edges is achieved by using priors that are based on the Student-t probability density function (pdf). The Variational methodology is used to solve the corresponding Bayesian model. Numerical experiments are presented that demonstrate the advantages of this model as compared to previous Gaussian based ones.

This paper proposes a novel approach for spam filtering based on the use of Deep Belief Networks (DBNs). In contrast to conventional feedfoward neural networks having one or two hidden layers, DBNs are feedforward neural networks with many hidden layers. Until recently it was not clear how to initialize the weights of deep neural networks, which resulted in poor solutions with low generalization capabilities. A greedy layer-wise unsupervised algorithm was recently proposed to tackle this problem with successful results. In this work we present a methodology for spam detection based on DBNs and evaluate its performance on three widely used datasets. We also compare our method to Support Vector Machines (SVMs) which is the state-of-the-art method for spam filtering in terms of classification performance. Our experiments indicate that using DBNs to filter spam e-mails is a viable methodology, since they achieve similar or even better performance than SVMs on all three datasets.

Video indexing requires the efficient segmentation of the video into scenes. In the method we propose, the video is first segmented into shots and key-frames are extracted using the global k-means clustering algorithm that represent each shot. Then an improved spectral clustering method is applied to cluster the shots into groups based on visual similarity and a label is assigned to each shot according to the group that it belongs to. Next, a method for segmenting the sequence of shot labels is applied, providing the final scene segmentation result. Numerical experiments indicate that the method we propose correctly detects most of the scene boundaries while preserving a good trade off between recall and precision.

We propose a new approach for image segmentation based on a hierarchical and spatially variant mixture model. According to this model, the pixel labels are random variables and a smoothness prior is imposed on them. The main novelty of this work is a new family of smoothness priors for the label probabilities in spatially variant mixture models. These Gauss-Markov random field-based priors allow all their parameters to be estimated in closed form via the maximum a posteriori (MAP) estimation using the expectation-maximization methodology. Thus, it is possible to introduce priors with multiple parameters that adapt to different aspects of the data. Numerical experiments are presented where the proposed MAP algorithms were tested in various image segmentation scenarios. These experiments demonstrate that the proposed segmentation scheme compares favorably to both standard and previous spatially constrained mixture model-based segmentation.

We propose an approach to analyzing functional neuroimages in which 1) regions of neuronal activation are described by a superposition of spatial kernel functions, the parameters of which are estimated from the data and 2) the presence of activation is detected by means of a generalized likelihood ratio test (GLRT). Kernel methods have become a staple of modern machine learning. Herein, we show that these techniques show promise for neuroimage analysis. In an on-off design, we model the spatial activation pattern as a sum of an unknown number of kernel functions of unknown location, amplitude, and/or size. We employ two Bayesian methods of estimating the kernel functions. The first is a maximum a posteriori (MAP) estimation method based on a Reversible-Jump Markov-chain Monte-Carlo (RJMCMC) algorithm that searches for both the appropriate model complexity and parameter values. The second is a relevance vector machine (RVM), a kernel machine that is known to be effective in controlling model complexity (and thus discouraging overfitting). In each method, after estimating the activation pattern, we test for local activation using a GLRT. We evaluate the results using receiver operating characteristic (ROC) curves for simulated neuroimaging data and example results for real fMRI data. We find that, while RVM and RJMCMC both produce good results, RVM requires far less computation time, and thus appears to be the more promising of the two approaches.

In this paper, we present an incremental method for model selection and learning of Gaussian mixtures based on the recently proposed variational Bayes approach. The method adds components to the mixture using a Bayesian splitting test procedure: a component is split into two components and then variational update equations are applied only to the parameters of the two components. As a result, either both components are retained in the model or one of them is found to be redundant and is eliminated from the model. In our approach, the model selection problem is treated locally, in a region of the data space, so we can set more informative priors based on the local data distribution. A modified Bayesian mixture model is presented to implement this approach, along with a learning algorithm that iteratively applies a splitting test on each mixture component. Experimental results and comparisons with two other techniques testify for the adequacy of the proposed approach.

We present a Bayesian method for mixture model training that simultaneously treats the feature selection and the model selection problem. The method is based on the integration of a mixture model formulation that takes into account the saliency of the features and a Bayesian approach to mixture learning that can be used to estimate the number of mixture components. The proposed learning algorithm follows the variational framework and can simultaneously optimize over the number of components, the saliency of the features, and the parameters of the mixture model. Experimental results using high-dimensional artificial and real data illustrate the effectiveness of the method.

In this paper, we propose a class of image restoration algorithms based on the Bayesian approach and a new hierarchical spatially adaptive image prior. The proposed prior has the following two desirable features. First, it models the local image discontinuities in different directions with a model which is continuous valued. Thus, it preserves edges and generalizes the on/off (binary) line process idea used in previous image priors within the context of Markov random fields (MRFs). Second, it is Gaussian in nature and provides estimates that are easy to compute. Using this new hierarchical prior, two restoration algorithms are derived. The first is based on the maximum a posteriori principle and the second on the Bayesian methodology. Numerical experiments are presented that compare the proposed algorithms among themselves and with previous stationary and non stationary MRF-based with line process algorithms. These experiments demonstrate the advantages of the proposed prior.

The probabilistic radial basis function (PRBF) network constitutes a probabilistic version of the RBF network for classification that extends the typical mixture model approach to classification by allowing the sharing of mixture components among all classes. The typical learning method of PRBF for a classification task employs the expectation-maximization (EM) algorithm and depends strongly on the initial parameter values. In this paper, we propose a technique for incremental training of the PRBF network for classification. The proposed algorithm starts with a single component and incrementally adds more components at appropriate positions in the data space. The addition of a new component is based on criteria for detecting a region in the data space that is crucial for the classification task. After the addition of all components, the algorithm splits every component of the network into subcomponents, each one corresponding to a different class. Experimental results using several well-known classification data sets indicate that the incremental method provides solutions of superior classification performance compared to the hierarchical PRBF training method. We also conducted comparative experiments with the support vector machines method and present the obtained results along with a qualitative comparison of the two approaches.

In this paper we propose a new hierarchical non stationary image prior for image restoration. This prior captures the directional edges using a continuous model and regularizes accordingly the restored images. In addition, the corresponding generative graphical model does not contain cycles, thus learning this model is easy and fast. Based on this prior image model, a maximum a posteriori (MAP) estimation algorithm is derived. Numerical experiments are provided that demonstrate the advantages of the proposed non stationary model as compared with algorithms that use stationary models.

An information brokerage environment for effective information structuring, indexing, and retrieval in the health-care administration sector is presented. The system is based on ontology modeling, natural language processing, extensible markup language, semantics analysis, and behavioral description. Semantics-based information acquisition is achieved through the uniform modeling, representation, and handling of domain-specific knowledge, both content-based and procedural. The system has been validated using information located on several repositories in the web and its performance is reported in terms of precision and recall.

In this paper, we propose a new methodology for analysis of microarray images. First, a new gridding algorithm is proposed for determining the individual spots and their borders. Then, a Gaussian mixture model (GMM) approach is presented for the analysis of the individual spot images. The main advantages of the proposed methodology are modeling flexibility and adaptability to the data, which are well-known strengths of GMM. The maximum likelihood and maximum a posteriori approaches are used to estimate the GMM parameters via the expectation maximization algorithm. The proposed approach has the ability to detect and compensate for artifacts that might occur in microarray images. This is accomplished by a model-based criterion that selects the number of the mixture components. We present numerical experiments with artificial and real data where we compare the proposed approach with previous ones and existing software tools for microarray image analysis and demonstrate its advantages.

Gaussian mixture models (GMMs) constitute a well-known type of probabilistic neural networks. One of their many successful applications is in image segmentation, where spatially constrained mixture models have been trained using the expectation-maximization (EM) framework. In this letter, we elaborate on this method and propose a new methodology for the M-step of the EM algorithm that is based on a novel constrained optimization formulation. Numerical experiments using simulated images illustrate the superior performance of our method in terms of the attained maximum value of the objective function and segmentation accuracy compared to previous implementations of this approach.

In this paper the blind image deconvolution (BID) problem is solved using the Bayesian framework. In order to find the parameters of the proposed Bayesian model we present a new generalization of the expectation maximization (EM) algorithm based on the variational approximation methodology. The proposed variational-based algorithm for BID can be derived in closed form and can be implemented in the discrete Fourier domain. Thus, it is very efficient even for very large images. We demonstrate with numerical experiments that the algorithm which was derived by the variational methodology yields promising improvements as compared to previous Bayesian algorithms for BID. Furthermore, the methodology presented here is very general with potential applications to other Bayesian models for this and other imaging problems.

MOTIVATION: This paper studies the problem of discovering subsequences, known as motifs, that are common to a given collection of related biosequences, by proposing a greedy algorithm for learning a mixture of motifs model through likelihood maximization. The approach adds sequentially a new motif to a mixture model by performing a combined scheme of global and local search for appropriately initializing its parameters. In addition, a hierarchical partitioning scheme based on kd-trees is presented for partitioning the input dataset in order to speed-up the global searching procedure. The proposed method compares favorably over the well-known MEME approach and treats successfully several drawbacks of MEME. RESULTS: Experimental results indicate that the algorithm is advantageous in identifying larger groups of motifs characteristic of biological families with significant conservation. In addition, it offers better diagnostic capabilities by building more powerful statistical motif-models with improved classification accuracy.

We propose a generative mixture model classifier that allows for the class conditional densities to be represented by mixtures having certain subsets of their components shared or common among classes. We argue that, when the total number of mixture components is kept fixed, the most efficient classification model is obtained by appropriately determining the sharing of components among class conditional densities. In order to discover such an efficient model, a training method is derived based on the EM algorithm that automatically adjusts component sharing. We provide experimental results with good classification performance.

A three-level hierarchical mixture model for classification is presented that models the following data generation process: (1) the data are generated by a finite number of sources (clusters), and (2) the generation mechanism of each source assumes the existence of individual internal class-labeled sources (subclusters of the external cluster). The model estimates the posterior probability of class membership similar to a mixture of experts classifier. In order to learn the parameters of the model, we have developed a general training approach based on maximum likelihood that results in two efficient training algorithms. Compared to other classification mixture models, the proposed hierarchical model exhibits several advantages and provides improved classification performance as indicated by the experimental results.

We present probabilistic models which are suitable for class conditional density estimation and can be regarded as shared kernel models where sharing means that each kernel may contribute to the estimation of the conditional densities of an classes. We first propose a model that constitutes an adaptation of the classical radial basis function (RBF) network (with full sharing of kernels among classes) where the outputs represent class conditional densities. In the opposite direction is the approach of separate mixtures model where the density of each class is estimated using a separate mixture density (no sharing of kernels among classes). We present a general model that allows for the expression of intermediate cases where the degree of kernel sharing can be specified through an extra model parameter. This general model encompasses both the above mentioned models as special cases. In all proposed models the training process is treated as a maximum likelihood problem and expectation-maximization algorithms have been derived for adjusting the model parameters.

We present a probabilistic neural network model which is suitable for classification problems. This model constitutes an adaptation of the classical RBF network where the outputs represent the class conditional distributions. Since the network outputs correspond to probability density functions, training process is treated as maximum likelihood problem and an expectation-maximization (EM) algorithm is proposed for adjusting the network parameters. Experimental results show that proposed architecture exhibits superior classification performance compared to the classical RBF network.

In cases where the signal-to-noise ratio (SNR) in ECGs is very poor, the correct definition of characteristics such as the isoelectric line and the J-point (beginning of the ST segment) is difficult. Inaccurate definition of those ECG characteristics can lead an automated ischemia detector to an incorrect diagnosis. We propose a method capable of extracting from noisy long duration ECG recordings those ECG characteristics that can be used for myocardial ischemia detection and analysis. We tested the performance of the method using noisy ECGs from the European Society of Cardiology ST-T database (ESC ST-T database). The results were more than satisfactory and the performance of our ischemia detector was improved in all cases. The proposed technique has low computational effort and can be executed in real time.

Partial differential equations (PDEs) with boundary conditions (Dirichlet or Neumann) defined on boundaries with simple geometry have been successfully treated using sigmoidal multilayer perceptrons in previous works. This article deals with the case of complex boundary geometry, where the boundary is determined by a number of points that belong to it and are closely located, so as to offer a reasonable representation. Two networks are employed: a multilayer perceptron and a radial basis function network. The later is used to account for the exact satisfaction of the boundary conditions. The method has been successfully tested on two-dimensional and three-dimensional PDEs and has yielded accurate results.

A general technique is proposed for embedding online clustering algorithms based on competitive learning in a reinforcement learning framework. The basic idea is that the clustering system can be viewed as a reinforcement learning system that learns through reinforcements to follow the clustering strategy we wish to implement. In this sense, the reinforcement guided competitive learning (RGCL) algorithm is proposed that constitutes a reinforcement-based adaptation of learning vector quantization (LVQ) with enhanced clustering capabilities. In addition, we suggest extensions of RGCL and LVQ that are characterized by the property of sustained exploration and significantly improve the performance of those algorithms, as indicated by experimental tests on well-known data sets.

We address the problem of probability density function estimation using a Gaussian mixture model updated with the expectation-maximization (EM) algorithm. To deal with the case of an unknown number of mixing kernels, we define a new measure for Gaussian mixtures, called total kurtosis, which is based on the weighted sample kurtoses of the kernels. This measure provides an indication of how well the Gaussian mixture fits the data. Then we propose a new dynamic algorithm for Gaussian mixture density estimation which monitors the total kurtosis at each step of the EM algorithm in order to decide dynamically on the correct number of kernels and possibly escape from local maxima. We show the potential of our technique in approximating unknown densities through a series of examples with several density estimation problems.

We present a method to solve initial and boundary value problems using artificial neural networks. A trial solution of the differential equation is written as a sum of two parts. The first part satisfies the initial/boundary conditions and contains no adjustable parameters. The second part is constructed so as not to affect the initial/boundary conditions. This part involves a feedforward neural network containing adjustable parameters (the weights). Hence by construction the initial/boundary conditions are satisfied and the network is trained to satisfy the differential equation. The applicability of this approach ranges from single ordinary differential equations (ODE's), to systems of coupled ODE's and also to partial differential equations (PDE's). In this article, we illustrate the method by solving a variety of model problems and present comparisons with solutions obtained using the Galekrkin finite element method for several cases of partial differential equations. With the advent of neuroprocessors and digital signal processors the method becomes particularly interesting due to the expected essential gains in the execution speed.

A neural network classifier is presented, which is based on geometrical fuzzy sets. Starting from the construction of the Voronoi diagram of the training patterns, an aggregation of Voronoi regions is performed leading to the identification of larger regions belonging exclusively to one of the pattern classes. The resulting scheme is a constructive algorithm that defines fuzzy clusters of patterns. Based on observations concerning the grade of membership of the training patterns to the created regions, decision probabilities are computed through which the final classification is performed. Experimental results concerning several classification problems indicate that the proposed method achieves high classification rates and compares favorably with other well-known approaches.

In this paper the Bipolar Random Network is described, which constitutes an extension of the Random Neural Network model and exhibits autoassociative memory capabilities. This model is characterized by the existence of positive and negative nodes and symmetrical behavior of positive and negative signals circulating in the network. The network's ability of acting as autoassociative memory is examined and several techniques are developed concerning storage and reconstruction of patterns. These approaches are either based on properties of the network or constitute adaptations of existing neural network techniques. The performance of the network under the proposed schemes has been investigated through experiments showing very good storage and reconstruction capabilities. Moreover, the scheme exhibiting the best behavior seems to outperform other well-known associative neural network models, achieving capacities that exceed 0.5n where n is the size of the network.

A multiscale method is described in the context of binary Hopfield-type neural networks. The appropriateness of the proposed technique for solving several classes of optimization problems is established by means of the notion of group update which is introduced here and investigated in relation to the properties of multiscaling. The method has been tested in the solution of partitioning and covering problems, for which an original mapping to Hopfield-type neural networks has been developed. Experimental results indicate that the multiscale approach is very effective in exploring the state-space of the problem and providing feasible solutions of acceptable quality, while at the same it offers a significant acceleration.

A technique is developed based on the use of a neural network model for performing information retrieval in a pictorial information system. The neural network provides autoassociative memory operation and allows the retrieval of stored symbolic images using erroneous or incomplete information as input. The network used is based on an adaptation of the random neural network model featuring positive and negative nodes and symmetrical behavior of positive and negative signals. The network architecture considered has hierarchical structure and allows two-level operation during learning and recall. An experimental software prototype, including an efficient graphical interface, has been implemented and tested. The performance of the system has been investigated through experiments under several schemes concerning storage and reconstruction of patterns. These schemes are either based on properties of the random network or constitute adaptations of known neural network techniques.>