Introducing a domain shift, such as a change in language or environment, to a well-trained speech enhancement system can cause severe performance degradation. Most current research assumes that a domain shift has already been detected and focuses on either supervised or unsupervised domain adaptation techniques. Here, we address the problem of automatically detecting when a domain shift has occurred. We present a domain shift detection method based on monitoring the confidence of a network that predicts the quality of enhanced speech. The experimental results show that our method can effectively detect a domain mismatch between the training and test sets.

Speech enhancement and dereverberation approaches based on neural networks are designed to learn a transformation from noisy to clean speech using supervised learning. However, networks trained in this way may fail to effectively handle languages, types of noise, or acoustic environments that were not included in the training data. To tackle this issue, the present study centers around unsupervised domain adaptation, specifically addressing scenarios characterized by substantial domain gaps. In this scenario, we have noisy speech data from the new domain, but the corresponding clean speech data is unavailable. We propose an adaptation method based on domainadversarial training followed by iterative self-training, where the estimated speech is used as pseudo labels, and the target samples are gradually introduced to the network based on their similarity to the source domain. The self-training also utilizes labeled samples from the source domain which are similar to the target domain. The experimental results show that our method effectively mitigates the domain mismatch between the training and test sets, thus outperforming the current baselines.

A classification model is calibrated if its predicted probabilities of outcomes reflect their accuracy. Calibrating neural networks is critical in medical analysis applications where clinical decisions rely upon the predicted probabilities. Most calibration procedures, such as temperature scaling, operate as a post processing step by using holdout validation data. In practice, it is difficult to collect medical image data with correct labels due to the complexity of the medical data and the considerable variability across experts. This study presents a network calibration procedure that is robust to label noise. We draw on the fact that the confusion matrix of the noisy labels can be expressed as the matrix product between the confusion matrix of the clean labels and the label noises. The method is based on estimating the noise level as part of a noise-robust training method. The noise level is then used to estimate the network accuracy required by the calibration procedure. We show that despite the unreliable labels, we can still achieve calibration results that are on a par with the results of a calibration procedure using data with reliable labels.

The integration of multi-document pre-training objectives into language models has resulted in remarkable improvements in multi-document downstream tasks.In this work, we propose extending this idea by pre-training a generic multi-document model from a novel crossdocument question answering pre-training objective.To that end, given a set (or cluster) of topically-related documents, we systematically generate semantically-oriented questions from a salient sentence in one document and challenge the model, during pre-training, to answer these questions while "peeking" into other topically-related documents.In a similar manner, the model is also challenged to recover the sentence from which the question was generated, again while leveraging cross-document information.This novel multidocument QA formulation directs the model to better recover cross-text informational relations, and introduces a natural augmentation that artificially increases the pre-training data.Further, unlike prior multi-document models that focus on either classification or summarization tasks, our pre-training objective formulation enables the model to perform tasks that involve both short text generation (e.g., QA) and long text generation (e.g., summarization).Following this scheme, we pre-train our model -termed QAMDEN -and evaluate its performance across several multi-document tasks, including multi-document QA, summarization, and query-focused summarization, yielding improvements of up to 7%, and significantly outperforms zero-shot GPT-3.5 and GPT-4. 1

Avi Caciularu, Ido Dagan, Jacob Goldberger, Arman Cohan. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

Ori Ernst, Avi Caciularu, Ori Shapira, Ramakanth Pasunuru, Mohit Bansal, Jacob Goldberger, Ido Dagan. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

Aligning sentences in a reference summary with their counterparts in source documents was shown as a useful auxiliary summarization task, notably for generating training data for salience detection.Despite its assessed utility, the alignment step was mostly approached with heuristic unsupervised methods, typically ROUGE-based, and was never independently optimized or evaluated.In this paper, we propose establishing summary-source alignment as an explicit task, while introducing two major novelties: (1) applying it at the more accurate proposition span level, and (2) approaching it as a supervised classification task.To that end, we created a novel training dataset for proposition-level alignment, derived automatically from available summarization evaluation data.In addition, we crowdsourced dev and test datasets, enabling model development and proper evaluation.Utilizing these data, we present a supervised proposition alignment baseline model, showing improved alignmentquality over the unsupervised approach.

In this study we present a mixture of deep experts (MoDE) neural-network architecture for single microphone speech enhancement. Our architecture comprises a set of deep neural networks (DNNs), each of which is an ‘expert’ in a different speech spectral pattern such as phoneme. A gating DNN is responsible for the latent variables which are the weights assigned to each expert’s output given a speech segment. The experts estimate a mask from the noisy input and the final mask is then obtained as a weighted average of the experts’ estimates, with the weights determined by the gating DNN. A soft spectral attenuation, based on the estimated mask, is then applied to enhance the noisy speech signal. As a byproduct, we gain reduction at the complexity in test time. We show that the experts specialization allows better robustness to unfamiliar noise types.1

Error correction codes are an integral part of communication applications, boosting the reliability of transmission. The optimal decoding of transmitted codewords is the maximum likelihood rule, which is NP-hard due to the curse of dimensionality. For practical realizations, sub-optimal decoding algorithms are employed; yet limited theoretical insights prevent one from exploiting the full potential of these algorithms. One such insight is the choice of permutation in permutation decoding. We present a data-driven framework for permutation selection, combining domain knowledge with machine learning concepts such as node embedding and self-attention. Significant and consistent improvements in the bit error rate are introduced for all simulated codes, over the baseline decoders. To the best of the authors' knowledge, this work is the first to leverage the benefits of the neural Transformer networks in physical layer communication systems.

Learning a mapping between word embeddings of two languages given a dictionary is an important problem with several applications.A common mapping approach is using an orthogonal matrix.The Orthogonal Procrustes Analysis (PA) algorithm can be applied to find the optimal orthogonal matrix.This solution restricts the expressiveness of the translation model which may result in sub-optimal translations.We propose a natural extension of the PA algorithm that uses multiple orthogonal translation matrices to model the mapping and derive an algorithm to learn these multiple matrices.We achieve better performance in a bilingual word translation task and a cross lingual word similarity task compared to the single matrix baseline.We also show how multiple matrices can model multiple senses of a word.

In this study we propose a deep clustering algorithm that extends the k-means algorithm. Each cluster is represented by an autoencoder instead of a single centroid vector. Each data point is associated with the autoencoder which yields the minimal reconstruction error. The optimal clustering is found by learning a set of autoencoders that minimize the global reconstruction mean-square error loss. The network architecture is a simplified version of a previous method that is based on mixture-of-experts. The proposed method is evaluated on standard image corpora and performs on par with state-of-the-art methods which are based on much more complicated network architectures.

In speech enhancement, the use of supervised algorithms in the form of deep neural networks (DNNs) has become tremendously popular in recent years. The target function of the DNN (and the associated estimators) is often either a masking function applied to the noisy spectrum, or the clean log-spectrum. In this work, we show that both separate cost functions are unsuitable for dealing with narrowband noise, and propose a new composite estimator in the log-spectrum domain. The new technique relies on a single DNN that outputs both a masking function and an estimated log-spectrum. Both outputs are used for the composite enhancement. The proposed estimator demonstrates superior performance for speech utterances contaminated by additive narrowband noise, while maintaining the enhancement quality of the baseline algorithms for wideband noise.

Man-made scenes are often densely packed, containing numerous objects, often identical, positioned in close proximity. We show that precise object detection in such scenes remains a challenging frontier even for state-of-the-art object detectors. We propose a novel, deep-learning based method for precise object detection, designed for such challenging settings. Our contributions include: (1) A layer for estimating the Jaccard index as a detection quality score; (2) a novel EM merging unit, which uses our quality scores to resolve detection overlap ambiguities; finally, (3) an extensive, annotated data set, SKU-110K, representing packed retail environments, released for training and testing under such extreme settings. Detection tests on SKU-110K, and counting tests on the CARPK and PUCPR+, show our method to outperform existing state-of-the-art with substantial margins.

Hagai Taitelbaum, Gal Chechik, Jacob Goldberger. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Hagai Taitelbaum, Gal Chechik, Jacob Goldberger. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

The information-bottleneck (IB) principle is defined in terms of mutual information. This study defines mutual information between two random variables using the Jensen-Shannon (JS) divergence instead of the standard definition which is based on the Kullback-Leibler (KL) divergence. We reformulate the information-bottleneck principle using the proposed mutual information and apply it to the problem of pairwise clustering. We show that applying IB to clustering tasks using JS divergences instead of KL yields improved results. This indicates that JS-based mutual information has an expressive power at least as the standard KL-based mutual information.

We address the problem of adding new classes to an existing classifier without hurting the original classes, when no access is allowed to any sample from the original classes. This problem arises frequently since models are often shared without their training data, due to privacy and data ownership concerns. We propose an easy-to-use approach that modifies the original classifier by retraining a suitable subset of layers using a linearly-tuned, knowledge-distillation regularization. The set of layers that is tuned depends on the number of new added classes and the number of original classes. We evaluate the proposed method on two standard datasets, first in a language-identification task, then in an image classification setup. In both cases, the method achieves classification accuracy that is almost as good as that obtained by a system trained using unrestricted samples from both the original and new classes.

Self-normalizing discriminative models approximate the normalized probability of a class without having to compute the partition function. In the context of language modeling, this property is particularly appealing as it may significantly reduce run-times due to large word vocabularies. In this study, we provide a comprehensive investigation of language modeling self-normalization. First, we theoretically analyze the inherent self-normalization properties of Noise Contrastive Estimation (NCE) language models. Then, we compare them empirically to softmax-based approaches, which are self-normalized using explicit regularization, and suggest a hybrid model with compelling properties. Finally, we uncover a surprising negative correlation between self-normalization and perplexity across the board, as well as some regularity in the observed errors, which may potentially be used for improving self-normalization algorithms in the future.

In this paper, we present a new control mechanism for LCMV beamforming. Application of the LCMV beamformer to speaker separation tasks requires accurate estimates of its building blocks, e.g. the noise spatial cross-power spectral density (cPSD) matrix and the relative transfer function (RTF) of all sources of interest. An accurate classification of the input frames to various speaker activity patterns can facilitate such an estimation procedure. We propose a DNN-based concurrent speakers detector (CSD) to classify the noisy frames. The CSD, trained in a supervised manner using a DNN, classifies noisy frames into three classes: 1) all speakers are inactive - used for estimating the noise spatial cPSD matrix; 2) a single speaker is active - used for estimating the RTF of the active speaker; and 3) more than one speaker is active - discarded for estimation purposes. Finally, using the estimated blocks, the LCMV beamformer is constructed and applied for extracting the desired speaker from a noisy mixture of speakers.

In this study, we introduce a new approach for learning language models by training them to estimate word-context pointwise mutual information (PMI), and then deriving the desired conditional probabilities from PMI at test time.Specifically, we show that with minor modifications to word2vec's algorithm, we get principled language models that are closely related to the well-established Noise Contrastive Estimation (NCE) based language models.A compelling aspect of our approach is that our models are trained with the same simple negative sampling objective function that is commonly used in word2vec to learn word embeddings.

Context representations are central to various NLP tasks, such as word sense disambiguation, named entity recognition, coreference resolution, and many more.In this work we present a neural model for efficiently learning a generic context embedding function from large corpora, using bidirectional LSTM.With a very simple application of our context representations, we manage to surpass or nearly reach state-of-the-art results on sentence completion, lexical substitution and word sense disambiguation tasks, while substantially outperforming the popular context representation of averaged word embeddings.We release our code and pretrained models, suggesting they could be useful in a wide variety of NLP tasks.

In this study we address the problem of training a neural network based on data with unreliable labels. We introduce an extra noise layer by assuming that the observed labels were created from the true labels by passing through a noisy channel whose parameters are unknown. We propose a method that simultaneously learns both the neural network parameters and the noise distribution. The proposed method is compared to standard back-propagation neural-network training that ignores the existence of wrong labels. The improved classification performance of the method is illustrated on several standard classification tasks. In particular we show that in some cases our approach can be beneficial even when the labels are set manually and assumed to be error-free.

In this study we address the problem of integrating information from several experts with unknown levels of expertise. In the usual setup each expert expresses her opinion by choosing one of the options. Here we assume that each expert provides her opinion in a soft manner via a distribution on the possible options. The goal is to find the reliability level of each expert and to optimally integrate their information. We develop an estimation algorithm which is an instance of the EM algorithm and an efficiently computed approximation of the E-step. Finally we present simulations that demonstrate the improved performance of the proposed approach.

In this paper, we present a single-microphone speech enhancement algorithm. A hybrid approach is proposed merging the generative mixture of Gaussians (MoG) model and the discriminative deep neural network (DNN). The proposed algorithm is executed in two phases, the training phase, which does not recur, and the test phase. First, the noise-free speech log-power spectral density is modeled as an MoG, representing the phoneme-based diversity in the speech signal. A DNN is then trained with phoneme labeled database of clean speech signals for phoneme classification with mel-frequency cepstral coefficients as the input features. In the test phase, a noisy utterance of an untrained speech is processed. Given the phoneme classification results of the noisy speech utterance, a speech presence probability (SPP) is obtained using both the generative and discriminative models. SPP-controlled attenuation is then applied to the noisy speech while simultaneously, the noise estimate is updated. The discriminative DNN maintains the continuity of the speech and the generative phoneme-based MoG preserves the speech spectral structure. Extensive experimental study using real speech and noise signals is provided. We also compare the proposed algorithm with alternative speech enhancement algorithms. We show that we obtain a significant improvement over previous methods in terms of speech quality measures. Finally, we analyze the contribution of all components of the proposed algorithm indicating their combined importance.

Classification of clustered breast microcalcifications into benign and malignant categories is an extremely challenging task for computerized algorithms and expert radiologists alike. In this paper we apply a multi-view-classifier for the task. We describe a two-step classification method that is based on a view-level decision, implemented by a logistic regression classifier, followed by a stochastic combination of the two view-level indications into a single benign or malignant decision. The proposed method was evaluated on a large number of cases from a standardized digital database for screening mammography (DDSM). Experimental results demonstrate the advantage of the proposed multi-view classification algorithm that automatically learns the best way to combine the views.

In this paper, we apply efficient implementations of integer linear programming to the problem of image segmentation. The image is first grouped into superpixels and then local information is extracted for each pair of spatially adjacent superpixels. Given local scores on a map of several hundred superpixels, we use correlation clustering to find the global segmentation that is most consistent with the local evidence. We show that, although correlation clustering is known to be NP-hard, finding the exact global solution is still feasible by breaking the segmentation problem down into subproblems. Each such sub-problem can be viewed as an automatically detected image part. We can further accelerate the process by using the cutting-plane method, which provides a hierarchical structure of the segmentations. The efficiency and improved performance of the proposed method is compared to several state-of-the-art methods and demonstrated on several standard segmentation data sets.

Massive knowledge resources, such as Wikidata, can provide valuable informa-tion for lexical inference, especially for proper-names. Prior resource-based ap-proaches typically select the subset of each resource’s relations which are relevant for a particular given task. The selection process is done manually, limiting these approaches to smaller resources such as WordNet, which lacks coverage of proper-names and recent terminology. This paper presents a supervised framework for auto-matically selecting an optimized subset of resource relations for a given target infer-ence task. Our approach enables the use of large-scale knowledge resources, thus providing a rich source of high-precision inferences over proper-names.1 1

Context representations are a key element in distributional models of word meaning. In contrast to typical representations based on neighboring words, a recently proposed ap-proach suggests to represent a context of a tar-get word by a substitute vector, comprising the potential fillers for the target word slot in that context. In this work we first propose a vari-ant of substitute vectors, which we find partic-ularly suitable for measuring context similar-ity. Then, we propose a novel model for rep-resenting word meaning in context based on this context representation. Our model outper-forms state-of-the-art results on lexical substi-tution tasks in an unsupervised setting. 1

Entailment rules between predicates are fundamental to many semantic-inference applications. Consequently, learning such rules has been an active field of research in recent years. Methods for learning entailment rules between predicates that take into account dependencies between different rules (e.g., entailment is a transitive relation) have been shown to improve rule quality, but suffer from scalability issues, that is, the number of predicates handled is often quite small. In this article, we present methods for learning transitive graphs that contain tens of thousands of nodes, where nodes represent predicates and edges correspond to entailment rules (termed entailment graphs). Our methods are able to scale to a large number of predicates by exploiting structural properties of entailment graphs such as the fact that they exhibit a “tree-like” property. We apply our methods on two data sets and demonstrate that our methods find high-quality solutions faster than methods proposed in the past, and moreover our methods for the first time scale to large graphs containing 20,000 nodes and more than 100,000 edges.

Open IE methods extract structured propositions from text.However, these propositions are neither consolidated nor generalized, and querying them may lead to insufficient or redundant information.This work suggests an approach to organize open IE propositions using entailment graphs.The entailment relation unifies equivalent propositions and induces a specific-to-general structure.We create a large dataset of gold-standard proposition entailment graphs, and provide a novel algorithm for automatically constructing them.Our analysis shows that predicate entailment is extremely context-sensitive, and that current lexical-semantic resources do not capture many of the lexical inferences induced by proposition entailment.

Most traditional distributional similarity models fail to capture syntagmatic patterns that group together multiple word features within the same joint context.In this work we introduce a novel generic distributional similarity scheme under which the power of probabilistic models can be leveraged to effectively model joint contexts.Based on this scheme, we implement a concrete model which utilizes probabilistic n-gram language models.Our evaluations suggest that this model is particularly wellsuited for measuring similarity for verbs, which are known to exhibit richer syntagmatic patterns, while maintaining comparable or better performance with respect to competitive baselines for nouns.Following this, we propose our scheme as a framework for future semantic similarity models leveraging the substantial body of work that exists in probabilistic language modeling.

We propose a new detection algorithm for MIMO communication systems employing a two-dimensional marginal of the Gaussian approximation of the exact discrete distribution of the transmitted data given the received data. From the 2D distributions we derive one-dimensional marginals by averaging all the 2D joint distributions related to a single input symbol. We prove that this strategy to obtain a 1D distribution from a set of not necessarily consistent 2D distributions is optimal (for a specified criterion). The improved performance of the proposed algorithm is demonstrated on several instances of the problem of MIMO detection.

This paper proposes an efficient high-performance detection algorithm for MIMO communication systems that is based on a sequence of optimal tree approximations of the Gaussian density of the unconstrained linear system. The finite-set constraint is then applied to obtain a cycle-free discrete distribution that is suitable for message-passing algorithms. The proposed GTA-SIC algorithm is iterative and is based on first decoding the most reliable symbol, then canceling its contribution and applying the message-passing decoding to the smaller system. The computational complexity of the proposed GTA-SIC algorithm and the MMSE-SIC are comparable. The significantly improved MIMO decoding performance of the algorithm proposed here compared to lattice-reduction aided MMSE-SIC is demonstrated on several examples of large MIMO systems with high-order QAM constellations.

Identifying entailment relations between predicates is an important part of applied semantic inference. In this article we propose a global inference algorithm that learns such entailment rules. First, we define a graph structure over predicates that represents entailment relations as directed edges. Then, we use a global transitivity constraint on the graph to learn the optimal set of edges, formulating the optimization problem as an Integer Linear Program. The algorithm is applied in a setting where, given a target concept, the algorithm learns on the fly all entailment rules between predicates that co-occur with this concept. Results show that our global algorithm improves performance over baseline algorithms by more than 10%.

We present a method for combining several segmentations of an image into a single one that in some sense is the average segmentation in order to achieve a more reliable and accurate segmentation result. The goal is to find a point in the "space of segmentations" which is close to all the individual segmentations. We present an algorithm for segmentation averaging. The image is first oversegmented into superpixels. Next, each segmentation is projected onto the superpixel map. An instance of the EM algorithm combined with integer linear programming is applied on the set of binary merging decisions of neighboring superpixels to obtain the average segmentation. Apart from segmentation averaging, the algorithm also reports the reliability of each segmentation. The performance of the proposed algorithm is demonstrated on manually annotated images from the Berkeley segmentation data set and on the results of automatic segmentation algorithms.

This paper proposes a new detection algorithm for MIMO communication systems employing high-order QAM constellations. The factor graph that corresponds to this problem is very loopy; in fact, it is a complete graph. Hence, a straightforward application of the Belief Propagation (BP) algorithm yields very poor results. Our algorithm is based on an optimal tree approximation of the Gaussian density of the unconstrained linear system. The finite-set constraint is then applied to obtain a cycle-free discrete distribution. Simulation results show that even though the approximation is not directly applied to the exact discrete distribution, applying the BP algorithm to the cycle-free factor graph outperforms current methods in terms of both performance and complexity. The improved performance of the proposed algorithm is demonstrated on the problem of MIMO detection.

In this study we present an efficient image categorization and retrieval system applied to medical image databases, in particular large radiograph archives. The methodology is based on local patch representation of the image content, using a "bag of visual words" approach. We explore the effects of various parameters on system performance, and show best results using dense sampling of simple features with spatial content, and a nonlinear kernel-based support vector machine (SVM) classifier. In a recent international competition the system was ranked first in discriminating orientation and body regions in X-ray images. In addition to organ-level discrimination, we show an application to pathology-level categorization of chest X-ray data, the most popular examination in radiology. The system discriminates between healthy and pathological cases, and is also shown to successfully identify specific pathologies in a set of chest radiographs taken from a routine hospital examination. This is a first step towards similarity-based categorization, which has a major clinical implications for computer-assisted diagnostics.

This paper presents a procedure for automatic extraction and segmentation of a class-specific object (or region) by learning class-specific boundaries. We describe and evaluate the method with a specific focus on the detection of lesion regions in uterine cervix images. The watershed segmentation map of the input image is modeled using a Markov random field (MRF) in which watershed regions correspond to binary random variables indicating whether the region is part of the lesion tissue or not. The local pairwise factors on the arcs of the watershed map indicate whether the arc is part of the object boundary. The factors are based on supervised learning of a visual word distribution. The final lesion region segmentation is obtained using a loopy belief propagation applied to the watershed arc-level MRF. Experimental results on real data show state-of-the-art segmentation results on this very challenging task that, if necessary, can be interactively enhanced.

Least squares (LS) fitting is one of the most fundamental techniques in science and engineering. It is used to estimate parameters from multiple noisy observations. In many problems the parameters are known a-priori to be bounded integer valued, or they come from a finite set of values on an arbitrary finite lattice. In this case finding the closest vector becomes NP-Hard problem. In this paper we propose a novel algorithm, the Tomographic Least Squares Decoder (TLSD), that not only solves the ILS problem, better than other sub-optimal techniques, but also is capable of providing the a-posteriori probability distribution for each element in the solution vector. The algorithm is based on reconstruction of the vector from multiple two-dimensional projections. The projections are carefully chosen to provide low computational complexity. Unlike other iterative techniques, such as the belief propagation, the proposed algorithm has ensured convergence. We also provide simulated experiments comparing the algorithm to other sub-optimal algorithms.

This paper proposes a new algorithm for the linear least squares problem where the unknown variables are constrained to be in a finite set. The factor graph that corresponds to this problem is very loopy; in fact, it is a complete graph. Hence, applying the Belief Propagation (BP) algorithm yields very poor results. The algorithm described here is based on an optimal tree approximation of the Gaussian density of the unconstrained linear system. It is shown that even though the approximation is not directly applied to the exact discrete distribution, applying the BP algorithm to the modified factor graph outperforms current methods in terms of both performance and complexity. The improved performance of the proposed algorithm is demonstrated on the problem of MIMO detection.

In this letter, we address the problem of urban-area extraction by using a feature-free image representation concept known as ldquoVisual Words.rdquo This method is based on building a ldquodictionaryrdquo of small patches, some of which appear mainly in urban areas. The proposed algorithm is based on a new pixel-level variant of visual words and is based on three parts: building a visual dictionary, learning urban words from labeled images, and detecting urban regions in a new image. Using normalized patches makes the method more robust to changes in illumination during acquisition time. The improved performance of the method is demonstrated on real satellite images from three different sensors: LANDSAT, SPOT, and IKONOS. To assess the robustness of our method, the learning and testing procedures were carried out on different and independent images.

Today, satellite and aerial images are the major source of information for landcover classification. An important usage of remotely sensed data is extracting urban regions to update GIS databases. However, in most cases human resources do not give a sufficient solution to the problem, since it can not entirely process such an enormous amount of remotely sensed data. In addition, most of the automatic methods for urban extraction that exist today are sensitive to atmospheric and radiometric parameters of the acquired image. In this paper we address the problem of urban areas extraction by using a visual representation concept known as "Bag of Words". This method, originally developed for text retrieval approaches, has been successfully applied to scenery image classification tasks. In this paper we introduce the "Bag of Words" approach into analysis of aerial and satellite images. Due to the fact that we implement a normalization process in our method, it is robust to changes in atmospheric conditions during acquisition time. The improved performance of the proposed method is demonstrated on IKONOS images. To assess the robustness of our method, the learning and testing procedures are performed on two different and independent images.

In this paper we introduce the MeanNN approach for estimation of main information theoretic measures such as differential entropy, mutual information and divergence. As opposed to other nonparametric approaches the MeanNN results in smooth differentiable functions of the data samples with clear geometrical interpretation. Then we apply the proposed estimators to the ICA problem and obtain a smooth expression for the mutual information that can be analytically optimized by gradient descent methods. The improved performance on the proposed ICA algorithm is demonstrated on standard tests in comparison with state-of-the-art techniques.

Mixture of Gaussians (MoG) model is a useful tool in statistical learning. In many learning processes that are based on mixture models, computational requirements are very demanding due to the large number of components involved in the model. We propose a novel algorithm for learning a simplified representation of a Gaussian mixture, that is based on the Unscented Transform which was introduced for filtering nonlinear dynamical systems. The superiority of the proposed method is validated on both simulation experiments and categorization of a real image database. The proposed categorization methodology is based on modeling each image using a Gaussian mixture model. A category model is obtained by learning a simplified mixture model from all the images in the category.

Low-density parity-check (LDPC) codes are usually decoded by running an iterative belief-propagation algorithm over the factor graph of the code. In the traditional message-passing schedule, in each iteration all the variable nodes, and subsequently all the factor nodes, pass new messages to their neighbors. Recently several studies show that serial scheduling, in which messages are generated using the latest available information, significantly improves the convergence speed in terms of number of iterations. It was observed experimentally in several studies that the serial schedule converges in exactly half the number of iterations compared to the standard parallel schedule. In this correspondence we provide a theoretical motivation for this observation by proving it for single-path graphs.

This work focuses on a general framework for image categorization, classification and retrieval that may be appropriate for medical image archives. The proposed methodology is comprised of a continuous and probabilistic image representation scheme using Gaussian mixture modeling (MoG) along with information-theoretic image matching measures (KL). A category model is obtained by learning a reduced model from all the images in the category. We propose a novel algorithm for learning a reduced representation of a MoG, that is based on the unscented-transform. The superiority of the proposed method is validated on both simulation experiments and categorization of a real medical image database.

In this paper we propose a new segmentation algorithm which combines patch-based information with edge cues under a probabilistic framework. We use a mixture of multiple Gaussians for building the statistical model with color and spatial features, and we incorporate edge information based on texture, color and brightness differences into the EM algorithm. We evaluate our results qualitatively and quantitatively on a large data-set of natural images and compare our results to other state-of-the-art methods.

In this study we apply a variant of a recently proposed linear subspace method, the neighbourhood component analysis (NCA), to the task of hyperspectral classification. The NCA algorithm explicitly utilizes the classification performance criterion to obtain the optimal linear projection. NCA assumes nothing about the form of the each class and the shape of the separating surfaces. Experimental studies were conducted on the basis of hyperspectral images acquired by two sensors: the airborne visible/infrared imaging spectroradiometer (AVIRIS) and AISA-EAGLE. Experimental results confirm the significant superiority of the NCA classifier in the context of hyperspectral data classification over methodologies that were previously suggested.

Conventionally, in each low-density parity-check (LDPC) decoding iteration all the variable nodes and subsequently all the check nodes send messages to their neighbors (flooding schedule). An alternative, more efficient, approach is to update the nodes' messages serially (serial schedule). A theoretical analysis of serial message passing decoding schedules is presented. In particular, the evolution of the computation tree under serial scheduling is analyzed. It shows that the tree grows twice as fast in comparison to the flooding schedule's one, indicating that the serial schedule propagates information twice as fast in the code's underlying graph. Furthermore, an asymptotic analysis of the serial schedule's convergence rate is done using the density evolution (DE) algorithm. Applied to various ensembles of LDPC codes, it shows that for long codes the serial schedule is expected to converge in half the number of iterations compared to the standard flooding schedule, when working near the ensemble's threshold. This observation is generally proved for the binary erasure channel (BEC) under some natural assumptions. Finally, an accompanying concentration theorem is proved.

We describe an algorithm for context-based segmentation of visual data. New frames in an image sequence (video) are segmented based on the prior segmentation of earlier frames in the sequence. The segmentation is performed by adapting a probabilistic model learned on previous frames, according to the content of the new frame. We utilize the maximum a posteriori version of the EM algorithm to segment the new image. The Gaussian mixture distribution that is used to model the current frame is transformed into a conjugate-prior distribution for the parametric model describing the segmentation of the new frame. This semisupervised method improves the segmentation quality and consistency and enables a propagation of segments along the segmented images. The performance of the proposed approach is illustrated on both simulated and real image data.

In this paper, we combine discrete and continuous image models with information-theoretic-based criteria for unsupervised hierarchical image-set clustering. The continuous image modeling is based on mixture of Gaussian densities. The unsupervised image-set clustering is based on a generalized version of a recently introduced information-theoretic principle, the information bottleneck principle. Images are clustered such that the mutual information between the clusters and the image content is maximally preserved. Experimental results demonstrate the performance of the proposed framework for image clustering on a large image set. Information theoretic tools are used to evaluate cluster quality. Particular emphasis is placed on the application of the clustering for efficient image search and retrieval.

An automated algorithm for tissue segmentation of noisy, low-contrast magnetic resonance (MR) images of the brain is presented. A mixture model composed of a large number of Gaussians is used to represent the brain image. Each tissue is represented by a large number of Gaussian components to capture the complex tissue spatial layout. The intensity of a tissue is considered a global feature and is incorporated into the model through tying of all the related Gaussian parameters. The expectation-maximization (EM) algorithm is utilized to learn the parameter-tied, constrained Gaussian mixture model. An elaborate initialization scheme is suggested to link the set of Gaussians per tissue type, such that each Gaussian in the set has similar intensity characteristics with minimal overlapping spatial supports. Segmentation of the brain image is achieved by the affiliation of each voxel to the component of the model that maximized the a posteriori probability. The presented algorithm is used to segment three-dimensional, T1-weighted, simulated and real MR images of the brain into three different tissues, under varying noise conditions. Results are compared with state-of-the-art algorithms in the literature. The algorithm does not use an atlas for initialization or parameter learning. Registration processes are therefore not required and the applicability of the framework can be extended to diseased brains and neonatal brains.

This paper proposes a dissimilarity measure between two Gaussian mixture models (GMM). Computing a distance measure between two GMMs that were learned from speech segments is a key element in speaker verification, speaker segmentation and many other related applications. A natural measure between two distributions is the Kullback-Leibler divergence. However, it cannot be analytically computed in the case of GMM. We propose an accurate and efficiently computed approximation of the KL-divergence. The method is based on the unscented transform which is usually used to obtain a better alternative to the extended Kalman filter. The suggested distance is evaluated in an experimental setup of speakers data-set. The experimental results indicate that our proposed approximations outperform previously suggested methods. 1.

In this paper we propose an efficient algorithm for reducing a large mixture of Gaussians into a smaller mixture while still preserv- ing the component structure of the original model; this is achieved by clustering (grouping) the components. The method minimizes a new, easily computed distance measure between two Gaussian mixtures that can be motivated from a suitable stochastic model and the iterations of the algorithm use only the model parameters, avoiding the need for explicit resampling of datapoints. We demon- strate the method by performing hierarchical clustering of scenery images and handwritten digits.

In this paper we propose a novel method for learning a Mahalanobis distance measure to be used in the KNN classification algorithm. The algorithm directly maximizes a stochastic variant of the leave-one-out KNN score on the training set. It can also learn a low-dimensional lin- ear embedding of labeled data that can be used for data visualization and fast classification. Unlike other methods, our classification model is non-parametric, making no assumptions about the shape of the class distributions or the boundaries between them. The performance of the method is demonstrated on several data sets, both for metric learning and linear dimensionality reduction.

In this paper, we describe a statistical video representation and modeling scheme. Video representation schemes are needed to segment a video stream into meaningful video-objects, useful for later indexing and retrieval applications. In the proposed methodology, unsupervised clustering via Gaussian mixture modeling extracts coherent space-time regions in feature space, and corresponding coherent segments (video-regions) in the video content. A key feature of the system is the analysis of video input as a single entity as opposed to a sequence of separate frames. Space and time are treated uniformly. The probabilistic space-time video representation scheme is extended to a piecewise GMM framework in which a succession of GMMs are extracted for the video sequence, instead of a single global model for the entire sequence. The piecewise GMM framework allows for the analysis of extended video sequences and the description of nonlinear, nonconvex motion patterns. The extracted space-time regions allow for the detection and recognition of video events. Results of segmenting video content into static versus dynamic video regions and video content editing are presented.

We present two new methods for approximating the Kullback-Liebler (KL) divergence between two mixtures of Gaussians. The first method is based on matching between the Gaussian elements of the two Gaussian mixture densities. The second method is based on the unscented transform. The proposed methods are utilized for image retrieval tasks. Continuous probabilistic image modeling based on mixtures of Gaussians together with KL measure for image similarity, can be used for image retrieval tasks with remarkable performance. The efficiency and the performance of the KL approximation methods proposed are demonstrated on both simulated data and real image data sets. The experimental results indicate that our proposed approximations outperform previously suggested methods.

We present a method for unsupervised clustering of image databases. The method is based on a recently introduced information-theoretic principle, the information bottleneck (IB) principle. Image archives are clustered such that the mutual information between the clusters and the image content is maximally preserved. The IB principle is applied to both discrete and continuous image representations, using discrete image histograms and probabilistic continuous image modeling based on mixture of Gaussian densities, respectively. Experimental results demonstrate the performance of the proposed method for image clustering on a large image database. Several clustering algorithms derived from the IB principle are explored and compared.

We address the problem of global registration between multiple d-dimensional point patterns with a given correspondence. The actual overlapping is not necessarily between pairs. Instead, it can be between any number of patterns. It is assumed that each pattern is a portion of an image of an unobserved object under a distinct rigid transformation. We derive an iterative solution for the problem of global registration of the patterns in order to reconstruct the original object. Our solution is based on the EM algorithm and it generalizes the well known solutions for the two-pattern case. We also suggest a very efficient method to implement the proposed algorithm. Experimental results demonstrate the improved performance of the proposed method.

A major limitation of hidden Markov model (HMM) based automatic speech recognition is the inherent assumption that successive observations within a state are independent and identically distributed (i.i.d.). The i.i.d. assumption is reasonable for some of the states (e.g., a state that corresponds to a steady state vowel). However, most states clearly violate this assumption (e.g., states corresponding to vowel-consonant transition, diphthongs, etc.) and are in fact characterized by a highly correlated and nonstationary speech signal. Previous alternative models have been proposed, that attempt to describe the dynamics of the signal within a phonetic unit. The new approach is generally known by the name segmental modeling, since the speech signal is modeled on a segment level base and not on a frame base (such as HMM). We propose a family of new segmental models that are composed of two elements. The first element is a nonparametric representation of the mean and variance trajectories, and the second is some parameterized transformation (e.g., random shift) of the trajectory that is global to the entire segment. The new model is in fact a continuous mixture of segment trajectories. We present recognition results on a large vocabulary task, and compare the model to alternative segment models on a triphone recognition task.

We present the concept of a scaled random segmental model, which aims to overcome the modeling problem created by the fact that segment realizations of the same phonetic unit differ in length. In the scaled model the variance of the random mean trajectory is inversely proportional to the segment length. The scaled model enables a Baum-Welch type parameter reestimation, unlike the previously suggested, non-scaled models, that require more complicated iterative estimation procedures. In experiments we have conducted with phoneme classification, it was found that the scaled model shows improved performance compared to the non-scaled model.