The growing sophistication of speech generated by Artificial Intelligence (AI) has introduced new challenges in audio deepfake detection. Text-to-speech (TTS) and voice conversion (VC) technologies can now produce convincing synthetic speech with high quality and intelligibility. This poses a serious threat to voice biometric security systems, such as automatic speaker recognition. It also increases the risks associated to the spread of spoken disinformation, where synthetic voices can be used to disseminate malicious content. In this study, we conduct an analysis of real and synthetic speech at phonetic and word levels. For that, a parallel dataset comprising real and synthetic speech signals were developed based on a subset of the LibriSpeech ASR corpus. Synthetic speech samples were generated using two TTS and one VC systems: Coqui TTS, VITS TTS, and StarGANv2 VC. We adopted HuBERT, a self-supervised speech model, to extract speech embeddings. The motivation for using this model stems from its ability to recognize sound units corresponding to the so-called pseudo phonemes. Our analysis is based on the KL divergence (KLD) between the distributions of synthetic and real phonemes, which allowed us to rank synthetic phonemes based on their alignment with their real counterpart. We also trained several classifiers per phoneme to distinguish between real and synthetic samples. We then compute the correlations between KLD and accuracies per phoneme. Besides showing a list of phonemes that are more discriminative, our findings suggest that vowels correlate better with the classifiers’ performance, suggesting that the KLD can be an indicator of the most distinguishable phonemes for deepfake detection.

Accurate crack segmentation in concrete transportation infrastructures is critical for ensuring structural integrity and facilitating timely maintenance interventions. This paper presents CrackSight, an end-to-end deep learning model for precise crack segmentation across varying observational ranges and extremely complex backgrounds. CrackSight seamlessly integrates crack detection and segmentation through two branches. The Detection Feature Extraction Branch (DFEB) provides global context for crack localization in complex backgrounds or at far observation ranges. It guides the segmentation model to focus on regions with the highest crack-prone potential. The segmentation branch leverages the fusion of multi-scale feature maps using dilated convolutions, allowing to capture subtle and complex crack patterns. The branch also incorporates the Dual-Attention Linear Focus Mechanism (DALFM) enhancing crack segmentation through saliency-driven improvements. Finally, CrackSight uses a novel hybrid contextual loss, which dynamically compensates for class imbalance and enhance crack discrimination against complex backgrounds. Our model is also lightweight and can be run in resource-constrained environments, making it suitable for real-world inspection using mobile platforms. Our results demonstrate that it significantly improves segmentation accuracy, setting a new benchmark for crack segmentation. The dataset and additional resources are available on GitHub.

Visual inspection is an important process for maintaining bridges in road transportation systems, and preventing catastrophic events and tragedies. In this process, accurate and automatic concrete defect classification and detection are major components to ensure early identification of any issue that can compromise the bridge safety and integrity. While a tremendous body of research has been proposed in the last decades for addressing these problems, the advent of deep learning unleashed huge opportunities for building more accurate and efficient methods. Our aim in this survey is to study the recent progress of vision-based concrete bridge defect classification and detection in the deep learning era. Our review encompasses major aspects underlying typical frameworks, which include concrete defect taxonomy, public datasets and evaluation metrics. We provide also a taxonomy of deep-learning-based classification and detection algorithms with a detailed discussion of their advantages and limitations. We also benchmark baseline models for classification and detection, using two popular datasets. We finally discuss important challenges of concrete defect classification and detection, and promising research avenues to build better models and integrate them in real-world visual inspection systems, which warrant further scientific investigation.

We present a unified statistical model for multivariate and multi-modal texture representation. This model is based on the formalism of finite mixtures of multivariate generalized Gaussians (MoMGG) which enables a compact and accurate representation of joint statistics of different sub-bands of multireslotion texture transform. This representation expresses correlation between sub-bands at different scales and orientations, and also between adjacent locations within the same subbands, providing a precise description of the texture layout. It enables also to combine different multi-scale transforms to build a richer and more representative texture signature. We successfully tested the model on traditional texture transforms such as wavelets and contourlets. Experiments on color-texture image retrieval have demonstrated the performance of our approach comparatively to state-of-art methods.

We propose a learning-based approach for automatic detection of fabric defects. Our approach is based on a statistical representation of fabric patterns using the redundant contourlet transform (RCT). The distribution of the RCT coefficients are modeled using a finite mixture of generalized Gaussians (MoGG), which constitute statistical signatures distinguishing between defective and defect-free fabrics. In addition to being compact and fast to compute, these signatures enable accurate localization of defects. Our defect detection system is based on three main steps. In the first step, a preprocessing is applied for detecting basic pattern size for image decomposition and signature calculation. In the second step, labeled fabric samples are used to train a Bayes classifier (BC) to discriminate between defect-free and defective fabrics. Finally, defects are detected during image inspection by testing local patches using the learned BC. Our approach can deal with multiple types of textile fabrics, from simple to more complex ones. Experiments on the TILDA database have demonstrated that our method yields better results compared with recent state-of-the-art methods.

We present a new statistical approach combining temporal and spatial information for robust online background subtraction (BS) in videos. Temporal information is modeled by coupling finite mixtures of generalized Gaussian distributions with foreground/background co-occurrence analysis. Spatial information is modeled by combining multiscale inter-frame correlation analysis and histogram matching. We propose an online algorithm that efficiently fuses both information to cope with several BS challenges, such as cast shadows, illumination changes, and various complex background dynamics. In addition, global video information is used through a displacement measuring technique to deal with pan-tilt-zoom camera effects. Experiments with comparison with recent state-of-the-art methods have been conducted on standard data sets. Obtained results have shown that our approach surpasses several state-of-the-art methods on the aforementioned challenges while maintaining comparable computational time.

The exponential use of digital cameras has raised a new problem: how to store/retrieve images/albums in very large photo databases that correspond to special events. In this paper, we propose a new probabilistic graphical model (PGM) to recognize events in photo albums stored by users. The PGM combines high-level image features consisting of scenes and objects detected in images. To consider the discriminative power of features, our model integrates the object/scene relevance for more precise prediction of semantic events in photo albums. Experimental results carried out on the challenging PEC dataset with 807 photo albums are presented.

This paper presents a new statistical approach combining temporal and spatial information for robust background subtraction (BS) in videos. Temporal information couples finite mixtures of generalized Gaussians (MoGG) and temporal cooccurrence analysis of forground/background data. Spatial information combines multi-scale correlation analysis and histogram matching. Our approach fuses both information to perform efficient BS in the presence of shadows, illumination changes and various complex background dynamics. Comparison with recent state-of-the-art methods on standard datasets has demonstrated the performance of our method in terms of precision and computational efficiency.

An approach is proposed for incorporating feature relevance in mutinomial kernel logistic regression (MKLR) for classification. MKLR is a supervised classification method designed for separating classes with non-linear boundaries. However, it assumes all features are equally important, which may decrease classification performance when dealing with high-dimensional or noisy data. We propose a feature weighting algorithm for MKLR which automatically tunes features contribution according to their relevance for classification and reduces data over-fitting. The proposed algorithm produces more interpretable models and is more generalizable than MKLR, Kernel-SVM and LASSO methods. Application to simulated data and video action classification has provided very promising results compared to the aforementioned classification methods.

In this paper, we develop a new framework for contourlet-based statistical modeling using finite Mixtures of Generalized Gaussian distributions ( MoGG). On the one hand, given the rich directional information provided by the contourlet transform (CT), we propose to use a redundant version of the CT, which describes texture structures more accurately. On the other hand, we use MoGG modeling of contourlet coefficients distribution, which allows for precise capturing of a wide range of histogram shapes and provides better description and discrimination of texture than single probability density functions (pdfs). Moreover, we propose three applications for the proposed approach, namely: (1) texture retrieval, (2) fabric texture defect detection, and 3) infrared (IR) face recognition. We compare two implementations of the CT: standard CT ( SCT) and redundant CT ( RCT). We show that the proposed approach yields better results in the applications studied compared to recent state-of-the-art methods.

This paper addresses statistical-based texture modeling using wavelets. We propose a new approach to represent the marginal distribution of the wavelet coefficients using finite mixtures of generalized Gaussian (MoGG) distributions. The MoGG captures a wide range of histogram shapes, which provides better description and discrimination of texture than using single probability density functions (pdf's), as proposed by recent state-of-the-art approaches. Moreover, we propose a model similarity measure based on Kullback-Leibler divergence (KLD) approximation using Monte Carlo sampling methods. Through experiments on two popular texture data sets, we show that our approach yields significant performance improvements for texture discrimination and retrieval, as compared with recent methods of statistical-based wavelet modeling.

In this paper, we address the texture retrieval problem using wavelet distribution. We propose a new statistical scheme to represent the marginal distribution of the wavelet coefficients using a mixture of generalized Gaussian distributions (MoGG). The MoGG allows to capture a wide range of histogram shapes, which provides a better description of texture and enhances texture discrimination. We propose a similarity measurement based on Kullback-Leibler distance (KLD), which is calculated using MCMC Metropolis-Hastings sampling algorithm. We show that our approach yields better texture retrieval results than previous methods using only a single probability density function (pdf) for wavelet representation, or texture energy distribution.

In this letter, we propose a clustering model that efficiently mitigates image and video under/over-segmentation by combining generalized Gaussian mixture modeling and feature selection. The model has flexibility to accurately represent heavy-tailed image/video histograms, while automatically discarding uninformative features, leading to better discrimination and localization of regions in high-dimensional spaces. Experimental results on a database of real-world images and videos showed us the effectiveness of the proposed approach.

In this paper, we propose an effective approach for tracking distribution of objects. The approach uses a competition between a tracked objet and background distributions using active contours. Only the segmentation of the object in the first frame is required for initialization. We evolve the object contour by assigning pixels in a fashion that maximizes the likelihood of the object versus the background. This maximization is implemented using an EM-like algorithm, which evolves the object contour exactly to its boundaries, and adapts the parameters of the object and background distributions.

Image segmentation combining boundary and region information has been the subject of numerous research works in the past. This combination is usually subject to arbitrary weighting parameters (hyper-parameters) that control the contribution of boundary and region features during segmentation. In this work, we investigate a new approach for estimating the hyper-parameters adaptively to segmentation. The approach takes its roots from the physical properties of the energy functional controlling segmentation and a Bayesian formulation of segmentation and hyper-parameters estimation.

Most image segmentation algorithms in the past are based on optimizing an objective function that aims to achieve the similarity between several low-level features to build a partition of the image into homogeneous regions. In the present paper, we propose to incorporate the relevance (selection) of the grouping features to enforce the segmentation toward the capturing of objects of interest. The relevance of the features is determined through a set of positive and negative examples of a specific object defined a priori by the user. The calculation of the relevance of the features is performed by maximizing an objective function defined on the mixture likelihoods of the positive and negative object examples sets. The incorporation of the features relevance in the object segmentation is formulated through an energy functional which is minimized by using level set active contours. We show the efficiency of the approach on several examples of object of interest segmentation and tracking where the features relevance is used.

In this paper, we propose a novel object tracking algorithm in video sequences. The method is based on object mixture matching between successive frames of the sequence by using active contours. Only the segmentation of the objects in the first frame is required for initialization. The evolution of the object contour on a current frame aims to find the maximum fidelity of the mixture likelihood for the same object between successive frames while having the best fit of the mixture parameters to the homogenous parts of the objects. To permit for a precise and robust tracking, region, boundary and shape information are coupled in the model. The method permits for tracking multi-class objects on cluttered and non-static backgrounds. We validate our approach on examples of tracking performed on real video sequences.

In this paper, we propose a novel automatic framework for variational color image segmentation based on unifying adaptive region information and mixture modelling. We consider a formulation of the region information based on the posterior probability of a mixture of general Gaussian (GG) pdfs where each region is represented by a pdf. The segmentation is formulated by the minimization of an energy functional according to the region contours and all the mixture parameters respectively. Two main objectives are achieved by the approach. A scheme is provided to extend easily the adaptive segmentation to an arbitrary number of regions and to perform it in a fully automatic fashion. Moreover, the segmentation recovers an accurate and representative mixture of pdfs. In the approach, we couple the boundary and region information of the image to steer the segmentation. We validate the method on the segmentation of real world color images.