With the rapid advancement of generative models, high-fidelity AI-generated images have become increasingly indistinguishable from real images, posing significant challenges to traditional detection methods that rely on explicit artifacts or uniform feature learning. We hypothesize that detection ambiguity originates from pattern coexistence: synthetic images simultaneously embed (a) authentic patterns inherited from real-image distributions and (b) synthetic patterns induced by generative architectures, whereas real images maintain consistent patterns. We validate this hypothesis through SHAP-based quantitative analysis, demonstrating that synthetic images inherently exhibit a dual distribution—simultaneously containing authentic patterns and synthetic traces—while real images show a unimodal distribution. Building on this insight, this paper proposes a Dual-Branch Asymmetric Discrepancy Learning (DADL) framework. The DADL leverages multi-scale feature extraction and Asymmetric Feature Discrepancy Loss to capture and amplify such pattern differences across multiple scales. Extensive experiments on three benchmarks (AIGCDetectBenchmark, GenImage, and Chameleon) show that DADL achieves state-of-the-art performance, with particular strengths in detecting high-fidelity synthetic images from diffusion models (e.g., Midjourney, SDv1.4, SDv1.5) and enhancing generalization across diverse generative paradigms. This study not only offers an effective approach for AIGI detection but also sheds light on the intrinsic properties of synthetic images, providing a new perspective for advancing AIGI forensics.

ABSTRACT High‐fidelity, speech‐driven 3D facial animation is crucial for immersive applications and virtual avatars. Nevertheless, advancement is impeded by two principal challenges: (1) a lack of high‐quality 3D data, and (2) inadequate modelling of the multi‐scale characteristics of speech signals. In this paper, we present Speech2Face3D, a novel two‐stage transfer‐learning framework that pretrains on large‐scale pseudo‐3D facial data derived from 2D videos and subsequently finetunes on smaller yet high‐fidelity 3D datasets. This design leverages the richness of easily accessible 2D resources while mitigating reconstruction noise through a simple temporal smoothing step. Our approach further introduces a Multi‐Scale Hierarchical Audio Encoder to capture subtle phoneme transitions, mid‐range prosody, and longer‐range emotional cues. Extensive experiments on public 3D benchmarks demonstrate that our method achieves state‐of‐the‐art performance on lip synchronization, expression fidelity, and temporal coherence metrics. Qualitative user evaluations validate these quantitative improvements. Speech2Face3D is a robust and scalable framework for utilizing extensive 2D data to generate precise and realistic 3D facial animations only based on speech.

Multimodal sentiment analysis (MSA) is important for quickly and accurately understanding people's attitudes and opinions about an event. However, existing sentiment analysis methods suffer from the dominant contribution of text modality in the dataset; this is called text dominance. In this context, we emphasize that weakening the dominant role of text modality is important for MSA tasks. To solve the above two problems, from the perspective of datasets, we first propose the Chinese multimodal opinion-level sentiment intensity (CMOSI) dataset. Three different versions of the dataset were constructed: manually proofreading subtitles, generating subtitles using machine speech transcription, and generating subtitles using human cross-language translation. The latter two versions radically weaken the dominant role of the textual model. We randomly collected 144 real videos from the Bilibili video site and manually edited 2557 clips containing emotions from them. From the perspective of network modeling, we propose a multimodal semantic enhancement network (MSEN) based on a multiheaded attention mechanism by taking advantage of the multiple versions of the CMOSI dataset. Experiments with our proposed CMOSI show that the network performs best with the text-unweakened version of the dataset. The loss of performance is minimal on both versions of the text-weakened dataset, indicating that our network can fully exploit the latent semantics in nontext patterns. In addition, we conducted model generalization experiments with MSEN on MOSI, MOSEI, and CH-SIMS datasets, and the results show that our approach is also very competitive and has good cross-language robustness.

Audio watermarking is a promising technology for copyright protection, yet traditional methods are limited that must be combined with auxiliary techniques against attacks. This article proposes a new audio watermarking method that embeds watermarks through a trained neural network. It adds small imperceptible perturbations to the original audio so that its deep features point to specific watermark features. Data augmentation and error correcting coding are employed to guarantee its practicable robustness. This method is robust against many attacks without auxiliary techniques and shows better performance than other deep learning-based methods.

Watermarking involves embedding a watermark in an image and later extracting it to prove the image’s copyright. In most cases, a complete image contains both smooth and textured regions. As a rule of thumb, the visual quality of an image with a watermark embedded in its textured regions is better than that of the same image with a watermark in smooth regions. This paper, by taking advantage of the fact, proposes a texture-aware local adaptive watermarking algorithm to maximize the watermark’s robustness while maintaining its imperceptibility. To identify textured regions in an image, we introduce the texture value, an efficient and proper metric of the richness of image texture. It combines the texture correlation of the AC coefficients, the luminance masking of the DC coefficient, and the distribution of image texture. A watermark is embedded adaptively into multiple non-overlapping textured regions of an image under the specified SSIM condition. Its adaptiveness comes from a novel texture-aware adaptive parameter model derived by multivariate regression analysis. Correct extraction of watermarks from multiple textured regions can be done by the cooperation of embedding and extraction strategies, with the assistance of RS-based watermark coding model. They allow for greater robustness, faster extraction, and adjustable watermark capacity. The simulation experiments on 100 images demonstrate that our proposed algorithm outperforms state-of-the-art algorithms with respect to imperceptibility, robustness, and adaptability.

Although text recognition methods based on deep neural networks have promising performance, there are still challenges due to the variety of text styles, perspective distortion, text with large curvature, and so on. To obtain a robust text recognizer, we have improved the performance from two aspects: data aspect and feature representation aspect. In terms of data, we transform the input images into S-shape distorted images in order to increase the diversity of training data. Besides, we explore the effects of different training data. In terms of feature representation, the combination of instance normalization and batch normalization improves the model's capacity and generalization ability. This paper proposes a robust scene text recognizer IBN-STR, which is an attention-based model. Through extensive experiments, the model analysis and comparison have been carried out from the aspects of data and feature representation, and the effectiveness of IBN-STR on both regular and irregular text instances has been verified. Furthermore, IBN-STR is an end-to-end recognition system that can achieve state-of-the-art performance.

Watermarking plays an important role in identifying the copyright of an image and related issues. The state-of-the-art watermark embedding schemes, spread spectrum and quantization, suffer from host signal interference (HSI) and scaling attacks, respectively. Both of them use a fixed embedding parameter, which is difficult to take both robustness and imperceptibility into account for all images. This paper solves the problems by proposing two novel blind watermarking schemes: a spread spectrum scheme with adaptive embedding strength (SSAES) and a differential quantization scheme with adaptive quantization threshold (DQAQT). Their adaptiveness comes from the proposed adaptive embedding strategy (AEP), which maximizes the embedding strength or quantization threshold by guaranteeing the peak signal-to-noise ratio (PSNR) of the host image after embedding the watermark, and strikes the balance between robustness and imperceptibility. SSAES is HSI free by factoring in the priori knowledge about HSI. In DQAQT, an effective quantization mode is proposed to resist scaling attacks by utilizing the difference between two selected DCT coefficients with high stability. Both SSAES and DQAQT can be easily applied to other watermarking frameworks. We introduce a notion called error threshold to theoretically analyze the performance of our proposed methods in details. The experimental results consistently demonstrate that SSAES and DQAQT outperform the state-of-the-art methods in terms of imperceptibility, robustness, computational cost, and adaptability.

Recently, image representation based on convolutional neural network (CNN) becomes more popular than SIFT based feature, such as Fisher vector (FV). However, which of the two works better for image retrieval is not entirely clear yet. In this paper, we propose to fuse CNN and FV to incorporate the advantages of both features for image retrieval. We extract CNN feature and FV from multi-scale regions, which makes the representation more robust to image noise. Then a query-adaptive feature fusion method is proposed, which is used jointly with 2-D inverted index under the framework of bag-of-words. Moreover, we make an evaluation of different CNN feature extraction methods for the region based method. Extensive experiments on four benchmark datasets demonstrate the effectiveness of our method with efficiency in both time cost and memory usage.

Many image retrieval systems adopt the bag-of-words model and rely on matching of local descriptors. However, these descriptors of keypoints, such as SIFT, may lead to false matches, since they do not consider the contextual information of the keypoints. In this paper, we incorporate the cues of meaningful regions where local descriptors are extracted. We describe a matching region estimation (MRE) method to find appropriate matching regions for local descriptor matching pairs. Then the region matching quality is evaluated and the true matched regions will enhance the similarity of local descriptors. Consequently, the image retrieval accuracy can be improved. Extensive experiments on benchmark datasets show the effectiveness of our method and our result compares favorably with the state-of-the-art.

In this paper, we present a robust Extremal Region (ER) based scene text detection system. To eliminate the vast non-text components generated by ER operator, a three-stage cascaded filter is proposed. In the first stage, a powerful character classifier enhanced by recursive local search is introduced to separate text components from noises. Then, an efficient heuristic pruning method is designed to further clean overlapped duplicate characters. Finally, after text line construction, a cascaded text line classification model integrating word entropy and sliding window based CNN is proposed to remove false text lines. Experiments on benchmarks show that our method achieves state-of-the-art performance.

Existing techniques usually adopt compact descriptors such as Fisher vector for mobile visual search, since compact descriptors are memory-efficient and suitable for fast transmission. In common Fisher vector methods, in order to make the size of image representations small enough for efficient transmission, only a small number of visual words are used. However, this choice usually sacrifices the search accuracy. In this paper, a Soft-Assignment Adjusting approach is proposed to just select informative components of descriptors for query. With this method, we can adopt more visual words to improve accuracy, while the memory usage is still low. Furthermore, efficient bitrate scalable codes are proposed in order to accommodate the network bandwidth variation. Experiments performed on benchmark datasets show that our proposed approach outperforms the state-of-the-art methods for mobile visual search.

Although many systems based on global or local descriptors have shown promising results for logo recognition, they have handled all logos with the same structure and not considered their diversities. Therefore, with the logo scale increasing, the general way cannot recognize each logo perfectly. To overcome this limitation, we propose a novel strategy to match query and each logo individually using these features. First, a new conception named logo density is introduced as important semantic information for logos. Second, matching density is given according to the logo density and by utilizing it in logistic function an individualized matching strategy is developed to obtain accurate similarity for query and a logo. Finally, we present a fast recognition algorithm based upon bag-of-words model to realize scalable logo recognition. Our method is evaluated on two challenging datasets (our 10,000-class logo dataset and FlickrLogos-27). Experiments demonstrate its superior performance comparing to previous methods.

Using hashing algorithms to learn binary codes representation of data for fast approximate nearest neighbor (ANN) search has attracted more and more attentions. Most existing hashing methods employ various hash functions to encode data. The resulting binary codes can be obtained by concatenating bits produced by those hash functions. These methods usually have two main steps: projection and thresholding. One problem of these methods is that every dimension of the projected data is regarded as the same importance and represented by one bit, which may result in ineffective codes. We introduce an adaptive bit allocation hashing (ABAH) method to encode data for ANN search. The basic idea is, according to the dispersion of every dimension after projection we use different number of bits to encode them. ABAH can effectively preserve the neighborhood structure in the original data space. Extensive experiments show that ABAH significantly outperforms three state-of-the-art methods.

This paper introduces a novel pre-locating algorithm for rapid logo detection in unconstrained color images. This work is distinguished by two major contributions. The first is a new method of representation for logo called “spatial connected component descriptor” (SCCD) containing connected component (CC) prediction model and effective-CC pixel distribution histogram. The former represents combinations between CCs based on color and spatial relationships of CCs. While the latter describes the pixel distribution information of effective CCs. The two parts capture the layout of logos from different points. The second is a logo pre-locating algorithm by the means of SCCD to search for logo prediction regions in test images, on which some content-based features are used for logo matching. Experimental results illustrate that our pre-locating algorithm speeds up logo detection to a great extent and shows precise location compared to previous systems.

It has always been very difficult to recognize realistic actions from unconstrained videos because there are tremendous variations from camera motion, background clutter, object appearance and so on. In this paper, a Single-Feature Hierarchical Latent Dirichlet Allocation model called SF-HLDA by extending Latent Dirichlet Allocation to the hierarchical one is first proposed for realistic action recognition. And then, by extending SF-HLDA, we present another model called Multi-Feature Hierarchical Latent Dirichlet Allocation model MF-HLDA which can effectively fuse several different features into one model for recognizing the realistic actions. Experiments demonstrate the effectiveness of our proposed models.

In this paper, we call the pattern classification problem that consists in assigning a category label to a long audio signal based on its semantic content as Generic Audio Document Categorization (GADC). A novel generative model is proposed to describe the generic audio document categories and solve the GADC problem. This model is a four-level hierarchical model in which two latent variables "audio topic" and "audio word" are introduced in addition to the two observed variables category and audio feature. We present an iterative learning algorithm including two Expectation-Maximization (EM) cycles to estimate the model parameters and give a discriminative document weighting procedure to make the model more discriminative. Subsequently, the distribution of "audio topic" in the well-trained model is utilized to represent each generic audio document category. This is same with some bag-of-word methods. However, our method is advanced since it does not require quantizing the continuous audio features to a vocabulary of "audio words". Finally, experiment results show the effectiveness of our approach.

The italic detection and slant rectification is a key step of optical character recognition (OCR). In this paper, a novel method is proposed to detect and rectify italic characters in Chinese advertising images. Based on observations on structures of many characters, the centroid angle is proposed and a statistical study on it is presented. According to the statistical results, the centroid angle of a Chinese character approximately obeys a Gaussian distribution with its slant angle. Moreover, a Markov Random Field (MRF) model, considering the font-face similarity of neighboring characters and the strong correlation between the centroid angle and the slant angle of a character, is then presented to estimate the slant angle of a character. The italic characters can be detected and rectified by the estimated angle. The experimental results demonstrate the proposed method is effective and applicable.

In this paper, a novel Chinese character localization method is proposed for texts in advertising images. To deal with the texts with gradient color, a color clustering method based on edge is introduced to separate the color image into homogeneous color layers. To solve the problem of locating characters varied in size, style and arranged in irregular direction, a novel character localization method is proposed, which integrates structure and CC-clustering to locate characters according to reliable features of characters. Finally, a new noise removal method based on stroke width histogram is employed to remove all non-characters connected components, and then all characters are located. The experimental results show that the proposed method can effectively locate characters in advertising images.

Automatic behavior recognition is one important task of community security and surveillance system. In this paper, a novel method is proposed for automatic selection of behavior models by iterative learning and abnormality recognition. The method is mainly composed of the following two steps: (1) The models of normal behaviors are automatically selected and trained by combining Dynamic Time Warping based spectral clustering and iterative learning; (2) Maximum A Posteriori adaptation technique is used to estimate the parameters of abnormal behavior models from those of normal behavior models. Compared with the related works in the literature, our method has three advantages: (1) automatic selection of the class number of normal behaviors from large unlabeled video data according to the process of iterative learning, (2) semi-supervised learning of abnormal behavior models, and (3) avoidance of the running risk of over-fitting during learning the Hidden Markov Models of behaviors in case of sparse data. Experiments demonstrate the effectiveness of our proposed method.

This paper presents a novel Global and Local Features based Latent Dirichlet Allocation model for scene recognition. The proposed model follows the bag-of-word framework like the Latent Dirichlet Allocation model. The traditional Latent Dirichlet Allocation model for scene recognition only uses the orderless bag of features called global features without considering spatial constraints on these features. Different from this model, our proposed model can combine both global features and local region features for improving the recognition performance. In our method, local region features are gotten by adding a simple spatial constraint on the orderless bag of features. Experiments on three scene datasets demonstrate the effectiveness of our proposed model.

We present a novel approach to measuring similarity between objects based on matching local “appearance contextual descriptor”. The descriptor has two components: Histogram of Oriented Gradient feature representing local patch appearance and the contextual descriptor capturing not only the spatial distribution of the non-reference patches relative to the reference patch but also the appearance similarities between the reference patch and the non-reference patches in the region. Corresponding patches within two similar objects will have similar contextual descriptors, though the patch appearances may have some difference. We treat recognition in a nearest-neighbor classification framework and match object in regions with no prior learning. We compare our method to commonly used methods and demonstrate its applicability to object detection and recognition.

We consider the image classification problem based on the similarities between images. The choice of the similarity is related to the particular applications, and it could be based on color, texture, bag-of-features, or even more complex kernels. As long as the pair-wise similarity matrix is transformed into a positive semidefinite one, the similarities of images could be treated as kernels. This transformation makes it possible for kernel methods to solve the similarity-based image classification problem. In this paper, we propose a novel kernelized classification framework based on sparse representation. This new framework casts the classification as finding a sparse linear representation of test image with respect to training images. Unlike the former works, we do this sparse coding procedure through a proposed kernelized orthogonal matching pursuit algorithm, which is performed in inner product space rather than Euclidean space. Through a proper choice of the similarity function, the proposed approach can be applied to diverse image classification problems. Comparative experiments between the proposed method and other existing methods, on two real datasets (Caltech-101 and Face Rec) show that our method performed better.

A fast image in painting method based on hybrid similarity-distance is proposed in this paper. In Criminisi et al.'s work, similarity distance are not reliable enough in many cases and the algorithm performs inefficiently. To solve these problems, we propose a new searching strategy to accelerate the algorithm. In addition, we modify the confidence-updating rule to make more reasonable the distributions of the confidences in source region. Besides, taking account of the stationarity of texture and the reliability of the source regions, we present a hybrid similarity-distance, which combines the distance in color space with the distance in spatial space by weight coefficients related to the confidence value. A more reasonable patch will be found out by this hybrid similarity-distance. The experiments verify that the proposed method yields qualitative improvements compared to Criminisi et al.'s work.

A novel approach based on the probabilistic latent semantic analysis model (pLSA) for automatic musical genre classification is proposed in this paper. Unlike traditional usage, the pLSA is used to model musical genre instead of single music signal in the proposed approach. First, an unsupervised clustering algorithm is utilized to group temporal segments in music signals into several natural clusters. By this means, each music signal is decomposed into a bag of ldquoaudio wordsrdquo. Subsequently, the pLSA model of each musical genre is trained through a new iterative training procedure and well-known EM algorithm. This training procedure can iteratively update the pLSA model parameters by discriminatively computing weight of each training music signal and evidently improve the model's discriminative performance. Finally, these models can be used to classify new unseen music signals. Experiments on two commonly utilized databases show that our pLSA based approach can give promising results and the iterative learning procedure is effective.

In this paper, three different approaches to Chinese-English bilingual phone modeling are investigated and compared. The first approach is to simply combine Chinese and English phone inventories together without phone sharing across the languages. The second one is to map language-dependent phones to the inventory of the International Phonetic Association (IPA) based on phonetic knowledge to construct the bilingual phone inventory. The third one is to merge the language-dependent phone models by an hierarchical phone clustering algorithm to get a compact bilingual inventory. In the third approach, two distance measures are used to perform the bottom-up clustering. One is the Bhattacharyya distance. The other is the acoustic likelihood distance. Experimental results show that the phone clustering approach outperforms the IPA-based phone mapping approach, and it can also achieve comparable performance to the simple combination of language-dependent phone inventories with fewer model parameters, especially when using acoustic likelihood distance measurement.

Adapting the parameters of a statistical speaker independent continuous speech recognizer to the speaker can significantly improve the recognition performance and robustness of the system. In this paper, we propose a novel target-driven speaker adaptation method, Generalized Joint Maximum a Posteriori (GJMAP), which extends and improves the previous successful method JMAP. GJMAP partitions the HMM parameters with respect to the adaptation data, using the priori phonetic knowledge. The generation of regression class trees is dynamically constructed on the target-driven principle in order to obtain the maximum increase of the auxiliary function. An off-line adaptation experiment on large vocabulary continuous speech recognition is carried out. The experimental results show GJMAP has more advantages than the conventional methods.

Some practical implementation issues on partially tied covariance (PTC) modeling are discussed. First, from the view of model complexity and computational load, a comparison is made for some variants of PTC. From the analysis, two representatives, STC and Ortho-STC are compared in detail. Second, based on these variants, two techniques are studied. One technique is joint optimization of both transformation and HMM parameters, which will exploit the potential of PTC. The other technique is model selection by hierarchical tree via Bayesian information criterion (BIC), which will decide the number and structure of transformation classes thus to assure the generalization capacity. Experiment results showed that STC always outperforms Ortho-STC due to the effect of parameter tying and by the application of above two techniques the system performance can be much improved.

A vector statistical piecewise polynomial (VPP) approximation algorithm is proposed for environment compensation in speech signals that are degraded by both additive and convolutive noise. By investigating the model of the telephone environment, we address a piecewise polynomial, namely two linear polynomials and a quadratic polynomial, to approximate the environment function precisely. The VPP is applied either to stationary noise, or to non-stationary noise. In the first case, batch EM is used in the log-spectral domain; in the second case, recursive EM with iterative stochastic approximation is developed in the cepstral domain. Both approaches are based on the minimum mean squared error (MMSE) sense. Experimental results are presented on the application of this approach in improving the performance of Mandarin large vocabulary continuous speech recognition (LVCSR) in background noise and different transmission channels (such as fixed telephone line and GSM). The method can reduce the average character error rate (CER) by about 18%.

This paper presents our approach to enhance the portability of acoustic models by mitigating the phonetic mismatch arising from a new testing task which is rather different from the training data. The approach is a hybrid one which combines knowledge-based context categorization to generate a context rich set of subword units, and data-driven-based acoustic model clustering on the level of context category. Compared with the conventional approach of only phonetic decision tree based model clustering and unseen model generation, the new approach improved greatly the desired subword coverage for the new testing domain, and achieved an error rate reduction by 10.8% for Chinese character accuracy in the recognition experiments. Together with the effect of the newly adopted basic units of 9 glottal stops, we achieved a total 23.5% error rate reduction in the testing compared to the baseline system.

In this paper, a distance-related unit association maximum entropy (DUAME) language modeling is proposed. This approach can model an event (unit subsequence) using the co-occurrence of full distance unit association (UA) features so that it is able to pursue a functional approximation to higher order N-gram with significantly less memory requirement. A smoothing strategy related to this modeling will also be discussed. Preliminary experimental results have shown that DUAME modeling is comparable to conventional N-gram modeling in perplexity with significantly small number of parameters.

As to traditional n-gram model, smaller n value is an inherent defect for estimating language probabilities in speech recognition, simply because that estimation could not be executed over farther word association but by means of short sequential word correlated information. This has an strong effect on the performance of speech recognition. This paper introduces an integrated language modeling with ngram model and word association model (abbreviated as WA model). This model integrated two kind of joint probabilities, traditional n-gram probability and word association probability, to estimate actual output probability. WA model are based on a combined probability estimation of orderly word association without distant and strict sequential limitation. In addition, two kinds of local linguistic constraints have also been incorporated into n-gram estimation for smoothing date sparse and adjusting special language unit score locally. A substantial improvement for the performance of Chinese phonetic-to-text transcription in speech recognition has been obtained.

While early machines adopted isolated syllable as input units and needed boring enrollment, our research focus on the speaker-independent, word-based dictation.A deliberately designed 120-speaker database was built for training ; inter-syllable context ,tonal and endpoint dependent acoustic model are applied with promising MFCC feature; Two-pass acoustic matching accelerates the recognition making fully advantage of the monosyllabic structure of Chinese speech; A complete word bigram and trigram serve as language processing module.With all efforts, the system reaches 90% character accuracy performing in almost real-time on Pentium PC without DSP help.