ABSTRACT The detection of small targets, particularly people and various types of vehicles, has long been recognised as a formidable and highly challenging task in remote sensing images. To improve the performance of small target detection, we propose IE‐YOLO, an information‐enhanced small target detection model based on YOLOv8. First, to enhance the model's ability to distinguish between the target and the background, or between the target and similar categories, the C2f module in the backbone is replaced with a custom‐designed vssC2f module. Second, to further amplify the features of small targets, a Feature Enhancement (FEB) Block is proposed to significantly strengthen overall feature extraction capability, enabling it to capture small target characteristics more effectively. Finally, an Information‐Enhanced Feature Pyramid Network (IE‐FPN) is proposed to fully utilise shallow features for retaining more small target information, significantly improving detection performance. Extensive experiments on the VisDrone2019 dataset show that the IE‐YOLO model achieves higher detection accuracy for small target detection than other advanced detection models. Compared with YOLOv8, the values of the mAP0.5 and the mAP0.5:0.95 have increased by 6.8% and 4.7%, respectively. It demonstrates significant application potential and practical value in the field of small target detection.

Automatic annotation of digital pictures is a key technology for managing and retrieving images from large image collections. Typical algorithms only deal with the problem of monolingual image annotation. In this paper, we propose a framework to deal with the problem of multilingual image annotation, which can annotate images in multiple languages. The framework can not only benefit users with different native languages, but also provide more accurate annotations. In this framework, image annotation is performed in two stages, including parallel monolingual image annotation and the fusion of annotation results in multiple languages. In the first stage, candidate annotations for each language are extracted by leveraging multilingual large scale Web image database. Due to the incompleteness and inaccuracy problem of candidate annotations, we proposed a multilingual annotation fusion algorithm (MAF). By modeling candidate annotations for each language as an n-partite graph, MAF algorithm can improve and re-rank multilingual annotations. Finally, annotations with the highest ranking values in each language are selected and translated as the result. Experimental results for English-Chinese image annotations demonstrate the effectiveness of the proposed framework.

In recent years, “bag-of-words” models, which treat an image as a collection of unordered visual words, have been widely applied in the multimedia and computer vision fields. However, their ignorance of the spatial structure among visual words makes them indiscriminative for objects with similar word frequencies but different word spatial distributions. In this paper, we propose a visual language modeling method (VLM), which incorporates the spatial context of the local appearance features into the statistical language model. To represent the object categories, models with different orders of statistical dependencies have been exploited. In addition, the multilayer extension to the VLM makes it more resistant to scale variations of objects. The model is effective and applicable to large scale image categorization. We train scale invariant visual language models based on the images which are grouped by Flickr tags, and use these models for object categorization. Experimental results show they achieve better performance than single layer visual language models and “bag-of-words” models. They also achieve comparable performance with 2-D MHMM and SVM-based methods, while costing much less computational time.

We study the problem of learning to rank images for image retrieval. For a noisy set of images indexed or tagged by the same keyword, we learn a ranking model from some training examples and then use the learned model to rank new images. Unlike previous work on image retrieval, which usually coarsely divide the images into relevant and irrelevant images and learn a binary classifier, we learn the ranking model from image pairs with preference relations. In addition to the relevance of images, we are further interested in what portion of the image is of interest to the user. Therefore, we consider images represented by sets of regions and propose multiple-instance rank learning based on the max margin framework. Three different schemes are designed to encode the multiple-instance assumption. We evaluate the performance of the multiple-instance ranking algorithms on real-word images collected from Flickr - a popular photo sharing service. The experimental results show that the proposed algorithms are capable of learning effective ranking models for image retrieval.

In image annotation refinement, word correlations among candidate annotations are used to reserve high relevant words and remove irrelevant words. Existing methods build word correlations on textual annotations of images. In this paper, visual contents of images are utilized to explore better word correlations by using multi-graph similarity reinforcement method. Firstly, image visual similarity graph and word correlations graph are built respectively. Secondly, the two graphs are iteratively reinforced by each other through image-word transfer matrix. Once the two graphs converge to steady states, the new word correlations graph is used to refine the candidate annotations. The experiments show that our method performs better than method not considering visual content of images.

In this paper, we propose a novel scheme for near-duplicate image detection, which is an important problem in variety of applications. While in general content based image retrieval, an image could be similar to the query image in infinitely various ways, the ways in which near-duplicate images deviate from the reference image are very limited. Based on this observation, we proposed to use examplar near-duplicate images, which can be obtained automatically, to improve the performance of near-duplicate image retrieval. We first use examplar near-duplicates to learn an effective distance measure and incorporate the learned metric into locality-sensitive hashing to achieve fast retrieval. We then use examplar near-duplicates to automatically expand the query to further improve the retrieval accuracy. The experimental results validate the effectiveness of the proposed algorithms.

Traditional image annotation approaches are only applicable for datasets with small and limited lexicon. Besides, annotation words are treated equally without considering the importance of each word in the real world. To address these problems, we propose TagRank, a method to model the relative importance of every candidate word. By exploiting tag clusters on Flickr, TagRank could be modeled as random walk with restarts, which incorporates both word frequency and word correlation information. As a result, a ranked annotation vocabulary could be built. By utilizing the tag importance in a real image annotation experiment, we show that TagRank is helpful for improving the performance of image annotation.

In this paper, we proposed an image ranking model called SocialRank for web image search engine, which leverages rich metadata and structures of virtual communities in online photo-sharing sites in Web 2.0. In SocialRank, three levels of static rankings for sites, users and images are computed, respectively. To achieve combined image rankings in a site, a graph model is proposed to integrate social relations between users and images to reinforce their static rankings. To obtain the unified image rankings of different sites, image rankings are normalized and combined by considering duplicate images and static site rankings. The diversity of search results is also an important factor of image search engines. So we proposed an offline static diversity ranking of images. The experimental results show that the proposed SocialRank performs comparably with Google Image Search.

Near-duplicate image detection is a critical task in copyright protection. More challenging than the common similarity search, this task requires not only the retrieval of the top similar images but also the detection of the entire near-duplicates collection from the Internet. The common similarity search algorithms are not capable to undertake the latter demand. This paper proposes the query oriented subspace shifting algorithm. The algorithm measures the similarity in various subspaces, which are dynamically generated based on the correlation between samples and the query image. An adaptive threshold is generated automatically to filter the near-duplicates in each subspace. As these subspaces are query oriented, the near-duplicates are less likely to be missed. Experiments shows that this method can effectively improve the detection recall while keeps the similar precision, comparing with the common similarity search algorithm.

Graph fusion is a widely adopted approach to multimodality fusion for image content analysis and retrieval. Typical graph fusion methods take into account both intragraph relationship and inter-graph relationship of images in the fusion process. Although intra-graph relationship can be built easily based on the similarities of images in each modality, it is not trivial to build the inter-graph relationship. In this paper, a novel method is proposed to construct the inter-graph relationship of images by introducing virtual links and setting the link weights to one. This scheme makes the iterative fusion process more efficient and effective. Experimental results on a public image dataset show that the proposed method outperforms the general similarity graph methods in image clustering and retrieval.

Modern web search engines are expected to return top-k results efficiently given a query. Although many dynamic index pruning strategies have been proposed for efficient top-k computation, most of them are prone to ignore some especially important factors in ranking functions, e.g. term proximity (the distance relationship between query terms in a document). The inclusion of term proximity breaks the monotonicity of ranking functions and therefore leads to additional challenges for efficient query processing. This paper studies the performance of some existing top-k computation approaches using term-proximity-enabled ranking functions. Our investigation demonstrates that, when term proximity is incorporated into ranking functions, most existing index structures and top-k strategies become quite inefficient. According to our analysis and experimental results, we propose two index structures and their corresponding index pruning strategies: Structured and Hybrid, which performs much better on the new settings. Moreover, the efficiency of index building and maintenance would not be affected too much with the two approaches.

Image search result clustering has become an active research topic. However, due to the limitations of current image search engines, the search result always exhibits partial clustering character, which makes the traditional clustering assumption unreasonable. In this paper, we apply Bregman bubble clustering (BBC), which clusters only a fraction of the whole data set, to image search result clustering. We show that relevant and irrelevant images are less mixed in the clusters produced by BBC. Therefore, we are able to incorporate a cluster based relevance feedback scheme to the clustering result and improve the relevance ranking of the search result according to user's feedback. Experiments on animal images from Flickr demonstrate the effectiveness of our clustering and re-ranking algorithms.

Image annotation refinement is crucial to improve the performance of automatic image annotation, in which the estimation of word correlation is a key issue. Typically, the word co-occurrence information may be utilized to estimate the word correlation. However, this approach is not accurate enough because it equally treats any word pair co-occurring in the training data and cannot extract synonymy relationship effectively. In this paper, a novel method is developed to estimate the word correlation based on the improved nearest spanning chains (NSC). It can extract more informative and reasonable relations among keywords. Obtaining the enhanced word correlation, a word-based graph is constructed, which is used to re-rank the candidate annotations for an untagged image. Experiments conducted on the typical Corel dataset demonstrate the effectiveness of the proposed method.

In this paper, a novel low-level feature, named texture moment, is designed to characterize the texture properties of grayscale images for content-based image retrieval. At first, seven attributes are defined for each pixel by applying seven orthogonal templates on its eight neighborhoods. The templates are derived from local Fourier transform. Then, the mean and variation of those seven attributes are calculated for all interior pixels respectively to form a 14-D feature vector. As this feature is highly complementary to other color features, properly combining it with color features together may produce good image retrieval results. Therefore, two feature combinations are also provided. Experiments on 5,000 general-purpose images demonstrate the effectiveness of the proposed texture moment feature and two feature combinations.

Online advertising has enjoyed exponential growth in recent years, and many advertisements appear in the form of images. Although it makes considerable profit, these advertisements tend to disturb the Internet surfing of normal users. Moreover, they always bring extra burden in indexing to commercial image search engines. Therefore, it is necessary to automatically detect those advertising images on the Web. In this paper, a classification based approach is proposed for advertising image detection, in which comprehensive features are exploited and effectively combined. Those features include visual content, link, text and visual layout in hosting Web pages. Promising experimental results are obtained on images collected from about 480 Web sites.

Automatic image annotation is an effective way for managing and retrieving abundant images on the internet. In this paper, we propose a novel search-based method for web image annotation. Firstly, surrounding text and other textual information in the hosting web pages are used as the candidate annotations. As the candidates are very noisy, two measures are defined to re-rank them and only top-ranked ones are reserved as the final annotations. One measure is based on the visual consistence between the given image and the image search results when the candidate annotation is submitted as a query. The other measure is based on the textual consistence between one candidate and others, which is calculated from the pairwise co-occurrence using image search engines. Finally, Dempster-Shafer multiple evidence combination approach is used to combine the visual and textual measures for the final ranking. Experimental results on web images demonstrate the effectiveness of the proposed method.

Automatic image annotation is crucial for keyword-based image retrieval because it can be used to improve the textual description of images efficiently. For this purpose, many methods have been developed. Due to the restrictions of computational complexity and small training set, the image annotation methods are usually based on the probability of individual word, instead of the joint probability of a set of words. Therefore the correlation between words is omitted. In this paper, we propose a method to approximate the joint probability of words in a progressive way. Given an image, the word with highest probability is first annotated. Then, the successive words are annotated by incorporating the information of previously annotated words. It can be seen as a "greedy" algorithm to calculate the joint probability of multiple words. The experiments show that the proposed progressive annotation method can effectively improve the annotation performance.

3D facial extraction from volume data is very helpful in virtual plastic surgery. Although the traditional Marching Cubes algorithm (MC) can be used for this purpose, it could not separate the facial surface from other tissue surfaces. This weakness greatly limited the accuracy of the facial model and its application in plastic surgery. In this paper a volume graph model is proposed, in which the facial surface extraction is formulated as a min-cut problem and can be solved by existing graph cut algorithm. Based on this model, irrelevant tissue surfaces are effectively excluded and a more accurate 3D virtual face can be built for plastic surgery.

Due to the well known semantic gap, content based image retrieval is a difficult problem. To bridge it, relevance feedback as an effective solution has been extensively studied in literatures. However, existing methods follow a single-line searching philosophy, which may lead to a local optimum in search space. To address the problem, we propose a human behavior consistent relevance feedback model for image retrieval in this paper. Simulating human behaviors, the proposed model enable the user to perform relevance feedback in three manners: Follow up, Go back, and Restart. Each manner is a way for the user to provide the system with his or her opinions about search results. The accumulated feedback information can be used to refine the user query and regulate the similarity metric. We adopt the graph ranking algorithm to model the retrieval process. Experiments conducted on standard Corel dataset and Pascal VOC 2006 dataset demonstrate the effectiveness of the proposed mechanism.

Image annotation has been an active research topic in recent years due to its potential impact on both image understanding and web image retrieval. Existing relevance-model-based methods perform image annotation by maximizing the joint probability of images and words, which is calculated by the expectation over training images. However, the semantic gap and the dependence on training data restrict their performance and scalability. In this paper, a dual cross-media relevance model (DCMRM) is proposed for automatic image annotation, which estimates the joint probability by the expectation over words in a pre-defined lexicon. DCMRM involves two kinds of critical relations in image annotation. One is the word-to-image relation and the other is the word-to-word relation. Both relations can be estimated by using search techniques on the web data as well as available training data. Experiments conducted on the Corel dataset and a web image dataset demonstrate the effectiveness of the proposed model.

Automatic image annotation is an effective way for managing and retrieving abundant images on the internet. In this paper, a bipartite graph reinforcement model (BGRM) is proposed for web image annotation. Given a web image, a set of candidate annotations is extracted from its surrounding text and other textual information in the hosting web page. As this set is often incomplete, it is extended to include more potentially relevant annotations by searching and mining a large-scale image database. All candidates are modeled as a bipartite graph. Then a reinforcement algorithm is performed on the bipartite graph to re-rank the candidates. Only those with the highest ranking scores are reserved as the final annotations. Experimental results on real web images demonstrate the effectiveness of the proposed model.

In general, digital images can be classified into photographs and computer graphics. This taxonomy is very useful in many applications, such as Web image search. However, there are no effective methods to perform this classification automatically. In this paper, we manage to solve this problem from two aspects. At first, we propose some novel low-level features that can reveal perceptional differences between photographs and graphics. Then, we adopt an effective algorithm to perform the classification. The experiments conducted on a large-scale image database indicate the effectiveness of our algorithm

Although relevance feedback has been extensively studied in content-based image retrieval in the academic area, no commercial Web image search engine has employed the idea. There are several obstacles for Web image search engines in applying relevance feedback. To overcome these obstacles, we proposed an efficient implicit relevance feedback mechanism. The proposed mechanism shows advantage over traditional relevance feedback methods in the following three aspects. Firstly, instead of enforcing the users to make explicit judgment on the results, our method regards user's click-through data as implicit relevance feedback which release burden from users. Secondly, a hierarchical image search results clustering algorithm is proposed to semantically organize the search results. Using the clustering results as features, our relevance feedback scheme could catch and reflect users' search intention precisely. Lastly, unlike traditional relevance feedback user interface which hardily substitutes subsequent results for previous ones, our method employed friendly recommendation rather than substitution to let the user narrow down on the refined images. To evaluate the implicit relevance feedback mechanism, comprehensive user studies were performed

As the amount of receded TV content is increasing rapidly, people need active and interactive browsing methods. In this paper, we use both text information from closed captions and visual information from video frames to generate links to enable one to explore not only the original video content but also augmented information from the Web. This solution especially shows its superiority when the video content cannot be well represented only by closed captions. A prototype system was implemented and some experiments were carried out to prove the effectiveness and efficiency

Finding visually identical images in large image collections is important for many applications such as intelligence propriety protection and search result presentation. Several algorithms have been reported in the literature, but they are not suitable for large image collections. In this paper, a novel algorithm is proposed to handle the situation, in which each image is compactly represented by a hash code. To detect duplicate images, only the hash codes are required. In addition, a very efficient search method is implemented to quickly group images with similar hash codes for fast detection. The experiments show that our algorithm can be both efficient and effective for duplicate detection in Web image search

With the pervasive use of camera phones, the embedded camera has been considered as a promising HCI manner for mobiles. With necessary technologies, it is possible to become a powerful tool to acquire the information in daily life. We have designed and implemented a system named Photo-to-Search to carry out queries from camera phones simply by taking some photos of interested objects. The captured pictures are compared with a large amount of Web images to select the ones which contain the same prominent object. Consequently, the related information is extracted from the Web pages where the matched images locate. In our demo, data of large buildings, storefronts and products are collected and these kinds of queries are specifically demonstrated to show the efficiency and the effectiveness of our system.

When browsing videos in mobile devices, people often feel that resolution greatly affects their perceptual experience in the limited screen size. In this paper, an attention based spatial video adaptation scheme is proposed to overcome display constraints by producing the region of interest. According to the size of the target display, we automatically detect and crop the informative region in each frame to generate a smooth sequence. To avoid costly fully encoding operations, we employ a set of transcoding techniques based on the H.264 standard. Experimental results show that this approach not only improves the perceptual quality but also saves the bandwidth and computation, especially for the videos which are not well edited

This paper proposes a generic model for unsupervised extraction of viewer's attention objects from color images. Without the full semantic understanding of image content, the model formulates the attention objects as a Markov random field (MRF) by integrating computational visual attention mechanisms with attention object growing techniques. Furthermore, we describe the MRF by a Gibbs random field with an energy function. The minimization of the energy function provides a practical way to obtain attention objects. Experimental results on 880 real images and user subjective evaluations by 16 subjects demonstrate the effectiveness of the proposed approach.

In this paper, we propose a general transductive learning framework named generalized manifold-ranking-based image retrieval (gMRBIR) for image retrieval. Comparing with an existing transductive learning method named MRBIR [12], our method could work well whether or not the query image is in the database; thus, it is more applicable for real applications. Given a query image, gMRBIR first initializes a pseudo seed vector based on neighborhood relationship and then spread its scores via manifold ranking to all the unlabeled images in the database. Furthermore, in gMRBIR, we also make use of relevance feedback and active learning to refine the retrieval result so that it converges to the query concept as fast as possible. Systematic experiments on a general-purpose image database consisting of 5,000 Corel images demonstrate the superiority of gMRBIR over state-of-the-art techniques.

In this paper, a novel offline supervised learning method is proposed to map low-level visual features to high-level semantic concepts for region-based image retrieval. The contributions of this paper lie in three folds. (1) For each semantic concept, a set of low-level tokens are extracted from the segmented regions of training images. Those tokens capture the representative information for describing the semantic meaning of that concept; (2) a set of posteriors are generated based on the low-level tokens through pairwise classification, which denote the probabilities of images belonging to the semantic concepts. The posteriors are treated as high-level features that connect images with high-level semantic concepts. Long-term relevance feedback learning is incorporated to provide the supervisory information needed in the above offline learning process, including the concept information and the relevant training set for each concept; (3) an integrated algorithm is implemented to combine two kinds of information for retrieval: the information from the offline feature-to-concept mapping process and the high-level semantic information from the long-term learned memory. Experimental evaluation on 10,000 images proves the effectiveness of our method.

In this paper, an optimization based learning method is proposed for image retrieval from graph model point of view. Firstly, image retrieval is formulated as a regularized optimization problem, which simultaneously considers the constraints from low-level feature, online relevance feedback and offline semantic information. Then, the global optimal solution is developed in both closed form and iterative form, providing that the latter converges to the former. The proposed method is unified in the senses that 1) it makes use of the information from various aspects in a global optimization manner so that the retrieval performance might be maximally improved; 2) it provides a natural way to support two typical query scenarios in image retrieval. The proposed method has a solid mathematical ground. Systematic experimental results on a general-purpose image database demonstrate that it achieves significant improvements over existing methods.

This paper addresses automatic image annotation problem and its application to multi-modal image retrieval. The contribution of our work is three-fold. (1) We propose a probabilistic semantic model in which the visual features and the textual words are connected via a hidden layer which constitutes the semantic concepts to be discovered to explicitly exploit the synergy among the modalities. (2) The association of visual features and textual words is determined in a Bayesian framework such that the confidence of the association can be provided. (3) Extensive evaluation on a large-scale, visually and semantically diverse image collection crawled from Web is reported to evaluate the prototype system based on the model. In the proposed probabilistic model, a hidden concept layer which connects the visual feature and the word layer is discovered by fitting a generative model to the training image and annotation words through an Expectation-Maximization (EM) based iterative learning procedure. The evaluation of the prototype system on 17,000 images and 7,736 automatically extracted annotation words from crawled Web pages for multi-modal image retrieval has indicated that the proposed semantic model and the developed Bayesian framework are superior to a state-of-the-art peer system in the literature.

In this paper, we present a method to retrieve natural images by sketch query. To measure the similarity between the sketch and an image, relevant regions are first located in that image through a multi-resolution search, and a normalized local shape similarity is proposed for image retrieval. Efficiency and other implementation issues are discussed. Experimental results show that it is an effective approach for content-based image retrieval

To better understand the content of multimedia, a lot of research efforts have been made on how to learn from multi-modal feature. In this paper, it is studied from a graph point of view: each kind of feature from one modality is represented as one independent graph; and the learning task is formulated as inferring from the constraints in every graph as well as supervision information (if available). For semi-supervised learning, two different fusion schemes, namely linear form and sequential form, are proposed. For each scheme, it is derived from optimization point of view; and further justified from two sides: similarity propagation and Bayesian interpretation. By doing so, we reveal the regular optimization nature, transductive learning nature as well as prior fusion nature of the proposed schemes, respectively. Moreover, the proposed method can be easily extended to unsupervised learning, including clustering and embedding. Systematic experimental results validate the effectiveness of the proposed method.

In this paper, a novel inhomogeneous clustering method is proposed for grouping web images. It is used to re-organize the search result of web image search engines into a hierarchical structure so that the users can conveniently browse the search result. This method takes into account various features associated with web images, and treats them in different ways. For the surrounding text extracted from the containing web pages, co-clustering approach is adopted; for low-level features of the image content and other features, one-way clustering approach is adopted. The clustering results of different approaches are combined together to produce the final image groups. Experimental results demonstrate the effectiveness of the proposed method.

In this paper, a novel learning based method is proposed for No-Reference image quality assessment. Instead of examining the exact prior knowledge for the given type of distortion and finding a suitable way to represent it, our method aims to directly get the quality metric by means of learning. At first, some training examples are prepared for both high-quality and low-quality classes; then a binary classifier is built on the training set; finally the quality metric of an un-labeled example is denoted by the extent to which it belongs to these two classes. Different schemes to acquire examples from a given image, to build the binary classifier and to model the quality metric are proposed and investigated. While most existing methods are tailored for some specific distortion type, the proposed method might provide a general solution for No-Reference image quality assessment. Experimental results on JPEG and JPEG2000 compressed images validate the effectiveness of the proposed method.

The canonical face recognition algorithm Eigenface and Fisherface are both based on one dimensional vector representation. However, with the high feature dimensions and the small training data, face recognition often suffers from the curse of dimension and the small sample problem. Recent research [4] shows that face recognition based on direct 2D matrix representation, i.e. 2DPCA, obtains better performance than that based on traditional vector representation. However, there are three questions left unresolved in the 2DPCA algorithm: I ) what is the meaning of the eigenvalue and eigenvector of the covariance matrix in 2DPCA; 2) why 2DPCA can outperform Eigenface; and 3) how to reduce the dimension after 2DPCA directly. In this paper, we analyze 2DPCA in a different view and proof that is 2DPCA actually a "localized" PCA with each row vector of an image as object. With this explanation, we discover the intrinsic reason that 2DPCA can outperform Eigenface is because fewer feature dimensions and more samples are used in 2DPCA when compared with Eigenface. To further reduce the dimension after 2DPCA, a two-stage strategy, namely parallel image matrix compression (PIMC), is proposed to compress the image matrix redundancy, which exists among row vectors and column vectors. The exhaustive experiment results demonstrate that PIMC is superior to 2DPCA and Eigenface, and PIMC+LDA outperforms 2DPC+LDA and Fisherface.

Retrospective news event detection (RED) is defined as the discovery of previously unidentified events in historical news corpus. Although both the contents and time information of news articles are helpful to RED, most researches focus on the utilization of the contents of news articles. Few research works have been carried out on finding better usages of time information. In this paper, we do some explorations on both directions based on the following two characteristics of news articles. On the one hand, news articles are always aroused by events; on the other hand, similar articles reporting the same event often redundantly appear on many news sources. The former hints a generative model of news articles, and the latter provides data enriched environments to perform RED. With consideration of these characteristics, we propose a probabilistic model to incorporate both content and time information in a unified framework. This model gives new representations of both news articles and news events. Furthermore, based on this approach, we build an interactive RED system, HISCOVERY, which provides additional functions to present events, Photo Story and Chronicle.

Most current content-based image retrieval systems are still incapable of providing users with their desired results. The major difficulty lies in the gap between low-level image features and high-level image semantics. To address the problem, this study reports a framework for effective image retrieval by employing a novel idea of memory learning. It forms a knowledge memory model to store the semantic information by simply accumulating user-provided interactions. A learning strategy is then applied to predict the semantic relationships among images according to the memorized knowledge. Image queries are finally performed based on a seamless combination of low-level features and learned semantics. One important advantage of our framework is its ability to efficiently annotate images and also propagate the keyword annotation from the labeled images to unlabeled images. The presented algorithm has been integrated into a practical image retrieval system. Experiments on a collection of 10,000 general-purpose images demonstrate the effectiveness of the proposed framework.

In this paper, a unified image retrieval framework based on both keyword annotations and visual features is proposed. In this framework, a set of statistical models are built based on visual features of a small set of manually labeled images to represent semantic concepts and used to propagate keywords to other unlabeled images. These models are updated periodically when more images implicitly labeled by users become available through relevance feedback. In this sense, the keyword models serve the function of accumulation and memorization of knowledge learned from user-provided relevance feedback. Furthermore, two sets of effective and efficient similarity measures and relevance feedback schemes are proposed for query by keyword scenario and query by image example scenario, respectively. Keyword models are combined with visual features in these schemes. In particular, a new, entropy-based active learning strategy is introduced to improve the efficiency of relevance feedback for query by keyword. Furthermore, a new algorithm is proposed to estimate the keyword features of the search concept for query by image example. It is shown to be more appropriate than two existing relevance feedback algorithms. Experimental results demonstrate the effectiveness of the proposed framework.

Content-related functionalities of image/video applications call for efficient tools that can automatically extract meaningful objects from images. However, traditional methods generally fail to capture objects of user interest because they totally neglect human visual attention perception. Aiming to address this problem, this study proposes a generic model for unsupervised extraction of viewer's attention objects from color images. We formulate the attention objects as a Markov random field (MRF). Then, the MRF is expressed in the form of a Gibbs random field with an energy function. The energy minimization that integrates visual attention and object growing provides a practical way to obtain attention objects. The proposed model works in a manner analogous to humans and has great promise to be a basic tool for content-based image/video applications. Experimental results show the effectiveness of the proposed model.

In this paper, we first apply the theory of wallpaper groups to natural images and extract a novel feature to depict the symmetry property of natural images. The original proposed algorithm takes autocorrelation and correlation as a preprocessing step, which is very time-consuming. Through further analysis, we develop a set of schemes to accelerate this algorithm. Experimental results demonstrate that in performing content-based image retrieval, the proposed symmetry feature outperforms the wavelet feature, which is a widely accepted descriptor of texture, and water-filling feature. The accelerated version of the algorithm improves the processing speed by a large margin while it produces little degradation in retrieval performance.

No-reference quality assessment is a relatively new topic and has been attracting more and more attention in recent years. Due to the limited understanding of the human vision system, most of the existing methods focus on measuring to what extent the image has been distorted. In this paper, by viewing all edge points in JPEG2000 compressed images as 'distorted' or 'un-distorted', we propose using principal component analysis (PCA) to extract the local feature of a given edge point, which indicates both blurring and ringing. We also propose using the probabilities of the given edge point being 'distorted' and 'un-distorted' to model the local distortion metric, which is straightforward and can be easily applied to any type of local feature. Experimental results demonstrate the effectiveness of our scheme.

Caused by light reflected off the subject's retina, red-eye is a troublesome problem in consumer photography. Although most of the cameras have the red-eye reduction mode, the result reality is that no on-camera system is completely effective. In this paper, we propose a fully automatic approach to detecting and correcting red-eyes in digital images. In order to detect red-eyes in a picture, a heuristic yet efficient algorithm is first adopted to detect a group of candidate red regions and then an eye classifier is utilized to confirm whether each candidate region is a human eye. Thereafter, for each detected redeye, we can correct it by the correction algorithm. In case that a red-eye cannot be detected automatically, another algorithm is also provided to detect red-eyes manually with the user's interaction by clicking on an eye. Experimental results on about 300 images with various red-eye appearances demonstrate that the proposed solution is robust and effective.

To improve the performance of image retrieval systems, the well-known semantic gap needs to be bridged. Relevance feedback provides a strategy for learning semantic concepts from visual features. This paper reports a novel framework to learn semantic concepts from the accumulated user feedback log. The semantic concepts consist of two categories: explicit semantics and implicit semantics. The former can be directly estimated by analyzing the user-provided feedback log. The latter is learned according to the obtained explicit semantics. Finally, both explicit and implicit semantics are applied to an image retrieval system. Experiments on 10,000 images show the superiority of the proposed method.

To really bridge the gap between high-level semantics and low-level features in content-based image retrieval (CBIR), a problem that must be solved is which features are suitable for explaining the current query concept. We propose a novel feature selection (FS) criterion based on a psychological similarity measurement, generalized feature contrast model, and implement an online feature selection algorithm in a boosting manner to select the most representative features and do classification during each feedback round. The advantages of the proposed method are: it does not require a Gaussian assumption for "relevant" images as other online FS methods do; it accounts for the intrinsic asymmetry between "relevant" and "irrelevant" image sets in CBIR online learning; it is very fast. Extensive experiments have shown our algorithm's effectiveness.

An entropy-based active learning scheme with support vector machines (SVMs) is proposed for relevance feedback in content-based image retrieval. The main issue in active learning for image retrieval is how to choose images for the user to label in the next interaction. According to information theory, we proposed an entropy-based criterion for good request selection. To apply the criterion with SVMs, probabilistic outputs are required. Since standard SVMs do not provide such outputs, two techniques are used to produce probabilities. One is to train the parameters of an additional sigmoid function. The other is to use the notion of version space. Experimental results on a database of 10,000 general-purpose images demonstrate the effectiveness of the proposed active learning scheme.

With the prevalence of digital cameras, the number of digital images increases quickly, which raises the demand for image quality assessment in terms of blur. Based on the edge type and sharpness analysis, using the Harr wavelet transform, a new blur detection scheme is proposed in this paper, which can determine whether an image is blurred or not and to what extent an image is blurred. Experimental results demonstrate the effectiveness of the proposed scheme.

In this paper, we propose a novel transductive learning framework named manifold-ranking based image retrieval (MRBIR). Given a query image, MRBIR first makes use of a manifold ranking algorithm to explore the relationship among all the data points in the feature space, and then measures relevance between the query and all the images in the database accordingly, which is different from traditional similarity metrics based on pair-wise distance. In relevance feedback, if only positive examples are available, they are added to the query set to improve the retrieval result; if examples of both labels can be obtained, MRBIR discriminately spreads the ranking scores of positive and negative examples, considering the asymmetry between these two types of images. Furthermore, three active learning methods are incorporated into MRBIR, which select images in each round of relevance feedback according to different principles, aiming to maximally improve the ranking result. Experimental results on a general-purpose image database show that MRBIR attains a significant improvement over existing systems from all aspects.

Web image search engine has become an important tool to organize digital images on the Web. However, most commercial search engines still use a list presentation while little effort has been placed on improving their usability. How to present the image search results in a more intuitive and effective way is still an open question to be carefully studied. In this demo, we present iFind, a scalable Web image search engine, in which we integrated two kinds of search result browsing interfaces. User study results have proved that our interfaces are superior to traditional interfaces.

In this paper, we propose and investigate a new user scenario for face annotation, in which users are allowed to multi-select a group of photographs and assign names to these photographs. The system will then attempt to propagate names from photograph level to face level, i.e. to infer the correspondence between name and face. Given the face similarity measure which combines methodologies from face recognition and content-based image retrieval, we formulate name propagation as an optimization problem. We define the objective function as the sum of similarities between each pair of faces of the same individual in different photographs, and propose an iterative optimization algorithm to infer the optimal correspondence. To make the propagation result reliable, a reject scheme is adopted to reject those with low confidence scores. Furthermore, we investigate the combination and alternation of browsing mode for propagation and viewer mode for annotation, so that each mode can benefit from additional inputs from the other mode. The experimental evaluation has been conducted within a typical family album of over one thousand photographs and the results show that the proposed approach is effective and efficient in automated face annotation in family albums.

Relevance feedback and region-based representations are two effective ways to improve the accuracy of content-based image retrieval systems. Although these two techniques have been successfully investigated and developed in the last few years, little attention has been paid to combining them together. We argue that integrating these two approaches and allowing them to benefit from each other will yield better performance than using either of them alone. To do that, on the one hand, two relevance feedback algorithms are proposed based on region representations. One is inspired from the query point movement method. By assembling all of the segmented regions of positive examples together and reweighting the regions to emphasize the latest ones, a pseudo image is formed as the new query. An incremental clustering technique is also considered to improve the retrieval efficiency. The other is the introduction of existing support vector machine-based algorithms. A new kernel is proposed so as to enable the algorithms to be applicable to region-based representations. On the other hand, a rational region weighting scheme based on users' feedback information is proposed. The region weights that somewhat coincide with human perception not only can be used in a query session, but can also be memorized and accumulated for future queries. Experimental results on a database of 10 000 general-purpose images demonstrate the effectiveness of the proposed framework.

Automatic human face detection from images in surveillance and biometric applications is a challenging task due to variations in image background, view, illumination, articulation, and facial expression. We propose a novel three-step face detection approach to addressing this problem. The approach adopts a simple-to-complex strategy. First, a linear-filtering algorithm is applied to enhance detection performance by removing most nonface-like candidates rapidly. Second, a boosting chain algorithm is adopted to combine the boosting classifiers into a hierarchical "chain" structure. By utilizing the inter-layer discriminative information, this algorithm reveals a higher efficiency than traditional approaches. Last, a postfiltering algorithm, consisting of image preprocessing; support vector machine-filter and color-filter, is applied to refine the final prediction. As only a few candidate windows remain in the final stage, this algorithm greatly improves detection accuracy with small computation cost. Compared with conventional approaches, this three-step approach is shown to be more effective and capable of handling more pose variations. Moreover, together with a two-level hierarchy in-plane pose estimator, a rapid multiview face detector is built. Experimental results demonstrate a significant performance improvement for the proposed approach over others.

We present a fully automatic system for face recognition in databases, with only a small number of samples (even a single sample) for each individual. The shape localization problem is formulated in the Bayesian framework. In the learning stage, the RankBoost approach is introduced to model the likelihood of local features associated with the fiducial point, while preserving the prior ranking order between the ground truth position and its neighbors; in the inferring stage, a simple efficient iterative algorithm is proposed to uncover the MAP shape by locally modeling the likelihood distribution around each fiducial point. Based on the accurately located fiducial points, two popular mutual enhancing texture features for human face representation are automatically extracted and integrated: global texture features, which are the normalized shape-free gray-level values enclosed in the mean shape: local texture features, which are represented by Gabor wavelets extracted at the fiducial points (eye corners, mouth, etc.). Global texture mainly encodes the low-frequency information of a face, while local texture encodes the local high-frequency components. Extensive experiments illustrate that our proposed shape localization approach significantly improves the shape location accuracy, robustness, and face recognition rate; moreover, experiments conducted on the FERET and Yale databases show that our algorithm outperforms the classical eigenfaces and fisherfaces, as well as other approaches utilizing shape and global and local textures.

An image retrieval framework that integrates efficient region-based representation in terms of storage and complexity and effective on-line learning capability is proposed. The framework consists of methods for region-based image representation and comparison, indexing using modified inverted files, relevance feedback, and learning region weighting. By exploiting a vector quantization method, both compact and sparse (vector) region-based image representations are achieved. Using the compact representation, an indexing scheme similar to the inverted file technology and an image similarity measure based on Earth Mover's Distance are presented. Moreover, the vector representation facilitates a weighted query point movement algorithm and the compact representation enables a classification-based algorithm for relevance feedback. Based on users' feedback information, a region weighting strategy is also introduced to optimally weight the regions and enable the system to self-improve. Experimental results on a database of 10,000 general-purposed images demonstrate the efficiency and effectiveness of the proposed framework.

We formulate the shape localization problem in the Bayesian framework. In the learning stage, we propose the Constrained Rank-Boost approach to model the likelihood of local features associated with the key points of an object, like face, while preserve the prior ranking order between the ground truth position of a key point and its neighbors; in the inferring stage, a simple efficient iterative algorithm is proposed to uncover the MAP shape by locally modeling the likelihood distribution around each key point via our proposed variational locally weighted learning (VLWL) method. Our proposed framework has the following benefits: 1) compared to the classical PCA models, the likelihood presented by the ranking prior likelihood model has more discriminating power as to the optimal position and its neighbors, especially in the problem with ambiguity between the optimal positions and their neighbors; 2) the VLWL method guarantees that the posterior probability of the derived shape increases monotonously; and 3) the above two methods are both based on accurate probability formulation, which spontaneously leads to a robust confidence measure for the discovered shape. Moreover, we present a theoretical analysis for the convergence of the Constrained Rank-Boost. Extensive experiments compared with the active shape models demonstrate the accuracy, robustness, and stability of our proposed framework.

With the development of content-based multimedia systems, there is a need for automatic extraction of objects from natural images. However, the objects extracted by most existing approaches are often inconsistent with human perception since these approaches totally neglect the viewer's attentions. To address this issue, a method is presented in this paper to automatically extract the viewer's attended objects from an image. Without fully understanding of the semantic content of an image, this method takes advantage of computational attention mechanisms and the seeded region growing technique. It may further facilitate the content-based image/video coding, indexing, and retrieval. Preliminary experimental evaluations on 200 real images demonstrate the effectiveness of this method.

Current image retrieval systems still have major difficulties in bridging the gap between high-level concept and low-level image representation. To overcome these difficulties, a memorization learning model is proposed in this paper. It memorizes the semantic knowledge of images in a database by simply accumulating the user-provided relevance feedback information. From the memorized knowledge, it then learns some hidden semantic information of images. Image retrieval is finally based on a seamless combination of low-level features, memorized semantic information, and estimated hidden semantic information. The model is easy to implement and can be efficiently applied to an image retrieval system. Preliminary experimental results on 10,000 images demonstrate the effectiveness of the proposed model.

This paper presents a new approach to motion blur identification based on Bayesian paradigm and maximum a posteriori (MAP) method. To represent general spatial-invariant motion blurs, especially those caused by nonuniform and nonstraight motions, a new blur model is proposed by incorporating the knowledge of blur type or partially known motion shape as a blur priori. Based on this model, an iterative method is adopted for the MAP blur identification. Experimental result shows that the proposed method offers an efficient way to precisely estimate the motion blur and to generate better restoration result, especially for the widely existing motion blurs caused by complicated motions and cannot be simplified as straight-and-uniform motion blurs.

In this paper, the application of support vector machines (SVM) in relevance feedback for region-based image retrieval is investigated. Both the one class SVM as a class distribution estimator and two classes SVM as a classifier are taken into account. For the latter, two representative display strategies are studied. Since the common kernels often rely on inner product or L/sub p/ norm in the input space, they are infeasible in the region-based image retrieval systems that use variable-length representations. To resolve the issue, a new kind of kernel that is a generalization of Gaussian kernel is proposed. Experimental results on a database of 10,000 general-purpose images demonstrate the effectiveness and robustness of the proposed approach.

We propose a novel approach to memorizing content relevance information accumulated in relevance feedback sessions (historical relevance feedback) in image retrieval. It is based on the idea that by storing identification numbers of several representative images for each positive image after a query session, it is possible to provide more precise and diverse retrieval results when this image is used later as a query in a new search session. To do so, a criterion for a "good" buddy image is proposed. Based on such criterion, an algorithm is designed to maximize the space spanned by the selected buddy images. Since the proposed approach requires only a constant storage space for each image, it has good scalability for a large size of image database. Experimental results on a database of 10,000 images show the high efficiency and good scalability of the proposed memorization scheme.

Automatic annotation of photographs is one of the most desirable needs in family photograph management systems. In this paper, we present a learning framework to automate the face annotation in family photograph albums. Firstly, methodologies of content-based image retrieval and face recognition are seamlessly integrated to achieve automated annotation. Secondly, face annotation is formulated in a Bayesian framework, in which the face similarity measure is defined as maximum a posteriori (MAP) estimation. Thirdly, to deal with the missing features, marginal probability is used so that samples which have missing features are compared with those having the full feature set to ensure a non-biased decision. The experimental evaluation has been conducted within a family album of few thousands of photographs and the results show that the proposed approach is effective and efficient in automated face annotation in family albums.

As current methods for content-based retrieval are incapable of capturing the semantics of images, we experiment with using spectral methods to infer a semantic space from user's relevance feedback, so that our system will gradually improve its retrieval performance through accumulated user interactions. In addition to the long-term learning process, we also model the traditional approaches to query refinement using relevance feedback as a short-term learning process. The proposed short- and long-term learning frameworks have been integrated into an image retrieval system. Experimental results on a large collection of images have shown the effectiveness and robustness of our proposed algorithms.

The region-based approach to image retrieval has emerged as one of the most active research directions in the past few years. There are two crucial problems in the region-based systems: the weighting of regions and the use of relevance feedback. The former plays an important role in computing the region-based similarity of two images, while the latter can improve the efficiency and effectiveness of any CBIR system, if properly employed. In this paper, we propose Key-Region, a novel region weighting scheme that is based on the user's relevance feedback information. The region weight that coincides with human perception can not only be used in a query session, but also be memorized and accumulated for future queries. Experimental results on a database of about 10,000 general-purposed images show the effectiveness of our weighting scheme.

We adopt a local Fourier transform as a texture representation scheme and derive eight characteristic maps for describing different aspects of cooccurrence relations of image pixels in each channel of the (SVcosH, SVsinH, V) color space. Then we calculate the first and second moments of these maps as a representation of the natural color image pixel distribution, resulting in a 48-dimensional feature vector. The novel low-level feature is named color texture moments (CTM), which can also be regarded as a certain extension to color moments in eight aspects through eight orthogonal templates. Experiments show that this new feature can achieve good retrieval performance for CBIR.

Relevance feedback (RF) has become a powerful technique in content-based image retrieval. Most RF methods assume that positive images follow the single Gaussian distribution, which is not sufficient to model the actual distribution of images due to the gap between the semantic concept and low-level features. In this paper, the Gaussian mixture model (GMM) is applied to represent the distribution of positive images in relevance feedback, and a novel method is proposed to estimate the parameters of the GMM. Both positive and negative examples are used to estimate the number of Gaussian components. Furthermore, due to the lack of training samples, unlabeled data are also incorporated to estimate the covariance matrices. Experimental results show that our GMM-based RF method outperforms that based on a single Gaussian model.

As current methods for content-based retrieval are incapable of capturing the semantics of images, we experiment with using spectral methods to infer a semantic space from user's relevance feedback, so the system will gradually improve its retrieval performance through accumulated user interactions. In addition to the long-term learning process, we also model the traditional approaches to query refinement using relevance feedback as a short-term learning process. The proposed short- and long-term learning frameworks have been integrated into an image retrieval system. Experimental results on a large collection of images have shown the effectiveness and robustness of our proposed algorithms.

We present a region-based image retrieval framework that integrates efficient region-based representation in terms of storage and retrieval and effective on-line learning capability. The framework consists of methods for image segmentation and grouping, indexing using modified inverted file, relevance feedback, and continuous learning. By exploiting a vector quantization method, a compact region-based image representation is achieved. Based on this representation, an indexing scheme similar to the inverted file technology is proposed. In addition, it supports relevance feedback based on the vector model with a weighting scheme. A continuous learning strategy is also proposed to enable the system to self improve. Experimental results on a database of 10,000 general-purposed images demonstrate the efficiency and effectiveness of the proposed framework.

Automatic generation of video summarization is one of the key techniques in video management and browsing. In this paper, we present a generic framework of video summarization based on the modeling of viewer's attention. Without fully semantic understanding of video content, this framework takes advantage of computational attention models and eliminates the needs of complex heuristic rules in video summarization. A set of methods of audio-visual attention model features are proposed and presented. The experimental evaluations indicate that the computational attention based approach is an effective alternative to video semantic analysis for video summarization.

MyPhotos is a prototype system for home photo management and processing. Several home user orientated image processing and analysis tools are provided. And several auto grouping methods can help user to organize photos. The system also provides a natural user interface and a workflow for easy browsing and searching.

MyVideos is a prototype system for managing home digital videos. It provides the basic functionalities of video segmentation, summarization, grouping, and editing. It also supports the video playback by shot, group, or highlight.

In this paper, a new boosting algorithm, called FloatBoost, is proposed to construct a strong face-nonface classifier. FloatBoost incorporates the idea of Floating Search into AdaBoost, and yields similar or higher classification accuracy than AdaBoost with a smaller number of weak classifiers. We also present a novel framework for fast multi-view face detection. A detector-pyramid architecture is designed to quickly discard a vast number of non-face sub-windows and hence perform multi-view face detection efficiently. This results in the first real-time multi-view face detection system which runs at 5 frames per second for 320x240 image sequence.

Automatic detection of image orientation is a very important operation in photo image management. In this paper, we propose an automated method based on the boosting algorithm to estimate image orientations. The proposed method has the capability of rejecting images based on the confidence score of the orientation detection. Also, images are classified into indoor and outdoor, and this classification result is used to further refine the orientation detection. To select features more sensitive to the rotation, we combine the features by subtraction operation and select the most useful features by boosting algorithm. The proposed method has several advantages: small model size, fast classification speed, and effective rejection scheme.

This article presents a unified approach to Chinese statistical language modeling (SLM). Applying SLM techniques like trigram language models to Chinese is challenging because (1) there is no standard definition of words in Chinese; (2) word boundaries are not marked by spaces; and (3) there is a dearth of training data. Our unified approach automatically and consistently gathers a high-quality training data set from the Web, creates a high-quality lexicon, segments the training data using this lexicon, and compresses the language model, all by using the maximum likelihood principle, which is consistent with trigram model training. We show that each of the methods leads to improvements over standard SLM, and that the combined method yields the best pinyin conversion result reported.

A novel idea of ubiquitous media agents is presented. Media agents are intelligent systems that are able to automatically collect and build personalized semantic indices of multimedia data on behalf of the user whenever and wherever he/she accesses/uses these multimedia data. The sources of these semantic descriptions are the textual context of the same documents that contain these multimedia data. The URLs of these multimedia data are indexed using these textual features. When the user wants to use these multimedia data once again, the media agents can also help the user find relevant multimedia data and provide proper suggestions based on the semantic indices. The media agents can also learn form the user's interaction records to refine the semantic indices and to model the user intentions and preferences. In our experiments, the media agents are effective in gathering relevant semantics for media objects and learning to provide precise suggestions when the user wants to re-use relevant media objects again.

No abstract available.

Abstract The popularity of digital images is rapidly increasing due to improving digital imaging technologies and convenient availability facilitated by the Internet. However, how to find user‐intended images from the Internet is nontrivial. The main reason is that the Web images are usually not annotated using semantic descriptors. In this article, we present an effective approach to and a prototype system for image retrieval from the Internet using Web mining. The system can also serve as a Web image search engine. One of the key ideas in the approach is to extract the text information on the Web pages to semantically describe the images. The text description is then combined with other low‐level image features in the image similarity assessment. Another main contribution of this work is that we apply data mining on the log of users' feedback to improve image retrieval performance in three aspects. First, the accuracy of the document space model of image representation obtained from the Web pages is improved by removing clutter and irrelevant text information. Second, to construct the user space model of users' representation of images, which is then combined with the document space model to eliminate mismatch between the page author's expression and the user's understanding and expectation. Third, to discover the relationship between low‐level and high‐level features, which is extremely useful for assigning the low‐level features' weights in similarity assessment.

Abstract A novel idea of media agent is briefly presented, which can automatically build a personalized semantic index of Web media objects for each particular user. Because the Web is a rich source of multimedia data and the text content on the Web pages is usually semantically related to those media objects on the same pages, the media agent can automatically collect the URLs and related text, and then build the index of the multimedia data, on behalf of the user whenever and wherever she accesses these multimedia data or their container Web pages. Moreover, the media agent can also use an off‐line crawler to build the index for those multimedia objects that are relevant to the user's favorites but have not accessed by the user yet. When the user wants to find these multimedia data once again, the semantic index facilitates text‐based search for her.

The paper presents a unified approach to Chinese statistical language modeling (SLM). Applying SLM techniques like trigrams to Chinese is challenging because: (1) there is no standard definition of words in Chinese, (2) word boundaries are not marked by spaces, and (3) there is a dearth of training data. Our unified approach automatically and consistently gathers a high-quality training data set from the Web, creates a high-quality lexicon, and segments the training data using this lexicon, all using a maximum likelihood principle, which is consistent with the trigram training. We show that each of the methods leads to improvements over standard SLM, and that the combined method yields the best pinyin conversion result reported.

a lot of problems, including speech recognition, handwriting, Chinese pinyin-input etc. In recognition, statistical language model, such as trigram, is used to provide adequate information to predict the probabilities of hypothesized word sequences. The traditional method relying on distribution estimation are sub-optimal when the assumed distribution form is not the true one, and that "optimality" in distribution estimation does not automatically translate into "optimality" in classifier design. This paper proposed a discriminative training method to minimize the error rate of recognizer rather than estimate the distribution of training data. Furthermore, lexicon is also optimized to minimize the error rate of the decoder through discriminative training. Compared to the traditional LM building method, our systems gets approximately 5%-25% recognition error reduction with discriminative training on language model building.

This paper presents two techniques for language model (LM) adaptation. The first aims to build a more general LM. We propose a distribution-based pruning of n-gram LMs, where we prune n-grams that are likely to be infrequent in a new document. Experimental results show that the distribution-based pruning method performed up to 9 % (word perplexity reduction) better than conventional cutoff methods. Moreover, the pruning method results in a more general n-gram backoff model, in spite of the domain, style, or temporal bias in the training data. The second aims to build a more task-specific LM. We propose an n-gram distribution adaptation method for LM training. Given a large set of out-of-task training data, called training set, andasmall set of task-specific training data, called seed set, we adapt the LM towards the task by adjusting the n-gram distribution in the training set to that in the seed set. Experimental results show non-trivial improvements over conventional methods. 1.

There is a special induction field in the figures of characters which is a physiological and psychological phenomenon and is highly related to human character recognition. In this paper, a formal definition of the field transform is introduced, in which the field of a character image is simulated by a 2-dimensional scalar field which is caused by all the stroke points in the character image. If can be applied to both binary and gray-level images. A very simple character recognition approach is also proposed based on the field transform and is applied to personal handwritten Chinese character recognition. Preliminary experimental results are very promising. Further efforts should be made to find out new ways of utilizing the field transform for character recognition and other applications.

The generalized Hough transform is a useful method for detecting arbitrary 2-dimensional shapes. Based on the consideration that handwritten Chinese characters are complex 2-dimensional shapes, we make an attempt to recognize them by using the generalized Hough transform. Features extracted from the parameter space are used to calculate the similarity measure between two character images. The algorithm is implemented in a look-up-table scheme which can be extended to other applications of pattern recognition. Experimental results on personal (writer-dependent) handwritten Chinese character recognition show that this algorithm is suitable for the pre-classification of personal handwritten Chinese character recognition.