One of the most important categories of online advertising is display advertising which provides publishers with significant revenue. Similar to other categories, the main goal in display advertising is to maximize user response rate for advertising campaigns, such as click through rates (CTR) or conversion rates. Previous studies have tried to optimize these parameters using objectives such as behavioral targeting. However, there is no published work so far to address the effect of the visual appearance of ads (creatives) on user response rate via a systematic data-driven approach. In this paper, we quantitatively study the relationship between the visual appearance and performance of creatives using large scale data in the world's largest display ads exchange system, RightMedia. We designed a set of 43 visual features, some of which are novel and others are inspired by related work. We extracted these features from real creatives served on RightMedia. We also designed and conducted a series of experiments to evaluate the effectiveness of visual features for CTR prediction, ranking and performance classification. Based on the evaluation results, we selected a subset of features that have the highest impact on CTR. We believe that the findings presented in this paper will be very useful for the online advertising industry in designing high-performance creatives. It also provides the research community with the first ever data set, initial insights into visual appearance's effect on user response propensity, and evaluation benchmarks for further study.

Display advertising has been a significant source of revenue for publishers and ad networks in online advertising ecosystem. One important business model in online display advertising is Ad Exchange marketplace, also called non-guaranteed delivery (NGD), in which advertisers buy targeted page views and audiences on a spot market through real-time auction. In this paper, we describe a bid landscape forecasting system in NGD marketplace for any advertiser campaign specified by a variety of targeting attributes. In the system, the impressions that satisfy the campaign targeting attributes are partitioned into multiple mutually exclusive samples. Each sample is one unique combination of quantified attribute values. We develop a divide-and-conquer approach that breaks down the campaign-level forecasting problem. First, utilizing a novel star-tree data structure, we forecast the bid for each sample using non-linear regression by gradient boosting decision trees. Then we employ a mixture-of-log-normal model to generate campaign-level bid distribution based on the sample-level forecasted distributions. The experiment results of a system developed with our approach show that it can accurately forecast the bid distributions for various campaigns running on the world's largest NGD advertising exchange system, outperforming two baseline methods in term of forecasting errors.

This paper is concerned with the problem of learning a model to rank objects (Web pages, ads and etc.). We propose a framework where the ranking model is both optimized and evaluated using the same information retrieval measures such as Normalized Discounted Cumulative Gain (NDCG) and Mean Average Precision (MAP). The main difficulty in direct optimization of NDCG and MAP is that these measures depend on the rank of objects and are not differentiable. Most learning-to-rank methods that attempt to optimize NDCG or MAP approximate such measures so that they can be differentiable. In this paper, we propose a simple yet effective stochastic optimization algorithm to directly minimize any loss function, which can be defined on NDCG or MAP for the learning-to-rank problem. The algorithm employs Simulated Annealing along with Simplex method for its parameter search and finds the global optimal parameters. Experiment results using NDCG-Annealing algorithm, an instance of the proposed algorithm, on LETOR benchmark data sets show that the proposed algorithm is both effective and stable when compared to the baselines provided in LETOR 3.0. In addition, we applied the algorithm for ranking ads in contextual advertising. Our method has shown to significantly improve relevance in offline evaluation and business metrics in online tests in a real large-scale advertising serving system. To scale our computations, we parallelize the algorithm in a MapReduce framework running on Hadoop.

The dynamic marketplace in online advertising calls for ranking systems that are optimized to consistently promote and capitalize better performing ads. The streaming nature of online data inevitably makes an advertising system choose between maximizing its expected revenue according to its current knowledge in short term (exploitation) and trying to learn more about the unknown to improve its knowledge (exploration), since the latter might increase its revenue in the future. The exploitation and exploration (EE) tradeoff has been extensively studied in the reinforcement learning community, however, not been paid much attention in online advertising until recently. In this paper, we develop two novel EE strategies for online advertising. Specifically, our methods can adaptively balance the two aspects of EE by automatically learning the optimal tradeoff and incorporating confidence metrics of historical performance. Within a deliberately designed offline simulation framework we apply our algorithms to an industry leading performance based contextual advertising system and conduct extensive evaluations with real online event log data. The experimental results and detailed analysis reveal several important findings of EE behaviors in online advertising and demonstrate that our algorithms perform superiorly in terms of ad reach and click-through-rate (CTR).

Learning to rank is a relatively new field of study, aiming to learn a ranking function from a set of training data with relevancy labels. The ranking algorithms are often evaluated using Information Retrieval measures, such as Normalized Discounted Cumulative Gain [1] and Mean Average Precision [2]. Until recently, most learning to rank algorithms were not using a loss function related to the above mentioned evaluation measures. The main difficulty in direct optimization of these measures is that they depend on the ranks of documents, not the numerical values output by the ranking function. We propose a probabilistic framework that addresses this challenge by optimizing the expectation of NDCG over all the possible permutations of documents. A relaxation strategy is used to approximate the average of NDCG over the space of permutation, and a bound optimization approach is proposed to make the computation efficient. Extensive experiments show that the proposed algorithm outperforms state-of-the-art ranking algorithms on several benchmark data sets.

The taxonomy is the most popular way for organizing a large volume of content. A taxonomy, which is typically a hierarchical representation of categories, provides a navigation structure for exploring and understanding the underlying corpus without sifting through a huge volume of documents. Creating and maintaining taxonomies with a large volume of documents remains a daunting task facing many enterprises. Content organization process typically involves four basic steps: (i) creating taxonomies; (ii) building classification models; (iii) populating taxonomies with documents; and (iv) deploying populated taxonomies in enterprise portals. Each step in the process may have unique requirements that determine what techniques and tools are suitable for the tasks. In this paper, we present a comprehensive suite of tools developed by Verity Inc. for accurately, collaboratively, and efficiently organizing enterprise content.

Verity Inc. has developed a comprehensive suite of tools for accurately and efficiently organizing enterprise content which involves four basic steps: (i) creating taxonomies, (ii) building classification models, (iii) populating taxonomies with documents, and (iv) deploying populated taxonomies in enterprise portals. A taxonomy is a hierarchical representation of categories. A taxonomy provides a navigation structure for exploring and understanding the underlying corpus without sifting through a huge volume of documents. Thematic Mapping automatically discovers a concept tree from a corpus of unstructured documents and assigns meaningful labels to concepts based on a semantic network. Integrating with Verity Intelligent Classifier's user-friendly GUI, a user can drill down a concept tree for navigation, perform a conceptual search to retrieve documents pertaining to a concept, build a taxonomy from the concept tree, as well as edit a taxonomy to tailor it into various views (customized taxonomies) of the same corpus. Classification rules can be generated automatically from concepts. These classification rules can be used for populating documents into the taxonomy.

In the recent years support vector machines (SVM) have been successfully applied to solve a large number of classification problems. Training an SVM, usually posed as a quadratic programming (QP) problem, often becomes a challenging task for the large data sets due to the high memory requirements and slow convergence. We propose to apply boosting to Platt's sequential minimal optimization (SMO) algorithm (1999) and to use resulting Boost-SMO method for speeding and scaling up the SVM training. Experiments on three commonly used benchmark data sets show that Boost-SMO achieves classification accuracy comparable to conventional SMO but is a factor of 3 to 10 faster. The speed-up could easily be orders of magnitude on the larger data sets.

The primary goal of pattern recognition is supervised or unsupervised classification. Among the various frameworks in which pattern recognition has been traditionally formulated, the statistical approach has been most intensively studied and used in practice. More recently, neural network techniques and methods imported from statistical learning theory have been receiving increasing attention. The design of a recognition system requires careful attention to the following issues: definition of pattern classes, sensing environment, pattern representation, feature extraction and selection, cluster analysis, classifier design and learning, selection of training and test samples, and performance evaluation. In spite of almost 50 years of research and development in this field, the general problem of recognizing complex patterns with arbitrary orientation, location, and scale remains unsolved. New and emerging applications, such as data mining, web searching, retrieval of multimedia data, face recognition, and cursive handwriting recognition, require robust and efficient pattern recognition techniques. The objective of this review paper is to summarize and compare some of the well-known methods used in various stages of a pattern recognition system and identify research topics and applications which are at the forefront of this exciting and challenging field.

This paper presents an efficient algorithm for lexicon-driven handwritten word recognition. In this algorithm, a word image is represented by a segmentation graph, and the lexicon is represented by a trie. As opposed to the standard lexicon-driven matching approach, where dynamic programming is invoked independently for matching each entry in the lexicon against the segmentation graph, the proposed algorithm matches the trie with the segmentation graph. Computation is saved by the efficient representation of the lexicon using the trie data structure. The performance of the proposed approach is compared with the standard dynamic programming algorithm. The proposed approach saves about 48.4% (excluding the trie initialization cost) and 15% of computation time from the standard algorithm when a dynamic lexicon is used. Better performance can be expected in static lexicon cases where the trie needs to be constructed only once.

This paper provides a brief review of the state-of-the-art of neural networks in off-line text recognition. We discuss the role that neural networks have played in text recognition. We also assess the state of the art of neural networks in character and word recognition. Despite the success of neural networks in character and word recognition, there are still many challenging problems.

This paper presents a cursive handwritten address recognition system, which consists of four main modules; 1) over-segmentor, 2) dynamic zip locator, 3) zip candidates generator, and 4) city-state-zip verifier. The dynamic zip locator and city-state-zip verifier are based on a flexible matcher for matching a sequence of graphemes with a list of generalized strings. The dynamic zip locator is able to locate zip without knowing exactly where the zip starts. The zip candidates generator uses a hidden Markov model with position-dependent state transition probabilities. A scheme for utilizing prefixes is designed to reduce computation and memory requirement. Finally, the system employs a mechanism for rejection based on rank features extracted from the matching. The overall system achieves an acceptance rate of 83.5% with 3.6% error for 5-digit encoding on 805 USPS cursive address images.

We present a method of combining multiple classifiers for optimizing word recognition. The proposed method combines the results of individual classifiers in such a way that the correct word is more likely to be hypothesized. This method provides a solution to the crucial issue of assigning reliable cost to the edges of the segmentation graph in the popular over-segmentation followed by dynamic programming approach for word recognition. Three combination functions are proposed and implemented. Experiments show that proposed method has a significant improvement on the word recognition accuracy.

We describe a prototype system for reading IATA flight coupons. The system exploits various characteristics of IATA coupons to determine reliably coupon types and field boundaries, and to minimize the amount of manual keying. In particular, we propose a method for extracting and recognizing fixed-pitch characters on noisy images with a complex background. The method does not require a complete drop-out of background, pre-printed text, or lines before recognition, and allows for recovering partially damaged characters (e.g., overlap with form content, handwritten annotations, etc.).

In the above paper by Mao-Jain (ibid., vol.7 (1996)), the Mahalanobis distance is used instead of Euclidean distance as the distance measure in order to acquire the hyperellipsoidal clustering. We prove that the clustering cost function is a constant under this condition, so hyperellipsoidal clustering cannot be realized. We also explains why the clustering algorithm developed in the above paper can get some good hyperellipsoidal clustering results. In reply, Mao-Jain state that the Wang-Xia failed to point out that their HEC clustering algorithm used a regularized Mahalanobis distance instead of the standard Mahalanobis distance. It is the regularized Mahalanobis distance which plays an important role in realizing hyperellipsoidal clusters. In conclusion, the comments made by Wang-Xia together with this response provide some new insights into the behavior of their HEC clustering algorithm. It further confirms that the HEC algorithm is a useful tool for understanding the structure of multidimensional data.

This paper presents a model-based form processing sub-system, which consists of a form model database and five modules: (i) form modeling, (ii) form recognition, (iii) form dropout, (iv) form definition tool, and (v) form reconstruction. The form modeling module builds explicit representations of scanned form templates to facilitate form recognition and dropout. It can also assist a user to define various fields on a form. The automatic form recognition eliminates the need for manually sorting input forms. The form dropout module effectively removes pre-printed form content to achieve a high data compression rate and to provide clean data for OCR. Our model-driven form dropout scheme has two major advantages over image-based subtraction methods in both dropout efficiency and quality preservation of filled-in data.

We propose a self-organizing network for hyperellipsoidal clustering (HEC). It consists of two layers. The first employs a number of principal component analysis subnetworks to estimate the hyperellipsoidal shapes of currently formed clusters. The second performs competitive learning using the cluster shape information from the first. The network performs partitional clustering using the proposed regularized Mahalanobis distance, which was designed to deal with the problems in estimating the Mahalanobis distance when the number of patterns in a cluster is less than or not considerably larger than the dimensionality of the feature space during clustering. This distance also achieves a tradeoff between hyperspherical and hyperellipsoidal cluster shapes so as to prevent the HEC network from producing unusually large or small clusters. The significance level of the Kolmogorov-Smirnov test on the distribution of the Mahalanobis distances of patterns in a cluster to the cluster center under the Gaussian cluster assumption is used as a compactness measure. The HEC network has been tested on a number of artificial data sets and real data sets, We also apply the HEC network to texture segmentation problems. Experiments show that the HEC network leads to a significant improvement in the clustering results over the K-means algorithm with Euclidean distance. Our results on real data sets also indicate that hyperellipsoidal shaped clusters are often encountered in practice.

Forms are an important means for information gathering, storage, retrieval, approval, and distribution in business and government organizations. This paper presents a form dropout approach which employs a distance transform and its gradient flow. Form templates are pre-processed off-line to obtain their distance transforms and gradient flows. We demonstrate that various components in the form dropout algorithm can derive benefit from rich geometric information about the form template which are made explicit in the distance transform and its gradient flow. Experimental results are also presented.

A nonlinear projection method is presented to visualize high-dimensional data as a 2D image. The proposed method is based on the topology preserving mapping algorithm of Kohonen. The topology preserving mapping algorithm is used to train a 2D network structure. Then the interpoint distances in the feature space between the units in the network are graphically displayed to show the underlying structure of the data. Furthermore, we present and discuss a new method to quantify how well a topology preserving mapping algorithm maps the high-dimensional input data onto the network structure. This is used to compare our projection method with a well-known method of Sammon (1969). Experiments indicate that the performance of the Kohonen projection method is comparable or better than Sammon's method for the purpose of classifying clustered data. Its time-complexity only depends on the resolution of the output image, and not on the size of the dataset. A disadvantage, however, is the large amount of CPU time required.

Classical feature extraction and data projection methods have been well studied in the pattern recognition and exploratory data analysis literature. We propose a number of networks and learning algorithms which provide new or alternative tools for feature extraction and data projection. These networks include a network (SAMANN) for J.W. Sammon's (1969) nonlinear projection, a linear discriminant analysis (LDA) network, a nonlinear discriminant analysis (NDA) network, and a network for nonlinear projection (NP-SOM) based on Kohonen's self-organizing map. A common attribute of these networks is that they all employ adaptive learning algorithms which makes them suitable in some environments where the distribution of patterns in feature space changes with respect to time. The availability of these networks also facilitates hardware implementation of well-known classical feature extraction and projection approaches. Moreover, the SAMANN network offers the generalization ability of projecting new data, which is not present in the original Sammon's projection algorithm; the NDA method and NP-SOM network provide new powerful approaches for visualizing high dimensional data. We evaluate five representative neural networks for feature extraction and data projection based on a visual judgement of the two-dimensional projection maps and three quantitative criteria on eight data sets with various properties.

Proposes a node saliency measure and a backpropagation type of algorithm to compute the node saliencies. A node-pruning procedure is then presented to remove insalient nodes in the network to create a parsimonious network. The optimal/suboptimal subset of features are simultaneously selected by the network. The performance of the proposed approach for feature selection is compared with Whitney's feature selection method. One advantage of the node-pruning procedure over classical feature selection methods is that the node-pruning procedure can simultaneously "optimize" both the feature set and the classifier, while classical feature selection methods select the "best" subset of features with respect to a fixed classifier.

Two models, called the multivariate rotation-invariant SAR (RISAR) model and the multiresolution SAR/RISAR (MRSAR/MRRISAR) model, are presented. The information used by the in MRSAR/MRRISAR model is more complete and more independent. Many experiments on image classification using RISAR model and MRRISAR model indicate that these models have very strong classification power, wide area in which they can be used, very good rotation invariance, and high speed for recognition.>