We present a new image search and ranking algorithm for retrieving unannotated images by collaboratively mining online search results which consist of online image and text search results. The online image search results are leveraged as reference examples to perform content-based image search over unannotated images. The online text search results are utilized to estimate the reference images' relevance to the search query. The key feature of our method is its capability to deal with unreliable online image search results through jointly mining visual and textual aspects of online search results. Through such collaborative mining, our algorithm infers the relevance of an online search result image to a text query. Once we obtain the estimate of query relevance score for each online image search result, we can selectively use query specific online search result images as reference examples for retrieving and ranking unannotated images. We tested our algorithm both on the standard public image datasets and several modestly sized personal photo collections. We also compared our method with two well-known peer methods. The results indicate that our algorithm is superior to existing content-based image search algorithms for retrieving and ranking unannotated images.

We propose a personalized re-ranking algorithm through mining user dwell times derived from a user’s previously online reading or browsing activities. We acquire document level user dwell times via a customized web browser, from which we then infer concept word level user dwell times in order to understand a user’s personal interest. According to the estimated concept word level user dwell times, our algorithm can estimate a user’s potential dwell time over a new document, based on which personalized webpage re-ranking can be carried out. We compare the rankings produced by our algorithm with rankings generated by popular commercial search engines and a recently proposed personalized

We propose a new personalized document summarization method that observes a user's personal reading preferences. These preferences are inferred from the user's reading behaviors, including facial expressions, gaze positions, and reading durations that were captured during the user's past reading activities. We compare the performance of our algorithm with that of a few peer algorithms and software packages. The results of our comparative study show that our algorithm can produce more superior personalized document summaries than all the other methods in that the summaries generated by our algorithm can better satisfy a user's personal preferences.

We propose a new method for personalized webpage recommendation. The method is capable of inferring a user's personal reading interest distribution according to implicit user feedbacks coming from the user's past online reading activities. With the inferred user reading interest distribution, we can recommend webpages in a search result set to a user in a personalized way. Our method is featured by its novel approach to observe the facial expressions and gaze positions of a user during the user's online reading activities as two types of implicit user feedbacks for estimating the user's reading interest distribution. To capture these implicit user feedbacks, we use an ordinary web camera and a customized web browser in the setup. The setup allows us to measure the distribution of the reading time a user spends in his or her reading activities over materials of different contents. With all the captured information, our method then estimates a user's reading interest distribution by finding correlations between the implicit feedbacks of a user with the contents of the read materials. Given the estimated user reading interest distribution, our algorithm can further predict the user's potential reading interest in any new webpage. Consequently, our algorithm can produce a personalized webpage recommendation for all the result webpages in an online search session. We compared the performance of our method with that of several mainstream commercial search engines as well as a recent personalized webpage ranking algorithm. The comparison results clearly show the superiority of our new method for personalized webpage recommendation.

We propose an optimal web content provider recommendation algorithm based on mining QoS (quality of service) information of the Internet. The QoS refers principally to the network bandwidth and waiting time (for a connection to be established). For contents replicated over multiple sites, our algorithm recommends a list of webpages having the desired content and ranked according to their QoSs for any specific user. The recommendation is generated through a data mining procedure based on known QoSs of connections between pairs of computers. Our user QoS mining procedure incrementally constructs a neural network group for QoS prediction based on clustering over the prediction errors. An accompanying decision tree algorithm is then used to select the most appropriate neural network among the neural network group to predict the QoS for a particular user connection. Based on our proposed recommendation algorithm, we have implemented a user-oriented search engine which can identify similar web content providers and make a ranked recommendation based on the prediction over the QoS experienced by individual users. Experiment results have verified that our QoS-based personal web content provider ranking algorithm can indeed produce a recommendation that improves the QoS experienced by individual users.

We propose a new document summarization algorithm which is personalized. The key idea is to rely on the attention (reading) time of individual users spent on single words in a document as the essential clue. The prediction of user attention over every word in a document is based on the user's attention during his previous reads, which is acquired via a vision-based commodity eye-tracking mechanism. Once the user's attentions over a small collection of words are known, our algorithm can predict the user's attention over every word in the document through word semantics analysis. Our algorithm then summarizes the document according to user attention on every individual word in the document. With our algorithm, we have developed a document summarization prototype system. Experiment results produced by our algorithm are compared with the ones manually summarized by users as well as by commercial summarization software, which clearly demonstrates the advantages of our new algorithm for user-oriented document summarization.

We propose a new recommendation algorithm for online documents, images and videos, which is personalized. Our idea is to rely on the attention time of individual users captured through commodity eye-tracking as the essential clue. The prediction of user interest over a certain online item (a document, image or video) is based on the user's attention time acquired using vision-based commodity eye-tracking during his previous reading, browsing or video watching sessions over the same type of online materials. After acquiring a user's attention times over a collection of online materials, our algorithm can predict the user's probable attention time over a new online item through data mining. Based on our proposed algorithm, we have developed a new online content recommender system for documents, images and videos. The recommendation results produced by our algorithm are evaluated by comparing with those manually labeled by users as well as by commercial search engines including Google (Web) Search, Google Image Search and YouTube. © 2008 ACM.

Abstract To imitate personal handwritings is non‐trivial. In this paper, we attempt to address the challenging problem of automatic handwriting facsimile. We focus on Chinese calligraphic writings due to their rich variation in style, high artistic values and also the fact that they are among the most difficult candidates for the problem. We first analyze the structures and shapes of the constituent components, i.e., strokes and radicals, of characters in sample calligraphic writings by the same writer. To generate calligraphic writing in the style of the writer, we facsimile the individual character elements as well as the layout relationships used to compose the character, both in the writer's personal writing style. To test our algorithm, we compare our facsimileing results of Chinese calligraphic writings with the original writings. Our results are found to be acceptable for most cases, some of which are difficult to differentiate from the real ones. More results and supplementary materials are provided in our project website at http://www.cs.hku.hk/~songhua/facsimile/ .