Multimedia data with associated semantics is omnipresent in today's social online platforms in the form of keywords, user comments, and so forth. This article presents a statistical framework designed to infer knowledge in the imaging domain from the semantic domain. Note that this is the reverse direction of common computer vision applications. The framework relates keywords to image characteristics using a statistical significance test. It scales to millions of images and hundreds of thousands of keywords. We demonstrate the usefulness of the statistical framework with three color imaging applications: 1) semantic image enhancement: re-render an image in order to adapt it to its semantic context; 2) color naming: find the color triplet for a given color name; and 3) color palettes: find a palette of colors that best represents a given arbitrary semantic context and that satisfies established harmony constraints.

Low-cost monitoring cameras/webcams provide unique visual information. To take advantage of the vast image dataset captured by a typical webcam, we consider the problem of retrieving weather information from a database of still images. The task is to automatically label all images with different weather conditions (e.g., sunny, cloudy, and overcast), using limited human assistance. To address the drawbacks in existing weather prediction algorithms, we first apply image segmentation to the raw images to avoid disturbance of the non-sky region. Then, we propose to use multiple kernel learning to gather and select an optimal subset of image features from a certain feature pool. To further increase the recognition performance, we adopt multi-pass active learning for selecting the training set. The experimental results show that our weather recognition system achieves high performance.

This thesis aims at developing methods to make digital devices more automatic and intuitive while focusing on image-related applications. We learn associations between image characteristics and keywords with a statistical framework based on large databases of annotated images. Such associations are widely exploitable and we demonstrate two main applications: semantic image enhancement and automatic color naming. Both applications show convincing results and suggest that the framework can be extended to other areas.

With the advent of social image-sharing communities, millions of images with associated semantic tags are now available online for free and allow us to exploit this abundant data in new ways. We present a fast non-parametric statistical framework designed to analyze a large data corpus of images and semantic tag pairs and find correspondences between image characteristics and semantic concepts. We learn the relevance of different image characteristics for thousands of keywords from one million annotated images. We demonstrate the framework's effectiveness with three different examples of semantic image enhancement: we adapt the gray-level tone-mapping, emphasize semantically relevant colors, and perform a defocus magnification for an image based on its semantic context. The performance of our algorithms is validated with psychophysical experiments.