In this paper, we present a new video database: CVD2014-Camera Video Database. In contrast to previous video databases, this database uses real cameras rather than introducing distortions via post-processing, which results in a complex distortion space in regard to the video acquisition process. CVD2014 contains a total of 234 videos that are recorded using 78 different cameras. Moreover, this database contains the observer-specific quality evaluation scores rather than only providing mean opinion scores. We have also collected open-ended quality descriptions that are provided by the observers. These descriptions were used to define the quality dimensions for the videos in CVD2014. The dimensions included sharpness, graininess, color balance, darkness, and jerkiness. At the end of this paper, a performance study of image and video quality algorithms for predicting the subjective video quality is reported. For this performance study, we proposed a new performance measure that accounts for observer variance. The performance study revealed that there is room for improvement regarding the video quality assessment algorithms. The CVD2014 video database has been made publicly available for the research community. All video sequences and corresponding subjective ratings can be obtained from the CVD2014 project page (http://www.helsinki.fi/psychology/groups/visualcognition/).

An effective interaction in augmented reality (AR) requires utilization of different modalities. In this study, we investigated orienting the user in bimodal AR. Using auditory perception to support visual perception provides a useful approach for orienting the user to directions that are outside of the visual field-of-view (FOV). In particular, this is important in path-finding, where points-of-interest (POIs) can be all around the user. However, the ability to perceive the audio POIs is affected by the ventriloquism effect (VE), which means that audio POIs are captured by visual POIs. We measured the spatial limits for the VE in AR using a video see-through head-worn display. The results showed that the amount of the VE in AR was approx. 5°–15° higher than in a real environment. In AR, spatial disparity between an audio and visual POI should be at least 30° of azimuth angle, in order to perceive the audio and visual POIs as separate. The limit was affected by azimuth angle of visual POI and magnitude of head rotations. These results provide guidelines for designing bimodal AR systems.

Most spatially interlacing stereoscopic 3D displays display odd and even rows of an image to either the left or the right eye of the viewer. The visual system then fuses the interlaced image into a single percept. This row-based interlacing creates a small vertical disparity between the images; however, interlacing may also induce horizontal disparities, thus generating depth artifacts. Whether people perceive the depth artifacts and, if so, what is the magnitude of the artifacts are unknown. In this study, we hypothesized and tested if people perceive interlaced edges on different depth levels. We tested oblique edge orientations ranging from 2 degrees to 32 degrees and pixel sizes ranging from 16 to 79 arcsec of visual angle in a depth probe experiment. Five participants viewed the visual stimuli through a stereoscope under three viewing conditions: noninterlaced, interlaced, and row averaged (i.e., where even and odd rows are averaged). Our results indicated that people perceive depth artifacts when viewing interlaced stereoscopic images and that these depth artifacts increase with pixel size and decrease with edge orientation angle. A pixel size of 32 arcsec of visual angle still evoked depth percepts, whereas 16 arcsec did not. Row-averaging images effectively eliminated these depth artifacts. These findings have implications for display design, content production, image quality studies, and stereoscopic games and software.

This paper presents a new database, CID2013, to address the issue of using no-reference (NR) image quality assessment algorithms on images with multiple distortions. Current NR algorithms struggle to handle images with many concurrent distortion types, such as real photographic images captured by different digital cameras. The database consists of six image sets; on average, 30 subjects have evaluated 12-14 devices depicting eight different scenes for a total of 79 different cameras, 480 images, and 188 subjects (67% female). The subjective evaluation method was a hybrid absolute category rating-pair comparison developed for the study and presented in this paper. This method utilizes a slideshow of all images within a scene to allow the test images to work as references to each other. In addition to mean opinion score value, the images are also rated using sharpness, graininess, lightness, and color saturation scales. The CID2013 database contains images used in the experiments with the full subjective data plus extensive background information from the subjects. The database is made freely available for the research community.

Television (TV) programs are increasingly augmented with second-screen applications that can provide complementary information and social media interaction related to the program. However, there is a lack of understanding about the optimal design of such applications. In this paper, we study the effects of complementary information and tweets on the media experience indexed by a comprehensive self-report questionnaire. We built a custom second-screen tablet application and conducted an experiment to investigate the effects of complementary information and tweets on the media experience of TV programs. Apart from reducing concentration and immersion, neither the presence of complementary information nor the presence of tweets affected the media experience of the TV programs, whereas the media experience of second-screen content depended on which program it accompanied. Our results demonstrate that the media experience of complementary information and tweets presented on second screens differs between TV programs.

The affective content of a movie is often considered to be largely determined by its style and aesthetics. Recently, studies have attempted to estimate affective movie content with computational features, but results have been mixed, one of the main reasons being a lack of data on perceptual stylistic and aesthetic attributes of film, which would provide a ground truth for the features. The distinctions between energetic and tense arousal as well as perceived and felt affect are also often neglected. In this study, we present a data set of ratings by 73 viewers of 83 stylistic, aesthetic, and affective attributes for a selection of movie clips containing complete scenes taken from mainstream movies. The affective attributes include the temporal progression of perceived and felt valence and arousal within the clips. The data set is aimed to be used to train algorithms that predict viewer assessments based on low-level computational features. With this data set, we performed a baseline study modeling the relation between a large selection of low-level computational features (i.e., visual, auditory, and temporal) and perceptual stylistic, aesthetic, and affective attributes of movie clips. Two algorithms were compared in a realistic prediction scenario: linear regression and the neural-network-based Extreme Learning Machine (ELM). Felt and perceived affect as well as stylistic attributes were shown to be equally easy to predict, whereas the prediction of aesthetic attributes failed. The performance of the ELM predictor was overall found to be slightly better than the linear regression. A feature selection experiment illustrated that features from all low-level computational modalities, visual, auditory and temporal, contribute to the prediction of the affect assessments. We have made our assessment data and extracted computational features publicly available.

In this paper we present Stage Framework, an HTML5 and CSS3 framework for digital book, magazine and newspaper publishing. The framework offers publishers the means and tools for publishing editorial content in the HTML5 format using a single web application. The approach is cross-platform and is based on open web standards. Stage Framework serves as an alternative for platform-specific native publications using pure HTML5 to deliver book, magazine and newspaper content while retaining the familiar gesture interaction of native applications. Available gesture actions include for example the page swipe and kinetic scrolling. The magazine browsing view relies entirely on CSS3 3D Transforms and Transitions, thus utilizing hardware acceleration in most devices and platforms. The web application also features a magazine stand which, can be used to offer issues of multiple publications. Developed as a part of master's thesis research, the framework has been published under the GPL and MIT licenses and is available to everyone via the framework website (http://stageframework.com) and the GitHub repository (http://github.com/ralatalo/stage).

Significant depth judgment errors are common in augmented reality. This study presents a visualization approach for improving relative depth judgments in augmented reality. The approach uses auxiliary augmented objects in addition to the main augmentation to support ordinal and interval depth judgment tasks. The auxiliary augmentations are positioned spatially near real-world objects, and the location of the main augmentation can be deduced based on the relative depth cues between the augmented objects. In the experimental part, the visualization approach was tested in the “X-ray” visualization case with a video see-through system. Two relative depth cues, in addition to motion parallax, were used between graphical objects: relative size and binocular disparity. The results show that the presence of auxiliary objects significantly reduced errors in depth judgment. Errors in judging the ordinal location with respect to a wall (front, at, or behind) and judging depth intervals were reduced. In addition to reduced errors, the presence of auxiliary augmentation increased the confidence in depth judgments, and it was subjectively preferred. The visualization approach did not have an effect on the viewing time.

Algorithmic image quality metrics have been based on the assumption that an image is only distorted by a single distortion type at a time. The performance of the current metrics is low if image concurrently includes more than one distortion. The aim of this study was to find efficient feature sets for predicting visual quality of real photographs which are subjected to many different distortion sources and types. Features should support each other and function with many concurrent image distortions. We used correlation based feature selector method and image database created with various digital cameras for feature selection. Based on the study the results are promising. Our general and scene-specific feature combinations correlate well with the human observations compared to the state-of-the-art metrics.

Most modern consumer cameras are capable of video capture, but their spatial resolution is generally lower than that of still images. The spatial resolution of videos can be enhanced with a hybrid camera system that combines information from high-resolution still images with low-resolution video frames in a process known as super-resolution. As this process is computationally intensive, we propose a camera system that uses the spatial and temporal information measures SI and TI standardized by ITU as camera parameters to determine during capture whether super-resolution processing would result in an increase in perceived quality. Experimental results show that the difference of these two measures can be used to determine the feasibility of super-resolution processing.

We present a system for automating magazine layout process and data on its performance in a user evaluation test. The purpose of the study is to find the feasibility of the key system variables on the end-user. This semi-automatic system is based on content-based image feature algorithms to automate the layout process. The image related automation includes image cropping, overlaying text on top of images, image color palette creation, and image alignment. The algorithms rely on principles of photography and are used here together in the context of graphic design. For example the rule-of-space and leading line concepts can be extended to layout contexts for improved alignment of text and images. The computation is based on automatic analysis, such as face detection, color saliency, and textureness. We used the automation to create a functional prototype, an iPad magazine, using an open HTML5 eBook framework, which relies on CSS3 media queries for the layout adaptation. User experiments (40 participants) were conducted where the system was compared against two commercial iPad magazine systems in terms of overall usability with emphasis on visual aspects. Questionnaires and free commenting was used in a task-based usage scenario. The experiments were video recorded and the user comments were transcribed and coded into attributes. Usability results mixed with qualitative observation are reported. Results show that usability, read- ability and visuality are important to the users. Our system was considered more usable than the other two systems with some of the defining aspects being simplicity in terms of usability, subjective readability and visual clarity.

In this study the perception of crowds was investigated in urban environment. The images of crowds were viewed non-stereoscopically and stereoscopically with HMD (head-mounted display). The task of the participants was to count the number of persons in the crowds. The results clearly indicate that stereoscopic viewing enhances perception of crowds. The counting task was determined to be easiest with stereoscopic viewing, its error rate was significantly smaller and it was significantly preferred to non-stereoscopic viewing. The viewing method did not differ statistically with respect to the completion time.

The development of visual retrieval methods requires information about user interaction with images, including their description and categorization. This article presents the development of a categorization model for magazine images based on two user studies. In Study 1, we elicited 10 main classes of magazine image categorization criteria through sorting tasks with nonexpert and expert users (N=30). Multivariate methods, namely, multidimensional scaling and hierarchical clustering, were used to analyze similarity data. Content analysis of category names gave rise to classes that were synthesized into a categorization framework. The framework was evaluated in Study 2 by experts (N=24) who categorized another set of images consistent with the framework and found it to be useful in the task. Based on the evaluation study the framework was solidified into a model for categorizing magazine imagery. Connections between classes were analyzed both from the original sorting data and from the evaluation study and included into the final model. The model is a practical categorization tool that may be used in workplaces, such as magazine editorial offices. It may also serve to guide the development of computational methods for image understanding, selection of concepts for automatic detection, and approaches to support browsing and exploratory image search.

One of the most important research goals in media science is a computational model for the human perception of visual quality, that is, how to predict the subjective visual quality experience. This research area has converged to developing new and investigating existing lower-level measurable quantities, physical, visual or computational, which could explain the high level experience. A principal research question, whether the prediction of the visual quality experience based on any lower-level objective measurements is possible at all, has received much less attention. This question is investigated in this study. First, we describe a large psychological experiment where true factors of the human quality experience are pair-wise resolved for dedicatedly selected samples. Second, we describe a ranking measure which reveals the relationship between selected measurable quantities and the human evaluation. Finally, the presented ranking method is used to provide quantitative evidence that visual quality experience can be predicted using lower-level measurable quantities. This result is novel and by simultaneously revealing the underlying lower-level factors it should re-direct the future research towards the true model.

The literature of visual quality is mainly concentrated on devising new physical, visual, or computational quality features which could indirectly reflect ldquotrue visual qualityrdquo. The problem is that the true visual quality is always a subjective and context sensitive judgement of a single individual or a group of individuals. Therefore, the developed methods are only loosely connected to this ultimate objective, and the existing de facto and official standards have been designed by forming a consensus among experts of a specific field (e.g., in the printing industry). In this study, we describe a large psychological experiment where true factors of the human quality experience are pair-wise resolved for dedicatedly selected samples. Then we describe a ranking measure which reveals the relationship between selected measurable quantities and the human evaluation trial. Finally by using the above framework, we devise the best combinations from a set of well-known measurable quantities. The devised combinations can be considered as optimal when agreement with the human visual quality experience is desired, and therefore, they also reveal completely novel information about measuring visual quality.