Aiming to develop of computational grammar system for visual information, we design a 4-tier framework that consists of four levels of 'visual grammar of images.' As a first step of realization, we propose a new dataset, named the PoB dataset, in which each image is annotated with multiple labels of armature patterns that compose the pictorial scene. The PoB dataset includes of a 10,000-painting dataset for art and a 4,959-image dataset for photography. In this paper, we discuss the consistency analysis of our dataset and its applicability. We also demonstrate how the armature patterns in the PoB dataset are useful in assessing aesthetic quality of images, and how well a deep learning algorithm can recognize these patterns. This paper seeks to set a new direction in image understanding with a more holistic approach beyond discrete objects and in aesthetic reasoning with a more interpretative way.

Multimedia retrieval task is faced with increasingly large datasets and variously changing preferences of users in every query. We realize that the high dimensional representation of physical data which previously challenges search algorithms now brings chances to cope with dynamic contexts. In this paper, we introduce a method of building a large-scale video frame retrieval environment with a fast search algorithm that handles user's dynamic contexts of querying by imagination and controlling response time. The search algorithm quickly finds an initial candidate, which has highest-match possibility, and then iteratively traverses along feature indexes to find other neighbor candidates until the input time bound is elapsed. The experimental studies based on the video frame retrieval system show the feasibility and effectiveness of our proposed search algorithm that can return results in a fraction of a second with a high success rate and small deviation to the expected ones. Moreover, its potential is clear that it can scale to large dataset while preserving its search performance.

To maintain and uplift the motivation for doing exercise which is tend to be too simple in rehabilitation and ergotherapy, doing exercise with music is one of the good solutions. In this paper, we confirm that doing exercise with music is fun. Next, we design five types of musical features which relate to exercises. These features are evaluated by doing five types of exercises with listening to the tunes. Based on the results obtained from the experiments, we found many new things, for example, tempo has the adjustment limitation based on how well the user knows the tune. In this paper, we report the latest results from the experiments and explain the way to progress of this research.

Our imagination-based query creation method is a new approach to image-recall functions reflecting color-based imaginations in human brains. This method dynamically represents user's imaginations and creates queries for image retrieval. The important aim of this method is to express user's abstract intentions in a computable objects and to recall those imaginations from image databases with color-analytical image retrieval. The main features of this method are: (1) a hierarchical model for color indexing to express image contexts, (2) color zooming for context-dependent color histogram generation by utilizing color names as contextual words, (3) integrating several histograms by using five histogram-combining operations and dynamic semantic weighting, and (4) threshold controlling for semantic correlation. This method exploits color information to retrieve intended images expressing a user's imagination. This paper also represents a “Database for Imagination Expression” (DBIE), which stores, shares and reuses created queries. Additionally, several qualitative experiments are also represented to examine the effectiveness and the feasibility of our method.