Abstract Traffic volume measurements are increasingly recognized as important for improving society’s infrastructure, by functions such as managing congestion and enhancing logistics. Dedicated traffic data capture devices that use sensors embedded in the road enable accurate measurements but have the problems of high cost and limited installation locations, which make it difficult to expand the coverage of traffic volume measurements. To address this issue, the approach that combines already deployed Closed-Circuit Television (CCTV) cameras with image recognition technology has attracted attention and offers practical performance in ordinary situations. One remaining problem is that accuracy is degraded by the presence of headlight flare at nighttime and occlusion by large vehicles on busy roads. In this paper, we propose a method for measuring traffic volume that automatically sets count-lines using the Kernel Support Vector Machine (Kernel SVM) at optimal positions less affected by these issues. In addition, to make the proposal robust to illumination changes and occlusion we introduce nonlinear count-lines. Extensive experiments on Japanese road video footage shows that our method improves accuracy by $$5.9\%$$ at night and $$2.1\%$$ in situations prone to occlusion compared to the most basic fixed count-line method. Additionally, experiments on a public dataset, UA-DETRAC, demonstrate the proposal’s effectiveness in countries other than Japan.

While person identification from multi/hyperspectral images of hands has advantages such as contactless and high flexibility in capturing images, it remains a difficult task because individual characteristics are not as clear as those of a face or fingerprints. The state-of-the-art method uses a 3D CNN classifier to capture detailed spectral information. However, this is computationally expensive and is negatively affected by undesired spectral variations caused by changes in hand pose. We propose a new method to address these problems. The key technical components of the proposed method are in the introduction of adversarial learning to learn pose-invariant features and in usage of the separable convolutions to decouple the operations in the channel and spatial directions to improve efficiency. Furthermore, these technical components are integrated into a unified supervised contrastive learning framework, which is suitable for person identification. Experimental results demonstrate that our method achieves higher accuracy than the existing method while significantly reducing computational complexity.

Aiming at capturing interactions between a human hand and a desk when seated at a desk, we address the task of localizing the position of a hand touching a desk from the contact sound.Our framework uses an array of contact microphones called piezoelectric devices mounted on the desk to regress the 2D coordinates of the hand position from the resulting contact sound.Training a qualified regression model requires a number of high-quality training samples, but collecting accurate ground truth coordinates is often costly.To address this problem, we propose a novel data augmentation technique customized to 2D regression problems named Bivariate Mixup (BiMixup).BiMixup is a generalization of Mixup, which is formulated as a univariate linear interpolation, to a bivariate quadratic interpolation.Experimental results using a real sound dataset show that our BiMixup reduces the error by 12.2 % compared to the original Mixup.

Japanese comics (called manga) are traditionally created in monochrome format. In recent years, in addition to monochrome comics, full color comics, a more attractive medium, have appeared. Unfortunately, color comics require manual colorization, which incurs high labor costs. Although automatic colorization methods have been recently proposed, most of them are designed for illustrations, not for comics. Unlike illustrations, since comics are composed of many consecutive images, the painting style must be consistent. To realize consistent colorization, we propose here a semi-automatic colorization method based on generative adversarial networks (GAN); the method learns the painting style of a specific comic from small amount of training data. The proposed method takes a pair of a screen tone image and a flat colored image as input, and outputs a colorized image. Experiments show that the proposed method achieves better performance than the existing alternatives.

This paper studies clothing and attribute recognition in the fashion domain. Specifically, in this paper, we turn our attention to the compatibility of clothing items and attributes (Fig 1). For example, people do not wear a skirt and a dress at the same time, yet a jacket and a shirt are a preferred combination. We consider such inter-object or inter-attribute compatibility and formulate a Conditional Random Field (CRF) that seeks the most probable combination in the given picture. The model takes into account the location-specific appearance with respect to a human body and the semantic correlation between clothing items and attributes, which we learn using the max-margin framework. Fig 2 illustrates our pipeline. We evaluate our model using two datasets that resemble realistic applica- tion scenarios: on-line social networks and shopping sites. The empirical evaluation indicates that our model effectively improves the recognition performance over various baselines including the state-of-the-art feature designed exclusively for clothing recognition. The results also suggest that our model generalizes well to different fashion-related applications.

This paper addresses the problem of unsupervised learning of binary hash codes for efficient cross-modal retrieval. Many unimodal hashing studies have proven that both similarity preservation of data and maintenance of quantization quality are essential for improving retrieval performance with binary hash codes. However, most existing cross-modal hashing methods mainly have focused on the former, and the latter still remains almost untouched. We propose a method to minimize the binary quantization errors, which is tailored to cross-modal hashing. Our approach, named Alternating Co-Quantization (ACQ), alternately seeks binary quantizers for each modality space with the help of connections to other modality data so that they give minimal quantization errors while preserving data similarities. ACQ can be coupled with various existing cross-modal dimension reduction methods such as Canonical Correlation Analysis (CCA) and substantially boosts their retrieval performance in the Hamming space. Extensive experiments demonstrate that ACQ can outperform several state-of-the-art methods, even when it is combined with simple CCA.

In this paper, we propose a novel operation support system that utilizes the latest Augmented Reality (AR) technology. Our system consists of a tablet device equipped with a camera. The system automatically recognizes an appliance by reading a QR code captured by the camera and estimates the relative position and orientation between the appliance and the camera in real-time to augment the guidance information to the appliance. Experiments show that our system can stably track camera movement and augment guidance information.

In this paper, we propose a novel method to detect boundaries and estimate figure/ground assignments simultaneously. The proposed approach is based on the observation that the mid-level feature expression for boundary detection can represent local shape of boundaries with high accuracy and high speed [1]. We use figure/ground information to enhance the mid-level features for occlusion boundaries, and propose an algorithm to integrate these mid-level features efficiently. In our global optimization process, efficient and accurate estimation is achieved by superpixel-based combinatorial optimization. Superpixel segmentation is used to reduce the boundary candidates while integrating neighboring classification responses reduces computation time and improves the accuracy of figure/ground assignment. Experiments show that the proposal can detect occlusion boundaries 10 times faster and conduct figure/ground assignment 7.1% more accurately than the current state-of-the-art alternative.

Image/video collection summarization is an emerging paradigm to provide an overview of contents stored in massive databases. Existing algorithms require at least O(N) time to generate a summary, which cannot be applied to online scenarios. Assuming that contents are represented as a sparse graph, we propose a fast image/video collection summarization algorithm using local graph clustering. After a query node is specified, our algorithm first finds a small sub-graph near the query without looking at the whole graph, and then selects fewer number of nodes diverse to each other. Our algorithm thus provides a summary in nearly constant time in the number of contents. Experimental results demonstrate that our algorithm is more than 1500 times faster than a state-of-the-art method, with comparable summarization quality.

A novel region-based approach is proposed to model semantic concepts using web images. Web images are mined to obtain multiple visual patterns automatically that then are used to model a semantic concept. First, the salient region groups corresponding to the representative visual patterns of a concept are mined and selected as positive samples. Next, a representative visual pattern is built in each salient region group by using a BDA classifier. Finally all the visual patterns are aggregated to describe the concept by using a BDA ensemble approach. Because the proposed method models a semantic concept utilizing multiple visual patterns, it enhances the visual variability of a visual model when learning from diverse web images and improves the robustness of the visual model in handling segmentation-related uncertainties. Experiment results demonstrate our method performs well on generic images including not only "object" concepts, but also complex "scene" concepts.

Browsing is a fundamental function in multimedia systems.This paper presents PanoramaExcerpts -a video browsing interface that shows a catalogue of two types of video icons: panoramic and keyframe icons.A panoramic icon is synthesized from a video segment taken with camera pan or tilt, and extracted using a camera operation estimation technique.A keyframe icon is extracted to supplement the panoramic icons; a shot-change detection algorithm is used.A panoramic icon represents the entire visible contents of a scene extended with camera pan or tilt, which is difficult to summarize using a single keyframe.For the automatic generation of PanoramaExcerpts, we propose an approach to integrate the following: (a) a shot-change detection method that detects instantaneous cuts as well as dissolves, with adaptive control over the sampling rate for efficient processing; (b) a method for locating segments that contain smooth camera pans or tilts, from which the panoramic icons can be synthesized; and (c) a layout method for packing icons in a space-efficient manner.We also describe the experimental results of the above three methods and the potential applications of PanoramaExcerpts.

No abstract available.