VLDB 2026 Research / reviewers in the wild / expert
Masahiro Toyoura
dblp:12/4116
· DBLP profile ↗
54ranked-venue papers
12as first author
16since 2021 · last 2026
0000-0002-5897-7573ORCID · verified
Domains — the database's venue-derived domains; a paper can count in several
Graphics, computer vision, multimedia, augmented reality and games · 48 · 9 first-author · 13 since 2021Human-computer interaction and ubiquitous computing · 24 · 5 first-author · 6 since 2021Artificial intelligence and machine learning · 3 · 2 first-author · 2 since 2021Applied, interdisciplinary, general and emerging computing · 2 · 1 first-author · 1 since 2021
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | MSGS: Multi-space Gaussian Splatting for mirror reflections
Zhankong Bao, Jiayi Xu 0002, Xuanxuan Huang, Masahiro Toyoura, Gefei Xie, Qianhong Xiang, Huaming Lin |
Comput. Graph. | 4 |
| 2026 | High-quality pattern decoding of large-scale color fabricabstractThis work develops a novel solution for generating binary patterns of large-scale fabrics using deep neural networks. It contributes to textile engineering by enabling the analysis of ancient and modern textile products. There are only two possible over-under relationships between warp and weft yarns at each crossing point, which can be simulated by a binary matrix. Generating a binary pattern from an observed fabric pattern can help designers save time and effort in reproducing fabrics. Deep neural networks have recently been applied in this field and can generate accurate binary patterns that match fabrics; however, these methods still require improvements. This paper introduces a feature point matching-based image stitching method to address the mismatch between the resolution of large fabric images and the network input requirements. Then, we preserve the contrast of color fabric patterns using principal component analysis for grayscale conversion. Finally, we propose a method for deriving pixel-wise confidence values of the label image based on receptive field size and stitching label images by accumulating weights. We show results on Jacquard fabric samples with an average of 266 thousand intersections. The ablation study showed that incorporating the two newly proposed methods achieved the highest accuracy for the textile binary pattern, with an average of 0.952 across samples. Masahiro Toyoura, Qingqi Huang, Renshu Gu, Gang Xu 0001 |
Eng. Appl. Artif. Intell. | 1 |
| 2026 | Unsupervised domain adaptation for cross-modal volumetric medical image segmentation by synergistic alignment and decoupled learning
Renshu Gu, Masahiro Toyoura, Gang Xu 0001 |
Pattern Recognit. | 8 |
| 2025 | Swin-WNet: Boundary-Aware Semantic Segmentation for Oral Squamous Cell CarcinomaabstractOral squamous cell carcinoma (OSCC) poses a significant threat to public health due to its severity and the laborintensive process of image analysis required by physicians. Intercellular bridges are bridge-like structures that connect adjacent cells and indicate the differentiation level of a cancer. Although intercellular bridges are known to disappear as differentiation decreases, pathologists and clinicians evaluate the presence of intercellular bridges to assess the degree of differentiation of cancer. While state-of-the-art (SOTA) deep learning methods, such as U-Net and its variants, perform well on uniform and clearly delineated objects (e.g., cell, lung, etc.), accurately segmenting intricate objects (e.g., intercellular bridge, retinal vessel) remains challenging due to their complex topologies, fine branches, and irregular morphological changes. This paper aims to propose a method that effectively utilize boundary information, particularly targeting intricate objects. Our approach, inspired by Swin-UNet, employs the Swin Transformer, comprising a feature encoder, two decoders (semantic decoder and boundary decoder) and an attention-guided fusion module to enhance the model's ability to segment intricate objects. By applying the constraints of the boundary decoder, the feature encoder's ability to encode structural information is enhanced without compromising semantic information extraction and representation. Furthermore, fusing the outputs of the boundary decoder and semantic decoder further strengthens the detail of structural information. To validate the generalizability of our method, we conducted comparative experiments on one private and one public dataset. The results demonstrate that our method outperforms SOTA methods. Zhenfei Wang, Zhenyang Zhu, Kunio Yoshizawa, Masahiro Toyoura, Naoki Oishi, Xiaoyang Mao |
CW | 4 |
| 2025 | Wavelet Integrated Convolutional Neural Network for ECG Signal Denoising
Takamasa Terada, Masahiro Toyoura |
MMM (4) | 2 |
| 2024 | A Semi-automatic Quality Assessment System for Capturing High-quality Fundus ImageabstractThe precision of medical diagnoses based on images is inextricably linked to the quality and clarity of the images. Poor image quality can impede accurate diagnosis and pose challenges for both physicians and machine learning algorithms in interpreting images. By automating the process of fundus image quality assessment during capture, we can ensure that only high-quality images are used, improving diagnostic accuracy. Accordingly, we propose a novel approach for automatically assessing the quality of fundus images using deep learning techniques. Our method incorporates retinal vessel segmentation into RGB images to create four-channel images and then trains a deep learning model on these images to identify the image focus and quality of the images. It can evaluate the quality of fundus images and request image recapture with the adjustment of specific parameters that are classified as poor quality. Our proposed method has the potential to improve the diagnostic accuracy and efficiency of retinal disease diagnosis, particularly in telemedicine settings. By automating the process of fundus image quality assessment, we can ensure that only high-quality images are used for diagnosis, thus improving diagnostic precision. It can serve as an efficient screening tool in the initial stage of acquiring high-quality fundus images. Asif Mohammed Arfi, Masahiro Toyoura, Kenji Kashiwagi, Satoshi Nishiguchi, Kentaro Go, Zhenyang Zhu, Xiaoyang Mao |
CW | 2 |
| 2024 | Diffusion-driven Cycle-consistent Domain Adaptation for Cross-modality Medical Image SegmentationabstractMedical image segmentation often suffers from performance degradation when applied to images from different domains. To address this, we propose DiMA-Seg (Diffusion Model Adaptation for Segmentation), a novel framework for unsupervised domain adaptation in medical image segmentation. DiMA-Seg combines GAN-based image translation with diffusion model-based feature extraction, leveraging the strengths of both approaches. Our method utilizes the hierarchical nature of diffusion models to extract multi-scale features for accurate segmentation in the target domain. Experiment on MMWHS dataset demonstrates that DiMA-Seg outperforms existing methods in segmentation accuracy. Renshu Gu, Xiangyang Wu 0001, Masahiro Toyoura, Gang Xu 0001 |
CW | 4 |
| 2024 | Whole Slide Image Annotation Support for Estimating Lesion ProportionsabstractThe annotation of medical images for the purpose of segmentation demands a high level of expertise and experience, and the generation of suitable datasets represents a significant challenge. In this study, we propose a method that facilitates annotation on a whole slide image (WSI) without requiring detailed specification of the lesion area. The objective is to enable annotation without the need for specialised knowledge. In particular, the WSI is divided into smaller regions for annotation purposes, with the proportion of lesions in each region being compared with reference images of similar regions, for which the proportion of lesions is known. The proportion of lesions in the reference small region image deemed to be most analogous is then designated as the annotation. The outcomes of an annotation experiment on multiple subjects based on this approach indicated that as the level of experience of the annotator increased, the variability in the accuracy rate diminished. This suggests that the annotators had acquired knowledge and skills through the process of annotation. Nana Takano, Satoshi Nishiguchi, Masahiro Toyoura |
CW | 3 |
| 2023 | Metamorphopsia Insepction System Based on Relevance FeedbackabstractPeople with metamorphopsia suffer from perceiving things in a distorted way. Various methods for examining metamorphopsia have been suggested in the current literature, with the most advanced techniques demonstrating the ability to yield quantitative measurements. However, these cutting-edge methods necessitate extended examination durations and impose challenging manipulations on patients. In this study, our objective is to enhance the time efficiency of the inspection process and alleviate the burden placed on the user. We propose a novel user-friendly quantitative inspection system which utilizes interactive reinforcement learning. Instead of having users directly operate the system, we ask them to evaluate the stimuli generated by the system. Based on their evaluations, the system gradually refines the deformation map representing the distortion perceived by the user. The reinforcement learning scheme is implemented using relevance feedback approach based on optimum-path forest classifier. To evaluate the effectiveness of the proposed system, subjective evaluation experiments involving simulated and real metamorphopsia participants were conducted in this study. The experimental findings reveal that, when compared to the state-of-the-art method, our proposed system yields comparable inspection out-comes while significantly reducing both the inspection duration and the mental workload. Zhenyang Zhu, Katsuhito Moritake, Kenji Kashiwagi, Masahiro Toyoura, Kentaro Go, Issei Fujishiro, Xiaoyang Mao |
SMC | 4 |
| 2023 | Textile image recoloring by polarization observation
Haipeng Luan, Masahiro Toyoura, Renshu Gu, Takamasa Terada, Haiyan Wu, Takuya Funatomi, Gang Xu 0001 |
Vis. Comput. | 2 |
| 2022 | LASOR: Learning Accurate 3D Human Pose and Shape via Synthetic Occlusion-Aware Data and Neural Mesh RenderingabstractA key challenge in the task of human pose and shape estimation is occlusion, including self-occlusions, object-human occlusions, and inter-person occlusions. The lack of diverse and accurate pose and shape training data becomes a major bottleneck, especially for scenes with occlusions in the wild. In this paper, we focus on the estimation of human pose and shape in the case of inter-person occlusions, while also handling object-human occlusions and self-occlusion. We propose a novel framework that synthesizes occlusion-aware silhouette and 2D keypoints data and directly regress to the SMPL pose and shape parameters. A neural 3D mesh renderer is exploited to enable silhouette supervision on the fly, which contributes to great improvements in shape estimation. In addition, keypoints-and-silhouette-driven training data in panoramic viewpoints are synthesized to compensate for the lack of viewpoint diversity in any existing dataset. Experimental results show that we are among the state-of-the-art on the 3DPW and 3DPW-Crowd datasets in terms of pose estimation accuracy. The proposed method evidently outperforms Mesh Transformer, 3DCrowdNet and ROMP in terms of shape estimation. Top performance is also achieved on SSP-3D in terms of shape prediction accuracy. Demo and code will be available at https://igame-lab.github.io/LASOR/. Kaibing Yang, Renshu Gu, Maoyu Wang, Masahiro Toyoura, Gang Xu 0001 |
IEEE Trans. Image Process. | 4 |
| 2022 | Personalized Image Recoloring for Color Vision Deficiency CompensationabstractSeveral image recoloring methods have been proposed to compensate for the loss of contrast caused by color vision deficiency (CVD). However, these methods only work for dichromacy (a case in which one of the three types of cone cells loses its function completely), while the majority of CVD is anomalous trichromacy (another case in which one of the three types of cone cells partially loses its function). In this paper, a novel degree-adaptable recoloring algorithm is presented, which recolors images by minimizing an objective function constrained by contrast enhancement and naturalness preservation. To assess the effectiveness of the proposed method, a quantitative evaluation using common metrics and subjective studies involving 14 volunteers with varying degrees of CVD are conducted. The results of the evaluation experiment show that the proposed personalized recoloring method outperforms the state-of-the-art methods, achieving desirable contrast enhancement adapted to different degrees of CVD while preserving naturalness as much as possible. Zhenyang Zhu, Masahiro Toyoura, Kentaro Go, Kenji Kashiwagi, Issei Fujishiro, Tien-Tsin Wong, Xiaoyang Mao |
IEEE Trans. Multim. | 2 |
| 2022 | Enhancing edge indicator for visual field loss compensation for homonymous hemianopia patients
Keisuke Ichinose, Issei Fujishiro, Masahiro Toyoura, Kenji Kashiwagi, Kentaro Go, Xiaoyang Mao |
Vis. Comput. | 4 |
| 2022 | LineM: assessing metamorphopsia symptom using line manipulation task
Zhenyang Zhu, Masahiro Toyoura, Issei Fujishiro, Kentaro Go, Kenji Kashiwagi, Xiaoyang Mao |
Vis. Comput. | 2 |
| 2021 | Eye-Tracker-Free Compensation for MetamorphopsiaabstractMetamorphopsia is a symptom caused by abnormalities in the retina, and people with metamorphopsia experience distortions in their field of view. To compensate for this disorder, computer-based distortion assessment and compensation methods have been proposed. In the state-of-the-art method, compensation is carried out by dynamically deforming the image with a manipulation map, which is bound to the symptom of an individual user with metamorphopsia, according to the user's gaze data captured by an eye tracker. However, this method suffers from the instability and inaccuracy of the eye tracker, which leads to a poor compensation effect. In this paper, we propose a novel method for metamorphopsia compensation without using an eye tracker. The proposed method generates a compensation image by simultaneously applying multiple manipulation maps to the input image. To evaluate the effectiveness of the proposed method, a preliminary subjective experiment using a reading task and involving 10 participants with normal vision was conducted. The evaluation results show that the proposed method can compensate for visual distortion during reading tasks. Katsuhito Moritake, Zhenyang Zhu, Masahiro Toyoura, Kentaro Go, Kenji Kashiwagi, Issei Fujishiro, Xiaoyang Mao |
CW | 3 |
| 2021 | Fast contrast and naturalness preserving image recolouring for dichromats
Zhenyang Zhu, Kentaro Go, Masahiro Toyoura, Xiaoyang Mao |
Comput. Graph. | 5 |
| 2020 | Using an Eye Tracking Device to Discriminate Different Symptoms in GlaucomaabstractWorldwide, 79.6 million people experience glaucoma, which can cause visual field loss to the individuals. Visual field examination plays an important role in the detection of glaucoma. However, current visual field examination approaches have disadvantages, such as requirements for expensive equipment, long testing time, location restrictions, and more. In the present study, we propose an assessment based on saccadic reaction time (SRT) to overcome the issues in the existing approaches. To confirm the effectiveness of our method, we simulated different stages of glaucoma using a visual field defect simulation system. The result of the visual experiment showed that the discrimination method using SRT can distinguish different symptoms with less testing time. Lina Chen, Kentaro Go, Yuichiro Kinoshita, Kenji Kashiwagi, Masahiro Toyoura, Issei Fujishiro, Xiaoyang Mao |
CW | 5 |
| 2020 | Visual Field Loss Compensation for Homonymous Hemianopia Patients Using Edge IndicatorabstractHomonymous hemianopia (HH) is one kind of visual field defect that the right or left half of the visual fields of both eyes are missing. HH is caused by damage to the neural pathways of the brain and a complete recovery is usually difficult. This study proposes a new information compensation method for HH patients using optical-see-through head mounted display. To avoid causing the occlusion, the proposed technique uses indicators placed at the boundary between the lost and remaining sides of visual field to notify patients about the changes in the lost side. Experiment involving simulated HH participants was conducted to verify the effectiveness of the proposed method and compare with the existing method using overlaid overview window. Objective and subjective evaluation results show that the proposed method can partially alleviate the occlusion problem of overlaid overview window approach and has lower physical and mental demand. Keisuke Ichinose, Issei Fujishiro, Kenji Kashiwagi, Xiaoyang Mao, Masahiro Toyoura, Kentaro Go |
CW | 6 |
| 2020 | Different Eye Movement Patterns on Simulated Visual Field Defects in a Video-watching TaskabstractVisual field defects (VFD) can be caused by a variety of conditions. Checking and tracking the progression of VFD is an important part of an eye assessment. Although the use of standard automatic perimetry (SAP) is very popular for VFD diagnosis, it limits the population because of its high requirement for patients. We used a video-watching task as a replacement modality, which precludes the long period of fixation and uses the on-screen gaze to replace the button response. We developed a simulation system to mimic the different types of VFD in people with a normal pattern.We hypothesize that patients with VFD need more eye movement to compensate for the unseen area. We proposed a metric that indicates the gross eye movements toward a specific direction and found a significant difference between the VFD and normal pattern. Furthermore, we found videos that show the unique eye movement pattern in different eye conditions. Changtong Mao, Kentaro Go, Yuichiro Kinoshita, Kenji Kashiwagi, Masahiro Toyoura, Issei Fujishiro, Jianjun Li 0001, Xiaoyang Mao |
CW | 5 |
| 2020 | Evaluation of Color Vision Compensation Algorithms for People with Varying Degrees of Color Vision DeficiencyabstractPeople with color vision deficiency (CVD) may have difficulty in discriminating colors. To improve their color perception, several compensation methods have been proposed which considered naturalness maintenance and contrast emphasis. All these methods are based on the simulation model of severe CVD and hence it is not clear whether are also effective for people with light CVD. In this paper, we conduct subjective study to evaluate the effectiveness of these methods for people with varying degrees of CVD. Zhenyang Zhu, Masahiro Toyoura, Xiaoyang Mao |
CW | 4 |
| 2020 | Enhancing visual performance of hemianopia patients using overview window
Issei Fujishiro, Kentaro Go, Masahiro Toyoura, Kenji Kashiwagi, Xiaoyang Mao |
Comput. Graph. | 4 |
| 2019 | Visual Assessment of Distorted View for Metamorphopsia Patient by Interactive Line ManipulationabstractThe number of individuals with Age-related Macular Degeneration (AMD) is rapidly increasing. One of the main symptoms of AMD is "metamorphopsia," or distorted vision, which not only makes it difficult for individuals with AMD to do detailed-oriented tasks but also makes sufferers more vulnerable to certain risks in day-to-day life. Traditional clinical approaches to assess metamorphopsia have lacked mechanisms for quantifying the degree of distortion in space, making it impossible to know exactly how individuals with the condition see things. This paper proposes a new method for quantifying distortion in space and visualizing AMD patients' distorted views via line manipulation. By visualizing the distorted views stemming from metamorphopsia, the method gives doctors and others an intuitive picture of how patients see the world and thereby enables a broad range of options for treatment and support. Hiromichi Ichige, Masahiro Toyoura, Kentaro Go, Kenji Kashiwagi, Issei Fujishiro, Xiaoyang Mao |
CW | 2 |
| 2019 | Computational Alleviation of Homonymous Visual Field Defect with OST-HMD: The Effect of Size and Position of Overlaid Overview WindowabstractVisual field defect (VFD) refers to a symptom in which a patient loses part of his/her field of view (FoV). Medical therapy can halt the progression of VFD, but complete recovery is impossible. In this paper, we propose a computational method for alleviating the restricted FoV with an optical see-through head-mounted display (OST-HMD), where an overview scene captured by the installed camera is overlaid on the persisting FoV. Since the overview window occludes with the real world scene, there is a trade-off between the augmented contextual information and the local unscreened information. We hypothesized that such a trade-off can be resolved by taking into consideration the size of the overview window and its displacement from the center of the unimpaired FoV. We therefore conducted an empirical evaluation through a Whac-A-Mole type of task with ten VFD-imitative subjects, where three sizes of an overview window with a fixed aspect ratio and seven positions in terms of elevation and azimuth were used combinatorially on an OST-HMD to find the best size and position of the overview window. It was statistically proven that for left-sided homonymous VFD-imitative subjects, the performance of the task was better when the medium-sized overview window was placed in the lower right position. The obtained result can legitimate default settings for the proposed VFD alleviation method. Kentaro Go, Kenji Kashiwagi, Masahiro Toyoura, Xiaoyang Mao, Issei Fujishiro |
CW | 4 |
| 2019 | Naturalness- and information-preserving image recoloring for red-green dichromatsabstractMore than 100 million individuals around the world suffer from color vision deficiency (CVD). Image recoloring algorithms have been proposed to compensate for CVD. This study has proposed a new recoloring algorithm to make up shortages of contrast enhancement and naturalness preservation of the state-of-the-art methods. The recoloring task is formulated as an optimization problem that is solved by using the colors in a simulated CVD color space to maximize contrast and to preserve the original color as much as possible. In addition, the dominant colors are extracted for recoloring. They are then propagated to the whole image so that the optimization problem could be solved at a reasonable cost independent of the image size. In the quantitative evaluation, the results of the proposed method are competitive with those of the best existing method. The evaluation involving subjects with CVD demonstrates that the proposed method outperforms the state-of-the-art method in preserving both the information and the naturalness of images. Zhenyang Zhu, Masahiro Toyoura, Kentaro Go, Issei Fujishiro, Kenji Kashiwagi, Xiaoyang Mao |
Signal Process. Image Commun. | 2 |
| 2019 | Generating Jacquard Fabric Pattern With Visual ImpressionsabstractWith jacquard fabric, designers can create complex patterns by freely defining the over-under relationships between warp yarns and weft yarns at each grid point or intersections in the fabric. Binary images are one way of representing the over-under relationships of warp and weft yarns at the grid points in a fabric pattern; an image requires an optimal number of warp-weft intersections-not too many, not too few-to produce a weave with both aesthetic and functional merits. This study proposes a method for generating jacquard fabric patterns that reproduce the visual impressions of given input images on jacquard fabric. Our method makes it possible to assign specific fabric dither masks to individual regions. These fabric dither masks can preserve the overall tone of the input image in the dithered image while appropriately controlling the number of warp-weft intersections in the pattern. As the fabric dither masks give users a considerable degree of freedom in defining texture frequencies and directional properties, the proposed method captures the visual impression of a given input image by enabling users to apply fabric dither masks to match textural features-either automatically or interactively. The former approach involves assigning the mask with the closest textural resemblance to each target region, ultimately producing a fabric pattern with a tone and texture that matches the input image. The interactive approach, meanwhile, provides the designer with an interactive interface for assigning masks. This allows designers to experiment with different expressions and track their progress visually, emphasizing or subduing specific areas at their own discretion. Masahiro Toyoura, Tetsuya Igarashi, Xiaoyang Mao |
IEEE Trans. Ind. Informatics | 1 |
| 2019 | Caricature synthesis with feature deviation matching under example-based framework
Masahiro Toyoura, Xiaoyang Mao |
Vis. Comput. | 2 |
| 2019 | Processing images for red-green dichromats compensation via naturalness and information-preservation considered recoloringabstractColor vision deficiency (CVD) is caused by anomalies in the cone cells of the human retina. It affects approximately 200 million individuals throughout the world. Although previous studies have proposed compensation methods, contrast and naturalness preservation have not been adequately and simultaneously addressed in the state-of-the-art studies. This paper focuses on red–green dichromats’ compensation and proposes a recoloring algorithm that combines contrast enhancement and naturalness preservation in a unified optimization model. In this implementation, representative color extraction and edit propagation methods are introduced to maintain global and local information in the recolored image. The quantitative evaluation results showed that the proposed method is competitive with state-of-the-art methods. A subjective experiment was also conducted and the evaluation results revealed that the proposed method obtained the best scores in preserving both naturalness and information for individuals with severe red–green CVD. Zhenyang Zhu, Masahiro Toyoura, Kentaro Go, Issei Fujishiro, Kenji Kashiwagi, Xiaoyang Mao |
Vis. Comput. | 2 |
| 2018 | Visual attention prediction for images with leading line structure
Issei Mochizuki, Masahiro Toyoura, Xiaoyang Mao |
Vis. Comput. | 2 |
| 2017 | Auto-framing based on user camera movementabstractWe propose a novel approach to assisting users with searching the optimal composition of a photograph. In existing studies, the process of detecting the object in a given photo occurred via only image processing, however the result does not always include the object of user's interest. A major technique contribution of our approach is to exploit the user's motion to understand the user's subjective interest in a scene. User's subjective interest and objective structure information of the scene are combined to estimate the best composition based on aesthetic measures. We named this system Auto-Framing. The evaluation result shows that estimated optimal composition closes to the ground-truth. We will embed our technique in an actual camera to enable both automatic detection of compositions and real-time guidance functionality. Tomoya Sawada, Masahiro Toyoura, Xiaoyang Mao |
CGI | 2 |
| 2017 | Synthesis of Facial Images Based on Relevance FeedbackabstractWe propose a dialogic system based on a relevance feedback strategy that allows for the semiautomatic synthesis of a facial image that only exists in a user's mind. The user is presented with several facial images and judges whether each one resembles the face that he or she is imagining. Based on the feedback from the user, a set of sample facial images are used to train an Optimum-Path Forest classifying the relevance of facial images. An interpolation method is then employed to synthesize new facial images that closely resemble the imagined face. A series of experiments are conducted to evaluate and verify the effectiveness and efficiency of the proposed technique. Caie Xu, Shota Fushimi, Masahiro Toyoura, Jiayi Xu 0002, Xiaoyang Mao |
CW | 3 |
| 2016 | Painterly Image Generation Using Scene-Aware Style TransferringabstractIn this paper, we propose a method for painterly image generation that uses an example painting with a similar scene to reflect the style of an original work in great detail. The styles of specific painters and methods often employ different colors and brushwork for each individual subject. Likewise, the connections between various subjects in a work also affect the colors and brushwork used. Our method takes input images, searches an example database for paintings with similar scenes, i.e., paintings in which the subjects have similar positional relationships and connections, and transfers the color and brushwork of the paintings to the corresponding subjects of the target images to generate painterly images that reflect specific styles in great detail. In order to ensure close linkage between various elements and to reproduce styles faithfully, our method applies the GIST approach proposed by Oliva et al. to the process of searching for paintings with similar scenes before performing style transfers. Masahiro Toyoura, Noriyuki Abe, Xiaoyang Mao |
CW | 1 |
| 2016 | Visualizing the lesson process in active learning classesabstractActive learning classes, which aim at increasing student participation in class, demand more management skills from the instructor than a conventional lecture class does. However, the instructor rarely recognizes how his/her lessons are different from those of others. The instructor cannot know exactly how one of his/her lessons is different from his/her previous week's lesson. This class-to-class comparison is effective in improving classes. This paper proposes a method for automatically visualizing the process and content of classes. Although there are ways to visualize the contents of classes manually, these approaches involve considerable investments of time and money. Machine learning techniques can automate the visualization. Our method estimated content with an average accuracy of 72.4%. Through our visualization, we confirmed that individual instructors use time differently from others and use their own time differently from lesson to lesson. Masahiro Toyoura, Mayato Sakaguchi, Xiaoyang Mao, Masanori Hanawa, Masayuki Murakami |
FIE | 1 |
| 2016 | Retrieval of clothing images based on relevance feedback with focus on collar designs
Masahiro Toyoura, Kazumi Shimizu, Xiaoyang Mao |
Vis. Comput. | 2 |
| 2016 | Example-based caricature generation with exaggeration control
Masahiro Toyoura, Jiayi Xu 0002, Fumio Ohnuma, Xiaoyang Mao |
Vis. Comput. | 2 |
| 2015 | Relevance Feedback Based Retrieval of Cloth Image with Focus on Collar DesignabstractAlthough many online shops allow users to search for clothing items by categories or keywords, it is usually a time consuming task to find the item of preferred design. Also users are usually not allowed to specify the details of design. This paper presents a new technology for extracting the feature vectors capturing the details of collar design. A prototype system based on relevance feedback is also developed allowing users search for cloth images with preferred collar design. The effectiveness of the proposed technique has been validated through a subject study. Kazumi Shimizu, Masahiro Toyoura, Xiaoyang Mao |
CW | 3 |
| 2015 | Hidden message in a deformation-based texture
Jiayi Xu 0002, Xiaoyang Mao, Xiaogang Jin 0001, Aubrey Jaffer, Shufang Lu, Li Li 0014, Masahiro Toyoura |
Vis. Comput. | 7 |
| 2014 | A Study on Perceived Similarity between Photograph and Shape Exaggerated CaricatureabstractThis paper investigates the relationship between the extent of exaggeration in a caricature and its face identification ability. As face recognition is largely influenced by facial deformations, we focused on finding the borderline between likeness and unlikeness by applying gradual alterations to the face shape of the subject being studied. Suggestions on manipulating the degree of similarity when generating a caricature will be given. The experimental environment in this research can be used as a user-friendly caricature generation system based on Exaggerating the Difference From the Mean face, which allows a user to freely control each generation step and design his or her own unique caricature portrait. Jiayi Xu 0002, Xiaoyang Mao, Masahiro Toyoura, Xiaogang Jin 0001 |
CW | 4 |
| 2014 | Example-Based Automatic Caricature GenerationabstractCaricature is a popular artistic media widely used for effective communications. The fascination of caricature lies in its expressive depiction of a person's prominent features, which is usually realized through the so called exaggeration technique. This paper proposes a new example based automatic caricature generation system supporting the exaggeration of visual appearance features. The system comprises the construction of a learning database and the generation of caricatures. The construction of the learning database links the pairs of facial images and corresponding caricatures. Given an input face, the system automatically compute the feature vectors of facial parts and hairstyle, and search the learning database for the exaggerated parts by using the most prominent features. Experimental results show that our system can achieve the control over the degree of exaggeration and the exaggerated results can better represent the features of the subjects. Kouki Tajima, Jiayi Xu 0002, Masahiro Toyoura, Xiaoyang Mao |
CW | 4 |
| 2014 | A natural click interface for AR systems with a single camera
Atsushi Sugiura, Masahiro Toyoura, Xiaoyang Mao |
Graphics Interface | 2 |
| 2014 | Mono-spectrum marker: an AR marker robust to image blur and defocus
Masahiro Toyoura, Haruhito Aruga, Matthew Turk 0001, Xiaoyang Mao |
Vis. Comput. | 1 |
| 2013 | Automatic pencil drawing generation using saliency mapabstractAn artist usually does not draw all the areas in a picture homogeneously, but tries to make the work more expressive by emphasizing what is important while eliminating irrelevant details. We present a technique for automatically converting an input image into a pencil drawing with such effect of emphasis and elimination [HATA, et al. 2012]. The technique combines Saliency Map [ITTI, et al. 1998] and Line Integral Convolution(LIC) based pencil drawing filter [MAO, et al. 2001]. Saliency map is used to predict the focus of attention in the input image. Multi-resolution pyramid is used to locally adapt the density and appearance of pencil strokes to the degree of attention defined with saliency map Michitaka Hata, Masahiro Toyoura, Xiaoyang Mao |
SAP | 2 |
| 2013 | Stego-Marbling-TextureabstractWe present stego-marbling-texture, a new and unique texture design method which allows users to deliver personalized messages with beautiful marbling textures. Our approach is inspired by the success of the recent work on modeling traditional marbling operations as mathematical functions. The encrypter transforms an input image or a text message into an intricate marbling pattern using marbling operations defined as reversible functions, and the decrypter recovers the input image or message through reversing the process of marbling operations. When applying marbling operations, the parameters of operations are automatically recorded, encrypted, and then invisibly embedded into the marbling pattern to create a stego-marbling-texture. In this way, the decrypter can be implemented as a stand along software, enabling the receiver to extract the hidden message from the stego-marbling-texture without requiring any extra information from the sender. To ensure that the message is unnoticeably and beautifully covered by the marbling texture, we propose a new technique for automatically creating a background which is harmonious with the input message based on a set of visual perception cues. Jiayi Xu 0002, Xiaoyang Mao, Xiaogang Jin 0001, Aubrey Jaffer, Shufang Lu, Li Li 0014, Masahiro Toyoura |
CAD/Graphics | 7 |
| 2013 | Clikable Virtual Button in Real SpaceabstractSummary form only given. Clicking a virtual object is the most fundamental and important interaction in Augmented Reality (AR). This paper presents a new natural click interface for AR systems. Through a primary study, we found the acceleration of fingertips provides cues for detecting click gesture and succeeded in use it for recognizing natural click gestures with a single camera. The proposed technique was evaluated through a virtual calculator application. Atsushi Sugiura, Masahiro Toyoura, Xiaoyang Mao |
CW | 2 |
| 2013 | Detecting Markers in Blurred and Defocused ImagesabstractPlanar markers enable an augmented reality (AR) system to estimate the pose of objects from images containing them. However, conventional markers are difficult to detect in blurred or defocused images. We propose a new marker and a new detection and identification method that is designed to work under such conditions. The problem of conventional markers is that their patterns consist of high-frequency components such as sharp edges which are attenuated in blurred or defocused images. Our marker consists of a single low-frequency component. We call it a mono-spectrum marker. The mono-spectrum marker can be detected in real time with a GPU. In experiments, we confirm that the mono-spectrum marker can be accurately detected in blurred and defocused images in real time. Using these markers can increase the performance and robustness of AR systems and other vision applications that require detection or tracking of defined markers. Masahiro Toyoura, Haruhito Aruga, Matthew Turk 0001, Xiaoyang Mao |
CW | 1 |
| 2013 | ActVis: Activity Visualization in VideosabstractWe present ActVis, which is a computer-aided video surveillance system for detecting and visualizing the activation levels of multiple objects in a video. ActVis indicates "something is happening" in a video. A user arranges panels indicating the regions of focusing objects on the video screen. Temporal differential as an activation level in a panel is detected by the system, and a corresponding seek bar representing the level is generated. In general, high-level features, such as body posture or facial direction/expression, cannot be extracted when the target object is partially occluded in video, or it is not human. By employing the temporal differential as a low-level feature and the metaphor of a level meter, our system can notify a user "when something happens." The user can explore high-level features of the moment. Potential applications of ActVis include the analysis of student activation levels in classroom for professional development of faculty, and observations of wild animals for ecological investigation. Masahiro Toyoura, Satoshi Nishiguchi, Xiaoyang Mao, Masayuki Murakami |
CW | 1 |
| 2013 | Film Comic Generation with Eye Tracking
Tomoya Sawada, Masahiro Toyoura, Xiaoyang Mao |
MMM (1) | 2 |
| 2012 | Using eye-tracking data for automatic film comic creationabstractA film comic is a kind of art work representing a movie story as a comic. It uses the images of the movie as panels. Verbal information such as dialogue and narrations is represented in word balloons. A key issue in creating film comics is how to select images which are significant in conveying the story of the movie. Such significance of images is inherently semantic and context-dependent and hence, technologies purely based on image analysis usually fail to produce good results. On the other hand, the word balloon arrangement requires understanding not only the semantic of images but also the verbal information, which is difficult except for the case the script of the movie is available. This paper describes a new attempt to use eye-tracking data for the automatic creation of a film comic from a movie. Patterns of eye movement are analyzed for detecting the change of scenes and gaze information is used for automatically finding the location for inserting and directing the word balloons. Our experiments showed that the proposed technique can largely improve the selection of significant images compared with the method using image features only and realize the automatic balloon arrangement. Masahiro Toyoura, Tomoya Sawada, Mamoru Kunihiro, Xiaoyang Mao |
ETRA | 1 |
| 2012 | Film Comic Reflecting Camera-Works
Masahiro Toyoura, Mamoru Kunihiro, Xiaoyang Mao |
MMM | 1 |
| 2012 | Hairstyle Suggestion Using Statistical Learning
Masahiro Toyoura, Xiaoyang Mao |
MMM | 2 |
| 2012 | Be-code: information embedding for logo imagesabstract2D image codes are widely used for inputting information to mobile devices with cameras. QR code is a typical example of such codes. Conventional image codes do not have semantics in images themselves. Such codes may spoil the quality of design when attached on commercial products. We propose a novel code, Be-code, for embedding data into logo images. Be-code is generated from a logo image by modifying some of the edge pixels according to the bits to be embedded, but avoiding causing obvious change to the appearance of the original logo image. Algorithm for extracting information from the camera captured Be-code is also presented. The original logo image is not required in decoding. In experiments, embedded data could be decoded at a high success rate. Kazuha Yamada, Yasuna Yamashita, Masahiro Toyoura, Xiaoyang Mao, Satoshi Takatsu, Chihiro Sugawara |
MoMM | 3 |
| 2012 | Automatic generation of accentuated pencil drawing with saliency map and LIC
Michitaka Hata, Masahiro Toyoura, Xiaoyang Mao |
Vis. Comput. | 2 |
| 2010 | BioMetal gloveabstractWe propose a new haptic device for rendering contact sensation of virtual objects in camera-based Augmented Reality (AR) environments. Haptic feedback can help a user to intuitively sense virtual objects. For vision-impaired users, it also means a transfer from optical information observed in the cameras to haptic information. In our system, the contact between the virtual objects and the user's hand is detected with cameras. Therefore, when presenting the contact sensation, optical markers on the hand should not be occluded from the cameras so as to avoid disturbing the estimation of 3D position and posture of the hand. To fulfill such a requirement, we used BioMetal, a promising and versatile light and thin material that shrinks when electric current is applied, which provides the haptic feedback. Strings of BioMetal were stitched onto our proposed BioMetal glove. Because BioMetal does not shrink instantly when energized, a major challenge is how to deal with the time lag. We address this problem by setting buffering regions for pre-heating the BioMetal strings. Masahiro Toyoura, Tatsuya Shono, Xiaoyang Mao |
VRST | 1 |
| 2008 | 3D shape reconstruction from incomplete silhouettes in multiple framesabstract3D shapes are reconstructed from silhouettes obtained by multiple cameras with the volume intersection method. In recent work, methods of integrating silhouettes in time sequences have been proposed. The number of silhouettes can be increased by integrating silhouettes in multiple frames. The silhouettes of a rigid object in multiple frames are integrated with its rigid motion. This motion is often estimated with 3D feature points extracted from silhouettes. When the estimated motion has large error, shapes are reconstructed with missing parts. This error is given by the incomplete extraction of 3D feature points, which is caused by additional and missing regions of extracted silhouettes. We cannot prevent silhouettes from being extracted with the additional and missing regions in real environments. Here, we propose an intelligent method of integrating incomplete silhouettes where outcrop points, which are 3D feature points for estimating motion, play an important role. The reconstructed shape can be evaluated referring to how many outcrop points have been included in the reconstructed shape of another frame. Although the evaluation does not represent the accuracy of estimated motion directly, it does guarantee that outstanding parts will be preserved in the reconstructed shape. Silhouettes in multiple frames can be integrated with fewer missing and additional parts based on this evaluation. Masahiro Toyoura, Masaaki Iiyama, Takuya Funatomi, Koh Kakusho, Michihiko Minoh |
ICPR | 1 |
| 2006 | Extraction of Outcrop Points from Visual Hulls for Motion EstimationabstractIn this article, we discuss 3D shape reconstruction of an object in a rigid motion with the volume intersection method. When the object moves rigidly, the cameras change their relative positions to the object at every moment. To estimate the motion correctly, we propose new feature points called outcrop points on the reconstructed 3D shape. These points are guaranteed to be located on the real surface of the object. If the rigid motion of the object can be correctly estimated, cameras at different moments serve as the cameras in different positions virtually. With these cameras in time sequences, we can increase accuracy of the reconstructed 3D shape without increasing the number of cameras. Based on this idea, we reconstruct an accurate shape of the object in motion from images obtained by limited number of cameras. As the result, we can acquire an accurate shape from images in time sequences Masahiro Toyoura, Masaaki Iiyama, Koh Kakusho, Michihiko Minoh |
ICME | 1 |