In this paper, we present a spherical magnetic joint for the inverted locomotion of a multi-legged robot. The permanent magnet’s spherical shape allows the robot to attach its foot to a steel surface without energy consumption. However, the robot’s inverted locomotion requires foot flexibility for placement and gait construction of the robot. Therefore, the spherical magnetic joint mechanism was designed and implemented for the robot feet to deal with angular placement. For decoupling the foot from the steel surface, the attractive force is adjusted by tilting the adjustable sleeve mechanism at an adequate angle between the surface and foot tip. Experimental results show that the spherical magnetic joint can maintain the attractive force at any angle, and the sleeve mechanism can reduce 20% of the reaction force for pulling the legs from the steel surfaces. Furthermore, the designed gait for inverted locomotion with a spherical magnetic joint was tested and compared to prove the concept of the spherical magnetic joint and sleeve mechanism.

In educational settings such as art galleries or museums, Augmented Reality (AR) has the potential to provide detailed information about exhibits. However, dealing with items that contain information in multiple sections or areas is still a significant challenge. For example, a large painting may contain many minute details, which requires a system that can explain its broader features rather than just a generic description. To address this challenge, we introduce an AR guidance system that uses an embodied agent to point out items and explain each piece and part of exhibit items in detail. We also designed and tested 3 different pointing interfaces for the embodied agent: gesture only, gesture with a dot laser, and gesture with line laser. To evaluate this interface, we conducted a user experiment simulating painting guidance to test interest and exhibit memory. During the experiment, the agent pointed to various areas of interest in the painting and provided a detailed description to participants. The result shows that the search times for target positions were the fastest with the line laser. However, no particular interface outperformed others in memory recall of exhibit content.

Drone navigation in complex environments poses many problems to teleoperators. Especially in three dimensional (3D) structures such as buildings or tunnels, viewpoints are often limited to the drone’s current camera view, nearby objects can be collision hazards, and frequent occlusion can hinder accurate manipulation. To address these issues, we have developed a novel interface for teleoperation that provides a user with environment-adaptive viewpoints that are automatically configured to improve safety and provide smooth operation. This real-time adaptive viewpoint system takes robot position, orientation, and 3D point-cloud information into account to modify the user’s viewpoint to maximize visibility. Our prototype uses simultaneous localization and mapping (SLAM) based reconstruction with an omnidirectional camera, and we use the resulting models as well as simulations in a series of preliminary experiments testing navigation of various structures. Results suggest that automatic viewpoint generation can outperform first- and third-person view interfaces for virtual teleoperators in terms of ease of control and accuracy of robot operation.

We present a human-centered designed social drone aiming to be used in a human crowd environment. Based on design studies and focus groups, we created a prototype of a social drone with a social shape, face and voice for human interaction. We used the prototype for a proxemic study, comparing the required distance from the drone humans could comfortably accept compared with what they would require for a nonsocial drone. The social shaped design with greeting voice added decreased the acceptable distance markedly, as did present or previous pet ownership, and maleness. We also explored the proximity sphere around humans with a social shaped drone based on a validation study with variation of lateral distance and heights. Both lateral distance and the higher height of 1.8 m compared to the lower height of 1.2 m decreased the required comfortable distance as it approached.

Low light situations pose a significant challenge to individuals working in a variety of different fields such as firefighting, rescue, maintenance and medicine. Tools like flashlights and infrared (IR) cameras have been used to augment light in the past, but they must often be operated manually, provide a field of view that is decoupled from the operator's own view, and utilize color schemes that can occlude content from the original scene. To help address these issues, we present VisMerge, a framework that combines a thermal imaging head mounted display (HMD) and algorithms that temporally and spectrally merge video streams of different light bands into the same field of view. For temporal synchronization, we first develop a variant of the time warping algorithm used in virtual reality (VR), but redesign it to merge video see-through (VST) cameras with different latencies. Next, using computer vision and image compositing we develop five new algorithms designed to merge non-uniform video streams from a standard RGB camera and small form-factor infrared (IR) camera. We then implement six other existing fusion methods, and conduct a series of comparative experiments, including a system level analysis of the augmented reality (AR) time warping algorithm, a pilot experiment to test perceptual consistency across all eleven merging algorithms, and an in-depth experiment on performance testing the top algorithms in a VR (simulated AR) search task. Results showed that we can reduce temporal registration error due to inter-camera latency by an average of 87.04%, that the wavelet and inverse stipple algorithms were perceptually rated the highest, that noise modulation performed best, and that freedom of user movement is significantly increased with visualizations engaged.

Drone navigation in complex environments poses many problems to teleoperators. Especially in 3D structures like buildings or tunnels, viewpoints are often limited to the drone's current camera view, nearby objects can be collision hazards, and frequent occlusion can hinder accurate manipulation. To address these issues, we have developed a novel interface for teleoperation that provides a user with environment-adaptive viewpoints that are automatically configured to improve safety and smooth user operation. This real-time adaptive viewpoint system takes robot position, orientation, and 3D pointcloud information into account to modify user-viewpoint to maximize visibility. Our prototype uses simultaneous localization and mapping (SLAM) based reconstruction with an omnidirectional camera and we use resulting models as well as simulations in a series of preliminary experiments testing navigation of various structures. Results suggest that automatic viewpoint generation can outperform first and third-person view interfaces for virtual teleoperators in terms of ease of control and accuracy of robot operation.

A better understanding of in vivo bio images is expected to contribute to the discovery of new drugs and mechanisms of disease. To improve the contributions of in vivo bioimaging, the extraction of a particular region is required in order to detect a particular cell's motion because manual image processing of a massive number of images is unrealistic. One of the issues for automatic image-segmentation is that conditions of image-taking are variable. Thus, some manual input and/or manual tuning of some parameters is required to adjust each image. To reduce manual operation for image processing of bone marrow cavity segmentation, we focused on the texture pattern of bone marrow cavity. In this paper, we propose a bone marrow cavity segmentation method using support vector machine and wavelet-based texture feature. The proposed method does not require manual inputs to obtain distribution of intensity before processing, because the texture patterns of bone marrow cavity regions are integrated into the system in advance. Moreover, it is applicable to a particular frame in an image sequence in which the condition of fluorescent material is variable because it does not require temporal variation or initial frame for the segmentation. In the experiment, we evaluated our method with nine types of mother wavelets and several sets of scale parameters. The bone marrow cavity segmentation, using graph-cuts with our texture pattern classification, performs well without manual inputs by a user.

A wide field of view augmented reality display is a special type of head-worn device that enables users to view augmentations in the peripheral visual field. However, the actual effects of a wide field of view display on the perception of augmentations have not been widely studied. To improve our understanding of this type of display when conducting divided attention search tasks, we conducted an in depth experiment testing two view management methods, in-view and in-situ labelling. With in-view labelling, search target annotations appear on the display border with a corresponding leader line, whereas in-situ annotations appear without a leader line, as if they are affixed to the referenced objects in the environment. Results show that target discovery rates consistently drop with in-view labelling and increase with in-situ labelling as display angle approaches 100 degrees of field of view. Past this point, the performances of the two view management methods begin to converge, suggesting equivalent discovery rates at approximately 130 degrees of field of view. Results also indicate that users exhibited lower discovery rates for targets appearing in peripheral vision, and that there is little impact of field of view on response time and mental workload.

This study evaluates the effectiveness of an AR-based context-aware assembly support system with AR visualization modes proposed in object assembly. Although many AR-based assembly support systems have been proposed, few keep track of the assembly status in real-time and automatically recognize error and completion states at each step. Naturally, the effectiveness of such context-aware systems remains unexplored. Our test-bed system displays guidance information and error detection information corresponding to the recognized assembly status in the context of building block (LEGO) assembly. A user wearing a head mounted display (HMD) can intuitively build a building block structure on a table by visually confirming correct and incorrect blocks and locating where to attach new blocks. We proposed two AR visualization modes, one of them that displays guidance information directly overlaid on the physical model, and another one in which guidance information is rendered on a virtual model adjacent to the real model. An evaluation was conducted to comparatively evaluate these AR visualization modes as well as determine the effectiveness of context-aware error detection. Our experimental results indicate the visualization mode that shows target status next to real objects of concern outperforms the traditional direct overlay under moderate registration accuracy and marker-based tracking.

We present an approach which enables real-time augmentation of an environment composed of materials with different texture and reflectance properties without the need of application-specific hardware or extensive preparation. Our solution uses a set of RGB images of a reconstructed model to optimize the reflectance parameters and light location. Each image is decomposed into its specular and diffuse components and we estimate the location of multiple light sources from specular highlights. The environment is stored in a voxel grid and we optimize the reflectance properties and colour of each voxel through inverse rendering. We verify our approach with a simulated environment and present results from a corresponding reconstructed environment.

We introduce an in-situ lighting and reflectance estimation method that does not require specific light probes and/or preliminary scanning. Our method uses images taken from multiple viewpoints while data accumulation and lighting and reflectance estimations run in the background of the primary AR system. As a result, our method requires little in the way of manipulations for image collection because it consists primarily of image processing and optimization. When used, lighting directions and initial optimization values are estimated via image processing. Eventually, the full parameters are obtained by optimization of the differences between real images. This system uses current best parameters because the parameter estimation and input image updates are run independently.

Context aware systems on smart-phones aim to provide useful information by analysing and recognizing users' situations from built-in sensors logs. Especially, predicting user actions is one of the important functions for the context aware systems on smart-phones because this function enables context aware systems to provide proactive and responsive services. Therefore the next location prediction is also an important function for context aware systems. This paper introduces a next location prediction method based on context recognition. In this method, we define a context as combinations of features which are extracted from a set of relational data generated from a phone's sensor logs. We applied Mixed Membership Stochastic Blockmodels (MMSB) to context extraction. We then collected sensor logs of a single user over a period of three months and conducted an evaluation using this collected dataset. An an evaluation using the dataset was conducted and the result shows that 60% of the test dataset ranked in the top 30% of all candidates of the next locations.

Our Pinch-n-Paste allows a user to touch or pinch one part of an object, copy and move its texture, and paste it onto another object, directly with his or her hand, in an augmented reality environment. To transfer texture appropriately from one part of an object to another, two texture images are generated by the Least Square Conformal Map (LSCM) technique. Two regions in the texture images corresponding to source and target areas of interest are then obtained using cross-boundary brushes. Target texel values are sampled from corresponding source texels by Moving Least Squares (MLS), and are finally mapped onto the target object. In this poster, we will describe the basic idea, implementation details, and example interaction results and a preliminary user study.

We propose a waist-mounted projector-camera (ProCam) system for asymmetric remote collaboration. A wearable camera is often used to transmit a worker's situation to a remote instructor, however 3D structure of the worker's environment is not always available and the instructor has a minimal flexibility in changing the camera's viewpoint. A stationary 3D measurement system is also commonly used for remote collaboration, however a narrow measurement area and occlusion from a worker's body can be a severe problem. Our waist-mounted ProCam system reconstructs worker's environment in real-time without occlusion from the worker's body. The remote instructor can give instructions simply by drawing annotations on the reconstructed environment on screen, and they are properly projected in front of the worker. Structured-light based reconstruction, vision-based localization, and visual annotation projection, are processed synchronously with a camera and modified shutter glasses so that both the camera and the worker observe only the information they need. Experimental results show that users prefer our system to a stationary ProCam system.

Our Touch-n-Paste allows a user to touch one part of an object, copy and move its texture, and paste it onto another object, directly with his or her hand, in an augmented reality environment. To transfer texture appropriately from one part of an object to another, two texture images are generated by the Least Square Conformal Map (LSCM) technique. Two regions in the texture images corresponding to source and target areas of interest are then obtained using cross-boundary brushes. Target texel values are sampled from corresponding source texels by Moving Least Squares (MLS), and are finally mapped onto the target object.

We report two user studies on a wearable hyperboloidal head mounted projective display (HHMPD) with a semi-transparent retro-reflective screen. First experiment revealed that a virtual image is perceived at a similar distance as the real image only when the observation distance is within 2.5m with monocular vision, whereas its threshold is further than 3m with stereo (binocular) vision. Second experiment revealed that users are able to identify visual stimuli in the periphery of the visual field up to ±50 degrees in horizontal, while paying attention to a real object in frontal direction.

People carry around mobile devices all the time in their daily life and get various information from the Internet in various situations. When searching for information (content) by using mobile devices, users' activities (e.g., walking and standing) and their situations (e.g., commuting in the morning and going out downtown in the evening) often change and this change may affect their degree of concentration on the display of mobile devices and their information needs. Therefore, search systems should provide users with an amount of information suitable for their activities and with a type of information suitable for their situations. In this paper, we present the design and implementation of a content search system considering mobile users' activities and situations, which aims to reduce users' load of operations in content searching. Our system recognizes user's activities and switches between two kinds of content search systems according to the user's activity: the location-based content search system runs when the user is standing, while the menu-based content search system runs when the user is walking. Both systems present information based on the user's situation. We also introduce a user's activity recognition method for mobile devices. This method classifies user's activity into standing, walking, and running using the sensors equipped in mobile devices.

Smart-phone users can search for information about surrounding facilities or a route to their destination. However, it is difficult to get or search for information while walking because of low legibility. To address this problem, users have to stop walking or enlarge the screen. Our previously proposed system for smart-phone switches the information presentation policies in response to the user's context. In this paper we describe our context recognition mechanism for this system. This mechanism estimates user context from sensors embedded in a smart-phone. We use a Support Vector Machine for the context classification and compare four types of feature values consisting of FFT and 3 types of Wavelet Transforms. Experimental results show that recognition rates are 87.2 % with FFT, 90.9 % with Gabor Wavelet, 91.8 % with Haar Wavelet, and 92.1 % with MexicanHat Wavelet.

Our Pinch-n-Paste allows a user to touch or pinch one part of an object, copy and move its texture, and paste it onto an other object, directly with his or her hand, in an augmented reality environment. To transfer texture appropriately from one part of an object to another, two texture images are generated by the Least Square Conformal Map (LSCM) technique. Two regions in the texture images corresponding to source and target areas of interest are then obtained using cross-boundary brushes. Target texel values are sampled from corresponding source texels by Moving Least Squares (MLS), and are finally mapped onto the target object.

In this paper, we propose a wide-view parallax-free eye-mark recorder with a hyperboloidal half-silvered mirror and a gaze estimation method suitable for the device. Our eye-mark recorder provides a wide field-of-view video recording of the user's exact view by positioning the focal point of the mirror at the user's viewpoint. The vertical angle of view of the prototype is 122 degree (elevation and depression angles are 38 and 84 degree, respectively) and its horizontal view angle is 116 degree (nasal and temporal view angles are 38 and 78 degree, respectively). We implemented and evaluated a gaze estimation method for our eye-mark recorder. We use an appearance-based approach for our eye-mark recorder to support a wide field-of-view. We apply principal component analysis (PCA) and multiple regression analysis (MRA) to determine the relationship between the captured images and their corresponding gaze points. Experimental results verify that our eye-mark recorder successfully captures a wide field-of-view of a user and estimates gaze direction with an angular accuracy of around 2 to 4 degree.

In this paper, we propose a wide-view parallax-free eye-mark recorder with a hyperboloidal half-silvered mirror. Our eye-mark recorder provides a wide field-of-view (FOV) video recording of the user's exact view by positioning the focal point of the mirror at the user's viewpoint. The vertical view angle of the prototype is 122 [deg] (elevation and depression angles are 38 and 84 [deg], respectively) and its horizontal view angle is 116 [deg] (nasal and temporal view angles are 38 and 78 [deg], respectively). We have implemented and evaluated a gaze estimation method for our eyemark recorder. Experimental results have verified that our eye-mark recorder successfully captures a wide FOV of a user and estimates a rough gaze direction.