This paper proposes Wearable Rhythm, a wearable system that encourages awareness of multi-layered, non-linear temporality. In contrast to dominant, efficiency-oriented views of time, the system ambiently conveys rhythms linked to bodily sensations. The current prototype maps physiological and behavioral rhythms into multisensory ambient cues, facilitating an exploration of how these layered temporalities shape an embodied awareness of time.

Reflection emerges when familiar scenes are revisited from a new angle, yet such moments do not always arise on their own during routine encounters. We present LudiScape, an ambient interface developed as a design probe to examine how staged visual augmentation shapes reflective experience through a five-level Visual Transformation Spectrum. LudiScape is instantiated in two metaphorical versions: Mirror for private reflection and Frame for asynchronous shared reflection. In a controlled within-subjects study (N=18), reflective experience increased relative to the baseline level in both versions. However, the most supportive level differed, peaking at Level 2 in Mirror and Level 4 in Frame. The findings suggest that degrees of reflective cue explicitness may support private and shared reflection differently, with lighter prompting appearing more effective in Mirror and more explicit traces appearing more effective in Frame.

We propose a novel technique for mitigating Visually Induced Motion Sickness (VIMS) during video viewing on virtual screens in a mixed-reality (MR) environment. The key idea is to counteract the motion of objects in a video (optical flow) by moving the virtual screen in an opposing direction and speed. We implemented this approach in a head mounted display (HMD) application and conducted the evaluation experiment to examine its effectiveness. The results indicate that the proposed method reduces VIMS in complex clips containing irregular rotations and translational motion. Moreover, it appears to be more effective than dynamic field of view restriction techniques. The proposed method also seems to enhance spatial presence relative to both conventional viewing and dynamic field of view restriction, therefore achieving VIMS mitigation and heightened immersion.

Mixed reality (MR) technologies have a high potential to enhance obstacle negotiation training beyond the capabilities of existing physical systems. Despite such potential, the feasibility of using MR for obstacle negotiation on typical training treadmill systems and its effects on obstacle negotiation performance remains largely unknown. This research bridges this gap by developing an MR obstacle negotiation training system deployed on a treadmill, and implementing two MR systems with a video see-through (VST) and an optical see-through (OST) Head Mounted Displays (HMDs). We investigated the obstacle negotiation performance with virtual and real obstacles. The main outcomes show that the VST MR system significantly changed the parameters of the leading foot in cases of Box obstacle (approximately 22 cm to 30 cm for stepping over 7cm-box), which we believe was mainly attributed to the latency difference between the HMDs. In the condition of OST MR HMD, users tended to not lift their trailing foot for virtual obstacles (approximately 30 cm to 25 cm for stepping over 7cm-box). Our findings indicate that the low-latency visual contact with the world and the user's body is a critical factor for visuo-motor integration to elicit obstacle negotiation.

Physical activity, particularly strength training, is crucial for a healthy lifestyle. Strong mental fortitude is required to endure the repetitive exercise through increased muscle fatigue necessary to enhance the effectiveness of strength training. This paper proposes a system that uses Virtual Reality to virtually alter the color, texture, and size of dumbbells, as well as verify the effects of the system. Our findings indicate a significant improvement in muscular endurance when using unweighted virtual dumbbells, despite the actual weight remaining constant. This improvement is likely attributed to the visual illusion of weight absence leading to increased mental fortitude. Future research should investigate how the color and texture of virtual dumbbells affect muscular endurance.

We present Tippy, a lifelog service designed to maintain human well-being through the practice of tiny habits. Fostering happiness in life is a crucial societal concern, and Tippy seeks to promote sustainable happiness by establishing an ongoing cycle of recording and revisiting positive personal visual moments. Additionally, we delve into some initial insights gleaned from our current prototype.

Virtual Reality (VR) shopping allows users to closely browse goods like the physical store, contributing to a novel shopping experience. Yet, it is still unknown the shopping experiences and the optimal store design for virtual goods that are merely used in virtual environments. We believe the shopping of virtual goods should be further studied, given its inherent distinctions from physical goods in terms of contextual information, complex motivation, and goods properties. In this work, we explore the user experience during virtual goods shopping by comparing diegetic and non-diegetic stores in a VR shooting game. The quantitative and qualitative results from our within-subject study (N=14) confirmed that the diegetic store enhanced the hedonic experiences. A mediation analysis further unveiled the contribution of social interaction to enjoyment and immersion in the diegetic store. Our study furnishes valuable implications for the future design of VR stores and the study of social VR.

Teleportation is a commonly used method in Virtual Reality environments, and specifying a teleportation destination in a 3D space poses a challenge. In this paper, we propose a new method called Gesture Velocity & Teleport (GVT) for teleportation in 3D space, which utilizes the controller’s velocity during a gesture to specify the teleportation target. To assess the effectiveness of GVT, we conduct a controlled lab experiment (N = 24) comparing it with an existing teleportation method.

RamenJourney is a lifelog service designed to recall daily positive memories of ramen geeks. It offers various display modes to highlight individual past positive visual moments. The design principles of RamenJourney are based on an interview conducted with a ramen geek who sought to amplify their positive moments using a social media platform. Furthermore, we have gleaned some preliminary insights from our current prototype.

Wearable telexistence robotic systems have risen in prominence after the COVID-19 pandemic, being vital for various applications. However, their design, fabrication, and evaluation require substantial resources and are unsupported by robot simulation systems. Therefore, we present V-TX, which is a VR-based system for deploying and evaluating wearable telexistence systems. V-TX extends key telexistence systems’ components with versatility for customizing different attributes based on the intended application. To demonstrate the potential of V-TX, we replicated a real telexistence robot in V-TX, called Piton, and showed a novel potential use for exploring and evaluating different robot designs, interaction methods, and deployments. We conducted a comparative evaluation between the real Piton robot and its V-TX implementation. The results show that V-TX gave results that match those of the real counterpart across the main factors of telexistence, indicating its viability as an alternative to traditional evaluation approaches using real robots. Lastly, we discuss the limitations and present future research directions.

Server-less architectures, specifically Function-as-a-Service (FaaS), are sometimes used for data processing in sensor networks. However, due to fluctuations in the start and completion times of FaaS tasks, the order can get reversed even when tasks are required to execute in sequence. As long as the start and end times of FaaS tasks are treated as delay fluctuated, it is not feasible to execute data processing as FaaS tasks. Therefore, this paper aims to clarify and formalize the conditions under which the execution order of periodic tasks by FaaS is maintained, assuming that fluctuations in the start and completion times of FaaS tasks can be predicted within a certain range. Furthermore, it presents a method to minimize the impact on data analysis when the completion of FaaS tasks is delayed beyond that predicted range. It is the first challenge that can provide the direction for future research on addressing the fluctuations.

Abstract To acquire information from the real world and respond appropriately to life's circumstances, vision is indispensable for humans. However, due to its ubiquitous nature, we often perceive the world unconsciously, thereby overlooking the opportunity to contemplate the significance of sight. Seeing goes beyond being a mere method of gathering information; it is an act of uncovering new perspectives and engaging in profound exploration. Theories on creative problem-solving strongly advocate for the advantages of adopting multiple viewpoints. By generating a multitude of alternatives through information gleaned from diverse perspectives, we enhance our ability to expand the range of choices available to us, thus facilitating more effective problem-solving. In this paper, we present Posthuman CollectiveEyes, a digital platform that enriches the human act of visual perception by integrating diverse viewpoints such as collective human, augmented human, and nonhuman viewpoints, and constructs posthuman viewpoints from the diverse viewpoints. In the design of Posthuman CollectiveEyes, we adopt the more-than-human perspective, widely employed in the social sciences to analyze the impact of technology on human actions and decision-making in organizations and societies. This perspective enables us to uncover knowledge that conventional human-centered approaches cannot capture, as the objective of Posthuman CollectiveEyes is to expand human cognitive capabilities through enhanced visual perception. The novel contribution of our approach lies in demonstrating that the design of innovative digital platforms aimed at enhancing human abilities necessitates a fresh design approach that incorporates the more-than-human perspective.

The spread of the new corona virus has imposed limitations on social physical interactions, increasing the importance of remote collaboration and work applications. This has especially effected tasks that require direct physical contact, such as skill-transfer. Accordingly, we have developed T2Remoter, a table tennis instruction system that combines robotics and VR. Our system consists of a VR table-tennis environment that is shared between the coach and the trainee, and a robot arm that is teleoperated by the coach and used to guide the hand-movements of the trainee. To the best of our knowledge, T2Remoter is the first teleoperated VR based table tennis training system. We conducted a series of focus groups to explore critical training routines and contexts for T2Remoter, involving 12 table tennis experts. We extract three main categories of training routines and contexts, and discuss their importance for implementation in T2Remoter.

We propose a Virtual Reality (VR) tennis serve training system based on the machine learning-based 3D skeleton estimation method. Our approach utilizes a single-view 3D skeleton estimation machine learning model to predict the serve motion of a coach from professional tennis players' videos. User movements are acquired from motion capture sensors and compared to the motions of the professional tennis players in real time, and feedback is provided based on the user's body motion. Compared to existing works, our approach can capture and generate a serve motion based on one video, which paves the way for utilizing existing videos of expert tennis players for tennis training in VR. We believe our system can provide a new method to motivate users to practice tennis by directly imitating their favorite expert tennis players. Moreover, we expect our approach to have performance advantages when compared to existing approaches, as it focuses on training that involves motion and posture of the user's entire body.

Telexistence refers to various technologies that enable a high sense of embodiment and interaction capabilities with remote environments. Although numerous telexistence systems have been explored in previous works of literature, no prior studies have investigated the usability of telexistence systems for daily use. Therefore, this study focuses on Piton, which is a novel wearable telexistence robot which has a high degree of freedom in a snake-shaped design. Piton offers a unique opportunity for remote interaction and manipulation in diverse environments, owing to its flexible movement, which enables users to adapt to various scenarios with ease. The robot's wearability is also an essential feature that allows the user to wear it as a backpack. However, despite its promising features, deeper investigation is necessary to explore potential use cases for Piton in users' daily lives and the challenge of Piton.

Recent advances in Augmented and Virtual Reality show that they will play a critical role in our daily lives. However, interaction in such virtual environments is challenging as digital contents cannot be physically felt. Although numerous researchers proposed different methods to achieve haptic feedback, most implementations are cumbersome and unsuitable for daily use. Accordingly, we developed a wearable robotic appendage that can provide various types of haptic feedback in AR. Our system comprises a 4 DoF robot worn on the user's forearm and equipped with a square end-effector. We demonstrate potential applications of our system for providing haptic experiences with UI's in AR and conduct a preliminary evaluation to explore users' impressions of our robotic appendage. Overall, the system is well-received by participants, yet they indicated challenges in weight and hand-tracking. Lastly, we discuss future research directions in light of the design and evaluation of our system.

Abstract Digital platforms are becoming critical infrastructures for supporting a variety of innovative services that enhance our everyday lives. These platforms need to offer not only rational services but also ludic or slow services that focus on human pleasure. One important aspect of creating innovative digital platforms is that their concrete requirements and potential opportunities are vague before they are designed. Thus, designing, prototyping and evaluating digital platforms iteratively is essential for refining or customizing them, as knowledge is gradually gained throughout these iterations. However, it is costly to develop prototype platforms and evaluate them with traditional methods. A better tool that can be used to reveal these platforms’ potential opportunities by conceiving them in a simple and rapid way is needed. In this paper, we present our journey to develop nine digital platforms that share collective human sight and hearing with the Human-Material-Pleasure (HMP) annotation method, which is a tool that we use to describe the visually structured annotations of multiple digital platforms based on the annotated portfolio method. The most significant part of the paper presents annotated portfolios based on the HMP annotation method for the nine digital platforms that we develop and shows how these annotated portfolios play an essential role in revealing and exploring the potential opportunities of our platforms during the refinement process. We also discuss how the HMP annotation method is used in the context of exploring the potential opportunities of wearable shape-changing robotic devices; these devices have significantly different characteristics from our digital platforms, which allows for showing insights more objectively by extracting diverse insights from an alternative angle.

Robotic avatar and telexistence systems have risen in prominence after the covid-19 pandemic, where current telecommunication methods are limited in terms of physical interaction abilities. Most existing systems focus on manual control of the remote robot, where the robot’s arms and head movements follow the user’s movements. Despite the effectiveness of such controls in conveying high levels of embodiment, such control methods jeopardize the efficiency of controls, especially for complex physical manipulation tasks, unclear environments, or unstable communication. Therefore, we propose an assistive-manipulation method to augment users’ control of a telexistence robot during physical manipulation tasks. Machine Learning (ML) was used in the remote environment to localize target objects. This information is sent to the local environment where an inverse kinematic (IK) solution to hold the intended object is generated. The generated IK solution is fused with the one generated by the user’s arm movements. The system enables generating various levels of IK fusion. However, an essential aspect of telexistence is to maintain high levels of embodiment and body ownership over the remote robot. Therefore, the evaluation in this paper focuses on investigating the effect of haptic feedback and the level of IK fusion on body ownership. The results indicate that haptic feedback induced a sense of assurance of task completion and enabling assistance from the system improved the user’s sense of control over the robotic arm.

People usually maintain their positive and openness by watching a variety of views in the world. However, the current situation such as epidemiological situations restricts people’s physical movement and may narrow people’s abilities. Additionally, current digital platforms, especially social media platforms, confine people to a narrow territory by filtering information to only their favorites. Thus, their happiness is achieved through a very narrow view. Sharing Collective Human Eye Views is a digital platform that people can share people’s diverse eye views with the world and show various scenes through people’s eye views.

The sharing economy, which uses digital technologies to share a variety of physical resources in a peer-to-peer manner, has been attracting attention in recent years. There are already widespread services including delivery services using personal cars, accommodation services using private homes, and outsourcing services using each person’s spare time. Investigating the feasibility of sharing other types of previous unexplored physical resources in traditional sharing economy services, particularly human bodies, offers promising opportunities with which to expand the current scope of the sharing economy. For example, if other physical resources, such as human senses, could be shared, we might see new possibilities that are different from the current sharing economy services.

The purpose of this study is to clarify which player perspective (the first-person perspective vs. the third-person perspective) is effective in promoting other-oriented empathy in persuasive virtual reality (VR) games and to provide preliminary insights into the design of persuasive VR games. To verify the hypothesis, participants played a persuasive VR game from each perspective, and their empathy orientation was investigated using a questionnaire.

Nowadays pedestrian navigation not only provides reliable route planning and efficient wayfinding instructions but also visualizes location-based information for supporting exploratory activities, for example, visualizing and recommending some Point-of-interests (POIs). With the Augmented Reality (AR) technology development, it provides more imagination in navigation field, especially in AR POIs visualization of map interface. However, a user’s experience during navigation activities is easy to suffer from information overload and difficulty of identifying the spatial knowledge from POIs. In this work, we aimed to explore a 3D landmark-based map interface in AR navigation to support location awareness and spatial knowledge acquisition. The results of a preliminary user study indicated that participants had positive attitudes to using 3D landmark-based map. The interviews showed it could help them recognize their locations better and learn spatial knowledge of POIs surrounding them, but it also increased mental and effort pressure during different phase of exploration, which engages us to modify our work in the future.

Locomotion systems used in virtual reality (VR) content have a significant impact on the content user experience. One of the most important factors of a walking system in VR is whether it can provide a plausible walking sensation because it is considered directly related to the user’s sense of presence. However, joystick-based and teleportation-based locomotion systems, which are commonly used today, can hardly provide an appropriate sense of presence to a user. To solve this problem, we present Swaying Locomotion, which is a novel VR-based locomotion system that uses head movements to support a user walking in a VR space while actually sitting in real space. Our user study suggests that Swaying Locomotion provides a better walking sensation than the traditional joystick-based approach.

We present a playful citizen science tool to explore various protein docking through dance like body actions for casual users. For more attracting casual users, the tool offers a social watching functionality based on a virtual reality platform that presents multiple persons' visual perspectives in a virtual space. We also investigate some preliminary insights of our current tool.

With the rapid progress of Mixed Reality (MR) technology development, a range of services will available for use in the near future. Given that MR technologies are still evolving, innovating new services that use them can be a difficult process. When designing a new service, scenarios play a vital role in helping to visualize the end product. By using a scenario, designers can explore the opportunities and pitfalls of the service. Prototypes, therefore, are highly desirable for enabling users to experience scenarios immersively. However, current MR devices have serious drawbacks such as a narrow field of view. In this paper, we explore the feasibility of a scenario experience method with virtual reality (VR) technologies for MR service design. We compared a VR based scenario with an MR based scenario by evaluating Ambient Bot, which is an MR service with a 3D character floating around the user that provides information when looked at. By comparing the scenario experience when using the VR device with that of the MR device, we were able to clarify the advantages and disadvantages of the scenario experience in VR.

One of the most important topics in future crowdsourcing is how to influence and coordinate collective people to perform micro-tasks towards their common goal achieved through crowdsourcing activities. For achieving the goal, crowdsourcing needs to take into account coordinating collective people not only encouraging individual people. This paper presents the Game as Rhetoric model that is a framework for designing digital rhetoric, which is embedded in the real world and influences participants' behavior. We then present Collective Action Crowdsourcing and its IoT-based prototype system as an example using the Game as Rhetoric model.

In this paper, we propose a playful tool to predict protein-protein docking. The tool uses human bodies to explore better protein-protein docking, where it offers natural interactions to dock protein molecules in a flexible way and playful interactions through two users' cooperative body movements. We present the design and implementation of the proposed tool and show potential opportunities and pitfalls of the current tool.

Our daily lives are becoming increasingly complex. We sometimes tend to forget what is essential in our daily lives. In the near future, wearable devices such as smart eyeglasses will have cameras, so people's eye views captured by the cameras of the devices may be shared with other people. Our approach projects collective human's eye views in a virtual reality space that can easily be used in our daily lives to help people reflect various issues in the world. To explore our approach, we built an initial prototype system and conducted a preliminary user study to investigate some future challenges of the currently developed prototype system.

Multipurpose wearable robots are an emerging class of devices, which offer intriguing interaction potential. The interoperability of multipurpose wearable robots and other types of wearable devices remains largely uninvestigated. We take the first steps to bridge this gap by presenting a framework for integrating augmented reality (AR) and multipurpose wearable robots. Our framework uses the publisher-subscriber model to expose different robot functionalities as services on a network. These can be invoked by the AR system. This model is advantageous in coping with different robot morphologies and various interaction methods. We implemented a prototype system using our framework by integrating an AR head-mounted display (HMD) and a wrist-worn robot, and demonstrate four experiences utilizing our prototype solution: 1) Robot status display, 2) shape-changing menus, 3) a media player and 4) a robot pose controller. To evaluate our approach, we performed a user study, which gauged user impressions and usability of developed experiences. Results indicate that our approach was well received, though participants highlighted a number of challenges in AR tracking when interacting within some of the experiences. Lastly, we discuss limitations and future research direction for our project.

This paper presents a mobile pervasive mixed reality game based on HoloMoL named Pervasive HoloMoL, where HoloMoL is a platform that helps us memorize different types of information by combining the method of loci memorization methodology and mixed reality (MR) contents delivered through Microsoft HoloLens. The HoloMoL is originally developed as an MR platform to develop information memorization enhancing applications, but we found that the platform can be used to develop broader applications beyond simple information memorization enhancements. In particular, multiple memorization methods used by different participants found in the initial experiment of HoloMoL can be useful as an effective game mechanic to increase a game's engagement. This paper reports an overview of HoloMoL. Then, how the multiple memorization methods in HoloMoL can be used in designing a mobile pervasive game as a primary game mechanic is described. Finally, we show some results and insights of the preliminary user study of Pervasive HoloMoL.

In virtual currency using the blockchain technology that has drawn significant attention in recent years, the mining work concentration problem, whereby machine power is monopolized when supporting its operation, is becoming a serious problem. In particular, with the use of blockchain technology in future advanced mobile computing environments, the problem becomes more serious.

The paper proposes a new way to encourage a community to increase the sustainability of their surroundings based on crowdsourcing. The most important characteristic in the approach is to automatically insert tasks that increase the community's well-being. Therefore, the more tasks the community's members perform that make their surroundings sustainable, the more they increase their well-being. The research is based on a microcrowdfunding crowdsourcing infrastructure. A new functionality to automatically add micro-tasks to increase a community's well-being is designed for enhancing the infrastructure. We also describe a preliminary user study to demonstrate the feasibility of our approach.

Recently, Mobile Virtual Reality (MVR) enclosures are becoming popular due to feasible price and quality. However, interaction in MVR comprises a number of challenges due to the lack of inputs' availability and variety as well as visual occlusion. Thus, we propose an adaptation architecture which enables the utilization of inputs/outputs, embedded in wearables and mobile devices, to interact with MVR contents in a variety of contexts. Our approach consists of two main aspects, context synchronization among MVR and external devices, and adaptive bindings between MVR interaction events and external device's Inputs/Outputs. To evaluate our approach, we implemented our adaptation architecture within two MVR applications. Initial assessment indicated that visual feedback, familiarity of adapted interaction's, and device's input/output attributes impacted the effectiveness of adaptations. Finally, we discuss our future direction.

Input Forager is a user-driven adaptation approach that enables users to flexibly borrow different embedded inputs from smartwatches, smartphones and tablets, then map borrowed inputs to interaction-events that effect Head Worn Display's (HWD) contents. Our approach allows users to form a diverse set of interaction paradigms that involve different device combinations, input types and interaction-event mappings that supplement the limitations of HWD's interaction methods. Such adaptation flexibility allows users to form interaction patterns that correspond to various HWDs' usage contexts. Accordingly, we developed a prototype that comprised an adaptation-client that ran on a smartwatch, a smartphone and a tablet, then carried a user study to evaluate Input Forager within a scenario that resembled HWDs usage contexts. Results indicated that our approach enabled users to reflect both common and distinct input and interaction-mapping preferences. Moreover, qualitative analysis revealed an overall positive usability results, while shortcomings were mainly in the adaptationclients' user interfaces. Finally, we presented the future direction of our project.

Everyday, we receive large amounts of information through our various senses. Since the majority of information is perceived through eye sight, other senses are often left unutilized. As a result, in order to utilize disengaged senses in our daily interactions, especially auditory, we propose a method to extract and transform real-world's visual information to auditory information. Our method is based on converting a real-world's objects' location and distance attributes to sound attributes, which are later be combined to form music. To verify our approach, we developed a prototype that we called Music Sonar, with which we carried out a preliminary user study followed by a questionnaire. The objective of Music Sonar is to reduce the cognitive overload due to visual information by converting visual feedback to ambient auditory feedback, allowing us to use our visual attention for more essential tasks. The user study results confirmed general validity of our approach, despite some shortcomings. Participants also provided a number of usability and interactivity related insights regarding the music based feedback. Finally, we provided our future direction of this research project.

Using computer technologies, the physical world in which we live and the virtual worlds generated by computers or popular cultures are coming closer together. Virtual agents are often designed that refer to humans in the physical world; at the same time, physical world products or services draw stories and emotions using characters. Computers can adjust virtual representations in the physical world based on the information they represent which is possessed by computers, and this adjustment leads to perceived personification. In this paper, we discuss the perception of personification and the possibility of application for persuasion. We developed a prototype application and then conducted surveys and a task-based user study. The results suggest the possibility of persuasion from personified agents superimposed close to an object.

Augmenting semiotic meaning of the real space is a promising approach to altering human attitudes and behaviors because low motivation is one of the biggest reasons why people do not engage in activism. Incorporating fictionality makes us use more persuasive expressions to change the meaning of the real space. However, there are a few discussions about strategies to enhance the meaning of the real space. In this paper, we analyze two case studies that aim to augment the real space. The case studies use a head mounted display to show the real space by capturing it through a camera, but the space is augmented from the true real space. Our focus, in particular, is criticizing how one can feel a sense of reality in the enhanced real space in these case studies because the persuasiveness of the enhanced real space is lost if a user does not feel a sense of reality. We extract effective insights and potential pitfalls from the analyses of the experiences of the researchers who developed and evaluated the case studies.

This paper introduces a new approach, named micro-crowdfunding, for motivating people to participate in achieving a sustainable society. Increasing people's awareness of how they participate in maintaining the sustainability of common resources, such as public sinks, toilets, shelves, and office areas, is central to achieving a sustainable society. Micro-crowdfunding, as proposed in the paper, is a new type of community-based crowdsourcing architecture that is based on the crowdfunding concept and uses the local currency idea as a tool for encouraging people who live in urban environments to increase their awareness of how important it is to sustain small, common resources through their minimum efforts. Because our approach is lightweight and uses a mobile phone, people can participate in micro-crowdfunding activities with little effort anytime and anywhere.

In this paper, a vCPU (virtual CPU) migration mechanism in order to improve real-time responsiveness in a GPOS (General Purpose Operating System) is presented in the embedded multicore virtualization platform and can also be applied to CPS environment. In a GPOS/RTOS (Real Time Operating System) virtualization system nowadays, tasks in GPOS, however, also need some degree of real-time services from the system. Unfortunately, in a traditional virtualization platform, a virtualization layer always gives RTOS vCPU higher priority and preempts the execution of GPOS. Therefore, a kernel module in the GPOS is added in our virtualization system to export and boost a GPOS vCPU contexts which needs higher priority against RTOS vCPU. At the same time, in order not to sacrifice the performance of the victim RTOS vCPU, a vCPU migration mechanism is added to our virtualization system, and migrates the RTOS vCPU to a low-loading CPU when this vCPU is preempted. Performance improvement of GPOS's real-time responsiveness is also given while a detailed analysis of the overhead of the RTOS vCPU is performed as well.

MoboQ is a location-based real-time social question answering service deployed in the field in China. Using MoboQ, people can ask temporal and geo-sensitive questions, such as how long is the line at a popular business right now, and then receive answers that crowdsourced from other users in a timely fashion. To obtain answers for questions, the system analyzes the live stream from public microblogging service Sina Weibo to identify people who are likely to currently be at the place that is associated with a question and sends them the unsolicited question through the microblogging service from which they were identified. MoboQ was deployed in China at the beginning of 2012, until October of the same year, it was used to ask 15,224 questions by 35,214 registered users, and it gathered 29,491 answers; 74.6% of the questions received at least one answer, 28% received a first response within 10 minutes, and 51% of the questions got first answer within 20 minutes. In total, 91% of the questions successfully found at least one answer candidate, and they were sent to 162,954 microblogging service users. We analyze the usage patterns and behaviors of the real-world end-users, discuss the lessons learned, and outline the future directions and possible applications that could be built on top of MoboQ.

System integrity monitors, such as rootkit detectors, rely critically on the ability to fetch and inspect pages containing code and data of a target system under study. To avoid being infected by malicious or compromised targets, state-of-the-art system integrity monitors rely on virtualization technology to set up a tamper-proof execution environment. Consequently, the virtualization infrastructure is part of the trusted computing base. However, modern virtual machine monitors are complex entities, with large code bases that are difficult to verify. In this paper, we present a new machine architecture called limited local memory (LLM), which we use to set up an alternative tamper-proof execution environment for system integrity monitors. This architecture builds upon recent trends in multicore chip design to equip each processing core with access to a small, private memory area. We show that the features of the LLM architecture, combined with a novel secure paging mechanism, suffice to bootstrap a tamper-proof execution environment without support for hardware virtualization. We demonstrate the utility of this architecture by building a rootkit detector that leverages the key features of LLM. This rootkit detector can safely inspect a target operating system without itself becoming the victim of infection.

In this paper, we introduce a lightweight processor abstraction layer named SPUMONE. SPUMONE provides virtual CPUs for respective guest OSes, and schedules them according to their priorities. In a typical case, SPUMONE schedules Linux with a low priority and an RTOS with a high priority. We first discuss why the traditional virtual machine monitor design is not appropriate for embedded systems, and how the features of SPUMONE allow us to design modern complex embedded systems with less efforts. Then, we describe two features of SPUMONE for the real-time resource management. SPUMONE also offers a novel mechanism to protect a critical component from malicious programs injected into the GPOS kernel.

Long hours of training are required to achieve perfect motion in certain sports such as martial arts and dance. In this paper, we are trying to develop a game-based learning system for martial arts that required emphasis of gesture movement with the aid of Kinect sensor. Emergence of controller-less sensors in gaming industry was said to exhibit the potential of bringing joy and fun into teaching and learning experience. We had seen many commercialized controller-less sensors that enhanced the game experience, brought us one step closer to the virtual reality. No doubts, games have unique opportunity to create experiences that unfold over long span of time. Why not we have them kill two birds with one stone? Besides the pleasure brought by the game, we can have the player indirectly learn the real sport through the game play and subsequently train them within the long span of time during game play. Our main idea is to unobtrusively teach player about martial arts during the game play. We have chosen Kinect sensor for its affordability and ease of programming. We have evaluated accuracy of Kinect sensor for motions capture by developing a simple program in Xcode with OpenNI framework and NITE middleware to estimate the player's height by retrieving joint points coordinates information. We have obtained the results of 100% user detection rate, high accuracy but low precision for height estimation with Kinect. This analysis convinced that Kinect is suitable to replace wearable sensor and to fit in our system with proper adjustment and arrangement.

When a real-time OS (RTOS) and a symmetric multiprocessing general purpose OS (SMP GPOS) are executed on the same multicore processor, there is a possibility to cause the lock holder preemption (LHP) problem, which is a well known problem of the sources of the performance degradation on an SMP GPOS. The problem occurs when a kernel mode thread holding a lock in a SMP GPOS is preempted by other OSes if these OSes share the same physical core. The existing solution does not take into account real-time responsiveness so it is not suitable to use in embedded systems. The paper proposes two new techniques to avoid the LHP problem for ensuring both the real-time responsiveness of real-time OS and the high throughput of SMP GPOS. The new proposed techniques have been implemented on our virtualization layer called SPUMONE, and we measured the results showing that the proposed new techniques reduce the LHP problem without degrading the real-time responsiveness.

System administrators have used integrity checkers to prevent the system from malicious infections. Especially, checking the integrity of the kernel is important, since the infections of the kernel affect the entire system. Most of the previous works to prevent such infections rely on the developers or administrators to write specifications to detect them. Those works require high engineering cost and may incur vulnerabilities. The other previous works use virtualization techniques to trace the memory usage of the target system. However, they require hardware supports for the virtualization to avoid significant overhead. Most of embedded systems do not have such hardware supports. In addition, the overhead of the integrity checking affects all of the guest OSes, because they check integrity of the target OS in the virtualization layer. Therefore, they are difficult to be applied to multi-core environment.In this paper, we propose a method to generate the integrity checker automatically. The integrity checker runs on a virtualization layer and checks the integrity of kernel data structures of the target OS kernel from the outside of it. The virtualization layer does not require a special hardware support for the virtualization, because the integrity checker only reads memory area used by the target OS. Moreover, the integrity checker is executed as a guest OS, and thereforeit does not affect the entire system performance when it runs on multicore environment. The integrity checker checks the kernel data structures using invariants of them. In order to generate the invariants automatically, our system analyzes obtained kernel data structures. However, checking all of the kernel data structures is not feasible, since there are a lot of kernel data structures and an analyzer uses relationships with them to generate invariants. Therefore, our challenge is to reduce the target kernel data structures while avoiding false positives and false negatives as much as possible.

A virtualization layer makes it possible to compose multiple functionalities on a multi-core processor with minimum modifications of OS kernels and applications. A multi-core processor is a good candidate to compose various software independently developed for dedicated processors into one multi-core processor to reduce both the hardware and development cost. In this paper, we present SPUMONE, which is a virtualization layer suitable for developing multi-core processor based-information appliances.

Since our daily life is becoming more and more complex, it is difficult to make a decision for a variety of issues about ourselves. Of course, the Internet gives us a possibility to reactively find necessary information through search engines. But, it requires users to understand what they like to know currently. Also, they need significant cognitive efforts to collect a set of necessary information reactively for making their decisions. Ambient social media offer an alternative way that people find information in the real world. The ambient social media will be pervasively integrated in our digital lives, and reflect the various facts in the real world. They do not need to use a browser for retrieving information from the Internet explicitly, and necessary information to people is shown in ambient social media proactively. This kind of new media is a promising approach to return feedback information to people for making a better decision or for avoiding a mindless decision. In this paper, we are developing two case studies to support the user's decision making in our daily life by returning necessary information proactively, and some design implications for developing future ambient social media.

In this paper, we propose a virtualization architecture for the multi-core embedded system to provide more system reliability and security while maintaining the same performance without introducing additional special hardware supports or having to implement complex protection mechanism in the virtualization layer. Virtualization has been widely used in embedded systems, especially in consumer electronics, albeit itself is not a new technique, because there are various needs for both GPOS (General Purpose Operating System) and RTOS (Real Time Operating System). The surge of the multi-core platform in the embedded system also helps the consolidation of the virtualization system for its better performance and lower power consumption. Embedded virtualization design usually uses two kinds of approaches. The first one is to use the traditional VMM, but it is too complicated for use in the embedded environment if there is no additional special hardware support. The other is the use of the micro kernel which imposes a modular design. The guest systems, however, would suffer from considerable amount of modifications because the micro kernel lets the guest systems to run in user space. For some RTOSes and theirs applications originally running in kernel space, it makes this approach more difficult to work because a lot of privileged instructions are used in those codes. To achieve better reliability and keep the virtualization layer design light weighted, a common hardware component adopted in the multi-core embedded processors is used in this work. In the most embedded platforms, vendors provide additional on-chip local memory for each physical core and these local memory areas are private only to their cores. By taking this memory architecture's advantage, we can mitigate above-mentioned problems at once. We choose to re-map the virtualization layer's program called SPUMONE, which it runs all its guest systems in kernel space, on the local memory. By doing so, it can provide additional reliability and security for the entire system because the SPUMONE's design in a multi-core platform has each instance being installed on a separated processor core which is different from the traditional virtualization layer design and the content of each SPUMONE is inaccessible to each others. We also achieve this goal without bringing any overhead to the overall performance.

In this paper we argue that energy conservation is largely a group phenomenon requiring group interventions to achieve change. Persuasive technology can help to provide these interventions. The present study explores the influence of group feedback and individual comparative feedback on energy consumption using an experimental simulation paradigm. To account for cultural differences in group orientation and the power of group feedback, two studies were conducted, one in the Netherlands and one in Japan, in which groups of participants received feedback on everyday tasks. As expected, Dutch participants saved more energy when individual comparison feedback was present, but not the Japanese participants. In contrast, as expected, group feedback caused Japanese participants to save more energy. Providing solely group feedback did not promote energy saving in the Netherlands. Group feedback made the Dutch save more energy only in combination with individual comparison feedback. These results suggest that persuasive technology can employ the power of feedback as a group intervention, but that relevant cultural orientations are crucial.

The augmented reality (AR)-based learning support has several advantages over virtual reality or PC applications. AR enables to maintain the physical interaction that an activity originally offers, thus the skills and knowledge acquired in an augmented learning process can be intuitively applied to practice use. Whereas lots of AR-based self-learning support systems have been developed in previous studies, it has not been sufficiently evaluated how it influences a learner's mindset and the efficiency of training. In this paper, we investigate the user experience brought by AR technologies in a self-learning process. We chose the game of Go as a study program, and developed the Augmented Reality Go (ARGo) system to compare the AR and conventional PC-based learning assistance. We found that the physical interaction with the original game apparatus enhanced the subjects' intrinsic motivation towards self-learning. Moreover, the original look-and-feel induced deeper concentration and higher elaboration on problem solving. Design issues are also discussed to generalize the concept of AR self-learning support towards broader application domains.

While recent embedded systems start to own more and more functionalities, security requirements become more and more important. In this paper we propose an approach to enhance the security of embedded systems. In this approach SPUMONE, a thin virtualization layer, is selected to build a multi-OS environment for its low overhead. Xv6 runs as trusted OS for executing the monitoring service that detects the violation of the integrity of the Linux kernel while Linux as a general purpose OS. The monitoring service checks whether Linux is compromised or not. A secure pager that offers the spatial isolation based on the core-local memory is proposed to protect the integrity of the xv6 kernel located in the main memory, which can make Linux and xv6 run in high security level.

In this paper, we propose a virtualization architecture for the multi-core embedded system to provide more sys-tem reliability and security while maintaining the same performance without introducing additional special hardware supports or having to implement complex protection mechanism in the virtualization layer. Embedded virtualization design usually uses two kinds of approaches, traditional VMM and microkernel approaches, but both of them suffer from performance or engineering cost problems. To achieve better reliability and keep the virtualization layer design light weighted, a common hardware component called local memory adopted in the multi-core embedded processors is used in this work. By taking this memory ar-chitecture's advantage, we can mitigate above-mentioned problems at once. We choose to re-map the virtualizationlayer's program called SPUMONE, which it runs all its guest systems in kernel space, on the local memory. By doing so, it can provide additional reliability and security for the entire system because the SPUMONE's design in a multi-core platform has each instance being installed on a separated processor core, which is different from the traditional virtualization layer design, and therefore the content of each SPUMONE in the local memory is inaccessible to each others.

In this paper we present a on-field study for evaluating a crowd sourcing mobile social search application. With the help of the local crowd via social medias, this application assists foreign visitors in Japan by answering their image-based questions at hand in a timely fashion. We ran a controlled field experiment for 6 weeks with 55 participants. We found that the mobile crowd sourcing model demonstrated a reliable performance on response speed and response quantity: half of the requests were answered within 10 minutes, 75% of requests were answered within 30 minutes, and on average every request had 4.2 answers. Especially in the afternoon, evening and night, nearly 88% requests were answered in average approximately 10 minutes, with more than 4 answers per request. In terms of participation motivation, we found the top active crowd workers were more driven by intrinsic motivations rather than any of the extrinsic incentives (gamification incentives and social incentives) we designed.

In order to enhance OS security, most of people use security patches to fix the vulnerabilities of the OS. However, the security patches may also incur vulnerabilities. These vulnerabilities are generated since most OSes has a lot of functionalities and their functionalities are very complex to manage the entire source code manually. Moreover, in order to use the security patch, rebooting the system is required. Some of systems such as enterprise servers and embedded systems cannot accept the rebooting. Therefore, we propose an external integrity checking system to enhance the OS security. The external integrity checking system and a target OS run on a hyper visor simultaneously, therefore, their operations do not affect each other. In addition, the integrity checking system is generated automatically with invariants. Therefore, the possibility of inserting vulnerabilities into the system is as small as possible, and the system can cover a lot of vulnerabilities.

In this paper, we introduce the AR (augmented reality) drum kit that supports a self-training process with visual guidance and feedback. While musical instruments play requires repetitive practice, the learner often easily loses interest and gives up making effort on learning. Therefore, our system was designed to keep intrinsic motivation by offering playful features, such as a game-like user interface and a variety of tasks with different difficulty levels. Moreover, as a drum kit offers distributed interaction points (i.e., drum pads), visual guidance information needs to consider that the player's attention would be fragmented during a performance. Available cognitive resources are limited for each task in such a multitasking environment, thus the information should be simple and lightweight in order not to require explicit attention. We developed several presentation styles, such as direct projection to drum pads and ambient projection on the wall, to evaluate the usability of the system. We also report preliminary user study results to identify further design issues for the future work.

Persuasive applications that change user behaviors and attitudes have been used for dietary support and healthcare and so on. When we use persuasive application to solve the social problems such as global warming and health issues, it is necessary that we extend the range of target users to people who are not interested in a problematic behavior. In this paper, we suggested an unconscious persuasive technique to design a persuasive application that is able to encourage people unconsciously. We have developed iDetective as a prototype application that persuades users unconsciously using agent persuasion and consciousness sensing. At the same time, we have evaluated the effectiveness of unconscious persuasion and influence on users through a user study using iDetective. As the result, although the unconscious persuasion on iDetective has yielded desirable effect, we could not find whether our application was able to attract people who are no interested in the target behavior. In addition, we have found that participants were not offended by the fact of unconscious persuasion and obtained possibilities and problems of unconscious persuasion.

In this demonstration, we introduce the augmented calligraphy system that aims at supporting a calligraphy learner's self-training process by giving feedback. In order to write characters well, body posture is a very important factor. However, it is difficult to keep proper posture without any assistance. Therefore, the system monitors the learner's posture by a web camera and notifies them if the posture moves into a bad shape. Several types of multimodal feedback were implemented, since we are particularly interested in how feedback design can decrease cognitive load.

Mobile user interfaces are currently not well designed with considering mobility. The usability of mobile services becomes degraded while moving, since a user cannot pay enough attention to the service in a multi-task environment. Thus, it is important for mobile services to convey information to a user with less cognitive workload. In this paper, we address design issues to enhance the mobility of mobile services, and evaluate the workload caused from cognitive processes in a multi-task environment. Taking cross-modal and cross-device interactions into consideration, three factors are focused as variable parameters in the experiment: device type, contents format and information volume. Based on the experimental results, we discuss how these factors affect a cognitive process in a mobile interaction. Future work is also implicated in order to apply the findings to actual mobile user interface design.

The recent increase in complexity and functionality in embedded systems makes them more vulnerable to rootkit-type attacks, raising the need for integrity management systems. However, as of today there is no such system that can guarantee the system's safety while matching the low-resource, real-time and multi-core requirements of embedded systems. In this paper, we present a Virtual Machine Monitor (VMM) based monitoring service for embedded systems that checks the actual kernel data against a safe data specification. However, due to the VMM and multi-core nature of the system, the guest OS can be preempted at any time, leading to the checking of potentially inconsistent states. We evaluated two approaches to solve this problem: detecting such invalid states by checking specific kernel data, and detecting system calls using the VMM.

Travelers in countries that use an unfamiliar script cannot use pocket translators or online translation services to understand menus, maps, signs and other important information, because they are unable to write the text they see. Solutions based on optical character recognition provide very limited performance in real-world situations and for complex scripts such as Chinese and Japanese. In this paper, we propose an alternative image translation solution based on crowdsourcing. A large number of human workers on mobile terminals are used to carry out the tasks of image recognition, translation and quality assurance. Compared to purely technical solutions, this human computation approach is also able to account for context and non-textual cues, and provide higher level information to the end-user. In this paper, we describe a preliminary user study to create a model of end-user requirements.

A hypervisor consolidates multiple numbers and multiple types of OSes into a single machine by multiplexing a real machine into a number of virtual machines (VMs). An example the typical application of a hypervisor is server consolidation which uses the same or similar types of VMs and OSes (symmetric). Another application is accommodating different types of OSes (asymmetric): open platform and closed platform, real-time and time sharing, general-purpose and dedicated purpose, emerging and legacy, etc. For instance, a mobile device consists of real-time OS, which controls radio transmissions, and general-purpose OS, which exposes highlevel API.

Multi-core processors are being increasingly adopted for embedded systems because they improve performance, power consumption and lower development cost. Composing multiple operating systems on a multi-core processor enhances the reusability of software when developing rich functional embedded systems. Multiple OS environments enable the product to use two versions of an operating system at the same time. In order to build multiple OS environments, a virtualization layer specialized for embedded systems is necessary, since most of processors for embedded systems support only two protection levels, and there is no hardware support for virtualization. In traditional approaches, an OS kernel runs at the user level to isolate the respective OS kernels to increase reliability, but this approach requires heavy modifications to the guest OSes if there is no proper hardware virtualization support that is rarely used in embedded systems. Therefore existing solutions are not preferred by the embedded system industry. In this paper, we propose a composition kernel where multiple OS kernels are running on top of a very thin hardware abstraction layer. A composition kernel can reduce the engineering cost of developing an embedded system by reusing existing OS kernels and application with minimum modification without assuming special hardware supports.

This paper proposes SPLiT (Scalable Performance Library Tool) as the methodology to improve performance of applications on multicore processors, through CPU, and cache optimizations. SPLiT analyzes applications based on cycle counts, and cache misses, and predicts behavior of the applications according to the analysis. With these acquired knowledge on the target application, SPLiT improves CPU resource allocation determined by operating systems. SPLiT, and SPLiT library are designed to achieve resource optimizations based on the combination of hardware information collected by Operating System kernel, and software information collected by applications. Additionally, all the features of SPLiT are immediately available with the small modifications to the application source codes, and the modifications are basically to implant SPLiT library calls into the source code. This simple requirement diminishes programmers' difficulties for performance tuning of multicore applications. Through empirical experiments, we validated the efficiency of SPLiT, and observed that performance of the web application was improved by 26%.

While pervasive technologies explore new gaming styles, traditional games, such as cards and tabletop games are still appealing and have various irreplaceable flavors. We point out that tangible game objects and spatial interactions amplify emotional impacts in gaming; and the advantage cannot be reproduced in completely digitalized games. Thus we propose the concept of augmented traditional games, which aims at extending game features without losing original look-and-feel. In this paper, we introduce a case study on augmenting the game of Go. Our prototype supports several game modes, for example, a self-training mode for beginners. Based on an experimental study with the prototype, we discuss human factor issues in game design. We also suggest an augmentation framework for a wider range of traditional games as future work.

Economic incentives are a powerful way of shaping consumer behavior towards more commercially efficient and environmentally sustainable patterns. In this paper, we explore the idea of combining pervasive computing techniques with electronic payment systems to create activity-based micro-incentives. Users who consume additional resources by e.g., occupying an air-conditioned space instead of a normal space are levied additional micro-payments. In an alternative approach, consumers who choose to save resources are rewarded with micro-rebates off the price of a service. As a result, the cost of using a service corresponds more closely with the resources used, leading market mechanisms to allocate resources efficiently. A key challenge is designing incentive mechanisms that alter consumer behavior in the desired fashion. We introduce four incentive models, and present evaluation results suggesting that consumers make different decisions depending on which model is used.

This paper introduces a new dependable operating system project, called DEOS, started in 2006, and scheduled to continue for six years. In this project, a safety extension mechanism called P-Bus is to be designed, and implemented in the Linux kernel so that a future dependability attribute is implemented with P-Bus. A hardware abstraction layer, called SPUMONE, is introduced so that a light-weight operating system, called ArcOS, and a monitoring service on top of ArcOS monitors the Linux kernel to provide a safety-net for the Linux kernel. New dependability metrics are being designed to enable developers and users to decide which hardware or software solution meets their dependability requirements, and thus can be used.

Our society depend on embedded and ubiquitous computing and the reliability of embedded software becomes more and more important. We have conducted a five years project with industries to develop software for realizing highly reliable embedded systems. We have tackled reliability issues from the following aspects. (1)Design environment: we have developed a UML design verification tool that apply model checking techniques to improve the design quality of application software. (2)Operating environment: we have developed operating system supports that realize multiple execution of real-time operating systems and also developed operating systems with enhanced resource management. Both technologies contribute the realization of robust run-time environment. (3) Real-time environment: we have developed real-time garbage collection techniques for Jave. They prevent the suspension of applications that violates the correct behavior of real-time applications. Also, they reduce the effort of application programmers to avoid garbage-collection during important execution timing. We have obtained fruitful results from these three research themes, and some of them are actually used in industries. Furthermore, we have integrated the results to make synergetic effect of them. In order to demonstrate the effectiveness, we have conduct an experiment. In this paper, we introduce the project and its results.

We present an architectural framework that provides the foundation for building smart object systems and uses a document centric approach utilizing a profile based artefact framework and a task based application framework. Our artefact framework represents an instrumented physical smart objects as a collection of service profiles and expresses these services in generic documents. Applications for smart objects are expressed as a collection of functional tasks (independent of the implementation) in a corresponding document. A runtime component provides the foundation for mapping these tasks to the corresponding service provider smart objects. There are three primary advantages of our approach- firstly, it allows developers to write applications in a generic way without prior knowledge of the smart objects that could be used by the applications. Secondly, smart object management (locating/accessing/etc.) issues are completely handled by the infrastructure thus application development becomes rapid and simple. Finally, the programming abstraction used in the framework allows extension of functionalities of smart objects and applications very easily. We describe an implemented prototype of our framework and show examples of its use in a real life scenario to illustrate its feasibility.

In this paper, a novel lightweight anomaly and fault detection infrastructure called anomaly detection by resource monitoring (Ayaka) is presented for information appliances. Ayaka provides a general monitoring method for detecting anomalies using only resource usage information on systems independent of its domain, target application, and programming languages. Ayaka modifies the kernel to detect faults and uses a completely application black-box approach based on machine learning methods. It uses the clustering method to quantize the resource usage vector data and learn the normal patterns with a hidden Markov Model. In the running phase, Ayaka finds anomalies by comparing the application resource usage with the learned model. The evaluation experiment indicates that our prototype system is able to detect anomalies, such as SQL injection and buffer overrun, without significant overheads.

In this paper, online system-level self-healing support is presented for embedded systems. Different from off-line log analysis methods used by conventional intrusion detection systems, our research focuses on analyzing runtime kernel data structures hence perform self-diagnosis and self-healing. Inside the infrastructure, self-diagnosis and self-healing solutions have been implemented based on several selected critical kernel data structures. They can fully represent current system status and are also closely related with system resources. At runtime once any system inconsistency has been detected, predefined recovery functions are invoked. Our prototype is developed based on a lightweight virtual machine monitor, above on which the monitored Linux kernel, runtime detection and recovery services run simultaneously. The proposed infrastructure requires few modifications to current Linux kernel source code, thus it can be easily adopted into existing embedded systems. It is also fully software-based without introducing any specific hardware, therefore it is cost-efficient. The evaluation experiment results indicate that our prototype system can correctly detect inconsistent kernel data structures caused by security attacks with acceptable penalty to system performance.

This paper presents EcoIsland, which is a system persuading individuals and families to change their lifestyle patterns to reduce CO2 emissions. EcoIsland visualizes the user's current eco-friendly behavior as an island shared by his/her family members. Several persuasive techniques developed in behaviorism, social psychology, and economy are used to offer incentives to him/her to encourage eco-friendly behavior. We examine and compare the implementation and effectiveness of different types of persuasive techniques in several user studies.

Ambient lifestyle feedback systems are embedded computer systems designed to motivate changes in a person's lifestyle by reflecting an interpretation of targeted behavior back to the person. Other interactive systems including serious games have been applied for the same purpose in areas such as nutrition, health and energy conservation, but they suffer from drawbacks such as inaccurate self-reporting, burdens placed on the user, and lack of effective feedback. Ambient lifestyle feedback systems overcome these challenges by relying on passive observation, calm presentation style and emotionally engaging feedback content. In this paper, we present an ambient lifestyle feedback system concept and provide insights from the design and implementation of two prototype systems, Virtual Aquarium and Mona Lisa Bookshelf. In particular, we discuss the theory and practice of effective feedback design by drawing on elementary behavioral psychology and small-scale user studies. The work is aimed at aiding in the design of ambient persuasive technologies and ambient interaction in general.

Recently, the engineering cost of embedded systems is rapidly increasing due to growing sophistication of services. To deal with the problem, hybrid operating system environments have been proposed. This enables to run a RTOS and a general purpose OS concurrently and to reuse software resources on both of them. This approach is efficient in reducing engineering costs. We reconfigured the requirement for these hybrid operating system environment and build a new architecture which fulfills these requirements by using virtualization techniques. Our system provides the facilities to build multiple operating system environment easily. There are two contributions in our systems. One is that the modification cost of the guest OS is small. The second contribution is improvement in system availability by enabling guest OS to reboot independently. Although we used virtualization layer to construct a hybrid operating system environment, the performance overhead is considering small. Therefor our approach is very practical and efficient for recent sophisticated embedded systems.

The kernel objects consist of critical kernel data structures and system call functions, which are the most important data for a system, should be protected as first-class candidates. In this paper, a lightweight system-level detection and recovery infrastructure is presented for embedded systems. Inside the infrastructure, specific runtime protections have been implemented for different kernel objects, kernel data structures are protected by the periodic detection and recovery, the interception of arguments is used to protect vulnerable system calls. At runtime once any system inconsistency has been detected, predefined recovery actions will be invoked. The consistency detection regulations and corresponding recovery actions can also be flexibly customized by system developers. The infrastructure requires few modifications to kernel source code, thus it is easy to integrate into existing embedded systems. The evaluation experiment results indicate our prototype system can correctly detect the inconsistent kernel data structures caused by security attacks and also prevent kernel from exploits due to vulnerable system calls with acceptable penalty to system performance. Moreover, it is fully software-based without introducing any specific hardware and requires no modifications to system call APIs, therefore legacy commercial-off-the-shelf (COTS) applications can be also easily reused.

The emergence of functional embedded systems such as cell-phones and digital appliances brought up a new issue, building a system supporting both real-time and rich services. One of the solutions is leveraging a hypervisor to integrate an RTOS and a commodity OS into a single device. However, this approach induces the limitation of application deployment; all the high priority tasks should reside in the RTOS. In this paper, we propose a task grain scheduling for a real-time hypervisor, which enables a flexible application deployment between an RTOS and a commodity OS. We constructed a prototype system with an existing hypervisor, an RTOS, and a commodity OS. We measured some basic overheads, and fixed some tasks which were missing their deadlines using the task grain scheduling to meet their deadlines. The overheads were small and the task grain scheduling achieved a flexible real-time scheduling for the hypervisor based system.

This paper explores system issues for involving end users in constructing and enhancing a smart home. In support of this involvement we present an infrastructure and a tangible deployment tool. Active participation of users is essential in a domestic environment as it offers simplicity, greater usercentric control, lower deployment costs and better support for personalization. Our proposed infrastructure provides the foundation for end user deployment utilizing a loosely coupled framework to represent an artefact and its augmented functionalities. Pervasive applications are built independently and are expressed as a collection of functional tasks. A runtime component, FedNet maps these tasks to corresponding service provider artefacts. The tangible deployment tool uses FedNet and allows end users to deploy and control artefacts and applications only by manipulating RFID cards. Primary advantages of our approach are two-fold. Firstly, it allows end users to deploy ubicomp systems easily in a Do-it-Yourself fashion. Secondly, it allows developers to write applications and to build augmented artefacts in a generic way regardless of the constraints of the target environment. We describe an implemented prototype and illustrate its feasibility in a real life deployment session by the end users. Our study shows that the end users might be involved in deploying future ubicomp systems if appropriate tools and supporting infrastructure are provided.

We present a deployment tool atop a document centric infrastraucture to support end users in constructing and extending smart object systems. This tool allows an ordinary individuals to deploy, extend and control smart object systems in a Do-It-Yourself (DIY) fashion. We have implemented two version

This paper explores system issues for enabling incremental deployment of pervasive application - the problem of how to deploy and gradually enhance the functionalities of applications in a pervasive environment. We present a system architecture, FedNet that provides the foundation for incremental d

Current mobile interaction is not well designed with considering mobility. Even though user attention is an important human factor for user interface design, current mobile service is too attention-consuming for moving users to perform tasks. This is because mobile service is still pursuing desktop-

Current mobile interaction is not well designed with considering mobility. Usability of a mobile service is degraded while on the move, since users can not pay enough attention to the service in such a dynamic and complicated mobile context. In this paper, we propose the mobile service design framew

In this paper, a lightweight system level monitoring infrastructure known as kernel objects monitoring infrastructure (KOMI) is presented for commercial-off-the-shelf (COTS) embedded systems. The kernel objects consist of certain critical kernel data structures and entry points of system calls, which are protected as first-class objects inside the system. KOMI provides specific runtime protections to different kernel objects: kernel data structures are protected by the periodic detection and recovery, the interception of arguments is used to protect vulnerable system calls. Both protection methods can provide not only consistency regulations but also recovery actions for the system. During its runtime deployment, once any system inconsistency has been detected, predefined recovery actions will be invoked. Since KOMI requires few modifications to kernel source code, it is easy to integrate into existing embedded systems. The evaluation experiment results indicate our prototype system can correctly detect the inconsistent kernel data structures caused by security attacks and also prevent kernel from exploits due to vulnerable system calls with acceptable penalty to the system performance. Moreover, KOMI is fully software-based without introducing any specific hardware and requires no modifications to system call APIs, therefore legacy applications can be also easily reused.

In recent studies, user interaction models for connecting information appliances to each other using real-world tangible interaction are limited to specified scenarios or applications. We developed a novel model for connecting appliances in every scenario based on the extension of the drag-and-drop interaction metaphor, which has proved acceptable to users. By applying this model with a camera-phone, we propose an interaction technique that is able to realize the direction of the data flow between connected appliances. One of the most important differences from existing research is that our technique enables users not only to use data files contained by one appliance on another appliance, but also to associate and configure networked appliances to each other, in a tangible manner.

User centric personalization plays an important role for the adoption of proactive applications. However, stipulating system support to facilitate personalization features in proactive applications generically is still an open issue. In this paper we have addressed this particular issue and presented Persona, a tool that enables adding personalization features in proactive applications in a generic manner. A key feature of Persona is portability that allows it to be injected in various pervasive middlewares as a plug-in. Consequently, existing proactive applications can easily be extended with Persona for personalization support. We have discussed the design and implementation rationale behind Persona and shown it's direct implications with two different middlewares and several proactive applications.

In this paper, we propose a novel computer gaming style Lifestyle Ubiquitous Gaming, where our experience is enriched in our daily lives by using gaming concepts. In lifestyle ubiquitous gaming, human daily activities are implicitly tracked and interpreted. Then, the feedback of the activities are returned to a user as a game system' presentation. The user can enjoy the game without too much conscious. This characteristic has a very big difference from traditional gaming. We describe the motivation and conceptual framework of lifestyle ubiquitous gaming, and show two case studies that we have conducted to demonstrate our proposal's feasibility and effectiveness.

In ubiquitous computing environments, our daily lives will be made convenient by embedded intelligent devices. Those devices, such as car navigation systems, personal digital assistances, and cellular phones, provide various kinds of the complex services. Those devices are networked with each other and provide complicated services, through the Internet. While they provide useful services, there is an increasing possibility of security attacks, which include the unexpected execution of unsecure codes. Current information appliances have not yet fully embodied a resource protection mechanism that prevents misbehaved applications from consuming the whole CPU capacity of system resources. In this paper, we propose accounting system, and describe its design and implementation. The system is a resource monitoring and restriction system that has the purpose of improving the system's reliability and security. We developed the system on Linux. Our system is a very generic to offer various services, such as security improvement, overload control, and class-based accounting, that require CPU resource control

In ubiquitous computing environments, contextawareness is one of the most important research topics. Computers embedded in our surrounding can extract information about a user, and the information makes it possible to offer personalized services according to the user’s preference. To extract a large amount of context information, wearable sensor devices will become more important in the near future. However, it is not easy to develop context-aware services that use context information from wearable sensor devices because of the gap between low-level sensor information and high-level context that the services require. In this paper, we propose a software infrastructure for wearable sensor networks. We first discuss the requirements to retrieve context information from wearable sensor networks. Then, we introduce our software infrastructure named Cinnamon that extracts high-level context from low-level context information retrieved from wearable sensor networks. The software infrastructure makes it dramatically easy to develop context-aware services for wearable sensor networks. We present the design and implementation of Cinnamon, and discuss our current prototype implementation.

In this paper, we propose a personal home sewer that makes it possible to coordinate a variety of home appliances. Since each person has a different personal home server, it allows him/her to personalize how to use the appliances according to his preferences. Also, a personal home server can discover and configure appliances at any places, such as railroad stations, cars, and streets as well as houses in a seamless way. We show the design and implementation of a personal home server, and present its current status.

Embedded operating system (OS) kernels provide basic execution primitives that can be used commonly by many appliances. The recent emergence of digital appliances requires more advanced features, such as networking and GUI. Those features dramatically complicate the appliances' software systems and increase their code sizes. In order to develop such large and complex software systems, we propose a system architecture that co-locates multiple embedded operating systems on a microkernel. The proposed architecture employs a microkernel to provide protected execution environments for the existing embedded kernels. In each protected execution environment, a kernel and its applications run just as they run directly on hardware since they share the same protection domain. Our approach can achieve the maximum reusability of the existing software resources including embedded OS kernels and their applications; thus, it protects the existing software resources. We have developed the system that consists of TL4 microkernel and a /spl mu/ITRON embedded kernel, and shows the feasibility of our approach through evaluations.

Using a single personal device as an universal controller for diverse services is a promising approach to solving the problem of too many controllers in ubiquitous multimodal environments. However, the current approaches to universal controllers cannot provide intuitive control because they are restricted to traditional mobile user interfaces such as small keys or small touch panels. We propose Vidgets, which is short for virtual tangible widgets}, as an approach to selecting and controlling ubiquitous services with virtually implemented tangible user interfaces based on a single sensing personal device equipped with a digital camera and several physical sensors. We classify the use of the universal controller into the three stages: (a) searching for a service, (b) grasping the service and (c) using the service. User studies with our prototype implementation indicate that the smooth transition and integration of the three stages improve the overall interaction with our universal controller.

EarlGray is a component-based Java virtual machine (JVM) that can be configured to satisfy various kinds of requirements for building future information appliances and embedded systems. While the modification and extension on an existing JVM tend to be done in an ad-hoc manner, EarlGray allows developers to customize the JVM in a systematic manner by decomposing it into components and the explicit descriptions of the relationship between the components. We also examine three case studies on the customization of the EarlGray: exchanging a scheduler, memory management, and class file verifier components. These case studies show the benefits and drawbacks of the current component-based technologies.

This paper presents a framework prototype for sentient environments. The framework provides a generic interface to the applications for interacting with sentient artefacts in a unified way regardless of their type and properties. As a result, application development is fairly simple, rapid and independent from the context-aware environments.

This paper describes a context information acquisition framework for a personal device that equips a variety of sensors. The framework captures context information about a user and his/her surrounding environment; and the information is used to adapt the behavior of applications running on the personal device. Our framework adopts the blackboard architecture to execute multiple analysis modules that analyze signals from respective sensors. Respective modules implement different algorithms to complement each other's results to retrieve more accurate and higher abstract context information.

Embedded computers pervade our lives. There are increasing opportunities for them to adopt embedded operating systems as they are required to have high functionality and complexity. Unfortunately, the reliability of such embedded systems is insufficient to support our lives in 24/7. Therefore, we propose the fine-grain proactive recovery (FPR) of operating system components. FPR enables an operating system to periodically restart its modules and thus to keep their states clean.

The number of various kinds of everyday objects that contain embedded computers is increasing due to the popularity of ubiquitous computing. While component-based software development becomes common in a variety of application domains, ubiquitous computing requires component frameworks that offer more advanced features than the current component frameworks. This paper explores a component framework in the context of a ubiquitous computing system. We design and implement a system coordinating various home appliances, called SENCHA. SENCHA is built on the OSGi component framework. Although the framework provides facilities for dynamically changing environments, this paper reveals several problems of the framework in our experience

Mixed reality is one of the most important techniques to achieve the vision of ubiquitous computing. Traditional middleware for mixed reality provide high level abstraction to hide complex algorithms for analyzing video images, but applications programmers still need to take into account distribution and automatic reconfiguration when developing mixed reality applications for ubiquitous computing. Our middleware infrastructure hides all the complexities to build mixed reality applications for ubiquitous computing. Therefore, the development does not require advanced skills for ubiquitous computing. The paper describes the design and implementation of our infrastructure, and presents some scenarios and the current status showing its effectiveness.

Our daily lives will be dramatically changed by embedded small computers in our environments. The environments are called ubiquitous computing environments. To realize the environments, it is important to reduce the cost to develop ubiquitous computing applications by encapsulating complex issues in middleware infrastructures that are shared by various applications.In this paper, we describe three middleware infrastructures for supporting ubiquitous computing, that have developed in our projects. Our infrastructures have tried to hide some complexities to make it easy to develop ubiquitous computing applications in an easy way. We also show some lessons learned in our projects.

We propose a personal home server that makes it possible to coordinate various home appliances. Since each person has a different personal home server, it allows us to personalize how to use the appliances according to his preference. Also, our personal home server can discover and configure various appliances at any places, such as railroad stations, cars, and streets as well as houses in a seamless way. We show the design and implementation of a personal home server, and discuss the current design and future work.

In this short article, we introduce Japan Embedded Linux Consortium. Then, we show the impact of ubiquitous computing researches to embedded system industries, and show a brief overview of our research activities.

The paper describes design and implementation of software infrastructure for building augmented reality applications for ubiquitous computing environments. Augmented reality is one of the most important techniques to achieve the vision of ubiquitous computing. Traditional toolkits for augmented reality provide the high level abstraction that makes it easy to build augmented reality applications. However, the applications programmers need to contemplate distribution and context-awareness that make the development of applications very hard, but they are necessary to build ubiquitous computing environments. Our infrastructure provides the high level abstraction and hides distribution and context-awareness from programmers. Therefore, the cost to develop augmented reality applications will be reduced dramatically by using our middleware infrastructure.

We propose universal interaction for networked home appliances, which is a simple mechanism to fill the gap between traditional user interface systems and advanced user interaction devices. Our system enables us to control appliances in a uniform way at any place, and the system allows us to select suitable input and output devices according to our preferences and situations. Also, the devices can be changed dynamically according to a user's preferences.

In this paper, we describe our currently ongoing work to build distributed middleware for networked audio and visual home appliances, which is executed on commodity software. The current prototype has adopted HAVi(Home Audio/Video Interoperability) as distributed middleware for controlling home appliances, which makes it possible to integrate a variety of home appliances and services. In our system, we have implemented HAVi in Java. Programs processing continuous media that emulate digital consumer devices are directly implemented on the Linux operating systems. Since Java and Linux are running on a variety of embedded platforms, our software can be ported to a target embedded system without the modification. Therefore, new home applications can be developed on a standard PC platform, and this makes the development cost of future advanced home appliances dramatically cheap.

We describe our ongoing work to build object-oriented distributed middleware for networked home appliances, which are executed on commodity operating systems. The current prototype has adopted HAVi (Home Audio/Video Interoperability) as distributed middleware for home appliances. We have implemented HAVi in Java, and have adopted Linux as an operating system. Since Java and Linux are running on a variety of platforms, our software can be ported to a target system without modification. Therefore, it becomes easy to develop a new application on a standard PC platform. The paper describes the design issues and the implementation of our system. We have actually implemented several programs that emulate home appliances such as a digital television. We also present some experiences with the current prototype.

In this paper, we propose a middleware system for building adaptive migratory continuous media applications. In future computing environments, a variety of objects at home and in offices will embed computers, and various applications will be moved among these computers according to the user's locations. For example, a computer that executes an application may be changed to another computer according to the location of the application's user for monitoring his behavior. However, since the computers may have dramatically different hardware and software configurations, the application cannot be moved without taking into account the configurations of the computers that the application is migrated. Therefore, migratory applications should be aware of environments where they are executed. The paper focuses on middleware for building adaptive migratory continuous media applications that are one of the most important classes of migratory applications. Our middleware consists of three components. The first component is an environment server that monitors computing environments that applications run, and it notifies the changes of the environments to the applications. The second component is a continuous media toolkit that enables us to build adaptive migratory continuous media applications easily. The toolkit enables us to build continuous media applications in a highly configurable way. This makes applications to be adapted to every computing environment by changing their configurations. The third component is a migration manager that makes applications migratory. We also show an example of a migratory application, and describe some experiences with building the application.

This paper proposes an extension to the standard CORBA interfaces to support explicit binding. The extended interface enables a program to select a suitable transport protocol for each method invocation, and it provides a flexible and versatile framework to control various transport protocols. The interface is compatible with the object model of CORBA, and it is carefully designed to take into account the portability of an application to other CORBA implementations. The extended interface has implemented in omniORB2, which is a CORBA 2.0 compliant system that has been developing at AT&T Laboratories, Cambridge.

We report supporting dynamic transport selection in omniORB2, which is a CORBA 2.0 compliant CORBA system that has been developed at AT&T Laboratories, Cambridge. We describe our design and implementation of supporting dynamic transport protocol selection in our system, and some initial experiments with the system. In our approach, IIOP can be selected for ensuring interoperability between applications. Thus, applications adopting our system can communicate with those adopting other CORBA systems, which run on the Internet environment. On the other hand, an application can select more suitable transport protocols for improving its performance or ensuring its real-time constraints if a server also supports the same transport protocol according to the characteristic of the application.

In this paper, we propose a software architecture for building environment-aware software that runs in various real-time computing environments by adapting programs to their respective environments. The framework is especially suitable for embedded computing environments, since the environments are required to support various types of computers.

In this paper, we propose software architecture using a notion of service proxies, that is a new framework for constructing applications in mobile computing environment. Our framework especially takes into account applications accessing services on Internet such as WWW browsers and MBONE conference tools. In our framework, an application is partitioned into two pieces, one piece runs on a mobile computer, and another piece runs on a stationary computer, where they are connected by wireless networks that may be replaced on the fly. The piece on a stationary computer is called a service proxy. The service proxy filters or caches data from servers before transmitting the data to the piece on a mobile computer. These two pieces are constructed by composing small objects whose composition can be dynamically reconfigured by adding or removing replaceable devices. The paper also presents a toolkit using our framework for building mobile applications easily.

A QOS control is one of the most important factors in continuous media applications. The QOS levels of the applications should be maintained even if several continuous media applications are running concurrently. Also, the QOS levels should be changed according to a number of applications and their respective importances when a system is overloaded. This paper presents our experience with a video-on-demand system that supports a dynamic QOS control scheme on Real-Time Mach, and shows the necessity of a real-time resource management for building such applications.

Synchronization and communication are two common sources of priority inversion which may make the behavior of systems unpredictable and unanalyzable. In microkernel-based systems, they are heavily used for building operating system servers and decomposing applications into several tasks. The management of priorities in IPC and synchronization should be integrated using a uniform mechanism since priority inversion occurs if such integration is not supported. Also, a highly preemptable server structure should be provided because the execution in a server may take up too much time. We propose an integrated real-time resource management model, and a real-time server model which solve the above problems. We implemented and evaluated the models in Real-Time Mach using a uniform mechanism. Our approach enables us to build operating system servers and to decompose applications into several tasks without incurring priority inversion.>