We report a mixed-methods field experiment of a conversational service robot deployed under everyday staffing discretion in a live bedding store. Over 12 days we alternated three conditions--Baseline (no robot), Robot-only, and Robot+Fixture--and video-annotated the service funnel from passersby to purchase. An explanatory sequential design then used six post-experiment staff interviews to interpret the quantitative patterns.

User willingness is a crucial element in the sales talk process that affects the achievement of the salesperson’s or sales system’s objectives. Despite the importance of user willingness, to the best of our knowledge, no previous study has addressed the development of automated sales talk dialogue systems that explicitly consider user willingness. A major barrier is the lack of sales talk datasets with reliable user willingness data. Thus, in this study, we developed a user willingness–aware sales talk collection by leveraging the ecological validity concept, which is discussed in the field of human–computer interaction. Our approach focused on three types of user willingness essential in real sales interactions. We created a dialogue environment that closely resembles real-world scenarios to elicit natural user willingness, with participants evaluating their willingness at the utterance level from multiple perspectives. We analyzed the collected data to gain insights into practical user willingness–aware sales talk strategies. In addition, as a practical application of the constructed dataset, we developed and evaluated a sales dialogue system aimed at enhancing the user’s intent to purchase.

Teleoperated customer service robots have attracted attention to improve customer service efficiency. However, operators experience boredom during long-duration operation due to monotony and idle time, leading to decreased task motivation. This study proposes and evaluates a method to reduce operator boredom through dialogue with the robot to be operated. Field experiments demonstrated that operator-robot dialogue significantly reduced boredom and contributed to maintaining task engagement during long-duration operation.

In this paper, we aim to understand how user motivation shapes human-robot interaction (HRI) in the wild. To explore this, we conducted a field study by deploying a fully autonomous conversational robot in a shopping mall over two days. Through sequential video analysis, we identified five patterns of interaction fluency (Smooth, Awkward, Active, Messy, and Quiet), four types of user motivation for interacting with the robot (Function, Experiment, Curiosity, and Education), and user positioning towards the robot. We further analyzed how these motivations and positioning influence interaction fluency. Our findings suggest that incorporating users' motivation types into the design of robot behavior can enhance interaction fluency, engagement, and user satisfaction in real-world HRI scenarios.

Turn-taking is a crucial aspect of human-robot interaction, directly influencing conversational fluidity and user engagement. While previous research has explored turn-taking models in controlled environments, their robustness in real-world settings remains underexplored. In this study, we propose a noise-robust voice activity projection (VAP) model, based on a Transformer architecture, to enhance real-time turn-taking in dialogue robots. To evaluate the effectiveness of the proposed system, we conducted a field experiment in a shopping mall, comparing the VAP system with a conventional cloud-based speech recognition system. Our analysis covered both subjective user evaluations and objective behavioral analysis. The results showed that the proposed system significantly reduced response latency, leading to a more natural conversation where both the robot and users responded faster. The subjective evaluations suggested that faster responses contribute to a better interaction experience.

In recent years, the demand for social robots has grown, requiring them to adapt their behaviors based on users’ states. Accurately assessing user experience (UX) in human-robot interaction (HRI) is crucial for achieving this adaptability. UX is a multi-faceted measure encompassing aspects such as sentiment and engagement, yet existing methods often focus on these individually. This study proposes a UX estimation method for HRI by leveraging multimodal social signals. We construct a UX dataset and develop a Transformer-based model that utilizes facial expressions and voice for estimation. Unlike conventional models that rely on momentary observations, our approach captures both short- and long-term interaction patterns using a multi-instance learning framework. This enables the model to capture temporal dynamics in UX, providing a more holistic representation. Experimental results demonstrate that our method outperforms third-party human evaluators in UX estimation.

In recent years, numerous studies have been conducted on dialogue robots powered by large language models,enabling sophisticated interactions such as providing guidance and engaging in small talk. However, the interaction performance remains imperfect, and the robots sometimes cause problems during interactions. In this study, we aim to automatically detect such anomalies in human-robot interactions by creating a dataset and developing anomaly detection models. To this end, we created a dataset by manually annotating videos of in-the-wild interactions collected from our field experiment designed to test a framework of parallel conversations in which a human intervenes when a problem occurs in the interaction. Using this dataset, we trained classification models to construct anomaly detection models. We then conducted another field experiment in which the model’s detection results were presented as alerts to operators within the parallel conversation framework. The results confirmed that providing alerts on the basis of the anomaly detection model was useful for facilitating operator intervention.

Generating and evaluating character-like utterances automatically is essential for applications ranging from character simulation to creative-writing support. Existing approaches primarily focus on basic aspects of character‐likeness, such as script-fidelity knowledge and conversational ability. However, achieving a higher level of character‐likeness in utterance generation and evaluation requires consideration of the character’s identity, which deeply reflects the character’s inner self. To bridge this gap, we identified a set of identity-centric character-likeness elements. First, we listed 27 elements covering various aspects of identity, drawing on psychology and identity theory. Then, to clarify the features of each element, we collected utterances annotated with these elements from a commercial smartphone game and analyzed them based on user evaluations regarding character-likeness and charm. Our analysis reveals part of element-wise effects on character‐likeness and charm. These findings enable developers to design practical and interpretable element-feature-aware generation methods and evaluation metrics for character-like utterances.

The rapid development of artificial intelligence (AI) has fundamentally transformed creative work practices in the design industry. Existing studies have identified both opportunities and challenges for creative practitioners in their collaboration with generative AI and explored ways to facilitate effective human-AI co-creation. However, there is still a limited understanding of designers' collaboration with AI that supports creative processes distinct from generative AI. To address these gaps, this study focuses on understanding designers' collaboration with decision-making AI, which supports the convergence process in the creative workflow, as opposed to the divergent process supported by generative AI. Specifically, we conducted a case study at an online advertising design company to explore how professional graphic designers at the company perceive the impact of decision-making AI on their creative work practices. The case company incorporated an AI system that predicts the effectiveness of advertising design into the design workflow as a decision-making support tool. Findings from interviews with 12 designers identified how designers trust and rely on AI, its perceived benefits and challenges, and their strategies for navigating the challenges. Based on the findings, we discuss design recommendations for integrating decision-making AI into the creative design workflow.

Teleoperating social robots requires operators to “speak as the robot,” as local users would favor robots whose appearance and voice match. This study focuses on real-time altered auditory feedback (AAF), a method to transform the acoustic traits of one’s speech and provide feedback to the speaker, to transform the operator’s self-representation toward “becoming the robot.” To explore whether AAF with voice transformation (VT) matched to the robot’s appearance can influence the operator’s self-representation and ease the task, we experimented with three conditions: no VT (No-VT), only VT (VT-only), and VT with AAF (VT-AAF), where participants teleoperated a robot to verbally serve real passersby at a bakery. The questionnaire results demonstrate that VT-AAF changed the participants’ self-representation to match the robot’s character and improved participants’ subjective teleoperating experience, while task performance and implicit measures of self-representation were not significantly affected. Notably, 87% of the participants preferred VT-AAF the most.

Various customer service robots’ teleoperation interfaces (I/Fs) have been developed for human-robot collaboration. However, a previous study has indicated a lack of operator enjoyment. This study aims to design an I/F that elicits operator enjoyment and identifies the factors that contribute to this enjoyment. We developed two I/Fs: a high degree of freedom I/F and a gradual flexibility degree of freedom I/F based on gamification to elicit operator enjoyment. Using these I/Fs, we conducted a field study in a real shopping mall to investigate the operators’ experiences. As a result, the operators in this experiment greatly enjoyed using our I/Fs. The results showed that our I/Fs could elicit operator enjoyment with the following three I/F factors suggested as potentially important design guidelines: ease of use through anonymity, moderate restriction of freedom, and sharing experiences with a group of operators.

Effective teleoperated avatar communication requires expressing social behaviors. Gaze behavior is one of the crucial social behaviors and includes reflexive reactions to the avatar’s surroundings and intentional responses to the operator’s speech and actions. Teleoperated avatars must have their gaze behavior controlled according to situational changes in both the avatar’s and operator’s contexts. However, it is not clear how to adjust the avatar’s gaze in response to changes in both situations. In this paper, we collect a dataset of gazing positions that the avatar is instructed to face, taking into account both avatar and operator situations, and annotation labels that represent both situations in detail. We then exploratorily analyze the ratio of gazing positions per situation through dynamic area-of-interest (AOI) analysis. Our analysis provides insights into determining the gaze behavior of teleoperated avatars.

In recent years, a framework of parallel conversations has been proposed to facilitate efficient conversations through cooperation between humans and dialogue systems. This approach aims to enable simultaneous conversations with multiple users by enabling the system to handle basic conversation and human operators to intervene when problems arise in the system’s conversation. Previous studies on parallel conversations have primarily focused on delegating simple exchanges such as greetings and acknowledgments to the system, with humans taking over for more complex interactions like providing guidance. Recent advancements in large language models may change this situation, enabling dialogue systems to engage in more advanced interactions. In this study, to examine which interventions will be made when large language models are utilized, we placed six dialogue robots based on large language models in an actual facility and conducted a field experiment involving parallel conversations for about a month. Our analysis of the collected data on dialogues and interventions showed that the most frequent interventions were made for supporting interactions when the system failed to react to the user utterances, indicating the limitations of using large language models alone and clarifying our next steps for facilitating smoother parallel conversations.

Dialogue systems are widely utilized in chatbots and call centers. However, it is often difficult for such systems to deliver fully autonomous dialogue. For users to have a better dialogue experience, a framework for human-system collaborative dialogue is proposed in which a human operator takes over the dialogue when needed, engaging in conversation with the user instead of the system (we call this process intervention). Operators join the dialogue in the middle; therefore, it is believed that dialogue summarization can be helpful for interventions. However, it is currently unclear whether dialogue summarization is actually useful. Therefore, in this study, we aim to investigate the usefulness of dialogue summaries for interventions through a field experiment conducted at an actual facility combining an aquarium and a zoo. The results of the field experiment revealed that dialogue summaries were more useful for intervention than dialogue history. Furthermore, we found no differences in the word categories included in the operator utterances during interventions irrespective of whether the dialogue history or dialogue format summary was presented to the operators, suggesting that dialogue format summary has content similar to that of dialogue history but improves the usefulness in intervention.

Service robots have increasingly been investigated in retailing. Previous studies mainly focused on the effectiveness of recommendation with regard to a single robot, and whether and how the use of two robots combined can achieve better performance remain unclear. In this study, we address this by exploring the combination power of two service robots for product recommendation in a bakery. We placed one robot inside the store for product recommendation and the other robot outside to promote the inside robot. Particularly, we are interested in the effects of the outside robot on the inside robot's performance in product recommendation. Our results indicate that using the outside robot to promote the inside robot achieved more purchases over using the inside robot alone. Particularly, we discovered that the outside robot increased the attention of customers toward the inside robot; hence, more customers checked and purchased the products. Based on the findings, we discuss the important points for the effective use of service robots.

Service robots are beginning to be used as a new kind of support for human labor. However, in many cases, we implement only specific task-dependent behaviors in robots according to the purpose of robot introduction, and rarely implement task-independent behaviors. In general, it is known that noninstrumental functions are one factor that improves user experience. Therefore, task-independent behavior of robots as an aspect of noninstrumental functions also has the potential to improve the impression made by robots and deliver a user experience beyond the users’ expectations, during human-robot interaction. This study aims to investigate the influence of task-dependent behavior and task-independent behavior on the impression made by robots, and user experience. We extracted, from previous studies, dialogue task-dependent and dialogue task-independent behaviors during human-robot interaction, and investigated the influence of these behaviors through a video-based survey. The result of the video-based survey shows that dialogue task-dependent behavior improves the functionality of robots and decreases factors of negative user experience, such as frustration, while also fulfilling users’ expectation for interaction with robots. It also shows that dialogue task-independent behavior builds a stronger relationship between users and robots and provides a user experience that exceeds users’ expectation regarding interaction with robots.

In recent years, communication robots have been employed to assist workers. However, it is known that users experience embarrassment when interacting with a robot in a public space, which may hinder their use. Previous studies investigated methods to reduce embarrassment when using robots and the factors that cause embarrassment. Although these studies have investigated the embarrassment experienced by users through only a single interaction, they have not investigated the embarrassment influenced by users’ past experiences. Therefore, in this study, we investigated changes in embarrassment and the factors causing embarrassment through repeated interactions with robots in public spaces. We conducted experiments in which the same participants used a robot in a public space multiple times and continuously experienced embarrassment through repeated interactions with the robot. The results show that repeated interactions with the robot reduce embarrassment, and that embarrassment is influenced by two factors: understanding of the user’s behavior from surrounding people and the user’s previous experience with the interaction.

Emotional voice conversion (VC) aims to convert a neutral voice to an emotional one while retaining the linguistic information and speaker identity. We note that the decoupling of emotional features from other speech information (such as content, speaker identity, etc.) is the key to achieving promising performance. Some recent attempts of speech representation decoupling on the neutral speech cannot work well on the emotional speech, due to the more complex entanglement of acoustic properties in the latter. To address this problem, here we propose a novel Source-Filter-based Emotional VC model (SFEVC) to achieve proper filtering of speaker-independent emotion cues from both the timbre and pitch features. Our SFEVC model consists of multi-channel encoders, emotion separate encoders, pre-trained speaker-dependent encoders, and the corresponding decoder. Note that all encoder modules adopt a designed information bottleneck auto-encoder. Additionally, to further improve the conversion quality for various emotions, a novel training strategy based on the 2D Valence-Arousal (VA) space is proposed. Experimental results show that the proposed SFEVC along with a VA training strategy outperforms all baselines and achieves the state-of-the-art performance in speaker-independent emotional VC with nonparallel data.

Research on self-recommending product agents is under way. Compared with product promotion by humanoid robot agents, self-recommending agents can call a passing cus-omer's attention to product. As the customer's interest is focused on the product, the self-recommending agent can then give instructions, such as “pick me up.” It has been found that customers tend to follow such instructions, and touching the product is known to effectively support sales promotion. From two experiments in a convenience store, a self-recommending agent in this study was successful in attracting customer interest, which included handling the products. However, we also found that, after being picked up, the products' monologue resulted in customers leaving the product behind on the shelf. Herein, we examine the reasons why.

Robot-mediated interaction may be one of the best approaches to overcome the implementation-related limitations of face-to-face interaction between children and older adults. However, there has been little research about development or demonstration of teleoperated social robot systems that could be implemented in nurseries. We report a preliminary experiment wherein older adults greet nursery pupils on their way to and from school using a teleoperated social robot. The results suggest that using teleoperated robots in dialogue-based tasks motivate the children to interact with the robot, and it increases the probability of learning from the tasks. Further, the older adult teleoperators enjoyed the task and wanted to continue the same. In addition, the teleoperators could inherit the close relationships built up by the previous teleoperator even if the teleoperator was replaced. This study provides a starting point for further research on technology-mediated interactions between children and older adults.

In this study, we explore the effective form of social presence for a tele-operated robot to provide customer service. Particularly, we address the question on if and how a tele-operated robot displays the presence of an operator, and what effects it would have on people's perception and behavior toward it. We launched a tele-operated robot in a supermarket and had it deliver recipe flyers. We adjusted the robot's social presence by showing or not showing the photo of an operator's face (F) and using or not using voice conversion (V), leading to three forms of presence: Wizard of Oz (F: no, V: yes), costume (F: yes, V: yes), and telepresence (F: yes, V: no), which indicated the operator's presence from a low to a high level. We determined that the customers behaved significantly different when they faced the tele-operated robot in different forms. Our robot that exhibited a moderate presence of the operator (costume form) achieved the overall best performance. Based on these findings, we discuss both the strengths and weaknesses of the three forms of presence for a tele-operated robot and recommend the appropriate form for various applications.

Customer service with teleoperated robots is suscep-tible to the same problems related to stress as is emotional labor in general, such as for in-person customer service representatives. In this study, we aimed to reduce that stress by constructing a buddy-like rapport between the robot, which is the target of teleoperation, and its operator. For this purpose, we designed an empathetic interaction between the robot and the operator and conducted a customer service experiment to verify its effectiveness. Our results demonstrate that the proposed interaction can build rapport between the robot and the operator and the operator can feel more reassured. Although the effect on stress could not be isolated directly from the data, detailed analyses of the response to the questionnaires indicated that the proposed interaction may be useful to relieve stress.

For a humanoid robot to make eye contact and initiate communication with a person, it is necessary to estimate the person's head position. However, eye contact becomes difficult due to the mechanical delay of the robot when the person is moving. Owing to these issues, it is important to conduct a head-position prediction to mitigate the effect of the delay in the robot motion. Based on the fact that humans turn their heads before changing direction while walking, we hypothesized that the accuracy of three-dimensional (3D) head-position prediction from a first-person view can be improved by considering the head pose. We compared our method with a conventional Kalman filter-based approach, and found our method to be more accurate. The experiment results show that considering the head pose helps improve the accuracy of 3D head-position prediction.

In this paper, we report on a field study in which we employed two service robots in a bakery store as a sales promotion. Previous studies have explored public applications of service robots public such as shopping malls. However, more evidence is needed that service robots can contribute to sales in real stores. Moreover, the behaviors of customers and service robots in the context of sales promotions have not been examined well. Hence, the types of robot behavior that can be considered effective and the customers' responses to these robots remain unclear. To address these issues, we installed two tele-operated service robots in a bakery store for nearly 2 weeks, one at the entrance as a greeter and the other one inside the store to recommend products. The results show a dramatic increase in sales during the days when the robots were applied. Furthermore, we annotated the video recordings of both the robots' and customers' behavior. We found that although the robot placed at the entrance successfully attracted the interest of the passersby, no apparent increase in the number of customers visiting the store was observed. However, we confirmed that the recommendations of the robot operating inside the store did have a positive impact. We discuss our findings in detail and provide both theoretical and practical recommendations for future research and applications.

Social robotics recommendations have been studied for a long time, and many existing studies have addressed in-store recommendations using robots. However, it has been pointed out that many studies conducted in "in-the-wild" field environments have only focused on the initial stages of customer purchase behavior, such as stopping and engaging in a conversation, and few have been able to induce an interest in the product and even purchase. One of the causes is that the robot itself attracts most of the customer’s attention, making it difficult for customers to be interested in the robot’s recommendations. To solve this problem, this study examines the inclusion of clear and specific instructions to customers in interactions in which the robot recommends services and products. We conducted a field experiment to confirm the effectiveness of such instructive interactions, and found that customers are more likely to be interested in the content recommended by the robot, rather than in the robot itself, through the instructive interaction.

Robot agents are increasingly used for user services, and society is becoming increasingly familiar with such robots. Most robots are of the humanoid type, which are easy to recognize as interaction partners. By interacting with a user, these agents may be able to generate user interest in a product and successfully sell it. However, according to previous studies, it has been suggested that users who are interested in the movement and appearance of the agent may focus only on these aspects and not listen to the recommendation. Therefore,we hypothesized that by making the product the agent, which we call a Self Recommendation Agent(SRA), the attention of the user would be focused on the product itself. Hence, if a user is interested in the agent, it is the same as paying attention to the product. Therefore, we expect that such an agent will be able to gain more attention from users than conventional agents while providing recommendations. To investigate the effectiveness of this agent, we set up a store in a shopping mall and conducted sales experiments. In this field study, we conducted an experiment to compare sales between a SRA and a robot agent.As a result, the SRA was able to make recommendations to many more users than the robot agent, and the users who received recommendations from the SRA remembered more about the recommended product than those who received recommendations from the robot agent.Based on these results, we confirm the possibility that the SRA is effective for advertising.

In this study, to mitigate cyberbullying in communication tools, we propose a method in which a chatbot pretending to be a member of a chat group defends the victim by proxy without permission (i.e., hijacking and proxy persuasion). An experiment was conducted to verify the effectiveness of the proposed method, the results of which showed that it increases the defending behavior of a user who was temporarily hijacked by the chatbot. However, compared with the existing method, the proposed method was found to have a problem with the continued use of the system.

Social robots are used in daily life. One of the applications of social robots is as recommendation systems. Previous research has mainly investigated how persuasive recommendations can be improved by focusing on the non-verbal/verbal behavior of robots. However, to use robots as recommendation systems every day, it is extremely important to examine the persistence of repeated persuasion over a long term, rather than the effect of one-time persuasion. Therefore, the objective of this study was to investigate the persistence of repeated persuasive of robots. For this purpose, robots with three types of behavior (Expert Behavior, Local Behavior, and Growth Behavior) recommended nutrition bars in a situation of daily consumption behavior for two weeks. We could confirm significant differences in the persistent persuasiveness in each behavior. The results suggested that the combination of value co-creation using local information and meta-trust expression had a significant impact on the persistence of the repeated persuasiveness of the robots in the longitudinal period. However, the acceptance of the recommendation robot system decreased due to the increase in the amount of information during recommendation; therefore, a new recommend system to solve this problem is desired.

In this study, we aimed to verify whether expanding a robot’s embodiment influenced the ease of inducing behavioral changes in passersby. We conducted a field experiment to call out to passersby for disinfection in a real shop using robots with various embodiments. As a result, the expansion of the robot embodiment attracted extensive attention of passersby, and then induced behavioral changes to the desired action, such as stopping in front of the robot and disinfecting their hands. Furthermore, the results revealed that in order to induce large behavioral changes, not only the presence of the robot’s body but also its movements with strong embodiment were necessary. These results indicated that expanding the robot’s embodiment was an important factor in inducing behavioral changes in passersby.

To explore the interaction design of an autonomous social robot stationed in a hotel room, we conducted a Wizard of Oz study. We developed a teleoperated robotic system that appears to move autonomously through voice-to-synthesis processing. Comparing the evaluation of the latest autonomous case with one of these teleoperated cases, the results show that it is possible to construct a robotic system that is more highly rated in terms of warmth, competence, and enjoyment of conversation. The results also suggest novel forms of the hotel room robot’s interactive services, such as a hotel-life management service and conversation partner service as a role of a listening presence, which draws out and understands with the guests’ talk.

In this study, we developed POP Cart that uses a shopping cart and an agent to recommend products. The agent on POP Cart is designed to call the customer by name, chatting and making recommendations to the customer casually like a friend. POP Cart has two advantages: first, the agent can form a positive relationship with the customer is in the store, which could be beneficial for a high recommendation success rate, as found in previous studies. Second, the agent can make multiple recommendations for different products according to the position of the customer while they are shopping in a large store. To evaluate the effectiveness of the recommendations made by the POP Cart agent, we conducted a field experiment in a real supermarket in Japan, where 49 participants shopped under three cart conditions. The results revealed that having an agent on a shopping cart is an effective way to recommend and sell products.

Under the circumstances that social robots are increasingly being developed and studied in service encounters at public spaces, are they introduced into residential environments (i.e., private space)? This study hypothesizes that a personal assistant device in residential environments should wear human-like appearance to engage in service as conversation partner. We implemented the interaction design that provides regular services as the current personal assistant and additional service as conversation partner, and then conducted a field experiment where the participants stayed in the hotel rooms with a smart speaker or a social robot. The results support the hypothesis in that of conversation amount and emotional experience by conversation. The results also suggest the possibility of commercial service, namely conversational advertisement through social robots.

As social robots are being increasingly employed in service encounters, effective interactive design should be considered. This paper focused on heartwarming interaction for hospitality, and explored an impact of a kind of robot's behavior, perceived cuteness of a robot, and users' personality on the interactions. The experimental results identified key factors which affects impression of heartwarming interaction.

It is an important functional behavior for humanoid robots to have face-to-face contact with humans. We predict future face position to achieve natural behavior that is similar to the communication between people. Robots gaze at a prediction point for reducing mechanical delay. The proposed system for robots to have face-to-face contact can reduce delay.

Emotional labor in text-based chat environments will be widespread in the near future. It is difficult for novice workers to make smooth conversation with an angry customer in a customer-support work- ing environment, one of the stressful emotional labors. In this pa- per, we propose a method to suppress the customer's anger based on a communication agent introduced into customer-support work in order to facilitate smooth communication. The agent first estab- lishes a positive relationship with the customer, and then the atti- tude changes to be sympathetic to the customer-support worker in order for the customer to follow sympathy and reduce anger. From the experimental evaluation, it is found that there is the possibil- ity of suppressing the participants' anger by this method, but the anger itself is at a higher level than in the common environment without the agent.

While more and more robots have been employed in the service industry, the impact of the human--robot social interaction on a heartwarming interaction service mostly remains an open question. For the purpose of exploring the possibility that a humanoid robot engages in a heartwarming interaction service, we developed herein the first prototype of the service system through a humanoid robot and conducted a field study, where a humanoid robot engages in the service for a customer at a hotel to collect a customer's impression of the service by questionnaire. The results demonstrate a humanoid robot's potential of engaging in a heartwarming interaction service that enhances customer satisfaction of the whole service. An exploratory analysis suggests the differences of the impact in sex and long-term interaction. Finally, we discuss the possibility of another application, namely, effective advertisement through a heartwarming interaction with a humanoid robot.

Brand advertising is a type of advertising that aims at increasing the awareness of companies or products. This type of advertising is well studied in economic, marketing, and psychological literature; however, there are no studies in the area of computational advertising because the effect of such advertising is difficult to observe. In this study, we consider a real-time biding strategy for brand advertising. Here, our objective to maximizes the total number of users who remember the advertisement, averaged over the time. For this objective, we first introduce a new objective function that captures the cognitive psychological properties of memory retention, and can be optimized efficiently in the online setting (i.e., it is a monotone submodular function). Then, we propose an algorithm for the bid optimization problem with the proposed objective function under the second price mechanism by reducing the problem to the online knapsack constrained monotone submodular maximization problem. We evaluated the proposed objective function and the algorithm in a real-world data collected from our system and a questionnaire survey. We observed that our objective function is reasonable in real-world setting, and the proposed algorithm outperformed the baseline online algorithms.