Self-supervised visual foundation models produce powerful embeddings that achieve remarkable performance on a wide range of downstream tasks. However, unlike vision-language models such as CLIP [67], self-supervised visual features are not readily aligned with language, hindering their adoption in open-vocabulary tasks. Our method, named dino.txt, unlocks this new ability for DINOv2 [63], a widely used self-supervised visual encoder. We build upon the LiT training strategy [97], which trains a text encoder to align with a frozen vision model but leads to unsatisfactory results on dense tasks. We propose several key ingredients to improve performance on both global and dense tasks, such as concatenating the [CLS] token with the patch average to train the alignment and curating data using both text and image modalities. With these, we successfully train a CLIP-like model with only a fraction of the computational cost compared to CLIP while achieving state-of-the-art results in zero-shot classification and open-vocabulary semantic segmentation.

The global housing crisis has led to the widespread adoption of micro-apartments, which present significant challenges due to their limited space. To address these issues, we developed oOoBOT, a modular robotic furniture system capable of transforming its form and function to meet diverse user needs. The design incorporates a Hoberman mechanism for expandable structures and an umbrella-inspired tabletop for flexible use. Enhanced mobility and reconfigurability are achieved through modular components. By optimizing space efficiency, oOoBOT enables multifunctional activities such as sleeping, working, and socializing, offering a practical solution for confined living environments.

We address the problem of generalized category discovery (GCD) that aims to partition a partially labeled collection of images; only a small part of the collection is labeled and the total number of target classes is unknown. To address this generalized image clustering problem, we revisit the mean-shift algorithm, i.e., a classic, powerful technique for mode seeking, and incorporate it into a contrastive learning framework. The proposed method, dubbed Contrastive Mean-Shift (CMS) learning, trains an embedding network to produce representations with better clustering properties by an iterative process of mean shift and contrastive update. Experiments demonstrate that our method, both in settings with and without the total number of clusters being known, achieves state-of-the-art performance on six public GCD benchmarks without bells and whistles.

A library serves as a repository of knowledge accessible to individuals of all ages, genders, educational backgrounds, social statuses, and economic levels. It stands as a communal space where community members can gather, bridging information disparities among various societal strata. To enhance accessibility to such libraries for a broader spectrum of people, we have introduced the CollaBot system. This system offers tailored services to users through the collaboration of robots. Our investigation encompassed the acceptance of robot types by users, robot characterization, and the prioritization of robot-provided services. Over the course of three stages of user evaluation, it became evident that participants preferred product-type robots over anthropomorphic robots. Furthermore, they expressed a preference for robots that assist other robots, even if these assisting robots exhibit clumsiness, as opposed to robots that exclusively excel in their designated tasks. Lastly, service prioritization varied based on the specific limitations or deficiencies faced by individual users.

Scaling down the transistor size enables a cost-effective high-performing solution in modern semiconductor designs. However, the added complexity of process and extremely small critical three-dimensional spacing of the transistor structure make it more challenging to maintain precise variation control. Defects introduced here can lead to different intermittent behaviors, some of which might not appear as fails during traditional testing procedures but might manifest as a system failure in the field. The emergence of artificial intelligence chip design has introduced high demands for a massive number of multi-cores connected in parallel with a huge memory array size in a chip. This further increases the importance of identifying rare tail events through in-depth diagnosis in advanced nodes. Various sophisticated algorithms have been proposed to improve defect coverage. However, the addition of algorithms may increase test cost tremendously while providing limited benefits for specific fault types which may not happen. Therefore, selecting a smart combination of algorithms that provide enough coverage for the specific product application is essential for cost-effective testing. In this paper, we review the electrical properties of marginal defects inserted in an SRAM device to evaluate test escapes in the latest technology. We also present a new algorithm to improve test coverage efficiency. A commercially available memory BIST tool was used to load a DFT compatible algorithm and operation set for memory test. A commercially available analog simulation tool was used in combination with a novel defect simulation flow to evaluate digital and analog behavior of the device.

We address the task of weakly-supervised few-shot image classification and segmentation, by leveraging a Vision Transformer (ViT) pretrained with self-supervision. Our proposed method takes token representations from the self-supervised ViT and leverages their correlations, via selfattention, to produce classification and segmentation predictions through separate task heads. Our model is able to effectively learn to perform classification and segmentation in the absence of pixel-level labels during training, using only image-level labels. To do this it uses attention maps, created from tokens generated by the self-supervised ViT backbone, as pixel-level pseudo-labels. We also explore a practical setup with “mixed” supervision, where a small number of training images contains ground-truth pixel-level labels and the remaining images have only image-level labels. For this mixed setup, we propose to improve the pseudo-labels using a pseudo-label enhancer that was trained using the available ground-truth pixel-level labels. Experiments on Pascal-5iand COCO-20idemonstrate significant performance gains in a variety of supervision settings, and in particular when little-to-no pixel-level labels are available.

Nunchi is a high context communication skill which makes an individual understand the counterpart’s indirect social cues and respond appropriately. Robotic things can perceive and recognize situations, and express appropriate responses to situations. Thus, this study was conducted with the expectation that the robotic things equipped with Nunchi, which understand the situation well and behave appropriately instead of the workers, would allow the workers to focus on their work. In order to investigate the effect of robotization of the objects on the degree to which Nunchi is required for the participant, task load, and the impression of objects according to the socio-relational context, a 2 (socio-relational context: stranger group vs. friend group) X 2 (robotization: robotic things vs. baseline things) mixed-participant experiment was designed. As a result, participants evaluated robotic things as more useful and social than baseline things. In addition, Nunchi was required less for the people in the stranger group when robotic things were used in their work than when baseline things were used. Finally, participants felt their task load reduced when the robotic things were used.

Voice cloning techniques using deep neural networks have been used for frauds, such as false financial transactions or fake news, limiting their widespread application. However, when incorporated with communication robots, voice cloning may increase social connectedness for isolated people. Herein, we suggested the concept of a voice-cloned communication robot (VCR), equipped with acquaintances’ voices to allow people in isolation to feel connected with their acquaintances. We developed a prototype VCR, conducted exploratory qualitative and quantitative studies, and verified its potential effectiveness in enhancing the mental health of isolated people.

The aim of our study was to investigate people’s preferred interaction with a robot -whether they prefer a robot that offers assistance based on its own judgment or a robot that follows specific verbal commands given by the user. To achieve this, we presented two types of sound-based interfaces. The first interface was a robot that judges the user’s needs based on the sound the user makes, without requiring any verbal commands. For example, if the user slaps a table with a pile of paper to organize it, the robot opens the drawer where the stapler is located. The second interface was a robot that follows the user’s verbal commands. In this interface, the user specifically asks the robot to find the stapler, and the robot opens the drawer accordingly. Our results showed that people preferred the verbal command interface in terms of usefulness, intelligence, appropriateness, and service evaluation. This indicates that individuals prefer to interact with a robot that follows their specific commands rather than a robot that infers people’s intention and reacts accordingly. Our findings suggest that designing robots to follow specific verbal commands may lead to more positive user experiences and perceptions of the robot’s intelligence and usefulness.

We introduce the integrative task of few-shot classification and segmentation (FS-CS) that aims to both classify and segment target objects in a query image when the target classes are given with a few examples. This task combines two conventional few-shot learning problems, few-shot classification and segmentation. FS-CS generalizes them to more realistic episodes with arbitrary image pairs, where each target class may or may not be present in the query. To address the task, we propose the integrative few-shot learning (iFSL) framework for FS-CS, which trains a learner to construct class-wise foreground maps for multi-label classification and pixel-wise segmentation. We also develop an effective iFSL model, attentive squeeze network (ASNet), that leverages deep semantic correlation and global self-attention to produce reliable foreground maps. In experiments, the proposed method shows promising performance on the FS-CS task and also achieves the state of the art on standard few-shot segmentation benchmarks.

To help people use their limited space efficiently, we propose an origami-based transformable robotic space called “PopupBot.” Specifically, we developed a robotic playhouse for children. A large origami structure with a bellows pattern is controlled by a servo motor and transforms into various types of furniture. For natural child-robotic space interaction, PopupBot perceives the child's intention for the space through speech recognition and provides appropriate space by inferring the space type matching the intention. We expect the PopupBot to provide a new space for people who suffer from staying in limited space especially due to the COVID-19 pandemic.

In order to prevent COVID-19 infection in the hospital environment, the medical staff is handling many treatments for patients non face-to-face, which reduces the efficiency of medical services. Robots may enable smooth non face-to-face interactions between medical staff and patients by providing cognitive and physical support to the medical staff. In this paper, we identified the medical staff's pain points and needs about the robots which would help them. In addition, researchers and medical staff together participated in generating the design concept of a robot needed in times of COVID-19 as design participants. We conducted qualitative interviews about robots with nurses working in a negative pressure isolation room (NPIR) where patients with COVID-19 are isolated while treatment. As a result, the needs for supporting increased workload including inventory monitoring, waste management, meal delivery, and medicine delivery as well as supporting communication in emergency including communication in patient's emergency and communication in medical staff's emergency were discovered. Based on the findings from the interview, we proposed a robot design concept that can satisfy medical staff's need in NPIR.

Reconstructing 3D objects in natural environments requires solving the ill-posed problem of geometry, spatially-varying material, and lighting estimation. As such, many approaches impractically constrain to a dark environment, use controlled lighting rigs, or use few handheld captures but suffer reduced quality. We develop a method that uses just two smartphone exposures captured in ambient lighting to reconstruct appearance more accurately and practically than baseline methods. Our insight is that we can use a flash/no-flash RGB-D pair to pose an inverse rendering problem using point lighting. This allows efficient differentiable rendering to optimize depth and normals from a good initialization and so also the simultaneous optimization of diffuse environment illumination and SVBRDF material. We find that this reduces diffuse albedo error by 25%, specular error by 46%, and normal error by 30% against single-and paired-image baselines that use learning-based techniques. Given that our approach is practical for everyday solid objects, we enable photorealistic relighting for mobile photography and easier content creation for augmented reality.

With becoming an aging society rapidly, the digital divide among the elderly is expected to emerge as a social problem. In order to bridge the digital gap between the elderly and the general public, it is necessary to find out what difficulties the elderly have in using information and communication technology (ICT) and what technologies and services they need. In order to figure out those issues, we conducted three phases of study: ICT literacy survey, interview, and design workshop (N=10). This study has three major findings. First, from the physical view, the elderly found ICT that reduces their movement distance attractive. Second, from the cognitive perspective, services which reduce the gap between the real physical ability of the elderly and the perceived ability of them by themselves are needed. Lastly, from the psychological view, an intuitive and natural interface is needed for the elderly who are afraid of learning new technology.

This study explores the effects of the positioning strategy of domestic social robots on the purchase intention of consumers. Specifically, the authors investigate the effects of robot positioning as companions with as assistants and as appliances. The study results showed that the participants preferred the domestic social robots positioned as assistants rather than as companions. Moreover, for male participants, the positioning of domestic social robots as appliances was also preferred over robots positioned as companions. The study results also showed that the effects of positioning on the purchase intention were mediated by the participants’ perception of usefulness regarding the robot.

Due to advances in technology, various types of existing products can be robotized. The interaction design between the robotized objects which are the robotized products and users has become an emerging issue since how to interact affects user acceptance of the new products. Because the robots can perceive, recognize and assist the users, robotized objects can assist users by reading the context and guessing their intentions in advance, or can assist users in response to users’ clear requests. In order to explore the effective interaction strategy of robotized object, we conduct 2 × 2 mixed-participant experiment. As a result, participants were more surprised by and felt positive emotion toward robotized objects with proactive interaction than with reactive interaction. In addition, the robotized object with proactive interaction was evaluated as more intelligent and easier to use than that with reactive interaction. The influence of interaction strategies on satisfaction of robotized objects was mediated by surprise and positive emotions. Furthermore, the robotized object with proactive interaction was evaluated as more positive when the user’s intention was expressed only by action cue than when it was expressed by both verbal cue and action cue.

Semi-supervised domain adaptation (SSDA) is to adapt a learner to a new domain with only a small set of labeled samples when a large labeled dataset is given on a source domain. In this paper, we propose a pair-based SSDA method that adapts a model to the target domain using self-distillation with sample pairs. Each sample pair is composed of a teacher sample from a labeled dataset (i.e., source or labeled target) and its student sample from an unlabeled dataset (i.e., unlabeled target). Our method generates an assistant feature by transferring an intermediate style between the teacher and the student, and then train the model by minimizing the output discrepancy between the student and the assistant. During training, the assistants gradually bridge the discrepancy between the two domains, thus allowing the student to easily learn from the teacher. Experimental evaluation on standard benchmarks shows that our method effectively minimizes both the inter-domain and intra-domain discrepancies, thus achieving significant improvements over recent methods.

Omnidirectional videos capture environmental scenes effectively, but they have rarely been used for geometry reconstruction. In this work, we propose an egocentric 3D reconstruction method that can acquire scene geometry with high accuracy from a short egocentric omnidirectional video. To this end, we first estimate per-frame depth using a spherical disparity network. We then fuse per-frame depth estimates into a novel spherical binoctree data structure that is specifically designed to tolerate spherical depth estimation errors. By subdividing the spherical space into binary tree and octree nodes that represent spherical frustums adaptively, the spherical binoctree effectively enables egocentric surface geometry reconstruction for environmental scenes while simultaneously assigning high-resolution nodes for closely observed surfaces. This allows to reconstruct an entire scene from a short video captured with a small camera trajectory. Experimental results validate the effectiveness and accuracy of our approach for reconstructing the 3D geometry of environmental scenes from short egocentric omnidirectional video inputs. We further demonstrate various applications using a conventional omnidirectional camera, including novel-view synthesis, object insertion, and relighting of scenes using reconstructed 3D models with texture.

In order to enable view-dependent appearance synthesis from the light fields of a scene, it is critical to evaluate the geometric relationships between light and view over surfaces in the scene with high accuracy. Perfect diffuse reflectance is commonly assumed to estimate geometry from light fields via multiview stereo. However, this diffuse surface assumption is invalid with real-world objects. Geometry estimated from light fields is severely degraded over specular surfaces. Additional scene-scale 3D scanning based on active illumination could provide reliable geometry, but it is sparse and thus still insufficient to calculate view-dependent appearance, such as specular reflection, in geometry-based view synthesis. In this work, we present a practical solution of inverse rendering to enable view-dependent appearance synthesis, particularly of scene scale. We enhance the scene geometry by eliminating the specular component, thus enforcing photometric consistency. We then estimate spatially-varying parameters of diffuse, specular, and normal components from wide-baseline light fields. To validate our method, we built a wide-baseline light field imaging prototype that consists of 32 machine vision cameras with fisheye lenses of 185 degrees that cover the forward hemispherical appearance of scenes. We captured various indoor scenes, and results validate that our method can estimate scene geometry and reflectance parameters with high accuracy, enabling view-dependent appearance synthesis at scene scale with high fidelity, i.e., specular reflection changes according to a virtual viewpoint.

We propose to address the problem of few-shot classification by meta-learning "what to observe" and "where to attend" in a relational perspective. Our method lever-ages relational patterns within and between images via self-correlational representation (SCR) and cross-correlational attention (CCA). Within each image, the SCR module transforms a base feature map into a self-correlation tensor and learns to extract structural patterns from the tensor. Between the images, the CCA module computes cross-correlation between two image representations and learns to produce co-attention between them. Our Relational Embedding Network (RENet) combines the two relational modules to learn relational embedding in an end-to-end manner. In experimental evaluation, it achieves consistent improvements over state-of-the-art methods on four widely used few-shot classification benchmarks of miniImageNet, tieredImageNet, CUB-200-2011, and CIFAR-FS.

Few-shot semantic segmentation aims at learning to segment a target object from a query image using only a few annotated support images of the target class. This challenging task requires to understand diverse levels of visual cues and analyze fine-grained correspondence relations between the query and the support images. To address the problem, we propose Hypercorrelation Squeeze Networks (HSNet) that leverages multi-level feature correlation and efficient 4D convolutions. It extracts diverse features from different levels of intermediate convolutional layers and constructs a collection of 4D correlation tensors, i.e., hypercorrelations. Using efficient center-pivot 4D convolutions in a pyramidal architecture, the method gradually squeezes high-level semantic and low-level geometric cues of the hypercorrelation into precise segmentation masks in coarse-to-fine manner. The significant performance improvements on standard few-shot segmentation benchmarks of PASCAL-5i, COCO-20i, and FSS-1000 verify the efficacy of the proposed method.

Smart speakers have different speech style depending on the installed artificial intelligence (AI). Furthermore, the AI agent’s appearances can make different impressions. Hence, it might give characters to the AI agent through speaking and appearance modalities. To determine how an AI agent’s speech and tactility types affect users’ perception, we designed a 2(speech types: assistant-like vs. companion-like) × 2(flexibility types: flexible vs. hard) × 2(roughness types: rough vs. smooth) mixed-participant experiment (N=48). As a result, when the speech style is like a companion, it is possible to give an impression of sociability through a flexible material. However, there was no significant difference by flexibility type when being an assistant. In addition, there was a significant interaction effect between speech types and flexibility types on usefulness. When the speech type is companion-like, the AI agent with flexible material was perceived as being more useful and providing better services than that with hard material. On the contrary, when the speech type is assistant-like, the opposite result was revealed. Regardless of the speech type, a flexible material increases the impression of sociability with higher service evaluation, and a rough finish gives the AI agent an impression of usefulness with higher service evaluation.

A study was conducted to investigate the effects of category labels of domestic robots on their consumer acceptance. The authors posited that compared to the label robots, a pre-existent category label such as home appliances would increase the consumers' evaluation of and purchase intention towards the products. It is suggested that the pre-existent category label helps consumers to perceive the functional values they stand to gain by consuming the product more than the label robots, which is often related to the concepts generated around cultural artifacts. The results of the study confirmed the hypotheses, and further discussions are provided in this paper.

The purpose of this study is to investigate a robot's impression perceived by users as well as the accuracy of perception of location information, which the robot provided according to the modality type of the robot. To explore this, we designed two 2 (verbal types: deictic vs. descriptive) x 2 (nose pointing: with nose vs. without nose) x 2 (eye pointing: with eyes vs. without eyes) mixed-participant studies. In the first study, we investigated the impacts of the robot's modality type in the imperative pointing situation. As a result, participants identified the robot's pointing gesture with nose as more effective, social, and positive, than the robot's pointing gesture without nose. Moreover, the descriptive speech robot was evaluated as more positive than the deictic speech robot. In terms of the accuracy of perception of location information, which the robot provided, participants identified the robot-designated chair more accurately when the robot delivered a deictic speech than when the robot delivered a descriptive speech. For the second study, we explored the effects of the robot's modality type in the declarative pointing situation. As a result, the robot's descriptive speech was rated as effective, social, natural, competent, trustworthy, and more positive than deictic speech. In the case of the robot's pointing gestures, pointing gesture with nose was evaluated as more effective, social, natural, competent, trustworthy, and positive than that without nose. In terms of the accuracy of location information perception, participants perceived the location of the object designated by the robot more accurately when the robot used descriptive speech, pointed with nose and without eyes.

In smart home environments, central interfaces that take commands from users and give orders to each relevant device appropriately are increasingly important. We investigated the type of central interface that consumers are more willing to adopt and whether these interfaces enhance the evaluation of services provided by smart home devices. This study confirms that speaker interfaces are preferred over social robots, speaker interfaces are perceived by users as more persuasive, and the adoption of central interfaces increases the overall service evaluation.

With the development of robotic technology, various types of robotic products have been developed, ranging from robotic objects that support small daily necessities (such as robotic umbrellas and frames) to robotic furniture (such as robotic chairs and drawers). Owing to consumerism, the consumer marketplace now overflows with surplus products, and storage and organizational products have been increasingly necessary. With this social stream, we focused on designing robotic storage furniture in this study. To create such a type of furniture that is acceptable by consumers, a qualitative user study was conducted with 16 subjects by analyzing the users' behaviors when using ordinary storage furniture. From this user study, we found strong user needs in terms of organizing and finding objects. Thus, we developed two types of robotic cabinets: the first one to assist users' organizing behavior and the second one to assist their finding behavior. To examine the effectiveness of the developed prototypes, we executed a 3 (behavior the robotic cabinet assists: baseline vs. organizing vs. finding) within-participants experiment. The result shows that significant effects vary depending on the type of robotic cabinet.

Robotic things and social robots have been introduced into home, and they are expected to change the relationships between humans. Our study examines whether the introduction of robotic things or social robots, and the way that they are organized, can change the social relationship between family members. To observe this phenomenon, we designed a living lab experiment that simulated a home environment and recruited two families to participate. Families were asked to conduct home activities within two different types of Internet of Robotic Things(IoRT):1)internet of only robotic things(IoRT without mediator condition), and 2)internet of robotic things mediated by a social robot(IoRT with mediator condition). We recorded the interactions between the family members and the robotic things during the experiments and coded them into a dataset for social network analysis. The results revealed relationship differences between the two conditions. The introduction of IoRT without mediator motivated younger generation family members to share the burden of caring for other members, which was previously the duty of the mothers. However, this made the interaction network inefficient to do indirect interaction. On the contrary, introducing IoRT with mediator did not significantly change family relationships at the actor-level, and the mothers remained in charge of caring for other family members. However, IoRT with mediator made indirect interactions within the network more efficient. Furthermore, the role of the social robot mediator overlapped with that of the mothers. This shows that a social robot mediator can help the mothers care for other members of the family by operating and managing robotic things. Additionally, we discussed the implications for developing the IoRT for home.

The objective of this study is to investigate the effect of socio-relational context and robot's tactility on the sense of social presence and user satisfaction. We executed a 2(socio-relational context: an intimate remote sender vs. a non-intimate remote sender) x 2(tactility: anthropomorphic tactility vs. non-anthropomorphic tactility) within-participants experiment (\pmbN=24). As a result, participants felt a stronger sense of a remote sender's presence when the remote sender was an intimate person than a non-intimate person, and felt a greater sense of a remote sender's presence when a robot had an anthropomorphic tactility than when a robot had a non-anthropomorphic tactility. In addition, participants preferred a telepresence robot with an anthropomorphic tactility in the context that they interacted with an intimate person, whereas a telepresence robot with a non-anthropomorphic tactility was preferred in the context that they interact with a non-intimate person in robot-mediated communication.

Genetic Programming has been successfully applied to learn to rank program elements according to their likelihood of containing faults. However, all GP-evolved formulæ that have been studied in the fault localization literature up to now are single expressions that only use a small set of basic functions. Based on recent theoretical analysis that different formulæ may be more effective against different classes of faults, we evaluate the impact of allowing ternary conditional operators in GP-evolved fault localization by extending our fault localization tool called FLUCCS. An empirical study based on 210 real world Java faults suggests that the simple inclusion of ternary conditional operator can help fault localization by placing up to 11% more faults at the top compared to our baseline, FLUCCS, which in itself can already rank 50% more faults at the top compared to the state-of-the-art SBFL formulæ.

This study aims to investigate the effects of information delivery types on users' perception toward the robot. We executed two experiments to explore appropriate information delivery types in each situation. In the first study, we compared which type of information delivery is suitable to the situation that the robot conveys environment states. In the second study, we examined a proper information delivery type in the situation that the robot conveys its internal states. We conducted a 3(information delivery types: speech vs. reflexive cue vs. none) within-participants experiment (N=24) both in the first study and in the second study. The results of the study showed that participants perceived a robot with the reflexive cue as more anthropomorphic and animate than one with speech and one without response. In addition, participants evaluated the service of the robot with the reflexive cue more positively than the robot with speech and the robot without the response in the both studies.