Recent advances in the diffusion models have significantly improved text-to-image generation. However, generating videos from text is a more challenging task than generating images from text, due to the much larger dataset and higher computational cost required. Most existing video generation methods use either a 3D U-Net architecture that considers the temporal dimension or autoregressive generation. These methods require large datasets and are limited in terms of computational costs compared to text-to-image generation. To tackle these challenges, we propose a simple but effective novel grid diffusion for text-to-video generation without temporal dimension in architecture and a large text-video paired dataset. We can generate a high-quality video using a fixed amount of GPU memory regardless of the number of frames by representing the video as a grid image. Additionally, since our method reduces the dimensions of the video to the dimensions of the image, various image-based methods can be applied to videos, such as textguided video manipulation from image manipulation. Our proposed method outperforms the existing methods in both quantitative and qualitative evaluations, demonstrating the suitability of our model for real-world video generation.

Representing wild sounds as images is an important but challenging task due to the lack of paired datasets between sound and images and the significant differences in the characteristics of these two modalities. Previous studies have focused on generating images from sound in limited categories or music. In this paper, we propose a novel approach to generate images from in-the-wild sounds. First, we convert sound into text using audio captioning. Second, we propose audio attention and sentence attention to represent the rich characteristics of sound and visualize the sound. Lastly, we propose a direct sound optimization with CLIPscore and AudioCLIP and generate images with a diffusion-based model. In experiments, it shows that our model is able to generate high quality images from wild sounds and outperforms baselines in both quantitative and qualitative evaluations on wild audio datasets.

Public spaces1 are a basis of urban lives. The mode and culture of sharing could be an indicator of the quality of life in the cities. For example, the buses and restaurants should comfort and bring satisfaction to individual visitors, including the vulnerable with special needs. However, their operation mostly occurs in rather a closed and exclusive manner [2]. An inherent limitation to such an inclusive sharing lies in the exclusive modes of traditional device interfaces; a variety of devices, called public devices hereafter, are installed in the public space and determine operational details of the space. As such, the space itself is shared, however, the public devices are controlled in exclusive ways. Could the sharing of the public space be operated in a democratic way?

Public spaces are equipped with 'public actuators', e.g., HVAC, lighting fixtures, speakers, or streaming TV channels to ensure their visitors' comfort. However, many public actuators rarely allow the visitors to adjust their operation, limiting their utility and fairness across the visitors. Also, the social bar is often too high to speak up one's preference and attempt to change an actuator's operation. Social control and use of IoT devices is an underexplored new direction of research even with its huge potential and implication, but comes with high complexity and scale. This paper proposes a novel architecture, namely, Social Control-and-Use Architecture for IoT Devices, which provides a systematic view and an effective tool to handle the complication and intricacy in system design. It also proposes Hivemind, a first-of-a-kind system developed, upon the architecture, for sharing IoT-enabled actuators in a public space. It transforms an exclusively-controlled actuator in a public space into a true public actuator, supporting visitors to instantly participate in the democratic collective control. Also, a myriad of off-the-shelf actuators are easily incorporated without modification to their implementation. The field deployment of Hivemind shows its comprehensive service coverage as well as the users' approval on the democratic collective control of public actuators.

Public spaces, where we gather, commune, and take a rest, are the essential parts of a modern urban landscape, enriching citizen's everyday life [3]. How we share these spaces are considered an indicator of the quality of life. Public spaces thus have a responsibility to provide comfort and satisfaction to any visitors. However, in most times, the operations of the spaces are managed in rather an exclusive manner.

Continuous mobile sensing applications (CMSAs) run 24 hours in the background, consuming a significant amount of energy. Due to CMSA's dynamic behavior according to the users' context, it is very difficult to predict the energy consumption of CMSA. User trace-based solutions have been proposed to effectively estimate the energy consumption of CMSA, but they suffer burdensome user trace collection. We propose Scenethesizer, a scenario-based energy estimation system, which generates and augments the user traces for unseen application usage scenarios and estimate the energy consumption of CMSA based on the generated user traces.

Deep-learning (DL) is receiving huge attention as enabling techniques for emerging mobile and IoT applications. It is a common practice to conduct DNN model-based inference using cloud services due to their high computation and memory cost. However, such a cloud-offloaded inference raises serious privacy concerns. Malicious external attackers or untrustworthy internal administrators of clouds may leak highly sensitive and private data such as image, voice and textual data. In this paper, we propose Occlumency, a novel cloud-driven solution designed to protect user privacy without compromising the benefit of using powerful cloud resources. Occlumency leverages secure SGX enclave to preserve the confidentiality and the integrity of user data throughout the entire DL inference process. DL inference in SGX enclave, however, impose a severe performance degradation due to limited physical memory space and inefficient page swapping. We designed a suite of novel techniques to accelerate DL inference inside the enclave with a limited memory size and implemented Occlumency based on Caffe. Our experiment with various DNN models shows that Occlumency improves inference speed by 3.6x compared to the baseline DL inference in SGX and achieves a secure DL inference within 72% of latency overhead compared to inference in the native environment.

Real-time remote interaction has become easier and richer powered by recent advances in mobile computing and communication. A number of research have been explored on enriching family interaction by augmenting an interaction channel with asynchronous communication [6] or additional sensory stimuli [5]. However, it is still far from achieving a sense of living together for family members involuntarily living apart, especially in context-aware impromptu interaction. For families living together, it is trivial to naturally perceive behavioral and situational contexts of the other and initiate a relevant interaction intuitively. For example, a wife starts a casual chat with asking her husband what he is going to cook when she sees him going to the kitchen or hears a simmering sound.

People in work-separated families have been heavily relying on cutting-edge face-to-face communication services. Despite their ease of use and ubiquitous availability, experiences in living together are still far incomparable to those through remote face-to-face communication. We envision that enabling a remote person to be spatially superposed in one's living space would be a breakthrough to catalyze pseudo living-together interactivity. We propose HomeMeld, a zero-hassle self-mobile robotic system serving as a co-present avatar to create a persistent illusion of living together for those who are involuntarily living apart. The key challenges are 1) continuous spatial mapping between two heterogeneous floor plans and 2) navigating the robotic avatar to reflect the other's presence in real time under the limited maneuverability of the robot. We devise a notion of functionally equivalent location and orientation to translate a person's presence into another in a heterogeneous floor plan. We also develop predictive path warping to seamlessly synchronize the presence of the other. We conducted extensive experiments and deployment studies with real participants.

No abstract available.