Indoor scene synthesis aims to automatically produce plausible, realistic, and diverse 3D indoor scenes, especially given arbitrary user requirements. Recently, the promising generalization ability of pre-trained large language models (LLM) assist in open-vocabulary indoor scene synthesis. However, the challenge lies in converting the LLM-generated outputs into reasonable and physically feasible scene layouts. In this paper, we propose to generate hierarchically structured scene descriptions with LLM and then compute the scene layouts. Specifically, we train a hierarchy-aware network to infer the fine-grained relative positions between objects and design a divide-and-conquer optimization to solve for scene layouts. The advantages of using hierarchically structured scene representation are two-fold. First, the hierarchical structure provides a rough grounding for object arrangement, which alleviates contradictory placements with dense relations and enhances the generalization ability of the network to infer fine-grained placements. Second, it naturally supports the divide-and-conquer optimization, by first arranging the sub-scenes and then the entire scene, to more effectively solve for a feasible layout. We conduct extensive comparison experiments and ablation studies with both qualitative and quantitative evaluations to validate the effectiveness of our key designs with the hierarchically structured scene representation. Our approach can generate more reasonable scene layouts while better aligned with the user requirements and LLM descriptions. We also present open-vocabulary scene synthesis and interactive scene design results to show the strength of our approach in the applications.

Despite the prevalence of Crowdsourced Live Video Streaming (CLVS), video viewers still suffer from low QoE particularly under rush hours, as the existing Content Delivery Network (CDN) is not scalable enough to handle the massive concurrent streaming. The rapid emergence of Web 3.0 provides new incentives for revisiting and applying the classical P2P networking in CLVS. However, the highly dynamic joining or leaving behavior of CLVS viewers frequently interrupts the real-time streaming and leads to low QoE, which demands to retrofit P2P. In this work, we bridge the gap by proposing a reliable P2P-assisted CLVS system named Venus, where viewers can share their streaming content smoothly, without video freeze regardless of viewers leaving. To realize Venus, different from the single-flow sharing in previous P2P video streaming, we design a novel multiflow framework with lightweight redundancy encoding, so as to handle the inherently high viewer dynamics. Correspondingly, we introduce a multiflow scheduler to enable QoE adaption concertedly over heterogeneous multiple flows. Real-world evaluation confirms the benefits of decentralized CLVS streaming, with Venus outperforming the state-of-the-art CDN solution by almost totally eliminating the video stall while enhancing the video quality by 10.2%.

Emotion recognition is a rising trend in related fields. Existing research mainly aims at the emotion recognition based on single or multi modalities on performance datasets. However, there is little research focusing on gesture-based emotion recognition using non-performance emotional datasets. In this article, we proposed a non-performance emotional data collection scheme using VR to construct a high-quality nonperformance dataset in gesture modality with fear labels. Then we proposed an optimized algorithm for gesture-based fear recognition using BLSTM on this dataset with the accuracy of up to 71.43%. Our work could be applied in many domains such as potentially domestic violence detection and VR game evaluation.

Being able to understand and communicate with domestic cats has always been fascinating to humans, although it is considered a difficult task even for phonetics experts. In this paper, we present our approach to this problem: Purrai, a neural-network-based machine learning platform to interpret cat’s language. Our framework consists of two parts. First, we build a comprehensively constructed cat voice dataset that is 3.7x larger than any existing public available dataset [1]. To improve accuracy, we also use several techniques to ensure labeling quality, including rule-based labeling, cross validation, cosine distance, and outlier detection, etc. Second, we design a two-stage neural network structure to interpret what cats express in the context of multiple sounds called sentences. The first stage is a modification of Google’s Vggish architecture [2] [3], which is a Convolutional Neural Network (CNN) architecture that focuses on the classification of nine primary cat sounds. The second stage takes the probability outputs of a sequence of sound classifications from the first stage and determines the emotional meaning of a cat sentence. Our first stage architecture generates a top-l and top-2 accuracy of 74.1% and 92.1%, better than that of the state-of-the-art approach: 64.9% and 83.4% [4]. Our sentence-based AI model achieves an accuracy of 81.1% for emotion prediction.