Generating human motions from textual descriptions has gained growing research interest due to its wide range of applications. However, only a few works consider human- scene interactions together with text conditions, which is crucial for visual and physical realism. This paper fo-cuses on the task of generating human motions in 3D in-door scenes given text descriptions of the human-scene in-teractions. This task presents challenges due to the multi-modality nature of text, scene, and motion, as well as the need for spatial reasoning. To address these challenges, we propose a new approach that decomposes the complex prob-lem into two more manageable sub-problems: (1) language grounding of the target object and (2) object-centric motion generation. For language grounding of the target ob-ject, we leverage the power of large language models. For motion generation, we design an object-centric scene rep-resentation for the generative model to focus on the target object, thereby reducing the scene complexity and facilitating the modeling of the relationship between human motions and the object. Experiments demonstrate the better motion quality of our approach compared to baselines and validate our design choices. Code will be available at link.

Recently, there has been a surge in cyberattacks targeting the Internet of Health Things (IoHT), increasing the urgency for advancing network intrusion detection systems (IDS). Machine learning techniques, especially deep neural networks (DNNs), are demonstrating potential in improving the precision of detection methods. Despite their advantages, the complexity of DNNs can obscure their decision-making process, impacting their acceptance in security-critical environments. To improve the transparency of DNN models, we propose a novel post-hoc interpretation method that applies a perturbation-based approach with an optimized mask applied to an autoencoder model for IDS. More importantly, we leverage contrastive learning to maintain perturbed samples within the original feature space, reducing the risk of misclassification due to sample drift and ensuring a clear interpretation of the mask. We validate our approach using the NSL-KDD and UNSW15 datasets, showing that it provides clearer and more robust explanations compared to existing methods. This enhancement in interpretability is pivotal for healthcare cybersecurity experts to gain insights into the decision-making processes of black-box models.

The recent breakthroughs in machine learning (ML) and deep learning (DL) have catalyzed the design and development of various intelligent systems over wide application domains. While most existing machine learning models require large memory and computing power, efforts have been made to deploy some models on resource-constrained devices as well. A majority of the early application systems focused on exploiting the inference capabilities of ML and DL models, where data captured from different mobile and embedded sensing components are processed through these models for application goals such as classification and segmentation. More recently, the concept of exploiting the mobile and embedded computing resources for ML/DL model training has gained attention, as such capabilities allow (i) the training of models via local data without the need to share data over wireless links, thus enabling privacy-preserving computation by design, (ii) model personalization and environment adaptation, and (iii) deployment of accurate models in remote and hardly accessible locations without stable internet connectivity. This work summarizes and analyzes state-of-the-art systems research that allows such on-device model training capabilities and provides a survey of on-device training from a systems perspective.

Conversational recommendation systems (CRS) aim to timely and proactively acquire user dynamic preferred attributes through conversations for item recommendation. In each turn of CRS, there naturally have two decision-making processes with different roles that influence each other: 1) director, which is to select the follow-up option (i.e., ask or recommend) that is more effective for reducing the action space and acquiring user preferences; and 2) actor, which is to accordingly choose primitive actions (i.e., asked attribute or recommended item) to estimate the effectiveness of the director’s option. However, existing methods heavily rely on a unified decision-making module or heuristic rules, while neglecting to distinguish the roles of different decision procedures, as well as the mutual influences between them. To address this, we propose a novel Director-Actor Hierarchical Conversational Recommender (DAHCR), where the director selects the most effective option, followed by the actor accordingly choosing primitive actions that satisfy user preferences. Specifically, we develop a dynamic hypergraph to model user preferences and introduce an intrinsic motivation to train from weak supervision over the director. Finally, to alleviate the bad effect of model bias on the mutual influence between the director and actor, we model the director’s option by sampling from a categorical distribution. Extensive experiments demonstrate that DAHCR outperforms state-of-the-art methods.

Conversational recommendation systems (CRS) aim to interactively acquire user preferences and accordingly recommend items to users. Accurately learning the dynamic user preferences is of crucial importance for CRS. Previous works learn the user preferences with pairwise relations from the interactive conversation and item knowledge, while largely ignoring the fact that factors for a relationship in CRS are multiplex. Specifically, the user likes/dislikes the items that satisfy some attributes (Like/Dislike view). Moreover social influence is another important factor that affects user preference towards the item (Social view), while is largely ignored by previous works in CRS. The user preferences from these three views are inherently different but also correlated as a whole. The user preferences from the same views should be more similar than that from different views. The user preferences from Like View should be similar to Social View while different from Dislike View. To this end, we propose a novel model, namely Multi-view Hypergraph Contrastive Policy Learning (MHCPL). Specifically, MHCPL timely chooses useful social information according to the interactive history and builds a dynamic hypergraph with three types of multiplex relations from different views. The multiplex relations in each view are successively connected according to their generation order in the interactive conversation. A hierarchical hypergraph neural network is proposed to learn user preferences by integrating information of the graphical and sequential structure from the dynamic hypergraph. A cross-view contrastive learning module is proposed to maintain the inherent characteristics and the correlations of user preferences from different views. Extensive experiments conducted on benchmark datasets demonstrate that MHCPL outperforms the state-of-the-art methods.

Widely linear modeling is an important quaternion signal processing technique for capturing the complete second-order statistics of quaternion signals. However, the algorithms based on widely linear modeling are computationally expensive due to the augmented variables and statistics. In this letter, a fast estimation technique based on a real augmented representation of widely linear modeling is proposed in the context of quaternion extreme learning machine with augmented hidden layer (QELMAH), resulting in a reduction of almost 93% of multiplications and 75% of additions. An equivalence between the proposed algorithm and the original QELMAH is theoretically established by proving that the trained networks with the proposed algorithm and the original one mathematically implement identical mapping. Such a technique is also applicable to the widely linear quaternion recursive least squares algorithm. The theoretical analysis and the effectiveness of the proposed algorithms are validated by simulations on two benchmark problems.

The cross-domain image captioning, which is trained on a source domain and generalized to other domains, usually faces the large domain shift problem. Although prior work has attempted to leverage both paired source and unpaired target data to minimize this shift, the performance is still unsatisfactory. One main reason lies in the large discrepancy in language expression between two domains, where diverse language styles are adopted to describe an image from different views, resulting in different semantic descriptions for an image. To tackle this problem, this paper proposes a Style-based Cross-domain Image Captioner (SCIC) which incorporates the discriminative style information into the encoder-decoder framework, and interprets an image as a special sentence according to external style instructions. Technically, we design a novel "Instruction-based LSTM", which adds the instruct gate to collect a style instruction, and then outputs a specified format according to that instruction. Two objectives are designed to train I-LSTM: 1) generating correct image descriptions and 2) generating correct styles, thus the model is expected to accurately capture the semantic meanings of an image by the special caption as well as understand the syntactic structure of the caption. We use MS-COCO as the source domain, and Oxford-102, CUB-200, Flickr30k as the target domains. Experimental results demonstrate that our model consistently outperforms the previous methods, and the style information incorporating with I-LSTM significantly improves the performance, with 5% CIDEr improvements at least on all datasets.

This paper presents a low-resolution infrared thermal dataset of people and thermal objects, such as a working laptop, in indoor environments. The dataset was collected by a far infrared thermal camera (32x24 pixels), which can capture the position and shape information of thermal objects without privacy issues that enable trustworthy computer vision applications. The dataset consists of 1770 thermal images with high-quality annotation collected from an indoor room with around 15°C. We implemented a privacy-preserving human detection method and trained a multiple object detection (MOD) model based on the dataset. The human detection method reaches 90.3% accuracy. On the other hand, the MOD model achieved 56.8% mean average precision (mAP). Researchers can implement interesting applications based on our dataset, for example, privacy-preserving people counting systems, occupancy estimation systems for smart buildings, and social distance detectors.

Baidu runs the largest commercial web search engine in China, serving hundreds of millions of online users every day in response to a great variety of queries. In order to build a high-efficiency sponsored search engine, we used to adopt a three-layer funnel-shaped structure to screen and sort hundreds of ads from billions of ad candidates subject to the requirement of low response latency and the restraints of computing resources. Given a user query, the top matching layer is responsible for providing semantically relevant ad candidates to the next layer, while the ranking layer at the bottom concerns more about business indicators (e.g., CPM, ROI, etc.) of those ads. The clear separation between the matching and ranking objectives results in a lower commercial return. The Mobius project has been established to address this serious issue. It is our first attempt to train the matching layer to consider CPM as an additional optimization objective besides the query-ad relevance, via directly predicting CTR (click-through rate) from billions of query-ad pairs. Specifically, this paper will elaborate on how we adopt active learning to overcome the insufficiency of click history at the matching layer when training our neural click networks offline, and how we use the SOTA ANN search technique for retrieving ads more efficiently (Here "ANN'' stands for approximate nearest neighbor search). We contribute the solutions to Mobius-V1 as the first version of our next generation query-ad matching system.

Beauty product retrieval is a challenging task due to the severe image variation issue in real-world scenes. In this work, to mitigate the data variation problem, we contribute a background-agnostic feature extractor, which is trained by a self-supervised salient object detection method. In particular, we first propose a foreground augmentation technique to acquire the augmentation image with its foreground mask. Next, a feature extractor with an attention pooling layer is proposed to learn background-agnostic representations by performing the salient object detection in a self-supervised manner. Finally, we ensemble the background-agnostic features of multiple models to perform the beauty product retrieval. Extensive experimental results have demonstrated the superiority of our proposed framework.