With the diversification of human-object interaction (HOI) applications and the success of capturing human meshes, HOI reconstruction has gained widespread attention. Existing mainstream HOI reconstruction methods often rely on explicitly modeling interactions between humans and objects. However, such a way leads to a natural conflict between 3D mesh reconstruction, which emphasizes global structure, and fine-grained contact reconstruction, which focuses on local details. To address the limitations of explicit modeling, we propose the End-to-End HOI Reconstruction Transformer with Graph-based Encoding (HOI-TG). It implicitly learns the interaction between humans and objects by leveraging self-attention mechanisms. Within the transformer architecture, we devise graph residual blocks to aggregate the topology among vertices of different spatial structures. This dual focus effectively balances global and local representations. Without bells and whistles, HOI-TG achieves state-of-the-art performance on BEHAVE and InterCap datasets. Particularly on the challenging InterCap dataset, our method improves the reconstruction results for human and object meshes by 8.9% and 8.6%, respectively.

Demystifying complex human-ground interactions is essential for accurate and realistic 3D human motion reconstruction from RGB videos, as it ensures consistency between the humans and the ground plane. Prior methods have modeled human-ground interactions either implicitly or in a sparse manner, often resulting in unrealistic and incorrect motions when faced with noise and uncertainty. In contrast, our approach explicitly represents these interactions in a dense and continuous manner. To this end, we propose a novel Ground-aware Motion Model for 3D Human Motion Reconstruction, named GraMMaR, which jointly learns the distribution of transitions in both pose and interaction between every joint and ground plane at each time step of a motion sequence. It is trained to explicitly promote consistency between the motion and distance change towards the ground. After training, we establish a joint optimization strategy that utilizes GraMMaR as a dual-prior, regularizing the optimization towards the space of plausible ground-aware motions. This leads to realistic and coherent motion reconstruction, irrespective of the assumed or learned ground plane. Through extensive evaluation on the AMASS and AIST++ datasets, our model demonstrates good generalization and discriminating abilities in challenging cases including complex and ambiguous human-ground interactions. The code will be available at https://github.com/xymsh/GraMMaR.

Crowd counting, which counts or estimates the number of people within a region, is critical in many applications, such as guided tours and disaster rescue. Several RF-based contact-free crowd counting techniques have been proposed in recent years, including WiFi, RFID, and millimeter wave radar. While promising in many aspects, one key limitation of current techniques is the small sensing range. However, many applications of crowd counting do require long-range sensing capability. In this work, we propose LoCount to significantly increase the sensing range of crowd counting using LoRa, which is a new wireless technology for long range communications among IoT devices. In particular, to solve the system performance degradation caused by different environments, we try to remove the components representing surrounding environments from the signal and use adversarial domain adaptation to extract environment-independent features. Considering that we may have multiple different source domains, for the target domain data, we comprehensively think over its similarity to each source domain and the prediction results to get the final result. We test LoCount in multiple large-scale scenes, and the results show that LoCount can achieve an average accuracy of 97.0% in the target domain without labeled data.

Recently, there has been an increasing concern about the privacy issue raised by identifiable information in machine learning. However, previous portrait matting methods were all based on identifiable images. To fill the gap, we present P3M-10k, which is the first large-scale anonymized benchmark for Privacy-Preserving Portrait Matting (P3M). P3M-10k consists of 10,421 high resolution face-blurred portrait images along with high-quality alpha mattes, which enables us to systematically evaluate both trimap-free and trimap-based matting methods and obtain some useful findings about model generalization ability under the privacy preserving training (PPT) setting. We also present a unified matting model dubbed P3M-Net that is compatible with both CNN and transformer backbones. To further mitigate the cross-domain performance gap issue under the PPT setting, we devise a simple yet effective Copy and Paste strategy (P3M-CP), which borrows facial information from public celebrity images and directs the network to reacquire the face context at both data and feature level. Extensive experiments on P3M-10k and public benchmarks demonstrate the superiority of P3M-Net over state-of-the-art methods and the effectiveness of P3M-CP in improving the cross-domain generalization ability, implying a great significance of P3M for future research and real-world applications. The dataset, code and models are available here (https://github.com/ViTAE-Transformer/P3M-Net).

Recently, there has been an increasing concern about the privacy issue raised by using personally identifiable information in machine learning. However, previous portrait matting methods were all based on identifiable portrait images. To fill the gap, we present P3M-10k in this paper, which is the first large-scale anonymized benchmark for Privacy-Preserving Portrait Matting. P3M-10k consists of 10,000 high-resolution face-blurred portrait images along with high-quality alpha mattes. We systematically evaluate both trimap-free and trimap-based matting methods on P3M-10k and find that existing matting methods show different generalization capabilities when following the Privacy-Preserving Training (PPT) setting, i.e., training on face-blurred images and testing on arbitrary images. To devise a better trimap-free portrait matting model, we propose P3M-Net, which leverages the power of a unified framework for both semantic perception and detail matting, and specifically emphasizes the interaction between them and the encoder to facilitate the matting process. Extensive experiments on P3M-10k demonstrate that P3M-Net outperforms the state-of-the-art methods in terms of both objective metrics and subjective visual quality. Besides, it shows good generalization capacity under the PPT setting, confirming the value of P3M-10k for facilitating future research and enabling potential real-world applications. The source code and dataset are available at https://github.com/JizhiziLi/P3M.

Digital twin (DT) systems that replicate the physical world digitally are powerful tools for monitoring physical systems and evaluating algorithms, but current DT systems are commonly not applicable for robotic deployment and investigation. Meanwhile, current 3-D simulation-based robotic platforms do not model the dynamics of the physical world on-the-fly as done in DT systems, limiting their potential for the development of robotics in challenging environments. To tackle this issue, we propose the first robot-centered smart DT framework, namely, Terra, to facilitate the deployment of robots in challenging environments. The proposed Terra framework introduces a comprehensive DT representation to encode the useful real-time dynamics of both the physical world and the robot agent deployed therein. A multiview multimodality perception module is further devised for Terra to obtain high-level semantics and deliver a precise description of the current status of the environment and the robot agent. By mapping the perceived results to the virtual replica of the physical environment, Terra actively updates the action policy and sends it back to the agent, forming an integral and real-time information feedback loop. In practice, to help demonstrate the effectiveness and feasibility of the proposed framework, we deliberately set up a challenging unordered physical environment with many obstacles and a very simple robot aiming to fulfill a navigation task. Empirical results show that the proposed Terra framework successfully facilitates the robot to accomplish the task without causing hazards.

Network Virtualization (NV) techniques allow multiple virtual network requests to beneficially share resources on the same substrate network, such as node computational resources and link bandwidth. As the most famous family member of NV techniques, virtual network embedding is capable of efficiently allocating the limited network resources to the users on the same substrate network. However, traditional heuristic virtual network embedding algorithms generally follow a static operating mechanism, which cannot adapt well to the dynamic network structures and environments, resulting in inferior nodes ranking and embedding strategies. Some reinforcement learning aided embedding algorithms have been conceived to dynamically update the decision-making strategies, while the node embedding of the same request is discretized and its continuity is ignored. To address this problem, a Continuous-Decision virtual network embedding scheme relying on Reinforcement Learning (CDRL) is proposed in our paper, which regards the node embedding of the same request as a time-series problem formulated by the classic seq2seq model. Moreover, two traditional heuristic embedding algorithms as well as the classic reinforcement learning aided embedding algorithm are used for benchmarking our prpposed CDRL algorithm. Finally, simulation results show that our proposed algorithm is superior to the other three algorithms in terms of long-term average revenue, revenue to cost and acceptance ratio.

Matrix Factorization based methods, e.g., the Concept Factorization (CF) and Nonnegative Matrix Factorization (NMF), have been proved to be efficient and effective for data clustering tasks. In recent years, various graph extensions of CF and NMF have been proposed to explore intrinsic geometrical structure of data for the purpose of better clustering performance. However, many methods build the affinity matrix used in the manifold structure directly based on the input data. Therefore, the clustering results are highly sensitive to the input data. To further improve the clustering performance, we propose a novel manifold concept factorization model with adaptive neighbor structure to learn a better affinity matrix and clustering indicator matrix at the same time. Technically, the proposed model constructs the affinity matrix by assigning the adaptive and optimal neighbors to each point based on the local distance of the learned new representation of the original data with itself as a dictionary. Our experimental results present superior performance over the state-of-the-art alternatives on numerous datasets.

A robust semi-supervised concept factorization (RSSCF) method is proposed in this paper, which not only makes good use of the available label information, but also addresses noise and extracts meaningful information simultaneously. In the proposed method, a constraint matrix is embedded into the basic concept factorization model to guarantee data with the same label share the same new representation. We utilize L2,1-norm on both loss function and regularization, thus this new model is not sensitive to outliers and the L2,1-norm regularization helps select useful information with joint sparsity. An efficient and elegant iterative updating scheme is also introduced with convergence and correctness analysis. Simulations are given to illustrate the effectiveness of our proposed method.