Abstract Human motion generation is a key task in computer graphics. While various conditioning signals such as text, action class, or audio have been used to harness the generation process, most existing methods neglect the case where a specific body is desired to perform the motion. Additionally, they rely on skeleton‐based pose representations, necessitating additional steps to produce renderable meshes of the intended body shape. Given that human motion involves a complex interplay of bones, joints, and muscles, focusing solely on the skeleton during generation neglects the rich information carried by muscles and soft tissues, as well as their influence on movement, ultimately limiting the variability and precision of the generated motions. In this paper, we introduce Shape‐conditioned Motion Diffusion model (SMD), which enables the generation of human motion directly in the form of a mesh sequence, conditioned on both a text prompt and a body mesh. To fully exploit the mesh representation while minimizing resource costs, we employ spectral representation using the graph Laplacian to encode body meshes into the learning process. Unlike retargeting methods, our model does not require source motion data and generates a variety of desired semantic motions that is inherently tailored to the given identity shape. Extensive experimental evaluations show that the SMD model not only maintains the body shape consistently with the conditioning input across motion frames but also achieves competitive performance in text‐to‐motion and action‐to‐motion tasks compared to state‐of‐the‐art methods.

ABSTRACT The Diffusion Probabilistic Model (DM) has emerged as a powerful generative model in the field of image synthesis, capable of producing high‐quality and realistic images. However, training DM requires a large and diverse dataset, which can be challenging to obtain. This limitation weakens the model's generalisation and robustness when training data is limited. To address this issue, EDG‐CDM, an innovative encoder‐guided conditional diffusion model was proposed for image synthesis with limited data. Firstly, the authors pre‐train the encoder by introducing noise to capture the distribution of image features and generate the condition vector through contrastive learning and KL divergence. Next, the encoder undergoes further training with classification to integrate image class information, providing more favourable and versatile conditions for the diffusion model. Subsequently, the encoder is connected to the diffusion model, which is trained using all available data with encoder‐provided conditions. Finally, the authors evaluate EDG‐CDM on various public datasets with limited data, conducting extensive experiments and comparing our results with state‐of‐the‐art methods using metrics such as Fréchet Inception Distance and Inception Score. Our experiments demonstrate that EDG‐CDM outperforms existing models by consistently achieving the lowest FID scores and the highest IS scores, highlighting its effectiveness in generating high‐quality and diverse images with limited training data. These results underscore the significance of EDG‐CDM in advancing image synthesis techniques under data‐constrained scenarios.

Hyperspectral image denoising is crucial for accurate extraction of spectral information. However, current convolutional neural network (CNN)-based methods have inherent limitations, while Transformer- based methods suffer from high computational complexity when processing global contextual information. To address this problem, we designed a hybrid Mamba-CNN context interaction module and constructed a U-shaped hierarchical encoder-decoder network (MUNet). The network takes the pixel-scale as input to maximize the preservation of image information and employs state-space model (SSM)-based Mamba blocks to efficiently capture global semantic information, while using convolution to extract local features. This enhances the modeling of global and local features for better denoising. Extensive experiments on synthetic and real hyperspectral image (HSI) datasets showed that the proposed MUNet achieves better performance than other state-of-the-art techniques.

Pedestrian intrusion detection is crucial to ensuring safe railway operation. Current pedestrian detection algorithms lack consideration for real-world railway scenarios, such as the reflective properties of screen doors and train windows, may mistakenly trigger pedestrian intrusion alerts. Scale variability and pedestrian overlap often lead to detection inaccuracy, making them inadequate for addressing the specific requirements of railway perimeter security. This letter introduces an innovative pedestrian detection algorithm that incorporates Mirror Translation Attention (MTA) and Feature Pooling Enhancement (FPE). MTA, including mirror flipping and offsetting the feature mapping, could significantly mitigate missed detection caused by reflective surfaces. Additionally, we introduce sparsity to the inputs of the self-attention, which significantly enhancing the model's inference speed. A multi-scale approach is adopted to accommodate the diversity in pedestrian sizes, while the FPE addresses occlusion issues across various scales. Compared to the advanced YOLOv8 model, the proposed method improves AP50 by 1.6% to 92.11% and reduces model parameters by 63.55% in our self-built railway pedestrian intrusion dataset.

Remote sensing images are different from ordinary images in that they have higher resolution, contain information of a larger area, and are characterized by strip-like objects in many scenes. The traditional Transformer model based on the moving window to calculate the attention is difficult to obtain the overall features when extracting the features of strip-shaped objects and is easily interfered by the surrounding features. To address this problem, this paper innovatively designs an axial Transformer module and constructs a U-shaped hierarchical encoder-decoder structure network (ATUNet). The network improves its ability to extract global features and resist interference from irrelevant features through the axial attention mechanism. We synthesize multiple test sets with noise levels for experiments using three datasets, NWPU-RESISC45, UCMerced_LandUse, and OPTIMAL-31. The experiments show that our network has good resistance to high noise and generalization ability.

Human motion tracking is a fundamental building block for various applications including computer animation, human-computer interaction, healthcare, and so on. To reduce the burden of wearing multiple sensors, human motion prediction from sparse sensor inputs has become a hot topic in human motion tracking. However, such predictions are non-trivial as (i) the widely adopted data-driven approaches can easily collapse to average poses, and (ii) the predicted motions contain unnatural jitters. In this work, we address the aforementioned issues by proposing a novel framework which can accurately predict the human joint moving angles from the signals of only four flexible sensors, thereby achieving the tracking of human joints in multi-degrees of freedom. Specifically, we mitigate the collapse to average poses by implementing the model with a Bi-LSTM neural network that makes full use of short-time sequence information; we reduce jitters by adding a median pooling layer to the network, which smooths consecutive motions. Although being bio-compatible and ideal for improving the wearing experience, the flexible sensors are prone to aging which increases prediction errors. Observing that the aging of flexible sensors usually results in drifts of their resistance ranges, we further propose a novel dynamic calibration technique to rescale sensor ranges, which further improves the prediction accuracy. Experimental results show that our method achieves a low and stable tracking error of 4.51 degrees across different motion types with only four sensors.

Abstract Nowadays, virtual reality (VR) is becoming an important technique for various educational subjects. However, Chinese calligraphy, as a unique artistic form, remains under-explored in terms of learning in a VR configuration. This deficiency is largely due to the challenge to render delicate haptic feedback of pen and brush during the process of writing. To achieve the purpose of haptic rendering, existing works mostly use the professional device (e.g. Phantom), which is expensive and not accessible to common users. Our work presents a novel yet simple approach to render haptic feedback for Chinese calligraphy in VR by using soft and deformable sponge as the medium between the handheld controller and writing surface. We compared three different feedback configurations using on-device vibration and sponge-enabled haptic feedback against the baseline configuration with no force feedback. Based on both the qualitative and quantitative results from user studies, we found that sponge-based haptic feedback not only provided a comfort experience of interactive virtual writing but also accelerated the learning performance of novices. Our approach is low cost, scalable and produces realistic user experience, which offers an alternative solution for future development of training systems for virtual Chinese calligraphy.

Butterflies are not only ubiquitous around the world but are also widely known for inspiring thrill resonance, with their elegant and peculiar flights. However, realistically modeling and simulating butterfly flights—in particular, for real-time graphics and animation applications—remains an under-explored problem. In this article, we propose an efficient and practical model to simulate butterfly flights. We first model a butterfly with parametric maneuvering functions, including wing-abdomen interaction. Then, we simulate dynamic maneuvering control of the butterfly through our force-based model, which includes both the aerodynamics force and the vortex force. Through many simulation experiments and comparisons, we demonstrate that our method can efficiently simulate realistic butterfly flight motions in various real-world settings.

Abstract Fluid‐like motion and linear wave propagation behavior will emerge when we impose boundary constraints and polarized conditions on crowds. To this end, we present a Lagrangian hydrodynamics method to simulate the fluid‐like motion of crowd and a triggering approach to generate the linear stop‐and‐go wave behavior. Specifically, we impose a self‐propulsion force on the leading agents of the crowd to push the crowd to move forward and introduce a Smoothed Particle Hydrodynamics‐based model to simulate the dynamics of dense crowds. Besides, we present a motion signal propagation approach to trigger the rest of the crowd so that they respond to the immediate leaders linearly, which can lead to the linear stop‐and‐go wave effect of the fluid‐like motion for the crowd. Our experiments demonstrate that our model can simulate large‐scale dense crowds with linear wave propagation.

We present a system for designing indoor scenes with convertible furniture layouts. Such layouts are useful for scenarios where an indoor scene has multiple purposes and requires layout conversion, such as merging multiple small furniture objects into a larger one or changing the locus of the furniture. We aim at planning the motion for the convertible layouts of a scene with the most efficient conversion process. To achieve this, our system first establishes object-level correspondences between the layout of a given source and that of a reference to compute a target layout, where the objects are re-arranged in the source layout with respect to the reference layout. After that, our system initializes the movement paths of objects between the source and target layouts based on various mechanical constraints. A joint space-time optimization is then performed to program a control stream of object translations, rotations, and stops, under which the movements of all objects are efficient and the potential object collisions are avoided. We demonstrate the effectiveness of our system through various design examples of multi-purpose, indoor scenes with convertible layouts.

Despite numerous progresses in the past decades, 3D shape acquisition techniques remain a threshold for various 3D face based applications. Moreover, advanced 2D data generative models based on the deep networks may not be directly applicable for 3D objects. In this work, we propose a geometry sampling approach to bridge the gap between unstructured 3D face models and the powerful deep networks towards an unsupervised 3D face generative model. Specifically, we devise a geometry sampling approach to obtain a structured representation of 3D faces, which enable us to adapt the 3D faces to the Deep Convolution Generative Adversarial Network (DCGAN) for 3D face generation. We have demonstrated the effectiveness of our generative model by producing a large variety of 3D faces with different facial expressions.

With the recent advances in data acquisition techniques, the compression of various dynamic mesh sequence data has become an important topic in the computer graphics community. In this article, we present a new spatio-temporal segmentation-based approach for the adaptive compression of the dynamic mesh sequences. Given an input dynamic mesh sequence, we first compute an initial temporal cut to obtain a small subsequence by detecting the temporal boundary of dynamic behavior. Then, we apply a two-stage vertex clustering on the resulting subsequence to classify the vertices into groups with optimal intra-affinities. After that, we design a temporal segmentation step based on the variations of the principal components within each vertex group prior to performing a PCA-based compression. Furthermore, we apply an extra step on the lossless compression of the PCA bases and coefficients to gain more storage saving. Our approach can adaptively determine the temporal and spatial segmentation boundaries to exploit both temporal and spatial redundancies. We have conducted extensive experiments on different types of 3D mesh animations with various segmentation configurations. Our comparative studies show the advantages of our approach for the compression of 3D mesh animations.

With the recent advances of data acquisition techniques, the compression of various 3D mesh animation data has become an important topic in computer graphics community. In this paper, we present a new spatio-temporal segmentation-based approach for the compression of 3D mesh animations. Given an input mesh sequence, we first compute an initial temporal cut to obtain a small subsequence by detecting the temporal boundary of dynamic behavior. Then, we apply a two-stage vertex clustering on the resulting subsequence to classify the vertices into groups with optimal intra-affinities. After that, we design a temporal segmentation step based on the variations of the principle components within each vertex group prior to performing a PCA-based compression. Our approach can adaptively determine the temporal and spatial segmentation boundaries in order to exploit both temporal and spatial redundancies. We have conducted many experiments on different types of 3D mesh animations with various segmentation configurations. Our comparative studies show the competitive performance of our approach for the compression of 3D mesh animations.

Abstract During the past decades, high‐fidelity realistic simulations of various flying insects exhibiting collective behavior have been broadly used in entertainment industries and virtual reality applications. However, due to the intrinsic complexity and high computational cost, shape constrained simulation of collective behaviors remains a challenging topic. In this paper, we present a robust multi‐agent model for large‐scale controllable shape constrained simulation of flying insects. Specifically, we design an internal force model to biologically mimic an individual insect. We also propose an external force model based on a trade‐off mechanic to guide the insects smoothly deforming into a target shape. Our experimental results and comparative studies show our method is able to simulate realistic and dynamic flying insects with various user‐specified shape constraints.

With the rapid growing advancement of animation technologies, 3D animated meshes are becoming one of the major data in the industry such as virtual reality. However, treating the animated mesh data efficiently remains a challenging task due to its large scale and limited feature descriptors. In this paper, we present an evolutionary signature for animated meshes based on tempo-spatial segmentation. In specific, we first conduct temporal segmentation to a given animated meshes with sub-motions, then apply spatial segmentation within each temporal segment, and intersect spatial segmentation result for over segmentation. Thirdly, we represent the segmentation results into graphs. Finally, we devise an edge evolution matrix based on the dynamic behaviour of each edge for the evolutionary signature of the input animated mesh. Our experimental results on similarity measurement by using the proposed signature reflect the effectiveness of our method.

With the rapid development of the display technologies, 3D shape data is becoming another important media kind. However, most of the existing 3D shape acquisition methods are either expensive or expertise-dependent. In this paper, we present an image representation for the 3D faces to bridge the gap between the feature-lacking 3D shapes and the powerful deep neural network learning tools. To achieve this, with the training set, we first extract the radial curves for each 3D face, and reform the curves into an image matrix, which enable to apply the classical Generative Adversarial Network model for the image synthesis. Finally, we propose a refining process to transform the output images into 3D synthetic faces. Our experimental results demonstrate the capability of our method which can correctly reflect the affinities among the different facial expressions and can generate the 3D faces.

Summary Multipath TCP (MPTCP) is a promising transport protocol that allows a multihomed device to simultaneously use multiple network interfaces to send application data over multiple paths. However, although applying MPTCP to data delivery introduces many and attractive benefits, the MPTCP is vulnerable to network attacks. When a path within the MPTCP connection suffers from some types of attacks (eg, a denial‐of‐service attack) and becomes underperforming, it will undoubtedly cause transmission interruption in the stable paths and thus degrade the application‐level performance. Unfortunately, the MPTCP path management mechanism is very simple and cannot timely prevent the usage of underperforming paths in multipath transmission. In this paper, we introduce a new “potentially underperforming” (PU) concept to MPTCP and propose a novel PU‐aware path usage management mechanism ((PU)2M2) for MPTCP aiming to (1) detect and declare an underperforming path and prevent the usage of underperforming paths in multipath transmission, (2) provide a finite‐state‐machine model to change per‐path's state accordingly and effectively manage multiple paths for data transmission, and (3) alleviate the packet reordering problem and make MPTCP avoid throughput performance degradation during network underperforming. We demonstrate the benefits of applying (PU)2M2 to MPTCP.

One major concern when applying Multipath TCP (MPTCP) to the real-time multimedia applications is related to MPTCP's fully-reliable and fully-ordered service nature, which will inevitably degrade users' Quality of Experience (QoE) for multimedia streaming services in a heterogeneous wireless network environment because asymmetric wireless links are commonly with different transmission characteristics and sensitive to variations. In this paper, we first discuss the design considerations of partially reliable-MPTCP associated with the real-time constraint of multimedia streaming. Then we propose a context-aware QoE-oriented MPTCP Partial Reliability extension (PR-MPTCP+) for providing partially reliable multimedia streaming service to an upper layer protocol. Finally, we evaluate the proposed PR-MPTCP+solution using a wide range of multimedia quality metrics.

One major concern of concurrent multipath transfer (CMT) in multi-homed Stream Control Transport Protocol (SCTP)-based heterogeneous wireless networks is that the utilization of different paths with diverse QoS-related networking parameters may cause packet reordering and buffer blocking. Although many efforts have been devoted to addressing the packet reordering issue, their sender-dependent-only scheduler does not consider balancing overhead and sharing load between the SCTP sender and receiver. This paper proposes a novel Receiver-driven Multipath Data Scheduling strategy for CMT (CMT-RMDS) necessitating the following aims: (1) alleviating the packet reordering problem, (2) improving the CMT performance, and (3) balancing overhead and sharing load between the sender and receiver. Simulation results show that the proposed CMT-RMDS solution outperforms the existing CMT solutions in terms of data delivery performance in heterogeneous wireless networks.

ABSTRACT We describe a compression method for three‐dimensional animation sequences that has notable advantages over existing techniques. We first aggregate the frame data by similarity and reorganize them into clusters, which results in the sequence split into several motion fragments of varying lengths. To minimize the number of clusters and obtain optimal clustering, we perform frame alignment, which eliminates the “global” rigid transformation from each frame data and use only “pose” when evaluating the similarity between frames. We then apply principal component analysis for each cluster, from which we get coordinates of corresponding frames in a reduced dimension. Because similar frames are considered, the number of coefficients required for each frame becomes smaller; thus, we obtain better dimension reduction for a given reconstruction error. Further, we perform intracluster compression based on linear coding. Because every motion fragment presents similar frames, conventional linear predictive coding can be replaced by key frame‐based linear coding to achieve minimal reconstruction error. Results show that our method can obtain a high compression ratio, with a limited reconstruction error. Copyright © 2013 John Wiley & Sons, Ltd.

A long standing research goal is to create robots capable of interacting with humans in dynamic environments. To realise this a robot needs to understand and interpret the underlying meaning and intentions of a human action through a model of its sensory data. The visual domain provides a rich description of the environment and data is readily available in most system through inexpensive cameras. However, such data is very high-dimensional and extremely redundant making modeling challenging.