Multimodal 3D segmentation involves a significant number of 3D convolution operations, which requires substantial computational resources and high-performance computing devices in MRI multimodal brain tumor segmentation. The key challenge in multimodal 3D segmentation is how to minimize network computational load while maintaining high accuracy. To address the issue, a novel lightweight parameter aggregation network (SuperLightNet) is proposed to realize the efficient encoder and decoder for the high accurate and low computation. A random multiview drop encoder is designed to learn the spatial structure of multimodal images through a random multi-view approach for solving the high computational time complexity that has arisen in recent years with methods relying on transformers and Mamba. A learnable residual skip decoder is designed to incorporate learnable residual and group skip weights for addressing the reduced computational efficiency caused by the use of overly heavy convolution and deconvolution decoders. Experimental results demonstrate that the proposed method achieves a leading reduction in parameter count by 95.59%, the 96.78% improvement in computational efficiency, the 96.86% enhancement in memory access performance, and the average performance gain of 0.21% on the BraTS2019 and BraTS2021 datasets in comparison with the state-of-the-art methods. Code is available at https://github.com/WTU-MIS-Laboratory/SuperLightNet.

Smart wearable systems can monitor users’ physiological data in real time, detect anomalies promptly through risk recognition technologies, provide early warnings, and assist users in taking preventive measures. However, single modal information is difficult to accurately recognize the behavioral state, expression state, and environmental conditions. Furthermore, multimodal data are often affected by noise and interference, complicating the accurate identification of risky behaviors. To address these challenges, we propose a smart wearable system based on the dual-frequency enhancement network (DFENet): 1) the multimodal sensor system is designed to combine behavioral recognition, expression recognition, and environmental recognition for comprehensive monitoring and recognition of multidimensional risk factors in complex scenarios; 2) the DFENet is proposed to overcome challenges in feature extraction and accurate classification in complex environments; and 3) the behavioral recognition dataset and the expression recognition dataset are built to verify the effectiveness of the designed smart wearable system. Experimental results indicate that the proposed system can real-time achieve risk recognition across physical activity, expression state, and environmental conditions, and the proposed DFENet achieves excellent performance in accuracy, parameters, and floating-point operations (FLOPs) metrics on the three datasets. The algorithm and datasets can be downloaded athttps://github.com/wtu1020/Multimodal-Wearable.

Brain-computer interface (BCI) offers a direct communication and control channel between the human brain and external devices, presenting new pathways for individuals with physical disabilities to operate robotic arms for complex tasks. However, achieving multiobject grasping tasks under low signal-to-noise ratio (SNR) consumer-grade EEG signals is a significant challenge due to the lack of robust decoding algorithms and precise visual tracking methods. This article proposes, ArmBCIsys, an integrated robotic arm system that combines a novel dual-branch frequency-enhanced network (DBFENet) to robustly decode EEG signals under noisy conditions with the high-precision vision-guided grasping module. The proposed DBFENet designs the scaling temporal convolution block (STCB) to extract multiscale spatiotemporal features from the time domain, while the designed DropScale projected Transformer (DSPT) utilizes discrete cosine transform (DCT) to capture key frequency-domain features, significantly improving decoding robustness. We fine-tune the masked-attention mask Transformer (Mask2Former) model on the Jacquard dataset and incorporate the multiframe centroid-intersection over union (IoU) tracking algorithm to build visual grasp segmenter (VisGraspSeg), enabling reliable segmentation and dynamic tracking for diverse daily objects. Experimental validations on both self-built code-modulated visual evoked potential (c-VEP) dataset (1344 samples) and two public c-VEP datasets demonstrate that DBFENet achieves the state-of-the-art recognition performance, and the system integrates the DBFENet and proposed vision-guided module and ensures stable multiobject selecting and automatic object grasping in dynamic environments, extending promising applications in healthcare robotics, assistive technology, and industrial automation. The self-built dataset has been made publicly accessible at https://github.com/wtu1020/ ArmBCIsys-Self-built-cVEP-Dataset.

Dynamic garment simulation plays a crucial role in applications such as virtual try-on and film production. Existing simulation methods face challenges including high computational time, video frame jitter, and limited garment styles. Therefore, we propose SmPhy, a method that takes real videos as input. To alleviate frame jitter in video generation, we employ a temporal perception network for motion smoothing. The temporal physics garment module introduces temporal dependency, utilizing the garment information output from the current frame as input for the next frame, and provides reliable physical constraints to enhance garment deformation effects. Qualitative and quantitative experiments demonstrate that SmPhy reduces time costs and successfully simulates 3D clothing animations closely resembling real-world behaviors. Access links to supporting materials are as follows: https://drive.google.com/file/d/1BIbSI4mT4YgCVFRorszGW9pxbPZo40SH/view?usp=drive_link

Intelligent wearable systems have been widely used in health monitoring, motion tracking, and engineering safety. However, the single function of current wearable systems cannot satisfy the requirements of complex scenarios, and the wearable systems cannot establish a relationship with the virtual 3D visualization platform. To address these issues, this paper proposes a novel intelligent wearable system with motion and emotion recognition. Multiple sensors are integrated into the system to collect motion and emotion information. In order to achieve accurate classification and recognition of multiple sensor information, we propose a novel human action recognition network called the three-branch spatial-temporal feature extraction network (TB-SFENet), which can obtain more robust features and achieve an accuracy of 97.04% on the UCI-HAR dataset and 92.68% on the UniMiB SHAR dataset. To establish the relationship between the real entity and virtual space, we use digital twin (DT) technology to establish the 3D display DT platform. The platform enables real-time information interaction, such as activity, emotion, location, and monitoring information. Additionally, we establish the TGAM electroencephalogram emotion classification (TEEC) dataset, which contains 120,000 pieces of data, for the proposed system. Experimental results indicate that the proposed system realizes virtual reality information interaction between the personal digital human and actual person based on the intelligent wearable system, which has great potential for applications in intelligent healthcare, virtual reality, and other fields.

Abstract Wearable human action recognition (HAR) has practical applications in daily life. However, traditional HAR methods solely focus on identifying user movements, lacking interactivity and user engagement. This paper proposes a novel immersive HAR method called MovPosVR. Virtual reality (VR) technology is employed to create realistic scenes and enhance the user experience. To improve the accuracy of user action recognition in immersive HAR, a multi‐scale spatio‐temporal attention network (MSSTANet) is proposed. The network combines the convolutional residual squeeze and excitation (CRSE) module with the multi‐branch convolution and long short‐term memory (MCLSTM) module to extract spatio‐temporal features and automatically select relevant features from action signals. Additionally, a multi‐head attention with shared linear mechanism (MHASLM) module is designed to facilitate information interaction, further enhancing feature extraction and improving accuracy. The MSSTANet network achieves superior performance, with accuracy rates of 99.33% and 98.83% on the publicly available WISDM and PAMPA2 datasets, respectively, surpassing state‐of‐the‐art networks. Our method showcases the potential to display user actions and position information in a virtual world, enriching user experiences and interactions across diverse application scenarios.

Abstract Human activity recognition (HAR) has significant potential in virtual sports applications. However, current HAR networks often prioritize high accuracy at the expense of practical application requirements, resulting in networks with large parameter counts and computational complexity. This can pose challenges for real‐time and efficient recognition. This paper proposes a hybrid lightweight DSANet network designed to address the challenges of real‐time performance and algorithmic complexity. The network utilizes a multi‐scale depthwise separable convolutional (Multi‐scale DWCNN) module to extract spatial information and a multi‐layer Gated Recurrent Unit (Multi‐layer GRU) module for temporal feature extraction. It also incorporates an improved channel‐space attention module called RCSFA to enhance feature extraction capability. By leveraging channel, spatial, and temporal information, the network achieves a low number of parameters with high accuracy. Experimental evaluations on UCIHAR, WISDM, and PAMAP2 datasets demonstrate that the network not only reduces parameter counts but also achieves accuracy rates of 97.55%, 98.99%, and 98.67%, respectively, compared to state‐of‐the‐art networks. This research provides valuable insights for the virtual sports field and presents a novel network for real‐time activity recognition deployment in embedded devices.

Intelligent garments, a burgeoning class of wearable devices, have extensive applications in domains such as sports training and medical rehabilitation. Nonetheless, existing research in the smart wearables domain predominantly emphasizes sensor functionality and quantity, often skipping crucial aspects related to user experience and interaction. To address this gap, this study introduces a novel real-time 3D interactive system based on intelligent garments. The system utilizes lightweight sensor modules to collect human motion data and introduces a dual-stream fusion network based on pulsed neural units to classify and recognize human movements, thereby achieving real-time interaction between users and sensors. Additionally, the system in- corporates 3D human visualization functionality, which visualizes sensor data and recognizes human actions as 3D models in realtime, providing accurate and comprehensive visual feedback to help users better understand and analyze the details and features of human motion. This system has significant potential for applications in motion detection, medical monitoring, virtual reality, and other fields. The accurate classification of human actions con- tributes to the development of personalized training plans and injury prevention strategies. This study has substantial implications in the domains of intelligent garments, human motion monitoring, and digital twin visualization. The advancement of this system is expected to propel the progress of wearable technology and foster a deeper comprehension of human motion.

In the fashion field, with the increasing diversity of clothing types and styles, accurate clothing classification becomes very important. However, the complex background and diverse styles of clothing images bring challenges to feature extraction. Classification based on texture features alone may focus too much on details and ignore the overall shape information, thus reducing the accuracy and stability of classification. In order to achieve fast and accurate clothing classification, this paper proposes a two-stream network structure clothing classification algorithm based on shape texture features and multi-feature fusion (TSFCloNet). Its main core is as follows: 1) using the two-stream network structure to extract texture and shape features from the input data set respectively; 2) in the shape feature extraction stream, the clothing shape acquisition module is first used to process the input clothing data set, and the obtained clothing shape data set is input into the ShapeNet feature extraction module to obtain shape feature information; 3) the FFCE (Feature Fusion Channel Enhancement) module is used to fuse the features obtained by the two branches of the structure respectively, and the DSAConv module is used to enhance feature extraction, and the final features are sent to the trained classifier to obtain the clothing style classification results. A large number of experimental results show that the proposed TSFCloNet network achieves higher classification accuracy when dealing with diverse and changeable fashion styles, significantly improving the performance of fashion image classification.

The goal of digital dress body animation is to produce the most realistic dress body animation possible. Although a method based on the same topology as the body can produce realistic results, it can only be applied to garments with the same topology as the body. Although the generalization-based approach can be extended to different types of garment templates, it still produces effects far from reality. We propose GSNet, a learning-based model that generates realistic garment animations and applies to garment types that do not match the body topology. We encode garment templates and body motions into latent space and use graph convolution to transfer body motion information to garment templates to drive garment motions. Our model considers temporal dependency and provides reliable physical constraints to make the generated animations more realistic. Qualitative and quantitative experiments show that our approach achieves state-of-the-art 3D garment animation performance.

Smart clothing is widely used for social safety, health monitoring, and sports monitoring. Current research focuses on the use of various materials or sensors to implement smart clothes with different functions, which implies that the functionality of smart clothing depends on the number of sensors used. For existing smart clothing systems, the greatest attention has been given to information processing algorithms and assembly of sensors, and the interaction between users and systems is ignored. To address this gap, this article considers a multifunctional smart clothing system constructed with several sensors. The smart clothing system proposed in this article mainly consists of a hardware module and a software module. Four types of sensors are incorporated into the hardware module to monitor the heart rate, blood oxygen saturation, body temperature, locating information, and activity states; the software module includes the 3-D model based on the user and the feedback system based on digital twin (DT) technology. The DT technology can map the fundamental states of users in terms of the monitoring indices from the hardware module, and give correspondent advice to users. This novel smart clothing system overcomes the lack of an interaction function in existing methods and introduces DT technology into smart wearable devices for the first time.

An image-based virtual try-on system transfers an in-shop garment to the corresponding garment region of a reference person, which has huge application potential and commercial value in online clothing shopping. Existing methods have difficulty preserving garment texture and body details because of rough garment alignment and imperfect detail-retention strategies. To address this problem, we propose a virtual try-on network based on semantic constraints and flow alignment. The key idea of the framework is as follows: 1) a global-local semantic predictor (GLSP) is proposed to generate a reasonable target semantic map, which clearly guides the correct alignment of the in-shop garment with the body and the generation of try-on result; and 2) a novel appearance flow-based garment alignment network (AFGAN) is proposed to align the in-shop garment with the body, which is important to preserve maximum garment detail and ensure natural and realistic warping; and 3) we propose a synthesis strategy to integrate the aligned garment and the human body to preserve maximum body detail for generating a realistic result and preventing cross-occlusion and pixel confusion between different body parts. Experiments on the existing benchmark dataset demonstrate that the proposed method achieves the best performance on qualitative and quantitative experiments among the state-of-the-art virtual try-on techniques.

With the growing trend of virtual try-on, multi-pose tasks attract researchers due to their higher commercial value. Prior methods lack an effective geometric deformation to maintain the original image details resulting in many details loss in the head and garment. To address this problem, we propose a new multi-pose virtual try-on network, which can fit a garment to the corresponding area of a person in arbitrary poses. First, the target pose’s body-semantic distribution is predicted by the target pose point. Second, the in-shop garment and human body are warped based on a human pose to solve the unnatural alignment and the lack of body details by the Deformation Module (DM). Finally, the human body in the given pose and garment is fine generated by the Filtering Synthesis Network (FSN). Compared to state-of-the-art methods with objective experiments on the MPV dataset, the proposed method achieves the best performance in metrics and the rich details in visual results.

3D virtual try-on receives widespread attention from scholars due to its great practical and commercial values. In prior methods, the fundamental problems lie in the limitations on texture retention during garment deformation and the lack of feature context capture during depth estimation. To address these problems, we propose a new 3D virtual try-on network via multi-scale characteristic capture (VTON-MC), which can produce an exact 3D model with the generated photo-realistic monocular image. The main processes are as follows: 1) predicting the human semantic-map and aligning the in-shop garment in the human pose using the appearance flow method, 2) synthesizing the human body and the warped garment to gain the image try-on result, and 3) estimating the human double-depth map of the image try-on result to reconstruct desired 3D try-on mesh by designed Depth Estimation Network (DEN). Extensive experiments on existing benchmark datasets demonstrate that VTON-MC outperforms state-of-the-art approaches efficiently.

Image-based virtual try-on aims to transfer a clothes onto a person while preserving both person's and cloth's attributes. However, the existing methods to realize this task require a target clothes, which cannot be obtained in most cases. To address this issue, we propose a novel user-friendly virtual try-on network (UF-VTON), which only requires a person image and an image of another person wearing a target clothes to generate a result of the person wearing the target clothes. Specifically, we adopt a knowledge distillation scheme to construct a new triple dataset for supervised learning, propose a new three-step pipeline (coarse synthesis, clothing alignment, and refinement synthesis) for try-on task, and utilize an end-to-end training strategy to further refine the results. In particular, we design a new synthesis network that includes both CNN blocks and swin-transformer blocks to capture global and local information and generate highly-realistic try-on images. Qualitative and quantitative experiments show that our method achieves the state-of-the-art virtual try-on performance.

During the process of pushing patch packets for Android application hotfix, the attacker can hijack and tamper with the dex file due to the lack of adding a digital signature, which leads to code injection with serious consequences. To address the above problems, an dynamic vulnerability detection system based on mitmproxy is primary proposed, which first utilizes mitmproxy to capture all the packets interacted between the client and the server while locating the dex file, then injects the test code into the dex and pushes it to the client for execution using a man-in-the-middle attack, and finally verifies through the log output by the application whether there is a code injection vulnerability. For 1000 applications in the application market, our system successfully detects 34 new unknown applications with dex injection, and the experimental results show that the system is effective in detecting real-world applications with vulnerabilities caused by hotfix.

During the process of pushing patch packets for Android application hotfix, the attacker can hijack and tamper with the dex file due to the lack of adding a digital signature, which leads to code injection with serious consequences. To address the above problems, an dynamic vulnerability detection system based on mitmproxy is primary proposed, which first utilizes mitmproxy to capture all the packets interacted between the client and the server while locating the dex file, then injects the test code into the dex and pushes it to the client for execution using a man-in-the-middle attack, and finally verifies through the log output by the application whether there is a code injection vulnerability. For 1000 applications in the application market, our system successfully detects 34 new unknown applications with dex injection, and the experimental results show that the system is effective in detecting real-world applications with vulnerabilities caused by hotfix.

Image-based virtual try-on systems have significant commercial value in online garment shopping. However, prior methods fail to appropriately handle details, so are defective in maintaining the original appearance of organizational items including arms, the neck, and in-shop garments. We propose a novel high fidelity virtual try-on network to generate realistic results. Specifically, a distributed pipeline is used for simultaneous generation of organizational items. First, the in-shop garment is warped using thin plate splines (TPS) to give a coarse shape reference, and then a corresponding target semantic map is generated, which can adaptively respond to the distribution of different items triggered by different garments. Second, organizational items are componentized separately using our novel semantic map-based image adjustment network (SMIAN) to avoid interference between body parts. Finally, all components are integrated to generate the overall result by SMIAN. A priori dual-modal information is incorporated in the tail layers of SMIAN to improve the convergence rate of the network. Experiments demonstrate that the proposed method can retain better details of condition information than current methods. Our method achieves convincing quantitative and qualitative results on existing benchmark datasets.

Image-based virtual try-on systems with the goal of transferring a target clothing item onto the corresponding region of a person have received great attention recently. However, it is still a challenge for the existing methods to generate photo-realistic try-on images while preserving non-target details(Fig. 1). To resolve this issue, we present a novel virtual try-on network, DP-VTON. First, a clothing warping module combines pixel transformation with feature transformation to transform the target clothing. Second, a semantic segmentation prediction module predicts a semantic segmentation map of the person wearing the target clothing. Third, an arm generation module generates arms of the reference image that will be changed after try-on. Finally, the warped clothing, semantic segmentation map, arms image and other non-target details (e.g. face, hair, bottom clothes) are fused together for try-on image synthesis. Extensive experiments demonstrate our system achieves the state-of-the-art virtual try-on performance both qualitatively and quantitatively.1

The image-based virtual try-on network transfers the target garment item to the corresponding region of the human body. Due to its commercial value in online garment shopping, it has attracted extensive attention from researchers. However, the previous virtual try-on methods are interfered heavily by garments in reference images, so they have defects in maintaining details of human upper limbs, neck, and given garment. Therefore, a novel High Fidelity Virtual Try-on Network via Semantic Adaptation (VTON-HF) is proposed to generate a result with better details. The main processes are as follows: 1) Thin Plate Spline (TPS) warps the target garment coarsely, 2) parsing network generates a target semantic map with the coarse warped garment, 3) our novel Semantic Map-based Image Adjustment Network (SMIAN) generates components separately to avoid interference between image parts with different semantics, 4) SMIAN fuses all components to generate the final result. VTON-HF can retain the maximum amount of detail in the reference garment than previous methods. Our novel architecture generates desired results by fusing separately generated components (garment, upper limb, and neck) and unchanging parts of the reference image. Moreover, our SMIAN incorporates a priori multimodal information in the tail layer, which effectively improves the convergence efficiency of the network. Our method achieves state-of-the-art quantitative results on IS, SSIM, PSNR, and FID using the VITON dataset. (see Fig. 1).

The pandemic of coronavirus disease 2019 (COVID-19) urgently calls for more sensitive molecular diagnosis to improve sensitivity of current viral nuclear acid detection. We have developed an anchor primer (AP)-based assay to improve viral RNA stability by bioinformatics identification of RNase-binding site of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) RNA and implementing AP dually targeting the N gene of SARS-CoV-2 RNA and RNase 1, 3, 6. The arbitrarily primed polymerase chain reaction (AP-PCR) improvement of viral RNA integrity was supported by (a) the AP increased resistance of the targeted gene (N gene) of SARS-CoV-2 RNA to RNase treatment; (b) the detection of SARS-CoV-2 RNA by AP-PCR with lower cycle threshold values (-2.7 cycles) compared to two commercially available assays; (c) improvement of the viral RNA stability of the ORF gene upon targeting of the N gene and RNase. Furthermore, the improved sensitivity by AP-PCR was demonstrated by detection of SARS-CoV-2 RNA in 70-80% of sputum, nasal, pharyngeal swabs and feces and 36% (4/11) of urine of the confirmed cases (n = 252), 7% convalescent cases (n = 54) and none of 300 negative cases. Lastly, AP-PCR analysis of 306 confirmed and convalescent cases revealed prolonged presence of viral loading for >20 days after the first positive diagnosis. Thus, the AP dually targeting SARS-CoV-2 RNA and RNase improves molecular detection by preserving SARS-CoV-2 RNA integrity and reveals the prolonged viral loading associated with older age and male gender in COVID-19 patients.

Data in a high-dimensional data space may reside in a low-dimensional manifold embedded within the high-dimensional space. Manifold learning discovers intrinsic manifold data structures to facilitate dimensionality reductions. We propose a novel manifold learning technique called fast [Formula: see text] selection for locally linear embedding or FSLLE, which judiciously chooses an appropriate number (i.e., parameter [Formula: see text]) of neighboring points where the local geometric properties are maintained by the locally linear embedding (LLE) criterion. To measure the spatial distribution of a group of neighboring points, FSLLE relies on relative variance and mean difference to form a spatial correlation index characterizing the neighbors’ data distribution. The goal of FSLLE is to quickly identify the optimal value of parameter [Formula: see text], which aims at minimizing the spatial correlation index. FSLLE optimizes parameter [Formula: see text] by making use of the spatial correlation index to discover intrinsic structures of a data point’s neighbors. After implementing FSLLE, we conduct extensive experiments to validate the correctness and evaluate the performance of FSLLE. Our experimental results show that FSLLE outperforms the existing solutions (i.e., LLE and ISOMAP) in manifold learning and dimension reduction. We apply FSLLE to face recognition in which FSLLE achieves higher accuracy than the state-of-the-art face recognition algorithms. FSLLE is superior to the face recognition algorithms, because FSLLE makes a good tradeoff between classification precision and performance.

In this paper, we present a thermal-aware file assignment technique called TIGER for reducing the cooling cost of storage clusters in data centers. We show that peak inlet temperatures of storage nodes depend on not only CPU utilization but also I/O activities, which rely on file assignments in a cluster. The TIGER scheme aims to lower peak inlet temperatures of storage clusters by dynamic thermal management through file placements. TIGER makes use of cross-interference coefficients to estimate the re-circulation of hot air from the outlets to the inlets of data nodes. TIGER first calculates the thresholds of disks in each data node based on its contribution to heat re-circulation in a data center. TIGER undertakes two steps to achieve high I/O performance while reducing cooling cost. First, TIGER assigns groups of files with similar service times to shorten I/O response times. Second, TIGER ensures that load imbalance does not exceed a specified threshold. We evaluate performance of TIGER in terms of both cooling energy conservation and response time of a storage cluster. Our results confirm that TIGER reduces cooling-power requirements for clusters by offering about 10 to 15 percent cooling-energy savings without significantly degrading I/O performance.

In this paper, we present thermal-aware file assignment technique called TIGER for reducing cooling cost of storage clusters in data centers. TIGER first calculates the thresholds of disks in each node based on its contribution to heat recirculation in a data center. Next, TIGER assigns files to data nodes according to calculated thresholds. We evaluated performance of TIGER in terms of both cooling energy conservation and response time of a storage cluster. Our results confirm that TIGER reduces cooling-power requirements for clusters by offering about 10 to 15 percent cooling energy savings without significantly degrading I/O performance.

This paper presents a novel Predictive Energy-Aware Management (PEAM) system that is able to reduce the energy costs of storage systems by appropriately selecting data transmission methods. In particular, we evaluate the energy costs of three methods (1. transfer data without archiving and compression, 2. archive and transfer data, 3. compress and transfer data) in preliminary experiments. According to the results, we observe that the energy consumption of data transmission greatly varies case by case. We cannot simply apply one method in all cases. Therefore, we design an energy prediction model that can estimate the total energy cost of data transmission by using particular transmission methods. Based on the model, our predictive energy-aware management system can automatically select the most energy efficient method for data transmission. Our experimental results show that our system performs better than simply selecting any one among the three methods for data transmission in terms of energy efficiency.

This paper analyzes the complexity of double disk fault tolerant storage scheme RAID5x under three types of states: normal, fault, and reconstruction. As a result, the space overhead of RAID5x is less than other code-mixing solution and is higher than XOR-based code solutions. Although RAID5x's mapping produces the computation overhead of the controller, it optimizes the data-accessing performance of the entire storage system. Its parity computing overhead of data-accessing is low like RAID5 at the normal state and almost keeps constant increase from the normal to the fault or between two reconstruction states.