We present SOPHY, a generative model for 3D physics-aware shape synthesis. Unlike existing 3D generative models that focus solely on static geometry or 4D models that produce physics-agnostic animations, our method jointly synthesizes shape, texture, and material properties related to physics-grounded dynamics, making the generated objects ready for simulations and interactive, dynamic environments. To train our model, we introduce a dataset of 3D objects annotated with detailed physical material attributes, along with an efficient pipeline for material annotation. Our method enables applications such as text-driven generation of interactive, physics-aware 3D objects and single-image reconstruction of physically plausible shapes. Furthermore, our experiments show that jointly modeling shape and material properties enhances the realism and fidelity of the generated shapes, improving performance on both generative geometry and physical plausibility. Project page: https://xjay18.github.io/SOPHY.

The advancement of generative artificial intelligence has led to the creation of more diverse and realistic fake facial images. This poses serious threats to personal privacy and can contribute to the spread of misinformation. Existing deepfake detection methods usually utilize prior knowledge about forged clues to design complex modules, achieving excellent performance in the intra-domain settings. However, their performance usually suffers from a significant decline in unseen forgery patterns. It is thus desirable to develop a generalized deepfake detection method using a neat network structure. In this paper, we propose a simple yet efficient framework to transfer a powerful large-scale vision model like ViT to the downstream deepfake detection task, namely the generalized deepfake detection framework (GenDF). Concretely, we first propose a deepfake-specific representation learning (DSRL) scheme to learn different discontinuity patterns across patches inside a fake facial image and continuity between patches within a real counterpart in a low-dimensional space. To further alleviate the distribution mismatch between generic real images and human facial images consisting of both real and fake, we introduce a feature space redistribution (FSR) scheme to separately optimize the distributions of real and fake feature space, enabling the model to learn more distinctive representations. Furthermore, to enhance the generalization performance on unseen forgery patterns produced by constantly evolving facial manipulation techniques and diverse variations on real faces, we propose a classification-invariant feature augmentation (CIFAug) function without trainable parameters. CIFAug expands the scopes of real and fake feature space along directions orthogonal to the classification direction, enabling the model to learn more generalizable features while preserving discrimination. Extensive experiments demonstrate that our method achieves state-of-the-art generalization performance in cross-domain and cross-manipulation settings with only 0.28M trainable parameters.

The generalization capability of a face anti-spoofing (FAS) model is critical to its practicality in the real world. Recent studies have theoretically and empirically uncovered that neural networks tend to exploit easy-to-learn frequency sets for decisions. These simplicity-biased representations, depending on what best simplifies the training objective, may hamper generalization. This paper thus focuses on mitigating the frequency shortcut learning of prior FAS models for improved generalization. Specifically, we introduce a frequency-aware autoencoder to retain more frequency details in intermediate features via reconstruction, facilitating comprehensive judgment of FAS. Based on the encoder output, we propose a dynamic frequency masking mechanism to select and suppress the probable shortcut bands during training, enabling broader horizons on under-explored frequencies. Moreover, we employ a style inhibited modulation to weaken stylized information in frequency space to reduce the reliance on spurious style features. Experiment results on generalized FAS benchmarks verify the superiority of our framework over existing methods. Our code has been integrated into this project: https://github.com/VISION-SJTU/UniDefense.

Representation learning in spatial and temporal domains has shown significant potential in EEG decoding, advancing the field of brain-computer interfaces (BCIs). However, the critical role of frequency information, closely tied to the brain's neurological mechanism, has been largely neglected. In this paper, we propose FSTNet, which integrates frequency-spatial-temporal domains synergistically. The network allows broadband EEG signals as input and adaptively learns informative frequency signatures. A frequency-aware module emphasizes the importance of frequency information by selectively assigning weights to latent representations in the frequency space. Subsequently, self-attention captures spatial and temporal dependencies, extracting discriminative neural signatures for EEG decoding. We conducted extensive experiments on EEG datasets for motor imagery and emotion recognition, achieving superior results on SEED, PhysioNet, and OpenBMI datasets in both individual and cross-subject scenarios. Additionally, visualization reveals that the network captures informative frequency ranges and spatial patterns associated with specific tasks, aligning with known physiological mechanisms. This enhances the transparency of the network's learning process. In conclusion, our method exhibits the potential for decoding EEG and advancing the understanding of neurological processes in the brain.

Recent studies have highlighted the promising application of NeRF in autonomous driving contexts. However, the complexity of outdoor environments, combined with the restricted viewpoints in driving scenarios, complicates the task of precisely reconstructing scene geometry. Such challenges often lead to diminished quality in reconstructions and extended durations for both training and rendering. To tackle these challenges, we present Lightning NeRF. It uses an efficient hybrid scene representation that effectively utilizes the geometry prior from LiDAR in autonomous driving scenarios. Lightning NeRF significantly improves the novel view synthesis performance of NeRF and reduces computational overheads. Through evaluations on real-world datasets, such as KITTI-360, Argoverse2, and our private dataset, we demonstrate that our approach not only exceeds the current state-of-the-art in novel view synthesis quality but also achieves a five-fold increase in training speed and a ten-fold improvement in rendering speed. Codes are available at https://github.com/VISION-SJTU/Lightning-NeRF.

While humans effortlessly discern intrinsic dynamics and adapt to new scenarios, modern AI systems often struggle. Current methods for visual grounding of dynamics either use pure neural-network-based simulators (black box), which may violate physical laws, or traditional physical simulators (white box), which rely on expert-defined equations that may not fully capture actual dynamics. We propose the Neural Material Adaptor (NeuMA), which integrates existing physical laws with learned corrections, facilitating accurate learning of actual dynamics while maintaining the generalizability and interpretability of physical priors. Additionally, we propose Particle-GS, a particle-driven 3D Gaussian Splatting variant that bridges simulation and observed images, allowing back-propagate image gradients to optimize the simulator. Comprehensive experiments on various dynamics in terms of grounded particle accuracy, dynamic rendering quality, and generalization ability demonstrate that NeuMA can accurately capture intrinsic dynamics. Project Page: https://xjay18.github.io/projects/neuma.html.

For boosting output power, frequency-up methods like gear train and plucking are introduced into pendulum-based electromagnetic energy harvesters (EEHs). Still, the gear train and plucking methods have the problems of high manufacturing costs and high energy loss, respectively. To address the above issues, inspired by biological grooming behavior, we present a low-cost, high-efficiency EEH utilizing 3D printing. This device is excited by an asymmetric stiffness pendulum, induced by an oblique cantilever beam, converting the ultra-low frequency of human motion into high-speed, unidirectional rotation of the rotor. The energy density comparison between the sandwich and back iron structure of the EEH is made by the simulation, and then parameters are selected. The bench-top swing experiment system is established to test the performance of the EEH, and the established equivalent electromagnetic model discusses the load resistance result. The feasibility of human walking for powering the Internet of Things (IoT) device is explored when wearing the EEH. The bench-top test result shows that the EEH can reach the normalized power of 1.07 · 10-4 W/(Hz ·°) and normalized power density of 6.6 W/(m3·°). Through human testing at a walking of 1 km/h, the IoT device can run normally, showing great potential for achieving self-power and cost-effective IoT devices.

Condition monitoring has played a significant role in reducing downtime and maintenance costs for key rotating machinery. However, traditional methods to power wireless sensor nodes depend highly on capacity-limited batteries or wiring from external source. Although the rotational energy harvesting technologies have been widely considered as a promising self-powered method, the output power under low-frequency occasions fails to supply the usable energy to wireless sensor nodes for condition monitoring. Therefore, a self-powered wireless condition monitoring system for rotating machinery in low-frequency occasions is presented in this article. A variable reluctance energy harvester is designed to convert rotational motion into electrical power. The ring-shaped stator contains magnets and tile silicon steel, while the rotor is composed of teethed silicon steel and coils. Besides, a ring-shaped circuit for low-speed occasions is designed to achieve power management and storage, signal detection and wireless transmission. In addition, an experimental test is carried out to verify the performance of the proposed self-powered wireless condition monitoring system. The results show that the output power of the proposed harvester reaches 336.7–851.8 mW at 200–328 rpm, while the average power after rectification and filtering is 203.3–602.5 mW. Moreover, the power test results show that the broadcasting, connecting, collecting and transmitting, and sleep modes of WiFi consume around 450, 206, 313, and 46 mW, respectively. By properly prolonging sleep mode, the average power consumption of wireless sensor networks can be significantly reduced. Furthermore, the condition monitoring performance of the proposed self-powered system is verified by acceleration detection.

Existing face forgery detectors mainly focus on specific forgery patterns like noise characteristics, local textures, or frequency statistics for forgery detection. This causes specialization of learned representations to known forgery patterns presented in the training set, and makes it difficult to detect forgeries with unknown patterns. In this paper, from a new perspective, we propose a forgery detection frame-work emphasizing the common compact representations of genuine faces based on reconstruction-classification learning. Reconstruction learning over real images enhances the learned representations to be aware of forgery patterns that are even unknown, while classification learning takes the charge of mining the essential discrepancy between real and fake images, facilitating the understanding of forgeries. To achieve better representations, instead of only using the encoder in reconstruction learning, we build bipartite graphs over the encoder and decoder features in a multi-scale fashion. We further exploit the reconstruction difference as guidance of forgery traces on the graph output as the final representation, which is fed into the classifier for forgery detection. The reconstruction and classification learning is optimized end-to-end. Extensive experiments on large-scale benchmark datasets demonstrate the superiority of the proposed method over state of the arts.

Semantic segmentation requires a large amount of densely annotated data for training and may generalize poorly to novel categories. In real-world applications, we have an urgent need for few-shot semantic segmentation which aims to empower a model to handle unseen object categories with limited data. This task is non-trivial due to several challenges. First, it is difficult to extract the class-relevant information to handle the novel class as only a few samples are available. Second, since the image content can be very complex, the novel class information may be suppressed by the base categories due to limited data. Third, one may easily learn promising base classifiers based on a large amount of training data, but it is non-trivial to exploit the knowledge to train the novel classifiers. More critically, once a novel classifier is built, the output probability space will change. How to maintain the base classifiers and dynamically include the novel classifiers remains an open question. To address the above issues, we propose a Dynamic Extension Network (DENet) in which we dynamically construct and maintain a classifier for the novel class by leveraging the knowledge from the base classes and the information from novel data. More importantly, to overcome the information suppression issue, we design a Guided Attention Module (GAM), which can be plugged into any framework to help learn class-relevant features. Last, rather than directly train the model with limited data, we propose a dynamic extension training algorithm to predict the weights of novel classifiers, which is able to exploit the knowledge of base classifiers by dynamically extending classes during training. The extensive experiments show that our proposed method achieves state-of-the-art performance on the PASCAL-5i and COCO-20i datasets. The source code is available at https://github.com/lizhaoliu-Lec/DENet.