Major progress in reward modeling over recent years has been driven by a paradigm shift from task-specific designs to generalist reward models. Despite this trend, developing effective reward models remains a fundamental challenge: the heavy reliance on large-scale labeled preference data. Pre-training on abundant unlabeled data offers a promising direction, but existing approaches fall short in instilling explicit reasoning capabilities into reward models. To bridge this gap, we propose a self-training approach that can leverage unlabeled data to scale up reward reasoning in reward models. Based on this approach, we develop GRAM-R² a generative reward model trained to produce not only preference labels but also accompanying reward rationales. GRAM-R² can serve as a foundation model for reward reasoning and can be applied to a wide range of tasks with minimal or no additional fine-tuning. It can support downstream applications such as policy optimization and task-specific reward tuning. Experiments on response ranking, task adaptation, and reinforcement learning from human feedback demonstrate that GRAM-R² consistently delivers strong performance, outperforming several strong discriminative and generative baselines.

Automatic modulation classification (AMC) has a wide range of applications in both civilian and military fields, such as industrial Internet of Things (IIoT) security, communication spectrum management, and military electronic countermeasures. However, label mislabeling often occurs in practical scenarios, significantly impacting the performance and robustness of deep neural networks (DNNs). In this article, we propose a meta-learning guided label noise distillation method to enhance the robustness of AMC models against label noise or errors. Specifically, we propose a teacher-student heterogeneous network (TSHN) to discriminate and distill label noise. Following the notion that labels represent information, a teacher network, utilizing trusted few-shot labeled samples, reevaluates and corrects labels for a considerable number of untrusted labeled samples through meta-learning. By dividing and conquering untrusted labeled samples according to their confidence levels, the student network learns more effectively. Additionally, we propose a multiview signal (MVS) method to further enhance the performance of hard-to-classify categories with few-shot trusted labeled samples. Extensive experiments on the RadioML2016 and HisarMod2019.1 data sets demonstrate that our methods significantly improve accuracy and robustness in signal AMC across diverse label noise scenarios, including symmetric, asymmetric, and mixed label noise. For example, compared to the baseline convolutional neural network with the cross-entropy loss, our proposed TSHN achieves a remarkable 1.26% to 36.84% accuracy improvement under symmetric label noise and 0.12% to 38.59% accuracy improvement under mixed label noise. Moreover, TSHN exhibits greater robustness to varying label noise rates compared to existing methods.

3D human mesh recovery from single RGB images or monocular videos is a challenging task. The twist representation utilized in existing inverse kinematics-based methods fails to accurately describe the twisting posture when the estimated bone direction is imprecise. Additionally, supervising SMPL shape parameters has the issue of shape estimation overfitting due to limited training data. This often results in compromised bone lengths that subsequently impair the precision of joint positions. To address these issues, we propose a framework that breaks down both human pose and shape into finer components, effectively managing and minimizing errors within each component. The proposed framework integrates two key advancements: the advanced Ortho-Twist and Swing Representation (OTSR) and the Skeleton-Focused Shape Refinement (SFSR). OTSR offers a more sophisticated representation for limb rotations compared to the traditional twist angle and swing representation to enhance the accuracy of twisting posture estimation. SFSR refines the estimated SMPL shape parameters by fitting bone lengths using the estimated joint positions, thereby significantly mitigating shape overfitting and enhancing joint position accuracy in the recovered mesh. We conduct experiments on the Human3.6 M and 3DPW datasets. The results demonstrate the superiority of the proposed framework in both single-image and video scenarios. Additionally, the ablation studies confirm the effectiveness of our proposed modules, and further generalizability experiments demonstrate that our two key advancements can serve as plug-and-play modules to enhance existing methods.

Most existing wideband signal detection and recognition (WSDR) methods rely on diverse, large-scale, and well-labeled training data, which are often difficult to obtain in practical application scenarios such as non-cooperative environments and novel signaling regimes. In this article, we propose a method for constructing a virtual signal large model (VSLM) and applying it to tackle the WSDR challenge under few-shot or even cross-domain few-shot scenarios. Firstly, we design two plug-and-play modules, virtual sample generation (VSG) and virtual category generation (VCG), for VSLM, respectively. VSG simulates the local and overall relationship between the burst signal and the constant signal, which is mainly completed by extracting time-frequency meta-block and data enhancement. Based on VSG and the multi-label concept, we further create virtual novel categories by injecting customizable semantic information into meta-blocks. Then, we further propose a dual decoupled network (DDN) to train the VSLM. DDN enhances signal details by decoupling low gray values (DLGV) in time-frequency representation, and alleviates conflicts during multi-task joint optimization by decoupling spectrum localization and signal classification. Finally, based on the wideband spectrogram dataset, extensive experiments have validated that our proposed methods can significantly improve the performance of WSDR under few-shot conditions.

Specific emitter identification (SEI) is crucial for attacking and defending Internet of Things (IoT) devices in untrusted scenarios or battlefield environments. However, existing SEI methods usually require annotation information, which is often unavailable in noncooperative communications and untrusted scenarios. In this article, we propose a signal contrastive self-supervised clustering (SCSC) method for unsupervised SEI applications. First, we propose SCSC with 1-D fingerprint pyramid feature extractor (1D-FPFE) for obtaining hierarchical subtle features of emitter signals. Then, we propose a bit-pulse selection (BPS) strategy and several signal data augmentation methods. By constructing signal positive and negative instance pairs through data augmentation, our approach generates cluster preference representations in a contrastive self-supervised learning manner. Extensive experimental results based on communication burst emitter dataset show that SCSC achieves an accuracy improvement of about 26% over the current best communication signal clustering algorithm. Moreover, SCSC also exhibits good performance and generalization for 30 emitter clustering and few-shot unlabeled signal clustering.

Internet of Things (IoT) networks are often subject to many malicious attacks in untrusted environments, and automatic modulation classification (AMC) is an effective way to combat IoT physical-layer threats. However, most existing AMC methods assume sufficient labeled signals and invariant signal distribution, which is often impossible in untrusted environments. In this article, a new meta-learning method is proposed for a few-shot AMC with distribution bias. First, a multi-frequency octave ResNet (MFOR) is constructed to learn coarse (low-frequency) and fine (high-frequency) features, which can efficiently identify the modulation type of the signal while saving computational resources. Second, a large number of classification-related meta-tasks are established for training MFOR to explore general knowledge in signal classification, and then transfer it to the AMC. Different with deep neural networks (DNNs) that learn a mapping by multiple instances, the MFOR with meta-learning (denoted as M-MFOR) can improve the generalization ability of new AMC tasks with very few instances and distribution bias. Furthermore, we find that the distribution bias between data can be reduced by adjusting the normalized distribution and propose a class-related mixup. Extensive experiments are taken on several datasets to investigate the effectiveness of M-MFOR. The results show its feasibility and superiority over existing methods.