Sound Event Detection (SED) plays a vital role in comprehending and perceiving acoustic scenes. Previous methods have demonstrated impressive capabilities. However, they are deficient in learning features of complex scenes from heterogeneous dataset. In this paper, we introduce a novel dual-branch architecture named Mutual-Assistance Tuning and Dual-Branch Aggregating for Heterogeneous Sound Event Detection (MTDA-HSED). The MTDA-HSED architecture employs the Mutual-Assistance Audio Adapter (M3A) to effectively tackle the multi-scenario problem and uses the Dual-Branch Mid-Fusion (DBMF) module to tackle the multi-granularity problem. Specifically, M3A is integrated into the BEATs block as an adapter to improve the BEATs’ performance by fine-tuning it on the multi-scenario dataset. The DBMF module connects BEATs and CNN branches, which facilitates the deep fusion of information from the BEATs and the CNN branches. Experimental results show that the proposed methods exceed the baseline of mpAUC by 5% on the DESED and MAESTRO Real datasets.

Multi-person motion prediction is an emerging and intricate task with broad real-world applications. Unlike single person motion prediction, it considers not just the skeleton structures or human trajectories but also the interactions between others. Previous methods achieve impressive predictions using various networks but often overlook the distinct representations of joint relations within individuals (intra-relations) and interactions among groups (inter-relations), inevitably leading to undesired dependencies. To address this issue, we introduce a new collaborative framework for multi-person motion prediction that explicitly modeling these relations: a GCN-based network for intra-relations and a novel reasoning network for inter-relations. Specifically, we propose a distance-aware cross-attention that incorporates physical distance constraints into inter-relation learning through a learnable distance weighting coefficient. Moreover, we propose a novel plug-and-play aggregation module called the Interaction Aggregation Module (IAM), which employs an aggregate-attention mechanism to seamlessly integrate these relations. Experiments indicate that the module can also be applied to other dual-path models. Extensive experiments on the 3DPW, 3DPW-RC, CMU-Mocap, MuPoTS-3D, as well as synthesized datasets Mix1 & Mix2 (9$\sim$15 persons), demonstrate that our method achieves state-of-the-art performance.

The dynamic information of the joints, such as the movement amplitude, is critical for forecasting precise human joint trajectories. Existing methods adopt global modeling in which all joints are treated as a whole to extract features for movement coordination. Though global modeling can exploit hidden relationships between joints, it also inevitably introduces undesired trajectory dependencies, which weakens the dynamic information effects and simplifies the constraints and kinetics model of joints. Therefore, we propose a dynamic pattern-based collaborative modeling framework (DPnet) that contains a keyframe enhanced module (KEM) and multi-channel feature extractor blocks (MFE-block). The KEM tackles the discontinuity between the last frame of observation and the first predicted one by duplicating the decisive frame. The MFE-block utilizes a multi-channel graph structure to enrich the dynamic information effects and recessive constraints of joints. To distinguish the dynamic information of each joint, we calculate the movement amplitude of the joints and propose three dynamic patterns, including active, inactive, and static patterns. We also propose a dynamic pattern-guided feature extractor (DP-FE) to alleviate the trajectory dependencies between joints with different dynamic patterns. We evaluate our approach on three standard benchmark datasets, including H3.6M [8], CMU-Mocap [44], and 3DPW [45]. Our approach achieves impressive results in both short-term and long-term predictions, confirming its effectiveness and efficiency.