Human-Object Interaction (HOI) detection remains challenging due to the semantic ambiguity of interaction categories and the limited discriminability of their feature representations. Existing approaches often improve recognition by employing sophisticated models or auxiliary textual annotations. While effective in certain gains, these solutions incur additional computational or annotation costs and struggle to capture intrinsic interaction regularities. To address these issues, we propose Pose-Guided Multi-Cue Explicit Query Construction (PM-EQC), a unified Transformer-based framework that builds upon collaborative modeling of appearance, spatial, and pose cues for discriminative interaction reasoning. At its core, the Collaborative Multi-Cue Query Constructor (CM-CQC) jointly models dependencies among visual cues to generate explicit query embeddings. CM-CQC further incorporates a hierarchical pose contextualization mechanism: global body configurations adaptively guide attention to local critical joints, yielding fine-grained pose embeddings and more precise interaction disambiguation. Owing to its modular design, PM-EQC integrates seamlessly with diverse backbones and benefits from their advances. Extensive experiments on PhysLab, HICO-DET, and V-COCO datasets demonstrate that PM-EQC achieves state-of-the-art performance, and the code is publicly available at https://github.com/ZMHSDUST/ PM-EQC.

Visual parsing of images and videos is critical for a wide range of real-world applications. However, progress in this field is constrained by limitations of existing datasets: (1) limited annotation diversity, which limits the support for diverse vision tasks within a unified dataset; (2) insufficient coverage of domains, particularly a lack of datasets tailored for educational scenarios; and (3) a lack of explicit procedural guidance, with weak logical rules and insufficient representation of a structured task process. To address these gaps, we introduce PhysLab, the first dataset that captures students conducting complex physics experiments. The dataset includes four representative experiments that feature diverse scientific instruments and rich human-object interaction (HOI) patterns. PhysLab comprises 620 long-form videos and provides multi-granularity annotations that support a variety of vision tasks, including action recognition, object detection, HOI analysis, etc. We establish baselines and perform extensive evaluations to highlight key challenges in the parsing of procedural educational videos. We expect PhysLab to serve as a valuable resource for advancing comprehensive visual parsing, facilitating intelligent classroom systems, and fostering closer integration among computer vision, multimedia, and educational technologies. The dataset and the evaluation toolkit are publicly available at https://github.com/ZMH-SDUST/PhysLab.

The prediction of remaining execution time is a critical area of research in business process monitoring. However, limited data availability and deficiencies in existing models have hindered progress in this area. To address these challenges, we introduce two production log datasets, coarse-grained log for television (CGL-TV) and fine-grained log for television (FGL-TV), collected from a semiautomated assembly line for television manufacturing. These datasets aim to address the issue of data scarcity in monitoring, analyzing, and optimizing the manufacturing process. Then, we investigate the significance of role information in semiautomated production processes and propose a novel feature selection strategy. That strategy replaces the traditional activity attributes with role attributes as the basis for prediction, resulting in a significant improvement in prediction accuracy. Furthermore, existing recurrent neural network (RNN)-based prediction methods have two major drawbacks: limited ability to capture long-term dependencies and the inability to parallelize computations. To address these issues, we propose a novel transformer-based remaining time prediction (TRTP) model. This model utilizes the self-attention mechanism instead of hidden state passing, which effectively incorporates global contextual information and enables potential parallel computing. The experimental results across four datasets demonstrate that our proposed method achieves state-of-the-art performance, e.g., a 5.2% performance gain on FGL-TV.

Action learning is a research area that aims to recognize the action category of each frame in the video. Context information is crucial for learning actions, but most existing methods face two challenges in exploiting this information: 1) They apply global attention to aggregate global features for action representation, resulting in inefficiency and redundancy. 2) They impose implicit action constraints to regularize the action distribution, leading to subjectivity, interpretability issues, and optimization difficulties. To address these challenges, we propose an end-to-end weakly-supervised Action Learning framework with Process Knowledge Decomposition (AL-PKD), which leverages the intrinsic characteristics of procedural task videos. To enhance the effectiveness and adaptability of context aggregation, we first design the TEAL-Net action recognition network. Specifically, the TEAL-Net accounts for the diverse neighbor distributions of action nodes across categories and collects local neighborhood features with different receptive fields through feature pyramids, improving the accuracy and efficiency of action representation. Moreover, to overcome the drawbacks of implicit constraint strategies, we next employ process mining techniques to extract three types of explicit action pair constraints: sequentiality, concurrency, and selectivity. These constraints guide the model’s predictions and improve the interpretability of the learning process. Finally, we use the Viterbi algorithm to dynamically infer the optimal action boundaries based on the frame-level predictions, which helps to eliminate local misclassifications. Experiments on three datasets of Breakfast, CrossTask, and PEVD demonstrate that our method achieves state-of-the-art performance.