Modeling complicated interactions among the ego-vehicle, road agents, and map elements has been a crucial part for safety-critical autonomous driving. Previous work on end-to-end autonomous driving relies on the attention mechanism to handle heterogeneous interactions, which fails to capture geometric priors and is also computationally intensive. In this paper, we propose the Interaction Scene Graph (ISG) as a unified method to model the interactions among the ego-vehicle, road agents, and map elements. With the representation of the ISG, the driving agents aggregate essential information from the most influential elements, including the road agents with potential collisions and the map elements to follow. Since a mass of unnecessary interactions are omitted, the more efficient scene-graph-based framework is able to focus on indispensable connections and leads to better performance. We evaluate the proposed method for end-to-end autonomous driving on the nuScenes dataset. Compared with strong baselines, our method significantly outperforms in full-stack driving tasks.

Large-scale monitoring represents a critical frontier in industrial IoT research, yet real-time tracking of heterogeneous machinery with prognostic capabilities remains a persistent challenge. Unresolved complexities include divergent feature dimensions across machinery, nonlinear interdependencies in multivariate data streams, and dynamic temporal behavior. To overcome these limitations, we introduce Kube-IPM, a Kubernetes-native platform designed for large-scale industrial process monitoring. First, we propose the attention-based padding (ABP) method that generates meaningful feature extensions instead of dummy values, preserving data integrity. Then we apply a temporal attention layer within an encoder-decoder architecture addressing time delays and variable coupling. Finally, we establish a Kubernetes-orchestrated cloud-edge deployment enabling scalable real-time monitoring. Taking the injection molding barrel heating process as an example, we validate our approach using a comprehensive dataset collected from two injection molding machines, one experimental platform, and two simulation models, featuring variable sensor configurations (4-6). We conduct comparisons with state-of-the-art predictive and padding methods. Experiments validate model’s performance, yielding MAE of 0.13± 0.003 (single-step) and 0.26± 0.009 (multi-step). It exceeds benchmarks by 38.1% and 38.0% against top predictive models, with 18.8% and 7.1% improvements over advanced padding techniques. Attention weight analysis validates that the temporal attention module effectively identifies temporal dependencies, enhancing interpretability in forecasting. Furthermore, Kube-IPM maintains 12 concurrent replicas within sampling intervals. Operational costs average $65 per machine, with marginal scaling expenses of $55 per unit, ensuring cost-efficiency for scalability. By integrating cloud-edge and attention-driven prediction technologies, this study constructs an intelligent manufacturing framework with both academic frontier and engineering implementation. Code availability: https://github.com/MangoLab-ZJU/ABP-TA-LSTM.git.

The heating process contributes to over 60% of the energy consumption in material manufacturing, such as injection molding, metallurgy, die casting, etc. Temperature prediction is the crucial first step to realize energy management. However, the complex heating process poses challenges, such as strong multivariate coupling, large time delay, and frequent changes of heat transfer characteristics. To address these issues, this article introduces a temporal re-attention long short-term memory (TRA-LSTM) method for multivariate time series prediction. We adapt an encoder–decoder framework. A temporal attention module is utilized for extracting key historical features. A re-attention incorporated loss layer is proposed to rectify the temporal attention. TRA-LSTM's performance is compared with five advanced time series prediction methods across six predictions on four real-world datasets collected, including data from the injection molding barrel heating process and the battery thermal runaway process. Results show that TRA-LSTM consistently outperformed the other methods with high accuracy and efficiency. The alignment of re-attention weights with tested time delays in both heating and cooling processes further demonstrates TRA-LSTM's advantage. Ablation experiments underscore the significant contribution of the re-attention module, while hyperparameter sensitivity analysis offers guidance on selection and validates TRA-LSTM's robustness. Overall, the proposed method enables precise multivariate time series prediction and shows substantial potential for optimizing energy management in industrial production.

Quality prediction plays an essential role in industrial process monitoring and optimization. However, the existed methods lack of the ability to extract features in multiphase and nonlinear time series data. To improve the quality prediction accuracy and simultaneously provide some suggestion of the process mechanism, this article proposes a phased dual-attention long short-term memory (PDA-LSTM) neural network, which consists of a phase segmentation module and a prediction module. The whole series data are first divided into phases in the phase segmentation module, and individual models are subsequently constructed for each phase in the prediction module to capture distinct patterns. The LSTM network with input and temporal attention mechanism is utilized to select the most relevant variables and time steps throughout the time sequence. The proposed method is verified with a numerical example and a real injection molding process. Compared with the existed methods, the PDA-LSTM method has the advantages of higher prediction accuracy and the ability to interpret the key input variable and time points of the process data.

Predicting future trajectories of pairwise traffic agents in highly interactive scenarios, such as cut-in, yielding, and merging, is challenging for autonomous driving. The existing works either treat such a problem as a marginal prediction task or perform single-axis factorized joint prediction, where the former strategy produces individual predictions without considering future interaction, while the latter strategy conducts conditional trajectory-oriented prediction via agentwise interaction or achieves conditional rollout-oriented prediction via timewise interaction. In this article, we propose a novel double-axis factorized joint prediction pipeline, namely, conditional goal-oriented trajectory prediction (CGTP) framework, which models future interaction both along the agent and time axes to achieve goal and trajectory interactive prediction. First, a goals-of-interest network (GoINet) is designed to extract fine-grained features of goal candidates via hierarchical vectorized representation. Furthermore, we propose a conditional goal prediction network (CGPNet) to produce multimodal goal pairs in an agentwise conditional manner, along with a newly designed goal interactive loss to better learn the joint distribution of the intermediate interpretable modes. Explicitly guided by the goal-pair predictions, we propose a goal-oriented trajectory rollout network (GTRNet) to predict scene-compliant trajectory pairs via timewise interactive rollouts. Extensive experimental results confirm that the proposed CGTP outperforms the state-of-the-art (SOTA) prediction models on the Waymo open motion dataset (WOMD), Argoverse motion forecasting dataset, and In-house cut-in dataset. Code is available at https://github.com/LiDinga/CGTP/.

Utilizing infrastructure and vehicle-side information to track and forecast the behaviors of surrounding traffic participants can significantly improve decision-making and safety in autonomous driving. However, the lack of real-world sequential datasets limits research in this area. To address this issue, we introduce V2X-Seq, the first large-scale sequential V2X dataset, which includes data frames, trajectories, vector maps, and traffic lights captured from natural scenery. V2X-Seq comprises two parts: the sequential perception dataset, which includes more than 15,000 frames captured from 95 scenarios, and the trajectory forecasting dataset, which contains about 80,000 infrastructure-view scenarios, 80,000 vehicle-view scenarios, and 50,000 cooperative-view scenarios captured from 28 intersections' areas, covering 672 hours of data. Based on V2X-Seq, we introduce three new tasks for vehicle-infrastructure cooperative (VIC) autonomous driving: VIC3D Tracking, Online-VIC Forecasting, and Offline-VIC Forecasting. We also provide benchmarks for the introduced tasks. Find data, code, and more up-to-date information at https://github.com/AIR-THU/DAIR-V2X-Seq.

Deep learning-based Video Frame Interpolation (Deep VFI) diminishes the visual traces of the conventional one such that it is challenging for the current VFI detectors. Therefore, it is necessary to identify the presence of deep interpolated frames (DIF) in a video. This paper proposed a hybrid neural network to localize the DIF by learning spatio-temporal representations from the residual and motion vector information in the compression domain. Firstly, the residual and motion vector of motion regions are maintained by an intra-prediction constraints. Then, inherent tampering traces are further highlighted through subtracting the estimate of the residual or motion vector by virtue of residual modulation or MV refinement network. Finally, an attention-based dual-stream network is designed to jointly learn discriminative representations from the enhancement traces. Deep VFI video datasets created by the state-of-the-art deep VFI methods, have been evaluated, and extensive experimental results clearly demonstrate that our approach can achieve state-of-the-art performance compared with conventional methods.

Deep learning-based video inpainting can fill the missing or undesired regions with spatial-temporal consistent contents without obvious visually distortion. Although the original purpose of deep inpainting is to repair flawed videos, it can also be adopted for malicious purposes, e.g., removal of specific objects. Therefore, automatically locating the inpainted regions is a challenging task in video forensics. This paper proposes a new forensic refinement framework to localize the deep inpainted regions by considering the spatial-temporal viewpoint. Firstly, we design a spatiotemporal convolution to suppress redundancy for highlighting deep inpainting traces. Then, a detection module is constructed with four concatenated ResNet blocks, and two upsampling layers to achieve a rough location map. Finally, a modified U-net based refinement module is developed for the pixel-wise localization map. Deep inpaiting video datasets created by the state-of-the-art deep inpainting method, have been evaluated, and extensive experimental results clearly demonstrate the efficacy of the proposed approach.

The prevalent scene text detection approach follows four sequential steps comprising character candidate detection, false character candidate removal, text line extraction, and text line verification. However, errors occur and accumulate throughout each of these sequential steps which often lead to low detection performance. To address these issues, we propose a unified scene text detection system, namely Text Flow, by utilizing the minimum cost (min-cost) flow network model. With character candidates detected by cascade boosting, the min-cost flow network model integrates the last three sequential steps into a single process which solves the error accumulation problem at both character level and text line level effectively. The proposed technique has been tested on three public datasets, i.e, ICDAR2011 dataset, ICDAR2013 dataset and a multilingual dataset and it outperforms the state-of-the-art methods on all three datasets with much higher recall and F-score. The good performance on the multilingual dataset shows that the proposed technique can be used for the detection of texts in different languages.