Engine anomaly detection is crucial in the marine systems sector as it enables the timely identification of potential faults, prevents major accidents, and ensures navigational safety. In this paper, we propose an unsupervised anomaly detection method to detect marine engines using a graph network autoencoder with multi-scale feature fusion. This method efficiently identifies temporal relationships in typical time series data, thus removing the requirement for labeled failure data. We selected the data with obvious anomalies for testing, compared the original data with the reconstructed data, marked the abnormal data, and achieved good results.

Graph neural networks (GNNs) have emerged as a powerful tool for fault diagnosis in industrial processes, where components and their relationships can be modeled as nodes and edges in a graph. However, most GNN models are based on the I.I.D. hypothesis and learn both causal and spurious correlations. In real-world applications, test data distributions often differ from training data, leading to changes in spurious correlations and inaccurate diagnosis results. To address this issue, this paper proposes a causal regularization graph attention network (CR-GAT) that decouples causal and confounding variables and eliminates spurious correlations. The method involves extracting high-level graph variables using differentiable pooling, measuring nonlinear dependence between variables with the Hilbert-Schmidt Independence Criterion (HSIC), and iteratively optimizing GAT loss and variable weights to learn true causal relationships. Comparative experiments on three-phase flow and power system datasets demonstrate the effectiveness of the proposed method.

Traditional intelligent fault diagnosis models are usually capable of diagnosing known types of faults. However, in the field of industrial fault diagnosis in open environments, it is almost impossible to collect training samples that cover all fault categories. Therefore, when encountering unknown types of fault, traditional methods tend to misclassify them as known categories. To address this issue, a causal counterfactual faithfulness generation method is proposed for open-set fault diagnosis of complex industrial processes. Initially, the signal data from fault sensors are processed into graph data composed of nodes and edges. Then, the features of nodes and their adjacent nodes are learned and integrated into graph architecture to generate new fault sample attributes. Subsequently, the causal generative model infers the category features and combines known fault categories to generate counterfactual samples. Finally, the sample’s classification as an unknown category is ultimately determined by testing the principle of consistency. The proposed method can significantly improve the accuracy of open-set diagnosis without affecting the accuracy of closed-set classification. Comparison experiments with multiple baseline models in two fault datasets illustrated that the proposed method shows an improvement in almost all indicators, which ultimately verified the effectiveness of the proposed method in the task of fault diagnosis in open environments.

Designing reward functions is a longstanding challenge in reinforcement learning (RL); it requires specialized knowledge or domain data, leading to high costs for development. To address this, we introduce Text2Reward, a data-free framework that automates the generation and shaping of dense reward functions based on large language models (LLMs). Given a goal described in natural language, Text2Reward generates shaped dense reward functions as an executable program grounded in a compact representation of the environment. Unlike inverse RL and recent work that uses LLMs to write sparse reward codes or unshaped dense rewards with a constant function across timesteps, Text2Reward produces interpretable, free-form dense reward codes that cover a wide range of tasks, utilize existing packages, and allow iterative refinement with human feedback. We evaluate Text2Reward on two robotic manipulation benchmarks (ManiSkill2, MetaWorld) and two locomotion environments of MuJoCo. On 13 of the 17 manipulation tasks, policies trained with generated reward codes achieve similar or better task success rates and convergence speed than expert-written reward codes. For locomotion tasks, our method learns six novel locomotion behaviors with a success rate exceeding 94%. Furthermore, we show that the policies trained in the simulator with our method can be deployed in the real world. Finally, Text2Reward further improves the policies by refining their reward functions with human feedback. Video results are available at https://text-to-reward.github.io/

We introduce Lemur and Lemur-Chat, openly accessible language models optimized for both natural language and coding capabilities to serve as the backbone of versatile language agents. The evolution from language chat models to functional language agents demands that models not only master human interaction, reasoning, and planning but also ensure grounding in the relevant environments. This calls for a harmonious blend of language and coding capabilities in the models. Lemur and Lemur-Chat are proposed to address this necessity, demonstrating balanced proficiencies in both domains, unlike existing open-source models that tend to specialize in either. Through meticulous pretraining using a code-intensive corpus and instruction fine-tuning on text and code data, our models achieve state-of-the-art averaged performance across diverse text and coding benchmarks. Comprehensive experiments demonstrate Lemur’s superiority over existing open-source models and its proficiency across various agent tasks involving human communication, tool usage, and interaction under fully- and partially- observable environments. The harmonization between natural and programming languages enables Lemur-Chat to significantly narrow the gap with proprietary models on agent abilities, providing key insights into developing advanced open-source agents adept at reasoning, planning, and operating seamlessly across environments. Our model and code have been open-sourced at https://github.com/OpenLemur/Lemur.

Autonomous agents that accomplish complex computer tasks with minimal human interventions have the potential to transform human-computer interaction, significantly enhancing accessibility and productivity. However, existing benchmarks either lack an interactive environment or are limited to environments specific to certain applications or domains, failing to reflect the diverse and complex nature of real-world computer use, thereby limiting the scope of tasks and agent scalability. To address this issue, we introduce OSWorld, the first-of-its-kind scalable, real computer environment for multimodal agents, supporting task setup, execution-based evaluation, and interactive learning across various operating systems such as Ubuntu, Windows, and macOS. OSWorld can serve as a unified, integrated computer environment for assessing open-ended computer tasks that involve arbitrary applications. Building upon OSWorld, we create a benchmark of 369 computer tasks involving real web and desktop apps in open domains, OS file I/O, and workflows spanning multiple applications. Each task example is derived from real-world computer use cases and includes a detailed initial state setup configuration and a custom execution-based evaluation script for reliable, reproducible evaluation. Extensive evaluation of state-of-the-art LLM/VLM-based agents on OSWorld reveals significant deficiencies in their ability to serve as computer assistants. While humans can accomplish over 72.36% of the tasks, the best model achieves only 12.24% success, primarily struggling with GUI grounding and operational knowledge. Comprehensive analysis using OSWorld provides valuable insights for developing multimodal generalist agents that were not possible with previous benchmarks. Our code, environment, baseline models, and data are publicly available at this https URL.

We present ClothesNet: a large-scale dataset of 3D clothes objects with information-rich annotations. Our dataset consists of around 4400 models covering 11 categories annotated with clothes features, boundary lines, and keypoints. ClothesNet can be used to facilitate a variety of computer vision and robot interaction tasks. Using our dataset, we establish benchmark tasks for clothes perception, including classification, boundary line segmentation, and keypoint detection, and develop simulated clothes environments for robotic interaction tasks, including rearranging, folding, hanging, and dressing. We also demonstrate the efficacy of our ClothesNet in real-world experiments. Supplemental materials and dataset are available on our project webpage at https://sites.google.com/view/clothesnet.

Differentiable Simulations have recently proven useful for various robotic manipulation tasks, including cloth manipulation. In robotic cloth simulation, it is crucial to maintain intersection-free properties. We present DiffClothAI, a differentiable cloth simulation with intersection-free friction contact and two-way coupling with articulated rigid bodies. DiffClothAI integrates the Project Dynamics and Incremental Potential Contact coherently and proposes an effective method to derive gradients in the Cloth Simulation. It also establishes the differentiable coupling mechanism between articulated rigid bodies and cloth. We conduct a comprehensive evaluation of DiffClothAI's effectiveness and accuracy and perform a variety of experiments in downstream robotic manipulation tasks. Supplemental materials and videos are available on our project webpage at https://sites.google.com/view/diffsimcloth.