Large language model (LLM) agents have been applied to sequential decision-making tasks such as web navigation, but without any environment-specific experiences, they often fail in these complex tasks.Moreover, current LLM agents are not designed to continually learn from past experiences during inference time, which could be crucial for them to gain these environment-specific experiences.To address this, we propose Contextual Experience Replay (CER), a training-free framework to enable efficient self-improvement for language agents in their context window.Specifically, CER accumulates and synthesizes past experiences into a dynamic memory buffer.These experiences encompass environment dynamics and common decision-making patterns, allowing the agents to retrieve and augment themselves with relevant knowledge in new tasks, enhancing their adaptability in complex environments.We evaluate CER on the challenging WEBARENA and VISUALWEBARENA benchmarks.On VISUALWEBARENA, CER achieves competitive performance of 31.9%.On WEBARENA, CER also gets a competitive average success rate of 36.7%, relatively improving the success rate of the GPT-4o agent baseline by 51.0%.We also conduct a comprehensive analysis on it to prove its efficiency, validity and understand it better.

Designing reward functions is a longstanding challenge in reinforcement learning (RL); it requires specialized knowledge or domain data, leading to high costs for development. To address this, we introduce Text2Reward, a data-free framework that automates the generation and shaping of dense reward functions based on large language models (LLMs). Given a goal described in natural language, Text2Reward generates shaped dense reward functions as an executable program grounded in a compact representation of the environment. Unlike inverse RL and recent work that uses LLMs to write sparse reward codes or unshaped dense rewards with a constant function across timesteps, Text2Reward produces interpretable, free-form dense reward codes that cover a wide range of tasks, utilize existing packages, and allow iterative refinement with human feedback. We evaluate Text2Reward on two robotic manipulation benchmarks (ManiSkill2, MetaWorld) and two locomotion environments of MuJoCo. On 13 of the 17 manipulation tasks, policies trained with generated reward codes achieve similar or better task success rates and convergence speed than expert-written reward codes. For locomotion tasks, our method learns six novel locomotion behaviors with a success rate exceeding 94%. Furthermore, we show that the policies trained in the simulator with our method can be deployed in the real world. Finally, Text2Reward further improves the policies by refining their reward functions with human feedback. Video results are available at https://text-to-reward.github.io/

We introduce Lemur and Lemur-Chat, openly accessible language models optimized for both natural language and coding capabilities to serve as the backbone of versatile language agents. The evolution from language chat models to functional language agents demands that models not only master human interaction, reasoning, and planning but also ensure grounding in the relevant environments. This calls for a harmonious blend of language and coding capabilities in the models. Lemur and Lemur-Chat are proposed to address this necessity, demonstrating balanced proficiencies in both domains, unlike existing open-source models that tend to specialize in either. Through meticulous pretraining using a code-intensive corpus and instruction fine-tuning on text and code data, our models achieve state-of-the-art averaged performance across diverse text and coding benchmarks. Comprehensive experiments demonstrate Lemur’s superiority over existing open-source models and its proficiency across various agent tasks involving human communication, tool usage, and interaction under fully- and partially- observable environments. The harmonization between natural and programming languages enables Lemur-Chat to significantly narrow the gap with proprietary models on agent abilities, providing key insights into developing advanced open-source agents adept at reasoning, planning, and operating seamlessly across environments. Our model and code have been open-sourced at https://github.com/OpenLemur/Lemur.

Chart understanding plays a pivotal role when applying Multimodal Large Language Models (MLLMs) to real-world tasks such as analyzing scientific papers or financial reports. However, existing datasets often focus on oversimplified and homogeneous charts with template-based questions, leading to an overly optimistic measure of progress. We demonstrate that although open-source models can appear to outperform strong proprietary models on these benchmarks, a simple stress test with slightly different charts or questions deteriorates performance by up to 34.5%. In this work, we propose CharXiv, a comprehensive evaluation suite involving 2,323 natural, challenging, and diverse charts from scientific papers. CharXiv includes two types of questions: 1) descriptive questions about examining basic chart elements and 2) reasoning questions that require synthesizing information across complex visual elements in the chart. To ensure quality, all charts and questions are handpicked, curated, and verified by human experts. Our results reveal a substantial, previously underestimated gap between the reasoning skills of the strongest proprietary model (i.e., GPT-4o), which achieves 47.1% accuracy, and the strongest open-source model (i.e., InternVL Chat V1.5), which achieves 29.2%. All models lag far behind human performance of 80.5%, underscoring weaknesses in the chart understanding capabilities of existing MLLMs. We hope that CharXiv facilitates future research on MLLM chart understanding by providing a more realistic and faithful measure of progress. Project website: https://charxiv.github.io/

Autonomous agents that accomplish complex computer tasks with minimal human interventions have the potential to transform human-computer interaction, significantly enhancing accessibility and productivity. However, existing benchmarks either lack an interactive environment or are limited to environments specific to certain applications or domains, failing to reflect the diverse and complex nature of real-world computer use, thereby limiting the scope of tasks and agent scalability. To address this issue, we introduce OSWorld, the first-of-its-kind scalable, real computer environment for multimodal agents, supporting task setup, execution-based evaluation, and interactive learning across various operating systems such as Ubuntu, Windows, and macOS. OSWorld can serve as a unified, integrated computer environment for assessing open-ended computer tasks that involve arbitrary applications. Building upon OSWorld, we create a benchmark of 369 computer tasks involving real web and desktop apps in open domains, OS file I/O, and workflows spanning multiple applications. Each task example is derived from real-world computer use cases and includes a detailed initial state setup configuration and a custom execution-based evaluation script for reliable, reproducible evaluation. Extensive evaluation of state-of-the-art LLM/VLM-based agents on OSWorld reveals significant deficiencies in their ability to serve as computer assistants. While humans can accomplish over 72.36% of the tasks, the best model achieves only 12.24% success, primarily struggling with GUI grounding and operational knowledge. Comprehensive analysis using OSWorld provides valuable insights for developing multimodal generalist agents that were not possible with previous benchmarks. Our code, environment, baseline models, and data are publicly available at this https URL.

This paper presents a magnetic integration scheme of LCL filter with symmetrical structure. This magnetic integration scheme uses EIE core to realize the inductor integration of symmetrical LCL filter, and realizes the decoupling and integration of upper and lower E-cores. The effectiveness of the magnetic integrated symmetrical LCL filter is verified by simulation and experiment. A 600 W single-phase SiC high-frequency inverter was used as the experimental platform to give the output current harmonic analysis results and conducted EMI noise. The symmetrical LCL filter with integrated structure not only has good harmonic suppression effect, but also greatly improves EMI suppression ability. The integrated filter is significantly smaller weight, helping to increase system power density.

Due to the aging of electric infrastructures, conventional power grid is being modernized toward smart grid that enables two-way communications between consumer and utility, and thus more vulnerable to cyber-attacks. However, due to the attacking cost, the attack strategy may vary a lot from one operation scenario to another from the perspective of adversary, which is not considered in previous studies. Therefore, in this paper, scenario-based two-stage sparse cyber-attack models for smart grid with complete and incomplete network information are proposed. Then, in order to effectively detect the established cyber-attacks, an interval state estimation-based defense mechanism is developed innovatively. In this mechanism, the lower and upper bounds of each state variable are modeled as a dual optimization problem that aims to maximize the variation intervals of the system variable. At last, a typical deep learning, i.e., stacked auto-encoder, is designed to properly extract the nonlinear and nonstationary features in electric load data. These features are then applied to improve the accuracy for electric load forecasting, resulting in a more narrow width of state variables. The uncertainty with respect to forecasting errors is modeled as a parametric Gaussian distribution. The validation of the proposed cyber-attack models and defense mechanism have been demonstrated via comprehensive tests on various IEEE benchmarks.

The planning of charging facilities (CFs) for electric vehicles (EVs) plays an important role for the extensive applications of EVs. Uncertainties existing in the development of future EV technology should be properly modeled to ensure the robustness of the planning scheme. The uncertainties concerned include EV development types, growth rate of load and traffic flow, and distributions of load and traffic flow in the smart grid. The existing single-stage planning model cannot fully evaluate the risks brought by all kinds of uncertainties. Given this background, the multistage CF planning problem considering uncertainties is studied in this work. First, several typical uncertainties in future smart grid with a high penetration of EVs are considered to generate development scenarios for multistage planning. Then, the well-established data envelopment analysis is utilized to evaluate the planning schemes while the novel EV expected energy not supplied cost is defined to measure the service ability of CFs. The final planning result obtained by the proposed framework will not only have good performance in the current stage but also exhibit robustness for all the considered scenarios in the future stage with respect to uncertainties. The application potential of the designed multistage planning framework is proved by an example with both the distribution network and traffic network included.

Grid-connected inverter is an important part of the grid-connected system. Compared with the traditional L or LC filter, LCL filter has a better high-frequency harmonic attenuation performance. However, LCL filter has resonant peak, which has a great influence on the stability of the system. This paper first analyzes the effect of passive damping method on the resonance peak; then a double closed-loop control strategy with the inner loop of capacitor current and the outer loop of grid current is introduced; finally, the LCL three-phase grid-connected inverter with double-loop current control significantly reduces the Total Harmonic Distortion (THD), which verifies the feasibility of parameters design scheme of control strategy.

Power switch devices which base on wide band-gap (WBG) semiconductor, such as silicon carbide metal-oxide-semiconductor-field-effect-transistor (SiC MOSFET) and gallium nitride high-electron-mobility transistor (GaN-HEMT) perform superior performance as compared with silicon (Si) MOSFET in high switching frequency, high blocking voltage, and high temperature operation. In this paper, a series of characteristic test and comparative evaluation of power loss and efficiency which based on Si MOSFET (IPW65R080CFDA), SiC MOSFET (C2M0080120D), and GaN HEMT (TPH3212PS) with similar current rating are conducted in the single-ended flyback converter. It shows that the WBG switch devices can reduce more power loss, improve efficiency, and enhance more power density instead of Si MOSFET.

Computer-aided diagnosis (CAD) systems have been verified to have the potential to assist radiologists in clinical diagnosis to detect and characterize focal liver lesions (FLLs) based on single- or multiphase contrast-enhanced computed tomography (CT) images. Features extracted from multiphase contrast-enhanced CT images carry more important diagnostic information i.e. enhancement pattern and demonstrate much stronger discriminative ability compared to those of single-phase CT images. In this paper, we propose a new method for multiphase image feature generation called the bag of temporal co-occurrence words (BoTCoW). A temporal co-occurrence image connecting intensity from multiphase images is constructed. Then the bag of visual word (BoVW) model is employed on the temporal co-occurrence images to extract temporal features. The proposed method effectively captures temporal enhancement information and demonstrates the distribution of the evolution patterns. The effectiveness of this method is validated in a retrieval system using 132 FLLs with confirmed pathology type. The preliminary results show that the proposed BoTCoW method outperforms the previously proposed temporal features and multiphase features based on the BoVW model.