Diffusion-based text-to-image generation models trade latency for quality: small models are fast but generate lower quality images, while large models produce better images but are slow. We present MoDM, a novel caching-based serving system for diffusion models that dynamically balances latency and quality through a mixture of diffusion models. Unlike prior approaches that rely on model-specific internal features, MoDM caches final images, allowing seamless retrieval and reuse across multiple diffusion model families. This design enables adaptive serving by dynamically balancing latency and image quality: using smaller models for cache-hit requests to reduce latency while reserving larger models for cache-miss requests to maintain quality. Small model image quality is preserved using retrieved cached images. We design a global monitor that optimally allocates GPU resources and balances inference workload, ensuring high throughput while meeting Service-Level Objectives (SLOs) under varying request rates. Our evaluations show that MoDM significantly reduces average serving time by 2.5× while retaining image quality, making it a practical solution for scalable and resource-efficient model deployment.

Collaborative perception in autonomous driving significantly enhances the perception capabilities of individual agents. Immutable heterogeneity, where agents have different and fixed perception networks, presents a major challenge due to the semantic gap in exchanged intermediate features without modifying the perception networks. Most existing methods bridge the semantic gap through interpreters. However, they either require training a new interpreter for each new agent type, limiting extensibility, or rely on a two-stage interpretation via an intermediate standardized semantic space, causing cumulative semantic loss. To achieve both extensibility in immutable heterogeneous scenarios and low-loss feature interpretation, we propose PolyInter, a polymorphic feature interpreter. It provides an extension point where new agents integrate by overriding only their specific prompts, which are learnable parameters that guide interpretation, while reusing PolyInter’s remaining parameters. By leveraging polymorphism, our design enables a single interpreter to accommodate diverse agents and interpret their features into the ego agent’s semantic space. Experiments on the OPV2V dataset demonstrate that PolyInter improves collaborative perception precision by up to 11.1% compared to SOTA interpreters, while comparable results can be achieved by training only 1.4% of PolyInter’s parameters when adapting to new agents. Code is available at https://github.com/yuchen-xia/PolyInter.

Large Language Models (LLMs) offer flexible reasoning capability but lack physical embodiment, while traditional automation systems can execute physical processes yet lack cognitive capability. This paper presents a layered architecture that bridges this gap by integrating LLMs with digital twins and physical automation systems, with reference to practical case studies as proof of concept. The proposed architecture comprises three layers: a cognitive layer powered by LLMs, a bridging layer based on digital twins, and a physical layer consisting of technical process and automation system. Within the digital twin layer, we introduce three design paradigms for structuring information to support effective LLM integration: state snapshot modeling, event message modeling, and plan sequence modeling. These paradigms are demonstrated through prototypical case studies on robotic automation control and process simulation. To address challenges such as hallucination, task complexity, and system reliability, we distill a set of practical strategies, including multi-agent system design, human validation, and test-driven development. Additionally, we propose the concept of "Return on Intelligence" as a conceptual tool for evaluating the efficacy of investments in intelligent automation. This research contributes to the theoretical foundation and the architecture design for developing intelligent, adaptive automation systems powered by LLMs.

Traditional industrial automation systems require specialized expertise to operate and complex reprogramming to adapt to new processes. Large language models offer the intelligence to make them more flexible and easier to use. However, LLMs’ application in industrial automation settings is underexplored. This paper introduces a framework for integrating LLMs to achieve end-to-end control of industrial automation systems. At the core of the framework is an agent system designed for industrial automation tasks. A structured prompting method and an event-driven modeling mechanism provide the information for LLMs to perform reasoning on different context levels, allowing them to semantically interpret the information, generate production plans, and control operations on the automation system. Furthermore, this framework facilitates the creation of structured datasets for fine-tuning LLMs on this specific downstream application. Our contribution includes a formal system design, proof-of-concept implementation, and a method for generating task-specific datasets for LLM fine-tuning and testing. This approach enables a more adaptive automation system that can respond to spontaneous events, allowing intuitive system operation and configuration through natural language. Demo videos and detailed evaluation data are accessible on GitHub: https://github.com/YuchenXia/LLM4IAS.

Oblivious RAM (ORAM) hides the memory access patterns, enhancing data privacy by preventing attackers from discovering sensitive information based on the sequence of memory accesses. The performance of ORAM is often limited by its inherent trade-off between security and efficiency, as concealing memory access patterns imposes significant computational and memory overhead. While prior works focus on improving the ORAM performance by prefetching and eliminating ORAM requests, we find that their performance is very sensitive to workload locality behavior and incurs additional management overhead caused by the ORAM stash pressure. This paper presents Palermo: a protocol-hardware co-design to improve ORAM performance. The key observation in Palermo is that classical ORAM protocols enforce restrictive dependencies between memory operations that result in low memory bandwidth utilization. Palermo introduces a new protocol that overlaps large portions of memory operations, within a single and between multiple ORAM requests, without breaking correctness and security guarantees. Subsequently, we propose an ORAM controller architecture that executes the proposed protocol to service ORAM requests. The hardware is responsible for concurrently issuing memory requests as well as imposing the necessary dependencies to ensure a consistent view of the ORAM tree across requests. Using a rich workload mix, we demonstrate that Palermo outperforms the RingORAM baseline by 2.9 ×, on average, incurring a negligible area overhead of 5.78mm2(less than 2% in 12th generation Intel CPU after technology scaling) and 2.14W without sacrificing security. We further show that Palermo also outperforms the state-of-the-art works PageORAM, PrORAM, and IR-ORAM.

We introduce MOSDT, the first algorithm designed for multi-agent offline safe reinforcement learning (MOSRL), alongside MOSDB, the first dataset and benchmark for this domain. Different from most existing knowledge distillation-based multi-agent RL methods, we propose policy self-distillation (PSD) with a new global information reconstruction scheme by fusing the observation features of all agents, streamlining training and improving parameter efficiency. We adopt full parameter sharing across agents, significantly slashing parameter count and boosting returns up to 38.4-fold by stabilizing training. We propose a new plug-and-play cost binary embedding (CBE) module, which binarizes cumulative costs as safety signals and embeds the signals into return features for efficient information aggregation. On the strong MOSDB benchmark, MOSDT achieves state-of-the-art (SOTA) returns in 14 out of 18 tasks (across all base environments including MuJoCo, Safety Gym, and Isaac Gym) while ensuring complete safety, with only 65% of the execution parameter count of a SOTA single-agent offline safe RL method CDT. Code, dataset, and results are available at this website: https://github.com/Lucian1115/MOSDT.git

This paper presents a novel design of a multi-agent system framework that applies large language models (LLMs) to automate the parametrization of simulation models in digital twins. This framework features specialized LLM agents tasked with observing, reasoning, decision-making, and summarizing, enabling them to dynamically interact with digital twin simulations to explore parametrization possibilities and determine feasible parameter settings to achieve an obj ective. The proposed approach enhances the usability of simulation model by infusing it with knowledge heuristics from LLM and enables autonomous search for feasible parametrization to solve a user task. Furthermore, the system has the potential to increase user-friendliness and reduce the cognitive load on human users by assisting in complex decision-making processes. The effectiveness and functionality of the system are demonstrated through a case study, and the visualized demos and codes are available at a GitHub Repository: https://github.comlYuchenXia/LLMDrivenSimulation

This paper presents a novel application of Large Language Models (LLMs) to improve Failure Mode and Effect Analysis (FMEA) for managing risks in technical processes and products. We designed an LLM multi-agent system that utilizes Retrieval Augmented Generation (RAG) for risk analysis and text generation. This system interfaces directly with an FMEA knowledge database to dynamically extract relevant information, perform reasoning based on user input and retrieved information, and generate targeted text recommendations for completing FMEA spreadsheets. By automating the analysis and synthesis of risk information, this system can provide expert knowledge and reduces the cognitive load on users, speeding up the FMEA process. The effectiveness and functionality of this LLM-enhanced FMEA application is demonstrated through a prototype, accessible via a GitHub Repository at: https://github.com/YuchenXia/LLMRiskAnalyzer.

In this paper, we present a novel framework that combines large language models (LLMs), digital twins and industrial automation system to enable intelligent planning and control of production processes. We retrofit the automation system for a modular production facility and create executable control interfaces of fine-granular functionalities and coarse-granular skills. Low-level functionalities are executed by automation components, and high-level skills are performed by automation modules. Subsequently, a digital twin system is developed, registering these interfaces and containing additional descriptive information about the production system. Based on the retrofitted automation system and the created digital twins, LLM-agents are designed to interpret descriptive information in the digital twins and control the physical system through service interfaces. These LLM-agents serve as intelligent agents on different levels within an automation system, enabling autonomous planning and control of flexible production. Given a task instruction as input, the LLM-agents orchestrate a sequence of atomic functionalities and skills to accomplish the task. We demonstrate how our implemented prototype can handle un-predefined tasks, plan a production process, and execute the operations. This research highlights the potential of integrating LLMs into industrial automation systems in the context of smart factory for more agile, flexible, and adaptive production processes, while it also underscores the critical insights and limitations for future work. Demos at: https://github.com/YuchenXia/GPT4IndustrialAutomation

The Asset Administration Shell (AAS) is a standardized data container for sharing data in the context of Industry 4.0. It allows different participants or systems to communicate their information based on shared meaning. Today, however, AAS makes data interoperable at the cost of extra development effort. Developers must map a proprietary information model to a standardized AAS model during the data transformation. In this work-in-process paper, a novel data transformation method based on transfer learning with neural language model is proposed to automatically map the data properties from an arbitrary information model into a standardized AAS model. The term "semantic fingerprint" is used to characterize a pivot intermediate vector generated by a neural network, containing latent conceptual meaning about a data property, which is in turn used for generating the mappings between data properties with similar conceptual meaning. By this means, the proposed approach fills the research gap on automated generation of AAS models with semantic analysis and is able to lower the barrier to adopting AAS with tool support.

The Freemium-Crowdsourcing platform replaces purchases in Freemium games with crowdsourcing tasks, which can effectively incentivize player participation using the psychological factor of reward attraction. However, no previous researches focus on quantifying this factor, nor do they make an effort on controlling the answer quality produced by the hybrid platform. We propose a novel online algorithm, IGATA, to allocate crowdsourcing tasks to Freemium game players on the hybrid Freemium-Crowdsourcing platform. In order to maximize the task satisfaction to reduce waste caused by task refusal, while keeping competitive expected profit output, we measure the players' quality using their in-game attributes, and quantify the task reward attraction as the key factor of allocation with the help of psychological knowledge. Evaluations conducted on real trace data show that IGATA makes a significant improvement in producing total attraction comparing to existing strategies.