Modern engineering, spanning electrical, mechanical, aerospace, civil, and computer disciplines, stands as a cornerstone of human civilization and the foundation of our society. However, engineering design poses a fundamentally different challenge for large language models (LLMs) compared with traditional textbook-style problem solving or factual question answering. Although existing benchmarks have driven progress in areas such as language understanding, code synthesis, and scientific problem solving, real-world engineering design demands the synthesis of domain knowledge, navigation of complex trade-offs, and management of the tedious processes that consume much of practicing engineers' time. Despite these shared challenges across engineering disciplines, no benchmark currently captures the unique demands of engineering design work. In this work, we introduce EngDesign, an Engineering Design benchmark that evaluates LLMs' abilities to perform practical design tasks across nine engineering domains. Unlike existing benchmarks that focus on factual recall or question answering, EngDesign uniquely emphasizes LLMs' ability to synthesize domain knowledge, reason under constraints, and generate functional, objective-oriented engineering designs. Each task in EngDesign represents a real-world engineering design problem, accompanied by a detailed task description specifying design goals, constraints, and performance requirements. EngDesign pioneers a simulation-based evaluation paradigm that moves beyond textbook knowledge to assess genuine engineering design capabilities and shifts evaluation from static answer checking to dynamic, simulation-driven functional verification, marking a crucial step toward realizing the vision of engineering Artificial General Intelligence (AGI).

While federated learning enables intelligent services and personalized user experiences, it raises privacy concerns due to regulatory requirements and user demands for data protection. Federated unlearning offers a potential solution to these issues. However, despite increasing demand for its practical implementation driven by right-to-be-forgotten regulations, the economic implications of federated unlearning on user behavior and platform profitability remain underexplored, potentially hindering its adoption. In this paper, we formulate a set of contract design problems for both unlearning-disabled and unlearning-enabled scenarios. Challenges arise when the unlearning-enabled platform jointly designs compensation for both learning and unlearning to incentivize users' sequential decisions to balance the expected revenue and unlearning cost. We first conduct a questionnaire survey that reveals that federated unlearning increases users' willingness to participate in federated learning. We then provide a necessary condition for maximizing the surplus of an unlearning-enabled platform, enabling the point-wise decomposition for the optimal contract design problem, based on which we minimize the incentive cost and maximize the surplus for the platform. Our further analysis reveals that i) the incentive effects of unlearning grow quadratically with users' privacy sensitivity, and ii) enabling unlearning may even profit more than disabling it when the training cost increases at a faster rate than the probability of privacy leakage as effort levels rise. Our numerical results show that the platform's profitability is primarily influenced by users' privacy sensitivity. When users have a relatively high privacy sensitivity, enabling unlearning can significantly improve profitability.

Evaluating the bias in Large Language Models (LLMs) becomes increasingly crucial with their rapid development.However, existing evaluation methods rely on fixed-form outputs and cannot adapt to the flexible open-text generation scenarios of LLMs (e.g., sentence completion and question answering).To address this, we introduce BiasAlert, a plug-and-play tool designed to detect social bias in open-text generations of LLMs.BiasAlert integrates external human knowledge with inherent reasoning capabilities to detect bias reliably.Extensive experiments demonstrate that BiasAlert significantly outperforms existing state-of-theart methods like GPT4-as-A-Judge in detecting bias.Furthermore, through application studies, we demonstrate the utility of BiasAlert in reliable LLM bias evaluation and bias mitigation across various scenarios.Model and code will be publicly released.