This paper introduces an evaluation method for large language models (LLMs) based on multi-model factual cognition consistency. Traditional evaluation methods, especially in terms of factuality assessments, face challenges in constructing extensive domain-specific question sets and relying on specific model answers. These methods fall short in the face of dynamic and diverse model development. To overcome these limitations, the proposed approach does not depend on a fixed set of standard answers. Instead, it utilizes the responses of multiple models to construct a dynamic, relative evaluation benchmark. We first developed a framework to capture and compare the cognitive consistency of different models when addressing specific questions. Subsequently, a dynamic iterative algorithm was designed to evaluate models based on these sets of answers. Experiments across multiple domains demonstrated the effectiveness of this method. This innovative evaluation strategy not only provides a more comprehensive and flexible approach to understanding and assessing the performance of LLMs in various scenarios but also offers practical guidance for future model development and improvement.

Chiplet-based deep neural network (DNN) accelerator is a promising solution to balance the performance and manufacturing cost. However, different from monolithic chips, interconnect network design and architectural partitioning for multiple chiplets would result in a huge design space and make it difficult to keep scalability and high hardware utilization. Moreover, how to efficiently map DNN workloads onto multiple DRAM dies and compute dies is another major challenge. To alleviate the above issues, in this work, we propose INDM, a chiplet-based interconnect network and dataflow mapping co-optimization for DNN accelerators. First, we propose an efficient hierarchical interconnect network composed of a multiring on-die network and a cluster-based interdie network, to facilitate the data reuse and traffic pattern in DNN workloads. Second, architectural partitioning and topology exploration for chiplet-based DNN accelerators are proposed to find the optimal architecture configurations. Third, an interdie communication-aware dataflow mapping is proposed to minimize traffic congestion during DNN layer switching. We implement the proposed chiplet-based interconnect network design and dataflow mapping algorithm for a set of popular DNN models, including VGG-16, ResNet-18, DarkNet-19, ResNet-50, and ResNet-101. Experimental results show that as compared with the state-of-the-art related work, such as NN-Baton and SIMBA, our work achieves 26.00%–73.81% energy-delay-product (EDP) reduction and 26.93%–79.78% latency reduction.

To ensure the delivery of high-performance and reliable services, data center networks (DCNs) are often over-provisioned for peak workload and traffic bursts. However, in real-world data centers, network traffic seldom reaches peak capacity of the network, resulting in significant energy waste. Traditional energy conservation approaches either suffer from high computational complexity and low solution quality, or their strategies cannot be dynamically adjusted to accommodate changes in data center network traffic. Deep reinforcement learning (DRL) provides an effective way to deal with these issues. However, most of the existing DRL-based schemes only consider either a continuous action space or a discrete action space, which greatly restricts the optimality of decisions. To solve these problems, this paper proposes a novel DRL-based DCN energy optimization framework, named SmartDCN. Specifically, SmartDCN consists of a traffic prediction module (TPM) and an energy optimization module (EOM). TPM incorporates an improved LSTM model JANET with an attention mechanism providing a high prediction accuracy, while EOM integrates our newly proposed parameterized DRL algorithm , named PAS-DQN, combining with the discrete-continuous hybrid action space. PAS-DQN implements a two-level control mechanism for the network, using TPM to predict future traffic in the data center as input. It is devoted to dynamically aggregating current traffic and makes tradeoffs between energy efficiency, performance, and robustness to optimize the network’s power consumption by dynamically calculating the minimum required network subset and turning off the non-involved network devices to achieve power savings. Experimental results show that SmartDCN significantly outperforms the existing state-of-the-art schemes in terms of energy savings under various network conditions.

Credit default swap transaction data repositories are frequently applied with credit default swap spread estimation and financial market risk assessment. However, in practical applications, there is poor liquidity, some missing data, and inaccurate definitions. Small samples tend to lead to poor prediction accuracy and poor adaptability of the statistical algorithm. Data generation can effectively increase the sample size and improve the effect of the risk assessment model. In this paper, a credit default swap data generation algorithm based on a sequence generative adversarial network (SeqGAN) is proposed, and the policy gradient algorithm in reinforcement learning is introduced to optimize the traditional generative adversarial network (GAN) algorithm to solve the gradient disappearance and poor data adaptability problems in the traditional algorithm. Gradient disappearance is due to the generator network in GAN being designed to be able to adjust the output continuously, which does not work on discrete data generation. The optimization algorithm proposed in this paper is used to train randomly distributed sequence data and generate credit default swap transactions with diversity and good model applicability. The credit default swap data generated in this paper are verified by the synthetic ranking agreement (SRA) index. The results show that SeqGAN can effectively synthesize various simulation samples, which can provide support for the risk discrimination model.

In recent years, deep learning (DL) has prospered again due to improvements in both computing and learning theory. Emerging studies mostly focus on the acceleration of refining DL models but ignore data preprocessing issues. However, data preprocessing can significantly affect the overall performance of end-to-end DL workflows. Our studies on several image DL workloads show that existing preprocessing backends are quite inefficient: they either perform poorly in throughput (30% degradation) or burn too many (>10) CPU cores. Based on these observations, we propose DLBooster, a high-performance data preprocessing pipeline that selectively offloads key workloads to FPGAs, to fit the stringent demands on data preprocessing for cutting-edge DL applications. Our testbed experiments show that, compared with the existing baselines, DLBooster can achieve 1.35×~2.4× image processing throughput in several DL workloads, but consumes only 1/10 CPU cores. Besides, it also reduces the latency by 1/3 in online image inference.