Recently, many deep learning models have been trained in geographically distributed data centers. The carbon emissions produced by training the models may pose a significant threat to climate change like increasing temperatures. Existing studies have a hardship in shifting the workload of training models to a data center with low carbon emissions. So, they fail to ensure low emissions of the workload during training, especially when long-term workloads like Large Language Models (LLMs) are trained. To cope with this problem, we propose a method that shifts the workload to a cloud with low carbon emissions while enduring a lack of computational resources. Specifically, we define a task scheduler that includes states and their transitions to migrate mini-batches dynamically. Next, we present a fault-tolerant control that optimizes a GPU frequency to adapt to workload variations of training models while guaranteeing its power consumption. Last, we conducted exhaustive experiments using real-world data in terms of carbon emissions, transfer time, and power consumption compared to state-of-the-art methods.

While humans naturally develop theory of mind (ToM), the capability to understand other people's mental states and beliefs, state-ofthe-art large language models (LLMs) underperform on simple ToM benchmarks.We posit that we can extend our understanding of LLMs' ToM abilities by evaluating key human ToM precursors-perception inference and perception-to-belief inference-in LLMs.We introduce two datasets, Percept-ToMi and Percept-FANToM, to evaluate these precursory inferences for ToM in LLMs by annotating characters' perceptions on ToMi and FANToM, respectively.Our evaluation of eight state-ofthe-art LLMs reveals that the models generally perform well in perception inference while exhibiting limited capability in perception-tobelief inference (e.g., lack of inhibitory control).Based on these results, we present PercepToM, a novel ToM method leveraging LLMs' strong perception inference capability while supplementing their limited perception-to-belief inference.Experimental results demonstrate that PercepToM significantly enhances LLM's performance, especially in false belief scenarios.

Abstract Warning: This paper contains examples of stereotypes and biases. The Bias Benchmark for Question Answering (BBQ) is designed to evaluate social biases of language models (LMs), but it is not simple to adapt this benchmark to cultural contexts other than the US because social biases depend heavily on the cultural context. In this paper, we present KoBBQ, a Korean bias benchmark dataset, and we propose a general framework that addresses considerations for cultural adaptation of a dataset. Our framework includes partitioning the BBQ dataset into three classes—Simply-Transferred (can be used directly after cultural translation), Target-Modified (requires localization in target groups), and Sample-Removed (does not fit Korean culture)—and adding four new categories of bias specific to Korean culture. We conduct a large-scale survey to collect and validate the social biases and the targets of the biases that reflect the stereotypes in Korean culture. The resulting KoBBQ dataset comprises 268 templates and 76,048 samples across 12 categories of social bias. We use KoBBQ to measure the accuracy and bias scores of several state-of-the-art multilingual LMs. The results clearly show differences in the bias of LMs as measured by KoBBQ and a machine-translated version of BBQ, demonstrating the need for and utility of a well-constructed, culturally aware social bias benchmark.

This paper discusses the main benefits of ZNS and shows why ZNS can be deprived of internal parallelism when downsizing its zone writable capacity. To this end, we use two production ZNS SSDs and quantitively analyze the performance degradation caused by inter-zone interference. We then suggest a simple mechanism to detect zone-to-zone relationships generating the interference and schedule I/O requests by being aware of internal parallelism. Our evaluation results using real production ZNS devices show that our mechanism can improve the bandwidth and latency of Linux's multi-queue I/O scheduler by 1.98 × and 2.2 ×, respectively.

Bill writing is a critical element of representative democracy.However, it is often overlooked that most legislative bills are derived, or even directly copied, from other bills.Despite the significance of bill-to-bill linkages for understanding the legislative process, existing approaches fail to address semantic similarities across bills, let alone reordering or paraphrasing which are prevalent in legal document writing.In this paper, we overcome these limitations by proposing a 5-class classification task that closely reflects the nature of the bill generation process.In doing so, we construct a human-labeled dataset of 4,721 billto-bill relationships at the subsection-level and release this annotated dataset to the research community.To augment the dataset, we generate synthetic data with varying degrees of similarity, mimicking the complex bill writing process.We use BERT variants and apply multi-stage training, sequentially fine-tuning our models with synthetic and human-labeled datasets.We find that the predictive performance significantly improves when training with both human-labeled and synthetic data.Finally, we apply our trained model to infer section-and bill-level similarities.Our analysis shows that the proposed methodology successfully captures the similarities across legal documents at various levels of aggregation. 1

We present a model to predict fine-grained emotions along the continuous dimensions of valence, arousal, and dominance (VAD) with a corpus with categorical emotion annotations.Our model is trained by minimizing the EMD (Earth Mover's Distance) loss between the predicted VAD score distribution and the categorical emotion distributions sorted along VAD, and it can simultaneously classify the emotion categories and predict the VAD scores for a given sentence.We use pre-trained RoBERTa-Large and fine-tune on three different corpora with categorical labels and evaluate on EmoBank corpus with VAD scores.We show that our approach reaches comparable performance to that of the state-of-the-art classifiers in categorical emotion classification and shows significant positive correlations with the ground truth VAD scores.Also, further training with supervision of VAD labels leads to improved performance especially when dataset is small.We also present examples of predictions of appropriate emotion words that are not part of the original annotations.

We introduce EfficientCL, a memory-efficient continual pretraining method that applies contrastive learning with novel data augmentation and curriculum learning.For data augmentation, we stack two types of operation sequentially: cutoff and PCA jittering.While pretraining steps proceed, we apply curriculum learning by incrementing the augmentation degree for each difficulty step.After data augmentation, we apply contrastive learning on projected embeddings of original and augmented examples.When fine-tuned on GLUE benchmark, our model outperforms baseline models, especially for sentence-level tasks.Additionally, this improvement is achieved with only 70% of computational memory compared to the baseline model. 1

With detailed sensor and visual data from automobiles, a data-driven model can learn to classify crash-related events during a drive. We propose a neural network model accepting time-series vehicle sensor data and forward-facing videos as input for learning classification of crash-related events and varying types of such events. To elaborate, a novel recurrent neural network structure is introduced, namely, denoising gated recurrent unit with decay, in order to deal with time-series automobile sensor data with missing value and noises. Our model detects crash and near-crash events based on a large set of time-series data collected from naturalistic driving behavior. Furthermore, the model classifies those events involving pedestrians, a vehicle in front, or a vehicle on either side. The effectiveness of our model is evaluated with more than two thousand 30-s clips from naturalistic driving behavior data. The results show that the model, including sensory encoder with denoising gated recurrent unit with decay, visual encoder, and attention mechanism, outperforms gated recurrent unit with decay, gated CNN, and other baselines not only in event classification and but also in event-type classification.