The surge of LLM studies makes synthesizing their findings challenging. Analysis of experimental results from literature can uncover important trends across studies, but the time-consuming nature of manual data extraction limits its use.Our study presents a semi-automated approach for literature analysis that accelerates data extraction using LLMs.It automatically identifies relevant arXiv papers, extracts experimental results and related attributes, and organizes them into a structured dataset, LLMEvalDB.We then conduct an automated literature analysis of frontier LLMs, reducing the effort of paper surveying and data extraction by more than 93% compared to manual approaches.We validate LLMEvalDB by showing that it reproduces key findings from a recent manual analysis of Chain-of-Thought (CoT) reasoning and also uncovers new insights that go beyond it, showing, for example, that in-context examples benefit coding & multimodal tasks but offer limited gains in math reasoning tasks compared to zero-shot CoT.Our automatically updatable dataset enables continuous tracking of target models by extracting evaluation studies as new data becomes available. Through LLMEvalDB and empirical analysis, we provide insights into LLMs while facilitating ongoing literature analyses of their behavior.

Teleoperated humanoid robot systems have made substantial advancements in recent years, offering a physical avatar that harnesses human skills and decision-making while safeguarding users from hazardous environments. However, current telelocomotion interfaces often fail to accurately represent the robot's environment, limiting the user's ability to effectively navigate the robot through unstructured terrain. This paper presents an initial telelocomotion framework that integrates the ForceBot locomotion interface with the small-sized humanoid robot, HECTOR V2. The framework utilizes ForceBot to simulate walking motion and estimate the user's Center of Mass (CoM) trajectory, which serves as a tracking reference for the robot. On the robot side, a model predictive control (MPC) approach, based on a reduced-order single rigid body model, is employed to track the user's scaled trajectory. We present experimental results on ForceBot's CoM estimation and the robot's tracking performance, demonstrating the feasibility of this approach.

Containers have become a critical component of cloud-native technologies, enabling organizations to run scalable and isolated workloads. However, in recent years there has been an increase in the sophistication of attacks targeting the cloud-native environment. It is crucial to implement security controls that protect containers at all stages of their lifecycle to mitigate risks such as privilege escalation by malicious applications. Numerous studies have aimed to develop security profiles for container applications that limit their privileges and protect the host system from compromise. Despite these efforts, the shared underlying kernel may still permit several successful attacks. In this study, we strive to develop a concise system call whitelist to address the problem of excessive privileges while ensuring the operational availability of applications. To achieve this, we propose to enhance static analysis with dynamic analysis to gather comprehensive information about the containerized application during two distinct execution phases: the initialization and the serving phases. Using this information, we determine the essential system calls for the application's operation and prevent all unwarranted system calls. We then perform crash analysis on the container under test to identify and incorporate any missing system calls. Through numerous experiments with popular server applications, we confirm that our approach is effective in discovering the necessary system calls for the operation of containerized applications. The system call whitelists produced by this method are more concise than Docker's default seccomp profile, consequently reducing the attack surface for a wide variety of applications and host systems significantly.

Despite significant interest in developing general purpose fact checking models, it is challenging to construct a large-scale fact verification dataset with realistic real-world claims.Existing claims are either authored by crowdworkers, thereby introducing subtle biases that are difficult to control for, or manually verified by professional fact checkers, causing them to be expensive and limited in scale.In this paper, we construct a large-scale challenging fact verification dataset called FAVIQ, consisting of 188k claims derived from an existing corpus of ambiguous information-seeking questions.The ambiguities in the questions enable automatically constructing true and false claims that reflect user confusions (e.g., the year of the movie being filmed vs. being released).Claims in FAVIQ are verified to be natural, contain little lexical bias, and require a complete understanding of the evidence for verification.Our experiments show that the stateof-the-art models are far from solving our new task.Moreover, training on our data helps in professional fact-checking, outperforming models trained on the widely used dataset FEVER or in-domain data by up to 17% absolute.Altogether, our data will serve as a challenging benchmark for natural language understanding and support future progress in professional fact checking.1

The main contribution of this paper is to present a straightforward yet effective historical input selection method for real-time traffic forecasting. Historical inputs are crucial elements in traffic forecasting because they help models recognize historical traffic patterns. However, selecting appropriate historical inputs becomes challenging, especially on irregular holidays with abnormal traffic patterns.As a solution, we suggest a method with three clear strengths. First, it is simple enough to employ in real-time computation. Second, our method provides more sophisticated features than existing studies which roughly categorize and match similar days. Finally, our method outperforms existing methods remarkably, given the unusual traffic patterns on special holidays.Our experiments verified the outperformance through an industrial prediction model and traffic dataset from a leading mobility service provider in South Korea.

Consistency training regularizes a model by enforcing predictions of original and perturbed inputs to be similar.Previous studies have proposed various augmentation methods for the perturbation but are limited in that they are agnostic to the training model.Thus, the perturbed samples may not aid in regularization due to their ease of classification from the model.In this context, we propose an augmentation method of adding a discrete noise that would incur the highest divergence between predictions.This virtual adversarial discrete noise obtained by replacing a small portion of tokens while keeping original semantics as much as possible efficiently pushes a training model's decision boundary.Experimental results show that our proposed method outperforms other consistency training baselines with text editing, paraphrasing, or a continuous noise on semi-supervised text classification tasks and a robustness benchmark 1 .

Gangwoo Kim, Hyunjae Kim, Jungsoo Park, Jaewoo Kang. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Near Field Communication (NFC) is a wireless communication technology using 13.56 MHz to support 2-way communications between two devices within ~10 cm. Such a short communication range may be considered as a shortcoming, but at the same time, this enables a secure data transfer within the connectivity region when compared to Bluetooth Low Energy (BLE) or Wi-Fi. Accordingly, NFC can be optionally utilized when secure data are transmitted in various services and applications focusing on ID validation. This demo presents a proof-of-concept for an alternative packet transmission method for achieving secure data exchange based on IP-over-NFC technology. For realizing this, there are a number of technical challenges. Utmost, the current NFC Linux kernel driver requires updates for supporting network-layer functionalities. Using such implementations, this demo demonstrates the procedures of transmitting secure data using NFC, while normal packet transmissions occur over WiFi simultaneously.