Recent advances in knowledge distillation (KD) have enabled smaller student models to approach the performance of larger teacher models. However, popular methods such as supervised KD and on-policy KD, are adversely impacted by the knowledge gaps between teacher-student in practical scenarios. Supervised KD suffers from a distribution mismatch between training with a static dataset and inference over final student-generated outputs. Conversely, on-policy KD, which uses student-generated samples for training, can suffer from low-quality training examples with which teacher models are not familiar, resulting in inaccurate teacher feedback. To address these limitations, we introduce Speculative Knowledge Distillation (SKD), a novel approach that leverages cooperation between student and teacher models to generate high-quality training data on-the-fly while aligning with the student's inference-time distribution. In SKD, the student proposes tokens, and the teacher replaces poorly ranked ones based on its own distribution, transferring high-quality knowledge adaptively. We evaluate SKD on various text generation tasks, including translation, summarization, math, and instruction following, and show that SKD consistently outperforms existing KD methods across different domains, data sizes, and model initialization strategies.

Recent studies show that large language models (LLMs) improve their performance through self-feedback on certain tasks while degrade on others.We discovered that such a contrary is due to LLM's bias in evaluating their own output.In this paper, we formally define LLM's self-bias -the tendency to favor its own generation -using two statistics.We analyze six LLMs (GPT-4, GPT-3.5, Gemini, LLaMA2, Mixtral and DeepSeek) on translation, constrained text generation, and mathematical reasoning tasks.We find that self-bias is prevalent in all examined LLMs across multiple languages and tasks.Our analysis reveals that while the self-refine pipeline improves the fluency and understandability of model outputs, it further amplifies self-bias.To mitigate such biases, we discover that larger model size and external feedback with accurate assessment can significantly reduce bias in the self-refine pipeline, leading to actual performance improvement in downstream tasks.The code and data are released at https://github. com/xu1998hz/llm_self_bias.

Direct alignment from preferences (DAP) has emerged as a promising paradigm for aligning large language models (LLMs) to human desiderata from pre-collected, offline preference datasets. While recent studies indicate that existing offline DAP methods can directly benefit from online training samples, we highlight the need to develop specific online DAP algorithms to fully harness the power of online training. Specifically, we identify that the learned LLM should adhere to the proximity of the behavior LLM, which collects the training samples. To this end, we propose online Preference Optimization in proximity to the Behavior LLM (BPO), emphasizing the importance of constructing a proper trust region for LLM alignment.We conduct extensive experiments to validate the effectiveness and applicability of our approach by integrating it with various DAP methods, resulting in significant performance improvements across a wide range of tasks when training with the same amount of preference data. Even when only introducing one additional data collection phase, our online BPO improves its offline DAP baseline from 72.0% to 80.2% on TL;DR and from 82.2% to 89.1% on Anthropic Helpfulness in terms of win rate against human reference text.

This paper introduces a voice-controlled Human Machine Interface (HMI) tailored for an assistive robotic exoskeleton glove, aimed at assisting patients coping with Brachial Plexus Injuries (BPI) in regaining their lost grasping functionality. The development of this HMI draws upon clinical experimentation results, forming a foundation for its design. The paper delves into the challenges encountered while employing a prior voice-based HMI, which necessitated an internet connection for complex computations and exhibited limitations in effectively processing concise commands. To address these issues, an innovative voice-controlled HMI system is proposed, featuring fixed-word detection to replace the speech-to-text (STT) converter and the Neutral Language Processor (NLP) to reduce computational overhead. Furthermore, the new HMI replaces the previous text-independent speaker verification with a text-dependent, one-shot learning approach. This enhancement streamlines custom retraining, significantly improving speaker verification accuracy for concise commands. Experimental results substantiate the applicability of the proposed voice-controlled HMI for assisting individuals with BPI through specialized exoskeleton gloves.

Abstract While large language models (LLMs) have shown remarkable effectiveness in various NLP tasks, they are still prone to issues such as hallucination, unfaithful reasoning, and toxicity. A promising approach to rectify these flaws is correcting LLMs with feedback, where the LLM itself is prompted or guided with feedback to fix problems in its own output. Techniques leveraging automated feedback—either produced by the LLM itself (self-correction) or some external system—are of particular interest as they make LLM-based solutions more practical and deployable with minimal human intervention. This paper provides an exhaustive review of the recent advances in correcting LLMs with automated feedback, categorizing them into training-time, generation-time, and post-hoc approaches. We also identify potential challenges and future directions in this emerging field.

Is it possible to train a general metric for evaluating text generation quality without humanannotated ratings?Existing learned metrics either perform unsatisfactorily across text generation tasks or require human ratings for training on specific tasks.In this paper, we propose SESCORE2, a self-supervised approach for training a model-based metric for text generation evaluation.The key concept is to synthesize realistic model mistakes by perturbing sentences retrieved from a corpus.The primary advantage of the SESCORE2 is its ease of extension to many other languages while providing reliable severity estimation.We evaluate SESCORE2 and previous methods on four text generation tasks across three languages.SESCORE2 outperforms unsupervised metric PRISM on four text generation evaluation benchmarks, with a Kendall improvement of 0.078.Surprisingly, SESCORE2 even outperforms the supervised BLEURT and COMET on multiple text generation tasks.The code and data are available at https://github.com/ xu1998hz/SEScore2 1 .

Building natural language inference (NLI) benchmarks that are both challenging for modern techniques, and free from shortcut biases is difficult.Chief among these biases is single sentence label leakage, where annotatorintroduced spurious correlations yield datasets where the logical relation between (premise, hypothesis) pairs can be accurately predicted from only a single sentence, something that should in principle be impossible.We demonstrate that despite efforts to reduce this leakage, it persists in modern datasets that have been introduced since its 2018 discovery.To enable future amelioration efforts, introduce a novel model-driven technique, the progressive evaluation of cluster outliers (PECO) which enables both the objective measurement of leakage, and the automated detection of subpopulations in the data which maximally exhibit it.

Automatically evaluating the quality of language generation is critical.Although recent learned metrics show high correlation with human judgement, these metrics do not provide explicit explanation of their verdict, nor associate the scores with defects in the generated text.To address this limitation, we present IN-STRUCTSCORE, a fine-grained explainable evaluation metric for text generation.By harnessing both explicit human instruction and the implicit knowledge of GPT-4, we fine-tune a text evaluation metric based on LLaMA, producing both a score for generated text and a human readable diagnostic report.We evaluate INSTRUCTSCORE on a variety of generation tasks, including translation, captioning, data-to-text, and commonsense generation.Experiments show that our 7B model surpasses all other unsupervised metrics, including those based on 175B GPT-3 and GPT-4.Surprisingly, our INSTRUCTSCORE, even without direct supervision from human-rated data, achieves performance levels on par with state-of-the-art metrics like COMET22, which were fine-tuned on human ratings.Prompt: You are evaluating a model output based on a reference.Reference: Normally the administration office downstairs would call me when there's a delivery.Output: Usually when there is takeaway, the management office downstairs will call.

This paper presents the development of an exoskeleton glove system for people who suffer from brachial plexus injuries, aiming to assist their lost grasping functionality. The robotic system consists of a portable glove system and an embedded controller. The glove system consists of Linear Series Elastic Actuators (LSEA), Rotary Series Elastic Actuators (RSEA), and optimized finger linkages to provide imitated human motion to each finger and a coupled motion of the hand. The design principles and optimization strategies were investigated to balance functionality, portability, and stability. The model-based force control strategy compensated with a backlash model and model-free force control strategy are presented and compared. Results show that our proposed model-free control method achieves the goal of accurate force control. Finally, experiments were conducted with the prototype of the developed integrated exoskeleton glove system. Results from 3 subjects with 150 trials show that our proposed exoskeleton glove system has the potential to be used as a rehabilitation device for patients.

Expert-layman text style transfer technologies have the potential to improve communication between members of scientific communities and the general public. High-quality information produced by experts is often filled with difficult jargon laypeople struggle to understand. This is a particularly notable issue in the medical domain, where layman are often confused by medical text online. At present, two bottlenecks interfere with the goal of building high-quality medical expert-layman style transfer systems: a dearth of pretrained medical-domain language models spanning both expert and layman terminologies and a lack of parallel corpora for training the transfer task itself. To mitigate the first issue, we propose a novel language model (LM) pretraining task, Knowledge Base Assimilation, to synthesize pretraining data from the edges of a graph of expert- and layman-style medical terminology terms into an LM during self-supervised learning. To mitigate the second issue, we build a large-scale parallel corpus in the medical expert-layman domain using a margin-based criterion. Our experiments show that transformer-based models pretrained on knowledge base assimilation and other well-established pretraining tasks fine-tuning on our new parallel corpus leads to considerable improvement against expert-layman transfer benchmarks, gaining an average relative improvement of our human evaluation, the Overall Success Rate (OSR), by 106%.

In this paper, we present a novel convex optimization approach to address the minimum-time speed planning problem over a fixed path with dynamic obstacle constraints and point-wise speed and acceleration constraints. The contributions of this paper are three-fold. First, we formulate the speed planning as an iterative convex optimization problem based on space discretization. Our formulation allows imposing dynamic obstacle constraints and point-wise speed and acceleration constraints simultaneously. Second, we propose a modified vertical cell decomposition method to handle dynamic obstacles. It divides the freespace into channels, where each channel represents a homotopy of free paths and defines convex constraints for dynamic obstacles. Third, we demonstrate significant improvement over previous work on speed planning for typical driving scenarios such as following, merging, and crossing.

This paper presents the development and experimental evaluation of a portable haptic exoskeleton glove system designed for people who suffer from brachial plexus injuries to restore their lost grasping functionality. The proposed glove system involves force perception, linkage-driven finger mechanism, and personalized voice control to achieve various grasping functionality requirements. The fully integrated system provides our wearable device with lightweight, portable, and comfortable characterization for grasping objects used in daily activities. Rigid articulated linkages powered by Series Elastic Actuators (SEAs) with slip detection on the fingertips provide stable and robust grasp for multiple objects. The passive abduction-adduction motion of each finger is also considered to provide better grasping flexibility for the user. The continuous voice control with bio-authentication also provides a hands-free user interface. The experiments with different objects verify the functionalities and capabilities of the proposed exoskeleton glove system in grasping objects with various shapes and weights used in activities of daily living (ADLs).

This paper presents the design and development of a novel, low-profile, exoskeleton robotic glove aimed for people who suffer from brachial plexus injuries to restore their lost grasping functionality. The key idea of this new glove lies in its new finger mechanism that takes advantage of the rigid coupling hybrid mechanism (RCHM) concept. This mechanism concept couples the motions of the adjacent human finger links using rigid coupling mechanisms so that the overall mechanism motion (e.g., bending, extension, etc.) could be achieved using fewer actuators. The finger mechanism utilizes the single degree of freedom case of the RCHM that uses a rack-and-pinion mechanism as the rigid coupling mechanism. This special arrangement enables to design each finger mechanism of the glove as thin as possible while maintaining mechanical robustness simultaneously. Based on this novel finger mechanism, a two-finger low-profile robotic glove was developed. Remote center of motion mechanisms were used for the metacarpophalangeal (MCP) joints. Kinematic analysis and optimization-based kinematic synthesis were conducted to determine the design parameters of the new glove. Passive abduction/adduction joints were considered to improve the grasping flexibility. A proof-of-concept prototype was built and pinch grasping experiments of various objects were conducted. The results validated the mechanism and the mechanical design of the new robotic glove and demonstrated its functionalities and capabilities in grasping objects with various shapes and weights that are used in activities of daily living (ADLs).

Trajectory planning in dynamic environments can be decomposed into two sub-problems: 1) planning a path to avoid static obstacles, 2) then planning a speed profile to avoid dynamic obstacles. This is also called path-speed decomposition. In this work, we present a novel approach to solve the first sub-problem, motion planning with static obstacles. From an optimization perspective, motion planning for autonomous vehicles can be viewed as non-convex constrained nonlinear optimization, which requires a good enough initial guess to start and is often sensitive to algorithm parameters. We formulate motion planning as convex spline optimization. The convexity of the formulated problem makes it able to be solved fast and reliably, while guaranteeing a global optimum. We then reorganize the constrained spline optimization into a recurrent formulation, which further reduces the computational time to be linear in the optimization horizon size. The proposed method can be applied to both trajectory generation and motion planning problems. Its effectiveness is demonstrated in challenging scenarios such as tight lane changes and sharp turns.

The detection of surrounding vehicles is an essential task in autonomous driving, which has been drawing enormous attention recently. When using laser scanners, L-Shape fitting is a key step for model-based vehicle detection and tracking, which requires thorough investigation and comprehensive research. In this paper, we formulate the L-Shape fitting as an optimization problem. An efficient search based method is then proposed to find the optimal solution. Our method does not rely on laser scan sequence information and therefore supports convenient data fusion from multiple laser scanners; it is efficient and involves very few parameters for tuning; the approach is also flexible to suit various fitting demands with different fitting criteria. On-road experiments with production-grade laser scanners have demonstrated the effectiveness and robustness of our approach.

We propose a robust object tracking algorithm for distance keeping. Taking advantage of a context-based region of interest, we are able to maximize the performance of each sensor, and reduce the computation time since we only focus on the targets inside the region. Tracking targets in road coordinates enables finding the distance-keeping target on any curved road, while a commercial Adaptive Cruise Control (ACC) system works best on straight roads. We demonstrate that the overall performance of the proposed algorithm is better than that of a commercial ACC system. The distance-keeping target can either be used for lane following for a standalone ACC system or an autonomous vehicle. Our object tracking algorithm can also be extended to find the target of interest for lane changing or ramp merging for an autonomous vehicle.

We present a motion planning framework for autonomous on-road driving considering both the uncertainty caused by an autonomous vehicle and other traffic participants. The future motion of traffic participants is predicted using a local planner, and the uncertainty along the predicted trajectory is computed based on Gaussian propagation. For the autonomous vehicle, the uncertainty from localization and control is estimated based on a Linear-Quadratic Gaussian (LQG) framework. Compared with other safety assessment methods, our framework allows the planner to avoid unsafe situations more efficiently, thanks to the direct uncertainty information feedback to the planner. We also demonstrate our planner's ability to generate safer trajectories compared to planning only with a LQG framework.

In this paper, an efficient real-time autonomous driving motion planner with trajectory optimization is proposed. The planner first discretizes the plan space and searches for the best trajectory based on a set of cost functions. Then an iterative optimization is applied to both the path and speed of the resultant trajectory. The post-optimization is of low computational complexity and is able to converge to a higher-quality solution within a few iterations. Compared with the planner without optimization, this framework can reduce the planning time by 52% and improve the trajectory quality. The proposed motion planner is implemented and tested both in simulation and on a real autonomous vehicle in three different scenarios. Experiments show that the planner outputs high-quality trajectories and performs intelligent driving behaviors.

In order to improve energy efficiency of transport systems, eco-driving strategies are studied world-widely. However, most literatures on eco-driving based on traditional traffic flow models, are greatly simplified, and can not evaluate the effects on detailed driving behaviors. By referring to robot motion planning approaches, in this research a microscopic vehicle model is developed and it can represent different driving behaviors, such as aggressive or conservative driving; a collision detection algorithm is proposed that takes O(1) time to check for a trajectory's collision, enabling realtime planning; and a traffic simulation system is developed by incorporating traffic rules, so that the driving behaviors such as observing or not observing traffic rules can also be represented. Experiments are conducted on the simulation platform, and the performance of different driving behaviors on travel time, mileage, comfort and eco is studied.