Off-policy evaluation (OPE) estimates the performance of a target policy using offline data collected from a behavior policy, and is crucial in domains such as robotics or healthcare where direct interaction with the environment is costly or unsafe. Existing OPE methods are ineffective for high-dimensional, long-horizon problems, due to exponential blow-ups in variance from importance weighting or compounding errors from learned dynamics models. To address these challenges, we propose STITCH-OPE, a model-based generative framework that leverages denoising diffusion for long-horizon OPE in high-dimensional state and action spaces. Starting with a diffusion model pre-trained on the behavior data, STITCH-OPE generates synthetic trajectories from the target policy by guiding the denoising process using the score function of the target policy. STITCH-OPE proposes two technical innovations that make it advantageous for OPE: (1) prevents over-regularization by subtracting the score of the behavior policy during guidance, and (2) generates long-horizon trajectories by stitching partial trajectories together end-to-end. We provide a theoretical guarantee that under mild assumptions, these modifications result in an exponential reduction in variance versus long-horizon trajectory diffusion. Experiments on the D4RL and OpenAI Gym benchmarks show substantial improvement in mean squared error, correlation, and regret metrics compared to state-of-the-art OPE methods.

While vision-language-action models (VLAs) have shown promising robotic behaviors across a diverse set of manipulation tasks, they achieve limited success rates when deployed on novel tasks out of the box. To allow these policies to safely interact with their environments, we need a failure detector that gives a timely alert such that the robot can stop, backtrack, or ask for help. However, existing failure detectors are trained and tested only on one or a few specific tasks, while generalist VLAs require the detector to generalize and detect failures also in unseen tasks and novel environments. In this paper, we introduce the multitask failure detection problem and propose SAFE, a failure detector for generalist robot policies such as VLAs. We analyze the VLA feature space and find that VLAs have sufficient high-level knowledge about task success and failure, which is generic across different tasks. Based on this insight, we design SAFE to learn from VLA internal features and predict a single scalar indicating the likelihood of task failure. SAFE is trained on both successful and failed rollouts and is evaluated on unseen tasks. SAFE is compatible with different policy architectures. We test it on OpenVLA, $\pi_0$, and $\pi_0$-FAST in both simulated and real-world environments extensively. We compare SAFE with diverse baselines and show that SAFE achieves state-of-the-art failure detection performance and the best trade-off between accuracy and detection time using conformal prediction. More qualitative results and code can be found at the project webpage: https://vla-safe.github.io/

Imitation Learning (IL) is a widely used framework for learning imitative behavior from demonstrations. It is especially appealing for solving complex real-world tasks where handcrafting reward function is difficult, or when the goal is to mimic human expert behavior. However, the learned imitative policy can only follow the behavior in the demonstration. When applying the imitative policy, we may need to customize the policy behavior to meet different requirements coming from diverse downstream tasks. Meanwhile, we still want the customized policy to maintain its imitative nature. To this end, we formulate a new problem setting called policy customization. It defines the learning task as training a policy that inherits the characteristics of the prior policy while satisfying some additional requirements imposed by a target downstream task. We propose a novel and principled approach to interpret and determine the trade-off between the two task objectives. Specifically, we formulate the customization problem as a Markov Decision Process (MDP) with a reward function that combines 1) the inherent reward of the demonstration; and 2) the add-on reward specified by the downstream task. We propose a novel framework, Residual Q-learning, which can solve the formulated MDP by leveraging the prior policy without knowing the inherent reward or value function of the prior policy. We derive a family of residual Q-learning algorithms that can realize offline and online policy customization, and show that the proposed algorithms can effectively accomplish policy customization tasks in various environments. Demo videos and code are available on our website: https://sites.google.com/view/residualq-learning.

This paper presents a novel online framework for safe crowd-robot interaction based on risk-sensitive stochastic optimal control, wherein the risk is modeled by the entropic risk measure. The sampling-based model predictive control relies on mode insertion gradient optimization for this risk measure as well as Trajectron++, a state-of-the-art generative model that produces multimodal probabilistic trajectory forecasts for multiple interacting agents. Our modular approach decouples the crowd-robot interaction into learning-based prediction and model-based control, which is advantageous compared to end-to-end policy learning methods in that it allows the robot's desired behavior to be specified at run time. In particular, we show that the robot exhibits diverse interaction behavior by varying the risk sensitivity parameter. A simulation study and a real-world experiment show that the proposed online framework can accomplish safe and efficient navigation while avoiding collisions with more than 50 humans in the scene.

In this paper, we consider motion as a means of sending messages between robots. We focus on a scenario in which a message is encoded in a sending robot's trajectory, and decoded by a receiver robot equipped with a monocular camera. The relative pose between the robots is unknown. We introduce an online Bayesian estimation algorithm based on the Multi-hypothesis Extended Kalman Filter for the receiving robot to simultaneously estimate its relative pose to the sender, and the trajectory class of the sender. The difficulty in this problem arises from the monocular vision model of the receiver and the unknown relative pose between robots, which brings inherent ambiguity into the trajectory identification, and hence the message decoding. An active vision-based control policy is derived and combined with the Bayesian estimation in order to deal with this difficulty. The policy is constructed online based on Monte Carlo Tree Search and aims at reducing the entropy over the trajectory class distribution. The algorithm has broad applications, e.g., to intent modeling and motion prediction for autonomous driving and autonomous drone operations. Simulation results demonstrate that the proposed estimation algorithm and the control policy result in an accurate trajectory classification.

In WLANs, collision-free communication is desired for real-time wireless video transmission (WVT) systems. A polling-based access scheme enables quality of service (QoS) and priority control over legacy WLAN stations (STAs). However, when multiple moving basic service sets (BSSs) are operating nearby in a polling-based access scheme, signal collisions between BSSs occur, and system performance of the WVT system degrades. Furthermore, the BSSs movements cause difficulties in arranging an access controller (AC) that manages the polling periods of multiple BSSs to avoid the signal collisions. This paper describes an adaptive access point (AP) cooperation scheme where each AP autonomously controls its polling period according to dynamic channel conditions. A network simulation using OPNET® demonstrates that the proposed scheme can enable the coexistence of up to five APs with a packet loss rate (PLR) of less than 10-6 and a delay of less than 50 ms for real-time communications.

Fractional sampling (FS) has been investigated to achieve path diversity with a single antenna in an OFDM system. However, in the FS-OFDM system, it is necessary to oversample a received signal and it leads to large power consumption in a small terminal. To shift the signal processing burdens for FS from a receiver to a transmitter, a precoded transmit path diversity scheme in OFDM has been proposed. It achieves path diversity without oversampling the received signal at the receiver side on time invariant channels. However, it has not taken time variant channels into consideration. On time variant channels, inter-carrier interference (ICI) deteriorates the bit-error-rate (BER) performance. In this study, the precoding for both ICI suppression and path diversity combining at the transmitter side has been proposed. Computer simulation shows that both the conventional scheme and the proposed scheme can achieve transmit path diversity. On a two path Rayleigh fading channel, the BER performance of the proposed scheme is better than that of the conventional scheme by about 1 dB. On ultra wideband (UWB) channels, the performance of the proposed scheme is 7–10 dB better at BER=10−2 than that of the conventional scheme. It can achieve ICI suppression and path diversity without any specific signal processing at the receiver. The effect of the ICI suppression is more significant as the number of the multipath is larger.

A diversity scheme with Fractional Sampling (FS) in OFDM receivers has been investigated recently. In the previous literature it has been shown that performance improvement with path diversity through FS depends on the bandwidth of the total channel response including transmit and receive baseband filters. However, if the signal on the adjacent channel exists, that signal causes interference to the desired signal with FS. In this paper, an adjacent channel interference cancellation scheme has been proposed for the FS OFDM receiver. The proposed scheme demodulates the signal on the adjacent channel and regenerates it to be subtracted from the received signal. Numerical results through computer simulation show that the proposed scheme achieves BER improvement under low signal-to-interference-ratio (SIR) conditions.

To achieve diversity with a single antenna, a fractional sampling (FS) scheme in OFDM has been proposed. FS achieves path diversity with a higher sampling rate and multiple demodulation branches. In the previous literatures for FS, the sampling interval is fixed even though the initial phase selection of the sampling has been investigated. The fixed sampling interval limits the diversity gain through FS. Thus, in this study, a non-uniform sampling point selection scheme according to the frequency response of a channel is proposed. Numerical results through computer simulation show that the proposed scheme improves the BER performance by about 2 dB when the number of the demodulation branches is 2. While the proposed scheme further improves bit error rate (BER) performance it requires a larger rate and more computational complexity for the selection of the sampling points. Therefore the complexity reduction for the sampling point selection is also investigated. The complexity reduction scheme eliminates the specific sets of the sampling points and reduces the computational complexity of the sampling point selection for the proposed scheme by a factor of 2. Computer simulation also shows that the low complexity version of the proposed scheme even achieves the equivalent BER compared to the original proposed scheme.

To achieve diversity with a single antenna, a fractional sampling (FS) scheme in OFDM has been investigated. In addition, to obtain more diversity gain through FS, a sampling point selection (SPS) scheme has been proposed. However, it requires a large number of multiplication operations for the SPS. This leads to large power consumption and costs for additional circuits. In this paper, a low-complexity SPS scheme is proposed. There is the trade-off between the diversity gain and the estimation accuracy. The serial process of the channel estimation requires only a DFT block with the size of N/4 as the additional circuit.

Fractional sampling (FS) and Doppler diversity equalization in OFDM receivers can achieve two types of diversity (path diversity and Doppler diversity) simultaneously on time-varying multipath channels. However FS with a higher sampling rate requires the large amount of complexity in demodulation. In this paper, a novel sampling point selection (SPS) scheme with MMSE equalization in FS-OFDM receivers is proposed. On fast time-varying multipath channels, the proposed scheme selects the appropriate samples from the fractionally sampled signals. Through the computer simulation, it is demonstrated that with the proposed scheme, both path diversity gain and Doppler diversity gain can increase as compared to a conventional non-SPS scheme.

Orthogonal frequency and code division multiplexing (OFCDM) has received large attention as a modulation scheme to realise high data rate transmission. The OFCDM system with fractional sampling (FS) is investigated. FS is a diversity scheme with a single antenna, which achieves path diversity through oversampling and parallel signal reception. However, the correlation among noise components may deteriorate the bit error rate performance at the receiver as the number of subcarriers and oversampling ratio increases. To overcome this problem, the alternative spreading code is used for the FS OFCDM system. Numerical results through computer simulation show that the proposed scheme can improve the performance of the receiver with the large oversampling ratio and the number of subcarriers.

In the IEEE 802.11a WLAN standard, orthogonal frequency division multiplexing (OFDM) modulation is employed. Diversity techniques are effective way to overcome multipath fading in the OFDM systems. A fractional sampling (FS) scheme is one of the diversity techniques with a single antenna. Through FS it is possible to separate multipath components and obtain diversity gain in OFDM. Though FS has been analysed numerically, the experiment of FS on actual indoor channels with real filter characteristics has not been carried out. In addition, though it has been suggested that in FS diversity a sharp filter limits the diversity gain, this limitation has not been confirmed especially with measurements. Therefore we investigate the effect of filter bandwidth in FS through simulation and an experiment. Numerical results through the simulation and the experiment indicate that the FS can achieve path diversity and improves the performance by about 2 dB if the bandwidth of the baseband filter is larger than the minimal bandwidth.

In this paper, a metric weighting scheme on a coded fractional sampling orthogonal frequency division multiplexing (FS OFDM) system is investigated. FS achieves path diversity with a single antenna through oversampling and subcarrier-based maximal ratio combining (MRC). Though the oversampling increases diversity order, correlation among noise components may deteriorate bit error rate (BER) performance. To clarify the relationship between the impulse response of the pulse shaping filter and the BER performance, five different pulse shaping filters are evaluated in the coded FS OFDM system. Numerical results obtained through computer simulation show that the metric weighting based on the Frobenius norm improves BER performance of the coded FS OFDM system.

A diversity scheme with fractional sampling (FS) in OFDM receivers is investigated recently. When a sharp filter is employed, the correlation of noise samples among the adjacent subcarriers increases, and the performance of this scheme is deteriorated. Therefore, low-complexity subcarrier based MMSE combining scheme with the subblock of the noise covariance matrix is proposed in this paper. Numerical results through computer simulation show that the MMSE combining scheme proposed in this paper can outperforms a conventional MRC scheme and a subcarrier based MMSE combining scheme, when the sharp filter is employed.

Through fractional sampling, it is possible to separate multi- path components and achieve diversity. However, power consumption grows as the sampling rate increases. In this paper, a novel sampling rate selection scheme in OFDM according to multipath environments is proposed. Numerical results through computer simulation show that the proposed sampling rate selection scheme reduces power consumption by decreasing the sampling rate when delay spread is small.