Deep neural network (DNN)-based policy models, such as vision-language-action (VLA) models, excel at automating complex decision-making from multi-modal inputs. However, scaling these models greatly increases computational overhead, complicating deployment in resource-constrained settings like robot manipulation and autonomous driving. To address this, we propose Saliency-Aware Quantized Imitation Learning (SQIL), which combines quantization-aware training with a selective loss-weighting strategy for mission-critical states. By identifying these states via saliency scores and emphasizing them in the training loss, SQIL preserves decision fidelity under low-bit precision. We validate SQIL's generalization capability across extensive simulation benchmarks with environment variations, real-world tasks, and cross-domain tasks (self-driving, physics simulation), consistently recovering full-precision performance. Notably, a 4-bit weight-quantized VLA model for robotic manipulation achieves up to 2.5x speedup and 2.5x energy savings on an edge GPU with minimal accuracy loss. These results underline SQIL's potential for efficiently deploying large IL-based policy models on resource-limited devices.

Recent works on machine learning for combinatorial optimization have shown that learning based approaches can outperform heuristic methods in terms of speed and performance. In this paper, we consider the problem of finding an optimal topological order on a directed acyclic graph (DAG) with focus on the memory minimization problem which arises in compilers. We propose an end-to-end machine learning based approach for topological ordering using an encoder-decoder framework. Our encoder is a novel attention based graph neural network architecture called \emph{Topoformer} which uses different topological transforms of a DAG for message passing. The node embeddings produced by the encoder are converted into node priorities which are used by the decoder to generate a probability distribution over topological orders. We train our model on a dataset of synthetically generated graphs called layered graphs. We show that our model outperforms, or is on-par, with several topological ordering baselines while being significantly faster on synthetic graphs with up to 2k nodes. We also train and test our model on a set of real-world computation graphs, showing performance improvements.

We consider local kernel metric learning for off-policy evaluation (OPE) of deterministic policies in contextual bandits with continuous action spaces. Our work is motivated by practical scenarios where the target policy needs to be deterministic due to domain requirements, such as prescription of treatment dosage and duration in medicine. Although importance sampling (IS) provides a basic principle for OPE, it is ill-posed for the deterministic target policy with continuous actions. Our main idea is to relax the target policy and pose the problem as kernel-based estimation, where we learn the kernel metric in order to minimize the overall mean squared error (MSE). We present an analytic solution for the optimal metric, based on the analysis of bias and variance. Whereas prior work has been limited to scalar action spaces or kernel bandwidth selection, our work takes a step further being capable of vector action spaces and metric optimization. We show that our estimator is consistent, and significantly reduces the MSE compared to baseline OPE methods through experiments on various domains.

We consider the offline reinforcement learning (RL) setting where the agent aims to optimize the policy solely from the data without further environment interactions. In offline RL, the distributional shift becomes the primary source of difficulty, which arises from the deviation of the target policy being optimized from the behavior policy used for data collection. This typically causes overestimation of action values, which poses severe problems for model-free algorithms that use bootstrapping. To mitigate the problem, prior offline RL algorithms often used sophisticated techniques that encourage underestimation of action values, which introduces an additional set of hyperparameters that need to be tuned properly. In this paper, we present an offline RL algorithm that prevents overestimation in a more principled way. Our algorithm, OptiDICE, directly estimates the stationary distribution corrections of the optimal policy and does not rely on policy-gradients, unlike previous offline RL algorithms. Using an extensive set of benchmark datasets for offline RL, we show that OptiDICE performs competitively with the state-of-the-art methods.

Monte-Carlo Tree Search (MCTS) is the state-of-the-art online planning algorithm for large problems with discrete action spaces. However, many real-world problems involve continuous action spaces, where MCTS is not as effective as in discrete action spaces. This is mainly due to common practices such as coarse discretization of the entire action space and failure to exploit local smoothness. In this paper, we introduce Value-Gradient UCT (VG-UCT), which combines traditional MCTS with gradient-based optimization of action particles. VG-UCT simultaneously performs a global search via UCT with respect to the finitely sampled set of actions and performs a local improvement via action value gradients. In the experiments, we demonstrate that our approach outperforms existing MCTS methods and other strong baseline algorithms for continuous action spaces.

Adversarial Imitation Learning alternates between learning a discriminator -- which tells apart expert's demonstrations from generated ones -- and a generator's policy to produce trajectories that can fool this discriminator. This alternated optimization is known to be delicate in practice since it compounds unstable adversarial training with brittle and sample-inefficient reinforcement learning. We propose to remove the burden of the policy optimization steps by leveraging a novel discriminator formulation. Specifically, our discriminator is explicitly conditioned on two policies: the one from the previous generator's iteration and a learnable policy. When optimized, this discriminator directly learns the optimal generator's policy. Consequently, our discriminator's update solves the generator's optimization problem for free: learning a policy that imitates the expert does not require an additional optimization loop. This formulation effectively cuts by half the implementation and computational burden of Adversarial Imitation Learning algorithms by removing the Reinforcement Learning phase altogether. We show on a variety of tasks that our simpler approach is competitive to prevalent Imitation Learning methods.

Stochastic variational inference has emerged as an effective method for performing inference on or learning complex models for data. Yet, one of the challenges in stochastic variational inference is handling high-dimensional data, such as sequential data, and models with non-differentiable densities caused by, for instance, the use of discrete latent variables. In such cases, it is challenging to control the variance of the gradient estimator used in stochastic variational inference, while low variance is often one of the key properties needed for successful inference. In this work, we present a new algorithm for stochastic variational inference of sequential models which trades off bias for variance to tackle this challenge effectively. Our algorithm is inspired by variance reduction techniques in reinforcement learning, yet it uniquely adopts their key ideas in the context of stochastic variational inference. We demonstrate the effectiveness of our approach through formal analysis and experiments on synthetic and real-world datasets.

Generative adversarial training for imitation learning has shown promising results on high-dimensional and continuous control tasks. This paradigm is based on reducing the imitation learning problem to the density matching problem, where the agent iteratively refines the policy to match the empirical state-action visitation frequency of the expert demonstration. Although this approach has shown to robustly learn to imitate even with scarce demonstration, one must still address the inherent challenge that collecting trajectory samples in each iteration is a costly operation. To address this issue, we first propose a Bayesian formulation of generative adversarial imitation learning (GAIL), where the imitation policy and the cost function are represented as stochastic neural networks. Then, we show that we can significantly enhance the sample efficiency of GAIL leveraging the predictive density of the cost, on an extensive set of imitation learning tasks with high-dimensional states and actions.

In this paper, the capacity of continuous-space electromagnetic channels is analyzed. We assume the regions confining the transceivers are filled with dielectric that is either lossy or lossless and show how the capacity is affected by the physical loss of the source region. Our analysis is based on calculation of exact power consumption in the presence of mutual coupling and accurate modeling of noise based on fluctuation-dissipation theorem. We also analytically derive the Q factor and show how the capacity is affected by the bandwidth constraint. Finally, we compare our work with the existing literature and analyze the maximum gain of spherical dielectric antennas in practical applications.

Recently in tandem with the spread of portable devices for reading electronic books, devices for digitizing paper books, called book scanners, are developed to meet the increased demand for digitizing privately owned books. However, conventional book scanners still have complex components to mechanically turn pages and to rectify the acquired images that are inevitably distorted by the curvy book surface. Here, we present the multi-scale mechanism that turns pages electronically using electroadhesive force generated by a micro-scale structure. Its another advantage is that perspective correction of image processing is applicable to readily reconstruct the distorted images of pages. Specifically, to turn one page at a time not two pages, we employ a micro-scale structure to generate near-field electroadhesive force that decays rapidly and accordingly attracts objects within tens of micrometers. We analyze geometrical parameters of the micro-scale structure to improve the decay characteristics. We find that the decay characteristics of electroadhesive force definitely depends upon the geometrical period of the micro-scale structure, while its magnitude depends on a variety of parameters. Based on this observation, we propose a novel electrode configuration with improved decay characteristics. Dynamical stability and kinematic requirements are also examined to successfully introduce near-field electroadhesive force into our digitizing process.

In this paper, we show the spatial degrees of freedom (DoF) in free space can be improved by using superdirectivity and such a gain is theoretically unlimited regardless of the physical size of the transmit and receive antenna arrays. Our conclusion is based on calculating the exact power consumption in the presence of mutual coupling and on calculating the exact spatial correlation of the noise field using the fluctuation dissipation theorem. As a result, high current excitation and high signal-to-noise ratio are available at the transmitter and the receiver, respectively. They were not believed to be possible in many previous works that did not consider such exact behaviors. We also analyze the effect of scattering in the near field region and discuss some solutions for many practical problems arising in implementing superdirectivity.

In this paper, the capacity of wireless channels is characterized based on electromagnetic and antenna theories with only minimal assumptions. We assume the transmitter can generate an arbitrary current distribution inside a spherical region and the receive antennas are uniformly distributed on a bigger sphere surrounding the transmitter. The capacity is shown to be (αP/N0) log e [bits/sec] in the limit of large number of receive antennas, where P is the transmit power constraint, a is the normalized density of the receive antennas and N0is the noise power spectral density. Although this result may look trivial, it is surprising in two ways. First, this result holds regardless of the bandwidth (bandwith can even be negligibly small). Second, this result shows that the capacity is irrespective of the size of the region containing the transmitter. This is against some previous results that claimed the maximum degrees of freedom is proportional to the surface area containing the transmitter normalized by the square of the wavelength. Our result has important practical implications since it shows that even a compact antenna array with negligible bandwidth and antenna spacing well below the wavelength can provide a huge throughput as if the array was big enough so that the antenna spacing is on the order of the wavelength.