This paper introduces a novel application of Supervised Contrastive Learning (SupCon) to Imitation Learning (IL), with a focus on learning more effective state representations for agents in video game environments. The goal is to obtain latent representations of the observations that capture better the action-relevant factors, thereby modeling better the cause-effect relationship from the observations that are mapped to the actions performed by the demonstrator, for example, “the player jumps whenever an obstacle appears ahead.” We propose an approach to integrate the SupCon loss with continuous output spaces, enabling SupCon to operate without constraints regarding the type of actions of the environment. Experiments on the 3D games Astro Bot and Returnal, and multiple 2D Atari games show improved representation quality, faster learning convergence, and better generalization compared to baseline models trained only with supervised action prediction loss functions.

We introduce HammerDrive, a novel architecture for task-aware visual attention prediction in driving. The proposed architecture is learnable from data and can reliably infer the current focus of attention of the driver in real-time, while only requiring limited and easy-to-access telemetry data from the vehicle. We build the proposed architecture on two core concepts: 1) driving can be modeled as a collection of sub-tasks (maneuvers), and 2) each sub-task affects the way a driver allocates visual attention resources, i.e., their eye gaze fixation. HammerDrive comprises two networks: a hierarchical monitoring network of forward-inverse model pairs for sub-task recognition and an ensemble network of task-dependent convolutional neural network modules for visual attention modeling. We assess the ability of HammerDrive to infer driver visual attention on data we collected from 20 experienced drivers in a virtual reality-based driving simulator experiment. We evaluate the accuracy of our monitoring network for sub-task recognition and show that it is an effective and light-weight network for reliable real-time tracking of driving maneuvers with above 90% accuracy. Our results show that HammerDrive outperforms a comparable state-of-the-art deep learning model for visual attention prediction on numerous metrics with ~13% improvement for both Kullback-Leibler divergence and similarity, and demonstrate that task-awareness is beneficial for driver visual attention prediction.

We consider the problem of imitation learning from a finite set of expert trajectories, without access to reinforcement signals. The classical approach of extracting the expert’s reward function via inverse reinforcement learning, followed by reinforcement learning is indirect and may be computationally expensive. Recent generative adversarial methods based on matching the policy distribution between the expert and the agent could be unstable during training. We propose a new framework for imitation learning by estimating the support of the expert policy to compute a fixed reward function, which allows us to re-frame imitation learning within the standard reinforcement learning setting. We demonstrate the efficacy of our reward function on both discrete and continuous domains, achieving comparable or better performance than the state of the art under different reinforcement learning algorithms.

Classifying human cognitive states from behavioral and physiological signals is a challenging problem with important applications in robotics. The problem is challenging due to the data variability among individual users, and sensor artefacts. In this work, we propose an end-to-end framework for real-time cognitive workload classification with mixture Hyper Long Short Term Memory Networks (m-HyperLSTM), a novel variant of HyperNetworks. Evaluating the proposed approach on an eye-gaze pattern dataset collected from simulated driving scenarios of different cognitive demands, we show that the proposed framework outperforms previous baseline methods and achieves 83.9% precision and 87.8% recall during test. We also demonstrate the merit of our proposed architecture by showing improved performance over other LSTM-based methods.

In this letter, we propose a novel manifold-based algorithm to solve the constant envelope (CE) precoding problem with interference exploitation. For a given power budget, we design the precoded symbols subject to the CE constraints, such that the constructive effect of the multiuser interference is maximized. While the objective function for the original problem is not complex differentiable, we consider the smooth approximation of its real representation, and map it onto a Riemannian manifold. By using the Riemmanian conjugate gradient algorithm, a local minimizer can be efficiently found. The complexity of the algorithm is analytically derived in terms of floating-points operations (flops) per iteration. Simulations show that the proposed algorithm outperforms the conventional methods on both symbol error rate and computational complexity.

We propose several low-complexity transmit antenna selection (TAS) and precoding schemes for massive multi-input multi-output (M-MIMO). It is well established that large antenna arrays in M-MIMO lead to particularly high hardware overheads as they require an equally large number of radio-frequency chains, and antenna selection is envisaged as a solution to reducing this hardware complexity. Accordingly, in the proposed schemes, both hardware and computational complexity of M-MIMO systems are addressed by jointly optimizing TAS and precoding. We first introduce a mixed-integer programming approach that simultaneously identifies the transmitting antennas subset and solves the precoding problem, by employing a unified metric based on constructive interference (CI) concept. We then propose three sub-optimal techniques that allow a reduction of the computational complexity required to solve the joint optimization. Our analyses and results prove that the proposed joint TAS and precoding schemes based on CI exploitation are able to outperform the state-of-the-art, while providing a favorable performance-complexity tradeoff.

We introduce a new approach to constant-envelope precoding (CEP) based on an interference-driven optimization region for generic phase-shift-keying modulations in the multi-user (MU) multiple-input-multiple-output downlink. While conventional precoding approaches aim to minimize the multi-user interference (MUI) with a total sum-power constraint at the transmitter, in the proposed scheme we consider MUI as a source of additional energy to increase the signal-to-interference-and-noise-ratio at the receiver. In our studies, we focus on two different CEP approaches: a first technique, where the power at each antenna is fixed to a specific value, and a two-step approach, where we first relax the power constraints to be lower than a defined parameter and then enforce CEP transmission. The algorithms are studied in terms of computational costs, with a detailed comparison between the proposed approach and the classical interference suppression schemes from the literature. Moreover, we analytically derive a robust optimization region to counteract the effects of channel-state estimation errors. The presented schemes are evaluated in terms of achievable symbol error rate in a perfect and imperfect channel-state information scenario for different modulation orders. Our results show that the proposed techniques further extend the benefits of classical CEP by judiciously relaxing the optimization region.

Low-complexity linear precoders are known to be asymptotically optimal in massive multi-input multi-output (MMIMO) systems. However, employing a very large set of antennas at the transmitter imposes an equally large number of radiofrequency (RF) chains. As a consequence, a MIMO approach for massive systems is prohibitive, as it leads to very high computational burdens and high hardware complexity at the transmitter. In this paper, a low complexity antenna selection (AS) scheme based on constructive interference is presented to reduce hardware complexity. We show that the proposed AS algorithm, combined with a simple matched filter (MF) precoder, outperforms more complex and computationally expensive AS techniques that involve channel inversion (CI) precoding. We derive the computational burdens of the proposed technique and analyze the benefits through the definition of a new power efficiency metric, that combines performances in terms of throughput with the circuit power required.

Communications in millimeter-wave (mm-wave) spectrum (30-300 GHz) have experienced a continuous increase in relevance for short-range, high-capacity wireless links, because of the wider bandwidths they are able to provide. In this work, we introduce a new mm-wave frequency transmission scheme that exploits a combination of the concepts of beamspace multi-input multi-output (B-MIMO) communications and beam selection to provide near-optimal performances with a low hardware-complexity transceiver. While large-scale MIMO approaches in mm-wave are affected by high dimensional signal space that increases considerably both complexity and costs of the system, the proposed scheme is able to achieve near-optimal performances with a reduced radio-frequency (RF) complexity thanks to beam selection. We evaluate the advantages of the proposed scheme via capacity computations, comparisons of numbers of RF chains required and by studying the trade-off between spectral and power efficiency. Our analytical and simulation results show that the proposed scheme is capable of offering a significant reduction in RF complexity with a realistic low-cost approach, for a given performance. In particular, we show that the proposed beam selection algorithms achieve higher power efficiencies than a full system where all beams are utilized.

We present a new millimeter-wave (mm-wave) transmission scheme that combines beamspace Multi-Input Multi-Output (B-MIMO) communications and beam selection techniques to achieve near-optimal performances with a low hardware-complexity transceiver. The use of mm-wave frequencies in MIMO systems is afflicted by high complexity and costs, caused by the high dimensional signal space. The proposed scheme is characterized by a reduced radio-frequency (RF) complexity, achieved through beam selection, while leading to near-optimal performances. The benefits of the proposed scheme are evaluated via performance computations, comparisons of numbers of required RF chains and a joint sum-rate and complexity metric that shows the trade-off between spectral and power efficiency. The analytical and numerical results presented in the paper show that the proposed scheme offers an improved power efficiency with respect to the full system with a realistic low-cost approach.