While autonomous driving technology has made remarkable strides, data-driven approaches still struggle with complex scenarios due to their limited reasoning capabilities. Meanwhile, knowledge-driven autonomous driving systems have evolved considerably with the popularization of visual language models. In this article, we propose LeapVAD, a novel method based on cognitive perception and dual-process thinking. Our approach implements a human-attentional mechanism to identify and focus on critical traffic elements that influence driving decisions. By characterizing these objects through comprehensive attributes-including appearance, motion patterns, and associated risks-LeapVAD achieves more effective environmental representation and streamlines the decision-making process. Furthermore, LeapVAD incorporates an innovative dual-process decision-making module mimicking the human-driving learning process. The system consists of an analytic process (System-II) that accumulates driving experience through logical reasoning and a heuristic process (System-I) that refines this knowledge via fine-tuning and few-shot learning. LeapVAD also includes reflective mechanisms and a growing memory bank, enabling it to learn from past mistakes and continuously improve its performance in a closed-loop environment. To enhance efficiency, we develop a scene encoder network that generates compact scene representations for rapid retrieval of relevant driving experiences. Extensive evaluations conducted on two leading autonomous driving simulators, CARLA and DriveArena, demonstrate that LeapVAD achieves superior performance compared with camera-only approaches despite limited training data. Comprehensive ablation studies further emphasize its effectiveness in continuous learning and domain adaptation. Project page: https://pjlab-adg.github.io/LeapVAD/.

This paper presented DriveArena, the first high-fidelity closed-loop simulation system designed for driving agents navigating in real scenarios. DriveArena features a flexible, modular architecture, allowing for the seamless interchange of its core components: Traffic Manager, a traffic simulator capable of generating realistic traffic flow on any worldwide street map, and World Dreamer, a high-fidelity conditional generative model with infinite autoregression. This powerful synergy empowers any driving agent capable of processing real-world images to navigate in DriveArena's simulated environment. The agent perceives its surroundings through images generated by World Dreamer and output trajectories. These trajectories are fed into Traffic Manager, achieving realistic interactions with other vehicles and producing a new scene layout. Finally, the latest scene layout is relayed back into World Dreamer, perpetuating the simulation cycle. This iterative process fosters closed-loop exploration within a highly realistic environment, providing a valuable platform for developing and evaluating driving agents across diverse and challenging scenarios. DriveArena signifies a substantial leap forward in leveraging generative image data for the driving simulation platform, opening insights for closed-loop autonomous driving. Code will be available soon on GitHub: https://github.com/PJLab-ADG/DriveArena

Robot programming is still an elaborate process in industrial assembly, especially in cases requiring a specific force profile for successfully executing an assembly step. In this work, we present the Haptic Knob, a low-cost haptic device (around 100 EUR on hardware costs) for (currently) one-dimensional robot movement programming on both position and force profiles. The Haptic Knob is equipped with a Field-Oriented Controlled (FOC) motor and a positional encoder that reads the human input position and generates feedback forces to the operator. The Haptic Knob is not equipped with an external Force/Torque sensor to measure the human input force, but works together with the impedance controller of the robot arm, which allows a dynamic force teach-in. A control interface is implemented to map the positional signal from the Haptic Knob to the movement of different actuators, as well as the force/torque signals from various devices to the Haptic Knob for force feedback. We showcase and validate the proposed hardware and control interface on the industrial use case of automotive fuse box assembly.

The echo separation issue of different transmit antennas is the most technical challenge in realizing multiple-input and multiple-output synthetic aperture radar (MIMO SAR) with same frequency band, especially for low-computing echo separation, making it extremely difficult towards the practical application for the spaceborne MIMO SAR. Based on the orthogonal frequency-division multiplexing (OFDM) chirp waveforms, this letter proposes an innovative echo separation scheme with digital beamforming (DBF) and bandpass filtering (BPF) on board and bandpass-null steering on the ground for the spaceborne MIMO SAR. This scheme transfers the complex computing process on board to the ground, thus significantly reduce the computational load and relieve the resource occupation on board. Also, the perfect separation of interested echoes from interference can be achieved by this scheme. Finally, performance comparisons and simulation results show the effectiveness of the proposed scheme. The proposed scheme enables a high-efficiency and great-performance echo separation for the spaceborne MIMO SAR and makes the MIMO SAR a more promising technique for future SAR missions.

Synthetic aperture radar (SAR) systems require a favorable waveform for imaging as the radar waveform directly affects the performance of SAR systems, such as image quality and resolution. It is well known that the nonlinear frequency modulation (NLFM) waveform can adjust the time-frequency relation to shape the power spectral density (PSD) and then provide a matched filtering output with lower sidelobes without losing signal-to-noise ratio (SNR). However, it will broaden the main lobe, which means the resolution of the SAR image decreases. Therefore, in this paper, a novel NLFM waveform is proposed. It employs the piecewise linear function (PWL) model to define the instantaneous chirp rate function, and genetic algorithm (GA) is then applied to optimize the waveform. The novel NLFM waveform promises enhanced performance and flexibility due to greater design freedom. Through this method, lower sidelobe is achieved within the same 3-dB main lobe width. Finally, simulation results are presented to verify the practicability of the proposed NLFM waveform.

The nonlinear frequency modulation (NLFM) waveform is a promising candidate for the linear frequency modulation (LFM) signal because its autocorrelation output exhibits low sidelobes without loss of signal-to-noise ratio (SNR), also avoiding the transmitting power loss for spaceborne synthetic aperture radar (SAR). However, the acquisition of the analytical expression of the NLFM waveform is often closely related to the indirect instantaneous frequency function generated by the principle of stationary phase (POSP), thus it is not inconvenient for the real-time generation of the efficient and precise NLFM signal on board. In this letter, based on the harmonic decomposition, closed-form expressions of the NLFM waveform, which are directly calculated by the predefined window, are derived in both time and frequency domains. Therefore, it is very beneficial to the real-time generation and process of the NLFM waveform for the SAR system. All the simulation results and analyses validate the promising potential of the closed-form expressions of the NLFM waveform for SAR application.

Multiple-input and multiple-output synthetic aperture radar (MIMO SAR) is a very potential technique for high-resolution and wide-width (HRWS) imaging due to the fact that it can provide more degrees of freedom. In this paper, an innovative MIMO-HRWS SAR imaging system is introduced with segmented phase coding (SPC) waveforms. However, it is well known that the echo separation issue is the most technical challenge for MIMO SAR. It has shown that the separation of orthogonal echoes from cross-correlation interference can be achieved by scan-on-receive (SCORE). However, this method often requires large computing resources on board, especially for null-steering process. For this, we propose a two-stage spaceborne-ground echo separation scheme. This scheme is mainly divided into two-steps: the first step is a real-time SCORE-beamforming process on board to reduce the downlink data volume, and the second step is a bandpass-null steering process to suppress the interfering echoes on the ground. Thus, this scheme allows for the low computing resources on board and sufficient suppression of the interference simultaneously. Following this scheme, the MIMO-HRWS SAR system enables its number of equivalent phase centers nearly double that of the azimuth multichannel SAR system, thus higher-resolution and wider-swath imaging. Finally, detailed simulation experiments for the MIMO-HRWS SAR imaging system are performed to verify the practicability and feasibility of the proposed scheme.

Multiple-input–multiple-output synthetic aperture radar (MIMO-SAR) has extensive application prospects, mainly including the acquisition of multidimensional scattering information, high-resolution and wide-width (HRWS) imaging, and moving target indication (MTI). Its echo separation is the most technical challenge, and so far, the confirmation for orthogonal waveform encoding MIMO-SAR by airborne experiments has not been reported in any literature. Here, an echo separation experiment based on the segmented phase code (SPC) waveforms and an airborne digital beamforming SAR (DBF-SAR) system is demonstrated for the first time. In the experiment, the SPC waveforms are cyclically transmitted within the adjacent pulse repetition intervals (PRIs) to simulate multiple transmitters, and the scattered echoes are received by the 16-channel antennas in elevation at the same time. In the postprocessing, the echo signals of continuous PRIs are added to obtain the mixed echoes, and a detailed echo separation method is adopted. In the method, the mixed echo signals from close arrival angles and far arrival angles are separated by the time shift and weighting, and by the bandpass filtering and DBF technique, respectively. Through the presented method, the mixed echoes of dual-transmit and 16-receive (2T16R) SAR imaging mode are separated and imaged successfully. The experimental results not only validate the echo separation scheme but also indicate that it is very promising in future MIMO-SAR missions.