The advancement of self-driving technology is driven by the need for robust and efficient perception systems along with frameworks for End-to-End testing, enabled by the CARLA simulator. We introduce C-Shenron, a novel integration of a realistic radar sensor model within CARLA, enabling researchers to develop and test navigation algorithms using radar data. It is the first realistic radar simulator which utilizes LiDAR and camera sensors to generate high-fidelity radar ADC measurements from physics based modeling of the environment. Utilizing this radar sensor and showcasing its capabilities in simulation, we demonstrate improved performance in end-to-end driving scenarios. Our setup aims to rekindle the interest in radar-based self-driving research and promote the development of algorithms that leverages its strengths.

The advancement of self-driving technology is driven by the need for robust perception and navigation systems. Simulators for autonomous driving facilitate the rapid development and testing of navigation algorithms; however, a key issue for most is their inaccurate modeling of the radar sensor. This is a significant drawback as radars offer robust sensing capabilities in adverse weather conditions and occlusions. CARLA, a widely adopted open-source simulator, provides a simplistic radar model that fails to capture the complex physical and material-dependent behavior of real-world radar. To address these limitations, we present CShenron, a radar simulation framework integrated into CARLA, which generates realistic radar measurements by fusing LiDAR and camera data. C-Shenron also supports configurable radar parameters, multiple sensor placements, and scalable dataset generation. Our evaluations demonstrate that radar-camera fusion models, trained with C-Shenron’s generated data, achieve performance equivalent to traditional LiDAR-camera baselines on key metrics from the CARLA leaderboard.

Reinforcement learning in the field of artificial intelligence has seen tremendous advances in recent years, but there are still several limitations standing in the way of its wider practical application, including sample inefficiency, generalization, and exploration-exploitation trade-off. Deep Q-Networks (DQN) have improved the performance of RL by using deep neural networks to approximate the Q-function and by using experience replay to store and reuse past experiences. Episodic memory in RL is a technique that allows an agent to store and reuse past experiences in order to improve its decision-making. However, current episodic memory-based RL techniques have some issues, such as generalization and slow learning, which can be improved by using methods such as experience replay compression and reducing the information of episodic memory into a parametric model. In this work, we propose cognitive-inspired episodic memory in DQN networks (CEMDQN) that reduces the priority weighting of old experiences over time, and the agent accesses recent experiences more frequently. The proposed model was evaluated on three different environments: StarGunner, BattleZone, and TimePilot. It was shown that when compared to standard episodic memory DQN, CEMDQN was more effective in test score performance for StarGunner (45.1 %), BattleZone (81 %), and TimePilot (53.54%) environments, respectively.

Crowd estimation and monitoring is an important surveillance task. We address the problem of estimating the “flow,” that is the number of persons passing a designated region in a unit time. We designate an area of the scene as a virtual trip wire and accumulate the total number of foreground pixels (in the trip wire) over a chosen time period. We show that cumulative pixel count is related to the number of persons passing through the trip-wire by a scale factor. This scale factor is highly sensitive to the “crowdedness” (levels of crowd density) of the scene which creates different levels of occlusion of the individuals walking/passing through the trip-wire. We use texture features to determine the crowdedness and choose the most appropriate scaling factor. Our method does not require detection and tracking of individuals and is robust to scene dynamics, background subtraction errors, and different crowd levels.

Color is a powerful attribute that is used to characterize objects for tracking and other surveillance tasks. Since color is dependent on ambient illumination and the imaging equipment, the reliability of color features decreases as the object moves through regions observed with different cameras and with different illumination conditions. Typically, this issue is addressed using alternative color spaces or highly simplified color transformation. In this paper, we propose a more methodical approach to achieving color consistency using colorimetric principles. We model the image analysis system as an observer and develop camera-specific transformations so that images of the same object appear similar to this observer. The transformations thus developed are approximated with 3D look-up tables for fast operation. Experimental results show improved color consistency even under two very different illumination conditions. The results are evaluated both qualitatively and in terms of the effect on a particle filter based object tracker.

Behavioral information can be inferred from the trajectory of a rigid object. We propose a method to detect anomalies in the approach of a vehicle by observing the patterns in its velocity and describe methods for more effective analysis of the velocity trajectory. First we define a hypothetical co-ordinate system in which the axes are specified with respect to the road and distances are true ground measurements. In this co-ordinate system the (axial) velocity is a one dimensional quantity. We estimate the “normal” trends in the velocities by activity path modeling after scaling the velocities by the vehicle's average speed. We detect an anomaly if a vehicle's velocity does not fall in the available path models. Finally, we use the shape of the trajectory to determine turns and other significant maneuvers. We also detect an anomaly if a vehicle's velocity falls in a path model which is inconsistent with the shape of its trajectory. We observe that our proposed co-ordinate system also improves trajectory shape analysis by suppressing false turns.

We present a framework for developing a color transformation between two display devices to achieve a desired perceptual match. The framework is realized by first developing an optimal color transformation between the two devices, and then formulating an optimization problem to develop a hardware resource-constrained approximation to the optimal transformation. We employ the framework to investigate how well the system performs when the transformation is constrained to be a single 3×3 linear transformation directly between the non-linear RGB spaces of two devices. The motivation for this constraint is to ease resource requirements in a real-time hardware implementation.

Color space transformations are very common in digital imaging and display systems. Their complex mathematics requires hardware friendly implementations to make them practical. In this paper, look-up tables (LUT) are described for implementation of the color transformations. We show that this is possible because an analytical model of the transform is available for training and cross-validation. We identify parameters that affect the performance of a LUT-based system and then formulate the solution as an optimization problem.