Transformers have gained significant attention in multimodal remote sensing fusion due to their strong global context modeling capability. Although Transformer-based methods excel at processing high-dimensional spectral sequences and joint spatial-spectral information, most current research remains focused on the spatial domain. Consequently, the exploration of frequency-domain features—particularly implicit frequency representations—is often neglected. Moreover, efficiently fusing multimodal data features while emphasizing more discriminative information remains a challenging task. To address these challenges, this paper proposes an Attention-guided Frequency Feature Decomposition Network (AF2DN) for Hyperspectral and LiDAR Data Classification. First, a Transformer-based Frequency Feature Decomposition(TFFD) method is proposed, employing window attention to capture distinct directional frequency components from multimodal remote sensing data. Through this approach, low-frequency components are utilized to characterize global structural information, while various high-frequency components are employed to extract diverse texture and edge features. Second, an Attention Frequency Modulation(AFM) module is developed, incorporating a weight learning matrix in the frequency domain. This matrix is designed to selectively amplify and suppress different frequency components, thereby reducing data redundancy resulting from frequency feature decomposition. Finally, an adaptive Multimodal Same-Frequency Feature Fusion (AMSF3) module is designed to achieve cross-modal feature integration at identical frequency bands. Extensive experiments are conducted on three benchmark datasets, and the results demonstrate that the proposed framework outperforms existing state-of-the-art methods while exhibiting stronger adaptability in complex environments.

This demonstration presents an advanced stereo vision system with high energy efficiency. An ov5640 binocular camera, operating at a maximum of 30 frames/second with FHD (1920×1080) resolution, is employed to capture image pairs. A Spartan-7 FPGA rectifies these images with a calibration map matrix and then channels the pixel stream to the stereo-matching processor in a 28nm CMOS process for depth estimation. The resulting depth map, crucial for tasks like obstacle detection and navigation, is displayed in real-time on the monitor for low-power stereo vision applications.

Binocular stereo vision is a depth estimation technique by imitating human eyes. It is widely used in various fields, such as self-driving cars, SLAM, and 3D reconstruction. However, designing a hardware architecture that can balance resource utilization, processing speed, and estimation accuracy remains a significant challenge. This paper proposes a compact and efficient hardware-based design that incorporates linear fitting-based cost fusion, disparity optimization with subpixel interpolation, and multi-directional occlusion filling techniques. Firstly, a gradient-enhanced pipelined matching costs architecture with a resource-saving scheme and re-computation paradigm is proposed to improve the accuracy of edge information. Then, we approximate the nonlinear exponential function by linear fitting to save the hardware resource. Moreover, we design the subpixel interpolation with an SRT radix-4 divider to refine the disparity, which significantly enhances the accuracy of the disparity map in real situations. Finally, we proposed a resource-reused architecture for synchronous hole filling and median filter in post-processing. The disparity map quality of the proposed architecture is evaluated on KITTI2015 datasets, which delivers leading accuracy compared to other state-of-the-art works. The architecture has been successfully implemented and demonstrated on the Stratix-V FPGA platform and achieved 54 frames per second operating at 112 MHz under a resolution of$1920\times 1080$.

With the increasing demand for high-resolution video and real-time processing, the limited efficiency of video-denoising algorithms has become a critical factor. This paper proposes a spatio-temporal video denoising co-processor to suppress an image sequence’s spatial and temporal noise. Temporal denoising is achieved by merging the current and previous frames at the pixel level in which the current frame is processed by a spatial filter. After exploiting noise estimation and motion detection, the Wiener filter calculates the merge ratio. Rather than buffering the entire previous frame, the JPEG-like codec can dynamically adjust the compression ratio through a predefined quantization table to satisfy the designed on-chip storage. The experimental results demonstrate that the spatio-temporal denoising co-processor can effectively eliminate the fluctuation of the grayscale value of the noise in videos. Simultaneously, the adaptive codec can reduce the storage space consumption for the frame buffer by at least 80% of the original size. To the best of our knowledge, this is the first fully integrated spatio-temporal denoising co-processor without any external memory. Additionally, the grayscale, RGB, and RAW versions of the co-processor are also implemented on the Stratix V FPGA platform and synthesized in 28nm CMOS technology.

Automotive radar sensors are widely used in vehicles so as to allow the “advanced driver assistance systems (ADAS)” to work. As far as we are concerned, no mandatory regulations were established so far for interference control in the cases where multiple vehicles equipped with automotive radars exist in close proximity. Consequently, those inter-vehicle cross-interference easily result in the radar performance degradation, e.g. incapability of target detection and identification. In this paper, an interference suppression algorithm is proposed which utilizes two Recursive-Least-Square (RLS) adaptive filters to estimate, the interference signals caused by the “aggressor” radar and the echo signals from the “ego” radar. The algorithm operates in the iterative mode with its initial input being the originally received beat signals involving interference. In each iteration, the estimates of the interference signals and of the clean signals are calculated sequentially by the two RLS filters, and the beat signal is updated with parts of the fragments substituted with their interference-mitigated counterparts. Such iterative operations stop when the convergence criterion, e.g. none of the fragments in the beat signals are considered being interfered, is achieved or the iteration number reaches the limit predefined due to the practical constraints. Simulation and measurement results demonstrate that the proposed dual-RLS-based algorithm outperforms the existing methods in terms of lower interference-to-signal ratio resultant and superior capability of retrieving the original range-Doppler profile.

The semi-global matching (SGM) algorithm in stereo vision is a well-known depth-estimation method since it can generate dense and robust disparity maps. However, the real-time processing and low power dissipation, the specifications of the Internet-of-Thing (IoT) applications, are challenging for their computational complexity. In this paper, we propose a hardware-oriented SGM algorithm with pixel-level pipeline and region-optimized cost aggregation for high-speed processing and low hardware-resource usage. Firstly, the matching costs in a region are integrated with an optimization strategy to significantly reduce memory usage and improve the processing speed of the cost aggregation. Then, a two-layer parallel two-stage pipeline (TPTP) architecture, which enables pixel-level processing, is designed to calculate two directions (0° and 135°) aggregation to further solve the crucial computational bottleneck of the SGM algorithm. Finally, the architecture is demonstrated on a low-cost XILINX Spartan-7 device and an advanced Stratix-V FPGA device for VGA ($640\times 480$) depth estimation. The experimental results show that the proposed architecture with compact hardware architecture also ensures accuracy. The pixel-level pipeline architecture enables a processing speed of 355 frames per second (fps) at 109MHz on the Spartan-7 FPGA device and 508 fps at 156MHz on the Stratix-V FPGA. Besides, the coprocessor respectively achieves an energy efficiency of 4.74 nJ/pixel with a power dissipation of 517mW and 4.29nJ/pixel with a power dissipation of 669mW on these two FPGAs.

In recent years, with the rapid development of Advanced Driver Assistance Systems (ADAS), vehicular millimeter-wave (mmWave) radar plays an important role in urban intelligent transportation system. Self-interference caused by multipath in radar channel is a widely concerned problem. However, the research on radar channel propagation characteristics is very limited and there are few models for radar channel available. In this paper, we propose a pioneering research for radar channel model. Based on measurement data, channel characteristics are analysed using the space-alternating generalized expectation-maximization (SAGE) algorithm. An empirical statistical model is established for the 77GHz mmWave radar channel in the underground parking lot scenario, which is one of the most challenging environments because of time-variation and abundant multipaths, filling the gap in the field. A path-association algorithm is used to obtain multipath component (MPC) trajectories in delay, Doppler frequency and power domains. Additionally, some trajectory (in range domain) features are analysed. Moreover, based on geometric analysis of trajectories in range domain, the deterministic component of trajectories is modeled, and then a novel method for discrimination between actual targets and ghost images is proposed and verified by simulation and measurement data.