Light field image depth estimation methods involve a large number of parameters and floating-point operations. This makes FPGA-based acceleration design a huge challenge, especially when pursuing high-precision acceleration design on resource-limited FPGAs. Motivated by this issue, a resource-efficient hardware accelerator based on a high-precision, low-bit lightweight light field image depth estimation network scheme is proposed, named RE-LFDE. First, we present a parameter-sharing, low-bit and lightweight network. It is able to improve accuracy, simplify the network structure and reduce network parameters. Secondly, we design a time-division multiplexing hardware-software co-design dataflow structure and build a resource-efficient acceleration engine, which can be deployed on the resource-limited FPGAs efficiently. Experimental results show that the average MSE on the 4D LF benchmark of RE-LFDE and its full-precision networks can be reduced to 3.265 and 3.363, respectively, while the weight parameters can be as low as 0.30MB and 0.04MB. Furthermore, on the ZCU104 platform, the consumption of BRAM and LUTRAM can be reduced to 22.44% and 13.64%, respectively. The code and model of the proposed method are available at https://github.com/sansi-zhang/RE-LFDE .

The 3-D vision computing is a key application in unmanned systems, satellites, and planetary rovers. Learning-based light field (LF) depth estimation is one of the major research directions in 3-D vision computing. However, conventional learning-based depth estimation methods involve a large number of parameters and floating-point operations, making it challenging to achieve low-power, fast, and high-precision LF depth estimation on a field-programmable gate array (FPGA). Motivated by this issue, an FPGA-based low-bit, lightweight LF depth estimation network (L$^{3}\text {FNet}$) is proposed. First, a hardware-friendly network is designed, which has small weight parameters, low computational load, and a simple network architecture with minor accuracy loss. Second, we apply efficient hardware unit design and software-hardware collaborative dataflow architecture to construct an FPGA-based fast, low-bit acceleration engine. Experimental results show that compared with the state-of-the-art works with lower mean-square error (mse), L$^{3}\text {FNet}$can reduce the computational load by more than 109 times and weight parameters by approximately 78 times. Moreover, on the ZCU104 platform, it requires 95.65% lookup tables (LUTs), 80.67% digital signal processors (DSPs), 80.93% BlockRAM (BRAM), 58.52% LUTRAM, and 9.493-W power consumption to achieve an efficient acceleration engine with a latency as low as 272 ns. The code and model of the proposed method are available athttps://github.com/sansi-zhang/L3FNet.

Abstract Light field (LF) depth estimation is a key task with numerous practical applications. However, achieving high‐precision depth estimation in challenging scenarios, such as occlusions and detailed regions (e.g. fine structures and edges), remains a significant challenge. To address this problem, the authors propose a LF depth estimation network based on multi‐region selection and guided optimisation. Firstly, we construct a multi‐region disparity selection module based on angular patch, which selects specific regions for generating angular patch, achieving representative sub‐angular patch by balancing different regions. Secondly, different from traditional guided deformable convolution, the guided optimisation leverages colour prior information to learn the aggregation of sampling points, which enhances the deformable convolution ability by learning deformation parameters and fitting irregular windows. Finally, to achieve high‐precision LF depth estimation, the authors have developed a network architecture based on the proposed multi‐region disparity selection and guided optimisation module. Experiments demonstrate the effectiveness of network on the HCInew dataset, especially in handling occlusions and detailed regions.