This paper introduces a tangible user interface based on projector-camera system. The interface is encoded by stripes in an imperceptible form. We make the first attempt to employ a neural network for extracting hand information from structured light images. The lightweight network of only 0.24 MB parameters infers hand existence, hand segmentation and fingertip locations. These results are utilized for decoding the imperceptible stripes. Experimental results show that the proposed network outperforms state-of-the-art methods, achieving a high IOU of hand segmentation (0.996), and fingertip localization precision (98.98%) within a 10-pixel range. The interactive system performs a touch detection precision of 98.78%, with an average fingertip localization error of 3.33 pixels. It supports real-time interaction with an average execution time is 24.87ms.

Camera calibration is a crucial step in many computer vision applications that require 3D information about the scene. The process involves estimating intrinsic and extrinsic parameters from a set of captured images. The main challenge is to obtain accurate corner position information from the captured patterns. The conventional method uses only one checkerboard pattern, which suffers from uneven noise distribution. To solve this problem, a calibration method based on complementary patterns is proposed, which reduces the influence of image noise and enhances the calibration accuracy. Experimental results show that the proposed method achieves a 27% reduction in the Root Mean Square of the re-projection error compared with the non-complementary pattern.

Stripe extraction is a deciding factor in the multi-line structured light systems. This paper introduces a method to extract the deteriorated stripes in a robust and accurate way. Based on the photoelectron crosstalk phenomenon of image sensors, mathematical expression of the ideal cross-section distribution for the captured stripes is deduced. Then, we establish a probability model for each pixel, with the consideration of how well the local distribution fits the ideal distribution, and how possible it is linked with the previous stripe centers. Therefore, the extraction process is transformed into a local searching scheme with dynamic update. Experimental results show that the robustness of the proposed scheme outperforms other methods. The root mean squared error of the reconstructed point cloud is 0.219mm, which proves the validity of the proposed method.