Thermal infrared imaging has attracted widespread attention in many fields due to the advantages of all-weather imaging and strong penetration. However, existing methods for thermal infrared novel-view synthesis often produce results with coarse details and floating artifacts, primarily caused by physical factors such as atmospheric transmission effects and thermal conduction. These challenges hinder accurate reconstruction of intricate structures and temperature distributions in thermal scenes, limiting the practical utility of previous approaches. To address these limitations, this paper introduces a physics-induced 3D Gaussian splatting method named Thermal3D-GS, the first novel-view synthesis method that relies exclusively on thermal infrared image. Thermal3D-GS begins by modeling atmospheric transmission effects and thermal conduction in three-dimensional media using neural networks. Additionally, considering the sparse features of infrared images, sparse feature priors are designed to improve the reconstruction accuracy of thermal infrared images. Furthermore, to validate the effectiveness of our method, the first large-scale benchmark dataset named Thermal Infrared Novel-view Synthesis Dataset (TI-NSD) is created. This dataset comprises 50 authentic thermal infrared video scenes, covering indoor, outdoor, traffic and uncrewed aerial vehicle (UAV) scenarios, with a total of 15,213 frames of thermal infrared image data. In addition, an expanded validation thermal infrared dataset, which includes three high-resolution scenes and five special scenes under varying atmospheric conditions and complex propagation media is constructed to assess generalization performance of the proposed method. Based on this dataset, this paper experimentally verifies the effectiveness of Thermal3D-GS. The results indicate that our method outperforms the baseline method with a 3.19 dB improvement in PSNR and significantly addresses the issues of floaters and indistinct edge features present in the baseline method.

Precise segmentation of thermal infrared images is crucial in domains like surveillance, medical diagnostics, intelligent transportation, accurate guidance and remote sensing. However, current thermal segmentation methods often oversimplify by treating thermal images as grayscale, neglecting vital physical factors such as thermal imaging effects and material information, thereby constraining segmentation precision. To address these limitations, we propose TherNet, a novel thermal infrared segmentation framework integrating thermal imaging effects and material physical information. The study elucidates the impacts of object radiation, inter-object thermal exchange, atmospheric scattering, and camera thermal inertia on thermal infrared imaging, developing four modules to model or rectify these physical processes. To validate the proposed framework, two large-scale infrared datasets were created: TI-Cityscapes for multi-class semantic segmentation in traffic scenes (4,200 frames, 18 classes), and TBRSD for single-object blindroad segmentation (5,180 frames from a pedestrian perspective). The proposed methods achieved SoTA performance across three infrared semantic segmentation datasets and the blind road segmentation dataset, underscoring the pivotal role of leveraging physical properties. TherNet provides innovative perspectives and robust benchmarks for future developments in the domain.

Computer vision-based walking assistants are prominent tools for aiding visually impaired people in navigation. Blind road segmentation is a key element in these walking assistant systems. However, most walking assistant systems rely on visual light images, which is dangerous in weak illumination environments such as darkness or fog. To address this issue and enhance the safety of vision-based walking assistant systems, we developed a thermal infrared blind road segmentation neural network (TINN). In contrast to conventional segmentation techniques that primarily concentrate on enhancing feature extraction and perception, our approach is geared towards preserving the inherent radiation characteristics within the thermal imaging process. Initially, we modelled two critical factors in thermal infrared imaging - thermal light atmospheric transmission and thermal inertia effect. Subsequently, we use an encoder-decoder architecture to fuse the feathers extracted by the two modules. Additionally, to train the network and evaluate the effectiveness of the proposed method, we constructed a large-scale thermal infrared blind road segmentation dataset named TBRSD consists 5180 pixel-level manual annotations. The experimental results demonstrate that our method outperforms existing techniques and achieves state-of-the-art performance in thermal blind road segmentation, as validated on benchmark thermal infrared semantic segmentation datasets such as MFNet and SODA. The dataset and our code are both publicly available in https://github.com/chenjzBUAA/TBRSD or http://xzbai.buaa.edu.cn/datasets.html.

Semantic segmentation in urban scenes is widely used in applications of intelligent transportation systems (ITS). In urban scenes, thermal infrared (TIR) images can be captured in weak illumination conditions or in the presence of obscuration (e.g., light fog, smoke). Therefore, TIR images have great potential to endow automated intelligent vehicles or assist navigation systems. However, TIR imaging is blurry and low-contrast due to the absorption by atmospheric gases and heat transfer effect. Hence, TIR semantic segmentation in urban scenes has rarely been explored even though it has a wide range of scenarios in ITS. To overcome this limitation, we analyze the light transport of TIR light. Our analysis reveals that contours are the reliable features shared by TIR and Visible Spectrum (VS) light. Inspired by this, we attempt to transfer joint features from VS domain to TIR domain. Thus, we propose a curriculum domain adaptation method to guide the TIR urban scene semantic segmentation task from VS domain through contours. Moreover, to evaluate the proposed model, we build TIR-SS: an open-for-request dataset consisting of TIR images and pixel level annotations of 8 classes in urban scenes. Qualitative and quantitative experimental results on the dataset indicate that the proposed domain adaptation method outperforms related methods on this TIR semantic segmentation task.