Autonomous driving relies heavily on advanced perception systems built on multi-sensor fusion, including LiDAR, cameras, GPS, and IMU. Among these, LiDAR is particularly valuable for providing accurate 3D spatial information, making it essential for many autonomous driving algorithms. However, LiDAR also presents challenges such as data sparsity, limited detection range (especially beyond 70 meters), and high computational costs. These limitations hinder the system's ability to detect distant objects or respond to sudden appearances and abrupt stops in real-time scenarios. In contrast, cameras offer rich texture and color information but struggle with depth estimation. To address the shortcomings of each sensor while leveraging their respective strengths, this paper proposes an integrated network architecture that combines LiDAR and camera data through early fusion. We introduce a lightweight, efficient fusion network capable of performing high-precision camera-based Panoptic Segmentation and depth completion simultaneously, thereby enhancing sparse LiDAR data. The proposed architecture achieves over 9 FPS inference speed on a single RTX 4070 GPU. For real-time 3D Semantic Segmentation, we fuse outputs from each task and deploy the system using ROS2 and Autoware. Experimental results demonstrate that our model effectively balances performance and computational efficiency. Its ability to handle a variety of perception tasks with fewer resources makes it suitable for practical deployment in autonomous driving and 3D scene reconstruction applications.

The complex mathematical model of autonomous vehicles makes it difficult for system identification due to a combination of non-linearity and uncertainty. Various strategies have been proposed to address the difficulty in system identification, as it significantly influences the precise path-tracking performance of autonomous vehicles. This paper proposes a Koopman operator approach data-driven optimal control algorithm for path-tracking of autonomous vehicles. To identify mathematical model’s various vehicle types of autonomous vehicle driving data were acquired in virtual simulation and real-world environments. An integrated linear model was identified using the Koopman operator neural network and the acquired driving data of autonomous vehicles. The identified integrated linear model was incorporated into a model predictive control algorithm designed for the path-tracking of autonomous vehicles. Reasonable path tracking performance was confirmed through performance evaluations conducted in path-tracking scenarios using various vehicle types for real and virtual vehicles in the real autonomous driving proving ground C-track and CARLA simulator environments. Index Terms—Koopman operator, deep neural network, model predictive control, autonomous vehicles, path-tracking

The state-of-the-art semantic segmentation tasks can be achieved by the variants of the fully convolutional neural networks (FCNs), which consist of the feature encoding and the deconvolution. However, they struggle with missing or inconsistent labels. To alleviate these problems, we utilize the image-level multi-class encoding as the global contextual information. By incorporating object classification into the objective function, we can reduce incorrect pixel-level segmentation. Experimental results show that our algorithm can achieve better performance than other methods on the same level training data volume.

The authors present a robust face recognition system for large-scale data sets taken under uncontrolled illumination variations. The proposed face recognition system consists of a novel illumination-insensitive preprocessing method, a hybrid Fourier-based facial feature extraction, and a score fusion scheme. First, in the preprocessing stage, a face image is transformed into an illumination-insensitive image, called an "integral normalized gradient image," by normalizing and integrating the smoothed gradients of a facial image. Then, for feature extraction of complementary classifiers, multiple face models based upon hybrid Fourier features are applied. The hybrid Fourier features are extracted from different Fourier domains in different frequency bandwidths, and then each feature is individually classified by linear discriminant analysis. In addition, multiple face models are generated by plural normalized face images that have different eye distances. Finally, to combine scores from multiple complementary classifiers, a log likelihood ratio-based score fusion scheme is applied. The proposed system using the face recognition grand challenge (FRGC) experimental protocols is evaluated; FRGC is a large available data set. Experimental results on the FRGC version 2.0 data sets have shown that the proposed method shows an average of 81.49% verification rate on 2-D face images under various environmental variations such as illumination changes, expression changes, and time elapses.

In this paper, we propose a novel method using gender information for achieving better performances of face recognition systems. Gender is one of the important factors for recognizing appearance of human faces and there are many studies on gender classifications such. However, the gender information is not actively applied in vision-based face recognition tasks, because we cannot find out human identity using only gender information. Therefore, we design the face recognition system based on the gender-based facial features with global facial features, and moreover, gender-based score normalization method for verification task. For fair evaluations, we use FRGC database known as a large size face image database.

The face recognition system based on the only single classifier considering the restricted information can not guarantee the generality and superiority of performances in a real situation. To challenge such problems, we propose the hybrid Fourier features extracted from different frequency bands and multiple face models. The hybrid Fourier feature comprises three different Fourier domains; merged real and imaginary components, Fourier spectrum and phase angle. When deriving Fourier features from three Fourier domains, we define three different frequency bandwidths, so that additional complementary features can be obtained. After this, they are individually classified by Linear Discriminant Analysis. This approach makes possible analyzing a face image from the various viewpoints to recognize identities. Moreover, we propose multiple face models based on different eye positions with a same image size, and it contributes to increasing the performance of the proposed system. We evaluated this proposed system using the Face Recognition Grand Challenge (FRGC) experimental protocols known as the largest data sets available. Experimental results on FRGC version 2.0 data sets has proven that the proposed method shows better verification rates than the baseline of FRGC on 2D frontal face images under various situations such as illumination changes, expression changes, and time elapses.

A novel heat kernel based Monte Carlo localization (HK-MCL) algorithm is presented to solve the degeneracy problem of conventional Monte Carlo localization: real-time global localization requires the number of initial samples to be small, whereas global localization may fail if the number of initial samples is small. The degeneracy problem is solved by an optimization approach called heat kernel based perturbation (HK-perturbation), which moves the samples towards the high likelihood area. HK-perturbation integrates the average local density and importance weight of samples to determine each sample's perturbation probability. The strategy improves simulated annealing algorithm via an obvious form of temperature, both in time and space, with respect to average local density and importance weight of samples. Systematic empirical results in global localization based on sonar illustrate superior performance, when compared to other state-of-the-art updating of Monte Carlo localization.

We present a novel discriminant analysis learning method which is applicable to non-linear data structures. The method can deal with pattern classification problems which have a multi-modal distribution for each class and samples of other classes may be closer to a class than those of the class itself. Conventional linear discriminant analysis (LDA) and LDA mixture model can not solve this linearly non-separable problem. Several local linear transformations are considered to yield locally transformed classes that maximize the between-class covariance and minimize the within-class covariance. The method invloves a novel gradient based algorithm for finding the optimal set of local linear bases. It does not have a local-maxima problem and stably converges to the global maximum point. The method is computationally efficienct as compared to the previous non-linear discriminant analysis based on the kernel approach. The method does not suffer from an overfitting problem by virtue of the linear base structure of the solution. The classification results are given for both simulated data and real face data. 1

In this paper, we propose an ICA (Independent Component Analysis) based face recognition algorithm, which is robust to illumination and pose variation. Generally, it is well known that the first few eigenfaces represent illumination variation rather than identity. Most PCA (Principal Component Analysis)-based methods have overcome illumination variation by discarding the projection to a few leading eigenfaces. The space spanned after removing a few leading eigenfaces is called the "residual face space". We found that ICA in the residual face space provides more efficient encoding in terms of redundancy reduction and robustness to pose variation as well as illumination variation, owing to its ability to represent non-Gaussian statistics. Moreover, a face image is separated into several facial components, local spaces, and each local space is represented by the ICA bases (independent components) of its corresponding residual space. The statistical models of face images in local spaces are relatively simple and facilitate classification by a linear encoding. Various experimental results show that the accuracy of face recognition is significantly improved by the proposed method under large illumination and pose variations.

We propose a method of efficient face description for facial image retrieval from a large data set. The novel descriptor is obtained by decomposing the face image into several components and then combining the component features. The decomposition combined with LDA (linear discriminant analysis) provides discriminative facial features that are less sensitive to light and pose changes. Each component is represented in its Fisher space and another LDA is then applied to compactly combine the features of the components. To enhance retrieval accuracy further, a simple pose classification and transformation technique is performed, followed by recursive matching. The experimental results obtained on the MPEG-7 data set show an impressive accuracy of our algorithm as compared with the conventional PCA/ICA/LDA methods.

We present a component-based face descriptor with LDA (Linear Discriminant Analysis) and a simple pose classification. Our algorithm has been developed to deal with face image retrieval in huge database such as those in internet environments. Such retrieval requires a compact face descriptor and an efficient recognition algorithm that is robust to variations in lighting and facial poses. Partitioning of a face image into components facilitates the development of an efficient and robust algorithm as follows. First, compensation for light and pose variations is much more easily done on individual components than on the whole image. Second, pose variation is compensated by classifying facial pose and aligning facial components. Finally, LDA is more effective at the component level which has simplified statistics than the whole image. Experimental results on MPEG-7 database show an impressive accuracy of our algorithm compared with conventional LDA methods. 1.

Presents a hybrid and parallel system based on artificial neural networks for a face invariant classifier and general pattern recognition problems. A set of face features is extracted by using the eigenpaxel method, which is based on principal component analysis (PCA) of a group of pixels, that is called a paxel. To classify subjects, multi-layer perceptron neural networks (NNs) are trained for each eigenpaxel. These parallel NN kernels provide sage, fast and efficient classification. To combine the results of parallel NNs, a novel judge analyzer is proposed based on bond rating classification and prediction. The proposed judge strategy can detect distinguishable face features even in arguable situations. The proposed method was evaluated on Olivetti and HongIk university (HIU) face databases and it yields a top recognition rate of 95.5% and 94.11% respectively, which are better results than the previous eigenpaxel and NN approach.