Graph convolutional networks (GCNs) have attracted considerable interest in skeleton-based action recognition. Existing GCN-based models have proposed methods to learn dynamic graph topologies generated from the feature information of vertices to capture inherent relationships. However, these models have two main limitations. Firstly, they struggle to effectively utilize high-dimensional or structural information, which limits their capacity for feature representation and consequently hinders performance improvement. Secondly, among these models, the multi-scale methods that aggregate information at different scales often over-capture unnecessary relationships between vertices. This leads to an over-smoothing problem where smoothed features are extracted, making it difficult to distinguish the features of each vertex. To address these limitations, we propose the multi-scale structural graph convolutional network (MSS-GCN) for skeleton-based action recognition. Within the MSS-GCN framework, the common intersection graph convolution (CI-GC) leverages the overlapped neighbor information, indicating the overlap between neighboring vertices for a given pair of root vertices. The graph topology of CI-GC is designed to compute the structural correlation between neighboring vertices corresponding to each hop, thereby enriching the context of inter-vertex relationships. Then, our proposed multi-scale spatio-temporal modeling aggregates local-global features to provide a comprehensive representation. In addition, we propose a Graph Weight Annealing (GWA) method, which is a graph scheduling method to mitigate the over-smoothing caused by multi-scale aggregation. By varying the importance between a vertex and its neighbors, we demonstrate that the over-smoothing problem can be effectively mitigated. Moreover, our proposed GWA method can easily be adapted to different GCN models to enhance performance. Combining the MSS-GCN model and the GWA method, we propose a powerful feature extractor that effectively classifies actions for skeleton-based action recognition in various datasets. We evaluate our approach on three benchmark datasets: NTU RGB+D, NTU RGB+D 120, and NW-UCLA. The proposed MSS-GCN achieves state-of-the-art performance on all three datasets, further validating the effectiveness of our approach.

Despite the remarkable success of deep learning , an optimal convolution operation on point clouds remains elusive owing to their irregular data structure . Existing methods mainly focus on designing an effective continuous kernel function that can handle an arbitrary point in continuous space. Various approaches exhibiting high performance have been proposed, but we observe that the standard pointwise feature is represented by 1D channels and can become more informative when its representation involves additional spatial feature dimensions. In this paper, we present Multidimensional Kernel Convolution (MKConv), a novel convolution operator that learns to transform the point feature representation from a vector to a multidimensional matrix. Unlike standard point convolution, MKConv proceeds via two steps. (i) It first activates the spatial dimensions of local feature representation by exploiting multidimensional kernel weights. These spatially expanded features can represent their embedded information through spatial correlation as well as channel correlation in feature space , carrying more detailed local structure information. (ii) Then, discrete convolutions are applied to the multidimensional features which can be regarded as a grid-structured matrix. In this way, we can utilize the discrete convolutions for point cloud data without voxelization that suffers from information loss. Furthermore, we propose a spatial attention module, Multidimensional Local Attention (MLA), to provide comprehensive structure awareness within the local point set by reweighting the spatial feature dimensions. We demonstrate that MKConv has excellent applicability to point cloud processing tasks including object classification, object part segmentation, and scene semantic segmentation with superior results.

The image-based lane detection algorithm is one of the key technologies in autonomous vehicles. Modern deep learning methods achieve high performance in lane detection, but it is still difficult to accurately detect lanes in challenging situations such as congested roads and extreme lighting conditions. To be robust on these challenging situations, it is important to extract global contextual information even from limited visual cues. In this paper, we propose a simple but powerful self-attention mechanism optimized for lane detection called the Expanded Self Attention (ESA) module. Inspired by the simple geometric structure of lanes, the proposed method predicts the confidence of a lane along the vertical and horizontal directions in an image. The prediction of the confidence enables estimating occluded locations by extracting global contextual information. ESA module can be easily implemented and applied to any encoder-decoder-based model without increasing the inference time. The performance of our method is evaluated on three popular lane detection benchmarks (TuSimple, CULane and BDD100K). We achieve state-of-the-art performance in CULane and BDD100K and distinct improvement on TuSimple dataset. The experimental results show that our approach is robust to occlusion and extreme lighting conditions.

Depth completion is the task of reconstructing dense depth images from sparse LiDAR data. LiDAR depth completion, for which LiDAR data is the only input, is an ill-posed and challenging problem owing to the underlying properties of LiDAR data: extremely few points, presence of discontinuities, and absence of texture information. Accordingly, most approaches are heavily dependent on guided color images, which leads to unsatisfactory results when the color images are degraded. To alleviate the dependency on color images but leverage this information during training, we present a deep convolutional neural network (CNN) consisting of depth and edge CNNs via transferring of knowledge. In order to compensate for the limitations of LiDAR data, we design the edge CNN to learn a gradient depth image from a powerful teacher network through theKnowledge-Distillationmethod. Since the teacher network is trained with color images, color-embedded information can be obtained in the test phase even if color images are not used as an input. We further propose aSelf-Distillationmethod for transferring the color-embedded features from the edge CNN to the depth CNN. Enforcing the depth features to contain edge information hardly observed in LiDAR data enables the depth CNN to generate more edge-attentive and structure-preserving results. Our novel methods show remarkable results in outdoor and indoor environments for KITTI and NYU-Depth-V2 datasets. Experiments performed with low-channel LiDAR data in KITTI and few depth points in the NYU-Depth-V2 dataset show that our method is robust to data sparsity and applicable in various scenarios.

Detecting moving objects has been widely studied since it plays vital many applications, such as video surveillance and intelligent transportation systems. It is necessary to accurately differentiate the foreground and the background in this technology to analyze object motions in the scene. Conventional detection methods use many reference frames to model the background to detect moving objects; however, the detection is inaccurate when immediate changes occur in the scene because the instant update of the background model is impossible. To be robust illumination changes and dynamic backgrounds, we propose an accurate and instant background modeling (AIBM) method that inpaints the background with superpixels and enhances it in detail with pixel-levels. Unlike the previous approaches, the proposed AIBM method utilizes spatio-temporal information of only two consecutive frames to eliminate the lengthy initialization and update period of the model. In this paper, we illustrate the importance of accurate and instant background modeling in detecting moving objects. The performance of our method is evaluated with three benchmark datasets (CDnet2014, LASIESTA, and SBI). The experimental results show that our AIBM method is robust to sudden changes in the scene and outperforms the other conventional methods with F-measure of 0.8911 and 0.9682 in detection accuracy in CDNet2014 and LASIESTA datasets, respectively. The accuracy of the generated background is also measured on the SBI dataset, which demonstrates the importance of high-quality background modeling.

Various sensors can be attached and added to autonomous vehicles, included visual cameras, radar, LiDAR (Light Detection And Ranging), and GNSS (Global Navigation Satellite System). These sensors have been studied in many research areas, in particular studies on building precise 3D maps. It is essential for autonomous driving to create an accurate 3D map of the surrounding scene. However, creating an accurate static 3D map is difficult due to changes in moving objects or dynamic environments. Spurious objects on the 3D map can be handled by removing or ignoring them for 3D mapping. Following this idea, we propose an object segmentation and inpainting network. The proposed network called SAM-Net, addresses the object duplication issue by segmenting the objects and inpainting them with the segmentation results. Conventional inpainting research has dealt with RGB images. No matter how well such approaches reconstruct holes or corrupted images, they do not establish 3D points’ relationship with the point cloud frame. Therefore, we suggest a depth inpainting method for outdoor object segmentation and inpainting tasks that utilizes a high-precision depth range sensor (Velodyne HDL-64E), which is not suggested before. Unfortunately, no dataset exists for the outdoor depth inpainting task. Thus, to train our model, we generate a new dataset by locating objects on a clean static background. Moreover, our proposed method shows outstanding depth performance compared to the previous visual inpainting method. Our dataset will be available at: “https://github.com/JunhyeopLee/lidar_inpainting”.

Recently, researchers have been leveraging LiDAR point cloud for higher accuracy in 3D vehicle detection. Most state-of-the-art methods are deep learning based, but are easily affected by the number of points generated on the object. This vulnerability leads to numerous false positive boxes at high recall positions, where objects are occasionally predicted with few points. To address the issue, we introduce Penetrated Point Classifier (PPC) based on the underlying property of LiDAR that points cannot be generated behind vehicles. It determines whether a point exists behind the vehicle of the predicted box, and if does, the box is distinguished as false positive. Our straightforward yet unprecedented approach is evaluated on KITTI dataset and achieved performance improvement of PointRCNN, one of the state-of-the-art methods. The experiment results show that precision at the highest recall position is dramatically increased by 15.46 percentage points and 14.63 percentage points on the moderate and hard difficulty of car class, respectively.

This paper presents the design and evaluation of FastAd, a fast and highly adaptive peer-to-peer (P2P) lookup system for medium-scale network where the number of nodes is not more than tens of thousands. Using routing tables larger than those of previous schemes, a lookup can be performed at just two hops, while maintenance cost of tables can be minimized by adopting lazy consistency with piggybacking. In addition, assigning dynamic node identifier (nodeId) instead of a fixed nodeId by hashing the node's IP address, FastAd network can be organized more adaptively and adjusted dynamically with system state such as load distribution, node dynamics, and so on. As a result, each node has different responsibility to exploit its heterogeneity.