Multi-View Stereo is widely utilized for reconstructing the dense geometric structure of objects from multiple viewpoints. Recently, learning-based PatchMatch MVS methods have attracted significant attention due to their high efficiency and accuracy. However, existing methods neglect the constraints of convolutional kernel receptive fields and uniform depth sampling. In this paper, we propose a novel PatchMatch MVS method named DU-PMVS, which concurrently supports adaptive feature extraction and depth sampling. Specifically, we design a Deformable Feature Pyramid Extractor to capture multi-scale features for each pixel, thereby enhancing the representation of contextual information. Additionally, we propose Uncertainty Probability Distribution-guided Depth Modeling, which addresses the issue of cumulative errors in coarse-to-fine structure by exploring richer uncertainty distributions and generating more effective depth hypotheses. Experimental results on the DTU and Tanks & Temples datasets demonstrate that our DU-PMVS can reconstruct more completed point clouds with low memory, particularly in challenging regions with weak textures.

Facial attribute editing plays a crucial role in synthesizing realistic faces with specific characteristics while maintaining realistic appearances. Despite advancements, challenges persist in achieving precise, 3D-aware attribute modifications, which are crucial for consistent and accurate representations of faces from different angles. Current methods struggle with semantic entanglement and lack effective guidance for incorporating attributes while maintaining image integrity. To address these issues, we introduce a novel framework that merges the strengths of latent-based and reference-based editing methods. Our approach employs a 3D GAN inversion technique to embed attributes from the reference image into a tri-plane space, ensuring 3D consistency and realistic viewing from multiple perspectives. We utilize blending techniques and predicted semantic masks to locate precise edit regions, merging them with the contextual guidance from the reference image. A coarse-to-fine inpainting strategy is then applied to preserve the integrity of untargeted areas, significantly enhancing realism. Our evaluations demonstrate superior performance across diverse editing tasks, validating our framework’s effectiveness in realistic and applicable facial attribute editing.

Due to the limitations of network communication conditions for online calling GPT, the onboard deployment of Large Language Models for autonomous driving is in need. In this paper, we propose Drive as Veteran, a fine-tuned LLaMA-7B model with driving tasks. A training set consisting of instructions, scenario descriptions and human-annotated driving tasks is established. Through LoRA fine-tuning, the capability of generating correct driving tasks of our model is demonstrated through a numerical experiment and the comparison to GPT-3.5 is presented. We show that smaller-sized Large Language Models could be deployed onboard with fast generation speed and high accuracy, which could serve as a core component for decision-making in autonomous driving.

In recent works, utilizing a deep network trained on meta-training set serves as a strong baseline in few-shot learning. In this paper, we move forward to refine novel-class features by finetuning a trained deep network. Finetuning is designed to focus on reducing biases in novel-class feature distributions, which we define as two aspects: class-agnostic and class-specific biases. Class-agnostic bias is defined as the distribution shifting introduced by domain difference, which we propose Distribution Calibration Module(DCM) to reduce. DCM owes good property of eliminating domain difference and fast feature adaptation during optimization. Class-specific bias is defined as the biased estimation using a few samples in novel classes, which we propose Selected Sampling(SS) to reduce. Without inferring the actual class distribution, SS is designed by running sampling using proposal distributions around support-set samples. By powering finetuning with DCM and SS, we achieve state-of-the-art results on Meta-Dataset with consistent performance boosts over ten datasets from different domains. We believe our simple yet effective method demonstrates its possibility to be applied on practical few-shot applications.

We propose a method to disentangle linear-encoded facial semantics from StyleGAN without external supervision. The method derives from linear regression and sparse representation learning concepts to make the disentangled latent representations easily interpreted as well. We start by coupling StyleGAN with a stabilized 3D deformable facial reconstruction method to decompose single-view GAN generations into multiple semantics. Latent representations are then extracted to capture interpretable facial semantics. In this work, we make it possible to get rid of labels for disentangling meaningful facial semantics. Also, we demonstrate that the guided extrapolation along the disentangled representations can help with data augmentation, which sheds light on handling unbalanced data. Finally, we provide an analysis of our learned localized facial representations and illustrate that the semantic information is encoded, which surprisingly complies with human intuition. The overall unsupervised design brings more flexibility to representation learning in the wild.

In recent years, deep learning algorithms have been widely used in semantic segmentation of aerial images. However, most of the current research in this field focus on images but not videos. In this paper, we address the problem of real-time aerial video semantic segmentation with BiSeNet[1]. Since BiSeNet is originally proposed for semantic segmentation of natural city scene images, we need a corresponding dataset to ensure the effect of transfer learning when applying it to aerial video segmentation. Therefore, we build a UAV streetscape sequence dataset (USSD) to fill the vacancy of dataset in this field and facilitate our research. Evaluation on USSD shows that BiSeNet outperforms other state-of-the-art methods. It achieves 79.26% mIoU and 93.37% OA with speed of 148.7 FPS on NVIDIA Tesla V100 for a 1920x1080 frame size input aerial video, which satisfies the demand of aerial video semantic segmentation with a competitive balance of accuracy and speed. The aerial video semantic segmentation results are provided at Our Repository.

Recently, impressive results have been achieved on many large-scale face recognition benchmarks, such as IJB-A Janus and Janus CS3. These datasets were designed to test robustness to nuisance transformations simultaneously such as pose, illumination, expression etc. We present a study paper, where we find that despite this goal in evaluation, there exists a significant frontal bias in yaw pose in these datasets. Therefore, high-performance on these recent datasets is misleading and does not reflect robustness to extreme pose in yaw. Moreover, many real-world applications only allow for a single frontal enrollment in a gallery (law enforcement, immigration etc.). As we show in our study, face recognition in this highly constrained setting with extreme pose variation in the probe images remains a highly challenging problem. Traditional approaches, performing well on datasets such as IJB-A Janus, perform much worse on older but highly controlled datasets such as CMU MPIE. To aid our study, we present a simple and practical method to handle pose variation in face recognition pipelines designed to deal with extremely off-angle faces. Our approach is to ignore the half of the face with any self-occlusion. This method allows our models to be highly robust to pose, and helps us achieve state-of-the-art results on several protocols using the CMU MPIE dataset as well as very accurate results on the CFP dataset, outperforming recent efforts using the same training data.

We motivate and present Ring loss, a simple and elegant feature normalization approach for deep networks designed to augment standard loss functions such as Softmax. We argue that deep feature normalization is an important aspect of supervised classification problems where we require the model to represent each class in a multi-class problem equally well. The direct approach to feature normalization through the hard normalization operation results in a non-convex formulation. Instead, Ring loss applies soft normalization, where it gradually learns to constrain the norm to the scaled unit circle while preserving convexity leading to more robust features. We apply Ring loss to large-scale face recognition problems and present results on LFW, the challenging protocols of IJB-A Janus, Janus CS3 (a superset of IJB-A Janus), Celebrity Frontal-Profile (CFP) and MegaFace with 1 million distractors. Ring loss outperforms strong baselines, matches state-of-the-art performance on IJB-A Janus and outperforms all other results on the challenging Janus CS3 thereby achieving state-of-the-art. We also outperform strong baselines in handling extremely low resolution face matching.

Although face detection has been intensely studied for decades, it is still a challenging topic due to numerous conditions, e.g. heavy occlusions, low resolutions, extreme poses, non-face patterns that look like human faces, etc. This paper proposes a novel region-based ConvNet to address these issues. Our approach enhances the interior deep facial features and explicitly incorporates the exterior deep features. The enhanced interior features provide fine details for small faces. The exterior features capture the local information surrounding the face, supporting the detection under challenging conditions. Experiments show that our proposed components improve the baseline method significantly. Additionally, our approach consistently achieves competitive performance in four challenging databases, i.e. Wider Face, AFW, PASCAL Faces, and FDDB. We also introduce a new challenging non-face dataset 1 of 6,000 images to benchmark false positive rates for future research.

The problems of hand detection have been widely addressed in many areas, e.g. human computer interaction environment, driver behaviors monitoring, etc. However, the detection accuracy in recent hand detection systems are still far away from the demands in practice due to a number of challenges, e.g. hand variations, highly occlusions, low-resolution and strong lighting conditions. This paper presents the Multiple Scale Faster Region-based Convolutional Neural Network (MS-FRCNN) to handle the problems of hand detection in given digital images collected under challenging conditions. Our proposed method introduces a multiple scale deep feature extraction approach in order to handle the challenging factors to provide a robust hand detection algorithm. The method is evaluated on the challenging hand database, i.e. the Vision for Intelligent Vehicles and Applications (VIVA) Challenge, and compared against various recent hand detection methods. Our proposed method achieves the state-of-the-art results with 20% of the detection accuracy higher than the second best one in the VIVA challenge.