In recent years, novel view synthesis from a monocular image has become a research hot-spot that attracts significant attention. Some recent work identifies latent vectors for high-quality view generation via iterative optimisation, which is a time-consuming process. In contrast, some others utilise an encoder learning a mapping function to approximately estimate optimal latent codes, which significantly reduces its processing time but sacrifices reconstruction quality. Consequently, how to balance synthesis quality and its generation efficiency still remains challenging. In this paper, we propose a residual-based encoder to incorporate with a 3D Generative Adversarial Networks (GAN), named ReE3D, for novel view synthesis. It applies an iterative prediction of latent codes to ensure much higher quality of novel view synthesis with an insignificant increase of processing time when compared to existing encoder-based 3D GAN inversion methods. Additionally, we enforce a novel geometric loss constraint on the encoder to predict view-invariant latent codes, thus effectively mitigating the trade-off between geometric and texture quality in 3D GAN inversion. Extensive experimental results demonstrate that our extended encoder-based method has achieved best trade-off performance in terms of novel view synthesis quality and its execution time. Our method has gained comparable synthesis quality with exponentially decreased processing time when compared to iterative optimisation methods, while improved synthesis performance of encoder-based methods significantly.

Adversarial training (AT) has shown impressive advantages in maintaining accuracy and enhancing robustness against adversarial examples. However, most existing AT techniques jointly optimize clean-example accuracy and adversarial-example robustness as dual objectives. This setting introduces an often overlooked issue; when optimizing hard samples near the decision boundary, the model may bolster robustness at the expense of accuracy or preserve accuracy to the detriment of robustness. To alleviate the accuracy-robustness sacrifices induced by hard samples, we propose mitigating hard sample interference (MHSI) from a sample-intervention perspective. MHSI aims to reduce the instability caused by hard samples during AT. Specifically, we introduce a weighted adaptive (WA) mechanism that strengthens the model's learning of clean samples, thereby reducing the negative impact of hard samples on accuracy. In addition, guided by an analysis of the gradient norm and the Hessian matrix, we design a dynamic calibration (DC) strategy that dynamically calibrates the probability outputs of hard samples to mitigate their damage to robustness. With these two modules, our approach significantly improves robustness without sacrificing accuracy. Extensive experiments on CIFAR-10, CIFAR-100, Tiny ImageNet, and SVHN demonstrate that MHSI effectively improves both accuracy and robustness and outperforms state-of-the-art methods under glass-box attacks. Notably, under an $l_{\infty }$ attack, MHSI yields up to a 6.22% robustness gain over the AT baseline. Our code is available at https://github.com/hubin111/MHSI.

Nonparametric estimation of the mean and covariance functions is ubiquitous in functional data analysis and local linear smoothing techniques are most frequently used. Zhang and Wang (2016) explored different types of asymptotic properties of the estimation, which reveal interesting phase transition phenomena based on the relative order of the average sampling frequency per subject $T$ to the number of subjects $n$, partitioning the data into three categories: “sparse”, “semi-dense”, and “ultra-dense”. In an increasingly available high-dimensional scenario, where the number of functional variables $p$ is large in relation to $n$, we revisit this open problem from a non-asymptotic perspective by deriving comprehensive concentration inequalities for the local linear smoothers. Besides being of interest by themselves, our non-asymptotic results lead to elementwise maximum rates of $L_2$ convergence and uniform convergence serving as a fundamentally important tool for further convergence analysis when $p$ grows exponentially with $n$ and possibly $T$. With the presence of extra $\log p$ terms to account for the high-dimensional effect, we then investigate the scaled phase transitions and the corresponding elementwise maximum rates from sparse to semi-dense to ultra-dense functional data in high dimensions. We also discuss a couple of applications of our theoretical results. Finally, numerical studies are carried out to confirm the established theoretical properties.

With the escalating demand for three-dimensional visual applications such as gaming, virtual reality, and autonomous driving, novel view synthesis has become a critical area of research. Current methods mainly depend on multiple views of the same subject to achieve satisfactory results, but there is often a significant lack of available data. Typically, only a single degraded image is available for reconstruction, which may be affected by occlusion, low resolution, or absence of color information. To overcome this limitation, we propose a two-stage feature matching approach designed specifically for single degraded images, leading to the synthesis of high-quality novel perspective images. This method involves the sequential use of an encoder for feature extraction followed by the fine-tuning of a generator for feature matching. Additionally, the integration of an information filtering module proposed by us during the GAN inversion process helps eliminate misleading information present in degraded images, thereby correcting the inversion direction. Extensive experimental results show that our method outperforms existing state-of-the-art single-view novel view synthesis techniques in handling challenges like occluded, grayscale, and low-resolution images. Moreover, the efficacy of our method remains unparalleled even when aforementioned method integrated with image restoration algorithms.

Guided depth super-resolution (GDSR) aims to enhance the level of detail in low-resolution depth images by utilizing the information present in the corresponding high-resolution RGB images. While existing methods utilize different approaches to guide the RGB image to the source image, they often ignore the texture similarity between these two images and usually suffer from unsatisfactory outline reconstruction of the depth map. In this article, we introduce an adaptive texture migration network (ATMNet) designed to mine rich feature information from RGB images and migrate them to the depth image. Specifically, we propose a multi-modal feature extractor (MMFE) to extract private and shared features between the depth map and RGB image. In addition, we present a texture migration module (TMM) to remap and fuse the features extracted from the raw image pairs. Last but not least, we develop a weighted adaptive loss to enhance the reconstruction of the edge areas in the depth map. Extensive experiments on public datasets such as Middlebury, NYUv2, and DIML demonstrate that our method outperforms the existing state-of-the-art GDSR methods and strikes a remarkable balance between performance and efficiency. The source code is available at https://github.com/MuggleTan/ATMNet .