Recently, Few-shot Learning (FSL) has become a popular task that aims to recognize new classes from only a few labeled examples and has been widely applied in fields such as natural science, remote sensing, and medical images. However, most existing methods focus only on the visual modality and compute prototypes directly from raw support images, which lack comprehensive and rich multimodal information. To address these limitations, we propose a novel Multimodal Prototype Augmentation FSL framework called MPA, including LLM-based Multi-Variant Semantic Enhancement (LMSE), Hierarchical Multi-View Augmentation (HMA), and an Adaptive Uncertain Class Absorber (AUCA). LMSE leverages large language models to generate diverse paraphrased category descriptions, enriching the support set with additional semantic cues. HMA exploits both natural and multi-view augmentations to enhance feature diversity (e.g., changes in viewing distance, camera angles, and lighting conditions). AUCA models uncertainty by introducing uncertain classes via interpolation and Gaussian sampling, effectively absorbing uncertain samples. Extensive experiments on four single-domain and six cross-domain FSL benchmarks demonstrate that MPA achieves superior performance compared to existing state-of-the-art methods across most settings. Notably, MPA surpasses the second-best method by 12.29% and 24.56% in the single-domain and cross-domain setting, respectively, in the 5-way 1-shot setting.

Monte Carlo integration is widely used in computer graphics, especially in rendering, but we identify two key limitations. First, for each sample, we can often obtain rich auxiliary information, such as sample position, or geometric information. However, a classical Monte Carlo estimator cannot effectively use this information and only averages the function values. Second, the Monte Carlo formulation makes it difficult to adapt the sampling distribution toward truly informative regions, which can be critical for reconstructing the signal. To address these limitations, we argue that a sampler and integrator beyond the standard Monte Carlo methods is needed. We therefore propose an end-to-end sampling-integration approach that jointly learns both a sampler and an integrator using neural networks, enabling samples to be drawn and used in a more coupled and principled manner. By training on a dataset of integrands, the estimator can further use learned priors over integrand structure and specialize to a family of problems. We evaluate our method on diverse applications in lighting, transmittance, generalized winding number, and walk-on-spheres, spanning both linear and nonlinear cases, and a broad range of low- and high-dimensional settings. Even though the networks add computational overheads, in the equal-sample setting, our method achieves substantial improvements, providing a powerful alternative to traditional quadrature rules and sampling methods.

Novel-view synthesis of specular objects like shiny metals or glossy paints remains a significant challenge. Not only the glossy appearance but also global illumination effects, including reflections of other objects in the environment, are critical components to faithfully reproduce a scene. In this paper, we present Neural Directional Encoding (NDE), a view-dependent appearance encoding of neural radiance fields (NeRF) for rendering specular objects. NDE transfers the concept of feature-grid-based spatial encoding to the angular domain, significantly improving the ability to model high-frequency angular signals. In contrast to previous methods that use encoding functions with only angular input, we additionally cone-trace spatial features to obtain a spatially varying directional encoding, which addresses the challenging interreflection effects Extensive experiments on both synthetic and real datasets show that a NeRF model with NDE (1) outperforms the state of the art on view synthesis of specular objects, and (2) works with small networks to allow fast (real-time) inference. The source code is available at: https://github.com/lwwu2/nde

Deep neural networks are shown to be vulnerable to adversarial examples. Recently, various methods have been proposed to improve the transferability of adversarial examples. However, most of the existing methods add perturbations to the whole image without discrimination, causing the visual quality of the adversarial examples to degrade drastically. In addition, existing attack methods ignore the gradient information of secondary features, which affects the accuracy of generating adversarial perturbations. In this work, we propose Adaptive Attention and Gradient Purification Attack (AAGP) to address such issues. Specifically, we judge the mean and standard deviation of the gradient values to find out where the model is interested. Since different models share similar regions of attention, adding perturbations only to such areas can reduce the addition of adversarial perturbation and can also lead to better transferability of adversarial examples to other models. In addition, we disrupt the correlation of pixels at the distribution of secondary features by random discarding pixels in low-attention areas, generating more transferable perturbations through more accurate gradient information. Experimental results on ImageNet show that our method enhances the visibility of the adversarial examples and their transferability compared with several advanced baselines.

Adversarial examples have the property of transferring across models, which has created a great threat for deep learning models. To reveal the shortcomings in the existing deep learning models, the method of the ensemble has been introduced to the generating of transferable adversarial examples. However, most of the model ensemble attacks directly combine the different models’ output but ignore the large differences in optimization direction of them, which severely limits the transfer attack ability. In this work, we propose a new kind of ensemble attack method called stochastic average ensemble attack. Unlike the existing approach of averaging the outputs of each model as an integrated output, we continuously optimize the ensemble gradient in an internal loop using the model history gradient and the average gradient of different models. In this way, the adversarial examples can be updated in a more appropriate direction and make the crafted adversarial examples more transferable. Experimental results on ImageNet show that our method generates highly transferable adversarial examples and outperforms existing methods.

Abstract Image hiding is a task that hides secret images into cover images. The purposes of image hiding are to ensure the secret images are invisible to the human and the secret images can be recovered. The current state‐of‐the‐art steganography methods run the risk of secret information leakage. A safe image hiding network (SIHNet) is presented to reduce the leakage of secret information. Based on some phenomena of image hiding methods which use invertible neural network, a reversible secret image processing (SIP) module is proposed to make the secret images suitable for hiding and make the stego images leak less secret information. Besides, a reversible lost information hiding (LIH) module is used to hide the lost information into the cover images, thus the method can recover the secret images better than the method that uses random noise to replace the lost information. Experimental results show that SIHNet outperforms other state‐of‐the‐art methods on the PSNR and SSIM values of the recovered secret images and the stego images. Besides, residual images of other state‐of‐the‐art methods all contain information about secret images while residual images of SIHNet leak almost no secret information. Thus the method can prevent the listener of transmission channel from obtaining the information of the secret image through the residual image, which means SIHNet performs better in security than other state‐of‐the‐art methods.

Inverse path tracing has recently been applied to joint material and lighting estimation, given geometry and multi-view HDR observations of an indoor scene. However, it has two major limitations: path tracing is expensive to compute, and ambiguities exist between reflection and emission. Our Factorized Inverse Path Tracing (FIPT) addresses these challenges by using a factored light transport formulation and finds emitters driven by rendering errors. Our algorithm enables accurate material and lighting optimization faster than previous work, and is more effective at resolving ambiguities. The exhaustive experiments on synthetic scenes show that our method (1) outperforms state-of-the-art indoor inverse rendering and relighting methods particularly in the presence of complex illumination effects; (2) speeds up inverse path tracing optimization to less than an hour. We further demonstrate robustness to noisy inputs through material and lighting estimates that allow plausible relighting in a real scene. The source code is available at: https://github.com/lwwu2/fipt

We introduce OpenIllumination, a real-world dataset containing over 108K images of 64 objects with diverse materials, captured under 72 camera views and a large number of different illuminations. For each image in the dataset, we provide accurate camera parameters, illumination ground truth, and foreground segmentation masks. Our dataset enables the quantitative evaluation of most inverse rendering and material decomposition methods for real objects. We examine several state-of-the-art inverse rendering methods on our dataset and compare their performances. The dataset and code can be found on the project page: https://oppo-us-research.github.io/OpenIllumination.

DIVeR builds on the key ideas of NeRF and its variants-density models and volume rendering – to learn 3D object models that can be rendered realistically from small numbers of images. In contrast to all previous NeRF methods, DIVeR uses deterministic rather than stochastic estimates of the volume rendering integral. DIVeR's representation is a voxel based field of features. To compute the volume rendering integral, a ray is broken into intervals, one per voxel; components of the volume rendering integral are estimated from the features for each interval using an MLP, and the components are aggregated. As a result, DIVeR can render thin translucent structures that are missed by other integrators. Furthermore, DIVeR's representation has semantics that is relatively exposed compared to other such methods – moving feature vectors around in the voxel space results in natural edits. Extensive qualitative and quantitative comparisons to current state-of-the-art methods show that DIVeR produces models that (1) render at or above state-of-the-art quality, (2) are very small without being baked, (3) render very fast without being baked, and (4) can be edited in natural ways. Our real-time code is available at: https://github.com/lwwu2/diver-rt

Graph Neural Network (GNN) is a graph representation learning approach for graph-structured data, which has witnessed a remarkable progress in the past few years. As a counterpart, the robustness of such a model has also received considerable attention. Previous studies show that the performance of a well-trained GNN can be faded by black-box adversarial examples significantly. In practice, the attacker can only query the target model with very limited counts, yet the existing methods require hundreds of thousand queries to extend attacks, leading the attacker to be exposed easily. To perform a step forward in addressing this issue, in this paper, we propose a novel attack methods, namely Graph Query-limited Attack (GQA), in which we generate adversarial examples on the surrogate model to fool the target model. Specifically, in GQA, we use contrastive learning to fit the feature extraction layers of the surrogate model in a query-free manner, which can reduce the need of queries. Furthermore, in order to utilize query results sufficiently, we obtain a series of queries with rich information by changing the input iteratively, and storing them in a buffer for recycling usage. Experiments show that GQA can decrease the accuracy of the target model by 4.8%, with only 1% edges modified and 100 queries performed.

Abstract Deep neural networks are widely adopted powerful tools for perceptual tasks. However, recent research indicated that they are easily fooled by adversarial examples, which are produced by adding imperceptible adversarial perturbations to clean examples. Here the steganalysis rich model (SRM) is utilized to generate noise feature maps, and they are combined with RGB images to discover the difference between adversarial examples and clean examples. In particular, a two‐stream pseudo‐siamese network that fuses the subtle difference in RGB images with the noise inconsistency in noise features is proposed. The proposed method has strong detection capability and transferability, and can be combined with any model without modifying its architecture or training procedure. The extensive empirical experiments show that, compared with the state‐of‐the‐art detection methods, the proposed approach achieves excellent performance in distinguishing adversarial samples generated by popular attack methods on different real datasets. Moreover, this method has good generalization, it trained by a specific adversary can defend against other adversaries effectively.

Abstract Arbitrary style transfer aims to stylize the content image with the style image. The key problem of style transfer is how to balance the global content structure and the local style patterns. A promising method to solve this problem is the attentional style transfer method, where a learnable embedding of image features enables style patterns to be flexibly recombined with the content image, so local style patterns will be well preserved in the stylized image. However, current attentional style transfer methods cannot well preserve the global content structure. To solve this problem, a novel attentional style transfer network is proposed, that relies on Optimal Transport (OT) for computing the attention map. The proposed OT‐based attention ensures the similarity between global distributions of the synthesized image and its corresponding style image. For the optimal transport computation, a regularized formulation is used, which not only allows an unbalanced optimal transport to address the deviational distributions but also improves the robustness of stylized results. The proposed method finds a well balance between the global content structure and local style patterns. Various experiments are conducted to demonstrate the superiority of the proposed method over state‐of‐the‐art methods.

Machine learning as a service (MLaaS) brings many benefits to people's daily life. However, the service mode of MLaaS will increase the risk of users' privacy leakage. Existing works focusing on privacy-preserving based on encryption, differential privacy, and distributed framework require high computing resources or cannot be applied in MLaaS. In this paper, we propose feature dilution (FD), a noise-based desensitization algorithm to remove sensitive information in raw data. In particular, FD continuously adds raw data features to the random noise until it meets the minimum amount for an effective query, and we call this noise weak-feature noise (WFN). By fine-tuning the MLaaS architecture, we have realized that users can utilize WFN to get normal services without exposing their local private data. Meanwhile, noise addition technology is introduced by us to reduce the risk of privacy leakage caused by “weak features”. Extensive experiments have demonstrated that users can use FD to obtain effective services without exposing their private data. Finally, we conducted practical tests on weak-feature noises and found that these noises are difficult to use by malicious service providers.

In recent years, tensor-based machine learning methods, in which the Support Tensor Machine (STM) is a typical technology, have gradually attracted the attention of researchers. Compared with Support Vector Machine (SVM), STM has superior generalization ability that can make full use of the structural information of data. However, it still faces many challenges due to the imperfection of its theoretical basis and model. In order to study the further development of STM, this paper provides a survey about the potential and existing problems in STM.