Introducing high-quality references can largely alleviate the uncertainty in blind face image restoration tasks, yet the equivocal utilization of reference priors makes it still a struggle to well preserve the human identity. We attribute the identity inconsistency to two deficiencies of existing reference-based face restoration methods, namely the inability to effectively determine which features need to be transferred, and the failure to preserve the structure and details of the selected features. This work mainly focuses on these two issues, and we present a novel blind face image restoration method that considers reference selection, transfer, and reconstruction (RefSTAR) to introduce proper features from reference images. Specifically, we construct a reference selection (RefSel) module, which can generate accurate masks to select reference features. For training the RefSel module, we construct a RefSel-HQ dataset through a mask generation pipeline, which contains annotated masks for 10,000 ground truth-reference pairs. To guarantee the exact introduction of selected reference features, a feature fusion paradigm is designed for reference feature transferring, and a Mask-Compatible Cycle-Consistency Loss is redesigned based on reference reconstruction to further ensure the presence of selected reference image features in the output image. Experiments on various backbone models demonstrate superior performance, showing better identity preservation ability and reference feature transfer quality.

Multiple cameras can provide comprehensive multi-view video coverage of a person. Fusing this multi-view data is crucial for tasks like behavioral analysis, although it traditionally requires camera calibration—a process that is often complex. Moreover, previous studies have overlooked the challenges posed by self-occlusion under multiple views and the continuity of human body shape estimation. In this study, we introduce a method to reconstruct the 3D human body from multiple uncalibrated camera views. Initially, we utilize a pre-trained human body encoder to process each camera view individually, enabling the reconstruction of human body models and parameters for each view along with predicted camera positions. Rather than merely averaging the models across views, we develop a neural network trained to assign weights to individual views for all human body joints, based on the estimated distribution of joint distances from each camera. Additionally, we focus on the mesh surface of the human body for dynamic fusion, allowing for the seamless integration of facial expressions and body shape into a unified human body model. Our method has shown excellent performance in reconstructing the human body on two public datasets, advancing beyond previous work from the SMPL model to the SMPL-X model. This extension incorporates more complex hand poses and facial expressions, enhancing the detail and accuracy of the reconstructions. Crucially, it supports the flexible ad-hoc deployment of any number of cameras, offering significant potential for various applications.

Recent state-of-the-art image restoration methods mostly adopt latent diffusion models with U-Net backbones, yet still facing challenges in achieving high-quality restoration due to their limited capabilities. Diffusion transformers (DiTs), like SD3, are emerging as a promising alternative because of their better quality with scalability. In this paper, we introduce DPIR (Dual Prompting Image Restoration), a novel image restoration method that effectivly extracts conditional information of low-quality images from multiple perspectives. Specifically, DPIR consits of two branches: a low-quality image conditioning branch and a dual prompting control branch. The first branch utilizes a lightweight module to incorporate image priors into the DiT with high efficiency. More importantly, we believe that in image restoration, textual description alone cannot fully capture its rich visual characteristics. Therefore, a dual prompting module is designed to provide DiT with additional visual cues, capturing both global context and local appearance. The extracted global-local visual prompts as extra conditional control, alongside textual prompts to form dual prompts, greatly enhance the quality of the restoration. Extensive experimental results demonstrate that DPIR delivers superior image restoration performance.

Diffusion-based methodologies have shown significant potential in blind face restoration (BFR), leveraging their robust generative capabilities. However, they are often criticized for two significant problems: 1) slow training and inference speed, and 2) inadequate recovery of fine-grained facial details. To address these problems, we propose a novel Truncated Diffusion model for efficient Blind Face Restoration (TD-BFR), a three-stage paradigm tailored for the progressive resolution of degraded images. Specifically, TD-BFR utilizes an innovative truncated sampling method, starting from low-quality (LQ) images at low resolution to enhance sampling speed, and then introduces an adaptive degradation removal module to handle unknown degradations and connect the generation processes across different resolutions. Additionally, we further adapt the priors of pre-trained diffusion models to recover rich facial details. Our method efficiently restores high-quality images in a coarse-to-fine manner and experimental results demonstrate that TD-BFR is, on average, 4.75× faster than current state-of-the-art diffusion-based BFR methods while maintaining competitive quality.

Recent advances in diffusion models have substantially improved text-driven image editing. However, existing frameworks based on discrete textual tokens struggle to support continuous control over camera parameters and smooth transitions in visual effects. These limitations hinder their applications to realistic, camera-aware, and fine-grained editing tasks. In this paper, we present CamEdit, a diffusion-based framework for photorealistic image editing that enables continuous and semantically meaningful manipulation of common camera parameters such as aperture and shutter speed. CamEdit incorporates a continuous parameter prompting mechanism and a parameter-aware modulation module that guides the model in smoothly adjusting focal plane, aperture, and shutter speed, reflecting the effects of varying camera settings within the diffusion process. To support supervised learning in this setting, we introduce CamEdit50K, a dataset specifically designed for photorealistic image editing with continuous camera parameter settings. It contains over 50k image pairs combining real and synthetic data with dense camera parameter variations across diverse scenes. Extensive experiments demonstrate that CamEdit enables flexible, consistent, and high-fidelity image editing, achieving state-of-the-art performance in camera-aware visual manipulation and fine-grained photographic control.

Real-world image super-resolution (RealSR) aims to enhance the visual quality of in-the-wild images, such as those captured by mobile phones. While existing methods leveraging large generative models demonstrate impressive results, the high computational cost and latency make them impractical for edge deployment. In this paper, we introduce PocketSR, an ultra-lightweight, single-step model that brings generative modeling capabilities to RealSR while maintaining high fidelity. To achieve this, we design LiteED, a highly efficient alternative to the original computationally intensive VAE in SD, reducing parameters by 97.5\% while preserving high-quality encoding and decoding. Additionally, we propose online annealing pruning for the U-Net, which progressively shifts generative priors from heavy modules to lightweight counterparts, ensuring effective knowledge transfer and further optimizing efficiency. To mitigate the loss of prior knowledge during pruning, we incorporate a multi-layer feature distillation loss. Through an in-depth analysis of each design component, we provide valuable insights for future research. PocketSR, with a model size of 146M parameters, processes 4K images in just 0.8 seconds, achieving a remarkable speedup over previous methods. Notably, it delivers performance on par with state-of-the-art single-step and even multi-step RealSR models, making it a highly practical solution for edge-device applications.

Traditional Visual Simultaneous Localization and Mapping (VSLAM) algorithms demonstrate good accuracy and robustness in well-lighting environments by extracting numerous feature points. However, in low-light conditions, insufficient illumination leads to low-contrast images, which hampers the ability of the front-end to extract adequate feature points, resulting in increased tracking errors or complete tracking failure. To overcome these limitations, this paper improves the performance of the ORB-SLAM3 in low-light environments with image enhancement capability. Specifically, we integrate the Histogram Equalization Prior-based (HEP) image enhancement module into ORB-SLAM3. Furthermore, we compare and analyze the results with two other image enhancement algorithms to ensure the effective fusion of image enhancement and ORB-SLAM3. Experiments were conducted and performance comparisons were made to validate the performance of ORB-SLAM3 with image enhancement in low-light environments. Experimental results indicate that the Root Mean Square Errors (RMSE) of the Absolute Pose Error (APE) are 0.99 cm and 0.76 cm on the two public ETH3D low-light datasets, respectively. Compared to the original ORB-SLAM3, the improvement is over 50%. Similarly, the RMSE of the Relative Pose Error (RPE) are 0.54 cm and 0.65 cm, with an improvement of 29% and 34%, respectively.

Protein-ligand interactions are increasingly profiled at high-throughput, playing a vital role in lead compound discovery and drug optimization. Accurate prediction of binding pose and binding affinity constitutes a pivotal challenge in advancing our computational understanding of protein-ligand interactions. However, inherent limitations still exist, including high computational cost for conformational search sampling in traditional molecular docking tools, and the unsatisfactory molecular representation learning and intermolecular interaction modeling in deep learning-based methods. Here we propose a geometry-aware attention-based deep learning model, GAABind, which effectively predicts the pocket-ligand binding pose and binding affinity within a multi-task learning framework. Specifically, GAABind comprehensively captures the geometric and topological properties of both binding pockets and ligands, and employs expressive molecular representation learning to model intramolecular interactions. Moreover, GAABind proficiently learns the intermolecular many-body interactions and simulates the dynamic conformational adaptations of the ligand during its interaction with the protein through meticulously designed networks. We trained GAABind on the PDBbindv2020 and evaluated it on the CASF2016 dataset; the results indicate that GAABind achieves state-of-the-art performance in binding pose prediction and shows comparable binding affinity prediction performance. Notably, GAABind achieves a success rate of 82.8% in binding pose prediction, and the Pearson correlation between predicted and experimental binding affinities reaches up to 0.803. Additionally, we assessed GAABind's performance on the severe acute respiratory syndrome coronavirus 2 main protease cross-docking dataset. In this evaluation, GAABind demonstrates a notable success rate of 76.5% in binding pose prediction and achieves the highest Pearson correlation coefficient in binding affinity prediction compared with all baseline methods.

Blind face restoration usually synthesizes degraded low-quality data with a pre-defined degradation model for training, while more complex cases could happen in the real world. This gap between the assumed and actual degradation hurts the restoration performance where artifacts are often observed in the output. However, it is expensive and infeasible to include every type of degradation to cover real-world cases in the training data. To tackle this robustness issue, we propose Diffusion-based Robust Degradation Remover (DR2) to first transform the degraded image to a coarse but degradation-invariant prediction, then employ an enhancement module to restore the coarse prediction to a high-quality image. By leveraging a well-performing denoising diffusion probabilistic model, our DR2 diffuses input images to a noisy status where various types of degradation give way to Gaussian noise, and then captures semantic information through iterative denoising steps. As a result, DR2 is robust against common degradation (e.g. blur, resize, noise and compression) and compatible with different designs of enhancement modules. Experiments in various settings show that our framework outperforms state-of-the-art methods on heavily degraded synthetic and real-world datasets.

As a representative of nonfunctional features, the Quality-of-Service (QoS) plays an important role in recommending the best service to users. To obtain the QoS value of web services, researchers have proposed many web service QoS prediction methods. However, most existing methods, e.g., the collaborative filtering method, leverage the historical user service invocation information to predict the QoS value, and only consider the QoS information of similar users or services, and ignore the attributes and characteristics of users and services. In this paper, we propose a Deep Feature Interaction Learning method, called DFIL for short, for web service QoS prediction. Specifically, (1) DFIL constructs a feature extraction method to effectively obtain the features of users and services. (2) DFIL trains a novel neural network for QoS prediction, which can accurately find the potential relationship between users and services based on the feature information, and provide good web service recommendation.

With the increasing prevalence of web services on the World Wide Web, a large number of functionally equivalent web services are provided by different providers. Quality-of-Service (QoS), representing the nonfunctional characteristics, plays an important role in dealing with how to recommend the optimal services to users among these candidates. Many existing methods for predicting QoS values of web services show that QoS values are intensively relevant to location due to the great influence of network distance and the internet connection between users and services. In this paper, we propose a novel location-aware feature interaction learning (LAFIL) method for predicting the QoS values of the user-service matrix and then making the recommendation by learning the underlying relation, which is hidden in the features concerning with location information. LAFIL can effectively solve the problems of data sparsity and cold-start by leveraging the location features of both users and services. To evaluate the performance of our proposed method, comprehensive experiments are conducted using a real-world dataset and the results show that our method achieves better QoS prediction accuracy compared to state-of-the-art approaches.

Fine-grained object categorization aims for distinguishing objects of subordinate categories that belong to the same entry-level object category. It is a rapidly developing subfield in multimedia content analysis. The task is challenging due to the facts that (1) training images with ground-truth labels are difficult to obtain, and (2) variations among different subordinate categories are subtle. It is well established that characterizing features of different subordinate categories are located on local parts of object instances. However, manually annotating object parts requires expertise, which is also difficult to generalize to new fine-grained categorization tasks. In this work, we propose a Weakly Supervised Part Detection Network (PartNet) that is able to detect discriminative local parts for the use of fine-grained categorization. A vanilla PartNet builds on top of a base subnetwork two parallel streams of upper network layers, which respectively compute scores of classification probabilities (over subordinate categories) and detection probabilities (over a specified number of discriminative part detectors) for local regions of interest (RoIs). The image-level prediction is obtained by aggregating element-wise products of these region-level probabilities, and meanwhile diverse part detectors can be learned in an end-to-end fashion under the image-level supervision. To generate a diverse set of RoIs as inputs of PartNet, we propose a simple Discretized Part Proposals module (DPP) that directly targets for proposing candidates of discriminative local parts, with no bridging via object-level proposals. Experiments on benchmark datasets of CUB-200-2011, Oxford Flower 102 and Oxford-IIIT Pet show the efficacy of our proposed method for both discriminative part detection and fine-grained categorization. In particular, we achieve the new state-of-the-art performance on CUB-200-2011 and Oxford-IIIT Pet datasets when ground-truth part annotations are not available.

In this work, we propose a novel method termed Frustum ConvNet (F-ConvNet) for amodal 3D object detection from point clouds. Given 2D region proposals in an RGB image, our method first generates a sequence of frustums for each region proposal, and uses the obtained frustums to group local points. F-ConvNet aggregates point-wise features as frustum-level feature vectors, and arrays these feature vectors as a feature map for use of its subsequent component of fully convolutional network (FCN), which spatially fuses frustum-level features and supports an end-to-end and continuous estimation of oriented boxes in the 3D space. We also propose component variants of F-ConvNet, including an FCN variant that extracts multi-resolution frustum features, and a refined use of F-ConvNet over a reduced 3D space. Careful ablation studies verify the efficacy of these component variants. F-ConvNet assumes no prior knowledge of the working 3D environment and is thus dataset-agnostic. We present experiments on both the indoor SUN-RGBD and outdoor KITTI datasets. F-ConvNet outperforms all existing methods on SUN-RGBD, and at the time of submission it outperforms all published works on the KITTI benchmark. Code has been made available at: https://github.com/zhixinwang/frustum-convnet.

A continuous function model is developed for efficient head-related impulse response (HRIR) representation. In this model, HRIR is modeled as the response of an IIR filter via common factor decomposition analysis, where HRIRs at the same azimuth but different elevations share the same IIR pole part and HRIRs at the same elevation but different azimuthes share the same IIR zero part. The pole and zero coefficients of the proposed common factor IIR (CF-IIR) filters are further modeled using sinusoidal functions of azimuth and elevation, respectively. A fast algorithm for evaluating sinusoidal function for efficient synthesis of virtual 3D sound is also developed.

In this letter, a Two-Dimension Common Factor Decomposition (2D-CFD) algorithm is proposed to represent the 3D (time, azimuth and elevation) head-related impulse response (HRIR) dataset with two factorized matrices. One matrix is made up of a set of common factor responses extracted from HRIRs with the same azimuth angle and carries azimuthal information contained in original HRIR dataset. The other matrix carries elevation information similarly. By using this dimension reduction, the storage requirement of HRIRs is reduced remarkably. In addition, the common factor responses in the matrix are further modeled with efficient low order IIR filters to reduce the computation complexity in 3D sound synthesis.

In this paper, Principal Component Analysis (PCA) is performed on HRIRs grouped according to azimuth and covariance to improve distortion performance while reducing storage requirement. By further combining Balanced Model Truncation (BMT) using a joint optimization scheme, a virtual 3D sound synthesis scheme with low computation complexity, little storage requirement and efficient linear interpolation is proposed. A smaller model distortion is achieved as high PCA model error of contralateral HRIR is decreased by grouping PCA strategy.

As investment in e-learning accelerates rapidly worldwide, it is important to improve the economic performance in the existing e-learning initiatives. This article presents a model for analyzing and evaluating the return on investment (ROI) in e-learning. The article also explains each component of the model in details. The model highlights the important issues that must be addressed to optimize the investment strategy. The model also further identifies the return on the investment by the mathematical method of fuzzy comprehensive evaluation.

A downside brought about by the explosive growth of online information and the shear amount of data traffic moving through our networks is the modern day headaches of information overload. The growth of handheld wireless devices can only mean that the end-users will continually be bombarded by emails and real-time information feeds, but now, even as they are on the go. Software agents provide mechanisms that have good potentials to lower the amount of information that an end- user has to deal with. Agent-enabled applications allow the end- user to personalize the way online resources are presented and can also filter out irrelevant or unwanted information. In this paper, we present our experience in using software agents and the Composite Capabilities/Preference Profiles (CC/PP) for customizing and delivering mobile services for Java 2 Micro Edition (J2ME) enabled and Wireless Application Protocol (WAP) enabled devices. We use software agents because such autonomous software entities have characteristics that can benefit mobile devices and the wireless environment, and the CC/PP is a standard for defining profiles for user preferences and device capabilities.