Despite the superiority of convolutional neural networks (CNNs) and Transformers in single-image rain removal, current multi-scale models still face significant challenges due to their reliance on single-scale feature pyramid patterns. In this paper, we propose an effective rain removal method, the dual-path multi-scale Transformer (DPMformer) for high-quality image reconstruction by leveraging rich multi-scale information. This method consists of a backbone path and two branch paths from two different multi-scale approaches. Specifically, one path adopts the coarse-to-fine strategy, progressively downsampling the image to 1/2 and 1/4 scales, which helps capture fine-scale potential rain information fusion. Simultaneously, we employ the multi-patch stacked model (non-overlapping blocks of size 2 and 4) to enrich the feature information of the deep network in the other path. To learn a richer blend of features, the backbone path fully utilizes the multi-scale information to achieve high-quality rain removal image reconstruction. Extensive experiments on benchmark datasets demonstrate that our model reaches superior performance, significantly improving the image deraining quality.

Heterogeneous networks based on multicolor visible light communication (VLC) and wireless fidelity (WiFi) have been considered as a key technology to achieve the capacity target in the future 6G mobile communication. However, the inter-cell interference (ICI) pattern in the multi-cell VLC scenario degrades the performance of the heterogeneous networks. To solve this problem, a multi-cell resource allocation mechanism based on interference control (MCRAMIC) is proposed, which includes a preparatory phase and an execution phase. Firstly, according to the different locations and data rate requirements of varied user equipments (UEs), the VLC access point (AP) selection algorithm based on interference avoidance and the UE priority assessment algorithm based on the requirement of UE are proposed.Then, according to the influence factor and priority factor, the multi-cell resource allocation algorithm based on interference control is proposed. According to the algorithm, the candidate VLC AP sets of the UEs are determined firstly, and the VLC APs and lightwave bands are selected for the UEs in turn. Meanwhile, some UEs are connected to the WiFi AP. Finally, the lightwave bands are further allocated to the UEs. Numeric results show that the proposed MCRAMIC outperforms the centralized resource allocation algorithm based on link conflict graph (LCG) and the resource allocation algorithm based on hypergraph theory in terms of the system throughput, UE satisfaction and service fairness.

In this paper, we focus on unsupervised learning for Video Object Segmentation (VOS) which learns visual correspondence (i.e., the similarity between pixel-level features) from unlabeled videos. Previous methods are mainly based on the contrastive learning paradigm, which optimize either in image level or pixel level. Image-level optimization (e.g., the spatially pooled feature of ResNet) learns robust high-level semantics but is sub-optimal since the pixel-level features are optimized implicitly. By contrast, pixel-level optimization is more explicit, however, it is sensitive to the visual quality of training data and is not robust to object deformation. To complementarily perform these two levels of optimization in a unified framework, we propose the In-aNd-Out (INO) generative learning from a purely generative perspective with the help of naturally designed class tokens and patch tokens in Vision Transformer (ViT). Specifically, for image-level optimization, we force the out-view imagination from local to global views on class tokens, which helps capture high-level semantics, and we name it as out-generative learning. As to pixel-level optimization, we perform in-view masked image modeling on patch tokens, which recovers the corrupted parts of an image via inferring its fine-grained structure, and we term it as in-generative learning. To discover the temporal information better, we additionally force the inter-frame consistency from both feature and affinity matrix levels. Extensive experiments on DAVIS-2017 val and YouTube-VOS 2018 val show that our INO outperforms previous state-of-the-art methods by significant margins.

Fine-grained image classification has drawn increasing attention as it is much closer to practical applications than generic image classification. The majority of current fine-grained approaches locate the discriminative regions and leverage the features of these regions for classification as their magic weapons. However, these approaches simply ignore the internal semantic region correlation. As is well known, the correlation reveals the salient information of images, which can further boost the performance of fine-grained image classification. To this end, we propose an Object Decoupling with Graph Correlation network (ODGC) to explore the informative potentials of region correlation. A Responsive Object Location Module (ROLM) is first introduced to obtain the fine-grained object within a bounding box automatically. A Semantic Decoupling Module (SDM) then segments the object into different parts. ODGC learns the representations of these parts by transferring these part features into a Graph Correlation Module (GCM). Consists of these three main modules, ODGC is trained for fine-grained image classification in an end-to-end way. Extensive experiments conducted on CUB-200-2011 demonstrate that the aforementioned modules significantly improve the ODGC, and it achieves a new state-of-the-art performance to 88.2% top-1 accuracy. Besides, we collect a practical business e-commercial dataset, named Ecom-15K. The evaluation on it further validates the applicability of our method in practical scenarios.

Face recognition across age progression remains one of the area's most challenging tasks, as the aging process affects both the shape and texture of a face. One possible solution is to apply a probabilistic model to represent a face simultaneously with its identity variable, which is stable through time, and its aging variable, which changes with time. However, as the aging process varies for different people, a person may look younger or older than another person, even though their ages are the same. Consequently, using the ‘real’ age labels given by existing face datasets for age-invariant face recognition will inevitably introduce ambiguity to learning algorithms. In this paper, an identity-inference model, based on age-subspace learning from appearance-age labels, is proposed. We first model human identity and aging variables simultaneously using Probabilistic Linear Discriminant Analysis (PLDA). Then, the aging subspace is learnt independently with the appearance-age labels, and the identity subspace is then determined iteratively with the Expectation-Maximization (EM) algorithm. We found that the learned aging subspace is insensitive to the training face images used, and is independent of the identity model. Consequently, the recognition of aging faces becomes simpler as identity inference no longer needs to consider age labels. Furthermore, in our algorithm, different identity features learnt from the identity model are further combined using Canonical Correlation Analysis (CCA), where their correlations are maximized for face recognition. A thorough experimental analysis of face recognition is performed on three public domain face-aging datasets: FGNET, MORPH, and CACD. Experiment results show that the proposed framework can achieve a comparable, or even better, performance against other state-of-the-art methods, especially when the age range is large.

Among the challenges faced by current active shape or appearance models, facial-feature localization in the wild, with occlusion in a novel face image, i.e. in a generic environment, is regarded as one of the most difficult computer-vision tasks. In this paper, we propose an Active Appearance Model (AAM) to tackle the problem of generic environment. Firstly, a fast face-model initialization scheme is proposed, based on the idea that the local appearance of feature points can be accurately approximated with locality constraints. Nearest neighbors, which have similar poses and textures to a test face, are retrieved from a training set for constructing the initial face model . To further improve the fitting of the initial model to the test face, an orthogonal CCA (oCCA) is employed to increase the correlation between shape features and appearance features represented by Principal Component Analysis (PCA). With these two contributions, we propose a novel AAM, namely the shape-appearance-correlated AAM (SAC-AAM), and the optimization is solved by using the recently proposed fast simultaneous inverse compositional (Fast-SIC) algorithm. Experiment results demonstrate a 5–10% improvement on controlled and semi-controlled datasets, and with around 10% improvement on wild face datasets in terms of fitting accuracy compared to other state-of-the-art AAM models.

Face recognition methods, which usually represent face images using holistic or local facial features, rely heavily on alignment. Their performances also suffer a severe degradation under variations in expressions or poses, especially when there is one gallery per subject only. With the easy access to high-resolution (HR) face images nowadays, some HR face databases have recently been developed. However, few studies have tackled the use of HR information for face recognition or verification. In this paper, we propose a pose-invariant face-verification method, which is robust to alignment errors, using the HR information based on pore-scale facial features. A new keypoint descriptor, namely, pore-Principal Component Analysis (PCA)-Scale Invariant Feature Transform (PPCASIFT)-adapted from PCA-SIFT-is devised for the extraction of a compact set of distinctive pore-scale facial features. Having matched the pore-scale features of two-face regions, an effective robust-fitting scheme is proposed for the face-verification task. Experiments show that, with one frontal-view gallery only per subject, our proposed method outperforms a number of standard verification methods, and can achieve excellent accuracy even the faces are under large variations in expression and pose.

In this paper, a novel algorithm based on the Active Shape Model (ASM) for locating landmarks on human faces is proposed. A challenge for detecting facial features is that faces may be under different poses, this makes the local appearance of each facial landmark vary greatly. To account for these variations, we propose an adaptive-profile scheme for ASM so that facial landmarks can be detected reliably and accurately under different poses. In our algorithm, a 2D profile is used for each landmark, and the 2D profiles of each landmark of the training face images are grouped to form a number of clusters. The corresponding shape vector for each of the clusters is then learned. For a query face image, the profiles to be used to locate the respective facial landmarks will be selected according to the face-shape vector in the current iteration. In other words, adaptive profiles are used in the search for landmarks. Face images from two subsets of the IMM Face Database are used for training, and the other two subsets are used for testing. The performance of our proposed algorithm is also evaluated using another dataset, namely the Bosphorus Dataset. Experiment results show that our proposed Adaptive-Profile Active Shape Model (APASM) can locate facial landmarks accurately under different face shapes, expressions, and poses.