ABSTRACT In recent years, with the rapid advancement of information and communications technology, consumer demands in the beauty and fashion industry have become increasingly diverse and personalised. To address real‐world application challenges, makeup transfer models must be capable of adapting to variations in head poses to ensure accurate makeup transfer across different angles and poses. In response to these challenges, we propose a new progressive makeup transfer model based on spatial style features and feature maps, termed progressive makeup transfer based on style features and feature maps (PMT‐SM). The PMT‐SM model is built upon a vision transformer (ViT) backbone and integrates several key components: feature pyramid network (FPN), makeup feature localiser (MFL), makeup style extractor (MSE) and progressive makeup generator (PMGen). This framework is designed to effectively handle large variations in head poses while ensuring precise and adaptable makeup transfer for real‐world applications. According to the experimental results, PMT‐SM model achieves scores of 0.94, 0.95 and 7.93 in ArcFace similarity, structural similarity index (SSIM) and Fréchet inception distance (FID), respectively, demonstrating superior ability to preserve identity and background information compared with existing makeup transfer models. In addition, the images generated by the proposed model achieve a best‐selected ratio (BSR) of 25.00%, indicating that the PMT‐SM model provides significant advantages in terms of visual quality and naturalness over other makeup transfer methods in the user study.

Screen content coding (SCC) in Versatile Video Coding (VVC) improves the coding efficiency of screen content videos (SCVs) significantly but results in high computational complexity due to the quad-tree plus multi-type tree (QTMT) structure of the coding unit (CU) partitioning. Therefore, we make the first attempt to reduce the encoding complexity from the perspective of CU partitioning for SCC in VVC. To this end, a fast CU partition prediction method is technically developed for VVC-SCC. First, to solve the problem of lacking sufficient SCC training data, SCVs are collected to establish a database containing CUs of various sizes and corresponding partition labels. Second, to determine the partition decision in advance, a novel WA-CNN model is proposed, which is capable of predicting two large CUs for VVC-SCC by adjusting the feature channels based on the size of input CU blocks. Finally, considering the imbalanced proportion of diverse partition decisions, a loss function with the weight that equalizes the contribution of imbalanced data is formulated to train the proposed WA-CNN model. Experimental results show that the proposed model reduces the SCC intra-encoding time by 35.65%${\sim }$38.31% with an average of 1.84%${\sim }$2.42% BDBR increase.

Compared with natural videos,screen content videos(SCVs) have particular features, such as fruitful sharper edges, lots of computer-generated graphics and texts, a large amount of flat areas. New tools are adopted toHEVC extensions on Screen Content Coding(HEVC-SCC), the traditional video rate control methods for natural videos are not effective for SCVs. For that, a3D-gradient guided rate control modelfor SCV coding, named 3DG-RC, is proposed to allocate bitrate more efficiently serving for SCVs. By considering the particular spatial-temporal characteristics of SCVs, the spatial and temporal feature extraction scheme is developed by using 3D-gradient filter and performed on the SCV to extract the spatial and temporal features simultaneously for guiding the bit allocation. The spatial-temporal feature similarity between three original reference SCV frames and their reconstructed ones is used to estimate the encoding parameters of the current block and frame. Experimental results demonstrate that compared with the classical and state-of-the-art rate control methods for HEVC-SCC, the proposed 3DG-RC algorithm achieves significant bitrate mismatch reduction and coding efficiency improvement for HEVC-SCC. In specific, the proposed 3DG-RC model outperforms the rate control model in SCM-8.8 with over 41.33% and 37.95% BD-BR savings on average, forlow delay B(LDB) andrandom access(RA) coding structure, respectively.

The amount of multi-modal data available on the Internet is enormous. Cross-modal hash retrieval maps heterogeneous cross-modal data into a single Hamming space to offer fast and flexible retrieval services. However, existing cross-modal methods mainly rely on the feature-level similarity between multi-modal data and ignore the relationship between relative rankings and label-level fine-grained similarity of neighboring instances. To overcome these issues, we propose a novelDeepCross-modalHashing based onSemanticConsistentRanking (DCH-SCR) that comprehensively investigates the intra-modal semantic similarity relationship. Firstly, to the best of our knowledge, it is an early attempt to preserve semantic similarity for cross-modal hashing retrieval by combining label-level and feature-level information. Secondly, the inherent gap between modalities is narrowed by developing a ranking alignment loss function. Thirdly, the compact and efficient hash codes are optimized based on the common semantic space. Finally, we use the gradient to specify the optimization direction and introduce the Normalized Discounted Cumulative Gain (NDCG) to achieve varying optimization strengths for data pairs with different similarities. Extensive experiments on three real-world image-text retrieval datasets demonstrate the superiority of DCH-SCR over several state-of-the-art cross-modal retrieval methods.

Recently, as one of the most promising biometric traits, the vein has attracted the attention of both academia and industry because of its living body identification and the convenience of the acquisition process. State-of-the-art techniques can provide relatively good performance, yet they are limited to specific light sources. Besides, it still has poor adaptability to multispectral images. Despite the great success achieved by convolutional neural networks (CNNs) in various image understanding tasks, they often require large training samples and high computation that are infeasible for palm-vein identification. To address this limitation, this work proposes a palm-vein identification system based on lightweight CNN and adaptive multi-spectral method with explainable AI. The principal component analysis on symmetric discrete wavelet transform (SMDWT-PCA) technique for vein images augmentation method is adopted to solve the problem of insufficient data and multispectral adaptability. The depth separable convolution (DSC) has been applied to reduce the number of model parameters in this work. To ensure that the experimental result demonstrates accurately and robustly, a multispectral palm image of the public dataset (CASIA) is also used to assess the performance of the proposed method. As result, the palm-vein identification system can provide superior performance to that of the former related approaches for different spectrums.

Formerly, dimensionality reduction techniques are effective ways for extracting statistical significance of features from their original dimensions. However, the dimensionality reduction also induces an additional complexity burden which may encumber the real efficiency. In this paper, a technique is proposed for the reduction of the dimension of samples rather than the features in the former schemes, and it is able to additionally reduce the computational complexity of the applied systems during the reduction process. This method effectively reduces the redundancies of a sample, in particular for those objects which possess partially symmetric property, such as human face, pedestrian, and license plate. As demonstrated in the experiments, based upon the premises of faster speeds in training and detection by a factor of 4.06 and 1.24, respectively, similar accuracies to the ones without considering the proposed method are achieved. The performance verifies that the proposed technique can offer competitive practical values in pattern recognition related fields.

Moving object detection is an important and fundamental step for intelligent video surveillance systems because it provides a focus of attention for post-processing. A multilayer codebook-based background subtraction (MCBS) model is proposed for video sequences to detect moving objects. Combining the multilayer block-based strategy and the adaptive feature extraction from blocks of various sizes, the proposed method can remove most of the nonstationary (dynamic) background and significantly increase the processing efficiency. Moreover, the pixel-based classification is adopted for refining the results from the block-based background subtraction, which can further classify pixels as foreground, shadows, and highlights. As a result, the proposed scheme can provide a high precision and efficient processing speed to meet the requirements of real-time moving object detection.

Memory requirements (for storing intermediate signals) and critical path are essential issues for 2-D (or multidimensional) transforms. This paper presents new algorithms and hardware architectures to address the above issues in 2-D dual-mode (supporting 5/3 lossless and 9/7 lossy coding) lifting-based discrete wavelet transform (LDWT). The proposed 2-D dual-mode LDWT architecture has the merits of low transpose memory (TM), low latency, and regular signal flow, making it suitable for very large-scale integration implementation. The TM requirement of theN×N2-D 5/3 mode LDWT and 2-D 9/7 mode LDWT are 2Nand 4N, respectively. Comparison results indicate that the proposed hardware architecture has a lower lifting-based low TM size requirement than the previous architectures. As a result, it can be applied to real-time visual operations such as JPEG2000, motion-JPEG2000, MPEG-4 still texture object decoding, and wavelet-based scalable video coding applications.

In this study, the current state-of-the-art in moving objects segmentation for intelligent video surveillance has been surveyed. An efficient Modified Directional Lifting-based 9/7 Discrete Wavelet Transform (MDLDWT) structure is proposed to further reduce the computational cost and preserve the fine shape information in low resolution image. Although perfect moving object detection in a practical environment is a challenging task due to the vague object shape issues in the low resolution configuration, the experimental results document that the proposed low complexity MDLDWT scheme can provide more precise detection rate for multiple moving objects, and the fine shape information can be effectively preserved for the real-time video surveillance applications in both indoor and outdoor environments.

This work proposes a new Stereo Vision-Based Self-Localization System (SVBSLS) for the RoboCup soccer humanoid league rules for the 2010 competition. The humanoid robot integrates the information from the pan/tilt motors and stereo vision to accomplish the self-localization and measure the distance of the robot and the soccer ball. The proposed approach uses the trigonometric function to find the coarse distances from the robot to the landmark and the robot to the soccer ball, and then it further adopts the artificial neural network technique to increase the precision of the distance. The statistics approach is also used to calculate the relationship between the humanoid robot and the position of the landmark for self-localization. The experimental results indicate that the localization system of SVBSLS in this research work has 100% average accuracy ratio for localization. The average error of distance from the humanoid soccer robot to the soccer ball is only 0.64 cm.

This paper presents a hierarchical scheme with block-based and pixel-based codebooks for foreground detection. The codebook is mainly used to compress information to achieve a high efficient processing speed. In the block-based stage, 12 intensity values are employed to represent a block. The algorithm extends the concept of the block truncation coding, and thus it can further improve the processing efficiency by enjoying its low complexity advantage. In detail, the block-based stage can remove most of the backgrounds without reducing the true positive rate, yet it has low precision. To overcome this problem, the pixel-based stage is adopted to enhance the precision, which also can reduce the false positive rate. Moreover, the short-term information is employed to improve background updating for adaptive environments. As documented in the experimental results, the proposed algorithm can provide superior performance to that of the former related approaches.

This work presents a new approach, symmetric mask-based scheme (SMBS), for moving object detection and tracking based on the symmetric mask-based discrete wavelet transform (SMDWT). This work presents a fast algorithm, called 2D SMDWT, to improve the critical issue of the 2D lifting-based Discrete Wavelet Transform (LDWT), and then obtains the benefit of low latency, reduced complexity, and low transpose memory for object detection. The successful moving object detection in a real surrounding environment is a difficult task due to noise issues such as fake motion or Gaussian noise. The SMBS approach can effectively reduce noises with low computing cost in both indoor and outdoor environments. The experimental results indicate that the proposed method can provide precise moving object detection and tracking.

This work presents a fast algorithm, namely 2-D Symmetric Mask-based Discrete Wavelet Transform (SMDWT), to address some critical issues of the 2-D Discrete Wavelet Transform (DWT). Unlike the traditional DWT involving dependent decompositions, the SMDWT itself is subband processing independent, which can significantly reduce complexity. Moreover, DWT cannot directly obtain target subbands, which leads to an extra wasting in transpose memory, critical path, and operation time. These problems can be fully improved with the proposed SMDWT. Nowadays, many applications employ DWT as the core transformation approach, the problems indicated above have motivated researchers to develop fast algorithms for DWT. The proposed SMDWT has been proved as a highly efficient independent processing to yield target subbands which can be applied to real-time visual applications, such as moving object detection and tracking, texture segmentation, image/video compression, and any DWT-based applications.

In this work, we propose a memory-efficient architecture of lifting based two-dimensional discrete wavelet transform (2-D DWT) for motion-JPEG2000. The proposed 2-D DWT architecture consists of a 1-D row processor, internal memory, and a 1-D column processor. The main advantage of this 2-D DWT is to reduce the internal memory requirement significantly. For an N×N image, only 2N and 4N sizes of internal memory are required for the 5/3 and 9/7 filters, respectively, to perform the one-level 2-D DWT decomposition. Moreover, it supports both lossless and lossy operation for 5/3 and 9/7 filters with high operation speed. The proposed 2-D DWT surpasses the existed lifting-based designs in the aspects of low internal memory requirement. It is suitable for VLSI implementation and can support various real-time image/video applications such as JPEG2000, motion-JPEG2000, MPEG-4 still texture object decoding, and wavelet-based scalable video coding.

Wavelet coding performs better than discrete cosine transform in visual processing. Moreover, it is scalable, which is important for modern video standards. The transpose memory requirement and operation speed are the two major concerns in 2-D lifting-based discrete wavelet transform (LDWT) implementation. This letter presents a novel algorithm, called 2-D symmetric mask-based discrete wavelet transform (SMDWT), to improve the critical issue of the 2-D LDWT, and then obtains the benefit of low-latency reduced complexity, and low transpose memory. The SMDWT also has the advantages of reduced complexity, regular signal coding, short critical path, reduced latency time, and independent subband coding processing. Furthermore, the 2-D LDWT performance can also be easily improved by exploiting an appropriate parallel method inherent to SMDWT. The proposed method has a significantly better lifting-based latency and complexity in 2-D DWT than normal 2-D 5/3 integer LDWT without degradation in image quality. The algorithm can be applied to real-time image/video applications.

In this work, we propose a high efficiency hardware architecture of embedded sub-band coding with optimal truncation (ESCOT) with pass concurrent context modeling (PCCM) scheme for wavelet-based scalable video coding (SVC). PCCM can merge the three-pass process of bit-plane coding into a single pass process. It improves the efficiency of the ESCOT algorithm and reduces the frequencies of memory access, which can reduce the power consumption. Furthermore we use the parallel architecture scheme of PCCM to encode 4 samples concurrently, which improves the operation speed and can reduce 40% of internal memory requirement. We use Artison TSMC 0.18μm 1P6M standard cell library to design and implement the proposed concurrent context modeling. The simulation results indicate that PCCM can have an operation speedup of 9.5 compared to the standard context modeling of ESCOT, and it can operate for 1080p with frame rate of 30fps at clock rate of 125MHz.

Wavelet coding has been shown to be better than discrete cosine transform (DCT) in image/video processing. Moreover, it has the feature of scalability, which is involved in modern video standards. This work presents novel algorithms, namely 2-D symmetric mask-based discrete wavelet transform (SMDWT), to improve the critical issue of the 2-D lifting-based discrete wavelet transform (LDWT), and then obtains the benefit of low latency, high-speed operation, and low temporal memory. The SMDWT also has the advantages of high-performance embedded periodic extension boundary treatment, reduced complexity, regular signal coding, short critical path, reduced latency time, and independent subband coding processing. Moreover, the 2-D lifting-based DWT performance can also be easily improved by exploiting appropriate parallel method inherently in SMDWT. Comparing with the normal 2-D 5/3 integer lifting-based DWT the proposed method significantly improves lifting-based latency and complexity in 2-D DWT without degradation in image quality. The algorithm can be applied to real-time image/video applications, such as JPEG2000, MPEG-4 still texture object decoding, and wavelet-based Scalable Video Coding (SVC).