Learning-based video compression has attracted more and more attention in recent years. Traditional video coding relies on block-based motion estimation and spatial frequency transformation. While these techniques can effectively compress videos, further enhancing the compression ratio becomes challenging. Introducing deep learning methods can overcome the limitations of manually designed algorithms. In this paper, we propose a learning-based video compression framework with Predicted Frame Refinement (PFR) to improve the compression efficiency. Firstly, a simple autoencoder is introduced to encode the motion information, eliminating the need for a complex optical-flow network. Then, we design a predicted frame refinement network with an attention feature fusion mechanism to generate predicted frames more suitable for extracting context. Finally, we introduce a context coding scheme to improve the compression ratio by jointly utilizing temporal prior and hyper prior. The entire network can be globally optimized and trained from scratch. The experimental result shows that the proposed compression framework outperforms previous methods. Our approach brings 31.2% more saved bit rate than x265 with veryslow preset. Our model also achieves a 7.1% gain in Multi-Scale Structural Similarity Index Measure (MS-SSIM) compared with the recent method proposed by Guo et al.(2023).

The current video coding techniques in the field of reducing redundancy between luma and chroma components have limitations, as they often overlook cross-component correlations. Previous research has employed linear and multi-model linear models to capture cross-component correlations, which are not tailored for screen content sequences. To address this issue, this paper proposes a self-aware cross-component prediction method based on template for screen content coding. With the neighboring reference samples, four prediction models are derived, and chroma prediction values at the template are calculated with the models. The sum of absolute transformed difference (SATD) cost between chroma prediction values and chroma reconstruction values at the template is computed for each model. Subsequently, the model with the lowest SATD cost is determined to be the selected model, which is used to generate prediction values for the current chroma block. Notably, the selected model is adaptively determined at both the encoder and decoder sides consistently, without signaling a model index. Experimental results show that the proposed method achieves 0.73%, 1.62% and 1.75% bit-rate savings on Y, U and V components respectively over ECM 6.0, for class TGM (Text and Graphics with Motion) under All-Intra (AI) configuration.

While current cross-component chroma prediction tools have achieved significant performance improvements, they still face challenges in handling the non-linear relationship between luma and chroma. The paper proposes a template-derived power-law cross-component prediction model, with the key advantage of improving the numerical distribution characteristics of pixels in an interpretable manner. It effectively compresses cross-component redundancy while avoiding overfitting. The model achieves BD-Rate gains of −0.03%, −1.33%, −1.38% under the All-Intra configuration.

Sufficient context is essential for modeling the geometric distribution of large-scale dynamic point clouds. However, previous methods gather the context without considering the distinctive characteristics of different contexts, which leads to suboptimal performances. In this paper, we propose an octree-based diverse context model that captures the large-scale context, local detailed context, and temporal context adaptively and separately. To effectively aggregate the large-scale context, we exploit large-range sibling and ancestor nodes with a dilated mask window. For the local detailed context, we aggregate adjacent encoded sibling nodes with a subsequent mask window. To incorporate temporal context, we propose a density network to take full advantage of the cross-frame information of dynamic point clouds. Experiments on LiDAR and dense object datasets show that our method saves 38.17% and 47.47% of bitrates compared to the MPEG G-PCC method, respectively.

Self-supervised contrastive learning methods have shown promising transferability in pretraining by maximizing the mutual information between two cropped regions as views from the same image. In order to effectively extract mutual information between views, the cropped regions need to be the same instance as prior hypothesis. However, the data collected in general scenes usually have multiple instances, so the two cropped regions probably contain different instances which will mislead the contrastive learning process. In this paper, we make the first attempt to exploit the spatial position relationships of the two cropped regions in self-supervised contrastive learning with images that include multiple instances. Then, we propose an effective method called Space-correlated Contrastive Learning (SpaceCL). Specifically, given two randomly cropped regions as contrastive pairs from the same image, we implement self-supervised contrastive learning by optimizing a space correspondence contrastive similarity loss. As a result, our method achieves state-of-the-art performance and remarkably outperforms other counterparts when pretrained on the COCO dataset of which images contain multiple instances. Experiments show our method outperforms ReSim with 2.6%AP on PASCAL VOC object detection, 0.8%APbband 0.6%APmkon COCO object detection and instance segmentation, 1.3%APmkon Cityscapes instance segmentation.

Recent learning-based intrinsic image decomposition methods have achieved remarkable progress. However, they usually require massive ground truth intrinsic images for supervised learning, which limits their applicability on real-world images since obtaining ground truth intrinsic decomposition for natural images is very challenging. In this paper, we present an unsupervised framework that is able to learn the decomposition effectively from a single natural image by training solely with the image itself. Our approach is built upon the observations that the reflectance of a natural image typically has high internal self-similarity of patches, and a convolutional generation network tends to boost the self-similarity of an image when trained for image reconstruction. Based on the observations, an unsupervised intrinsic decomposition network (UIDNet) consisting of two fully convolutional encoder-decoder sub-networks, i.e., reflectance prediction network (RPN) and shading prediction network (SPN), is devised to decompose an image into reflectance and shading by promoting the internal self-similarity of the reflectance component, in a way that jointly trains RPN and SPN to reproduce the given image. A novel loss function is also designed to make effective the training for intrinsic decomposition. Experimental results on three benchmark real-world datasets demonstrate the superiority of the proposed method.

The number of Internet of Things (IoT) devices has exploded in recent years. Due to the simple implementation and difficult-to-patch firmware, IoT devices are vulnerable to malware attacks. Static analysis is a feasible way to understand the behavior of IoT malware for detection and mitigation. However, unlike traditional malware on personal computers or smartphones, the diversity of processor architecture on IoT devices brings a variety of challenges for researchers. Current malware detection methods based on operation code or byte code cannot address the multi-architecture issue well. In this paper, we propose a cross-architecture IoT malware detection method based on graph neural network(GNN). We represent each binary file as a function call graph(FCG), since FCG is a higher-level architecture-independent feature. Natural language processing model is used to extract semantic information from operation code in our method. Enable semantic information as node feature and then we use GNN to extract structural information from FCG. Our method takes both semantic and structural information into account to identify malware. We also create a dataset that covers 5 different processor architectures to evaluate our method. The experiment we conduct over the dataset shows that our method performs better than other methods and is capable to detect unknown malware.

Recent image aesthetic assessment methods have achieved remarkable progress due to the emergence of deep convolutional neural networks (CNNs). However, these methods focus primarily on predicting generally perceived preference of an image, making them usually have limited practicability, since each user may have completely different preferences for the same image. To address this problem, this paper presents a novel approach for predicting personalized image aesthetics that fit an individual user’s personal taste. We achieve this in a coarse to fine manner, by joint regression and learning from pairwise rankings. Specifically, we first collect a small subset of personal images from a user and invite him/her to rank the preference of some randomly sampled image pairs. We then search for the K -nearest neighbors of the personal images within a large-scale dataset labeled with average human aesthetic scores, and use these images as well as the associated scores to train a generic aesthetic assessment model by CNN-based regression. Next, we fine-tune the generic model to accommodate the personal preference by training over the rankings with a pairwise hinge loss. Experiments demonstrate that our method can effectively learn personalized image aesthetic preferences, clearly outperforming state-of-the-art methods. Moreover, we show that the learned personalized image aesthetic benefits a wide variety of applications.

Most state-of-the-art binary image steganographic techniques only consider the flipping distortion according to the human visual system, which will be not secure when they are attacked by steganalyzers. In this paper, a binary image steganographic scheme that aims to minimize the embedding distortion on the texture is presented. We extract the complement, rotation, and mirroring-invariant local texture patterns (crmiLTPs) from the binary image first. The weighted sum of crmiLTP changes when flipping one pixel is then employed to measure the flipping distortion corresponding to that pixel. By testing on both simple binary images and the constructed image data set, we show that the proposed measurement can well describe the distortions on both visual quality and statistics. Based on the proposed measurement, a practical steganographic scheme is developed. The steganographic scheme generates the cover vector by dividing the scrambled image into superpixels. Thereafter, the syndrome-trellis code is employed to minimize the designed embedding distortion. Experimental results have demonstrated that the proposed steganographic scheme can achieve statistical security without degrading the image quality or the embedding capacity.

The XOR-based visual cryptography (VC) is a possible methodology to solve the poor visual quality problem without darkening the background in VC. However, investigations on XOR-based VC are not sufficient. In this paper, we exploit some extended capabilities for XOR-based VC. Actually, two XOR-based VC algorithms are proposed, namely XOR-based VC for general access structure (GAS) and adaptive region incrementing XOR-based VC. The first algorithm aims to implement complicated sharing strategy using GAS, while maintaining merits such as perfect reconstruction of secret, no pixel expansion, and no code book requirement. In the second algorithm, the concept of adaptive security level is first introduced, where the security levels are recovered in accordance with the qualified sets instead of the quantity of stacked shares. Adaptive region incrementing XOR-based VC further enriches the application scenarios. Theoretical analysis on the proposed algorithms are provided, as well as extensive experimental results and evaluations for demonstrating the effectiveness and advantages of the two methods.

In recent years, random grid (RG) received much attention for constructing visual secret sharing (VSS) scheme without pixel expansion. But recovered secret image with low visual quality reveals due to the stacking operation. To improve the recovered image quality, XOR-based VSS is adopted. However, shares constructed from XOR-based VSS are random-looking. The noise-like appearance further increases the chance of suspicion on secret image communication and imposes difficulty for managing the shares. In this work, a novel (2,2) generalized RG-based VSS is introduced. By adopting the (2,2) VSS, we propose a XOR-based meaningful VSS where shares with meaningful contents are constructed. Moreover, superior visual quality is provided by this method as well.

Random grid (RG) is a method to implement visual cryptography (VC) without pixel expansion. However, a reconstructed secret with lower visual quality reveals in RG-based VC due to the fact that average light transmission of a share is fixed at 1/2. In this work, we introduce the concept of generalized RG, where the light transmission of a share becomes adjustable, and adopt generalized RG to implement different VC schemes. First, a basic algorithm, a (2,2) generalized RG-based VC, is devised. Based on the (2,2) scheme, two VC schemes including a (2,n) generalized RG-based VC and a (n,n) xor-based meaningful VC are constructed. The two derived algorithms are designed to solve different problems in VC. In the (2,n) scheme, recovered image quality is further improved. In the (n,n) method, meaningful shares are constructed so that the management of shadows becomes more efficient, and the chance of suspicion on secret image encryption is reduced. Moreover, superior visual quality of both the shares and recovered secret image is achieved. Theoretical analysis and experimental results are provided as well, demonstrating the effectiveness and advantages of the proposed algorithms.

Visual secret sharing (VSS) is a way to protect a secret image among a group of participants by using the notions of perfect ciphers and secret sharing. However, each share generated by conventional VSS is m times as big as the original secret image, where m is called pixel expansion. Random grid (RG) is an alternative approach to implement VSS without pixel expansion. However, reported RG-based VSS methods are threshold schemes. In this study, RG-based VSS for general access structures is presented. Secret image is encoded into n RGs while qualified sets can recover the secret visually and forbidden sets cannot. The proposed scheme is a generalisation of the threshold methods, where those reported RG-based schemes can be considered as the special cases of the proposed scheme. Experimental results are provided, demonstrating the effectiveness and advantages of the proposed scheme.

Engineering graph plays an important role in design and manufacture, such as architecture, machinery, manufacture, military and so on. However, almost no persons consider the security and copyright of two-dimensional engineering graph. A novel method for two-dimensional engineering graph watermark based U system is proposed in this paper. Watermarks generated by this technique can be successfully extracted even after rotated, translated, and scaling transformed.