Vector-Quantized Image Modeling (VQIM) is a fundamental research problem in image synthesis, which aims to represent an image with a discrete token sequence. Existing studies effectively address this problem by learning a discrete codebook from scratch and in a code-independent manner to quantize continuous representations into discrete tokens. However, learning a codebook from scratch and in a code-independent manner is highly challenging, which may be a key reason causing codebook collapse, i.e., some code vectors can rarely be optimized without regard to the relationship between codes and good codebook priors such that die off finally. In this paper, inspired by pretrained language models, we find that these language models have actually pretrained a superior codebook via a large number of text corpus, but such information is rarely exploited in VQIM. To this end, we propose a novel codebook transfer framework with part-of-speech, called VQCT, which aims to transfer a well-trained codebook from pretrained language models to VQIM for robust codebook learning. Specifically, we first introduce a pretrained codebook from language models and part-of-speech knowledge as priors. Then, we construct a vision-related codebook with these priors for achieving codebook transfer. Finally, a novel codebook transfer network is designed to exploit abundant semantic relationships between codes contained in pretrained codebooks for robust VQIM codebook learning. Experimental results on four datasets show that our VQCT method achieves superior VQIM performance over previous state-of-the-art methods.

Vector quantization (VQ) is a key technique in high-resolution and high-fidelity image synthesis, which aims to learn a codebook to encode an image with a sequence of discrete codes and then generate an image in an auto-regression manner. Although existing methods have shown superior performance, most methods prefer to learn a single-modal codebook (\emph{e.g.}, image), resulting in suboptimal performance when the codebook is applied to multi-modal downstream tasks (\emph{e.g.}, text-to-image, image captioning) due to the existence of modal gaps. In this paper, we propose a novel language-guided codebook learning framework, called LG-VQ, which aims to learn a codebook that can be aligned with the text to improve the performance of multi-modal downstream tasks. Specifically, we first introduce pre-trained text semantics as prior knowledge, then design two novel alignment modules (\emph{i.e.}, Semantic Alignment Module, and Relationship Alignment Module) to transfer such prior knowledge into codes for achieving codebook text alignment. In particular, our LG-VQ method is model-agnostic, which can be easily integrated into existing VQ models. Experimental results show that our method achieves superior performance on reconstruction and various multi-modal downstream tasks.

Detecting and eliminating clouds is a crucial step in remote sensing image (RSI) preprocessing. The removal of clouds can significantly enhance the performance of subsequent remote sensing applications. Existing deep learning (DL)-based cloud detection methods extract semantic information to improve feature representation and, subsequently, detection performance. However, these methods do not fully utilize the potential of context semantic information. Besides, to capture semantics from large receptive fields, they employ convolution operators with large kernel sizes, which results in high computational costs. Thus, these computationally heavy models are not suitable for resource-limited devices, particularly satellites. To address this issue, we propose a cloud detection model, LGCNet. This model efficiently extracts both local and global contextual information, fully utilizing semantics while reducing resource usage. LGCNet is built on an encoder-decoder structure. Specifically, the encoder extracts local scale-aware semantics through proposed local semantic blocks (LSBs), which are then skip-connected to the decoder. This approach provides adaptive and diverse local contextual information. On the top of the encoder, the high-level global semantics are captured via the proposed global feature TransBlock (GFTB). A variety of extracted semantics ensure improved detection performance. We evaluate the proposed method using two public datasets: LandSat8 and Moderate-Resolution Imaging Spectroradiometer (MODIS). We conducted experiments on both a server and an edge computing device. Our extensive experiments revealed that LGCNet outperforms other lightweight cloud detection and semantic segmentation methods in terms of performance and computational load.

Aiming at the problem of the sick limb condition of the exoskeleton rehabilitation robot affects the smoothness and stability of the robot system during rehabilitation training, this paper proposed an impedance control model for the lower limb exoskeleton rehabilitation robot. The model realizes the flexibility of the robot system by adjusting the impedance control parameters in real time. To verify the validity of the model, we used SCONE software to realize forward dynamics simulation of walking gait. The classical PID impedance control system and fuzzy adaptive impedance control system are simulated respectively. The results show that the fuzzy adaptive control system is more effective to adapt to the changes of limb condition for the impedance control system of lower limb exoskeleton rehabilitation robot.

In the case of network applications become more common and computer network security issues become more prominent, intercepting and analyzing network packets is important. However, network environment traffic is large. How to intercept and analyze network packet effectively is a crucial issue. This paper research the process of NDIS intermediate driver capture packets in the Windows platform and propose a mode of shared memory blocks that transport data packets to the application layer software to analyze. This way can improve the efficiency of communication between kernel layer and application layer and reduce the probability of losses during the packets transmission.

Computation of the optical flow from a sequence of images remains open in the community of computer vision. Two classical models for this problem are the global smoothness algorithm proposed by Horn-Schunck and the oriented smoothness algorithm by Nagel and Enkelmann. In order to increase the accuracy of motion discontinuity, we propose a new optical flow model which incorporates the harmonic smoothness constraint borrowed from the harmonic gradient vector flow (HGVF) model into the oriented smoothness constraint. In particular, we combine the curl term of the harmonic constraint with the oriented smoothness to control the direction of the displacement vectors together and introduce two spatially varying weighting functions to control the above-mentioned two terms. The benefit of the suggested strategies is illustrated qualitatively on the synthetical image and quantitatively on the Middlebury optical flow benchmark. Compared with the classical Horn-Schunck and Nagel-Enkelmann methods, this method can provide more accurate estimation of optical flow around motion discontinuities.