ABSTRACT Semi‐supervised medical image segmentation (SSMIS) has proven to be an effective solution that leverages limited labelled data and abundant unlabeled data, thereby significantly reducing the labour and cost associated with manual annotation. However, most of the existing teacher‐student frameworks are prone to suffer from confirmation bias during training, adversely affecting the performance of SSMIS. To address this challenge, we propose the Dual Student Discrepancy Correction framework (DSDC), which extends the Mean Teacher (MT) framework by incorporating an additional student model with identical architecture but independently updated parameters. This design mitigates the parameter coupling issue that may arise when updating the teacher model via Exponential Moving Average (EMA) in conventional single‐student paradigms. Moreover, the prediction discrepancy between the two student models is leveraged for error detection and correction, enabling the network to identify and rectify its own cognitive biases, ultimately enhancing segmentation accuracy. Comprehensive experiments on two public benchmarks, an MRI dataset (LA) and a CT dataset (Pancreas‐NIH), reveal that our DSDC framework surpasses current State‐of‐the‐Art (SOTA) approaches across all evaluation metrics. These findings substantiate the framework's effectiveness in SSMIS tasks. Code is accessible at https://github.com/Sangfugui/DSDC .

ABSTRACT With the explosive growth in the volume of image usage, selective image encryption (SIE) has emerged as an efficient method to enhance encryption efficiency. The challenge of how to identify images containing sensitive content has long been a difficult issue. Deep learning technology, with its powerful semantic extraction capabilities, has naturally become an auxiliary tool for recognizing images containing specific content. This review primarily focuses on recent advancements in SIE integrated with semantic understanding. First, it reviews the current state of development in image ROI encryption. Subsequently, it proposes two semantic‐aware SIE schemes based on image‐to‐image and text‐to‐image search paradigms. The review also introduces evaluation metrics for assessing both the encryption algorithms and deep learning models involved in such SIE systems. Finally, it analyzes potential security issues in SIE, such as privacy protection of deep learning models and leakage of ROI edge regions, as well as possible optimization directions, including model lightweighting and encryption parallelization to enhance efficiency. In conclusion, this review indicates that selective encryption is not limited to ROI‐based approaches but also includes semantic retrieval followed by targeted encryption. Moreover, with the integration of deep learning models, considerations regarding security and efficiency have become more complex, representing key areas for further exploration in future research.

ABSTRACT In semi‐supervised segmentation, deep co‐training has emerged as a promising paradigm by utilizing prediction agreement and unlabelled‐data‐driven supervisory signals. Despite its effectiveness, insufficient diversity between the collaborating models often causes them to become highly consistent at an early stage, weakening the benefit of mutual learning and eventually making the framework behave similarly to self‐training. In an effort to postpone this premature convergence, different initial perturbations were incorporated, but unfortunately, this resulted in a decrease in the quality of the model's pseudo‐labels, which further deteriorated the model's performance. To tackle this issue, we propose an Uncertainty Aware and Diverse Co‐training model (UADC) which is in line with the essential postulates of co‐training. The model features a novel co‐training framework working in hybrid spatial‐frequency domain. Specifically, after revisiting the basic hypotheses of co‐training, we devise a novel multi‐view training approach that addresses the issues of dependency and role inequality inherent in traditional methods. Targeting the misconceptions in previous co‐training architectures, we incorporate Segformer and frequency domain training to decouple and thereby counteract the tendency towards self‐training degeneration during later training stages. Furthermore, to enhance the quality of pseudo‐labels, we delve into the impact of uncertainty on model performance. We alter the structure of the model, designing a method grounded in the epistemic uncertainty of the segmentation model to optimize the quality of pseudo labels. We conduct extensive experiments on three public medical datasets of ISIC, Kvasir and CVC‐ClinicDB, and compared our model with other state‐of‐the‐art approaches, demonstrating its effectiveness and competitive performance.

ABSTRACT Accurate identification of signet ring cells (SRCs) in histopathological images is essential for the auxiliary evaluation of signet ring cell carcinoma (SRCC). Although deep learning has greatly advanced automated medical image interpretation, reliable SRC segmentation remains difficult because pixel‐level annotations are limited and SRCs often appear in crowded, adherent, or overlapping patterns. These characteristics increase the complexity of distinguishing individual cellular regions from surrounding tissues. To address these issues, this work proposes RGD‐UNet, a dedicated semantic segmentation network designed for SRC analysis. The model adopts an encoder–decoder framework and is optimized to capture discriminative pathological features while maintaining efficient computation. Unlike methods that rely on additional refinement procedures, RGD‐UNet can directly delineate clustered and overlapping SRC regions, thereby simplifying the segmentation workflow and improving practical usability. Furthermore, to facilitate the development and evaluation of SRC segmentation algorithms in clinical‐oriented scenarios, the DigestPath 2019 dataset was further extended by adding complete SRC mask annotations. Experimental results demonstrate that RGD‐UNet achieves superior segmentation precision compared with representative state‐of‐the‐art approaches while retaining low computational cost. These findings suggest that the proposed method has strong potential for assisting computer‐aided pathological diagnosis of SRCC.

ReRAM-based Processing-In-Memory (PIM) architectures are compelling contenders for deep learning due to their ability to perform matrix-vector multiplications (MVMs) directly within the memory, significantly reducing data movement and enhancing computational efficiency. However, since MVMs occur in the analog domain, analog-to-digital converters (ADCs) dominate power consumption and area overhead in current implementations. To this end, we propose RedPIM, an efficient ReRAM-based PIM accelerator design for deep neural networks (DNNs) that reduces the number of analog-to-digital conversions. RedPIM exploits the fact that in ReRAM-based PIM accelerators, the overall energy consumption generally increases with the number of activated analog-to-digital conversions. Specifically, we introduce a novel training algorithm that is aware of the ADC overhead during activation value quantization and optimizes accuracy concurrently. From a hardware design perspective, we develop a lookup table (LUT)-based quantization module to enable efficient and low-cost activation value quantization. In addition, we propose an efficient adaptive operation unit (OU) size assignment scheme that further minimizes analog-to-digital conversions by considering activation sparsity and weight distribution. Extensive experimental results show our RedPIM reduces latency to 27.72% and energy consumption to 10.15% of the baseline, with minimal accuracy loss, making it a promising solution for enhancing DNN acceleration. The code for this project is available at: https://github.com/JialeLiLab/ADC_aware_Learning.git .

The advancement in the field of machine learning is inextricably linked with the concurrent progress in domain-specific hardware accelerators such as GPUs and TPUs. However, the rapidly growing computational demands necessitated by larger models and increased data have become a primary bottleneck in further advancing machine learning, especially in mobile and edge devices. Currently, the neuromorphic computing paradigm based on memristors presents a promising solution. In this study, we introduce a memristor-based MobileNetV3 neural network computing paradigm and provide an end-to-end framework for validation. The results demonstrate that this computing paradigm achieves over 90% accuracy on the CIFAR10 dataset while saving inference time and reducing energy consumption. With the successful development and verification of MobileNetV3, the potential for realizing more memristor-based neural networks using this computing paradigm and open-source framework has significantly increased. This progress sets a groundbreaking pathway for future deployment initiatives.

As the need for real-time, low-power deep learning at the edge increases, efficient hardware acceleration becomes crucial. Traditional edge hardware designs often scale to accommodate neural network sizes, which can degrade overall performance by taxing the hardware. To solve this, we propose a novel Layer-wise Hardware Acceleration (LHA) approach for Deep Neural Network (DNN) inference, leveraging progressive quantization and Partial Reconfiguration (PR). We first apply progressive quantization to systematically reduce the bit-width of network weights and activations, lowering computational and memory demands. Then, we utilize Field Programmable Gate Arrays (FPGAs) with PR capabilities to dynamically reconfigure hardware for each quantized network layer in sequence. This method optimizes FPGA resource usage, tailors to each layer’s needs, and reallocates freed resources to boost overall performance. Experiments show that LHA significantly enhances resource efficiency while maintaining inference performance on edge devices.

Recent advancements in deep neural networks have created significant challenges for deploying these models on edge devices due to their computational and memory demands. We propose a novel integrated compression framework that combines nonlinear orthogonality-based channel pruning with progressive power-of-two (PoT) quantization to achieve efficient model compression for edge computing. Our framework first employs Radial Basis Function (RBF) kernel-based nonlinear orthogonality measurement to identify and remove redundant channels while preserving essential feature representations, then applies a layer-wise progressive power-of-two quantization scheme that enables efficient hardware implementation through bit-shift operations. Comprehensive experiments on CIFAR-10 and ImageNet demonstrate the effectiveness of our approach. On VGG16 with CIFAR-10, our method achieves 92.36% accuracy while reducing model size by 98.7% and computational complexity by 98.4%. On ResNet50 with ImageNet, we maintain 75.01% accuracy while achieving 95.93% model size reduction and 97.24% computational complexity reduction. Our framework significantly outperforms existing methods in terms of compression ratio and hardware efficiency while maintaining competitive accuracy.

ABSTRACT Image inpainting represents a fundamental task in computer vision, focusing primarily on the generation of missing content within an image to restore its integrity and aesthetics. Existing GAN‐based approaches often produce content with ambiguity and require a high training difficulties. Moreover, they tend to focus narrowly on damaged regions, leading to edge distortions that hinder generalisation. To address these challenges, we propose an algorithm that consist of two distinct networks. The first network, called Edge‐e4e, is designed for initial image restoration and integrates a pre‐trained StyleGAN2 as the generator to mitigate edge distortions. This network employs an encoder‐StyleGAN2 architecture, where only the encoder part is trained, thereby reducing training costs compared to traditional GAN methods. To resolve ambiguities in the restored content, we incorporate edge information into the damaged regions, guiding the network to generate content that is consistent with the original image. The second network, called Appending network, includes two style‐based encoders and a generator to improve the similarity between the images restored by Edge‐e4e and the original images. Specifically, we subtract the restored images from the input images in the channel dimension to obtain distortion maps, which serve as a prior to refine the restored images from Edge‐e4e. To further enhance the quality of refined images, we propose incorporating plugin and modulate plugin modules for style extraction and fusion. These modules utilise information from the input images and seamlessly integrate it into the style‐based generator. Experimental results demonstrate that our algorithm achieves high‐fidelity restoration and excellent generalisation, with optimal FID and Lpips metrics of 0.0631 and 0.875, respectively. The code is publicly available at: https://github.com/MengZhen‐Chi/Edge‐Pries‐Image‐Inpainting‐with‐StyleGAN2 .

Multi-modal emotion recognition plays a crucial role in human-computer interaction. Nowadays, many studies have developed fusion algorithms for this purpose. However, two challenges are still present, i.e., insufficient cross-modal information sharing and weak fusion feature representations. To this end, we develop a novel framework, namely CH-Net, for multi-modal emotion recognition with physiological signals. It is based on cross-modal attention and hypercomplex domain fusion. First, our learnable cross-modal attention mechanism adaptively aligns features across modalities, enhancing both complementarity and modality-specific discrepancies. Second, a hypercomplex fusion module encodes these features, yielding more robust representations while reducing parameter overhead. Two benchmark datasets, i.e., MAHNOB-HCI and DEAP, are utilized to train and test our model. Extensive experiments demonstrate that CH-Net is effective and outperforms state-of-the-art (SOTA) methods. Our code will be available athttps://github.com/xuxusky/CH-Net.

Remote sensing change detection (RSCD) has become an essential tool in observing and analyzing geographical information. However, existing deep learning approaches dependent solely on visual modalities may encounter challenges in discerning subtle variations amidst noise interference. To overcome these issues, we propose a multiscale semantic-guided synergistic interaction network (MSSI-Net), which utilizes the advanced multimodal semantic representations for enhancing the capacity to perceive hierarchical changes. Specifically, we first devise a multiscale interaction module (MIM) which leverages multiscale attention mechanism to guide the interaction between the coarse and fine stages of different visual features. The fine-grained visual features subsequently complement the semantic features through scale weight reassignment to enhance the discriminative capability of vision-language features. Furthermore, driven by the semantic-guided synergistic interaction mechanism, our developed cross-modal feature fusion module (CFFM) exploits both homogeneous and heterogeneous features among modalities. This ensures that the generated vision-language features are semantically representative. Finally, we formulate a manifold differential perception head (MDPH) to optimize the detection of changes by efficiently fusing diverse differential feature representations, achieving comprehensive performance enhancement. Extensive experiments conducted on four benchmark datasets (LEVIR-CD, CDD, SYSU-CD and WHU-CD) indicate that the designed MSSI-Net achieves state-of-the-art performance compared to existing methods.

Transformers have recently gained significant attention in medical image segmentation due to their ability to capture long-range dependencies. However, the presence of excessive background noise in large regions of medical images introduces distractions and increases the computational burden on the fine-grained self-attention (SA) mechanism, which is a key component of the transformer model. Meanwhile, preserving fine-grained details is essential for accurately segmenting complex, blurred medical images with diverse shapes and sizes. Thus, we propose a novel Multi-scale Dynamic Sparse Attention (MDSA) module, which flexibly reduces computational costs while maintaining multi-scale fine-grained interactions with content awareness. Specifically, multi-scale aggregation is first applied to the feature maps to enrich the diversity of interaction information. Then, for each query, irrelevant key-value pairs are filtered out at a coarse-grained level. Finally, fine-grained SA is performed on the remaining key-value pairs. In addition, we design an enhanced downsampling merging (EDM) module and an enhanced upsampling fusion (EUF) module for building pyramid architectures. Using MDSA to construct the basic blocks, combined with EDMs and EUFs, we develop a UNet-like model named MDSA-UNet. Since MDSA-UNet dynamically processes only a small subset of relevant fine-grained features, it achieves strong segmentation performance with high computational efficiency. Extensive experiments on four datasets spanning three different types demonstrate that our MDSA-UNet, without using pre-training, significantly outperforms other non-pretrained methods and even competes with pre-trained models, achieving Dice scores of 82.10% on DDTI, 80.20% on TN3K, 90.75% on ISIC2018, and 91.05% on ACDC. Meanwhile, our model maintains lower complexity, with only 6.65 M parameters and 4.54 G FLOPs at a resolution of 224 × 224, ensuring both effectiveness and efficiency. Code is available at URL.

Existing High Efficiency Video Coding (HEVC) selective encryption algorithms only consider the encoding characteristics of syntax elements to keep format compliance, but ignore the semantic features of video content, which may lead to unnecessary computational and bit rate costs. To tackle this problem, we present a content-aware tunable selective encryption (CATSE) scheme for HEVC. First, a deep hashing network is adopted to retrieve groups of pictures (GOPs) containing sensitive objects. Then, the retrieved sensitive GOPs and the remaining insensitive ones are encrypted with different encryption strengths. For the former, multiple syntax elements are encrypted to ensure security, whereas for the latter, only a few bypass-coded syntax elements are encrypted to improve the encryption efficiency and reduce the bit rate overhead. The keystream sequence used is extracted from the time series of a new improved logistic map with complex dynamic behavior, which is generated by our proposed sine-modular chaotification model. Finally, a reversible steganography is applied to embed the flag bits of the GOP type into the encrypted bitstream, so that the decoder can distinguish the encrypted syntax elements that need to be decrypted in different GOPs. Experimental results indicate that the proposed HEVC CATSE scheme not only provides high encryption speed and low bit rate overhead, but also has superior encryption strength than other state-of-the-art HEVC selective encryption algorithms.

Existing selective encryption schemes for High Efficiency Video Coding (HEVC) only focus on the encoding characteristics of syntax elements in entropy coding and lack an understanding of the video content. Consequently, a large amount of unnecessary encryption operations are utilized to protect insensitive video frames, resulting in low encryption efficiency. In this article, we propose a content-aware selective encryption scheme for H.265/HEVC, which encrypts only the groups of pictures (GOPs) containing sensitive content and thus offers high efficiency. In our scheme, a deep hashing network is first adopted to retrieve video frames to determine the content-sensitive GOPs. Then, multiple prediction and residual syntax elements in sensitive GOPs are encrypted using a keystream sequence generated by the hyper-chaotic Lorenz system. In addition, the direct current coefficient of each \(4\times 4\) transform block is exchanged with a pseudo-randomly selected non-zero alternating current coefficient to further offer stronger visual distortion. Finally, the sign bits used for marking each GOP-type are reversibly embedded into the encrypted syntax elements to facilitate the decoder to distinguish the GOPs that need to be decrypted. Experimental results indicate that the proposed content-aware selective encryption scheme can efficiently protect sensitive content and is robust against all common attacks. Furthermore, it outperforms other state-of-the-art HEVC selective encryption algorithms in terms of security performance.

The popular methods for semi-supervised semantic segmentation mostly adopt a unitary network model using convolutional neural networks (CNNs) and enforce consistency of the model’s predictions over perturbations applied to the inputs or model. However, such a learning paradigm suffers from two critical limitations: a) learning the discriminative features for the unlabeled data; b) learning both global and local information from the whole image. In this paper, we propose a novel Semi-supervised Learning (SSL) approach, called Transformer-CNN Cohort (TCC), that consists of two students with one based on the vision transformer (ViT) and the other based on the CNN. Our method subtly incorporates the multi-level consistency regularization on the predictions and the heterogeneous feature spaces via pseudo-labeling for the unlabeled data. First, as the inputs of the ViT student are image patches, the feature maps extracted encode crucial class-wise statistics. To this end, we propose class-aware feature consistency distillation (CFCD) that first leverages the outputs of each student as the pseudo labels and generates class-aware feature (CF) maps for knowledge transfer between the two students. Second, as the ViT student has more uniform representations for all layers, we propose consistency-aware cross distillation (CCD) to transfer knowledge between the pixel-wise predictions from the cohort. We validate the TCC framework on Cityscapes and Pascal VOC 2012 datasets, which outperforms existing SSL methods by a large margin. Project page: https://vlislab22.github.io/TCC/.

Imagined speech production with electrocorticography (ECoG) plays a crucial role in brain-computer interface system. A challenging issue is the great variation underlying the frequency bands of the ECoG signals’ encode information, which makes current methods difficult to generate imagined speech with stable quality among different persons. To this end, we propose a robust model to generate high-quality imagined speech from ECoG. A frequency enhancement branch is first designed to adaptively modulate the frequency information, whose product is fed into the following multi-scale channel attention module for robust feature extraction and fusion. By incorporating both the mel-spectrum and audio as training constraints, a multi-constraint decoder branch is finally constructed for imagined speech production. The performance of our model is evaluated on a high-quality dateset, i.e, Single Word Production Dutch-iBIDS. It yields Pearson correlation scores that are all above 0.8, and the standard deviationsare are all below 0.2 in different volunteers. Experimental results demonstrate that our model is effective and robust for ECoG based imagined speech production, and has advantages over peer methods.

Abstract The edges of objects are of great significance to the task of instance segmentation. However, most of the current popular deep neural networks do not pay much attention to the object edge information. More importantly, using the down‐sampling pooling layer in the deep learning network, the edge detail information of the object will be lost. To address this issue, inspired by the manual annotation process, we propose Mask Point R‐CNN aiming at promoting the neural network's attention to the object boundary. Specifically, we introduce the auxiliary task of object contour point detection on the Mask R‐CNN framework, which can effectively improve the gradient flow between different tasks by multi‐task learning and repairing objects' boundary information via feature fusion. Consequently, the model can be more sensitive to the edges of the object and capture more geometric features. Quantitatively, the experimental results show that our Mask Point R‐CNN outperforms vanilla Mask R‐CNN by 3.8% on the Cityscapes dataset and 0.8% on the COCO dataset.

The dynamical system perspective has been used to build efficient image classification networks and semantic segmentation networks. Furthermore, the Runge–Kutta (RK) methods are powerful tools for building networks from the dynamical systems perspective. Hence, the Runge–Kutta segmentation network (RKSeg) for medical image segmentation was born. Skip connections and multiple scaling are often used in common models but lack mathematical explanations. RKSeg interprets and uses skip connections based on the RK methods. Therefore, RKSeg greatly improves segmentation efficiency. However, it does not explain and use multiple scales from a dynamical system perspective but only inherits the multi-scale scheme of existing models. We compensate for this shortcoming by interpreting and using multiple scales based on the RK methods. In addition, the network structure also limits the excellent image classification networks as the backbones of RKSegs. Therefore, we modify the network structure to support more image classification networks as backbones. As a result, we propose a novel network structure RKSeg+. Our proposed RKSeg+ achieves better segmentation results with fewer parameters than RKSeg. Furthermore, RKSeg+, well configured with few parameters, outperforms state-of-the-art models on six of the ten organ datasets in the Medical Segmentation Decathlon.

Existing H.265/HEVC selective encryption (SE) schemes do not take into account the semantic features of input videos, nor do they adjust the encryption syntax elements according to the sensitivity of video content, which greatly limits their applicability. In this paper, we propose a chaos-based tunable H.265/HEVC SE scheme with semantic understanding. First, a deep hashing network is employed to identify content-sensitive videos by analyzing the semantic features of video sequences. Then, the non-sensitive videos and the retrieved sensitive ones are encrypted with different encryption strengths, respectively. Specifically, for non-sensitive videos, seven syntax elements with bypass-coded bins are selected for encryption at a constant bit rate. Hence, the encrypted bitstream keeps exactly the same compression ratio. To provide heavier visual distortion for content-sensitive videos, the regular-coded bins of four syntax elements and the intra prediction mode (IPM) are encrypted based on their corresponding encoding characteristics as well. Additionally, the selected syntax elements are all masked using a keystream generated by a chaotic system to ensure real-time constraints. Experimental results demonstrate that our suggested scheme offers format compatibility and is secure against all common attacks. Meanwhile, it outperforms state-of-the-art SE schemes in terms of security strength. Furthermore, the proposed scheme can be flexibly used in a wide range of applications according to the user’s requirements for encryption strength and bit rate.

Joint image compression and encryption algorithms are intensively investigated due to their powerful capability of simultaneous image data compression and sensitive information protection. Unfortunately, most of the existing algorithms suffered from either poor compression efficiency or weak encryption strength, making them vulnerable to cryptanalysis. To address these limitations, we propose a chaos-based JPEG-compatible joint image compression and encryption algorithm. We separate the luminance and chrominance coefficients to preserve file size and encrypt the discrete cosine transform (DCT) coefficients in parallel. The proposed inter-block DC encryption strategy achieves high encryption intensity based on the permutation-substitution structure. In addition, we apply both inter- and intra-block permutations to AC coefficients and strengthen the encryption using an inter-block substitution for non-zero AC coefficients. The results of security and performance analyses demonstrate that the proposed algorithm offers robust encryption of image data while maintaining compression efficiency for real-time transmission.

Video-based services have become popular. Clients often outsource their videos to the cloud to relieve local maintenance. However, privacy has become a major concern, since many videos contain sensitive information. Although retrieving (unencrypted) videos has been extensively investigated, retrieving encrypted multimedia has received relatively rare attention, at best in a limitation of image-based similarity searches. We initiate the study of scalable encrypted video search, enabling clients to query videos similar to an image search. Our modular framework leverages intrinsic attributes of videos, such as semantics and visuals, to effectively capture their contents. We propose a two-step approach whereby lightweight searchable encryption techniques are used for pre-screening, followed by an interactive approach for fine-grained search. Furthermore, we present three instantiations, including one centralized-writer instantiation and two distributed-writer instantiations, to effectively cater to varying needs and scenarios: (1) The centralized one employs forward and backward private searchable encryption [CCS 2017] over deep hashing [CVPR 2020]. (2) Motivated by distributed computing, the multi-writer instantiations building atop HSE [Usenix Security 2022] allows searching the relevant videos contributed by multiple intuitions collaboratively. Our experimental results illustrate their practical performance over multiple real-world datasets, whether in a centralized setting or distributed setting.

Recent breakthroughs in semi-supervised semantic segmentation have been developed through contrastive learning. In prevalent pixel-wise contrastive learning solutions, the model maps pixels to deterministic representations and regularizes them in the latent space. However, there exist inaccurate pseudo-labels which map the ambiguous representations of pixels to the wrong classes due to the limited cognitive ability of the model. In this paper, we define pixel-wise representations from a new perspective of probability theory and propose a Probabilistic Representation Contrastive Learning (PRCL) framework that improves representation quality by taking its probability into consideration. Through modelling the mapping from pixels to representations as the probability via multivariate Gaussian distributions, we can tune the contribution of the ambiguous representations to tolerate the risk of inaccurate pseudo-labels. Furthermore, we define prototypes in the form of distributions, which indicates the confidence of a class, while the point prototype cannot. More- over, we propose to regularize the distribution variance to enhance the reliability of representations. Taking advantage of these benefits, high-quality feature representations can be derived in the latent space, thereby the performance of se- mantic segmentation can be further improved. We conduct sufficient experiment to evaluate PRCL on Pascal VOC and CityScapes to demonstrate its superiority. The code is available at https://github.com/Haoyu-Xie/PRCL.

The ability of scene understanding has sparked active research for panoramic image semantic segmentation. However, the performance is hampered by distortion of the equirectangular projection (ERP) and a lack of pixel-wise annotations. For this reason, some works treat the ERP and pinhole images equally and transfer knowledge from the pinhole to ERP images via unsupervised domain adaptation (UDA). However, they fail to handle the domain gaps caused by: 1) the inherent differences between camera sensors and captured scenes; 2) the distinct image formats (e.g., ERP and pinhole images). In this paper, we propose a novel yet flexible dual-path UDA framework, DPPASS, taking ERP and tangent projection (TP) images as inputs. To reduce the domain gaps, we propose cross-projection and intra-projection training. The cross-projection training includes tangent-wise feature contrastive training and prediction consistency training. That is, the former formulates the features with the same projection locations as positive examples and vice versa, for the models' awareness of distortion, while the latter ensures the consistency of cross-model predictions between the ERP and TP. Moreover, adversarial intra-projection training is proposed to reduce the inherent gap, between the features of the pinhole images and those of the ERP and TP images, respectively. Importantly, the TP path can be freely removed after training, leading to no additional inference cost. Extensive experiments on two benchmarks show that our DPPASS achieves + 1.06% mIoU increment than the state-of-the-art approaches. https://vlis2022.github.io/cvpr23/DPPASS

Semi-Supervised Semantic Segmentation (S4) aims to train a segmentation model with limited labeled images and a substantial volume of unlabeled images. To improve the robustness of representations, powerful methods introduce a pixel-wise contrastive learning approach in latent space (i.e., representation space) that aggregates the representations to their prototypes in a fully supervised manner. However, previous contrastive-based S4 methods merely rely on the supervision from the model’s output (logits) in logit space during unlabeled training. In contrast, we utilize the outputs in both logit space and representation space to obtain supervision in a collaborative way. The supervision from two spaces plays two roles: 1) reduces the risk of over-fitting to incorrect semantic information in logits with the help of representations; 2) enhances the knowledge exchange between the two spaces. Furthermore, unlike previous approaches, we use the similarity between representations and prototypes as a new indicator to tilt training those under-performing representations and achieve a more efficient contrastive learning process. Results on two public benchmarks demonstrate the competitive performance of our method compared with state-of-the-art methods.

Video-based services have become popular. Clients often outsource their videos to the cloud to relieve local maintenance. However, privacy has emerged as a major concern since many videos contain sensitive information. While retrieving (unencrypted) videos has been widely studied, encrypted multimedia retrieval receives rare attention, at best in a limited form of similarity searches on images.

Recent years have witnessed an increasing popularity of wireless body area network (WBAN), with which continuous collection of physiological signals can be conveniently performed for healthcare monitoring. Energy consumption is a critical issue because it directly affects the duration of the equipped sensors. In this article, we propose a low-cost and confidential electrocardiogram (ECG) acquisition approach for WBAN. The compressed sensing (CS) is employed for low-cost signal acquisition, and its cryptographic features are exploited for promoting the framework's confidentiality. In particular, the RIPless measurement matrix is used to give CS the resistance against plaintext attack, while the first-order Σ∆ quantizer is employed to embed the cryptographic diffusion feature into the whole system. Two chaotic systems are employed for generating the required secret elements for the acquisition and encryption. Experiment results well demonstrate the signal reconstruction and security performance of the proposed framework.

This paper presents a new chaos-based medical image encryption algorithm using iterative permutation and substitution operation. To address the drawbacks encountered when using the invertible area-preserving chaotic maps, we introduce a pixel swapping-based image scrambling method with permutation keystream sequence generated from the logistic map. In the substitution stage, each keystream element is generated from a 1-D chaotic map selected from a group of three 1-D chaotic maps according to the value of the pixel previous to the one it applies to, making the keystream sequence dependent on plain-image. As a result, the robustness of the proposed algorithm against chosen-plaintext attack is ensured and the diffusion intensity is increased. The results of NPCR and UACI tests indicate that the proposed algorithm takes only two cipher rounds to achieve a desired diffusion effect. Moreover, each of the three 1-D chaotic maps is constructed by combining of two existing 1-D chaotic maps (seed maps). Compared with its corresponding seed maps, the compound map has larger chaotic ranges and more complex chaotic properties while remaining simplicity, making it a good candidate for constructing image ciphers with a sufficiently large key space and high computational efficiency. Our theoretical analysis and experimental results indicate that the proposed algorithm has a high level of security.

This paper suggests a new chaos-based color image cipher with an efficient substitution keystream generation strategy. The hyperchaotic Lü system and logistic map are employed to generate the permutation and substitution keystream sequences for image data scrambling and mixing. In the permutation stage, the positions of colored subpixels in the input image are scrambled using a pixel-swapping mechanism, which avoids two main problems encountered when using the discretized version of area-preserving chaotic maps. In the substitution stage, we introduce an efficient keystream generation method that can extract three keystream elements from the current state of the iterative logistic map. Compared with conventional method, the total number of iterations is reduced by 3 times. To ensure the robustness of the proposed scheme against chosen-plaintext attack, the current state of the logistic map is perturbed during each iteration and the disturbance value is determined by plain-pixel values. The mechanism of associating the keystream sequence with plain-image also helps accelerate the diffusion process and increase the degree of randomness of the keystream sequence. Experimental results demonstrate that the proposed scheme has a satisfactory level of security and outperforms the conventional schemes in terms of computational efficiency.

Recently, a number of chaos-based image ciphers with permutation-substitution structure have been suggested. Generally, the permutation and substitution are considered as two independent procedures, and individual chaotic maps or systems are iterated and quantified to produce the keystreams for the two stages. In this paper, we suggest a fast color image encryption scheme with both permutation and substitution keystreams quantified from a sequence extracted from the orbit of chaotic Chen's system. As the total number of iterations is reduced by half, the computational efficiency is improved. To confuse the relationship between the ciphertext and the secret key, the positions of subpixels in each color channel are scrambled across the entire color space using a pixel-swapping strategy, which can avoid the periodicity problem encountered by the discretized version of area-preserving chaotic maps. Experimental results demonstrate that the proposed scheme has a satisfactory level of security and outperforms the conventional scheme in term of computational efficiency.

This paper presents a new chaos-based image cipher using a plaintext-related permutation. The cat map and Lorenz system are employed to shuffle the positions of image pixels and generate the diffusion keystream, respectively. The control parameters of the cat map, i.e. the permutation key, are determined by the Murmur2 hash value of the original image. Owing to the avalanche property of hash functions, completely different shuffled images will be produced even if there is a tiny difference between the original ones, and it helps accelerate the diffusion process. Experimental results indicate that the proposed scheme requires only one and two cipher cycles to achieve an acceptable and a satisfactory diffusion properties, respectively, whereas two and three cipher cycles are needed by typical schemes to achieve the same properties. Thorough security analysis is carried out, and the results demonstrate the satisfactory security of the proposed scheme.