As semantic communication (SemCom) attracts growing attention as a novel communication paradigm, ensuring the security of transmitted semantic information over open wireless channels has become a critical issue. However, traditional encryption methods often introduce significant additional communication overhead to maintain stability, and conventional learning-based secure SemCom methods typically rely on a channel capacity advantage for the legitimate receiver, which is challenging to guarantee in real-world scenarios. In this paper, we propose a coding-enhanced jamming method that eliminates the need to transmit a secret key by utilizing shared knowledge–potentially part of the training set of the SemCom system–between the legitimate receiver and the transmitter. Specifically, we leverage the shared private knowledge base to generate a set of private digital codebooks in advance using neural network (NN)-based encoders. For each transmission, we encode the transmitted data into digital sequence Y1and associate Y1with a sequence randomly picked from the private codebook, denoted as Y2, through superposition coding. Here, Y1serves as the outer code and Y2as the inner code. By optimizing the power allocation between the inner and outer codes, the legitimate receiver can reconstruct the transmitted data using successive decoding with the index of Y2shared, while the eavesdropper’s decoding performance is severely degraded, potentially to the point of random guessing. Experimental results demonstrate that our method achieves security comparable to state-of-the-art approaches while significantly improving the reconstruction performance of the legitimate receiver by more than 1 dB across varying channel signal-to-noise ratios (SNRs) and compression ratios.

Sensor-equipped tool condition monitoring (TCM) is crucial for automated machining, but missing data poses a significant challenge. Existing methods struggle with the complex patterns and substantial data loss common in these dynamic processes. This paper introduces a novel duration-aware part attention mechanism for robust TCM. Unlike existing attention mechanisms, ours explicitly models time-duration dependencies within sensor signals, capturing multi-scale representations of tool degradation even with incomplete data. The part-attention operator, adapted from the Swin Transformer, can dynamically weight different time segments based on their duration and relevance. We further incorporate a cross-dimensional self-attention mechanism to fuse information across multiple sensors and time steps, capturing complex relationships indicative of tool wear. We evaluate our method on real-world machining datasets with varying levels of missing data, demonstrating its superior ability to accurately monitor tool condition compared to existing methods. The results show that the duration-aware part-attention effectively captures crucial temporal dependencies, leading to robust TCM even with substantial data loss.

This paper addresses the challenge of achieving security in semantic communication (SemCom) over a wiretap channel, where a legitimate receiver coexists with an eavesdropper experiencing a poorer channel condition. Despite previous efforts to secure SemCom against eavesdroppers, guarantee of approximately zero information leakage remains an open issue. In this work, we propose a secure SemCom approach based on superposition code, aiming to provide quantifiable and controllable security for digital SemCom systems. The proposed method employs a double-layered constellation map, where semantic information is associated with satellite constellation points and cloud center constellation points are randomly selected. By carefully allocating power between these two layers of constellation, we ensure that the symbol error probability (SEP) of the eavesdropper when decoding satellite constellation points is nearly equivalent to random guessing, while maintaining a low SEP for the legitimate receiver to successfully decode the semantic information. Simulation results demonstrate that the peak signal-to-noise ratio (PSNR) and mean squared error (MSE) of the eavesdropper's reconstructed data, under the proposed method, can range from decoding Gaussian-distributed random noise to approaching the variance of the data. This validates the effectiveness of our method in nearly achieving the experimental upper bound of security for digital SemCom systems when both eavesdroppers and legitimate users utilize identical decoding schemes. Furthermore, the proposed method consistently outperforms benchmark techniques, showcasing superior data security and robustness against eavesdropping. The implementation code is publicly available at:https://github.com/1weixuanchen/A-Superposition-Code-Based-Semantic-Communication.

Wireless communication systems operating in fading channels often demand pilots for channel estimation and data recovery, leading to substantial transmission overhead. In this paper, we propose a novel pilot-free semantic communication system designed for transmitting images over multi-user MIMO (MU-MIMO) fading channels. Specifically, our method involves extracting multi-scale semantic features from the source image at the transmitter, effectively embedding pilot-like information. At the receiver, we extract channel features from these semantic features at each scale, enabling the reconstruction of the source image without requiring explicit channel estimation and signal detection. To enhance the image reconstruction process, we introduce a novel module, called Resnet Transformer, which combines multi-head self-attention (MHSA) with Resnet block. Our experimental results demonstrate that this pilot-free system outperforms existing pilot-aided semantic communication methods in terms of perceptual quality and transmission efficiency.

Adaptive rate control for deep joint source and channel coding (JSCC) is considered as an effective approach to transmit sufficient information in scenarios with limited communication resources. We propose a deep JSCC scheme for wireless image transmission with entropy-aware adaptive rate control, using a single deep neural network to support multiple rates and automatically adjust the rate based on the feature maps of the input image and their entropy, as well as the channel conditions. In particular, we maximize the entropy of the feature maps to increase the average information carried by each transmitted symbol during the training. We further decide which feature maps should be activated based on their entropy, which improves the efficiency of the transmitted symbols. We also propose a pruning module to remove less important pixels in the activated feature maps in order to further improve transmission efficiency. The experimental results demonstrate that our proposed scheme learns an effective rate control strategy that reduces the required channel bandwidth while preserving the quality of the reconstructed images.

In recent years, deep learning-based lossy image compression have achieved great success. However, the problem of their huge overhead in terms of computational and parametric costs has still not been adequately addressed. Inspired by the classical image compression methods, deep learning based models are usually combined with an entropy model to maintain the compression performance. Existing methods also introduce side information to serve as a prior on the parameters of the entropy model, which have achieved better rate-distortion performance. Based on the role of side information in learned image compression models, we propose an efficient pruning method for such models. In particular, the proposed pruning approach automatically searches for the optimal decoder architecture based on the extent to which each hidden layer in the decoder utilizes side information. The experiment results demonstrate the effectiveness of the proposed method and show that it outperforms all existing related studies in terms of compression performance.

In order to improve the efficiency and safety of autonomous vehicles’ decision-making process in complex urban scenarios, a decision-making method for hierarchical processing of static traffic law information and dynamic traffic participant information is proposed in this paper. In the decision-making method, the candidate behavior set is constructed by extracting the element of traffic laws and fully considering the traffic laws constraints to ensure the legality and effectiveness of the decision-making algorithm. Besides, four evaluation indicators and two-level entry threshold are designed to select the optimal driving behavior with the consideration of driving efficiency, ride safety and macro path requirement. The advantage of proposed method is that it avoids the problem of poor adaptability to traffic laws and regulations as the existing methods usually make decisions under the condition of mixed traffic laws and traffic participant information. A complete driving task simulation and analysis, including six typical urban traffic scenarios, is given under Matlab environment. The results show that the proposed decision-making method is able to make reasonable and feasible decisions and highly consistent with the actual driver’s decision-making behavior in complex urban scenarios, which verifies the effectiveness of the proposed method.

Oblivious RAM (ORAM) conceals memory access pattern by translating a single read/write operation into the accesses to a set of randomized locations. The obliviousness is achieved by adding redundancy to the memory system, which comes at the expense of increased performance overhead. In memory systems, locality has always been a critical factor, as accessing the data with temporal or spatial locality would result in a performance gain. Although the two design considerations of obliviousness and locality may seem contradictory at first glance, combining them in a unified design can potentially hide long memory access latency in ORAM without sacrificing provable data security.This paper presents LolliRAM, a cross-layer design to exploit data locality in Oblivious RAM. LolliRAM optimizes ORAM system through two different layers: (i) the data structure in ORAM, and (ii) the fast and secure cache in ORAM controller. The reuse of redundant memory accesses at the layer of data structure in ORAM can effectively reduce memory footprints. Both temporal and spatial locality can be exploited through the elastic grouping of blocks and the optimization at the layer of ORAM controller. We conduct a set of experiments using realistic workloads that are generated from standard benchmarks. Experimental results show that LolliRAM can reduce access latency by 71.57% on average (up to 81.50%) with negligible space overhead in comparison with representative schemes.

Many important physical phenomena involve subtle signals that are difficult to observe with the unaided eye, yet visualizing them can be very informative. Current motion magnification techniques can reveal these small temporal variations in video, but require precise prior knowledge about the target signal, and cannot deal with interference motions at a similar frequency. We present DeepMag, an end-to-end deep neural video-processing framework based on gradient ascent that enables automated magnification of subtle color and motion signals from a specific source, even in the presence of large motions of various velocities. The advantages of DeepMag are highlighted via the task of video-based physiological visualization. Through systematic quantitative and qualitative evaluation of the approach on videos with different levels of head motion, we compare the magnification of pulse and respiration to existing state-of-the-art methods. Our method produces magnified videos with substantially fewer artifacts and blurring whilst magnifying the physiological changes by a similar degree.

Unobtrusive and accurate ambulatory methods are needed to monitor long-term sleep patterns for improving health. Previously developed ambulatory sleep detection methods rely either in whole or in part on self-reported diary data as ground truth, which is a problem, since people often do not fill them out accurately. This paper presents an algorithm that uses multimodal data from smart-phones and wearable technologies to detect sleep/wake state and sleep onset/offset using a type of recurrent neural network with long-short-term memory (LSTM) cells for synthesizing temporal information. We collected 5580 days of multimodal data from 186 participants and compared the new method for sleep/wake classification and sleep onset/offset detection to, first, nontemporal machine learning methods and, second, a state-of-the-art actigraphy software. The new LSTM method achieved a sleep/wake classification accuracy of 96.5%, and sleep onset/offset detection F1 scores of 0.86 and 0.84, respectively, with mean absolute errors of 5.0 and 5.5 min, respectively, when compared with sleep/wake state and sleep onset/offset assessed using actigraphy and sleep diaries. The LSTM results were statistically superior to those from non-temporal machine learning algorithms and the actigraphy software. We show good generalization of the new algorithm by comparing participant-dependent and participant-independent models, and we show how to make the model nearly realtime with slightly reduced performance.

Deep convolutional neural networks (CNNs) are widely adopted in intelligent systems with unprecedented accuracy but at the cost of a substantial amount of data movement. Although the emerging processing-in-memory (PIM) architecture seeks to minimize data movement by placing memory near processing elements, memory is still the major bottleneck in the entire system. The selection of hyper-parameters in the training of CNN applications requires over hundreds of kilobytes cache capacity for concurrent processing of convolutions. How to jointly explore the computation capability of the PIM architecture and the highly parallel property of neural networks remains a critical issue. This paper presents Para-Net, that exploits Parallelism for deterministic convolutional neural Networks on the PIM architecture. Para- Net achieves data-level parallelism for convolutions by fully utilizing the on-chip processing engine (PE) in PIM. The objective is to capture the characteristics of neural networks and present a hardware-independent design to jointly optimize the scheduling of both intermediate results and computation tasks. We formulate this data allocation problem as a dynamic programming model and obtain an optimal solution. To demonstrate the viability of the proposed Para-Net, we conduct a set of experiments using a variety of realistic CNN applications. The graph abstractions are obtained from deep learning framework Caffe. Experimental results show that Para-Net can significantly reduce processing time and improve cache efficiency compared to representative schemes.

Convolutional neural networks (CNNs) have been successfully applied in artificial intelligent systems to perform sensory processing, sequence learning, and image processing. In contrast to conventional computing-centric applications, CNNs are known to be both computationally and memory intensive. The computational and memory resources of CNN applications are mixed together in the network weights. This incurs a significant amount of data movement, especially for high-dimensional convolutions. The emerging Processing-in-Memory (PIM) alleviates this memory bottleneck by integrating both processing elements and memory into a 3D-stacked architecture. Although this architecture can offer fast near-data processing to reduce data movement, memory is still a limiting factor of the entire system. We observe that an unsolved key challenge is how to efficiently allocate convolutions to 3D-stacked PIM to combine the advantages of both neural and computational processing. This paper presents MemoNet, a memory-efficient data allocation strategy for convolutional neural networks on 3D PIM architecture. MemoNet offers fine-grained parallelism that can fully exploit the computational power of PIM architecture. The objective is to capture the characteristics of neural network applications and perfectly match the underlining hardware resources provided by PIM, resulting in a hardware-independent design to transparently allocate data. We formulate the target problem as a dynamic programming model and present an optimal solution. To demonstrate the viability of the proposed MemoNet, we conduct a set of experiments using a variety of realistic convolutional neural network applications. The extensive evaluations show that, MemoNet can significantly improve the performance and the cache utilization compared to representative schemes.

Animated GIFs are widely used on the Internet to express emotions, but their automatic analysis is largely unexplored. Existing GIF datasets with emotion labels are too small for training contemporary machine learning models, so we propose a semi-automatic method to collect emotional animated GIFs from the Internet with the least amount of human labor. The method trains weak emotion recognizers on labeled data, and uses them to sort a large quantity of unlabeled GIFs. We found that by exploiting the clustered structure of emotions, the number of GIFs a labeler needs to check can be greatly reduced. Using the proposed method, a dataset called GIFGIF+ with 23,544 GIFs over 17 emotions was created, which provides a promising platform for affective computing research.

A growing number of studies show links between changes in tongue appearance and human health conditions. This paper studies tongue color changes in the context of stress to explore the feasibility of providing a novel and non-invasive stress measurement method. In a laboratory study, 24 participants were asked to perform a calm and a stressful math task and to take a photo of their tongue right after each of the tasks. We observed subtle but consistent color differences between calm and stress tasks for up to 75% of the participants, which was consistent with both self-report and physiological metrics of stress. Moreover, we observed significant correlations of up to 0.72 between certain tongue colors and long-term stress assessed with the 10-item Perceived Stress Scale questionnaire. We discuss the potential implications of this work and highlight some lines of future research.

In this paper we present Organic Primitives, an enabling toolbox that expands upon the library of input-output devices in HCI and facilitates the design of interactions with organic, fluid-based systems. We formulated color, odor and shape changing material primitives which act as sensor-actuators that convert pH signals into human-readable outputs. Food-grade organic molecules anthocyanin, vanillin, and chitosan were employed as dopants to synthesize materials which output a spectrum of colors, degrees of shape deformation, and switch between odorous and non-odorous states. We evaluated the individual output properties of our sensor-actuators to assess the rate, range, and reversibility of the changes as a function of pH 2-10. We present a design space with techniques for enhancing the functionality of the material primitives, and offer passive and computational methods for controlling the material interfaces. Finally, we explore applications enabled by Organic Primitives under four contexts: environmental, cosmetic, edible, and interspecies.

Vital signs, cognitive load, and stress can be remotely measured from human faces using video-capturing devices under ambient light, which raises both wide applications and privacy issues. To avoid immoral use of this technology, there is a need for methods to eliminate physiological information from facial videos without affecting their visual appearance. To meet the need, we develop a novel algorithm based on motion component magnification that inputs a video and outputs its replica with physiological signals removed. Facial video data has been collected from 18 participants in a study to assess the performance of our algorithm in thwarting heart rate measurement based on remote photoplethysmography. Our results show that the mean absolute error of heart rate measurement averaged among participants was increased from 0.254 beats per minute to above 17 beats per minute without causing visible artifact. This is the first demonstration of an algorithm that can achieve this kind of functionality.

Animated GIFs are widely used on the Internet to express emotions, but their automatic analysis is largely unexplored before. To help with the search and recommendation of GIFs, we aim to predict their emotions perceived by humans based on their contents. Since previous solutions to this problem only utilize image-based features and lose all the motion information, we propose to use 3D convolutional neural networks (CNNs) to extract spatiotemporal features from GIFs. We evaluate our methodology on a crowd-sourcing platform called GIFGIF with more than 6000 animated GIFs, and achieve a better accuracy then any previous approach in predicting crowd-sourced intensity scores of 17 emotions. It is also found that our trained model can be used to distinguish and cluster emotions in terms of valence and risk perception.