Neural Representations for Videos (NeRV) has emerged as a promising implicit neural representation (INR) approach for video analysis, which represents videos as neural networks with frame indexes as inputs. However, NeRV-based methods are time-consuming when adapting to a large number of diverse videos, as each video requires a separate NeRV model to be trained from scratch. In addition, NeRV-based methods spatially require generating a high-dimension signal (i.e., an entire image) from the input of a low-dimension timestamp, and a video typically consists of tens of frames temporally that have a minor change between adjacent frames. To improve the efficiency of video representation, we propose Meta Neural Representations for Videos, named MetaNeRV, a novel framework for fast NeRV representation for unseen videos. MetaNeRV leverages a meta-learning framework to learn an optimal parameter initialization, which serves as a good starting point for adapting to new videos. To address the unique spatial and temporal characteristics of video modality, we further introduce spatial-temporal guidance to improve the representation capabilities of MetaNeRV. Specifically, the spatial guidance with a multi-resolution loss aims to capture the information from different resolution stages, and the temporal guidance with an effective progressive learning strategy could gradually refine the number of fitted frames during the meta-learning process. Extensive experiments conducted on multiple datasets demonstrate the superiority of MetaNeRV for video representations and video compression.

Large language models(LLMs) have garnered significant attention and demonstrated impressive capabilities in a wide range of applications. However, due to their enormous computational costs, the deployment and application of LLMs are often severely limited. To address this issue, structured pruning is an effective solution to compress the parameters of LLMs. Determining the importance of each sub-module in LLMs and minimizing performance loss are critical issues that need to be carefully addressed in structured pruning. In this paper, we propose an effective and fast structured pruning method named SlimLLM for large language models. For channel and attention head pruning, we evaluate the importance based on the entire channel or head, rather than merely aggregating the importance of individual elements within a sub-module. This approach enables a more holistic consideration of the interdependence among elements within the sub-module. In addition, we design a simple linear regression strategy for the output matrix to quickly recover performance. We also propose layer-based importance ratio to determine the pruning ratio for each layer. Based on the LLaMA benchmark results, our SlimLLM outperforms other methods and achieves state-of-the-art performance.

Healthcare data frequently contain a substantial proportion of missing values, necessitating effective time series imputation to support downstream disease diagnosis tasks. However, existing imputation methods focus on discrete data points and are unable to effectively model sparse data, resulting in particularly poor performance for imputing substantial missing values. In this paper, we propose a novel approach, ImputeINR, for time series imputation by employing implicit neural representations (INR) to learn continuous functions for time series. ImputeINR leverages the merits of INR in that the continuous functions are not coupled to sampling frequency and have infinite sampling frequency, allowing ImputeINR to generate fine-grained imputations even on extremely sparse observed values. Extensive experiments conducted on eight datasets with five ratios of masked values show the superior imputation performance of ImputeINR, especially for high missing ratios in time series data. We also validate that applying ImputeINR to impute missing values in healthcare data enhances the performance of downstream disease diagnosis tasks.

The rapid growth of the Internet of Things (IoT) has led to the widespread use of companion apps for device management. However, these apps expose a critical vulnerability in the IoT ecosystem: insufficient verification procedures during device firmware updates (DFU), often resulting in firmware leakage. Once leaked, the firmware reveals sensitive design details, creating a straightforward path for attackers to reverse-engineer devices. To address this issue, we designed an automated analysis tool called FirmProj. It systematically evaluates firmware leakage risks by examining IoT companion apps. FirmProj combines advanced static analysis techniques with large language models to identify DFU modules, extract firmware files, and detect security vulnerabilities. In a large-scale study involving 10,047 IoT companion apps, FirmProj successfully retrieved 3,434 firmware files, uncovering severe flaws in DFU implementations that can lead to firmware leakage. These findings resulted in the assignment of 35 CVE IDs. Our results highlight the urgent need to strengthen firmware protection mechanisms throughout the IoT ecosystem.

With the rapid advancement of deep learning, computer-aided diagnosis and treatment have become crucial in medicine. UNet is a widely used architecture for medical image segmentation, and various methods for improving UNet have been extensively explored. One popular approach is incorporating transformers, though their quadratic computational complexity poses challenges. Recently, State-Space Models (SSMs), exemplified by Mamba, have gained significant attention as a promising alternative due to their linear computational complexity. Another approach, neural memory Ordinary Differential Equations (nmODEs), exhibits similar principles and achieves good results. In this paper, we explore the respective strengths and weaknesses of nmODEs and SSMs and propose a novel architecture, the nmSSM decoder, which combines the advantages of both approaches. This architecture possesses powerful nonlinear representation capabilities while retaining the ability to preserve input and process global information. We construct nmSSM-UNet using the nmSSM decoder and conduct comprehensive experiments on the PH2, ISIC2018, and BU-COCO datasets to validate its effectiveness in medical image segmentation. The results demonstrate the promising application value of nmSSM-UNet. Additionally, we conducted ablation experiments to verify the effectiveness of our proposed improvements on SSMs and nmODEs.

Transformers have become foundational architectures for both natural language and computer vision tasks. However, the high computational cost makes it quite challenging to deploy on resource-constraint devices. This paper investigates the computational bottleneck modules of efficient transformer, i.e., normalization layers and attention modules. LayerNorm is commonly used in transformer architectures but is not computational friendly due to statistic calculation during inference. However, replacing LayerNorm with more efficient BatchNorm in transformer often leads to inferior performance and collapse in training. To address this problem, we propose a novel method named PRepBN to progressively replace LayerNorm with re-parameterized BatchNorm in training. Moreover, we propose a simplified linear attention (SLA) module that is simple yet effective to achieve strong performance. Extensive experiments on image classification as well as object detection demonstrate the effectiveness of our proposed method. For example, our SLAB-Swin obtains $83.6%$ top-1 accuracy on ImageNet-1K with $16.2$ms latency, which is $2.4$ms less than that of Flatten-Swin with $0.1%$ higher accuracy. We also evaluated our method for language modeling task and obtain comparable performance and lower latency. Codes are publicly available at https://github.com/xinghaochen/SLAB and https://github.com/mindspore-lab/models/tree/master/research/huawei-noah/SLAB.

Multi-channel distribution of Android apps offers convenience to users, yet simultaneously introduces security concerns. Although apps published on Google Play and third-party markets share the same version code, differences in app content may still arise. Notably, a recent incident involving the third-party market version of Pinduoduo app containing malicious code highlights the intentionally-differentiated implementations of app functionalities by developers between Google Play and third-party markets. The case of Pinduoduo may be just the tip of the iceberg, underscoring the need for a comprehensive investigation of the disparities between Google Play and third-party market versions of apps.In this work, we systematically analyze the differences in security and privacy of cross-market apps that claim to share the same version code. Specifically, we propose three research questions that cover differences in app protection, security threats, and permission usage. To answer these questions, we constructed a dataset containing 17,218 app pairs (filtered from 236,731 apps) and permission mappings (27,046 SDK mappings, 1,656 ContentProvider mappings, and 309 Intent mappings) for API levels 16 - 33. This dataset enables us to perform a comprehensive differential analysis. Consequently, our investigation unveiled a series of captivating and insightful findings. Approximately 29.02% of apps show differences in one or all three aspects. For example, the third-party market versions of apps often request more permissions compared to their Google Play counterparts, particularly among apps in the game category. Our work can help developers and app store operators improve cross-market app consistency, enhancing the quality of the Android app ecosystem and user experience.

Change detection for high resolution remote sensing images is an important but challenging task. In this article, we propose a spatial-temporal-channel attention Unet++ (STC-Unet++) for remote sensing image change detection. The STC-Unet++ takes advantage of the Unet++ structure, combining semantic information to change detection. In addition, it employs a spatial-temporal-channel attention mechanism, extracting features more discriminatively and improving the change detection accuracy without increasing training time. Finally, experiments are carried out on the LEVIR-CD dataset, and the results show that the STC-Unet++ can effectively detect the changes, achieving 89.0% recall, 88.3% accuracy, 88.4% F1-score, 79.49% IoU and 94.1% AUC.

Change detection in SAR images is an important but challenge task. Due to the difficulty of SAR interpretation, reliable training samples are lacking, limiting the application of deep learning technology in SAR image change detection. To overcome this problem, this article proposes an unsupervised SAR image change detection method based on slow feature analysis theory with convolutional neural network (SAR-SFAnet). It adopts SDAEs to automatically extract features from SAR data, and employs slow feature analysis theory to project the extracted multi -dimensional features into a new space. In addition, an alternative optimization strategy is introduced, making the features learned by bi - temporal stacked denoising auto-encoder (SDAEs) have more consistent representations, as well as making the change detection map more accurate. Finally, comparative experiments are carried out on two real SAR data sets, demonstrating the effectiveness of the proposed method.