Artificial Intelligence (AI) technology has been widely used in malware detection and has significantly improved defense against cyberattacks. Existing deep learning-based methods rely on training with large-scale data and only on predefined categories, making them inadequate for rapidly responding to novel malware attacks. Malware classification based on few-shot learning has made some progress in identifying unknown malware using limited data. However, existing methods struggle to achieve high performance because they typically focus on a single malicious feature, such as a single malware image or an API call sequence, thereby ignoring the multi-dimensional nature of malware. To deal with these challenges, we propose a multi-view few-shot learning method for malware classification. We propose a multi-view malicious feature engineering scheme, which combines domain knowledge and expert experience to analyze the malware from various perspectives. Furthermore, we propose a support-query prototype generation method based on multi-view malicious features to generate higher-quality malware prototypes, which enhances the representation of novel malware family distributions. Extensive experiments show that the proposed method outperforms existing state-of-the-art approaches. With only two samples per family, the accuracy still exceeds 90%. Our method demonstrates superior cross-dataset recognition capabilities, thereby fully illustrating its robustness and generalizability across different data distributions.

In low-light environments, human vision is severely limited by weak light sources, leading to significantly reduced visual capabilities. Similarly, in machine vision, low-light recognition tasks such as nighttime autonomous driving and surveillance tasks involving the detection of small faces in low-light conditions are more challenging than tasks in normal lighting. Current low-light face detection models lack adaptability to different low-light conditions, and the accuracy of face detection remains unsatisfactory. In this paper, we propose a novel face detection framework DSLL-Face, specifically designed to tackle the challenges of face detection in low-light environments. Our proposed DarkHead, featuring a specialized branch designed to predict the distribution of bounding boxes, thereby substantially enhances the supervision of bounding box localization. This innovative approach effectively resolves the issue of blurry bounding boxes and significantly increases the accuracy of predicted positions. We employ a novel loss function tailored for detecting small faces, enhancing the sensitivity and effectively addressing the blurriness issues in small face detection. Furthermore, we leverage the Channel Grouping and Partial Convolution block (CGP) to enhance multi-scale expression capabilities. We develop the EMNet-pro model with the aim of further enhancing images to improve their adaptability under various low-light conditions. Extensive experiments demonstrate that our model exhibits outstanding capability in low-light face detection on the DARK FACE dataset and achieves significantly better performance compared to existing state-of-the-art frameworks.

An unending stream of malware variations presents a severe threat to the Internet community as a way of initiating cyber-attacks. Although few-shot learning-based malware classification methods have achieved some success in detecting unknown malware and using limited data for training, the majority of current techniques still struggle with classification performance because they only take into account a single malware image or API call sequence feature, ignoring the multi-dimensional nature of malware. To deal with these challenges, this paper proposes a malware image and opcode joint learning method for few-shot malware classification. We employ a cross-modal attention mechanism to determine the weight representing the correlation between the malware’s binary and assembly codes. Furthermore, we compute a weighted prototype based on the fused feature vector of binary and assembly codes to enhance the prototype’s generalizability. Extensive experiments demonstrate the superiority of our method compared to existing few-shot malware classification models, with an average accuracy of more than 83% in the 5-way settings with only two samples on both LargePE and VirusShare datasets.

The transferability of adversarial examples is pivotal in black-box attacks on deep learning models. The existing transfer-based attacks typically rely on a single data augmentation technique, which hampers the diversity of generated adversarial examples. Additionally, applying a single adversarial noise generation path may impose limitations on the perturbation strength of the generated noise, thereby compromising the transferability of these examples. To address these issues, we propose a framework called Joint Training and Dual Feature Mixiup (JFM), which comprises the dual feature mixup module and joint training module. The dual feature mixup module performs feature mixing between benign and augmented images, enabling the comprehensive extraction of benign example features and enhancing the diversity of adversarial examples. Furthermore, the joint training module designs a dual-path prediction loss function that incorporates both the loss between mixed feature examples and benign examples, as well as the loss between augmented examples and benign examples, thereby enhancing the transferability of the generated examples. Empirical evaluation of the ImageNet-compatible dataset demonstrates that our JFM method exhibits superior attack capability and significantly outperforms state-of-the-art methods.

The growing popularity of contact-free smart sensing has contributed to the development of the Artificial Intelligence of Things (AIoT). The contact-free sensory data has great potential to mine and analyze the hidden information for AIoT-enabled applications. However, due to the limited storage resource of contact-free smart sensing devices, data is naturally stored in the cloud, which is at risk of privacy leakage. Cloud storage is generally considered insecure. On one hand, the openness of the cloud environment makes the data easy to be attacked, and the complex AIoT environment also makes the data transmission process vulnerable to the third party. On the other hand, the Cloud Service Provider (CSP) is untrusted. In this article, to ensure the security of data from contact-free smart sensing devices, a Cloud-Edge-End cooperative storage scheme is proposed, which takes full advantage of the differences in the cloud, edge, and end. Firstly, the processed sensory data is stored separately in the three layers by utilizing well-designed data partitioning strategy. This scheme can increase the difficulty of privacy leakage in the transmission process and avoid internal and external attacks. Besides, the contact-free sensory data is highly time-dependent. Therefore, combined with the Cloud-Edge-End cooperation model, this article proposes a delta-based data update method and extends it into a hybrid update mode to improve the synchronization efficiency. Theoretical analysis and experimental results show that the proposed cooperative storage method can resist various security threats in bad situations and outperform other update methods in synchronization efficiency, significantly reducing the synchronization overhead in AIoT.

We propose PAniC-3D, a system to reconstruct stylized 3D character heads directly from illustrated (p)ortraits of (ani)me (c)haracters. Our anime-style domain poses unique challenges to single-view reconstruction; compared to natural images of human heads, character portrait illustrations have hair and accessories with more complex and diverse geometry, and are shaded with non-photorealistic contour lines. In addition, there is a lack of both 3D model and portrait illustration data suitable to train and evaluate this ambiguous stylized reconstruction task. Facing these challenges, our proposed PAniC-3D architecture crosses the illustration-to-3D domain gap with a line-filling model, and represents sophisticated geometries with a volumetric radiance field. We train our system with two large new datasets (11.2k Vroid 3D models, 1k Vtuber portrait illustrations), and evaluate on a novel AnimeRecon benchmark of illustration-to-3D pairs. PAniC-3D significantly outper-forms baseline methods, and provides data to establish the task of stylized reconstruction from portrait illustrations.

With the rapid development of artificial intelligence (AI), various industries have been propelled forward. In the field of medicine, AI has proven to be invaluable in aiding doctors to gain further insights into medical conditions, medical imaging is a prime example. Indeed, the availability of large-scale datasets like ImageNet has significantly contributed to the success of deep learning models in various computer vision tasks. However, when it comes to medical image classification, the availability of large, labeled datasets is relatively limited. As a result, the number of models specifically trained for medical image classification is comparatively smaller. This paper presents a novel neural network, named Channel Grouping and Partial Convolution Network (CGPNet), built upon the foundation of the InceptionNext model. Leveraging the commonly used lightweight technique, along with the introduction of grouped partial convolutions and dynamic convolutions, our proposed network exhibits promising performance. In particular, experiments were conducted on the ISIC-2019 (International Skin Imaging Collaboration) dataset, which demonstrates that our model achieves 2.61% improvement in top-1 accuracy compared to InceptionNext.

Smart contract vulnerability detection is an important security practice aimed at identifying and fixing potential vulnerabilities. This detection technique involves using static and dynamic analysis methods to inspect and test contract code, in order to identify code patterns and logical errors that may lead to security vulnerabilities. However, summarizing previous research reveals limitations in terms of scalability and generalizability, which can result in higher rates of false positives and false negatives in detection results. Therefore, we propose a novel smart contract detection framework called TSCSG: Two-Stage Smart Contract Vulnerability Detection Combining Semantic Features and Graph Features. In the graph extraction stage, TSCSG utilizes the data flow graph and control flow graph of smart contracts to extract the required contract graph. After processing the graph data, TSCSG employs our proposed RTMP network to extract smart contract graph features. In the semantic extraction stage of contract vulnerabilities, TSCSG utilizes smart contract data propagation chains to extract semantic features of smart contract vulnerabilities, which are then combined with the graph features to obtain the final detection results. Our large-scale empirical study on the EtherScan dataset demonstrates that TSCSG achieves satisfactory results in detecting reentrancy and timestamp vulnerabilities, outperforming 9 state-of-the-art vulnerability detection methods.

Social media plays a significant role in our lives, providing convenience in accessing information and expressing opinions through various platforms. However, this convenience has also led to the proliferation of rumors. Therefore, detecting rumors on social media has become increasingly important. Many existing works focus on detecting rumors by analyzing source posts and comments posted by users, utilizing pre-trained language models to capture text representations. However, simply averaging the comments from different users fails to capture the correlation between comments and source posts, which is crucial for rumor detection. To address this issue, in this paper, we propose a multi-scale feature aggregation method. We utilize a BERT-based pre-trained language model to encode the source posts and comment texts, obtaining token scale feature representations. Furthermore, to differentiate between different comments, we perform secondary aggregation of features at the comment scale, mitigating the limitations of previous methods that treat all comments equally. We then employ an attention layer to obtain a correlation weight matrix between the source posts and comments, which represents their correlation. This approach significantly extracts comment content most relevant to the current event being detected. Experimental results on existing datasets in both Chinese and English demonstrate the superiority of our method compared to other existing approaches. Further experiments also confirm the importance of the multi-scale approach in mining the correlation between source posts and comments for rumor detection.

Human pose information is a critical component in many downstream image processing tasks, such as activity recognition and motion tracking. Likewise, a pose estimator for the illustrated character domain would provide a valuable prior for assistive content creation tasks, such as reference pose retrieval and automatic character animation. But while modern data-driven techniques have substantially improved pose estimation performance on natural images, little work has been done for illustrations. In our work, we bridge this domain gap by efficiently transfer-learning from both domain-specific and task-specific source models. Additionally, we upgrade and expand an existing illustrated pose estimation dataset, and introduce two new datasets for classification and segmentation subtasks. We then apply the resultant state-of-the-art character pose estimator to solve the novel task of pose-guided illustration retrieval. All data, models, and code will be made publicly available.

The frequent outbreaks of agriculture pests have caused heavy losses in crop production. And the small size and high similarity of agricultural pests bring challenges to the prompt and accurate pest detection using imaging technologies. The key impetus of this paper is to achieve a good balance between efficiency and accuracy for pest detection on the basis of agricultural image data mining. This paper proposes Pest-YOLO which is a real-time agriculture pest detection method based on the improved convolutional neural network (CNN) and YOLOv4. First, a squeeze-and-excitation attention mechanism module is introduced to CNN for mining image data, extracting key features, and suppressing unrelated features. Then, a cross-stage multi-feature fusion method is designed to improve the structure of feature pyramid network and path aggregation network, thus enhancing the feature expressiveness of small targets like pests. Finally, our Pest-YOLO realizes end-to-end real-time pest detection with high accuracy based on improved CNN and YOLOv4. We evaluate the performance of our method on a typical large-scale pest dataset including 28k images and 24 classes. Experimental results demonstrate that our method outperforms the state-of-the-art solutions including Faster R-CNN and YOLO-based detectors, and achieves good performance with 71.6% mAP and 83.5% Recall. The proposed method is effective and applicable for accurate and real-time intelligent pest detection without expertise feature engineering.

We introduce Neural Radiosity, an algorithm to solve the rendering equation by minimizing the norm of its residual, similar as in classical radiosity techniques. Traditional basis functions used in radiosity, such as piecewise polynomials or meshless basis functions are typically limited to representing isotropic scattering from diffuse surfaces. Instead, we propose to leverage neural networks to represent the full four-dimensional radiance distribution, directly optimizing network parameters to minimize the norm of the residual. Our approach decouples solving the rendering equation from rendering (perspective) images similar as in traditional radiosity techniques, and allows us to efficiently synthesize arbitrary views of a scene. In addition, we propose a network architecture using geometric learnable features that improves convergence of our solver compared to previous techniques. Our approach leads to an algorithm that is simple to implement, and we demonstrate its effectiveness on a variety of scenes with diffuse and non-diffuse surfaces.

Most current speech enhancement models use spectrogram features that require an expensive transformation and result in phase information loss. Previous work has overcome these issues by using convolutional networks to learn the temporal correlations across high-resolution waveforms. These models, however, are limited by memory-intensive dilated convolution and aliasing artifacts from upsampling. We introduce an end-to-end fully recurrent neural network for single-channel speech enhancement. The network structured as an hourglass-shape that can efficiently capture long-range temporal dependencies by reducing the features resolution without information loss. Also, we use residual connections to prevent gradient decay over layers and improve the model generalization. Experimental results show that our model outperforms state-of-the-art approaches in six quantitative evaluation metrics.

Emotion recognition in dyadic communication is challenging because: 1. Extracting informative modality-specific representations requires disparate feature extractor designs due to the heterogenous input data formats. 2. How to effectively and efficiently fuse unimodal features and learn associations between dyadic utterances are critical to the model generalization in actual scenario. 3. Disagreeing annotations prevent previous approaches from precisely predicting emotions in context. To address the above issues, we propose an efficient dyadic fusion network that only relies on an attention mechanism to select representative vectors, fuse modality-specific features, and learn the sequence information. Our approach has three distinct characteristics: 1. Instead of using a recurrent neural network to extract temporal associations as in most previous research, we introduce multiple sub-view attention layers to compute the relevant dependencies among sequential utterances; this significantly improves model efficiency. 2. To improve fusion performance, we design a learnable mutual correlation factor inside each attention layer to compute associations across different modalities. 3. To overcome the label disagreement issue, we embed the labels from all annotators into a k-dimensional vector and transform the categorical problem into a regression problem; this method provides more accurate annotation information and fully uses the entire dataset. We evaluate the proposed model on two published multimodal emotion recognition datasets: IEMOCAP and MELD. Our model significantly outperforms previous state-of-the-art research by 3.8%-7.5% accuracy, using a more efficient model.

Multimodal affective computing, learning to recognize and interpret human affect and subjective information from multiple data sources, is still challenging because:(i) it is hard to extract informative features to represent human affects from heterogeneous inputs; (ii) current fusion strategies only fuse different modalities at abstract levels, ignoring time-dependent interactions between modalities. Addressing such issues, we introduce a hierarchical multimodal architecture with attention and word-level fusion to classify utterance-level sentiment and emotion from text and audio data. Our introduced model outperforms state-of-the-art approaches on published datasets, and we demonstrate that our model's synchronized attention over modalities offers visual interpretability.

We examine the utility of linguistic content and vocal characteristics for multimodal deep learning in human spoken language understanding. We present a deep multimodal network with both feature attention and modality attention to classify utterance-level speech data. The proposed hybrid attention architecture helps the system focus on learning informative representations for both modality-specific feature extraction and model fusion. The experimental results show that our system achieves state-of-the-art or competitive results on three published multimodal datasets. We also demonstrated the effectiveness and generalization of our system on a medical speech dataset from an actual trauma scenario. Furthermore, we provided a detailed comparison and analysis of traditional approaches and deep learning methods on both feature extraction and fusion.

In this paper, we present a novel deep multimodal framework to predict human emotions based on sentence-level spoken language. Our architecture has two distinctive characteristics. First, it extracts the high-level features from both text and audio via a hybrid deep multimodal structure, which considers the spatial information from text, temporal information from audio, and high-level associations from low-level handcrafted features. Second, we fuse all features by using a three-layer deep neural network to learn the correlations across modalities and train the feature extraction and fusion modules together, allowing optimal global fine-tuning of the entire structure. We evaluated the proposed framework on the IEMOCAP dataset. Our result shows promising performance, achieving 60.4% in weighted accuracy for five emotion categories.

Human conversation analysis is challenging because the meaning can be expressed through words, intonation, or even body language and facial expression. We introduce a hierarchical encoder-decoder structure with attention mechanism for conversation analysis. The hierarchical encoder learns word-level features from video, audio, and text data that are then formulated into conversation-level features. The corresponding hierarchical decoder is able to predict different attributes at given time instances. To integrate multiple sensory inputs, we introduce a novel fusion strategy with modality attention. We evaluated our system on published emotion recognition, sentiment analysis, and speaker trait analysis datasets. Our system outperformed previous state-of-the-art approaches in both classification and regressions tasks on three datasets. We also outperformed previous approaches in generalization tests on two commonly used datasets. We achieved comparable performance in predicting co-existing labels using the proposed model instead of multiple individual models. In addition, the easily-visualized modality and temporal attention demonstrated that the proposed attention mechanism helps feature selection and improves model interpretability.

We present a deep learning framework for fast 3D activity localization and tracking in a dynamic and crowded real world setting. Our training approach reverses the traditional activity localization approach, which first estimates the possible location of activities and then predicts their occurrence. Instead, we first trained a deep convolutional neural network for activity recognition using depth video and RFID data as input, and then used the activation maps of the network to locate the recognized activity in the 3D space. Our system achieved around 20cm average localization error (in a 4m × 5m room) which is comparable to Kinect's body skeleton tracking error (10--20cm), but our system tracks activities instead of Kinect's location of people.

We introduce the Concurrent Activity Recognizer (CAR) - an efficient deep learning structure that recognizes complex concurrent teamwork activities from multimodal data. We implemented the system in a challenging medical setting, where it recognizes 35 different activities using Kinect depth video and data from passive RFID tags on 25 types of medical objects. Our preliminary results showed our system achieved an 84% average accuracy with 0.20 F1-Score.

Summary As mobile social networks (MSNs) are booming and gaining tremendous popularity, there have been an increasing number of communications and interactions among users. Taking this advantage, users in MSNs make decisions via collecting and combining trust information from different users. Hence, trust evaluation technology has become a key requirement for network security in MSNs. In such MSNs, however, the community/group structures are dynamically changing, and users may belong to multiple communities/groups. Therefore, trust evaluation plays a critical role in inferring trustworthy contacts among users. In this paper, an innovative trust inference model is proposed for MSNs, in which multiple dimensional trust metrics are incorporated to reflect the complexity of trust. To infer trust relations between users in MSNs with complex communities, we first construct dynamic implicit social behavioral graphs (DynISBG) based on dynamic complex community/group structures and propose an efficient detection algorithm forDynISBGunder fuzzy degreeκ. We then present a multi‐dimensional fuzzy trust inferring approach that involves four metrics, that is, static attribute trust factor, dynamic behavioral trust factor, long‐term trust evolution factor, and recommendation‐based trust opinion. Moreover, to obtain the recommendation‐based trust opinion about indirect connected users, we discuss the trust aggregation and propagation along trust path. Finally, we evaluate the performance of our novel approach with simulations. The results show that, compared with the existing approaches, the proposed model provides a more detailed analysis in trust evaluation with higher accuracy. Copyright © 2016 John Wiley & Sons, Ltd.