Abstract Early diagnosis of Alzheimer's disease (AD) plays a key role in preventing and responding to this neurodegenerative disease. It has shown that, compared with a single imaging modality‐based classification of AD, synergy exploration among multimodal neuroimages is beneficial for the pathological identification. However, effectively exploiting multimodal information is still a big challenge due to the lack of efficient fusion methods. Herein, a multimodal fusion network based on attention mechanism is proposed, in which magnetic resonance imaging (MRI) and positron emission computed tomography (PET) images are converted into feature vectors with the same dimension, while the demographic information and clinical data are preprocessed and converted into feature vectors through embedding. This attention model can focus on important feature points, fuse the multimodal information more effectively, and thus provide accurate diagnosis and prediction for different pathological stages. The results show that the model achieves an accuracy of 84.1% for triple classification tasks in normal cognition (NC) versus mild cognitive impairment (MCI) versus AD and 93.9% prediction accuracy in stable MCI (sMCI) versus progressive MCI (pMCI). In contrast to the existing multimodal diagnosis methods, our model yields a state‐of‐the‐art accuracy of AD diagnosis, which is powerful and promising in clinical practice.

The goal of Multi-View Stereo is to reconstruct the 3D point cloud model from multiple views. With the development of deep learning, more and more learning-based research has achieved remarkable results. However, existing methods ignore the fine-grained features of the bottom layer, which leads to the poor quality of model reconstruction, especially in terms of completeness. Besides, current methods still rely on a large amount of consumed memory resources because of the application of 3D convolution. To this end, this paper proposes a Multiple Granularities Feature Fusion Network for Multi-View Stereo, an end-to-end depth estimation network combining global and local features, which is characterized by fine-granularity multi-feature fusion. Firstly, we propose a dense feature adaptive connection module, which can adaptively fuse the global and local features in the scene, provide a more complete and effective feature map for inferring a more detailed depth map, and make the ultimate model more complete. Secondly, in order to further improve the accuracy and completeness of the reconstructed point cloud, we introduce normal and edge loss futead of only using depth loss functions as in the existing methods, which makes the network more sensitive to small depth structures. Finally, we propose distributed 3D convolution instead of traditional 3D convolution, which reduces memory consumption. The experimental results on the DTU and Tanks & Temples datasets demonstrate that the proposed method in this papaer achieves the state-of-the-art performance, which proves the accuracy and effectiveness of the MG-MVSNet proposed in this paper.

To balance the inference speed and detection accuracy of a grasp detection algorithm, which are both important for robot grasping tasks, we propose an encoder–decoder structured pixel-level grasp detection neural network named the attention-based efficient robot grasp detection network (AE-GDN). Three spatial attention modules are introduced in the encoder stages to enhance the detailed information, and three channel attention modules are introduced in the decoder stages to extract more semantic information. Several lightweight and efficient DenseBlocks are used to connect the encoder and decoder paths to improve the feature modeling capability of AE-GDN. A high intersection over union (IoU) value between the predicted grasp rectangle and the ground truth does not necessarily mean a high-quality grasp configuration, but might cause a collision. This is because traditional IoU loss calculation methods treat the center part of the predicted rectangle as having the same importance as the area around the grippers. We design a new IoU loss calculation method based on an hourglass box matching mechanism, which will create good correspondence between high IoUs and high-quality grasp configurations. AEGDN achieves the accuracy of 98.9% and 96.6% on the Cornell and Jacquard datasets, respectively. The inference speed reaches 43.5 frames per second with only about 1.2 × 106 parameters. The proposed AE-GDN has also been deployed on a practical robotic arm grasping system and performs grasping well. Codes are available at https://github.com/robvincen/robot_gradet .

BACKGROUND: Drug-Drug interactions (DDIs) are a challenging problem in drug research. Drug combination therapy is an effective solution to treat diseases, but it can also cause serious side effects. Therefore, DDIs prediction is critical in pharmacology. Recently, researchers have been using deep learning techniques to predict DDIs. However, these methods only consider single information of the drug and have shortcomings in robustness and scalability. RESULTS: In this paper, we propose a multi-type feature fusion based on graph neural network model (MFFGNN) for DDI prediction, which can effectively fuse the topological information in molecular graphs, the interaction information between drugs and the local chemical context in SMILES sequences. In MFFGNN, to fully learn the topological information of drugs, we propose a novel feature extraction module to capture the global features for the molecular graph and the local features for each atom of the molecular graph. In addition, in the multi-type feature fusion module, we use the gating mechanism in each graph convolution layer to solve the over-smoothing problem during information delivery. We perform extensive experiments on multiple real datasets. The results show that MFFGNN outperforms some state-of-the-art models for DDI prediction. Moreover, the cross-dataset experiment results further show that MFFGNN has good generalization performance. CONCLUSIONS: Our proposed model can efficiently integrate the information from SMILES sequences, molecular graphs and drug-drug interaction networks. We find that a multi-type feature fusion model can accurately predict DDIs. It may contribute to discovering novel DDIs.

Abstract Recently, graph convolutional networks have shown excellent results in skeleton‐based action recognition. This paper presents a multi‐stage part‐aware graph convolutional network for the problems of model over complication, parameter redundancy and lack of long‐dependence feature information. The structure of this network has a multi‐stream input and two‐stream output, which can greatly reduce the complexity and improve the accuracy of the model without losing sequence information. The two branches of the network have the same backbone, which includes 6 multi‐order feature extraction blocks and 3 temporal attention calibration blocks, and the outputs of the two branches are fused together. In multi‐order feature extraction block, a channel‐spatial attention mechanism and a graph condensation module are proposed, which can extract more distinguishable feature and identify the relationship between parts. In temporal attention calibration block, the temporal dependencies between frames in the skeleton sequence are modeled. Experimental results show that the proposed network outperforms many mainstream methods on NTU and Kinetics datasets, for example, it achieves 92.4% accuracy on the cross‐subject benchmark of NTU‐RGBD60 dataset.

Designing a novel and efficient architecture is the thrust of medical image segmentation. In this paper, we introduce a novel network named Multi-scale Feedback Feature Refinement U-Net (MFFRU-Net) for medical image segmentation. We design a simple and effective multi-scale feedback structure. Up-sampling and 1 × 1 convolution are used to feedback the feature maps of different scales in the decoder to the encoder, so that multiple high-level and low-level features are fused to obtain more representative features. Specifically, we propose a feature refinement module (FRM) based on the dual attention mechanism in the middle layer of the network. FRM block can enhance the use of spatial and channel information of image features. We evaluate the MFFRU-Net on two datasets. Comprehensive experimental results show that the proposed method is superior to the original U-Net method and other advanced methods.