Accurate target segmentation from computed tomography (CT) scans is crucial for surgical robots to perform clinical surgeries successfully. However, the lack of medical image data and annotations has been the biggest obstacle to learning robust medical image segmentation models. Self-supervised learning can effectively address this problem by providing a strategy to pre-train a model with unlabeled data, and then fine-tune downstream tasks with limited labeled data. Existing self-supervised methods fail to simultaneously utilize the abundant global anatomical structure information and local feature differences in medical imaging. In this work, we propose a new strategy for the pre-training framework, which uses the three-dimensional anatomical structure of medical images and specific task and background cues to segment volumetric medical images with limited annotations. Specifically, we propose (1) learning intrinsic patterns of volumetric medical image structures through multiple sub-tasks, and (2) designing a multi-level background cube contrastive learning strategy to enhance the target feature representation by exploiting the differences between the specific target and background. We conduct extensive evaluations on two publicly available datasets. Under limited annotation settings, the proposed method yields significant improvements compared to other self-supervised learning techniques. The proposed method achieves within 6% of the baseline performance using only five labeled CT volumes for training.

Multi-view Comprehensive Representation Learning (MCRL) aims to synthesize information from multiple views to learn comprehensive representations of data items. Prevalent deep MCRL methods typically concatenate synergistic view-specific representations or average aligned view-specific representations in the fusion stage. However, the performance of synergistic fusion methods inevitably degenerate or even fail when partial views are missing in real-world applications; the aligned based fusion methods usually cannot fully exploit the complementarity of multi-view data. To eliminate all these drawbacks, in this work we present a Progressive Deep Multi-view Fusion (PDMF) method. Considering the multi-view comprehensive representation should contain complete information and the view-specific data contain partial information, we deem that it is unstable to directly learn the mapping from partial information to complete information. Hence, PDMF employs a progressive learning strategy, which contains the pre-training and fine-tuning stages. In the pre-training stage, PDMF decodes the auxiliary comprehensive representation to the view-specific data. It also captures the consistency and complementarity by learning the relations between the dimensions of the auxiliary comprehensive representation and all views. In the fine-tuning stage, PDMF learns the mapping from the original data to the comprehensive representation with the help of the auxiliary comprehensive representation and relations. Experiments conducted on a synthetic toy dataset and 4 real-world datasets show that PDMF outperforms state-of-the-art baseline methods. The code is released at https://github.com/winterant/PDMF.

Unmanned aerial vehicle (UAV) carrying intelligent reflecting surface (IRS) can serve as an aerial platform to improve the coverage area and transmission performance of traditional wireless network. In this paper, we investigate the weighted sum rate maximization problem for a UAV-assisted and IRS-based multi-user communication system. Specifically, by applying the alternating optimization approach, we propose a two-phase approach to effectively optimize the number of activated reflective elements, the precoding matrix, the phase shift, and the UAV location. Numerical results validate that the proposed approach can converge at a faster rate and improve the weighted sum rate performance.

In this paper, we investigate the secrecy outage probability (SOP) for a multi-tier unmanned aerial vehicular (UAV) assisted cognitive communication network. Specifically, the low-altitude rotary-wing (LARW) UAV relays harvest radio frequency (RF) energy from the transmission of a high-altitude fixed-wing (HAFW) UAV and forward the information to a ground destination under a decode-and-forward protocol while a ground eavesdropper tries to capture the relay signal. The multi-tier UAV system transmits in an underlay fashion over a licensed spectrum of a primary user. We study the exact statistics of SOP assuming that the number of UAV relays follows a Poisson point process (PPP). The simulation results are presented to illustrate and verify the analytical results.

Abstract Image classification is one of the mainstream tasks of computer vision. However, the most existing methods use labels of the same granularity level for training. This leads to ignoring the hierarchy that may help to differentiate different visual objects better. Embedding hierarchical information into the convolutional neural networks (CNNs) can effectively regulate the semantic space and thus reduce the ambiguity of prediction. To this end, a multi‐task learning framework, named as Hierarchical Bilinear Convolutional Neural Network (HB‐CNN), is developed by seamlessly integrating CNNs with multi‐task learning over the hierarchical visual concept structures. Specifically, the labels with a tree structure are used as the supervision to hierarchically train multiple branch networks. In this way, the model can not only learn additional information (e.g. context information) as the coarse‐level category features, but also focus the learned fine‐level category features on the object properties. To smoothly pass hierarchical conceptual information and encourage feature reuse, a connectivity pattern is proposed to connect features at different levels. Furthermore, a bilinear module is embedded to generalise various orderless texture feature descriptors so that our model can capture more discriminative features. The proposed method is extensively evaluated on the CIFAR‐10, CIFAR‐100, and ‘Orchid’ Plant image sets. The experimental results show the effectiveness and superiority of our method.

As an essential role in smart city applications, personalized recommender systems help users to find their potentially interested items from their historically generated data. Recently, researchers have started to utilize the massive user-generated multimodal contents to improve recommendation performance. However, previous methods have at least one of the following drawbacks: 1) employing shallow models, which cannot well capture high-level conceptual information; 2) failing to capture personalized user visual preference. In this article, we present a deep users’ multimodal preferences-based recommendation (UMPR) method to capture the textual and visual matching of users and items for recommendation. We extract textual matching from historical reviews. We construct users’ visual preference embeddings to model users’ visual preference and match them with items’ visual embeddings to obtain the visual matching. We apply UMPR on two applications related to smart city: restaurant recommendation and product recommendation. Experiments show that UMPR outperforms competitive baseline methods.

Merchants of e-commerce Websites expect recommender systems to entice more consumption which is highly correlated with the customers' purchasing propensity. However, most existing recommender systems focus on customers' general preference rather than purchasing propensity often governed by instant demands which we deem to be well conveyed by the questions asked by customers. A typical recommendation scenario is: Bob wants to buy a cell phone which can play the game PUBG. He is interested in HUAWEI P20 and asks “can PUBG run smoothly on this phone?” under it. Then our system will be triggered to recommend the most eligible cell phones to him. Intuitively, diverse user questions could probably be addressed in reviews written by other users who have similar concerns. To address this recommendation problem, we propose a novel Question-Driven Attentive Neural Network (QDANN) to assess the instant demands of questioners and the eligibility of products based on user generated reviews, and do recommendation accordingly. Without supervision, QDANN can well exploit reviews to achieve this goal. The attention mechanisms can be used to provide explanations for recommendations. We evaluate QDANN in three domains of Taobao. The results show the efficacy of our method and its superiority over baseline methods.

Online Shopping has become a part of our daily routine, but it still cannot offer intuitive experience as store shopping. Nowadays, most e-commerce Websites offer a Question Answering (QA) system that allows users to consult other users who have purchased the product. However, users still need to wait patiently for others’ replies. In this paper, we investigate how to provide a quick response to the asker by plausible answer identification from product reviews. By analyzing the similarity and discrepancy between explicit answers and reviews that can be answers, a novel multi-task deep learning method with carefully designed attention mechanisms is developed. The method can well exploit large amounts of user generated QA data and a few manually labeled review data to address the problem. Experiments on data collected from Amazon demonstrate its effectiveness and superiority over competitive baselines.

We are concerned with using user-tagged images to learn proper hashing functions for image retrieval. The benefits are two-fold: (1) we could obtain abundant training data for deep hashing models; (2) tagging data possesses richer semantic information which could help better characterize similarity relationships between images. However, tagging data suffers from noises, vagueness and incompleteness. Different from previous unsupervised or supervised hashing learning, we propose a novel weakly-supervised deep hashing framework which consists of two stages: weakly-supervised pre-training and supervised fine-tuning. The second stage is as usual. In the first stage, rather than performing supervision on tags, the framework introduces a semantic embedding vector (sem-vector) for each image and performs learning of hashing and sem-vectors jointly. By carefully designing the optimization problem, it can well leverage tagging information and image content for hashing learning. The framework is general and does not depend on specific deep hashing methods. Empirical results on real world datasets show that when it is integrated with state-of-art deep hashing methods, the performance increases by 8-10%.

Product reviews are valuable for upcoming buyers in helping them make decisions. To this end, different opinion mining techniques have been proposed, where judging a review sentence's orientation (e.g., positive or negative) is one of their key challenges. Recently, deep learning has emerged as an effective means for solving sentiment classification problems. A neural network intrinsically learns a useful representation automatically without human efforts. However, the success of deep learning highly relies on the availability of large-scale training data. We propose a novel deep learning framework for product review sentiment classification which employs prevalently available ratings as weak supervision signals. The framework consists of two steps: (1) learning a high level representation (an embedding space) which captures the general sentiment distribution of sentences through rating information; and (2) adding a classification layer on top of the embedding layer and use labeled sentences for supervised fine-tuning. We explore two kinds of low level network structure for modeling review sentences, namely, convolutional feature extractors and long short-term memory. To evaluate the proposed framework, we construct a dataset containing 1.1M weakly labeled review sentences and 11,754 labeled review sentences from Amazon. Experimental results show the efficacy of the proposed framework and its superiority over baselines.