Surgical smoke caused poor visibility during laparoscopic surgery, the smoke removal is important to improve the safety and efficiency of the surgery. We propose the Multilevel-feature-learning Attention-aware based Generative Adversarial Network for Removing Surgical Smoke (MARS-GAN) in this work. MARS-GAN incorporates multilevel smoke feature learning, smoke attention learning, and multi-task learning together. Specifically, the multilevel smoke feature learning adopts the multilevel strategy to adaptively learn non-homogeneity smoke intensity and area features with specific branches and integrates comprehensive features to preserve both semantic and textural information with pyramidal connections. The smoke attention learning extends the smoke segmentation module with the dark channel prior module to provide the pixel-wise measurement for focusing on the smoke features while preserving the smokeless details. And the multi-task learning strategy fuses the adversarial loss, cyclic consistency loss, smoke perception loss, dark channel prior loss, and contrast enhancement loss to help the model optimization. Furthermore, a paired smokeless/smoky dataset is synthesized for elevating smoke recognition ability. The experimental results show that MARS-GAN outperforms the comparative methods for removing surgical smoke on both synthetic/real laparoscopic surgical images, with the potential to be embedded in laparoscopic devices for smoke removal.

Mobile crowdsensing has been regarded as an efficient paradigm for performing large-scale sensing tasks. In this paper, we consider a specific scenario, where the crowdsensing platform needs to collect the sensing data in a requested time window (RTW), and mobile users would bid for their sensing time windows. This process could be modeled as a reverse auction. In this context, the Vickrey-Clark-Groves (VCG) mechanism becomes a generic auction mechanism that uniquely guarantees both truthfulness and efficiency, but it is vulnerable to false-name bidding and generates high overpayment for the platform. Thus in this paper, we design the core-selecting mechanism to solve VCG’s vulnerability and improve the revenue. We demonstrate that our proposed mechanism achieves the properties of RTW feasibility, efficiency, individual rationality, and false-name-proofness. Besides, to minimize the incentives of users to deviate from truthful-telling, we adopt a VCG-nearest payment rule and propose an efficient algorithm called CCG-TWC. Our extensive simulation results show that the core-selecting mechanism could reduce VCG’s overpayment by about 10%.

Multi-label text classification (MLTC) is an important task in natural language processing, which assigns multiple labels to each text in the dataset. Typical method like Binary Relevance (BR) is arguably the most intuitive solution for the task. It works by decomposing the multi-label learning task into a number of independent binary learning tasks while ignoring the correlation between labels. Recently, neural network models attract much attention. Researchers view the MLTC task as a sequence generation problem. Although some new methods based on generative model (e.g. sequence-to-sequence), such as novel decoder structure and various attention mechanisms, can improve the performance. These methods still have some short-comings, such as unreasonable loss function, unclear ordering of target labels. To address these limitations, we propose a simple and effective novel model, which combines the merits of neural network and BRs methods. Our model also takes into account the categories and levels of labels. We decompose the MLTC problem to binary classification, together with global and local extractor to avoid the impact of label ordering and cumulative error. Experimental results show that our model achieves an improvement of 3.0% micro-F1 and a reduction of 6.0% hamming loss on AAPD dataset compared with the state-of-the-art work. And obtained good performance on RCV1-V2 dataset.

We propose a weakly supervised semantic segmentation algorithm that uses image tags for supervision. We apply the tags in queries to collect three sets of web images, which encode the clean foregrounds, the common backgrounds, and realistic scenes of the classes. We introduce a novel three-stage training pipeline to progressively learn semantic segmentation models. We first train and refine a class-specific shallow neural network to obtain segmentation masks for each class. The shallow neural networks of all classes are then assembled into one deep convolutional neural network for end-to-end training and testing. Experiments show that our method notably outperforms previous state-of-the-art weakly supervised semantic segmentation approaches on the PASCAL VOC 2012 segmentation benchmark. We further apply the class-specific shallow neural networks to object segmentation and obtain excellent results.

Converting a color image to a grayscale image, namely decolorization, is an important process for many real-world applications. Previous methods build contrast loss functions to minimize the contrast differences between the color images and the resultant grayscale images. In this paper, we improve upon a widely used decolorization method with two extensions. First, we relax the need for heuristics on color orders, which the baseline method relies on when computing the contrast differences. In our method, the color orders are incorporated into the loss function and are determined through optimization. Moreover, we apply a nonlinear function on the grayscale contrast to better model human perception of contrast. Both qualitative and quantitative results on the standard benchmark demonstrate the effectiveness of our two extensions.

Keyword-based image color re-rendering is a convenient way to enhance the color of images. Most methods only focus on the color characteristics of the image while ignoring the semantic meaning of different regions. We propose to incorporate semantic information into the color re-rendering pipeline through semantic segmentation. Using semantic segmentation masks, we first generate more accurate correlations between keywords and color characteristics than the state-of-the-art approach. Such correlations are then adopted for rerendering the color of the input image, where the segmentation masks are used to indicate the regions for color rerendering. Qualitative comparisons show that our method generates visually better results than the state-of-the-art approach. We further validate with a psychophysical experiment, where the participants prefer the results of our method.

Convolutional Neural Networks (CNNs) have been widely adopted for many imaging applications. For image aesthetics prediction, state-of-the-art algorithms train CNNs on a recently-published large-scale dataset, AVA. However, the distribution of the aesthetic scores on this dataset is extremely unbalanced, which limits the prediction capability of existing methods. We overcome such limitation by using weighted CNNs. We train a regression model that improves the prediction accuracy of the aesthetic scores over state-of-the-art algorithms. In addition, we propose a novel histogram prediction model that not only predicts the aesthetic score, but also estimates the difficulty of performing aesthetics assessment for an input image. We further show an image enhancement application where we obtain an aesthetically pleasing crop of an input image using our regression model.

In this paper, we propose a novel algorithm for improving and visualizing image search results. The proposed algorithm improves user's image search experience by three steps: (1) re-rank the initial image search results by the random walk refinement based on visual consistency and saliency cues, (2) project the re-ranked images into a 2-dimentional panel according to their saliency information and correlations, (3) detect and extract the saliency regions in each image for final visualization. To evaluate the performance of our algorithm, user study has been conducted. Experimental results demonstrate that our visualization algorithm provides more pleasing image search experience than the conventional image search methods.

A hierarchical neurocontroller architecture, which comprises two artificial neural network (ANN) systems for inversion kinematics solution and motion control of robotic redundant manipulators is presented. The solution of inverse kinematics is realized by a Hopfield network, in which the global planning of a collision-free trajectory is based on the potential functions using the necessary conditions of minimum for an integral type criterion. A direct servo-level controller is utilized by a multilayer feedforward network based on backpropagation algorithm, in which the computed torque technique is employed to control manipulator's joints to track the trajectory. The stability of the both sub-controllers is analyzed. Another major contribution of this paper is to provide an approach for the most difficult problem in using neurocontroller-how to efficiently train the designed ANNs.