Representation learning is widely used to project high-dimensional data to low-dimensional subspace for feature extraction in image recognition tasks. However, many related methods barely explore the fuzziness and uncertainty between data classes. Besides, the classical unsupervised sparse constraint weakens the evaluation of feature importance and neglects the preservation of discriminant information during sparse representation. To solve these issues, a novel fuzzy discriminative block representation learning (FDBRL) algorithm is proposed for image feature extraction. FDBRL aims to enhance the discriminability of subspace by designing effective constraints for projection learning. Specifically, based on the label information and the fuzzy relation between data, we construct a fuzzy block weight matrix and embed it into the${l_{2,1}}$norm regularization term to realize supervised sparse constraint for the representation learning. Next, the low-rank constraint is used to capture the inherent global structure information of data. Finally, we introduce a classification loss term with transformation matrix for joint optimization, such that the projection learning is not limited to number of classes, and the discriminative ability is further improved. Comprehensive experimental results on six benchmarks verify that our method achieves promising performance with other state-of-the-arts in both robustness and effectiveness.

Shadow removal is a computer-vision task that aims to restore the image content in shadow regions. While almost all recent shadow-removal methods require shadow-free images for training, in ECCV 2020 Le and Samaras introduces an innovative approach without this requirement by cropping patches with and without shadows from shadow images as training samples. However, it is still laborious and time-consuming to construct a large amount of such unpaired patches. In this paper, we propose a new G2R-ShadowNet which leverages shadow generation for weakly-supervised shadow removal by only using a set of shadow images and their corresponding shadow masks for training. The proposed G2R-ShadowNet consists of three sub-networks for shadow generation, shadow removal and refinement, respectively and they are jointly trained in an end-to-end fashion. In particular, the shadow generation sub-net stylises non-shadow regions to be shadow ones, leading to paired data for training the shadow-removal sub-net. Extensive experiments on the ISTD dataset and the Video Shadow Removal dataset show that the proposed G2R-ShadowNet achieves competitive performances against the current state of the arts and outperforms Le and Samaras’ patch-based shadow-removal method.

Cellular communications have been regarded as promising approaches to deliver high-broadband communication links for unmanned aerial vehicles (UAVs), which have been widely deployed to conduct various missions, e.g., precision agriculture, forest monitoring, and border patrol. However, the unique features of aerial UAVs, including high-altitude manipulation, 3-D mobility, and rapid velocity changes, pose challenging issues to realize reliable cellular-enabled UAV communications, especially with the severe intercell interference generated by UAVs. To deal with this issue, we propose a novel position-based robust beamforming algorithm through complementarily integrating the navigation information and wireless channel information to improve the performance of cellular-enabled UAV communications. Specifically, in order to achieve the optimal beam weight vector, the navigation information of the UAV system is innovatively exploited to predict the changes of the direction-of-Arrival (DoA) angle. To fight against the high mobility of UAV operations, an optimization problem is formed by considering the tapered surface of DoA angle and solved to correct the inherent position error. Comprehensive simulation experiments are conducted and the results show that the proposed robust beamforming algorithm could achieve over 90% DoA estimation error reduction and up to 14-dB SINR gain compared with five benchmark beamforming algorithms, including linearly constrained minimum variance (LCMV), position-based beamforming, diagonal loading (DL), robust capon beamforming (RCB), and robust LCMV algorithm.

Shadow removal can significantly improve the image visual quality and has many applications in computer vision. Deep learning methods based on CNNs have become the most effective approach for shadow removal by training on either paired data, where both the shadow and underlying shadow-free versions of an image are known, or unpaired data, where shadow and shadow-free training images are totally different with no correspondence. In practice, CNN training on unpaired data is more preferred given the easiness of training data collection. In this paper, we present a new Lightness-Guided Shadow Removal Network (LG-ShadowNet) for shadow removal by training on unpaired data. In this method, we first train a CNN module to compensate for the lightness and then train a second CNN module with the guidance of lightness information from the first CNN module for final shadow removal. We also introduce a loss function to further utilise the colour prior of existing data. Extensive experiments on widely used ISTD, adjusted ISTD and USR datasets demonstrate that the proposed method outperforms the state-of-the-art methods with training on unpaired data.

By involving only subtle motions of body parts, video-based microaction recognition is a very important but challenging problem. Most existing action recognition methods are developed for general actions, and the current state-of-the-art methods usually largely rely on high-layer features learned from convolutional neural networks (CNNs). High-layer CNN features usually contain more semantic information but less detailed information. However, detailed information can be important for microactions due to the motion subtleness of such actions. In this paper, we propose to more effectively learn midlayer CNN features for enhancing microaction recognition. More specifically, we develop a new dual-branch network for microaction recognition: one branch uses the high-layer CNN features for classification, and the second branch further explores the midlayer CNN features for classification. In the second branch, we introduce a novel subtle motion detector consisting of three modules: 1) a discriminative spatial-temporal feature learning module, which further learns the subtle motion features corresponding to the discriminative spatial-temporal regions, 2) a parallel multiplier attention module, which further refines the features learned in channels and spatial-temporal domains, and 3) an activation fusion module, which fuses the max and average activations from midlayer CNN features for classification. In the experiments, we build a new microaction video dataset, where the micromotions of interest are mixed with other larger general motions such as walking. Comprehensive experimental results verify that the proposed method yields new state-of-the-art performance in two microaction video datasets, while its performance on two generalaction video datasets is also very promising.

Recently, data-driven deep saliency models have achieved high performance and have outperformed classical saliency models, as demonstrated by results on datasets such as the MIT300 and SALICON. Yet, there remains a large gap between the performance of these models and the inter-human baseline. Some outstanding questions include what have these models learned, how and where they fail, and how they can be improved. This article attempts to answer these questions by analyzing the representations learned by individual neurons located at the intermediate layers of deep saliency models. To this end, we follow the steps of existing deep saliency models, that is borrowing a pre-trained model of object recognition to encode the visual features and learning a decoder to infer the saliency. We consider two cases when the encoder is used as a fixed feature extractor and when it is fine-tuned, and compare the inner representations of the network. To study how the learned representations depend on the task, we fine-tune the same network using the same image set but for two different tasks: saliency prediction versus scene classification. Our analyses reveal that: 1) some visual regions (e.g. head, text, symbol, vehicle) are already encoded within various layers of the network pre-trained for object recognition, 2) using modern datasets, we find that fine-tuning pre-trained models for saliency prediction makes them favor some categories (e.g. head) over some others (e.g. text), 3) although deep models of saliency outperform classical models on natural images, the converse is true for synthetic stimuli (e.g. pop-out search arrays), an evidence of significant difference between human and data-driven saliency models, and 4) we confirm that, after-fine tuning, the change in inner-representations is mostly due to the task and not the domain shift in the data.

In this paper, we propose a novel video coding scheme to significantly reduce the coding complexity and enhance overall coding efficiency in videos acquired by high mobility devices such as unmanned aerial vehicles (UAVs). In order to reduce the encoded data bits and encoding time to facilitate real-time data transmission, as well as minimize the image distortion caused by the jitter of onboard camera, a sensor-assisted global motion estimation (GMV) algorithm is designed to calculate perspective transformation model and global motion vectors, which are used in both the inter-frame coding to improve the coding efficiency and intra-frame coding to reduce block search complexity. We conducted comprehensive simulation experiments on official HM-16.10 codec and the performance results show the proposed method can achieve faster block search by 50% to 60% speedup and lower bitrate by 15% to 30% compared with standard HEVC coding software.

Unmanned Aerial Vehicles (UAVs) have demonstrated exceptional capabilities in many real-world applications such as remote sensing, emergency medicine delivery and precision agriculture. LTE networks are regarded as key candidates to offer high performance broadband wireless services to support UAV applications and safe deployment. However, the unique features of aerial UAVs including high-altitude manipulation, three-dimension (3D) mobility and rapid velocity changes, pose challenging issues for optimising LTE wireless communications to support UAVs, especially under the severe inter-cell interference generated by UAVs in the sky. To deal with this issue, we propose a novel position-based robust beamforming algorithm to improve the performance of LTE networks to serve UAVs. For obtaining optimal weight vectors that could tolerate Direction-of-Arrival (DoA) estimation errors, we propose a hybrid method to integrate the channel information and UAV flight information to accurately estimate the DoA angle range. In order to validate the performance of the proposed robust beamforming algorithm, we conduct comprehensive simulation experiments under practical configurations. The results show that the proposed robust beamforming algorithm outperforms benchmark Linearly Constrained Minimum Variance (LCMV) beamforming and GPS-based beamforming algorithms.

Recognizing micro actions from videos is a very challenging problem since they involve only subtle motions of body parts. In this paper, we propose a new deep-learning based method for micro action recognition by building a segment-level temporal pyramid to better capture the motion details. More specifically, we first temporally sample the input video for short segments and for each of the video segment, we employ a two-stream convolutional neural networks (CNNs) followed by a temporal pyramid for extracting deep features. Finally, the features derived from all the video segments are combined for action classification. We evaluate the proposed method on a micro-action video dataset, as well as a general-action video dataset, with very promising results.

Simultaneous wireless information and power transfer (SWIPT) offers novel designs that could enhance the sustainability and resilience of communication systems. Due to the very limited receiving power from radio frequency (RF) signals, optimal splitting strategies play an essential role for many SWIPT systems. This paper investigates optimal power splitting from the outage perspective by formulating the power, information and joint outage performance using a Markov chain, and studying the boundary conditions for achieving an energy- neutral state. Our results show that: 1)the power splitting factor is the key factor of intrinsic trade-off between power and information outage; and 2) the proposed polynomial method can obtain the optimal power splitting factor which leads to the minimum joint outage probability.

In this paper, we apply the attention mechanism to autonomous driving for steering angle prediction. We propose the first model, applying the recently introduced sparse attention mechanism to visual domain, as well as the aggregated extension for this model. We show the improvement of the proposed method, comparing to no attention as well as to different types of attention.

Recognizing human actions in wearable camera videos, such as videos taken by GoPro or Google Glass, can benefit many multimedia applications. By mixing the complex and non-stop motion of the camera, motion features extracted from videos of the same action may show very large variation and inconsistency. It is very difficult to collect sufficient videos to cover all such variations and use them to train action classifiers with good generalization ability. In this paper, we develop a new approach to train action classifiers on a relatively smaller set of fixed-camera videos with different views, and then apply them to recognize actions in wearable-camera videos. In this approach, we temporally divide the input video into many shorter video segments and transform the motion features to stable ones in each video segment, in terms of a fixed view defined by an anchor frame in the segment. Finally, we use sparse coding to estimate the action likelihood in each segment, followed by combining the likelihoods from all the video segments for action recognition. We conduct experiments by training on a set of fixed-camera videos and testing on a set of wearable-camera videos, with very promising results.

Automatic tracking of large-scale crowded targets are of particular importance in many applications, such as crowded people/vehicle tracking in video surveillance, fiber tracking in materials science, and cell tracking in biomedical imaging. This problem becomes very challenging when the targets show similar appearance and the interslice/ inter-frame continuity is low due to sparse sampling, camera motion and target occlusion. The main challenge comes from the step of association which aims at matching the predictions and the observations of the multiple targets. In this paper we propose a new groupwise method to explore the target group information and employ the within-group correlations for association and tracking. In particular, the within-group association is modeled by a nonrigid 2D Thin-Plate transform and a sequence of group shrinking, group growing and group merging operations are then developed to refine the composition of each group. We apply the proposed method to track large-scale fibers from microscopy material images and compare its performance against several other multi-target tracking methods. We also apply the proposed method to track crowded people from videos with poor inter-frame continuity.

This paper addresses the problem of detecting coherent motions in crowd scenes and presents its two applications in crowd scene understanding: semantic region detection and recurrent activity mining. It processes input motion fields (e.g., optical flow fields) and produces a coherent motion field named thermal energy field. The thermal energy field is able to capture both motion correlation among particles and the motion trends of individual particles, which are helpful to discover coherency among them. We further introduce a two-step clustering process to construct stable semantic regions from the extracted time-varying coherent motions. These semantic regions can be used to recognize pre-defined activities in crowd scenes. Finally, we introduce a cluster-and-merge process, which automatically discovers recurrent activities in crowd scenes by clustering and merging the extracted coherent motions. Experiments on various videos demonstrate the effectiveness of our approach.