In this article, we study the multiple dictionaries-based compressed sensing, which has sparse signals recovered with several different dictionaries, and then takes the average as the final recovery. Statistically, the averaging method will reduce the errors with zero mean, but can hardly deal with the errors with nonzero means. In practice, the nonzero-mean errors are usually inevitable since sparse recovery will drop relatively small elements. To reduce such kind of errors, we analytically show that the error distributions of different dictionaries should be different to each other. Interestingly, the condition is hard to achieve, even though the dictionaries are learned independently. This is because the dictionaries, which are trained on the same dataset tend to converge to close points and, thus, yield similar recovery errors. In an attempt to address the issue, we propose a collaborative dictionary learning model to enhance the difference between dictionaries. The model is implemented with a block coordinate descent algorithm, with guaranteed convergence. Experiments show that our collaboratively learned dictionaries perform better than the independently learned ones.

Large-scale data classification poses great challenges to computation and storage. There are two major solutions to address the problem: the data dimension reduction and quantization. In the paper, we study the method of first reducing data dimension by random projection and then quantizing the projections to ternary and binary codes, which has been widely applied in practice. Often, the extreme quantization would degrade the accuracy of classification due to high quantization errors. Interestingly, however, we observe that the quantization could result in performance improvement, rather than degradation, if the data for quantization are preprocessed by sparse transforms. Also, the quantization gain could be obtained with the random projections of the data, if both the data and random projection matrices are sparse enough, such that the resulting projections remain sparse. The intriguing performance is verified and analyzed with extensive experiments.

For deep networks with complex nonlinearity, the structure analysis and design remain challenging. In the letter, we propose to understand and build deep networks as a cascade of compressed sensing. Each compressed sensing module consists of two layers, corresponding to the two data transforms involved in compressed sensing, namely the sparse transform over dictionaries and the measurement over random projection. The two transforms can be viewed as a process of feature generation and extraction, thus amenable to network construction. The proposed construction has two advantages. First, it can be analyzed with compressed sensing theory. Second, it enables networks to be learned layer by layer by back-propagating the target via compressed sensing, and empirically the layerwise learning method tends to outperform the conventional error back-propagation method in the presence of noise.

Learning to adapt to a series of different goals in visual navigation is challenging. In this work, we present a model-embedded actor-critic architecture for the multigoal visual navigation task. To enhance the task cooperation in multigoal learning, we introduce two new designs to the reinforcement learning scheme: inverse dynamics model (InvDM) and multigoal colearning (MgCl). Specifically, InvDM is proposed to capture the navigation-relevant association between state and goal and provide additional training signals to relieve the sparse reward issue. MgCl aims at improving the sample efficiency and supports the agent to learn from unintentional positive experiences. Besides, to further improve the scene generalization capability of the agent, we present an enhanced navigation model that consists of two self-supervised auxiliary task modules. The first module, which is named path closed-loop detection, helps to understand whether the state has been experienced. The second one, namely the state-target matching module, tries to figure out the difference between state and goal. Extensive results on the interactive platform AI2-THOR demonstrate that the agent trained with the proposed method converges faster than state-of-the-art methods while owning good generalization capability. The video demonstration is available at https://vsislab.github.io/mgvn.

Imitation learning enables robots to learn a task by simply watching the demonstration of the task. Current imitation learning methods usually require the learner and demonstrator to occur in the same context. This limits their scalability to practical applications. In this paper, we propose a more general imitation learning method which allows the learner and the demonstrator to come from different contexts, such as different viewpoints, backgrounds, and object positions and appearances. Specifically, we design a robotic system consisting of three models: context translation model, depth prediction model and multi-modal inverse dynamics model. First, the context translation model translates the demonstration to the context of learner from a different context. Then combining the color observation and depth observation as inputs, the inverse model maps the multi-modal observations into actions to reproduce the demonstration, where the depth observation is provided by a depth prediction model. By performing the block stacking tasks both in simulation and real world, we prove the cross-context learning advantage of the proposed robotic system over other systems.

Visual object tracking is a fundamental and time-critical vision task. However, most trackers such as SiamFC and CFNet missed the object movement and simply defined the searching region centered at the location of the target in the previous frame. So they tend to fail in the cases with severe occlusion or a large displacement of the target. In this paper, we consider the object tracking as a dual-task problem of guessing and matching. A guess module is to estimate the motion trend of the target by reinforcement learning based on the observations on appearance changes and motion history. Rather than using the previous location of the target, we may have a more accurate center to locate the searching region. Benefited from such improved searching region, the match module becomes less prone to the object drift problem, and can easily identify the target from the potential distractors in the background. Extensive experimental results on benchmark datasets such as RGBT, OTB-2013, OTB-50 and OTB-100, show that the proposed method achieves leading performance compared to state-of-the-art trackers. Moreover, the proposed tracker could maintain real-time speed, giving itself the potential in practical applications.

In this paper, we propose a novel deep neural network based attention model to learn the representative local regions from a video sequence for person re-identification. Specifically, we propose a multi-scale spatial-temporal attention (MSTA) model to measure the regions of each frame in different scales from the perspective of whole video sequence. Compared to traditional temporal attention models, MSTA focuses on exploiting the importance of local regions of each frame to the whole video representation in both spatial and temporal domains. A new training strategy is designed for the proposed model by incorporating the image-to-image mode with the videoto- video mode. Extensive experiments on benchmark datasets demonstrate the superiority of the proposed model over state-ofthe- art methods.

Image emotion recognition aims to automatically categorize the emotion conveyed by an image. The potential of deep representation has been demonstrated in recent research on image emotion recognition. To better understand how CNNs work in emotion recognition, we investigate the deep features by visualizing them in this work. This study shows that the deep models mainly rely on the image content but miss the image style information such as color, texture, and shapes that are low-level visual features but are vital for evoking emotions. To form a more discriminative representation for emotion recognition, we propose a novel CNN model that learns and integrates the content information from the high layers of the deep network with the style information from the lower layers. The uncertainty of image emotion labels is also investigated in this paper. Rather than using the emotion labels for training directly, as in previous work, a new loss function is designed by including the emotion labeling quality to optimize the proposed inference model. Extensive experiments on benchmark datasets are conducted to demonstrate the superiority of the proposed representation.

Task scheduling in cloud computing is an NP-complete problem. It concerns how to properly arrange task execution process using a set of necessary cloud resources. Existing works assume that tasks can be interrupted without any overhead, based on which task schedules can be optimized to achieve some objectives (e.g., maximize resource utilization). We observe that interrupting tasks, as well as subsequent task recovery process, will inevitably impose overheads in terms of consuming additional CPU time on corresponding physical machines. In addition, not all tasks can be interrupted. Those observations motivate us to consider a more general scenario where a job consists of both interruptible and non-interruptible tasks with specific deadline and resource requirements. Accordingly, we design algorithms to minimize task interruption overhead while ensuring task completion deadline. Specifically, we first formulate an Integer Linear Program (ILP) for offline optimization. A heuristic algorithm is then proposed for online task scheduling, and is compared with the optimal ILP solution. Numerical results confirm the correctness of our ILP and show the efficiency of the proposed heuristic.

Learning to adapt to a series of different goals in visual navigation is challenging. In this work, we present a model-embedded actor-critic architecture for the multi-goal visual navigation task. To enhance the task cooperation in multi-goal learning, we introduce two new designs to the reinforcement learning scheme: inverse dynamics model (InvDM) and multi-goal co-learning (MgCl). Specifically, InvDM is proposed to capture the navigation-relevant association between state and goal, and provide additional training signals to relieve the sparse reward issue. MgCl aims at improving the sample efficiency and supports the agent to learn from unintentional positive experiences. Extensive results on the interactive platform AI2-THOR demonstrate that the proposed method converges faster than state-of-the-art methods while producing more direct routes to navigate to the goal. The video demonstration is available at: https://youtube.com/channel/UCtpTMOsctt3yPzXqe_JMD3w/videos.

Resource scheduling is one of the main problems for multi-satellite Tracking, Telemetry and Command (TT&C) networks. Traditional multi-resource joint scheduling algorithms are with long solution time, low efficiency, high computational cost, and simple description on the system. Deep Neural Network (DNN) provides a possible new way to solve those problems, but it is difficult to handle correlations among the input data. This motivates our work to solve the strong correlation problem based on the accumulated historical data, and thus enables DNN for TT&C resource scheduling. By discretizing the data, multiple constraints and related attributes are transformed into different flags, and some binary bits of the data are used to reflect the constraint relationship. Then, we can use DNN model and construct an intelligent TT&C resource scheduling system to handle multiple constraints and data attributes (such as priorities among tasks and others). This improves the efficiency of TT&C resources utilization and automation. Effectiveness of the proposed model is verified by simulations.

Learning actions from human demonstration is an emerging trend for designing intelligent robotic systems, which can be referred as video to command. The performance of such approach highly relies on the quality of video captioning. However, the general video captioning methods focus more on the understanding of the full frame, lacking of consideration on the specific object of interests in robotic manipulations. We propose a novel deep model to learn actions from human demonstration video for robotic manipulation. It consists of two deep networks, grasp detection network (GNet) and video captioning network (CNet). GNet performs two functions: providing grasp solutions and extracting the local features for the object of interests in robotic manipulation. CNet outputs the captioning results by fusing the features of both full frames and local objects. Experimental results on UR5 robotic arm show that our method could produce more accurate command from video demonstration than state-of-the-art work, thereby leading to more robust grasping performance.

Siamese networks are prevalent in person re-identification (re-id) tasks to address the similarity and dissimilarity among video frames. It mainly focuses on the inter-video variation between spatio-temporal features extracted from different videos, while the variation between features of the same video has been rarely discussed. In this paper, we introduce the concept of “mean-body” and define an intra-video loss to address the variation between spatio-temporal features of the same video. A novel loss is presented to boost the training of the re-id networks by combining the proposed intra-video loss and the Siamese loss. Specifically, the intra-video loss uses the unique mean-body of each camera viewpoint to make the video sequence more clustered, while the Siamese loss is to make the wrong matching videos more separated. To train the whole network, we update the network and the mean-body in an iterative manner. As a result, the proposed loss is expected to improve the generalization capability of the re-id networks on the testing set. Extensive results demonstrate that the presented approach outperforms the state-of-the-art algorithms on the publicly available data sets, such as PRID2011, iLIDS-VID, and MARS, in terms of re-id accuracy.

Video-based person re-identification (re-id) matches two tracks of persons from different cameras. Features are extracted from the images of a sequence and then aggregated as a track feature. Compared to existing works that aggregate frame features by simply averaging them or using temporal models such as recurrent neural networks, we propose an intelligent feature aggregate method based on reinforcement learning. Specifically, we train an agent to determine which frames in the sequence should be abandoned in the aggregation, which can be treated as a decision making process. By this way, the proposed method avoids introducing noisy information of the sequence and retains these valuable frames when generating a track feature. On benchmark data sets, experimental results show that our method can boost the re-id accuracy obviously based on the state-of-the-art models.

This paper studies a class of binary matrices with correlations between distinct columnsequal to zero or one, which has reported comparable performance with random matrices inrecent studies of compressed sensing. For such matrix, we analyze its structure propertyand provide an improved performance estimation.

Recently developed dual domain image denoising (DDID) algorithm and its variants, such as dual domain filter (DDF), achieve remarkable results by combining bilateral filter with frequency-based method. However, this kind of algorithms require large patches to guarantee the denoising performance and most of them produce ringing artifacts due to the Gibbs phenomenon induced by high-contrast details. To address these issues, we propose a Foveated Nonlocal Dual Denoising (FNDD) algorithm by unifying foveated nonlocal means and frequency-based methods. In this way, the ability to preserve the high-contrast details is noticeably improved by exploiting foveated self-similarity (patch similarity) instead of pixel similarity, thus leading to void of artifacts. Moreover, we propose an entropy-based back projection step for compensating the detail loss to further improve the performance. Experimental results validate that FNDD significantly outperforms DDID in terms of both quantitative metrics and subjective visual quality under much smaller patches, and even achieves comparable results against state-of-the-art competitors.

This letter studies the sensing performance of random Bernoulli matrices with column size$n$much larger than row size$m$. It is observed that as the compression ratio$n/m$increases, this kind of matrices tends to present a performance floor regarding the guaranteed signal sparsity. The performance floor is effectively estimated with the formula${1 \over 2}(\sqrt {\pi m/2} + 1)$. To the best of our knowledge, it is the first time in compressed sensing, a theoretical estimation is successfully proposed to reflect the real performance.

Recently sparse representation has been successfully applied to single object tracking by observing the reconstruction error of candidate object with sparse representation. In practice, sparse representation also shows competitive performance on multi-class classification, and thus is potential for multi-object tracking. In this paper we explore this technique for on-line multi-object tracking through a simple tracking-by-detection scheme, with background subtraction for object detection and sparse representation for object recognition. Final experiments demonstrate that the proposed approach only combining color histogram and 2-dimensional coordinates as features, achieves favorable performance over state-of-the-art work in persistent identity tracking.

We present the sparse binary matrix defined by low-density parity-check (LDPC) codes as measurement matrix in compressed sensing. This kind of matrix owns much stronger orthogonality than current other main measurement matrices. In practice, for a matrix given size, the optimal column degree for high orthogonality can also be determined by progressive edge-growth (PEG) algorithm.