The attributed scattering center (ASC) can provide a concise physical description of radar targets, which has broad applications for physical-induced radar recognition. However, the ASC extraction is of great difficulty due to its high-dimensional parametric modeling. In this letter, a modified Alternative Direction Method of Multipliers (ADMM) unfolded-net is proposed for ASC decomposition of synthetic aperture radar (SAR) targets. By transforming the ASC decomposition challenge into a sparse reconstruction problem, the modified ADMM-net can fast attain ASC parameters as well as reconstruct a complete signal from sparse observations. Instead of tremendous iterations with fixed hyper-parameters as traditional methods, the modified ADMM-net unrolls several iterations into a few layers with learned hyper-parameters. Therefore, the proposed algorithm can realize fast and precise ASC decomposition. Experimental results confirm its effectiveness and superiority.

Abstract Image quality enhancement aims to recover rich details from degraded images, which is applied into many fields, such as medical imaging, filming production and autonomous driving. Deep convolutional neural networks (CNNs) have enabled rapid development of image quality enhancement. However, most existing CNN‐based methods lack versatility when targeting different subtasks in terms of the design of networks. Besides, they often fail to balance precise spatial representations and necessary contextual information. To deal with these problems, this paper proposes a novel unified framework for low‐light image enhancement, image denoising and image super‐resolution. The core of this architecture is a residual hybrid attention block (RHAB), which consists of several dynamic down‐sampling modules (DDM) and hybrid attention up‐sampling modules (HAUM). Specifically, multi‐scale feature maps are fully interacted with each other with the help of nested subnetworks so that both high‐resolution spatial details and high‐level contextual information can be combined to improve the representation ability of the network. Further, a hybrid attention network (HAN) is proposed and evaluations on three separate subtasks demonstrate its good performance. Extensive experiments on the authors' synthetic dataset, a more complex version, show that the authors' method achieve better quantitative and visual results compared to other state‐of‐the‐art methods.

Multi-view correlation learning has attracted great attention with the proliferation of heterogeneous data. Typical methods, such as Canonical Correlation Analysis (CCA) and its variants, usually maximize one-to-one corresponding correlation of inter-view data, while most of them neglect discriminative multi-label information and local structure of each view data. In this paper, we propose multi-label Semantics and Locality Preserving Correlation Projections method (SLPCP), which seeks for a semantic common subspace by jointly learning view-specific linear projections from intra-view and interview perspectives simultaneously. SLPCP can be easily optimized with generalized eigen value decomposition via concatenating the projections of multi-views. Applied to retrieval tasks of image and text data in experiments, SLPCP out performs state-of-the-art methods on a widely used dataset NUS-WIDE. The extensive experiments also validate that it is effective to preserve the multi-label semantics and locality of multi-view data.

With the explosive growth of web data, effective and efficient technologies are in urgent need for retrieving semantically relevant contents of heterogeneous modalities. Previous studies devote efforts to modeling simple cross-modal statistical dependencies, and globally projecting the heterogeneous modalities into a measurable subspace. However, global projections cannot appropriately adapt to diverse contents, and the naturally existing multilevel semantic relation in web data is ignored. We study the problem of semantic coherent retrieval, where documents from different modalities should be ranked by the semantic relevance to the query. Accordingly, we propose TINA, a correlation learning method by adaptive hierarchical semantic aggregation. First, by joint modeling of content and ontology similarities, we build a semantic hierarchy to measure multilevel semantic relevance. Second, with a set of local linear projections and probabilistic membership functions, we propose two paradigms for local expert aggregation, i.e., local projection aggregation and local distance aggregation. To learn the cross-modal projections, we optimize the structure risk objective function that involves semantic coherence measurement, local projection consistency, and the complexity penalty of local projections. Compared to existing approaches, a better bias-variance tradeoff is achieved by TINA in real-world cross-modal correlation learning tasks. Extensive experiments on widely used NUS-WIDE and ICML-Challenge for image-text retrieval demonstrate that TINA better adapts to the multilevel semantic relation and content divergence, and, thus, outperforms state of the art with better semantic coherence.

Presents corrections to the paper, "Cross-modal correlation learning by adaptive hierarchical semantic aggregation," (Hua, Y., et al) IEEE Trans. Multimedia, vol. 18, no. 6, pp. 1201-1216, Jun. 2016.

With the explosive growth of online multi-media data, methodologies of retrieving documents from heterogeneous modalities are indispensable to facilitate information acquisition in real applications. Most of existing research efforts are focused on building correlation learning models on hand-crafted features for visual and textual modalities. However, they lack the ability to capture the meaningful patterns from complicated visual modality, and are not able to identify the true correlation between modalities during feature learning process. In this paper, we propose a novel cross-modal correlation learning method with well-designed deep convolutional network to learn representations from visual modality. A cross-modal correlation layer with a linear projection is added on the top of the network by maximizing semantic consistency with large margin principle. All the parameters are jointly optimized with stochastic gradient descent. With the deep architecture, our model is able to disentangle the complex visual information, and learn the semantically consistent patterns in a layer-by-layer fashion. Experimental results on widely used NUS-WIDE dataset show that our model outperforms state-of-the-art correlation learning methods built on 6 hand-crafted visual features for image-text retrieval.

With the explosive growth of web data, effective and efficient technologies are in urgent needs for retrieving semantically relevant contents of heterogeneous modalities. Previous studies construct global transformations to project the heterogeneous data into a measurable subspace. However, global projections cannot appropriately adapt to diverse contents, and the naturally existing multi-level semantic relation in web data is ignored. We study the problem of semantic coherent retrieval, where documents from different modalities should be ranked by the semantic relevance to the queries. Accordingly, we propose TINA, a correlation learning method by Adaptive Hierarchical Semantic Aggregation. First, by joint modeling of content and ontology similarities, we build a semantic hierarchy to measure multi-level semantic relevance. Second, with a set of local linear projections aggregated by gating functions, we optimize the structure risk objective function that involves semantic coherence measurement, local projection consistency and the complexity penalty of local projections. Therefore, semantic coherence and a better bias-variance trade-off can be achieved by TINA. Extensive experiments on widely used NUS-WIDE and ICML-Challenge datasets demonstrate that TINA outperforms state-of-the-art, and achieves better adaptation to the multi-level semantic relation and content divergence.

The Mahalanobis metric learning is an effective tool for constructing semantic consistent distance among data in single modal data analysis. However, distance metric learning is a more challenging issue for cross modal data, where less attention has been paid in previous studies. In this paper, we propose Cross mOdal Large mArgin metric leaRning (COLAR) with multi-level semantic relevance. With large margin principle, we model different levels of the semantic relations across modalities, e.g., the one-to-one correspondence and intra-class relation, while traditional correlation learning approaches (such as CCA and its variants) can only handle the one-to-one correspondence or treat them indiscriminatively. As a result, the distances of multi-level relevance among cross modal data are optimized based on a regularized learning framework. Promising performance is achieved on cross modal retrieval, i.e., image-to-text retrieval and text-to-image retrieval.

In this paper, we propose a new depth sampling method based on a learned Mahalanobis distance (DSMD), instead of traditional Euclidean distance (DSED), for depth extraction from a single image. This metric is learned with the goal that images with similar and dissimilar 3D structures to the query image are separated by a large margin. Thus the learned distance metric can better measure the similarity of 3D structure between images than Euclidean distance. We also propose a simple method based on Gaussian weighting function (D-FGW) for depth fusion of the sampled images. Experiments show that our DSMD method produces more accurate depth estimation of the query image than the DSED method does. Our DFGW method is fast and produces decent results. With the depth fusion method based on energy function minimization [1], our DSMD method achieves state-of-the-art results on Make 3D dataset.

The goal of depth inference from a single image is to assign a depth to each pixel in the image according to the image content. In this paper, we propose a deep learning model for this task. This model consists of a convolutional neural network (CNN) with a linear regressor being as the last layer. The network is trained with raw RGB image patches cropped by a large window centered at each pixel of an image to extract feature representations. Then the depth map of a test image can be efficiently obtained by forward-passing the image through the trained model plus a simple up-sampling. Contrary to most previous methods based on graphical model and depth sampling, our method alleviates the needs for engineered features and for assumptions about semantic information of the scene. We achieve state-of-the-art results on Make 3D dataset, while keeping low computational time at the test time.

Attention models for free-viewing of images are commonly developed in a bottom-up (BU) manner. However, in this paper, we propose to include memory-oriented top-down spatial attention cues into the model. The proposed generative saliency model probabilistically combines a BU module and a top-down (TD) module and can be applied to both static and dynamic scenes. For static scenes, the experience of attention distribution to similar scenes in long-term memory is mimicked by linearly mapping a global feature of the scene to the long-term top-down (LTD) saliency. And for dynamic scenes, we add on the influence of short-term memory to form a HMM-like chain to guide the attention distribution. Our improved BU module utilizes low-level feature contrast, spatial distribution and location information to highlight saliency regions. It is tested on 1000 benchmark images and outperforms state-of-the-art BU methods. The complete model is examined on two video datasets, one with manually labeled saliency regions, and another with recorded eye-movement fixations. Experimental results show that our model achieves significant improvement in predicting human visual attention compared with existing saliency models.

In this paper we propose a depth recovery approach for monocular videos with or without camera motion. By combining geometric information and moving object extraction, not only the depth of background but also the depth of foreground can be recovered. Furthermore, for cases involving complex camera motion such as fast moving, translating, vertical movement, we propose a novel global motion estimation (GME) method including effective outlier rejection to extract moving objects, and experiments demonstrate that the proposed GME method outperforms most of the state-of-the-art methods. The depth recovery approach we propose is tested on four video sequences with different camera movements. Experimental results show that our approach produces more accurate depth of both background and foreground than existing depth recovery methods.