We present PanoGPS, a new approach for generating indoor panoramas, capable of following a room layout sketch and description text to generate panoramas with geometrically-plausible room layouts and semantically-consistent contents. Overall, our idea is to inform the generative model collectively with the two inputs, to inspire it to implicitly generate and refine a unified latent code for panorama generation. Specifically, we first propose using the semantic layout map to encode the room geometry to condition the generative model with plausible geometric information. Second, we establish a pipeline to integrate features from the two inputs and generate a single unified latent code that describes both the room layout and room contents of the target panorama. Third, we progressively refine the latent code to produce a coarse panorama to enforce seamless left-right border connections, while iteratively upsizing the coarse panorama with details. Our method outperforms existing approaches by generating high-quality panoramas with geometrically plausible structures and semantically meaningful content. Code is available at: https://github.com/wumengyangok/PanoGPS

Oriented object detection in aerial images is a crucial link in earth observation. As a special parameter in oriented object representation, angle is the key to achieving high-precision detection. However, the widely-used regression-based methods suffer from boundary discontinuity problem due to the periodicity of angle. To address this issue, we proposed a novel angle prediction method called Fixed Step Trigonometric Coder (FSTC). Exploiting the innate periodicity of trigonometric functions, FSTC can encode angles cyclically in a succinct, continuous, and uniform manner. Based on FSTC, we designed a single-shot oriented object detector, namely, Trigonometric-coded Refined Detector (TRDet), for high-precision object detection in real-time. TRDet consists of two modules: the Angle Optimization Module (AOM) and the Object Detection Module (ODM). AOM employs FSTC to generate high-quality rotated anchors. In ODM, a Dynamically Weighted Loss (DWL) was proposed to make the model focus on hard samples with higher angle deviation. Extensive experiments on DOTA and HRSC2016 show that both FSTC and TRDet can achieve competitive performance compared with peer works.

This article presents a new approach based on deep learning to automatically extract colormaps from visualizations. After summarizing colors in an input visualization image as a Lab color histogram, we pass the histogram to a pre-trained deep neural network, which learns to predict the colormap that produces the visualization. To train the network, we create a new dataset of ∼ 64K visualizations that cover a wide variety of data distributions, chart types, and colormaps. The network adopts an atrous spatial pyramid pooling module to capture color features at multiple scales in the input color histograms. We then classify the predicted colormap as discrete or continuous, and refine the predicted colormap based on its color histogram. Quantitative comparisons to existing methods show the superior performance of our approach on both synthetic and real-world visualizations. We further demonstrate the utility of our method with two use cases, i.e., color transfer and color remapping.

This paper presents a new approach to recognizing vanishing-point-constrained building planes from a single image of street view. We first design a novel convolutional neural network (CNN) architecture that generates geometric segmentation of per-pixel orientations from a single street-view image. The network combines two-stream features of general visual cues and surface normals in gated convolution layers, and employs a deeply supervised loss that encapsulates multi-scale convolutional features. Our experiments on a new benchmark with fine-grained plane segmentations of real-world street views show that our network outperforms state-of-the-arts methods of both semantic and geometric segmentation. The pixel-wise segmentation exhibits coarse boundaries and discontinuities. We then propose to rectify the pixel-wise segmentation into perspectively-projected quads based on spatial proximity between the segmentation masks and exterior line segments detected through an image processing. We demonstrate how the results can be utilized to perspectively overlay images and icons on building planes in input photos, and provide visual cues for various applications.

This paper presents a new approach to recognize elements in floor plan layouts. Besides walls and rooms, we aim to recognize diverse floor plan elements, such as doors, windows and different types of rooms, in the floor layouts. To this end, we model a hierarchy of floor plan elements and design a deep multi-task neural network with two tasks: one to learn to predict room-boundary elements, and the other to predict rooms with types. More importantly, we formulate the room-boundary-guided attention mechanism in our spatial contextual module to carefully take room-boundary features into account to enhance the room-type predictions. Furthermore, we design a cross-and-within-task weighted loss to balance the multi-label tasks and prepare two new datasets for floor plan recognition. Experimental results demonstrate the superiority and effectiveness of our network over the state-of-the-art methods.

Vision-based object tracking is a challenging problem. In the tracking process, the object is usually first recognized in a given image. Then a bounding box is used to describe the position of the target object. Normally, a vector [x, y, width, height] is adopted to represent the bounding box. Under this viewpoint, the tracking problem can be treated as a regression problem if we handle the image sequence frame by frame. Due to the recent advancement in machine learning, many researchers apply neural networks to solve the visual tracking problem. This greatly improves the accuracy of bounding box prediction. Actually, the neural network based approaches are more suitable for end-to-end systems. In this paper, we propose to train and use a single neural network to tackle the tracking task. With the cropped candidate image patch as the input to the network, the output is the bounding box that indicates the target position. In our network, we first have a mask map to identify the target. It is a binary image and is divided into two classes. The positive class denotes the foreground while the negative class denotes the background. The mask map is then used for the estimation of the bounding box vector. The task now becomes an image mapping problem. We have achieved a good balance between accuracy and computational efficiency. Our tracker can reach an average speed of 178 frames per second(fps) and a maximum of 334 fps in the OTB benchmark.

Liquid cooling shows great potential in resolving the huge thermal obstacle in 3D ICs. However, it brings new challenges including large thermal gradient and high pumping requirement. In this paper, liquid cooling networks with flexible topology are investigated to achieve more desirable trade-offs between energy efficiency and thermal profile. Specifically, a fast thermal model for the cooling network is proposed and analyzed, followed by our optimization methodologies to construct cooling networks targeting at pumping power saving and thermal gradient reduction, respectively. Experimental results show that, under the same constraints, the cooling network can save as much as 84.03% pumping power or reduce 37.65% thermal gradient compared to straight microchannels.