A phrase contains an object-orienting noun and some attribution-associating words. Therefore, focusing on phrases could better generate images with the objects and their tightly relevant characteristics. We propose a Phrase-boost Gener-ative Adversarial Network (PhraseGAN) with threefold im-provement for scene level text-to-image generation. First, we propose a Transformer-based encoder to encode the in-put words and sentences and encode related words and their targeting nouns into phrases by text correlation analysis. Sec-ond, we utilize Graph Convolution Networks to measure fine-grained text-image similarity, which could gain constraints on relative positions between different objects. Finally, we de-sign a phrase-region discriminator to discriminate the qual-ity of the generated objects and the consistency between the phrases and their corresponding objects. Experimental results on the Microsoft COCO dataset demonstrate that PhraseGAN can generate better images from texts than state-of-the-art methods.

The problem of color and texture distortion remains unsolved in the photorealistic style transfer task. It is mainly caused by the interference between color and texture during transferring. To address this problem, we propose a end-to-end network via adaptive filtering and channel separation. Given a pair of content image and reference image, we firstly decompose them into two structure layers through adaptive weighted least squares filter (AWLSF), which could better perceive the color structure and illumination. Then, we carry out RGB transfer in a channel separation way on the two generated structure layers. To deal with texture in a relatively independent manner, we use a module and a subtraction operation to get more complete and clear content features. Finally, we merge the color structure and texture detail into the ultimate result. We conduct solid quantitative experiments on four metrics NIQE, AG, SSIM, and PSNR, and make a user study. The experimental results demonstrate that our method is able to produce better results than previous state-of-the-art methods, and validate the effectiveness and superiority of our method.

A narrative collage is an interesting image editing method for summarizing the main theme or storyline behind an image collection. We present a novel method to generate narrative images with plausible semantic scene structures. To achieve this goal, we introduce a layer graph and a scene graph to represent the relative depth order and semantic relationship between image objects, respectively. We first cluster the input image collection to select representative images, and then we extract a group of semantic salient objects from each representative image. Both layer graphs and scene graphs are constructed and combined according to our specific rules for reorganizing the extracted objects in every image. We design an energy model to appropriately locate every object on the final canvas. The experimental results show that our method can produce competitive narrative collage results and that it performs well on a wide range of image collections.

In recent years, quality of experience (QoE) has been investigated and proved to have both influential factors on user's visual quality and perceptual quality, while the perceptual quality means user's requirement on personalized content should be acquired in optimized quality. That's to say, those segments holding user interested content such as highlights need to be allocated more network resource in a resource-limited streaming scenario. However, all the existing HTTP-based adaptive methods only focus the content-agnostic bitrate adaptation according to limited network resources or energy resource, since they ignored user perceived semantics on some important segments, which suffered less quality on the important segments than on those ordinary ones, so as to hurt the overall QoE. In this paper, we have proposed a new semantic-aware adaptation scheme for MPEG-DASH services, which decides how to preserve bandwidth and buffering time depending on content descriptors for the perceived important content to users. Further, a semantic-aware probe and adaptation (SMA-PANDA) algorithm has been implemented in a DASH client to compare with conventional bitrate adaptions. Preliminary results show that SMA-PANDA achieves better QoE and flexibility on streaming user's interested content on MPEG-DASH platform, and it also aggressively helps user interested content compete more resource to deliver high quality presentation.

The paper presents novel content-aware adaptation schemes for QoE optimized HTTP-based adaptive video streaming services. In recent years, content type has been investigated and proofed to have a deep influence on user's acceptable quality of experience (QoE). However, most of the existing HTTP-based adaptive video strategies only conduct bitrate switching according to network resources without a consideration of video content types. This may cause a non-optimal network resource allocation which may not achieve the optimized QoE for delivered video streaming services. In this paper, we proposed two content-aware adaptation schemes, named as Content-aware Probe and Adapt (C-PANDA) and Content-aware Dynamic Programming (C-DP), which are able to decide the video bitrate for the next video segment, based on not only the available bandwidth, the existing buffer capacity, but also the video content type. We implemented both content-aware adaptation algorithms in player prototypes based on libdash 3.0 library. Preliminary results show that content-aware adaptation schemes achieve better QoE when compared with conventional PANDA scheme. The proposed schemes are able to optimize QoE according to contents. They can achieve higher than acceptable QoE for most attractive contents such as those with high motion intensity to improve user experience for the most interesting scenes; select same video representations for segments belonging to a scene to avoid quality oscillation; and reserve buffer resource during low motion scenes to avoid stalling for the following high motion scenes. The work will be helpful in designing QoE-aware DASH schemes in the future.