Editing images via instruction provides a natural way to generate interactive content, but it is a big challenge due to the higher requirement of scene understanding and generation. Prior work utilizes a chain of large language models, object segmentation models, and editing models for this task. However, the understanding models provide only single modality ability, restricting the editing quality. We aim to bridge understanding and generation via a new multi-modality model that provides the intelligent abilities to instruction-based image editing models for more complex cases. To achieve this goal, we individually separate the instruction editing task with the multi-modality chain of thought prompts, i.e., Chain-of-Thought (CoT) planning, editing region reasoning, and editing. For Chain-ofThought planning, the large language model could reason the appropriate sub-prompts considering the instruction provided and the ability of the editing network. For editing region reasoning, we train an instruction-based editing region generation network with a multi-modal large language model. Finally, a hint-guided instruction-based editing network is proposed for editing image generations based on the sizeable text-to-image diffusion model to accept the hints for generation. Extensive experiments demonstrate our method has competitive editing abilities on complex real-world images. The source code will be publicly available.

Solving the task of inverse imaging problems can restore unknown clean images from input measurements that have incomplete information. Utilizing powerful generative models, such as denoising diffusion models, could better tackle the ill-posed issues of inverse problems with the distribution prior of the unknown clean images. We propose a learnable state-estimator-based diffusion model to incorporate the measurements into the reconstruction process. Our method makes efficient use of the pre-trained diffusion models with computational feasibility compared to the conditional diffusion models, which need to be trained from scratch. In addition, our pipeline does not require explicit knowledge of the image degradation operator or make the assumption of its form, unlike many other works that use the pre-trained diffusion models at the test time. The experiments on three typical inverse imaging problems (both linear and non-linear), inpainting, deblurring, and JPEG compression restoration, have comparable results with the state-of-the-art methods.

Since the advent of social media platforms, image selection based on social preference is a challenging task that all users inherently undertake before sharing images with the public. In our user study for this problem, human choices of images based on perceived social preference are largely inaccurate (58.7% accuracy). The challenge of this task, also known as image popularity assessment, lies in its subjective nature caused by visual and non-visual factors. Especially in the social media setting, social feedback on a particular image largely differs depending on who uploads it. Therefore social preference model should be able to account for this user-specific image aspect of the task. To address this issue, we present a retrieval-augmented approach that leverages both image features and user-specific statistics for neural image popularity assessment. User-specific statistics are derived by retrieving past images with their statistics from a memory bank. By combining these statistics with image features, our approach achieves 79.5% accuracy, which significantly outperforms human and baseline models on the pairwise ranking of images from the Instagram Influencer Dataset. Our source code will be publicly available.

With the widespread adoption of the fifth-generation (5G) cellular network and Mobile Edge Computing (MEC), numerous Internet of Things (IoT) applications are emerging in many critical areas. IoT applications are typically running on mobile devices to provide real-time interaction with users by connecting with smart IoT devices and remote cloud services. In order to ensure the performance of IoT applications, Quality of Service (QoS) is commonly used as a key metric for the selection and adaptation of high-quality services at runtime. Collaborative QoS prediction methods have been proposed in the literature to predict personalized QoS values, enabling QoS-based selection and adaptation. However, privacy issues in collaborative QoS prediction discourage users from collaborating by sharing data in practice. Furthermore, there are untrusted users in the unreliable MEC environment, which makes the prediction encounter serious reliability issues. As a result, privacy and reliability issues have become key challenges to make QoS prediction approaches feasible. In this paper, we proposed a credible and privacypreserving QoS prediction approach by leveraging federated learning techniques and developing reputation mechanisms to address this critical challenge. We evaluate the method on a large- scale real QoS dataset and the experimental results demonstrate the effectiveness and efficiency of the method.