Each photo in an image burst can be considered a sam-ple of a complex 3D scene: the product of parallax, diffuse and specular materials, scene motion, and illuminant vari-ation. While decomposing all of these effects from a stack of misaligned images is a highly ill-conditioned task, the conventional align-and-merge burst pipeline takes the other extreme: blending them into a single image. In this work, we propose a versatile intermediate representation: a two-layer alpha-composited image plus flow model constructed with neural spline fields - networks trained to map input coordinates to spline control points. Our method is able to, during test-time optimization, jointly fuse a burst image capture into one high-resolution reconstruction and decom-pose it into transmission and obstruction layers. Then, by discarding the obstruction layer, we can perform a range of tasks including seeing through occlusions, reflection sup-pression, and shadow removal. Tested on complex in-the-wild captures we find that, with no post-processing steps or learned priors, our generalizable model is able to out-perform existing dedicated single-image and multi-view ob-struction removal approaches.

We present a novel perceptually-motivated interactive tool for using color contrast to enhance details represented in the lightness channel of images and video. Our method lets users adjust the perceived contrast of different details by manipulating local chromaticity while preserving the original lightness of individual pixels. Inspired by the use of similar chromaticity mappings in painting, our tool effectively offers contrast along a user-selected gradient of chromaticities as additional bandwidth for representing and enhancing different details in an image. We provide an interface for our tool that closely resembles the familiar design of tonal contrast curve controls that are available in most professional image editing software. We show that our tool is effective for enhancing the perceived contrast of details without altering lightness in an image and present many examples of effects that can be achieved with our method on both images and video.

Driven by Service-Oriented Computing, time-aware service recommendation aims to support personalized mashup development, adapting to the rapid shifts of users’ dynamic preferences. Recently, users’ social connections have shown significant benefits to time-aware service recommendation, and graph neural networks have demonstrated great success in learning the pattern of information flow among users. However, the current paradigm always presumes a given social network, which is not necessarily consistent with the similarities of service preferences among users and is expensive to collect for most service platforms. We propose a novel idea to learn the graph structure among historical mashups and make time-aware service recommendation for dynamic mashup creation collectively in a coupled framework. This idea raises two challenges, i.e., scalability and accuracy. To solve both challenges simultaneously, we introduce the Dynamic Relation Graph Learning (DRGL) framework for time-aware service recommendation. For scalability, our framework has a coarse-to-fine recalling strategy to learn the graph structure among the mashups, which enables the exploration of potential links among all historical mashups while maintaining a tractable amount of computation. For accuracy, we leverage recent advances in self-attention mechanisms to the mashup modeling and propose a transformer-based mashup encoder, which considers long-range dependencies in dense mashups for more accurate mashup representations. Extensive experiments show that the DRGL model consistently outperforms the state-of-the-art methods in terms of prediction accuracy for mashup creation.

Multi-view image generation attracts particular attention these days due to its promising 3D-related applications, e.g., image viewpoint editing. Most existing methods follow a paradigm where a 3D representation is first synthesized, and then rendered into 2D images to ensure photo-consistency across viewpoints. However, such explicit bias for photo-consistency sacrifices photo-realism, causing geometry artifacts and loss of fine-scale details when these methods are applied to edit real images. To address this issue, we propose ray conditioning, a geometry-free alternative that relaxes the photo-consistency constraint. Our method generates multi-view images by conditioning a 2D GAN on a light field prior. With explicit viewpoint control, state-of-the-art photo-realism and identity consistency, our method is particularly suited for the viewpoint editing task.

Next location recommendation services play a pivotal role in Location-Based Social Networks (LBSNs) due to their ability to provide personalized recommendations of attractive destinations, resulting in substantial benefits for both users and service providers. Recent research indicates that these services are influenced by both sequential and geographical factors. However, we argue that most of these services fail to fully exploit the latent multi-group knowledge of location semantics and user preferences, resulting in suboptimal performance. Therefore, we propose STMGCL, a novel spatial-temporal multi-group contrastive learning-based method to discover intrinsic multi-group information for improving next location recommendation services. Specifically, STMGCL designs Spatial Group Contrastive Learning (SGCL) to extract multiple group knowledge regarding location semantics. Additionally, it develops Temporal Group Contrastive Learning (TGCL) to explore multiple user preference group information through a self-attention based encoder. Finally, we leverage a multi-task learning strategy and a generalized Expectation Maximization (EM) algorithm to ensure that STMGCL is optimized end-to-end with guaranteed convergence. Extensive experiments conducted on four real-world datasets demonstrate the superior performance of STMGCL over baselines.

As an emerging concept, Knowledge as a Service (KaaS) aims to provide on-demand content-based (data, information, knowledge) delivery to meet the needs of users. With the prosperity of knowledge services, the prediction of the usage tendency of knowledge services has become an important and timely research topic. This study focuses on speculating the possible popularity of knowledge services in the next period of time, which can assist other downstream service tasks such as service recommendations. The interactions among knowledge services and their rich information (such as historical usage observation and text information) provide grounding for predicting the usage trend of services. However, recent spatial-temporal prediction based on graph neural networks usually depends heavily on the quality of manually created graphs, which may be expensive for knowledge services. To tackle such a limitation, this article proposes a novel Multi-modal Reciprocal SpatioTemporal (MRST) framework, which can jointly mine spatial dependencies and model time patterns for spatiotemporal coupling prediction. Two types of Edge Inference Networks (called EIN-o and EIN-t) are designed to sufficiently discover the spatial dependencies among knowledge services based on the data of usage observation sequences and service descriptions, respectively, and generate multi-modal directed weighted knowledge service graphs. Based on these graphs, MRST integrates GCN-based spatiotemporal prediction models as backbones to make predictions. Particularly, MRST features a unique reciprocal framework. On the one hand, EINs infer and generate multi-modal graphs to serve GCNs; on the other hand, GCNs utilize such spatial dependencies to make predictions and then introduce feedback to optimize EINs. In the meantime, to facilitate reproducible research, we collect a new knowledge service dataset fromWikipediacalled Wiki-EN dataset. Experiments on this real data set show that the proposed MRST framework significantly surpasses the baselines and can learn meaningful spatial dependencies outside the predefined graphic structure.

We present ReCapture, a system that leverages AR-based guidance to help users capture time-lapse data with hand-held mobile devices. ReCapture works by repeatedly guiding users back to the precise location of previously captured images so they can record time-lapse videos one frame at a time without leaving their camera in the scene. Building on previous work in computational re-photography, we combine three different guidance modes to enable parallel hand-held time-lapse capture in general settings. We demonstrate the versatility of our system on a wide variety of subjects and scenes captured over a year of development and regular use, and explore different visualizations of unstructured hand-held time-lapse data.

Web service recommendation remains a highly demanding yet challenging task in the field of services computing. In recent years, researchers have started to employ side information comprised in a heterogeneous Web service ecosystem to address the issues of data sparsity and cold start in Web service recommendation. Some recent works have exploited the deep learning techniques to learn user/Web service representations accumulating information from multiplex sources. However, we argue that they still struggle to utilize multi-source information in a discriminating, unified and flexible manner. To tackle this problem, this paper presents a novel multi-source information graph-based Web service recommendation framework (MGASR), which can automatically and efficiently extract multifaceted knowledge from the heterogeneous Web service ecosystem. Specifically, different node-type and edge-type dependent parameters are designed to model corresponding types of objects (nodes) and relations (edges) in the Web service ecosystem. We then leverage graph neural networks (GNNs) with an attention mechanism to construct a multi-source information neural network (MIN) layer, for mining diverse significant dependencies among nodes. By stacking multiple MIN layers, each node can be characterized by a highly contextualized representation due to capturing high-order multi-source information. As such, MGASR can generate representations with rich semantic information toward supporting Web service recommendation tasks. Extensive experiments conducted over three real-world Web service datasets demonstrate the superior performance of our proposed MGASR as compared to various baseline methods.

Service recommendation for composition creation is a widely applied technique, which expedites mashup development by reusing existing services. The core of service recommendations is to simultaneously understand user needs as well as the functions of available services. However, the descriptions provided by users and service providers may not always be accurate or up to date, which poses significant challenges to composition creating. To tackle this problem, in this paper we propose a deep learning-based service recommendation framework named coACN, short for Collaborative Attention Convolutional Network, which can effectively learn the bilateral information toward service recommendation. On the one hand, a domain-level attention module is constructed to refine user needs embeddings by drawing messages from related service domains. On the other hand, a graph convolutional network is established to excavate the service-composition graph and fuse structured information into service embeddings. For a service node in the graph, the information of its compositions as its first-order neighbor nodes is used to supplement the latest functions and features of the service; and the information of the services as its second-order neighbor nodes may bring collaborative relationships into the service. Extensive experiments on the real-world ProgrammableWeb dataset show the significant improvement of our proposed coACN framework over state-of-the-art methods.

The number of available online services increases sharply with the development of the Internet. These services typically belong to varying service domains. To address the data-sparse issue, cross-domain recommendation techniques are proposed to transfer the information in relevant service domains to improve the recommendation effects. In this paper, we presented a novel end-to-end cross-domain service recommendation learning framework, named EATN, short for End-to-end Attention Transfer Network, which is different from most existing cross-domain step-by-step learning frameworks. To realize this end-to-end framework, we design a workflow to achieve user preferences cross-domain matching procedure. We capture fine-grained and multi-faceted user preferences by using multiple Multi-Layer Perceptron layers. To reasonably integrate multi-faceted transfer preferences, we design a service-level attention module, which learns weight based on the relevance to services. Finally, it can improve the recommendation effect of cold-start users in the target domain. Extensive experiments on the real-world Amazon dataset show the significant improvement of our proposed EATN framework.