Conventional WebRTC media flows rely on Transport-Wide Congestion Control (TWCC) to handle delay- and loss-based signals for video bit rate adaptation. Emerging Low Latency Low Loss, and Scalable Throughput (L4S) technology enables applications to exploit Explicit Congestion Notification (ECN) signals and scalable congestion control to achieve sub-millisecond queuing latency. Specifically, for WebRTC media flows, this requires client-side support for IETF RFC 8888 ECN feedback and L4S-compliant congestion control at the server.

We present a demonstration of Network Based Media Processing (NBMP) standard-compliant cloud service for reconstructing and Augmented Reality (AR) display of fully textured 3D human model using 2-5 images captured with a smartphone.

This paper proposes a 3D map augmented photo gallery mobile application that allows user to virtually transit from 2D image space to the 3D map space, to expand the field of view to surrounding environments that are not visible in the original image, and to change viewing angles among different global registered images during the image browsing. The processing of images consists of two main steps: in the first step, the client application uploads an image to the GeoImage Engine which extracts the geo-metadata and returns them back to the client; in the second step, the client application requests augmented content from server, and then renders 3D view of images on the screen of mobile devices.

Object registration has been widely discussed with the development of various range sensing technologies. In most work, however, the point clouds of reference and target are generated by the same technology, such as a Kinect range camera, LiDAR sensor, or Structure from Motion technique. Cases in which reference and target point clouds are generated by different technologies are rarely discussed. Due to the significant differences across various point cloud data in terms of point cloud density, sensing noise, scale, occlusion etc., object registration between such different point clouds becomes extremely difficult. In this study, we address for the first time an even more challenging case in which the differently-sourced point clouds are acquired from a real street view. One is generated on the basis of an image sequence through the SfM process, and the other is produced directly by the LiDAR system. We propose a two-stage matching and registration algorithm to achieve object registration between these two different point clouds. The experiments are based on real building object point cloud data and demonstrate the effectiveness and efficiency of the proposed solution. The newly proposed solution can be further developed to contribute to several related applications, such as Location Based Service.

This paper proposes a video playback system that allows user to expend the field of view to surrounding environments that are not visible in the original video frame, arbitrarily change the viewing angles, and see the superimposed point-of-interest (POIs) data in an augmented reality manner during the video playback. The processing consists of two main steps: in the first step, client uploads a video to the GeoVideo Engine, and then the GeoVideo Engine extracts the geo-metadata and returns them back to the client; in the second step, client requests POIs from server, and then the client renders the video with POIs.

Augmented reality applications are gaining popularity due to increased capabilities of modern mobile devices. Creating AR content however is tedious and traditionally done on desktop environments by professionals, with extensive knowledge and/or even programming skills required. In this demo, we demonstrate a complete mobile approach for creating vision-based AR in both indoor and outdoor environment. Using hyperlinks, Web mashups are built to dynamically augment the physical world by normal users without programing skills.

While many attempts have been done towards creating mixed reality platforms for mobile client devices, there have not been any significant efforts at the server/infrastructure side. We demonstrate our Mixed Reality Web Service Platform (MRS-WS) dedicated to enabling rapid creation of mixed reality solutions, those being either desktop or mobile. Focusing on common interfaces and functions across user generated and commercial geo-content, we provide an appealing developer offering, which we are currently evaluating via a closed set of university partners. Our plan is to gradually expand the developer API access to more partners, before deciding if it is ready for fully public developer access.

In this paper we describe a system for creating a navigable mirror world, utilizing community photographs of real life environments. We present the essential architecture and a prototype solution for not only geotagging, but also spatially structuring content. Mash-up interfaces are available towards 3rdparties, for linking and georeferencing their content.