Traditional frame-based imaging systems face trade-offs especially under low-light conditions: long-exposure frames have high signal-to-noise ratio (SNR) but suffer from motion blur and low frame rate, while short-exposure frames are sharp but suffer from noise. Event cameras can complement these modalities by capturing brightness changes and exposure-invariant information at very high temporal resolution. We propose a novel multi-modal framework for lowlight enhancement by integrating information from long-exposure, short-exposure, and event data. We introduce an enhanced lightup mechanism to improve short-exposure frames using illumination from long-exposure frames, and an SNR-guided fusion strategy for noise suppression and detail preservation. We use event information to improve feature selection, especially in the low SNR regions. Experimental results on public datasets demonstrate that our method can effectively handle noise and motion blur. We obtain 14dB improvement in PSNR over state-of-the-art methods based on conventional frames and 4dB improvement over methods based on frames and events.

This paper describes the background, design and application of a new SEI message – the Multiplane Image Information SEI message, which has recently been adopted into the Technology under Consideration (TuC) document of the JVET committee for potential inclusion in the VSEI standard (ITU-T H.274 and ISO/IEC 23002-7). The paper also provides preliminary compression experiment results and analysis on the implications of the coding efficiency and functionality of the different packing options supported in the SEI message.

This paper describes a new video coding tool in the Versatile Video Coding standard (VVC) named as luma mapping with chroma scaling (LMCS). Experimental compression performance results for LMCS and non-normative examples for deriving LMCS parameter values are also provided. LMCS has two main components: 1) a process for mapping input luma code values to a new set of code values for use inside the coding loop; and 2) a luma-dependent process for scaling chroma residue values. The first process, luma mapping, aims at improving the coding efficiency for standard and high dynamic range video signals by making better use of the range of luma code values allowed at a specified bit depth. The second process, chroma scaling, manages relative compression efficiency for the luma and chroma components of the video signal. The luma mapping process of LMCS is applied at the pixel sample level, and is implemented using a piecewise linear model. The chroma scaling process is applied at the chroma block level, and is implemented using a scaling factor derived from reconstructed neighboring luma samples of the chroma block.

This paper describes a video coding scheme submitted in response to the joint call for proposals (CfP) on video compression for capability beyond HEVC issued by ITU-T SG16 Q.6 (VCEG) and ISO/IEC JTC1/SC29/WG11(MPEG) in October 2017. It includes video coding techniques for the standard dynamic range (SDR) and high dynamic range (HDR) categories. Design of the core SDR codec in the response is based on the joint exploration model (JEM) reference software developed by the joint video exploration team (JVET). Some of key coding tools in the JEM are significantly simplified to reduce both average and worst-case complexity for hardware design with negligible coding performance loss. Furthermore, two additional coding technologies, namely multi-type tree (MTT) and decoder-side intra mode derivation (DIMD), are used to further improve coding efficiency. For the HDR category, besides the tools used in SDR category, two additional coding tools: an in-loop reshaper and a luma-based QP prediction method are used to further improve HDR coding efficiency. Simulation results demonstrate the high coding efficiency achieved by the proposed video codec at the expense of moderate coding complexity over HEVC. For random access configuration, it achieves average bit rate savings of 35.7% and 4.00% over the HM and JEM anchors with decoding time of 263% and 33%, respectively, for the SDR sequences. For the HDR sequences, the proposed in-loop reshaper is configured to maximize HDR objective metrics, it achieves average bit rate savings of 31.3% and 4.6% over the HM and JEM for wPSNRY metrics for the HDR PQ content.

ICTCP is a colour difference signal representation recently standardized in Rec. ITU-R BT.2100. One design criteria of ICTCP for HDR/WCG imagery is to address issues in Non-Constant Luminance (NCL) Y'CbCr with HDR/WCG signals. In this paper, the superior performance of ICTCP in preserving constant luminance over the NCL Y'CbCr is analyzed in detail. The fundamental limitations of Y'CbCr luma adjustment algorithm which aims at fixing chroma subsampling artifact of Y'CbCr is discussed as well. This paper also provides guidance on signaling of ICTCP and feasible encoder settings with HEVC Main 10 codec. Experimental results suggest that for HDR content with real BT. 2100 (BT. 2020) colour gamut, to perform chroma subsampling and compression in ICTCP have significant value.

This paper describes the Orthogonal Muxing Frame Compatible Full Resolution (OM-FCFR) technology that provides efficient compression and reconstruction of full resolution stereoscopic video while maintaining compatibility with stereoscopic AVC Frame Compatible (FC) bitstreams. The OM-FCFR is a dual-layer coding system and is built upon the MVC Stereo High Profile of the AVC. Orthogonal muxing is used to design the enhancement layer video signal. A “Reference Process Unit (RPU)” is employed for creating the inter-layer reference picture that can serve as a good prediction for an enhancement layer picture. Simulation results show that significant improvement over the frame compatible solution is achieved in both subjective and objective evaluations. This technology was submitted as a proposal in response to the call for proposals (CfP) on MPEG Frame- Compatible (MFC) enhancement technology issued by ISO/IEC JTC1/SC29/WG11 (MPEG) and the specification was adopted as the working draft for the MFC standard and AVC Amendment 5.

The growth in digital, stereographic motion pictures has been surprising and strong, providing a new revenue generator for the industry. After a sluggish start for the digital cinema industry, 3-D movies have become the primary driver of digital cinema adoption. Those experienced in digital stereographic techniques are clamoring for more tools and capabilities to support their creative visions. Meanwhile, the large industry of movie production workers is clamoring for information on the basic methods of creating and presenting stereographic movies. This paper presents the techniques and tradeoffs in employing 3-D camera capture, 3-D computer generated imagery, and 2-D-to-3-D conversion in developing stereographic images. It further reviews the current status and future possibilities for 3-D viewing systems in the exhibition cinemas.