Recent video codecs, e.g. AV1, VVC, apply a Non-rectangular (NR) partitioning to combine prediction signals using a smooth blending around the boundary, followed by a rectangular transform (TX) on the whole block. TX on each NR residual separately is not yet supported. A recent NR TX technique [1] demonstrated promising gains in an experimental setup outside the reference software. This method employs the regular inverse-DCT at the decoder to reconstruct a rectangular signal while discarding the signal outside the region of interest. This design is appealing due to the minimal changes required at the decoder. The method uses a partitioned 2D DCT as a dictionary to find a sparse representation of the NR signal, with scaled representations serving as TX coefficients. These coefficients typically exhibit properties distinct from those of DCT TX coefficients. Therefore, the established entropy coding schemes in video codecs, which are primarily designed for DCT coefficients, are not well-suited for optimally encoding these TX coefficients.

The VVC codec is applied to the task of multispectral image (MSI) compression using adaptive and scalable coding structures. In a “plain” VVC approach, concepts from picture-to-picture temporal prediction are employed for decorrelation along the MSI’s spectral dimension. The popular principle component analysis (PCA) for spectral decorrelation is further evaluated in combination with VVC intra-coding for spatial decorrelation. This approach is referred to as PCA-VVC. A novel adaptive MSI compression algorithm, named HPCLS, is introduced, that uses PCA and inter-prediction for spectral and VVC intra-coding for spatial decorrelation. Further, a novel adaptive scalable approach is proposed, that provides a separately decodable spectrally scaled preview of the MSI in the compressed file. Information contained in the preview is exploited in order to reduce the overall file size. All schemes are evaluated on images from the ARAD HS data set containing outdoor scenes with a high variety in brightness and color. We found that “Plain” VVC is outperformed by both PCA-VVC and HPCLS. HPCLS shows advantageous rate-distortion (RD) behavior compared to PCA-VVC for reconstruction quality above 51 dB PSNR. The performance of the scalable approach is compared to the combination of an independent RGB preview and one of HPCLS or PCA-VVC denoted as simulcast. The scalable approach shows significant benefit especially at higher preview qualities. A more detailed version of this article can be found on arXiv1.