Point Clouds (PCs) have gained considerable interest as a potential format for representing tridimensional (3D) data in various applications, including augmented reality and autonomous vehicles. In the realm of multimedia, where applications and technologies such as 3DTV revolve around human interaction, it is crucial to assess how humans perceive the visual quality of PCs. To address this need, research on Point Cloud Quality Assessment (PCQA) metrics, incorporating aspects of the human visual perception, has gained prominence. This research aims to facilitate PC quality enhancement, optimize PC registration pipelines, and improve the performance of PC codecs. However, assessing the quality of PCs poses inherent challenges due to its irregularity and sparsity, necessitating the identification of mathematical relationships within PCs for accurate quality predictions. In this work, we adopt a projection-based approach, representing a PC as a structured set of bidimensional (2D) projections-regular images derived from the 3D structure of the PC. We leverage the power of Neural Networks (NNs) and Machine Learning (ML) to create a perceptual-driven, full-reference metric for PCQA. We assess these images using established Image Quality Assessment (IQA) methods, widely acknowledged in the state-of-the-art (SOTA) for traditional 2D (non-immersive) visual content. The generated scores are then deposited into a vector, serving as input for a regressor model to predict the final quality score. Our results demonstrate the competitiveness of our model compared to SOTA metrics.

Real-time health monitoring is revolutionizing healthcare delivery nowadays. Using everyday settings, especially due to the recent wearable health devices, it is possible to monitor individuals at any place and moment, allowing the detection and prevention of many diseases. Among the various technologies present in wearable devices that allow continuous health monitoring, Photoplethysmography (PPG) is one of the most important techniques. PPG is non-invasive, low-cost, easy-to-implement, and, therefore, convenient to track physiological signals, such as oxygen saturation in the bloodstream, heart rate variability, respiration rate, etc. Due to these advantages, PPG is widely used in diverse health applications, notably in commercial wearable apparatuses. However, despite its advantages, PPG presents a main drawback of being highly susceptible to motion artifacts and environmental noises, which impair PPG-based applications, especially when PPG signals are recorded via wearable devices. Therefore, to enable reliable measurements, signal quality must be assessed, and unreliable signals should be rejected. With such signal reliability needs, the most important thing is Signal Quality Assessment (SQA). In this paper, we introduce a novel SQA method that projects the 1D PPG signals into 2D images and then uses a Vision Transformer (ViT) to classify their quality. Results show that the proposed method presents a competitive quality prediction accuracy when compared with the state-of-the-art.

Point Clouds (PCs) are collections of points distributed in the 3D space, containing attributes such as color, normals, transparency, and specularity. Dynamic Point Clouds (DPCs) correspond to sequences of points in the 3D space that vary over time like pixels vary over time in a conventional video. Dynamic PCs are a suitable way to represent volumetric videos that can be used in augmented or virtual reality applications. This representation, however, requires a large number of points to achieve a high quality of experience and needs to be compressed before storage and transmission. Therefore, reliable quality metrics are needed in order to automatically estimate the perceptual quality of dynamic PC contents. Since currently there are several quality assessment metrics for static PC, a possible approach solution consists of using temporal pooling functions to combine the quality scores predicted for each of the frames. In this paper, we study the effects of different temporal pooling strategies on the performance of dynamic PC quality assessment methods. Our experimental tests were performed using a recent publicly-available database, demonstrating the efficiency of the evaluated temporal pooling models. More specifically, the work provides a recipe on how to apply a temporal pooling function to combine frame-based quality predictions generated with texture-based static PC quality assessment methods to estimate the quality of dynamic PCs.

Point Clouds (PCs) have recently been adopted as the preferred data structure for representing 3D visual contents. Examples of Point Cloud (PC) applications range from 3D representations of small objects up to large scenes, both still or dynamic in time. PC adoption triggered the development of new coding, transmission, and display methodologies that culminated in new international standards for PC compression. Along with these, in the last couple of years, novel methods have been developed for evaluating the visual quality of PC contents. This paper presents a new objective full-reference visual quality assessment metric for static PC contents, named BitDance, which uses color and geometry texture descriptors. The proposed method first extracts the statistics of color and geometry information of the reference and test PCs. Then, it compares the color and geometry statistics and combines them to estimate the perceived quality of the test PC. Using publicly available PC quality assessment datasets, we show that the proposed PC quality assessment metric performs very well when compared to state-of-the-art quality metrics. In particular, the method performs well for different types of PC datasets, including the ones where both geometry and color are not degraded with similar intensities. BitDance is a low complexity algorithm, with an optimized C++ source code that is available for download at github.com/rafael2k/bitdance-pc_metric.

The popularity of smartphones, virtual reality headsets, and head-mounted devices is fomenting immersive applications that employ realistic representations of the real world. Among these, Point Cloud (PC) contents have recently gained prominence in academia and industry. However, there is some consensus that PC objective quality assessment methods are still an open problem. In this paper, we introduce a PC quality metric based on multiple distances between reference and test PCs. These distances are computed in both still points and texture spaces. Distances computed in still points space consider the direct differences between reference and test points. Distances in the texture space are measured after computing the Local Binary Pattern (LBP) descriptor of PCs. Since PCs are not equally geometrically distributed, we adapted the LBP descriptor to make the nearest points as the neighborhood pixels of the descriptor. The difference between the LBP statistics of reference and test PCs is used to assess the quality of the test PC. Experimental results show the proposed method performs well when compared with the state-of-the-art Point Cloud Quality Assessment (PCQA) methods.

In recent years image compression techniques based on deep learning have achieved great success and their performances are gradually reaching the methods crafted by experts, such as JPEG, WebP, and Better Portable Graphics (BPG). A technique that is fundamental for modern image and video codecs is intra prediction, which takes advantage of local redundancy to predict the pixels from previously encoded neighbors. In this paper, we use Convolutional Neural Networks (CNN) to develop a new intra-picture prediction mode. More specifically, we propose a multi-mode intra prediction approach that uses two CNN-based prediction modes and all intra modes previously implemented in the High Efficiency Video Coding (HEVC) standard. We also propose a bit allocation technique that increases the bitstream only if the reconstruction error is significantly reduced. Experimental results evince a significant and consistent performance increase compared to other approaches that use a similar backbone architecture, with 28% bitrate reduction compared to the baseline codec.

In recent years, there has been an increase in the popularity of Point Clouds (PC) as the preferred data structure for representing 3D visual contents. Examples of PC applications range from 3D representations of small objects up to large maps. The advent of PC adoption triggered the development of new coding, transmission, and presentation methodologies. And, along with these, novel methods for evaluating the visual quality of PC contents. This paper presents a new objective full-reference visual quality metric for PC contents, which uses a proposed descriptor entitled Local Luminance Patterns (LLP). It extracts the statistics of the luminance information of reference and test PCs and compares their statistics to assess the perceived quality of the test PC. The proposed PC quality assessment method can be applied to both large and small scale PCs. Using publicly available PC quality datasets, we compared the proposed method with current state-of-the-art PC quality metrics, obtaining competing results.

The proliferation of devices such as mobile phones, virtual reality headsets, and head-mounted displays has increased the popularity of immersive applications that deliver realistic representations of the real world. Among the technologies that enable such applications, the point cloud (PC) technology seems to be one of the most mature alternatives, gaining prominence in academia, industry, and standardization committees. Although PC technologies have been used in entertainment, automotive, and geographical location industries, the design of objective quality assessment methods for PC contents is still an open problem. In this paper, we introduce a texture-based objective quality assessment method for PC contents. The method analyzes the texture of the PC content using the Local Binary Pattern (LBP) descriptor. Unlike points in still (2D) images, the points in a PC are not equally distributed in space. Therefore, we adapted the LBP descriptor to allow processing a PC point and its neighboring points. The statistics of the LBP outputs, for both reference and test PCs, are computed and compared to obtain a quality estimate for the test (impaired) PC content. Experimental results show that the proposed PC quality metric has a good correlation with subjective quality scores, outperforming state-of-the-art PC quality metrics.

Due to the rapid development of multimedia technologies, over the last decades image quality assessment (IQA) has become an important topic. As a consequence, a great research effort has been made to develop computational models that estimate image quality. Among the possible IQA approaches, blind IQA (BIQA) is of fundamental interest as it can be used in most multimedia applications. BIQA techniques measure the perceptual quality of an image without using the reference (or pristine) image. This paper proposes a new BIQA method that uses a combination of texture features and saliency maps of an image. Texture features are extracted from the images using the local binary pattern (LBP) operator at multiple scales. To extract the salient of an image, i.e. the areas of the image that are the main attractors of the viewers' attention, we use computational visual attention models that output saliency maps. These saliency maps can be used as weighting functions for the LBP maps at multiple scales. We propose an operator that produces a combination of multiscale LBP maps and saliency maps, which is called the multiscale salient local binary pattern (MSLBP) operator. To define which is the best model to be used in the proposed operator, we investigate the performance of several saliency models. Experimental results demonstrate that the proposed method is able to estimate the quality of impaired images with a wide variety of distortions. The proposed metric has a better prediction accuracy than state-of-the-art IQA methods.

This paper proposes a new general-purpose no-reference image quality assessment (NR-IQA) method based on color texture analysis. Specifically, the proposed method uses the statistics of the orthogonal color planes pattern (OCPP) descriptor to characterize image quality. The OCPP descriptor, proposed in this paper, is an extension of the local binary pattern operator that incorporates color information. To make NR-IQA methods more generic, that is, more sensitivity to different types of degradation (e.g., color and contrast degradation), it is important to take into consideration the color information. In the proposed NR-IQA method, we use the statistics of the OCPP descriptor as an input vector to a regression algorithm, which models the nonlinear relationship between the OCPP statistics and the subjective opinion scores. Experimental results show that proposed IQA method is quite efficient when compared to popular state-of-the-art NR-IQA methods.

In this paper, we propose a new no-reference image quality assessment (NR-IQA) method that uses a machine learning technique based on Local Ternary Pattern (LTP) descriptors. LTP descriptors are a generalization of Local Binary Pattern (LBP) texture descriptors that provide a significant performance improvement when compared to LBP. More specifically, LTP is less susceptible to noise in uniform regions, but no longer rigidly invariant to gray-level transformation. Due to its insensitivity to noise, LTP descriptors are not able to detect milder image degradation. To tackle this issue, we propose a strategy that uses multiple LTP channels to extract texture information. The prediction algorithm uses the histograms of these LTP channels as features for the training procedure. The proposed method is able to blindly predict image quality, i.e., the method is no-reference (NR). Results show that the proposed method is considerably faster than other state-of-the-art no-reference methods, while maintaining a competitive image quality prediction accuracy.

Inverse halftoning techniques are known to introduce visible distortions (typically, blurring or noise) into the reconstructed image. To reduce the severity of these distortions, we propose a novel training approach for inverse halftoning algorithms. The proposed technique uses a coupled dictionary (CD) to match distorted and original images via a sparse representation. This technique enforces similarities of sparse representations between distorted and non-distorted images. Results show that the proposed technique can improve the performance of different inverse halftone approaches. Images reconstructed with the proposed approach have a higher quality, showing less blur, noise, and chromatic aberrations.