Recent applications of decorrelation methods to the multi-head attention layers and output embeddings of transformer-based models have resulted in improvements in efficiency and accuracy. Despite these advancements, there is a lack of research focused on the influence of decorrelation on transformer interpretation techniques. This study investigated the impact of two decorrelation methods on interpreting the decision-making logic of a Bidirectional Encoder Representations from Transformers (BERT) model. Two metrics, namely Comprehensiveness and Sufficiency, were used to quantify the interpretation quality, while the changes in correlation within each multi-head self-attention layer was statistically analyzed. Results indicate that decorrelating BERT embeddings leads to a sparser distribution of weights in the middle attention layers and a significantly improved interpretation quality. Conversely, decorrelating the attention maps of specific attention layers increases the correlation in the corresponding attention weight matrices, yielding a less marked improvement in interpretation quality and, in some instances, degraded model performance.

Speech foundation models are remarkably successful in various consumer applications, prompting their extension to clinical use-cases. This is challenged by small clinical datasets, which precludes effective fine-tuning. We tested the efficacy of two models to classify participants by segmental (Wav2Vec2.0) and suprasegmental (Trillsson) speech analysis windows. Analysis at both time scales has shown differences in the context of cognitive decline. Speakers were classified as healthy controls (HC), Amyloid-β+ (Aβ+), mild cognitive impairment (MCI), or dementia. A subset of W2V2 and Trillsson representations showed large effect size between HC and each risk factor. Cross-validation showed W2V2 consistently outperforms Trillsson. Mean macro-F1 of 54.1%, 63.5%, and 72.0% in were found for classifying Aβ+, MCI, and dementia from HC. Repeatability of Trillsson and W2V2 showed intraclass correlations of 0.30 and 0.41. Reliability of such models must be enhanced for clinical speech analysis and longitudinal tracking.

Training robust clinical speech-based models that generalize requires large sample sizes because speech is variable and high-dimensional. Researchers have turned to foundational models, such as the Bidirectional Encoder Representations from Transformers (BERT), to generate lower-dimensional embeddings, and then finetuned the models for a specific down-stream clinical task. While there is empirical evidence that this approach is helpful, a recent study reveals that the embeddings generated by BERT models tend to be highly correlated, which makes the downstream models difficult to fine-tune, particularly in the small sample size regime. In this work, we propose a new regularization scheme to penalize correlated embeddings during fine tuning of BERT and apply the approach to speech-based assessment of cognitive impairment. Compared to existing methods, the proposed method yields lower estimation errors and smaller false alarm rates in a Mini-Mental State Examination (MMSE) score regression task.

Acoustic analysis plays an important role in the assessment of dysarthria. Out of a public health necessity, telepractice has become increasingly adopted as the modality in which clinical care is given. While there are differences in software among telepractice platforms, they all use some form of speech compression to preserve bandwidth, with the most common algorithm being the Opus codec. Opus has been optimized for compression of speech from the general (mostly healthy) population. As a result, for speech-language pathologists, this begs the question: is the remotely transmitted speech signal a faithful representation of dysarthric speech? Existing high-fidelity audio recordings from 20 speakers of various dysarthria types were encoded at three different bit rates defined within Opus to simulate different internet bandwidth conditions. Acoustic measures of articulation, voice, and prosody were extracted, and mixed-effect models were used to evaluate the impact of bandwidth conditions on the measures. Significant differences in cepstral peak prominence, degree of voice breaks, jitter, vowel space area, pitch, and vowel space area were observed after Opus processing, providing insight into the types of acoustic measures that are susceptible to speech compression algorithms.

A common assumption in stereo matching is that the corresponding pixels in stereo images have similar pixel values. Unfortunately, such an assumption may not be true due to radiometric variations in different views, leading to severely degraded matching results. In this letter, we propose a radiometrically invariant stereo matching algorithm called Optimal Local Adaptive Radiometric Compensation (LARAC). In LARAC, we approximate the spatially varying Pixel Value Correspondence Function (PVCF) between a corresponding pixel pair as a locally consistent polynomial within an optimal local adaptive window. The optimal polynomial coefficients are obtained for each candidate disparity value and are used to compute the matching cost. Meanwhile, a self-correction property is achieved by the proposed LARAC, leading to reduced matching errors for the outlier pixels. Experimental results suggest that the proposed LARAC outperforms other state-of-the-art stereo matching algorithms.

The emerging cloud gaming technology has been growing fast, driving up huge mobile consumer demands. The video streaming based cloud gaming scenario renders the game scenes in the cloud servers, and streams the encoded sequences to the thin clints where the game scenes are decoded and displayed to the players. However, current existing clouding gaming services have some problems, such as the latency and bandwidth limitation. The size of the video stream is usually quite large which requires heavy transmission. Worse still, the frame data rate will burst when the game scenes contain fast translation or rotation, resulting in strong latency problem. In this paper, we propose a novel video streaming based cloud gaming algorithm which reduces the burst of the frame rate significantly. There are mainly two innovations in this paper. Firstly, based on the analysis of the motion estimation strategy in the video codec, we introduce image homography technique for better motion prediction. Meanwhile, according to the rasterization rules of the game engine, we present a special designed interpolation algorithm named Edge Preserved Interpolation (EPI), for more accurate edge interpolation and further reduce the residues in the edge regions. The proposed algorithm is implemented on the x264 platform. Experimental results show that our algorithm has 18.0% BD-rate reduction compared with x264.

We study the problem of formulating the discrete dense stereo matching using continuous convex optimization. One of the previous work derived a relaxed convex formulation by establishing the relationship between the disparity vector and a warping matrix. However it suffers from high computational complexity. In this paper, the previous convex formulation is translated into an equivalent quadratic programming (QP). Then redundant variables and constraints are eliminated by exploiting the internal sparse property of the warping matrix. The resulting QP can be efficiently tackled using interior point solvers. Moreover, enhanced smoothness term and effective post-processing procedures are also incorporated to further improve the disparity accuracy. Experimental results show that the proposed method is much faster and better than the previous convex formulation, and provides competitive results against existing convex approaches.

In this paper, we present a novel view synthesis method named Visto, which uses a reference input view to generate synthesized views in nearby viewpoints. We formulate the problem as a joint optimization of inter-view texture and depth map similarity, a framework that is significantly different from other traditional approaches. As such, Visto tends to implicitly inherit the image characteristics from the reference view without the explicit use of image priors or texture modeling. Visto assumes that each patch is available in both the synthesized and reference views and thus can be applied to the common area between the two views but not the out-of-region area at the border of the synthesized view. Visto uses a Gauss–Seidel-like iterative approach to minimize the energy function. Simulation results suggest that Visto can generate seamless virtual views and outperform other state-of-the-art methods.

3D technologies such like three-dimensional television and free viewpoint television have caught enormous attentions in the consumer market recently. However, because of the inaccurate camera configuration and environmental constraint, there are errors in the assumed equally-spaced camera intervals. In this paper, we propose a novel camera spacing correction algorithm to detect the spacing errors among the multiple cameras by making the corresponding points co-linear in the epipolar plane images. Experimental results show that the proposed algorithms are robust and can achieve good performance even if the corresponding pixels are not well detected. Meanwhile, our algorithm can be solved by convex optimization with an extremely low complexity.

Halftone image watermarking has been explored and developed rapidly over the past decade. However, there are still issues to be studied. This paper presents a data hiding method called Data Hiding by Dual Color Conjugate Error Diffusion (DHDCCED) to hide a binary secret pattern into two error diffused color halftone images, such that when the two color halftone images are overlaid, the secret pattern will be revealed. The experimental results show that DHDCCED can significantly improve the performances when comparing both the correct decoding rate and the visual quality of the revealed secret pattern to the existing method Color Conjugate Error Diffusion (CCED).

Cost aggregation is the most essential step for dense stereo correspondence searching, which measures the similarity between pixels in the stereo images. In this paper, based on the analysis of the optimal adaptive weight, we propose a novel support aggregation strategy by adaptive weighting selection. The proposed method calculates the aggregation cost by the joint optimization of both left and right matching cost. By assigning more reasonable weighting coefficients, we exclude the occlusion pixels while preserving sufficient support region for accurate matching. The proposed optimal strategy can be integrated by any other adaptive weighting based cost aggregation method to generate more reasonable similarity measurement. Experimental results show that, compare with traditional methods, our algorithm can reduce the foreground fatten phenomenon while increasing the accuracy in the high texture regions.

View synthesis is a technique to generate images/videos in a virtual viewpoint. In this paper, the dis-occlusion/hole problem in view synthesis is resolved from the temporal domain. By the fact that dis-occlusions belong to the background, firstly we build an online background under a newly designed Switchable Gaussian Model (SGM), owning to its computationally simplicity and scene adaptivity. Then, real textures in the dis-occlusions are able to be recovered with the built background. Experimental results have verified the improvements in rendering quality and computation complexity by comparing to the conventional spatial filling methods and other temporal filling methods.

In depth map coding for 3D video coding systems, coding errors in edges can severely affect the synthesis quality. Edge errors mainly compose of two parts: one is blurring and ringing artifact around sharp edge and the other is fake edge caused by blocking artifact. In this paper, we propose an adaptive depth map filter to remove blocking artifacts while preserving depth edges. The proposed filter is designed based on bilateral filter, in which the range kernel parameter is changed adaptively considering the strength of edges and blocking artifacts. Experimental results demonstrate that the proposed depth map filter can achieve up to 0.41 dB gain on the synthesis quality compared to the deblocking filter in MVC at low bit rate.

In this paper, we present a view synthesis method named Visto which aims to generate seamless novel views from a monocular view input. We formulate the problem as joint optimization of inter-view texture similarity and geometry preservation, which significantly differs from traditional view synthesis framework. In this way, the image characteristics of virtual view are inherently inherited from the reference view without introducing any image prior or texture modeling technique. The energy function is minimized using Gauss-Seidel-like approach, and the quality of the virtual view is refined iteratively. The proposed approach also tolerates small depth map errors. Further more, the algorithm is parallel friendly. The simulation results outperform several existing state-of-the-art monocular view synthesis systems.

View synthesis offers a great flexibility in generating free viewpoint television (FTV) and 3D video (3DV). However, the depth-image-based view synthesis approach is very sensitive to errors in the camera parameters or poorly estimated depth maps (also called depth images). Because of these errors, three kinds of artifacts (blurring, contour, hole) are possibly introduced during the general synthesis process. Comparing to conventional methods which implement the view synthesis only in ideal case, in this paper, we propose to design an error compensation and reliability based view synthesis system where the potential errors are considered. The main contributions are highlighted as follows: Firstly, the camera parameter errors are compensated by a global homography transformation matrix. Secondly, the depth maps are classified into both reliable and unreliable regions and the reliability based weighting masks are built to blend synthesized images from two different views together. Finally, a reliability depth map based hole-filling technique is used to fill the existing holes. The experimental results demonstrate that these artifacts are efficiently reduced in the synthesized images.

We present a novel convex-optimization approach to solving the dense stereo matching problem in computer vision. Instead of directly solving for disparities of pixels, by establishing the connection between a permutation matrix and a disparity vector, we directly formulate the stereo matching problem as a continuous convex quadratic program in a simple, elegant and straightforward manner without performing any complicated relaxation or approximation. By using CVX, the Matlab software for disciplined convex programming, our method is extremely simple to implement.

Single camera stereo system utilizes mirrors and a single camera for computational stereo, where the mirrors provide extra views needed for stereo and 3D reconstruction. In this paper, we investigate the basic epiploar geometric properties of the single camera stereo image and propose a novel image rectification technique to map the epipolar lines in the original image into the horizontally aligned lines in the rectified image. Besides, the rotation angles of the single camera corresponding to the planar mirror are derived during rectification. Experimental results show the robustness and accuracy of our method.

When encrypted data need to be sent to an untrusted computer for processing, it is highly desirable to have a homomorphic cryptosystem that allows the untrusted computer to process the encrypted signals totally without decryption such that, when the encrypted domain result is decrypted, the decrypted value is the same as the equivalent plaintext domain operation. These operations range from basic arithmetic operations to complicated transformations. In this paper, we analyze the existence of equivalent operations in the encrypted domain for four of the common bitwise operations: OR, AND, NOR and NAND in plaintext domain. We will show that such equivalent operations should not exist. For otherwise, if such operations exist, the RSA cryptosystems can be broken by a low-complexity attack.

Depth map is widely adopted and available in the 3D research area. Combining the depth map with the matting techniques is helpful to the original matte and depth image based rendering in 3D. Herein, in this paper, a novel adaptive depth map assisted matting approach with concise integration is presented and applied to achieve favorable matting results. In this approach, the Lagrange-multiplier-free closed form solution is firstly derived to reduce the computation complexity and to increase matting accuracy. Based on the work of Levin et al. on closed form matting, an improved alpha matte is then achieved by introducing an adaptive smoothness criterion which is the function of depth map variance. Finally, the matting system is capable of working in a full automatical way by generating the trimap from the depth information. Simulation results demonstrate that the proposed method is able to efficiently generate an alpha matte with an roughly user specified scribbles or an automatically generated trimap.

We address the problem of boundary estimation by formulating it as inter-region contour and intra-region information analysis in the framework of graph-based segmentation. Given an image without any prior information about object model and class, we seek to approximate one's instant perception of visual similarity. The method can serve as a preprocessing step for many higher level operations that require regional support, such as scene understanding and object recognition. We show in this paper that the defined region comparison predicate makes a better boundary estimator than efficient graph-based image segmentation (EGS) - a well known and widely used segmentation method. We further illustrate, by making a small relaxation, further improvement of segmentation performance can be achieved. Experimental results have demonstrated the effectiveness of our proposed method.

In this paper we give a step-by-step detailed analysis on the performance of shortest spanning tree (SST) and its revised version, recursive SST (RSST). We further propose a novel segmentation scheme based on recursive SST in the warped domain produced by density estimation. The proposed method is robust for variant natural image input and is easy to implement. Experimental results and comparisons with other methods have illustrated the effectiveness and robustness of the proposed method.

Accurate estimation of wind speed can improve control capabilities of wind turbines. The wind speed is generally different at every point on the surface covered by the blades due to three dimensional and time-varied wind fields; effective wind speed cannot be measured by an anemometer. In this paper the problem of effective wind speed estimation is considered as soft sensor. Soft sensor modeling utilizes technique of support vector machine, which is efficient for the problem characterized by small sample, nonlinearity, high dimension, local minima and has high generalization. Comparing with Kalman filter, simulation results show support vector machine is an effective method for soft sensor modeling. Effective wind speed estimation can exactly track the trend of wind speed and has high estimation precision.