Traditional skin-contact physical sensors typically detect changes of blood volume to predict the periodicity of heartbeat by analyzing the absorption spectra of hemoglobin. However, the contact on human skin may cause uncomfortable feeling and induce difficulty for long-term monitoring. Recently, video-based remote photoplethysmography (rPPG) estimation approaches analyze the periodic facial color changes for matching cardiac cycle in a contactless manner. Nevertheless, the inherent relationship between the changes of facial color and blood volume is not fully exploited. Besides the influence of blood volume (i.e., hemoglobin), there are also other factors such as lighting and reflection that cause the change on facial color. We exploit the physical principles that cause skin color variations to separate the hemoglobin factor driven by blood volume. Based on the physical prior of the reflection of human skin, we introduce an rPPG estimation network assisted by decoupled hemoglobin sequence, named HemNet, which first explicitly leverages hemoglobin to assist rPPG signal estimation. To obtain meaningful hemoglobin from facial video, we design a human skin color disentangler that decouples the facial color variations into four significant features, i.e., hemoglobin, melanin, shading, and specular. We then present a multi-modality rPPG estimator that utilizes cross-covariance attention to extract fused feature from hemoglobin and RGB video inputs. Finally, an adaptive negative Pearson loss is proposed to effectively address phase misalignment between the blood volume in the finger and facial region during the training phase. We evaluate our HemNet on four widely used public benchmark datasets. The superiority of our method is demonstrated in both intra-dataset and cross-dataset test settings. The code is available at https://github.com/jingang-cv/hemnet.

Multi-task learning (MTL) has been successfully applied to a wide range of real-world applications. However, MTL models often suffer from performance degradation with negative transfer due to sharing all features without distinguishing their helpfulness for all tasks. To this end, many works on feature selection for multi-task learning (FS-MTL) have been proposed to alleviate negative transfer between tasks by learning features selectively for each specific task. However, due to latent confounders between features and task targets, the correlations captured by the feature selection modules proposed in these works may fail to reflect the actual effect of the features on the targets. This paper explains negative transfer in FS-MTL from a causal perspective and presents a novel architecture called Causal Feature Selection for Multi-task Learning(CFS-MTL). This method incorporates the idea of causal inference into feature selection for multi-task learning via pseudo-intervention. It aims to select features with more stable causal effects rather than spurious correlations for each task by regularizing the distance between feature ITEs and feature importance. We conduct extensive experiments based on three real-world datasets to demonstrate that our proposed CFS-MTL outperforms state-of-the-art MTL models significantly in the AUC metric.

Multi-task learning (MTL) aims at solving multiple related tasks simultaneously and has experienced rapid growth in recent years. However, MTL models often suffer from performance degeneration with negative transfer due to learning several tasks simultaneously. Some related work attributed the source of the problem is the conflicting gradients. In this case, it is needed to select useful gradient updates for all tasks carefully. To this end, we propose a novel optimization approach for MTL, named GDOD, which manipulates gradients of each task using an orthogonal basis decomposed from the span of all task gradients. GDOD decomposes gradients into task-shared and task-conflict components explicitly and adopts a general update rule for avoiding interference across all task gradients. This allows guiding the update directions depending on the task-shared components. Moreover, we prove the convergence of GDOD theoretically under both convex and non-convex assumptions. Experiment results on several multi-task datasets not only demonstrate the significant improvement of GDOD performed to existing MTL models but also prove that our algorithm outperforms state-of-the-art optimization methods in terms of AUC and Logloss metrics.

Multi-task learning (MTL) has experienced rapid growth in recent years. A typical way of conducting MTL with deep neural networks (DNNs) is either establishing a sort of global feature sharing mechanism across all tasks or assigning each task an individual set of parameters with cross-connections. However, these existing approaches leverage DNNs only to share features of a certain order. Several modelsdemonstrated that explicitly modeling feature sharing with both low-order and high-order features can boost performance. To this end, we propose a model-agnostic sparse routing architecture called MASR, which emphasizes arbitrary order feature sharing for multi-task learning. It is able to choose specific orders of features to route for a given task through learnable latent variables. Moreover, MASR is model-agnostic and can be combined with existing MTL models to share features of both low-order and high-order. Extensive experimental results on several real-world datasets not only confirm the significant improvement of MASR performed to existing MTL models but also outperform existing hybrid architectures in terms of AUC metric.