Visual Dialog aims to answer multi-round, interactive questions based on the dialog history and image content. Existing methods either consider answer ranking and generating individually or only weakly capture the relation across the two tasks implicitly by two separate models. The research on a universal framework that jointly learns to rank and generate answers in a single model is seldom explored. In this paper, we propose a contrastive learning-based framework UTC to unify and facilitate both discriminative and generative tasks in visual dialog with a single model. Specifically, considering the inherent limitation of the previous learning paradigm, we devise two inter-task contrastive losses i.e., context contrastive loss and answer contrastive loss to make the discriminative and generative tasks mutually reinforce each other. These two com-plementary contrastive losses exploit dialog context and target answer as anchor points to provide representation learning signals from different perspectives. We evaluate our proposed UTC on the VisDial v1.0 dataset, where our method outperforms the state-of-the-art on both discriminative and generative tasks and surpasses previous state-of-the-art generative methods by more than 2 absolute points on Recall@1.

Self-supervised learning, especially contrastive learning, has made an outstanding contribution to the development of many deep learning research fields. Recently, researchers in the acoustic signal processing field noticed its success and leveraged contrastive learning for better music representation. Typically, existing approaches maximize the similarity between two distorted audio segments sampled from the same music. In other words, they ensure a semantic agreement at the music level. However, those coarse-grained methods neglect some inessential or noisy elements at the frame level, which may be detrimental to the model to learn the effective representation of music. Towards this end, this paper proposes a novel Positive-nEgative frame mask for Music Representation based on the contrastive learning framework, abbreviated as PEMR. Concretely, PEMR incorporates a Positive-Negative Mask Generation module, which leverages transformer blocks to generate frame masks on Log-Mel spectrogram. We can generate self-augmented negative and positive samples by masking important components or inessential components, respectively. We devise a novel contrastive learning objective to accommodate both self-augmented positives/negatives sampled from the same music. We conduct experiments on four public datasets. The experimental results of two music-related downstream tasks, music classification and cover song identification, demonstrate the generalization ability and transferability of music representation learned by PEMR.

We present a method (DMESH) for nonrigid cyclic motion analysis using a series of velocity images covering the cycle acquired, for example, from phase-contrast magnetic resonance imaging. The method is based on fitting a dynamic finite-element mesh model to velocity samples of an extended region, at all time frames. The model offers a flexible tradeoff between accuracy and reproducibility with controllable built-in spatiotemporal smoothing, which is determined by the fineness of the initially defined mesh and the richness of included Fourier harmonics. The method can further provide a prediction of the analysis reproducibility, along with the estimated motion and deformation quantities. Experiments have been conducted to validate the method and to verify the reproducibility prediction. Use of the method for motion analysis using displacement information (e.g., from magnetic resonance tagging) has also been explored.

Presents a method for nonrigid cyclic motion analysis using a series of velocity images, acquired for example, from phase contrast magnetic resonance imaging. The method is based on fitting a global spatiotemporal finite element mesh model, which has controllable builtin spatial/temporal smoothing that is determined by the finesse of the initially defined mesh and richness of included Fourier harmonics, to velocity samples of an extended region at all time frames. If desired, the method can further provide a prediction of the analysis reproducibility along with the estimated motion and deformation quantities. Experiments have been conducted to validate the method and verify the reproducibility prediction.

Phase contrast magnetic resonance imaging (MRI) can provide in vivo myocardial velocity field measurements. These data allow densely spaced material points to be tracked throughout the whole heart cycle using, for example, the Fourier tracking algorithm. To process the tracking results for myocardial deformation and strain quantification, we developed a method that is based on fitting the tracking results to an appropriate local deformation model. We further analyzed the accuracy and precision of the method and provided performance predictions for several local models. In order to validate the method and the theoretical performance analysis, we conducted controlled computer simulations and a phantom study. The results agreed well with expectations. Human heart data were also acquired and analyzed, and provided encouraging results. At the signal-to-noise ratio (SNR) level and spatial resolution expected in clinical settings, the study predicts strain quantification accuracy and precision that may allow the technique to become a practical and powerful noninvasive approach for the study of cardiac function, although clinically acceptable data acquisition strategies for three-dimensional (3-D) data are still a challenge.