Diffusion models have been adopted for behavioral cloning in a sequence modeling fashion, benefiting from their exceptional capabilities in modeling complex data distributions. The standard diffusion-based policy iteratively denoises action sequences from random noise conditioned on the input states and the model is typically trained with a singular diffusion loss. This paper explores the potential enhancements in such models when the denoising process is informed by a better visual representation. We study the scenario where the model is jointly optimized using the standard diffusion loss alongside an auxiliary objective based on self-supervised learning. After experimenting with various objectives, we introduce Crossway Diffusion, a simple yet effective way to enhance diffusion-based visuomotor policy learning via a state decoder and an auxiliary reconstruction objective. During training, the state decoder reconstructs raw image pixels and other states from the intermediate representations of the model. Experiments demonstrate the effectiveness of our method in various simulated and real-world tasks, confirming its consistent advantages over the standard diffusion-based policy and other baselines.

The best performance in Air-tissue boundary (ATB) segmentation of real-time Magnetic Resonance Imaging (rtMRI) videos in speech production is known to be achieved by a 3-dimensional convolutional neural network (3D-CNN) model. However, the evaluation of this model, as well as other ATB segmentation techniques reported in the literature, is done using Dynamic Time Warping (DTW) distance between the entire original and predicted contours. Such an evaluation measure may not capture local errors in the predicted contour. Careful analysis of predicted contours reveals errors in regions like the velum part of contour1 (ATB comprising of upper lip, hard palate, and velum) and tongue base section of contour2 (ATB covering jawline, lower lip, tongue base, and epiglottis), which are not captured in a global evaluation metric like DTW distance. In this work, we automatically detect such errors and propose a correction scheme for the same. We also propose two new evaluation metrics for ATB segmentation separately in contour1 and contour2 to explicitly capture two types of errors in these contours. The proposed detection and correction strategies result in an improvement of these two evaluation metrics by 61.8% and 61.4% for contour1 and by 67.8% and 28.4% for contour2. Traditional DTW distance, on the other hand, improves by 44.6% for contour1 and 4.0% for contour2.

The SegNet model has been shown to provide the best performance in air-tissue boundary (ATB) segmentation in real-time Magnetic Resonance Imaging (rtMRI) videos in seen subject conditions. The SegNet model uses overall binary cross entropy as the loss function. However, such a global loss function does not give enough emphasis on regions which are more prone to errors. In this work, together with global loss, we explore the use of regional loss functions which focus on areas of the contours which have been analysed as error prone in the past. Evaluation is done using global Dynamic Time Warping (DTW) distance as well as regional metrics. The regional metrics used are EVEL and VELrDTW for contour1, and ETB and TBrDTW for contour2. We show that using such combinations of regional and global losses improves the regional, as well as global, evaluation metrics. For the best combination of losses, the two regional metrics show an improvement of 37.2 and 25.3 for contour1 and 23.9 and 28.4 for contour2, over a baseline model which uses only global loss. Global DTW distance, on the other hand, improves by 11.2 for contour1 and 5.6 for contour2.