Shared autonomy combines human user and AI copilot actions to control complex systems such as robotic arms. When a task is challenging, requires high dimensional control, or is subject to corruption, shared autonomy can significantly increase task performance by using a trained copilot to effectively correct user actions in a manner consistent with the user’s goals. To significantly improve the performance of shared autonomy, we introduce Diffusion Sequence Copilots (DiSCo): a method of shared autonomy with diffusion policy that plans action sequences consistent with past user actions. DiSCo seeds and inpaints the diffusion process with user-provided actions with hyperparameters to balance conformity to expert actions, alignment with user intent, and perceived responsiveness. We demonstrate that DiSCo substantially improves task performance in simulated driving and robotic arm tasks. Project website: https://sites.google.com/view/disco-shared-autonomy/

Multimodal deep learning systems are deployed in dynamic scenarios due to the robustness afforded by multiple sensing modalities. Nevertheless, they struggle with varying compute resource availability (due to multi-tenancy, device heterogeneity, etc.) and fluctuating quality of inputs (from sensor feed corruption, environmental noise, etc.). Statically provisioned multimodal systems cannot adapt when compute resources change over time, while existing dynamic networks struggle with strict compute budgets. Additionally, both systems often neglect the impact of variations in modality quality. Consequently, modalities suffering substantial corruption may needlessly consume resources better allocated towards other modalities. We propose ADMN, a layer-wise Adaptive Depth Multimodal Network capable of tackling both challenges - it adjusts the total number of active layers across all modalities to meet compute resource constraints, and continually reallocates layers across input modalities according to their modality quality. Our evaluations showcase ADMN can match the accuracy of state-of-the-art networks while reducing up to 75% of their floating-point operations.

Unsupervised Domain Adaptation (UDA) leverages labeled data from the source domain to align the target domain distribution, yet its performance is constrained by the lack of supervision in the target domain, resulting in a significant performance gap compared to fully supervised methods. Semi-Supervised Learning (SSL) combines limited labeled data with abundant unlabeled data but assumes identical distributions between labeled and unlabeled data, failing to address cross-domain challenges in medical imaging. To address these limitations, this paper proposes a Semi-Supervised Domain Adaptation (SSDA) framework based on Local-Global Hybridization and Dual-Teacher Collaborative Distillation, aiming to enhance the robustness and accuracy of medical image segmentation. The main contributions are summarized as follows:(1) FMix is introduced to utilize frequency-domain information for generating mask regions simulating real organ structures, enabling the model to focus on local features;(2) Mixup is incorporated to enhance domain invariance by simulating inter-domain transitions, guiding the model to prioritize global features;(3) A complementary dual-teacher distillation model is constructed, leveraging FMix-Mixup co-enhanced training to synergize local-global feature learning;(4) Monte Carlo Dropout is adopted to filter low-confidence pseudo-labels and iteratively update the target-domain labeled dataset. Extensive experiments on the BraTS2018 benchmark dataset demonstrate that the proposed framework significantly improves cross-modality medical image segmentation performance. With only one labeled target sample, it achieves over 10% improvement in Dice coefficient. Compared to state-of-the-art SSDA methods, the framework achieves a 4.7% higher Dice score, approaching the performance of fully supervised learning.