Computational complexity of Bayesian learning is impeding its adoption in practical, large-scale tasks, despite demonstrations of significant merits such as improved robustness and resilience to unseen or out-of-distribution inputs over their non-Bayesian counterparts. Although, Deep ensemble methods (Seligmann et al. 2024; Lakshminarayanan, Pritzel, and Blundell 2017) have proven to be highly effective for Bayesian deep learning, their practical application is hindered by substantial computational cost. In this study, we introduce an innovative framework to mitigate the computational burden of ensemble Bayesian deep learning. We explore a more feasible alternative, inspired by the recent success of low-rank adapters, we introduce Bayesian Low-Rank LeArning (Bella). We show, i) Bella achieves a dramatic reduction in the number of trainable parameters required to approximate a Bayesian posterior; and ii) it not only maintains, but in some instances, surpasses the performance–in accuracy and out-of-distribution generalisation–of conventional Bayesian learning methods and non-Bayesian baselines. Our extensive empirical evaluation in large-scale tasks such as ImageNet, CAMELYON17, DomainNet, VQA with CLIP, LLaVA demonstrate the effectiveness and versatility of Bella in building highly scalable and practical Bayesian deep models for real-world applications.

Social intelligence is essential for understanding complex human expressions and social interactions.While large multimodal models (LMMs) have demonstrated remarkable performance in social intelligence question answering (SIQA), they are still inclined to generate responses relying on language priors and ignoring the relevant context due to the dominant prevalence of text-based data in the pretraining stage.To interpret the aforementioned language bias of LMMs, we employ a structure causal model and posit that counterfactual reasoning can mitigate the bias by avoiding spurious correlations between LMMs' internal commonsense knowledge and the given context.However, it is costly and challenging to construct multimodal counterfactual samples.To tackle the above challenges, we propose an output Distribution Calibration network with Virtual Counterfactual (DCVC) data augmentation framework.DCVC devises a novel output distribution calibration network to mitigate the impact of negative language biases while preserving beneficial priors.Perturbations are introduced to the output distributions of LMMs to simulate the distribution shifts from counterfactual manipulations of the context, which is employed to construct counterfactual augmented data virtually.Experiments on multiple datasets demonstrate the effectiveness and generalizability of our proposed method.

Large Multimodal Models (LMMs) have achieved strong performance across a range of vision and language tasks.However, their spatial reasoning capabilities are underinvestigated.In this paper, we construct a novel VQA dataset, Spatial-MM, to comprehensively study LMMs' spatial understanding and reasoning capabilities.Our analyses on object-relationship and multi-hop reasoning reveal several important findings.Firstly, bounding boxes and scene graphs, even synthetic ones, can significantly enhance LMMs' spatial reasoning.Secondly, LMMs struggle more with questions posed from the human perspective than the camera perspective about the image.Thirdly, chain of thought (CoT) prompting does not improve model performance on complex multi-hop questions involving spatial relations.Lastly, our perturbation analysis on GQA-spatial reveals that LMMs are much stronger at basic object detection than complex spatial reasoning.We believe our new benchmark dataset and in-depth analyses can spark further research on LMMs spatial reasoning.1