Hamed Damirchi, Imezadelajara, Ehsan Abbasnejad, Afshar Shamsi, Zhen Zhang, Javen Qinfeng Shi. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Contrastive Language-Image Pretraining (CLIP) stands out as a prominent method for image representation learning. Various architectures, from vision transformers~(ViTs) to convolutional networks (ResNets) have been trained with CLIP to serve as general solutions to diverse vision tasks. This paper explores the differences across various CLIP-trained vision backbones. Despite using the same data and training objective, we find that these architectures have notably different representations, different classification performance across datasets, and different robustness properties to certain types of image perturbations. Our findings indicate a remarkable possible synergy across backbones by leveraging their respective strengths. In principle, classification accuracy could be improved by over 40 percentage with an informed selection of the optimal backbone per test example. Using this insight, we develop a straightforward yet powerful approach to adaptively ensemble multiple backbones. The approach uses as few as one labeled example per class to tune the adaptive combination of backbones. On a large collection of datasets, the method achieves a remarkable increase in accuracy of up to 39.1\% over the best single backbone, well beyond traditional ensembles.

Conventionally, deep odometry networks use objective functions that only penalize short-term deviations from the true path. Since such an objective does not impose any constraints on the long-term deviations from the path, a second consistency-based loss term may be added to lower long-term drift. However, maintaining a balance between the two loss terms is challenging and often treated as a design hyperparameter. To mitigate this balancing issue, we propose to use the uncertainty over both odometry and the long-term transformations in a maximum likelihood setting and allow the network to tune the weighting between the two loss terms. To this end, we derive the odometry uncertainty alongside the pose outputs using the network itself and to derive the covariance matrix over the integrated transformation, we propose to propagate the odometry uncertainty through each iteration. This formulation provides an adaptive and statistically consistent method to weigh the incremental and integrated loss terms against each other, noting the increase in uncertainty as more steps are integrated over. We show that our approach to consistency-based losses allows the network to surpass the accuracy of the state-of-the-art visual odometry approaches. Then, the efficacy of the derived uncertainty as weighting medium is visualized and the performance benefits of uncertainty quantification are shown in a pose-graph based localization scenario.

Ease of calibration and high-accuracy task-space state-estimation purely based on onboard sensors is a key requirement for enabling easily deployable cable robots in real-world applications. In this work, we incorporate the onboard camera and kinematic sensors to drive a statistical fusion framework that presents a unified localization and calibration system which requires no initial values for the kinematic parameters. This is achieved by formulating a Monte-Carlo algorithm that initializes a factor-graph representation of the calibration and localization problem. With this, we are able to jointly identify both the kinematic parameters and the visual odometry scale alongside their corresponding uncertainties. We demonstrate the practical applicability of the framework using our state-estimation dataset recorded with the ARAS-CAM suspended cable driven parallel robot, and published as part of this manuscript.

Human Activity Recognition (HAR) is a crucial factor in assisted living systems and elderly care solutions where the activity of the subject can be used to ensure the safety of the elderly and provide more efficient services. HAR is a challenging problem that needs advanced solutions than using handcrafted features to achieve a desirable performance. Deep learning has been proposed as a solution to obtain more accurate HAR systems being robust against noise. In this paper, we introduce ARC-Net and propose the utilization of capsules to fuse the information from multiple inertial measurement units (IMUs) to predict the activity performed by the subject. We hypothesize that this network will be able to tune out the unnecessary information and will be able to make more accurate decisions through the iterative mechanism embedded in capsule networks. We provide heatmaps of the priors, learned by the network, to visualize the utilization of each of the data sources by the trained network. Then, gradient based interpretations are provided and further discussed. By using the proposed network, we were able to increase the accuracy of the state-of-the-art approaches by 2%. Furthermore, we investigate the directionality of the confusion matrices of our results and discuss the specificity of the activities based on the provided data.