Most existing methods for unsupervised domain adaptation focus on learning domain-invariant representations. However, recent works have shown that the generalization on the target domain can fail due to the trade-off between marginal distribution alignment and joint error under a large domain shift. A few labeled target data points can enhance adaptation quality, but the distribution shift between labeled and unlabeled target data is often overlooked. Therefore, we propose a novel learning theory to address the joint error in semi-supervised domain adaptation that can reduce the mutual distribution shift between pairs from labeled and unlabeled domains. Furthermore, we introduce a discrepancy measurement between hypotheses to tackle the inconsistency of the loss functions in the algorithm and theory. Extensive experiments demonstrate that our method consistently outperforms baseline approaches, particularly in scenarios with large domain shifts and scarce labeled target data.

Existing domain adaptation systems can hardly be applied to real-world problems with new classes presenting at deployment time, especially regarding source-free scenarios where multiple source domains do not share the label space despite being given a few labeled target data. To address this, we consider a challenging problem: multi-source semi-supervised open-set domain adaptation and propose a learning theory via joint error, effectively tackling strong domain shift. To generalize the algorithm into source-free cases, we introdcue a computationally efficient and architecture-flexible attention-based feature generation module. Extensive experiments on various data sets demonstrate the significant improvement of our proposed algorithm over baselines.

Unsupervised domain adaptation (UDA) tries to overcome the tedious work of labeling data by leveraging a labeled source dataset and transferring its knowledge to a similar but different target dataset. Meanwhile, current vision-language models exhibit remarkable zero-shot prediction capabilities. In this work, we combine knowledge gained through UDA with the inherent knowledge of vision-language models. We introduce a strong-weak guidance learning scheme that employs zero-shot predictions to help align the source and target dataset. For the strong guidance, we expand the source dataset with the most confident samples of the target dataset. Additionally, we employ a knowledge distillation loss as weak guidance. The strong guidance uses hard labels but is only applied to the most confident predictions from the target dataset. Conversely, the weak guidance is employed to the whole dataset but uses soft labels. The weak guidance is implemented as a knowledge distillation loss with (adjusted) zero-shot predictions. We show that our method complements and benefits from prompt adaptation techniques for vision-language models. We conduct experiments and ablation studies on three benchmarks (OfficeHome, VisDA, and DomainNet), outperforming state-of-the-art methods. Our ablation studies further demonstrate the contributions of different components of our algorithm.

Unsupervised domain adaptation (UDA) tries to overcome the need for a large labeled dataset by transferring knowledge from a source dataset, with lots of labeled data, to a target dataset, that has no labeled data. Since there are no labels in the target domain, early misalignment might propagate into the later stages and lead to an error build-up. In order to overcome this problem, we propose a gradual source domain expansion (GSDE) algorithm. GSDE trains the UDA task several times from scratch, each time reinitializing the network weights, but each time expands the source dataset with target data. In particular, the highest-scoring target data of the previous run are employed as pseudo-source samples with their respective pseudo-label. Using this strategy, the pseudo-source samples induce knowledge extracted from the previous run directly from the start of the new training. This helps align the two domains better, especially in the early training epochs. In this study, we first introduce a strong baseline network and apply our GSDE strategy to it. We conduct experiments and ablation studies on three benchmarks (Office-31, OfficeHome, and DomainNet) and outperform state-of-the-art methods. We further show that the proposed GSDE strategy can improve the accuracy of a variety of different state-of-the-art UDA approaches.

The requirement for large labeled datasets is one of the limiting factors for training accurate deep neural networks. Unsupervised domain adaptation tackles this problem of limited training data by transferring knowledge from one domain, which has many labeled data, to a different domain for which little to no labeled data is available. One common approach is to learn domain-invariant features for example with an adversarial approach. Previous methods often train the domain classifier and label classifier network separately, where both classification networks have little interaction with each other. In this paper, we introduce a classifier-based backprop-induced weighting of the feature space. This approach has two main advantages. Firstly, it lets the domain classifier focus on features that are important for the classification, and, secondly, it couples the classification and adversarial branch more closely. Furthermore, we introduce an iterative label distribution alignment method, that employs results of previous runs to approximate a class-balanced dataloader. We conduct experiments and ablation studies on three benchmarks Office-31, Office-Home, and DomainNet to show the effectiveness of our proposed algorithm.

Video prediction is a complicated task as countless possible future frames exist that are equally plausible. While recent work have made progress in the prediction and generation of future video frames, these work have not attempted to disentangle different features of videos such as an object’s structure and its dynamics. Such a disentanglement would allow one to control these aspects to some extent in the prediction phase, while at the same time maintain the object’s intrinsic properties that are learned as the model’s internal representation. In this work, we propose Ladder Variational Recurrent Neural Networks (LVRNN). We employ a type of ladder autoencoder shown to be effective for feature disentanglement on images and apply it to the Variational Recurrent Neural Network (VRNN) architecture, which has been used for video prediction. We rely on extracted keypoints in each frame to separate the structure from the visual features. We then show how different levels of the ladder network learn to disentangle features and demonstrate that each of these levels can be used for controlling different aspects of future frames such as structure and dynamics. We evaluate our method on the Human3.6M and BAIR robot datasets. We show that our method is able to perform hierarchical disentanglement, yet provide reasonable results compared to similar methods.

Source-free Domain Adaptation (SFDA) aims to adapt a model trained on a given (source) environment to the new (target) environment, without directly accessing the source data. Due to the lack of labeled source data, it is often difficult for SFDA methods to provide reliable class representations for the target data. To overcome this issue, we propose the idea of Confidence-based Subsets Feature Alignment (CSFA). CSFA divides the target data into two subsets: confident subset that consists of samples having low entropy class predictions from the source model, and non-confident subset with samples that do not. By using the pseudo-labels from the confident subset, we can frame the original SFDA problem as a Universal Domain Adaptation (UniDA) problem, and provide reliable class representations for the target data by aligning feature distributions of the two subsets. Specifically, we propose a multi-task framework that simultaneously applies a standard SFDA algorithm in combination with a UniDA-inspired algorithm, which further infuses class representations into the adaption process. We evaluate the proposed method on a wide range of cross-domain object recognition tasks and achieve higher or comparable accuracy compared to existing SFDA methods. Ablation studies are conducted to verify the effectiveness of the proposed method.

Backhoe loads sediment onto the bed of dump trucks during earthmoving work. The prediction of backhoe loading time is essential for ensuring safe cooperation between the backhoe and dump trucks. However, it is difficult to predict the instant at which the backhoe is ready to load sediment, because of the similarity in motions observed during gathering sediment. Moreover, since operators have different skill levels, the prediction requires a unique model for each operator. In this study, we attempt to predict the instant at which the backhoe is ready to load sediment into the dump truck. For this purpose, the beta-process hidden Markov model (BP-HMM) is employed to build a backhoe motion model for a specific operator. Time series data of backhoe loading motions for crushed rocks and wood chips, which were measured using 6-axis inertial measurement unit (IMU) sensors equipped at the cab, boom, and arm of the backhoe, were used for modeling with the BP-HMM. Several primitive motions of the backhoe, which occur at the completion of preparation before the loading process begins, were discovered as a result of the motion modeling based on the BP-HMM. We developed the prediction of the instant using three primitive motions. At best, the proposed method could predict the instant with a probability of 67% and 100%, at 6.0 s and 0.7 s before the loading motions began, respectively. This phased prediction can be used to reduce the idle time and risk for dump trucks during earthmoving work with the backhoe.

In large industrial plants, the inspection of production lines is a heavy and costly task that puts human inspectors at high risk. In order to overcome these challenges, we have developed an autonomous plant inspection system using a mobile tracked vehicle. In this paper, we propose an autonomous navigation method for tracked vehicles based on a contact force model that enables the robot to compensate for collisions with obstacles. The model considers the influence of the contact force on the linear and angular motion of the robot. Using the model, the controllable velocity range is derived during collisions. The experimental results show that the robot is safely controlled by complying with velocity constraints. In addition, our method can generate motions such as leaving wall, L-shaped curve and crosswise locomotion in straight passage while navigation alongside the walls. The method allows the robot to smoothly follow a target path, despite colliding with obstacles.

Perception is the main key in enabling robots to react to and interact with their environment. Particularly, for multi-floor operations, the robot must robustly detect and localize stairs to allow for safe climbing. In this paper, we develop a graph-based stairway detection method for point cloud data, that can detect a large variety of stairways. Our approach first segments planar regions and extracts the stair tread- and stair riser-shaped segments. With these segments, a dynamic graph model is initialized that is used to detect stairs including the railing system in the surroundings. We show that our system can accurately detect and localize different stairways from a variety of different positions, including descending stairs. Our system's accuracy is higher than those of most state-of-the-art stairway detection methods even in case of sparse point cloud data.