Previous continual learning setups for embodied intelligence focused on executing low-level actions based on human commands, neglecting the ability to learn high-level planning and multi-level knowledge.To address these issues, we propose the Hierarchical Embodied Continual Learning Setups (HEC) that divide the agent's continual learning process into two layers: high-level instructions and low-level actions, and define five embodied continual learning sub-setups.Building on these setups, we introduce the Task-aware Mixture of Incremental LoRA Experts (Task-aware MoILE) method.This approach achieves task recognition by clustering visual-text embeddings and uses both a task-level router and a token-level router to select the appropriate LoRA experts.To effectively address the issue of catastrophic forgetting, we apply Singular Value Decomposition (SVD) to the LoRA parameters obtained from prior tasks, preserving key components while orthogonally training the remaining parts.The experimental results show that our method stands out in reducing the forgetting of old tasks compared to other methods, effectively supporting agents in retaining prior knowledge while continuously learning new tasks.

Integrating complementary strengths of raw data and logical rules to improve the learning generalization has been recently shown promising and effective, e.g., abductive learning is one generic framework that can learn the perception model from data and reason between rules simultaneously. However, the performance would be seriously decreased when inaccurate logical rules appear, which may be even worse than baselines using only raw data. Efforts on this issue are highly desired while remain to be limited. This paper proposes a simple and effective safe abductive learning method to alleviate the harm caused by inaccurate rules. Unlike the existing methods which directly use all rules without correctness checks, it utilizes them selectively by constructing a graphical model with an adaptive reasoning process to prevent performance hazards. Theoretically, we show that induction and abduction are mutually beneficial, and can be rigorously justified from a classical maximum likelihood estimation perspective. Experiments on diverse tasks show that our method can tolerate at least twice as many inaccurate rules as accurate ones and achieve highly competitive performance while other methods can't. Moreover, the proposal can refine inaccurate rules and works well in extended weakly supervised scenarios.

Abductive learning models (ABL) and neural-symbolic predictive models (NeSy) have been recently shown effective, as they allow us to infer labels that are consistent with some prior knowledge by reasoning over high-level concepts extracted from sub-symbolic inputs. However, their generalization ability is affected by reasoning shortcuts: high accuracy on given targets but leveraging intermediate concepts with unintended semantics. Although there have been techniques to alleviate reasoning shortcuts, theoretical efforts on this issue remain to be limited. This paper proposes a simple and effective analysis to quantify harm caused by it and how can mitigate it. We quantify three main factors in how NeSy algorithms are affected by reasoning shortcuts: the complexity of the knowledge base, the sample size, and the hypothesis space. In addition, we demonstrate that ABL can reduce shortcut risk by selecting specific distance functions in consistency optimization, thereby demonstrating its potential and approach to solving shortcut problems. Empirical studies demonstrate the rationality of the analysis. Moreover, the proposal is suitable for many ABL and NeSy algorithms and can be easily extended to handle other cases of reasoning shortcuts.

Learning-based multi-view stereo (MVS) is gaining prominence as a method for 3D reconstruction. However, existing methods in the process of feature learning fail to focus on the structural information implied in the scene. This oversight prevents the network from perceiving the geometric properties of the scene and weakens the generalizability of the network. Therefore, we propose a novel framework named Point Feature Relation Network for Multi-view Stereo (PFR-MVSNet), which is composed of a Dynamic Structure Perception (DSP) module, an Adaptive Feature Exploration (AFE) module, and a Point Transformer Block (PTB) module, to solve the problems caused by the oversight. The DSP module first augments the feature of the 3D point cloud from multi-view 2D features, then establishes spatial structure relations within local regions on the point cloud and guides the feature learning of points through the aggregated structure information. After the network has fully learned the intra-region structure features, the AFE module repartitions perception regions with similar features. The point features within the regions are further learned by the PTB module. We evaluate our method on three benchmark datasets: DTU, Tanks & Temples, and ETH3D. The experimental results show that our method achieves superior accuracy of 0.289 mm on the DTU dataset and exhibits more robust generalization on the Tanks & Temples and ETH3D datasets compared with other learning-based MVS methods.

Thanks to the recent breakthroughs in deep learning (DL) methods and ever-increasing computation power, nowadays DL models are being increasingly applied to all kinds of fields. They, along with other software modules, compose complex DL systems such as autonomous driving systems. Unfortunately, fatalities happen to these systems as is reported in real-life situations, e.g., traffic accidents involving autonomous driving vehicles. Further analyses show that this is because DL systems contain defects. To this end, understanding defects in DL systems is critical for preventing such accidents. To help improve the reliability of DL systems, many researchers have devoted efforts to their testing. In this paper, we investigated state-of-the-art methods through in-depth exploration. Firstly, we made a detailed analysis of deep learning system defects. We proposed a defect classification scheme, and summarized the characteristics of defects and their impacts on the system. Secondly, we systematically investigated the detection methods and tools and evaluated their capability in defect detection.

This work discusses how to achieve worst-case Out-Of-Distribution (OOD) generalization for a variety of distributions based on a relatively small labeling cost. The problem has broad applications, especially in non-i.i.d. open-world scenarios. Previous studies either rely on a large amount of labeling cost or lack of guarantees about the worst-case generalization. In this work, we show for the first time that active model adaptation could achieve both good performance and robustness based on the invariant risk minimization principle. We propose \textsc{Log}, an interactive model adaptation framework, with two sub-modules: active sample selection and causal invariant learning. Specifically, we formulate the active selection as a mixture distribution separation problem and present an unbiased estimator, which could find the samples that violate the current invariant relationship, with a provable guarantee. The theoretical analysis supports that both sub-modules contribute to generalization. A large number of experimental results confirm the promising performance of the new algorithm.