We present a field deployment of a collaborative scientist-robot system that enables improved scientific gain for planetary science missions. In this system, the scientist interacts with the autonomy through a computer interface to specify prior disciplinary knowledge, hypothesis information, and refine the mission objectives using preferences and ratings. The goal of this interaction is to use autonomy where it works best, optimizing robot data collection paths under a set of constraints, while enabling scientists to communicate priors and objectives in a straightforward and efficient manner. The system was deployed with a quadruped during a planetary analog field science mission at White Sands National Park, New Mexico, to support the investigation of the surface and shallow subsurface properties of the Martian analog dunes. The system was used to generate robot paths for three different mission scientists, who had varying priors and objectives. A quadruped robot executed the path providing data on the stiffness of the surface measured through ground-leg interactions. The generated dense stiffness measurements were used by scientists to select refined locations for follow-up data collection. While, we found that more work is needed to fully incorporate human priors, our deployed system enabled scientists to collect mission-critical data more efficiently, and they were able to adapt the system to their science needs.

Traditional Multi-View Stereo (MVS) methods, which rely on hand-crafted features, often fail to capture overall scene structures, particularly under varying lighting conditions, in low-textured regions, or in reflective areas. Recent learning-based approaches, although demonstrating state-of-the-art (SOTA) performance, still face generalization challenges due to their reliance on fine-tuning. In this paper, we propose SF-MVSNet to mitigates this issue. SF-MVSNet enhances the diversity and isolation of features by introducing a Siamese-like Fusion network in the depth estimation stage, thereby improving generalizability. More specifically, we divide the correlation volume workflow into two parallel branches with shared weights, which can be viewed as group-wise correlation, providing unique insights for measuring feature similarities. Then, we introduce a Three-dimensional Attentional Feature Fusion network (3D AFF) to combine the similarities and differences. This process augments the network’s capability to handle intricate scenes, which is particularly important when confronted with the unseen and more challenging dataset such as the Tanks and Temples benchmarks. Experimental results demonstrate that our method achieves competitive results compared with recent SOTA approaches. Results can be found on the Tanks and Temples official evaluation website leaderboard section. Ablation studies further validate the efficiency of the designed module. Code is available at https://github.com/AsDeadAsAD-odo/SF-MVSNet.

Knowledge distillation is a model compression method that transforms complex models into efficient ones. Traditional distillation, with two-stage training, is time-consuming and computationally expensive. Self-distillation methods can alleviate this deficiency by adopting a one-stage strategy. However, most of them lack a meticulously designed auxiliary network, resulting in the learned knowledge being simplistic and insufficient, which limits the ability of distillation. To address this challenge, we propose a training framework named deep-supervised Knowledge Enhancement Self-Distillation (KED) which organizes the teacher model hierarchically through stacking auxiliary classifiers after each shallow block to form the student models. Specifically, two modules, i.e., logit decouple distillation (LDD) and attention map generation (AT-Gen), are embedded in the framework to enhance the distillation knowledge. LDD divides the output of each classifier (logits) into target and non-target classes, making the knowledge from outputs more flexible and efficient. Moreover, AT-Gen extracts attention maps from the features of each block, emphasizing the knowledge from intermediate layers through attention maps' integration and interaction. Finally, knowledge from different sources works together to guide the training of the student models, compensating for the bias of the auxiliary networks. Experiments on public datasets demonstrate that our method outperforms other state-of-the-art methods.

Our analysis of human sampling decision data reveals that scientists adapt their sampling strategies to balance multiple objectives based on two key factors: the current level of information about the environment, and the availability of sampling location options with large potential rewards. While this work is only a beginning step towards the development of cognitive-compatible robotic decision algorithms, our findings show by better understanding human decision processes, robots can use extremely simple algorithms to connect experts' high-level objectives to desired sampling locations while balancing multiple objectives. Going forward, exploring how humans coordinate and prioritize multiple objectives under more sophisticated scientific exploration scenarios, such as with multiple competing hypotheses, with hypotheses regarding multiple variables, or with additional sampling objectives, would be helpful to explore. These understandings could help our robots produce explainable sampling strategies that are well-aligned with humans' high level goals, and improve humans' trust and confidence during teaming. These cognitive understandings could also allow robots to identify potential vulnerabilities in human decisions, such as biases and fatigue, and provide targeted support to enhance scientific outcomes. In addition, we expect that these cognitive insights could complement existing robotic decision methods by informing which algorithms to use, and eventually empower robots to become intelligent teammates that can truly participate in the decision-making process.

Truly collaborative scientific field data collection between human scientists and autonomous robot systems requires a shared understanding of the search objectives and tradeoffs faced when making decisions. Therefore, critical to developing intelligent robots to aid human experts is an understanding of how scientists make such decisions and how they adapt their data collection strategies when presented with new information in situ . In this study, we examined the dynamic data collection decisions of 108 expert geoscience researchers using a simulated field scenario. Human data collection behaviors suggested two distinct objectives: an information-based objective to maximize information coverage and a discrepancy-based objective to maximize hypothesis verification. We developed a highly simplified quantitative decision model that allows the robot to predict potential human data collection locations based on the two observed human data collection objectives. Predictions from the simple model revealed a transition from information-based to discrepancy-based objective as the level of information increased. The findings will allow robotic teammates to connect experts’ dynamic science objectives with the adaptation of their sampling behaviors and, in the long term, enable the development of more cognitively compatible robotic field assistants.

While videoconferencing is prevalent, concurrent participation channels are limited. People experience challenges keeping up with the discussion, and misunderstanding frequently occurs. Through a formative study, we probed into the design space of providing real-time transcripts as an extra communication space for video meeting attendees. We then present MeetScript, a system that provides parallel participation channels through real-time interactive transcripts. MeetScript visualizes the discussion through a chat-alike interface and allows meeting attendees to make real-time collaborative annotations. Over time, MeetScript gradually hides extraneous content to retain the most essential information on the transcript, with the goal of reducing the cognitive load required on users to process the information in real time. In an experiment with 80 users in 22 teams, we compared MeetScript with two baseline conditions where participants used Zoom alone (business-as-usual), or Zoom with an adds-on transcription service (Otter.ai). We found that MeetScript significantly enhanced people's non-verbal participation and recollection of their teams' decision-making processes compared to the baselines. Users liked that MeetScript allowed them to easily navigate the transcript and contextualize feedback and new ideas with existing ones.