Matching images and sentences demands a fine understanding of both modalities. In this article, we propose a new system to discriminatively embed the image and text to a shared visual-textual space. In this field, most existing works apply the ranking loss to pull the positive image/text pairs close and push the negative pairs apart from each other. However, directly deploying the ranking loss on heterogeneous features (i.e., text and image features) is less effective, because it is hard to find appropriate triplets at the beginning. So the naive way of using the ranking loss may compromise the network from learning inter-modal relationship. To address this problem, we propose the instance loss, which explicitly considers the intra-modal data distribution. It is based on an unsupervised assumption that each image/text group can be viewed as a class. So the network can learn the fine granularity from every image/text group. The experiment shows that the instance loss offers better weight initialization for the ranking loss, so that more discriminative embeddings can be learned. Besides, existing works usually apply the off-the-shelf features, i.e., word2vec and fixed visual feature. So in a minor contribution, this article constructs an end-to-end dual-path convolutional network to learn the image and text representations. End-to-end learning allows the system to directly learn from the data and fully utilize the supervision. On two generic retrieval datasets (Flickr30k and MSCOCO), experiments demonstrate that our method yields competitive accuracy compared to state-of-the-art methods. Moreover, in language-based person retrieval, we improve the state of the art by a large margin. The code has been made publicly available.

JPL has developed the worlds first rock climbing robot. This video presents climbing trials at vertical, overhanging, and inverted angles, and a zero-g drill for astronauts.

Lemur IIa, the second-generation Limbed Excursion Mechanical Utility Robot, is a six-limbed walking robot developed at NASA' s Jet Propulsion Laboratory. Lemur is distinct in part because each limb was designed be used for both walking and manipulation. It is intended to be a prototype of a dexterous, operationally flexible robot for on-orbit assembly, inspection, and maintenance. This video illustrates some of the current capabilities of this platform. Demonstrations included are omnidirectional walking without rotating the robot, body pose control from a fixed stance, visually guided docking with a recharging station, and manipulation and inspection with multiple tools (on multiple limbs). These capabilities represent a sample of the current tasks Lemur is performing. Please see www-robotics.jpl.nasa.gov for more information and demonstrations about Lemur.

Lemur IIa, the second-generation Limbed Excursion Mechanical Utility Robot, is a six-limbed walking robot developed at NASA' s Jet Propulsion Laboratory. Lemur is distinct in part because each limb was designed be used for both walking and manipulation. It is intended to be a prototype of a dexterous, operationally flexible robot for on-orbit assembly, inspection, and maintenance. This video illustrates Lemur completing an autonomous assembly task. Lemur, as part of a robotic team, is helping to assemble a structure. In this demonstration, Lemur autonomously walks through a truss, locates a task board, and tightens a bolt on that board. These demonstrations represent a sample of the current tasks Lemur is performing. Please see www-robotics.jpl.nasa.gov for more information and demonstrations about Lemur.

Exploration of high risk terrain areas such as cliff faces and site construction operations by autonomous robotic systems on Mars requires a control architecture that is able to autonomously adapt to uncertainties in knowledge of the environment. We report on the development of a software/hardware framework for cooperating multiple robots performing such tightly coordinated tasks. This work builds on our earlier research into autonomous planetary rovers and robot arms. Here, we seek to closely coordinate the mobility and manipulation of multiple robots to perform examples of a cliff traverse for science data acquisition, and site construction operations including grasping, hoisting, and transport of extended objects such as large array sensors over natural, unpredictable terrain. In support of this work we have developed an enabling distributed control architecture called control architecture for multirobot planetary outposts (CAMPOUT) wherein integrated multirobot mobility and control mechanisms are derived as group compositions and coordination of more basic behaviors under a task-level multiagent planner. CAMPOUT includes the necessary group behaviors and communication mechanisms for coordinated/cooperative control of heterogeneous robotic platforms. In this paper, we describe CAMPOUT, and its application to ongoing physical experiments with multirobot systems at the Jet Propulsion Laboratory in Pasadena, CA, for exploration of cliff faces and deployment of extended payloads.

We have developed a cliff robot that is capable of descending a cliff and autonomously navigating to way-points on the cliff wall. This aggressive mobility system consists of an ensemble of three tethered robots, which cooperate under tight coordinated control and collective state estimation. The distributed task is described as a behavior network, which consists of a network of controllers spread across the robots and which interact through communication links to achieve a collective control objective. Fielded experimental results show that the cliff robot is capable of navigating to designated way-points on a cliff wall using the proposed control scheme.