This paper proposes a novel dynamic model and a locomotion framework based on it for nonholonomic wheeled mobile robots (NWMR) with a general configuration. The general configuration is established on a nonholonomically constrained wheeled mobile platform driven by two motors, into which a 3-DoFs waist-leg mechanism, a torso, and a 7-DoFs dual-arm system are integrated. Based on this configuration, a novel terrain-adaptive NWMR dynamics model (TAND) is formulated to operate on arbitrary inclined planes, which unifies the dynamic equations of NWMRs with those of conventional 6-DoFs floating-base robots. Consequently, traditional 3-DoFs floating-base NWMR dynamic models are encompassed as specific cases within the proposed TAND. Furthermore, by incorporating hierarchical optimization-based whole-body control (HOWBC) with TAND, a terrain-adaptive HOWBC (TAHC) is introduced, enabling the achievement of motion tracking, posture maintenance, and manipulability optimization (MTO) across varied terrains. For the MTO sub-task, a new whole-body manipulability index specifically designed for NWMR is proposed, and gradient optimization methods at the acceleration level are applied to enhance it. The proposed TAND, MTO, and TAHC are validated through comprehensive simulations, demonstrating their effectiveness in enabling coordinated locomotion across different terrains.

Multirobot exploration (MR-Exploration) is a primary task providing the location and map for many multirobot applications. To improve system performance, convolutional neural network (CNN) is introduced by recent researches into critical components in MR-Exploration, such as feature-point extraction (FE) and place recognition (PR). This CNN-based MR-Exploration needs to simultaneously run multiple CNN models and complex postprocessing algorithms. This significantly challenges the hardware platforms of embedded systems. Previous researches reveal that an FPGA is ideal for CNN processing on embedded platforms. Such accelerators usually process different models in sequence, while they cannot schedule multiple tasks at runtime. Furthermore, the postprocessing of CNNs is computationally intensive and becomes the bottleneck of the whole system. To handle such problems, we propose an interruptible CNN accelerator for multirobot exploration (INCAME) framework to rapidly deploy the robot applications on FPGAs. In INCAME, we propose an interrupt method based on virtual instructions to support multitasking on CNN accelerators. INCAME also includes hardware modules for accelerating the postprocessing of the CNN-based components. Organically, it integrates the postprocessing and CNN backbone by sharing memory. Experimental results reveal that INCAME enables multitask scheduling on the CNN accelerator with negligible performance degradation (0.3%). INCAME enables embedded FPGAs to perform MR-Exploration in real time (20 fps) via the multitask support and postprocessing acceleration.

Collaborative exploration in an unknown environment without external positioning under limited communication is an essential task for multi-robot applications. For inter-robot positioning, various Distributed Simultaneous Localization and Mapping (DSLAM) systems share the Place Recognition (PR) descriptors and sensor data to estimate the relative pose between robots and merge robots’ maps. As maps are constantly shared among robots in exploration, we design a map-based DSLAM framework, which only shares the submaps, eliminating the transfer of PR descriptors and sensor data. Our framework saves 30% of total communication traffic. For exploration, each robot is assigned to get much unknown information about environments with paying little travel cost. As the number of sampled points increases, the goal would change back and forth among sampled frontiers, leading to the downgrade in exploration efficiency and the overlap of trajectories. We propose an exploration strategy based on Multi-robot Multi-target Potential Field (MMPF), which can eliminate goal’s back-and-forth changes, boosting the exploration efficiency by 1.03 ×∼1.62 × with 3 % ∼ 40 % travel cost saved. Our SubMap-based Multi-robot Exploration method (SMMR-Explore) is evaluated on both Gazebo simulator and real robots. The simulator and the exploration framework are published as an open-source ROS project at https://github.com/efc-robot/SMMR-Explore.

In recent years, Convolutional Neural Network (CNN) has been widely used in robotics, which has dramatically improved the perception and decision-making ability of robots. A series of CNN accelerators have been designed to implement energy-efficient CNN on embedded systems. However, despite the high energy efficiency on CNN accelerators, it is difficult for robotics developers to use it. Since the various functions on the robot are usually implemented independently by different developers, simultaneous access to the CNN accelerator by these multiple independent processes will result in hardware resources conflicts.To handle the above problem, we propose an INterruptible CNN Accelerator (INCA) to enable multi-tasking on CNN accelerators. In INCA, we propose a Virtual-Instruction-based interrupt method (VI method) to support multi-task on CNN accelerators. Based on INCA, we deploy the Distributed Simultaneously Localization and Mapping (DSLAM) on an embedded FPGA platform. We use CNN to implement two key components in DSLAM, Feature-point Extraction (FE) and Place Recognition (PR), so that they can both be accelerated on the same CNN accelerator. Experimental results show that, compared to the layer-by-layer interrupt method, our VI method reduces the interrupt respond latency to 1%.

Feature-point extraction is a fundamental step in many applications, such as image matching and Simultaneous Localization and Mapping (SLAM). The CNN-based feature-point extraction methods have made significant signs of progress in both feature-point detection and descriptor generation compared with handcrafted processes. However, the computational and storage complexity makes it difficult for CNN to run on real-time embedded systems. In this paper, we aim to deploy the advanced CNN-based feature-point extraction methods onto real-time embedded FPGA systems. We optimize the softmax data flow so that the computation of softmax and NMS can be reduced by 64×. We generate the normalized descriptors after picking the feature-points with the highest confidence so that the computation cost of normalization is reduced by 1500×. We use fixed-point in both of the CNN backbone and the postprocessing operations, and implement them on the ZCU102 FPGA platform. The experimental results show that our proposed hardware-software co-design CNN-based feature-point extraction method outperforms the handcrafted techniques. Our feature-point extraction on the embedded platform runs at the speed of 20 fps, meeting the real-time requirement.

Multi-Robot Exploration (MR-Exploration) that provides the location and map is a basic task for many multi-robot applications. Recent researches introduce Convolutional Neural Network (CNN) to critical components in MR-Exploration, like Feature-point Extraction (FE) and Place Recognition (PR), to improve the system performance. Such CNN-based MR-Exploration requires running multiple CNN models simultaneously, together with complex post-processing algorithms, greatly challenges the hardware platforms, which are usually embedded systems. Previous researches have shown that FPGA is a good candidate for CNN processing on embedded platforms. But such accelerators usually process different models sequentially, lacking the ability to schedule multiple tasks at runtime. Furthermore, post-processing of CNNs in FE is also computation consuming and becomes the system bottleneck after accelerating the CNN models. To handle such problems, we propose an INterruptible CNN Accelerator for Multi-Robot Exploration (INCAME) framework for rapid deployment of robot applications on FPGA. In INCAME, we propose a virtual-instruction-based interrupt method to support multi-task on CNN accelerators. INCAME also includes hardware modules to accelerate the post-processing of the CNN-based components. Experimental results show that INCAME enables multi-task scheduling on the CNN accelerator with negligible performance degradation (0.3%). With the help of multi-task supporting and post-processing acceleration, INCAME enables embedded FPGA to execute MR-Exploration in real time (20 fps).

Sparse signal processing theory has been applied to SAR imaging. The estimation of sparsity is crucial for sparse SAR imaging. But the true value of sparsity is unknown. Adaptive parameter estimation for sparse SAR imaging can achieved by the automatic regularization parameter estimating methods. However, these methods are deduced based on measurement matrix, which will cause huge computational and memory costs. Also, the adaptive estimated sparsity is often greater than the true value due the noise and sidelobes. In this paper, we propose improved adaptive parameter estimation method for sparse SAR imaging. The complex-image-based sparse SAR imaging is adopted to pre-estimate the parameter. Then, azimuth-range decouple operators are introduced into parameter estimation method. Simulation and real data experimental results show the effectiveness of the proposed method.

Sparse signal processing has been applied in synthetic-aperture radar (SAR) imaging. As a typical sparse reconstruction model, L1regularization often underestimates the intensities of the targets. The estimated radar cross section (RCS) is related to the pixel intensity. Thus, the linear relationship between the targets' intensities cannot kept. The underestimation will also cause radiometric errors and affect the quantitative use of the SAR data. In this letter, we present a SAR imaging method based on generalized minimax concave (GMC) penalty. GMC is a nonconvex penalty and its cost function is convex. GMC can avoid the underestimation of pixel intensity. In the iteration, the azimuth-range decouple operators are used to avoid the huge memory and computational costs. The performance of the proposed method is verified using real data.

Accurate haptic feedback is a critical challenge for surgical robots, especially for flexible endoscopic surgical robots whose transmission systems are Tendon-Sheath Mechanisms (TSMs) with highly nonlinear friction profiles and force hysteresis. For distal end haptic sensing of TSMs, this paper, for the first time, proposes to measure the compression force on the sheath at the distal end so that the tension force on the tendon, which equals the compression force on the sheath, can be obtained. A new force sensor, i.e., a nitinol tube attached with an optical Fiber Bragg Grating (FBG) fiber, is proposed to measure the compression force on the sheath. This sensor, with similar diameter and configuration (hollow) as the sheath, can be compactly integrated with TSMs and surgical end-effectors. In this paper, mechanics analysis and verification tests are presented to reveal the relationship between the tension force on the tendon and the compression force on the sheath. The proposed force sensor was calibrated in tests with a sensitivity of 24.28 pm/N and integrated with a tendon-sheath driven grasper to demonstrate the effectiveness of the proposed approach and sensor. The proposed approach and sensor can also be applied for a variety of TSMs-driven systems, such as robotic fingers/hands, wearable devices, and rehabilitation devices.

Multi-channel sliding spotlight SAR can achieve high-resolution and wide-swath imaging. The sparse reconstruction algorithm can improve the quality of imaging. By applying the sparse reconstruction algorithm to multichannel sliding spotlight SAR imaging, azimuth ambiguities, noise and clutter can be suppressed effectively. In this paper, ll regularization based multi-channel sliding spotlight SAR imaging method is proposed. The proposed method combines the DPCA imaging operators with the l1 regularization scheme to solve the nonuniform sampling and azimuth ambiguities problem. The proposed method can suppress azimuth ambiguities more effectively than the reconstruction filter algorithm based DPCA technology in the case of a lower PRF. The experiment results verify the effectiveness of the proposed method.