Vision-based tactile sensing, an economical and widely utilized methodology, has the potential to offer crucial contact geometry information for localizing objectives even in cases of visual occlusion. However, this kind of fingertip sensor is limited. When a person picks up a relatively small object placed on a flat surface with two fingers, they may not only use the pads of their fingers depending on the size of the object but also use their fingernails for small or thin objects. Fingers with nail structures have been shown to be effective in picking up objects like this in robot hands as well. Moreover, in actual work, accidental contact between sensors and surrounding objects such as tables often occurs. Sensors with fingernails can avoid this situation in advance by having the fingernails touch the object before the fingertip touches the object. In this work, we present the NailTact, which can detect the force applied to both the fingertip part and the nail part from the same camera image using a single camera. Using the prototype robot finger, we will verify the sensor response characteristics to the load on the nail and the sensor response when grasping an object with the nail and the situation when the finger makes contact with a table. We also present a simple model that illustrates the relationship between the force applied to the nail and the movement of the marker. In the card-grasping experiment, we not only successfully grasped a very thin object but also measured the grasping force.

Inspection and maintenance work at heights car-ries significant risks and is time consuming for human workers. Therefore, aerial manipulators are expected to replace these tasks. This paper presents a multirotor UAV equipped with a single horizontal thruster. This minimal configuration en-ables physical contact while keeping the airframe's attitude horizontal for non-destructive inspection work on vertical wall surfaces. The thrust required to move forward and backward during a physical contact task is independent of the thrust required for hovering, simplifying control of the UAV. Utilizing onboard sensors, the UAV automatically maintains a forward-facing posture against the wall, initiates and sustains contact, and disengages when necessary. Additionally, the UAV in this study incorporates an ultrasonic thickness measurement device, allowing for the verification of automated measurements while in flight.

Pruning work at high altitude is a dangerous work with a high risk of accidents for human workers. In this research, we propose a multirotor flying robot that is equipped with a wire-suspended device and performs pruning task. We use a saber saw as a cutting tool. If the cutting tool is installed on the body of the multirotor platform, it is difficult for the flying robot to approach the desired work point if there are obstacles such as other branches around the target branch to be pruned. Therefore, in this study, we propose a saber saw suspended from the body of the multirotor platform with two wires. The wire-suspended device is equipped with a saber saw and four ducted fans that produces thrust in any direction on the horizontal plane. This ducted fan system can be used to suppress the swing of the wire-suspended device to make positioning of the saber saw blade to the target point easier, and to improve the efficiency of the cutting and reduce the cutting time by providing a pushing force to the saber saw. As a result, the pruning work could be performed efficiently. Experiments have demonstrated that aerial pruning is possible using the long-reach wire-suspended saber saw.

We describe the concept, design and implementation of a unique manipulator system for a multirotor aerial vehicle. The proposed manipulator system consists of three robotic arms attached to a multirotor airframe with the objective to provide the ability to manipulate single or multiple objects as well as aid in complex navigation tasks by doing contact based obstacle avoidance and acting as adaptive landing gear in uneven terrain. In this paper we primarily focus on the description and experimentation of one of the tasks achievable by the proposed aerial multi-manipulator system: landing a multirotor aerial vehicle on an uneven surface. Deploying the on-board manipulator as landing gears reduces the hardware carried by the vehicle while additionally providing the ability to land in unstructured and commonly difficult to land terrain.

In this work, we present a method for the individual, simultaneous teleoperation of multiple multirotors. Instead of using a traditional four axis remote controller to steer the small sized quadrotors used in this work, all degrees of freedom of the device are intuitively controlled by a human operator in virtual reality, allowing high precision maneuvering of the vehicles through obstacles in three dimensional space. The proposed method enables the human operator to precisely control multiple micro aerial vehicles simultaneously, without the use of formation flight or other grouping methods. The teleoperation control performance of this approach is compared to a traditional four axis control method in several experiments.

We have proposed airborne docking using two multi-rotor aerial robots. This paper presents a transport multi-rotor UAV with winch mechanism and a small multi-rotor with onboard locolization and mobile manipulation system. The winch mechanism enables the UAV to lower and raise a bar to transport another UAV attached to it. The airborne docking method used in our work is chosen in order to avoid the effect of downwash generated by the multi-rotors. With experiments we have verified the possibility of airborne docking, and evaluated how it influences the transport multi-rotor UAV as the load is changed, using the IMU data of UAV.

We have proposed a multimodal sensing with optical device and its application to robotic grasp control. The proposed device provides both tactile and proximity information with high spatial resolution on the same scale of coordinates, that is necessary while grasping small and complicated-shaped objects using a robotic hand. In our proposed device, tactile information is obtained as infrared image through a light conductive plate, which allows the capture of a tactile image of high spatial resolution using a camera. Proximity information is detected based on stereo matching of a pair of images obtained by two cameras. By using a compound-eye camera, we constructed a compact device so that the sensor can be mounted on the robotic hand. Through experiments with 6 DoF robotic arm and robotic hand equipped with the proposed sensing device, we show that the robotic motion for searching, approaching and grasping can be controlled based on sensor information obtained from only the proposed device. In addition, our combined sensing approach enables an adaptive grasping motion of the robotic hand.

While aerial vehicles have become more and more popular in the hobbyist sector as a platform for photography and cinematography, currently there is little utilization of aerial vehicles in an industrial environment. In this work, we describe our efforts to develop an aerial robotics platform with the ability to traverse a ceiling with high stability and precision using a top mounted omni wheel drive system and an AR-marker system. The marker system has been extended to incorporate the use of dual camera sensor fusion. The proposed system is specifically designed and planned to be suitable for a real world industrial assignment, namely the setting of painted ink-marker used as an orientation for drilling, measuring or maintenance tasks on the ceiling in an infrastructure inspection work environment. We show with test flights the feasibility of this approach and the ability of the developed platform to do high precision localization and positioning relative to an AR-marker in order to perform an ink-marker placement task.

We describe an implementation of torsional work at high altitude employing an aerial robot equipped with an upward directed hand on top of a hexarotor airframe, considering its workspace above the airframe. The aerial robot in this study consists of a small hexarotor platform and a robotic hand module including a gripper. After grasping the object, the torsional work is executed by using yaw rotation of the body of the robot. For increasing a torque for the torsional task, an impact device is embedded at the base of the hand module. A maximum instant torque generated with this impact device was more than 4 N·m. In addition, the number of rotations of the body is measured through images captured by an onboard camera mounted beside the hand. Through experiments, we verified the feasibility of the present robot by successfully unscrewing a light bulb set to the socket above the robot and safely landing with the detached bulb.

We describe an aerial robot equipped with an upward directed hand on top of a quadrotor airframe for realizing an aerial manipulation task at high altitude, considering a work space above the robot. The aerial robot in this study consists of a small quadrotor platform, a robotic hand module including a gripper and slider mechanism, and an embedded vision system. Positioning of the hand is autonomously performed through a vision-based control system. In this system, position and orientation of a bar-like object is measured and detected through a monocular camera image and FPGA-based on-board processing. Through experiments, the grasping of a bar located above the robot, keeping the grasp with shut off propellers and succeeding take off by releasing a bar was achieved.

A silicon retina is an intelligent vision sensor that can execute real-time image preprocessing by using a parallel analog circuit that mimics the structure of the neuronal circuits in the vertebrate retina. For enhancing the sensor's robustness to changes in illumination in a practical environment, we have designed and fabricated a silicon retina on the basis of a computational model of brightness constancy. The chip has a wide-dynamic-range and shows a constant response against changes in the illumination intensity. The photosensor in the present chip approximates logarithmic illumination-to-voltage transfer characteristics as a result of the application of a time-modulated reset voltage technique. Two types of image processing, namely, Laplacian-Gaussian-like spatial filtering and computing the frame difference, are carried out by using resistive networks and sample/hold circuits in the chip. As a result of these processings, the chip exhibits brightness constancy over a wide range of illumination. The chip is fabricated by using the 0.25- μm complementary metal-oxide semiconductor image sensor technology. The number of pixels is 64 × 64, and the power consumption is 32 mW at the frame rate of 30 fps. We show that our chip not only has a wide-dynamic-range but also shows a constant response to the changes in illumination.

We describe a binocular vision system that emulates disparity computation in the neuronal circuit of the primary visual cortex (V1). The system consists of two sets of silicon retinas and simple cell chips that correspond to the binocular vision and field programmable gate array (FPGA) circuit. This arrangement mimics the hierarchical architecture of the visual system of the brain. The silicon retina is an analog very large scale integrated (aVLSI) circuit and possesses a Laplacian-Gaussian-like spatial filter similar to the receptive field of the vertebrate retina. The simple cell chip generates a Gabor-like spatial filter similar to the orientation-selective receptive field of the simple cell in V1 by aggregating several pixels of the silicon retina. The FPGA receives the outputs from the two simple cell chips corresponding to binocular inputs from the left and right eyes and calculates the binocular disparity in real-time based on the disparity energy model. The system provides output images tuned to five different disparities in parallel. The disparity map is obtained by comparing these disparity energy outputs. Due to the combination of the parallel and analog computation of the aVLSIs and the pixel-wise computation with hard-wired digital circuits, the present system can efficiently compute the binocular disparity using compact hardware and low power dissipation in real-time.

We designed a mixed analog-digital neuromorphic vision system that can replicate the response of complex cells in the primary visual cortex (V1). Using the system, the binocular disparity energy maps were calculated in natural scenes in real time. Because of its compact hardware and low power dissipation, the neuromorphic vision system developed in the present study is suitable to robotic vision. More interestingly, it provides insights to explore the visual function of the neuronal network of the brain, visualizing neural images inferred from physiological experiments.

A high resolution neuromorphic multi-chip vision system was fabricated to emulate the orientation selective response of the simple cell in the primary visual cortex. The vision system consists of two types of analog chips: a silicon retina and a simple cell chip. The center-surround concentric receptive fields of the silicon retina are aggregated in the simple cell chip, mimicking the hierarchical architecture in the visual system of the brain. Both chips have 100 times 100 pixels. Using the orientation-selective outputs obtained from the multi-chip system, texture segregation was conducted based on a computational model inspired by neurophysiology. The texture image was filtered by the two orthogonally oriented receptive fields of the multi-chip system and the filtered images were combined to segregate the area of different texture orientation with the aid of field programmable gate array (FPGA). The present multi-chip system is useful to emulate and verify computational models for texture segregation of the cortical cells

A high resolution neuromorphic aVLSI was fabricated to emulate the orientation selective response of the simple cell in the primary visual cortex. The aVLSI circuits consist of two analog chips: a silicon retina and an orientation chip. The center-surround concentric receptive fields of the silicon retina are aggregated in the orientation chip, mimicking the hierarchical architecture in the visual system of the brain. Both chips have 100 /spl times/ 100 pixels and therefore, this multi-chip system is applicable to robotic vision. Using the orientation-selective outputs obtained from the multi-chip system, a texture segregation was conducted based on a similar algorithm of the energy computation. The texture image was filtered by the two orthogonally oriented receptive fields of multi-chip system and the filtered images were combined to segregate the area of different texture orientation with the aid of a PC. The study demonstrated that the orientation-selective multi-chip system developed is useful to emulate the texture segregation employing a fundamental architecture to generate the simple cell response in the primary visual cortex and is applicable to robotic vision.

In this paper, we designed and fabricated a multichip neuromorphic analog very large scale integrated (aVLSI) system, which emulates the orientation selective response of the simple cell in the primary visual cortex. The system consists of a silicon retina and an orientation chip. An image, which is filtered by a concentric center-surround (CS) antagonistic receptive field of the silicon retina, is transferred to the orientation chip. The image transfer from the silicon retina to the orientation chip is carried out with analog signals. The orientation chip selectively aggregates multiple pixels of the silicon retina, mimicking the feedforward model proposed by Hubel and Wiesel. The chip provides the orientation-selective (OS) outputs which are tuned to 0 degrees, 60 degrees, and 120 degrees. The feed-forward aggregation reduces the fixed pattern noise that is due to the mismatch of the transistors in the orientation chip. The spatial properties of the orientation selective response were examined in terms of the adjustable parameters of the chip, i.e., the number of aggregated pixels and size of the receptive field of the silicon retina. The multichip aVLSI architecture used in the present study can be applied to implement higher order cells such as the complex cell of the primary visual cortex.