Image-based sports analytics enable automatic retrieval of key events in a game to speed up the analytics process for human experts. However, most existing methods focus on structured television broadcast video datasets with a straight and fixed camera having minimum variability in the capturing pose. In this paper, we study the case of event detection in sports videos for unstructured environments with arbitrary camera angles. The transition from structured to unstructured video analysis produces multiple challenges that we address in our paper. Specifically, we identify and solve two major problems: unsupervised identification of players in an unstructured setting and generalization of the trained models to pose variations due to arbitrary shooting angles. For the first problem, we propose a temporal feature aggregation algorithm using person re-identification features to obtain high player retrieval precision by boosting a weak heuristic scoring method. Additionally, we propose a data augmentation technique, based on multi-modal image translation model, to reduce bias in the appearance of training samples. Experimental evaluations show that our proposed method improves precision for player retrieval from 0.78 to 0.86 for obliquely angled videos. Additionally, we obtain an improvement in F1 score for rally detection in table tennis videos from 0.79 in case of global frame-level features to 0.89 using our proposed player-level features. Please see the supplementary video submission at https://ibm.biz/BdzeZA.

We propose a recommendation method that considers the user's individual preference and influence from other users in social media. This method predicts the user's individual preference and influence from other users by applying the probability of divergence from random-selection based on a statistical hypothesis test as a form of modified content-based filtering. We evaluated the proposed method by focusing on the rate at which items that have recommended tags are contained among all items. The proposed method is shown to have higher accuracy than traditional content-based filtering. It is especially effective when some percentage of the items have recommendation tags.

Wearable robots have finally reached a point at which they can be put to more practical use. However, methods for assessing their safety have yet to be properly established. Furthermore, for ethical reasons, such assessments must be conducted without exposing the human user to unnecessary danger. Therefore, in this study, a lower-leg dummy for the safety assessment of a wearable robot was developed to evaluate the effects of a misaligned knee joint.

This paper proposes an action generation model which consists of many motor primitive modules. The motor primitive modules output motor commands based on sensory information. Complicated behavior is generated by sequentially switching the modules. The model also has a prediction unit. This unit predicts which module will be used for current action generation. We have confirmed the effectiveness of the model by applying it to a robot navigation task simulation, and have investigated the influence of the prediction on the action generation.

Stroke is the leading cause of adult disability, and training in voluntary movement has been shown to be effective in rebuilding motor skills. We propose an EEG-FES system that can assist in the reconstruction of a closed-loop between motor commands and sensory feedbacks in stroke patients. The system uses event-related desynchronization (ERD) to reflect motor intentions. We did two pilot studies. One focused on extracting clear ERD from healthy subjects who must generally be trained for a few month to extract clear ERD. We also tested the effect of online visual feedback training. The results indicated that for some subjects, but not others, ERD could be extracted in a few days. The other study focused on whether FES (sensory feedback) affects ERD. The results indicated that there was no FES effect (leg stimulation) in leg motor area (Cz) ERD.

Understanding the mechanisms which the brain uses to control body movements have been important in the both fields of robotics and neuroscience. Recent neuroscience studies have shown that humans can acquire a neural representation of the relation between motor command and movement, i.e., learn an internal model of the environment dynamics. In this paper, we investigate motor adaptation of human arm movements to external dynamics. In an experiment, we tried to determine whether humans can learn an internal model of a mixed force field (V+P) that was the sum of a velocity-dependent force field (V) and a position-dependent force field (P). The experimental results show that the subjects did not learn the internal model of V+P accurately and they compensated for the loads by using impedance control. Our results suggest that humans use impedance control when internal models become inaccurate because of the complexity of the external dynamics.

Several studies have been made on intelligent systems typified by a robot. For these systems to behave appropriately in a complicated environment, an action generation model is necessary. This paper proposes an action generation model which consists of many motor primitive modules. The motor primitive modules output motor commands based on sensory information. Complicated behavior is generated by sequentially switching the modules. The model also has a prediction unit. This unit predicts which module will be used for current action generation. A current action is generated by the module chosen based on both the prediction and the current sensor input. Hence, the proposed model can produce different actions even when the current input information is the same. The proposed model is constructed by using a competitive neural network and a recurrent neural network. The modules and the prediction unit are acquired by learning from continuous sensory-motor flow. We have confirmed the effectiveness of the model by applying it to a robot navigation task simulation, and have investigated the influence of the prediction on the action generation.

MR compatible apparatus is essential to design new tasks for fMRI study. This paper described the development of an MR compatible manipulandum actuated by the ultrasonic motors, which was able to work within MRI scanner and to perform fMRI task continuously during finger movements. The prototype of manipulandum was able to produce two discriminable force fields: position dependent and velocity dependent force field. The control law was derived by the characteristic of velocity-phase difference-torque relationship of the ultrasonic motor. After MR compatibility was verified with signal noise ratio of the images, it was shown that the proposed prototype was able to use for fMRI study

Regardless of complex, unknown, and dynamically-changing environments, living creatures can recognize situated environments and behave adaptively by theirselves in real-time. However it is impossible to prepare optimal motion trajectories with respect to every possible situations in advance. The key concept for realizing suitable environmental cognition and motor adaptation is a context-based elicitation of constraints which are canalizing well-suited sensorimotor coordination. For this aim, in this study, we propose a polymorphic neural networks model called CTRNN+ NM (CTRNN with neuromodulatory bias). The proposed model is applied to two dimensional arm-reaching movement control in various viscous curl force fields. The model parameters were optimized by GA. Simulation results reveal that the proposed model inherits high robustness even though it is situated in unexperienced environment, which has same curl but different size of viscous force, since it evolved "how to adapt" instead of "how to move."

Biological motor control system includes some time delay within the neural transmission loop, which makes the motor control unstable. On the other hand, it is seen that the low-pass filtering for the actuator output can improve the system stability. It is estimated that the Renshaw cell in the spinal reflex loop may provide the motor output with the low-pass filtering property. The present paper represents the motor control mechanism as a hierarchical distributed control system including the time delay, and it is shown that the Renshaw cell could contribute toward stabilizing the time delay loop in the hierarchical distributed control system. In addition it is shown that the control system can manages a complex task such as a cooperative problem between posture and motion control

The paper proposes a periodic motion pattern generation model inspired by biological brain motor systems. It consists of three processing elements called, brain, CPG and body parts. The brain part is a time-series pattern discriminator modeled by RNN, which works as a predictive parameter selector for a subsequent CPG part. Moreover, the CPG part is a rhythmic pattern generator for a lower level motor control, which is represented by a neural oscillator model, and the body part corresponds to a physical dynamics between a controlled object and its environment. In the proposed schema, environmental perturbations can be stabilized by a mutual entrainment characteristic between the CPG and the body. In addition, the brain part can recognize several kinds of environmental changes via its proprioceptive feedback time-series stem from own actions, and it can predictively modulate the motion patterns by recalling well-suited CPG parameters according to the body dynamics.

The present paper proposes a learning control method for the musculoskeletal system of arm based on reinforcement learning. An optimization for the hand trajectory and muscle's force distribution is needed to acquire the reaching motion. The proposed architecture can acquire an optimized motion through learning the task. However, the biological control system composed of musculoskeletal system is not able to sense the state without time delay. The time delay causes instability of learning. The proposed scheme consists of the reinforcement learning part and neural internal model. Neural internal model is employed to compensate for the time delay by estimating the state of musculoskeletal system. Then, there must be a modeling error if some noise is included. Thus we introduce the minimum modeling error criterion for reinforcement learning, which gives not only the reduction of total muscle level but also the smoothness of the hand trajectory. The effectiveness and the biological plausibility of the present model is demonstrated by several simulations.

In this paper, a time series identifying and modeling method using neural networks is developed as an approximation tool for time series. Such a method can capture homeostatic dynamics of the system under the influence of exogenous event. The results show that financial time series include both predictable deterministic and unpredictable random components. Neural networks can identify the properties of homeostatic dynamics and model the dynamic relation between endogenous and exogenous variables in financial time series input-output system. In addition, we investigate the impact of the number of model inputs and the number of hidden layer neurons on time series analysis and financial forecasting.

Iriki et al. reported interesting results regarding the visual receptive field of two kinds of neurons in the parietal cortex of a monkey. A monkey did a task to reach its hand or tool to a target. The receptive field of one kind of neuron was enlarged when the monkey used the tool grasped by its hand. The receptive field of the other type of neurons moved together with its hand even though the hand was hidden under an opaque plate. They discussed those results in relation to high-order cognitive functions such as body image and symbolization (Iriki et al., 1996; Iriki, 1998; Obayashi et al., 2000). A hypothesis is posited that these neurons contribute to generate the critic output (state evaluation in a given task) and are obtained through reinforcement learning. Thereby, tool use is considered to be the change of link length for simplicity; a layered neural network learns hand reaching by a manipulator based on reinforcement learning. Inputs of the network are visual sensory signals and the state of the manipulator. Outputs are the critic and joint torques as the actor. After learning, the manipulator came to move its hand toward the target on the visual sensor when the target was located within the hand's reach. Both types of neurons observed in experiments of Iriki et al. were found in the hidden layer of the neural network.

This is a review paper on Mobiligence research. In order to solve the mechanism of intelligence with which higher organisms can behave even in unknown environment, and to discover design principle of artificial systems that have such intelligence, a new concept called Mobiligence is introduced based on assumption that the intelligence must be emerged through the interaction of the system with the environment caused by its mobility. The research on the Mobiligence is characterized by embodied plasticity, co-embodiment with environment, and internal generation of abductive information. The framework of the research on Mobiligence is presented in this paper as well as the originality of the research.

The present paper discusses an optimal control method of biological robot arm which has redundancy of the mapping from the control input to the task goal. The control input space is divided into a couple of subspaces according to a priority order depending on the progress and stability of learning. In the proposed method, the search noise which is required for reinforcement learning is restricted within the first priority subspace. Then the constraint is relaxed with the progress of learning, and the search space extends to the second priority subspace in accordance with the history of learning. The method was applied to the musculoskeletal system as an example of biological control systems. Dynamic manipulation is obtained through reinforcement learning with no previous knowledge of the arm's dynamics. The effectiveness of the proposed method is shown by computational simulation.

Proposes a computational emotion formation model and attempt to apply the model to speech communication. As speech communication plays an essential part of our daily life, the utilization of speech sound is expected to make human-robot communication smooth. The proposed model forms the state of emotion based on the prosodic components because the emotional aspects of human speech sounds are found in the prosodic change. The model is based on an agent network architecture. Also the model is taken account of the motor command generation according to the state of emotion. The validity of the model is examined by the computer simulation. From the results of the simulation, it is shown that the formation of emotion in the model is adequate for speech sounds, and the motor command generation is also adequate.

In the recent robotics, much attention has been focused on utilizing reinforcement learning for designing robot controllers. However, there still exists difficulties, one of them is well known as state space explosion problem. As the state space for learning system becomes continuous and high dimensional, the learning process results in time-consuming since its combinational states explodes exponentially. In order to adopt reinforcement learning for such complicated systems, it should be taken not only "adaptability" but "computational efficiencies" into account. In the paper, we propose an adaptive state space recruitment strategy for reinforcement learning, which enables the system to divide state space gradually according to task complexity and progress of learning. Some simulation results and real robot implementation show the validity of the method.

We propose autonomous learning algorithm based on the internal state of the associative memory for intelligent robots. The proposed associative memory model consists of structural unstable oscillators and a common field such as chemical concentration. In computer simulations, we use the binary pattern as the stimuli. When the pattern memorized in the network is given to the network from the outer world, the internal state of the network becomes a periodic state. On the other hand, when the pattern has not been memorized is given to the network, the state becomes an intermittently chaotic and the output of the network travels around the input and some memorized patterns. This chaotic state is regarded as "I don't know" state. Further, when the proposed autonomous learning algorithm is applied to the proposed network, the network can learn only the novel patterns automatically without destroying the previously memorized patterns.

In this paper, a new approach is examined for controlling the prosthetic hand: the torque control of each joint. The joint torque is estimated from EMG signals using an artificial neural network. The learning system is based on a feedback error learning schema. Two controlled objects were used in the experiments, i.e. a hand dynamic model and prosthetic hand. These experimental results show the possibility to apply the proposed method to the prosthetic hand control under the conditions that the controlled object is a second order delay system.

A learning and dynamic pattern generating system for acquiring the skills in dynamic manipulation of objects using robotic manipulators is to be established, where the desired space trajectories for the manipulators are not specified explicitly. Robotic batting is taken as a task example. The problem is approached so as to result in an iterative learning of the joint driving torque patterns of the manipulator that are considered as the task skills and learned against several typically given desired ball velocities. A multi-layered artificial neural network is used to learn and generalize the joint driving torque against various desired ball velocities, and an iterative optimal control algorithm is adopted to generate the supervisory joint driving torque signals for the neural network. Computer simulation examples of a three-degree-of-freedom manipulator are outlined, the results are depicted to explain the idea and verify the proposed approach, and the robustness issues are discussed qualitatively.

In this paper, we propose a direct torque control method for the prosthetic hand. In order to estimate the joint torque from EMG signals, an artificial neural network by the feedback error learning schema is used. 2-DOF motions, i.e. hand grasping/opening and arm flexion/extension, are picked up. In the experiments, two measurement conditions of EMG signal are prepared: the forearm from which the EMG signal is measured is free or fixed. Then it is verified that the neural network can learn the relation between the EMG signal and the joint torque under these two measurement conditions.

RBF (radial basis function)-based networks have been widely used because they can learn a strong nonlinear function fast and easily due to their local learning characteristics. Among them, Gaussian soft-max networks have generalization ability better than regular RBF networks because of their extrapolation ability. However, since the RBF-based network has no hidden unit which can represent some global information, the internal representation cannot be obtained. Accordingly even if the knowledge which could be obtained through the previous sets of learning is utilized effectively in the present learning, the network has to learn from scratch. Multi-layered neural networks are able to form the internal representation in the hidden layer through learning. The paper proposes a Gauss-sigmoid neural network for learning with continuous input signals. The input signals are put into a RBF network, and then the outputs of the RBF network are put into a sigmoid-based multi-layered neural network. After learning based on backpropagation, the localized signals from the RBF network are integrated and an appropriate space for the given learning is reconstructed in the hidden layer of the sigmoid-based neural network. Once the hidden space is constructed, both the advantage of the local learning and the global generalization ability can exist together.

We propose an autonomous dynamical pattern recognition and learning system. It is demonstrated that when the embedded pattern, i.e., known pattern, is given to the network, the firing pattern of the network immediately goes to the relevant embedded pattern and the network state reduces to the oscillatory state at once. Second, when no embedded pattern, i.e., unknown pattern, is given to the network, the network state oscillates chaotically. It is considered as "I don't know" state proposed by Freeman and coworkers. Finally, when Hebb rule is applied to the network under the external stimuli that are unknown patterns, the internal state of the network is inversely bifurcated from the chaotic state to the periodic state according to the progress of learning. By using this phase transition as an index of the progress of learning, the network can learn new patterns without any external observers.

We believe that communication in multi-agent system has two major meanings. One of them is to transmit one agent's observed information to the other. The other meaning is to transmit what an agent is thinking. Here we focus the latter and aim to the emergence of the autonomous and decentralized arbitration communication among some agents. Communication contents, strategy and representation are not prescribed and are acquired by learning using a reinforcement signal which is given to the agent after its action. The reinforcement signal is not shared with the other agents. In order to realize this learning, the agent often has to make a decision not only from the present communication signals but also from the past signals. Accordingly the system architecture using recurrent type (Elman) neural network is proposed. The ability of this architecture was examined by two and four agents negotiation problems. A variety of negotiation strategies emerged among the agents through the learning to avoid some conflict after their decisions.

Traditionally, tasks for a robotic system are specified by a desired timed trajectory. There, however, are many tasks in which the desired motions are specified by the state of system rather than time, such as contour following tasks. For such tasks, passive velocity field control (PVFC) has been proposed and the geometry of the controlled systems was analyzed. In this paper, a method to apply the PVFC to cooperative multiple mobile robotic systems whose sub-system is under nonholonomic constraints and which convey a common rigid object in a horizontal plane is proposed, and the effectiveness of the control method is demonstrated by several numerical simulations.

This paper proposes a new dynamical memory system based on chaotic neural networks, and its learning scheme. It is demonstrated that, when no embedded pattern, i.e., unknown pattern, is applied to the system, the output pattern travels around the embedded patterns, and the traveling phases depend on a external parameter of the networks such as the input from the other neurons or cortex. Further, the temporal output of the networks reflects a hierarchical structure of the memorized patterns.

Task-oriented motion skill formation for kinematically redundant robot manipulators as an alternative approach to their task planning and control problem, is discussed. Elasto-viscous structures analogous to human arm muscle-skeleton structures are assumed to be the base of the joint actuation, and their dynamic impedance properties (stiffness and viscosity), together with their equilibrium trajectory profiles, are defined as the task skill. An iterative computing algorithm based on variant calculus is derived, and task examples on a 3 DOF planar manipulator are presented.

Traditionally, tasks for a manipulator are specified by a desired timed trajectory. There, however, are many tasks in which the desired motions are specified by the state of system rather than time, such as contour following tasks. For such tasks, passive velocity field control (PVFC) has been proposed and the geometry of the controlled systems was analyzed. In this paper, a method to extend the PVFC to cooperative multiple manipulators is proposed, and the effectiveness of the control method is demonstrated by numerical simulations, and some properties are shown.

Autonomous distributed control (ADC) is one of the most attractive approaches for more versatile and autonomous robot systems. The paper proposes a parallel and distributed trajectory generation method for redundant manipulators through cooperative and competitive interactions among subsystems composing the ADC that is based on a concept of virtual arms. The virtual arm has the same kinematic structure as the manipulator except that its end point is located on a joint or link of the manipulator. Then the redundant manipulator can be represented by a set of the virtual arms. Trajectory generation and point to point control of the redundant manipulator are discussed, and it is shown that the kinematic redundancy of the manipulator can be utilized positively in the generated trajectories by using the virtual arms.

Impedance control is one of the most effective control methods for the manipulators in contact with their environments. The characteristics of force and motion control, however, is determined by a desired impedance parameter of a manipulator's end-effector that should be carefully designed according to a given task and an environment. The present paper proposes a new method to regulate the impedance parameter of the end-effector through learning of neural networks. Three kinds of the feed-forward networks are prepared corresponding to position, velocity and force control loops of the end-effector before learning. First, the neural networks for position and velocity control are trained using iterative learning of the manipulator during free movements. Then, the neural network for force control is trained for contact movements. During learning of contact movements, a virtual trajectory is also modified to reduce control error. The method can regulate not only stiffness and viscosity but also inertia and virtual trajectory of the end-effector. Computer simulations show that a smooth transition from free to contact movements can be realized by regulating impedance parameters before a contact.

Many researchers have studied bilateral master-slave teleoperator systems, especially since Anderson and Spong (1989) proposed a novel and simple method to overcome the instability due to transmission delays. In this paper we propose a bilateral master-slave manipulator system in which the slave impedance can be adjusted by the operator in a bilateral fashion. The stability of the system for the time delay is also discussed, and the effectiveness of the system is shown by experiments.

When two robot arms holding an object together they form a closed kinematic chain which enables generation and control of arbitrary internal forces on the object. The internal forces balance within the closed kinematic chain and therefore do not contribute to the resultant force that generates motion. However, they change the effective compliance realized on the object. Therefore when the object held by the robots makes a contact with a rigid environment the compliance at the contact location does not only depend on the inherent compliances and the posture of each robot arm but also depends on the internal forces exerted on the object. In cases when the contact location moves relative to each robot's tip, the compliance at the contact depends also on the variation of the contact location. In this study we propose a method that utilizes the internal forces to keep the contact compliance in primarily important directions close to the desired values regardless of the variation of the contact location and robots' posture.

Visual information processing in biological systems is performed simultaneously and in parallel at each neural unit. That is, a global task with no supervisor is achieved by the interaction among many subsystems. In this sense, the visual process shares the common functional structure with 'autonomous decentralized systems' which have attracted much attention in the field of system engineering. We attempt to design a pictorial pattern recognition system based on the concept of the autonomous decentralized system. The local interaction among subsystems is described by reaction-diffusion equations, in which each subsystem utilizes only the neighboring information around itself.>

This paper discusses the compliance control of an ultrasonic motor powered prosthetic forearm which utilizes cutaneously measured electromyogram (EMG) signals sensed with the electrodes over the muscles as means of detecting motor commands sent by the central nervous system (CNS). Compliance control of the artificial limb was studied by implementing the bilinear model of the forearm and hand. This model emphasizes the role of the visco-elastic properties of the musculo-skeletal system of the actual limb in controlling its net configuration and movement. The flexor and extensor muscles extending over a joint influence the total joint impedance and determine the equilibrium position of the joint. Relaxing both flexor and extensor muscles makes the joint compliant to external forces, while activating both muscles increases the impedance of the joint.>

Studies multiple-robot manipulator cooperative manipulation of an unknown dynamic object. The authors show a simple approach for each robot to identify the object dynamics through cooperation. The authors decentralize the position/internal force hybrid controller such that each robot has its controller and determines its control torque from its own feedback information. Experiments show the effectiveness of the control strategy.>

This paper discusses compliant manipulation of multiple robot manipulator's cooperative in a dynamical environment. Cooperative compliant manipulation requires the simultaneous control of both the object's position and the interaction forces between the robots and the object. It is shown that position-controlled manipulators can not perform this kind of task successfully. It is proved that, in order to avoid excessive pressing, the frequency bandwidth of the overall control systems including the manipulators and the object should be adjusted according to the object dynamics. A unified model-matching control approach is developed in which each autonomous robot uses its own position and interaction force information to adjust the system's frequency bandwidth and to maintain the internal force in a desired performance. The reference models of the object's position and the internal force are selected with respect to the object's dynamics, and the force feedback controllers are designed accordingly to realize the reference models. Simulation studies show the effectiveness of this control approach.

A distributed feedback control for redundant manipulators based on virtual arms is proposed. The virtual arm has the same kinematic structure as the manipulator except that its end-point is located on the joint or link of the manipulator. Therefore, the configuration of the redundant manipulator can be represented as a set of virtual arms. Dynamic behavior of each subsystem is based on an end-point feedback control of the virtual arm on the operational space. Position control of the manipulator utilizing kinematic redundancy can be realized through cooperative and competitive interactions between subsystems, and the dynamic behavior of the whole system is always stable. The advantages of the method are discussed.>

One of the basic characteristics of the dual-arm robots which manipulate a common object is the redundancy of joint degrees of freedom The present paper proposes a compliance control method utilizing kinematic redundancy of the dual-arm robot. The method presented here can regulate the compliance or stiffness of several points on the dual arm- robot as well as the object compliance through regulation of the joint stiffness.

Proposes a new conventional method to reconstruct 3D motion of rigid polyhedral objects from a sequence of monocular images. In general the problem is ill-posed, therefore additional information is required to recover depth. The authors utilize line correspondence between sequential images and existence of parallel line segments in the scene. The relation between the coordinates of points can be described only by a rotation matrix, if it is formulated relatively. Then if there exists a pair of parallel line segments, the matrix can be solved by using linear equations. After that the translation vector is computed. It is necessary that there exists at least one pair of parallel line segments in the scene in order to obtain motion parameters. They also propose the method to extract pairs of parallel line segments in the image. Finally some experimental results for simulated data are demonstrated. >