While Large Language Models (LLMs) achieve near-human performance on standard benchmarks, their capabilities often fail to generalize to complex, real-world problems. To bridge this gap, we introduce DeepQuestion, a scalable, automated framework that systematically elevates the cognitive complexity of existing datasets. Grounded in Bloom's taxonomy, DeepQuestion generates (1) scenario-based problems to test the application of knowledge in noisy, realistic contexts, and (2) instruction-based prompts that require models to create new questions from a given solution path, assessing synthesis and evaluation skills. Our extensive evaluation across ten leading open-source and proprietary models reveals a stark performance decline with accuracy dropping by up to 70% as tasks ascend the cognitive hierarchy. These findings underscore that current benchmarks overestimate true reasoning abilities and highlight the critical need for cognitively diverse evaluations to guide future LLM development.

Having an appropriate flapping path to yield efficient propulsion is an interesting issue in fish robotics. In most works, especially two-segmental structures, the flapping motion is limited to sinusoidal functions. In this paper, to cope with the aforementioned limitation, a conceptual non-sinusoidal path is proposed. The proposed flapping path and the conventional one, both are optimized for a sample fish robot. According to some simulation results, it is shown that if a proper actuator is employed to generate both optimized paths, the proposed approach yields more propulsion efficiency. Furthermore, it is discussed that our method can better imitate fish muscle output power. Finally, through experiments, some practical issues are considered.

Due to the lack of enough generalization in the state space, common methods of reinforcement learning suffer from slow learning speed, especially in the early learning trials. This paper introduces a model-based method in discrete state spaces for increasing the learning speed in terms of required experiences (but not required computation time) by exploiting generalization in the experiences of the subspaces. A subspace is formed by choosing a subset of features in the original state representation. Generalization and faster learning in a subspace are due to many-to-one mapping of experiences from the state space to each state in the subspace. Nevertheless, due to inherent perceptual aliasing (PA) in the subspaces, the policy suggested by each subspace does not generally converge to the optimal policy. Our approach, called model-based learning with subspaces (MoBLeSs), calculates the confidence intervals of the estimated Q -values in the state space and in the subspaces. These confidence intervals are used in the decision-making, such that the agent benefits the most from the possible generalization while avoiding from the detriment of the PA in the subspaces. The convergence of MoBLeS to the optimal policy is theoretically investigated. In addition, we show through several experiments that MoBLeS improves the learning speed in the early trials.

Developing analytical models of efficient locomotion in biology is one of the most interesting goals in bio- inspired robotics. This paper presents a mathematical framework in order to model one of the most energy efficient locomotion types in flying animals; i.e., thermal soaring. Unlike the legged locomotion, in flying, modeling the environmental effects on animals' behaviors is very important. In doing so, we develop our model by assuming thermals as bubbles of rising air. According to pieces of real evidence, this kind of modeling is more compatible with the nature of thermal soaring. Moreover, we present a simple hybrid control strategy for obtaining the optimal path in order to maximize benefit from the updraft of air-flow. By using this control strategy, the flying robot can plan a path for traveling between thermals without flapping; i.e., energy efficient flying. So as to investigate the compatibility of presented model and controller with reality, we set their parameters based on the biological evidences. As a result, in simulations, it is observed that the generated flying behavior is comparable with the thermal soaring behavior of real birds. This observation provides a confirmation for generality and applicability of the presented approach.

Albatross has an energy efficient flying pattern (dynamic soaring) among seabirds. This interesting point encourages us to exploit its flying natural dynamics so as to control the flying robots on energy efficient and robust gaits. In doing so, we study the albatross dynamic soaring from analytical and biological perspectives and realize that to generate the dynamic soaring instead of trajectory control, the mechanical energy should be regulated. Accordingly, the control objective is set to mechanical energy regulation, and the bank angle and lift coefficient are computed to satisfy this objective. The presented method is simulated on a standard albatross model and generates two different types of dynamic soaring; O-shaped and a-shaped patterns. In addition, by means of simulations, it is investigated that the presented method is robust in face of variations in initial conditions and unexpected disturbances in the environment's model; i.e., they cannot disturb the stability and cyclic behavior of the system. Moreover, the simulation results are compared with pieces of natural evidence from albatross and interesting similarities are observed.

Social robots are becoming a companion in everyday life. To be well accepted by humans, they should efficiently understand meanings of their partners' motions and body language and respond accordingly. Learning concepts by imitation brings them this ability in a user-friendly way. This paper presents a fast and robust model for incremental learning of concepts by imitation (ILoCI). In ILoCI, observed multimodal spatiotemporal demonstrations are incrementally abstracted and generalized based on their perceptual and functional similarities during the imitation. Perceptually similar demonstrations are abstracted by a dynamic model of the mirror neuron system. The functional similarities of demonstrations are also learned through a limited number of interactions with the teacher. Incremental relearning of acquired concepts together through memory rehearsal enables the learner to gradually extract and utilize the common structural relations among demonstrations to expedite the learning process especially at the initial stages. Performance of ILoCI is assessed using a standard benchmark dataset and a human-robot interaction task in which a humanoid robot learns to abstract teacher's hand motions during imitation. Its performance is also evaluated on occluded observations that are probable in real environments. The results show efficiency of ILoCI in concept acquisition, recognition, prediction, and generation in addition to its robustness to occlusions and high variability in observations.

In this paper, we answer two interleaved questions. The first one is, having a redundant serial manipulator with a given cyclic task, how can we benefit simultaneously from both natural dynamics modification (NDM) and kinematic redundancy resolution to reduce the actuators' torque? Here, the NDM is done by devising parallel nonlinear monoarticular compliances (MACs), which span one joint, and nonlinear biarticular compliances (BACs), which pass over two joints. We take advantage of kinematic redundancy to exploit the robot's natural dynamics. The second question is how do kinematic redundancy resolution and the NDM interact to minimize the cost? To answer these questions, we cast the problem of simultaneous modification and exploitation of natural dynamics into a constrained multiobjective optimization problem. We show that the set of optimal compliances has an analytical solution as a parametric function of joint trajectories. Accordingly, we study how the components of cost function affect the profile of optimal compliant elements. The proposed method is implemented on a simulated planar 3-DoF manipulator and a simulated nonplanar 4-DoF manipulator for three different tasks. The results shed light on how kinematic redundancy resolution influences efficiency of using MACs and BACs and, consequently, increases attainable gains from the NDM. Moreover, analysis of the results specifies the roles of mono- and BACs and especially explains the reason behind the particular importance of having BACs to reduce the actuation cost.

In this paper, we propose a new method for implementing a rotational nonlinear spring with user defined profile, based on the combination of a linear spring with a nonlinear transmission mechanism. The proposed structure consists of a non-circular cam, a roller which moves along outer circumference of the cam, and a stretched translational linear spring which is connected between center of the cam and center of the roller. We obtain a set of differential equations to design shape of the cam for any given torque-angle profile. Also, it will be shown that profiles with both positive and negative values can be implemented by the proposed method. At last, the cam of some popular nonlinear springs are designed, including constant, cubic, hyperbolic tangent and sinusoidal springs.

In this paper, we present the dynamical equations of a nonlinear adaptive natural oscillator (NANO) in order to exploit the natural dynamics in robotic systems. The presented oscillator tries to minimize an energy-based cost function by adapting the shape and frequency of the reference trajectory. Stability, convergence, and optimality of this oscillator are guaranteed analytically. Moreover, the performance of this oscillator is investigated by applying it to three different types of robotic models; i.e., the pendulum, the adaptive-toy, and the hopper-leg.

Recently, online input selection has gained an increasing attention in evolving fuzzy models. In this paper, we proposed a new evolving fuzzy system referred to as evolving heterogeneous fuzzy inference system (eHFIS), which can simultaneously perform local input selection and system identification in an evolving and integrative manner. The introduced eHFIS is structured by some fuzzy rules with different effective input variables. This was achieved through inclusion of some parameters (local input selectors) in the structure of the Takagi-Sugeno system. An online learning algorithm is proposed to identify the eHFIS, where 1) the premise parameter learning and rule evolution take place with the usage of an incremental and evolving clustering for partitioning the data space; 2) a local input selection strategy based on switching to a neighboring model is adopted, and then, all fuzzy rules with the same input structure form a new category; and 3) for each category, the parameters of linear models, in consequent parts, are updated by weighted recursive fuzzily weighted least-squares estimator. The performance of the proposed eHFIS is evaluated and compared through several simulations on hand made as well as real-life datasets.

In a wheeled-tip manipulation system, the wheels at the tip of robot fingers grasp objects and manipulate them by rolling over their boundaries. In this paper, we introduce a 2-D kinematic modeling for wheeled-tip manipulation systems in order to handle rolling over objects' corners in the manipulation. The modeling facilitates motion planning through projecting grasp stability conditions into configuration space, which unifies the planning of the grasp stability and obstacle avoidance. It is done through defining a static force closure map independent of the obstacles and the robot mechanism. This results in a straightforward planning in PRM framework. The simulation and experimental results support the approach.

In an incomplete information game, a big challenge is to find the best way of exploiting available information for optimal decision making of the agents. In this paper, two decision making methods, namely model-based and learning-based bidding strategies, are proposed and compared, for repeated Cournot competition of the generators in a day-ahead electricity market. The sum of the rivals' offered quantities (SROQ) is considered as the state of the agent and its value is estimated using an adaptive expectation method. In the model-based approach, the convergence of the agents' strategies to the Nash equilibrium point is also studied in two different cases. In the learning-based approach, the optimal bidding strategy is learned through combination of state estimation and a reinforcement learning method. Using the estimated state (SROQ), the optimal decision is learned through a fuzzy Q-learning algorithm. Through a case study, which is performed on the three-bus benchmark Cournot model, the convergence of the generators' bids to the Nash-Cournot equilibrium is examined.

We present a data-driven method for designing parallel compliance. Designing such compliance helps the system to improve energy efficiency, mainly by reducing negative work. The core idea is to design a controller first and then find springs working in parallel with each actuator such that force-displacement graph is lined up around displacement axis. By doing so, we simply shape the natural dynamics for performing the task efficiently. Maximum torque reduction for actuators is a byproduct of this design method. The method can be used in different cyclic robotic application, especially in legged locomotion systems. In this paper, we design a spinal compliance for a bounding quadruped robot in Webots. The results show that the power consumption and the maximum torque are reduced significantly.

We studied the effect of the control of an active spine versus a fixed spine, on a quadruped robot running in bound gait. Active spine supported actuation led to faster locomotion, with less foot sliding on the ground, and a higher stability to go straight forward. However, we did no observe an improvement of cost of transport of the spine-actuated, faster robot system compared to the rigid spine.

Bouncing, balancing and swinging the leg forward can be considered as three basic control tasks for bipedal locomotion. Defining the trunk by an unstable inverted pendulum, balancing as being translated to trunk stabilization is the main focus of this paper. The control strategy is to generate a hip torque to have upright trunk to achieve robust hopping and running. It relies on the Virtual Pendulum (VP) concept which is recently proposed for trunk stabilization, based on human/animal locomotion analysis. Based on this concept, a control approach, named Virtual Pendulum Posture control (VPPC) is presented, in which the trunk is stabilized by redirecting the ground reaction force to a virtual support point. The required torques patterns generated by the controller, could partially be exerted by elastic structures like hip springs. Hybrid Zero Dynamics (HZD) control approach is also applied as an exact method of keeping the trunk upright. Stability of the motion which is investigated by Poincaré map analysis could be achieved by hip springs, VPPC and HZD. The results show that hip springs, revealing muscle properties, could facilitate trunk stabilization. Compliance in hip produces acceptable performance and robustness compared with VPPC and HZD, while it is a passive structure.

This paper presents a new control approach to achieve robust hopping with upright trunk in the sagittal plane. It relies on an innovative concept for trunk stabilization, called Virtual Pendulum concept, recently proposed, based on experimental finding in animal locomotion. With this concept, the trunk is stabilized by redirecting the ground reaction force to a virtual support point, named Virtual Pivot Point (VPP). This concept is combined with a new leg adjustment scheme to induce stable hopping when an extended trunk is added to SLIP model. The stability is investigated by Poincaré map analysis. With fixed VPP position, stability, disturbance rejection and moderate robustness are achieved, but with low convergence speed. To improve the performances and attain higher robustness, event based control of VPP position is introduced, using feedback of the system state at apex. Dead beat control and Discrete LQR are alternatively considered to adjust the feedback gains. In both cases, considerable enhancements with respect to stability, convergence speed and robustness against perturbations are achieved.

In general, imitation is imprecisely used to address different levels of social learning from high-level knowledge transfer to low-level regeneration of motor commands. However, true imitation is based on abstraction and conceptualization. This article presents a model for conceptual imitation through interaction with the teacher to abstract spatio-temporal demonstrations based on their functional meaning. Abstraction, concept acquisition, and self-organization of proto-symbols are performed through an incremental and gradual learning algorithm. In this algorithm, Hidden Markov Models (HMMs) are used to abstract perceptually similar demonstrations. However, abstract (relational) concepts emerge as a collection of HMMs irregularly scattered in the perceptual space but showing the same functionality. Performance of the proposed algorithm is evaluated in two experimental scenarios. The first one is a human-robot interaction task of imitating signs produced by hand movements. The second one is a simulated interactive task of imitating whole body motion patterns of a humanoid model. Experimental results show efficiency of our model for concept extraction, proto-symbol emergence, motion pattern recognition, prediction, and generation.

Representing an image as a set of its key and interesting lines facilitates the image understanding and classification. In this paper, we propose a method to extract the significant and interesting lines of the scene, which probably are useful in image classification. The proposed method is inspired from the Visual Attention, which is a perceptual mechanism in human and other primates that direct their perceptions to the limited regions of the scene. The attended regions are usually valuable in performing the task. Since the approach of using the lines to classify the images is particularly useful for omnidirectional images, we specialize our method to deal with these kinds of images. In the experiments, we demonstrate how our proposed methods improve the image classification performance with processing only small parts of the input images.

Application of the compliance to increase the robustness of the bipedal walker locomotion is the main target of this paper. The control of bipedal robots with point feet is one of the most challenging problems in the domain of hybrid dynamical systems. In order to make an exponentially stable periodic walking for an Acrobot as a planar biped with only one degree of freedom, a controller is designed based on the hybrid zero dynamics analysis. Then, through appropriate alterations in robot structure, the controller complexities are reduced which result in fewer parameters for tuning. Improvement of the robot structure via compliance insertion not only compensates the lower degree of freedom of the control design process, but also expands the domain of stability of the closed-loop system. In this regard, a nonlinear damper is inserted between two legs of Acrobot to enable it to walk on a wider range of slopes. The main controller is designed for walking on a flat terrain and the compliance of the damper empowers it to make stable walking on slopes up to 17°. The simulation results confirmed the efficiency of the proposed approach.

In this letter, we propose a learning system, active decision fusion learning (ADFL), for active fusion of decisions. Each decision maker, referred to as a local decision maker, provides its suggestion in the form of a probability distribution over all possible decisions. The goal of the system is to learn the active sequential selection of the local decision makers in order to consult with and thus learn the final decision based on the consultations. These two learning tasks are formulated as learning a single sequential decision-making problem in the form of a Markov decision process (MDP), and a continuous reinforcement learning method is employed to solve it. The states of this MDP are decisions of the attended local decision makers, and the actions are either attending to a local decision maker or declaring final decisions. The learning system is punished for each consultation and wrong final decision and rewarded for correct final decisions. This results in minimizing the consultation and decision-making costs through learning a sequential consultation policy where the most informative local decision makers are consulted and the least informative, misleading, and redundant ones are left unattended. An important property of this policy is that it acts locally. This means that the system handles any nonuniformity in the local decision maker's expertise over the state space. This property has been exploited in the design of local experts. ADFL is tested on a set of classification tasks, where it outperforms two well-known classification methods, Adaboost and bagging, as well as three benchmark fusion algorithms: OWA, Borda count, and majority voting. In addition, the effect of local experts design strategy on the performance of ADFL is studied, and some guidelines for the design of local experts are provided. Moreover, evaluating ADFL in some special cases proves that it is able to derive the maximum benefit from the informative local decision makers and to minimize attending to redundant ones.

With growing attention to ensemble learning, in recent years various ensemble methods for face recognition have been proposed that show promising results. Among diverse ensemble construction approaches, random subspace method has received considerable attention in face recognition. Although random feature selection in random subspace method improves accuracy in general, it is not free of serious difficulties and drawbacks. In this paper we present a learning scheme to overcome some of the drawbacks of random feature selection in the random subspace method. The proposed learning method derives a feature discrimination map based on a measure of accuracy and uses it in a probabilistic recall mode to construct an ensemble of subspaces. Experiments on different face databases revealed that the proposed method gives superior performance over the well-known benchmarks and state of the art ensemble methods.

This paper introduces a new method for learning top-down and task-driven visual attention control along with physical actions in interactive environments. Our method is based on the Reinforcement Learning of Visual Classes(RLVC) algorithm and adapts it for learning spatial visual selection in order to reduce computational complexity. Proposed algorithm also addresses aliasings due to not knowing previous actions and perceptions. Continuing learning shows our method is robust to perturbations in perceptual information. Our method also allows object recognition when class labels are used instead of physical actions. We have tried to gain maximum generalization while performing local processing. Experiments over visual navigation and object recognition tasks show that our method is more efficient in terms of computational complexity and is biologically more plausible.

Designing an intelligent situated agent is a difficult task because the designer must see the problem from the agent's viewpoint, considering all its sensors, actuators, and computation systems. In this paper, we introduce a bio-inspired hybridization of reinforcement learning, cooperative co-evolution, and a cultural-inspired memetic algorithm for the automatic development of behavior-based agents. Reinforcement learning is responsible for the individual-level adaptation. Cooperative co-evolution performs at the population level and provides basic decision-making modules for the reinforcement-learning procedure. The culture-based memetic algorithm, which is a new computational interpretation of the meme metaphor, increases the lifetime performance of agents by sharing learning experiences between all agents in the society. In this paper, the design problem is decomposed into two different parts: 1) developing a repertoire of behavior modules and 2) organizing them in the agent's architecture. Our proposed cooperative co-evolutionary approach solves the first problem by evolving behavior modules in their separate genetic pools. We address the problem of relating the fitness of the agent to the fitness of behavior modules by proposing two fitness sharing mechanisms, namelyuniformandvalue-basedfitness sharing mechanisms. The organization of behavior modules in the architecture is determined by our structure learning method. A mathematical formulation is provided that shows how to decompose the value of the structure into simpler components. These values are estimated during learning and are used to find the organization of behavior modules during the agent's lifetime. To accelerate the learning process, we introduce a culture-based method based on our new interpretation of the meme metaphor. Our proposed memetic algorithm is a mechanism for sharing learned structures among agents in the society. Lifetime performance of the agent, which is quite important for real-world applications, increases considerably when the memetic algorithm is in action. Finally, we apply our methods to two benchmark problems: an abstract problem and a decentralized multirobot object-lifting task, and we achieve human-competitive architecture designs.

Similar to humans and primates, artificial creatures like robots are limited in terms of allocation of their resources to huge sensory and perceptual information. Serial processing mechanisms used in the design of such creatures demands engineering attentional control mechanisms. In this paper, we present a new algorithm for learning top-down sequential visual attention control for agents acting in interactive environments. Our method is based on the key idea, that attention can be learned best in concert with visual representations through automatic construction and discretization of the visual state space. The tree representing the top-down attention is incrementally refined whenever aliasing occurs by selecting the most appropriate saccadic direction. The proposed approach is evaluated on action-based object recognition and urban navigation tasks, where obtained results support applicability and usefulness of developed saccade movement method for robotics.

Most of the central pattern generator (CPG) models are based on defining explicit dynamical systems and finding the appropriate parameters. In this paper, we propose a novel CPG model that is based on altering a nonlinear oscillator to obtain desired limit cycle behavior. This CPG model benefits from an explicit basin of attraction and also fast convergence behavior. The presented CPG model is used in an imitation model that tries to learn the proper periodical behavior by looking at a mentor. First, a mentor performs the desired periodical behavior. Then, a hand-eye coordination process, inspired from infant babbling, is initiated to extract proper motor actions from what is observed. The extracted motor actions are finally embedded into the CPG model for smooth reproduction. This imitation model is implemented on a robotic marionette behavior learning task. The outcome of the final performance of the robotic marionette is behaviorally understandable smooth actions.

In this paper, we propose a fast algorithm for gesture recognition based on the saliency maps of visual attention. A tuned saliency-based model of visual attention is used to find potential hand regions in video frames. To obtain the overall movement of the hand, saliency maps of the differences of consecutive video frames are overlaid. An improved characteristic loci feature extraction method is introduced and used to code obtained hand movement. Finally, the extracted feature vector is used for training SVMs to classify the gestures. The proposed method along a hand-eye coordination model is used to play a robotic marionette and an approval/rejection phase is used to interactively correct the robotic marionette's behavior.

Similarities between foot stability in single support phase of dynamic legged locomotion and dynamic grasp during nonprehensile carrying of an object on a palm are studied. Both foot stability and dynamic grasp conditions are driven mathematically. Then it is shown that these two conditions share the same basic dynamical concepts. It is also revealed that dynamic grasp and foot stability conditions have structurally the same equations when the palm -in object manipulation- and the ground -in legged locomotion- are horizontal. In this situation, parameters of the equations are very similar. It is demonstrated that the effects of violation of foot stability and dynamic grasp conditions are behaviorally the same. In addition to analytical discussions, some simulation examples in ADAMS are provided to validate the presented models and the results.

In this paper, we present a robotic pole climber that can be used for moving up lighting poles, telephone line poles, or similar poles for the purpose of cleaning, maintenance, or inspection. The proposed design is inspired from human climbers, which use a strap and their weight to stably climb poles. The major advantage of this design is in its capability of carrying high payloads and its natural stability. The other features of the robot are its simple design, ease of control, light weight, simple mechanism and fast climbing speed.

It is generally agreed that faces are not recognized only by utilizing some holistic search among all learned faces, but also through a feature analysis that aimed to specify more important features of each specific face. This paper addresses a novel decision strategy that efficiently uses both holistic and facial component (left eye, right eye, nose and mouth) feature analysis to recognize faces. The proposed algorithm uses the whole face features in the first step of recognition task. If the decision machine fails to assign a class (with high confidence) then the individual facial components are processed and the resulting information are combined with those obtained from the whole face to assign the output. Simulation studies justify the superior performance of the proposed method as compared to that of Eigenface method. Experimental results also show that the proposed system is robust against small errors in facial component extractor.

Cooperation in learning (CL) can be realized in a multiagent system, if agents are capable of learning from both their own experiments and other agents' knowledge and expertise. Extra resources are exploited into higher efficiency and faster learning in CL as compared to that of individual learning (IL). In the real world, however, implementation of CL is not a straightforward task, in part due to possible differences in area of expertise (AOE). In this paper, reinforcement-learning homogenous agents are considered in an environment with multiple goals or tasks. As a result, they become expert in different domains with different amounts of expertness. Each agent uses a one-step Q-learning algorithm and is capable of exchanging its Q-table with those of its teammates. Two crucial questions are addressed in this paper: "How the AOE of an agent can be extracted?" and "How agents can improve their performance in CL by knowing their AOEs?" An algorithm is developed to extract the AOE based on state transitions as a gold standard from a behavioral point of view. Moreover, it is discussed that the AOE can be implicitly obtained through agents' expertness in the state level. Three new methods for CL through the combination of Q-tables are developed and examined for overall performance after CL. The performances of developed methods are compared with that of IL, strategy sharing (SS), and weighted SS (WSS). Obtained results show the superior performance of AOE-based methods as compared to that of existing CL methods, which do not use the notion of AOE. These results are very encouraging in support of the idea that "cooperation based on the AOE" performs better than the general CL methods.

Designing an intelligent situated agent is a difficult task as the designer must see the problem from the agent's standpoint considering all its sensors and actuators. We have devised a co-evolutionary/reinforcement learning hybrid method to automate the design of hierarchical behavior-based systems. In our approach, the design problem is decomposed into two separate parts: developing a repertoire of behaviors and organizing those behaviors in a structure. Mathematical formulation shows how to decompose the value of the structure to simpler components. These components can be estimated and used to find the optimal organization of behaviors during the agent's lifetime. Moreover, a novel co-evolutionary mechanism is suggested that evolves each type of behavior separately in their own genetic pool. Our method is applied to the decentralized multi-robot object lifting task which results in human-competitive performance.

A multiprocessor scheduling problem is defined as the assignment of a given set of tasks to a set of processors. These tasks should be assigned in a way such that the total execution time is minimized and certain criteria are met. A wide range of solutions and heuristics have been proposed to solve this important system optimization problem. In this paper, we propose a novel representation to solve the task scheduling problem using genetic algorithm (GA). This representation is novel not only in the way it presents task scheduling, but also in that the length of that representation is intelligently adaptable to the given problem. Task duplication is allowed in our method and it is capable of spanning a large proportion of the solution space without the need for penalty/rewards or adding repair mechanisms whilst always generating valid chromosomes. Due to this new representation, order of the search space has been reduced; consequently, the proposed approach outperforms some recently studied GA based scheduling methods over 120 times with respect to the number of fitness evaluations.

The ever-growing complexity of the modern chips is forcing fundamental changes in the way systems are designed. System-on-a-Programmable-Chip (SOPC) concept is bringing a major revolution in the design of integrated circuits, due to the fact that it makes unprecedented levels of in-field integration possible. Genetic Algorithm (GA) is a powerful function optimizer that is used successfully to solve problems in many different disciplines. A major drawback of GA is that it needs huge computation time for sequential execution on PCs. Therefore, the hardware implementation of GA has been the focus of some recent studies. Parallel GA (PGA) is particularly important for efficient hardware implementation and promise substantial gains in performance and results. In this paper, a SOPC-based PGA framework is proposed. Our proposed framework can be used in real-time applications. We have implemented our proposed system on an Altera ® Stratix Development Kit and we compare its performance with the corresponding software simulation. The results obtained indicate a speedup of up to 50 times in the elapsed computation time.

Behavior-based systems have been successfully used in control and robotics applications. In traditional behavior-based systems, only a single behavior controls the agent in any time step. However, this behavior arbitration is not appropriate for many complex tasks. In this paper, we propose Hierarchical Soft Behavior-based Architecture that uses the concept of soft suppression to coordinate flexibly between behaviors. In our method, we use reinforcement learning to find an appropriate amount of suppression for each behavior in the architecture, in addition to learn the internal mechanism of each behavior. Several experiments are provided to show the effectiveness of our method in the mobile robot navigation task.

A novel multi-agent image segmentation system for MR Images is proposed. The agents are behavior-based autonomous entities that are situated in the image as their environment. The agents have local knowledge and cooperate in an implicit and simple manner to achieve globally rational team behavior. Incremental design of agents and their simple interactions facilitate incorporation of expert knowledge and properties of the specific application in the agents' mind. The system has been applied to MRI brain scans for segmentation of three pairs of structures. The results are satisfactory in terms of both average performance and robustness. The system is also proved to be robust against initialization of the agents.

Using each other's knowledge and expertise in learning - what we call cooperation in learning- is one of the major existing methods to reduce the number of learning trials, which is quite crucial for real world applications. In situated systems, robots become expert in different areas due to being exposed to different situations and tasks. As a consequence, areas of expertise (AOE) of the other agents must be detected before using their knowledge, especially when the exchanged knowledge is not abstract, and simple information exchange might result in incorrect knowledge, which is the case for Q-learning agents. In this paper we introduce an approach for extraction of AOE of agents for cooperation in learning using their Q-tables. The evaluating robot uses a behavioral measure to evaluate itself, in order to find a set of states it is expert in. That set is used, then, along with a Q-table-based feature for extraction of areas of expertise of other robots by means of a classifier. Extracted areas are merged in the last stage. The proposed method is tested both in extensive simulations and in real world experiments using mobile robots. The results show effectiveness of the introduced approach, both in accurate extraction of areas of expertise and increasing the quality of the combined knowledge, even when, there are uncertainty and perceptual aliasing in the application and the robot

In this paper we study learning in cooperative object pushing systems. The proposed approach is based on the idea of learning individual skills and then mapping the required cooperative behaviors on the learned skills. The control point for each individual robot is chosen so as to simplify the design of the reinforcement signal and reduce the role of delayed reward on individual learning in addition to make cooperative protocol simpler. A fuzzy Q-learning system for learning individual object pushing is proposed, together with a method for coordination among the robots to push the object cooperatively. The coordination method takes into account the dynamics of the object and the learned individual skills. The idea of the coordination protocol is based on the notion of ability and active regions defined in this paper. In effect, the cooperation protocol is mapped to a Q-value based policy. Simulation results supportively show that the robots learn individual and cooperative object pushing efficiently

Decision trees, being human readable and hierarchically structured, provide a suitable mean to derive state-space abstraction and simplify the inclusion of the available knowledge for a reinforcement learning (RL) agent. In this paper, we address two approaches to combine and purify the available knowledge in the abstraction trees, stored among different RL agents in a multi-agent system, or among the decision trees learned by the same agent using different methods. Simulation results in nondeterministic football learning task provide strong evidences for enhancement in convergence rate and policy performance

This paper is concerned with the derivation of the kinematics model of the University of Tehran-Pole Climbing Robot (UT-PCR). As the first step, an appropriate set of coordinates is selected and used to describe the state of the robot. Nonholonomic constraints imposed by the wheels are then expressed as a set of differential equations. By describing these equations in terms of the state of the robot an underactuated driftless nonlinear control system with affine inputs that governs the motion of the robot is derived. A set of experimental results are also given to show the capability of the UT-PCR in climbing a stepped pole.

Imitation equips robots with a simple and natural interface to learn new tasks. Although abstraction is a remarkable feature of imitation that discriminates it from mimicking, there has been no enough research on this dimension of imitation. Relational concepts are the simplest type of abstract concepts and can be an appropriate start point. These concepts may be learned by combining perceptual categorization and classical conditioning. The paper will first formalize relational concept learning within an imitative context. Internal modules of the learning agent are considered to be functions. We will prove that in this case the concept-motor mapping becomes one-to-one which simplifies learning. A learning algorithm for the model will be also proposed and evaluated in a phoneme acquisition experiment with a large number of highly overlapped samples.

Many real world applications would involve grasp of large objects in unstructured environments. Agent-based approach to multi-robot grasp of objects would prove useful under the above circumstances. In this paper, the problem of form closure grasp for planar convex objects by multiple robots is tackled. Contrary to the previous approaches, no a priori information about the shape of the object is assumed, and the robots are not allowed to fully communicate among themselves. A distributed multi-agent based approach using Q-learning is proposed. The state space, action set and learning algorithm are formulated. The results are verified through simulations using a developed Q-learning test bed.

Hand-design of an intelligent agent's behaviors and their hierarchy is a very hard task. One of the most important steps toward creating intelligent agents is providing them with capability to learn the required behaviors and their architecture. Architecture learning in a behavior-based agent with subsumption architecture is considered in this paper. Overall value function is decomposed into easily calculate-able parts in order to learn the behavior hierarchy. Using probabilistic formulations, two different decomposition methods are discussed: storing the estimated value of each behavior in each layer, and storing the ordering of behaviors in the architecture. Using defined decompositions, two appropriate credit assignment methods are designed. Finally, the proposed methods are tested in a multi-robot object-lifting task that results in satisfactory performance.

The field of robotics is currently undergoing a change toward creation of robots that can naturally interact with humans. For achieving this, interactive robots must be endowed with natural interfaces that can sense and respond in real-time. Vision can provide handy information for this purpose by detecting and tracking human limbs to analyze gestures, actions and even emotions. However, real-time processing of visual information is a challenging bottleneck. In this paper, we introduce a novel method, namely "self-organized contours", that can distinctly accelerate contour initialization, which is the slowest phase in visual tracking. Although the proposed method is general-purpose, it allows immediate initialization of active contours due to its similarity with snake structure. The proposed method is inspired from group behavior in insects and animals, particularly fishes.

A distributed and cooperative fault-clearing strategy for a team of object lifting robots is introduced It is assumed that one of the robots misses a portion of its lifting power and the robots must redistribute the load among themselves to perform their task. Two distributed and cooperative methods for the load reallocation among the position-controlled robots without requiring them to change their grasp positions are introduced. The first method benefits from the existing redundancy in the number and in the lifting power of the robots. In the second method, the object is tilted in order to move the ZMP (Zero Moment Point) away from the faulty robot and, consequently, redistributing the load. Difficulties in controlling ZMP movements are pointed out. Therefore, the second fault clearing procedure is designed such that the ZMP position is controlled without resorting to sophisticated or centralized control algorithms. The stability of the proposed methods is discussed. Simulation results are given to support the developed methods.

By using other agents' experiences and knowledge, a learning agent may learn faster, make fewer mistakes, and create some rules for unseen situations. These benefits would be gained if the learning agent can extract proper rules from the other agents' knowledge for its own requirements. One possible way to do this is to have the learner assign some expertness values (intelligence level values) to the other agents and use their knowledge accordingly. Some criteria to measure the expertness of the reinforcement learning agents are introduced. Also, a new cooperative learning method, called weighted strategy sharing (WSS) is presented. In this method, each agent measures the expertness of its teammates and assigns a weight to their knowledge and learns from them accordingly. The presented methods are tested on two Hunter-Prey systems. We consider that the agents are all learning from each other and compare them with those who cooperate only with the more expert ones. Also, the effect of communication noise, as a source of uncertainty, on the cooperative learning method is studied. Moreover, the Q-table of one of the cooperative agents is changed randomly and its effects on the presented methods are examined.

Studying the system dynamics, this research is an attempt to understand and design the basic robot behavior, information system, and distributed cooperation strategies required by a group of cooperative behavior-based mobile robots for handling an object. A new concept to develop distributed cooperation strategies to carry a load is introduced. In this method, the task of transferring the object is divided into two independent executable subtasks: constraining and moving the load. Each subtask is assigned to a group of distributed mobile robots. Based on this idea (constrain-move concept), two distributed cooperation strategies to turn the object about a fixed point and to move it along a straight line are introduced. The constrain-move concept is generalized for lifting and lowering the object. It is noted that weight of the payload can be used by the robots as a natural constraint on the object. Utilizing this natural constraint, a distributed cooperation strategy and an information system to lift and lower the object are introduced. The coordination protocols are devised in such a way that the robots can control movements of the object using their own sensory information and some static data communicated between the team members. Simulation and experimental results are given to support the proposed approach.

Cooperative learning in a multi-agent system can improve the learning quality and learning speed. The improvement can be gained if each agent detects the expert agents and uses their knowledge properly. In the paper, a cooperative learning method, called weighted strategy sharing (WSS) is introduced. Also some criteria are introduced to measure the expertness of agents. In WSS, based on the amount of its team-mate expertness, each agent assigns a weight to their knowledge. These weights are used in sharing knowledge among agents in our system. WSS and the expertness criteria are tested on two simulated hunter-prey problems and on object pushing systems.

This paper extends the constrain-move strategy for four robots lifting an object in a distributed manner. Ahmadabadi et al. (1996, 1998) have successfully applied constrain-move to three robots, but due to existence of redundancy in vertical forces in a team of four robots, this protocol is not applicable to such systems in its original form. In the presented cooperation strategy, the robots support the load on its bottom face and there is no lateral artificial constraint on the object. The distributed coordination protocol is designed in such a way that with minimum volume of communication and implementing their own local sensors, the robots can lift the object while its Euler angles and robot-object contact are controlled. Stability condition for the system is attained mathematically. Simulation results are given to support the applicability of the presented system.

Describes a multi-robot system which incorporates a behavior-based dynamic cooperation strategy for object transportation with various requirements on task performance. This cooperation strategy is realized in two steps: designing the distributed robot's cooperative behavioral attributes according to its abilities and task dynamics, and organizing these behavioral attributes so that the team cooperation is realized. Through the behavior design, this strategy allows the dynamic cooperation control to be distributed equally to each robot. Constraint-based behavior is designed to cope with the requirements of task performance. In organizing robots' abilities, the concept of form closure is introduced as the basic strategy and additional closure rules are introduced to realize various constrained motions.

Based on the constrain-move concept, previously introduced by the authors (1997), a unified approach to develop cooperation strategies for a group of multiple robots to lift/lower the object and to carry it along a desired path is introduced. In this method, when transferring the load, a group of robots, called constrain making robots, constrain the object's movements which are orthogonal to its desired path. To lift/lower the object, these robots confine angular and horizontal movements of the load by constraining its sides. The remaining robots push the object toward its goal. It is discussed that, the weight of the load can be used by the robots as a natural constrain on the object. Doing so, there is no need for the robots to confine the object any more. Instead, the object lifting robots must lift the object in such a way that the natural constraint on the object is held. A distributed cooperation strategy for the robots to fulfil this requirement is reexamined. Some experimental results are also given to show the effectiveness of the proposed approach.

A new concept to develop distributed cooperation strategies for a group of object handling mobile robots to carry the load is introduced. In the proposed method, the task of transferring the object is divided into two independent subtasks; constraining and moving the load. Then, each subtask is assigned to a group of robots. Based on the proposed idea, a distributed cooperation strategy for a team of robots to turn an object about a defined point is presented. This cooperation protocol is designed in such a way that a group of robots constrain the object passively. The remaining robots undertake pushing the object in order to turn it. Validity of the proposed cooperation protocol is verified through computer simulations. Using the simulation results, effect of incorporating compliance elements in the robot arms on preventing the system from jamming, caused by error in the position of the robots, is also discussed.

Based on the constrain-move concept previously proposed by the authors (1997), two distributed cooperation strategies for a group of multiple mobile robots to turn an object about a fixed point and move it along a straight line are introduced. In these protocols, some of the robots passively constrain the object and the other ones move the load. It is assumed that the robots grasp the object through some friction contacts. The cooperation strategies are designed in such a way that the task of stabilizing the robot-object contact is simplified. Moreover, task allocation in the group is performed in such a way that the size of the robot team is reduced. Information system and robot behavior required by the team transferring the object are also discussed. In addition, simulation and experimental results are reported to show the effectiveness and applicability of the proposed approach.

The ultimate goal of this research is to study information, cooperation strategies, and behaviors required by a group of behavior based mobile robots for lifting and transferring an object with unknown mass and center of gravity. This paper presents some of the investigation results of the first stage of the project, lifting the object. After discussing the problems facing a group of object lifting robots, a distributed cooperation strategy, sensory system, and required behaviors for multiple cooperative object lifting robots are introduced. In the proposed cooperation strategy, when the object's tilt angle passes a defined value, the robot(s) having the lowest contact point with the object moves upward faster while the others slow down and stop. The sensory system is designed so that each robot is able to measure the object's tilt angle and communicate with the other robots when required. Sufficient conditions for applicability of the proposed algorithm are provided and the validity of the studied approach is verified through computer simulations and experiments.