One of the simplest approach in Reinforcement Learning (RL) is updating Q-table using Bellman operator. While theoretical expectations hint at the potential convergence achieved by modeling the discrete Q-table with the Bellman operator, practical limitations surface in real-world scenarios. The main challenges associated with it include the exponential growth of the Q-table size with an increasing number of state dimensions and the inability to use the Q-table in continuous state spaces. Alternative approaches, such as employing neural networks to approximate the parameterized Q-function, may not necessarily result in convergence.In response to these challenges, this paper introduces an simple innovative methodology inspired by the Bellman method updating. The proposed method utilizes fuzzy rules to discretize the state space, leading to the direct use of the Bellman operator for updating the fuzzy neural network weights, effectively acting as the Fuzzy Q-table. Instead of approximating the Q-function utilizing neural network/deep neural network based on gradient approaches, the proposed method establishes a Fuzzy Q-table and updates it using the Bellman equation. This strategic decision helps to solve the convergence problem in addition to prevent entrapment in local minima problems, a common challenge faced by conventional gradient methods. The efficacy of the proposed approach is demonstrated through its application to trading in the Bitcoin Futures Market, showcasing its ability to navigate complexities and uncertainties. Beyond financial markets, this methodology presents a versatile solution applicable to a diverse range of reinforcement learning problems, addressing limitations faced by traditional Q-tables or DQN.

Digital computers are built of analog circuit components that "twin" conceptual digital entities such as switches or logic gates. Employing such "analog doppelgangers" to twin conceptual discrete components modulo thresholding and clocking constraints was a necessary step in implementing modern digital computer. These digital computers have gone on to be so successful that in many circles the word "digital" has almost become synonymous with "computational". We may see some of this effect even the term "digital twin". This paper speculatively addresses the idea that the twin need not be digital in the strictest sense so long as it actually models, indistinguishably, the behavior of some other system. We will draw from previous work using fully-recurrent continuous time and continuous valued neural networks to twin optimal controllers for a simplified legged-locomotion agent. We will then argue that the use of "analog doppelgangers" to twin systems directly has benefits and utility that strictly digital twins may not possess in equal measure. Finally, we will examine and discuss what types of practical problems might be positively impacted by this alternative view.

Algorithm selection and hyperparameter tuning are critical steps in both academic and applied machine learning (ML). These steps are becoming increasingly delicate due to the extensive rise in the number, diversity, and distributed nature of ML resources. Multi-agent systems, when applied to the design of ML platforms, bring about several distinctive characteristics, such as scalability, flexibility, and robustness, just to name a few. This article proposes a fully automatic and collaborative agent-based mechanism for selecting distributed ML algorithms and simultaneously tuning their hyperparameters. Our method builds upon an existing agent-based hierarchical ML platform and augments its query structure to support the aforementioned functionalities without being limited to specific learning, selection, and tuning mechanisms. We have conducted theoretical assessments, formal verification, and analytical study to demonstrate the correctness, resource utilization, and computational efficiency of our technique. According to the results, our solution is algorithmically correct and exhibits linear time and space complexity in relation to the size of available resources. To further verify its correctness and demonstrate its effectiveness and flexibility across a range of algorithmic options and datasets, the article also presents a series of empirical results on a system composed of 24 algorithms and 9 datasets. The findings not only highlight the efficiency and scalability of the proposed approach, but also show its flexibility and openness to responding to the dynamic and distributed ML ecosystem.

Human resources are a necessary cost for the search and rescue of people lost or stranded in the forest, which cannot be easily accessed in person. UAVs can be used to observe forests from up in the air without visiting directly. While UAVs are fly searching for people, they cannot be easily detected due to being too small and potentially being overlapped by obstacles such as grass, cars, and even the shadows of trees. We focus on detecting small objects even overlapped or hindered in the field and design an enhanced human detection Network (EHDNet) focusing on small objects. EHDNet enhances detection performance via an attentive feature pyramid network (AFPN) focused on small objects.

Cyber-Physical Systems (CPS) have a physical part that can interact with sensors and actuators.The data that is read from sensors and the one generated to drive actuators is crucial for the correct operation of this class of devices.Most implementations trust the data being read from sensors and the outputted data to actuators.Real-time validation of the input and output of data for any system is crucial for the safety of its operation.This paper proposes an architecture for handling this issue through smart data guards detached from sensors and controllers and acting solely on the data.This mitigates potential issues of malfunctioning sensors and intentional sensor and controller attacks.The data guards understand the expected data, can detect anomalies and can correct them in real-time.This approach adds more guarantees for fault-tolerant behavior in the presence of attacks and sensor failures.

Hierarchical Multi-agent Systems provide convenient and relevant ways to analyze, model, and simulate complex systems composed of a large number of entities that interact at different levels of abstraction. In this article, we introduce HAMLET (Hierarchical Agent-based Machine LEarning plaTform), a hybrid machine learning platform based on hierarchical multi-agent systems, to facilitate the research and democratization of geographically and/or locally distributed machine learning entities. The proposed system models machine learning solutions as a hypergraph and autonomously sets up a multi-level structure of heterogeneous agents based on their innate capabilities and learned skills. HAMLET aids the design and management of machine learning systems and provides analytical capabilities for research communities to assess the existing and/or new algorithms/datasets through flexible and customizable queries. The proposed hybrid machine learning platform does not assume restrictions on the type of learning algorithms/datasets and is theoretically proven to be sound and complete with polynomial computational requirements. Additionally, it is examined empirically on 120 training and 4 generalized batch testing tasks performed on 24 machine learning algorithms and 9 standard datasets. The provided experimental results not only establish confidence in the platform’s consistency and correctness but also demonstrate its testing and analytical capacity.

In autonomous driving research, one of the bottlenecks is the shortage of a well-annotated dataset to train deep neural networks for object detection. Specifically, a dataset focusing on harsh weather conditions is insufficient. The purpose of this research is to explore the power of utilizing synthetic data for training object detection deep neural networks under harsh weather conditions. We introduce a state-of-the-art automated pipeline to collect synthetic images from a high realism video game and generate training data which can be used for training an autonomous driving object detection neural network. We use our synthetic dataset, KITTI, and Cityscapes to train three separate object detection neural networks and employ the PASCAL object detection criteria to evaluate each neural networks' performance. The results from the experiment indicate that the neural network trained by our synthetic dataset outperforms its counterparts and achieves higher average precision (AP) in detecting images under harsh weather conditions. The result sheds a light on employing synthetic data to resolve the challenges in the real world.

This paper introduces new methods for gaining sensitive information from Internet of Things (IoT) devices. Generally, manufacturers tend to forget about security when putting a new product on the market, this notion does not exclude IoT. This has been proven by current research using simple techniques and devices to obtain unencrypted information. Using similar attacks, the IoT hubs themselves was chosen as a target instead of the devices connected to them. Two different attacks will be used. First, using various methods for sniffing Hub traffic, we will attempt to gain credentials on the uplink side. Secondly, port scans will be used to gain any exploited services in order to obtain root access. After our attempts, it was found that IoT hubs can be very susceptible to tech savvy attackers using simple Man in the Middle attacks. If network access was achieved, the security of ones home is greatly compromised allowing intruders an 8-10 minute window to enter a house undetected. Using modest methodologies, consumers and companies can prevent future exploits.

Previously, we introduced Evolutionary Model Consistency Checking (EMCC) as an adjunct to Evolvable and Adaptive Hardware (EAH) methods. The core idea was to dual-purpose objective function evaluations to simultaneously enable EA search of hardware configurations while simultaneously enabling a model-based inference of the nature of the damage that necessitated the hardware adaptation. We demonstrated the efficacy of this method by modifying a pair of EAH oscillators inside a simulated Flapping-Wing Micro Air Vehicle (FW-MAV). In that work, we were able to show that one could, while online in normal service, evolve wing gait patterns that corrected altitude control errors cause by mechanical wing damage while simultaneously determining, with high precision, what the wing lift force deficits that necessitated the adaptation. In this work, we extend the method to be able to also determine wing drag force deficits. Further, we infer the now extended set of four unknown damage estimates without substantially increasing the number of objective function evaluations required. In this paper we will provide the outlines of a formal derivation of the new inference method plus experimental validation of efficacy. The paper will conclude with commentary on several practical issues, including better containment of estimation error by introducing more in-flight learning trials and why one might argue that these techniques could eventually be used on a true free-flying flapping wing vehicle.

This paper introduces the design and experiments of a gas tracking algorithm, named a circular movement algorithm, which is for natural indoor environment without a strong and constant wind. A graph of gas concentration shows tumultuous flow under the condition [8]. To solve the turbulent gas concentration problem, we defined cumulative-gas concentration (CGC) value to compute gradient of gas concentration and compare gas concentration values. Using the circular movement algorithm, the robot performs circular movement and collects gas concentration data to determine directions of gas source. The algorithm makes the robot able to track the directions of the gas source. This paper also introduces a tracking method using relative coordinate system for the tracking robot in the gas leaking circumstances which can calculate the location of the gas leakage without an absolute coordinate of a room or a building. As a result of experiments, the robot was able to detect a location of possible gas source by using the proposed circular movement algorithm and the relative coordinate system.

The next generation of ambient assisted living services will be based on eco-systems or organizations of intelligent artificial agents embodied in companion robots and smart objects. To provide, anywhere and anytime, smart assistance services to people, these agents need to be endowed with advanced knowledge representation, reasoning and communication capabilities. In this paper, we propose a distributed cognitive architecture allowing to integrate seamlessly the actors of the ambient system and an expressive model for commonsense knowledge representation and reasoning on events. This model allows a common description of the actors in an open world and a management of interactions with humans using natural language. A scenario dedicated to the cognitive assistance of frail people is implemented and analyzed for validation purposes of the proposed approach.

To quantify social tags' relatedness in an image collection, we examine the betweenness centrality measure. We depict the image collection as a multi-graph representation, where nodes are the social tags and edges bind an image's social tags. We present our weighted betweenness centrality algorithm and compare it to the unweighted version on sparse and dense graphs. The MIRFLICKR and ImageCLEF benchmark image collections are used in our experimental evaluation. We notice an 11% increase in the computation runtime with weighted edges in determining shortest paths within our image collections. We discuss the intended impact of our approach in conjunction with a node importance evaluation, via the k-path centrality algorithm, for determining situation-aware path planning applications.

Ambient intelligence, ubiquitous and networked robots, cloud robotics, are new research hot topics that start to gain popularity among the robotics community. They enable robots to acquire richer functionalities and open the way for the composition of a variety of robotic services with three functions: semantic perception, reasoning and actuation. This paper introduces the recent challenges and future trends of these topics.

As robots become more pervasive and ubiquitous in the lives of humans, they become increasingly involved in everyday tasks formerly executed by humans. Humans should expect robots to take on tasks to simplify our lives, by working with humans just as other humans do, in normal organizations and societies. This labor specialization allows humans more comfort, time or focus on higher level desires or tasks. To further this unification of relationships, the defined line between humans and other non-humans must become more indistinguishable. This ever increasing degree of indistin-guishability provides we care less about who or what executes a task or solves a goal, as long as that entity is capable and available. In this paper, we propose a model and a simple example implementation which minimizes the strict line between humans, software agents, robots, machines and sensors (HARMS) and reduces the distinguishability between these actors.

Firefighters put themselves in harm's way while saving others and may even lose their lives in certain situations, such as toxic fumes, extreme heat, or inhaling smoke. In order to protect firefighters from the risks and, at the same time, to save others' lives, firefighting and firefighter assistant robots have been developed. This paper will compare different firefighting and firefighter assistant robots and their functionalities. The main thrust of the paper, however, is to discuss the transition from tele-operated robots first to voice-operated and, eventually, to fully autonomous ones, which is where robotic intelligence resides. We will introduce HARMS, the human-agent-robot-machine-sensor collaborative effort, and explain why using natural language as the basis of their communication is not only optimal but also feasible and affordable with the Ontological Semantic Technology.

It has been said that the only constant in life is change. This rule can also be directly applied to the lives of organizations. Any organization of non-trivial size, scope, life expectancy or function, is destined to change. An organization without

In harsh or dangerous environments, robots can lose function in multiple sensors and effectors over the mission, thus reducing their overall capability. If the capabilities provided by these sensors/effectors are necessary for mission completion, having an adaptive system that can overcome these losses is critical to mission accomplishment. In this paper, we propose a solution using multiple agents, organized as a team, to give robots the ability to adapt and overcome sensor/effector loss. When a sensor/effector is lost, the team can reorganize to provide the robot highest operational utility, given its current capabilities. The robot can adapt to such losses by substituting other sets of sensors/effectors to provide the best overall capability. While the robot may operate at a lower level of effectiveness, it will be able continue its mission, if possible.

The use of multiple robots to accomplish tasks has been common for many years. The term, cooperative, is used to designate the nature of the interactions among a team of robots. However, there are many differences in how these robots interact. Some groups of robots appear, on the surface, to cooperate but are not actually aware of the other robots. Other groups share goals and objectives. Our research investigated a diverse collection of multiple-robot systems. This paper highlights eight examples. The variations of interaction that we found were used to create a robotic-interaction taxonomy. Each robotic team is measured against three dimensions for placement into the taxonomy. Terminology for different types of cooperation is suggested.

This paper reports our progress in applying multiagent systems analysis and design in the area of cooperative robotics. In this paper, we apply the Multiagent Systems Engineering (MaSE) methodology to design a team of autonomous, heterogeneous search and rescue robots. MaSE provides a top-down approach to building cooperative robotic systems instead of the bottom-up approach employed in most robotic implementations. We follow the MaSE steps and discuss various approaches and their impact on the final system design.