Creating an immutable repository for vital robot and environmental data, ensuring long-term accessibility, and functioning in the absence of GPS or mapping are crucial for visual homing navigation systems. We focus on the intersection of blockchain and robotics, particularly in visual homing. Our research involves an in-depth analysis of various blockchain consensus mechanisms, highlighting their suitability for visual homing applications. The heart of blockchain functionality lies in its consensus mechanism, which facilitates agreement among network nodes. In our first study part, we conduct a comprehensive comparative analysis of key consensus algorithms, emphasizing visual homing’s decentralization, fault tolerance, latency, and throughput requirements. This analysis serves as a valuable reference for researchers and developers, emphasizing the importance of aligning the chosen consensus mechanism with specific blockchain application needs. The second part of our work involves extensive experiments exploring the connection between blockchain and visual homing. We assess prominent consensus mechanisms like Proof of Work (PoW), Proof of Stake (PoS), Delegated Proof of Stake (DPoS), and Proof of Authority (PoA) within a virtual environment in Gazebo, leveraging wide area visual navigation (WAVN). Our research implementation is grounded in the ROS framework and the Gazebo simulation environment.

This paper introduces a novel approach to GPS-denied visual navigation of a robot team over a wide (i.e., out of line of sight) area which we call WAVN (Wide Area Visual Navigation). Application domains include small-scale precision agriculture as well as exploration and surveillance. The proposed approach requires no exploration or map generation, merging, and updating, some of the most computationally intensive aspects of multi-robot navigation, especially in dynamic environments and for long-term deployments. In contrast, we extend the visual homing paradigm to leverage visual information from the entire team to allow a robot to home to a distant location. Since it only employs the latest imagery, the approach can be resilient to the current state of the environment. WAVN requires three components: identification of common landmarks between robots, a communication infrastructure, and an algorithm to find a sequence of common landmarks to navigate to a goal. The principal contribution of this paper is the navigation algorithm in addition to simulation and physical robot results characterizing performance. The approach is also compared to more traditional map-based approaches.

It is increasingly common for software developers to leverage the features and ease-of-use of different languages in building software systems. Nonetheless, interaction between different languages has proven to be a source of software engineering concerns. Existing static analysis tools handle the software engineering concerns of monolingual software but there is little general work for multilingual systems despite the increasing visibility of these systems. While recent work in this area has greatly extended the scope of multilingual static analysis systems, the focus has still been on a primary, host language interacting with subsidiary, guest language functions. In this paper we propose a novel approach that does not privilege any one language and has a modular way to include new languages. We present an approach to multilingual taint analysis (a security oriented static analysis method) as a ‘meta-level’ algorithm which includes monolingual static analysis as a special case. A complexity analysis of the taint analysis algorithm is presented along with a detailed ‘deep’ multilingual example with Python and C/C++ software. A performance analysis is presented on a collection of 20 public, multilingual repositories selected from github. Our results show an average of 76% improved coverage using our algorithm when compared to monolingual taint analysis.

Large software systems often comprise programs written in different programming languages. In the case when cross-language interoperability is accomplished with a Foreign Function Interface (FFI), for example pybind11, Boost.Python, Emscripten, PyV8, or JNI, among many others, common software engineering tools, such as call-graph analysis, are obstructed by the opacity of the FFI. This complicates debugging and fosters potential inefficiency and security problems. One contributing issue is that there is little rigorous software design advice for multilingual software. In this paper, we present our progress towards a more rigorous design approach to multilingual software. The approach is based on the existing approach to the design of large-scale C++ systems developed by Lakos. The FFI is an aspect of physical rather than logical architecture. The Lakosian approach is one of the few design methodologies to address physical design rather than just logical design. Using the MLSA toolkit developed in prior work for analysis of multilingual software, we focus in on one FFI – the pybind11 FFI. An extension to the Lakosian C++ design rules is proposed to address multilingual software that uses pybind11. Using a sample of 50 public GitHub repositories that use pybind11, we measure how many repositories would currently satisfy these rules. We conclude with a proposed generalization of the pybind11-based rules for any multilingual software using an FFI interface.

Artificially intelligent assistive agents are playing an increased role in our work and homes. In contrast with currently predominant conversational agents, whose intelligence derives from dialogue trees and external modules, a fully autonomous domestic or workplace robot must carry out more complex reasoning. Such a robot must make good decisions as soon as possible, learn from experience, respond to feedback, and rely on feedback only as much as necessary. In this research, we narrow the focus of a hypothetical robot assistant to a room-tidying task in a simulated domestic environment. Given an item, the robot chooses where to put it among many destinations, then optionally receives feedback from a human operator. We frame the problem as a contextual bandit, a reinforcement learning approach frequently used in Web recommendation systems. We evaluate epsilon-greedy and LinUCB action selection methods under a variety of infrequent feedback scenarios, with several methods for managing the lack of feedback. Our empirical results show that, while early-episode performance and overall accuracy of epsilon-greedy action selection can be improved through learning from no-response feedback and careful management of remembered training episodes, a baseline LinUCB approach outperforms epsilon-greedy action selection in early-episode performance, overall accuracy, and simplicity.

editorial Free Access Share on Dependability in Cyber-Physical Systems and Applications Authors: Md Zakirul Alam Bhuiyan Fordham University, New York, NY, USA Fordham University, New York, NY, USAView Profile , Sy-yen Kuo National Taiwan University, Taipei, Taiwan National Taiwan University, Taipei, TaiwanView Profile , Damian Lyons Fordham University, New York, NY, USA Fordham University, New York, NY, USAView Profile , Zili Shao The Hong Kong Polytechnic University, Hung Hom, Hong Kong The Hong Kong Polytechnic University, Hung Hom, Hong KongView Profile Authors Info & Claims ACM Transactions on Cyber-Physical SystemsVolume 3Issue 1January 2019 Article No.: 1pp 1–4https://doi.org/10.1145/3271432Published:29 September 2018Publication History 4citation680DownloadsMetricsTotal Citations4Total Downloads680Last 12 Months116Last 6 weeks9 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteView all FormatsPDF

Analysis of multilingual codebases is a topic of increasing importance. In prior work, we have proposed the MLSA (MultiLingual Software Analysis) architecture, an approach to the lightweight analysis of multilingual codebases, and have shown how it can be used to address the challenge of constructing a single call graph from multilingual software with mutual calls. This paper addresses the challenge of constructing monolingual call graphs in a lightweight manner (consistent with the objective of MLSA) which nonetheless yields sufficient information for resolving language interoperability calls. A novel approach is proposed which leverages information from a compiler-generated AST to provide the quality of call graph necessary, while the program itself is written using an Island Grammar that parses the AST providing the lightweight aspect necessary. Performance results are presented for a C/C++ implementation of the approach, PAIGE (Parsing AST using Island Grammar Call Graph Emitter) showing that despite its lightweight nature, it outperforms Doxgen, is robust to changes in the (Clang) AST, and is not restricted to C/C++.

Developer preferences, language capabilities and the persistence of older languages contribute to the trend that large software codebases are often multilingual – that is, written in more than one computer language. While developers can leverage monolingual software development tools to build software components, companies are faced with the problem of managing the resultant large, multilingual codebases to address issues with security, efficiency, and quality metrics. The key challenge is to address the opaque nature of the language interoperability interface: one language calling procedures in a second (which may call a third, or even back to the first), resulting in a potentially tangled, inefficient and insecure codebase. An architecture is proposed for lightweight static analysis of large multilingual codebases – the MLSA architecture. Its modular and table-oriented structure addresses the open-ended nature of multiple languages and language interoperability APIs. We focus here as an application on the construction of call-graphs that capture both inter-language and intra-language calls. The algorithms for extracting multilingual call-graphs from codebases are presented, and several examples of multilingual software engineering analysis are discussed. The state of the implementation and testing of MLSA is presented, and the implications for future work are discussed.

Navigation is challenging for an autonomous robot operating in an unstructured environment. Visual homing is an AI local navigation technique used to direct a robot to a previously seen location, and is inspired by biological models. Most visual homing uses a panoramic camera. Prior work has shown that exploiting depth cues in homing from, e.g., a stereo-camera, leads to improved performance. However, many stereo-cameras have a limited field of view (FOV). We present a stereovision database methodology for visual homing. We use two databases we have collected, one indoor and one outdoor, to evaluate the effect of FOV on the performance of our homing with stereovision algorithm. Based on over 120,000 homing trials, we show that contrary to intuition, a panoramic field of view does not necessarily lead to the best performance, and we discuss the implications of this.

A key challenge in the automatic verification of robot mission software, especially critical mission software, is to be able to effectively model the performance of a human operator and factor that into the formal performance guarantees for the mission. We present a novel approach to modelling the skill level of the operator and integrating it into automatic verification using a linear Gaussians model parameterized by experimental calibration. Our approach allows us to model different skill levels directly in terms of the behavior of the lumped, robot plus operator, system. Using MissionLab and VIPARS (a behavior-based robot mission verification module), we present a comparison of our predicted performance guarantees for two missions in which a teleoperated quadrotor identifies a target for an autonomous ground robot to intercept: one mission in which the operator flies the quadrotor by line of sight to locate the target and one where the operator flies the quadrotor using its video feed. We demonstrate the effectiveness of our approach by comparing predicated performance to experimentally measured performance.

Localization and mapping algorithms can allow a robot to navigate well in an unknown environment. However, whether such algorithms enhance any specific robot mission is currently a matter for empirical validation. In this paper we apply our MissionLab/VIPARS mission design and verification approach to an autonomous robot mission that uses probabilistic localization software. Two approaches to modeling probabilistic localization for verification are presented: a high-level approach, and a sample-based approach which allows run-time code to be embedded in verification. Verification and experimental validation results are presented for two different missions, each using each method, demonstrating the accuracy of verification, and both are compared with verification of an odometry-only mission, to show the mission-specific benefit of localization.

The effective use of autonomous robot teams in highly-critical missions depends on being able to establish performance guarantees. However, establishing a guarantee for the behavior of an autonomous robot operating in an uncertain environment with obstacles is a challenging problem. This paper addresses the challenges involved in building a software tool for verifying the behavior of a multi-robot waypoint mission that includes uncertain environment geometry as well as uncertainty in robot motion. One contribution of this paper is an approach to the problem of a-priori specification of uncertain environments for robot program verification. A second contribution is a novel method to extend the Bayesian Network formulation to reason about random variables with different subpopulations, introduced to address the challenge of representing the effects of multiple sensory histories when verifying a robot mission. The third contribution is experimental validation results presented to show the effectiveness of this approach on a two-robot, bounding overwatch mission.

Certain robot missions need to perform predictably in a physical environment that may have significant uncertainty. One approach is to leverage automatic software verification techniques to establish a performance guarantee. The addition of an environment model and uncertainty in both program and environment, however, means that the state space of a model-checking solution to the problem can be prohibitively large. An approach based on behavior-based controllers in a process-algebra framework that avoids state-space combinatorics is presented here. In this approach, verification of the robot program in the uncertain environment is reduced to a filtering problem for a Bayesian network. Validation results are presented for the verification of a multiple-waypoint and an autonomous exploration robot mission.

We have developed an approach that can be used by mission designers to determine whether or not a performance guarantee for their mission software, when carried out under the uncertain conditions of a real-world environment, will hold within a threshold probability. In this paper we demonstrate its utility for verifying multirobot missions, in particular a bounding overwatch mission.

Certain robot missions need to perform predictably in a physical environment that may only be poorly characterized in advance. We have previously developed an approach to establishing performance guarantees for behavior-based controllers in a process-algebra framework. We extend that work here to include random variables, and we show how our prior results can be used to generate a Dynamic Bayesian Network for the coupled system of program and environment model. Verification is reduced to a filtering problem for this network. Finally, we present validation results that demonstrate the effectiveness of the verification of a multiple waypoint robot mission using this approach.

This paper describes the need and methods required to construct an integrated software verification and mission specification system for use in robotic missions intended for counter-weapons of mass destruction (c-WMD) operations, as part of a 3-year effort for the Defense Threat Reduction Agency. The overall system architecture is described. The principal tool for verification is a process algebra, PARS, based on port automata theory. PARS is introduced, emphasizing its ability to represent probabilistic programs and uncertain and dynamic environments, followed by the analysis of mission properties for an example robotic mission.

A team of robots cooperating to quickly produce a map needs to share landmark information between members so that the local maps can be accurately merged. However, the appearance of landmarks as seen by members of the team can change dramatically due to the phenomenon of occlusion. We have previously presented an approach to landmark representation using Terrain Spatiograms an extension to image spatiograms in which the spatial information relates to the scene rather than the image. Because this representation preserves depth structure, it is possible to identify and filter potential occlusions. We present an approach to identifying and filtering occlusions using terrain spatiograms, and report experimental results on 20 landmark datasets for varying states of occlusion. We show that occlusion can be detected and filtered, resulting in improved landmark matching scores.

A team of robots working to explore and map an area may need to share information about landmarks so as to register their local maps and to plan effective exploration strategies. In previous papers we have introduced a combined image and spatial representation for landmarks: terrain spatiograms. We have shown that for manually selected views, terrain spatiograms provide an effective, shared representation for occlusion filtering and for combination of multiple views. In this paper, we present a landmark saliency architecture (LSA) that will automatically select candidate landmarks given some preference settings. Using a dataset of 21 outdoor stereo images generated by LSA, we show that the terrain spatiogram representation reliably recognizes automatically selected landmarks. The terrain spatiogram results are shown to improve on two purely appearance based approaches: template matching and image histogram matching.

In this paper we evaluate the use of a novel spatial histogram, the terrain spatiogram, as a common representation for exchanging landmark information between robots working as a team to map an area. Individual robots use range sensors to provide the spatial dimension of the spatiogram and video for the image dimension. We have previously shown that terrain spatiograms can be shared between robots in a heterogeneous team to recognize landmarks and to fuse observations from multiple sensors or multiple platforms. A terrain spatiogram using a mixture of Gaussians (MOG) model is introduced and a corresponding normalized spatiogram similarity measure defined. Two methods to generate a MOG terrain spatiogram are presented and compared experimentally using indoor and outdoor landmark information transferred between two different models of robot equipped with differently configured stereocameras.

We address the problem of sensor fusion for stereo and ultrasound depth measurements for map building for a robot operating in a cluttered environment. In such a situation it's difficult to make useful and realistic assumptions about the sensor or environment statistics. Combinatorial Fusion Analysis is used to develop an approach to fusion with unknown sensor and environment statistics. A metric is proposed that shows when fusion from a set of fusion alternatives will produce a more accurate estimation of depth than either sonar or stereo alone and when not. The metric consists of two criteria: (a) the performance ratio PR(A,B) between sensors A and B, and (b) the diversity d(A,B) between A and B as captured by the rank-score function fA and fB. Experimental results are reported to illustrate that these two CFA criteria are viable predictors to distinguish between positive cases (the combined system performs better than or equal to the individual systems) and negative cases.

The ADAPT project is a collaboration of researchers in linguistics, robotics and artificial intelligence at three universities. We are building a complete robotic cognitive architecture for a mobile robot designed to interact with humans in a range of environments, and which uses natural language and models human behavior. This paper concentrates on the HRI aspects of ADAPT, and especially on how ADAPT models and interacts with humans.

We address the problem of automated video tracking of targets when targets undergo multiple mutual occlusions. Our approach is based on the idea that as targets are occluded, selection of feature subsets and combinations of those features are effective in identifying the target and improving tracking performance. We use combinatorial fusion analysis to develop a metric to select which subset of features will produce the most accurate tracking. In particular we show that the combination of a pair of features A and B will improve the accuracy only if (a) A and B have relative high performance, and (b) A and B are diverse. We present experimental results to illustrate the performance of the proposed metric

In many useful video tracking situations, targets move through repeated mutual occlusions. As targets undergo occlusions, the feature subsets and combinations of those features that are effective in identifying the target and improving tracking performance may change. We use combinatorial fusion analysis to select and evaluate criteria by which to identify the combination of features that will produce the most accurate tracking. In particular we show that the combination of a pair of features A and B will improve the accuracy only if (a) A and B have relative high performance, and (b) A and B are diverse. We present experimental results from three diverse video sequences to illustrate the performance of the proposed criteria

Automated video tracking is useful in a number of applications such as surveillance, multisensor networks, robotics and virtual reality. In this paper we investigate an approach to tracking based on fusing the output of a collection of video trackers, each attending to a different feature or cue on the target. We show both theoretically and experimentally that the method used to prune the growth of target hypotheses can have a great impact on the trackers performance, and indirectly, change the benefit of using linear score combination as opposed to a non-linear rank combination for fusion. We also show that the rank-score graph defined by Hsu and Taksa can be used to select a subset of features to fuse to reduce classification error.

An attractive approach to improve tracking performance for visual surveillance is to use information from multiple visual sensory cues such as position, color, shape, etc. Previous work in fusion for tracking has tended to focus on fusion by numerically combining the scores assigned by each cue. We argue that for video scenes with many targets in a crowded situation, the splitting and merging of regions associated with targets, and the subsequent dramatic changes in cue values and reliabilities, renders this form of fusion less effective. In this paper we present experimental results showing that use of cue rank information in fusion produces a significantly better tracking result in crowded scenes. We also present a formalization of this fusion problem as a step in understanding why this effect occurs and how to build a tracking system that exploits it.

It is important to be able to guarantee the safety and effectiveness of robot behavior in applications where robots must operate alongside people or in hazardous situations. A modeling framework based on port automata and asynchronous communication is introduced in this paper. By looking at the internal transitions between port communications, an analysis approach is developed that removes the combinatoric issues of looking at an asynchronous combination of robot and environment. An example application of the approach to wheel slippage in a mobile robot is presented.

The paper introduces an approach to the problem of choosing when to zoom a moving camera so as to follow a designated video surveillance target. Rather than trying to maintain a simple viewing constraint (e.g., target >10% of image), the potential misrecognition of the target is also used to decide when to zoom. A discrete-event approach is used to develop two models of appearance change as well as a model that represents the viewing constraints for target surveillance. Disagreement between the appearance models is taken to indicate a potential loss of target. Supervisory discrete event control theory is used to construct automatically a controller that selects zoom actions to prevent loss of target. The implementation of this controller is overviewed and results presented.

We study a novel approach to the problem of fusion of sensory information in tracking multiple targets in CCTV surveillance video. The approach, called "rank and fuse" (RAF) is based on multiple feature ranking and merging as opposed to a more typical combination of all scores (similarity or probability) in a single ranking. This has the advantages of low computational complexity, easy scalability to multiple features, and low-latency. Experimental results are presented to illustrate two aspects of the RAF approach for a "difficult" example from CCTV surveillance: the advantage of rank versus score combination, and the use of the rank versus score curve to decide which features to fuse.

A robot system operating in an environment in which there is uncertainty and change needs to combine the ability to react with the ability to plan ahead. In a previous paper we proposed a solution to the problems of integrating planning and reaction: cast planning as adaptation of a reactive system, the planner-reactor approach. In this paper, we present our first experimental results from this approach. The results indicate that the planner-reactor approach is an attractive option for integrating planning and reaction in a robot system, allowing smooth, online updates to the behavior, of a reactive system with little time and behavior penalties accruing from the use of a planner.>

A robot system operating in an environment in which there is uncertainty and change needs to combine the ability to react with the ability to plan ahead. In a previous paper we proposed a solution to the problems of integrating planning and reaction: cast planning as adaptation of a reactive system. The planner, asynchronously tuning the reactor, decides on the appropriate parts of the reactor to be modified based on perceptions: information gathered in the reactor for the express purpose to inform the planner. In this paper, we show the benefits of using perception to adapt the reactor where it needs to be updated most and present our first experimental results from the planner-reactor architecture.>

The paper describes the software design approach and implementation of a stable grasp strategy to control a multifingered robot hand in a dynamic and uncertain environment. The overall design starts with a reactive system, called the Grasp Reactor, which measures the environment, and produces actions based on the environmental situations present. To improve the robustness of the Grasp Reactor, it is augmented with a deliberative component which executes concurrently with the Grasp Reactor. This component, called the Grasp Advisor, communicates global constraints to the Grasp Reactor to improve its decision making capability.

The problem of constructing a plan representation that can deal with the complexity of representing and analyzing robot behavior in uncertain and dynamic environments is addressed. A concurrent-process based representation is developed which represents both the plan (or controller) and the uncertain and dynamic environment in which the plan operates. A methodology is outlined for analyzing the behavior of this interacting system of plan and world. This methodology is illustrated with a mixed-batch example from the domain of robotic kitting. To balance the theoretical work, a description of the implemented robot kitting cell is presented.>

A robot system operating in an environment in which there is uncertainty and change needs to combine the ability to react with the ability to plan ahead. The authors had previously proposed a solution to the problems of integrating planning and reaction: cast planning as adaptation of a reactive system. They extend the theoretical treatment of the planner, including the effect of its iterated adaptation on the reactor, and describe the current implementation environment and a working example.>

Classical artificial intelligence planning is not sufficiently robust in uncertain and dynamic environments. Reactive approaches are robust in some environments-namely those for which they have been programmed. An approach that integrates a priori planning with reaction to increase robustness is presented. As motivation, a practical robot problem, the kitting robot, is presented. Solving this problem demands a system that can make timely and robust actions in an uncertain environment. A solution to the kitting robot problem in which planning is cast as adaptation of a reactive system to suit changes in the goals or environment is outlined. The reactive system (the reactor) is based on a formal model for representing flexible robot plans, the RS model. Thus, it was possible to formalize the mechanisms by which the planner improves the behavior of the reactor. This system was implemented to control a Puma-560 robot equipped with visual sensing.>

A methodology is presented for building integrated planning-reacting systems. The work is based on a formal process-based approach to building the reactive component; this makes it possible to formalize the concept of a planner improving a reactor. The planner design emphasizes how the planner can use the reactor to focus its reasoning, as well as how the reactor is guided by the planner to improve its behavior. The reactive component (the reactor) is based on a process-based model of robot computation, the RS model. This gives a powerful representation for actions with precise formal semantics. The duty of the planning component (the planner) is to adapt the reactor to suit a set of objectives and the possibilities afforded by the environment. Planner and reactor both operate continually, separately and in a complementary fashion.>

A process-based model of robot task plan representation is introduced, and some of its advantages are discussed. By process-based it is meant that the plan is represented as a set of concurrent communicating computational processes, rather than as a graph or a set of local propositions. A description is given of how the authors have developed process composition operators that allow them to represent explicitly and succinctly the relationship between plan components to express sensing and reaction, dynamic resource allocation, and error monitoring and recovery. The approach has formal benefits as well as representation benefits; using the techniques of the (extensive) process algebra literature, they introduce a way to analyze plans (manually or automatically) for liveness and efficiency properties and indicate how the method could help in plan generation.>

Robot applications, represented as plans, are used to outline a viewpoint that robustness needs to be emphasized in two areas: in the plan representation and in the underlying system software. Robot applications are inherently distributed, since the hardware usually comprises a set of independent actuators and sensors, with the robot programs acting as links between them. A special model of distributed computation, the RS (Robot Schemas) model, has been designed to handle the issues of robot plan representation, and an overview of the model is presented. An initial implementation of the model with minimal execution support demonstrated that the domain-dependent aspect of robustness on its own was not sufficient for robust behavior. Consequently, the OS has been augmented with real-time scheduling, and monitoring facilities.>

It is noted that almost all attempts to construct special-purpose robot programming languages have proceeded by taking a computer-programming language and adding some special primitives. Here, the authors have taken the approach of trying to define computation at its most primitive level in terms of the characteristics of the robot domain. They construct a special model of computation, called RS (robot schemas), with properties designed to facilitate sensory-based robot programming. This approach offers the potential to construct robot task representations which are easy to use and concise, and which execute in an efficient manner. The authors define the model formally using port automata. These definitions ensure consistency and well-definedness, and the facilitate plan verification and automatic plan generation.>

This paper suggests a solution to the specification problem inherent in writing programs for manipulators with many degrees of freedom. It introduces a mechanism called tagged potential fields, which can be used to economically specify a complex manipulator configuration. Some examples are demonstrated for preshaping a dextrous hand, and some of the possible future applications previewed.

This paper describes a high-level control mechanism for a dextrous hand. It overviews a distributed environment for robot control currently under development. A simple flexible set of grasps for a dextrous hand is then suggested and their implementation discussed. Finally the paper speculates about the role the grasp can play in obstacle avoidance and vice-versa.