Summary Distributed virtual environments provide a shared sense of time and space to geographically distributed users. They depend on a limited ability to ensure system nodes at different locations maintain consistent state data. This paper presents system models and algorithms designed to find optimal update scheduling parameters that minimize the effects of inconsistent simulation state data. The first model is concerned with ensuring distributed virtual environments that present a fair experience to all users while simultaneously providing adequate levels of system performance. In the second model, we introduce the concept of plausibility limits and address their use in ensuring all participants in an interaction to see the same result.

Loss of user engagement with automation in increasingly complex environments, such as driving a car, can have increasingly unexpected and consequential impacts. This paper presents a three-step process for designing cooperative automation into existing systems to maximize system performance with the constraint of maintaining user engagement. The first step identifies beneficial areas for automation through user testing, focusing on areas where human capabilities constrain performance. Next, an automation is created to perform a portion of the task leading to the constraint. User tests at this point compare different automation types to choose an automation that improves on human-only performance. Lastly, the automation trigger is adjusted to allow the human to maintain engagement with the system at a level desired to avoid the risks associated with loss of user engagement. This process is demonstrated through three human-in-the-loop experiments which illustrate the trade-space between maintaining human engagement and improving system performance.

Sequence pattern mining (SPM) seeks to find multiple items that commonly occur together in a specific order. One common assumption is that the relevant differences between items are captured through creating distinct items. In some domains, this leads to an exponential increase in the number of items. This paper presents a new SPM, Sequence Mining of Temporal Clusters (SMTC), that allows item differentiation through attribute variables for domains with large numbers of items. It also provides a new technique for addressing interleaving, a phenomena that occurs when two sequences occur simultaneously resulting in their items alternating. By first clustering items temporally and only focusing on sequences after the temporal clusters are established, it sidesteps the traditional interleaving issues. SMTC is evaluated on a digital forensics dataset, a domain with a large number of items and frequent interleaving. Its results are compared with Discontinuous Varied Order Sequence Mining (DVSM) with variables added (DVSM-V). By adding variables, both algorithms reduce the data by 96 percent, and identify 100 percent of the events while keeping the false positive rate below 0.03 percent. SMTC mines the data in 20 percent of the time it takes DVSM-V and provides a lower false positive rate even at higher similarity thresholds.

The Neural Evolution of Augmenting Topologies (NEAT) algorithm is a popular technique to evolve neural networks. While NEAT has been applied to many problems, such as control systems and game playing, the underlying speciation rules governing NEAT have not been explored. This paper proposes a new speciation technique, called Natural Evolution NEAT (NENEAT), that replaces NEAT's speciation strategy with a new stategy inspired by cladistics. The cladistic formation of species forms a tree of common descent. NENEAT can find smaller network solutions in fewer generation and population member evaluations than NEAT using the same number of children.

Pursuit and evasion game related tasks are one application area envisioned for future multi- vehicle Unmanned Aircraft systems (UAs). Pursuit and evasion games relate to tasks such as search and rescue operations, surveillance, and search and capture missions. They include two sub-tasks of finding evaders and then capturing them. Because there are two sub-tasks, there are different roles that each pursuer must take on during the game. This paper examines the amount of communication required to determine what role each UAs pursuer should take to achieve a quick location and capture of the evader. Results demonstrate that more robust internal control strategies offset role communication when the communications become limited.

Recently, a framework has been developed to automate the training of prediction algorithms but has only been tested on one system. In order to generalize the approach a few key functions must be performed. One of these functions is load generation. Unfortunately, a valid load generator has not been developed for a Microsoft Windows active directory environment. In this paper we introduce and detail a tool that we have developed to make the implementation of this new framework possible in a Microsoft domain, we present data generated by our tool to demonstrate its efficacy, and finish with several extensions and applications for our tool.

Automated feature matching of nuclear detonations (NUDETs) enables three‐dimensional point cloud reconstruction, and establishment a volume‐based model to reduce uncertainty in estimating the yield of NUDETs solely from video. Establishing a volume‐based model requires feature correspondences between wide viewpoints of 58°–110° that are larger than scale‐invariant feature transform‐based techniques can reliably match. The presented technique detects relative bright features in the NUDET known as ‘hotspots,’ and matches them across wide viewpoints using a spherical based object model. Results show that hotspots can be detected with a 71.95% hit rate and 86.03% precision. Hotspots are matched to films from different viewpoints with 76.6% correctness and a standard deviation of 16.4%. Hotspot descriptors are also matched in time sequence with 99.6% correctness and a standard deviation of 1.07%. The results demonstrate that a spherical model can serve as a viable descriptor model for matching across wide viewpoints when the object is known to be spherical. It also demonstrates an automated feature detection and matching combination that enables features to be matched from unsynchronised video across wide viewpoints of 58°–110° on spherical objects where state‐of‐the‐art techniques are insufficient.

Large-scale systems are part of a growing trend in distributed computing, and coordinating control of them is an increasing challenge. This paper presents a cooperative agent system that scales to one million or more nodes in which agents form coalit

Hyperspectral-based change detection is often inadvertently affected by image artifacts, reducing the accuracy of the change detector. We present a Hyperspectral Change Detection (HSCD) process to distinguish parallax-induced change from legitimate change. Image parallax decreases the accuracy of change detection results. The approach introduced in this paper utilizes a combination of a spectral change detector and stereo geometry to reduce parallax-induced false alarms. Image parallax is determined by considering the error in the epipolar constraint, meaning the corresponding points between two images must lie on epipolar lines. Experimental analysis shows a false alarm reduction by nearly one order of magnitude on synthetic hyperspectral imagery and nearly two orders of magnitude on real hyperspectral imagery.

Ant colony optimization (ACO) algorithms can generate quality solutions to combinatorial optimization problems. However, like many stochastic algorithms, the quality of solutions worsen as problem sizes grow. In an effort to increase performance, we added the variable step size off-policy hill-climbing algorithm called PDWoLF (Policy Dynamics Win or Learn Fast) to several ant colony algorithms: Ant System, Ant Colony System, Elitist-Ant System, Rank-based Ant System, and Max-Min Ant System. Easily integrated into each ACO algorithm, the PDWoLF component maintains a set of policies separate from the ant colony's pheromone. Similar to pheromone but with different update rules, the PDWoLF policies provide a second estimation of solution quality and guide the construction of solutions. Experiments on large traveling salesman problems (TSPs) show that incorporating PDWoLF with the aforementioned ACO algorithms that do not make use of local optimizations produces shorter tours than the ACO algorithms alone.

When an autonomous vehicle operates in an unknown environment, it must remember the locations of environmental objects and use those object to maintain an accurate location of itself. This vehicle is faced with Simultaneous Localization and Mapping (SLAM), a circularly defined robotics problem of map building with no prior knowledge. The SLAM problem is a difficult but critical component of autonomous vehicle exploration with applications to search and rescue missions. This paper presents the first SLAM solution combining stereo cameras, inertial measurements, and vehicle odometry into a Multiple Integrated Navigation Sensor (MINS) path. The FastSLAM algorithm, modified to make use of the MINS path, observes and maps the environment with a LIDAR unit. The MINS FastSLAM algorithm closes a 140 meter loop with a path error that remains within 1 meter of surveyed truth. This path reduces the error 79% from an odometry FastSLAM output and uses 30% of the particles.

When a camera is affixed on a dynamic mobile robot, image stabilization is the first step towards more complex analysis on the video feed. This paper presents a novel electronic image stabilization (EIS) algorithm for highly dynamic mobile robotic platforms. The algorithm combines optical flow motion parameter estimation with angular rate data provided by a strapdown inertial measurement unit (IMU). A discrete Kalman filter in feedforward configuration is used for optimal fusion of the two data sources. Performance evaluations are conducted using a simulated video truth model (capturing the effects of image translation, rotation, blurring, and moving objects), and live test data. Live data was collected from a camera and IMU affixed to the DAGSI Whegs mobile robotic platform as it navigated through a hallway. Template matching, feature detection, optical flow, and inertial measurement techniques are compared and analyzed to determine the most suitable algorithm for this specific type of image stabilization. Pyramidal Lucas-Kanade optical flow using Shi-Tomasi good features in combination with inertial measurement is the EIS algorithm found to be superior. In the presence of moving objects, fusion of inertial measurement reduces optical flow root-mean-squared (RMS) error in motion parameter estimates by 40%.

Computer network cyber-security is a very serious concern in many commercial, industrial, and military environments. This paper proposes a new computer network security approach defined by self organized agent swarms (SOMAS) which provides a novel computer network security management framework based upon desired overall system behaviors. The SOMAS structure evolves based upon the partially observable Markov decision process (POMDP) formal model and the more complex interactive-POMDP and decentralized-POMDP models. Example swarm specific and network based behaviors are formalized and simulated. This paper illustrates through various statistical testing techniques, the significance of this proposed SOMAS architecture.

Coalition formation algorithms are generally not applicable to real-world robotic collectives since they lack mechanisms to handle uncertainty. Those mechanisms that do address uncertainty either deflect it by soliciting information from others or apply reinforcement learning to select an agent type from within a set. This paper presents a coalition formation mechanism that directly addresses uncertainty while allowing the agent types to fall outside of a known set. The agent types are captured through a novel agent modeling technique that handles uncertainty through a belief-based evaluation mechanism. This technique allows for uncertainty in environmental data, agent type, coalition value, and agent cost. An investigation of both the effects of adding agents on processing time and of model quality on the convergence rate of initial agent models (and thereby coalition quality) is provided. This approach handles uncertainty on a larger scale than previous work and provides a mechanism readily applied to a dynamic collective of real-world robots.

This paper merges hierarchical reinforcement learning (HRL) with ant colony optimization (ACO) to produce a HRL ACO algorithm capable of generating solutions for large domains. This paper describes two specific implementations of the new algorithm: the first a modification to Dietterich's MAXQ-Q HRL algorithm, the second a hierarchical ant colony system algorithm. These implementations generate faster results, with little to no significant change in the quality of solutions for the tested problem domains. The application of ACO to the MAXQ-Q algorithm replaces the reinforcement learning, Q-learning, with the modified ant colony optimization method, Ant-Q. This algorithm, MAXQ-AntQ, converges to solutions not significantly different from MAXQ-Q in 88% of the time. This paper then transfers HRL techniques to the ACO domain and traveling salesman problem (TSP). To apply HRL to ACO, a hierarchy must be created for the TSP. A data clustering algorithm creates these subtasks, with an ACO algorithm to solve the individual and complete problems. This paper tests two clustering algorithms, k-means and G-means. The results demonstrate the algorithm with data clustering produces solutions 20 times faster with 5-10% decrease in solution quality due to the effects of clustering.

Current autonomous and semi-autonomous robotic platforms are limited to functioning in highly structured environments such as buildings and roads. Autonomous robots that could explore and navigate rugged terrain and highly unstructured environments such as collapsed buildings would have large dividends in civilian and military applications. In this work, we present the next generation of Whegstrade robots, DAGSI Whegstrade, which has been completed and extensively field tested. Several innovations have made the robot more rugged and well suited to autonomous operation. Specifically, an actively controlled, passively compliant body joint has been tested in three different modes of operation to judge the usefulness of the mechanism. A two-dimensional dynamic simulation of the robot has also been constructed, and has been used to study the effects of weight distribution on obstacle climbing and to investigate future autonomous climbing strategies. Moving the center of mass forward allowed the robot to climb taller obstacles. DAGSI Whegstrade can climb rectangular obstacles as tall as 2.19 times the length of a leg.

The development of coherent and dynamic behaviors for mobile robots is an exceedingly complex endeavor ruled by task objectives, environmental dynamics and the interactions within the behavior structure. This paper discusses the use of genetic programming techniques and the unified behavior framework to develop effective control hierarchies using interchangeable behaviors and arbitration components. Given the number of possible variations provided by the framework, evolutionary programming is used to evolve the overall behavior design. Competitive evolution of the behavior population incrementally develops feasible solutions for the domain through competitive ranking. By developing and implementing many simple behaviors independently and then evolving a complex behavior structure suited to the domain, this approach allows for the reuse of elemental behaviors and eases the complexity of development for a given domain. Additionally, this approach has the ability to locate a behavior structure which a developer may not have previously considered, and whose ability exceeds expectations. The evolution of the behavior structure is demonstrated using agents in the Robocode environment, with the evolved structures performing up to 122 percent better than one crafted by an expert.

When a digital forensics investigator suspects that steganography has been used to hide data in an image, he must not only determine that the image contains embedded information but also identify the method used for embedding. The determination of the embedding methodor stego fingerprint — is critical to extracting the hidden information. This paper focuses on identifying stego fingerprints in JPEG images. The steganography tools targeted are F5, JSteg, Model-Based Embedding, OutGuess and StegHide. Each of these tools embeds data in a dramatically different way and, therefore, presents a different challenge to extracting the hidden information. The embedding methods are distinguished using features developed from sets of stego images that are used to train a multi-class support vector machine (SVM) classifier. For new images, the image features are calculated and evaluated based on their associated label to the most similar class, i.e., clean or embedding method feature space. The SVM results demonstrate that, in the worst case, embedding methods can be distinguished with 87% reliability.

Steganography (stego) is used primarily when the very existence of a communication signal is to be kept covert. Detecting the presence of stego is a very difficult problem which is made even more difficult when the embedding technique is not known. This article presents an investigation of the process and necessary considerations inherent in the development of a new method applied for the detection of hidden data within digital images. We demonstrate the effectiveness of learning vector quantization (LVQ) as a clustering technique which assists in discerning clean or non-stego images from anomalous or stego images. This comparison is conducted using 7 featuresover a small set of 200 observations with varying levels of embedded information from 1% to 10% in increments of 1%. The results demonstrate that LVQ not only more accurately identify when an image contains LSB hidden information when compared to k-means or using just the raw feature sets, but also provides a simple method for determining the percentage of embedding given low information embedding percentages.

Reinforcement learning is one of the more attractive machine learning technologies, due to its unsupervised learning structure and ability to continually learn even as the operating environment changes. Additionally, by applying reinforcement learning to multiple cooperative software agents (a multi-agent system) not only allows each individual agent to learn from its own experience, but also opens up the opportunity for the individual agents to learn from the other agents in the system, thus accelerating the rate of learning. This research presents the novel use of fuzzy state aggregation, as the means of function approximation, combined with the fastest policy hill climbing methods of Win or Lose Fast (WoLF) and policy-dynamics based WoLF (PD-WoLF). The combination of fast policy hill climbing and fuzzy state aggregation function approximation is tested in two stochastic environments: Tileworld and the simulated robot soccer domain, RoboCup. The Tileworld results demonstrate that a single agent using the combination of FSA and PHC learns quicker and performs better than combined fuzzy state aggregation and Q-learning reinforcement learning alone. Results from the multi-agent RoboCup domain again illustrate that the policy hill climbing algorithms perform better than Q-learning alone in a multi-agent environment. The learning is further enhanced by allowing the agents to share their experience through a weighted strategy sharing.

This paper introduces hyper-ellipsoids as an improvement to hyper-spheres as intrusion detectors in a negative selection problem within an artificial immune system. Since hyper-spheres are a specialization of hyper-ellipsoids, hyper-ellipsoids retain the benefits of hyper-spheres. However, hyper-ellipsoids are much more flexible, mostly in that they can be stretched and reoriented. The viability of using hyper-ellipsoids is established using several pedagogical problems. We conjecture that fewer hyper-ellipsoids than hyper-spheres are needed to achieve similar coverage of nonself space in a negative selection problem. Experimentation validates this conjecture. In pedagogical benchmark problems, the number of hyper-ellipsoids to achieve good results is significantly (~50%) smaller than the associated number of hyper-spheres.

Abstract—Robot mapping even today is one of the most challenging problems in robot programming. Most successful methods use some form of occupancy grid to represent a mapped region. This approach becomes problematic if the robot is mapping a large environment, the map quickly becomes too large for processing and storage. Rather than storing the map as an occupancy grid, our robot (equipped with sonars) sees the world as a series of connected spaces. These spaces are initially mapped as an occupancy grid in a room by room fashion. As the robot leaves a space, denoted by passing through a doorway, the grids are converted to a polygonal representation. This polygonal representation is stored as rooms and hallways as a set of Absolute Space Representations (ASRs) representing the space connections. Using this representation makes navigation and localization easier for the robot to process. I.