Abstract This work introduces an image quality assessment (IQA)-driven optimization framework for synthetic data generation (SDG) to improve the simulation-to-real-world (sim2real) transfer performance of synthetic vision systems. By using IQA metrics—particularly structural similarity (SSIM) based—as fitness functions for genetic algorithms (GAs), we automate the tuning of synthetic environments to enhance the generalizability of synthetic-trained object detectors in new or unobserved real-world scenarios. Digital-twin environments were constructed using both 3D Gaussian splatting from real images and by procedural generation methods. Two GAs were developed: a pre-process GA that modifies the environment before image generation for background focused similarity, and a post-process GA that adjusts noise and visual properties in the synthetic images, focusing on foreground-object similarity. The Synthetic Image Quality Analysis Calculator (SIQAC) software was extended to define regions of interest and evaluate transferability. Experimental results show sim2real accuracy improvements of up to 36% with the pre-process GA, 19% with the post-process GA, and 33% when combined, reaching near real vision system performance levels. These findings demonstrate the benefits of IQA-informed optimization in reducing the sim2real gap for synthetic vision systems.

The Model AI Assignments session seeks to gather and disseminate the best assignment designs of the Artificial Intelligence (AI) Education community. Recognizing that assignments form the core of student learning experience, we here present abstracts of six AI assignments from the 2021 session that are easily adoptable, playfully engaging, and flexible for a variety of instructor needs. Assignment specifications and supporting resources may be found at http://modelai.gettysburg.edu.

Code authorship identification can assist in identifying creators of malware, identifying plagiarism, and giving insights in copyright infringement cases. Taking inspiration from facial recognition work, we apply recent advances in metric learning to the problem of authorship identification and verification. The metric learning approach makes it possible to measure similarity in the learned embedding space. Access to a discriminative similarity measure allows for the estimation of probability distributions that facilitate open-set classification and verification. We extend our analysis to verification based on sets of files, a previously unexplored problem domain in large-scale author identification. On closed-set tasks we achieve competitive accuracies, but do not improve on the state of the art.

The deployment of autonomous and semi-autonomous robots is likely to increase dramatically over the next decade. Recent autonomous vehicle prototypes illustrate both the rapid progress of the underlying technology and the commercial possibilities of robotics. The next few years are likely to see increased interest in robotics among both students and employers.

To make progess in understanding human visuomotor behavior, we will need to understand its basic components at an abstract level. One way to achieve such an understanding would be to create a model of a human that has a sufficient amount of complexity so as to be capable of generating such behaviors. Recent technological advances have been made that allow progress to be made in this direction. Graphics models that simulate extensive human capabilities can be used as platforms from which to develop synthetic models of visuomotor behavior. Currently, such models can capture only a small portion of a full behavioral repertoire, but for the behaviors that they do model, they can describe complete visuomotor subsystems at a useful level of detail. The value in doing so is that the body's elaborate visuomotor structures greatly simplify the specification of the abstract behaviors that guide them. The net result is that, essentially, one is faced with proposing an embodied “operating system” model for picking the right set of abstract behaviors at each instant. This paper outlines one such model. A centerpiece of the model uses vision to aid the behavior that has the most to gain from taking environmental measurements. Preliminary tests of the model against human performance in realistic VR environments show that main features of the model show up in human behavior.

Much of the allocation of human resources to tasks is studied under the rubric of "attention". However this is a very low-dimensional characterization of a system that has many degrees of freedom. To make progess in understanding human brain resource allocations, we will need to understand its basic functions at an abstract level. One way of accomplishing such an integration is to create a model of a human that has a useful amount of complexity. Essentially, one is faced with proposing an embodied "operating system" model that can be tested against human performance. Recently, technological advances have been made that allow progress in this direction. Graphic models that simulate extensive human capabilities can be used as platforms to develop synthetic models of visuo-motor behavior. Currently, such models can capture only a small portion of a full behavioral repertoire, but for the behaviors that they do model, they can describe complete visuo-motor subsystems at a level of detail that can be tested against human performance in realistic environments. This paper outlines one such model and shows both that it can produce interesting new hypotheses as to the role of vision and also that it can greatly enhance our understanding of a more multifacted characterization attention in visuo-motor tasks.

University of Rochester Rochester, NY 14627 [email protected] Recent eye tracking studies in natural tasks suggest that there is a tight link between eye movements and goal directed motor actions. However, most existing models of human eye movements provide a bottom up ac- count that relates visual attention to attributes of the visual scene. The purpose of this paper is to introduce a new model of human eye move- ments that directly ties eye movements to the ongoing demands of be- havior. The basic idea is that eye movements serve to reduce uncertainty about environmental variables that are task relevant. A value is assigned to an eye movement by estimating the expected cost of the uncertainty that will result if the movement is not made. If there are several candidate eye movements, the one with the highest expected value is chosen. The model is illustrated using a humanoid graphic figure that navigates on a sidewalk in a virtual urban environment. Simulations show our protocol is superior to a simple round robin scheduling mechanism.