Discrete data-based learning approaches have facilitated the wide applications of AI models, especially the notably favored foundation models. However, simply scaling the diversity and quantity of training data is still inadequate to achieve human-like thinking and action competency. A shift of learning paradigm from spatially–temporally discrete, weakly correlated, and noninteractive samples to spatially–temporally continuous, strongly correlated, and interactive scenarios is expected to go beyond the element-level understanding and promote the relation, trend, as well as situation awareness abilities of AI models. This article systematically structures the methodology of scenarios engineering (SE) and proposes a three-layer SE roadmap consisting of the scenarios development layer, scenarios organization layer, and scenarios cognition layer. This roadmap is designed to foster the flexible and efficient construction, organization, and utilization of scenarios. Building on this foundation and parallel intelligence, we introduce the framework of scenarios intelligence (SI) that leverages scenarios as next-generation data resources and microworld models to cultivate embodied AI agents, facilitating the development of descriptive, predictive, and prescriptive intelligence in tasks like perception, decision-making, and action. Experiments are conducted with unmanned aerial vehicles (UAVs) to illustrate the effectiveness of the proposed method in environmental understanding, risk assessment, and active perception.

Reidentification (Re-ID) is a crucial computer vision application with a variety of potential uses in many maritime scenarios, including search, rescue, and surveillance. However, the development of advanced boat reidentification (Boat Re-ID) algorithms necessitates the availability of large-scale Re-ID datasets for model training and evaluation. Inspired by scenarios engineering, this study proposes a new framework for automatically generating a realistic synthetic dataset for boat Re-ID investigation. The synthetic dataset contains 107 boat models and various visual conditions in 36 real backgrounds. The use of synthetic datasets enables the learning-based Re-ID algorithm’s performance to be quantitatively verificated under varying imaging conditions. Nonetheless, our experiments prove that synthetic datasets are inadequate to handle real-world challenges. Therefore, we present a domain adaptation approach that integrates both real and synthetic data to create trustworthy models. This approach employs a multistep training strategy, gradient reversal layer and novel loss functions to preserve the features from two distribution dataset domains. The results of the experiments demonstrate that 1) synthetic datasets can be employed to train boat Re-ID algorithms and quantitatively test the performance of these algorithms under diverse imaging conditions and 2) our approach utilizes the attributes of the two data domains (real and synthetic) to achieve exceptional performance in real-world applications.

Foreground detection (FD) plays an important role in the domain of video surveillance for highway. The design of advanced FD algorithms requires large-scale and diverse video dataset. However, collecting and labeling real dataset is still time-consuming, labor-intensive, and highly subjective. To address this issue, we first use computer graphics (CG) to clone real highway scenarios (HS) and generate synthetic multi-challenge video datasets, called “Synthetic-HS (CG)”, automatically labeled with accurate pixel-level ground truth. The Synthetic-HS (CG) dataset contains eight imaging condition sequences for computer vision research. Then, we design an image translation (IT) model that translates source domain (Synthetic-HS (CG)) to target domain (real). This model uses skip connections and attention module to generate realistic synthetic images “Synthetic-HS (IT)”. We use publicly available Synthetic-HS in combination with the corresponding real video sequence to conduct experiments. The experiment results suggest that: 1) The Synthetic-HS (CG) dataset enables us to provide precise quantitative evaluation of the drawbacks of foreground detection methods 2) The realistic Synthetic-HS (IT) images can be used to promote the visual perception in highway video surveillance.

Foundation models are used to train a broad system of general data to build adaptations to new bottlenecks. Typically, they contain hundreds of billions of hyperparameters that have been trained with hundreds of gigabytes of data. However, this type of black-box vulnerability places foundation models at risk of data poisoning attacks that are designed to pass on misinformation or purposely introduce machine bias. Moreover, ordinary researchers have not been able to completely participate due to the rise in deployment standards. This study introduces the theoretical framework of scenarios engineering (SE) for building accessible and reliable foundation models in metaverse, namely, “SE-enabled foundation models in metaverse.” Particularly, the research framework comprises a six-layer architecture (infrastructure layer, operation layer, knowledge layer, intelligence layer, management layer, and interaction layer), which can provide controllability, trustworthiness, and interactivity for the foundation models in metaverse. This creates closed-loop, virtual–real, and human–machine environments that provides the best indices and goals for the foundation models, which allows us to fully validate and calibrate the corresponding models. Then, examples of use cases from the automotive industry are listed to provide transparency on the possible use and benefits of our approach. Finally, the open research topics of related frameworks are discussed.

Virtual simulated scenes are becoming a critical part of autonomous driving. In the context of knowledge automation and machine learning, simulated images are widely used for visual environmental perception. However, even the most inspirational applications have not fully exploited the potential of simulated images in solving real-world problems. In this article, we propose a novel framework “ParallelEye Pipeline,” which uses image-to-image translation and simulated images to automatically generate realistic synthetic images with multiple ground-truth annotations. Specifically, this method has three steps: first, we use Unity3D software to simulate driving scenarios and generate simulated image pairs (including raw images and six ground-truth labels) from the simulated scenes; second, advanced image-to-image translation algorithms can generate realistic and high-resolution synthetic images from simulated image pairs; third, we exploit publicly datasets, simulated images, and synthetic images to conduct experiments for visual perception. The experimental results suggest: 1) synthetic images and simulated images can improve the performance of detectors in real autonomous driving scenarios and 2) image-to-image translation algorithms can be affected by occlusion condition.

Abstract There are many methods proposed for generating polycube polyhedrons, but it lacks the study about the possibility of generating polycube polyhedrons. In this paper, we prove a theorem for characterizing the necessary condition for the skeleton graph of a polycube polyhedron, by which Steinitz's theorem for convex polyhedra and Eppstein's theorem for simple orthogonal polyhedra are generalized to polycube polyhedra of any genus and with non‐simply connected faces. Based on our theorem, we present a faster linear algorithm to determine the dimensions of the polycube shape space for a valid graph, for all its possible polycube polyhedrons. We also propose a quadratic optimization method to generate embedding polycube polyhedrons with interactive assistance. Finally, we provide a graph‐based framework for polycube mesh generation, quadrangulation, and all‐hex meshing to demonstrate the utility and applicability of our approach.

Dataset plays an essential role in the training and testing of traffic vision algorithms. However, the collection and annotation of images from the real world is time-consuming, labor-intensive, and error-prone. Therefore, more and more researchers have begun to explore the virtual dataset, to overcome the disadvantages of real datasets. In this paper, we propose a systematic method to construct large-scale artificial scenes and collect a new virtual dataset (named “ParallelEye”) for the traffic vision research. The Unity3D rendering software is used to simulate environmental changes in the artificial scenes and generate ground-truth labels automatically, including semantic/instance segmentation, object bounding boxes, and so on. In addition, we utilize ParallelEye in combination with real datasets to conduct experiments. The experimental results show the inclusion of virtual data helps to enhance the per-class accuracy in object detection and semantic segmentation. Meanwhile, it is also illustrated that the virtual data with controllable imaging conditions can be used to design evaluation experiments flexibly.

Offline training and testing are playing an essential role in design and evaluation of intelligent vehicle vision algorithms. Nevertheless, long-term inconvenience concerning traditional image datasets is that manually collecting and annotating datasets from real scenes lack testing tasks and diverse environmental conditions. For that virtual datasets can make up for these regrets. In this paper, we propose to construct artificial scenes for evaluating the visual intelligence of intelligent vehicles and generate a new virtual dataset called “ParallelEye-CS”. First of all, the actual track map data is used to build 3D scene model of Chinese Flagship Intelligent Vehicle Proving Center Area, Changshu. Then, the computer graphics and virtual reality technologies are utilized to simulate the virtual testing tasks according to the Chinese Intelligent Vehicles Future Challenge (IVFC) tasks. Furthermore, the Unity3D platform is used to generate accurate ground-truth labels and change environmental conditions. As a result, we present a viable implementation method for constructing artificial scenes for traffic vision research. The experimental results show that our method is able to generate photorealistic virtual datasets with diverse testing tasks.

Polycube construction and deformation are essential problems in computer graphics. In this paper, we present a robust, simple, efficient, and automatic algorithm to deform the meshes of arbitrary shapes into polycube form. We derive a clear relationship between a mesh and its corresponding polycube shape. Our algorithm is edge-preserving, and works on surface meshes with or without boundaries. Our algorithm outperforms previous ones with respect to speed, robustness, and efficiency. Our method is simple to implement. To demonstrate the robustness and effectivity of our method, we have applied it to hundreds of models of varying complexity and topology. We demonstrate that our method compares favorably to other state-of-the-art polycube deformation methods.