Deploying object detectors in embedded devices such as unmanned aerial vehicles (UAVs) comes with many challenges. This is due to both the UAV itself having low embedded resources in terms of computation and memory, and also due to the nature of the captured visual data with the variations in objects' scale, orientation, density, viewpoint, distribution, shape, context and others. It is crucial for the object detector to be robust with high accuracy, real-time with fast inference and light-weight to be applicable. Inspired by YOLO architecture, we propose a novel single-stage detection architecture. Our contributions are, first, feature fusion spatial pyramid pooling (FFSPP) block that applies attention-based feature fusion across both time and space utilizing the information of subsequent frames and scales in an efficient manner. Secondly, we introduce a multi-dilated attention-based cross-stage partial connection (MDACSP) block that helps in increasing the receptive field and producing per-channel modulation weights after aggregating the feature maps across their spatial domain. Third, scaled feature fusion head (SFFH) fuses both the FFSPP block features and the connected MDACSP block features specific for this head. For a more robust result across different scenarios, we perform cross-ensembling with three of the top UAV/traffic surveillance datasets: UAVDT, UA-DETRAC and VisDrone. Our ablation study shows how every contribution improves over the baseline. Our approach yielded the state-of-the-art results in all the aforementioned datasets achieving 89.3% mAP, 93.5% mAP, and 42.9% mAP respectively. Testing the model performance on NVIDIA Jetson Xavier NX board shows a desirable balance between the inference time and the memory cost. We also show qualitatively the model robustness and efficiency across the diverse complex scenarios of these datasets. We hope this work facilitates the advancement of the UAV-based perception in such crucial industrial applications.

Recently, the feature-level generation has demonstrated the effectiveness of pseudo-stereo synthesis in Monocular 3D Detection (M3D). In this paper, we aim to further bridge the gap between the stereo and the monocular 3D object detectors in autonomous driving through direct image-level pseudo-stereo generation. We propose a novel Cycled Generative Pseudo-Stereo (CGPS) architecture to generate the right-view image from the left-view for constructing a pseudo-stereo pair to stereo 3D object detectors while maintaining the natural of M3D with the left-view image as the only input. Moreover, we use a triplet consistency loss to focus on the detected objects in the pseudo-stereo generation. Besides, we demonstrate that the proposed CGPS is an ad-hoc module to adapt top stereo 3D object detectors into monocular 3D object detectors. The proposed framework with CGPS achieves 74.80%, 55.28%, and 46.70% 3DAP for easy, moderate, and hard difficulty levels in monocular 3D detection on the KITTI benchmark with comparable performance to the stereo 3D object detectors but using a monocular image as the only input. Till the submission, the proposed M3D framework ranks 1stwith dramatic improvements against the existing monocular 3D detectors on the KITTI benchmark.

Developing realistic and interactive virtual environments is a major hurdle in the progress of Metaverse. At present, majority of Metaverse applications necessitate the manual construction of 3D models which is both time-consuming and costly. Additionally, it is challenging to design environments that can promptly react to users' actions. To address this challenge, this paper proposes a novel approach to generate virtual worlds using digital twin (DT) technology and AI through a Text-to-Metaverse pipeline. This pipeline converts natural language input into a scene JSON, which is used to generate a 3D virtual world using two engines: Generative Script Engine (GSE) and Generative Metaverse Engine (GME). GME generates a design script from the JSON file, and then uses it to generate 3D objects in an environment. It aims to use multimodal AI and DT technology to produce realistic and highly detailed virtual environments. The proposed pipeline has potential applications including education, training, architecture, healthcare and entertainment, and could change the way designers and developers create virtual worlds. While this short paper covers an abstract as per the Doctorial Symposium's guidelines, it contributes to the research on generative models for multimodal data and provides a new direction for creating immersive virtual experiences.