We propose a novel object detection framework tailored for mixed reality (MR), combining a customizable synthetic dataset with a lightweight attention-enhanced detection model. Our dataset generation pipeline synthesizes planetary and telescope foregrounds with hybrid real-synthetic backgrounds, enabling robust learning across variable lighting and occlusion scenarios—challenges common in educational MR environments. At the core of our architecture is the Dual Focus Multiscale Attention (DFMA) module, which simultaneously refines spatial and channel-wise features at multiple scales. Integrated into a YOLO-based (You Only Look Once) backbone and FPN, DFMA significantly improves feature discrimination while preserving real-time efficiency. On MS COCO our model improves mean Average Precision (mAP) across Intersection over Union (IoU) thresholds from 0.5 to 0.95 ([email protected]:0.95) over state-of-the-art nano detectors from 39.3% to 41.3% (± 2%) at only +6% params and +3% GFLOPs, with a notable reduction in false positives on visually similar, low-textured objects. We further demonstrate real-time deployment in a Unity-based MR application, highlighting the system's effectiveness in immersive astronomy-focused educational scenarios. Our results underscore the potential of synthetic data and multiscale attention to bridge accuracy, speed, and realism in next generation MR systems.

We introduce a compact video-classification pipeline for real-time dynamic hand-gesture recognition in mixed-reality (MR) settings. The network marries a MobileNetV3 backbone with two purpose-built temporal components: (1) a Gated Discriminative Temporal Shift Module (G-DiTSM) that inserts first-order motion differences and learns channel-wise gates to fuse them adaptively, and (2) a lightweight Depth-Efficient Channel Attention (DepthECA) block that recalibrates spatial features on the fly. Operating on eight sparsely sampled frames per clip (Temporal Segment Network paradigm), the resulting model contains 2.65 M parameters and requires only 0.084 GFLOPs per inference. Evaluated on the RGB-only 20BN Jester benchmark (148k clips spanning 27 gesture classes) recorded from front-facing viewpoints. The system reaches 95.34% Top-1 and 99.80% Top-5 accuracy, surpassing recent 3D CNNs and transformer baselines while being an order of magnitude lighter. Ablations confirm that DepthECA and G-DiTSM provide complementary gains (+18.78% and +0.93% Top-1, respectively, over the MobileNetV3 baseline). Because all components are plug-and-play and introduce minimal overhead, the architecture is well suited to the tight latency and power budgets of standalone MR headsets, paving the way for natural grab, rotate, and command interactions using only on-board RGB cameras

Real-time object detection in indoor settings is a challenging area of computer vision, faced with unique obstacles such as variable lighting and complex backgrounds. This field holds significant potential to revolutionize applications like augmented and mixed realities by enabling more seamless interactions between digital content and the physical world. However, the scarcity of research specifically fitted to the intricacies of indoor environments has highlighted a clear gap in the literature. To address this, our study delves into the evaluation of existing datasets and computational models, leading to the creation of a refined dataset. This new dataset is derived from OpenImages v7[14], focusing exclusively on 32 indoor categories selected for their relevance to real-world applications. Alongside this, we present an adaptation of a CNN detection model, incorporating an attention mechanism to enhance the model’s ability to discern and prioritize critical features within cluttered indoor scenes. Our findings demonstrate that this approach is not just competitive to existing state-of-the-art models in accuracy and speed but also opens new avenues for research and application in the field of real-time indoor object detection.

In this paper, a custom Single Shot Multi-box Detector (SSD) [1] is proposed for object detection on difficult scenes. The fruit 360 dataset [2], with low-textured images of different fruits and vegetables, is used as a training and validation data set. The purpose of this research is to implement the detector on mobile devices for mixed and augmented reality experiences, so a lighter weight SSD [1] model was designed while retaining its performance. The custom model is 4 times faster than the original SSD [1] model and the tests showed that it is even more accurate on the designated data set. The model is implemented in Python using Tensorflow and will soon be available on GitHub for public use.