Real-time recognition and localization of keyboard keys combined with real-time hand tracking and detection of finger-press actions are critical functions of vision-based virtual keyboards. These are not trivial tasks because there are numerous types of keyboard layouts and typist behaviors. Furthermore, key detection is especially difficult in uncontrolled real-world scenarios, where users' hands occlude significant portions of the keyboard while typing. Also, the detection finger press-down actions is further complicated in-the-wild by frequently changing camera viewpoints without direct line of sight to the finger pressing a key. In this work, we address the challenge of complete keyboard segmentation and detection of hand-occluded keys by proposing a deep learning approach for realtime finger-press detection, visible-key detection, and occluded-key recovery. Our models were trained and evaluated on Kaggle's Keyboard Key Detection dataset [1] and further empirically tested on the MSU Typing Behavior Database [3]. Our best models achieved 88% finger press-down detection and a key detection performance of 0.91 IOU and 97.8% mAP@75, while running at 60 fps.

Vision Transformers (ViTs) have continuously achieved new milestones in object detection. However, the considerable computation and memory burden compromise their efficiency and generalization of deployment on resource-constraint devices. Besides, efficient transformer-based detectors designed by existing works can hardly achieve a realistic speedup, especially on multi-core processors (e.g., GPUs). The main issue is that the current literature solely concentrates on building algorithms with minimal computation, oblivious that the practical latency can also be affected by the memory access cost and the degree of parallelism. Therefore, we propose SpeedDETR, a novel speed-aware transformer for end-to-end object detectors, achieving high-speed inference on multiple devices. Specifically, we design a latency prediction model which can directly and accurately estimate the network latency by analyzing network properties, hardware memory access pattern, and degree of parallelism. Following the effective local-to-global visual modeling process and the guidance of the latency prediction model, we build our hardware-oriented architecture design and develop a new family of SpeedDETR. Experiments on the MS COCO dataset show SpeedDETR outperforms current DETR-based methods on Tesla V100. Even acceptable speed inference can be achieved on edge GPUs.

In this paper, we provide: (a) An efficient method for simplifying MMPP source models, which can be used to solve multi-media queueing problems with complicated aggregate input traffic. The effectiveness of the method is verified through numerical examples. (b) An approximation algorithm for analyzing multi-link queueing networks with MMPP traffic sources and arbitrary configurations. In this algorithm, a queueing network is analyzed by decomposing it into individual queues, each with an arrival rate appropriately reduced by the packet dropping occurred in other queues. Each queue is then solved individually by using the simplified source models of (a) while the interdependencies with the other queues are accounted for through this reduced-load approximation. Any desirable end-to-end performance measure characterizing the quality of service (QOS) of the multi-media traffic can be calculated in this manner. The results obtained by the algorithm have been compared with simulation results, and the observed relative error is satisfactory.