In this paper, we propose a lightweight model for hand gesture recognition using an RGB camera. The proposed model enables recognition of first-person hand gestures using a single camera and achieves near-real-time computational performance on both high-end and low-end computing devices. The proposed framework utilizes multi-task multi-modal learning and self-distillation to deal with the challenges in hand gesture recognition. We integrate additional modalities (depth) and a future prediction mechanism to enhance the model’s ability to learn spatio-temporal information. Furthermore, we employ self-distillation to compress the model, achieving a balance between accuracy and computational efficiency. We compared the proposed hand gesture recognition model with the state-of-the-art method, and our model outperforms the SOTA by 0.88% and 3.52% on the EgoGesture and NVGesture datasets, respectively. In terms of computational efficiency, our model takes only 161ms in average to recognize a gesture on a device with low-end GPUs (NVIDIA Jetson TX2), which is acceptable for interaction in XR applications.

Deep reinforcement learning has shown its success in game playing. However, 2.5D fighting games would be a challenging task to handle due to ambiguity in visual appearances like height or depth of the characters. Moreover, actions in such games typically involve particular sequential action orders, which also makes the network design very difficult. Based on the network of Asynchronous Advantage Actor-Critic (A3C), we create an OpenAI-gym-like gaming environment with the game of Little Fighter 2 (LF2), and present a novel A3C+ network for learning RL agents. The introduced model includes a Recurrent Info network, which utilizes game-related info features with recurrent layers to observe combo skills for fighting. In the experiments, we consider LF2 in different settings, which successfully demonstrates the use of our proposed model for learning 2.5D fighting games.

Smart Lenses refer to detachable, orientable and zoomable lenses that stream live videos over wireless networks to heterogeneous computing devices, including tablets and smartphones. Various novel applications are made possible by smart lenses, including mobile photography, smart surveillance cameras, and Unmanned Aerial Vehicle (UAV) cameras. However, to our best knowledge, existing smart lenses are closed and proprietary, and thus we initiate an open-source project called Smart Beholder for end-to-end solutions of smart lenses. The code and documents of Smart Beholder can be found at our website http://www.smartbeholder.org. Our Smart Beholder platform are useful to researchers for fast prototyping, developers for rapid development, and amateurs for hobbies. We have implemented Smart Beholder server (camera) using a popular embedded Linux platform, called Raspberry Pi. We have also realized Smart Beholder client (controller) on various OS's, including Android. Our experimental results show the practicality and efficiency of our proposed Smart Beholder: we outperform commercial products in the market in terms of both objective and subjective metrics. We believe the release of Smart Beholder will stimulate future studies on novel multimedia applications enabled by smart lenses.

Abstract Motivation: Robust large-scale sequence analysis is a major challenge in modern genomic science, where biologists are frequently trying to characterize many millions of sequences. Here, we describe a new Java-based architecture for the widely used protein function prediction software package InterProScan. Developments include improvements and additions to the outputs of the software and the complete reimplementation of the software framework, resulting in a flexible and stable system that is able to use both multiprocessor machines and/or conventional clusters to achieve scalable distributed data analysis. InterProScan is freely available for download from the EMBl-EBI FTP site and the open source code is hosted at Google Code. Availability and implementation: InterProScan is distributed via FTP at ftp://ftp.ebi.ac.uk/pub/software/unix/iprscan/5/ and the source code is available from http://code.google.com/p/interproscan/. Contact: http://www.ebi.ac.uk/support or [email protected] or [email protected]