Full-body motion tracking plays an essential role in AR/VR applications, bridging physical and virtual interactions. However, it is challenging to reconstruct realistic and diverse full-body poses based on sparse signals obtained by head-mounted displays, which are the main devices in AR/VR scenarios. Existing methods for pose reconstruction often incur high computational costs or rely on separately modeling spatial and temporal dependencies, making it difficult to balance accuracy, temporal coherence, and efficiency. To address this problem, we propose KineST, a novel kinematics-guided state space model, which effectively extracts spatiotemporal dependencies while integrating local and global pose perception. The innovation comes from two core ideas. Firstly, in order to better capture intricate joint relationships, the scanning strategy within the State Space Duality framework is reformulated into kinematics-guided bidirectional scanning, which embeds kinematic priors. Secondly, a mixed spatiotemporal representation learning approach is employed to tightly couple spatial and temporal contexts, balancing accuracy and smoothness. Additionally, a geometric angular velocity loss is introduced to impose physically meaningful constraints on rotational variations for further improving motion stability. Extensive experiments demonstrate that KineST has superior performance in both accuracy and temporal consistency within a lightweight framework.

Real-time localization of endoscope is significant for the navigation and automation of endoscopic diagnosis and minimally invasive surgery. However, traditional localization based on optical tracking or magnetic tracking is easily influenced by occlusion or electromagnetic interference, while the implementation is complicated. Meanwhile, transformation and correlation information in image pairs are still ignored in existing visual localization methods for endoscopy. In this work, a novel relative pose regression framework is proposed for relative pose estimation and absolute pose tracking of endoscope based on endoscopic videos. Firstly, scene features and transformation features are respectively extracted from endoscopic observations and the corresponding optical flow by the proposed feature encoder based on gated convolution, which can prevent gradient vanishing when training the encoder from scratch on endoscopic data. Furthermore, a novel correlation module based on cross-attention is proposed to extract correlation features from two input images, which can capture more key features in endoscopic frames with more limited vision from local to global. Moreover, a novel pose decoder with upsampling and downsampling on the channel dimension is utilized to extract richer representation from the concatenated feature map for relative transformation vector prediction. The proposed method outperforms the state-of-the-art methods on the datasets from nasal endoscopy and colonoscopy, with an average localization error of less than 5%. The further experiments also demonstrate the efficiency of the proposed method. The demo videos of visual localization can be found on https://endoloc.netlify.app/

Real-time ego-motion tracking for endoscope is a significant task for efficient navigation and robotic automation of endoscopy. In this paper, a novel framework is proposed to perform real-time ego-motion tracking for endoscope. Firstly, a multi-modal visual feature learning network is proposed to perform relative pose prediction, in which the motion feature from the optical flow, the scene features and the joint feature from two adjacent observations are all extracted for prediction. Due to more correlation information in the channel dimension of the concatenated image, a novel feature extractor is designed based on an attention mechanism to integrate multi-dimensional information from the concatenation of two continuous frames. To extract more complete feature representation from the fused features, a novel pose decoder is proposed to predict the pose transformation from the concatenated feature map at the end of the framework. At last, the absolute pose of endoscope is calculated based on relative poses. The experiment is conducted on three datasets of various endoscopic scenes and the results demonstrate that the proposed method outperforms state-of-the-art methods. Besides, the inference speed of the proposed method is over 30 frames per second, which meets the real-time requirement. The project page is here: remote-bmxs.netlify.app

Ego-motion estimation is a critical task for both monocular odometry and depth estimation, which are significant for navigation and scene perception in endoscopy. Most of existing ego-motion estimation frameworks only utilize a single modality of visual feature. In this work, a novel multimodal visual feature-based framework is proposed to perform real-time vision-based ego-motion estimation for visual odometry and depth estimation in endoscopic scenes. In the framework, to extract correlation information of two adjacent endoscopic frames, a channel attention-based module is proposed to integrate multidimensional features from the concatenated image and a feature interaction module is proposed to fuse scene features from two observations. Moreover, a novel pose decoder based on depthwise separable convolution is proposed to extract multiscale feature representation. Furthermore, a fully supervised pipeline and a self-supervised pipeline with the proposed framework are designed for monocular odometry and depth estimation, respectively. In the experiments, the proposed framework is compared with several advanced methods on five different datasets of various endoscopic scenes. Both quantitative and qualitative results demonstrate the pipeline with the proposed framework provides the most accurate monocular odometry and depth estimation in different kinds of endoscopic scenes.

SUMMARY: The lollipop-diagram is one of the widely used graphical representations to visualize and explore translational effects of genetic mutations in cancer genomics. However, an easy-to-use lollipop-diagram tool with full functionality is still lacking. Here, we introduce g3viz, an R package that enables researchers to explore genetic mutation data using a lollipop-diagram in a web browser. With a few lines of R code, users can interactively visualize data details, annotate findings and export resultant diagrams in high-quality figures. Because of usefulness and usability, g3viz can be generally exploited by researchers with different levels of bioinformatics skills and programming experience. AVAILABILITY AND IMPLEMENTATION: The R package is freely available under the MIT license from CRAN (http://cran.r-project.org/web/packages/g3viz). The g3lollipop JavaScript package is freely available under MIT license at GitHub (https://github.com/g3viz/g3lollipop.js). SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.