Omnidirectional depth estimation predicts 360-degree depth information using multiple fisheye cameras arranged in a surround-view configuration. However, due to the lack of reference panorama and differences between the predicted depth viewpoint and input cameras, it is challenging to construct and utilize semantic information to improve depth accuracy, resulting in limited accurate in complex regions such as non-overlapping, weak textures, object boundaries and occlusions. This paper proposes a novel model architecture that effectively extracts and leverages semantic information to enhance the accuracy of omnidirectional depth estimation. Specifically, the proposed algorithm combines the variance and mean of multi-view image features to construct the fused matching cost and utilize both geometry and semantic constraints. The model extracts 360-degree semantic context during matching cost aggregation, and predict the corresponding panoramas jointly with omnidirectional depth maps. A semantic-aware spatial propagation module is then employed to further refine the depth estimation. We leverage a multi-scale multi-task learning strategy to supervise the prediction of omnidirectional depth maps and panoramas jointly. The proposed approach achieves state-of-the-art performance on public datasets, and also demonstrates high-precision results on real-world data. The experiments with varying camera configurations validate the generalization ability and flexibility of the algorithm.

Abstract Online detection of action start is a significant and challenging task that requires prompt identification of action start positions and corresponding categories within streaming videos. This task presents challenges due to data imbalance, similarity in boundary content, and real‐time detection requirements. Here, a novel Time‐Attentive Fusion Network is introduced to address the requirements of improved action detection accuracy and operational efficiency. The time‐attentive fusion module is proposed, which consists of long‐term memory attention and the fusion feature learning mechanism, to improve spatial‐temporal feature learning. The temporal memory attention mechanism captures more effective temporal dependencies by employing weighted linear attention. The fusion feature learning mechanism facilitates the incorporation of current moment action information with historical data, thus enhancing the representation. The proposed method exhibits linear complexity and parallelism, enabling rapid training and inference speed. This method is evaluated on two challenging datasets: THUMOS’14 and ActivityNet v1.3. The experimental results demonstrate that the proposed method significantly outperforms existing state‐of‐the‐art methods in terms of both detection accuracy and inference speed.

The Online Detection of Action Start (ODAS) has attracted the attention of researchers because of its practical applications in areas such as security and emergency response. However, online detection of activity boundaries remains a challenging task due to the inherent ambiguity of boundary definition and the significant imbalance in the number of boundaries and nonboundary points. To address this issue, this study proposes a novel Distribution-aware Activity Boundary Representation (DABR) method that utilizes a continuous probability density function to smooth the probability of moments near activity boundaries. The proposed DABR reduces the penalty for detecting moments near ground-truth boundary points, while increasing the number of samples related to boundary points. Additionally, we introduce a two-stage framework that incorporates class-informed information in temporal localization for more efficient activity boundary localization. Extensive experiments demonstrate that our method achieves state-of-the-art results on two standard datasets, particularly exhibiting a significant improvement of 11.5% at average p-mAP on the THUMOS'14 dataset.

Bacterial infections often involve virulence factors that play a crucial role in the pathogenicity of bacteria. Accurate detection of virulence factor genes (VFGs) is essential for precise treatment and prognostic management of hypervirulent bacterial infections. However, there is a lack of rapid and accurate methods for VFG identification from the metagenomic data of clinical samples. Here, we developed a Reads-based Virulence Factors Scanner (RVFScan), an innovative user-friendly online tool that integrates a comprehensive VFG database with similarity matrix-based criteria for VFG prediction and annotation using metagenomic data without the need for assembly. RVFScan demonstrated superior performance compared to previous assembly-based and read-based VFG predictors, achieving a sensitivity of 97%, specificity of 98% and accuracy of 98%. We also conducted a large-scale analysis of 2425 clinical metagenomic datasets to investigate the utility of RVFScan, the species-specific VFG profiles and associations between VFGs and virulence phenotypes for 24 important pathogens were analyzed. By combining genomic comparisons and network analysis, we identified 53 VFGs with significantly higher abundances in hypervirulent Klebsiella pneumoniae (hvKp) than in classical K. pneumoniae. Furthermore, a cohort of 1256 samples suspected of K. pneumoniae infection demonstrated that RVFScan could identify hvKp with a sensitivity of 90%, specificity of 100% and accuracy of 98.73%, with 90% of hvKp samples consistent with clinical diagnosis (Cohen's kappa, 0.94). RVFScan has the potential to detect VFGs in low-biomass and high-complexity clinical samples using metagenomic reads without assembly. This capability facilitates the rapid identification and targeted treatment of hvKp infections and holds promise for application to other hypervirulent pathogens.