With the advantages of low latency, low power consumption, and high parallelism, optical neural networks (ONN) offer a promising solution for time-sensitive and resource-limited artificial intelligence applications. However, the performance of the ONN model is often diminished by the gap between the ideal simulated system and the actual physical system. To bridge the gap, this work conducts extensive experiments to investigate systematic errors in the optical physical system within the context of image classification tasks. Through our investigation, two quantifiable errors—light source instability and exposure time mismatches—significantly impact the prediction performance of ONN. To address these systematic errors, a physics-constrained ONN learning framework is constructed, including a well designed loss function to mitigate the effect of light fluctuations, a CCD adjustment strategy to alleviate the effects of exposure time mismatches and a ’physics-prior based’ error compensation network to manage other systematic errors, ensuring consistent light intensity across experimental results and simulations. In our experiments, the proposed method achieved a test classification accuracy of 96.5% on the MNIST dataset, a substantial improvement over the 61.6% achieved with the original ONN. For the more challenging QuickDraw16 and Fashion MNIST datasets, experimental accuracy improved from 63.0% to 85.7% and from 56.2% to 77.5%, respectively. Moreover, the comparison results further demonstrate the effectiveness of the proposed physics-constrained ONN learning framework over state-of-the-art ONN approaches. This lays the groundwork for more robust and precise optical computing applications.

Weakly-supervised Temporal Action Localization (W-TAL) is a significant task in video understanding, intending to recognize the category and pinpoint the temporal boundaries of action segments in untrimmed videos based on the video-level labels. Due to the lack of frame-level annotations, recognizing the spatiotemporal relationships among action snippets and precisely separating the foreground and background are two arduous challenges. In this paper, we propose a novel Feature Enhancement and Foreground-Background Separation (FE-FBS) method to address these problems. Specifically, we construct a spatiotemporal cooperation enhancement scheme utilizing residual graph convolutional to capture the spatial and temporal dependencies between the current snippet and others, generating snippet features with spatial and temporal correlations, thereby ensuring a complete feature representation for action localization. Furthermore, we propose a two-branch explicit foreground-background joint attention mechanism to guide foreground and background modeling, combined with an inverse enhancement strategy to enhance action weight for better foreground and background distinction, thereby improving the accuracy of action localization. We use THUMOS’14 and ActivityNet v1.3 datasets to achieve accuracy rates of 66.7% and 38.9%, respectively, and compare our method to other methods, gaining superiority.