Weakly Supervised temporal Article Grounding (WSAG) is an important task in the field of video understanding and retrieval, which aims to ground sentences from the article with the corresponding video clips (segments), using only video-text pairwise annotations during training. Existing methods typically rely on a proposal-based approach that utilizes contrastive learning to score candidate video clips. However, these approaches often face two main drawbacks: (1) They mainly focus on global video-level alignment with the article while neglecting the multi-level relationships between video clips and text hierarchy; (2) They overlook intra-modal feature learning, failing to adequately distinguish salient patterns. To address these issues, we propose a novel multi-granularity game-theoretic method, namely Hierarchical Cooperative Network (HCN), which models video clips and article units (i.e., at word, sentence, and paragraph layers) as players in a game. Specifically, our proposed method introduces two mechanisms: (1) Intra-modal feature cooperation: Within each modality, we encourage cooperation among the features to obtain a salient feature set as the effective representations, enabling a sharper distinction between semantically overlapping video clips and enhancing discriminative textual cues; (2) Layered cross-modal cooperation: For different semantic layers of the article, we employ the Shapley interaction index to quantify and promote synergistic effects to further enhance cross-modal matching. By decomposing multi-modal interactions into intra-modal and cross-modal cooperations with multi-layer semantics, HCN achieves fine-grained alignment and strengthens salient feature representations for the WSAG task. Experimental results demonstrate that HCN achieves the state-of-the-art performances on representative WSAG benchmarks, significantly outperforming existing weakly supervised methods and competitive video large language models.

Surgical procedures are inherently complex and dynamic, with intricate dependencies and various execution paths. Accurate identification of the intentions behind critical actions, referred to as Primary Intentions (PIs), is crucial to understanding and planning the procedure. This paper presents a novel framework that advances PI recognition in instructional videos by combining top-down grammatical structure with bottom-up visual cues. The grammatical structure is based on a rich corpus of surgical procedures, offering a hierarchical perspective on surgical activities. A grammar parser, utilizing the surgical activity grammar, processes visual data obtained from laparoscopic images through surgical action detectors, ensuring a more precise interpretation of the visual information. Experimental results on the benchmark dataset demonstrate that our method outperforms existing surgical activity detectors that rely solely on visual features. Our research provides a promising foundation for developing advanced robotic surgical systems with enhanced planning and automation capabilities.

With the rapid evolution of artificial intelligence (AI) technologies, supporting computing-intensive and latencysensitive applications in resource-constrained environments has become increasingly challenging. In response, we propose APPOMNLS, a multi-objective optimization approach for Satellite Edge Computing (SEC) that targets application response latency, energy consumption, and on-time completion rates. It integrates a proximal policy optimization (PPO) with a self-attention mechanism under a multi-neighborhood local search framework. The Transformer-based self-attention module enhances the PPO network's representational capability, while multi-neighborhood local search switches flexibly between global and local exploration of the solution space. Experiments based on Iridium-NEXT constellation Two-Line Element (TLE) data demonstrate that our approach clearly outperforms its peers in terms of terminal response speed, energy efficiency and on-time application completion rates. APPO-MNLS brings value to SEC with the capability of guaranteeing reliable and effective global satellite network service.

By equipping the synthetic aperture radar (SAR) sensors on multiple unmanned aerial vehicles (UAVs) to form a UAV swarm (UAVS) and operate collaboratively, UAVS-SAR presents the significant advantages of rapid echoes acquisition, high imaging frame rate as well as high system survivability for advanced SAR applications. However, due to the flexible trajectories as well as the distributed configuration, the problem of the spectrum blurring in the UAVS-SAR is more complicated than that of the conventional monostatic/bistatic SAR configurations, which makes the current fast time domain algorithms (FTDAs) difficult to achieve high imaging performance. In this paper, a novel fast time domain algorithm (FTDA) is developed for UAVS-SAR imaging with both high efficiency and promising accuracy. By developing the hierarchical framework based on the designed spectrum alignment function, the imaging procedures can be realized recursively where back projection (BP) operations are reduced dramatically, and then, the total computational burden are decreased consequently. Moreover, the trajectory difference of the UAVS formation is particularly considered for practical applications, which will inevitably degrade the coherence among the sub-images from different UAV platforms. To address this problem, a correction procedure is designed according to the distributed geometrical configuration. As the coherence between the sub-images is adequately maintained, the data-driven motion compensation (MOCO) is readily developed to remove the residual phase errors to achieve desirable imaging performance. Simulations and raw data experiments are presented to validate the advantages of the proposed algorithm.

Accurate and reliable annotation of anatomical landmarks in laparoscopic surgery remains a challenge due to varying degrees of landmark visibility and changing shapes of human tissues during a surgical procedure in videos. In this paper, we propose a knowledge-driven framework that integrates prior surgical expertise with visual data to address this problem. Inspired by visual reasoning knowledge of tool-anatomy interactions, our framework models a spatio-temporal graph to represent the static topology of tool and tissue and dynamic transitions of landmarks' temporal behavior. By assigning explainable features of the surgical scene as node attributes in the graph, the surgical context is incorporated into the knowledge space. An attention-guided message passing mechanism across the graph dynamically adjusts the focus in different scenarios, enabling robust tracking of landmark states throughout the surgical process. Evaluations on the clinical dataset demonstrate the framework's ability to effectively use the inductive bias of explainable features to label landmarks, showing its potential in tackling intricate surgical tasks with improved stability and reliability.

The study of classroom teachers’ teaching behavior aims to track the development process of teacher behavior, which provides evidence for teachers’ reflection on classroom teaching. Although the recent attempts propose several promising directions for the analysis of teaching behavior, the existing public datasets are still insufficient to meet the need for these potential solutions due to lacking of varied classroom environment, fine-grained teaching scene behavior data. In this paper, we construct a large-scale, diverse, scenario-specific, and multi-task teacher behavior dataset named TBU. On top of it, we systematically investigate representative methods on multiple tasks on TBU, which can serve as a benchmark for the research towards a more comprehensive understanding of teaching video data. The dataset can be available at: https://github.com/cai-KU/TBU.

The growing need for applications that require significant computational power and high responsiveness has significantly driven the advancement of multi-access edge computing (MEC), with satellite edge computing (SEC) emerging as a formidable solution for regions where devices are marooned in areas with sparse computational resources. We present a study on enhancing task scheduling and resource allocation efficiency under the SEC framework, introducing a novel system model that simulates a heterogeneous network characterized by variable bandwidths, channel gains, and transmission powers. We propose a tailored SEC architecture that addresses stringent latency requirements and devise a dynamic scheduling method that adjusts task priorities based on urgency. Our experiments, grounded in realistic parameters from the Iridium and OneWeb satellite constellations, demonstrate the efficacy of our algorithm. The findings underscore significant improvements in managing the SEC landscape, providing robust solutions that enhance overall system performance and reliability in global service networks.

Surgical workflow analysis integrates perception, comprehension, and prediction of the surgical workflow, which helps real-time surgical support systems provide proper guidance and assistance for surgeons. This article promotes the idea of critical actions, which refer to the essential surgical actions that progress towards the fulfillment of the operation. Fine-grained workflow analysis involves recognizing current critical actions and previewing the moving tendency of instruments in the early stage of critical actions. Aiming at this, we propose a framework that incorporates operational experience to improve the robustness and interpretability of action recognition in in-vivo situations. High-dimensional images are mapped into an experience-based explainable feature space with low dimensions to achieve critical action recognition through a hierarchical classification structure. To forecast the instrument's motion tendency, we model the motion primitives in the polar coordinate system (PCS) to represent patterns of complex trajectories. Given the laparoscopy variance, the adaptive pattern recognition (APR) method, which adapts to uncertain trajectories by modifying model parameters, is designed to improve prediction accuracy. The in-vivo dataset validations show that our framework fulfilled the surgical awareness tasks with exceptional accuracy and real-time performance.

Because fast factorized back-projection (FFBP) algorithm is not limited by the assumption of azimuth-invariant of echo signal, it has significant advantages for maneuvering platform synthetic aperture radar (MP-SAR) imaging. Due to the curved trajectory of MP-SAR, the focusing quality of the SAR image becomes very sensitive to the terrain scene elevation, and the range and angular histories are difficult to be solved in polar coordinate system for FFBP implementation. In this letter, a new image reconstruction algorithm integrated with elevation information is proposed for MP-SAR imaging with high efficiency. The proposed algorithm is based on the hybrid coordinate system which is incorporated with elevation information in the FFBP process. In the proposed algorithm, a flexible matching region is introduced to efficiently solve the range and angular histories in FFBP recursion which can reconstruct high-quality images of terrain scene with both high accuracy and efficiency. Simulation experiments are implemented and analyzed to validate the superior performance of the proposed algorithm.

Fast factorized back-projection (FFBP) has significant advantages for bistatic forward-looking synthetic aperture radar (BFSAR) imaging with arbitrary geometry and complex configuration. Conventional FFBP is generally based on the polar coordinate system (PCS) for recursive processing; however, it involves huge interpolations and causes computational inefficiency. In this article, a novel FFBP is developed for BFSAR focusing based on the Cartesian coordinate system (CCS), which is referred to as Cartesian fast factorized back-projection (CFFBP). In the new algorithm, a two-step spectrum correction is designed to avoid spectrum aliasing, and the Nyquist sampling requirement (NSR) for the BFSAR image spectrum can be decreased significantly. With low NSR in CCS, subimage merging can be implemented with noninterpolation processing, so that the proposed algorithm can achieve high performance in both accuracy and efficiency. Moreover, the practical problem of motion error is particularly considered in algorithm development, and well-adapted data-driven motion compensation (DDMC) is integrated with CFFBP based on which a new Cartesian fast time-domain (CFTD) processing framework is developed for BFSAR application. Promising results from both simulation and raw data experiments are provided and analyzed to validate the high performance of the proposed algorithm.

Fast factorized back-projection (FFBP) is a classic fast time-domain algorithm (FTDA), which is not limited by the assumption of azimuth-invariant of echo signal and is suitable for the bistatic synthetic aperture radar (BiSAR) process of arbitrary geometric configuration. However, when the conventional FFBP processing is employed for continuous imaging of multiple full-apertures, the processing efficiency will be decreased significantly, and difficulty will be introduced in motion compensation (MOCO) development. The main contributions in this article include the following two aspects: 1) a new FTDA framework based on FFBP implementation is developed for continuous imaging where echo data are divided into several full-aperture data blocks and then processed separately by FFBP implementation to reduce redundant BP operations for achieving high efficiency and 2) an efficient and effective data-driven MOCO methodology is developed based on the new FTDA framework for high focusing quality. In MOCO, because the phase error functions of subimages are estimated in the phase history domain from different local polar coordinate systems, these phase error functions are actually discontinuous in the spatial domain, which will bring significant discontinuity and defocusing into the final image. To address this problem, the correspondence of error functions between the spatial domain and the wavenumber domain is revealed based on which the phase error functions are reconstructed to remove the discontinuity for high focusing quality. Promising results from both simulation and raw data experiments are provided and analyzed to validate the high performance of the proposed algorithm.

Conventional sparsity-driven synthetic aperture radar (SAR) imagery often encounters the sensitivity of nonsystematic errors and highly computational load. In this article, a cooperative multitask learning algorithm is proposed based on an autocalibrated alternating direction method of multipliers (AutoCal-ADMM) framework, by which the sparse feature of the scenes/targets-of-interests can be enhanced, and simultaneously the nonmodeled motion errors of either airborne platform or moving target can be autocalibrated in a synergistic manner. By leveraging the entropy and sparsity regularizers in the AutoCal-ADMM framework, the proposed algorithm is particularly tailored to obtain focused SAR images with enhanced sparsity. A reasonable surrogate function is designed for a convex objective function, so that an analytical proximal mapping of the entropy regularizer can be derived. Both nonsystematic range cell migration (NsRCM) and azimuthal phase errors (APEs) are concerned and coherently compensated. A linear and complex soft-thresholding operator is introduced for the sparse solution. The proposed algorithm is capable of greatly alleviating “error propagation” between multiple tasks, where an optima balance between the sparse and focusing features can be achieved. Superior performance in terms of convergence and efficiency can be guaranteed. Both raw SAR and canonical ground moving target imaging (GMTIm) data sets are processed and comparisons with conventions are performed, where the effectiveness and superiority of the proposed AutoCal-ADMM algorithm are validated.

Time-domain algorithms have special advantages for bistatic forward-looking synthetic aperture radar (BFSAR) applications with complex geometric configuration. In this paper, a improved fast time-domain algorithm based on orthogonal elliptical polar (OEP) coordinate is proposed for BFSAR imaging, which has prominently reduced burden in computation. In addition, the non-systematic range cell migration (NsRCM) is also analyzed and corrected in the motion compensation (MOCO) process. Simulation experiments are presented and analyzed to validate the superiority of the proposed algorithm.

The original simplicial method (OSM), a variant of the classic Kelley’s cutting plane method, has been shown to converge to the minimizer of a composite convex and submodular objective, though no rate of convergence for this method was known. Moreover, OSM is required to solve subproblems in each iteration whose size grows linearly in the number of iterations. We propose a limited memory version of Kelley’s method (L-KM) and of OSM that requires limited memory (at most n+ 1 constraints for an n-dimensional problem) independent of the iteration. We prove convergence for L-KM when the convex part of the objective g is strongly convex and show it converges linearly when g is also smooth. Our analysis relies on duality between minimization of the composite convex and submodular objective and minimization of a convex function over the submodular base polytope. We introduce a limited memory version, L-FCFW, of the Fully-Corrective Frank-Wolfe (FCFW) method with approximate correction, to solve the dual problem. We show that L-FCFW and L-KM are dual algorithms that produce the same sequence of iterates; hence both converge linearly (when g is smooth and strongly convex) and with limited memory. We propose L-KM to minimize composite convex and submodular objectives; however, our results on L-FCFW hold for general polytopes and may be of independent interest.

For a synthetic aperture radar (SAR) system with curved trajectory, which is different from the conventional SAR, the downward velocity and the acceleration result in highly complicated range history, making it hard to achieve a focused target response. The traditional SAR imaging algorithms are not accurate enough to compensate the phase errors introduced from the highly complicated range history. In this letter, considering the impact of complex range history, an improved hyperbolic range equation is proposed to access the 2-D spectrum for curved trajectory SAR imaging. Based on the derived spectrum, frequency-domain imaging algorithm can be performed to focus targets. By analyzing the phase error and comparing with other current range models, the proposed range model is proved to be precise enough to deal with the complex motion model happened in curved trajectory SAR imaging. Simulation experiments are implemented to evaluate the imaging performance of the proposed approach.

In this letter, a new spectrum-oriented fast factorized backprojection (FFBP) algorithm is proposed for synthetic aperture radar (SAR) imaging on a maneuvering platform. Specifically, an analytical SAR image spectrum is derived in a novel quasi-polar coordinate system based on the FFBP, which makes it easy to incorporate with an autocalibration process for both systematic and nonsystematic errors. Different from the conventional FFBP algorithms developed in polar grid, the proposed algorithm devised in quasi-polar gird conducts the motion-induced phase error as a space-invariant component, which will definitely facilitate the phase autofocusing process during the FFBP recursions. Subsequently, a phase autofocusing process is incorporated in the resultant SAR image formation algorithm. Simulations and discussions are presented to show the focusing quality improvement made by the proposed algorithm.

Because of flexible geometric configuration and trajectory designation, time-domain algorithms become popular for unmanned aerial vehicle (UAV) synthetic aperture radar (SAR) applications. In this paper, a new quasi-polar-coordinate-based fast factorized back-projection (FFBP) algorithm combined with data-driven motion compensation is proposed for miniature UAV-SAR imaging. By utilizing wavenumber decomposition, the analytical spectrum of a quasi-polar grid image is obtained, where the phase errors arising from the trajectory deviations can be conveniently investigated and the phase autofocusing can be compatibly incorporated. Different from the conventional FFBP based on a polar coordinate system, the proposed algorithm operates in a quasi-polar coordinate system, where the phase errors become spacial invariant and can be accurately estimated and easily compensated. Moreover, the relationship between phase errors and nonsystematic range cell migration (NsRCM) is revealed according to the analytical image spectrum, based on which the NsRCM correction is developed to further improve the image focusing quality for high-resolution SAR applications. Promising experimental results from the raw data experiments of miniature UAV-SAR test bed are presented and analyzed to validate the advantages of the proposed algorithm.

In this paper, the analytic expression of synthetic aperture radar (SAR) image spectrum in the polar grid system is derived based on the wavenumber analysis. By revealing the relationship between wavenumber variable and image spectrum in the polar system, we can better understand the mechanism of fast factorized BP (FFBP) processing. Moreover, the form of phase error in spectral domain can be possibly revealed which will facilitate motion compensation and autofocusing in FFBP processing. Simulation results are presented and analyzed to demonstrate the validity of the derived spectrum.

Forward velocity extraction is a very important process for obtaining a high-quality unmanned aerial vehicle (UAV) synthetic aperture radar (SAR) image. Because of the constraints of low flying altitude and small platform size, the flight path of the UAV is easily disturbed by the atmospheric turbulence. The complex motion error of the UAV's flight path makes the forward velocity difficult to be extracted from raw SAR data. To address this problem, an adaptive notch filtering (ANF)-based approach for forward velocity extraction is proposed. Based on the kinetic characteristics of the UAV, the variation of Doppler centroid frequency is analyzed and exploited to remove most components of the cross-track acceleration in the low-frequency range. Then, by regarding the forward velocity component as a narrow-band component, ANF processing is employed to extract it from the estimated Doppler rate. Comparing with the methods reported in the literature, the ANF method can achieve higher accuracy and efficiency due to its excellent notching performance and strong suppression for narrow-band signals. Promising results from raw data experiments are presented to demonstrate the validity and superiority of the proposed method.

In airborne/stationary bistatic-synthetic-aperture-radar imaging, translational invariance was no longer valid. After range cell migration correction, the range-compressed signal under the same range gate exhibited azimuth-dependent FM rates that made the motion-induced phase error difficult to separate from the echoes. To solve this problem, an azimuth-dependent phase gradient autofocus (PGA) algorithm was proposed. Different from the conventional PGA, the residual quadratic phase arising from the azimuth-dependent FM rates was additionally estimated and compensated. As the influence of the azimuth-dependent FM rates was greatly reduced, a phase gradient estimator was subsequently applied for accurate phase error retrieval. Acquired raw data were analyzed to verify the proposed algorithm.

In the above paper (ibid., vol. 10, no, 6, pp. 1290-1294, Nov. 2013), there is an error in equation (5). The correction is presented here.

A fuzzy approach to assess service satisfaction values of retailers is discussed in this paper, which has been developed from fuzzy assessment methods and applied to questionnaire analysis of retailers' service satisfaction. Subjective judgments are often vague and it is not easy for retailers to express the satisfaction of service quality using an exact numerical value, and suggest the necessity of using a fuzzy approach. Applying the fuzzy approach, service satisfaction values are identified and organized into five basic service types service of client manager, ordering service, delivering service, consulting and complaint service, and business environment. Finally, the uncertainties of the service satisfaction value sets are quantified by the measure of fuzzy entropy.