Recent advancements in multimodal large models have significantly bridged the representation gap between diverse modalities, catalyzing the evolution of video multimodal interpretation, which enhances users' understanding of video content by generating correlated modalities. However, most existing video multimodal interpretation methods primarily concentrate on global comprehension with limited user interaction. To address this, we propose a novel task, Controllable Video Segmentation and Captioning (SegCaptioning), which empowers users to provide specific prompts, such as a bounding box around an object of interest, to simultaneously generate correlated masks and captions that precisely embody user intent. An innovative framework, Scene Graph-guided Fine-grained SegCaptioning Transformer (SG-FSCFormer), is designed to integrate a Prompt-guided Temporal Graph Former to effectively capture and represent user intent through an adaptive prompt adaptor, ensuring that the generated content aligns well with the user's requirements. Furthermore, our model introduces a Fine-grained Mask-linguistic Decoder to collaboratively predict high-quality caption-mask pairs using a Multi-entity Contrastive loss, while providing fine-grained alignment between each mask and its corresponding caption tokens, thereby enhancing the user's comprehension of videos. Comprehensive experiments conducted on two benchmark datasets demonstrate that SG-FSCFormer achieves remarkable performance, effectively capturing user intent and generating precise multimodal outputs tailored to user specifications. Our code is available at https://github.com/XuZhang1211/SG-FSCFormer.

The success of most existing cross-modal retrieval methods heavily relies on the assumption that the given queries follow the same distribution of the source domain. However, such an assumption is easily violated in real-world scenarios due to the complexity and diversity of queries, thus leading to the query shift problem. Specifically, query shift refers to the online query stream originating from the domain that follows a different distribution with the source one. In this paper, we observe that query shift would not only diminish the uniformity (namely, within-modality scatter) of the query modality but also amplify the gap between query and gallery modalities. Based on the observations, we propose a novel method dubbed Test-time adaptation for Cross-modal Retrieval (TCR). In brief, TCR employs a novel module to refine the query predictions (namely, retrieval results of the query) and a joint objective to prevent query shift from disturbing the common space, thus achieving online adaptation for the cross-modal retrieval models with query shift. Expensive experiments demonstrate the effectiveness of the proposed TCR against query shift. Code is available at https://github.com/XLearning-SCU/2025-ICLR-TCR.

The fabrication of Nanoimprint Lithography (NIL) masters serves as the initial process in the manufacturing of devices such as silicon photonic chips using NIL. Repairing defects and modifying structures on an NIL master accurately is critical for NIL manufacturing. This study proposes an innovative scanning electron microscope (SEM)-based in-situ nanomanipulation technique to address the macro-micro-nano cross-scale nanopositioning issues necessary for repair functions. A macro/micro closed-loop control system was designed, which includes a frequency/voltage (f/u) proportional controller, a real-time direct inverse hysteresis compensation feedforward controller, a grating displacement sensor, and a cross-scale nanopositioning platform. The performance testing of the nano-manipulation approach resulted in a repair range of 22.18×20.92×10.06 mm3, meeting the repair requirements for most silicon photo chip NIL masters in terms of size. The system achieved a repair accuracy at 4.946 nm (X-axis), 4.663 nm (Y-axis), and 4.679 nm (Z-axis). As a demonstrate, the repair functionality testing confirmed the ability of the system to remove target structures such as cantilever beams and detach adhered particles from a master surface.

Contouring control of 3D trajectory is critical in multi-axial machine tool, scanning stage and other precision automation systems. Currently, most contouring controllers are based on Frenet frames, thus their limited applicability to nonzero-curvature 3D trajectories rather than zero-curvature ones which exist ubiquitously in multi-axial motion systems. This paper proposes an optimal vector-referenced moving frame based contouring controller (OVRMFCC) suitable for contouring control of 3D zero-curvature trajectories. Firstly, the optimal vector-referenced moving frame (OVRMF) capable of framing arbitrary finite-length smooth 3D trajectory regardless of its curvature was proposed. Then, a contouring controller (OVRMFCC) based on OVRMF was designed, followed by derivation of its analytical form and proof of its convergence. Finally, this controller was deployed to an FPGA-based controller target with comprehensive comparison experiments on a triaxial system. Experimental results indicates that OVRMFCC reduces at least 46.6% maximum contour error, and 25.0% root-mean-square contour error compared to cross-coupled controller. Besides, OVRMFCC achieves almost the same precision on trajectories with nonzero curvature or curvature singularities compared to TCF. It still maintains high-precision contour tracking on trajectories with continuous zero-curvature segments or planned discrete trajectories with sharp curvature changes, while TCF crashes or leads to several times larger contour error. Note to Practitioners—This work is motivated by the increasing need of contouring control of 3D trajectory in precision automation systems. The mainstream 3D contouring controllers, like task coordinated frame method and model predicted contouring controller, are invalid for zero-curvature 3D trajectory which ubiquitously exists in motion system since they are based on Frenet frame which fails to be defined where curvature is zero. Although cross-coupled controller can tackle zero-curvature 3D trajectory, it proves inefficient in reducing contour error as it is commonly model-free and not specifically designed. To solve this problem, we proposed an optimal vector referenced moving frame (OVRMF) for arbitrary infinite-length smooth 3D trajectory framing and proves its existence strictly. The main advantage of OVRMF lies in its independence of curvature, thus its existence everywhere as long as the curve is$C^{1}$continuous. And based on OVRMF a contouring controller (OVRMFCC) is designed which combines the benefit of both TCF and OVRMF. With this, OVRMFCC can track any infinite-length$C^{3}$-continuous 3D trajectory, which fill the gap of traditional 3D contouring controller. Experimental results reveal that OVRMFCC maintains high precision regardless of trajectory curvature, which outperforms both cross-coupled controller and TCF.

Generalized zero-shot learning (GZSL) shows great potential for improving generalization to unseen classes in real-world scenarios. However, most GZSL methods depend on benchmark datasets with per-class attribute annotations, which creates a large semantic gap and worsens the domain shift problem in the visual-semantic space. To address these challenges, instance-level attributes offer an intuitive solution, but they require expensive manual annotation. In this paper, we propose a simple yet effective approach called per-instance attribute synthesis (PIAS) to generate diverse semantic representations for each instance. Our method first uses the Vision Transformer (ViT) model to extract visual features and then generates per-instance attributes. The patch splitting, positional embedding, and multi-head self-attention mechanisms in ViT improve the discriminability of both visual and semantic representations. Next, we define the generated attributes of class-average images as class anchor points. These anchor points are calibrated in the semantic space by minimizing the cosine similarity between the anchor points and per-class attribute annotations. Finally, we improve the diversity of generated per-instance attributes by aligning the topological structure between per-class attribute annotations and synthesized per-instance attributes with that between class-average visual features and per-instance visual features. We conduct comprehensive experiments on three challenging ZSL datasets: AWA2, CUB, and SUN. The results show that PIAS significantly outperforms state-of-the-art methods under both ZSL and GZSL settings. We further demonstrate the generalization ability of PIAS by applying it to attribute-based zero-shot image retrieval tasks.

Exemplar-based image translation has garnered significant interest from researchers due to its broad applications in multimedia/multimodal processing. Existing methods primarily employ Euclidean-based losses to implicitly establish cross-domain correspondences between exemplar and conditional images, aiming to produce high-fidelity images. However, these methods often suffer from two challenges: 1) Insufficient excavation of domain-invariant features leads to low-quality cross-domain correspondences, and 2) Inaccurate correspondences result in errors propagated during the translation process due to a lack of reliable prior guidance. To tackle these issues, we propose a novel prior-guided diffusion model with global-local contrastive learning (PROMOTE), which is trained in a self-supervised manner. Technically, global-local contrastive learning is designed to align two cross-domain images within hyperbolic space and reduce the gap between their semantic correlation distributions using the Fisher-Rao metric, allowing the visual encoders to extract domain-invariant features more effectively. Moreover, a prior-guided diffusion model is developed that propagates the structural prior to all timesteps in the diffusion process. It is optimized by a novel prior denoising loss, mathematically derived from the transitions modified by prior information in a self-supervised manner, successfully alleviating the impact of inaccurate correspondences on image translation. Extensive experiments conducted across seven datasets demonstrate that our proposed PROMOTE significantly exceeds state-of-the-art performance in diverse exemplar-based image translation tasks. The source code is publicly available at http://github.com/zgj77/PROMOTE.

Cracks are one of the most common and significant types of road surface damage, posing a threat to the safety of pedestrians and vehicles. If left untreated, cracks can lead to severe consequences such as road and bridge collapse. Therefore, it is essential to develop an efficient road crack detection method. Traditional crack identification methods have the problem of being largely affected by the environment and having low recognition accuracy. In this paper, we propose a road crack detection model based on an improved You Only Look Once version 5 (YOLOv5) model that addresses the limitations of existing state-of-the-art crack detection methods in terms of accuracy and detection speed. First, we replace the intersection over union (IoU) loss function with the SCYLLA-IoU (SIoU) loss function for better accuracy. Second, to enhance detection performance, we replace the feature pyramid network (FPN) with a bi-directional feature pyramid network (BiFPN). Finally, to better extract spatial feature information of different sizes, we modify the original Spatial Pyramid Pooling-Fast (SPPF) module of YOLOv5 by using Spatial Pyramid Pooling Cross-Stage Partial Connections (SPPCSPC). We evaluated our YOLOv5s-BiFPN-SPPCSPC-SIoU (YOLOv5s-BSS) method on the dataset from the IEEE 2020 Global Road Damage Detection Challenge (GRDDC) and achieved promising results on road damage datasets from China, Japan, and the United States. The [email protected] of different cracks in three datasets reached 84.9%, 54.6%, and 71%. Our method outperforms related methods, with an increase of 0.7%, 0.7%, and 2.8% over YOLOv5s.

This paper presents a precision centimeter-range positioner based on a Lorentz force actuator using flexure guides. An additional digital-to-analog converter and an operational amplifier (op amp) circuit together with a suitable controller are used to enhance the positioning accuracy to the nanometer level. First, a suitable coil is designed for the actuator based on the stiffness of the flexure guide model. The flexure mechanism and actuator performance are then verified with finite element analysis. Based on these, a means to enhance the positioning performance electronically is presented together with the control scheme. Finally, a prototype is fabricated, and the performance is evaluated. This positioner features a range of 10 mm with a resolution of 10 nm. The proposed scheme can be extended to other systems.