Recent advances in vision-language models (VLMs) and reinforcement learning (RL) have driven progress in GUI automation. However, most existing methods rely on static, one-shot visual inputs and passive perception, lacking the ability to adaptively determine when, whether, and how to observe the interface. We present GUI-Eyes, a reinforcement learning framework for active visual perception in GUI tasks. To acquire more informative observations, the agent learns to make strategic decisions on both whether and how to invoke visual tools, such as cropping or zooming, within a two-stage reasoning process. To support this behavior, we introduce a progressive perception strategy that decomposes the decision-making into coarse exploration and fine-grained grounding, coordinated by a two-level policy. In addition, we design a spatially continuous reward function tailored to tool usage, which integrates both location proximity and region overlap to provide dense supervision and alleviate the reward sparsity common in GUI environments. On the ScreenSpot-Pro benchmark, GUI-Eyes-3B achieves 44.8% grounding accuracy using only 3k labeled samples, significantly outperforming both supervised and RL-based baselines. These results highlight that tool-aware active perception, enabled by staged policy reasoning and fine-grained reward feedback, is critical for building robust and data-efficient GUI agents.

Human-robot interaction plays a critical role in scientific experiments by ensuring efficient and reliable execution of experimental tasks. To achieve successful task completion, robots must adapt in real time to unexpected task variations, external disturbances, and safety constraints. In this work, we propose a reactive task and motion planning framework designed to address these challenges. By formulating interaction tasks using Linear Temporal Logic (LTL), our approach introduces Planning Decision Tree and Augmented Planning Decision Tree approach to dynamically adjust task sequences in response to environmental changes. At the execution layer, we employ a Model Predictive Path Integral controller, which ensures both efficient and safe control. Additionally, the planning interface effectively coordinates the planning and execution layers, ensuring strict adherence to experimental task specifications. The effectiveness of the proposed reactive planning framework is demonstrated through physical experiments using a 7-DoFs robot. Project website: https://sites.google.com/view/rtlp-iros/

As an effective pruning method, dynamic pruning introduces gate modules that allow different input data to choose different channels. This shows that the choice of channels strongly depends on data. However, this creates an additional computational burden because of the gate modules. In this paper, we propose a simple, efficient and transferable channel pruning method via attention module and memory curve, dubbed as CPAM, which not only takes advantage of the strong correlation between data and channels, but also does not impose any additional computational burden on the model. Inspired by the memory curve, we use a progressive method without any sparse operation. Moreover, our method has been demonstrated effective for many advanced CNN architectures. Notably, on CIFAR-10, CPAM reduces 50% FLOPs on ResNet-56 with 0.31% relative accuracy improvement, which has advanced the state-of-the-art.

Vision transformer has emerged as a new paradigm in computer vision, showing excellent performance while accompanied by expensive computational cost. Image token pruning is one of the main approaches for ViT compression, due to the facts that the complexity is quadratic with respect to the token number, and many tokens containing only background regions do not truly contribute to the final prediction. Existing works either rely on additional modules to score the importance of individual tokens, or implement a fixed ratio pruning strategy for different input instances. In this work, we propose an adaptive sparse token pruning framework with a minimal cost. Specifically, we firstly propose an inexpensive attention head importance weighted class attention scoring mechanism. Then, learnable parameters are inserted as thresholds to distinguish informative tokens from unimportant ones. By comparing token attention scores and thresholds, we can discard useless tokens hierarchically and thus accelerate inference. The learnable thresholds are optimized in budget-aware training to balance accuracy and complexity, performing the corresponding pruning configurations for different input instances. Extensive experiments demonstrate the effectiveness of our approach. Our method improves the throughput of DeiT-S by 50% and brings only 0.2% drop in top-1 accuracy, which achieves a better trade-off between accuracy and latency than the previous methods.

Learning subtle representation about object parts plays a vital role in fine-grained visual recognition (FGVR) field. The vision transformer (ViT) achieves promising results on computer vision due to its attention mechanism. Nonetheless, with the fixed size of patches in ViT, the class token in deep layer focuses on the global receptive field and cannot generate multi-granularity features for FGVR. To capture region attention without box annotations and compensate for ViT shortcomings in FGVR, we propose a novel method named Adaptive attention multi-scale Fusion Transformer (AFTrans). The Selective Attention Collection Module (SACM) in our approach leverages attention weights in ViT and filters them adaptively to correspond with the relative importance of input patches. The multiple scales (global and local) pipeline is supervised by our weights sharing encoder and can be easily trained end-to-end. Comprehensive experiments demonstrate that AFTrans can achieve SOTA performance on three published fine-grained benchmarks: CUB-200-2011, Stanford Dogs and iNat2017.

We introduce a Power-of-Two post-training quantization( PTQ) method for deep neural network that meets hardware requirements and does not call for long-time retraining. PTQ requires a small set of calibration data and is easier for deployment, but results in lower accuracy than Quantization-Aware Training( QAT). Power-of-Two quantization can convert the multiplication introduced by quantization and dequantization to bit-shift that is adopted by many efficient accelerators. However, the Power-of-Two scale has fewer candidate values, which leads to more rounding or clipping errors. We propose a novel Power-of-Two PTQ framework, dubbed RAPQ, which dynamically adjusts the Power-of-Two scales of the whole network instead of statically determining them layer by layer. It can theoretically trade off the rounding error and clipping error of the whole network. Meanwhile, the reconstruction method in RAPQ is based on the BN information of every unit. Extensive experiments on ImageNet prove the excellent performance of our proposed method. Without bells and whistles, RAPQ can reach accuracy of 65% and 48% on ResNet-18 and MobileNetV2 respectively with weight INT2 activation INT4. We are the first to propose PTQ for the more constrained but hardware-friendly Power-of-Two quantization and prove that it can achieve nearly the same accuracy as SOTA PTQ method. The code will be released.

Accurate detection of the key components of transmission lines is an important part of smart grid construction. However, the detection of key components of transmission lines faces the problems of severe occlusion, irregular shape, and large size differences, which present a great challenge for anchor-based object detectors. We propose the anchor-based and anchor-free (ABAF) model, an object detection algorithm for both general and special object datasets. The ABAF detector is jointly trained with the anchor-based branch and anchor-free branch. At the time of inference, the predicted results of both are fused to yield the final detections. The experimental results show that the anchor-based branch is good at detecting shape-regular objects and the anchor-free branch is better at detecting irregularly shaped objects than anchor-based branch, and our fusion model ABAF has strong robustness for different datasets with excellent and stable performance. ABAF with ResNet-50 achieves 89.69% mAP on the transmission line dataset, a 3.72% improvement over one-stage anchor-based detector RetinaNet and we achieve 79.03% mAP on the PASCAL VOC dataset.