As eye-tracking becomes increasingly common in modern mobile devices, the potential for hands-free, gaze-based interaction grows, but current gesture sets are largely expert-designed and often misaligned with how users naturally move their eyes. To address this gap, we introduce a two-phase methodology for developing intuitive gaze gestures. First, four co-design workshops with 20 non-expert participants generated 102 initial concepts. Next, four gaze interaction experts reviewed and refined these into a set of 32 gestures. We found that non-experts, after a brief introduction, intuitively anchor gestures in familiar metaphors and develop a compositional grammar; i.e., activation (dwell) + action (gaze gesture or blink), to ensure intentionality and mitigate the classic Midas Touch problem. Experts prioritized gestures that are ergonomically sound, aligned with natural saccades, and reliably distinguishable. The resulting user-grounded, expert-validated gesture set, along with actionable design principles, provides a foundation for developing intuitive, hands-free interfaces for gaze-enabled devices.

Event cameras respond in microseconds, enabling them to capture rapid changes in eye movements. However, the inherent variability in eye movements usually introduces uncertainty, complicating accurate predictions based solely on temporal trends. To address this, we propose an event-based gaze estimation method that leverages spatial-temporal interactions to integrate both spatial and temporal distributions for enhanced eye movement prediction. The temporal features are generated by the gated recurrent unit (GRU), while the multi-level spatial features are extracted by convolutions. To preserve fine-grained spatial details, cross-channel information fusion is employed to merge spatiotemporal features, with skip connections ensuring retention of the original input’s spatial information. Experimental results on the 3ET+ benchmark dataset demonstrate that our method achieves a competitive accuracy of 1.52 pixels.

Event-driven eye-based emotion recognition has attracted increasing attention due to the high temporal resolution and dynamic range inherent to event cameras. The intrinsic spatial sparsity of event data, combined with the eye-based emotion recognition task's reliance on localized features such as eyebrows and eyelids, makes it intuitive and efficient to discard less informative regions. However, integrating such sparsification into CNNs remains challenging due to their reliance on dense grid-based operations. In this paper, we propose an efficient vision transformer framework for eye-based emotion recognition with event cameras. Specifically, we present window selection and token selection schemes tailored for event data and eye-based emotion recognition, which can diminish computing demands while enhancing performance. Firstly, we estimate the importance of all local windows and discard those with limited information, reducing computational cost while emphasizing attention on the periocular region. Secondly, we further introduce an adaptive token pruning mechanism that jointly evaluates the input event data and tokens to predict a binary decision mask, identifying and discarding uninformative tokens. Extensive experiments validate that the proposed approach outperforms existing state-of-the-art methods in accuracy by a significant margin.

Underwater images suffer from severe color distortion, due to the wavelength-dependent light attenuation and scattering. Various underwater image enhancement methods have been developed to improve the quality of degraded underwater images. However, contemporary approaches often overlook the impact of different scene colors on the overall process, potentially leading to undesired outcomes, such as enhanced images exhibiting excessive redness. In this paper, we observe that the color tones of degraded underwater images exhibit variability under the influence of different underwater targets and scenes. Each degraded color channel can be utilized to guide the color correction of other channels. Given this, a light-weight underwater color correction network, dubbed UCCNet, is presented to alleviate the issue of color corruption. In UCCNet, three parallel branches are designed to excavate the residual information within each color channel, subsequently leveraging these features to improve the quality of underwater images. Moreover, facing the challenge of effectively enhancing underwater images in diverse and complex scenes, the model UCCNet-KT is established based on UCCNet. In UCCNet-KT, the technology of knowledge transfer is designed to improve the generalization ability by enriching the dataset and constructing the loss function. Extensive experiments on various underwater datasets indicate the impressive performance of the UCCNet and UCCNet-KT qualitatively and quantitatively.

Underwater images usually suffer from color casts and low contrast due to the absorption and scattering of light by the water medium. The degradation is caused not only by the light attenuation on the scene-sensor path but also by the light attenuation on the water surface-scene path. To eliminate the dual-path light attenuation, we propose a novel unified total variation method based on an extended underwater imaging model. Unlike previous variation-based methods that only consider light propagation along the scene-sensor path, we additionally include light propagation along the water surface-scene path in the underwater imaging model. In the proposed variational framework, we transform underwater image enhancement into two subproblems and construct different prior knowledge-guided optimization functions for them. The two subproblems aim to remove the light attenuation along the scene-sensor and surface-scene paths. Moreover, we present an alternating direction minimization algorithm based on an augmented Lagrange multiplier to address the optimization problems. The subjective and objective experimental results on underwater images with different attenuation characteristics demonstrate that the proposed method achieves good performance in underwater image enhancement.

The presence of suspended particles in turbid media not only superimposes a veil on the scene but also changes the color of the scene, resulting in poor visibility and low contrast of images. It is essential to overcome these effects for further applications of the images. However, these effects are difficult to eliminate simultaneously and thus the restoration for the images becomes multitasked. In this letter, we propose an extended image formation model that introduces color constancy theory to discuss the change of light color when natural light penetrates the medium to the object. Based on the model, the image restoration task is divided into two parts: dehazing and color correction, and for these two parts, scene depth fusion-based dehazing and adaptive color constancy are proposed. The dehazing approach introduces a weighted fusion strategy to estimate scene depth to achieve a better dehazing effect. The color constancy approach improves the Gray World hypothesis with gain factors to adaptively compensate for the chromatic loss and extends the color deviation solved by color constancy from color temperature to medium. Extensive experiments on images of different scenes prove the effectiveness of the proposed method in image restoration.

In this letter, a simple yet effective dehazing framework is proposed, which consists of a novel color correction and a contrast enhancement. Most of the existing dehazing works focus on enhancing the contrast of the degraded images, but rarely of them concern about the color cast, which is ubiquitous in the scattering medium. To address the color distortion, an affine transform model-based color correction method is first proposed to improve the appearance of the image while preserving the details, which is inspired by the traditional color transfer. The color transfer alters the color values of a source image by sharing the global color statistics of a reference image, which makes it unsuitable to address the locally variable color deviations encountered in highly color distorted images as in poor visibility conditions (sandstorms and underwater). To alter color correction locally, we add local color fidelity and gradient constraint to the proposed technique, which overcomes the limitation that the traditional method depends too much on the global color statistics of the reference image and encourages it to handle the degraded image with various color casts and light conditions. In addition, a multiscale gradient-domain processing is applied to enhance the contrast. In this procedure, by extracting the information of different layers, we can easily restore the contrast while limiting the significant amplification of noise. The extensive qualitative and quantitative experiments reveal that the color and the contrast can be significantly improved by the proposed technique.

Image taken in turbid media generally exists poor visibility and low contrast, which results from attenuation of the propagated light. In this letter, an effective polarization-based image dehazing method is proposed, which relies on the relationship between the angle of polarization (AoP) from the Stokes vector and the scattered light. To avoid the influence of noise, AoP is optimized based on regularization constraints. The regularization function is made using an assumption that adjacent pixels with similar colors have similar values of AoP. Moreover, according to the revised AoP information, all the key parameters can be effectively and automatically estimated without considering the no-object region (or the sky region) exists or not, which relies on a frequency prior strategy. Extensive experiments on real-world images demonstrate that the proposed method is more effective than several previous image restoration or enhancement works.

This letter introduces an underwater image enhancement method to handle low contrast and color cast of underwater images. Firstly, with the help of hierarchical searching technique, we propose a novel backscattered light estimation method. And in this procedure, a novel scoring formula is considered into our method, which comprehensively considers multiple prior knowledge. Then, we generalize underwater dark channel prior (UDCP) approach to obtain more robust transmission estimation. In addition, we also develop a white balance method to further modify the appearance of the resultant image. Extensive experiments on real-world images demonstrate that the proposed method outperforms several previous image restoration or enhancement works.

In natural rainy scenes, visibility is significantly degraded by two types of phenomena: specular highlights of nearby individual rain streaks and atmospheric veiling effect caused by distant accumulated rain. However, most existing deraining methods only take the first kind of degradation into consideration, which limits their potential application in heavy rain. In this study, a joint rain and atmospheric veil removal framework is proposed to address this problem. Since rain streaks and rain accumulation are entangled with each other, which is intractable to simulate, causing clean/rainy image pairs of real‐world are hard to generate. Hence, after introducing a generalised rain model, which can represent both rain streaks and atmospheric veil physically, the authors do not learn the mapping function between image pairs using deep‐learning architecture, but estimate the rain streaks, transmission, and atmospheric light via Gaussian mixture model patch prior and dark channel prior to solve the rain model instead. According to the comprehensive experimental evaluations, the proposed method outperforms other state‐of‐the‐art methods in terms of both high visibility and vivid colour, especially in natural heavy rain scenario.