Appearance-based supervised methods with full-face image input have made tremendous advances in recent gaze estimation tasks. However, intensive human annotation requirement inhibits current methods from achieving industrial level accuracy and robustness. Although methods based on generative AI can synthesize eye images to expand self-annotated eye data, these methods usually have limited model capacity and cannot effectively inject gaze information, resulting in poor quality, monotonous texture, and inaccurate gaze direction of the generated eye images. To alleviate the above challenge, we propose a novel gaze data synthesizer framework, in which a 3D-eye model that can flexibly manipulate the gaze direction is used to finely control eye image synthesis based on a stable diffusion large generative model, so that high-quality eye images with arbitrary gaze angle can be synthesized. At the same time, when training the gaze feature extractor, we propose a cross-domain feature alignment module to minimize the feature distribution discrepancy between real samples and synthetic ones, to pursue domain-invariant (shape, texture, etc.) gaze representation. Both qualitative and quantitative experimental results demonstrate that the proposed scheme generates high-quality gaze images and also achieves superior gaze estimation performances over state-of-the-art.

Efficient and highly accurate lightweight gaze estimation method has been receiving increasing research attention due to the emergence of mobile interactive platforms such as mobile device and AR/VR. State-of-the-art deep learning based gaze estimation models suffer from either heavy computational architecture which is infeasible for mobile deployment or limited generalization capability which cannot deal with large diversity in eye texture or distinguish subtle/frequent pupil movement. To mitigate the above challenges, we propose a novel lightweight network structure featuring a deformable approximate large kernel which can effectively extend the receptive field to handle complicated eye movement and highly varying eye/gaze region appearance with very tight computational budget. In the meantime, we embed the training of the gaze estimator into a control information extraction module, which serves as a gaze-parameter input that modularizes a large generative model (Stable Diffusion V1.5) to output gaze-specific eye images. In this way, the great generalization capability of large generative model could be implicitly distilled/pursued into our lightweight gaze model. Extensive comparisons with various state-of-the-art gaze estimation methods demonstrate the superiority of our proposed model and training scheme in terms of both accuracy and model complexity.