The goal of gaze object prediction (GOP) is to predict human gaze objects and categories. However, existing methods require additional head priors or filter the results before evaluation, which is an obstacle for real-world applications. To this end, this paper proposes aTransformer-basedGazeObjectPrediction underReal-world setting (TransGOP-R), which does not rely on any head prior input and evaluates end-to-end. We first design a head location module to generate human head location information from a head query. Then, an error analysis demonstrates that the primary error source of the existing GOP model is in gaze estimation, which is caused by the difficulty in predicting gaze points by directly regressing heatmaps. Therefore, we introduce cone prediction into the model training stage, allowing the middle-layer features of the gaze regressor to build the relationship between the target human and objects before regressing the gaze point. An oriented gradient mechanism is proposed in this process to ensure the object detection performance is not affected by cone information. Finally, we conducted very detailed and sufficient experiments to verify the superiority of our method on the GOO-Synth and GOO-Real datasets. At the same time, we also achieve advantages compared to the human-target gaze estimation methods on the GazeFollowing, VideoAttentionTarget, and ChildPlay datasets.

This paper presents Core, a conceptually simple, effective and communication-efficient model for multi-agent cooperative perception. It addresses the task from a novel perspective of cooperative reconstruction, based on two key insights: 1) cooperating agents together provide a more holistic observation of the environment, and 2) the holistic observation can serve as valuable supervision to explicitly guide the model learning how to reconstruct the ideal observation based on collaboration. Core instantiates the idea with three major components: a compressor for each agent to create more compact feature representation for efficient broadcasting, a lightweight attentive collaboration component for cross-agent message aggregation, and a reconstruction module to reconstruct the observation based on aggregated feature representations. This learning-to-reconstruct idea is task-agnostic, and offers clear and reasonable supervision to inspire more effective collaboration, eventually promoting perception tasks. We validate Core on two large-scale multi-agent percetion dataset, OPV2V and V2X-Sim, in two tasks, i.e., 3D object detection and semantic segmentation. Results demonstrate that Core achieves state-of-the-art performance, and is more communication-efficient.

Reflection symmetry detection receives increasing attentions in recent years. The state-of-the-art algorithms mainly use the matching of intensity-based features (such as the SIFT) within a single image to find symmetry axes. This paper proposes a novel approach by establishing the correspondence of locally affine invariant edge-based features, which are superior to the intensity based in the aspects that it is insensitive to illumination variations, and applicable to textureless objects. The locally affine invariance is achieved by simple linear algebra for efficient and robust computations, making the algorithm suitable for detections under object distortions like perspective projection. Commonly used edge detectors and a voting process are, respectively, used before and after the edge description and matching steps to form a complete reflection detection pipeline. Experiments are performed using synthetic and real-world images with both multiple and single reflection symmetry axis. The test results are compared with existing algorithms to validate the proposed method.

We propose robust affine-invariant contour descriptor and measure for shape matching under nonlinear deformations. The descriptor is formed by orthonormal configuration matrix of local contour. The geodesic distance on Grassmann manifold is used to measure similarities of shapes under locally affine transformations, which can approximate complex deformations like articulations. A rigorous perturbation analysis proves that condition numbers of configuration matrices are critical for robustness. Then a method to improve matching stability using the condition numbers is deduced. Commonly used contour matchers, e.g., dynamical programming and others, are all applicable to the descriptor to obtain satisfied matching. Experimental evaluations are given using both synthetic and real-world images.

This paper proposes a novel application of image inpainting techniques for the edge enhancement problems in image deblurring and denoising. The edge enhancement effect is achieved by the jumps of pixel values at edge locations resulted from an inpainting process. The process is formulated by the eikonal partial differential equation (PDE) to rule the inpainting priority of pixels in automatically erased regions. The equation is then numerically solved by the fast marching method. A solution of the Laplace's equation is also embedded in the numerical scheme to assure the smoothness in non-edge locations. Experiments are performed and comparisons are made with other PDE-based enhancement methods to demonstrate the merit of the proposed algorithm.

This work proposes novel locally affine invariant descriptors for shape representation. The descriptors are theoretically simple and solid, derived from the matrix theories. They can be used for matching and retrieval of shapes under affine transformation, articulated motion or nonrigid deformation. Comparisons of the work with the state-of-the-art shape descriptors are performed based on synthetic and some well-known databases. The experiments validate that the proposed descriptors achieve higher retrieval accuracy and have faster running speed than most of other approaches.

This paper proposes an affine invariant matching algorithm for shape correspondence problems in arbitrary dimensions. Formulating shapes by configuration matrices of landmarks, and using the fact that subspaces (e.g. range spaces) of these matrices are invariant to affine transformations, the shape correspondence is modelled as a permutation relation between orthogonal projection matrices of the subspaces. Then the matching result is solved by an efficient factorization procedure for rank-deficient matrices. The algorithm is compact, fast, and independent of dimensions. Experimental results for 1D, 2D and 3D matchings of synthetic and real data are provided, which demonstrate potential applications of the algorithm to shape analysis, and to other related problems like wide baseline stereo matching and range data registration.

A new algorithm to preserve edges in image magnification is proposed. The algorithm is based on an inpainting technique: Blurry edges in the initially magnified image are erased using automatic edge detections, then the erased regions are filled with the filling priority guided by the watershed concept. The algorithm results in sharp patterns along the edges of the initial image, and the discontinuity across the edges is guaranteed. Experiments demonstrate that the algorithm is superior to some popular methods such as edge-directed interpolations and anisotropic diffusions.

Most existing methods for inpainting structure images fail in handling thick unknown regions, resulting in over-smoothing effects in the filled areas. In this paper, a novel inpainting algorithm, termed as isophote propagation, is presented to solve the problem. The algorithm propagates isophote information while maintaining the continuity of isophotes directions. It is simple and fast, and performs better in preserving sharp edges than existing algorithms. Effectiveness of the algorithm is demonstrated in experiments.

A two-stage face recognition method is presented in this paper. In the first stage, the set of candidate patterns is narrowed down with the global similarity being taken into account. In the second stage, synergetic approach is employed to perform further recognition. Face image is segmented into meaningful regions, each of which is represented as a prototype vector. The similarity between a given region of the test pattern and a stored sample is obtained as the order parameter which serves as an element of the order vector. Finally, a modified definition of the potential function is given, and the dynamic model of recognition is derived from it. The effectiveness of the proposed method is experimentally confirmed.

Some models for super-resolution restoration assume that low-resolution (LR) images are formed from high-resolution (HR) ones by a warping first, blurring second (followed by decimating) imaging process; whereas others adopt a blurring first, warping second imaging constraint. These models are abused to some extent and respective conditions for usability of them are not identified. Such ambiguities are analyzed and cleared up in this letter. Conclusions are: The warping-blurring model coincides with the general imaging physics, but it is usable only if the motion among HR images is known a priori. When the motion is estimated from LR images, this model may cause systematic error, but the use of the blurring-warping model is more appropriate, and leads to better performance.

To solve the problem of super-resolution restoration that algorithm performances are strongly affected by model uncertainties, e.g. inaccurate motion estimations, a new super-resolution algorithm is presented. The video restoration problem is modeled as a time-varying discrete linear dynamical system, and the model uncertainties are considered as perturbations to the system. The algorithm to solve this model is based on a modified Kalman filter, and perturbations are estimated from the motion-compensated prediction residual (MCPR). Experiments for real world video were taken to show the robustness of this algorithm to model uncertainties.