Parkinson's Disease (PD) is often characterized by abnormal gait patterns, which can be objectively and quantitatively diagnosed using Vertical Ground Reaction Force (VGRF) signals. Previous studies have demonstrated the effectiveness of deep learning in VGRF signal analysis. However, the inherent graph structure of VGRF signals has not been adequately considered, limiting the representation of dynamic gait characteristics. To address this, we propose a Multi-Scale Adaptive Directed Graph Neural Network (MS-ADGNN) approach to distinguish the gaits between Parkinson's patients and healthy controls. This method models the VGRF signal as a multi-scale directed graph, capturing the distribution relationships within the plantar sensors and the dynamic pressure conduction during walking. MS-ADGNN integrates an Adaptive Directed Graph Network (ADGN) unit and a Multi-Scale Temporal Convolutional Network (MSTCN) unit. ADGN extracts spatial features from three scales of the directed graph, effectively capturing local and global connectivity. MSTCN extracts multi-scale temporal features, capturing short to long-term dependencies. The proposed method outperforms existing methods on three widely used datasets. In cross-dataset experiments, the average improvements in terms of accuracy, F1-score, and geometric mean are 2.46$\%$, 1.25$\%$, and 1.11$\%$ respectively. Meanwhile, in 10-fold cross-validation experiments, the improvements are 0.78$\%$, 0.83$\%$, and 0.81$\%$ respectively.

Hierarchical decoders are widely employed in semantic segmentation architectures, where feature upsampling and fusion are critical at each layer. However, inaccuracies in feature position restoration during upsampling and misalignments in multi-scale feature fusion limit model performance. To address these issues, we propose the Aligned Feature Aggregation Upsampler (AFAU), a novel upsampling module. AFAU consists of two types of Aligned Feature Aggregation Modules (AFAM), which partition feature maps from both the encoder and decoder into multiple aligned patches. By applying attention mechanisms between corresponding patches, AFAM facilitates more effective contextual information fusion. Additionally, AFAM corrects feature position inaccuracies by calculating the similarity between linear embeddings of low-level (query) features and high-level features. This process allows semantic information aggregation guided by fine-grained high-resolution features. We conducted extensive experiments across multiple advanced semantic segmentation frameworks, demonstrating the universality and efficiency of AFAU. Notably, our results indicate that decoders incorporating AFAU significantly outperform SegFormer across all sizes of MiT encoders on the ADE20K and Cityscapes datasets.

The reading of arbitrarily-shaped text has received increasing research attention. However, existing text spotters are mostly built on two-stage frameworks or character-based methods, which suffer from either Non-Maximum Suppression (NMS), Region-of-Interest (RoI) operations, or character-level annotations. In this paper, to address the above problems, we propose a novel fully convolutional Point Gathering Network (PGNet) for reading arbitrarily-shaped text in real-time. The PGNet is a single-shot text spotter, where the pixel-level character classification map is learned with proposed PG-CTC loss avoiding the usage of character-level annotations. With a PG-CTC decoder, we gather high-level character classification vectors from two-dimensional space and decode them into text symbols without NMS and RoI operations involved, which guarantees high efficiency. Additionally, reasoning the relations between each character and its neighbors, a graph refinement module (GRM) is proposed to optimize the coarse recognition and improve the end-to-end performance. Experiments prove that the proposed method achieves competitive accuracy, meanwhile significantly improving the running speed. In particular, in Total-Text, it runs at 46.7 FPS, surpassing the previous spotters with a large margin.

Detecting scene text of arbitrary shapes has been a challenging task over the past years. In this paper, we propose a novel segmentation-based text detector, namely SAST, which employs a context attended multi-task learning framework based on a Fully Convolutional Network (FCN) to learn various geometric properties for the reconstruction of polygonal representation of text regions. Taking sequential characteristics of text into consideration, a Context Attention Block is introduced to capture long-range dependencies of pixel information to obtain a more reliable segmentation. In post-processing, a Point-to-Quad assignment method is proposed to cluster pixels into text instances by integrating both high-level object knowledge and low-level pixel information in a single shot. Moreover, the polygonal representation of arbitrarily-shaped text can be extracted with the proposed geometric properties much more effectively. Experiments on several benchmarks, including ICDAR2015, ICDAR2017-MLT, SCUT-CTW1500, and Total-Text, demonstrate that SAST achieves better or comparable performance in terms of accuracy. Furthermore, the proposed algorithm runs at 27.63 FPS on SCUT-CTW1500 with a Hmean of 81.0% on a single NVIDIA Titan Xp graphics card, surpassing most of the existing segmentation-based methods.

Visual attention is a dynamic process of scene exploration and information acquisition. However, existing research on attention modeling has concentrated on estimating static salient locations. In contrast, dynamic attributes presented by saccade have not been well explored in previous attention models. In this paper, we address the problem of saccadic scanpath prediction by introducing an iterative representation learning framework. Within the framework, saccade can be interpreted as an iterative process of predicting one fixation according to the current representation and updating the representation based on the gaze shift. In the predicting phase, we propose a Bayesian definition of saccade to combine the influence of perceptual residual and spatial location on the selection of fixations. In implementation, we compute the representation error of an autoencoder-based network to measure perceptual residuals of each area. Simultaneously, we integrate saccade amplitude and center-weighted mechanism to model the influence of spatial location. Based on estimating the influence of two parts, the final fixation is defined as the point with the largest posterior probability of gaze shift. In the updating phase, we update the representation pattern for the subsequent calculation by retraining the network with samples extracted around the current fixation. In the experiments, the proposed model can replicate the fundamental properties of psychophysics in visual search. In addition, it can achieve superior performance on several benchmark eye-tracking data sets.

Visual attention is a dynamic search process of acquiring information. However, most previous studies have focused on the prediction of static attended locations. Without considering the temporal relationship of fixations, these models usually cannot explain the dynamic saccadic behavior well. In this paper, an iterative representation learning framework is proposed to predict the saccadic scanpath. Within the proposed framework, saccade can be explained as an iterative process of finding the most uncertain area and updating the representation of scenes. In implementation, a deep autoencoder is employed for representation learning. The current fixation is predicted to be the most salient pixel, with saliency estimated by the reconstruction residual of the deep network. Image patches around this fixation are then sampled to update the network for the selection of subsequent fixations. Compared with existing models, the proposed model shows the state-of-the-art performance on several public data sets.

Research on visual perception indicates that the human visual system is sensitive to center-surround (C-S) contrast in the bottom-up saliency-driven attention process. Different from the traditional contrast computation of feature difference, models based on reconstruction have emerged to estimate saliency by starting from original images themselves instead of seeking for certain ad hoc features. However, in the existing reconstruction-based methods, the reconstruction parameters of each area are calculated independently without taking their global correlation into account. In this paper, inspired by the powerful feature learning and data reconstruction ability of deep autoencoders, we construct a deep C-S inference network and train it with the data sampled randomly from the entire image to obtain a unified reconstruction pattern for the current image. In this way, global competition in sampling and learning processes can be integrated into the nonlocal reconstruction and saliency estimation of each pixel, which can achieve better detection results than the models with separate consideration on local and global rarity. Moreover, by learning from the current scene, the proposed model can achieve the feature extraction and interaction simultaneously in an adaptive way, which can form a better generalization ability to handle more types of stimuli. Experimental results show that in accordance with different inputs, the network can learn distinct basic features for saliency modeling in its code layer. Furthermore, in a comprehensive evaluation on several benchmark data sets, the proposed method can outperform the existing state-of-the-art algorithms.

The center-surround comparison principle is widely used in existing bottom-up saliency estimation models. However, most of them are based on local image processing techniques which are hard to handle texture regions well as a relatively large neighborhood is required to represent textures. In this paper, we propose a nonlocal patch-based reconstruction approach to reformulate the center-surround comparison. In the proposed approach, the saliency is measured by the reconstruction residual of representing the central patch with a linear combination of its surrounding patches. As a generalization of Itti et al.'s classical center-surround comparison scheme, the proposed approach performs well on images with symmetric structures where Itti et al.'s method fails, as well as on general natural images. Numerical experiments show the proposed approach produces better results compared to the state-of-the-art algorithms on several public databases.

Depth acquisition for moving objects becomes increasingly critical for some applications such as human facial expression recognition. This paper presents a method for capturing the depth maps of moving objects that uses a one-shot black-and-white stripe pattern with the features of simplicity and easily generation. Considering the accuracy of a matching is crucial for a precise depth map but the matching of variant-width stripes is sparse and rough, the phase differences extracted by Gabor filter to achieve a pixel-wise matching with sub-pixel accuracy are used. The details of the derivation are presented to prove that this method based on the phase difference calculated by Gabor filter is valid. In addition, the periodic ambiguity of the encoded stripe is eliminated by the epipolar segment covering a given depth range at a camera-projector calibrating stage to decrease the calculation complexity. Experimental results show that our method can get a dense and accurate depth map of a moving object.

In this paper, we introduce a novel just noticeable difference (JND) estimation model based on the unified brain theory, namely the free-energy principle. The existing pixel-based JND models mainly consider the orderly factors and always underestimate the JND threshold of the disorderly region. Recent research indicates that the human visual system (HVS) actively predicts the orderly information and avoids the residual disorderly uncertainty for image perception and understanding. Thus, we suggest that there exists disorderly concealment effect which results in high JND threshold of the disorderly region. Beginning with the Bayesian inference, we deduce an autoregressive model to imitate the active prediction of the HVS. Then, we estimate the disorderly concealment effect for the novel JND model. Experimental results confirm that the proposed JND model outperforms the relevant existing ones. Furthermore, we apply the proposed JND model in image compression, and around 15% of bit rate can be reduced without jeopardizing the perceptual quality.

In this paper, we propose a novel motion estimation framework based on the sparsity associated with gradients of the parametrized motion field. Beginning with Shen and Wu’s sparse model for optic flow estimation [15], we show the sparsity of the motion field can be enhanced by increasing the degree of freedom of the parametrized motion model. With such an enhancement, we formulate the motion estimation as an ‘0 optimization problem. Along with an ‘1 norm regularization to the instant constancy assumption, this problem is solved by a reweighted ‘1 optimization approach. Experiments on constant, pure translational, and affine motion models certify that the enhanced sparsity provides improved accuracy for motion estimation.

This paper introduces a new sparse model for robust and reliable optical flow estimation. We show that the sparsity is directly related to gradient fields of optical flow. According to theory on sparse signal recovery, we rigorously formulate the optical flow estimation as an ℓ0optimization problem. Considering the piecewise continuous nature of motion, the basic optical flow constraint is regularized by an ℓ1norm. Then, with convex relaxation, the solution is obtain via the ordinary or reweighted ℓ1optimization approach. Experimental results show the proposed method performs better than traditional methods and deals well with the discontinuities on motion boundaries.

Nodes layout is a critical issue affecting the quality of service of camera networks. The layout aims at adopting the least number of cameras to cover the whole scene with expected observation quality. This is generally formulated as a non-convex optimization problem which is hard to be solved in polynomial time. In this paper, we propose an efficient near-optimal convex solution for layout guaranteeing satisfactory observation quality based on an anisotropic sensing model of camera. We construct the sensing model based on the physical imaging process. This model provides an accurate measurement of observation quality for the directional nonuniform sensing field of cameras. The layout is treated as selecting the best subset of cameras from a redundant initial layout with numerous cameras. Based on this characteristic and the sensing model, we formulate the layout as a sparse ℓ0problem which is further relaxed to a convex ℓ1minimization. Therefore, the near-optimal layout is efficiently obtained via convex optimization. Simulation results confirm the effectiveness of the proposed approaches.

A pixel-wise adaptive model for estimating the just-noticeable difference (JND) in spatial domain is proposed in this paper. As the human visual system (HVS) can be considered as a multichannel system, we assume that there exist two channels in the HVS, which deliver luminance adaption factor and texture masking factor, respectively. Both channels affect the JND threshold in a cooperative manner. The texture regions are with abundant redundancy and can tolerate much noise. The disorder degree and spatial masking of the texture are considered to estimate the texture masking effect, for deducing such JND threshold that coincides with the HVS. Finally, the luminance adaptation factor and texture masking factor are combined nonlinearly. Various experiments confirm the improved model has a better visual effect than models proposed before.

Detecting and segmenting out the regions of interest (ROIs) is one of the foundations in image processing and analysis. Because the final information sink of images is human, for segmenting out the ROIs effectively, we need to study human visual system (HVS) and imitate the behaviors when human viewing a scene. Researchers have found several factors which affect human attentions by studying eye movements when one views an image. In this paper, a method is proposed to detect the ROIs automatically based on HVS. In the proposed algorithm, the properties of pixels such as the contrast, location and edges are analyzed, and the pixels are enhanced according to the sensitivity of HVS. Then these factors are combined to a salient map, which classifies each pixel of the image in relation to its perceptual importance. Finally, the ROIs are segmented according to the salient map. This algorithm is easy to work, and can segment the objects from complex background efficiently.

In this paper, an approach for object tracking is proposed based on the online color information fusion scheme. The fusion scheme, is performed by projecting multi-channel color images to a one-dimensional pseudo gray scale space. This dimensionality reduction simplifies the designation of the tracking algorithm. The fusing coefficients are determined by taking the Fisher linear discriminant analysis to maximize the discriminative capability after taking the projection. The robustness of the approach lies in exploiting the appearance discrimination of the object in cluttered scenarios with varying illumination conditions. This scheme is embedded into a mean-shift tracking system and the experimental results show our scheme can enhance the discriminant characters in the changing environment and hence producing robust tracking results.

In this paper, we address a novel automatic distorted grid recognition algorithm and its application to identify spots grid in microarray images. The proposed method contains two main steps. The first step is a top-down procedure which retrieves the global information, horizontal spacing, vertical spacing and rotation angle, of grid by utilizing Gaussian mixture model and document spectrum. The second step, which is bottom-up, rebuilds the grid structure with the aid of the data obtained in the first step. The estimation of global information is precise and the reconstruction progress is effective. The accuracy and efficiency of the approach are validated by applying it to synthesis images, real distorted grid and microarray images. Furthermore, our approach is automatic and insensitive to the absence of grid elements.