Federated learning (FL) enables multiple clients to collaboratively train machine learning (ML) models without exposing their private data. The recent surge in poisoning attacks and privacy leakage against FL has driven the development of secure federated learning (SFL) solutions. However, existing SFL schemes show inadequate performance when confronted with non-independent and identically distributed ( non-IID) data. In addition, traditional SFL architectures are prone to single point of failure (SPOF) issues due to the heavy computational and communication burdens imposed on a single server. In response to these issues, a novel blockchain-based secure federated learning (BSFL) framework is proposed in this paper. Specifically, we devise a proof of verification (PoV) consensus protocol to identify poisoning attacks under non-IID situations, while preventing the waste of computational and communication resources. Subsequently, we present a personalized differential privacy (PDP) mechanism, which achieves comprehensive privacy protection with lower noise levels. Furthermore, the integration of the blockchain with the proposed reward mechanism overcomes SPOF and fosters constructive participation through transparent processes. Formal theoretical analysis demonstrates the security, privacy, and efficiency of our framework. Extensive experimental evaluations indicate that BSFL exhibits strong resilience against various poisoning attacks and achieves better model accuracy compared to existing SFL solutions.

Generic deep learning (DL) networks for image restoration like denoising and interpolation lack mathematical interpretability, require voluminous training data to tune large parameter sets, and are fragile in the face of covariate shift. To address these shortcomings, we build interpretable networks by unrolling variants of a graph-based optimization algorithm of different complexities. Specifically, for a general linear image formation model, we first formulate a convex quadratic programming (QP) problem with a new $\ell _{2}$ -norm graph smoothness prior called gradient graph Laplacian regularizer (GGLR) that promotes piecewise planar (PWP) signal reconstruction. To solve the posed unconstrained QP problem, instead of computing a linear system solution straightforwardly, we introduce a variable number of auxiliary variables and correspondingly design a family of ADMM algorithms. We then unroll them into variable-complexity feedforward networks, amenable to parameter tuning via back-propagation. More complex unrolled networks require more labeled data to train more parameters, but have better overall performance. The unrolled networks have periodic insertions of a graph learning module, akin to a self-attention mechanism in a transformer architecture, to learn pairwise similarity structure inherent in data. Experimental results show that our unrolled networks perform competitively to generic DL networks in image restoration quality while using only a fraction of parameters, and demonstrate improved robustness to covariate shift.

Lightweight super-resolution (SR) has garnered attention for balancing performance and efficiency in resource-constrained environments. In lightweight SR tasks, traditional CNN-based methods are constrained by limited receptive fields, leading to suboptimal SR performance. In contrast, ViT-based models achieve remarkable results but suffer from significant computational burden due to the self-attention mechanism. In this paper, we propose adaptive Pixel Classification and equivalent Large Kernels Network (PCLKN), a novel lightweight SR model that addresses the limitations of traditional CNN-based and ViT-based methods. PCLKN utilizes equivalent large kernels to expand the receptive field while relying solely on convolutional operations, significantly reducing computational overhead. Additionally, it integrates global priors with spatial and channel attention to enhance feature extraction and leverages adaptive pixel classification to utilize similar pixel information for reconstruction. Experimental results on benchmark datasets demonstrate that PCLKN achieves superior SR performance with an excellent trade-off between performance and computational complexity.

Image manipulation has sparked widespread concern due to its potential security threats on the Internet. The boundary between the authentic and manipulated region exhibits artifacts in image manipulation localization (IML). These artifacts are more pronounced in heterogeneous image splicing and homogeneous image copy-move manipulation, while they are more subtle in removal and inpainting manipulated images. However, existing methods for image manipulation detection tend to capture boundary artifacts via explicit edge features and have limitations in effectively addressing subtle artifacts. Besides, feature redundancy caused by the powerful feature extraction capability of large models may prevent accurate identification of manipulated artifacts, exhibiting a high false-positive rate. To solve these problems, we propose a novel edge-aware network (EAN) to capture boundary artifacts effectively. This network treats the image manipulation localization problem as a segmentation problem inside and outside the boundary. In EAN, we develop an edge-aware mechanism to refine implicit and explicit edge features by the interaction of adjacent features. This approach directs the encoder to prioritize the desired edge information. Also, we design a multi-feature fusion strategy combined with an improved attention mechanism to enhance key feature representation significantly for mitigating the effects of feature redundancy. We perform thorough experiments on diverse datasets, and the outcomes confirm the efficacy of the suggested approach, surpassing leading manipulation localization techniques in the majority of scenarios.

Since parathyroid glands (PG) regulate the body’s calcium levels and significantly impact human health, developing methods for their automatic detection during endoscopic thyroid surgery is of utmost clinical significance. However, existing parathyroid detection works suffer from color variations, target deformation, blur, and lighting effects in intricate surgical environments. To address the above shortcomings, in this paper, we propose a novel double-layer graph attention network for PG detection, which explicitly facilitates local augmentation via key visual features (e.g., texture and shape) identification and global interactions. It can robustly combat image blur and better differentiate the PG targets and background parts, thus improving the detection precision. Furthermore, we observe most prior works fail to deeply understand the spatial relation among targets and unavoidably suffer from false or missed detection, which is heavily due to total ignorance or insufficient utilization of depth information, especially under lighting variations and occlusions. To fill the gap, we propose a depth relation augmentation component to adaptively capture the prominent relative positional relations between targets based on depth information and incorporate it into the proposed GNN framework, significantly deepening spatial understandings and naturally enhancing generalizability. Due to lacking a thyroid endoscopy surgery benchmark for evaluating this task, we meticulously established a novel dataset from 838 actual surgeries conducted (via the fully laparoscopic thoracic-breast approach) at the Fujian Medical University Union Hospital. Extensive experiments show that our framework achieves superior PG detection accuracy compared to current state-of-the-art counterparts while keeping real-time efficiency.

We revisit the well-studied image segmentation problem from a soft labeling perspective: instead of estimating integer labels per pixel indicating a finite set of classes, each pixel is assigned a real number that conveys the level of uncertainty in the estimated class label. Soft labels are useful, for example, for subsequent human editing or composition. Specifically, given a set of pre-computed super-pixel labels and feature vectors per pixel, we formulate a convex optimization objective regularized by signal-dependent gradient graph Laplacian regularizers (GGLR), which promotes piecewise planar (PWP) signal reconstruction. Unlike a previous well-known soft segmentation scheme that requires expensive computation of the first 100 eigenvectors, our optimization can be solved efficiently in linear time via conjugate gradient (CG). Experimental results show that our method produces satisfactory soft labels per pixel for images in two public datasets at a reduced computation cost compared to the previous soft segmentation scheme.

Obtaining pixel-accurate and topologically complete retinal vessel segmentation is challenging due to many factors, for instance, vascular structure complexity, image contrast variations, and limitations of valuable datasets. In this paper, we introduce a novel network structure that applies dual-branch: directional reweighted branch and skeletonized branch. In the direction reweighted branch, we propose adaptive directional enhancement and connectivity consistency enhancement, which can be used to extract favorable directional channel information and model the bidirectional relationship between pixels, respectively. In the skeletonized branch, we employ morphological skeletonization to align the ground truth with the predicted segmentation map. By doing that, we effectively preserve the vessel’s topological structure from a global perspective. Extensive experiments on publicly available retinal datasets DRIVE, CHASE_DB1, and STARE show that our proposed approach has achieved significant results in preserving vessel structure and accurate segmentation.

The health-related Internet of Things (IoT) play an irreplaceable role in the collection, analysis, and transmission of medical data. As a device of the health-related IoT, the electroencephalogram (EEG) has long been a powerful tool for physiological and clinical brain research, which contains a wealth of personal information. Due to its rich computational/storage resources, cloud computing is a promising solution to extract the sophisticated feature of massive EEG signals in the age of big data. However, it needs to solve both response latency and privacy leakage. To reduce latency between users and servers while ensuring data privacy, we propose a privacy-preserving feature extraction scheme, called LightPyFE, for EEG signals in the edge computing environment. In this scheme, we design an outsourced computing toolkit, which allows the users to achieve a series of secure integer and floating-point computing operations. During the implementation, LightPyFE can ensure that the users just perform the encryption and decryption operations, where all computing tasks are outsourced to edge servers for specific processing. Theoretical analysis and experimental results have demonstrated that our scheme can successfully achieve privacy-preserving feature extraction for EEG signals, and is practical yet effective.

Viral information like rumors or fake news is spread over a communication network like a virus infection in a unidirectional manner: entity$i$conveys information to a neighbor$j$, resulting in two equally informed (infected) parties. Existing graph diffusion processes focus only on bidirectional diffusion on an undirected graph. Instead, leveraging recent research in graph signal processing (GSP), we propose a new directed acyclic graph (DAG) diffusion process to estimate the probability$x_{i}(t)$of node$i$'s infection at time$t$given an initial infected source node$s$, where$x_{i}(\infty)=1$. Specifically, given an undirected positive graph modeling node-to-node communication, we first estimate its graph embedding: a latent coordinate for each graph node in an assumed low-dimensional manifold space via extreme eigenvectors computed using LOBPCG. Next, we construct a DAG based on Euclidean distances between latent coordinates. Spectrally, we prove that the asymmetric DAG Laplacian matrix contains real non-negative eigenvalues, and that the DAG diffusion converges to the all-infection vector$\mathbf{x}(\infty)=1$as$t\rightarrow\infty$. Simulations show that our DAG diffusion process accurately estimates the probabilities of node infection over a variety of graph structures at different time instants.

Due to the technical limitation of pen tablets, there are sensing points data loss from the touch screen when the handwriting speed is fast. This problem will cause discrete, segmented, and unsmooth handwriting curves. In order to recover the unknown point coordinates from the observed corrupted curve of handwriting, we propose a curve interpolation algorithm by combining gradient graph Laplacian regularizer and cyclic shift. We first define the gradient of 2D curve and create the related gradient graph. Then the handwriting curve is interpolated by the gradient graph Laplacian regularizer. For handwriting stroke offset, we introduce a cyclic shift of handwriting for translation invariance. Experimental results on synthetic curves and handwriting datasets show that the interpolation quality of our proposed algorithm is better than other competing algorithms, and it promotes the curve smoothness of the turning points.

In the graph signal processing (GSP) literature, it has been shown that signal-dependent graph Laplacian regularizer (GLR) can efficiently promote piecewise constant (PWC) signal reconstruction for various image restoration tasks. However, for planar image patches, like total variation (TV), GLR may suffer from the well-known “staircase” effect. To remedy this problem, we generalize GLR to gradient graph Laplacian regularizer (GGLR) that provably promotes piecewise planar (PWP) signal reconstruction for the image interpolation problem—a 2D grid with random missing pixels that requires completion. Specifically, we first construct two higher-order gradient graphs to connect local horizontal and vertical gradients. Each local gradient is estimated using structure tensor, which is robust using known pixels in a small neighborhood, mitigating the problem of larger noise variance when computing gradient of gradients. Moreover, unlike total generalized variation (TGV), GGLR retains the quadratic form of GLR, leading to an unconstrained quadratic programming (QP) problem per iteration that can be solved quickly using conjugate gradient (CG). We derive the means-square-error minimizing weight parameter for GGLR, trading off bias and variance of the signal estimate. Experiments show that GGLR outperformed competing schemes in interpolation quality for severely damaged images at a reduced complexity.

Despite the success of pedestrian detection, there is still a significant gap in the performance of the detection of pedestrians at different scales. Detecting small-scale pedestrians is extremely challenging due to the low resolution of their convolution features which is essential for downstream classifiers. To address this issue, we observed pedestrian datasets and found that pedestrians often gather together in crowded public places. Then we propose MagnifierNet, a simple but effective small-scale pedestrian detector towards multiple dense regions. MagnifierNet uses our proposed sweep-line based grouping algorithm to find dense regions based on the number of pedestrians in the grouped region. And we adopt a new definition of small-scale pedestrians through grid search and KL-divergence. Besides, our grouping method can also be used as a new strategy for pedestrian data augmentation. The ablation study demonstrates that MagnifierNet improves the representation of small-scale pedestrians. We validate the effectiveness of MagnifierNet on CityPersons and KITTI datasets. Experimental results show that MagnifierNet achieves the best small-scale pedestrian detection performance on CityPersons benchmark without any external data, and also achieves competitive performance for detecting small-scale pedestrians on KITTI dataset without bells and whistles.

Image cosegmentation is a newly emerging research area in image processing. It refers to the problem of segmenting the common objects simultaneously in multiple images by utilizing the similarity of foreground regions among these images. In this paper, a new active contour model is proposed by using shape-similarity and foreground discovery scheme. The foreground discovery scheme is used to obtain the rough contours of the common objects which are used as initial evolution curves. The energy function of the proposed model includes two parts: an intra-image energy and an inter-image energy. The intra-image energy explores the differences between foreground regions and background regions in each image. And the inter-image energy is used to explore the similarities of the common objects among target images, which composes of a region color feature energy term and a shape constraint energy term. The region color feature term indicates the foreground consistency and the background consistency among the images; and the shape constraint energy term allows the global changes of shapes and truncates the local variation caused by misleading features. Experimental results show that the proposed model can improve the accuracy of the image cosegmentation significantly through regularizing the changes of shapes.

Natural image modeling plays a key role in many vision problems such as image denoising. Image priors are widely used to regularize the denoising process, which is an ill-posed inverse problem. One category of denoising methods exploit the priors (e.g., TV, sparsity) learned from external clean images to reconstruct the given noisy image, while another category of methods exploit the internal prior (e.g., self-similarity) to reconstruct the latent image. Though the internal prior based methods have achieved impressive denoising results, the improvement of visual quality will become very difficult with the increase of noise level. In this paper, we propose to exploit image external patch prior and internal self-similarity prior jointly, and develop an external patch prior guided internal clustering algorithm for image denoising. It is known that natural image patches form multiple subspaces. By utilizing Gaussian mixture models (GMMs) learning, image similar patches can be clustered and the subspaces can be learned. The learned GMMs from clean images are then used to guide the clustering of noisy-patches of the input noisy images, followed by a low-rank approximation process to estimate the latent subspace for image recovery. Numerical experiments show that the proposed method outperforms many state-of-the-art denoising algorithms such as BM3D and WNNM.

In this paper we introduce a new shape-driven approach for object segmentation. Given a training set of shapes, we first use deep Boltzmann machine to learn the hierarchical architecture of shape priors. This learned hierarchical architecture is then used to model shape variations of global and local structures in an energetic form. Finally, it is applied to data-driven variational methods to perform object extraction of corrupted data based on shape probabilistic representation. Experiments demonstrate that our model can be applied to dataset of arbitrary prior shapes, and can cope with image noise and clutter, as well as partial occlusions.

In this paper, a novel variational model based on prior shapes for simultaneous object classification and segmentation is proposed. Given a set of training shapes of multiple object classes, a sparse linear combination of training shapes in a low-dimensional representation is used to regularize the target shape in variational image segmentation. By minimizing the proposed variational functional, the model is able to automatically select the reference shapes that best represent the object by sparse recovery and accurately segment the image, taking into account both the image information and the shape priors. For some applications under an appropriate size of training set, the proposed model allows artificial enlargement of the training set by including a certain number of transformed shapes for transformation invariance, and then the model remains jointly convex and can handle the case of overlapping or multiple objects presented in an image within a small range. Numerical experiments show promising results and the potential of the method for object classification and segmentation.

In this paper, we propose a novel model for simultaneous image segmentation and object recognition. Our model is different from previous prior-based level set variatioinal image segmentation in two aspects. The first is the use of the shape sparse representation, which is able to integrate shape priors by linear combination into variational image segmentation. The second is that segmentation and recognition procedures are carried out automatically. The sparsest solution will determine the identity of the target. In addition, our model can handle more general shape priors. Numerical experiments show promising results on synthetic and real images.

This paper presents a modified patchwork watermarking scheme for digital images by incorporating the Watson's perceptual model into the watermarking process. Watermarking occurs in the DCT domain. Watson's perceptual model provides a measure of distortions that each DCT coefficient can resist based on the human visual system (HVS). Perceptual information is incorporated into the watermarking scheme by minimizing the Watson's distance between the host image and watermarked image using quadratic programming. Compared with patchwork schemes which do not consider perceptual models, experiments indicate that the proposed method has increased robustness and fidelity of the watermarking system.