Generative AI offers transformative potential for high-stakes domains such as healthcare and finance, yet privacy and regulatory barriers hinder the use of real-world data. To address this, differentially private synthetic data generation has emerged as a promising alternative. In this work, we introduce a unified benchmark to systematically evaluate the utility and fidelity of text datasets generated under formal Differential Privacy (DP) guarantees. Our benchmark addresses key challenges in domain-specific benchmarking, including choice of representative data and realistic privacy budgets, accounting for pre-training and a variety of evaluation metrics. We assess state-of-the-art privacy-preserving generation methods across five domain-specific datasets, revealing significant utility and fidelity degradation compared to real data, especially under strict privacy constraints. These findings underscore the limitations of current approaches, outline the need for advanced privacy-preserving data sharing methods and set a precedent regarding their evaluation in realistic scenarios.

Transformer-Based models have turned out to be very successful in many artificial intelligence (AI) tasks, outperforming traditional convolutional neural networks (CNNs), especially in the field of Natural Language Processing (NLP). Their success relies upon a self-attention mechanism which, when compared to CNNs, has a global rather than a local receptive domain. This article proposes an energy-efficient edge-device Transformer inference processor termed Smart Transformer Processing Element (STPE). Firstly, STPE sets up a Multi-Mode Indexing and Sparsity Scheme (MISS) for token association, and further reduces the computational load through in-situ computation; secondly, STPE exploits the Local Properties of Attention Mechanism (LPAM) to further reduce redundant and repetitive calculations in Transformer operations by means of a search band calculation and error correction mechanism; thirdly, STPE has designed a Quantization and Compression Parallel Method (QCPM) to improve the computing speed and hardware utilization under weak related (WR) token. Employing 28nm CMOS synthesis tools, the area of the proposed STPE processor is 7.33 mm2. Its peak energy efficiency is 84.15TOPS/W, which is 14.7 times higher than that of the H100 graphics processing unit (GPU) and 3.06 times higher than that of the most advanced Transformer processor.

This work presents an efficient decoding pipeline for neuromorphic implantable brain-machine interfaces (Neu-iBMI), leveraging sparse neural event data from an event-based neural sensing scheme. We introduce a tunable event filter (EvFilter), which also functions as a spike detector (EvFilter-SPD), significantly reducing the number of events processed for decoding by 192× and 554×, respectively. The proposed pipeline achieves high decoding performance, up to R2= 0.73, with ANN- and SNN-based decoders, eliminating the need for signal recovery, spike detection, or sorting, commonly performed in conventional iBMI systems. The SNN-Decoder reduces computations and memory required by 5 − 23× compared to NN-, and LSTM-Decoders, while the ST-NN-Decoder delivers similar performance to an LSTM-Decoder requiring 2.5× fewer resources. This streamlined approach significantly reduces computational and memory demands, making it ideal for low-power, on-implant, or wearable iBMIs.

Recently, Graph Neural Networks (GNNs) have shown great potential in terms of accuracy for problems that are well-described by graph representations, such as problems of path planning. However, implementing GNNs on mobile platforms is challenging as it requires a significant amount of computation and large memory. This article proposes a High-Performance Edge GNN Inference Processor termed GPE (GNN Processing Element). Firstly, GPE sets up Multi-Dimensional Indexing and Dynamic Pruning Schemes (MIDPS) for GNN networks, and achieves cross layer interconnection of multiple neighboring nodes via NOC (Network on Chip); secondly, GPE utilizes Graph Structure Adjacency Table Information (GSATI) of a GNN to further reduce redundant and repetitive calculations by means of repeated matching and difference transfer mechanisms; thirdly, GPE has a graph-based Multi Parallelism Simplification and Operation Method (MPSOM) to improve computing speed and hardware utilization under small data volumes. Using 28nm CMOS synthesis tools, the area of the proposed GPE processor is 5.37 square millimeters. Its peak energy efficiency is 21.5TOPS/W, which is 3.76 times higher than that of the H100 GPU (Graphics Processing Unit), while the energy consumption of GNN is 80.9% lower than the previous SOTA (State of Art) work.

The functional analysis of the left atrium (LA) is important for evaluating cardiac health and understanding diseases like atrial fibrillation. Cine MRI is ideally placed for the detailed 3D characterization of LA motion and deformation but is lacking appropriate acquisition and analysis tools. Here, we propose tools for the Analysis of Left Atrial Displacements and DeformatIons using online learning neural Networks (Aladdin) and present a technical feasibility study on how Aladdin can characterize 3D LA function globally and regionally. Aladdin includes an online segmentation and image registration network, and a strain calculation pipeline tailored to the LA. We create maps of LA Displacement Vector Field (DVF) magnitude and LA principal strain values from images of 10 healthy volunteers and 8 patients with cardiovascular disease (CVD), of which 2 had large left ventricular ejection fraction (LVEF) impairment. We additionally create an atlas of these biomarkers using the data from the healthy volunteers. Results showed that Aladdin can accurately track the LA wall across the cardiac cycle and characterize its motion and deformation. Global LA function markers assessed with Aladdin agree well with estimates from 2D Cine MRI. A more marked active contraction phase was observed in the healthy cohort, while the CVD $\text {LVEF}_{\downarrow } $ group showed overall reduced LA function. Aladdin is uniquely able to identify LA regions with abnormal deformation metrics that may indicate focal pathology. We expect Aladdin to have important clinical applications as it can non-invasively characterize atrial pathophysiology. All source code and data are available at: https://github.com/cgalaz01/aladdin_cmr_la.

The use of deep neural networks for modelling system dynamics is increasingly popular, but long-term prediction accuracy and out-of-distribution generalization still present challenges. In this study, we address these challenges by considering the parameters of dynamical systems as factors of variation of the data and leverage their ground-truth values to disentangle the representations learned by generative models. Our experimental results in phase-space and observation-space dynamics, demonstrate the effectiveness of latent-space supervision in producing disentangled representations, leading to improved long-term prediction accuracy and out-of-distribution robustness.

Deep reinforcement learning (DRL) has the potential to train robots to perform complex tasks in the real world without requiring accurate models of the robot or its environment. However, agents trained with these algorithms typically lack the explainability of more traditional control methods. In this work, we use a combination of out-of-distribution generalisation tests and post hoc interpretability methods in order to understand what strategies DRL-trained agents use to perform a reaching task. To do so, we train agents under different conditions, using comparison to better interpret both quantitative and qualitative results; this allows us to not only provide local explanations, but also broad categorisations of behaviour. A key aim of our work is to understand how agents trained with visual domain randomisation (DR)—a technique which allows agents to generalise from simulation-based-training to the real world—differ from agents trained without. Our results show that the primary outcome of DR is more robust, entangled representations, accompanied by greater spatial structure in convolutional filters. Furthermore, even with an improved saliency method introduced in this work, we show that qualitative studies may not always correspond with quantitative measures, necessitating the combination of inspection tools in order to provide sufficient insights into the behaviour of trained agents. We conclude with recommendations for applying interpretability methods to DRL agents.

Although atrial fibrillation (AF) is the most common sustained atrial arrhythmia, treatment success for this condition remains suboptimal. Information from magnetic resonance imaging (MRI) has the potential to improve treatment efficacy, but there are currently few automatic tools for the segmentation of the atria in MR images. In the study, we propose a LA-Net, a multi-task network optimised to simultaneously generate left atrial segmentation and edge masks from MRI. LA-Net includes cross attention modules (CAMs) and enhanced decoder modules (EDMs) to purposefully select the most meaningful edge information for segmentation and smoothly incorporate it into segmentation masks at multiple-scales. We evaluate the performance of LA-Net on two MR sequences: late gadolinium enhanced (LGE) atrial MRI and atrial short axis balanced steady state free precession (bSSFP) MRI. LA-Net gives Hausdorff distances of 12.43 mm and Dice scores of 0.92 on the LGE (STACOM 2018) dataset and Hausdorff distances of 17.41 mm and Dice scores of 0.90 on the bSSFP (in-house) dataset without any post-processing, surpassing previously proposed segmentation networks, including U-Net and SEGANet. Our method allows automatic extraction of information about the LA from MR images, which can play an important role in the management of AF patients.

Hindsight experience replay (HER) is a goal relabelling technique typically used with off-policy deep reinforcement learning algorithms to solve goal-oriented tasks; it is well suited to robotic manipulation tasks that deliver only sparse rewards. In HER, both trajectories and transitions are sampled uniformly for training. However, not all of the agent's experiences contribute equally to training, and so naive uniform sampling may lead to inefficient learning. In this paper, we propose diversity-based trajectory and goal selection with HER (DTGSH). Firstly, trajectories are sampled according to the diversity of the goal states as modelled by determinantal point processes (DPPs). Secondly, transitions with diverse goal states are selected from the trajectories by using k-DPPs. We evaluate DTGSH on five challenging robotic manipulation tasks in simulated robot environments, where we show that our method can learn more quickly and reach higher performance than other state-of-the-art approaches on all tasks.

Unsupervised learning is of growing interest because it unlocks the potential held in vast amounts of unlabeled data to learn useful representations for inference. Autoencoders, a form of generative model, may be trained by learning to reconstruct unlabeled input data from a latent representation space. More robust representations may be produced by an autoencoder if it learns to recover clean input samples from corrupted ones. Representations may be further improved by introducing regularization during training to shape the distribution of the encoded data in the latent space. We suggest denoising adversarial autoencoders (AAEs), which combine denoising and regularization, shaping the distribution of latent space using adversarial training. We introduce a novel analysis that shows how denoising may be incorporated into the training and sampling of AAEs. Experiments are performed to assess the contributions that denoising makes to the learning of representations for classification and sample synthesis. Our results suggest that autoencoders trained using a denoising criterion achieve higher classification performance and can synthesize samples that are more consistent with the input data than those trained without a corruption process.

Generative adversarial networks (GANs) learn a deep generative model that is able to synthesize novel, high-dimensional data samples. New data samples are synthesized by passing latent samples, drawn from a chosen prior distribution, through the generative model. Once trained, the latent space exhibits interesting properties that may be useful for downstream tasks such as classification or retrieval. Unfortunately, GANs do not offer an ``inverse model,'' a mapping from data space back to latent space, making it difficult to infer a latent representation for a given data sample. In this paper, we introduce a technique, inversion, to project data samples, specifically images, to the latent space using a pretrained GAN. Using our proposed inversion technique, we are able to identify which attributes of a data set a trained GAN is able to model and quantify GAN performance, based on a reconstruction loss. We demonstrate how our proposed inversion technique may be used to quantitatively compare the performance of various GAN models trained on three image data sets. We provide codes for all of our experiments in the website (https://github.com/ToniCreswell/InvertingGAN).

The authors propose a novel deep learning model for classifying medical images in the setting where there is a large amount of unlabelled medical data available, but the amount of labelled data is limited. They consider the specific case of classifying skin lesions as either benign or malignant. In this setting, the authors’ proposed approach – the semi‐supervised, denoising adversarial autoencoder – is able to utilise vast amounts of unlabelled data to learn a representation for skin lesions, and small amounts of labelled data to assign class labels based on the learned representation. They perform an ablation study to analyse the contributions of both the adversarial and denoising components and compare their work with state‐of‐the‐art results. They find that their model yields superior classification performance, especially when evaluating their model at high sensitivity values.

Computer vision remains an under-exploited technology for assistive devices. Here, we propose a navigation technique using low-resolution images from wearable or hand-held cameras to identify landmarks that are indicative of a user’s position along crowdsourced paths. We test the components of a system that is able to provide blindfolded users with information about location via tactile feedback. We assess the accuracy of vision-based localisation by making comparisons with estimates of location derived from both a recent SLAM-based algorithm and from indoor surveying equipment. We evaluate the precision and reliability by which location information can be conveyed to human subjects by analysing their ability to infer position from electrostatic feedback in the form of textural (haptic) cues on a tablet device. Finally, we describe a relatively lightweight systems architecture that enables images to be captured and location results to be served back to the haptic device based on journey information from multiple users and devices.

Vision is one of the most important of the senses, and humans use it extensively during navigation. We evaluated different types of image and video frame descriptors that could be used to determine distinctive visual landmarks for localizing a person based on what is seen by a camera that they carry. To do this, we created a database containing over 3 km of video-sequences with ground-truth in the form of distance travelled along different corridors. Using this database, the accuracy of localization—both in terms of knowing which route a user is on—and in terms of position along a certain route, can be evaluated. For each type of descriptor, we also tested different techniques to encode visual structure and to search between journeys to estimate a user’s position. The techniques include single-frame descriptors, those using sequences of frames, and both color and achromatic descriptors. We found that single-frame indexing worked better within this particular dataset. This might be because the motion of the person holding the camera makes the video too dependent on individual steps and motions of one particular journey. Our results suggest that appearance-based information could be an additional source of navigational data indoors, augmenting that provided by, say, radio signal strength indicators (RSSIs). Such visual information could be collected by crowdsourcing low-resolution video feeds, allowing journeys made by different users to be associated with each other, and location to be inferred without requiring explicit mapping. This offers a complementary approach to methods based on simultaneous localization and mapping (SLAM) algorithms.

Visual object recognition is just one of the many applications of camera-equipped smartphones. The ability to recognise objects through photos taken with wearable and handheld cameras is already possible through some of the larger internet search providers; yet, there is little rigorous analysis of the quality of search results, particularly where there is great disparity in image quality. This has motivated us to develop the Small Hand-held Object Recognition Test (SHORT). This includes a dataset that is suitable for recognising hand-held objects from either snapshots or videos acquired using hand-held or wearable cameras. SHORT provides a collection of images and ground truth that help evaluate the different factors that affect recognition performance. At its present state, the dataset is comprised of a set of high quality training images and a large set of nearly 135,000 smartphone-captured test images of 30 grocery products. In this paper, we will discuss some open challenges in the visual object recognition of objects that are being held by users. We evaluate the performance of a number of popular object recognition algorithms, with differing levels of complexity, when tested against SHORT.

The ubiquity of smartphones with high quality cameras and fast network connections will spawn many new applications. One of these is visual object recognition, an emerging smartphone feature which could play roles in high-street shopping, price comparisons and similar uses. There are also potential roles for such technology in assistive applications, such as for people who have visual impairment. We introduce the Small Hand-held Object Recognition Test (SHORT), a new dataset that aims to benchmark the performance of algorithms for recognising hand-held objects from either snapshots or videos acquired using hand-held or wearable cameras. We show that SHORT provides a set of images and ground truth that help assess the many factors that affect recognition performance. SHORT is designed to be focused on the assistive systems context, though it can provide useful information on more general aspects of recognition performance for hand-held objects. We describe the present state of the dataset, comprised of a small set of high quality training images and a large set of nearly 135,000 smartphone-captured test images of 30 grocery products. In this version, SHORT addresses another context not covered by traditional datasets, in which high quality catalogue images are being compared with variable quality user-captured images; this makes the matching more challenging in SHORT than other datasets. Images of similar quality are often not present in “database” and “query” datasets, a situation being increasingly encountered in commercial applications. Finally, we compare the results of popular object recognition algorithms of different levels of complexity when tested against SHORT and discuss the research challenges arising from the particularities of visual object recognition from objects that are being held by users.

The Map Seeking Circuit (MSC) has been suggested to address the inverse problem of transformation discovery as found in signal processing, vision, inverse kinematics and many other natural tasks. According to this idea, a parallel search in the transformation space of a high dimensional problem can be decomposed into parts efficiently using the ordering property of superpositions. Deterministic formulations of the circuit have been suggested. Here, we provide a probabilistic interpretation of the architecture whereby the superpositions of the circuit are seen as a series of marginalisations over parameters of the transform. Based on this, we interpret the weights of the MSC as importance weights. The latter suggests the incorporation of Monte-Carlo approaches in the MSC, providing improved resolution of parameter estimates within resource constrained implementations. As a final contribution, we model mixed serial/parallel search strategies of biological vision to reduce the problem of collusions, a common problem in the standard MSC approach.

Using a modified version of the linear cellular neural network (CNN) filtering paradigm recently proposed by the authors, we designed a nonseparable spatiotemporal bandpass filter with tunable spatiotemporal passband volumes. The filter presented here qualitatively resembles spatiotemporal receptive field models for the primary visual cortex. Numerical simulation results confirm the bandpass characteristic of our filtering network.

This paper presents an alternative approach to existing and widely used correlation metrics through the use of orientation information. The gradient field correlation method presented here utilises derivative of Gaussian (DoG) operators for estimating directional derivatives of an image for two matching applications: classical planar object detection and point correspondence matching. The experimental results confirm that a suitably normalised gradient vector field, which emphasises gradient direction information in an image, leads to better selectivity when applied to classical template matching problems. For the case of establishing point correspondences, combinations of gradient vector field metrics yield higher in lying match percentages (by RANSAC) relative to normalised cross-correlation with little extra computational cost, particularly at smaller patch sizes. It is also shown that pixel-wise field component normalisation is critical to the success of this approach.

The steering framework significantly reduces the computational cost of detecting and extracting oriented low-level spatial features in images by linearly combining the responses of a small set of basis filters at fixed orientations to yield filter responses at arbitrary orientations. However, the characteristics of the steered filter, such as orientation and scale selectivity, remain the same as those of the basis filters. We show that linear combination with the appropriate basis filters and weights can achieve more. We present a novel approach for sharpening the orientation selectivity of polar-separable filters by linear combination and well-known trigonometric identities which reduces the need for more basis filters with higher orientation selectivity. We describe the principles for and issues with designing quadrature sharpening filters. We also qualitatively and quantitatively compare 4-basis sharpened filter kernels with an equivalent steerable set of 6-basis filters

The dominant orientation at any point, P, in an image is the direction from P in which there is the least gray-level variance. It is often defined by using gradient estimates, but may be extended to employ neighbourhood operators that provide some degree of phase invariance. However, the approach to estimating the dominant orientation at P depends on the scale or size of the (usually) non-trivial neighbourhood being considered. Multiscale PCA-based orientation estimation techniques involve computationally-heavy solving of eigensystems at each scale and location. In contrast, we propose two methods which use a measure of anisotropy to select or weight orientations respectively at different scales in order to provide a single estimate of orientation at any given point. This is believed to be closer to human perception of contour direction. Results are presented for two simple orientation estimation techniques against comparisons with multiscale PCA estimation for human perceptual boundaries.

This work addresses the design of a novel complex steerable wavelet construction, the generation of transform-space feature measurements associated with corner and edge presence and orientation properties, and the application of these measurements directly to image denoising. The decomposition uses pairs of bandpass filters that display symmetry and antisymmetry about a steerable axis of orientation. While the angular characterization of the bandpass filters is similar to those previously described, the radial characteristic is new, as is the manner of constructing the interpolation functions for steering. The complex filters have been engineered into a multirate system, providing a synthesis and analysis subband filtering system with good reconstruction properties. Although the performance of our proposed denoising strategy is currently below that of recently reported state-of-the-art techniques in denoising, it does compare favorably with wavelet coring approaches employing global thresholds and with an "Oracle" shrinkage technique, and presents a very promising avenue for exploring structure-based denoising in the wavelet domain.

A method of vessel tracking has been developed for quantification of vessel diameters of retinal images. This method utilises twin Gaussian functions to model the distribution of grey level over a vessel cross section. The diameter of the vessel at the cross section can then be calculated using the functions. The variation of vessel diameter in the direction of vessel longitude axis has been described by a tracking technique based on parameters of modelled intensity distribution curves over every cross section. This enables us to obtain an average diameter over any length of a vessel and to develop more parameters for diagnosis and study of vascular diseases.

We describe a multi-rate, compact Hough transform for shape detection. The approach is novel in its application of 2D steerable shape kernels at each level of a multi-rate pyramidal decomposition to construct accumulator spaces. Employing such filtering operations can provide efficiency gains in searching for particular closed shapes in digital images, whilst maintaining code simplicity and freedom from early thresholding operations.

We present a method for the segmentation of blood vessels in retinal images based upon the second derivative of the intensity image which gives information about its topology and overcomes the problem of image intensity variations. The minimum eigenvalue and the magnitude of its gradient are used as features for a region growing procedure which is defined in two stages. For the first stage, growth is restricted to regions with low gradients, allowing vessels to grow where the values of the minimum eigenvalue lie within a wide interval and allowing rapid growth of background regions outside of the vessel boundaries. For the second stage, in which the borders between classes are defined, the algorithm grows vessel and background classes simultaneously without the gradient restriction.

Scale and orientation steerable 2D filters are constructed using a frame of Erlang functions in the Fourier domain. Erlang forms for the radial frequency characteristic are shown to provide complex quadrature filters which can be steered in a scale parameter, #. Oriented, spatial domain filters are constructed by imposing an appropriate angular selectivity in the frequency domain. A filter bank is designed, and outputs of the filters of the bank are steered to construct an oriented scale-space decomposition of an image subband. Some applications are discussed. 1 Introduction The use of sub-band methods in image analysis is perhaps not as widespread as for coding. Nevertheless, quite successful and powerful analysis algorithms have been demonstrated [1]. One can make good use of sub-band processing when image features exist at several scales, so that operators of different sizes must be employed: a key issue then becomes the design of these operators. This paper, in a manner similar to...