In recent years, the rapid advancement of generative AI tools, such as Stable Diffusion and Generative Adversarial Networks, have enabled the realistic manipulation of images, raising concerns about misinformation, fraud, and the integrity of imagery. While these tools have been predominantly applied to conventional images, there is a substantial risk that geospatial images, in particular satellite images, could be manipulated using these tools. Satellite images are being increasingly utilized in journalism, environmental monitoring, and military and humanitarian planning, due to their ever-increasing spatial resolution and accessibility. Therefore, we can determine that there is a substantial risk that these images may become an attractive target for malicious manipulation. In this work, we introduce the development of a novel dataset consisting of high-resolution context-aware manipulated satellite images, designed to reflect realistic adversarial scenarios. Furthermore, we develop a multimodal, geospatially-aware manipulation detection system, which utilizes contextual information and deep learning to identify manipulated regions within satellite images. Our approach aims to advance the state of the art in satellite image forensics and contribute to a robust framework to mitigate the risk posed by manipulated satellite images. We report a macro-averaged F1 score of 0.82, representing a 43% improvement over the current state-of-the-art benchmark.

In radiology, particularly in lung cancer diagnosis, diagnostic errors and cognitive biases pose substantial challenges. These issues, including perceptual errors, interpretive mistakes, and cognitive biases such as anchoring and premature closure, are often unnoticed by experienced radiologists. To address these challenges, we propose the Multi-Eyes principle approach, which utilises multiple deep learning models to reduce bias and potentially improve diagnostic accuracy. Inspired by the Four-Eyes principle in business and cybersecurity, this methodology employs various 3D and 2D (for validation) deep learning architectures and three uncertainty quantification techniques: Monte Carlo Dropout, Deep Ensemble, and Ensemble Monte Carlo Dropout. Each model functions as an independent reviewer, similar to blind reviews. With entropy selected as the uncertainty measurement, it is averaged, followed by ensemble averaging of predictions. The effectiveness of this approach was demonstrated using the LIDC-IDRI dataset for lung cancer classification. Statistical analysis of the uncertainty's distribution reveals that with more models, uncertainty in incorrect predictions becomes more peaked and left skewed, indicating consensus on uncertainty levels. This results in accuracy and F1 score improvements, even with the best performing model, addressing overconfidence in single-model systems. These findings highlight the potential of the Multi-Eyes principle to significantly improve diagnostic performance in computer-aided diagnostic systems. Future research may explore different uncertainty quantification methods and feedback mechanisms for further advancement.

We show that correlations between the camera used to acquire an image and the class label of that image can be exploited by convolutional neural networks (CNN), resulting in a model that “cheats” at an image classification task by recognizing which camera took the image and inferring the class label from the camera. We show that models trained on a dataset with camera / label correlations do not generalize well to images in which those correlations are absent, nor to images from unencountered cameras. Furthermore, we investigate which visual features they are exploiting for camera recognition. Our experiments present evidence against the importance of global color statistics, lens deformation and chromatic aberration, and in favor of high frequency features, which may be introduced by image processing algorithms built into the cameras.

This paper investigates the application of deep convolutional neural networks with prohibitively small datasets to the problem of macular edema segmentation. In particular, we investigate several different heavily regularized architectures. We find that, contrary to popular belief, neural architectures within this application setting are able to achieve close to human-level performance on unseen test images without requiring large numbers of training examples. Annotating these 3D datasets is difficult, with multiple criteria required. It takes an experienced clinician two days to annotate a single 3D image, whereas our trained model achieves similar performance in less than a second. We found that an approach which uses targeted dataset augmentation, alongside architectural simplification with an emphasis on residual design, has acceptable generalization performance- despite relying on fewer than 15 training examples.

With the growing significance of graphs as an effective representation of data in numerous applications, efficient graph analysis using modern machine learning is receiving a growing level of attention. Deep learning approaches often operate over the entire adjacency matrix - as the input and intermediate network layers are all designed in proportion to the size of the adjacency matrix - leading to intensive computation and large memory requirements as the graph size increases. It is therefore desirable to identify efficient measures to reduce both run-time and memory requirements allowing for the analysis of the largest graphs possible. The use of reduced precision operations within the forward and backward passes of a deep neural network along with novel specialised hardware in modern GPUs could offer promising avenues towards efficiency. In this paper, we provide an in-depth exploration of the use of reduced-precision operations, easily integrable into the highly popular PyTorch framework, and an analysis of the effects of Tensor Cores on graph convolutional neural networks. We perform an extensive experimental evaluation of three GPU architectures and two widely-used graph analysis tasks (vertex classification and link prediction) using well-known benchmark and synthetically generated datasets. Thus allowing us to make important observations on the effects of reduced-precision operations and Tensor Cores on computational and memory usage of graph convolutional neural networks - often neglected in the literature.

In this paper, a methodology to disclose ideological features using data analytics techniques aimed at interpreting national security instabilities is proposed. The analysis is based on two concepts, namely, authoritarianism and an attribute connected to it, hostility. Different computational techniques are used to address this a problem suchlike natural language processing, machine learning and deep learning models. The methodology proposed in this paper forms part of and enhances a previously reported holistic social media analysis framework for national security. The robustness and effectiveness of our approach are tested on one real-world event related to disruptive activity, protests in Puerto Rico in 2019.

Fencingis a fast-paced sport played with swords which are Épée, Foil, and Sabre. However, such fast-pace can cause referees to make wrong decisions. Review of slow-motion camera footage in tournaments helps referees' decision-making, but it interrupts the match and may not be available for every organisation. Motivated by the need for better decision-making, analysis and availability, we introduce the first fully-automated deep learning classification and detection system for fencing body moves at the moment a touch is made. This is an important step towards creating a fencing analysis system, with player profiling and decision tools that will benefit the fencing community. The proposed architecture combines You Only Look Once version three (YOLOv3) with a ResNet-34 classifier, trained on ImageNet settings, to obtain 83.0 % test accuracy on the fencing moves. These results are exciting development in the sport, providing immediate feedback and analysis along with accessibility, hence making it a valuable tool for trainers and fencing match referees.

Abstract In this work, we explore the problems of recommending and visualising makeup products based on images of customers. Focusing on mascara, we propose a two-stage approach that first recommends products to a new customer based on the preferences of other customers with similar visual appearance and then visualises how the recommended products might look on the customer. For the initial product recommendation, we train a Siamese convolutional neural network, using our own dataset of cropped eye regions from images of 91 female subjects, such that it learns to output feature vectors that place images of the same subject close together in high-dimensional space. We evaluate the trained network based on its ability to correctly identify existing subjects from unseen images, and then assess its capability to identify visually similar subjects when an image of a new subject is used as input. For product visualisation, we train per-product generative adversarial networks to map the appearance of a specific product onto an image of a customer with no makeup. We train models to generate images of two mascara formulations and assess their capability to generate realistic mascara lashes while changing as little as possible within non-lash image regions and simulating the different effects of the two products used.

Optical Character Recognition (OCR), is extraction of textual data from scanned text documents to facilitate their indexing, searching, editing and to reduce storage space. Although OCR systems have improved significantly in recent years, they still suffer in situations where the OCR output does not match the text in the original document. Deep learning models have contributed positively to many problems but their full potential to many other problems are yet to be explored. In this paper we propose a post-processing approach based on the application deep learning to improve the accuracy of OCR system (minimizing the error rate). We report on the use of neural network language models to accomplish the task of correcting incorrectly predicted characters/words by OCR systems. We applied our approach to the IAM handwriting database. Our proposed approach delivers significant accuracy improvement of 20.41% in F-score, 10.86% in character level comparison using Levenshtein distance and 20.69% in document level comparison over previously reported context based OCR empirical results of IAM handwriting database.

Graphs have become a crucial way to represent large, complex and often temporal datasets across a wide range of scientific disciplines. However, when graphs are used as input to machine learning models, this rich temporal information is frequently disregarded during the learning process, resulting in suboptimal performance on certain temporal inference tasks. To combat this, we introduce Temporal Neighbourhood Aggregation (TNA), a novel vertex representation model architecture designed to capture both topological and temporal information to directly predict future graph states. Our model exploits hierarchical recurrence at different depths within the graph to enable exploration of changes in temporal neighbourhoods, whilst requiring no additional features or labels to be present. The final vertex representations are created using variational sampling and are optimised to directly predict the next graph in the sequence. Our claims are supported by experimental evaluation on both real and synthetic benchmark datasets, where our approach demonstrates superior performance compared to competing methods, outperforming them at predicting new temporal edges by as much as 23% on real-world datasets, whilst also requiring fewer overall model parameters.

The study of National Security and its associated considerations is a sensitive and complex paradigm. It encapsulates both the protection of the territorial integrity and sovereignty of a state, as well as guaranteeing the security of its population. Known as Human Security, human-centred threats arising from radical activities need to be mitigated else they may escalate and have implications on National Security. The modern era has introduced further disruptive challenges, known as Hybrid Threats, that use non-traditional tools (Hybrid Tools) to intensify the impact of a likely threat. Social Media is a clear illustration of such tools, where the stability of the state and its people can be compromised by the dissemination of material. The ability to identify behaviour bordering on criminality within the deregulated world of Social Media is a Human Security imperative for governments. This paper follows on from our earlier work to detect affected National Security variables through the analysis of social media communication and trigger an alert when a likely threat is detected. As a result, a set of crisis interpretation processes are started to construe the event, such as radical behaviour analysis.This paper details the methodological approach to analyse one Hybrid Tool (Social Media) in order to identify likely instability scenarios based on the Human Security spectrum and therefore extract, detect and interpret dissimilar behavioural patterns that outline radical behavioural traits for National Security. The proposed methodology focuses on five steps, namely Instability Scenarios, Entity Extraction, Wordlists Creation, Content Analytics, and Data Interpretation.

Deep Convolutional Neural Networks (CNN) have been extensively applied in various computer vision tasks. Although such approaches have demonstrated exceptionally high performance in various open challenges, adapting them to more specialised tasks can be non-trivial. In this paper we discuss our design and implementation of a batchcode detection system capable of accurate segmentation of batchcode regions within images of consumer products. A batchcode is a unique identifier printed on the packaging of many products that encodes useful information such as date and location of manufacture. Detection of batchcodes in images of products is a useful step in many processes, including quality control, supply chain tracking and counterfeit detection. Beginning with a unique dataset of product images and a set of crowdsourced coarse annotations that roughly correspond to the locations of batchcodes, we demonstrate that such annotations are insufficient for training a reliable model, and subsequently describe a novel label refinement process, which we call the Maximally Stable Global Region (MSGR) method, that we use to generate accurate ground-truth data suitable for training a robust neural network. We also show that detection accuracy can be further improved by applying MSGR to the output of the neural network. We evaluate our approach using a manually labelled test dataset of images of shampoo bottles, and demonstrate the efficacy of the proposed method for accurate real-time batchcode detection.

Graph embeddings have become a key and widely used technique within the field of graph mining, proving to be successful across a broad range of domains including social, citation, transportation and biological. Unsupervised graph embedding techniques aim to automatically create a low-dimensional representation of a given graph, which captures key structural elements in the resulting embedding space. However, to date, there has been little work exploring exactly which topological structures are being learned in the embeddings, which could be a possible way to bring interpretability to the process. In this paper, we investigate if graph embeddings are approximating something analogous to traditional vertex-level graph features. If such a relationship can be found, it could be used to provide a theoretical insight into how graph embedding approaches function. We perform this investigation by predicting known topological features, using supervised and unsupervised methods, directly from the embedding space. If a mapping between the embeddings and topological features can be found, then we argue that the structural information encapsulated by the features is represented in the embedding space. To explore this, we present extensive experimental evaluation with five state-of-the-art unsupervised graph embedding techniques, across a range of empirical graph datasets, measuring a selection of topological features. We demonstrate that several topological features are indeed being approximated in the embedding space, allowing key insight into how graph embeddings create good representations.

Enhancement, followed by segmentation, quantification and modelling of blood vessels in retinal images plays an essential role in computer-aided retinopathy diagnosis. In this paper, we introduce the bowler-hat transform method a new approach based on mathematical morphology for vessel enhancement. The proposed method combines different structuring elements to detect innate features of vessel-like structures. We evaluate the proposed method qualitatively and quantitatively and compare it with the state-of-the-art methods using both synthetic and real datasets. Our results establish that the proposed method achieves high-quality vessel-like structure enhancement in both synthetic examples and clinically relevant retinal images. The bowler-hat transform is shown to be able to detect fine vessels while still remaining robust at junctions.

Graphs are a commonly used construct for representing relationships between elements in complex high dimensional datasets. Many real-world phenomenon are dynamic in nature, meaning that any graph used to represent them is inherently temporal. However, many of the machine learning models designed to capture knowledge about the structure of these graphs ignore this rich temporal information when creating representations of the graph. This results in models which do not perform well when used to make predictions about the future state of the graph - especially when the delta between time stamps is not small. In this work, we explore a novel training procedure and an associated unsupervised model which creates graph representations optimised to predict the future state of the graph. We make use of graph convo-lutional neural networks to encode the graph into a latent representation, which we then use to train our temporal offset reconstruction method, inspired by auto-encoders, to predict a later time point - multiple time steps into the future. Using our method, we demonstrate superior performance for the task of future link prediction compared with none-temporal state-of-the-art baselines. We show our approach to be capable of outperforming non-temporal baselines by 38% on a real world dataset.

The paper presents an Alert Mechanism for analysing and detecting National Security threats using Social Media posts as the primary source of information. This mechanism is meant to be an early warning system that can identify situations where a critical mass of individuals feel attracted towards a disruptive cause, based on emotions and Human Security aspects, which may lead to societal tipping points. Comprehensive experiments on real-world events related to disruptive and non-disruptive cases demonstrate both the robustness and effectiveness of the proposed mechanism.

Handling large corpuses of documents is of significant importance in many fields, no more so than in the areas of crime investigation and defence, where an organisation may be presented with a large volume of scanned documents which need to be processed in a finite time. However, this problem is exacerbated both by the volume, in terms of scanned documents and the complexity of the pages, which need to be processed. Often containing many different elements, which each need to be processed and understood. Text recognition, which is a primary task of this process, is usually dependent upon the type of text, being either handwritten or machine-printed. Accordingly, the recognition involves prior classification of the text category, before deciding on the recognition method to be applied. This poses a more challenging task if a document contains both handwritten and machine-printed text. In this work, we present a generic process flow for text recognition in scanned documents containing mixed handwritten and machine-printed text without the need to classify text in advance. We realize the proposed process flow using several open-source image processing and text recognition packages. The evaluation is performed using a specially developed variant, presented in this work, of the IAM handwriting database, where we achieve an average transcription accuracy of nearly 80% for pages containing both printed and handwritten text.

Macular holes are blinding conditions, where a hole develops in the central part of retina, resulting in reduced central vision. The prognosis and treatment options are related to a number of variables, including the macular hole size and shape. High-resolution spectral domain optical coherence tomography allows precise imaging of the macular hole geometry in three dimensions, but the measurement of these by human observers is time-consuming and prone to high inter- and intra-observer variability, being characteristically measured in 2-D rather than 3-D. We introduce several novel techniques to automatically retrieve accurate 3-D measurements of the macular hole, including: surface area, base area, base diameter, top area, top diameter, height, and minimum diameter. Specifically, we introduce a multi-scale 3-D level set segmentation approach based on a state-of-the-art level set method, and we introduce novel curvature-based cutting and 3-D measurement procedures. The algorithm is fully automatic, and we validate our extracted measurements both qualitatively and quantitatively, where our results show the method to be robust across a variety of scenarios. Our automated processes are considered a significant contribution for clinical applications.

In this paper we study three state-of-the-art, but competing, approaches for generating graph embeddings using unsupervised neural networks. Graph embeddings aim to discover the `best' representation for a graph automatically and have been applied to graphs from numerous domains, including social networks. We evaluate their effectiveness at capturing a good representation of a graph's topological structure by using the embeddings to predict a series of topological features at the vertex level. We hypothesise that an `ideal' high quality graph embedding should be able to capture key parts of the graph's topology, thus we should be able to use it to predict common measures of the topology, for example vertex centrality. This could also be used to better understand which topological structures are truly being captured by the embeddings. We first review these three graph embedding techniques and then evaluate how close they are to being `ideal'. We provide a framework, with extensive experimental evaluation on empirical and synthetic datasets, to assess the effectiveness of several approaches at creating graph embeddings which capture detailed topological structure.

Helical objects occur in medicine, biology, cosmetics, nanotechnology, and engineering. Extracting a 3D parametric curve from a 2D image of a helical object has many practical applications, in particular being able to extract metrics such as tortuosity, frequency, and pitch. We present a method that is able to straighten the image object and derive a robust 3D helical curve from peaks in the object boundary. The algorithm has a small number of stable parameters that require little tuning, and the curve is validated against both synthetic and real-world data. The results show that the extracted 3D curve comes within close Hausdorff distance to the ground truth, and has near identical tortuosity for helical objects with a circular profile. Parameter insensitivity and robustness against high levels of image noise are demonstrated thoroughly and quantitatively.

The accurate analysis of biological networks, enabled by the precise capture of their individual components, can reveal important underlying biological principles. Efficient image processing techniques are required to precisely identify and quantify the networks from complex images. In this paper, we present a novel approach for a weighted and undirected graph-based network reconstruction and quantification from 2D images using an adaptive rectangular mesh refinement approach. The proposed approach is able to efficiently identify the organizational principles of the network, capturing the underlying network structure, and computing relevant network topological properties. We validate the proposed approach by comparing it with the state-of-the-art method.

Runways are vital descriptive features of airports and knowledge of their location is important to many aviation and military applications. With the recent wide availability of remote sensing data, there is demand for an automatic process of extracting runway geometry from satellite imagery. In particular, Very High Resolution (VHR) data makes it feasible to extract a runway's area precisely. In this paper we establish a novel method for accurate and precise extraction of geometric polygons for an arbitrary number of runways in VHR remote sensing imagery. Validated results are demonstrated for a dataset of twelve images of six different airports, at 61 cm resolution from the QuickBird II satellite.

BACKGROUND: Microscopy image segmentation lays the foundation for shape analysis, motion tracking, and classification of biological objects. Despite its importance, automated segmentation remains challenging for several widely used non-fluorescence, interference-based microscopy imaging modalities. For example in differential interference contrast microscopy which plays an important role in modern bacterial cell biology. Therefore, new revolutions in the field require the development of tools, technologies and work-flows to extract and exploit information from interference-based imaging data so as to achieve new fundamental biological insights and understanding. RESULTS: We have developed and evaluated a high-throughput image analysis and processing approach to detect and characterize bacterial cells and chemotaxis proteins. Its performance was evaluated using differential interference contrast and fluorescence microscopy images of Rhodobacter sphaeroides. CONCLUSIONS: Results demonstrate that the proposed approach provides a fast and robust method for detection and analysis of spatial relationship between bacterial cells and their chemotaxis proteins.

MOTIVATION: Fungi form extensive interconnected mycelial networks that scavenge efficiently for scarce resources in a heterogeneous environment. The architecture of the network is highly responsive to local nutritional cues, damage or predation, and continuously adapts through growth, branching, fusion or regression. These networks also provide an example of an experimental planar network system that can be subjected to both theoretical analysis and experimental manipulation in multiple replicates. For high-throughput measurements, with hundreds of thousands of branches on each image, manual detection is not a realistic option, especially if extended time series are captured. Furthermore, branches typically show considerable variation in contrast as the individual cords span several orders of magnitude and the compressed soil substrate is not homogeneous in texture making automated segmentation challenging. RESULTS: We have developed and evaluated a high-throughput automated image analysis and processing approach using Phase Congruency Tensors and watershed segmentation to characterize complex fungal networks. The performance of the proposed approach is evaluated using complex images of saprotrophic fungal networks with 10(5)-10(6) edges. The results obtained demonstrate that this approach provides a fast and robust solution for detection and graph-based representation of complex curvilinear networks. AVAILABILITY AND IMPLEMENTATION: The Matlab toolbox is freely available through the Oxford e-Research Centre website: http://www.oerc.ox.ac.uk/research/bioimage/software CONTACTS: [email protected].

Many biomedical applications require detection of curvilinear structures in images and would benefit from automatic or semiautomatic segmentation to allow high-throughput measurements. Here, we propose a contrast-independent approach to identify curvilinear structures based on oriented phase congruency, i.e., the phase congruency tensor (PCT). We show that the proposed method is largely insensitive to intensity variations along the curve and provides successful detection within noisy regions. The performance of the PCT is evaluated by comparing it with state-of-the-art intensity-based approaches on both synthetic and real biological images.

Abstract Motivation: Advances in the field of microscopy have brought about the need for better image management and analysis solutions. Novel imaging techniques have created vast stores of images and metadata that are difficult to organize, search, process and analyze. These tasks are further complicated by conflicting and proprietary image and metadata formats, that impede analyzing and sharing of images and any associated data. These obstacles have resulted in research resources being locked away in digital media and file cabinets. Current image management systems do not address the pressing needs of researchers who must quantify image data on a regular basis. Results: We present Bisque, a web-based platform specifically designed to provide researchers with organizational and quantitative analysis tools for 5D image data. Users can extend Bisque with both data model and analysis extensions in order to adapt the system to local needs. Bisque's extensibility stems from two core concepts: flexible metadata facility and an open web-based architecture. Together these empower researchers to create, develop and share novel bioimage analyses. Several case studies using Bisque with specific applications are presented as an indication of how users can expect to extend Bisque for their own purposes. Availability: Bisque is web based, cross-platform and open source. The system is also available as software-as-a-service through the Center of Bioimage Informatics at UCSB. Contact: [email protected]; [email protected] Supplementary information: The supplementary material is available at Bioinformatics online, including screen shots, metadata XML descriptions and implementation details.

BACKGROUND: We present a biosegmentation benchmark that includes infrastructure, datasets with associated ground truth, and validation methods for biological image analysis. The primary motivation for creating this resource comes from the fact that it is very difficult, if not impossible, for an end-user to choose from a wide range of segmentation methods available in the literature for a particular bioimaging problem. No single algorithm is likely to be equally effective on diverse set of images and each method has its own strengths and limitations. We hope that our benchmark resource would be of considerable help to both the bioimaging researchers looking for novel image processing methods and image processing researchers exploring application of their methods to biology. RESULTS: Our benchmark consists of different classes of images and ground truth data, ranging in scale from subcellular, cellular to tissue level, each of which pose their own set of challenges to image analysis. The associated ground truth data can be used to evaluate the effectiveness of different methods, to improve methods and to compare results. Standard evaluation methods and some analysis tools are integrated into a database framework that is available online at http://bioimage.ucsb.edu/biosegmentation/. CONCLUSION: This online benchmark will facilitate integration and comparison of image analysis methods for bioimages. While the primary focus is on biological images, we believe that the dataset and infrastructure will be of interest to researchers and developers working with biological image analysis, image segmentation and object tracking in general.

This paper describes ongoing work on creating a benchmarking and validation dataset for biological image segmentation. While the primary target is biological images, we believe that the dataset would be of help to researchers working in image segmentation and tracking in general. The motivation for creating this resource comes from the observation that while there are a large number of effective segmentation methods available in the research literature, it is difficult for the application scientists to make an informed choice as to what methods would work for her particular problem. No one single tool exists that is effective on a diverse set of application contexts and different methods have their own strengths and limitations. We describe below three different classes of data, ranging in scale from subcellular to cellular to tissue level images, each of which pose their own set of challenges to image analysis. Of particular value to the image processing researchers is that the data comes with associated ground truth information that can be used to evaluate the effectiveness of different methods. The analysis and evaluation are also integrated into a database framework that is available online at http://dough.ece.ucsb.edu.