Determining early-stage prognostic markers and stratifying patients for effective treatment are two key challenges for improving outcomes for melanoma patients. Previous studies have used tumour transcriptome data to stratify patients into immune subgroups, which were associated with differential melanoma specific survival and potential predictive biomarkers. However, acquiring transcriptome data is a time-consuming and costly process. Moreover, it is not routinely used in the current clinical workflow. Here, we attempt to overcome this by developing deep learning models to classify gigapixel haematoxylin and eosin (H&E) stained pathology slides, which are well established in clinical workflows, into these immune subgroups. We systematically assess six different multiple instance learning (MIL) frameworks, using five different image resolutions and three different feature extraction methods. We show that pathology-specific self-supervised models using 10x resolution patches generate superior representations for the classification of immune subtypes. In addition, in a primary melanoma dataset, we achieve a mean area under the receiver operating characteristic curve (AUC) of 0.80 for classifying histopathology images into 'high' or 'low immune' subgroups and a mean AUC of 0.82 in an independent TCGA melanoma dataset. Furthermore, we show that these models are able to stratify patients into 'high' and 'low immune' subgroups with significantly different melanoma specific survival outcomes (log rank test, P< 0.005). We anticipate that MIL methods will allow us to find new biomarkers of high importance, act as a tool for clinicians to infer the immune landscape of tumours and stratify patients, without needing to carry out additional expensive genetic tests.

Urban infrastructure assets (e.g. roads, water pipes) perform critical functions to the health and well-being of society. Although it has been widely recognised that different infrastructure assets are highly interconnected, infrastructure management in practice such as planning, installation and maintenance are often undertaken by different stakeholders without considering these dependencies due to the lack of relevant data and cross-domain knowledge, which may cause unexpected cascading social, economic and environmental effects. In this paper, we present a knowledge based decision support system for urban infrastructure inter-asset management. By considering various infrastructure assets (e.g. road, ground, cable), triggers (e.g. pipe leaking) and potential consequences (e.g. traffic disruption) as a holistic system, we model each sub-domain using a modular ontology and encapsulate the interdependence between them using a set of rules. Moreover, qualitative likelihood is assigned to each rule by domain experts (e.g. civil engineers) to encode the uncertainty of knowledge, and an inference engine is applied to predict the potential consequences of a given trigger with location specific data and the encoded rules. A web-based prototype system has been developed based on the above concept and demonstrated to a wide range of stakeholders. The system can assist in the process of decision making by aiding data collation and integration, as well as presenting potential consequences of possible triggers, advising on whether additional information is needed or suggesting ways of obtaining such information. The work shows an intelligent approach to integrate and process multi-source data to pioneer a novel way to aid a complex decision process with a high social impact.

We present an interactive decision support system for assisting city infrastructure inter-asset management. It combines real-time site specific data retrieval, a knowledge base co-created with domain experts and an inference engine capable of predicting potential consequences and risks resulting from the available data and knowledge. The system can give explanations of each consequence, cope with incomplete and uncertain data by making assumptions about what might be the worst case scenario, and making suggestions for further investigation. This demo presents multiple real-world scenarios, and demonstrates how modifying assumptions (parameter values) can lead to different consequences.

We present an uncertainty management scheme in rule-based systems for decision making in the domain of urban infrastructure. Our aim is to help end users make informed decisions. Human reasoning is prone to a certain degree of uncertainty but domain experts frequently find it difficult to quantify this precisely, and thus prefer to use qualitative (rather than quantitative) confidence levels to support their reasoning. Secondly, there is uncertainty in data when it is not currently available (missing). In order to incorporate human-like reasoning within rule-based systems we use qualitative confidence levels chosen by domain experts in urban infrastructure. We introduce a mechanism for the representation of confidence of input facts and inference rules, and for the computation of confidence in the inferred facts. We also present a mechanism for computing inferences in the presence of missing facts, and their effect on the confidence of inferred facts.

The problem of automatically recognizing and fitting hyperbolae from ground-penetrating radar (GPR) images is addressed, and a novel technique computationally suitable for real-time on-site application is proposed. After preprocessing of the input GPR images, a novel thresholding method is applied to separate the regions of interest from background. A novel column-connection clustering (C3) algorithm is then applied to separate the regions of interest from each other. Subsequently, a machine learnt model is applied to identify hyperbolic signatures from outputs of the C3 algorithm, and a hyperbola is fitted to each such signature with an orthogonal-distance hyperbola fitting algorithm. The novel clustering algorithm C3 is a central component of the proposed system, which enables the identification of hyperbolic signatures and hyperbola fitting. Only two features are used in the machine learning algorithm, which is easy to train using a small set of training data. An orthogonal-distance hyperbola fitting algorithm for “south-opening” hyperbolae is introduced in this work, which is more robust and accurate than algebraic hyperbola fitting algorithms. The proposed method can successfully recognize and fit hyperbolic signatures with intersections with others, hyperbolic signatures with distortions, and incomplete hyperbolic signatures with one leg fully or largely missed. As an additional novel contribution, formulas to compute an initial “south-opening” hyperbola directly from a set of given points are derived, which make the system more efficient. The parameters obtained by fitting hyperbolae to hyperbolic signatures are very important features; they can be used to estimate the location and size of the related target objects and the average propagation velocity of the electromagnetic wave in the medium. The effectiveness of the proposed system is tested on both synthetic and real GPR data.

We propose a personalized ConvNet pose estimator that automatically adapts itself to the uniqueness of a person's appearance to improve pose estimation in long videos. We make the following contributions: (i) we show that given a few high-precision pose annotations, e.g. from a generic ConvNet pose estimator, additional annotations can be generated throughout the video using a combination of image-based matching for temporally distant frames, and dense optical flow for temporally local frames, (ii) we develop an occlusion aware self-evaluation model that is able to automatically select the high-quality and reject the erroneous additional annotations, and (iii) we demonstrate that these high-quality annotations can be used to fine-tune a ConvNet pose estimator and thereby personalize it to lock on to key discriminative features of the person's appearance. The outcome is a substantial improvement in the pose estimates for the target video using the personalized ConvNet compared to the original generic ConvNet. Our method outperforms the state of the art (including top ConvNet methods) by a large margin on three standard benchmarks, as well as on a new challenging YouTube video dataset. Furthermore, we show that training from the automatically generated annotations can be used to improve the performance of a generic ConvNet on other benchmarks.

The transport network in many countries relies on extended portions which run underground in tunnels. As tunnels age, repairs are required to prevent dangerous collapses. However repairs are expensive and will affect the operational efficiency of the tunnel. We present a decision support system (DSS) based on supervised machine learning methods that learns to predict the risk factor and the resulting repair urgency in the tunnel maintenance planning of a European national rail operator. The data on which the prototype has been built consists of 47 tunnels of varying lengths. For each tunnel, periodic survey inspection data is available for multiple years, as well as other data such as the method of construction of the tunnel. Expert annotations are also available for each 10m tunnel segment for each survey as to the degree of repair urgency which are used for both training and model evaluation. We show that good predictive power can be obtained and discuss the relative merits of a number of learning methods.

Assessing the Underworld (ATU) is a large interdisciplinary UK research project, which addresses challenges in integrated inter-asset maintenance. As assets on the surface of the ground (e.g. roads or pavements) and those buried under it (e.g. pipes and cables) are supported by the ground, the properties and processes of soil affect the performance of these assets to a significant degree. In order to make integrated decisions, it is necessary to combine the knowledge and expertise in multiple areas, such as roads, soil, buried assets, sensing, etc. This requires an underpinning knowledge model, in the form of an ontology. Within this context, we present a new ontology for describing soil properties (e.g. soil strength) and processes (e.g. soil compaction), as well as how they affect each other. This ontology can be used to express how the ground affects and is affected by assets buried under the ground or on the ground surface. The ontology is written in OWL 2 and openly available from the University of Leeds data repository: http://doi.org/10.5518/54 .

The objective of this work is to estimate upper body pose for signers in TV broadcasts. Given suitable training data, the pose is estimated using a random forest body joint detector. However, obtaining such training data can be costly. The novelty of this paper is a method of transfer learning which is able to harness existing training data and use it for new domains. Our contributions are: (i) a method for adapting existing training data to generate new training data by synthesis for signers with different appearances, and (ii) a method for personalising training data. As a case study we show how the appearance of the arms for different clothing, specifically short and long sleeved clothes, can be modelled to obtain person-specific trackers. We demonstrate that the transfer learning and person specific trackers significantly improve pose estimation performance.

Generic solutions for misalignment correction in image stacks are based on slice-byslice registration to a reference volume. Slice misalignment in cardiac Magnetic Resonance cine series typically includes translational and rotational components. Recovery of rotation in medial down to apical slices with the basic slice-to-volume registration may provide unreliable results because of rotational symmetry of the short-axis view of the left ventricle. This paper describes a novel method for correction of stack misalignment in cardiac cine series with recovery of the rotational component. The core of the presented method is a custom spatial transform which parametrises a set of independent centred rigid transforms, one per slice in a stack. The stack alignment transform was designed to circumvent the weaknesses associated with slice-to-volume registration and improve the reliability of misalignment correction over the slice-by-slice registration approach. The image similarity metric for every iteration during the stack alignment transform optimisation is calculated on the whole image stack with all slice correction parameters contributing to the result. The presented method was evaluated on 50 clinical datasets and the results demonstrate an improvement of misalignment correction with the proposed transform. The study also provides evidence of the importance of recovery of slice rotations in this application; the presented method is general enough to be applied to similar problems in other image modalities.

An efficient and general framework for the incorporation of statistical prior information, based on a wide variety of detectable point features, into level set based object tracking is presented. Level set evolution is based on the maximisation of a set of likelihoods on mesh values at features, which are located using a stochastic sampling process. This evolution is based on the interpolation of likelihood gradients using kernels centred at the features. Feature detectors implemented are based on moments of colour histogram segmented images and learned image patches located using normalised correlation, although a wide variety of feature detectors could be used. A computationally efficient level set implementation is presented along with a method for the incorporation of a motion model into the scheme. 1

A qualitative image description grammar with automatic image fitting and object modelling algorithms is presented. The grammar is based on as-signing a square sub-region of an image one of a finite number of qualitative labels, based on the occurrence of object boundaries within this region and how these intersect the region boundary. In the general case there is an in-finite number of such labels, however the use of a multi-scale approach al-lows a finite (small) number of labels at each scale. This makes the problem tractable within a constraint satisfaction type framework. Constraints are put on neighboring labels based on the premise that all object boundaries are con-tinuous, having no ending within an image. A minimum description length (MDL) approach is suggested for description hypothesis selection (based on colour histograms) and methods for (constraint based) hypothesis genera-tion/adaption and (Hidden Markov Model based) a-priori shape modelling are presented. 1

In this paper we present a system that combines the benefits of 3D deformable models and level set methods for medical volume segmentation. Our 3D deformable model is a very computationally efficient method for segmenting medical volumes, however it is not currently able to segment features, such as renal arteries, that are small relative to the imaging slice thickness used. Level Set methods are an alternative approach to deformable models that re-pose the volume segmentation problem as the calculation of the steady state of an initial value Partial Differential Equation (PDE) system on a regular rectilinear or cubic mesh. The segmentation obtained is parameterised by the zero value level set of this mesh (analogous to an iso-surface). These methods are very computationally expensive, but have the advantage of being able to segment relatively small features such as renal arteries. The problem domain explored in this paper is the segmentation of arterial structures. The results of these segmentations are to be used in the assessment of patient suitability for minimally invasive (keyhole) surgical procedures in patients with abnormal aortic aneurysms. An abdominal aortic aneurysm (AAA) is a dilation of the abdominal aorta. AAAs usually increase in size with time, and if left untreated eventually rupture causing catastrophic haemorrhage. An AAA may be treated by conventional surgical methods, but increasingly minimally invasive techniques, where a stent graft is placed in the lumen, are being used. Patient suitability is assessed using CT data and a calibrated projection angiogram - only about 10% of patients are suitable for the keyhole repair. Once a candidate has been assessed as suitable, measurements are made from the same images to determine the key dimensions of the required stent. Our overall aim is to automate both of these stages of image analysis, ensuring that the full 3D nature of the CT is used. In this paper we describe a segmentation aproach that combines the benefits of a 3D deformable model

A system for the tracking and classification of livestock movements is pre-sented. The combined ‘tracker-classifier ’ scheme is based on a variant of Is-ard and Blakes ‘Condensation ’ algorithm [6] known as ‘Re-sampling Con-densation ’ in which a second set of samples is taken from each image in the input sequence based on the results of the initial Condensation sampling. This is analogous to a single iteration of a genetic algorithm and serves to incor-porate image information in sample location. Re-sampling Condensation relies on the variation within the spatial (shape) model being separated into pseudo-independent components (analogous to genes). In the system a hierarchical spatial model based on a variant of the Point DistributionModel [16] is used to model shape variation accurately. Re-sults are presented that show this algorithm gives improved tracking perfor-mance, with no computational overhead, over Condensation alone. Separate Cyclic Hidden Markov Models are used to model ‘Healthy ’ and ‘Lame ’ movements within the Condensation framework in a competitive man-ner such that the model best representing the data will be propagated through the image sequence. 1

A method of generating complementary eigenspaces optimised for interclass and intra-class separability respectively is presented. The objective of creating these spaces is to improve the efficiency of eigenspace search algorithms. The inter-class optimised space may also be used to improve classification and a quantitative evaluation of this against conventional Principal Component Analysis and Canonical analysis (based on Linear Discriminant Analysis) is presented. A qualitative comparison of the intra-class optimised space and spaces produced by Principal Component Analysis on single class data is also presented. 1 Introduction Principal Component Analysis (PCA) is an efficient way of parameterising the variance within a multivariate data set such that the dimensionality may be reduced without greatly affecting approximation accuracy. This is done by finding the eigenvectors of a covariance matrix formed from the data set and forming an `eigenspace' based on these. In many ...

A method of extracting, classifying and modelling non-rigid shapes from an image sequence is presented. Shapes are approximated by polygons where the number of sides is related to the physical features of a shape class rather than any particular shape. A method of `seeding' the polygonal approximation is given where `seeds' are automaticallyextracted from a set of data. Multiple models are built using polygons with different numbers of sides to allow for feature occlusion. Principal component analysis (PCA) is performed on vector representations of the sides of the polygons which are normalised by the total perimeter. This removes the need for normalisation of scale and translation as required in the Point DistributionModel [16]. A `fit score' metric is defined which gives an indicationof how well a given shape fits a model. 1 Introduction There has been much work carried out on the extraction and classification of non-rigid shapes from image sequences. In this paper a new ...