Abstract While self-supervised learning has improved anomaly detection in computer vision and natural language processing, it is unclear whether tabular data can benefit from it. This paper explores the limitations of self-supervision for tabular anomaly detection. We conduct several experiments spanning various pretext tasks on 26 benchmark datasets to understand why this is the case. Our results confirm representations derived from self-supervision do not improve tabular anomaly detection performance compared to using the raw representations of the data. We show this is due to neural networks introducing irrelevant features, which reduces the effectiveness of anomaly detectors. However, we demonstrate that using a subspace of the neural network’s representation can recover performance.

Inferring the connectivity of biological neural networks from neural activation data is an open problem. We propose that the analogous problem in artificial neural networks is more amenable to study and may illuminate the biological case. Here, we study the specific problem of assigning artificial neurons to locations in a network of known architecture, specifically the LeNet image classifier. We evaluate a supervised learning approach based on features derived from the eigenvectors of the activation correlation matrix. Experiments highlighted that for an image dataset to be effective for accurate localisation, it should fully activate the network and contain minimal confounding correlations. No single image dataset was found that resulted in perfect assignment, however perfect assignment was achieved using a concatenation of features from multiple image datasets.

When extrinsic rewards are sparse, artificial agents struggle to explore an environment. Curiosity, implemented as an intrinsic reward for prediction errors, can improve exploration but it is known to fail when faced with action-dependent noise sources (‘noisy TVs’). In an attempt to make exploring agents robust to Noisy TVs, we present a simple solution: aleatoric mapping agents (AMAs). AMAs are a novel form of curiosity that explicitly ascertain which state transitions of the environment are unpredictable, even if those dynamics are induced by the actions of the agent. This is achieved by generating separate forward predictions for the mean and aleatoric uncertainty of future states, with the aim of reducing intrinsic rewards for those transitions that are unpredictable. We demonstrate that in a range of environments AMAs are able to circumvent action-dependent stochastic traps that immobilise conventional curiosity driven agents. Furthermore, we demonstrate empirically that other common exploration approaches—previously thought to be immune to agent-induced randomness—can be trapped by stochastic dynamics.

Recent efforts have shown that neural text processing models are vulnerable to adversarial examples, but the nature of these examples is poorly understood. In this work, we show that adversarial attacks against CNN, LSTM and Transformer-based classification models perform word substitutions that are identifiable through frequency differences between replaced words and their corresponding substitutions. Based on these findings, we propose frequency-guided word substitutions (FGWS), a simple algorithm exploiting the frequency properties of adversarial word substitutions for the detection of adversarial examples. FGWS achieves strong performance by accurately detecting adversarial examples on the SST-2 and IMDb sentiment datasets, with F1 detection scores of up to 91.4% against RoBERTa-based classification models. We compare our approach against a recently proposed perturbation discrimination framework and show that we outperform it by up to 13.0% F1.

Research shows that natural language processing models are generally considered to be vulnerable to adversarial attacks; but recent work has drawn attention to the issue of validating these adversarial inputs against certain criteria (e.g., the preservation of semantics and grammaticality). Enforcing constraints to uphold such criteria may render attacks unsuccessful, raising the question of whether valid attacks are actually feasible. In this work, we investigate this through the lens of human language ability. We report on crowdsourcing studies in which we task humans with iteratively modifying words in an input text, while receiving immediate model feedback, with the aim of causing a sentiment classification model to misclassify the example. Our findings suggest that humans are capable of generating a substantial amount of adversarial examples using semantics-preserving word substitutions. We analyze how human-generated adversarial examples compare to the recently proposed TextFooler, Genetic, BAE and SememePSO attack algorithms on the dimensions naturalness, preservation of sentiment, grammaticality and substitution rate. Our findings suggest that human-generated adversarial examples are not more able than the best algorithms to generate natural-reading, sentiment-preserving examples, though they do so by being much more computationally efficient.

Many operations of vision require image regions to be isolated and inter-related. This is challenging when they are different in detail and extent. Practical methods of Computer Vision approach this through the tools of downsampling, pyramids, cropping and patches. In this paper we develop an ideal geometric structure for this, compatible with the existing scale space model of image measurement. Its elements are apertures which view the image like fuzzy-edged portholes of frosted glass. We establish containment and cause/effect relations between apertures, and show that these link them into cross-scale atlases. Atlases formed of Gaussian apertures are shown to be a continuous version of the image pyramid used in Computer Vision, and allow various types of image description to naturally be expressed within their framework. We show that views through Gaussian apertures are approximately equivalent to the jets of derivative of Gaussian filter responses that form part of standard Scale Space theory. This supports a view of the simple cells of mammalian V1 as implementing a system of local views of the retinal image of varying extent and resolution. As a worked example we develop a keypoint descriptor scheme that outperforms previous schemes that do not make use of learning.

The role of anomaly detection in X-ray security imaging, as a supplement to targeted threat detection, is described, and a taxonomy of anomaly types in this domain is presented. Algorithms are described for detecting appearance anomalies of shape, texture, and density, and semantic anomalies of object category presence. The anomalies are detected on the basis of representations extracted from a convolutional neural network pre-trained to identify object categories in photographs, from the final pooling layer for appearance anomalies, and from the logit layer for semantic anomalies. The distribution of representations in normal data is modeled using high-dimensional, full-covariance, Gaussians, and anomalies are scored according to their likelihood relative to those models. The algorithms are tested on X-ray parcel images using stream-of-commerce data as the normal class, and parcels with firearms present the examples of anomalies to be detected. Despite the representations being learned for photographic images and the varied contents of stream-of-commerce parcels, the system, trained on stream-of-commerce images only, is able to detect 90% of firearms as anomalies, while raising false alarms on 18% of stream-of-commerce.

We present a method for automated car detection in xraytransmission images of freight containers. A random forest classifier was used to classify image sub-windows as “car” and “non-car” based on image features such as intensity and log-intensity, as well as local structures and symmetries as encoded by Basic Image Features (BIFs) and oriented Basic Image Features (oBIFs). The proposed approach was validated using a dataset of stream of commerce X-ray images. A car detection rate of 100% was achieved while maintaining a false alarm rate of 1.23%. Further reduction in false alarm rate, potentially at the cost of detection rate, was possible by tweaking the classification confidence threshold. This work establishes a framework for the automated classification of X-ray transmission cargo images and their content, paving the way towards the development of tools to assist custom officers faced with an ever increasing number of images to inspect.

Texture classification and segmentation have been extensively researched over the last thirty years. Early on the Brodatz album[1] quickly became the de facto standard in which a texture class comprised a set of nonoverlapping sub-images cropped from a single photograph. Later, as the focus shifted to investigating illuminationand pose-invariant algorithms, the CUReT database[3] became popular and the texture class became the set of photographs of a single physical sample captured under a variety of imaging conditions. While extremely successful algorithms have been developed to address classification problems based on these databases, the challenging problem of measuring perceived inter-class texture similarity has rarely been discussed. This paper makes use of a new texture collection[4]. It comprises 334 texture samples, including examples of embossed vinyl, woven wall coverings, carpets, rugs, window blinds, soft fabrics, building materials, product packaging, etc. Additionally, an associated perceptual similarity matrix is provided. This was obtained from a grouping experiment using 30 observers. The similarity scores, S(Ii, I j), for each texture pair were calculated simply by dividing the number of observers that grouped the pair into the same sub-set by the number of observers that had the opportunity do so. A dissimilarity matrix was then defined as dsim(Ii, I j) = 1− S(Ii, I j). Hence dsim(Ii, Ii) = 0 for all images Ii, and dsim(Ii, I j) = 1 if none of the participants grouped images Ii together with I j.

Characters extracted from images or graphics pose a challenge for traditional character recognition techniques. The high degree of intraclass variation along with the presence of clutter makes accurate recognition difficult, yet the semantic information conveyed by sections of text within images or graphics makes their recognition an important problem. Previous work has shown that, on the two most commonly used datasets of such characters, Histogram of Oriented Gradient (HOG) descriptors have outperformed other methods. In this work we consider two extensions of the HOG descriptor to include features at multiple scales, and evaluate their performance using characters taken from images and graphics. We demonstrate that, by combining pairs of oriented gradients at different scales, it's possible to achieve an increase in performance of 12.4% and 5.6% on the two datasets.

We consider the measurement of image structure using linear filters, in particular derivative-of-Gaussian (DtG) filters, which are an important model of V1 simple cells and widely used in computer vision, and whether such measurements can determine local image symmetry. We show that even a single linear filter can be sensitive to a symmetry, in the sense that specific responses of the filter can rule it out. We state and prove a necessary and sufficient, readily computable, criterion for filter symmetry-sensitivity. We use it to show that the six filters in a second order DtG family have patterns of joint sensitivity which are distinct for 12 different classes of symmetry. This rich symmetry-sensitivity adds to the properties that make DtG filters well-suited for probing local image structure, and provides a set of landmark responses suitable to be the foundation of a nonarbitrary system of feature categories.

Many successful recent approaches to texture classification model texture images as distributions over a set of discrete features, or textons, which correspond to a partitioning of the space of responses to local descriptors such as filter banks or image patches. This partitioning is learned by unsupervised clustering of descriptor responses taken from the dataset to be analysed. Here, we explore a quantization of filter responses into a dictionary of discrete features which is based on geometrical, rather than statistical, considerations, resulting in a simple texture description based on a dictionary of dasiavisual wordspsila which is independent of the images to be described. A multi-scale classification scheme built on this dictionary is evaluated. The results presented are, to the best of our knowledge, state-of-the-art for the UIUCTex and KTH-TIPS datasets, and close to the state-of-the-art for CUReT, despite using a less sophisticated classifier.

Most successful object recognition systems are based on a visual alphabet of quantised gradient orientations. Here, we introduce two richer image feature alphabets for use in object recognition. The two alphabets are evaluated using the PASCAL VOC challenge 2007 dataset. The results show that both alphabets perform as well as or better than the 'standard' gradient orientation based one.

Zipf's Law describes a power law between the frequency and rank of an event which is observed, most famously, in word distributions in human languages [1]; where it has been argued [2] that it represents an optimally efficient communication code. From an analogy between image coding schemes and natural language, we hypothesize that conforming to Zipf's Law is a necessary property of a good image description code, i.e. one that provides optimal local descriptors for vision tasks such as object categorization and recognition. Following the analogy between images and documents, we have developed an 'alphabet' of Basic Image Features (BIFs) [3]. This alphabet allows us to label each pixel, at each scale, with one of a small number of letters. 'Words' in this approach are then fixed square patches of BIF 'letters'. We test whether the frequency statistics of BIF words obeys Zipf's law, and find that as we vary certain parameters the Zipfness varies systematically. Remarkably, we find that for certain parameter settings we get extremely Zipf behaviour. Moreover the encoding at these settings looks optimal to eyeball measure.

Characterization of second order local image structure by a 6D vector (or jet) of Gaussian derivative measurements is considered. We consider the affect on jets of a group of transformations-affine intensity-scaling, image rotation and reflection, and their compositions-that preserve intrinsic image structure. We show how this group stratifies the jet space into a system of orbits. Considering individual orbits as points, a 3D orbifold is defined. We propose a norm on jet space which we use to induce a metric on the orbifold. The metric tensor shows that the orbifold is intrinsically curved. To allow visualization of the orbifold and numerical computation with it, we present a mildly-distorting but volume-preserving embedding of it into euclidean 3-space. We call the resulting shape, which is like a flattened lemon, the second order local-image-structure solid. As an example use of the solid, we compute the distribution of local structures in noise and natural images. For noise images, analytical results are possible and they agree with the empirical results. For natural images, an excess of locally 1D structure is found.

A new hierarchical segmentation algorithm is described. Its computational complexity and memory requirements are detailed, showing it to be practicably applicable to images of useful size. A simple modification of the algorithm adapts it to produce hierarchical segmentations that satisfy a constraint set. Results are given showing that this adapted algorithm can be used as the basis of a semi-automatic object definition tool or as the interface between a low-level image description module and a high-level module coding for knowledge and expectation.

Feature extraction by applying a threshold window to image intensity values is a simple and common image processing technique. We con-sider the case of 3D images where intensity based feature extraction is used to determine the volume of objects of interest. We show that the accuracy of volume determination is limited by partial volume (PV) effects. We outline a new method for correcting for PV effects based on object geometry and object intensity. Although this PV correction has been developed with respect to a specific application in magnetic resonance imaging, it is applicable to volume extraction by threshold-ing in any image and can possibly be extended to other intensity-based extraction techniques. 1

In the absence of a priori information on scales of interest vision systems should initially process in a scale invariant manner. The fact that any signal can only be sampled discretely further constrains the initial processing. The paper argues that a representation satisfying these requirements is an hierarchical segmentation of scalespace. An algorithm is presented to compute such representations. The algorithm has been designed so that its operation is scale invariant in the following sense: the addition of finer scale information only ever adds to the computed representation and never changes what was discoverable from coarser scales. It is noted that such a scheme has benefits even when the scales of interest are known. 1

The authors show the close relationship between two algorithms developed in their laboratory: (1) for segmenting grey-scale images and (2) for grouping patterns of dots or disconnected edge fragments. In both cases the output consists of objects each of which is bounded by a closed contours and contains a skeleton whose components are explicitly linked to boundary components. The second algorithm accomplishes a type of perceptual grouping which has often been regarded as implying a different type of processing from earlier segmentation steps. The authors suggest that early segmentation and later perceptual grouping can be, and should be, considered as fundamentally similar processes.>