Abstract In multiple-instance learning (MIL), the existing bag encoding and attention-based pooling approaches assume that the instances in the bag have no relationship among them. This assumption is unsuited, as the instances in the bags are rarely independent in diverse MIL applications. In contrast, the instance relationship assumption-based techniques incorporate the instance relationship information in the classification process. However, in MIL, the bag composition process is complicated, and it may be possible that instances in one bag are related and instances in another bag are not. In present MIL algorithms, this relationship assumption is not explicitly modeled. The learning algorithm is trained based on one of two relationship assumptions (whether instances in all bags have a relationship or not). Hence, it is essential to model the assumption of instance relationships in the bag classification process. This paper proposes a robust approach that generates vector representation for the bag for both assumptions and the representation selection process to determine whether to consider the instances related or unrelated in the bag classification process. This process helps to determine the essential bag representation vector for every individual bag. The proposed method utilizes attention pooling and vision transformer approaches to generate bag representation vectors. Later, the representation selection subnetwork determines the vector representation essential for bag classification in an end-to-end trainable manner. The generalization abilities of the proposed framework are demonstrated through extensive experiments on several benchmark datasets. The experiments demonstrate that the proposed approach outperforms other state-of-the-art MIL approaches in bag classification.

Gastrointestinal (GI) endoscopy has been an active field of research motivated by the large number of highly lethal GI cancers. Early GI cancer precursors are often missed during the endoscopic surveillance. The high missed rate of such abnormalities during endoscopy is thus a critical bottleneck. Lack of attentiveness due to tiring procedures, and requirement of training are few contributing factors. An automatic GI disease classification system can help reduce such risks by flagging suspicious frames and lesions. GI endoscopy consists of several multi-organ surveillance, therefore, there is need to develop methods that can generalize to various endoscopic findings. In this realm, we present a comprehensive analysis of the Medico GI challenges: Medical Multimedia Task at MediaEval 2017, Medico Multimedia Task at MediaEval 2018, and BioMedia ACM MM Grand Challenge 2019. These challenges are initiative to set-up a benchmark for different computer vision methods applied to the multi-class endoscopic images and promote to build new approaches that could reliably be used in clinics. We report the performance of 21 participating teams over a period of three consecutive years and provide a detailed analysis of the methods used by the participants, highlighting the challenges and shortcomings of the current approaches and dissect their credibility for the use in clinical settings. Our analysis revealed that the participants achieved an improvement on maximum Mathew correlation coefficient (MCC) from 82.68% in 2017 to 93.98% in 2018 and 95.20% in 2019 challenges, and a significant increase in computational speed over consecutive years.

The prevalent use of surveillance cameras in public places and advancements in computer vision warrants most sought-after research in the domain of anomalous activity detection. Several approaches have been proposed for the detection of an anomaly in videos. Spatio-temporal features using 3D Convolutional Network (C3D) is a state-of-the-art approach for this problem where deep multiple instance ranking framework is being investigated. However, this approach requires the segmentation of videos before feature extraction that can produce unstable segmentation results and can have a large memory footprint. In this paper, we extract video features using the Inception-v3 deep learning network which eliminates segmentation. Moreover, to improve the robustness of the backbone classifier we propose to use a homogeneous bagging ensemble of the 3-Layer Fully Connected (FC) Network. Experiments are conducted on the UCF-Anomaly detection dataset and exhibit improved performance over existing approaches.

Handwriting biometrics is the science of identifying the behavioral aspect of an individual’s writing style and exploiting it to develop automated writer identification and verification systems. This paper presents an efficient handwriting identification system which combines scale-invariant feature transform (SIFT) and RootSIFT descriptors in a set of Gaussian mixture models (GMMs). In particular, a new concept of similarity and dissimilarity Gaussian mixture models (SGMM and DGMM) is introduced. While an SGMM is constructed for every writer to describe the intra-class similarity that is exhibited between the handwritten texts of the same writer, a DGMM represents the contrast or dissimilarity that exists between the writer’s style on one hand and other different handwriting styles on the other hand. Furthermore, because the handwritten text is described by a number of key point descriptors where each descriptor generates an SGMM/DGMM score, a new weighted histogram method is proposed to derive the intermediate prediction score for each writer’s GMM. The idea of weighted histogram exploits the fact that handwritings from the same writer should exhibit more similar textual patterns than dissimilar ones, hence, by penalizing the bad scores with a cost function, the identification rate can be significantly enhanced. Our proposed system has been extensively assessed using six different public datasets (including three English, two Arabic, and one hybrid language), and the results have shown the superiority of the proposed system over the state-of-the-art techniques.

Retinal image analysis may disclose severity and causes of many diabetic diseases e.g. for Diabetic Macular Edema inspection. Many techniques have been introduced for automatic classification of exudate lesion to speed up the diagnosis of diabetic disease. In almost all previous work, exudate lesion detection is either modelled as binary or multiclass classification problem. However, along with the classification of normal / abnormal regions, other information needs to be simultaneously classified such as patient's age, ethnicity, race, diabetic's type etc. In this work, we presented a new technique, namely Multi-label learning model to improve the classification of exudate lesions. Features are extracted using multi-scale local binary patterns. Multi label k nearest neighbour (ML-kNN), Multi-label Ranking Support Vector Machine Learning (ML-Rank SVM), Multi-label Learning Neural Network Radial Base Function (MLNN-RBF) and Multi-label Learning Neural Network Back-Propagation (MLNN-BP) are evaluated as the classification models and compared with traditional binary multi-class classifiers. Experiment results show that multi-label framework is very useful for diabetic retinopathy differentiation and can improve retinal image classification.

Diabetic retinopathy (DR) is one of the most important cause of vision loss in diabetic patients. The most primary sign of DR is the presence of exudates, and detecting these in early screening is crucial in preventing vision loss. This paper proposes a system for automatic exudate detection using a combination of texture features, extracted from different local binary pattern (LBP) variants, with an artificial neural network (ANN) classifier. The publicly available database DIARETDB0 of colour fundus images was used for testing purposes and the values of sensitivity, specificity and accuracy found were 98.68%, 94.81 % and 96.73% respectively for the neural network based classification. These results have also been shown to outperform existing work.

Writer identification is an important field in forensic document examination. Typically, a writer identification system consists of two main steps: feature extraction and matching and the performance depends significantly on the feature extraction step. In this paper, we propose a set of novel geometrical features that are able to characterize different writers. These features include direction, curvature, and tortuosity. We also propose an improvement of the edge-based directional and chain code-based features. The proposed methods are applicable to Arabic and English handwriting. We have also studied several methods for computing the distance between feature vectors when comparing two writers. Evaluation of the methods is performed using both the IAM handwriting database and the QUWI database for each individual feature reaching Top1 identification rates of 82 and 87 % in those two datasets, respectively. The accuracies achieved by Kernel Discriminant Analysis (KDA) are significantly higher than those observed before feature-level writer identification was implemented. The results demonstrate the effectiveness of the improved versions of both chain-code features and edge-based directional features.

Intermodality face matching or Heterogeneous face recognition involves matching faces from different modalities such as infrared images, sketch images and low/high resolution visual images. This problem is further alleviated due to inherit problems in face recognition such as pose, expression, illumination, occlusion etc. Existing face recognition algorithms fail to address the existing feature gap exist between images of different modalities. To solve this problem, we propose a new method inspired from Probabilistic Linear Discriminant Analysis (PLDA). PLDA is a generative probabilistic method which models the face into signal and noise components. This method reports outstanding results when compared to other contemporary approaches. But PLDA is designed to apply the image data in only one modality. In this paper, its efficacy has been extended to more generic problem of handling faces captured in different modalities. Experiments conducted on HFB (VIS-NIR), Biosecure (Low-High or Webcam-Digitalcam) face databases validate its robustness and superiority over other methods.

Face recognition subject to uncontrolled illumination and blur is challenging. Interestingly, image degradation caused by blurring, often present in real-world imagery, has mostly been overlooked by the face recognition community. Such degradation corrupts face information and affects image alignment, which together negatively impact recognition accuracy. We propose a number of countermeasures designed to achieve system robustness to blurring. First, we propose a novel blur-robust face image descriptor based on Local Phase Quantization (LPQ) and extend it to a multiscale framework (MLPQ) to increase its effectiveness. To maximize the insensitivity to misalignment, the MLPQ descriptor is computed regionally by adopting a component-based framework. Second, the regional features are combined using kernel fusion. Third, the proposed MLPQ representation is combined with the Multiscale Local Binary Pattern (MLBP) descriptor using kernel fusion to increase insensitivity to illumination. Kernel Discriminant Analysis (KDA) of the combined features extracts discriminative information for face recognition. Last, two geometric normalizations are used to generate and combine multiple scores from different face image scales to further enhance the accuracy. The proposed approach has been comprehensively evaluated using the combined Yale and Extended Yale database B (degraded by artificially induced linear motion blur) as well as the FERET, FRGC 2.0, and LFW databases. The combined system is comparable to state-of-the-art approaches using similar system configurations. The reported work provides a new insight into the merits of various face representation and fusion methods, as well as their role in dealing with variable lighting and blur degradation.

Visual concept detection and action recognition are one of the most important tasks in content-based multimedia information retrieval (CBMIR) technology. It aims at annotating images using a vocabulary defined by a set of concepts of interest including scenes types (mountains, snow, etc.) or human actions (phoning, playing instrument). This paper describes our system in the ImageCLEF@ICPR10, Pascal VOC 08 Visual Concept Detection and Pascal VOC 10 Action Recognition Challenges. The proposed system ranked first in these large-scale tasks when evaluated independently by the organizers. The proposed system involves state-of-the-art local descriptor computation, vector quantization via clustering, structured scene or object representation via localized histograms of vector codes, similarity measure for kernel construction and classifier learning. The main novelty is the classifier-level and kernel-level fusion using Kernel Discriminant Analysis and Spectral Regression (SR-KDA) with RBF Chi-Squared kernels obtained from various image descriptors. The distinctiveness of the proposed method is also assessed experimentally using a video benchmark: the Mediamill Challenge along with benchmarks from ImageCLEF@ICPR10, Pascal VOC 10 and Pascal VOC 08. From the experimental results, it can be derived that the presented system consistently yields significant performance gains when compared with the state-of-the art methods. The other strong point is the introduction of SR-KDA in the classification stage where the time complexity scales linearly with respect to the number of concepts and the main computational complexity is independent of the number of categories.

Linear Regression Classifier (LRC) is state-of-the-art face recognition method that represent a probe image as a linear combination of class specific models. However, this method views the image as a point in a feature space, and thus LRC cannot accommodate severe luminance alterations. Histogram-based features, such as Multiscale Local Phase Quantisation histogram (MLPQH) have gained reputation as powerful and attractive texture descriptors showing excellent results in terms of accuracy and computational complexity in face recognition. In this paper, MLPQH features are integrated with "face" features to confront the illumination problem in LRC. The main novelty is the fusion of histogram and face features using z-score normalisation and LRC classifier. The proposed system is evaluated on two benchmarks: ORL and Extended Yale B. The results indicate a significant increase in the performance when compared with state-of the-art face recognition methods.

Visual concept detection is one of the most important tasks in image and video indexing. This paper describes our system in the ImageCLEF@ICPR Visual Concept Detection Task which ranked first for large-scale visual concept detection tasks in terms of Equal Error Rate (EER) and Area under Curve (AUC) and ranked third in terms of hierarchical measure. The presented approach involves state-of-the-art local descriptor computation, vector quantisation via clustering, structured scene or object representation via localised histograms of vector codes, similarity measure for kernel construction and classifier learning. The main novelty is the classifier-level and kernel-level fusion using Kernel Discriminant Analysis with RBF/Power Chi-Squared kernels obtained from various image descriptors. For 32 out of 53 individual concepts, we obtain the best performance of all 12 submissions to this task.

We consider the problem of learning a linear combination of pre-specified kernel matrices in the Fisher discriminant analysis setting. Existing methods for such a task impose an ¿1norm regularisation on the kernel weights, which produces sparse solution but may lead to loss of information. In this paper, we propose to use ¿2norm regularisation instead. The resulting learning problem is formulated as a semi-infinite program and can be solved efficiently. Through experiments on both synthetic data and a very challenging object recognition benchmark, the relative advantages of the proposed method and its ¿1counterpart are demonstrated, and insights are gained as to how the choice of regularisation norm should be made.

This paper considers on a number of issues that arise when a trainable machine vision system learns directly from humans. We contrast this to the “normal” situation where machine learning (ML) techniques are applied to a “cleaned” data set which is considered to be perfectly labeled with complete accuracy. This paper is done within the context of a generic system for the visual surface inspection of manufactured parts; however, the issues treated are relevant not only to wider computer vision applications such as medical image screening but also to classification more generally. Many of the issues we consider arise from the nature of humans themselves: They will be not only internally inconsistent but also will often not be completely confident about their decisions, particularly if they are making decisions rapidly. People will also often differ systematically from each other in the decisions they make. Other issues may arise from the nature of the process, which may require the ML to have the capacity for real-time online adaptation in response to users' input. Because of this, it may be that the users cannot always provide input to a consistent level of detail. We describe how all of these issues may be tackled within a coherent methodology. By using a range of classifiers trained on data sets from a compact disc imprint production process, we present results which demonstrate that training methods designed to take proper consideration of these issues may actually lead to improved performance.

This paper discusses the suitability of reconfigurable computing to speedup classification problems using Nearest Neighbour (1NN) classifier. 1NN classifier is widely used in the literature especially in real-time applications such as face recognition, on-line hand-written character recognition and medical applications where the performance enhancement in terms of speed is desirable. To evaluate the effectivness of our implementation on Field Programmable Gate Arravs (FPGAs), experiments were carried out on two medical data sets. Results have shown that the classification accuracy is exactly same for both FPGAs and microprocessor (µP) based solutions with FPGA has superior speed performances.

The nearest-neighbor (NN) classifier has long been used in pattern recognition, exploratory data analysis, and data mining problems. A vital consideration in obtaining good results with this technique is the choice of distance function, and correspondingly which features to consider when computing distances between samples. In this paper, a new ensemble technique is proposed to improve the performance of NN classifier. The proposed approach combines multiple NN classifiers, where each classifier uses a different distance function and potentially a different set of features (feature vector). These feature vectors are determined for each distance metric using Simple Voting Scheme incorporated in Tabu Search (TS). The proposed ensemble classifier with different distance metrics and different feature vectors (TS-DF/NN) is evaluated using various benchmark data sets from UCI Machine Learning Repository. Results have indicated a significant increase in the performance when compared with various well-known classifiers. Furthermore, the proposed ensemble method is also compared with ensemble classifier using different distance metrics but with same feature vector (with or without Feature Selection (FS)).

Quantitative cell imagery in cancer pathology has progressed greatly in the last 25 years. The application areas are mainly those in which the diagnosis is still critically reliant upon the analysis of biopsy samples, which remains the only conclusive method for making an accurate diagnosis of the disease. Biopsies are usually analyzed by a trained pathologist who, by analyzing the biopsies under a microscope, assesses the normality or malignancy of the samples submitted. Different grades of malignancy correspond to different structural patterns as well as to apparent textures. In the case of prostate cancer, four major groups have to be recognized: stroma, benign prostatic hyperplasia, prostatic intraepithelial neoplasia, and prostatic carcinoma. Recently, multispectral imagery has been used to solve this multiclass problem. Unlike conventional RGB color space, multispectral images allow the acquisition of a large number of spectral bands within the visible spectrum, resulting in a large feature vector size. For such a high dimensionality, pattern recognition techniques suffer from the well-known "curse-of-dimensionality" problem. This paper proposes a novel round-robin tabu search (RR-TS) algorithm to address the curse-of-dimensionality for this multiclass problem. The experiments have been carried out on a number of prostate cancer textured multispectral images, and the results obtained have been assessed and compared with previously reported works. The system achieved 98%-100% classification accuracy when testing on two datasets. It outperformed principal component/linear discriminant classifier (PCA-LDA), tabu search/nearest neighbor classifier (TS-1NN), and bagging/boosting with decision tree (C4.5) classifier.

Grey level co-occurrence matrix (GLCM), one of the best known tool for texture analysis, estimates image properties related to second-order statistics. These image properties commonly known as Haralick texture features can be used for image classification, image segmentation, and remote sensing applications. However, their computations are highly intensive especially for very large images such as medical ones. Therefore, methods to accelerate their computations are highly desired. This paper proposes the use of reconfigurable hardware to accelerate the calculation of GLCM and Haralick texture features. The performances of the proposed co-processor are then assessed and compared against a microprocessor based solution.