Abstract Human Activity Recognition (HAR) covers methods for automatically identifying human activities from a stream of data. End‐users of HAR methods cover a range of sectors, including health, self‐care, amusement, safety and monitoring. In this survey, the authors provide a thorough overview of deep learning based and detailed analysis of work that was performed between 2018 and 2023 in a variety of fields related to HAR with a focus on device‐free solutions. It also presents the categorisation and taxonomy of the covered publication and an overview of publicly available datasets. To complete this review, the limitations of existing approaches and potential future research directions are discussed.

Mammography is the primary screening method for lesion visualization and detecting early potentially cancerous changes in breast tissue. The application of deep learning based computer-aided diagnosis (CADx) systems to mammography mass classification poses several challenges: confounding information is learned by a deep learning model, and it can be difficult for mammographic readers to understand how and why it makes a specific decision. In this work, we present a framework for interpretable convolutional neural network-based mammographic abnormality classification. In addition to predicting whether a mass lesion is benign or malignant, our work aims to follow the reasoning processes of mammographic readers in detecting clinically relevant semantic features, such as the shape characteristics of the mass. The framework includes model training that incorporates a combination of data with original images and data with pixel-wise annotations, leading to improved performance of the model. The proposed training method based on DenseNet121 achieved an improved accuracy of 86 . 1 ± 3 . 4 % compared to 67 . 9 ± 3 . 8 % for the original model on mass classification. The results show the proposed method highlighted the classification-relevant parts of the image, whereas the original method highlighted healthy tissue and confounding information. An interpretable algorithm is developed that explains the model using features representing specific clinical characteristics, thereby aiding in prediction. This allows mammographic readers to verify the model’s output for plausibility instead of relying on it blindly.

Breast cancer is the most commonly diagnosed female malignancy worldwide. Recent developments in deep convolutional neural networks have shown promising performance for breast cancer detection and classification. However, due to variations in appearance and small datasets, biased features can be learned by the networks in distinguishing malignant and benign instances. To investigate these aspects, we trained a densely connected convolutional network (DenseNet) to obtain representative features of breast tissue, selecting texture features representing different physical morphological representations as the network’s inputs. Connectivity estimation, represented by a connection matrix, is proposed for feature learning. To make the network provide an unbiased prediction, we used k-nearest neighbors to find k training samples whose connection matrices are closest to the test case. When evaluated on OMI-DB we achieved improved diagnostic accuracy 73.89±2.89% compared with 71.35±2.66% for the initial CNN model, which showed a statistically significant difference (p=0.00036). The k training samples can provide visual explanations which are useful in understanding the model predictions and failures of the model.

CONTEXT AND BACKGROUND: Breast cancer is one of the most common diseases threatening the human lives globally, requiring effective and early risk analysis for which learning classifiers supported with automated feature selection offer a potential robust solution. MOTIVATION: Computer aided risk analysis of breast cancer typically works with a set of extracted mammographic features which may contain significant redundancy and noise, thereby requiring technical developments to improve runtime performance in both computational efficiency and classification accuracy. HYPOTHESIS: Use of advanced feature selection methods based on multiple diagnosis criteria may lead to improved results for mammographic risk analysis. METHODS: An approach for multi-criterion based mammographic risk analysis is proposed, by adapting the recently developed multi-label fuzzy-rough feature selection mechanism. RESULTS: A system for multi-criterion mammographic risk analysis is implemented with the aid of multi-label fuzzy-rough feature selection and its performance is positively verified experimentally, in comparison with representative popular mechanisms. CONCLUSIONS: The novel approach for mammographic risk analysis based on multiple criteria helps improve classification accuracy using selected informative features, without suffering from the redundancy caused by such complex criteria, with the implemented system demonstrating practical efficacy.

Breast cancer has a high incidence among women worldwide. This, together with the recent developments in deep learning based convolutional networks, have motivated research towards the enhancement of Computer Aided Diagnosis (CAD) systems. In this paper, the performance of a densely connected convolutional network (DenseNet) for breast cancer was investigated for the malignant/benign classification of mammographic masses. Different mammography data sets were collected to investigate the capacity of this network for learning a combination of these databases. To achieve this, internal low-level, mid-level and high-level features/abstracts were extracted from the model together with hand-crafted features, generating a vast amount of data. Using the distributed rough set based feature selection approach (Sp-RST), significant features were selected from both deep learning based features and hand-crafted ones, and fed into a learning model with separate and combined data approaches for the classification of mammographic masses. Results show that by using Sp-RST as a powerful technique capable of performing big data preprocessing, DenseNet had the representational capacity to learn mammographic abnormalities.

Network intrusion is one of the most serious threats to network infrastructure and digital assets. Intrusion-detection systems detects un-authorized network access which violates the permissions settings of computer systems, networks, and information systems. Data mining is the one of the businesslike techniques amongst which are available for intrusion detection. Many data mining methods, including decision trees and rough set theory, would work better on discrete or binarized data. In data sets, on account of the existence in very large numbers, data discretization is deemed to be the data pretreatment method and such technology is very important. In this paper, the discretization methods of DFSFDP, DBSCAN, CAIM and K-means are implemented and applied to the KDD intrusion detection data set. Then four evaluation indexes are used to evaluate the effect of discretization. Moreover, with regard to the four discretization methods, their performance and accuracy are put forward and contrasted. The results of experiment acquired through using these methods on NSL-KDD dataset prove that used DFSFDP which is selected by the index of inconsistency measure performs well in terms of classification accuracy.

Computer-aided diagnosis (CAD) has great potential in providing real benefits to doctors and patients. Recent studies have, however, found lack of trust in CAD by radiologists in clinical diagnostic decision making. One of the main reasons is the lack of an appropriate confidence measure. This paper presents the first-ever study of classification confidence in the context of breast mass classification. We evaluated 11 state-of-the-art classification algorithms on breast mass image data using their confidence of classification metric, in addition to other standard evaluation metrics including accuracy and area under the curve (ROC). Experimental results show that although most classifiers produced very similar results with less than 2% difference in terms of accuracy and ROC, their performances are significantly different in terms of confidence levels. We suggest that the confidence measure should be used in conjunction with the existing performance metrics such as accuracy and ROC.

Considering the importance of early diagnosis of breast cancer, a supervised patch‐wise texton‐based approach has been developed for the classification of mass abnormalities in mammograms. The proposed method is based on texture‐based classification of masses in mammograms and does not require segmentation of the mass region. In this approach, patches from filter bank responses are utilised for generating the texton dictionary. The methodology is evaluated on the publicly available Digital Database for Screening Mammography database. Using a naive Bayes classifier, a classification accuracy of 83% with an area under the receiver operating characteristic curve of 0.89 was obtained. Experimental results demonstrated that the patch‐wise texton‐based approach in conjunction with the naive Bayes classifier constructs an efficient and alternative approach for automatic mammographic mass classification.

Computer Aided Detection (CAD) systems are being developed to assist radiologists in diagnosis. For breast cancer the emphasis is shifting from detection to classification of abnormalities. The presented work concentrates on the benign versus malignant classification of micro-calcification clusters, which are a specific type of mammographic abnormality associated with the early development of breast cancer. After segmentation (automatic or manual), tree-based representations were used to distinguish between benign and malignant clusters, which takes into account clinical criteria such as the number of micro-calcifications in the clusters and their distribution and is based on the topology of the trees and the connectivity of the micro-calcifications. The idea of using tree structure based on the distance of individual calcifications for the classification of benign and malignant micro-calcification clusters is novel and closely related to clinical perception. Tree structures used in this study are distinct from decision trees classifiers being used in many machine learning approaches. Initial evaluation on the Digital Database for Screening Mammography (DDSM) data shows promising results, with an accuracy equal to 91 %, which is comparable to state of the art CAD systems and is in line with clinical perception of the morphology and appearance of benign and malignant micro-calcification clusters.

Breast lesion detection using ultrasound imaging is considered an important step of computer-aided diagnosis systems. Over the past decade, researchers have demonstrated the possibilities to automate the initial lesion detection. However, the lack of a common dataset impedes research when comparing the performance of such algorithms. This paper proposes the use of deep learning approaches for breast ultrasound lesion detection and investigates three different methods: a Patch-based LeNet, a U-Net, and a transfer learning approach with a pretrained FCN-AlexNet. Their performance is compared against four state-of-the-art lesion detection algorithms (i.e., Radial Gradient Index, Multifractal Filtering, Rule-based Region Ranking, and Deformable Part Models). In addition, this paper compares and contrasts two conventional ultrasound image datasets acquired from two different ultrasound systems. Dataset A comprises 306 (60 malignant and 246 benign) images and Dataset B comprises 163 (53 malignant and 110 benign) images. To overcome the lack of public datasets in this domain, Dataset B will be made available for research purposes. The results demonstrate an overall improvement by the deep learning approaches when assessed on both datasets in terms of True Positive Fraction, False Positives per image, and F-measure.

The need to reduce the dimensionality of a dataset whilst retaining its inherent manifold structure is key to many pattern recognition, machine learning, and computer vision problems. This process is often referred to as manifold learning since the structure is preserved during dimensionality reduc tion by learning the intrinsic low-dimensional manifold that the data lies upon. In this paper a heuristic approach is presented to tackle this problem by approximating the manifold as a set of piecewise linear models. By merging these linear models in an order defined by their global topology a globally stable and locally accurate model of the manifold can be obtained. A detailed analysis of the proposed approach is presented along with comparison with existing manifold learning techniques. Results obtained on both artificial and image based data show that in many cases this heuristic approach to manifold learning is able to out-perform traditional techniques.

This paper describes a computer-aided diagnosis method for targeted prostate biopsies within the peripheral zone in T2-Weighted MRI. We subdivided the peripheral zone into four regions and compare each sub region's grey level histogram with malignant and normal histogram models, and use specific metrics to estimate the presence of abnormality. The initial evaluation based on 200 MRI slices taken from 40 different patients and we achieved 87% correct classification rate with 89% and 86% sensitivity and specificity, respectively. The main contribution of this paper is a novel approach of Computer Aided Diagnosis which is using grey level histograms analysis between sub regions. In clinical point of view, the developed method could assist clinicians to perform targeted biopsies which are better than the random ones which are currently used.

The Brain Extraction Tool (BET) developed by Smith is widely used for brain segmentation due to its simplicity, accuracy and insensitivity to parameter settings. However, it typically requires a large number of iterations to generate acceptable results. It also sometimes fails to recognize boundaries of the brain. Moreover, obvious under-segmentation occurs for some datasets. In this paper, we present an improved BET method where at each iteration, we enhance the vertex displacement, add a new search path and embed an independent surface reconstruction process. These strategies lead to much faster convergence. Furthermore, a scheme based on fuzzy c-means is proposed to refine the segmentation. Experimental results based on various datsets demonstrated that the proposed method significantly outperforms the original BET and other competing methods.

In this paper, a fully automatic method is proposed for the detection of prostate cancer within the peripheral zone. The method starts by filtering noise of the original image followed by feature extraction and smoothing which is based on Discrete Cosine Transform. Next, we identify the peripheral zone area using quadratic equation and divide it into left and right regions. Subsequently, peak detection is performed on both regions. Finally, we calculate the percentage similarity and Ochiai coefficients to decide whether abnormality occurs. The initial evaluation of the proposed method is based on 70 prostate MRI images from 20 patients and 81.43% of the slices were classified correctly with 11.42% false negative and 7.14% of false positive results, respectively.

In mammographic imaging, the presence of microcalcifications, small deposits of calcium in the breast, is a primary indicator of breast cancer. However, not all microcalcifications are malignant and their distribution within the breast can be used to indicate whether clusters of microcalcifications are benign or malignant. Computer-aided diagnosis (CAD) systems can be employed to help classify such microcalcification clusters. In this paper a novel method for classifying microcalcification clusters is presented by representing discrete mereotopological relations between the individual microcalcifications over a range of scales in the form of a mereotopological barcode. This barcode based representation is able to model complex relations between multiple regions and the results on mammographic microcalcification data shows the effectiveness of this approach. Classification accuracies of 95% and 80% are achieved on the MIAS and DDSM datasets, respectively. These results are comparable to existing state-of-the art methods. This work also demonstrates that mereotopological barcodes could be used to help trained clinicians in their diagnosis by providing a clinical interpretation of barcodes that represent both benign and malignant cases.

Thermal imaging technology can be used to detect stress levels in humans based on the radiated heat from their face. In this paper, we use thermal imaging to monitor the periorbital region's thermal variations and test whether it can offer a discriminative signature for detecting deception. We start by presenting an overview on automated deception detection and propose a novel methodology, which we validate experimentally on 492 thermal responses (249 lies and 243 truths) extracted from 25 participants. The novelty of this paper lies in scoring a larger number of questions per subject, emphasizing a within-person approach for learning from data, proposing a framework for validating the decision making process, and correct evaluation of the generalization performance. A $k$ -nearest neighbor classifier was used to classify the thermal responses using different strategies for data representation. We report an 87% ability to predict the lie/truth responses based on a within-person methodology and fivefold cross validation. Our results also show that the between-person approach for modeling deception does not generalize very well across the training data.

This paper presents a novel method for breast density segmentation in mammograms. The global structure of dense tissue is analysed based on a topographic map of the whole breast, which is a hierarchical representation, obtained from the upper level sets of the image. A shape tree is constructed to represent the topological and geometrical structure of the topographic map. The saliency and independency of shapes are analysed based on the shape tree to detect the candidate dense tissue regions. The geometric moments of the candidates are computed to remove incorrect dense regions. The segmentation results are evaluated based on the full MIAS database. Qualitative evaluation indicates realistic segmentation with respect to breast tissue density. For mammographic risk assessment, the obtained classification accuracy is 76% and 90% for BIRADS and low/high density classification.

This paper presents a combination of existing advanced methods to solve the partial volume segmentation problem. It uses region-based active surface modelling in a hierarchical scheme to eliminate segmentation errors, followed by an alpha matting step to further refine the segmentation. This method can have an interest in several applications in medical imaging. We have validated our method on real PET images of head-and-neck cancer patients as well as custom designed phantom PET images. Experiments show that our method can generate more accurate segmentation than existing approaches.

Manifold learning is a powerful tool for reducing the dimensionality of a dataset by finding a low-dimensional embedding that retains important geometric and topological features. In many applications it is desirable to add new samples to a previously learnt embedding, this process of adding new samples is known as the out-of-sample extension problem. Since many manifold learning algorithms do not naturally allow for new samples to be added we present an easy to implement generalized solution to the problem that can be used with any existing manifold learning algorithm. Our algorithm is based on simple geometric intuition about the local structure of a manifold and our results show that it can be effectively used to add new samples to a previously learnt embedding. We test our algorithm on both artificial and real world image data and show that our method significantly out performs existing out-of-sample extension strategies.

We present a framework for the reduction of dimensionality of a data set via manifold learning. Using the building blocks of local hyperplanes we show how a global manifold can be reconstructed by iteratively merging these hyperplanes. A Minimum Spanning Tree provides the skeleton needed to traverse the manifold so that the local hyperplanes can be used to build a global, locally stable, embedding. We show state of the art results when compared against existing manifold learning approaches using benchmark synthetic data. We also show how our technique can be used on real world image data.

In this paper we present a novel image classification methodology based on texture signature. The approach consists of four distinct steps: 1) feature extraction from texture images without using any prior knowledge (e.g. viewpoint, illumination condition); 2) textures are modelled as texture signatures; 3) model selection and reduction is used to remove noise and outliers; 4) texture image classification using Columbia-Utrecht (CUReT) texture database. Classification performance was 91% accuracy for all 61 materials (2806 images) present in the CUReT database. The results are compared with texton based classification and effects due to various parameter settings are discussed.

Manifold learning is a widely used statistical tool which reduces the dimensionality of a data set while aiming to maintain both local and global properties of the data. We present a novel manifold learning technique which aligns local hyper planes to build a global representation of the data. A Minimum Spanning Tree provides the skeleton needed to traverse the manifold so that the local hyper planes can be merged using parallel projections to build a global hyper plane of the data. We show state of the art results when compared against existing manifold learning algorithm on both artificial and real world image data.

The accuracy of methods for the assessment of mammographic risk analysis is heavily related to breast tissue characteristics. Previous work has demonstrated considerable success in developing an automatic breast tissue classification methodology which overcomes this difficulty. This paper proposes a unified approach for the application of a number of rough and fuzzy-rough set methods to the analysis of mammographic data. Indeed this is the first time that fuzzy-rough approaches have been applied to this particular problem domain. In the unified approach detailed here feature selection methods are employed for dimensionality reduction developed using rough sets and fuzzy-rough sets. A number of classifiers are then used to examine the data reduced by the feature selection approaches and assess the positive impact of these methods on classification accuracy. Additionally, this paper also employs a new fuzzy-rough classifier based on the nearest neighbour classification algorithm. The novel use of such an approach demonstrates its efficiency in improving classification accuracy for mammographic data, as well as considerably removing redundant, irrelevant, and noisy features. This is supported with experimental application to two well-known datasets. The overall result of employing the proposed unified approach is that feature selection can identify only those features which require extraction. This can have the positive effect of increasing the risk assessment accuracy rate whilst additionally reducing the time required for expert scrutiny, which in-turn means the risk analysis process is potentially quicker and involves less screening.

It has been shown that the accuracy of mammographic abnormality detection methods is strongly dependent on the breast tissue characteristics, where a dense breast drastically reduces detection sensitivity. In addition, breast tissue density is widely accepted to be an important risk indicator for the development of breast cancer. Here, we describe the development of an automatic breast tissue classification methodology, which can be summarized in a number of distinct steps: 1) the segmentation of the breast area into fatty versus dense mammographic tissue; 2) the extraction of morphological and texture features from the segmented breast areas; and 3) the use of a Bayesian combination of a number of classifiers. The evaluation, based on a large kappa = 0.81 and 0.67 for the two data sets) between automatic and expert-based Breast Imaging Reporting and Data System mammographic density assessment.

A recent trend in digital mammography is computer-aided diagnosis systems, which are computerised tools designed to assist radiologists. Most of these systems are used for the automatic detection of abnormalities. However, recent studies have shown that their sensitivity is significantly decreased as the density of the breast increases. This dependence is method specific. In this paper we propose a new approach to the classification of mammographic images according to their breast parenchymal density. Our classification uses information extracted from segmentation results and is based on the underlying breast tissue texture. Classification performance was based on a large set of digitised mammograms. Evaluation involves different classifiers and uses a leave-one-out methodology. Results demonstrate the feasibility of estimating breast density using image processing and analysis techniques.

We present a framework for the registration and correspondence of magnetic resonance (MR) (three-dimensional data, 3D, data) and x-ray (two-dimensional data, 2D, data) mammographic images. The robustness of this work relies on the development of a novel method to establish nonlinear correspondence between modalities of different dimensionality, which also represent different physical tissue aspects. The correspondence is based on a 2D–2D matching process, which takes into account features from internal linear structures from both images and a measure of global similarity between modalities (Martí et al., International Journal of Pattern Recognition and Artificial Intelligence, vol. 16, no. 3, pp. 331–340, 2002). The 2D–3D correspondence relies on an intermediate step, which establishes registration between the 2D x-ray image and a projection of the 3D MR data. Initial quantitative and qualitative evaluation results, based on a small data set, are presented that show the validity of the developed approach.

We describe methods for detecting linear structures in mammograms, and for classifying them into anatomical types (vessels, spicules, ducts, etc). Several different detection methods are compared, using realistic synthetic images and receiver operating characteristic (ROC) analysis. There are significant differences (p < 0.001) between the methods, with the best giving an Az value for pixel-level detection of 0.943. We also investigate methods for classifying the detected linear structures into anatomical types, using their cross-sectional profiles, with particular emphasis on recognising the "spicules" and "ducts" associated with some of the more subtle abnormalities. Automatic classification results are compared with expert annotations using ROC analysis, demonstrating useful discrimination between anatomical classes (Az = 0.746). Some of this discrimination relies on simple attributes such as profile width and contrast, but important information is also carried by the shape of the profile (Az = 0.653). The methods presented have potentially wide application in improving the specificity of abnormality detection by exploiting additional anatomical information.

Scale-orientation signature space represents a non-linear transformation from normal grey-level image space. From this point of view there are similarities with other representations such as radial basis functions, Fourier, Mellin and wavelet transforms. Morphological related techniques are used to generate the scale-orientation space. A series of filtering techniques, based on se-lectively removing values in scale-orientation space, have been developed. Transportation across scales, but not orientations, in scale-orientation space is discussed as a way to preserve the base of image structures. Evaluation of the described techniques is based on a set of standard images which con-tain linear structures superimposed on a complex texture background. Both quantitative and qualitative results are presented.

A novel method to obtain point correspondence in pairs of images is presented. Our approach is based on automatically establishing correspondence between linear structures which appear in images using robust features such as orientation, width and curvature extracted from those structures. The extracted points can be used to register sets of images. The potential of the developed approach is demonstrated on mammographic images.

A method to correspond linear structures in mammographic images is pre-sented. Our approach is based on automatically establishing correspondence between linear structures which appear in images using robust features such as orientation, width and curvature extracted from those structures. The re-sulting correspondence is used to track linear structures and regions in mam-mographic images taken at different times. 1

We have developed a novel similarity measure to evaluate image correspondence. Our method is based on the mutual information between images. The main difference to other mutual information approaches is that we incorporate spatial information using grey-level co-occurrence matrices, leading to a more general measurement. We have used this technique to evaluate two registration algorithms (local affine transformation and thin plate splines) applied to a dataset of mammographic images.

There have been several approaches to the classification of texture in images. Most approaches will take certain local attributes or features into account and base the classification on these measures. In here we demonstrate the use of a statistical approach to separate the structure and texture background present in images. Modelling is based on normal images which only contain a texture background. The resulting model is applied to images which contain abnor-mal image structures as well as a normal texture background. Especially for mammographic (and other medical application) this can provide useful in-formation which can be used as a pre-processing tool to obtain the structures present in the image and at the same time get a robust classification of the background texture. 1

In mammography the presence of subtle abnormalities such as stellate patterns and architectual distortions indicates possible malignancy. Radiologists do not always detect these abnormalities in screening mammograms; this has led to interest in computer--aided mammographic interpretation where the radiologist is presented with computer--generated 'prompts' for abnormalities. A first step in this process is the detection of the "orientation", "scale" and "strength" of linear structures in the mammograms. We discuss several generic methods for extracting this information from images and compare their performance using synthetic images intended to simulate the appearance of mammograms. We show significant differences in performance between the different methods. We also show results obtained for real mammograms. 1 Introduction The UK Breast Screening Programme alone generates 1.5 million mammograms per annum. Potential malignancies can be detected from subtle abnormalities in ...