Early diagnosis of cardiovascular diseases is essential for an effective treatment, potentially preventing severe health complications and improving clinical outcomes. Electrocardiogram (ECG) and phonocardiogram (PCG) are cost-effective, noninvasive diagnostic tools providing crucial and complementary information about the heart’s electrical and mechanical activities. This paper presents a novel approach to the assessment of cardiovascular health through the multimodal analysis of simultaneously recorded ECG and PCG signals. Combining multimodal analysis and transfer learning on publicly available data, the most successful multimodal approach achieved an accuracy of 82.79%, a ROC AUC score of 91.26%, and a recall of 93.10% demonstrating the potential of these techniques. This study provides a foundation for future research aimed at enhancing the performance of multimodal cardiac abnormality detection systems.

OBJECTIVE: Murmurs are abnormal heart sounds, identified by experts through cardiac auscultation. The murmur grade, a quantitative measure of the murmur intensity, is strongly correlated with the patient's clinical condition. This work aims to estimate each patient's murmur grade (i.e., absent, soft, loud) from multiple auscultation location phonocardiograms (PCGs) of a large population of pediatric patients from a low-resource rural area. METHODS: The Mel spectrogram representation of each PCG recording is given to an ensemble of 15 convolutional residual neural networks with channel-wise attention mechanisms to classify each PCG recording. The final murmur grade for each patient is derived based on the proposed decision rule and considering all estimated labels for available recordings. The proposed method is cross-validated on a dataset consisting of 3456 PCG recordings from 1007 patients using a stratified ten-fold cross-validation. Additionally, the method was tested on a hidden test set comprised of 1538 PCG recordings from 442 patients. RESULTS: The overall cross-validation performances for patient-level murmur gradings are 86.3% and 81.6% in terms of the unweighted average of sensitivities and F1-scores, respectively. The sensitivities (and F1-scores) for absent, soft, and loud murmurs are 90.7% (93.6%), 75.8% (66.8%), and 92.3% (84.2%), respectively. On the test set, the algorithm achieves an unweighted average of sensitivities of 80.4% and an F1-score of 75.8%. CONCLUSIONS: This study provides a potential approach for algorithmic pre-screening in low-resource settings with relatively high expert screening costs. SIGNIFICANCE: The proposed method represents a significant step beyond detection of murmurs, providing characterization of intensity, which may provide an enhanced classification of clinical outcomes.

This work considers the problem of segmenting heart sounds into their fundamental components. We unify statistical and data-driven solutions by introducing Markov-based Neural Networks (MNNs), a hybrid end-to-end framework that exploits Markov models as statistical inductive biases for an Artificial Neural Network (ANN) discriminator. We show that an MNN leveraging a simple one-dimensional Convolutional ANN significantly outperforms two recent purely data-driven solutions for this task in two publicly available datasets: PhysioNet 2016 (Sensitivity: 0.947 ±0.02; Positive Predictive Value : 0.937 ±0.025) and the CirCor DigiScope 2022 (Sensitivity: 0.950 ±0.008; Positive Predictive Value: 0.943 ±0.012). We also propose a novel gradient-based unsupervised learning algorithm that effectively makes the MNN adaptive to unseen datum sampled from unknown distributions. We perform a cross dataset analysis and show that an MNN pre-trained in the CirCor DigiScope 2022 can benefit from an average improvement of 3.90% Positive Predictive Value on unseen observations from the PhysioNet 2016 dataset using this method.

Cardiac auscultation is one of the most cost-effective techniques used to detect and identify many heart conditions. Computer-assisted decision systems based on auscultation can support physicians in their decisions. Unfortunately, the application of such systems in clinical trials is still minimal since most of them only aim to detect the presence of extra or abnormal waves in the phonocardiogram signal, i.e., only a binary ground truth variable (normal vs abnormal) is provided. This is mainly due to the lack of large publicly available datasets, where a more detailed description of such abnormal waves (e.g., cardiac murmurs) exists. To pave the way to more effective research on healthcare recommendation systems based on auscultation, our team has prepared the currently largest pediatric heart sound dataset. A total of 5282 recordings have been collected from the four main auscultation locations of 1568 patients, in the process, 215780 heart sounds have been manually annotated. Furthermore, and for the first time, each cardiac murmur has been manually annotated by an expert annotator according to its timing, shape, pitch, grading, and quality. In addition, the auscultation locations where the murmur is present were identified as well as the auscultation location where the murmur is detected more intensively. Such detailed description for a relatively large number of heart sounds may pave the way for new machine learning algorithms with a real-world application for the detection and analysis of murmur waves for diagnostic purposes.

Heart sounds are difficult to interpret due to events with very short temporal onset between them (tens of milliseconds) and dominant frequencies that are out of the human audible spectrum. Computer-assisted decision systems may help but they require robust signal processing algorithms. In this paper, we propose a new algorithm for heart sound segmentation using a hidden semi-Markov model. The proposed algorithm infers more suitable sojourn time parameters than those currently suggested by the state of the art, through a maximum likelihood approach. We test our approach over three different datasets, including the publicly available PhysioNet and Pascal datasets. We also release a pediatric dataset composed of 29 heart sounds. In contrast with any other dataset available online, the annotations of the heart sounds in the released dataset contain information about the beginning and the ending of each heart sound event. Annotations were made by two cardiopulmonologists. The proposed algorithm is compared with the current state of the art. The results show a significant increase in segmentation performance, regardless the dataset or the methodology presented. For example, when using the PhysioNet dataset to train and to evaluate the HSMMs, our algorithm achieved average an F-score of [Formula: see text] compared to [Formula: see text] achieved by the algorithm described in [D.B. Springer, L. Tarassenko, and G. D. Clifford, "Logistic regressionHSMM-based heart sound segmentation," IEEE Transactions on Biomedical Engineering, vol. 63, no. 4, pp. 822-832, 2016]. In this sense, the proposed approach to adapt sojourn time parameters represents an effective solution for heart sound segmentation problems, even when the training data does not perfectly express the variability of the testing data.

This paper studies the use of deep convolutional neural networks to segment heart sounds into their main components. The proposed methods are based on the adoption of a deep convolutional neural network architecture, which is inspired by similar approaches used for image segmentation. Different temporal modeling schemes are applied to the output of the proposed neural network, which induce the output state sequence to be consistent with the natural sequence of states within a heart sound signal (S1, systole, S2, diastole). In particular, convolutional neural networks are used in conjunction with underlying hidden Markov models and hidden semi-Markov models to infer emission distributions. The proposed approaches are tested on heart sound signals from the publicly available PhysioNet dataset, and they are shown to outperform current state-of-the-art segmentation methods by achieving an average sensitivity of 93.9% and an average positive predictive value of 94% in detecting S1 and S2 sounds.

This paper proposes a computer assisted diagnostic (CAD) system for the detection of melanoma in dermoscopy images. Clinical findings have concluded that in case of melanoma, the lesion borders exhibit differential structures such as pigment networks and streaks as opposed to normal skin spots, which have smoother borders. We aim to validate these findings by performing segmentation of the skin lesions followed by an extraction of the peripheral region of the lesion that is subjected to feature extraction and classification for detecting melanoma. For segmentation, we propose a novel active contours based method that takes an initial lesion contour followed by the usage of Kullback-Leibler divergence between the lesion and skin to fit a curve precisely to the lesion boundaries. After segmentation of the lesion, its periphery is extracted to detect melanoma using image features that are based on local binary patterns. For validation of our algorithms, we have used the publicly available PH dermoscopy dataset. An extensive experimental analysis reveals two important findings: 1). The proposed segmentation method mimics the ground truth data accurately, outperforming the other methods that have been used for comparison purposes, and 2). The most significant melanoma characteristics in the lesion actually lie on the lesion periphery.

Rheumatic heart disease can result from repeated episodes of acute rheumatic fever, which damages the heart valves and reduces their functionality. Early manifestations of heart valve damage are visible in echocardiography in the form of valve thickening, shape changing and mobility reduction. The quantification of these features is important for a precise diagnosis and it is the main motivation for this work. The first step to make this quantification is to accurately identify and track the anterior mitral leaflet throughout the cardiac cycle. An accurate segmentation and tracking with minimum user interaction is still an open problem in literature due to low image quality, speckle noise, signal dropout and nonrigid deformations. In this work, we propose a novel approach for the identification of the anterior mitral valve leaflet in all frames. The method requires a single user-specified point on the posterior wall of the aorta as input, in the first frame. The echocardiography videos are converted into a new image space, the Virtual M-mode, which samples the original echocardiography image over automatically estimated scanning lines. This new image space not only provides the motion pattern of the posterior wall of the aorta, the anterior wall of the aorta and the posterior wall of the left atrium, but also provides the location of the structures in each frame. The location information is then used to initialize the localized active contours, followed by segmenting the anterior mitral leaflet. Results shown that the new image space has robustly identified the anterior mitral valve leaflet, without any failure. The median modified Hausdorff distance error of the proposed method was 2.3 mm, with a recall of 0.94.

This work proposes a new approach for the segmentation of the extensor tendon in ultrasound images of the second metacarpophalangeal joint (MCPJ). The MCPJ is known to be frequently involved in early stages of rheumatic diseases like rheumatoid arthritis. The early detection and follow up of these diseases is important to start and adapt the treatments properly and, in that way, preventing irreversible damage of the joints. This work relies on an active contours framework, preceded by a phase symmetry preprocessing and with prior knowledge energies, to automatically identify the extensor tendon. Active contours methods are widely used in ultrasound images because of their robustness to speckle noise and ability to join unconnected smaller regions into a coherent shape. The tendon is formulated as a line so open ended active contours were used. Phase symmetry highlights the tendon, by setting a proper scale range and angle span. The distance between structures and the tendon slope were also included to enforce the model based on anatomical characteristics. And finally, the concavity measures were used because, given the anatomy of the finger, we know that the tendon line should have less than two concavities. To solve the active contours energy minimization a genetic algorithm approach was used. Several energy metric configurations were compared using the modified Hausdorff distance and results showed that this segmentation is not only possible, but exhibits errors smaller than 0.5 mm with a confidence of 95% with the phase symmetry preprocessing and energies based on the line neighborhood, area ratio, slope, and concavity measurements.

Automatic and simultaneous electrocardiogram (ECG) and phonocardiogram (PCG) segmentation is a good example of current challenges when designing multi-channel decision support systems for healthcare. In this paper, we implemented and tested a Montazeri coupled hidden Markov model (CHMM), where two HMM's cooperate to recreate the “true” state sequence. To evaluate its performance, we tested different settings (two fully connected and two partially connected channels) on a real dataset annotated by an expert. The fully connected model achieved 71% of positive predictability (P+) on the ECG channel and 67% of P+on the PCG channel. The partially connected model achieved 90% of P+on the ECG channel and 80% of P+in the PCG channel. These results validate the potential of our approach for real world multichannel application systems.

The design of computer-assisted decision (CAD) systems for different biomedical imaging scenarios is a challenging task in computer vision. Sometimes, this challenge can be attributed to the image acquisition mechanisms since the lack of control on the cameras can create different visualizations of the same imaging site under different rotation, scaling, and illumination parameters, with a requirement to get a consistent diagnosis by the CAD systems. Moreover, the images acquired from different sites have specific colors, making the use of standard color spaces highly redundant. In this paper, we propose to tackle these issues by introducing novel region-based texture, and color descriptors. The proposed texture features are based on the usage of analytic Gabor filters (for compensation of illumination variations) followed by the calculation of first- and second-order statistics of the filter responses and making them invariant using some trivial mathematical operators. The proposed color features are obtained by compensating for the illumination variations in the images using homomorphic filtering followed by a bag-of-words approach to obtain the most typical colors in the images. The proposed features are used for the identification of cancer in images from two distinct imaging modalities, i.e., gastroenterology and dermoscopy . Experiments demonstrate that the proposed descriptors compares favorably to several other state-of-the-art methods, elucidating on the effectiveness of adapted features for image characterization.

We describe the architecture, problems and lessons learned from building a Health Kiosk from commercial, off-the shelf Personal Health Devices and a computer with a touch-screen interface. The kiosk is used autonomously by patients to measure vital data prior to a consultation, in the scope of a population screening, or for routinely monitoring. The prototype was tested in multiple events with different user groups, aided by observer reporting and user questionnaires to assess problems and difficulties with the interface and device usage.

In this paper, a novel subject-adaptable heartbeat classification model is presented, in order to address the significant interperson variations in ECG signals. A multiview learning approach is proposed to automate subject adaptation using a small amount of unlabeled personal data, without requiring manual labeling. The designed subject-customized models consist of two models, namely, general classification model and specific classification model. The general model is trained using similar subjects out of a population dataset, where a pattern matching based algorithm is developed to select the subjects that are "similar" to the particular test subject for model training. In contrast, the specific model is trained mainly on a small amount of high-confidence personal dataset, resulting from multiview-based learning. The learned general model represents the population knowledge, providing an interperson perspective for classification, while the specific model corresponds to the specific knowledge of the subject, offering an intraperson perspective for classification. The two models supplement each other and are combined to achieve improved personalized ECG analysis. The proposed methods have been validated on the MIT-BIH Arrhythmia Database, yielding an average classification accuracy of 99.4% for ventricular ectopic beat class and 98.3% for supraventricular ectopic beat class, which corresponds to a significant improvement over other published results.

Cardiovascular variability and homeostasis control in response to precise noxious stimuli and different analgesic doses were analyzed. Cardiovascular variability was altered in response to noxious stimulation, and amplitude responses varied in a dose-dependent manner with the analgesic. Responses were more pronounced to laringoscopy/intubation, when compared to tetanic and incision stimuli. Homeostasis control was also altered in response to stimulation, demonstrating that dynamic cardiovascular control relations are modified in response to precise noxious stimuli and anesthetic drugs.

We evaluated a population of 7199 children between 2 and 19 years old to study the relations between the observed demographic and physiological features in the occurrence of a pathological/non-pathological heart condition. The data was collected at the Real Hospital Português, Pernambuco, Brazil. We performed a feature importance study, with the aim of categorizing the most relevant variables, indicative of abnormalities. Results show that second heart sound, weight, heart rate, height and secondary reason for consultation are important features, but not nearly as decisive as the presence of heart murmurs. Quantitatively speaking, systolic murmurs and a hyperphonetic second heart sound increase the odds of having a pathology by a factor of 320 and 6, respectively.

Segmentation is a vital step for pattern recognition systems used in in-body imaging scenarios. In this paper we compare the performance of three popular segmentation algorithms (mean shift, normalized cuts, level-sets) when applied to two distinct in-body imaging scenarios: chromoen-doscopy and narrow-band imaging. Observation shows that the model-based algorithm did not perform well, when compared to its segmentation by clustering alternatives. Normalized cuts obtained the best performance although future work hints that texture similarity should be further explored in order to increase segmentation performance in this type of scenarios.

In this work we study the impact of a set of bag-of-features strategies for the recognition of cancer in gastroen-terology images. By using the SIFT descriptor, we analyzed the importance and performance impact of term weighting functions for the construction of visual vocabularies. Further analyzes were conducted in order to ascertain the robustness of multiclass decomposition rules for Support Vector Machines with different kernels. Our study was extended by tailoring a decomposition rule that explores prior knowledge according the four grades of the Singh taxonomy (SDR). We found that SDR coupled with a frequency term weight function attained the best overall results (80%) when trained with an intersection kernel. It also outperformed standard decomposition rules when using a χ2kernel and attained competitive performances with a linear kernel.

Classification of texture images, especially in cases where the images are subjected to arbitrary rotation and scale changes due to dynamic imaging conditions is a challenging problem in computer vision. This paper proposes a novel methodology to obtain rotation and scale invariant texture features from the images. The feature extraction for a given image involves the calculation of the averages of Gabor filter responses at various scales and orientations. For rotation and scaling of images, these averages indicate the respective shifts in the features. These shifts are normalized by doing summations of Gabor responses across scales and then taking the magnitude of Discrete Fourier Transforms across the resulting features and vice versa thus giving us scale and rotation invariant texture features. The proposed features are used for identifying cancer in the vital stained magnification endoscopy images. Experiments demonstrate the superiority of the proposed feature set over several other state-of-the-art texture feature extraction methods with around 90% classification accuracy for identifying cancer in gastroenterology images.

The DigiScope project aims at developing a digitally enhanced stethoscope capable of using state of the art technology in order to help physicians in their daily medical routine. One of the main tasks of DigiScope is to build a repository of auscultations (sound and medical related data). In this work, we present a preliminary analysis and study of the first auscultations performed on children of a Brazilian hospital. Results indicate that classifiers can be obtained that distinguish reasonably well patients with cardiac pathologies from those that do not have pathologies.

In this paper, we consider the problem of separating a set of independent components when only one movable sensor is available to record the mixtures. We propose to exploit the quasi-periodicity of the heart signals to transform the signal from this one moving sensor, into a set of measurements, as if from a virtual array of sensors. We then use ICA to perform source separation. We show that this technique can be applied to heart sounds and to electrocardiograms.

In-body imaging technologies such as vital-stained magnification endoscopy pose novel image processing challenges to computer-assisted decision systems given their unique visual characteristics such as reduced color spaces and natural textures. In this paper we will show the potential of using adapted color features combined with local binary patterns, a texture descriptor that has exhibited good adaptation to natural images, for classifying gastric regions into three groups: normal, pre-cancer and cancer lesions. Results exhibit 91% accuracy, confirming that specific research for in-body imaging could be the key for future computer assisted decision systems for medicine.

Endoscopic capsule is a recent medical technology with important clinical benefits but suffering from a practical handicap: long exam annotation times. This paper proposes and compares two approaches (Bayesian and support vector machines) that can be used to segment the gastrointestinal tract into its four major topographic areas, allowing the automatic estimation of the clinically relevant gastric and intestinal sections and corresponding transit times. According to medical specialists, this can reduce exam annotation times by up to 12% (15 min). This automatic tool has been integrated into our CapView annotation software that is currently being used by three medical institutions.

The endoscopic capsule is a recent medical technology with important clinical benefits but suffering from a practical handicap: long exam annotation times. This paper shows how support vector machines can be used to segment the gastrointestinal tract into its four major topographic areas, allowing the automatic estimation of the clinically relevant gastric and intestinal transit times. According to medical specialists, this can reduce exam annotation times by up to 12%

Recent advances in miniaturization led to the development of what is now called the endoscopic capsule. This small device is swallowed by a patient and films the whole gastrointestinal tract, allowing the detection of abnormalities. Currently, a doctor typically needs up to two hours to analyze a full exam, so automation is desirable. This paper presents a methodology for measuring the potential of selected visual MPEG-7 descriptors for the task of specific medical event detection such as blood, ulcers. Experiments show that the best results are obtained by the Scalable Color and Homogenous Texture descriptors, especially if only relevant coefficients are used.

Video encoding standards, namely MPEG-2, store large amounts of information obtained for compression purposes that can be accessed with minimal decoding. This paper shows that, with proper filtering of motion vectors and DCT coefficients, accurate segmentation results can be achieved by combining both reliable motion estimation and background subtraction. We further present a fine segmentation step that exploits specific blob characteristics to reduce segmentation noise and solve some occlusion problems. Examples using real videos from underground station CCTV cameras show that compressed domain information can be the key for successful surveillance applications where very fast algorithms with high accuracy are required.