Deep learning models often struggle with class imbalance and low-resolution medical images, where critical spatial details and minority-class features are underrepresented. We introduce the Adaptive Distribution-aware Vision Transformer (AdaptiveViT), a novel hybrid CNN-Transformer architecture that unifies fine-grained local feature extraction with global contextual modelling. AdaptiveViT incorporates a distribution-aware modulation mechanism that adaptively adjusts feature emphasis according to the severity of class imbalance. In addition, a Distribution-aware Adaptive (DA) Loss incorporates the dataset imbalance ratio into an adaptive focusing scheme, enhancing minority-class sensitivity. Experiments on five skin lesion datasets with varying image resolutions and imbalance ratios (as high as 1:10 for melanoma versus non-melanoma) demonstrate that AdaptiveViT consistently outperforms state-of-the-art CNN, Transformer, and hybrid baselines in F1 and AUC, while maintaining stable convergence across imbalance levels. Validation on gastrointestinal endoscopy datasets further demonstrates AdaptiveViT's domain-agnostic generalisation beyond skin lesion data, which share similar imbalance characteristics. All experiments are conducted using patient-disjoint splits, with a threshold-free evaluation protocol to ensure fair, unbiased, and clinically reliable comparisons. Overall, AdaptiveViT establishes a hybrid framework for medical image classification under class imbalance and image-resolution variability. The code is available at https://github.com/mmu-dermatology-research/AdaptiveViT.

The development of driver monitoring systems (DMS) has emerged as a pivotal advancement in automotive safety to monitor driver behaviors and prevent road accidents. Driver emotions are one of the critical factors that affect driving safety. While numerous studies have explored facial macro-expressions (FMaEs) of drivers, these facial expressions can be consciously controlled, allowing individuals to conceal their true negative emotions. However, inaccurate capturing of true emotions can lead to severe accidents. In this paper, we focus on facial micro-expressions (FMEs), which are brief and involuntary facial movements that reveal deeper insights into genuine emotions. Thereby, they provide a more accurate measure of a driver's emotional condition, and are crucial for assessing driver readiness and safety. To facilitate FME studies of drivers, we analyzed driver videos from an affective driver-pedestrian interaction experiment, providing corresponding facial action units and FME annotations, and built a real driver FME dataset. By analyzing the driver data, we found that there are more FMEs when drivers respond to negative stimuli compared with positive stimuli, and there are more FMaEs when drivers respond to positive stimuli compared with negative stimuli. To protect driver identities while maintaining the integrity of the facial movements, we released a synthesized driver FME dataset. Furthermore, we benchmarked deep learning-based architectures on the real and synthetic driver FME datasets, uncovering the challenges inherent in recognizing FMEs in real-world driving scenarios.

Facial micro-expressions (MEs) are involuntary movements of the face that occur spontaneously when a person experiences an emotion but attempts to suppress or repress the facial expression, typically found in a high-stakes environment. In recent years, substantial advancements have been made in the areas of ME recognition, spotting, and generation. However, conventional approaches that treat spotting and recognition as separate tasks are suboptimal, particularly for analyzing long-duration videos in realistic settings. Concurrently, the emergence of multimodal large language models (MLLMs) and large vision-language models (LVLMs) offers promising new avenues for enhancing ME analysis through their powerful multimodal reasoning capabilities. The ME grand challenge (MEGC) 2025 introduces two tasks that reflect these evolving research directions: (1) ME spot-then-recognize (ME-STR), which integrates ME spotting and subsequent recognition in a unified sequential pipeline; and (2) ME visual question answering (ME-VQA), which explores ME understanding through visual question answering, leveraging MLLMs or LVLMs to address diverse question types related to MEs. All participating algorithms are required to run on this test set and submit their results on a leaderboard. More details are available at https://megc2025.github.io.

Abstract This work conducts a review of all chronic wound segmentation deep learning studies meeting specific criteria that have been published since research first started in this domain 8 years ago (2015–2023). Management of chronic wounds represents a serious ongoing concern for hospitals and outpatient clinics world-wide. There is a clear need for technological interventions using deep learning approaches that could have a potential significant impact in the automated monitoring of such wounds. We review the existing literature and perform R-squared statistical analysis to form a fresh understanding of the field to gain deeper insights into the issues that are presenting obstacles to research progress. Our findings show a negative correlation between small test set size and test metrics (Dice similarity coefficient and mean intersection over union), indicating smaller test sets are associated with higher test metrics. We also identify other major hurdles in the field, such as a lack of data understanding, a lack of data availability, and a lack of research transparency. The focus of this body of work is to increase understanding of the underlying issues that have pervaded in deep learning chronic wound research. A clear presentation of findings in this work can be used by researchers as a guide to avoiding common pitfalls, and to advance research knowledge.

Facial micro-expressions (MEs) are involuntary spontaneous movements of the face that typically appear in high-stakes situations where a person attempts to conceal a certain emotion from being known. A decade after the inception of the widely used CASME II and SMIC datasets, research in computational analysis of MEs has now advanced toward new pathways, exploring problems crucial to model generalization and real-world practicality. It is often challenging to design robust algorithms or models for spotting micro-expressions due to the high variability across diverse cultural backgrounds. Also, treating spotting and recognition as separate tasks is undesirable when handling long-spanning videos under realistic settings. This Grand Challenge comprises two distinct tracks: the Cross-Cultural Spotting (CCS) track, and the Spot-Then-Recognize (STR) track. All participating solutions submitted their results to a leaderboard, and several submissions performed well surpassing their respective baseline results. More details are available at: https://megc2024.github.io.

Monitoring the healing progress of diabetic foot ulcers is a challenging process. Accurate segmentation of foot ulcers can help podiatrists to quantitatively measure the size of wound regions to assist prediction of healing status. The main challenge in this field is the lack of publicly available manual delineation, which can be time consuming and laborious. Recently, methods based on deep learning have shown excellent results in automatic segmentation of medical images, however, they require large-scale datasets for training, and there is limited consensus on which methods perform the best. The 2022 Diabetic Foot Ulcers segmentation challenge was held in conjunction with the 2022 International Conference on Medical Image Computing and Computer Assisted Intervention, which sought to address these issues and stimulate progress in this research domain. A training set of 2000 images exhibiting diabetic foot ulcers was released with corresponding segmentation ground truth masks. Of the 72 (approved) requests from 47 countries, 26 teams used this data to develop fully automated systems to predict the true segmentation masks on a test set of 2000 images, with the corresponding ground truth segmentation masks kept private. Predictions from participating teams were scored and ranked according to their average Dice similarity coefficient of the ground truth masks and prediction masks. The winning team achieved a Dice of 0.7287 for diabetic foot ulcer segmentation. This challenge has now entered a live leaderboard stage where it serves as a challenging benchmark for diabetic foot ulcer segmentation.

International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multicenter study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), image preprocessing (97%), data curation (79%), and post-processing (66%). The “typical” lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work.

Facial micro-expressions (MEs) are involuntary movements of the face that occur spontaneously when a person experiences an emotion but attempts to suppress or repress the facial expression, typically found in a high-stakes environment. Unfortunately, the small sample problem severely limits the automation of ME analysis. Furthermore, due to the weak and transient nature of MEs, it is difficult for models to distinguish it from other types of facial actions. Therefore, ME in long videos is a challenging task, and the current performance cannot meet the practical application requirements. Addressing these issues, this challenge focuses on ME and the macro-expression (MaE) spotting task. This year, in order to evaluate algorithms' performance more fairly, based on CAS(ME)2, SAMM Long Videos, SMIC-E-long, CAS(ME)3 and 4DME, we build an unseen cross-cultural long-video test set. All participating algorithms are required to run on this test set and submit their results on a leaderboard with a baseline result.

Micro-expressions are facial movements that are extremely short and not easily detected, which often reflect the genuine emotions of individuals. Micro-expressions are important cues for understanding real human emotions and can be used for non-contact, non-perceptual deception detection, or abnormal emotion recognition. It has broad application prospects in national security, judicial practice, health prevention, and clinical practice. However, micro-expression feature extraction and learning are highly challenging because they are typically short in duration, low intensity, and have local facial asymmetry. In addition, the intelligent micro-expression analysis combined with deep learning technology is also plagued by the problem of relatively small data samples. Not only is micro-expression elicitation very difficult, micro-expression annotation is also very time-consuming and laborious. More importantly, the micro-expression generation mechanism is not yet clear, which shackles the application of micro-expressions in real scenarios. FME'23 is the inaugural workshop in this area of research, with the aim of promoting interactions between researchers and scholars from within this niche area of research. This year we hope to discuss the growing ethical conversations when using face data, and how we can come to a consensus on micro-expression standards within affective computing.

Various deep learning methods have been proposed to segment breast lesions from ultrasound images. However, similar intensity distributions, variable tumor morphologies and blurred boundaries present challenges for breast lesions segmentation, especially for malignant tumors with irregular shapes. Considering the complexity of ultrasound images, we develop an adaptive attention U-net (AAU-net) to segment breast lesions automatically and stably from ultrasound images. Specifically, we introduce a hybrid adaptive attention module (HAAM), which mainly consists of a channel self-attention block and a spatial self-attention block, to replace the traditional convolution operation. Compared with the conventional convolution operation, the design of the hybrid adaptive attention module can help us capture more features under different receptive fields. Different from existing attention mechanisms, the HAAM module can guide the network to adaptively select more robust representation in channel and space dimensions to cope with more complex breast lesions segmentation. Extensive experiments with several state-of-the-art deep learning segmentation methods on three public breast ultrasound datasets show that our method has better performance on breast lesions segmentation. Furthermore, robustness analysis and external experiments demonstrate that our proposed AAU-net has better generalization performance in the breast lesion segmentation. Moreover, the HAAM module can be flexibly applied to existing network frameworks. The source code is available on https://github.com/CGPxy/AAU-net.

Micro-expressions are facial movements that are extremely short and not easily detected, which often reflect the genuine emotions of individuals. Micro-expressions are important cues for understanding real human emotions and can be used for non-contact non-perceptual deception detection, or abnormal emotion recognition. It has broad application prospects in national security, judicial practice, health prevention, clinical practice, etc. However, micro-expression feature extraction and learning are highly challenging because micro-expressions have the characteristics of short duration, low intensity, and local asymmetry. In addition, the intelligent micro-expression analysis combined with deep learning technology is also plagued by the problem of small samples. Not only is micro-expression elicitation very difficult, micro-expression annotation is also very time-consuming and laborious. More importantly, the micro-expression generation mechanism is not yet clear, which shackles the application of micro-expressions in real scenarios. FME'22 is the inaugural workshop in this area of research, with the aim of promoting interactions between researchers and scholars from within this niche area of research and also including those from broader, general areas of expression and psychology research. The complete FME'22 workshop proceedings are available at: https://dl.acm.org/doi/proceedings/10.1145/3552465.

Facial micro-expressions (MEs) are involuntary movements of the face that occur spontaneously when a person experiences an emotion but attempts to suppress or repress the facial expression, typically found in a high-stakes environment. Unfortunately, the small sample problem severely limits the automation of ME analysis. Furthermore, due to the brief and subtle nature of ME, ME spotting is a challenging task, and the performance is still not satisfactory yet. This challenge focuses on two tasks, i.e., the micro- and macro-expression spotting task, and the ME Generation task.

Facial expression spotting is the preliminary step for micro- and macro-expression analysis. The task of reliably spotting such expressions in video sequences is currently unsolved. Current best systems depend upon optical flow methods to extract regional motion features, before categorisation of that motion into a specific class of facial movement. Optical flow is susceptible to drift error, which introduces a serious problem for motions with long-term dependencies, such as high frame-rate macro-expression. We propose a purely deep learning solution which, rather than tracking frame differential motion, compares via a convolutional model, each frame with two temporally local reference frames. Reference frames are sampled according to calculated micro- and macro-expression duration. As baseline for MEGC2021 using leave-one-subject-out evaluation method, we show that our solution performed better in a high frame-rate (200 fps) SAMM long videos dataset (SAMM-LV) than a low frame-rate (30 fps) (CAS(ME)2) dataset. We introduce a new unseen dataset for MEGC2022 challenge (MEGC2022-testSet) and achieves F1-Score of 0.1531 as baseline result.

The International Skin Imaging Collaboration (ISIC) datasets have become a leading repository for researchers in machine learning for medical image analysis, especially in the field of skin cancer detection and malignancy assessment. They contain tens of thousands of dermoscopic photographs together with gold-standard lesion diagnosis metadata. The associated yearly challenges have resulted in major contributions to the field, with papers reporting measures well in excess of human experts. Skin cancers can be divided into two major groups - melanoma and non-melanoma. Although less prevalent, melanoma is considered to be more serious as it can quickly spread to other organs if not treated at an early stage. In this paper, we summarise the usage of the ISIC dataset images and present an analysis of yearly releases over a period of 2016 - 2020. Our analysis found a significant number of duplicate images, both within and between the datasets. Additionally, we also noted duplicates spread across testing and training sets. Due to these irregularities, we propose a duplicate removal strategy and recommend a curated dataset for researchers to use when working on ISIC datasets. Given that ISIC 2020 focused on melanoma classification, we conduct experiments to provide benchmark results on the ISIC 2020 test set, with additional analysis on the smaller ISIC 2017 test set. Testing was completed following the application of our duplicate removal strategy and an additional data balancing step. As a result of removing 14,310 duplicate images from the training set, our benchmark results show good levels of melanoma prediction with an AUC of 0.80 for the best performing model. As our aim was not to maximise network performance, we did not include additional steps in our experiments. Finally, we provide recommendations for future research by highlighting irregularities that may present research challenges. A list of image files with reference to the original ISIC dataset sources for the recommended curated training set will be shared on our GitHub repository (available at www.github.com/mmu-dermatology-research/isic_duplicate_removal_strategy).

Facial micro-expressions (FMEs) are involuntary facial movements that occur spontaneously when a person experiences an emotion but tries to suppress or repress the facial expression and usually occur in high-risk situations. Thus, FMEs are very short in duration, an important feature that distinguishes them from ordinary facial expressions. And MEs are considered to be one of the most valuable cues for complex human emotion understanding and lie detection. Since 2014, the computational analysis and automation of MEs have been an emerging area of face research. The workshop will explore various dimensions of the human mind through emotion understanding and FME analysis, as well as extended research based on multi modal approaches.

Facial image inpainting is a problem that is widely studied, and in recent years the introduction of Generative Adversarial Networks, has led to improvements in the field. Unfortunately some issues persists, in particular when blending the missing pixels with the visible ones. We address the problem by proposing a Wasserstein GAN combined with a new reverse mask operator, namely Reverse Masking Network (R-MNet), a perceptual adversarial network for image inpainting. The reverse mask operator transfers the reverse masked image to the end of the encoder-decoder network leaving only valid pixels to be inpainted. Additionally, we propose a new loss function computed in feature space to target only valid pixels combined with adversarial training. These then capture data distributions and generate images similar to those in the training data with achieved realism (realistic and coherent) on the output images. We evaluate our method on publicly available dataset, and compare with state-of-the-art methods. We show that our method is able to generalize to high-resolution inpainting task, and further show more realistic outputs that are plausible to the human visual system when compared with the state-of-the-art methods. https://github.com/Jireh-Jam/R-MNet-Inpainting-keras

This paper presents baseline results for the Third Facial Micro-Expression Grand Challenge (MEGC 2020). Both macro-and micro-expression intervals in CAS(ME)2and SAMM Long Videos are spotted by employing the method of Main Directional Maximal Difference Analysis (MDMD). The MDMD method uses the magnitude maximal difference in the main direction of optical flow features to spot facial movements. The single-frame prediction results of the original MDMD method are post-processed into reasonable video intervals. The metric F1-scores of baseline results are evaluated: for CAS(ME)2, the F1-scores are 0.1196 and 0.0082 for macro-and micro-expressions respectively, and the overall F1-score is 0.0376; for SAMM Long Videos, the F1-scores are 0.0629 and 0.0364 for macro-and micro-expressions respectively, and the overall F1-score is 0.0445. The baseline project codes are publicly available at https://github.com/HeyingGithub/ Baseline-project-for-MEGC2020_spotting.

The recent emergence of automatic facial micro-expression analysis has attracted a lot of attention in the last five years. Compared to the advances made in micro-expression recognition, the task of micro-expression spotting from long videos is tremendously in need of more effective methods. This paper summarises the 3rd Facial Micro-Expression Grand Challenge (MEGC 2020) held in conjunction with the 15th IEEE Conference on Automatic Face and Gesture Recognition (FG) 2020. In this workshop, we propose a new challenge of spotting both macro- and micro-expressions from long videos, to spur the community to develop new techniques for micro-expression spotting and also to extend facial micro-expression analysis to more complex real-world scenarios where micro-expressions are likely to be intertwined among normal expressions. In this paper, we outline the evaluation protocols for the challenge task, and describe the datasets involved. Then, we summarize the methods from the accepted challenge papers, present the comparison and analysis of results, as well as future directions.

Facial micro-expression can be characterized by its short duration and subtle movements. In facial micro-expression recognition, these subtle movements require more specific feature descriptors due to only a few parts of the face produce information that helps us to recognize micro-expressions. Over the past decade, researchers designed different Region of Interests (ROIs) to study specific face regions in micro-expressions recognition. To further study this aspect, we proposed a region-based method with an adaptive mask for facial micro-expression recognition. Based on the most frequent Action Units on the two publicly available datasets, i.e. CASME II and SAMM, 14 ROIs are defined where the adaptive mask is created by calculating the optical flow after Gaussian smoothing. Further, LBP-TOP features are extracted from each ROIs and Sequential Minimal Optimization is used to classify the micro-expressions. When evaluating our proposed method on CASME II and SAMM, we achieved the accuracy of 68.2% and 56.1%. In terms of F1-Score, our proposed method achieved 0.57 on CASMEII and the best performance of 0.50 on SAMM.

With the growth of popularity of facial micro-expressions in recent years, the demand for long videos with micro- and macro-expressions remains high. Extended from SAMM, a micro-expressions dataset released in 2016, this paper presents SAMM Long Videos dataset for spontaneous micro- and macro-expressions recognition and spotting. SAMM Long Videos dataset consists of 147 long videos with 343 macro-expressions and 159 micro-expressions. The dataset is FACS-coded with detailed Action Units (AUs). We compare our dataset with Chinese Academy of Sciences Macro-Expressions and Micro-Expressions (CAS(ME)2) dataset, which is the only available fully annotated dataset with micro- and macro-expressions. Furthermore, we preprocess the long videos using OpenFace, which includes face alignment and detection of facial AUs. We conduct facial expression spotting using this dataset and compare it with the baseline of MEGC III. Our spotting method outperformed the baseline result with F1-score of 0.3299.

This paper presents two methods for the first Micro-Expression Spotting Challenge 2019 by evaluating local temporal pattern (LTP) and local binary pattern (LBP) on two most recent databases, i.e. SAMM and CAS(ME)2. First we propose LTP-ML method as the baseline results for the challenge and then we compare the results with the LBP-χ2-distance method. The LTP patterns are extracted by applying PCA in a temporal window on several facial local regions. The micro-expression sequences are then spotted by a local classification of LTP and a global fusion. The LBP-χ2-distance method is to compare the feature difference by calculating χ2distance of LBP in a time window, the facial movements are then detected with a threshold. The performance is evaluated by Leave-One-Subject-Out cross validation. The overlap frames are used to determine the True Positives and the metric F1-score is used to compare the spotting performance of the databases. The F1-score of LTP-ML result for SAMM and CAS(ME)2are 0.0316 and 0.0179, respectively. The results show our proposed LTP-ML method outperformed LBP-χ2-distance method in terms of F1-score on both databases.

Automatic facial micro-expression (ME) analysis is a growing field of research that has gained much attention in the last five years. With many recent works testing on limited data, there is a need to spur better approaches that are both robust and effective. This paper summarises the 2nd Facial Micro-Expression Grand Challenge (MEGC 2019) held in conjunction with the 14th IEEE Conference on Automatic Face and Gesture Recognition (FG) 2019. In this workshop, we proposed challenges for two micro-expression (ME) tasks- spotting and recognition, with the aim of encouraging rigorous evaluation and development of new robust techniques that can accommodate data captured across a variety of settings. In this paper, we outline the evaluation protocols for the two challenge tasks, the datasets involved, and an analysis of the best performing works from the participating teams, together with a summary of results. Finally, we highlight some possible future directions.

We address two issues for tackling face recognition across pose, one is disentangled representation learning and the other is training data augmentation. To have better training properties, we propose the Representation-Learning Wasserstein-GAN (RL-WGAN) with three component networks for learning the disentangled facial representation. As the learning based on imbalanced data often leads to biased estimation, we proposed a data augmentation scheme that exploits the 3D Morphable Model (3DMM) for generating faces of desired poses. The RL-WGAN and the data augmentation are verified in the experiments with benchmark databases, and compared with contemporary approaches for performance evaluation.

Facial micro-expression datasets lack consistency and standardisation, with different research groups using various experimental settings, in particular, where the datasets are varied in resolution and frame rates. To provide new insights into the roles of frame rate and resolution, we conduct an investigation into the use of different frame rates and resolution on current benchmark datasets (SMIC and CASME II). By using Temporal Interpolation Model, we subsample SMIC (original frame rate is 100 fps) to 50 fps and CASME II (original frame rate is 200 fps) into 100 fps and 50 fps. In addition, the resolution settings are adjusted to three scaling factors: 100% (original resolution), 75% and 50%. Three feature types are used to test the performance of these settings, which are Local Binary Patterns in Three Orthogonal Planes, 3D Histograms of Oriented Gradient and Histogram of Oriented Optical Flow. The results showed that the frame rate and resolution could affect the performance of micro-expression recognition, which behave distinctively dependent on feature types. This work provides new guidelines for future research in selecting frame rate, resolution and feature descriptors in micro-expressions recognition.

Current practice for diabetic foot ulcers (DFU) screening involves detection and localization by podiatrists. Existing automated solutions either focus on segmentation or classification. In this work, we design deep learning methods for real-time DFU localization. To produce a robust deep learning model, we collected an extensive database of 1775 images of DFU. Two medical experts produced the ground truths of this data set by outlining the region of interest of DFU with an annotator software. Using five-fold cross-validation, overall, faster R-CNN with InceptionV2 model using two-tier transfer learning achieved a mean average precision of 91.8%, the speed of 48 ms for inferencing a single image and with a model size of 57.2 MB. To demonstrate the robustness and practicality of our solution to real-time prediction, we evaluated the performance of the models on a NVIDIA Jetson TX2 and a smartphone app. This work demonstrates the capability of deep learning in real-time localization of DFU, which can be further improved with a more extensive data set.

Micro-facial expressions are regarded as an important human behavioural event that can highlight emotional deception. Spotting these movements is difficult for humans and machines, however research into using computer vision to detect subtle facial expressions is growing in popularity. This paper proposes an individualised baseline micro-movement detection method using 3D Histogram of Oriented Gradients (3D HOG) temporal difference method. We define a face template consisting of 26 regions based on the Facial Action Coding System (FACS). We extract the temporal features of each region using 3D HOG. Then, we use Chi-square distance to find subtle facial motion in the local regions. Finally, an automatic peak detector is used to detect micro-movements above the proposed adaptive baseline threshold. The performance is validated on two FACS coded datasets: SAMM and CASME II. This objective method focuses on the movement of the 26 face regions. When comparing with the ground truth, the best result was an AUC of 0.7512 and 0.7261 on SAMM and CASME II, respectively. The results show that 3D HOG outperformed for micro-movement detection, compared to state-of-the-art feature representations: Local Binary Patterns in Three Orthogonal Planes and Histograms of Oriented Optical Flow.

This paper presents baseline results for the first Facial Micro-expressions Grand Challenge (MEGC) 2018 by evaluating LBP-TOP, HOOF and 3DHOG on CASME II and SAMM. We further improve the result of composite database evaluation (Task B of the challenge) by introducing selective block-based features fusion representation. Base on objective classes, this task combines CASME II and SAMM into a single composite database and uses Leave-One-Subject-Out crossvalidation to evaluate the performance. Our proposed method achieve F1-Score of 0.579, which outperformed LBP-TOP, HOOF and 3DHOG with 0.523, 0.527 and 0.436, respectively.

This paper summarises the Facial Micro-Expression Grand Challenge (MEGC 2018) held in conjunction with the 13th IEEE Conference on Automatic Face and Gesture Recognition (FG) 2018. In this workshop, we aim to stimulate new ideas and techniques for facial micro-expression analysis by proposing a new cross-database challenge. Two state-of-the-art datasets, CASME II and SAMM, are used to validate the performance of existing and new algorithms. Also, the challenge advocates the recognition of micro-expressions based on AU-centric objective classes rather than emotional classes. We present a summary and analysis of the baseline results using LBP-TOP, HOOF and 3DHOG, together with results from the challenge submissions.

Micro-facial expressions are spontaneous, involuntary movements of the face when a person experiences an emotion but attempts to hide their facial expression, most likely in a high-stakes environment. Recently, research in this field has grown in popularity, however publicly available datasets of micro-expressions have limitations due to the difficulty of naturally inducing spontaneous micro-expressions. Other issues include lighting, low resolution and low participant diversity. We present a newly developed spontaneous micro-facial movement dataset with diverse participants and coded using the Facial Action Coding System. The experimental protocol addresses the limitations of previous datasets, including eliciting emotional responses from stimuli tailored to each participant. Dataset evaluation was completed by running preliminary experiments to classify micro-movements from non-movements. Results were obtained using a selection of spatio-temporal descriptors and machine learning. We further evaluate the dataset on emerging methods of feature difference analysis and propose an Adaptive Baseline Threshold that uses individualised neutral expression to improve the performance of micro-movement detection. In contrast to machine learning approaches, we outperform the state of the art with a recall of 0.91. The outcomes show the dataset can become a new standard for micro-movement data, with future work expanding on data representation and analysis.

Breast lesion detection using ultrasound imaging is considered an important step of computer-aided diagnosis systems. Over the past decade, researchers have demonstrated the possibilities to automate the initial lesion detection. However, the lack of a common dataset impedes research when comparing the performance of such algorithms. This paper proposes the use of deep learning approaches for breast ultrasound lesion detection and investigates three different methods: a Patch-based LeNet, a U-Net, and a transfer learning approach with a pretrained FCN-AlexNet. Their performance is compared against four state-of-the-art lesion detection algorithms (i.e., Radial Gradient Index, Multifractal Filtering, Rule-based Region Ranking, and Deformable Part Models). In addition, this paper compares and contrasts two conventional ultrasound image datasets acquired from two different ultrasound systems. Dataset A comprises 306 (60 malignant and 246 benign) images and Dataset B comprises 163 (53 malignant and 110 benign) images. To overcome the lack of public datasets in this domain, Dataset B will be made available for research purposes. The results demonstrate an overall improvement by the deep learning approaches when assessed on both datasets in terms of True Positive Fraction, False Positives per image, and F-measure.

Annotation of data is fundamental for training any modern facial tracking system. Methods, such as deep and machine learning, require large amounts of pre-annotated data to produce results found in many state-of-the-art systems. 2- Dimensional (images) annotations rely on the texture information of the face to annotate the features. However, in 3-Dimensions (models), the task becomes more complex. In 3D, the conventional 2D approaches are ineffective as facial landmarks are difficult to accurately identify without texture information. There has been little research into the accuracy of methods for annotating 3D facial models. This paper proposes a method for annotating 3D models which uses texture information by aligning a model to a 2D image to compare its accuracy and throughput to the conventional methods of 3D annotation. For evaluation, 16 nonexpert volunteers were recruited and instructed to annotate three models using both the proposed method and the conventional method. The resultant annotations were compared to ground truth data generated by an experienced annotator. The results demonstrate significant improvement in the throughput of the proposed annotation method compared to the conventional approach without significant differences in accuracy. The proposed method also highlights that the conventional method does not successfully identify all the facial landmarks. The proposed method will be made freely available to use online.

Diabetic Foot Ulcer (DFU) is a major complication of Diabetes, which if not managed properly can lead to amputation. DFU can appear anywhere on the foot and can vary in size, colour, and contrast depending on various pathologies. Current clinical approaches to DFU treatment rely on patients and clinician vigilance, which has significant limitations such as the high cost involved in the diagnosis, treatment and lengthy care of the DFU. We introduce a dataset of 705 foot images. We provide the ground truth of ulcer region and the surrounding skin that is an important indicator for clinicians to assess the progress of ulcer. Then, we propose a two-tier transfer learning from bigger datasets to train the Fully Convolutional Networks (FCNs) to automatically segment the ulcer and surrounding skin. Using 5fold cross-validation, the proposed two-tier transfer learning FCN Models achieve a Dice Similarity Coefficient of 0.794 (±0.104) for ulcer region, 0.851 (±0.148) for surrounding skin region, and 0.899 (±0.072) for the combination of both regions. This demonstrates the potential of FCNs in DFU segmentation, which can be further improved with a larger dataset.

The analysis of human motion as a clinical tool can bring many benefits such as the early detection of disease and the monitoring of recovery, so in turn helping people to lead independent lives. However, it is currently under used. Developments in depth cameras, such as Kinect, have opened up the use of motion analysis in settings such as GP surgeries, care homes and private homes. To provide an insight into the use of Kinect in the healthcare domain, we present a review of the current state of the art. We then propose a method that can represent human motions from time-series data of arbitrary length, as a single vector. Finally, we demonstrate the utility of this method by extracting a set of clinically significant features and using them to detect the age related changes in the motions of a set of 54 individuals, with a high degree of certainty (F1-score between 0.9-1.0). Indicating its potential application in the detection of a range of age-related motion impairments.

Facial wrinkle is one of the most prominent biological changes that accompanying the natural aging process. However, there are some external factors contributing to premature wrinkles development, such as sun exposure and smoking. Clinical studies have shown that heavy smoking causes premature wrinkles development. However, there is no computerised system that can automatically assess the facial wrinkles on the whole face. This study investigates the effect of smoking on facial wrinkling using a social habit face dataset and an automated computerised computer vision algorithm. The wrinkles pattern represented in the intensity of 0–255 was first extracted using a modified Hybrid Hessian Filter. The face was divided into ten predefined regions, where the wrinkles in each region was extracted. Then the statistical analysis was performed to analyse which region is effected mainly by smoking. The result showed that the density of wrinkles for smokers in two regions around the mouth was significantly higher than the non-smokers, at p-value of 0.05. Other regions are inconclusive due to lack of large-scale dataset. Finally, the wrinkle was visually compared between smoker and non-smoker faces by generating a generic 3D face model.

Analysis and quantification of human motion to support clinicians in the decision-making process is the desired outcome for many clinical-based approaches. However, generating statistical models that are free from human interpretation and yet representative is a difficult task. In this paper, we propose a framework that automatically recognizes and evaluates human mobility impairments using the Microsoft Kinect One depth sensor. The framework is composed of two parts. First, it recognizes motions, such as sit-to-stand or walking 4 m, using abstract feature representation techniques and machine learning. Second, evaluation of the motion sequence in the temporal domain by comparing the test participant with a statistical mobility model, generated from tracking movements of healthy people. To complement the framework, we propose an automatic method to enable a fairer, unbiased approach to label motion capture data. Finally, we demonstrate the ability of the framework to recognize and provide clinically relevant feedback to highlight mobility concerns, hence providing a route toward stratified rehabilitation pathways and clinician-led interventions.

Digital image processing systems are complex, being usually composed of different computer vision libraries. Algorithm implementations cannot be directly used in conjunction with algorithms developed using other computer vision libraries. This paper formulates a software solution by proposing a processor with the capability of handling different types of image processing algorithms, which allow the end users to install new image processing algorithms from any library. This approach has other functionalities like capability to process one or more images, manage multiple processing jobs simultaneously and maintain the manner in which an image was processed for later use. It is a computational efficient and promising technique to handle variety of image processing algorithms. To promote the reusability and adaptation of the package for new types of analysis, a feature of sustainability is established. The framework is integrated and tested on a medical imaging application, and the software is made freely available for the reader. Future work involves introducing the capability to connect to another instance of processing service with better performance. Copyright © 2015 John Wiley & Sons, Ltd.

Detecting micro-facial movements in a video sequence is the first step in realising a system that can pick out rapid movements automatically as a person is being recorded. This paper proposes a new method of micro-movement detection by applying Histogram of Oriented Gradients as a feature descriptor on our in-house high-speed video dataset of spontaneous micro facial movements. Firstly the algorithm aligns and crops faces for each video using automatic facial point detection and affine transformation. Then a de-noising algorithm is applied to each video before splitting them into blocks where the Histogram of Oriented Gradient features are calculated for each frame in every video block. The Chi-Squared distance measure is then used to calculate dissimilarity in the spatial appearance between frames at a set interval. The final feature vector is calculated after normalisation of the raw distance values and peak detection is applied to 'spot' micro-facial movements. An individualised baseline threshold is used to determine the value a peak must exceed to be classed as a movement. The result is compared with a benchmark algorithm - feature difference analysis techniques for micro-facial movements using Local Binary Patterns. Results indicate the proposed method achieves higher Recall of 0.8429 and F1-measure of 0.7672.

The majority of current facial age estimation methods are based on appearance based features. However, wrinkle based research has not been widely addressed. In this paper, we propose a novel method based on multi-scale aging patterns (MAP). These directly extract the features from local patches without extensive geometric modelling. First, we locate facial landmarks by using the Face++ detector and then normalize the face by using a linear transformation. We define a face template which consists of ten predefined wrinkle regions. Then, for each region, we detect wrinkles and construct aging patterns by using the MAP. Finally, the age is estimated by implementing the sequential minimal optimization (SMO). The performance of the algorithms is assessed by using mean absolute error (MAE) on the benchmark database - FERET. We observe that MAP produces a lower MAE of 4.87 on FERET compared to the benchmark algorithms. Therefore, we conclude that wrinkle could be used as a feature on face age estimation. Future work would involve improvements of the algorithm by combining other descriptors such as non-wrinkle descriptor and appearance parameters.

The recognition of human activity is a challenging topic for machine learning. We present an analysis of Support Vector Machines (SVM) and Random Forests (RF) in their ability to accurately classify Kinect kinematic activities. Twenty participants were captured using the Microsoft Kinect performing ten physical rehabilitation activities. We extracted the kinematic location, velocity and energy of the skeletal joints at each frame of the activity to form a feature vector. Principle Component Analysis (PCA) was applied as a pre-processing step to reduce dimensionality and identify significant features amongst activity classes. SVM and RF are then trained on the PCA feature space to assess classification performance, we undertook an incremental increase in the dataset size. We analyse the classification accuracy, model training and classification time quantitatively at each incremental increase. The experimental results demonstrate that RF outperformed SVM in classification rate for six out of the ten activities. Although SVM has performance advantages in training time, RF would be more suited to real-time activity classification due to its low classification time and high classification accuracy when using eight to ten participants in the training set.

The rise of service-oriented architectures and technology has, in recent years, started to lead to the opportunity for data sharing on a large-scale. In this paper we report upon how a service-oriented framework has been leveraged to support the development of a training application for radiologists. The application and framework have been developed separately - but sympathetically - with experience in both domains being leveraged to develop the prototype of an end-to-end training system. While the initial system utilises previously collected data, it is intended that in the future the system will interoperate with systems deployed within hospital environments.