Transformers have rapidly become the dominant architecture for analyzing sequential data, utilizing their self-attention mechanism to effectively capture long-term temporal patterns, outperforming recurrent-based methods across various applications. In this paper, we explore the application of transformers to wearable sensor data, focusing on the analysis of human gait, which is often complex and sensitive. We propose two novel frameworks: Data Efficient Sensor Transformer (DesT) for centralized learning and Federated Data Efficient Sensor Transformer (FeDesT) for federated learning (FL) in edge-computing environments. Both frameworks employ knowledge distillation to improve the generalization of transformers, which can be prone to over-fitting due to the limited labeled data available in wearable sensor applications. Experimental results using human gait data collected from uneven and irregular surfaces show that DesT improves the accuracy by 14.8% when compared to existing transformers. FeDesT reduces computational demands on edge devices while outperforming traditional FL methods for transformers. This work demonstrates the potential of transformers for wearable sensor data analysis in both centralized and federated contexts, particularly where privacy and computational efficiency is paramount.

Photo-identification (photo-id) is one of the main non-invasive capture-recapture methods utilised by marine researchers for monitoring cetacean (dolphin, whale, and porpoise) populations. This method has historically been performed manually resulting in high workload and cost due to the vast number of images collected. Recently automated aids have been developed to help speed-up photo-id, although they are often disjoint in their processing and do not utilise all available identifying information. Work presented in this paper aims to create a fully automatic photo-id aid capable of providing most likely matches based on all available information without the need for data pre-processing such as cropping. This is achieved through a pipeline of computer vision models and post-processing techniques aimed at detecting cetaceans in unedited field imagery before passing them downstream for individual level catalogue matching. The system is capable of handling previously uncatalogued individuals and flagging these for investigation thanks to catalogue similarity comparison. We evaluate the system against multiple real-life photo-id catalogues, achieving mAP@IOU[0.5] = 0.91, 0.96 for the task of dorsal fin detection on catalogues from Tanzania and the UK respectively and 83.1, 97.5% top-10 accuracy for the task of individual classification on catalogues from the UK and USA.