The advancement of technology has revolutionized the agricultural industry, transitioning it from labor-intensive farming practices to automated, AI-powered management systems. In recent years, more intelligent livestock monitoring solutions have been proposed to enhance farming efficiency and productivity. This work presents a novel approach to animal activity recognition and movement tracking, leveraging tiny machine learning (TinyML) techniques, wireless communication framework, and microcontroller platforms to develop an efficient, cost-effective livestock sensing system. It collects and fuses accelerometer data and vision inputs to build a multimodal network for three tasks: image classification, object detection, and behavior recognition. The system is deployed and evaluated on commercial microcontrollers for real-time inference using embedded applications, demonstrating up to 270× model size reduction, less than 80ms response latency, and on-par performance comparable to existing methods. The incorporation of the wireless communication technique allows for seamless data transmission between devices, benefiting use cases in remote locations with poor Internet connectivity. This work delivers a robust, scalable IoT-edge livestock monitoring solution adaptable to diverse farming needs, offering flexibility for future extensions.

In recent years, the development of smart edge computing systems to process information locally is on the rise. Many near-sensor machine learning (ML) approaches have been implemented to introduce accurate and energy efficient template matching operations in resource-constrained edge sensing systems, such as wearables. To introduce novel solutions that can be viable for extreme edge cases, hybrid solutions combining conventional and emerging technologies have started to be proposed. Deep Neural Networks (DNN) optimised for edge application alongside new approaches of computing (both device and architecture -wise) could be a strong candidate in implementing edge ML solutions that aim at competitive accuracy classification while using a fraction of the power of conventional ML solutions. In this work, we are proposing a hybrid software-hardware edge classifier aimed at the extreme edge near-sensor systems. The classifier consists of two parts: (i) an optimised digital tinyML network, working as a front-end feature extractor, and (ii) a back-end RRAM-CMOS analogue content addressable memory (ACAM), working as a final stage template matching system. The combined hybrid system exhibits a competitive trade-off in accuracy versus energy metric with$E_{front-end}$=$96.23 nJ$and$E_{back-end}$=$1.45 nJ$for each classification operation compared with 78.06 μJ for the original teacher model, representing a 792-fold reduction, making it a viable solution for extreme edge applications.

People living with dementia are at risk of social isolation, and conversational AI agents can potentially support such individuals by reducing their loneliness. In our study, a conversational AI agent, called MindTalker, co-designed with therapists and utilizing the GPT-4 Large Language Model (LLM), was developed to support people with early-stage dementia, allowing them to experience a new type of “social relationship” that could be extended to real life. Eight PwD engaged with MindTalker for one month or even longer, and data was collected from interviews. Our findings emphasized that participants valued the novelty of AI, but sought more consistent, deeper interactions. They desired a personal touch from AI, while stressing the irreplaceable value of human interactions. The findings underscore the complexities of AI engagement dynamics, where participants commented on the artificial nature of AI, highlighting important insights into the future design of conversational AI for this population.

The quantification of emotional states is an important step to understanding wellbeing. Time series data from multiple modalities such as physiological and motion sensor data have proven to be integral for measuring and quantifying emotions. Monitoring emotional trajectories over long periods of time inherits some critical limitations in relation to the size of the training data. This shortcoming may hinder the development of reliable and accurate machine learning models. To address this problem, this article proposes a framework to tackle the limitation in performing emotional state recognition: (1) encoding time series data into coloured images; (2) leveraging pre-trained object recognition models to apply a Transfer Learning (TL) approach using the images from step 1; (3) utilising a 1D Convolutional Neural Network (CNN) to perform emotion classification from physiological data; (4) concatenating the pre-trained TL model with the 1D CNN. We demonstrate that model performance when inferring real-world wellbeing rated on a 5-point Likert scale can be enhanced using our framework, resulting in up to 98.5% accuracy, outperforming a conventional CNN by 4.5%. Subject-independent models using the same approach resulted in an average of 72.3% accuracy (SD 0.038). The proposed methodology helps improve performance and overcome problems with small training datasets.

Abstract Involving and engaging people with intellectual disabilities on issues relating to their mental wellbeing is essential if relevant tools and solutions are to be developed. This research explores how inclusive and participatory co-design techniques and principles can be used to engage people with intellectual disabilities in designing innovations in mental wellbeing tangible technologies. In particular, individuals with intellectual disabilities participated in a co-design process via a series of workshops and focus groups to design tangible interfaces for mental wellbeing as their wellbeing challenges are often diagnostically overshadowed. The workshops helped participants explore new technologies, including sensors and feedback mechanisms that can help monitor and potentially improve mental wellbeing. The adopted co-design approach resulted in a range of effective and suitable interfaces being developed for varying ages.

Mental health problems are on the rise globally and strain national health systems worldwide. Mental disorders are closely associated with fear of stigma, structural barriers such as financial burden, and lack of available services and resources which often prohibit the delivery of frequent clinical advice and monitoring. Technologies for mental well-being exhibit a range of attractive properties, which facilitate the delivery of state-of-the-art clinical monitoring. This review article provides an overview of traditional techniques followed by their technological alternatives, sensing devices, behaviour changing tools, and feedback interfaces. The challenges presented by these technologies are then discussed with data collection, privacy, and battery life being some of the key issues which need to be carefully considered for the successful deployment of mental health toolkits. Finally, the opportunities this growing research area presents are discussed including the use of portable tangible interfaces combining sensing and feedback technologies. Capitalising on the data these ubiquitous devices can record, state of the art machine learning algorithms can lead to the development of robust clinical decision support tools towards diagnosis and improvement of mental well-being delivery in real-time.

The novel coronavirus, designated by the World Health Organization as COVID-19 has required many countries around the world to close work spaces, schools and public venues. This has required policy makers and venue managers to investigate practical mitigation strategies using technology to exit the lockdown safely and enable the reopening of public spaces. This paper introduces Digital personal protective equipment (PPE), a dynamic and affordable wearable approach that remind people to keep their distance and keep track of their contact traces. This IoT based BLE probing technique approach empowers employers, city and venue managers to encourage social-distancing and trigger a friendly alert using vibration when social distancing is violated in privacy-preserving manner.

Abstract In recent years, machine learning has developed rapidly, enabling the development of applications with high levels of recognition accuracy relating to the use of speech and images. However, other types of data to which these models can be applied have not yet been explored as thoroughly. Labelling is an indispensable stage of data pre-processing that can be particularly challenging, especially when applied to single or multi-model real-time sensor data collection approaches. Currently, real-time sensor data labelling is an unwieldy process, with a limited range of tools available and poor performance characteristics, which can lead to the performance of the machine learning models being compromised. In this paper, we introduce new techniques for labelling at the point of collection coupled with a pilot study and a systematic performance comparison of two popular types of deep neural networks running on five custom built devices and a comparative mobile app (68.5–89% accuracy within-device GRU model, 92.8% highest LSTM model accuracy). These devices are designed to enable real-time labelling with various buttons, slide potentiometer and force sensors. This exploratory work illustrates several key features that inform the design of data collection tools that can help researchers select and apply appropriate labelling techniques to their work. We also identify common bottlenecks in each architecture and provide field tested guidelines to assist in building adaptive, high-performance edge solutions.

In recent years, mobile phone technology has taken tremendous leaps and bounds to enable all types of sensing applications and interaction methods, including mobile journaling and self-reporting to add metadata and to label sensor data streams. Mobile self-report techniques are used to record user ratings of their experiences during structured studies, instead of traditional paper-based surveys. These techniques can be timely and convenient when data are collected “in the wild”. This paper proposes three new viable methods for mobile self-reporting projects and in real-life settings such as recording weather information or urban noise mapping. These techniques are Volume Buttons control, NFC-on-Body, and NFC-on-Wall. This work also provides an experimental and comparative analysis of various self-report techniques regarding user preferences and submission rates based on a series of user experiments. The statistical analysis of our data showed that pressing screen buttons and screen touch allowed for higher labelling rates, while Volume Buttons proved to be more valuable when users engaged in other activities, e.g. while walking. Similarly, based on participants’ preferences, we found that NFC labelling was also an easy and intuitive technique when used in the context of self-reporting and place-tagging. Our hope is that by reviewing current self-reporting interfaces and user requirements, we will be able to enable new forms of self-reporting technologies that were not possible before.

Abstract Pervasive sensing has opened up new opportunities for measuring our feelings and understanding our behavior by monitoring our affective states while mobile. This review paper surveys pervasive affect sensing by examining and considering three major elements of affective pervasive systems, namely “sensing,” “analysis,” and “application.” Sensing investigates the different sensing modalities that are used in existing real-time affective applications, analysis explores different approaches to emotion recognition and visualization based on different types of collected data, and application investigates different leading areas of affective applications. For each of the three aspects, the paper includes an extensive survey of the literature and finally outlines some of challenges and future research opportunities of affective sensing in the context of pervasive computing.

Participate was a 3-year collaboration between industry and academia to explore how mobile, Web and broadcast technologies could combine to deliver environmental campaigns. In a series of pilot projects, schools used mobile sensors to enhance science learning; visitors to an ecological attraction employed mobile phones to access and generate locative media; and the public played a mobile phone game that challenged their environmental behaviours. Key elements of these were carried forward into an integrated trial in which participants were assigned a series of environmental missions as part of an overarching narrative that was delivered across mobile, broadcast and Web platforms. These experiences use a three-layered structure for campaigns that draw on experts, local groups and the general public, who engage through a combination of playful characterisation and social networking.

This work proposes a system for rating shops and for monitoring the cell phone-based emotion responses of customers in a shopping mall environment. To measure customer satisfaction in a shopping environment, a mobile, non-intrusive and comfortable wearable biosensor is used to measure the Electro dermal Activity (EDA) of the shopper. The users' proximity to the store is detected using NFC tags that report to the custom application on the mobile phone. The custom emotion recognition software analyses these streams of data in real-time and associates emotion levels to each event. The aim of this project is to demonstrate the possibility of using pervasive affective computing to explicate consumer behavior towards the stores in shopping malls. By triggering positive emotions through enhancing services and improving advertising campaigns, retailers can trigger positive emotional states, which ultimately contribute to a positive and memorable shopping experience.

Social Networks offer a context for people to meet, communicate and collaborate. Tools to support communities usually provide a communication medium and functionalities to find communication partners. This paper describes our mobile framework EnvAware which engages the public in location sensitive experiences and in municipal monitoring of their environment, available both on users' mobile phones, and online. This is centered on the use of smart phones as sources and sinks of information. It involves coordination among multiple phones as well as sensors deployed in the environment. This mobile forum is based on Cell-ID positioning and GPRS communications. It stores and receives information from a remote server which analyses and processes the scientific data received from a scalable mobile sensing framework called mFeel. The processed data and exchanged information are made available to local communities through EnvAware.