ABSTRACT Human action recognition in sports videos is a challenging computer vision task due to fast motion, frequent occlusions and fine‐grained visual similarities among action classes. This work presents TESA‐Net (Temporal‐Efficient Spatial Attention Network), an efficient dual‐stream architecture for basketball action recognition that achieves state‐of‐the‐art performance while maintaining computational efficiency. Unlike existing methods that rely on expensive 3D convolutions or full spatio‐temporal attention mechanisms, TESA‐Net employs a pre‐trained 2D ResNet‐50 backbone with lightweight temporal aggregation. The key innovation is a novel Court Line Detection module that augments the appearance stream with edge‐based geometric features, enabling accurate discrimination between shot types that differ primarily in shooting distance. We evaluate TESA‐Net on two complementary benchmarks: Basketball‐51, which targets fine‐grained shot classification in professional broadcasts, and MultiSubjects, which addresses coarse‐grained action recognition in amateur gymnasium recordings. On Basketball‐51, TESA‐Net achieves a validation accuracy of 94.37%, surpassing the previous state‐of‐the‐art HAQT (92.76%) by 1.61 percentage points. On MultiSubjects, TESA‐Net reaches 96.12% accuracy, matching transformer‐based approaches while using significantly fewer parameters. Owing to its efficient design, TESA‐Net requires substantially less memory than competing 3D‐based methods, enabling practical deployment without high‐end hardware.

Head pose estimation (HPE) is crucial in various real-world applications, like human–computer interaction and biometric framework enhancement. This research aims to leverage network curvature to predict head pose from a single image. In networks, certain groups of nodes fulfill significant functional roles. This study focuses on the interactions of facial landmarks, considered as vertices in a weighted graph. The experiments demonstrate that the underlying graph geometry and topology enable the detection of similarities among various head poses. Two independent notions of discrete Ricci curvature for graphs, namely Ollivier–Ricci and Forman–Ricci curvatures, are investigated. These two types of Ricci curvature, each reflecting distinct geometric properties of the network, serve as inputs to the regression model. The results from the BIWI, AFLW2000, and Pointing‘04 datasets reveal that the two discretizations of Ricci’s curvature are closely related and outperform state-of-the-art methods, including both landmark-based and image-only approaches. This demonstrates the effectiveness and promise of using network curvature for HPE in diverse applications. • The topology of the underlying graph can identify similarities across head poses. • Analyzes the performance differences between Ollivier and Forman Ricci curvatures. • Ollivier–Ricci-based method shows competitive results on three different datasets. • The Forman–Ricci-based method offers a computationally efficient alternative.

Facial Expression recognition is a computer vision problem that took relevant benefit from the research in deep learning. Recent deep neural networks achieved superior results, demonstrating the feasibility of recognizing the expression of a user from a single picture or a video recording the face dynamics. Research studies reveal that the most discriminating portions of the face surfaces that contribute to the recognition of facial expressions are located on the mouth and the eyes. The restrictions for COVID pandemic reasons have also revealed that state-of-the-art solutions for the analysis of the face can severely fail due to the occlusions of using the facial masks. This study explores to what extend expression recognition can deal with occluded faces in presence of masks. To a fairer comparison, the analysis is performed in different occluded scenarios to effectively assess if the facial masks can really imply a decrease in the recognition accuracy. The experiments performed on two public datasets show that some famous top deep classifiers expose a significant reduction in accuracy in presence of masks up to half of the accuracy achieved in non-occluded conditions. Moreover, a relevant decrease in performance is also reported also in the case of occluded eyes but the overall drop in performance is not as severe as in presence of the facial masks, thus confirming that, like happens for face biometric recognition, occluded faces by facial mask still represent a challenging limitation for computer vision solutions.

Face-based recognition methods usually need the image of the whole face to perform, but in some situations, only a fraction of the face is visible, for example wearing sunglasses or recently with the COVID pandemic we had to wear facial masks. In this work, we propose a network architecture made up of four deep learning streams that process each one a different face element, namely: mouth, nose, eyes, and eyebrows, followed by a feature merge layer. Therefore, the face is segmented into the part of interest by means of ROI masks to keep the same input size for the four network streams. The aim is to assess the capacity of different combinations of face elements in recognizing the subject. The experiments were carried out on the Masked Face Recognition Database (M2FRED) which includes videos of 46 participants. The obtained results are 96% of recognition accuracy considering the four face elements; and 92%, 87%, and 63% of accuracy for the best combination of three, two, and one face elements respectively.

Head pose estimation is not only a crucial challenge for many real-world applications, such as driver attention detection analysis, but it represents an interesting strategy to support biometric frameworks as well. This paper aims to estimate head pose from a single image by applying notions of network curvature. In the real world, many complex networks have groups of nodes that are well connected to each other with significant functional roles. Similarly, the interactions of facial landmarks can be represented as complex dynamic systems modeled by weighted graphs. The functionality of such a system is therefore intrinsically linked to the topology and geometry of the underlying graph. In this work, using the geometric notion of Ollivier-Ricci curvature (ORC) on weighted graphs as input to the XGBoost regression model, we show that the intrinsic geometric basis of ORC offers a natural approach to discovering underlying common structure within a pool of poses. Experiments on the BIWI, AFLW2000 and Pointing '04 datasets show that the ORC_XGB method performs well compared to state-of-the-art methods, both landmark-based and image-only.

Smart factories are fostered by integrating intelligent systems and ICT technologies. The role they play is crucial in the spread of Industry 4.0 and the economic growth of developed countries. Smart factories can be empowered by using several sensors aimed at making them more and more “smart.” Unfortunately, work accidents are still very common resulting in human losses and permanent injuries. This makes it urgent and key to implementing security and safety measures also in the context of a smart factory. In this article, a novel framework for supporting smart devices in a smart factory, using multiple sensors to monitor different biometric features, both physical and behavioral is proposed. Thanks to the fusion of several biometric traits with the support of machine learning technologies working together with different kinds of sensors, it is possible to guarantee three fundamental aspects within the interaction between an operator of a smart device and the device itself: continuous authentication (i.e., continuous face recognition), drowsiness detection, and liveness detection. With the application of the proposed framework, it is possible to significantly improve the safety of operators avoiding fatal accidents for them. Experiments made using COTS-hardware showed that the authors’ idea is easy to implement in a large-scale smart factory and further improves the spread of Industry 4.0.

Abstract Head pose estimation represents an important computer vision technique in different contexts where image acquisition cannot be controlled by an operator, making face recognition of unknown subjects more accurate and efficient. In this work, starting from partitioned iterated function systems to identify the pose, different regression models are adopted to predict the angular value errors (yaw, pitch and roll axes, respectively). This method combines the fractal image compression characteristics, such as self-similar structures in order to identify similar head rotation, with regression analysis prediction. The experimental evaluation is performed on widely used benchmark datasets, i.e., Biwi and AFLW2000, and the results are compared with many existing state-of-the-art methods, demonstrating the robustness of the proposed fusion approach and excellent performance.

Recent advances in Machine Learning have unveiled interesting possibilities for real-time investigating about user characteristics and expressions like, but not limited to, age, sex, body posture, emotions and moods. These new opportunities lay the foundations for new HCI tools for interactive applications that adopt user emotions as a communication channel.

Today, identity verification is required in many common activities, and it is arguably true that most people would like to be authenticated in the easiest and most transparent way, without having to remember a personal identification number. To this regard, this paper presents a multibiometric system based on the observation that the instinctive gesture of responding to a phone call can be used to capture two different biometrics, namely ear and arm gesture, which are complementary due to their, respectively, physical and behavioral nature. We conducted a comprehensive set of experiments aimed at assessing the contribution of each of the two biometrics as well as the advantage in their fusion to the system's overall performance. Experiments also provide objective measurement of both saliency and correlation of data captured by each sensor involved (accelerometer, gyroscope, and camera) according to various features extraction, features matching, and data-fusion techniques. The reports provide evidences about the potential of the proposed system and method for user authentication “in-the-wild,” whilst its eventual usage for person identification is also investigated. All of the experiments have been carried out on a specifically built, publicly available ear-arm database, including multibiometric captures of more than 100 subjects performed during different sessions, that represents an additional contribution of this paper.

Context awareness is major component of Ambient Intelligence. In fact, Ambient Intelligent environments are designed to combine ubiquity, awareness, intelligence and natural interaction. Awareness is defined as the ability by the system to locate and recognize people and objects, and their intentions. Then, intelligence is the ability of the system to analyze the detected context and to adapt its behavior to people and situations, and to learn over time, in order to provide users with personalized services. These concepts date back to late'90, but nowadays the widespread and ubiquitous availability of mobile devices, equipped with several different sensors, allows to put them into practice in a number of unexpected ways. This work presents a preliminary investigation on the possibility to use some of the smartphone sensors, namely the accelerometer and the gyroscope, to identify the bodily context when the user lifts the device to answer a call. The arm gesture, i.e., the way it is performed, is classified into 4 different states: while standing, sitting, walking or running. This information can be used to trigger context-sensitive system actions.

In the last fifteen years, smartphones have become very popular amongst the population, with the subsequent development of dozens of applications aimed at providing security to these portable devices. Nowadays, the cutting edge devices are also provided with biometric sensors (e.g., fingerprint sensors) allowing the users to access them without using the out-of-date alphanumerical password. In this work, we present a method that realizes iris recognition by means of Self Organizing Maps (SOM). In order to obtain a better refined and discriminative feature map, the RGB data of the iris, previously segmented, have been combined with two statistical descriptors. The algorithm has been designed specifically to require a low processing power, making it an ideal choice in the context of mobile devices.

Smartphones are arguably candidates to become the platform of choice for ubiquitous biometric-based identity verification, thanks to their embedded sensors, reasonably good computing power and widespread diffusion. While applications of the most established biometrics like face, fingerprint and even iris have already been proposed on mobile devices, other less exploited identifiers could also be worth investigating. To this regard, a novel multi-modal approach to person authentication based on ear biometrics and gesture analysis is proposed in this paper. The idea is to coupling the discriminant power of ear, captured during the act of responding to a phone call, with the user's arm dynamics affecting the smartphone motion pattern due to behavioral and anatomical characteristics involved in this gesture. According to experiments conducted on a specifically built multi-modal database comprising a hundred subjects, we confirm that the “responding gesture” has significant discriminating power and combined to ear features provides even greater robustness and accuracy in mobile authentication scenarios.

Mixed reality has been proposed for industrial applications such as AR-assisted manteinance and repair in which real equipment are augmented by visual aids. A possible complementary use of this technology for the same context can be achieved by performing virtual training in a “mixed reality environment” enabling to observe actual-size virtual replicas of real equipment within the physical space, providing an advantage in terms of perceptual impact and learning efficacy compared to traditional computer-based-training techniques. To this aim we present an effective hybrid tracking approach, exploiting both optical markers and inertial sensors to measure user's head position and rotation, providing both accuracy and robustness to fast head movements. A field-deployable tridimensionally-arranged marker-set, specifically designed for this kind of application, represents a further contribute of this paper, providing reliable close-to-medium distance optical tracking capabilities and increased robustness to challenging lighting conditions. First experiments confirm the potential of the proposed solution for this context and for many other applications as well.

In the last years, the growing diffusion of lightweight portable computing device like netbooks, tablets, and smartphones, featuring adequate processing power coupled with trackpad/touchpad interface, one or two webcams and eventually additional sensors (accelerometers, gps, gyroscopes, digital compass, etc.) has provided a low-cost platform to augmented reality applications, usually relying on more dedicated but also expensive and bulky technologies like motion tracking systems and see-through head mounted displays. In this paper we present and describe an AR application aimed to showcase how it is possible to effectively augment AV (Audio-Video) components, a kind of hi-tech gear today diffused in most home environments, by means of context dependent graphics contents. Visual aids in the form of both static and animated graphics are displayed according to the current status of the component (outputted via a serial interface) or simply based on the selection operated by the user through the trackpad. Moreover the system is able to help user to focus his/her attention on the physical interface on the AV component (e.g. a knob, a button or a connector in the back panel) either via an augmenting strategy (e.g. by adding virtual info) or by means of a diminishing approach (hiding all the other not relevant features). The proposal is easily extendible to a broad category of low-cost AR applications as it is based on cheap hardware (netbooks/tablets) and exploits marker based motion tracking through external webcam and the lcd screen as a see-through display.

VIVIE is a system for video sequence indexing. Video frames are annotated according to the identities of appearing subjects. Different interacting modules perform different processing steps, and each can be possibly substituted with a different one performing the same task using a different method. Classification and clustering are the most challenging activities. Differently from most existing similar systems, VIVIE accounts for the concomitant appearance of two identities in the same clip, and exploits such information for identity mapping. VIVIE was tested on 7 video clips and on a subset of the SCFace database to assess its performances.

Virtual simulators have been used in the last twenty years for applications ranging from flight simulation to computer-based training, just to name a few. More recently a new level of simulation has been introduced thanks to haptic interfaces able to reproduce kinesthetic and/or tactile feedback, typically experimented during interaction with real-world objects. In this paper visual simulation and haptic interfaces are integrated in a novel training system for Intra Cytoplasmic Sperm Injection (ICSI), an in-vitro fertilization technique which is now a standard for the treatment of human infertility. We describe a virtual micromanipulation simulator made by two hand-based Cyberforce haptic devices (a synthetic replica of the actual manipulation gear) and a visual-haptic engine simulating the shape and the dynamic behavior of the main components in the artificial fertilization process: the human egg, the selected sperm and the micro needles required to inject the latter into the egg's cytoplasm. Our first tests, conducted so far, are encouraging.

Thanks to the advances of voltage regulator (VR) technologies and haptic systems, virtual simulators are increasingly becoming a viable alternative to physical simulators in medicine and surgery, though many challenges still remain. In this study, a pervasive visual-haptic framework aimed to the training of obstetricians and midwives to vaginal delivery is described. The haptic feedback is provided by means of two hand-based haptic devices able to reproduce force-feedbacks on fingers and arms, thus enabling a much more realistic manipulation respect to stylus-based solutions. The interactive simulation is not solely driven by an approximated model of complex forces and physical constraints but, instead, is approached by a formal modeling of the whole labor and of the assistance/intervention procedures performed by means of a timed automata network and applied to a parametrical 3-D model of the anatomy, able to mimic a wide range of configurations. This novel methodology is able to represent not only the sequence of the main events associated to either a spontaneous or to an operative childbirth process, but also to help in validating the manual intervention as the actions performed by the user during the simulation are evaluated according to established medical guidelines. A discussion on the first results as well as on the challenges still unaddressed is included.

Face's tri-dimensional shape represents a highly discriminating yet challenging biometric identifier due to different issues, some of which related to capture, alignment and normalization. This paper presents an improved normal map based face recognition approach, which relies on a novel method to automatically align a captured 3D face mesh to a reference template, allowing a more precise face comparison. The alignment algorithm exploits pyramidal-normal-map metric, a coarse to finer measurement of angular distance between two surfaces computed through normal maps with progressively increasing resolution. After the registration has been performed, the normalized face can be rapidly compared to any other template in the gallery database for authentication or identification purposes using standard normal map metric. The alignment approach avoids the need for a rough or manual face pre-alignment and maximizes recognition precision, requiring a fraction of the time needed by the iterative closest point (ICP) method to operate. We show preliminary experimental results on a 3D dataset featuring 235 different subjects.

During the last few years, many algorithms have been proposed in particular for face recognition using classical 2-D images. However, it is necessary to deal with occlusions when the subject is wearing sunglasses, scarves and such. In the same way, ear recognition is arising as a new promising biometric for people recognition, even if the related literature appears to be somewhat underdeveloped. In this paper, several hybrid face/ear recognition systems are investigated. The system is based on IFS (Iterated Function Systems) theory that are applied on both face and ear resulting in a bimodal architecture. One advantage is that the information used for the indexing and recognition task of face/ear can be made local, and this makes the method more robust to possible occlusions. The distribution of similarities in the input images is exploited as a signature for the identity of the subject. The amount of information provided by each component of the face and the ear image has been assessed, first independently and then jointly. At last, results underline that the system significantly outperforms the existing approaches in the state of the art.

Face represents a rich biometric identifier whose potential in term of discriminating power has not been fully exploited yet. This paper addresses face recognition through a multi-modal approach operating on face's 3D (geometry) and 2D (skin texture) features by means of two different metrics: augmented normal map and principal component analysis. Augmented normal map includes shape (surface normals represented as 24 bit colour pixels) and texture info (additional 8 bit for skin colour) into one 32 bit image. The proposed two-staged method firstly performs a fast one-to-many comparison of facial geometry exploiting normal map metric. Then, to further improve recognition precision and reliability, best rank faces are compared to probe by PCA resulting in a final score. Other advantages are robustness to facial expressions and the ability to selectively filter face's non-skin regions (beard, moustaches). We include preliminary experimental results on a dataset of 101 textured 3D faces.

This paper presents a 3D face recognition method aimed to biometric applications. The proposed method compares any two faces represented as 3D polygonal surfaces through their corresponding normal map, a bidimensional array which stores local curvature (mesh normals) as the pixel's RGB components of a color image. The recognition approach, based on the computation of a difference map resulting from the comparison of normal maps, is simple yet fast and accurate. A weighting mask, automatically generated for each subject using a set of expression variations, improves the robustness to a broad range of facial expressions. First results show the effectiveness of the method on a database of 3D faces featuring different genders, ages and expressions.

Nowadays face recognition is gaining great attention from the researcher respect to other biometrics, because it represents a good compromise between reliability and people acceptance. While this growth largely is driven by growing application demands, such as identification for law enforcement and authentication, for banking and security system access. The recognition task is difficult because of image variation in terms of position, size, expression, and pose. In this paper a new IFS based recognition method is presented. It exploits the IFS theory, largely studied in still image compression and indexing, but not enough for the face recognition task.

The problem of performance evaluation of business processes supported by Workflow Management Systems is a recent research issue. In this paper, we propose an approach to the performance evaluation of automated business processes based on the measurement language WPQL (Workflow Performance Query Language). The paper first describes the WPQL architecture together with a selection mechanism by means of which the workflow entities to measure are isolated. Then, the main constructs of WPQL for measure definition and measure application are presented and exemplified. Finally, we show a working session of the support tool and discuss some guideline for further research.

The literature offers several examples of executable specification languages, ranging from mathematically based notation to visual formalisms. In this paper, an analysis environment for executable system specifications based on the language RSF (Requirement Specification Formalism) is presented. The analysis environment also contains two other linguistic tools — RSQ (Requirement Specification Querying), and SEF (Specification Execution Filtering). Using RSQ, classes of execution paths with certain properties can be exercised, so that selected behavioral aspects can be observed. Using SEF, the amount of information and the times at which it is output can be controlled, making the behavior analysis more effective. The paper shows how the notation of RSF, with its few basic concepts, is naturally exploited as a nucleus for the other tools, which are easily integrated to produce the final analysis environment. The combination of RSQ and SEF allows the planning of testing and analysis activities. A working session is shown for a specification describing a telephone switch call handler.

A system for the definition, querying and analysis of executable specification is presented in the paper. After a short review RSF, a specification language for the definition of systems with time constraints, the authors consider the problem of querying the specifications. The query language RSQ allows one to characterize sub-classes of possible specification execution behaviours in terms of queries directed to the specification. The problem of selection of significant information produced during the execution of the specification is also considered. The system proposes three linguistic tools, one for the definition, one for the querying and the last for the filtering of executable software specifications. The linguistic tools are quite homogeneous both in their syntax and semantics and support the organization of software behaviour inspection and analysis in the RSF prototyping environment.>