Positive Computing involves the utilisation of digital technology to foster psychological wellbeing and human potential. We will present an overview of Positive Computing and what it implies for multimodal systems for public speaking. The position of this paper is that a Positive Computing approach can make such systems more effective and improve user experience. We will focus on three of the tenets of Positive Computing viz. awareness, autonomy and stress-reduction. We will discuss different existing multimodal systems for public speaking within the context of Positive Computing.

This paper presents a fast approach for computing imagestixels from a sparse edge-based disparity map. The use ofedge-based disparity maps speeds up the computation of thestixels as only a few pixels must be processed compared toapproaches which use dense disparity maps. The proposedapproach produces as output the stixels in one of the viewsof the stereo-pair and a segmentation of the edge-pointsinto obstacle. Additionally the proposed approach allowsthe identification of partially occluded objects by allowingmore than one stixel per image column. The proposed ap-proach is fast to compute with no loss on accuracy.

A multimodal system for public speaking with real time feedback has been developed using the Microsoft Kinect. The system has been developed within the paradigm of positive computing which focuses on designing for user wellbeing. The system detects body pose, facial expressions and voice. Visual feedback is displayed to users on their speaking performance in real time. Users can view statistics on their utilisation of speaking modalities. The system also has a mentor avatar which appears alongside the user avatar to facilitate user training. Autocue mode allows a user to practice with set text from a chosen speech.

We have developed a multimodal prototype for public speaking with real time feedback using the Microsoft Kinect. Effective speaking involves use of gesture, facial expression, posture, voice as well as the spoken word. These modalities combine to give the appearance of self-confidence in the speaker. This initial prototype detects body pose, facial expressions and voice. Visual and text feedback is displayed in real time to the user using a video panel, icon panel and text feedback panel. The user can also set and view elapsed time during their speaking performance. Real time feedback is displayed on gaze direction, body pose and gesture, vocal tonality, vocal dysfluencies and speaking rate.

A prototype has been developed for a digital system and multi-modal user interface to analyze social signals displayed during public speaking. User testing on the prototype has commenced to evaluate the most effective way to display real-time feedback to users on their speaking performance.

Current interest-point tracking algorithms are either local-based or spatial-based. We propose a hybrid local-spatial interest point (IP) matching algorithm for articulated human body tracking. The first stage is local-based and finds matched pairs of IPs from two lists of reference and target IPs through a local-feature-descriptors-based matching method. The second stage of the algorithm is spatial-based. It starts with the confidently matched pairs, and recovers more matched pairs from the remaining unmatched IPs through graph matching and cyclic string matching. To compensate for the problem of Reference List Leakage (RLL), which decreases the number of reference IPs throughout the frame sequence and causes failure of tracking, an IP List Scoring and Refinement (LSR) strategy is proposed to maintain the number of reference IPs around a specific level. The proposed algorithm benefits from: local-based IP matching firstly to avoid the expense of distance and neighborhood comparison of spatial-based methods; and then spatial-based IP matching algorithm to compensate for the drawback of local-based matching stage when it fails in ambiguous situations. Experimental results show that the proposed combined local-spatial IP matching algorithm improves the matching process in comparison with local-based IP matching algorithms and tackles the problem of RLL.

The interest point (IP) matching algorithms match the points either locally or spatially. We propose a local-spatial IP matching algorithm usable for articulated human body tracking. The local-based stage finds matched IP pairs of two reference and target IP lists using a local-feature-descriptors-based matching method. Then, the spatial-based stage recovers more matched pairs from the remaining unmatched IPs through based on the result of the previous stage using the Shape Contexts (SC) feature vectors. The proposed approach benefits from the speed of local matching algorithms as well as the accuracy and robustness of spatial matching methods. Experimental results show that not only the proposed algorithm increases the precision rate from 44.71% to 97.41%, but also it improves the recall rate from 80.88% to 84.96%.

This study addresses the problem of vision‐based sign language recognition, which is to translate signs to English. The authors propose a fully automatic system that starts with breaking up signs into manageable subunits. A variety of spatiotemporal descriptors are extracted to form a feature vector for each subunit. Based on the obtained features, subunits are clustered to yield codebooks. A boosting algorithm is then applied to learn a subset of weak classifiers representing discriminative combinations of features and subunits, and to combine them into a strong classifier for each sign. A joint learning strategy is also adopted to share subunits across sign classes, which leads to a more efficient classification. Experimental results on real‐world hand gesture videos demonstrate the proposed approach is promising to build an effective and scalable system.

Recently, a promising research direction has emerged in sign language recognition (SLR) aimed at breaking up signs into manageable subunits. This paper presents a novel SL learning technique based on boosted subunits. Three main contributions distinguish the proposed work from traditional approaches: 1) A novel boosting framework is developed to recognize SL. The learning is based on subunits instead of the whole sign, which is more scalable for the recognition task. 2) Feature selection is performed to learn a small set of discriminative combinations of subunits and SL features. 3) A joint learning strategy is adopted to share subunits across sign classes, which leads to a better performance classifiers. Our experiments show that compared to Dynamic Time Warping (DTW) when applied on the whole sign, our proposed technique gives better results.

This paper describes a method of real-time facial-feature extraction which is based on matching techniques. The method is composed of facial-area extraction and mouth-area extraction using colour histogram matching, and eye-area extraction using template matching. By the combination of these methods, we can realize real-time processing, user-independent recognition and tolerance to changes of the environment. Also, this paper touches on neural networks which can extract characteristics for recognizing the shape of facial parts. The methods were implemented in an experimental image processing system, and we discuss the cases that the system is applied to man-machine interface using facial gesture and to sign language translation.