Kaligo-based Intelligent Handwriting Teacher (KIHT) is a bi-nationally funded research project. The aim of this joint project is to develop an intelligent learning device for automated handwriting, composed of existing components, which can be made available to as many students as possible. With KIHT, we specifically address the challenging task of using inertial sensors to retrace the trajectory of a pen without relying on external reference systems. The nearly unlimited freedom to let the pen glide over the paper has not yet provided a satisfactory solution to this challenge in the state-of-the-art methods, even with sophisticated algorithms and AI approaches. The final phase of the project is now being launched and together with partners from industry and academia, we are taking a holistic approach by considering the entire chain of components, from the pen to the embedded processing system, the algorithms and the app.

This paper focuses on the problem of online action detection for interactive systems, with a special emphasis on earliness. Online Action Detection (OAD) refers to the challenging task of recognizing gestures in untrimmed, streaming videos where the actions occur in unpredictable orders and durations. To address these challenges, we present a skeleton-based system for OAD incorporating a decision mechanism to accurately detect ongoing gestures. This allows us to provide instance-level output, achieving a high level of stream understanding. This mechanism relies on a novel Connectionist Temporal Classification (CTC) loss design that restricts the path possibilities according to the action boundaries. We also present a mechanism to tune the trade-off between accuracy and earliness according to the needs of the interactive system using a weighted label prior. This system includes a 3D CNN network, referred to as DOLT-C3D, exploiting the spatial–temporal information provided by the euclidean skeleton representation. We extensively evaluate our approach on eight publicly available datasets, demonstrating its superior performance compared to state-of-the-art methods in terms of both accuracy and earliness. We also successfully applied our approach to early 2D gestures detection. Furthermore, our system shows real-time performance, making it a suitable choice for interactive systems.

Early recognition of untrimmed handwritten gestures is the task of recognizing as soon as possible gestures drawn in a continuous stream, one after another. This is particularly challenging for multi-touch gestures because it is impossible to know when the gesture has started and finished. For mono-stroke gestures, in an application context where the finger is never removed from the device between gestures, the recognition is even more complex. In this work we present an extension of the Online Long-Term Convolutional 3D (OLT-C3D) network to address the task of early recognition of untrimmed gestures which have been addressed by very few works. To evaluate our approach, we created two synthetic datasets using freely available benchmarks, MTGSetB and ILGDB, simulating the streaming data in two different application scenarios. Furthermore, we propose a new evaluation metric for this specific task. Our approach achieves good performances on the two new datasets and will be a baseline for future works on this challenging task.

International audience

Online handwriting recognition is challenging but an already well-studied topic. However, recent advances in the development of convolutional neural networks (CNN) make us believe that these networks could still improve the state of the art especially in the much more challenging context of online children handwritten letters recognition. This is because, children handwriting is, at an early stage of learning, approximate and includes deformed letters. To evaluate the potential of these networks, we study the early and late fusions of different input channels that can provide a CNN with information about the handwriting dynamics in addition to the static image of the characters. The experiments on a real children handwriting dataset with 27 000 characters acquired in primary schools, show that using multiple channels with CNN, improves the accuracy performance of different CNN architectures and different fusion settings for character recognition.

In this paper, we present an extension of the bi-dimensional grammar called LCD-CMG (Layered Context Driven Constraint Multiset Grammar), a generic formalism able to model and analyse different types of structured documents. We place ourselves in the context of eager interpretation of the user's strokes, on pen-based tablets. The main constraint is therefore the real-time user interaction. LCD-CMG context sensitiveness reduces the combinatory associated to the analysis of the document structure (spacial/geometric relations between elements, e.g. the relation between a door and its associated wall). However, when we introduce complex semantic notions in the document (rooms in the architecture domain, polygons, quadrilaterals, etc in geometry), we are confronted with a combinatorial explosion. To deal with this issue, we extend the grammar parser with a semantic context analyser. This analyser models a global vision of the document, i.e. high-level relations between the elements, whereas the LCD-CMG parser models a more local vision of the document structure. The coupling of this semantic context analyser with the CD-CMG parser considerably reduces the analysis complexity, enhances the expressiveness of the formalism, and can be generalized to different types of structured documents. Experiments on two application domains, the creation of architecture plans and that of geometric shapes, show the relevance and the improvement of the extension.

This paper is based on a research project aiming at improving learning arithmetic operations at school using pen-based tablets. Given an arithmetic operation instruction, the goal is to analyze a child's handwritten answer. This comes down to find if any mistakes are made and their nature. An adapted representation and similarity search are needed for this analysis. In this paper, we propose to use a valued graph representation for handwritten arithmetical operations. To produce the analysis, we compute a similarity search with the corresponding expected answer using Graph Edit Distance (GED). To make up for the uncertainty of the noisy handwritten input recognition, we produce several segmented graph hypotheses for a single answer. Using the GED, we are able to correlate each hypothesis to the instruction graph. It enables to highlight multiple kinds of mistakes a child can make. The GED computation being a NP-complete problem, we propose to use sub-graph isomorphism: we partially match the instruction on each hypothesis in polynomial time to cut part of the tree search. Experiments were conducted on an in-house dataset composed of 400 handwritten arithmetical additions written by children on pen-based tablet. The time required for the GED computation is evaluated. We are able to match the complete operation in reasonable time on larger graphs while finding most of the time the best corresponding hypothesis.

Working with a non-stationary stream of data requires for the analysis system to evolve its model (the parameters as well as the structure) over time. In particular, concept drifts can occur, which makes it necessary to forget knowledge that has become obsolete. However, the forgetting is subjected to the stability-plasticity dilemma, that is, increasing forgetting improve reactivity of adapting to the new data while reducing the robustness of the system. Based on a set of inference rules, Evolving Fuzzy Systems - EFS - have proven to be effective in solving the data stream learning problem. However tackling the stability-plasticity dilemma is still an open question. This paper proposes a coherent method to integrate forgetting in Evolving Fuzzy System, based on the recently introduced notion of concept drift anticipation. The forgetting is applied with two methods: an exponential forgetting of the premise part and a deferred directional forgetting of the conclusion part of EFS to preserve the coherence between both parts. The originality of the approach consists in applying the forgetting only in the anticipation module and in keeping the EFS (called principal system) learned without any forgetting. Then, when a drift is detected in the stream, a selection mechanism is proposed to replace the obsolete parameters of the principal system with more suitable parameters of the anticipation module. An evaluation of the proposed methods is carried out on benchmark online datasets, with a comparison with state-of-the-art online classifiers (Learn++.NSE, PENsemble, pclass) as well as with the original system using different forgetting strategies.

This paper proposes a new architecture of incremental fuzzy inference system (also called Evolving Fuzzy System EFS). In the context of classifying data stream in non stationary environment, concept drifts problems must be addressed. Several studies have shown that EFS can deal with such environment thanks to their high structural flexibility. These EFS perform well with smooth drift (or incremental drift). The new architecture we propose is focused on improving the processing of brutal changes in the data distribution (often called brutal concept drift). More precisely, a generalized EFS is paired with a module of anticipation to improve the adaptation of new rules after a brutal drift. The proposed architecture is evaluated on three datasets from UCI repository where artificial brutal drifts have been applied. A fit model is also proposed to get a "reactivity time" needed to converge to the steady-state and the score at end. Both characteristics are compared between the same system with and without anticipation and with a similar EFS from stateof-the-art. The experiments demonstrates improvements in both cases.

This paper presents a fuzzy visibility graph representation for handwritten mathematical expressions (HME) computed over segmented symbols using learned fuzzy landscape (FL) models. The learned FL models define the relative positioning of a pair of symbols using both their morphology, their typology and their context. A Random Forest Classifier uses this relative positioning to qualify relationships between symbols. The valued fuzzy visibility graph with the FL membership is produced from this classifier's output. This graph offers an explicit representation of the HME bi-dimensional structure which is then parsed with a set of rules to produce the recognized HME. We evaluate the performance of this system on the task of HME structure recognition using provided segmented symbols with experimental results on both CROHME 2014 and 2016 datasets. We obtain results up to par with the state-of-the-art thus proving that our fuzzy visibility graphs are a strong representation for mathematical expression parsing.

This paper presents a generic method for eager interpretation of online hand-drawn sketches. This work takes place within the ACTIF project, with the objective of designing an e-learning system for geometry lessons in middle-schools. The goal is to analyze on the fly the child's inputs (using a pen-based interaction on tablet), and to give real-time feedback (visual, corrective, or guidance ones). We are therefore faced with the constraint of real-time strokes interpretation. We propose to use an incremental analysis process enabling to decrease the search space size. Nevertheless, we encounter combinatorial problems given the application field is complex. For example, create subfigures from an existing one means the parser has to consider all possible combinations of connected segments that are contained in the original shape. To tackle this issue, we extend the definition and the parsing of bi-dimensional grammars by formalizing a hierarchy between production rules. Therefore, we propose alternative exploration strategies and reduce the search space of applicable production rules at any given level of the analysis. Thus, the parser is able to dynamically switch between the exploration strategies in the analysis process. The system has been tested with different drawing scenarios of geometric figures sketching. It reached interesting real-time performance, without loosing the generic aspect of the approach.

Exergames involve using the fullbody to interact with an immersive world, which raises the challenge of capturing, processing and recognizing the action of the user even for cheap mocap systems such as the Microsoft Kinect. In fact, these recent technological advances have renewed interest in skeleton-based action recognition. Our review of related literature reveals that the issues encountered are not the result of random processes, which could simply be studied by using statistical tools, but are instead due to the fact that the pattern to be recognized, i.e. an action, was produced by a human being. 2D hand-drawn symbols are further examples of patterns resulting from a human motion. Therefore, the main contribution of this paper is to examine the validity of transferring the expertise of hand-drawn symbol representation to better recognize actions based on skeleton data. Principally, we propose a new action representation, namely the 3DMM, as an initial case-study illustrating how such transfer could be conducted. The experimental results, obtained over two benchmarks, confirm the soundness of our approach and encourage more thorough examination of the transfer.

In this paper a multi-classifier method for early recognition of handwritten gesture is presented. Unlike the other works which study the early recognition problem related to the time, we propose to make the recognition according to the quantity of incremental drawing of handwritten gestures. We train a segment length based multi-classifier for the task of recognizing the handwritten touch gesture as early as possible. To deal with potential similar parts at the beginning of different gestures, we introduce a reject option to postpone the decision until ambiguity persists. We report results on two freely available datasets: MGSet and ILG. These results demonstrate the improvement we obtained by using the proposed reject option for the early recognition of handwritten gestures.

The papers in this special section focus on handwriting and drawing processes for user-centered systems. The papers provide a wide and updated overview of the frontier of research in the field of humancentered systems based on drawing and handwriting processing. Through the papers, some of the most relevant directions of further research are highlighted with specific attention to components related to human–machine interaction. The Guest Editors hope that this issue brings forth the importance of automated systems related to automatic processing of drawing and handwriting.

We propose in this paper a new online Multi-User Multi-Touch handwritten diagram DataBase (MUMTDB) for evaluating recognition systems under the multi-user situation. The data is collected according to two predefined mind map scenarios which contains 9 classes of graphical symbols. Each scenario is completed by involving two users at the same time. Since the users are given freedom to draw the symbols as they want, the dataset contains a diversity of multi-stroke and even multi-touch symbols. It allows addressing new challenging problems regarding the recognition of simultaneous composition of structured documents. The dataset is freely available on-line.

Action recognition based on human skeleton structure represents nowadays a prosper research field. This is mainly due to the recent advances in terms of capture technologies and skeleton extraction algorithms. In this context, we observed that 3D skeleton-based actions share several properties with handwritten symbols since they both result from a human performance. We accordingly hypothesize that the action recognition problem can take advantage of trial and error already carried out on handwritten patterns. Therefore, inspired by one of the most efficient and compact handwriting feature-set, we propose in this paper a skeleton descriptor referred to as Handwriting-Inspired Features (HIF3D). First of all a data preprocessing is applied to joint trajectories in order to handle the variabilities among actor's morphologies. Then we extract the HIF3D features from the processed joint locations according to a time partitioning scheme so as to additionally encode the temporal information over the sequence. Finally, we selected the Support Vector Machine (SVM) to achieve the classification step. Evaluations conducted on two challenging datasets, namely HDM05 and UTKinect, testify the soundness of our approach as the obtained results outperform the state-of-the-art algorithms that rely on skeleton data.

The present study was designed to ascertain how far flagging up potential errors can improve the automatic interpretation of technical documents. We used the resources model to analyze the supervised retro-conversion of architectural floor plans from the perspective of distributed cognition. Results showed that automated assistance helps users to correct errors spotted by the system and saves time. Surprisingly, they also showed that flagging up possible errors may make users less effective in identifying and correcting errors that go unnoticed by the system. Responses to a questionnaire probing the participants’ confidence in the system suggested that they were so trusting that they lowered their vigilance in those areas that had not been signaled by the system, leading to the identification of fewer errors there. Thus, although the participants’ confidence in the automated assistance system led to improved performances in those areas it highlighted, it also meant that areas to which the system did not draw attention were less thoroughly checked.

Touch sensitive interface enables new interaction methods, like using gesture commands. The use of gesture commands give rise to a cross-learning situation where the user has to learn and memorize the command gestures and the classifier has to learn and recognize drawn gestures. To easily memorize more than a dozen of gesture commands, it is important to be able to customize them. The classification task associated with the use of customized gesture commands is complex because the classifier only has very few samples per class to start learning from. We thus need an evolving recognition system that can start from very few data samples and that will learn incrementally to achieve good performance after some using time. This article presents the impact of using rejection based user interactions to supervise the on-line training of the evolving classifier. The objective is to obtain a gesture command system that cooperates as best as possible with the user: to learn from its mistakes without soliciting him too often. To detect confusing classes we apply confusion reject principles to our evolving recognizer, which is based on a first order fuzzy inference system. A significant user experiment has been performed on 63 persons that validates our approach. This user experiment shows the interest of optimizing user interactions by taking into account the confusion detection capability of our recognition system.

In most applications of touch based human computer interaction, multi-touch gestures are used for directly manipulating the interface such as scaling, panning, etc. In this paper, we propose using multi-touch gesture as indirect command, such as redo, undo, erase, etc., for the operating system. The proposed recognition system is guided by temporal, spatial and shape information. This is achieved using a graph embedding approach where all previous information are used. We evaluated our multi-touch recognition system on a set of 18 different multi-touch gestures. With this graph embedding method and a SVM classifier, we achieve 94.50% recognition rate. We believe that our research points out a possibility of integrating together raw ink, direct manipulation and indirect command in many gesture-based complex application such as a sketch drawing application.

Touch sensitive interfaces enable new interaction methods like using gesture commands. To easily memorize more than a dozen of gesture commands, it is important to be able to customize them. The classifier used to recognize drawn symbols must hence be customisable, able to learn from very few data, and evolving, able to learn and improve during its use. This work studies different supervision strategies for the online training of the evolving classifier. We compare six supervision strategies, depending on user interaction (solicitation by the system), and self-evaluation capacities (notion of reject). In particular, there is a trade-off between the number of user interactions, to supervise the online training, and the error rate of the classifier. We show in this paper that the strategy giving the best results is to learn from data validated by the user, when the confidence of the recognition is too low, and from data implicitly validated.

This paper presents a new method to help users defining personalized gesture commands (on pen-based devices) that maximize recognition performance from the classifier. The use of gesture commands give rise to a cross-learning situation where the user has to learn and memorize the command gestures and the classifier has to learn and recognize drawn gestures. The classification task associated with the use of customized gesture commands is complex because the classifier only has very few samples per class to start learning from. We thus need an evolving recognition system that can start from scratch or very few data samples and that will learn incrementally to achieve good performance after some using time. Our objective is to make the user aware of the recognizer difficulties during the definition of commands, by detecting confusion among gesture classes, in order to help him define a gesture set that yield good recognition performance from the beginning. To detect confusing classes we apply confusion reject principles to our evolving recognizer, which is based on a first order fuzzy inference system. A realistic experiment has been made on 55 persons to validate our confusion detection technique, and it shows that our method leads to a significant improvement of the classifier recognition performance.

In this paper, we present the impact of user interaction in the recognition of off-line structured documents. This interaction requires solving two major problems: how interpretation results will be presented to the user, and how the user will interact with analysis process. We propose to study the effects of those two aspects in the context of an interactive method (IMISketch) for off-line handwritten 2D architectural floor plan recognition. The use tests are realized in collaboration with researchers in cognitive psychology (more than 100 persons participated in the tests). The experiments demonstrate that (i) a progressive presentation of the analysis results, (ii) user interventions during it and (iii) the user solicitation by the analysis process are an efficient strategy for interactive recognition of offline documents.

In this paper, we present an optimized approach, based on a competitive breadth-first exploration of the analysis tree, for an interactive interpretation of off-line sketch. The competitive breadth-first exploration of the analysis tree, allows to compare several hypotheses of interpretation to deal with confusion. Unfortunately, in practice these methods are rarely used because they often induce a large combinatory. This paper presents an optimization strategy to minimize the combinatory. The aim is to demonstrate the relevance of a competitive breadth-first exploration in off-line document analysis, in particular when the approach is interactive, ie the user is involved into the loop analysis. This paper demonstrates this optimized interactive analysis method on off-line handwritten 2D architectural floor plans.

We work on new strategies of exploration for interpretation of off-line sketches. A first approach (call IMISketch) was based on a competitive breadth-first exploration of the analysis tree allowing to evaluate simultaneously several possible hypotheses of recognition in a dynamic local context of document. A great advantage of this strategy is to be able to solicit the user during the decision process to avoid error accumulation in the analysis step. IMISketch strategy is very interesting but it can lead combinatory problems when addressing complex sketches. In this paper, we propose a new hybrid strategy for exploration. The recognition process alternates between a breadth-first and depth-first exploration. The strategy is totally driven by the grammatical description of the document. The paper demonstrates the interest of this new hybrid strategy method on handwritten 2D architectural floor plans containing walls, opening and furnitures.

Pen-based interfaces allow users to interact with the help of a stylus and/or fingers. Thanks to a recognition system, users can execute commands by drawing gestures. Because of the complexity of the software, the set of gesture commands can be very large. Consequently, it becomes difficult for the users to remember all the commands. In this paper we introduce "Semi-customizable gestural commands". The idea is to take into account the commands usage frequency: to facilitate the memorization of the most used commands we leave users the freedom/liberty to define their associated gestures (by) themselves. The remaining gesture commands will be automatically generated on the base of the defined gestures according to their hierarchical categorization. Several comparative tests demonstrate that this new approach improved the progressive memorization of the gestural commands.

This paper tackles the problem of decremental learning of an evolving classification system. We study the use of decremental learning to improve performance of evolving recognizers in non-stationary scenarios. Our on-line recognizer is based on an evolving fuzzy inference system. In this paper, we propose a new strategy to introduce decremental learning, with the use of a sliding window, in the optimization of fuzzy rules conclusions. This approach is based on a downdating technique of least squares solutions for unlearning old data. This technique is evaluated on handwritten gesture recognition tasks. In particular, it is shown that this downdating techniques allow to adapt to concept drifts and that we face a precision reactiveness trade-off. It is also demonstrated that decremental learning is necessary to maintain the system learning capacity over time, making decremental learning essential for the life-time use of an evolving classification system.

Relative positioning between components of a structured object plays a key role for its interpretation. Fuzzy relative positioning templates are a description framework for 2D handwritten patterns, that is based on positioning models specifically designed for dealing with variability and imprecision of handwriting. In this work, we present fuzzy positioning templates and investigate the idea of recognizing structured handwritten symbols by considering the relative positioning of the components, rather than the shapes of the components themselves or the global shape of the symbol. The templates are automatically trained from data without requiring any prior knowledge. Experiments on a database of on-line symbols prove that this original strategy is a promising approach for interpretation of structured patterns.

In this paper, we present a new generic method for an interactive interpretation of sketches. This method is based on a competitive breadth-first exploration of the analysis tree. As opposed to well known structural approaches, this method allows to evaluate simultaneously several possible hypotheses of recognition in a dynamic local context of document. At each step of the analysis, the decision process selects the best hypotheses. If it detects an ambiguity, it will solicit the user to select the right hypothesis. In fact, the user participation has a great impact to avoid error accumulation during the analysis step and overcomes the combinatory due to the sketch complexity. This paper demonstrates this interactive method on 2D architectural floor plans.

In this paper, we present a new type of Marking menus. Continuous Marking Menus are specifically dedicated to pen-based interfaces, and designed to define a set of cursive, realistic handwritten gestures. In menu mode, they offer a continuous visual feedback and fluent exploration of menu hierarchy, inviting the user to execute cursive gestures for invoking the desired commands. In marking mode, a specific gesture recognition method is proposed and proved to be very efficient for recognizing cursive gestures.

In this paper, we present a new method to design customizable self-evolving fuzzy rule-based classifiers. The presented approach combines an incremental clustering algorithm with a fuzzy adaptation method in order to learn and maintain the model. We use this method to build an evolving handwritten gesture recognition system, that can be integrated into an application to provide personalization capabilities. Experiments on an on-line gesture database were performed by considering various user personalization scenarios. The experiments show that the proposed evolving gesture recognition system continuously adapts and evolve according to new data of learned classes, and remains robust when introducing new unseen classes, at any moment during the lifelong learning process.

We present in this paper a new method for the design of evolving neuro-fuzzy classifiers. The presented approach is based on a first-order Takagi-Sugeno neuro-fuzzy model. We propose a modification on the premise structure in this model and we provide the necessary learning formulas, with no problem-dependent parameters. We demonstrate by the experimental results the positive effect of this modification on the overall classification performance.

In this paper, we present a new method to design customizable self-evolving fuzzy rule-based classifiers. The presented approach combines an incremental clustering algorithm with a fuzzy adaptation method in order to learn and maintain the model. We use this method to build an evolving handwritten gesture recognition system. The self-adaptive nature of this system allows it to start its learning process with few learning data, to continuously adapt and evolve according to any new data, and to remain robust when introducing a new unseen class at any moment in the life-long learning process.

In this paper, we present a new incremental learning strategy for handwritten character recognition systems.This learning strategy enables the recognition system to learn ldquorapidlyrdquo any new character from very few examples.The presented strategy is driven by a confusion detection mechanism in order to control the learning process. Artificial characters generation techniques are used to overcome the problem of lack of learning data when introducing a new character from unseen class. The results show that a good recognition rate (about 90%) is achieved after only 5 learning examples. Moreover, the rate quickly rises to 94% after 10 examples, and approximately 97% after 30 examples. A reduction of error of 40% is obtained by using the artificial characters generation techniques.

In this paper, we present a new method for on-line Chinese character recognition that relies on an explicit description of characters structure. Contrary to most of known structural approaches, this model can describe characters written in a fluent style, thanks to a flexible fuzzy modeling of shapes and positioning of their structural components (primitives and radicals). We designed a process for incremental training of the models cooperated with automatic structural labeling for minimizing the required manual task in model design. First experiments show that the method is able to recognize non-regularly written characters and has a convincing generalization ability.

In this paper we investigate the use of linguistic information given by language models to deal with word recognition errors on handwritten sentences. We focus especially on errors due to out-of-vocabulary (OOV) words. First, word posterior probabilities are computed and used to detect error hypotheses on output sentences. An SVM classifier allows these errors to be categorized according to defined types. Then, a post-processing step is performed using a language model based on part-of-speech (POS) tags which is combined to the n-gram model previously used. Thus, error hypotheses can be further recognized and POS tags can be assigned to the OOV words. Experiments on on-line handwritten sentences show that the proposed approach allows a significant reduction of the word error rate.

This paper presents a word extraction approach based on the use of a confidence index to limit the total number of segmentation hypotheses in order to further extend our online sentence recognition system to perform "on-the-fly" recognition. Our initial word extraction task is based on the characterization of the gap between each couple of consecutive strokes from the online signal of the handwritten sentence. A confidence index is associated to the gap classification result in order to evaluate its reliability. A reconsideration process is then performed to create additional segmentation hypotheses to ensure the presence of the correct segmentation among the hypotheses. In this process, we control the total number of segmentation hypotheses to limit the complexity of the recognition process and thus the execution time. This approach is evaluated on a test set of 425 English sentences written by 17 writers, using different metrics to analyze the impact of the word extraction task on the whole sentence recognition system performances. The word extraction task using the best reconsideration strategy achieves a 97.94% word extraction rate and a 84.85% word recognition rate which represents a 33.1% word error rate decrease relatively to the initial word extraction task (with no segmentation hypothesis reconsideration).

In this paper, we propose an original hybrid statistical-structural method for on-line Chinese character recognition. We model characters thanks to fuzzy inference rules combining morphological and contextual information formalized in a homogeneous way. For that purpose, we define a set of primitives modeling all the stroke classes that can be found in handwritten Chinese characters. Thus, each analyzed stroke can be classified as primitive without any segmentation process. Inference rules are built from the coupling of a priori information about the primitives constituting the characters and automatic modeling of their relative positioning. The fuzzy inference system aggregates these rules for decision making. First experiments validate this method with a recognition rate of 97.5% on a subset of Chinese characters.

In this paper we investigate the integration of a confusion network into an on-line handwritten sentence recognition system. The word posterior probabilities from the confusion network are used as confidence scored to detect potential errors in the output sentence from the Maximum A Posteriori decoding on a word graph. Dedicated classifiers (here, SVMs) are then trained to correct these errors and combine the word posterior probabilities with other sources of knowledge. A rejection phase is also introduced in the detection process. Experiments on handwritten sentences show a 28.5 % relative reduction of the word error rate.

In this paper, we present an approach for separating text and non-text ink strokes in online handwritten Japanese documents based on Markov random fields (MRFs), which effectively utilize the spatial relationship between strokes. Support vector machine (SVM) classifiers are trained for individual stroke and stroke pair classification, and on converting the SVM outputs to probabilities, the likelihood clique potentials of MRF are derived. In experiments on the TUAT Kon-date database, the proposed MRF approach yield superior performance compared to individual stroke classification and sequence classification based on hidden Markov models (HMMs).

This study presents an automatic online adaptation mechanism to the handwriting style of a writer for the recognition of isolated handwritten characters. The classifier we use here is based on a Fuzzy Inference System (FIS) similar to those we have designed for handwriting recognition. In this FIS each premise rule is composed of a fuzzy prototype which represents intrinsic properties of a class. Furthermore, the conclusion part of rules associates a score to the prototype for each class. The adaptation mechanism affects both the conclusions of the rules and the fuzzy prototypes by recentering and reshaping them thanks to a new approach called ADAPT inspired by the Learning Vector Quantization. Thus the FIS is automatically fitted to the handwriting style of the writer that currently uses the system. Our adaptation mechanism is compared with well known adaptation techniques. The tests were based on eight different writers and the results illustrate the benefits of the method in terms of error rate reduction (86% in average). This allows such kind of simple classifiers to achieve up to 98.4% of recognition accuracy on the 26 Latin letters in a writer dependent context.

In this paper, we present a new approach that explicitly exploits the spatial context of strokes to drive the shape recognition. We call this recognition method "context driven recognition" (CDR). The underlying idea is that only a sub-set of all possible symbols can be recognized in a specific spatial context. The main challenge is to detect and model automatically the context areas of interest so that the recognition method can be independent of any specific information on the targeted pen-based application. The paper details the learning scheme of the CDR method and how the obtained model is used during the recognition process. The results on a real-world pen-based recognition problem show that the method can reach better performances than a classical approach by decreasing the shape recognition complexity.

We present an automatic on-line adaptation mechanism to the writer's handwriting style for the recognition of isolated handwritten characters. The classifier is based on a fuzzy inference system (FIS). This FIS is composed of fuzzy prototypes which represent the intrinsic properties of the classes and it uses numeric conclusions. The proposed adaptation mechanism affects both the conclusions of the rules and the fuzzy prototypes of the premises by re-centering and re-shaping them. Doing so, the FIS is automatically fitted to the handwriting style of the writer that is currently using the system. This adaptation mechanism has been tested with 8 different writers. The results show the adaptation mechanism is able to improve the recognition rate from 88% to 98.2% in average for the 26 Latin letters.

This paper investigates the integration of a statistical language model into an on-line recognition system in order to improve word recognition in the context of handwritten sentences. Two kinds of models have been considered: n-gram and n-class models (with a statistical approach to create word classes). All these models are trained over the Susanne corpus and experiments are carried out on sentences from this corpus which were written by several writers. The use of a statistical language model is shown to improve the word recognition rate and the relative impact of the different language models is compared. Furthermore, we illustrate the interest to define an optimal cooperation between the language model and the recognition system to re-enforce the accuracy of the system.

This paper presents a method to generate an equivalent on-line signal of a letter image. To do this, an off-line drawing order has to be recovered using handwriting knowledge. Our approach consists in looking for several start and the end points. Then a reconstruction algorithm is applied and finally the best path is chosen. This approach has been validated on a database of handwritten letters including both on-line and off-line signals. An on-line recognition system has been used to recognize these letters.

This paper presents a lexical post-processing optimization for handwritten word recognition. The aim of this work is to explore the combination of different lexical post-processing approaches in order to optimize the recognition rate, the recognition time and memory requirements. The present method focuses on the following tasks: a lexicon organization with word filtering, based on holistic word features to deal with large vocabulary (creation of static sublexicon compressed in a tree structure); a dedicated string matching algorithm for online handwriting (to compensate for the recognition and the segmentation errors); and a specific exploration strategy of the results provided by the analytical word recognition process. Experimental results are reported using several lexicon sizes (about 1000, 7000 and 25000 entries) to evaluate different optimization strategies according to the recognition rate, computational cost and memory requirements.

In our previous works, a recognition system named ResifCar was designed specifically for on-line handwritten character recognition. This system is based on an explicit modeling by hierarchical fuzzy rules. Thus, it is understandable an optimizable after the learning stage. We present in this article a new classifier that is an extension of ResifCar. Indeed it tries to combine ResifCar's advantages with a generic aspect to handle different recognition problems. This new hybrid system combines two complementary levels. The first one uses a robust modeling by an intrinsic fuzzy clustering of each class and determines their confusing areas. The second level, based on fuzzy decision trees, operates a progressive discrimination inside these areas. Both levels are formalized by fuzzy inference systems organized hierarchically and fused for final decision. Experiments were conducted on the one hand on classical benchmarks and on the other hand on on-line handwritten digits and lower-case letters. For all of these cases, the classifier achieves good recognition rates without final optimization. 1.

This paper presents a new hybrid learning method for the construction of fuzzy decision trees. The main principle of this approach is to automatically generates a hierarchical organization of the knowledge coupled with local choice of the best feature subspace. To improve the representation, a double level of modeling is used. Firstly a pre-classification level searches fuzzy decision regions to operate a natural discrimination between classes. The second level refines the previous one, doing an intrinsic fuzzy modeling of the classes represented in the fuzzy regions. Moreover, the best feature subspace is determined locally by a genetic algorithm for each partitioning. Finally, to have an understandable and "transparent" representation, the fuzzy decision tree is formalized as a fuzzy inference system which is easily modifiable and can be optimized a posteriori. First experimental results conducted on classical benchmarks and on a handwritten digits database show the capacity of the hybrid learning approach to provide reliable and compact classification system.

A new handwriting modeling and segmentation approach is introduced for cursive letter and word analysis. For the letter analysis, the proposed method is based on the detection of a set of "perceptual anchorage points" to extract a priori pertinent strokes. This physical segmentation of the handwritten drawing enables us to conduct a logical modeling of letters with respect to the most stable strokes of each letter class. For the handwritten word analysis, we present a constructive segmentation approach to overcome the word segmentation problem. The main idea is to locate "anchorage structures" in the word drawing based on the most robust strokes of the letters. This new approach of handwriting analysis has been implemented in a writer-independent online handwriting recognition system. Experimental results are reported using a lexicon context of 1128, 7000 and 25,000 words.