Handwriting perceptual classification and synthesis using discriminate HMMs and progressive iterative approximation

  • Hala BezineEmail author
  • Adel M. Alimi


This paper handles the problem of online handwriting synthesis. Indeed, this work presents a probabilistic model using the embedded hidden Markov models (HMMs) for the classification and modeling of perceptual sequences. At first, we start with a vector of perceptual points as input seeking a class of basic shape probability as output. In fact, these perceptual points are necessary for the drawing and the recovering of each basic shape where each one is designed with an HMM built and trained with its components. Each path through these possibilities of control points represents an observation that serves as input for the following step. Secondly, the already detected sequences of observations which represent a segment formed an initial HMM and the concatenation of multiple ones leads to a global HMM. To classify a global HMM, we should codify it by searching the best path of initial HMM. The best path is obtained by computing the maximum of likelihood of the different basic shapes. In order to synthesize the handwritten trace, and to recover the best control points sequences, we investigated the progressive iterative approximation. The performance of the proposed model was assessed using samples of scripts extracted from IRONOFF and MAYASTROON databases. In fact, these samples served for the generation of the set of control points used for the HMMs training models. In experiments, good quantitative agreement and approximation were found for the generated trajectories and a more reduced representation of the scripts models was designed.


Cursive handwriting synthesis Embedded hidden Markov models Visual perceptual codes Control points Progressive iterative interpolation 



The authors would like to acknowledge the financial support of this work by grants from General Direction of Scientific Research (DGRST), Tunisia, under the ARUB 01/UR/11/02 program.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. 1.
    Alvaro F, Sanchez J-A, Benedi J-M (2014) Recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models. Pattern Recogn Lett 35:58–67CrossRefGoogle Scholar
  2. 2.
    Aycard O, Mari JF, Washington R (2004) Learning to automatically detect features for mobile robots using second-order hidden Markov models. Int J Adv Robot Syst 1(4):29CrossRefGoogle Scholar
  3. 3.
    Bag S, Bhowmick P, Harit G (2011) Recognition of Bengali handwritten characters using skeletal convexity and dynamic programming. In: International conference on emerging applications of information technology, pp 265–268Google Scholar
  4. 4.
    Bag S, Bhowmick P, Harit G (2012) Detection of structural concavities in character images—a writer-independent approach. In: First Indo-Japan conference, PerMIn, pp 260–268Google Scholar
  5. 5.
    Berio D, Akten M, Leymarie F, Grierson M, Plamondon R (2017) Calligraphic stylization learning with a physiologically plausible model of movement and recurrent neural networks. In: 4th International conference on movement computing, MOCO’2017,
  6. 6.
    Bezine H, Alimi AM (2013) Development of an Arabic handwriting learning educational system. Int J Softw Eng Appl 4(2):33–49Google Scholar
  7. 7.
    Bezine H, Alimi AM (2016) Analysis and synthesis of handwriting movements via the enhanced Beta-elliptic model. In: International conference on systems, signals and devices, pp 295–300Google Scholar
  8. 8.
    Bezine H, Ghanmi W, Alimi MA (2014) A HMM model based on perceptual codes for on-line handwriting generation. In: International conference on advances cognitive technologies and applications, Cognitive’ Italy, pp 126–132Google Scholar
  9. 9.
    Bezine H, Kefi M, et Alimi AM (2007) On the Beta-elliptic model for the control of the human arm movement. Int J Pattern Recognit Artif Intell 1(21):5–19CrossRefGoogle Scholar
  10. 10.
    Bouaziz S, Magnan A (2007) Contribution of the visual perception and graphic production systems to the copying of complex geometrical drawings: a developmental study. Cognit Dev 22(1):5–15. CrossRefGoogle Scholar
  11. 11.
    Bullock D, Grossberg S, Mannes C (1993) A neural network model for cursive script production. Biol Cybern 70:15–28zbMATHCrossRefGoogle Scholar
  12. 12.
    Chang WD, Shin J (2012) A statistical handwriting model for style-preserving and variable character synthesis. Int J Doc Anal Recognit 15:1–19CrossRefGoogle Scholar
  13. 13.
    Chatzis S, Kosmopoulos D, Papadourakis GA (2016) non stationary hidden Markov model with approximately infinitely-long time-dependencies. Int J Artif Intell Tools 25(5):51–62CrossRefGoogle Scholar
  14. 14.
    Choi H, Kim J (2003) Generation of handwritten characters with Bayesian network based on-line handwriting recognizers. In: Proceedings of ICDAR’03, England, pp 995–999Google Scholar
  15. 15.
    Choi T, Li M, Fu K, Lin L (2018) Music sequence prediction with mixture Hidden Markov models. arXiv preprint arXiv:1809.00842
  16. 16.
    Cui Y, Mousas C (2018) Master of puppets: an animation-by-demonstration computer puppetry authoring framework. 3D Res 9(1):1–14CrossRefGoogle Scholar
  17. 17.
    Danna J, Fontaine M, Paz-Villagran V, Gondre C, Thoret E, Aramaki M, Kronland MR, Ystad S, Velay J-L (2015) The effect of real-time auditory feedback on learning new characters. Hum Mov Sci 43:216–228CrossRefGoogle Scholar
  18. 18.
    Dean TA, SinghS S, Jasra A, Peters GW (2014) Parameter estimation for hidden Markov models with intractable likelihoods. J Stat 41(4):970–987MathSciNetzbMATHGoogle Scholar
  19. 19.
    Ding S, Bian W, Liao H, Sun T, Xue Y (2017) Combining Gabor filtering and classification dictionaries learning for fingerprint enhancement. IET Biom 6(6):438–447CrossRefGoogle Scholar
  20. 20.
    Ding S, Zhao X, Xu H, Zhu Q, Xue Y (2018) NSCT-PCNN image fusion based on image gradient motivation. IET Comput Vis 12(4):377–383CrossRefGoogle Scholar
  21. 21.
    Flash T, Hogan N (1985) Moving gracefully: quantitative theories of motor coordination. Neuroscience 10(4):170–174Google Scholar
  22. 22.
    Forney G (1972) The viterbi algorithm. Proc IEEE 61(3):268–278MathSciNetCrossRefGoogle Scholar
  23. 23.
    Gangadhar G, Chakravarthy VS, Joseph D (2007) An oscillatory neuromotor model of handwriting generation. Int J Doc Anal Recognit (IJDAR) 10(2):69–84. CrossRefGoogle Scholar
  24. 24.
    Gilet E (2009) Modélisation bayésienne d’une boucle perception-action: Application à la lecture et à l’écriture. Grenoble, France, Ph.D., University Joseph FourieGoogle Scholar
  25. 25.
    Gilloux M (1994) Hidden Markov models in handwriting recognition. Springer, Berlin, pp 264–288Google Scholar
  26. 26.
    Graves A (2014) Generating sequences with recurrent neural networks. Neural Evol Comput 14:1–43MathSciNetGoogle Scholar
  27. 27.
    Grossberg S, Paine RW (2000) A neural model of corticocerebellar interactions during attentive imitation and predictive learning of sequential handwriting movements. Neural Netw 2:999–1046CrossRefGoogle Scholar
  28. 28.
    Guyon I (1996) Handwriting synthesis from handwritten glyphs. In: Proceedings of IWFHR’96, England, pp 309–312Google Scholar
  29. 29.
    Hollerbach J (1981) An oscillation theory of handwriting. Biol Cybern 39:139–156CrossRefGoogle Scholar
  30. 30.
    Hu L, Zanibbi R (2011) Segmenting handwritten math symbols using AdaBoost and multi-scale shape context features. In: International conference on document analysis and recognition ICDAR’2011, pp 1180–1184Google Scholar
  31. 31.
    Hu Z, Xu Y, Huang L, Leung H (2009) A Chinese handwriting system with automatic error detection. Int J Softw Spec Issue Adv Distance Learn Technol 4(2):101–107Google Scholar
  32. 32.
    Ioannidou ZS, Theodoropoulou MC, Papandreou NC, Willis JH, Hamodrakas SJ (2014) Cutprotfam-pred: detection and classification of putative structural cuticular proteins from sequence alone based on profile hidden Markov models. Insect Biochem Mol Biol 52:51–59CrossRefGoogle Scholar
  33. 33.
    Jawahar CV, Balasubramanian A, Nambo AM (2009) Retrieval of online handwriting by synthesis and matching. Int J Pattern Recogn 42:1445–1457zbMATHCrossRefGoogle Scholar
  34. 34.
    Kalveram KT (1998) A neural oscillator model learning given trajectories. Motor Control and Human skill: A multi-disciplinary perspective, 127–140Google Scholar
  35. 35.
    Kherallah M, Haddad L, Alimi AM (2009) A new approach for online Arabic hand-writing recognition. In: Proceedings of 2nd international conference on Arabic language resources and tools, pp 22–23Google Scholar
  36. 36.
    Kuo S, Agazzi O (1994) Keywords spotting in poorly printed documents using psuedo 2-D hidden Markov models. IEEE Trans Pattern Anal Mach Intell 16:842–848CrossRefGoogle Scholar
  37. 37.
    Langrock R, Kneib T, Sohn A, DeRuiter SL (2015) Non parametric inference in hidden Markov models using p-splines. Biometrics 71(2):520–528MathSciNetzbMATHCrossRefGoogle Scholar
  38. 38.
    Lee Y-S, Cho S-B (2011). Activity recognition using hierarchical hidden Markov models on a smartphone with 3D accelerometer. In: International conference on hybrid artificial intelligence systems, pp 460–467Google Scholar
  39. 39.
    Lin Z, Wan L (2007) Style-preserving English handwriting synthesis. Pattern Recog. 40(7):2097–2109zbMATHCrossRefGoogle Scholar
  40. 40.
    Ltaief M, Bezine H, Alimi MA (2012) A neuro-Beta-elliptic model for handwriting generation movements. In: International conference on frontiers in handwriting recognition, ICFHR’2012, Italy, pp 799–803Google Scholar
  41. 41.
    Ltaief M, Bezine H, Alimi MA (2016) A Spiking neural network model for complex handwriting movements generation. Int J Comput Sci Inf Sec IJCSIS 14(7):319–327Google Scholar
  42. 42.
    Ltaief M, Njah S, Bezine H, Alimi MA (2012) Genetic algorithms for perceptual codes extraction. Int J Intell Learn Syst Appl 4:256–265Google Scholar
  43. 43.
    Malaviya A, Peters L, Camposano R (1993) A fuzzy online handwriting recognition system: FOHRES. In: International conference on fuzzy theory and technology, USA, pp 1–15Google Scholar
  44. 44.
    Mari J, Fohr D, Junqua J (1996) A second-order hmm for high performance word and phoneme-based continuous speech recognition. Int Conf Acoust Speech Signal Process 1:435–438Google Scholar
  45. 45.
    Mousas C (2017) Full-body locomotion reconstruction of virtual characters using a single inertial measurement unit. Sensors 17:11. CrossRefGoogle Scholar
  46. 46.
    Mousas C (2018) Performance-driven dance motion control of a virtual partner character. In: International conference on virtual reality and 3D user interfaces, pp 57– 64Google Scholar
  47. 47.
    Mousas C, Anagnostopoulos C-N (2017) Real-time performance-driven finger motion synthesis. Comput Gr 65:1–11CrossRefGoogle Scholar
  48. 48.
    Njah S, Nouma B, Bezine H, Alimi AM (2012) MAYASTROUN: a multi language handwriting database. In: International conference on frontiers in handwriting recognition ICFHR’2012, Italy, pp 308–312Google Scholar
  49. 49.
    Plamondon R (1989) A handwriting model based on differential geometry. In: Plamondon R, Suen CY, Simner M (eds) Computer recognition and human production of handwriting. World Scientific Publisher, Singapore, pp 179–192CrossRefGoogle Scholar
  50. 50.
    Plamondon R, Guerfali W (1998) The generation of handwriting with delta-lognormal synergies. Biol Cybern 78:119–132zbMATHCrossRefGoogle Scholar
  51. 51.
    Qiao S, Shen D, Wang X, Han N, Zhu W (2015) A self-adaptive parameter selection trajectory prediction approach via hidden Markov models. IEEE Trans Intell Trans Syst 16(1):284–296CrossRefGoogle Scholar
  52. 52.
    Rabiner L (1989) A tutorial on HMM and selected applications in speech recognition. Proc IEEE 77(2):257–286CrossRefGoogle Scholar
  53. 53.
    Ramaiah C, Plamondon R, Govindaraju V (2014) A sigma-lognormal model for handwritten text CAPTCHA generation. In: Proceedings of international conference on pattern recognition, ICPR’2014, pp 250–254Google Scholar
  54. 54.
    Rémi C, Frelicot C, Courtellemont P (2002) Automatic analysis of the structuring of children’s drawing and writing. Pattern Recogn 35(5):1059–1069zbMATHCrossRefGoogle Scholar
  55. 55.
    Rusu A, Govindaraju V (2004) Handwritten CAPTCHA: using the difference in the abilities of humans and machines in reading handwritten words. In: Proceedings of IWFHR’2004, pp 586–591Google Scholar
  56. 56.
    Schomaker L (1991) Simulation and recognition of handwriting movements: a vertical approach to modeling Human motor behavior. Netherlands, Dissertation, University NijmegenGoogle Scholar
  57. 57.
    Senatore R, Marcelli A (2012) A neural scheme for procedural motor learning of handwriting. In: International conference on frontiers in handwriting recognition. ICFHR’2012, pp 659–666Google Scholar
  58. 58.
    Shao L, Zhou H (1996) Curve fitting with Bezier cubics. Gr Models Image Process 58:223–228CrossRefGoogle Scholar
  59. 59.
    Shi D, Elliott RJ, Chen T (2016) Event-based state estimation of discrete-state hidden Markov models. Automatica 65:12–26MathSciNetzbMATHCrossRefGoogle Scholar
  60. 60.
    Simonnet D, Anquetil E, Bouillon M (2017) Multi-criteria handwriting quality analysis with online fuzzy models. Pattern Recogn 69:310–324CrossRefGoogle Scholar
  61. 61.
    Sin B-K, Kim J (1998) Network-based approach to Korean handwriting analysis. Int J Pattern Recogn Artif Intell 12(2):233–249CrossRefGoogle Scholar
  62. 62.
    Song C, Qu Z, Blumm N, Barabasi A-L (2010) Limits of predictability in human mobility. Science 327:1018–1021MathSciNetzbMATHCrossRefGoogle Scholar
  63. 63.
    Srihari SN, Cha S-H, Arora H, Lee S (2002) Individuality of handwriting. J Forensic Sci 44(4):856–872Google Scholar
  64. 64.
    Tamposis IA, Theodoropoulou MC, Tsirigos KD, Bagos PG (2018) Extending hidden Markov models to allow conditioning on previous observations. J Bioinf Comput Biol 1:2. CrossRefGoogle Scholar
  65. 65.
    Taweechai N, Natasha D (2013) Approximating handwritten curves using progressive-iterative approximation. In: 10th IEEE international conference computer graphic, imaging and visualisation, USA, pp 17–22Google Scholar
  66. 66.
    Taweechai N, Natasha D (2012) Approximating online handwriting image by Bezier curves. In: 10th IEEE international conference on computer graphic, imaging and visualisation (CGIV), USA, pp 33–37Google Scholar
  67. 67.
    Thammano A, Rugkunchon S (2006) A neural network model for online handwritten mathematical symbol recognition. In: International conference on intelligent computing, pp 292–298Google Scholar
  68. 68.
    Uno M, Suzuki R, Kawato M (1989) Minimum muscle-tension change model which reproduces human arm movement. In: 4th symposium on biological and physiological engineering, pp 299–305Google Scholar
  69. 69.
    Viard-Gaudin C, Lallican PM, Knerr S (1999) The ireste on/off (IRONOFF) dual handwriting database. In: International conference on document analysis and recognition.
  70. 70.
    Wada Y, Kawato M (2004) A via-point time optimization algorithm for complex sequential trajectory formation. Neural Netw 17:353–364zbMATHCrossRefGoogle Scholar
  71. 71.
    Wada Y, Ohkawa K, Sumita K (2001). Generation of diversity form characters using a computational handwriting model and a genetic algorithm. In: ICANN’01: LNCS, Springer, Hedelberg. vol 2130, pp 1217–1224Google Scholar
  72. 72.
    Wang J, Wu C, Xu Q-Y, Shum Y-H (2004) Combining shape and physical models for online cursive handwriting synthesis. Int J Doc Anal Recogn 7(4):1433–2833Google Scholar
  73. 73.
    Wang J, Wu C, Xu Y, Shum H, Ji L (2002) Learning-based cursive handwriting synthesis. In: Proceedings of IWFHR’2002, pp 157–162Google Scholar
  74. 74.
    Wheeler TJ, Clements J, Finn RD (2014) Skylign: a tool for creating informative, interactive logos representing sequence alignments and profile hidden Markov models. BMC Bioinf 15:1. CrossRefGoogle Scholar
  75. 75.
    Yanhong L, David LO, Zheng Q (2007) Similarity measures between intuitionistic fuzzy (vague) sets: a comparative analysis. Pattern Recogn Lett 28:278–285CrossRefGoogle Scholar
  76. 76.
    Zeng K, Ding S, Jia W (2019) Single image super-resolution using a polymorphic parallel CNN. Appl Intell 49(1):292–300. CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.REGIM-Lab: Research Group on Intelligent Machines Laboratory, National School of EngineersUniversity of SfaxSfaxTunisia

Personalised recommendations