Predicting Unemployment with Machine Learning Based on Registry Data

  • Markus ViljanenEmail author
  • Tapio Pahikkala
Conference paper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 385)


Many statistical models have been developed to understand the causes of unemployment, but predicting unemployment has received less attention. In this study, we develop a model to predict the labour market state of a person based on machine learning trained with a large administrative unemployment registry. The model specifies individuals as Markov chains with person specific transition rates. We evaluate the model on three tasks, where the goal is to predict who has the highest risk of escaping unemployment, becoming unemployed, and being unemployed at any given time. We obtain good performance (AUC: 0.80) for the machine learning model of lifetime unemployment, and very good performance (AUC: 0.90+) to the near future when we know the recent labour market state of a person. We find that person information affects the predictions in an intuitive way, but there still are significant differences that can be learned by utilizing labour market histories.


Unemployment Machine learning Prediction 


  1. 1.
    Ernst, E., Rani, U.: Understanding unemployment flows. Oxford Rev. Econ. Pol. 27(2), 268–294 (2011)CrossRefGoogle Scholar
  2. 2.
    Shimer, R.: Reassessing the ins and outs of unemployment. Rev. Econ. Dyn. 15(2), 127–148 (2012)CrossRefGoogle Scholar
  3. 3.
    Ahn, H.J., Hamilton, J.D.: Heterogeneity and unemployment dynamics. J. Bus. Econ. Stat. 1–26 (2019)Google Scholar
  4. 4.
    Honkanen, P.: Odotelaskelmat työllisyyden, työttömyyden ja eläkeajan arvioinnissa. KELA Working Papers, No. 137 (2018)Google Scholar
  5. 5.
    Pedersen, P.J., Westergård-Nielsen, N.C.: Unemployment. A review of the evidence from panel data. In: Economics of Unemployment. Edward Elgar Publishing (2000)Google Scholar
  6. 6.
    Wanberg, C.R.: The individual experience of unemployment. Ann. Rev. Psychol. 63, 369–396 (2012)CrossRefGoogle Scholar
  7. 7.
    Kettunen, J.: Education and unemployment duration. Econ. Educ. Rev. 16(2), 163–170 (1997)CrossRefGoogle Scholar
  8. 8.
    Ollikainen, V.: The determinants of unemployment duration by gender in Finland. VATT Discussion Papers, No. 316 (2003)Google Scholar
  9. 9.
    Kyyrä, T.: Partial unemployment insurance benefits and the transition rate to regular work. Eur. Econ. Rev. 54(7), 911–930 (2010)CrossRefGoogle Scholar
  10. 10.
    Rokkanen, M., Uusitalo, R.: Changes in job stability: evidence from lifetime job histories. IZA Discussion Papers, No. 4721 (2010)Google Scholar
  11. 11.
    Asplund, R.: Unemployment among finnish manufacturing workers. Who gets unemployed and from where? ETLA Discussion Papers, No. 711 (2000)Google Scholar
  12. 12.
    Eriksson, T., Pehkonen, J.: Unemployment flows in Finland, 1969–95: a time series analysis. Labour 12(3), 571–593 (1998)CrossRefGoogle Scholar
  13. 13.
    Peltola, M.: Työmarkkinasiirtymät Suomessa. Työllisyyden päättymisen jälkeinen työmarkkinasiirtymien dynamiikka vuosina 1995–1999. VATT Discussion Papers, No. 360 (2005)Google Scholar
  14. 14.
    Heckman, J.J., Borjas, G.J.: Does unemployment cause future unemployment? Definitions, questions and answers from a continuous time model of heterogeneity and state dependence. Economica 47(187), 247–283 (1980)CrossRefGoogle Scholar
  15. 15.
    Flinn, C.J., Heckman, J.J.: New methods for analyzing individual event histories. Sociol. Methodol. 13, 99–140 (1982)CrossRefGoogle Scholar
  16. 16.
    Mühleisen, M., Zimmermann, K.F.: A panel analysis of job changes and unemployment. Eur. Econ. Rev. 38(3–4), 793–801 (1994)CrossRefGoogle Scholar
  17. 17.
    D’Amuri, F., Marcucci, J.: The predictive power of Google searches in forecasting US unemployment. Int. J. Forecast. 33(4), 801–816 (2017)CrossRefGoogle Scholar
  18. 18.
    Tuhkuri, J.: ETLAnow: a model for forecasting with big data-forecasting unemployment with Google searches in Europe. No. 54. ETLA Report (2016)Google Scholar
  19. 19.
    Katris, C.: Prediction of unemployment rates with time series and machine learning techniques. Comput. Econ. 55, 673–706 (2019). Scholar
  20. 20.
    de Troya, Í.M.R., et al.: Predicting, explaining and understanding risk of long-term unemployment. In: 32nd Conference on Neural Information Processing Systems (2018)Google Scholar
  21. 21.
    Kütük, Y., Güloğlu, B.: Prediction of transition probabilities from unemployment to employment for Turkey via machine learning and econometrics: a comparative study. J. Res. Econ. 3(1), 58–75 (2019)Google Scholar
  22. 22.
    Beyersmann, J., Allignol, A., Schumacher, M.: Competing Risks and Multistate Models with R. Springer, Heidelberg (2011). Scholar
  23. 23.
    Tutz, G., Schmid, M.: Modeling Discrete Time-to-Event Data. Springer, Cham (2016). Scholar
  24. 24.
    Duchateau, L., Janssen, P.: The Frailty Model. Springer, Heidelberg (2007). Scholar
  25. 25.
    Cook, R.J., Lawless, J.: The Statistical Analysis of Recurrent Events. Springer, Heidelberg (2007). Scholar
  26. 26.
    Rausand, M., Høyland, A.: System Reliability Theory: Models, Statistical Methods, and Applications, vol. 396. Wiley, Hoboken (2003)zbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Turun YliopistoTurkuFinland

Personalised recommendations