Annals of Telecommunications

, Volume 74, Issue 7–8, pp 501–515 | Cite as

On the rewards of self-adaptive IoT honeypots

  • Adrian Pauna
  • Ion Bica
  • Florin PopEmail author
  • Aniello Castiglione


In an era of fully digitally interconnected people and machines, IoT devices become a real target for attackers. Recent incidents such as the well-known Mirai botnet, have shown that the risks incurred are huge and therefore a risk assessment is mandatory. In this paper we present a novel approach on collecting relevant data about IoT attacks. We detail a SSH/Telnet honeypot system that leverages reinforcement learning algorithms in order to interact with the attackers, and we present the results obtained in view of defining optimal reward functions to be used. One of the key issues regarding the performance of such algorithms is the direct dependence on the reward functions used. The main outcome of our study is a full implementation of an IoT honeypot system that leverages Apprenticeship Learning using Inverse Reinforcement Learning, in order to generate best suited reward functions.


Internet of things Honeypot systems Inverse reinforcement learning Neural network Self-adaptive honeypot systems Reinforcement learning 



We express our thanks to all reviewers for their valuable comments and remarks.

Funding information

The research presented in this paper is supported by the following projects: ATLAS (PN-III-P1-1.2-PCCDI-2017-0272) and ROBIN (PN-III-P1-1.2-PCCDI-2017-0734).


  1. 1.
    Online dictionaries. (2017) Definition: “Internet of things (IoT)”. Accessed on November 2018
  2. 2.
    Alper Erdal, “Cisco internet of things,” October 2015. Available: Accessed on November 2018
  3. 3.
    Barcena, Mario Ballano, and Candid Wueest. Insecurity in the internet of things. Security response, symantec (2015)Google Scholar
  4. 4.
    Markowsky, Linda, and George Markowsky (2015) Scanning for vulnerable devices in the internet of things. In Intelligent data acquisition and advanced computing systems: technology and applications (IDAACS), 2015 IEEE 8th international conference on, vol. 1, pp. 463–467. IEEEGoogle Scholar
  5. 5.
    NSA (2016) The next wave - the internet of things: it’s a wonderfully integrated life, vol. 21, no. 2Google Scholar
  6. 6.
    Pa Y, Pa M, Suzuki S, Yoshioka K, Matsumoto T, Kasama T, Rossow C (2016) Iotpot: a novel honeypot for revealing current iot threats. J Inf Proc 24(3):522–533Google Scholar
  7. 7.
    Alaba FA, Othman M, Hashem IAT, Alotaibi F (2017) Internet of things security: a survey. J Netw Comput Appl 88:10–28, ISSN 1084–8045. CrossRefGoogle Scholar
  8. 8.
    Antonakakis M, April T, Bailey M, Bernhard M, Bursztein E, Cochran J, Durumeric Z et al. (2017) Understanding the mirai botnet. In USENIX security symposium, pp. 1092–1110Google Scholar
  9. 9.
    Aceto G, Botta A, Marchetta P, Persico V, Pescapé A (2018) A comprehensive survey on internet outages. J Netw Comput Appl 113:36–63, ISSN 1084-8045. CrossRefGoogle Scholar
  10. 10.
    Cui A, and Stolfo SJ (2010) A quantitative analysis of the insecurity of embedded network devices: results of a wide-area scan. In Proceedings of the 26th annual computer security applications conference, pp. 97–106. ACM.
  11. 11.
    He H, Maple C, Watson T, Tiwari A, Mehnen J, Jin Y, and Gabrys B (2016) The security challenges in the IoT enabled cyber-physical systems and opportunities for evolutionary computing & other computational intelligence. In Evolutionary computation (CEC), 2016 IEEE congress on, pp. 1015–1021. IEEEGoogle Scholar
  12. 12.
    Kumar P, Kunwar RS, and Sachan A (2016) A survey report on: security & challenges in internet of things. In Proc National Conference on ICT & IoT, pp. 35–39Google Scholar
  13. 13.
    Senpai A (2016) Mirai-source-code. Available: Accessed on November 2018
  14. 14.
    Vlasenko D (2008) Busybox: The swiss army knife of embedded linux. Available:, Accessed on November 2018
  15. 15.
    Catuogno L, Castiglione A, Palmieri F (2015) A honeypot system with honeyword-driven fake interactive sessions. In: 2015 international conference on high performance computing and simulation (HPCS), pp. 187–194. IEEE, JulyGoogle Scholar
  16. 16.
    Pauna A, and Bica I (2014) RASSH-reinforced adaptive SSH honeypot. In Communications (COMM), 2014 10th international conference on, pp. 1–6. IEEE.
  17. 17.
    Pauna A, Iacob A-C, and Bica I (2018) QRASSH-A self-adaptive SSH honeypot driven by Q-learning. In 2018 international conference on communications (COMM), pp. 441–446. IEEE.
  18. 18.
    Wagener G, State R, Engel T, and Dulaunoy A (2011) Adaptive and self-configurable honeypots. In 12th IFIP/IEEE international symposium on integrated network management (IM 2011) and workshops, pp. 345–352. IEEE.
  19. 19.
    Abbeel P, and Ng AY (2004) Apprenticeship learning via inverse reinforcement learning. In Proceedings of the twenty-first international conference on machine learning, p. 1. ACMGoogle Scholar
  20. 20.
  21. 21.
    Cheswick B (1992) An evening with Berferd in which a cracker is lured, endured, and studied. In Proc. winter USENIX conference, San Francisco, pp. 20–24Google Scholar
  22. 22.
    Wagener G Thesis: self-adaptive honeypots coercing and assessing attacker behaviour Accessed on November 2018
  23. 23.
    Pauna A (2012) Improved self adaptive honeypots capable of detecting rootkit malware. In Communications (COMM), 2012 9th international conference on, pp. 281–284. IEEE,
  24. 24.
    Kippo:, Accessed on November 2018
  25. 25.
    Sutton RS, Barto AG (2018) Reinforcement learning: An introduction. MIT press. Accessed 19 Dec 2018
  26. 26.
    Luo T, Xu Z, Jin X, Jia Y, and Ouyang X (2017) Iotcandyjar: towards an intelligent-interaction honeypot for iot devices. Black HatGoogle Scholar
  27. 27.
    La QD, Quek TQS, and Lee J (2016) A game theoretic model for enabling honeypots in IoT networks. In Communications (ICC), 2016 IEEE international conference on, pp. 1–6. IEEE.
  28. 28.
    Yin Minn Pa Pa, Suzuki S, Yoshioka K, Matsumoto T, Kasama T, and Rossow C (2015) IoTPOT: analysing the rise of IoT compromises. In Proceedings of the 9th USENIX conference on offensive technologies (WOOT’15), Aurélien Francillon and Thomas Ptacek (Eds.). USENIX Association, Berkeley, CA, USA, 9–9Google Scholar
  29. 29.
    Bhangwar NH, Halepoto IA, Khokhar S, Laghari AA (2017) On routing protocols for high performance. Stud Inf Control 26(4):441–448Google Scholar
  30. 30.
    Guarnizo JD, Tambe A, Bhunia SS, Ochoa M, Tippenhauer NO, Shabtai A, and Elovici Y (2017) SIPHON: towards scalable high-interaction physical honeypots. In Proceedings of the 3rd ACM workshop on cyber-physical system security (CPSS ‘17). ACM, New York, NY, USA, 57–68.
  31. 31.
    Phype. Telnet IoT honeypot. Accessed on November 2018
  32. 32.
    Honeything. Accessed on November 2018
  33. 33.
    Dowling S, Schukat M and Melvin H (2017) A ZigBee honeypot to assess IoT cyberattack behaviour, 28th Irish Signals and Systems Conference (ISSC), Killarney, 2017, pp. 1–6.
  34. 34.
    Krishnaprasad P (2017) “Capturing attacks on IoT devices with a multi-purpose IoT honeypot”. PhD thesis, Indian Institute of Technology KanpurGoogle Scholar
  35. 35.
    Krawetz N (2004) Anti-honeypot technology, in IEEE Security & Privacy, vol. 2, no. 1, pp. 76–79.
  36. 36.
    T. Holz and F. Raynal (2005) Detecting honeypots and other suspicious environments, Proceedings from the Sixth Annual IEEE SMC Information Assurance Workshop, West Point, NY, USA, pp. 29–36.
  37. 37.
    Fioriti V, Chinnici M (2017) Node seniority ranking in networks. Stud Inf Control 26(4):397–402Google Scholar
  38. 38.
    Năstase L, Sandu IE, Popescu N (2017) An experimental evaluation of application layer protocols for the internet of things. Stud Inf Control 26(4):403–412Google Scholar
  39. 39.
    Bellman R (1957) A Markovian decision process. J Math Mech 6:679–684. Accessed 19 Dec 2018
  40. 40.
    Ng AY, Russell SJ (2000) Algorithms for inverse reinforcement learning. In: Langley P (ed) Proceedings of the seventeenth international conference on machine learning (ICML ‘00). Morgan Kaufmann Publishers Inc., San Francisco, CA, pp 663–670Google Scholar
  41. 41.
    Li H, Wei T, Ren AO, Qi Z, and Wang Y (2017) Deep reinforcement learning: framework, applications, and embedded implementations. In Computer-aided design (ICCAD), 2017 IEEE/ACM international conference on, pp. 847–854. IEEE.
  42. 42.
    Cowrie:, Accessed on November 2018
  43. 43.
    IRASSH-T:, Accessed on November 2018
  44. 44.
    Miguel Sousa Lobo, Lieven Vandenberghe, Stephen Boyd, Hervé Lebret, Applications of second-order cone programming, Linear Algebra Appl, volume 284, issues 1–3,1998, Pages 193–228, ISSN 0024-3795,
  45. 45.
    Quadratic Programming in Python Accessed on November 2018
  46. 46.
    Dulaunoy A, Wagener G, Mokaddem S, and Wagner C (2017) An extended analysis of an IoT malware from a blackhole network. TNC17Google Scholar
  47. 47.
    Alaa M, Zaidan AA, Zaidan BB, Talal M, Kiah MLM (2017) A review of smart home applications based on internet of things. J Netw Comput Appl 97:48–65. CrossRefGoogle Scholar
  48. 48.
    Popa D, Pop F, Serbanescu C, and Castiglione A (2019) Deep learning model for home automation and energy reduction in a smart home environment platform. Neural Comput Applic (in press)Google Scholar
  49. 49.
    Mohd BJ, Hayajneh T, Vasilakos AV (2015) A survey on lightweight block ciphers for low-resource devices: comparative study and open issues. J Netw Comput Appl 58:73–93, ISSN 1084-8045. CrossRefGoogle Scholar
  50. 50.
    Ishitaki T, Obukata R, Oda T, and Barolli L (2017) Application of deep recurrent neural networks for prediction of user behavior in Tor networks. In Advanced information networking and applications workshops (WAINA), 2017 31st international conference on, pp. 238–243. IEEE,
  51. 51.
    Chifor B-C, Bica I, Patriciu V-V, Pop F (2018) A security authorization scheme for smart home internet of things devices. Futur Gener Comput Syst 86:740–749CrossRefGoogle Scholar
  52. 52.
    Esposito C, Castiglione A, Palmieri F, Ficco M, Dobre C, Iordache GV, Pop F (2018) Event-based sensor data exchange and fusion in the internet of things environments. J Parallel Distrib Comput 118:328–343CrossRefGoogle Scholar
  53. 53.
    Zhimin Yu, Chong-zhi Gao, Zhengjun Jing, Brij Bhooshan Gupta, Qiuru Cai. A practical public key encryption scheme based on learning parity with noise. IEEE Access, 2018., 6, 31918, 31923
  54. 54.
    Yang L, Han Z, Huang Z, Ma J (2018) A remotely keyed file encryption scheme under mobile cloud computing. J Netw Comput Appl 106:90–99CrossRefGoogle Scholar
  55. 55.
    Tan C, Li Y, and Cheng Y (2017) An inverse reinforcement learning algorithm for semi-Markov decision processes. In Computational intelligence (SSCI), 2017 IEEE symposium series on, pp. 1–6. IEEE.

Copyright information

© Institut Mines-Télécom and Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Faculty of Military Electronic and Information SystemsMilitary Technical AcademyBucharestRomania
  2. 2.Faculty of Automatic Control and Computers“Politehnica” University of BucharestBucharestRomania
  3. 3.National Institute for Research and Development in Informatics (ICI)BucharestRomania
  4. 4.Department of Computer ScienceUniversity of SalernoFiscianoItaly

Personalised recommendations