Deep Online Hierarchical Unsupervised Learning for Pattern Mining from Utility Usage Data

  • Saad MohamadEmail author
  • Damla Arifoglu
  • Chemseddine Mansouri
  • Abdelhamid Bouchachia
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 840)


Machine learning approaches for non-intrusive load monitoring (NILM) have focused on supervised algorithms. Unsupervised approaches can be more interesting and of more practical use in real case scenarios. More specifically, they do not require labelled training data to be collected from individual appliances and the algorithm can be deployed to operate on the measured aggregate data directly. In this paper, we propose a fully unsupervised NILM framework based on Deep Belief network (DBN) and online Latent Dirichlet Allocation (LDA). Firstly, the raw signals of the house utilities are fed into DBN to extract low-level generic features in an unsupervised fashion, and then the hierarchical Bayesian model, LDA, learns high-level features that capture the correlations between the low-level ones. Thus, the proposed method (DBN-LDA) harnesses the DBN’s ability of learning distributed hierarchies of features to extract sophisticated appliances-specific features without the need of precise human-crafted input representations. The clustering power of the hierarchical Bayesian models helps further summarise the input data by extracting higher-level information representing the residents’ energy consumption patterns. Using Deep-Hierarchical models reduces the computational complexity since LDA is not directly applied to the raw data. The computational efficiency is crucial as our application involves massive data from different types of utility usages. Moreover, we develop a novel online inference algorithm to cope with this big data. Another novelty of this work is that the data is a combination of different utilities (e.g., electricity, water and gas) and some sensors measurements. Finally, we propose different methods to evaluate the results and preliminary experiments show that the DBN-LDA is promising to extract useful patterns.


Unsupervised non-intrusive load monitoring Pattern recognition Online Latent Dirichlet Allocation Deep belief network 



This work was supported by the Energy Technology Institute (UK) as part of the project: High Frequency Appliance Disaggregation Analysis (HFADA). A. Bouchachia was supported by the European Commission under the Horizon 2020 Grant 687691 related to the project: PROTEUS: Scalable Online Machine Learning for Predictive Analytics and Real-Time Interactive Visualization.


  1. 1.
    Bonfigli, R., Squartini, S., Fagiani, M., Piazza, F.: Unsupervised algorithms for non-intrusive load monitoring: an up-to-date overview. In: 2015 IEEE 15th International Conference on Environment and Electrical Engineering (EEEIC). IEEE (2015)Google Scholar
  2. 2.
    Salakhutdinov, R., Tenenbaum, J.B., Torralba, A.: Learning with hierarchical-deep models. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1958–1971 (2013)CrossRefGoogle Scholar
  3. 3.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)CrossRefGoogle Scholar
  4. 4.
    Deng, L.: A tutorial survey of architectures, algorithms, and applications for deep learning. APSIPA Trans. Sig. Inf. Process. (2014)Google Scholar
  5. 5.
    Bengio, Y., et al.: Learning deep architectures for AI. Found. Trends\(\textregistered \) Mach. Learn. 2(1), 1–127 (2009)Google Scholar
  6. 6.
    Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3(Jan), 993–1022 (2003)zbMATHGoogle Scholar
  8. 8.
    Fischer, C.: Feedback on household electricity consumption: a tool for saving energy? Energ. Effi. 1(1), 79–104 (2008)CrossRefGoogle Scholar
  9. 9.
    Kelly, J., Knottenbelt, W.: The UK-DALE dataset, domestic appliance-level electricity demand and whole-house demand from five UK homes. Sci. Data 2, 150007 (2015)CrossRefGoogle Scholar
  10. 10.
    Filip, A.: BLUED: a fully labeled public dataset for event-based non-intrusive load monitoring research. In: 2nd Workshop on Data Mining Applications in Sustainability (SustKDD) (2011)Google Scholar
  11. 11.
    Kolter, J.Z., Johnson, M.J.: REDD: a public data set for energy disaggregation research. In: Workshop on Data Mining Applications in Sustainability (SIGKDD). San Diego, CA (2011)Google Scholar
  12. 12.
    Hoffman, M.D., Blei, D.M., Wang, C., Paisley, J.: Stochastic variational inference. J. Mach. Learn. Res. 4(1), 1303–1347 (2013)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Makonin, S., Popowich, F., Bartram, L., Gill, B., Bajic, I.V.: AMPds: a public dataset for load disaggregation and eco-feedback research. In: 2013 IEEE Electrical Power & Energy Conference (EPEC). IEEE (2013)Google Scholar
  14. 14.
    Makonin, S., Ellert, B., Bajić, I.V., Popowich, F.: Electricity, water, and natural gas consumption of a residential house in Canada from 2012 to 2014. Sci. Data 3, 160037 (2016)CrossRefGoogle Scholar
  15. 15.
    Hart, G.W.: Nonintrusive appliance load monitoring. In: Proceedings of the IEEE (1992)Google Scholar
  16. 16.
    Liang, J., Ng, S.K., Kendall, G., Cheng, J.W.: Load signature study part I: basic concept, structure, and methodology. IEEE Trans. Power Delivery 25(2), 551–560 (2010)CrossRefGoogle Scholar
  17. 17.
    Kolter, J.Z., Batra, S., Ng, A.Y.: Energy disaggregation via discriminative sparse coding. In: Advances in Neural Information Processing Systems (2010)Google Scholar
  18. 18.
    Srinivasan, D., Ng, W., Liew, A.: Neural-network-based signature recognition for harmonic source identification. IEEE Trans. Power Delivery 21(1), 398–405 (2006)CrossRefGoogle Scholar
  19. 19.
    Berges, M.,  Goldman, E., Matthews, H.S., Soibelman, L.: Learning systems for electric consumption of buildings. In: Computing in Civil Engineering (2009)Google Scholar
  20. 20.
    Ruzzelli, A.G., Nicolas, C., Schoofs, A., O’Hare, G.M.: Real-time recognition and profiling of appliances through a single electricity sensor. In: 2010 7th Annual IEEE Communications Society Conference on Sensor Mesh and Ad Hoc Communications and Networks (SECON). IEEE (2010)Google Scholar
  21. 21.
    Kelly, J., Knottenbelt, W.: Neural nilm: deep neural networks applied to energy disaggregation. In: Proceedings of the 2nd ACM International Conference on Embedded Systems for Energy-Efficient Built Environments. ACM (2015)Google Scholar
  22. 22.
    Lai, Y.-X., Lai, C.-F., Huang, Y.-M., Chao, H.-C.: Multi-appliance recognition system with hybrid SVM/GMM classifier in ubiquitous smart home. Inf. Sci. 230, 39–55 (2013)CrossRefGoogle Scholar
  23. 23.
    Kim, H., Marwah, M., Arlitt, M., Lyon, G., Han, J.: Unsupervised disaggregation of low frequency power measurements. In: Proceedings of the 2011 SIAM International Conference on Data Mining. SIAM (2011)Google Scholar
  24. 24.
    Kolter, J.Z., Jaakkola, T.: Approximate inference in additive factorial HMMs with application to energy disaggregation. In: Artificial Intelligence and Statistics (2012)Google Scholar
  25. 25.
    Johnson, M.J., Willsky, A.S.: Bayesian nonparametric hidden semi-Markov models. J. Mach. Learn. Res. 14(Feb), 673–701 (2013)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Wytock, M., Kolter, J.Z.: Contextually supervised source separation with application to energy disaggregation. In: AAAI (2014)Google Scholar
  27. 27.
    Saad, M., Abdelhamid, B.: Online Gaussian LDA for unsupervised pattern mining from utility usage data. In: ECML-PKDD (2018, submitted)Google Scholar
  28. 28.
    Hoffman, M., Bach, F.R., Blei, D.M.: Online learning for latent Dirichlet allocation. In: Advances in Neural Information Processing Systems (2010)Google Scholar
  29. 29.
    Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22, 400–407 (1951)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Mohamad, S., Bouchachia, A., Sayed-Mouchaweh, M.: Asynchronous stochastic variational inference. arXiv preprint. arXiv:1801.04289 (2018)
  31. 31.
    Mohamad, S., Sayed-Mouchaweh, M., Bouchachia, A.: Active learning for classifying data streams with unknown number of classes. Neural Netw. 98, 1–5 (2018)CrossRefGoogle Scholar
  32. 32.
    Mohamad, S., Bouchachia, A., Sayed-Mouchaweh, M.: A bi-criteria active learning algorithm for dynamic data streams. IEEE Trans. Neural Netw. Learn. Syst. (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Saad Mohamad
    • 1
    Email author
  • Damla Arifoglu
    • 1
  • Chemseddine Mansouri
    • 1
  • Abdelhamid Bouchachia
    • 1
  1. 1.Department of ComputingBournemouth UniversityPooleUK

Personalised recommendations