Information Systems Frontiers

, Volume 18, Issue 5, pp 981–994 | Cite as

On the empirical justification of theoretical heuristic transference and learning

  • Stuart H. Rubin
  • Thouraya Bouabana-Tebibel
  • Yasmine Hoadjli


The solution of intractable problems implies the use of heuristics. Quantum computers may find use for optimization problems, but have yet to solve any NP-hard problems. This paper demonstrates results in game theory for domain transference and the reuse of problem-solving knowledge through the application of learned heuristics. It goes on to explore the possibilities for the acquisition of heuristics for the solution of the NP-hard TSP problem. Here, it is found that simple heuristics (e.g., pairwise exchange) often work best in the context of more or less sophisticated experimental designs. Often, these problems are not amenable to exclusive logic solutions; but rather, require the application of hybrid approaches predicated on search. In general, such approaches are based on randomization and supported by parallel processing. This means that heuristic solutions emerge from attempts to randomize the search space. The paper goes on to present a constructive proof of the unbounded density of knowledge in support of the Semantic Randomization Theorem (SRT). It highlights this result and its potential impact upon the community of machine learning researchers.


Domain transference Heuristics Machine learning N-puzzle Randomization Reuse SRT 



Stuart would like to extend special thanks to Michael Leyton, Rutgers Department of Mathematics, for reviewing his tremendously exciting Theory of Randomization (Rubin 2007) and to the kind genius, Sukarno Mertoguno for applicative discussions on cybersecurity. Stuart would also like to extend his thanks to the Office of Naval Research (ONR) for providing financial backing for this research with the support of his Branch Head, Ken Simonsen. This work was produced by a U.S. government employee as part of his official duties and is not subject to copyright. It is approved for public release with an unlimited distribution.


  1. Behbood, V., Lu, J., & Zhang, G. (2011). Long term bank failure prediction using fuzzy refinement-based transductive transfer learning. In IEEE International Conference on Fuzzy Systems (pp. 2676–2683). Taiwan: IEEE.Google Scholar
  2. Behbood, V., Lu, J., & Zhang, G. (2013a). Fuzzy bridged refinement domain adaptation: Long-term bank failure prediction. International Journal of Computational Intelligence and Applications, 12(01).Google Scholar
  3. Behbood, V., Lu, J., & Zhang, G. (2013b). Text categorization by fuzzy domain adaptation. In IEEE International Conference on Fuzzy Systems. Hyderabad: IEEE.Google Scholar
  4. Behbood, V., Lu, J., & Zhang, G. (2014). Fuzzy refinement domain adaptation for long term prediction in banking ecosystem. IEEE Transactions on Industrial Informatics, 10(2), 1637–1646.CrossRefGoogle Scholar
  5. Caruana, R. (1993). Multitask learning: a knowledge-based source of inductive bias. In Proceedings of the tenth international conference on machine learning (pp. 41–48). MA, USA.Google Scholar
  6. Caruana, R. (1997). Multitask learning. Machine learning, 28(1), 41–75.CrossRefGoogle Scholar
  7. Celiberto, L.A., Matsuura, J.P., Lopez de Mantaras, R., & Bianchi, R.A.C. (2011). Using cases as heuristics in reinforcement learning: a transfer learning application. In International Joint Conference on Artificial Intelligence. Barcelona.Google Scholar
  8. Chaitin, G.J. (1975). Randomness and mathematical proof. Scientific American, 232(5), 47–52.CrossRefGoogle Scholar
  9. Chopra, S., Balakrishnan, S., & Gopalan, R. (2013). DLID: Deep Learning for domain adaptation by interpolating between domains. In ICML Workshop on challenges in representation learning, Vol. 2. Atlanta.Google Scholar
  10. Cireşan, D.C., Meier, U., & Schmidhuber, J. (2012). Transfer learning for latin and chinese characters with deep neural networks. In International Joint Conference on Neural Networks (IJCNN). Australia: IEEE.Google Scholar
  11. Cousins, N., & Eccles, J.C. (1985). Nobel Prize Conversations with Sir John Eccles, Roger Sperry, Ilya Prigogine, Brian Josephson. CA: Saybrook Publishers.Google Scholar
  12. Eccles, J.C. (1976). The understanding of the brain. New York: McGraw-Hill Co. 2d ed.Google Scholar
  13. Feigenbaum, E.A., & McCorduck, P. (1983). The fifth generation: Artificial intelligence and Japan’s computer challenge to the world reading. MA: Addison-Wesley Pub. Co.Google Scholar
  14. Fogel, D.B. (2001). Blondie24: Playing at the edge of AI. Mountain View: Morgan Kaufmann Publishers, Inc.Google Scholar
  15. Huang, J.-T., Li, J., Yu, D., Deng, L., & Gong, Y. (2013). Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers. Vancouver: IEEE.CrossRefGoogle Scholar
  16. Kandaswamy, C., Silva, L.M., Alexandre, L.A., Santos, J.M., & de Sá, J.M. (2014). Improving deep neural network performance by reusing features trained with transductive transference. In 24Th international conference on artificial neural networks and machine learning–ICANN (pp. 265–272). Hampurg: Springer.Google Scholar
  17. Kfoury, A.J., Moll, R.N., & Arbib, M.A. (1982). A programming approach to computability. New York: Springer Verlag Inc.CrossRefGoogle Scholar
  18. Koçer, B., & Arslan, A. (2010). Genetic transfer learning. Expert Systems with Applications, 37(10), 6997–7002.CrossRefGoogle Scholar
  19. Lin, J.-H., & Vitter, J.S. (1991). Complexity results on learning by neural nets. Machine Learning, 6(3), 211–230.Google Scholar
  20. Lu, J., Behbood, V., Hao, P., Zuo, H., Xue, S., & Zhang, G. (2015). Transfer learning using computational intelligence: a survey. Knowledge-Based Systems, 80, 14–23.CrossRefGoogle Scholar
  21. Luis, R., Sucar, L.E., & Morales, E.F. (2010). Inductive transfer for learning bayesian networks. Machine learning, 79(1-2), 227– 255.CrossRefGoogle Scholar
  22. Ma, Y., Luo, G., Zeng, X., & Chen, A. (2012). Transfer learning for cross-company software defect prediction. Information and Software Technology, 54(3), 248–256.CrossRefGoogle Scholar
  23. Michalski, R.S., Carbonell, J.G., & Mitchell, T.M. (1983). Machine learning: an artificial intelligence approach (volume 1). Palo Alto: Tioga Publishing Co.CrossRefGoogle Scholar
  24. Mitchell, T.M., Carbonell, J.G., & Michalski, R.S. (Eds.) (1986). Machine Learning: A guide to current research. New York: Springer-Verlag Inc.Google Scholar
  25. Niculescu-Mizil, A., & Caruana, R. (2007). Inductive Transfer for bayesian network structure learning. In 11Th international Conference on Artificial Intelligence and Statistics, Puerto Rico (pp. 339–346).Google Scholar
  26. Nilsson, N.J. (1980). Principles of artificial intelligence. Mountain View: Morgan Kaufmann Publishers Inc.Google Scholar
  27. Oyen, D., & Lane, T. (2013). Bayesian discovery of multiple bayesian networks via transfer learning. In 13Th international conference on data mining (ICDM) (pp. 577–586). Dallas: IEEE.Google Scholar
  28. Pan, S.J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions Knowledge and Data Engineering, 22(10), 1345–1359.CrossRefGoogle Scholar
  29. Rubin, S.H. (2004). On the auto-randomization of knowledge. In: Information Reuse and Integration. In Proceedings of the 2004 IEEE International Conference On Information Reuse and Integration (pp. 308–313): IEEE.Google Scholar
  30. Rubin, S.H. (2007). On randomization and discovery. Information Sciences, 177(1), 170–191.CrossRefGoogle Scholar
  31. Rubin, S.H. (2012). On creativity and intelligence in computational systems. In Advances in Reasoning-Based Image Processing Intelligent Systems (pp. 383–421). Berlin: Springer.CrossRefGoogle Scholar
  32. Rubin, S.H., & Bouabana-Tebibel, T. (2015a). Naval intelligent authentication and support through randomization and transformative search. In To appear New approaches in intelligent control and image analysis - techniques, methodologies and applications. Intelligent Systems Reference Library: Springer.Google Scholar
  33. Rubin, S.H., & Bouabana-Tebibel, T. (2015b). NNCS: Randomization and informed search for novel naval cyber strategies. In Recent Advances in Computational Intelligence in Defense and Security. Studies in Computational Intelligence, (Vol. 621 pp. 193–223): Springer.Google Scholar
  34. Rubin, S.H., Bouabana-Tebibel, T., Hoadjli, Y., Habib, K., & Belamiri, B.Y. (2015). On heuristic randomization and reuse as an enabler of domain transference. In The 16th IEEE international conference on information reuse and integration IEEE IRI 2015. San Francisco, USA, August 13–15 (pp. 411–418).CrossRefGoogle Scholar
  35. Rubin, S.H., Chen, S.-C., Law, J.B., & Lee, G.K. (2005). On the inherent necessity of heuristic proofs. In 2005 IEEE International Conference on Systems, Man and Cybernetics, (Vol. 4 pp. 3890–3896): IEEE.Google Scholar
  36. Rubin, S.H., Murthy, S.N.J., Smith, M.H., & Trajković, L. (2004). Kaser: knowledge amplification by structured expert randomization. IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics, 34(6), 2317–2329.CrossRefGoogle Scholar
  37. Shell, J. (2013). Fuzzy transfer learning. PhD thesis: De Montfort University.Google Scholar
  38. Shell, J., & Coupland, S. (2012). Towards fuzzy transfer learning for intelligent environments, (Vol. 7683 pp. 145–160): Springer.Google Scholar
  39. Shell, J., & Coupland, S. (2015). Fuzzy transfer learning: methodology and application. Information Sciences, 293, 59–79.CrossRefGoogle Scholar
  40. Silver, D.L., & Poirier, R. (2007). Context-sensitive MTL networks for machine lifelong learning. In 20th florida artificial intelligence research society conference (pp. 628–633). Key West.Google Scholar
  41. Swietojanski, P., Ghoshal, A., & Renals, S. (2012). Unsupervised cross-lingual knowledge transfer in DNN-based LVCSR. In IEEE Workshop on spoken language technology (SLT) (pp. 246–251). Miami: IEEE.CrossRefGoogle Scholar
  42. Tesar, B. (2000). Learnability in optimality theory: Mit Press.Google Scholar
  43. Uspenskii, V.A. (1987). Gödel’s incompleteness theorem. Translated from Russian. Ves Mir Publishers: Moscow.Google Scholar

Copyright information

© Springer Science+Business Media New York (outside the USA) 2016

Authors and Affiliations

  • Stuart H. Rubin
    • 1
  • Thouraya Bouabana-Tebibel
    • 2
  • Yasmine Hoadjli
    • 2
  1. 1.Space and Naval Warfare Systems Center PacificSan DiegoUSA
  2. 2.Ecole nationale Supérieure d’InformatiqueLCSI LaboratoryAlgerAlgeria

Personalised recommendations