Industry Panel on Defining Industry Standards for Benchmarking Artificial Intelligence

  • Raghunath NambiarEmail author
  • Shahram Ghandeharizadeh
  • Gary Little
  • Christoph Boden
  • Ajay Dholakia
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11135)


Introduced in 2009, the Technology Conference on Performance Evaluation and Benchmarking (TPCTC) is a forum bringing together industry experts and researchers to develop innovative techniques for evaluation, measurement and characterization. This panel at the tenth TPC Technology Conference on Performance Evaluation and Benchmarking (TPCTC 2018) brought together industry experts and researchers from a broad spectrum of interests in the field of Artificial Intelligence (AI).



The panelists acknowledge the contributions of the TPC members to the industry and academic community and look forward to working with the TPC AI committee in creating a new standard for benchmarking AI systems. The panelists also thank Shane Handy for his comments and feedback with this document.


  1. 1.
    Stonebraker, M.: A new direction for TPC? In: Nambiar, R., Poess, M. (eds.) TPCTC 2009. LNCS, vol. 5895, pp. 11–17. Springer, Heidelberg (2009). Scholar
  2. 2.
    Nambiar, R., Poess, M.: Keeping the TPC relevant! PVLDB 6(11), 1186–1187 (2013)Google Scholar
  3. 3.
    Nambiar, R., Wakou, N., Carman, F., Majdalany, M.: Transaction Processing Performance Council (TPC): state of the council 2010. In: Nambiar, R., Poess, M. (eds.) TPCTC 2010. LNCS, vol. 6417, pp. 1–9. Springer, Heidelberg (2011). Scholar
  4. 4.
    Nambiar, R.: Towards an industry standard for benchmarking artificial intelligence systems. In: ICDE 2018, pp. 1679–1680 (2018)Google Scholar
  5. 5.
    TPC Press Release Transaction Processing Performance Council (TPC) Establishes Artificial Intelligence Working Group (TPC-AI).
  6. 6.
    Cisco Blog: Towards an Industry Standard for Benchmarking AI.
  7. 7.
    Huppler, K.: The art of building a good benchmark. In: Nambiar, R., Poess, M. (eds.) TPCTC 2009. LNCS, vol. 5895, pp. 18–30. Springer, Heidelberg (2009). Scholar
  8. 8.
    McKinsey Global Institute: Artificial Intelligence the Next Digital Fortier? Decision Paper, June 2017Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Raghunath Nambiar
    • 1
    Email author
  • Shahram Ghandeharizadeh
    • 2
  • Gary Little
    • 3
  • Christoph Boden
    • 4
  • Ajay Dholakia
    • 5
  1. 1.AMD, Inc.Santa ClaraUSA
  2. 2.University of Southern CaliforniaLos AngelesUSA
  3. 3.Nutanix, Inc.San JoseUSA
  4. 4.Technischen Universität BerlinBerlinGermany
  5. 5.Lenovo Group LimitedMorrisvilleUSA

Personalised recommendations