Abstract
Competitions have become a popular tool in the data science community to solve hard problems, assess the state of the art and spur new research directions. Companies like Kaggle and open source platforms like Codalab connect people with data and a data science problem to those with the skills and means to solve it. Hence, the question arises: What, if anything, could NIPS add to this rich ecosystem?
In 2017, we embarked to find out. We attracted 23 potential competitions, of which we selected five to be NIPS 2017 competitions. Our final selection features competitions advancing the state of the art in other sciences such as “Classifying Clinically Actionable Genetic Mutations” and “Learning to Run”. Others, like “The Conversational Intelligence Challenge” and “Adversarial Attacks and Defences” generated new data sets that we expect to impact the progress in their respective communities for years to come. And “Human-Computer Question Answering Competition” showed us just how far we as a field have come in ability and efficiency since the break-through performance of Watson in Jeopardy. Two additional competitions, DeepArt and AI XPRIZE Milestions, were also associated to the NIPS 2017 competition track, whose results are also presented within this chapter.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
- 2.
- 3.
- 4.
- 5.
Possible scores were from one to five with former corresponding to the bad and the latter to the excellent dialogue quality.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
Links to code of non-targeted attacks: https://www.kaggle.com/c/6864/discussion/40420, targeted attacks: https://www.kaggle.com/c/6866/discussion/40421, defenses: https://www.kaggle.com/c/6867/discussion/40422
- 19.
Disclosure: Alexander Ecker, Leon Gatys, Łukasz Kidziński and Matthias Bethge are founders and shareholders of DeepArt UG (haftungsbeschränkt), the company operating https://deepart.io.
- 20.
- 21.
To detect cheating, we logged voters’ IP addresses and other meta information. We identified a handful of cheaters and removed their images from the competition usually within a day.
References
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01540, 2016.
Scott L Delp, Frank C Anderson, Allison S Arnold, Peter Loan, Ayman Habib, Chand T John, Eran Guendelman, and Darryl G Thelen. Opensim: open-source software to create and analyze dynamic simulations of movement. IEEE transactions on biomedical engineering, 54(11):1940–1950, 2007.
Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. Image style transfer using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2414–2423, 2016.
Łukasz Kidziński, Ong Carmichael Sharada, Mohanty, Jennifer Hicks, Sean Francis, Sergey Levine, Marcel Salathé, and Scott Delp. Learning to run challenge. In Sergio Escalera and Markus Weimer, editors, NIPS 2017 Competition Book. Springer, Springer, 2018.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, and Michael Bernstein. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211–252, 2015.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. 2015.
Acknowledgements
The NIPS 2017 Competition track was sponsored by NIPS and ChaLearn.
The Conversational Intelligence Challenge was partially sponsored by Facebook, Flint Capital, IVADO, Microsoft Maluuba, Element AI.
The Learning to Run Challenge was organized by the Mobilize Center at Stanford University, a National Institutes of Health Big Data to Knowledge (BD2K) Center of Excellence supported through Grant U54EB020405, and by the crowdAI.org platform. The challenge was partially sponsored by NVIDIA, Amazon Web Services and Toyota Research Institute.
The Neural Art Challenge was sponsored by a number of sponsors, who we would like to thank. DeepArt.io sponsored the high-resolution renderings. ChaLearn.org sponsored the printing of the posters. Prices were sponsored by NVIDIA.
The IBM Watson AI XPRIZE is sponsored by IBM Watson.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Escalera, S. et al. (2018). Introduction to NIPS 2017 Competition Track. In: Escalera, S., Weimer, M. (eds) The NIPS '17 Competition: Building Intelligent Systems. The Springer Series on Challenges in Machine Learning. Springer, Cham. https://doi.org/10.1007/978-3-319-94042-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-94042-7_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-94041-0
Online ISBN: 978-3-319-94042-7
eBook Packages: Computer ScienceComputer Science (R0)