Skip to main content
SpringerLink
Log in

T-Brain: A Collaboration Platform for Data Scientists

  • Conference paper
  • First Online:
Security with Intelligent Computing and Big-data Services (SICBS 2017)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 733))

  • 1124 Accesses

Abstract

When data were generated easily and rapidly with mobile services and computing power can increase on demand with the cloud computation service, data scientists who work with huge data can solve challenging problems. Smart intelligent applications such as Go, healthcare and self-driving vehicles show great improvement recently. In addition to those problems, there are still more complex problem such as weather impacts analysis, financial crisis prediction and crime prevention and so on. To overcome those challenging problems, many crossdisciplinarity or interdisciplinary experts have to collaborate for the solutions. In the paper, we propose a collaboration platform and a system design for data scientists to share data, write analytic scripts and discuss topics related with those problems. In current status, eleven dataset were collect ed such as spam mail, malware data, honeynet log, Hadoop workload log and some other open data and based on those dataset and improvement local cache design (i.e., average response time improvement 92.36% and request availability improvement 70%). With the platform, many education and competition activities can be hold successfully on the collaboration platform.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., et al.: Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016)

    Article  Google Scholar 

  2. Sadek, I., Elawady, M., Shabayek, A.E.R.: Automatic classification of bright retinal lesions via deep network features. arXiv preprint arXiv:1707.02022 (2017)

  3. Huval, B., Wang, T., Tandon, S., Kiske, J., Song, W., Pazhayampallil, J., et al.: An empirical evaluation of deep learning on highway driving. arXiv preprint arXiv:1504.01716 (2015)

  4. InnoCentive|Open Innovation & Crowdsourcing Platform. https://www.innocentive.com/

  5. Kaggle: Your home for data science. https://www.kaggle.com/. Accessed 1 Dec 2017

  6. CrowdANALYTIX: Automating business processes using artificial intelligence

    Google Scholar 

  7. CodaLab - Home. https://worksheets.codalab.org/. Accessed 1 Dec 2017

  8. Topcoder|Deliver Faster through Crowdsourcing. https://www.topcoder.com/

  9. HackerRank|Technical Recruiting|Hiring the Best Engineers. https://www.hackerrank.com/. Accessed 1 Dec 2017

  10. Merkel, D.: Docker: lightweight Linux containers for consistent development and deployment. Linux J. 2014, 2 (2014)

    Google Scholar 

  11. Docker Swarm overview|Docker Documentation. https://docs.docker.com/swarm/overview/. Accessed 1 Dec 2017

  12. Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51, 107–113 (2008)

    Article  Google Scholar 

  13. Project Jupyter|Home. http://jupyter.org/. Accessed 1 Dec 2017

  14. Kluyver, T., Ragan-Kelley, B., Pérez, F., Granger, B.E., Bussonnier, M., Frederic, J., et al.: Jupyter Notebooks-a publishing format for reproducible computational workflows. In: ELPUB, pp. 87–90 (2016)

    Google Scholar 

  15. GitHub - jupyterhub/jupyterhub: Multi-user server for Jupyter notebooks. https://github.com/jupyterhub/jupyterhub. Accessed 1 Dec 2017

  16. Overview of RESTful API Description Languages - Wikipedia. https://en.wikipedia.org/wiki/Overview_of_RESTful_API_Description_Languages. Accessed 1 Dec 2017

  17. Deploying JupyterHub for Education. https://developer.rackspace.com/blog/deploying-jupyterhub-for-education/. Accessed 1 Dec 2017

  18. Redis. https://redis.io. Accessed 1 Dec 2017

  19. Celery: Distributed task queue. http://www.celeryproject.org

  20. Selenium - Web Browser Automation. http://www.seleniumhq.org

  21. keras-mnist-tutorial/MNIST in Keras.ipynb at master · wxs/keras-mnist-tutorial · GitHub. https://github.com/wxs/keras-mnist-tutorial/blob/master/MNIST%20in%20Keras.ipynb

  22. OpenChorus Project: The Dawn of The Data Science Movement|Dell EMC Big Data. http://bigdatablog.emc.com/2012/11/09/openchorus-project-the-dawn-of-the-data-science-movement/

  23. Zeilenga, K.: Lightweight directory access protocol (LDAP): technical specification road map (2006)

    Google Scholar 

  24. Pierson, N.: Overview of Active Directory Federation Services in Windows Server 2003 R2. Microsoft Corporation, October 2005

    Google Scholar 

  25. Hellerstein, J.M., Ré, C., Schoppmann, F., Wang, D.Z., Fratkin, E., Gorajek, A., et al.: The MADlib analytics library: or MAD skills, the SQL. Proc. VLDB Endow. 5, 1700–1711 (2012)

    Article  Google Scholar 

  26. RC Team: R language definition. R Foundation for Statistical Computing, Vienna, Austria (2000)

    Google Scholar 

  27. Trusted Analytics. https://github.com/trustedanalytics. Accessed 1 Dec 2017

  28. Kafka, A.: A high-throughput, distributed messaging system, vol. 5 (2014). kafka.apache.org

  29. Zawodny, J.: Redis: lightweight key/value store that goes the extra mile. Linux Magazine, vol. 79 (2009)

    Google Scholar 

  30. Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauley, M., et al.: Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, p. 2 (2012)

    Google Scholar 

  31. Vora, M.N.: Hadoop-HBase for large-scale data. In: 2011 International Conference on Computer Science and Network Technology (ICCSNT), pp. 601–605 (2011)

    Google Scholar 

  32. Chodorow, K.: MongoDB: The Definitive Guide: Powerful and Scalable Data Storage. O’Reilly Media, Inc. (2013)

    Google Scholar 

  33. H2O.ai. https://www.h2o.ai. Accessed 1 Dec 2017

  34. RStudio - Open source and enterprise-ready professional software for R. https://www.rstudio.com

  35. Pérez, F., Granger, B.E.: IPython: a system for interactive scientific computing. Comput. Sci. Eng. 9 (2007)

    Google Scholar 

  36. HackNTU. https://www.facebook.com/hackNTU/posts/1146642025421019:0. Accessed 3 Dec 2017

  37. Yeh, C.-C., Zhou, J., Chang, S.-A., Lin, X.-Y., Sun, Y., Huang, S.-K.: BigExplorer: a configuration recommendation system for big data platform. In: 2016 Conference on Technologies and Applications of Artificial Intelligence (TAAI), pp. 228–234 (2016)

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by Trend Micro, National Center for High-Performance Computing, the Institute for Information Industry under the grant 106-EC-17-D-11-1502 and G367JA1210, Taiwan Information Security Center (TWISC), Academia Sinica, and the Ministry of Science and Technology, Taiwan under the grant 105-2221-E-009-122, and 106-3114-E-009-007.

Author information

Authors and Affiliations

  1. Department of Computer Science, National Chiao Tung University, Hsinchu, 300, Taiwan

    Chao-Chun Yeh & Shih-Kun Huang

  2. Information Technology Service Center, National Chiao Tung University, Hsinchu, 300, Taiwan

    Shih-Kun Huang

  3. Computational Intelligence Technology Center, Industrial Technology Research Institute, National Chiao Tung University, Hsinchu, 300, Taiwan

    Chao-Chun Yeh, Sheng-An Chang, Yi-Chin Chu, Xuan-Yi Lin, Yichiao Sun & Jiazheng Zhou

Authors
  1. Chao-Chun Yeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sheng-An Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yi-Chin Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xuan-Yi Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yichiao Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jiazheng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Shih-Kun Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao-Chun Yeh .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Information Engineering, National Dong Hwa University, Hualien, Taiwan

    Sheng-Lung Peng

  2. Department of Information Management, Central Police University, Taoyuan, Taiwan

    Shiuh-Jeng Wang

  3. Department of Automation and Applied Informatics, Faculty of Engineering, Aurel Vlaicu University of Arad, Arad, Romania

    Valentina Emilia Balas

  4. School of Computer Technology, Jingzhou, China

    Ming Zhao

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yeh, CC. et al. (2018). T-Brain: A Collaboration Platform for Data Scientists. In: Peng, SL., Wang, SJ., Balas, V., Zhao, M. (eds) Security with Intelligent Computing and Big-data Services. SICBS 2017. Advances in Intelligent Systems and Computing, vol 733. Springer, Cham. https://doi.org/10.1007/978-3-319-76451-1_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-76451-1_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-76450-4

  • Online ISBN: 978-3-319-76451-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation