Current Challenges in Statistics

Abstract

Much has been written on the future of statistics in the era of rapid technological innovations, and huge data quantities that require quick analysis. The number of statistics and biostatistics departments at universities is growing. The number of Ph.D. degrees conferred to students is in the hundreds, and there are many positions available to statisticians in industries, hospitals, business firms, and universities. Although it seems that the state of Statistical Science is strong, it is challenged now by the trend to develop departments of Data Science. In the past, departments of Computer Science focused attention on teaching hardware engineering or software engineering. Their students were directed at many universities to study statistics and applied probability in statistics or mathematics departments. In the future, this cooperation might end. Even among statisticians there are calls to change the way we teach statistics from models to ready-made algorithms (see Kaplan, Am Stat, 7:89–96, 2018). To be a good data scientist one should be strong in mathematics, statistics, applied probability, operations research (optimization), and computers. It seems to me that a BS degree in mathematics and MS degree in applied statistics, with a strong emphasize on big data analysis, like courses on data mining, machine learning, and neuro-networks, might give the students an excellent preparation for data analysis. Algorithms are computational instructions and as such cannot replace models. Science is based on theory and models, if data refutes a theory a new theory is required, or a modification of the old one. Experiments in science laboratories are of moderate size and are based on experimental design. Usually experiments are done in order to test a hypothesis. For this we still need statistical methodology based on models. Thus basic statistical education is required for anyone who analyzes data. In the following section some ideas based on experience are presented about teaching statistics to data scientists. Departments of statistics could serve well departments of data science. This requires creating courses with appropriate syllabuses. This is a real challenge that should be taken seriously.