Engaging Earth- and Environmental-Science Undergraduates Through Weather Discussions and an eLearning Weather Forecasting Contest
- 513 Downloads
For students who major in meteorology, engaging in weather forecasting can motivate learning, develop critical-thinking skills, improve their written communication, and yield better forecasts. Whether such advances apply to students who are not meteorology majors has been less demonstrated. To test this idea, a weather discussion and an eLearning weather forecasting contest were devised for a meteorology course taken by third-year undergraduate earth- and environmental-science students. The discussion consisted of using the recent, present, and future weather to amplify the topics of the week’s lectures. Then, students forecasted the next day’s high temperature and the probability of precipitation for Woodford, the closest official observing site to Manchester, UK. The contest ran for 10 weeks, and the students received credit for participation. The top students at the end of the contest received bonus points on their final grade. A Web-based forecast contest application was developed to register the students, receive their forecasts, and calculate weekly standings. Students who were successful in the forecast contest were not necessarily those who achieved the highest scores on the tests, demonstrating that the contest was possibly testing different skills than traditional learning. Student evaluations indicate that the weather discussion and contest were reasonably successful in engaging students to learn about the weather outside of the classroom, synthesize their knowledge from the lectures, and improve their practical understanding of the weather. Therefore, students taking a meteorology class, but not majoring in meteorology, can derive academic benefits from weather discussions and forecast contests. Nevertheless, student evaluations also indicate that better integration of the lectures, weather discussions, and the forecasting contests is necessary.
KeywordsMeteorology Forecasting Education Contest Evaluation
The number-one attraction for students who take atmospheric science classes is weather forecasting (Knox and Ackerman 2005). Besides its popularity with students, active learning through repeated practice is the best way for students to gain experience and improve (Roebber and Bosart 1996). Also, tomorrow’s forecast is not found among the answers in the back of the book! Weather forecasting also encourages critical-thinking skills, involving the highest levels in the cognitive domain of Bloom’s taxonomy (e.g., application, analysis, synthesis, evaluation) (Cervato et al. 2009). In addition, forecasting contests and discussions of the current weather in the classroom can motivate student learning (Harrington et al. 1991; Skeeter 2006), inspire better grades (Hilliker 2008), improve their written communication skills (Market 2006), and result in better forecasts (Market 2006). Finally, weather discussions help close the gap between the knowledge-seeking professors and the goal-seeking students (Roebber 2005) because an otherwise sterile set of equations illustrating complicated physical concepts comes alive when being used to illustrate the current weather in the news or out the window.
Teaching weather in the classroom can also stimulate educational and scientific research. The results from weather forecasting contests have motivated published journal articles on the forecast process (e.g., Sanders 1963, 1967, 1973, 1979, 1986; Bosart 1975, 1983; Gyakum 1986; Bosart and Landin 1994; Hamill and Wilks 1995; Roebber et al. 1996; Bond and Mass 2009; Decker 2012). Weather discussions at the University at Albany deriving from undergraduate and graduate laboratory research courses have motivated student-led published research articles (e.g., Schultz et al. 1997; Bals-Elsholz et al. 2001), and weather education at universities has even led to commercial ventures (Roebber et al. 2010; www.weatherquest.co.uk).
Even for students who are not majoring in meteorology, students in other disciplines who take a meteorology course may develop an appreciation for how the weather affects society and the economy, an understanding of what is happening outside and how it affects their lives, and a practical knowledge about the weather and how it evolves. Yet, whether these students would benefit from similar forecasting exercises as meteorology majors is less well known. Specifically, Hilliker (2008) showed that students in a general education course and geology majors who participated in a forecast contest could improve as they developed more experience, although Seuss (2011) did not find such improvement among students in a survey course for nonmajors. In addition, students’ performance in the contest may (e.g., Hilliker 2008) or may not (e.g., Bond and Mass 2009) be correlated with their academic performance.
An opportunity to test the validity of these ideas in a different context arose in 2010. A new meteorology course (EART 30551) was being developed at the University of Manchester for about 45 third-year earth- and environmental-science students. As there is no degree program specific to meteorology at Manchester, all students had little previous exposure to meteorology before this class. The goals of the course were for the students to see the practical application of the lecture material, to look at the weather and weather forecasts outside of the lecture period, and to engage their critical-thinking skills. The course was designed to have a 20- to 30-min weather discussion presented by the instructor in lecture, coupled with a weather forecasting contest. The weather discussion explored the recent and present weather in the context of the week’s lecture material and then discussed the forecast. With the support of the Faculty eLearning Team, an application was built to collect the student forecasts, score the forecasts, and display the results. This contest was run for two consecutive years: autumn semesters 2010 and 2011. The performance of the students was assessed, and student evaluations were collected at the end of the semesters asking about the utility of the weather discussions and the forecast contest to their learning. The purpose of this article is to describe the contest and discussions and to assess whether students not majoring in meteorology benefit from such activities. In particular, do students gain critical-thinking skills, are students more motivated to learn about the weather, and did the forecasting contest close the gap between the instructor and students?
The Contest Rules and the Weather Discussion
The eLearning Team led the effort to design and build MetCast, a Web site that the students could log into to enter their forecasts, check the results of the previous forecasts, and see their rankings in the contest. The Web site was accessible either directly or through the University of Manchester virtual learning environment Blackboard. The contest was run in two consecutive autumn semesters (2010 and 2011) to 40 and 44 students, respectively.
The purpose of the forecasting contest is to get you to think about the weather at least once a week in more detail than “Do I need my wellies today?” During weeks 2–11, you will be responsible for making a forecast once a week for two quantities:
the maximum temperature (in whole °C) at the official Met Office observing site at Woodford east of Manchester (WMO code = 03348) for the 24-h period starting 0000 UTC Saturday, and
your assessment of the likelihood of precipitation at Woodford for the same 24-h period. Your assessment of the probability of precipitation (POP) will be determined by an integer between 0 and 10, where 0 means zero chance of precipitation and 10 means a certain (100 %) likelihood of precipitation.
Participation in the forecast contest is mandatory and is included as part of your evaluation (10 % of the final class grade). Quality of the forecasts will not be part of the evaluation. However, the game will be scored. The best six scores will receive the following bonus points added onto their final course grades (ranked from first to sixth): 6, 5, 4, 3, 2, and 1. (In other words, if you have the best score in the forecast contest and received a 65 in the class assignments, your final score will be increased to 71.) [In 2010, this section read, “The student with the best score in the class will receive an additional 8 marks, the second best score will receive an additional 6 marks, the third best score will receive 4 marks, and the fourth best score will receive 2 marks.”] Any student who beats the score from the average of the class forecasts (called consensus) will receive an additional 2 marks (in addition to the marks above if you are one of the top scores).
Error points will be assigned to individual forecasts as the sum of the temperature error [abs (observed temperature − forecast temperature) × 8] and the precipitation error [(observed POP − forecast POP)2], where the observed POP is 0 if no precipitation is measured, 5 if a trace of precipitation is measured, and 10 if precipitation is measured (0.1 mm or more). The sum of the error points from all ten individual forecasts over the class will be summed to produce the total score. The student with the lowest score will be declared the winner.
If you do not submit a forecast by 2300 UTC, you will receive 100 error points for the week’s forecast. This error will likely be damaging to your ranking, but may still keep you in the game. Furthermore, missing a forecast will affect your mark for the 10 % assessment.
In making your forecast, you may consult any source of information that you wish: your own local knowledge of weather in Manchester, Internet, Met Office, TV or radio weather forecasts, other students, tarot cards, etc. A basic set of URLs will be provided within Blackboard. Enter your forecast from the Web site http://metcast.seaes.manchester.ac.uk, which is also available through Blackboard.
To start, login to the Metcast Web site http://metcast.seaes.manchester.ac.uk with your university account and create a username/pseudonym (does not have to be your real name or your university account name, but must be a clean and appropriate name).
Because the precipitation error is squared, you should only forecast 0 or 10 if you are absolutely certain it will not or will precipitate. If you are not expecting it to rain, make a forecast of 1 or 2. It is better to take 1 or 4 error points if it does not rain than to take 81 or 64 points if it does rain.
- 2.Climatology (the average conditions at Manchester or CLIMO) is worth knowing. The average monthly high temperatures and POPs are listed below.
October: 14 °C, 5
November: 9 °C, 6
December: 7 °C, 6
Some changes were made to the contest after 2010. First, only the four best students in 2010 received bonus points added onto their final course grades (8, 6, 4, and 2 bonus points, respectively). These bonus points were changed in 2011 to allow more students (six) to benefit from the contest. Second, the number of error points from missing a forecast was dropped from 150 points in 2010 to 100 points in 2011. This decrease was to keep students who miss one forecast from being knocked too far out of contention. These weights between temperature and precipitation errors are based on the lead author’s experience from past forecast contests.
For comparison, a no-brainer forecast of climatology each week would have yielded 346 points in 2010 and 380 points in 2011. A student forecasting climatology would have ended up in 11th place overall in 2010 and 35th place overall in 2011, neither of which would have placed the student in a position to win bonus marks.
In both years, at the end of each lecture, the instructor gave a 20- to 30-min weather discussion. The discussion was meant to elaborate upon the recent lecture material and to show the practical application of the lecture material to the real atmosphere. The weather discussion involved a presentation of the recent past, present, and future weather. Discussion of the future weather involved providing the Met Office forecast, showing some forecast weather maps, and giving some general guidance to the students in making their forecast for this week’s contest.
Description of the Forecast Contest Application
eLearning application is thought to be most effective when learners can associate the object of learning directly rather than trying to understand how to use the system (Constabile et al. 2005). Therefore, the user interface for the students was kept as simple as possible. The Web application authenticated users through the university’s Central Authentication System so that it integrates better with other university systems. In this way, students can access the site via the university’s virtual learning environment more seamlessly. The registering of students for the contest was controlled via release of an entry password as this avoided the additional technical effort in integrating with the university’s student management system.
The Web application was developed to manage multiple competitions at the same time, although in practice only one competition ever ran at a time. Students could enter or edit forecasts up to the deadline. They could also review their past performance by comparing their previous forecasts against the observed readings. A student’s ranking against the other participants in the contest was presented as a simple bar chart, which also displayed the class consensus (Fig. 2).
These efforts in the design and implementation of MetCast appeared to pay off. At the end of 2010, the students were asked to comment on the MetCast application, whether there were technical issues or aspects that were confusing. Only a few students noted minor technical problems, but none were persistent. No serious problems were noted in 2011 either.
Results of the Contests
There were some differences between the forecast contest results from 2010 and 2011. One difference was that the 2011 students outperformed the 2010 students. In 2011, six students beat consensus, which had a score of 128 points. By comparison, only one student in 2010 beat consensus, which had a score of 275 points. (Recall that receiving fewer points is a better score.) Anecdotally, the weather situations in 2010 seemed to have been harder to forecast for than in 2011. The second difference is that students were more likely to take greater risks in 2011 than in 2010 (e.g., only forecasting 0 or 100 % probability of precipitation). The third difference is that fewer students missed forecasts in 2011 than in 2010. Finally, the students in 2011 seemed to have been more engaged with the course than the students in 2010. The reasons for these last three differences are unknown.
The errors in probability of precipitation forecasts as a function of week for 2010 and 2011 are shown in Fig. 3c,d. On average, the smaller errors in 2011 are indicative of the tendency for students to have taken greater risks (and to have it pay off) in the probability of precipitation forecasts mentioned earlier (Fig. 3c,d). Otherwise, there is little indication that the forecasts showed improvement over the 10 weeks. Instead, the higher errors were associated with more difficult forecasts that did not agree with the majority of the students’ (and the instructor’s) forecasts.
At the end of 2010 and 2011, the students were asked for their opinions about the contest and its contribution to the course. Changes implemented in 2011 (to be discussed below) improved the ratings that the students gave the discussion and the contest.
Indeed, in teaching the course in 2011, a greater effort at integration was made. More detailed notes (including annotated graphics) from the forecast discussion were made available online after lecture for students to study from. These actions seemed to be responsible for the improvement of students’ learning. A larger proportion of students felt that the contest was a good use of their time (73 % agreed or strongly agreed, with another 24 % in between; Fig. 5) and felt that the contest helped them understand the lectures (50 % agreed or strongly agreed, with another 25 % in between; Fig. 6). Nearly all students said that the weekly discussions of the weather in lecture helped them understand the lecture material (85 % agreed or strongly agreed, with another 11 % in between; Fig. 7). Although these improvements are heartening, more work remains in integrating the forecast contest with the lecture material and the weather discussions.
How Students Made the Forecasts
Given that the students had no prior experience in weather forecasting, how did they do it? What resources did they use? How did the best students win?
Sometimes I was prompted to read the more detailed sections on the websites about the weather. It certainly helped my awareness of how accurate/inaccurate forecasting is and made me more interested in the weather.
I just used the BBC website and the Met Office website, and if those were vastly different I would use ‘weatheronline.co.uk’, then just decided upon an average each week.
I found the map discussions very interesting regarding the forecasts we had just done and for instance why unpredicted events occurred.
I mainly used the Met office website. But then I noticed their temperature results were often a couple of degrees out. So then I started looking at other news sources like the BBC. … In the final few weeks I came to the rough conclusion the Met Office would probably predict the temp to be around one or two degrees lower than the actual temperature so then I put the temperature higher taking into account the other sources’ predictions.
An email survey of the winners from both years yielded similar results. Students who did well examined at least two Web sites for forecasts, and two mentioned waiting until the deadline before submitting their forecast to capture any last-minute changes to the weather that may affect their forecast. A couple of students mentioned that, before this course, they followed the weather regularly anyway because of a personal interest in it, including one who was an outdoor enthusiast.
The forecast contest and weather discussion implemented in this course demonstrated that even non-meteorologists could appreciate the fun and the challenge of weather forecasting. The contest also encouraged the students to look at the weather outside of the lecture period, look at weather forecasts from sources outside the classroom (Met Office, BBC, internet sites), and be more conscious of how to obtain this information. The results showed that the best students academically were not the ones who necessarily performed well in the forecast contest, contrary to some previously published results. Specifically, students who may not necessarily be book-smart can be competitive.
Based on the feedback from the 2010 students, the course was altered for 2011. Changes included more students receiving bonus marks, more integration between the lecture material, forecast contest and weather discussion, and annotated notes from the weather discussion made available online. Student evaluations indicated that the forecast contest and weather discussions helped the students grasp concepts from lecture and was a good use of their time.
Thus, our results support the conclusions from the previous literature that forecast contests help develop students’ critical-thinking skills from the highest levels in the cognitive domain of Bloom’s taxonomy. Specifically, students gather weather information from several sources and apply it to the construction of their forecast, students analyze the quality and the source of the weather information in creating their own forecast, students synthesize information from these disparate sources, and they critically evaluate the information they collect in constructing their forecast. The student evaluations showed that the weather discussion and the forecast contest motivated the students to learn about the weather in order to succeed in the contest, although better integration is still needed. Finally, because the answers to tomorrow’s forecast are not in the back of the book, the contest helps turn students into knowledge-seekers rather than goal-seekers, closing the gap between instructors and students.
We thank Prof. Dan Keyser of the State University of New York at Albany for the inspiration to hold a weather forecasting contest for non-meteorology majors, the students for their participation in the contest and their feedback, eLearning Manager Alison Hamilton for her support of this project, and the anonymous reviewer for prompt and constructive feedback that improved the manuscript.
- Bosart LF (1983) An update on trends in skill of daily forecasts of temperature and precipitation at the State University of New York at Albany. Bull Am Meteor Soc 64:346–354Google Scholar
- Constabile MF, De Marsico M, Lanzilotti R, Plantamura VL, Roselli T (2005) On the usability evaluation of e-Learning applications. In: Proceedings of 38th Hawaii international conference on system sciences, 3–6 January 2005, IEEE, pp 1–10Google Scholar
- Decker SG (2012) Experiences with a probabilistic forecasting game for meteorology students. Preprints, 21st symposium on education, New Orleans, LA, Am Meteor Soc 3.4Google Scholar
- Gedzelman SD (1978) Forecasting skill of beginners. Bull Am Meteor Soc 59:1305–1309Google Scholar
- Hilliker J (2008) Assessment of a weather forecasting contest in multi-leveled meteorology classes. J Geoscience Educ 56(2):160–165Google Scholar
- Sanders F (1986) Trends in skill of Boston forecasts made at MIT, 1966–84. Bull Am Meteor Soc 67:170–176Google Scholar
- Seuss EJ, Cervato C, Gallus WA, Jr, Hobbs J (2011) Weather forecasting as a learning tool in a large service course: does practice make perfect? Preprints, 20th symposium on education, American Meteorological Society, 6 pp. Available online at http://ams.confex.com/ams/91Annual/webprogram/Manuscript/Paper179788/Suess et al. 2010 extended abstract.pdf