Abstract
Due to COVID19, universities across Canada were forced to undergo a transition from classroombased facetoface learning and invigilated assessments to onlinebased learning and noninvigilated assessments. This study attempts to empirically measure the impact of COVID19 on students’ marks from eleven science, technology, engineering, and mathematics (STEM) courses using a Bayesian linear mixed effects model fitted to longitudinal data. The Bayesian linear mixed effects model is designed for this application which allows studentspecific error variances to vary. The novel Bayesian missing value imputation method is flexible which seamlessly generates missing values given complete data. We observed an increase in overall average marks for the courses requiring lowerlevel cognitive skills according to Bloom’s Taxonomy and a decrease in marks for the courses requiring higherlevel cognitive skills, where larger changes in marks were observed for the underachieving students. About half of the disengaged students who did not participate in any course assessments after the transition to online delivery were in special support.
Introduction
The spread of the novel coronavirus disease 2019 (COVID19) started in December 2019 in Wuhan, China [32]. Due to the rising health concerns, many universities around the world transitioned from facetoface to online course delivery and assessments [24]. In the province of British Columbia (BC), Canada, all oncampus classes and activities in the postsecondary schools were cancelled starting March 15, 2020. At Thompson Rivers University (TRU), the facetoface classes shifted to alternative modes of delivery mostly conducted through the online learning management systems. In order to adapt to the changes, most, if not all, of remaining course assessments were switched to openbook noninvigilated exams with adjusted weights for the evaluation components. In this study, we proposed a Bayesian statistical model to evaluate the effects of the sudden change from classroombased to online delivery and assessments in response to COVID19 pandemic on students’ academic performance. It is important to understand comprehensively how students’ performance has changed after March 15, 2020 due to the COVID19 effect and to investigate factors that potentially contributed to the changes.
Due to the unique situation of COVID19 and its social and educational ramifications, there have been a few published articles discussing how COVID19 and the shutdown of universities have impacted students’ performance in postsecondary education. For instance, Sintema [26] investigated the possible impacts that the closure of secondary schools due to COVID19 in Zambia would have on the general performance of students in specific subject areas. Having interviewed STEM^{Footnote 1} educators at a public secondary school in Zambia, the study concluded that learners’ performance would be negatively affected in STEM subjects in the upcoming national examination if the COVID19 epidemic is not being lessened in the shortest possible time. This is because of not only the lack of contact and meaningful interactions between learners and teachers, but also insufficient elearning tools to facilitate such interactions. Basilaia and Kvavadze [3] studied the capacities of the country Georgia and its population to continue the education process at the schools in the online form of distance learning. The study was conducted in a private school during the COVID19 pandemic which reviewed different available platforms that were used with the support of the government for online education and live communications. Their findings showed that the transition from traditional to online education system was successful in terms of adaptability and gained skills by students, teachers and administrative staff. Nevertheless, none of these studies compared students’ performances empirically.
During the Severe Acute Respiratory Syndrome (SARS) epidemic in Hong Kong, when all schools and universities were ordered closed and governments invoked quarantine laws to isolate those who might be the carriers, Wong [29] conducted a study which briefly described the impact of ecommerce on the local community with an emphasis on the use of elearning technology as a contingency measure in tertiary institutions. Their study showed that, given limited time available for the course design and delivery, the examination result of the elearning class was slightly better than the traditional class. However, their study lacks the rigorous empirical comparisons of students’ marks using a sound statistical model. They suggested further rigorous study.
There are some studies evaluating how students perform in online versus facetoface delivery in postsecondary mathematics education. Jones and Vena [13] focused on equity in learning as reflected in the final grades of online and onsite students from the same postsecondary mathematics course taught repeatedly over 10 semesters. Onsite students attended regular class sessions, while online students only attended an orientation session and a final exam. For both groups, the evaluations were invigilated. Their findings revealed significant statistical differences in online and onsite students’ final grades, in favor of onsite student achievement. Rey [23] examined the associations between taking basic skills mathematics courses online versus facetoface and student success and persistence. The study stressed on the difficulties associated with effective communication of mathematics topics and ideas via the Internet and found no noticeable associations with learning outcomes or persistence. Their study also pointed out that the quality of education gained from online basic skills mathematics courses is relatively equivalent to facetoface courses. Weems [27] compared two sections of beginning algebra course where one was taught online and the other was taught on site. The study reported no significant difference for exam average between the two formats, but highlighted a decrease in performance by the online students across the exams. Ashby et al. [2] compared student success in a Developmental Math course offered in three different learning environments: online, blended, and facetoface. Using a one way analysis of variance (ANOVA), the authors showed that there were significant differences between learning environments with the students in the blended courses having the least success. With regards to the efficiency of learning outcomes in the online pedagogy, Arias et al. [1] studied the effectiveness of online delivery relative to facetoface delivery by randomly assigning students into two categories: online and facetoface. These two sections were taught by the same instructor and the course objectives and exams were the same for both sections as well. The authors concluded that “both course objectives and the mechanism used to assess the relative effectiveness of the two modes of education may play an important role in determining the relative effectiveness of alternative delivery methods.” One other area of interest in this respect is the notion of exams and their competence in the digital space. In a study by Williams and Wong [28], the authors examined the efficacy of closedbook invigilated final exams versus openbook and openweb (OBOW) final exams in a completely online university. They analyzed students’ experience who had completed both formats of the exam by surveying them on the merits of each exam format. Their findings showed that \(100\%\) of students found OBOW preferable to traditional closedbook exams. On the issue of academic integrity, their results indicated that in both exam formats “there has been equal opportunity of plagiarism in the view of students.” On the other hand, there are some studies indicating that online exams and access to technology provide more opportunities for dishonest behaviors [14]. Even though there are some similarities and differences with the aforementioned studies, the research environment of our study is qualitatively different. In our study, a Bayesian statistical model is proposed to compare students’ performances reflected by their empirical marks received in a sudden unexpected scenario where both classes and evaluations were virtually forced to be conducted online.
On the other hand, the closure of universities and transitioning to online teaching may have prompted mental, psychological and educational challenges for students and faculty members. A recent study by Statistics Canada based on crowdsourcing data completed by over 100,000 postsecondary students from April 19 to May 1, 2020 provides insight into how students’ academic life was impacted by the COVID19 pandemic [11]. Admittedly, the adoption of alternate modes of delivery and distance learning under these circumstances are not ideal. Students accustomed to oncampus learning have expressed concerns over the loss of social and interactive side of education. In addition, the rapid transition to online instruction poses challenges to students who were not equipped to adjust to this mode of learning either because they learn better inperson, lack appropriate tools, suffer financial hardship, or do not have a home environment suitable for learning online [11]. The transition of classes to online from inperson lectures due to COVID19 had an even more severe impact on students with mental health disabilities who needed extra care and frequent facetoface interactions with the instructors in order to keep maintaining motivation, interest and persistence to succeed in the course [9]. In a recent study, Sahu [24] highlighted the impacts of COVID19 on education and mental health of students and academic staff and indicated the challenges posed by the closure of universities and other restrictive measures such as the shift in the delivery mode of the course, the change in the format of assessments, the travel restrictions on students, etc. In another study, Cao et al. [5] investigated students’ anxiety level during COVID19 in a Chinese medical college located in Hubei Province. Their findings showed that about a quarter of students in their sample had experienced mild, moderate or severe levels of anxiety during COVID19 outbreak. In this paper, we have studied some aspects of stress among students with disabilities and special needs that contributed to their disengagement in any of the remaining evaluation components after the emergence of COVID19.
We developed a Bayesian hierarchical linear mixed effects model to measure the effects of COVID19 on students’ marks. The literature of random effects model goes back to Laird and Ware [16]. The classical maximum likelihood estimation and inference of the linear mixed effects model using expectation maximization (EM) algorithm were shown by Laird et al. [15] and Lindstrom et al. [19]. These articles approached the problem of model building and inference from a frequentist viewpoint. On the other hand, the general design of Bayesian method for the linear mixed effects model was described by Zhao et al. [31]. The missing value imputation for linear mixed effects model, using a frequentist viewpoint, was proposed by Schafer et al. [25]. In this paper, we redesigned the linear mixed effects model for the change of students’ raw marks before and after COVID19 effect in Winter 2020 semester at TRU. We described the complete Bayesian methodology for the proposed model using conjugate and semiconjugate prior distributions. We then derived the full conditional posterior distributions of the parameters and proposed the Gibbs sampling [17] to generate Markov Chain Monte Carlo (MCMC) samples [12] from the posterior distributions. In order to impute missing values of marks, we assumed the mechanism of missingness completely at random [4]. Our novel contributions are the design and implementation of the fully Bayesian missing value imputation method in a linear mixed effects modeling setup. While most of the classical statistical missing value imputation methods are performed only once given complete data before the model building [7], our method of Bayesian missing value imputation is flexible which seamlessly generates missing values before every generation of the Markov chain from the posterior distributions of the missing values given observed data. In our model, we allow studentspecific error variances to vary which is a considerable extension of the methodology in a Bayesian linear mixed effects hierarchical modeling setup. We wrote our codes for the proposed fully Bayesian hierarchical model using R [22]. The R codes and data are available on GitHub (https://github.com/jhtomal/covid19_impact.git). We are in the process of wrapping up the codes in an R package to publish in the Comprehensive R Archive Network (CRAN) so that the broad scientific community can apply our method in their applications.
We hypothesize that the COVID19 has negative effects on students’ performances, reflected on their marks, and on their stress level especially on students needing special supports. As such, this article aims to explore the following questions:

(i)
How are the raw marks of all students in a course compared before and after the university switched to online delivery due to COVID19?

(ii)
How are the raw marks of individual students within a course compared before and after the transition to online delivery?

(iii)
Are the stronger or weaker students getting higher or lower average raw marks due to online delivery relative to inperson delivery?

(iv)
Are there disengaged students who did not participate in any evaluation components in a course after the university shifted to online delivery? Are these students on special support?

(v)
Are the trends of marks consistent across all courses or departments in this study? If there are different trends, what are the potential factors that make the difference?
In order to cope with the the unexpected transition to online delivery and openbook noninvigilated evaluations, the instructors from different departments in the Faculty of Science at TRU came up with new sets of weighted schemes which were strikingly different from what they had at the beginning of the semester. We thus analyze students’ raw marks, as opposed to weighted aggregated marks, because the raw numbers reflect the unbiased scenario. Subsequently in Sect. 5, we discussed the results according to the level of cognitive skills and handson experiences required in the courses we investigated. In our analysis, we classified the cognitive skills with reference to the Bloom’s Taxonomy of Knowledge.^{Footnote 2}
Design of the study and description of data
The analysis of empirical data starts with some summary statistics followed by a fully Bayesian linear mixed effects model. The posterior statistical inferences of the model parameters proceed following posterior model inspections.
The study population consists of all of the students in the Faculty of Science at Thompson Rivers University. We collect longitudinal data by sampling 11 courses (with total number of students equals 326; see Table 1 for details) across three departments from the Faculty of Science, including seven courses in Mathematics (MATH) and Statistics (STAT), two courses in Computing Science (COMP) and two courses in Architectural and Engineering Technology (ARET). Each course has been taught, marked,^{Footnote 3} and graded by the same instructor from the start to the end of the Winter 2020 semester (before and after COVID19 effects) which ensures instructor’s effects being adjusted while leaving only students’ effects to compare. Moreover, the Cronbach’s alpha [8] test of reliability analysis has been performed to each course to make sure that the course data are reliable. After the selection of the courses, studentspecific raw marks are recorded over time using a series of evaluations such as multiple assignments, quizzes, tests, exams, and projects. Specific to each course, we have analyzed marks from all sorts of evaluations and ensured covering all possible grading practices. The raw marks specific to each evaluation component are then converted into percents (from 0 to 100) where larger numbers indicate better marks. We also record whether the marks are observed or missing. Given that all the classes went online starting March 15, 2020, the index variable is simply defined as a vector of boolean values indicating whether the effects of COVID19 have occurred (before March 15 vs after March 15). As there is no randomness in the index variable, we consider it deterministic. A Bayesian hierarchical model with missing value imputation technique is then applied to determine whether students’ raw marks were increased or decreased before and after March 15. After model fitting, statistical analysis and inference are performed to student specific raw marks in each course. Our findings generalize to the Faculty of Science at TRU, and may eventually to all faculties of science across universities in Canada. Figure 1 shows the overall design of the study.
Methods
The linear mixed effects model
Let \(Y_{it}\) be the raw marks in percent for the ith student \((i = 1, \ldots , k)\) evaluated at time t \((t = 1, \ldots , n)\) in a particular course. The subscript t represents a series of evaluations conducted over time from multiple assignments, quizzes, tests, exams, and projects. Let \(x_{it}\) be another variable measured at time t for the ith student which can explain the variation in \(Y_{it}\). We consider the following linear mixed effects model
where
and \(\epsilon _{it} \sim \text {Normal}(0, \sigma ^2_i)\) with studentspecific error variance \(\sigma ^2_i\). The sampling distribution of \(Y_{it}\) is
In this model, \(\beta _{0i}\) and \(\beta _{0i} + \beta _{1i}\) are the average marks of the ith student before and after March 15, 2020, respectively. Here, \(\beta _{1i}\) is the difference of average marks before and after March 15, 2020 for the ith student. Let \(\varvec{\beta }_i = \left( \beta _{0i}, \beta _{1i}\right) ^T\) represents the vector of regression coefficients for the ith student. We assume that the ith student is a randomly and independently selected student from the pool of students in a specific course in the faculty of science at Thompson Rivers University (TRU). This leads us to consider the sampling distribution for \(\varvec{\beta }_i\) as
where \(\varvec{\theta } = \left( \theta _0, \theta _1\right) ^T\) and \(\varSigma \) is the variancecovariance matrix. This part of the model explains that \(\theta _0\) and \(\theta _0 + \theta _1\) are the average marks of all students in a course before and after March 15, 2020, respectively.
The sampling distribution for the studentspecific error variance is
defined in terms of shape and rate parameters, where the course specific error variance is \(\sigma ^2_0\) with strength \(\nu _0\) \((> 0)\). Here, the large values of \(\nu _0\) will force \(1/\sigma ^2_i\) to be tightly clustered around \(\sigma _0^2\) and vice versa. On the other hand, large values of \(\sigma ^2_0\) represent large betweenstudent variability and small withinstudent variability \(\sigma ^{2}_i\). The overall picture of the hierarchical model is shown below in Fig. 2.
The likelihood function for the parameters of the linear mixed effects model is:
where dnorm, dmvnorm, and dgamma are the density functions for the normal, multivariate normal, and gamma distributions, respectively. Please note that the likelihood function provides information contained in the data for the unknown parameters of interests.
The prior distributions
In addition to the information contained in the data, some extra knowledge may come from experimenter’s prior experience of the system giving rise to the data. Incorporation of this prior knowledge about the system in model building may increase the precision of the estimates for the parameters of interest.
The prior distribution of a parameter represents experimenter’s prior belief via hyperparameters. We note that the prior belief should be unbiased: a belief which reflects the expected truth of the system that generates the data. In addition, we emphasize that the prior belief should not be strong unless there is enough evidence towards the belief. This is the case because the strong prior belief might pull the posterior belief towards itself. In situations where there is weak or no prior belief, we suggest to be objective and follow the lead by the data.
The prior distribution for the coursespecific error variance \(\sigma _0^2\) is considered as
where the prior belief regarding \(\sigma ^2_0\) is represented by the hyperparameters \(\alpha _1\) (the shape parameter) and \(\alpha _2\) (the rate parameter). Specifically, the expected prior belief about \(\sigma ^2_0\) is expressed as \(E(\sigma ^2_0) = \alpha _1/\alpha _2\). Here, small and large numbers of \(\alpha _1\) represent weak and strong prior belief regarding the coursespecific error variance \(\sigma ^2_0\).
We restrict \(\nu _0\) to be a whole number and choose the prior on \(\nu _0\) to be a discrete analogue of exponential distribution on \(\{1, 2, 3, \ldots \}\) as following:
where \(\alpha _3\) reflects the strength of prior belief about \(\nu _0\). Specifically, small values of \(\alpha _3\) represent weak belief about \(\nu _0\) and vice versa.
The parameter vector for the coursespecific mean \(\varvec{\theta }\) is considered to follow the following distribution
The prior belief about the coursespecific mean vector is considered to be \(\varvec{\theta _0}\) (i.e., \(E(\varvec{\theta }) = \varvec{\theta }_0\)) and the strength of the prior belief is represented by the variancecovariance matrix \(\varSigma _0\). Here, \(\varSigma _0\) is a positivedefinite matrix where the diagonal elements contain the variances (large values of the variances represent weak prior belief) and the offdiagonal elements contain the covariances (small values of absolute covariance represent weak correlation between \(\theta \)’s).
The prior distribution corresponding to the variancecovariance matrix \(\varSigma \) of the coursespecific mean vector \(\varvec{\theta }\) is
where the prior belief about \(\varSigma \) is represented by \(S_0\) (i.e., \(E(\varSigma ) = S_0/(\eta _0  d  1)\), where d be the dimension of \(\varvec{\theta }\)) and the strength of prior belief is represented by \(\eta _0\). As in other prior distributions, large and small values of \(\eta _0\) represent strong and weak prior belief, respectively.
The posterior distributions
After collecting data, we combine the information from the data with prior belief to obtain posterior belief. In other words, the posterior belief is our updated belief after observing the data.
The posterior distribution for the overall error variance \(\sigma _0^2\) is Gamma which is obtained using Eqs. (4) and (6) as following:
The posterior distribution for the strength parameter \(\nu _0\) for the overall error variance \(\sigma _0^2\) is obtained using Eqs. (4) and (7):
The posterior distribution for the studentspecific error variance \(1/\sigma ^2_i\) is independent Gamma which is obtained by using Eqs. (2) and (4):
where \(\text {SSE}(\varvec{\beta }_i) =\left( \mathbf {y}_i \mathbf {x}_i\varvec{\beta }_i\right) ^T\left( \mathbf {y}_i \mathbf {x}_i\varvec{\beta }_i\right) \) be the studentspecific sum of squares of errors.
The posterior distribution for the studentspecific mean vector \(\varvec{\beta }_i\) is independent MultivariateNormal which is obtained using Eqs. (2) and (3) as following:
with posterior mean
and posterior variancecovariance matrix
The posterior distribution for the overall coursespecific mean vector \(\varvec{\theta }\) is Multivariate Normal which is obtained using Eqs. (3) and (8):
with posterior mean
and posterior variancecovariance matrix
The posterior distribution of \(\varSigma \) is InverseWishart which is obtained via Eqs. (3) and (9) as following
where \(S_{\varvec{\theta }} = \sum _{i=1}^k \left( \varvec{\beta }_i \varvec{\theta }\right) \left( \varvec{\beta }_i  \varvec{\theta }\right) ^T\).
Distribution of missing values
Let \(Y_{i, o}\) and \(Y_{i, m}\) be the marks that are observed and missing, respectively, for the ith student in a course. Given the distribution of the marks
the missing marks for the ith student are imputed independently by generating data from the following distribution
where
and
are obtained using the properties of conditional multivariate normal distribution with
and
Note that, in generating the missing values, the \(\varvec{\beta }_i\), \(\varSigma \) and \(\sigma ^2_i\) are generated first from their respective posterior distributions. The computational details of the missing value imputation is provided in Sect. 3.5 with specifics in step 7 of the Gibbs sampling algorithm.
Many missing value imputation methods exist in the literature such as: mean imputation, row average imputation, ordinary least squares imputation, linear model based imputation, local least squares imputation [7], regression imputation, imputation of longitudinal data [30], singular value decomposition, principal component analysis [18], and expectation maximization [25]. Most of the above missing value imputation methods are classical methods which allow the imputation of missing values only once given the observed data. Our method of missing value imputation is fully Bayesian which allows seamless imputation of missing values before every generation of the MCMC scans of the parameters from their posterior distributions.
The Gibbs sampling algorithm
The approximation of posterior distribution via Gibbs sampling is briefly presented below. For a given state of the parameters
and \(\mathbf {y}_i = \{\mathbf {y}_{i, o}, \mathbf {y}_{i, m}^{(s)}\}\), a new state is generated as follows:

1.
Sample \(\nu _0\) using
$$\begin{aligned} \nu _0^{(s+1)} \sim P\left( \nu _0  \sigma _0^{2(s)}, \sigma _1^{2(s)}, \ldots , \sigma _k^{(2)}\right) , \end{aligned}$$where the posterior distribution is specified in Eq. (11).

2.
Sample \(\sigma _0^2\) using
$$\begin{aligned} \sigma _0^{2(s+1)} \sim P\left( \sigma _0^2  \sigma _1^{2(s)}, \ldots , \sigma _k^{2(s)}, \nu _0^{(s+1)}\right) , \end{aligned}$$with posterior distribution specified in Eq. (10).

3.
For each \(i \in \{1, 2, \ldots , k\}\), independently sample \(\sigma ^2_i\) using
$$\begin{aligned} \sigma _i^{2(s+1)} \sim P\left( \sigma _i^2  \mathbf {y}_{i, o}, \mathbf {y}_{i, m}^{(s)}, \mathbf {x}_i, \varvec{\beta }_i^{(s)}, \nu _0^{(s+1)}, \sigma _0^{2(s+1)} \right) , \end{aligned}$$where the posterior distribution is specified in Eq. (12).

4.
For each \(i \in \{1, 2, \ldots , k\}\), independently sample \(\varvec{\beta }_i\) using
$$\begin{aligned} \varvec{\beta }_i^{(s+1)} \sim P\left( \varvec{\beta }_i\mathbf {y}_{i,o}, \mathbf {y}_{i,m}^{(s)}, \mathbf {x}_i, \varvec{\theta }^{(s)}, \sigma _i^{2(s+1)}, \varSigma ^{(s)}\right) , \end{aligned}$$with posterior distribution specified in Eq. (13).

5.
Sample \(\varvec{\theta }\) using
$$\begin{aligned} \varvec{\theta }^{(s+1)} \sim P\left( \varvec{\theta }\varvec{\beta }_1^{(s+1)}, \ldots , \varvec{\beta }_k^{(s+1)}, \varSigma ^{(s)}\right) , \end{aligned}$$where the posterior distribution is specified in Eq. (14).

6.
Sample \(\varSigma \) using
$$\begin{aligned} \varSigma ^{(s+1)} \sim P\left( \varSigma  \varvec{\theta }^{(s+1)}, \varvec{\beta }_1^{(s+1)}, \ldots , \varvec{\beta }_k^{(s+1)}\right) , \end{aligned}$$with posterior distribution specified in Eq. (15).

7.
For each \(i \in \left\{ 1, 2, \ldots , k\right\} \), independently sample the missing marks using
$$\begin{aligned} \mathbf {y}_{i,m}^{(s+1)} \sim P\left( \mathbf {Y}_{i,m}  \mathbf {y}_{i, o}, \mathbf {y}_{i, m}^{(s)}, \mathbf {x}_i, \varvec{\beta }_i^{(s+1)}, \varSigma ^{(s+1)}, \sigma _i^{2(s+1)}\right) , \end{aligned}$$where the posterior distribution of missing data given the observed data is specified in Eq. (17).
The order in which the new parameters and missing data are generated does not matter. What does matter is that each parameter is updated conditional upon the current value of the remaining parameters and imputed missing values. The Gibbs sampling algorithm is implemented using the R [22] language.
Specification of hyperparameters
The hyperparameters for the prior distributions are chosen as described below. The hyperparameters for the prior distribution of overall error variance \(\sigma _0^2\) (Eq. (6)) are chosen as \(\alpha _1 = 1\) and \(\alpha _2 = 1/100\). This specification implies that \(E(\sigma ^2_0) = 100\). In other words, our prior specification considers large variability between \(\sigma _1^{2}\), \(\sigma _2^{2}, \ldots , \sigma _k^{2}\). That is, our belief implies large overall error variability and small within student error variability. At the same time, we consider \(\alpha _3\) (the hyperparameter for the prior of \(\nu _0\)) to be 1 as the small values of \(\alpha _3\) represent weak prior belief about \(\nu _0\). Given that we have no prior knowledge about the overall error variance and the student specific error variances, a weak prior imposes less subjectivity and lets the data objectively determine the parameter values.
To choose the hyperparameters for the prior distribution of \(\varvec{\theta }\) (Eq. (8)), we fitted ordinary least squares (OLS) regression to studentspecific marks and saved the OLS coefficients. The OLS coefficients are averaged to specify \(\varvec{\theta }_0\) (the prior mean vector for \(\varvec{\theta }\)). The prior variancecovariance matrix \(\varSigma _0\) is considered to be the sample covariance of the OLS coefficients. Such a prior distribution represents belief that is aligned with the information contained in the data.
Similarly, we consider the prior sum of squares matrix \(S_0\) (Eq. (6)) to be equal to the sample covariance of the ordinary least squares estimates of the coefficients. This specification ensures the lead by the data as the expectation of the prior distribution of \(\varSigma \) equals to the ordinary least squares estimates of the coefficients. But at the same time, we consider \(\eta _0 = d + 2 = 4\). This specification makes the prior distribution flat or diffuse to make the prior belief weak. Such a prior specification ensures less subjectivity or more objectivity.
In order to initiate the Gibbs sampling algorithm, the initial parameter values are obtained from the OLS estimates of the coefficients. The initial missing values are obtained by simple average of the studentspecific observed marks. We used the first 1000 scans of the Gibbs sampler as burnin and threw the values of the realizations out. This eliminates the effect of the initial values we have chosen to start the Gibbs sampler. We then ran the Gibbs sampler for another 10,000 scans and saved every 10th scan to produce a sequence of 1000 values for each parameter. We checked if the Markov chain for each parameter obtained stationarity or not. The autocorrelation values and plots for the sequence of saved scans for each parameter are also examined. After confirming convergence in terms of stationarity and minimal autocorrelation, we proceeded next to perform Bayesian posterior inference.
Figure 3 shows the MCMC traceplots against thinned scans of the chains for \(\theta _0\) (topleft) and \(\theta _1\) (topright). These plots show the generated values of \(\theta _0\) and \(\theta _1\) from their respective posterior distributions saved at every 10th scan of the chain after throwing out the burnedin scans. It is visible that the chain has achieved stationarity. The bottom panels of Fig. 3 show the autocorrelation functions (ACFs) for \(\theta _0\) (bottomleft) and \(\theta _1\) (bottomright). It is also clear that the generated thinned scans of the chains are nearly uncorrelated. In fact, their effective sample sizes are straight 1000 in each. Similarly, we have confirmed stationarity for the parameters for every other course selected in our study.
Results
We summarized the number of students, the percentage of disengaged students, and the percentage of missing marks within each course in Table 1. The percentage of disengaged students is defined as the percentage of students within a course who did not participate in any of the assessment/evaluation components after classes and assessments/evaluations transitioned to online. The percentage of disengaged students varies from \(0\%\) to \(5.56\%\) across different courses. Even though these numbers are relatively small, the implications are significant. We investigated the reasons as to why students disengaged after March 15, 2020. According to students’ records/responses, at least 43% of them were in need of special supports and accommodations due to mental illness caused by concussion, severe disability in coping with stress, and being slow in processing information. Most of such students had difficulties following the course content in an online delivery mode without facetoface interactions with the instructors. The other 57% of disengaged students were those who struggled the most with the course contents. Since there were no marks available for the evaluation components for the disengaged students after March 15, 2020, we excluded them from further analysis.
The percentage of missing marks is defined as the percentage of missing evaluation components relative to all evaluation components for the remaining students in a course. These numbers vary from 3.73 to \(14.07\%\). As shown in Table 2, the percentage of missing marks are higher for the courses in which more direct handson supports were required in the form of labs, coding, programming, and seminars than the courses evaluated mainly using assignments, quizzes, tests, and exams.
Coursespecific analysis
Figure 4 and Table 4 show the comparisons of overall performances (topleft panel) and three studentspecific performances (topright, bottomleft and bottomright panels) before and after March 15 for the course MATH 1070: Mathematics for Business and Economics. The overall marks for all the students in this course went up from a median of \(66.683\%\) (\(95\%\) credible interval of 61.203–71.763%) to \(74.991\%\) (with \(95\%\) credible interval of 69.353–80.647%). Even though there is a small overlap in the \(95\%\) credible intervals for the overall marks, studentspecific marks for some students increased by a large margin. The first case (Student 5) made a significant shift of marks from a failing letter grade to a passing letter grade of C (TRU grading scales are summarized in Table 3). This student was most likely failing as his/her average marks had a \(95\%\) credible interval ranging from \(47.384\%\) to \(50.470\%\). The second case (Student 14) made an even larger shift from a failing letter grade to a strong letter grade of \(A\). The third case (Student 27) also showed a large shift from a barely passing letter grade of D to a passing letter grade of A. None of the distributions of marks before and after March 15 had any overlaps, making the shifts highly statistically significant.
Figures 5, 6, 7 and Tables 5, 6, 7 show the comparisons of overall performance and studentspecific cases in each of the courses MATH 1240: Calculus 2, MATH 1250: Calculus for Biological Sciences 2, and MATH 1640: Technical Mathematics 1. After the transition to online delivery, the overall medians in these courses were increased by about \(20\%\), \(8\%\) and \(7\%\), respectively. The overall increase of marks was statistically significant for the course MATH 1240. On the other hand, the increase of studentspecific marks within the course was not uniform. For example, the performance of Student 5 in MATH 1240 improved by only \(1.67\%\), while Students 6 and 29 had an increase of marks by \(33.66\%\) and \(29.18\%\), respectively. The last two students shifted their marks from a failing grade to passing letter grades of \(A^\) and B, respectively. A more or less similar trend is observed for Students 6 and 24 in MATH 1250, with a small variability in their grades. In other words, a few weaker students managed to improve their marks consistently to a significantly higher range of grades after March 15. However, some goodstanding students, such as Student 26 in MATH 1250 and Student 5 in MATH 1640, experienced a decline in their marks. Note that the decline of marks for these good students were statistically insignificant. While the decline of marks for Student 5 in MATH 1640 was practically insignificant, the decline for Student 26 in MATH 1250 was considered practically significant as the change of the letter grade was from \(A^+\) to A.
Figures 8, 9 and Tables 8, 9 show the comparisons of overall marks and studentspecific cases for the secondyear courses MATH 2200: Introduction to Analysis, and MATH 2240: Differential Equations 1, respectively. MATH 2200 is a proof course, required for Math Majors and a gateway to heavyproof math courses. After the transition to online delivery, the overall median marks were increased by about \(10\%\). For most of the students the improvement was not notable. However, for some students, such as Students 5 and 9, the improvements in marks (\(41.96\%\) and \(41.28\%\), respectively) were practically and statistically significant. Both of the students’ marks moved from a failing letter grade to passing letter grades of \(B^+\) and \(C^\), respectively. In MATH 2240, the overall marks increased by about \(11\%\) with almost zero overlap before and after COVID19. A similar trend of growth in the marks, especially for the struggling students, has been observed in this course. For instance, Students 14 and 21 have shown a significant jump in their marks after March 15. On the other hand, the \(A^+\) level Student 29 experienced a very minimum change in his/her grade. This change is insignificant both practically and statistically.
Figure 10 and Table 10 display the comparisons of marks before and after March 15 for the course STAT 2000: Probability and Statistics. The overall marks went up slightly from a median of 65.653–67.675%. As there is an overlap in the two distributions, the increase is not statistically significant. Regarding the studentspecific cases, Students 5 and 22 who were barley passing the course had an increase by about \(15\%\) and \(16\%\), respectively. None of the two distributions overlap each other before and after March 15, hence, the shifts are highly statistically significant. On the other hand, a few goodstanding students experienced a decline in their marks from higher percentages to lower percentages. For example, Student 12 was in the \(A^+\) grade range before March 15, while his/her marks went down significantly by around \(11\%\) to the letter grade of \(B+\) after March 15.
Figures 11, 12 and Tables 11, 12 compare the overall performance and studentspecific cases for the courses ARET 1400: Civil Technology 1, and ARET 2600: Statics and Strength of Materials. The overall students’ performances in these two courses declined after the transition to online delivery. In ARET 1400, the median marks decreased significantly by \(12.14\%\), whereas in ARET 2600, the median marks decreased by \(4.93\%\). The decrease in ARET 2600 is not statistically significant. The percentage of decrease or increase of marks varies from one student to another. For example, the decrease in marks for Student 29 in ARET 1400 and Student 15 in ARET 2600 were \(30.75\%\) and \(18.74\%\), respectively. On the other hand, Student 6 in ARET 1400 and Student 4 in ARET 2600 did not experience a large drop in their marks. For Student 4 in ARET 2600, the decrease is insignificant both statistically and practically. There are a few exceptions to this trend as well. For example, Student 12 in ARET 1400 and Student 12 in ARET 2600 did experience some increase in their marks after transitioning to online delivery mode. Again, the increase of \(2.36\%\) for Student 12 in ARET 1400 was insignificant both statistically and practically.
Figures 13, 14 and Tables 13, 14 show overall performances and studentspecific cases for the courses COMP 2680: Web Development, and COMP 4980: Bioinformatics. Students’ performances in these Computing Science courses were negatively affected by the transition to online delivery mode. The median of overall marks went down by \(3\%\) for COMP 2680 and by 19% for COMP 4980. The decrease is statistically insignificant for COMP 2680, while significant for COMP 4980. The marks decreased for a large number of students after March 15. For example, the performance of Students 18 and 22 in COMP 2680 decreased by \(14.86\%\) and \(33.82\%\) and the performance of Students 8 and 11 in COMP 4980 decreased by \(1.95\%\) and \(40.56\%\), respectively. On the other hand, some goodstanding students were able to maintain their good performance after March 15, such as Student 16 in COMP 2680 and Student 19 in COMP 4980. However, these increases were insignificant both practically and statistically.
Discussion of results
Among the 11 courses in this study, both increasing and decreasing trends in students’ marks were observed. Specifically, a general increase in the marks is shown in theorybased courses requiring lowerlevel cognitive skills according to Bloom’s Taxonomy of Knowledge, whereas a general decrease in the marks are shown in the courses requiring either interactive handson support or higherlevel cognitive skills.
Rising trend in courses requiring lowerlevel cognitive skills
Universitylevel math courses are normally delivered in a traditional lecture format with the instructor teaching core concepts and theories accompanied by related examples and applications varying from direct applications to more conceptual and intricate ones. Student assessment is then composed of several inclass quizzes, written homeworks, one or two midterms and a final exam with generally more weight on summative assessments than formative ones. Knowing that a standard first or secondyear math course is often taken by a large group of students enrolled in various university programs with a wide range of backgrounds in math, assessments in these courses are mainly focused on questions with a mediumdifficulty level in order to reasonably evaluate students’ learning. In other words, according to Bloom’s Taxonomy of Knowledge, inperson math exams are normally testing low to mediumlevel skills and abilities with limited allocation of questions to higher level skills such as analysis and synthesis. However, transitioning to online and openbook exams did change all equations.
The unprecedented closure of universities due to COVID19 imposed an unexpected shock to academia in particular to the traditional culture of course delivery and assessment in mathematics. Given the very limited time for instructors to prepare for switching to online modes of delivery, creating online openbook tests and restructuring inperson exams to be suitable for the online format was rather infeasible. Moreover, although some instructors did attempt to design tests relatively different than inperson exams in order to target deeper levels of understanding, they were faced with students’ complaints and resistance. This is completely understandable though on account of the lack of training throughout the semester for such exams. Students in MATH 1250, for example, even in normal circumstances, are mostly categorized in the struggling group with levels of math anxiety; hence it is unrealistic to expect them to perform well in an exam format to which they are not used to. Therefore, most MATH and STAT courses maintained a format similar to inperson exams with the online assessments being openbook.
Nevertheless, with the availability of resources in an openbook exam the low and mediumlevel question types that target memory and comprehension skills such as recalling, defining, describing or explaining concepts were no longer truly examining students’ learning as these can be easily found in textbooks, class notes, etc. It was observed that students had a better performance in these question types in the online version of exams compared to similar facetoface exams prior to university closure. For instance, students in MATH 1240 are normally in the category of moderately strong Science students and their performance improved after the transition to online delivery. The improvement took place mostly because the assessment components and their structures did not change, except the exams being openbook, and students could adapt to the new delivery method with relative ease. Similarly, in MATH 2240 which is a secondyear course, the overall performance has grown significantly which can be partly attributed to openbook exams maintaining a similar structure to facetoface ones.
Furthermore, one can consider the role of technology and online resources available to students during an online math exam. Tools were no longer limited to a basic scientific calculator. Advanced online calculators and math programs, along with many online forums were at hand during an openbook exam. For example, at the beginning of the Winter 2020 semester, all students in MATH 2200 were struggling with the course content as writing a rigorous proof is a skill never taught in the firstyear courses. After about a month, many students in this class could improve their proof writing skills and consequently improved their grades to some extent. When the course transitioned to online delivery, the assessment components remained unchanged, most of the students continued their upward trend and their overall performance improved. But this transition might have provided some weaker students the opportunity to seek other resources for answers in noninvigilated tests. This pattern can be seen for students 5 and 9 (Fig. 8). This is also observed in Student 24 in MATH 1250 and Student 21 in MATH 2240, whose large grade improvements turned their grade from failing to the letter grades of \(\mathrm {C}^+\) and B, respectively. The performance of Student 29 in MATH 2240 and Student 17 in MATH 2200, however, did not change significantly after March 15. These students were among the average to highperforming students.
The statistics course (STAT 2000) we investigated in this study tests mediumlevel cognitive skills of students, as per Bloom’s Taxonomy, and falls slightly above the Math courses. This course requires students to understand the methods, organize the information in the data, apply the methodology to the data to gain insightful knowledge, and provide summary and explanations of the findings. Nurturing these skills requires some discussions between the instructor and the students. After the transition to online delivery, the support that students needed were provided to the best of the instructor’s ability, especially to the students who requested support via online meetings and discussions. As the structure of the course is very similar to Math courses with some added applications, the overall performance of students improved slightly by about \(2\%\). This increase of overall performance is not as large as the Math courses which are aligned with lowerlevel cognitive skills. Moreover, when most of the weaker students in this course improved their grades by a large margin, the top students experienced a slight decrease in their marks despite their potential.
Decreasing trend in courses requiring higherlevel cognitive skills and interactive handson support
In ARET courses, application and analysis of learned theories and concepts play a key role in the assessments. While assessments in math courses revolve around defining, calculating or reproducing facts pertaining to a topic, ARET courses demand students’ expertise in applying the knowledge learned to novel applications/situations. Assessments for the ARET courses investigated in this study entail developing problem solving skills with emphasis on analyzing and devising the concepts and principles in applied situations. These skills are categorized as medium to highlevel cognitive skills, as per Bloom’s Taxonomy. Students in these ARET courses experienced a decline in their grades after March 15 mainly because they needed some handson and facetoface support to develop their analyzing and problem solving skills required for the final exam. Providing such handson support was not feasible after March 15 and, as a result, many students in ARET 1400 and ARET 2600 did not perform well in their final exam. A few of these students happened to be among the stronger cohort of students: for example, Student 29 in ARET 1400 and Student 15 in ARET 2600.
Computing Science is known as a ‘learningbydoing’ subject, and most COMP courses require enormous interactive supports in handson programming and laboratories, without which it is hard for students to succeed in these courses. Table 2 shows that the computing science courses at TRU are more geared towards handson practices in lab components compared to MATH and STAT courses. For the two COMP courses investigated in this study, students in COMP 2680 learn web development skills in lectures, then practice and apply these skills in handson laboratories. COMP 4980 is an interdisciplinary course where students learn how to apply computing science skills to analyze, synthesize and interpret the biological data. It requires not only handson laboratories to practice problem solving skills, but also supports from instructor’s domainspecific knowledge to help students connect the biological problems that they try to solve with computational models and interpret their results both biologically and mathematically. In other words, COMP courses investigated in this study test students’ medium to highlevel cognitive skills, as per Bloom’s Taxonomy.
Unsurprisingly, students’ performances in both courses were negatively affected by the rapid switch from facetoface to online delivery mode. The marks dropped after March 15 for most students as shown in Figs. 13 and 14, including students with good standings (e.g., Student 18 in COMP 2680 and Student 8 in COMP 4980) and relatively weak students (e.g., Student 22 in COMP 2680 and Student 11 in COMP 4980). There might be a few reasons that potentially caused the decrease of students’ marks after March 15 in these courses. For instance, most students did better in the first half of the semester, because the first few topics of the course were introductory topics which were easy to pick up, and more difficult topics were introduced in the second half of the semester (after March 15). During the facetoface delivery, students could also get handson help from instructors or TAs in lectures, labs, or Computing Science Help Center, but the handson labs and the Help Center were no longer accessible after March 15. For COMP 4980, there was a group project in the last three weeks of the semester, but students could not get facetoface interaction with each other and did not receive the same level of support from the instructor due to the online delivery mode, therefore, some students struggled in the term project.
Conclusion
We report results from a moderately large scale study from 11 courses, where the effects of COVID19 on students’ performance were compared with empirical rigor. This study shows that a sudden change of delivery mode has an immense impact on students’ marks. After switching to online delivery mode and assessments due to COVID19, students’ marks were increased in theorybased courses that required lowerlevel cognitive skills based on Bloom’s Taxonomy, whereas in courses with handson lab, coding and programming components, or courses that required higherlevel cognitive skills, the marks were decreased. The larger increase (for MATH and STAT courses) or decrease (for COMP and ARET courses) of marks are mainly observed for weaker students as opposed to stronger students. The group of stronger students experienced a smaller decrease of marks, while some very hard working students were able to maintain a good standing of marks towards their credentials. The impact has been much more significant on students with special needs who disengaged from the course after March 15. We also emphasize on the fact that the COVID19 outbreak, lockdown and closure of schools have exposed students to an extraordinary stress level. Students faced the sudden shock of online transition with virtually no education and training on how to take ownership over their submitted work in the online space and be accountable for that.
The authors of this paper observed similar trends in results across Canada which was discussed in many educational workshops and meetings, such as SSC Webinar on Teaching Statistics Online^{Footnote 4} and CMS COVID19 Research and Education Meeting (CCREM).^{Footnote 5} Hence, the results of the study can also be generalized across Canada. This is because most, if not all, of the universities across Canada follow the same educational system and experienced moving towards online teaching and assessment at around the same time.
Our novel contribution is analysing and comparing COVID19 effects on students’ marks. In addition to this novel application, we also designed and developed novel computational models. A Bayesian linear mixed effect model was designed to fully address the comparison of marks. The implementation of Bayesian missing value imputation is novel both in terms of statistics and application.
In this paper, we considered a normal distribution (Eq. (2)) for the response variable of interest. As alternatives, one may wish to use other probability distributions as they fit. For example, in the presence of unusually small or large numbers in studentspecific data, one may wish to use heavytailed noncentral tdistribution. The use of alternative distributions may complicate the computational process for posterior realizations in situations when the posterior distributions are not in closed form. In such a situation, one may need to use Metropolis or MetropolisHastings algorithm instead of Gibbs sampling. On the other hand, the applications of opensource MCMC software, such as JAGS [10], WinBUGS [20], or Stan [6] may appear handy to improve computational issues.
This paper considered STEM courses offered under the faculty of science at TRU. On the other hand, consideration of more courses across multiple faculties in University might be of interest. Such interest may also extend to multiple universities in a country or across the world. However, such augmentation of data may require one to use the multilevel linear mixed effects model [21].
Notes
 1.
STEM is a curriculum based on the idea of educating students in four specific disciplines—Science, Technology, Engineering and Mathematics—in an interdisciplinary and applied approach.
 2.
Bloom’s Taxonomy of Knowledge is a set of three hierarchical models used to classify educational learning objectives into levels of complexity and specificity: https://cft.vanderbilt.edu/guidessubpages/bloomstaxonomy/.
 3.
There is no teaching assistant for marking.
 4.
 5.
References
 1.
Arias, J.J., Swinton, J., Anderson, K.: Online vs. facetoface: a comparison of student outcomes with random assignment. eJ. Bus. Educ. Scholarsh. Teach. 12(2), 23 (2018). https://files.eric.ed.gov/fulltext/EJ1193426.pdf
 2.
Ashby, J., Sadera, W.A., McNary, S.W.: Comparing student success between developmental math courses offered online, blended, and facetoface. J. Interact. Online Learn. 10(3), 10 (2011). https://www.ncolr.org/jiol/issues/pdf/10.3.2.pdf
 3.
Basilaia, G., Kvavadze, D.: Transition to online education in schools during a SARSCoV2 Coronavirus (COVID19) pandemic in Georgia. Pedag. Res. 5, 1–9 (2020). https://doi.org/10.29333/pr/7937
 4.
Bhaskaran, K., Smeeth, L.: What is the difference between missing completely at random and missing at random? Int. J. Epidemiol. 43(4), 1336–1339 (2014). https://doi.org/10.1093/ije/dyu080
 5.
Cao, W., Fang, Z., Hou, G., Han, M., Xu, X., Dong, J., Zheng, J.: The psychological impact of the covid19 epidemic on college students in china. Psychiatry Res. 287(112934), (2020). https://doi.org/10.1016/j.psychres.2020.112934
 6.
Carpenter, B., Gelman, A., Hoffman, M.D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., Riddell, A.: Stan: a probabilistic programming language. J. Stat. Softw. 76(1), 1–32 (2017). https://doi.org/10.18637/jss.v076.i01. https://www.jstatsoft.org/v076/i01
 7.
Celton, M., Malpertuy, A., Lelandais, G., Brevern, A.G.d.: Comparative analysis of missing value imputation methods to improve clustering and interpretation of microarray experiments. BMC Genomics 11(15) (2010). https://doi.org/10.1186/147121641115
 8.
Cronbach, L.J.: Coefficient alpha and the internal structure of tests. Psychometrika 16, 297–334 (1951). https://doi.org/10.1007/BF02310555
 9.
Dennon, A.: Coronavirus impacts on students and online learning (2020). https://www.bestcolleges.com/blog/coronavirusimpactsonstudents/
 10.
Denwood, M.J.: runjags: an R package providing interface utilities, model templates, parallel computing methods and additional distributions for MCMC models in JAGS. J. Stat. Softw. 71(9), 1–25 (2016). https://doi.org/10.18637/jss.v071.i09
 11.
Doreleyers, A., Knighton, T.: COVID19 pandemic: academic impacts on postsecondary students in canada. Statistics Canada (2020). https://www150.statcan.gc.ca/n1/pub/45280001/2020001/article/00015eng.htm
 12.
Geyer, C.J.: Practical Markov chain Monte Carlo. Stat. Sci. 7(4), 473–483 (1992). https://www.jstor.org/stable/2246094
 13.
Jones, S.J., Vena, M.L.: Learning equity between online and onsite mathematics courses. MERLOT J. Online Learn. Teach. 9(1), 1–12 (2013). https://jolt.merlot.org/vol9no1/jones_0313.pdf
 14.
Kleiner, C., Lord, M.: The cheating game: ‘everyone’s doing it,’ from grade school to graduate school. U.S. News World Rep. 127, 54–63 (1999)
 15.
Laird, N., Lange, N., Stram, D.: Maximum likelihood computations with repeated measures: application of the EM algorithm. J. Am. Stat. Assoc. 82(397), 97–105 (1987). https://doi.org/10.2307/2289129. http://www.jstor.org/stable/2289129
 16.
Laird, N.M., Ware, J.H.: Randomeffects models for longitudinal data. Biometrics 38(4), 963–974 (1982). https://doi.org/10.2307/2529876. http://www.jstor.org/stable/2529876
 17.
Lawrence, C., Altschul, S., Boguski, M., Liu, J., Neuwald, A., Wootton, J.: Detecting subtle sequence signals: a Gibbs sampling strategy for multiple alignment. Science 262(5131), 208–214 (1993). https://doi.org/10.1126/science.8211139. https://science.sciencemag.org/content/262/5131/208
 18.
Lin, W., Tsai, C.F.: Missing value imputation: a review and analysis of the literature (2006–2017). Artif. Intell. Rev. 53, 1487–1509 (2019)
 19.
Lindstrom, M.J., Bates, D.M.: Newton–Raphson and EM algorithms for linear mixedeffects models for repeatedmeasures data. J. Am. Stat. Assoc. 83(404), 1014–1022 (1988). https://doi.org/10.2307/2290128. http://www.jstor.org/stable/2290128
 20.
Lunn, D.J., Thomas, A., Best, N., Spiegelhalter, D.: Winbugs—a bayesian modelling framework: concepts, structure, and extensibility. Stat. Comput. 10, 325–337 (2000). https://doi.org/10.1023/A:1008929526011
 21.
Pinheiro, J.C., Chao, E.C.: Efficient Laplacian and adaptive gaussian quadrature algorithms for multilevel generalized linear mixed models. J. Comput. Graph. Stat. 15(1), 58–81 (2006). https://doi.org/10.1198/106186006X96962
 22.
R Core Team: R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna (2020). https://www.Rproject.org
 23.
Rey, J.G.: The effects of online courses for student success in basic skills mathematics classes at california community colleges. Ph.D. thesis, University of Southern California (2010). https://search.proquest.com/openview/8872a71d22481e7919cdb8d71eda5992/1?pqorigsite=gscholar&cbl=18750&diss=y
 24.
Sahu, P.: Closure of universities due to coronavirus disease 2019 (COVID19): impact on education and mental health of students and academic staff. Cureus 12(4), e7541 (2020). https://doi.org/10.7759/cureus.7541. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7198094/
 25.
Schafer, J.L., Yucel, R.M.: Computational strategies for multivariate linear mixedeffects models with missing values. J. Comput. Graph. Stat. 11(2), 437–457 (2002). https://doi.org/10.1198/106186002760180608
 26.
Sintema, E.: Effect of COVID19 on the performance of grade 12 students: implications for STEM education. EURASIA J. Math. Sci. Technol. Educ. 16(7), em1851 (2020). https://doi.org/10.29333/ejmste/7893
 27.
Weems, G.H.: Comparison of beginning algebra taught onsite versus online. J. Dev. Educ. 26(1), 10 (2002). https://www.learntechlib.org/p/95604/
 28.
Williams, J.B., Wong, A.: The efficacy of final examinations: a comparative study of closedbook, invigilated exams and openbook, openweb exams. Br. J. Educ. Technol. 40, 227–236 (2009). https://doi.org/10.1111/j.14678535.2008.00929.x
 29.
Wong, E.T.: Impact of Elearning during SARS outbreak in Hong Kong. The social and cognitive impacts of ecommerce on modern organizations, pp. 185–203 (2004). https://doi.org/10.4018/9781591402497.ch009
 30.
Zhang, Z.: Missing data imputation: focusing on single imputation. Ann. Transl. Med. 4(1) (2016). http://atm.amegroups.com/article/view/8839
 31.
Zhao, Y., Staudenmayer, J., Coull, B.A., Wand, M.P.: General design bayesian generalized linear mixed models. Stat. Sci. 21(1), 35–51 (2006). https://doi.org/10.2307/27645735. http://www.jstor.org/stable/27645735
 32.
Zu, Z.Y., Jiang, M.D., Xu, P.P., Chen, W., Ni, Q.Q., Lu, G.M., Zhang, L.J.: Coronavirus disease 2019 (COVID19): a perspective from china. Radiology 0(0), 200490 (2020). https://doi.org/10.1148/radiol.2020200490
Acknowledgements
We thank Mr. Shi Shi for sharing the course information of COMP 2680, and Dr. Musfiq Rahman for his discussions earlier in this study. We also thank the Editor, and two anonymous reviewers for their insightful comments, which substantially strengthened the arguments presented here.
Author information
Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Tomal, J., Rahmati, S., Boroushaki, S. et al. The Impact of COVID19 on Students’ Marks: A Bayesian Hierarchical Modeling Approach. METRON (2021). https://doi.org/10.1007/s40300021002001
Received:
Accepted:
Published:
Keywords
 COVID19
 Science education
 Hierarchical Linear Modeling
 Missing value estimation