1 Introduction

Working life and everyday practices are increasingly dependent on technology and digitalisation (OECD 2015). To prepare students for changes in society and the demands of the new global economy, one of the key focuses of Finnish basic education reform is to promote new learning environments that are suitable for fostering students’ twenty-first-century skills such as digital literacy and collaboration (The Ministry of Education and Culture 2016). The education reform has highlighted the need to increase the use of digital technologies for learning and assessing such skills (Ahonen and Kankaanranta 2015; Göçen et al. 2020). Digital learning also needs an appropriate pedagogical approach that requires teachers to integrate digital competence into their practice (Aslan and Zhu 2016; Tammaro and D'Alessio 2016). This involves not only teachers’ technical skills in using digital technologies, but also the ability to use and apply digital technologies properly in their own teaching. However, several studies revealed that there is a lack of skills in using digital technologies among teachers, and a need for pedagogical support in their work with ICT (Mishra and Mehta 2017; Sumardi et al. 2020).

This study is part of a larger project, OpenDigi, funded by the Finnish Ministry of Education and Culture. The main goal of the OpenDigi project is to build teachers’ communities, including not only in-service teachers but also teacher educators and pre-service teachers, in order to enhance digital pedagogical skills and the use of digital assessment tools (for further information, see OpenDigi n.d.). In this study, a digital pedagogy training aimed at promoting the use of digital assessment tools was organised for elementary and lower secondary school in-service teachers. This present study aims to investigate the impact of digital pedagogy training on in-service teachers’ attitudes towards digital technologies.

1.1 Digital competence

To prepare schools for the demands of the twenty-first century, teachers have been challenged to expand their use of digital technologies in their teaching (Albion et al. 2015). Researchers have shown that only few teachers were satisfied with their knowledge and skills in working with digital technologies in the classroom, and they were not prepared to integrate the promotion of mandatory digital competence into their practice (Brigas et al. 2016; Pittman and Gaines 2015). Even when some teachers integrated digital competence into their work, researchers found that their integration of digital technologies was rather limited to a basic level and for demonstrative purposes (Brigas et al. 2016; Aslan and Zhu 2016).

Digital competence is a concept that is widely used and can be defined in different ways by researchers and policy makers (see Spante et al. 2018 for a review). In this study, teachers’ digital competence is their ability to use ICT with proper pedagogical understanding and awareness of its impact on the students’ learning process (Krumsvik 2014). According to Ilomäki et al. (2016), digital competence consists of four components: (1) the technical skills needed to use digital technologies; (2) the abilities to use and apply digital technologies in different working situations; (3) the ability to evaluate digital technologies critically for their ethical issues, limitations, and challenges; and (4) the motivation to participate in and commit to the digital culture. On this definition, digital competence goes beyond the technical skills needed to use digital technologies; teachers must also be able to justify their decisions as to which digital technologies are to be implemented in various working situations and how (Krumsvik 2014).

1.2 Digital technologies for learning and assessment

In the past few decades, there has been an increasing amount of literature concerning the use of digital technologies in education (OECD 2015; Sung et al. 2016). Digital technologies serve different educational purposes, and offer various advantages compared with traditional learning tools (Faber et al. 2017). Experimental evidence has shown that digital technologies not only can be used as effective tools to promote learning achievement and collaborative learning (Laakso et al. 2018; Kurvinen et al. 2018; Kurvinen et al. 2020), but also can be useful assessment tools for tracking students’ learning progress and the improvement of teachers’ own teaching (Faber et al. 2017; Kurvinen et al. 2016; Laakso et al. 2018).

There are several digital technologies such as digital learning environments available for teachers to use in their teaching (Laakso et al. 2018; Umek et al. 2017). Some of these environments, such as Moodle and Blackboard, are more dominant and have been widely used in different major areas (Alokluk 2018; Umek et al. 2017). While the primary objective of any digital learning environment is to support students’ learning, only a few of them include automatic assessment features that can be advantageous for both teachers and students. ViLLE, for example, is a web-based learning environment with additional unique functions that support collaborative learning and enable automatically assessed exercises with immediate feedback (see Laakso et al. 2018 for an overview of ViLLE). Unlike the traditional learning environment, the design of the ViLLE exercise-based learning environment was predominantly driven by constructivism (Bruner 1979) and constructionism (Papert 1980), which expects exercises to facilitate active learning (Laakso et al. 2018). To enhance collaborative learning, ViLLE enables a group mode function in which two or more students can do interactive exercises in groups. Moreover, the automatic digital assessment features in ViLLE allow students to reflect on their learning performance and can be a beneficial tool for teachers to track students’ learning progress for a process-oriented assessment. Recent studies by Laakso et al. (2018) and Kurvinen et al. (2016, 2018, 2020) showed that ViLLE is an efficient digital learning environment that can enhance students’ motivation and their performance in various heterogeneous educational setups.

1.3 Factors influencing teachers to use digital technologies

Even though digital technologies can enhance students’ learning and motivation (Faber et al. 2017; Laakso et al. 2018), previous literature identified several challenges of integrating digital technologies into classroom practice (Aslan and Zhu 2016; Durff and Carter 2019; Ertmer 1999). Ertmer (1999) explained these challenges in terms of barriers to change, which are the extrinsic and intrinsic factors that hinder teachers’ efforts to implement innovations into their practice. Some teachers may be constrained by the external barriers, such as limited access to resources, training, and support, while others may struggle to overcome intrinsic barriers linked to teachers’ confidence, beliefs, attitudes, and the perceived value of technology (Ertmer et al. 2012). This explanation is in line with a model of technology integration proposed by Christensen and Knezek (2008). In this model, Christensen and Knezek (2008) distinguished three major elements for a high level of technology integration, including teachers’ will to use digital technologies in their classroom (teachers’ attitudes), their skills in using digital technologies (digital competence), and adequate access to digital technology as a tool (access to technology). The results indicated that this model could explain more than 90% of the variance in the level of classroom technology integration (Morales 2006).

Attitudes and beliefs are two concepts that are closely related and sometimes used interchangeably. According to Op ‘t Eynde, P., De Corte, E., and Verschaffel, L. et al. (2002), beliefs are subjective conceptions that one regards as true, while attitudes are constructed based on clusters of beliefs and, therefore, can be regarded as the sum of beliefs around a particular object or situation (Instefjord and Munthe 2017). For decades, various researchers have acknowledged a powerful influence of attitudes and beliefs on teachers’ behaviours (Instefjord and Munthe 2017; Mumtaz 2000; Pepin and Roesken-Winter 2015; Teo et al. 2016). A review of the literature (Ertmer and Ottenbreit-Leftwich 2010; Mumtaz 2000) shows that attitudes towards technology and confidence in ICT use are significant predictors of teachers’ use of technology. The study by Cox et al. (1999), for instance, showed that teachers who are already regular users of ICT have confidence in ICT use, perceive ICT to be useful for their personal work and teaching, and plan to extend their use further in the future. In a survey of 764 teachers, Wozney et al. (2006) revealed that there was a strong connection between teachers’ use of technology and their confidence. These results are in line with the European Union study (2013) showing that teachers’ confidence levels in their operational ICT use can have a potential influence on how often teachers use ICT-based activities in their teaching. Professional development programmes or teacher training have been found to play an essential role in promoting teachers’ confidence (see Ertmer and Ottenbreit-Leftwich 2010 for a review). Teachers who received ICT training tend to be less anxious and more confident in their ICT use, and usually value ICT more highly (European Union 2013).

Teachers’ attitudes towards technology can also be regarded in terms of their perception of the usefulness of technology (Instefjord and Munthe 2017; Teo et al. 2016). Teo et al. (2016) defined the perceived usefulness of technology as the degree to which teachers believe that using technology could enhance their performance in teaching. In this study, we also included students’ learning perspectives in the definition. Thus, here the perceived usefulness refers to teachers’ perception of the extent to which using technology would improve teaching and learning. Previous studies revealed that teachers who are unconvinced about the benefits of technology on instructional productivity are less likely to use technology in their practice (Kim et al. 2013).

Apart from these important factors, Kim et al. (2013) underlined the relationship between teachers’ beliefs about teaching and learning and their technology integration practices. There is an assumption that integration of ICT into teaching practices require an approach to teaching and learning based on constructivism and collaboration (Ertmer 1999). Some researchers have suggested that the use of ICT will eventually change teaching practices towards more collaborative learning (Dillenbourg 1999; Scardamalia and Bereiter 1994). However, several studies have presented somewhat contradictory results. Valtonen (2011), for example, indicated that teachers’ conceptions of teaching and learning do not necessarily fully correlate collaborative learning with ICT. The results showed that many teachers adopted a teacher-centred approach based on transmitting knowledge, and used collaborative learning activities as a minor additional element to the main learning activities. These results are in line with the results of Kim et al. (2013) who reported that teachers’ ICT practices could also operate, for example, within a traditional teacher-centred approach where developing ICT skills are in focus.

1.4 Aims

This study aims to investigate the impact of digital pedagogy training on in-service teachers’ attitudes towards digital technologies. In particular, we attempted to answer these two research questions:

  1. 1.

    Does the digital pedagogy training have an impact on teachers’ attitudes towards digital technologies?

  2. 2.

    Is the impact of the digital pedagogy training on teachers’ attitudes towards digital technologies dependent on their confidence in ICT use?

2 Method

2.1 Participants and context

The participants of the study were 98 in-service teachers working in schools located in southern Finland. A majority of the participants were elementary school teachers (76.5%), followed by lower secondary school teachers (19.4%), and those who taught some specific subjects at both levels (4.1%). These 98 in-service teachers volunteered to participate in a digital pedagogy training, the aim of which was to get familiar with the use of digital assessment tools in the ViLLE learning platform. The training was organised for one seminar lasting around three hours in spring 2018. The training facilitator was a ViLLE specialist who worked in OpenDigi as a coordinator and digital pedagogy expert teacher. In the training session, the in-service teachers were encouraged to explore different functions of ViLLE, such as automatically assessed exercises with immediate feedback and learning, and analytical tools for process-oriented assessment. In addition, several issues regarding how to use these assessment tools effectively to promote teaching and learning were discussed. An additional goal of the training was to recruit teachers for the OpenDigi project.

2.2 Measures

This study used a single group pre-test post-test design. Teachers’ attitudes towards digital technologies were measured with a questionnaire separated into three different parts including (1) beliefs about oneself as a user of ICT, (2) beliefs about ICT as a teaching and learning tool, and (3) beliefs about the use of digital assessment tools. The first part contained 12 items concerning beliefs about oneself as a user of ICT. Factor analysis was conducted on these items. Five items were excluded from the analyses because they loaded onto different factors in the pre- and post-tests. Two factors were constructed on the beliefs about oneself as a user of ICT: confidence in ICT use and the need for ICT support. The factors explained 70.32% and 68.08% of the variance for the pre- and post-test, respectively. The second part consisted of 7 items to measure teacher beliefs about ICT as a teaching and learning tool. Two items were omitted from the analyses since they loaded onto different factors in the pre- and post-tests. Two factors were constructed, including the perceived usefulness of digital technologies and ICT in supporting collaborative skills. The factors explained 70.37% and 77.48% of the variance for the pre- and post-test, respectively.

The last part contained 7 items to measure teachers’ beliefs about the use of digital assessment tools. Factor analysis was conducted. Four items were excluded from the analyses because they loaded onto different factors in the pre- and post-tests. One factor was constructed on the beliefs related to process-oriented assessment and assessment tools. The factors explained 61.04% and 62.73% of the variance for the pre- and post-test, respectively.

2.3 Procedure

All participants were asked to complete a web-based self-report questionnaire concerning teachers’ attitudes towards digital technologies. The pre-test was administered to the in-service teachers at the beginning of the digital pedagogy training (in spring of 2018) organised for these teachers to get them familiar with the use of digital assessment tools in the ViLLE learning platform. The post-test was conducted later in autumn of 2019 to measure the in-service teachers’ attitudes towards digital technologies after the teachers had been working with their peers in their own schools. There were 22 in-service teachers (around 22%) who completed both pre- and post-test. Around two thirds of the respondents (N = 16) were elementary school teachers, followed by lower secondary school teachers (N = 5), and one teacher who taught in some specific subjects at both levels (N = 1).

2.4 Analysis

Table 1 presents five factors of attitudes towards digital technologies. Cronbach’s alphas of all scales in pre- and post-test are between .62 to .82, which meet the threshold criteria commonly adopted in social science (Cronbach alpha > .60), indicating reasonable reliability (Hair et al. 2006). To investigate the impact of the digital pedagogy training, analyses used in the present study are separated into two parts. The first part examined the impact of the digital pedagogy training on teachers’ attitudes towards digital technologies. The second part used k-means clustering to categorise teachers into groups based on their confidence in ICT use. In addition, the paired-samples T-test was carried out to determine whether the impact of the digital pedagogy training on teachers’ attitudes towards digital technologies depending on their confidence in ICT use.

Table 1 Five factors contributed to teachers’ attitudes towards digital technologies
Full size table

3 Results

3.1 Associations between different factors contributed to attitudes towards digital technologies

To explore overall relationships between different variables that explain teachers’ attitudes towards digital technologies, Pearson correlations were calculated based on the pre-test data (included only those teachers who completed both pre- and post-test; N = 22), and the correlation matrix is shown in Table 2. The results reveals associations between the need for ICT support and the other two factors including teachers’ confidence in ICT use (r[22] = 0.49, p = 0.02) and their perceived usefulness of technology (r[22] = 0.54, p = 0.01). However, ICT in supporting collaborative skills and process-oriented assessment and assessment tools have no association with any of these variables.

Table 2 Pearson’s correlation matrix between the main variables
Full size table

3.2 The impact of the digital pedagogy training

Next, the paired-samples T-test was conducted to investigate the impact of the digital pedagogy training on the five main variables regarding teachers’ attitudes towards digital technologies. The results indicate that there was significant difference between the need for ICT support pre- (M = 4.11, SD = 0.52) and post-test (M = 3.71, SD = 0.49); t(21) = 3.78, p = 0.001. The teachers had a lower need for ICT support after receiving the digital pedagogy training. In addition, there was no significant difference between teachers’ confidence in ICT use, beliefs about ICT in supporting collaborative skills, and the use of digital assessment tools pre- and post-test (see Table 3). The results show that teachers’ perception of the usefulness of technology was lower when compared pre- (M = 3.55, SD = 0.47) and post-test (M = 3.33, SD = 0.51); t(21) = 2.45, p = 0.023.

Table 3 Descriptive information concerning the main variables pre- and post-test by initial confidence level
Full size table

3.3 The role of teachers’ confidence in ICT use on the impact of digital pedagogy training

To investigate the role of teachers’ confidence in ICT use on the impact of digital pedagogy training, first teachers were categorised based on their initial confidence in ICT use. K-means clustering was conducted. As a result, teachers were classified into two groups: low (LC) (N = 9) and high initial confidence (HC) in ICT use (N = 13). Descriptive information concerning five variables of teachers’ attitudes towards technologies pre- and post-test is presented in Table 3. Then the independent samples test was conducted, and the results show a clear distinction between the two groups. The teachers in the LC group had a lower level of confidence in ICT use than those in the HC group both pre- (t(20) = 6.89, p < 0.001) and post-test (t(20) = 6.89, p < 0.001). Also there was a significant difference between groups in the perceived usefulness of technology. The teachers in the LC group had a lower level of perceived usefulness than HC-group teachers both pre- (t(20) = 2.29, p = 0.033) and post-test (t(20) = 2.13, p = 0.046). However, there was no significant difference between the two groups in terms of the need for ICT support, beliefs about ICT in supporting collaborative skills, and process-oriented assessment and assessment tools both pre- and post-test.

Finally, the paired-samples T-test was performed to determine whether the impact of the digital pedagogy training depended on their confidence in ICT use. The results show that teachers in the LC group (M = 2.69, SD = 0.43) had a higher level of confidence in ICT use post-test (M = 3.14, SD = 0.49); t(8) = 3.11, p = 0.014, suggesting a positive impact of the digital pedagogy training on the LC group teachers’ confidence level. In contrast, the confidence level in ICT use for the teachers in the HC group showed no significant changes between pre- (M = 3.94, SD = 0.41) and post-test (M = 3.98, SD = 0.67); t(12) = 0.27, p = 0.790. In addition, the need for ICT support of the teachers in the HC group (M = 4.26, SD = 0.41) was significant lower post-test (M = 3.77, SD = 0.48); t(12) = 4.16, p = 0.001. For teachers in the LC group, there was no significant difference in the need for ICT support between pre- (M = 3.89, SD = 0.60) and post-test (M = 3.63, SD = 0.51). Further investigation indicated that there were no significant differences between pre- and post-test regarding the perceived usefulness of digital technology, beliefs about ICT in supporting collaborative skills, and process-oriented assessment and assessment tools in both the LC and HC groups.

4 General discussion

Teachers have faced several challenges in seeking to expand their use of digital technologies in their teaching in order to meet the demands of the twenty-first century (Albion et al. 2015). Previous studies have shown that teachers did not have adequate skills in using digital technologies, and were not prepared to integrate the promotion of mandatory digital competence into their practice (Aslan and Zhu 2016; Brigas et al. 2016; Pittman and Gaines 2015). These studies suggested that there is a need for pedagogical support in their work with ICT (Mishra and Mehta 2017; Sumardi et al. 2020). In this present study, a digital pedagogy training was organised for in-service teachers to promote the use of digital assessment tools with the ViLLE learning platform. The study investigated the association between different factors contributing to teachers’ attitudes towards digital technologies and the effectiveness of the training programme on in-services’ attitudes towards digital technologies.

Overall, the results showed that there was a relationship between the need for ICT support and the other two factors, including teachers’ confidence in ICT use and their perception of the usefulness of technology. However, the results were slightly controversial, and do not accord with all the observations from the previous literature. For instance, earlier literature found that teachers who often use ICT show confidence in ICT use and perceive ICT to be useful, and plan to expand their use of ICT in the future (Cox et al. 1999). These results suggested that there might be a connection between teachers’ confidence in ICT use and the perceived usefulness of technology. This present study, however, showed that there was no significant association between confidence in ICT use and the perceived usefulness of technology. One explanation for this could be that the two factors are not necessarily directly related. For example, although some teachers are not regular users of ICT and have low confidence in ICT use, they could still perceive ICT to be useful and want to learn how to use ICT. Moreover, the present study showed that there was also no correlation found between process-oriented assessment and assessment tools as well as ICT in supporting collaborative skills with all the other factors. Compared to the other factors, confidence in ICT use, the need for ICT support and perceived usefulness represent basic attitudes. In contrast, the use of ICT in supporting collaborative skills is still a relatively new approach, as is process-oriented assessment and especially the use of digital tools to support the process.

Another aim of this study was to investigate the impact of the training programme on in-service teachers’ attitudes towards digital technologies. The results showed that the teachers had a lower need for ICT support after receiving the digital pedagogy training. Moreover, teachers’ perception of the usefulness of technology decreased from pre- to post-test. These results could be explained in terms of a novelty-effect that might influence teachers’ enthusiasm to learn the new ICT tool. The teachers had rather high levels of perceived usefulness of digital technology in the beginning, which later began to diminish over time.

Lastly, we further investigated the role of teachers’ confidence level in ICT use. The results suggested that the impact of the digital pedagogy training on teachers’ attitudes towards digital technologies are various depending on the level of confidence in ICT use. The teachers in the LC group had a higher confidence level after the training, suggesting a positive impact of the digital pedagogy training on the LC group teachers’ confidence in using ICT. These findings are in line with previous studies indicating an essential role of professional development programmes in promoting teachers’ confidence in ICT use (European Union 2013). In contrast, the confidence level in ICT use for the teachers in the HC group showed no significant change after the training. This might be because their level of confidence in ICT use was already high at the beginning. Moreover, the results showed that the need for ICT support of the teachers in the HC group was lower after the training, while there were no significant changes in the need for ICT support of the teachers in the LC group. These results suggested that the training might fulfil teachers’ needs for ICT support, particularly for the teachers in the HC group.

Further investigation showed that there were no significant changes after the training regarding teachers’ perception of the usefulness of digital technology and beliefs about ICT in supporting collaborative skills in both the LC and HC groups. These results are in line with the findings by Valtonen (2011), who explained that although the use of ICT was expected to influence teachers’ beliefs and teaching practices towards more collaborative learning, it was not necessarily the case. Many teachers still employed a teacher-centred approach based on the transmitting of knowledge and used collaborative learning activities as a minor additional element to the main learning activities (Valtonen 2011). For future study, it would be meaningful to also investigate teachers’ beliefs about teaching and learning. In addition, although the automatic digital assessment features in ViLLE could be beneficial tools for teachers to follow students’ learning progress for process-oriented assessment, the results of this study showed that there were no significant changes in teachers’ beliefs about process-oriented assessment and assessment tools after the training programme in both the LC and HC groups. One possible explanation could be that learning analytics and process-oriented assessment are rather new concepts, and it might take a longer time for teachers to change their practices and commit to the digital culture.

5 Conclusions and limitations of the study

To promote teachers’ digital competence in teaching and learning, in this study we organised a digital pedagogy training that focused on the use of digital assessment tools with the ViLLE learning platform in which teachers experienced different kinds of assessment tools that could be beneficial in classroom practice, particularly more advanced tools for process-oriented assessment. Even though the results of this study suggested a positive impact of the programme on teachers’ ICT confidence in the LC group, the limitation of a single group pre-test post-test design in this study must be considered. The participants of the study volunteered to participate in the training, and this may imply that their interest level towards this particular topic might be different from other in-service teachers in general. In addition, unlike regular intervention programmes that typically followed systematic guidelines, in this study we were concerned about ecological validity and tried to maintain natural settings as much as possible. Researchers did not interfere at all with the participants after the training. Because of the nature of the web-based survey and the inability of researchers to contact participants directly (Morton et al. 2012), the response rate for the post-test was rather low (around 22%). Despite this rather small number of respondents, researchers have suggested that it is still an acceptable response rate to gain meaningful information (at least a 21% response rate for 100 participants) (Chapman and Joines 2017).

An additional limitation is that this study used self-developed questionnaire. Even though Cronbach’s alphas of all scales in pre- and post-test present reasonable reliability according to the guidelines of Hair et al. (2006), several important items were discarded during the factor analyses process. Moreover, the in-service teachers’ attitudes towards digital technologies were measured by self-assessments with a questionnaire. Consequently, the results of the study do not necessarily represent their actual competence or practices of using ICT. Information regarding teachers’ utilisation of the ViLLE learning platform would provide insightful information on how the teachers used the learning platform in their teaching. However, due to ethical issues in this study, we could not access such information since it can be linked to students’ personal information and their assessment. For future research, it would be important to prepare for additional students’ and guardians’ permissions. Teachers’ experience with the ViLLE learning platform should also be examined, for example through semi-structured interviews.

Several studies indicated a number of challenges of integration of digital technologies into classroom practice (Aslan and Zhu 2016; Durff and Carter 2019). This study showed that a short-term training could have an impact on teachers’ ICT confidence level, particularly teachers in the LC group. This could be the first crucial step in promoting teachers’ digital pedagogy skills. In order to have a lasting change in the way teachers use technology to support teaching and learning in the classroom, this may require the use of long-term professional development activities in a supportive environment.