Keywords

1 Introduction

The first international conference on information technology in educational management (ITEM) took place in Jerusalem in 1994. Since then, the Working Group 3.7 of the International Federation for Information Processing (IFIP) has held a total of ten international meetings. During the conferences, researchers, academics and professionals have discussed and contributed to a better development and understanding of all the issues associated with the use of information technology (IT) in educational management. Most of these contributions, generally presented as papers, have been published, eventually becoming an important body of knowledge and experience.

ITEM studies have been considerably enriched by the varied profiles and backgrounds of the participants in these working conferences, to the point of acquiring certain characteristics of their own. This situation was described in ‘ITEM: synthesis of experience, research and future perspectives on computer-assisted school information systems’ [1]. ITEM analysis has been approached from very different perspectives that correspond to diverse academic disciplines, which is probably the underlying reason for the area’s conceptual richness. After twenty years of uninterrupted periodic meetings with outstanding academic results, it seems worthwhile to pause to reflect on what has been achieved during this period.

In the year 2006, during the conference that took place in Hamamatsu (Japan), a synthesis study was presented on the scientific production of Working Group 3.7 in its first ten years of existence [2]. The aim of the present paper is to study the literature resulting from the international ITEM conferences organised by the IFIP Working Group 3.7, comparing two periods, the first from 1994 to 2006, and the second from 2008 to 2014. The same structure will be followed as in the initial study in order to facilitate the comparison of the two periods.

2 Analysis Methodology

Since the WG 3.7 began its activity, a total of 11 conferences have been held on all the continents. The first, where the foundations of the group were established, took place in Jerusalem in 1994. All the conferences have been followed by publications comprising the most relevant papers presented. These publications have been edited by prestigious companies specialised in publishing scientific papers. Table 1 shows the city and country where the conference took place, the year, book title, publisher, year of publication, and the number of articles published in each book.

Table 1. International ITEM conferences and resulting publications.
Full size table

The publication corresponding to the 2014 Conference includes a total of 28 contributions, of which 16 are directly related to the ITEM field. This is due to the fact that the 2014 Conference joined two IFIP working groups, the WG 3.4 on Professional and Vocational Education in ICT, and the aforementioned WG 3.7 on Information Technology in Educational Management. Therefore, only the 16 papers corresponding to ITEM have been analysed.

The information resulting from papers’ analysis has been classified into two groups: first, according to the topics addressed in each article; and second, considering the research methodology applied, in order to further group them considering the type of approach employed. This type of organization was chosen to maintain the format adopted in the publication corresponding to the review of the first 10 years of WG 3.7 (1994–2004). In the tables, the data have been added from the study of the next ten years of the work group, until the group’s conference which took place in Potsdam (Germany) in 2014. Tables have been included with the data since the first publication to facilitate comparison of the two periods. In order to classify the articles under a specific topic and research methodology, the titles and abstracts of all 96 papers published during the 2006–2014 period were studied. In those cases where the classification was unclear the text was revised. In case the paper could be assigned to more than one topic or the research methodology was unclear, the two researchers that carried out the classification process confronted their opinions and jointly agreed the assignment to a specific category.

3 Data Analysis

Tables 2 and 3 show a set of general topics discussed at the international conferences of IFIP Working Group 3.7. Table 2 covers the 1994–2004 period, while Table 3 covers the 2006–2014 period. Both tables show the number of papers on each topic and the percentage they represent of the total number of papers per publication, as well as the accumulated total. The last column in Table 3 summarizes the total numbers for the whole period (twenty years).

Table 2. Papers classified by research subject (first period 1994–2004).
Full size table
Table 3. Papers classified by research subject (second period 2006–2014).
Full size table

The topics comprise rather large classification areas in an attempt to obtain significant results. Otherwise, a more detailed classification might have made the results excessively fragmented. For example, Grover et al. [14] suggests 20 IT related topics applicable to any study area, and authors such as Claver et al. [15] increase this number to 30. If it was considered each different type of educational management application as a research topic, this would probably result in an excessively large number of themes. In this case, it was decided to group all these applications in one single category, making no distinctions between the specific applications. This approach could offer a clearer vision of the attention given to this topic at the different conferences, which would be more difficult to perceive if the analysis was more fragmented. A total of 9 topics were identified, although some have an irregular appearance over time.

An initial comparison of Tables 2 and 3 shows a difference in the number of papers published between the two periods. This is due to the fact that in the first period 6 conferences were held, compared to 5 in the second period, yielding 117 and 96 publications, respectively. The last column in Table 2 shows that the most popular topics during the first ten years were Assimilation and integration of IT into educational management (23%), followed by Assessment of IT support for educational management (17.1%), IT applications in educational management (17.1%), and National, regional and local experience in the use of IT for educational management (15.4%). Together, these topics represent 76.2% of all the papers published in that period. Interestingly, the most recurrent topic during the first 10 years of publications was Assessment of IT support for educational management, which has been repeatedly addressed because of the importance of analysing the results obtained after implementing new strategies, policies, techniques or tools. However, this topic has progressively lost importance in the second period, judging by the decreasing number of papers published about it.

In the 20 years of publications about ITEM, Assimilation and integration of IT into educational management is the topic that has received most attention (24.9%). In both periods, this topic has led the number of publications.

The tables show that of the three topics with the greatest number of publications in the first ten years of WG 3.7 publications, two of them continue to figure among the three most frequent topics in the second period. These topics are: Assimilation and integration of IT into educational management, and IT applications in educational management. The topic Assessment of IT support for educational management was no longer among the most repeated topics in the second period, ceding its position to the topic of IT applications for teaching. Although the latter is not directly related to the scope of the WG 3.7, this theme gradually gained importance in the second period of analysis, doubling its former percentage and occupying the third position in the number of publications in the second period (16.7%). However, it is proposed that ITEM practitioners and researchers consider the reasons for this increase, given that this topic corresponds rather to other fields of study, such as IT use for educational purposes.

In both periods into which the analysis was divided, the following three topics produced fewer publications: ITEM state of the art, the discipline’s present situation and trends; Mathematical tools employed to create models for educational management; and Teacher and manager training in the use of IT for educational management.

4 Research Methods

Regarding the research methodology, the papers that were reviewed can be divided into theoretical studies and empirical studies. The theoretical studies were grouped into conceptual and illustrative categories, and the empirical studies were classified as case studies and field studies. Each of these methods will be described briefly.

Theoretical studies are fundamentally based on ideas, structures and speculations, rather than a systematic observation of reality. Although non-empirical articles may contain some empirical observations or facts, they will be of secondary importance. In other words, the emphasis is on ideas rather than facts. Theoretical studies can have a conceptual and illustrative nature. Conceptual studies describe structures, models or theories, and they offer explanations and reasons. The illustrative ones, on the other hand, are designed to guide practice and make recommendations for action or establish stages for attending to certain circumstances. The emphasis is on what and how, rather than why.

The essence of the research carried out in empirical studies is to observe the reality being investigated. This is where case studies can be placed. These kinds of studies are becoming more numerous in the field of IS/IT, mostly for the following reasons [16]: (a) the researcher can study IS/IT in its natural environment, learn about the state of the art, and generate theories based on practice; (b) case studies allow researchers to answer questions about how and why and, therefore, understand the nature and complexity of the process that is taking place; and (c) they are appropriate for investigation in areas with few previous studies, and they are often the first stage of empirical research. However, case studies have often been criticised for their lack of scientific rigour, although this is not due to a problem in the method itself, but rather to the fact that often the label ‘case study’ has been given to what is merely a recounting of anecdotes [17].

Field study is another empirical research method that analyses one or several organisations commonly with regard to a set of variables. There is an experimental design, but no experimental control, which means that the researcher collects information about uncontrolled situations. The object of study operates in its usual fashion while the research is conducted. The aim is to relate results to certain explanatory variables. It is similar to the case study in that phenomena are analysed in their natural environment without introducing any variations in it. However, the methods differ in that field study is not interested in the whole phenomenon, but only specific aspects or variables. Moreover, the analysis of information in case studies is mostly qualitative, whereas field studies generally use quantitative methods.

Tables 4 and 5 show a classification of papers considering the research methodology employed. As in Tables 2 and 3, Tables 4 and 5 show the total number of papers per period, as well as the percentage of each according to the methodology used, and the progression of each method over time, expressed for each of the publications considered. The last column in Table 4 shows total values for the first period, while the last two columns in Table 5 summarize the total values for the second period and for the entire range of years.

Table 4. Papers classified by research methodology (first period 1994–2004).
Full size table
Table 5. Papers classified by research methodology (second period 2006–2014).
Full size table

In the 20-year period analysed, it can be observed that the proportion of publications based on theoretical studies is quite similar to the proportion based on empirical studies, being slightly more in the latter group.

An analysis of Table 4 shows that most of the articles published during the first period are empirical, exceeding the theoretical ones by more than ten percentage points. However, for the second period, Table 5 shows that the tendency of the first period was inverted, with the number of publications based on theoretical studies being greater than the number based on empirical studies. Among the theoretical studies, the illustrative type is the most frequent and doubles the conceptual type in each period and for the 20 years under study. By contrast, for publications based on empirical studies, the tendency was inverted, going from a greater percentage of publications based on case studies during the first ten years to a greater role of field studies during the second period. However, in the total calculation, there is still a predominance of empirical research based on case studies.

5 Conclusions

The interest in overcoming problems derived from adapting a new and changing technology such as IT to educational management has set the standards for ITEM research during the 20-year existence of Working Group 3.7. In this period of time, eleven books have been published containing the main conclusions of the work conferences, which allows us to confirm that this area of study has reached maturity.

Nevertheless, ahead of us is the task of completing a compact body of knowledge consisting of theories that can help ITEM to acquire its own identity. This is not an easy task, due to its multidisciplinary nature and its strong dependence on a changing support like IT.