Keywords

1 Introduction

Traditional assessment methods (e.g., written exam, quizzes, and presentations) enable teachers to make judgments about the students learnings, translated in the assignment of grades. These methods of assessment can have a direct impact on the quality of student learning as many of them assume that the focus of exams and assignments reflects the educational goals most valued by an instructor, directing their learning and studying accordingly (McKeachie and Svinicki 2013). Changing this view is somehow difficult since it is rooted in many teaching systems. However, alternatives to the traditional class assessment can try to mitigate some of those problems.

Assessment Centers in Schools (ACiSs) is a pilot research project in the area of school education that aims to investigate the possible improvement of learning outcomes and stress reduction, by modifying the traditional processes of summative and formative assessment of students in middle schools – although applicable to any level of education. These centers, supported in an information technology (IT) system, will be managed by teachers who do not necessarily teach students in class. Students can, individually, request a written evaluation exam on a certain subject module, choosing between summative or formative examination. Furthermore, students will also be scrutinized through a conversation with the teacher who evaluates their knowledge in the great context of the contents, understanding-relationally a wider area of knowledge. It is intended to bring the teacher back in class, to the role of master teacher that is no longer in opposition with the student, lowering the intense interpersonal comparison and stress, commonly linked to traditional assessments in class. It is also believed to contribute to the improvement of dysfunctionalities such as indiscipline, bullying and violence at school, which can benefit the most disadvantaged and create opportunities to reduce early school leaving (Noel et al. 2015).

The objective is, therefore, to have a more rigorous and demanding assessment, to increase the success of learning for a larger number of students, to increase the quality and quantity of the different potentialities of each student, to increase the possibility of such potentialities to flourish asymmetrically adjusting to different students, and at the same time, as already mentioned, reducing the intense interpersonal comparison and stress.

The technical request behind the operation and implementation of a ACiSs are very low as it requires a server and a, possibly devoted, classrooms equipped with some mean of access (e.g., computers) to the exams wordings. In an extreme case, tablets or smart-phones owned by the students can also be used to access the exams.

This paper describes the efforts being made to build such a system. The platform will manage users, subjects, subject chapters and curricular goals (CG), questions, exams, grades, etc. Teachers upload questions to the platform (namely, question wording, subject, chapter, CG involved in its resolutions, resolution time estimation, degree of difficulty, and one or more answers). Then, exams (formative or summative) are generated taking into consideration those attributes (e.g., subject, CG, solving time, degree of difficulty, etc.). Upon solving an exam, students upload their solutions to the system, to be classified by a teacher which will mark the CG successfully attained by the students. The exams can latter be retrieved given an identifier, which allows to associate exams to students. In this context, exams can also be semi-automatically generated by manually adjusting automatically generated exams. Beside being used to maintain the users’ grades and assessment, CG attainment by the students are used to generate exams adequate to the student, implementing the formative assessment.

This paper is divided as follows The next section presents the motivation, context and background of the problem. Section 3 makes a brief description of the ACiSs system. Section 4 outlines the assessment platform and some examples of the required set up. The final section presents a conclusion and future work.

2 Context and Background

2.1 Pedagogy for Well-Being – Comprehensive-Relationship (PWb–CR)

The ACiSs project arises in the context of previous research carried out at the Institute of Education of the University of London and the University of Algarve, within the framework of an education centred on well-being, inspiring a Pedagogy for Well-being (PWb).

The Education for Well-being, departs of current views (White 1990), coupled with those of the classical age, as summarized in (Williams 1981, pp. 20), which are the following: (i) The purpose of life is to live it in well-being, whereas education is the complex process of learning to live the good life and in well-being; (ii) Well-being is a reflective tranquility; (iii) Such reflexive tranquility originates in the autonomy of the person (self-sufficiency); (iv) Autonomy is the greater capacity to be with one’s neighbor, originated in the capacity of the person to be alone with himself and in reflective tranquility; (v) But there are important aspects that are not under the control of the person, being of the domain of luck and affected by the contingent enemies of the reflexive tranquility, provoking disorder in the person. (vi) Such disorder may be, for the purposes of practical guidance in schools, summarized in the complex and interrelated action of the following seven forms of disorder (FDs), here postulated as follows: (vi.1) Intense interpersonal comparison through competition, envy, jealousy, vanity, prestige, relationships of superiority and inferiority, podiums of winners and losers, comparisons of physical, artistic, intellectual, or industrial capacities; (vi.2) Corruption of intention; (vi.3) Dependence on substances, persons, objects, organizations and traditions; (vi.4) Division by nationalities, regions, languages, professions, sexual orientation, “races”, social classes, religions, gender, ethnic tribalism, physical or mental impaired and not impaired, old and new people, human and nonhuman animals, etc.; (vi.5) Fear, prominent in schools, like fear of examinations, fear of showing ignorance in public, fear of student to teacher, fear of teacher to students, fear of colleagues and of the culture of cruelty and humiliation of mobs in the classroom or in the corridors, fear of public speaking, etc.; (vi.6) Self disintegration for lack of basic goods for the body, such as shelter, clothing, food, but above all, caused by affectations of the mind that can be inscribed under the title of neuroses, such as depressions and anxieties; (vi.7) Violence that can take forms of oppression by domination, power, exploitation, greed, hatred, punishment, and humiliation.

That is, the person in the absence of these forms of disorder, can enjoy autonomy with reflexive tranquility, being the educated person. Thus, education can be seen as a complex process of learning to live a life in well-being that promotes an environment in the absence of the forms of disorder and the sensitivity to it, so as to let flourish the autonomous person, enjoying reflexive tranquility.

A pedagogy inspired by an education for well-being is especially sensitive in creating learning environments that are focused on removing or mitigating the presence of forms of disorder, fitting in the double purpose of preparing students for the life of work, and being an active contributor to the social cohesion. With this fundamental background, PWb aims at:  

Forms of Disorder.:

Creation of learning environments in the absence of the seven forms of disorder;

Luck, Interdependence and Forms of Disorder.:

Sensitize students to the prominence of luck in the origin and unfolding of personal life and to the prominence of interdependence between all people, in the conditioning of our lives, and to the occurrence of a myriad of complex situations where forms of disorder occur.

Greater Framework.:

Provide the student with opportunities of a greater framework, helping him/her throughout his/her evolution, to position himself/herself, threefold: before the universe, before society and before his/her neighbor, as a reflection of his positioning with himself;

Outside Order and Inner Order.:

To seek to harmonize the inquiry by the search for the truth of the external order in nature and society, science, with the inquiry of the search for the truth of the inner order of the student, namely, for example, by “mindfulness”, or others spiritual inquiries.

Comprehensive-Relationship (CR) in the Curriculum:

Comprehensive -Relational learning of curricular subjects, with great attention to the power of “concept words”. Permanently, in double movement, goes from the particular to the general framework, and vice versa, being especially careful with key words and symbols. This is in the sense that one does not manipulate only the word mimetically, but rather touches more closely the reality that it is supposed to represent. Such movement must be done even within the topic matter, vertically, and still with the other topics under study, horizontally, comprehensively and relational. This is a permanent effort to broaden the great frameworks of the outer order as currently described by the sciences and the humanities. For example, the study of mathematical equations may, vertically, require a careful explanation of algebra, the origin of the word and history, the explanation of the branches of mathematics and their relation to the sciences, and the enquiring of what is a science. Horizontally it can lead to the key concept of energy in physics and mathematics there used, or to the quantitative study of racism in society.

Soft Skills.:

Provide and encourage opportunities for the acquisition and training of soft transversal skills, such as: creativity and entrepreneurship, verbal communication in public, orientation of meetings, financial literacy, debate of ideas, etc.

 

PWb contrasts with the criticized “exam pedagogy,” too focused on mechanical answers to the most likely exam questions. For example, it is usually not surprising that in a mathematics examination questions are asked about what is Algebra, its history, its framing with other chapters of mathematics, its relation with the great framework in the investigation by the external order, on the part of diverse Sciences. The student solves the restricted problem of equations by solving them, but tends not to enjoy the broader understanding of their relational understanding of the world. Thus PWb-CR assumes at its heart a concern for a more holistic approach, at the levels of the outer and inner order of the person.

2.2 The Discomfort in Schools, Absolute Superiority, and Two Paths of Solution

Schools, in general, are currently the scene of several dysfunctions widely reported in the literature, as very serious: indiscipline, bullying and various types of violence that can exceptionally escalate to bloody shootings. Such disturbances are believed to potentiate problems that are also widely reported, such as early school leaving, school failure, the discrimination of the most disadvantaged, which end up being the most affected.

What is assumed here as the cause of such dysfunctional behaviours is that, unconsciously, by tradition and in the eagerness to help students acquire the indispensable instrumental knowledge for their performance and success in this “knowledge society,” families and schools, adopt intense but subtle practices, permeated by FDs. Thus, they seriously erode students’ emotional robustness and self-confidence, weakening them to success at work and in life for well-being. More directly, student violence, in and against school, is seen here as a response to what they feel to be, unconsciously and emotionally, the violence of the school over them.

As prominent FDs in school, one can point to intense interpersonal comparison and fear. Hume (1992, pp. 594) gravely credits this type of comparison with the removal of sympathy, making us feel pain with the happiness of the other, and feeling happiness in his pain. Focusing on school, da Silva (2000, pp. 235) states that “the student is especially educated to keep his work to himself, to” shine “at the expense of others, if necessary; The machine of notes and podiums of honours, of vigilance in the tests and of the lessons recited, leads, although perhaps not for this purpose, to an almost monstrous development of selfishness; The child is accustomed to not helping anyone.” And further on (da Silva 2000, pp. 270), “comparative praise,” “medals,” and “the notes given much less to know of individual value than to grade the whole class and to make the differences between students very clear.”

Note that da Silva (2000) stresses that the problem is not notes and exams per se. It would be absurd not to make gauges, to encourage students to improve and succeed in blossoming their potential, or to want to level them all underneath. What is called “monstrous” is the use of grades, to install an intense interpersonal comparison that generates certain dominant feelings of superiority in some, and dominant feelings of inferiority in others. These are not feelings of superiority-inferiority, morally and emotionally innocuous. It is normal for any student to have relative superiority, for example as to competence in mathematics, compared to others. Having such an advantage, it can keep the humanity of the contact, helping their peers. So the problem is not the superiority relative to functionalities, in the sciences, arts, sport, etc. The crucial problem is when the complex, because it frustrates any complete description, interaction of school and social devices generate and attribute to some students an “absolute superiority”, making them believe to be better and more deserving human beings than others. And, making others, the majority, feel symmetrical negative feelings, believing themselves less valuable and incompetent, eroding their self-confidence and self-esteem. As Macmurray (2004, pp. 151) points out, “If the inequalities of functional life are not subordinated to the deeper equality of human fellowship, they become absolute and community perishes.”

Conscious of this central problem of the disruptive violence of human fellowship, contained in the absolute superiority-inferiority in which the school community perishes, and where the different dysfunctional violent reactions such as indiscipline and bullying emerge, two solutions present themselves. First, the Finnish solution, which removed exams from pre-university schools, as argued by White (2014) in the book “Who Needs Examinations?”. But, in some social and political traditions, the examinations occupy a very important place to dethrone, so we are taken to the second solution. Second, the implementation of ACiSs that, keeping grades, further increase the number of possible exams and their rigor, reinforcing confidence for the success of learning, but mitigating the intense interpersonal comparison and fear, as described ahead, in the context of PWb-CR.

3 Brief Description of the ACiSs

ACiSs are conceptualized as spaces within the school, having teachers assigned to it exclusively or as a complementary task to teaching, intervening in the assessments of students who do not attend their classes. Using predominantly an IT system to generate tests/exams wordings from questions in a database, with generosity of time to reduce fear/stress, students can make written exams in a particular subject modules, at their request. These written exams can have the form of summative or formative evaluation, being the latter generated to improve the students competences according with the subject’s CG. However, in addition to this written assessment, the student should have an oral evaluation (CR conversation) with the teacher, to ensure that he/she masters the great concepts of knowledge framing, avoiding the strong incidence of a “predominant pedagogy for exams”, in which some questions/problems can be solved, in writing, without really knowing the main concepts.

The minimal initial requirements is a room, equipped with computers. Supported by a teacher, the student can individually request the assessment in the ACiSs, for a comfortable number of times, so as to remove as much stress as possible from “single occasion exams.” Among other benefits, the following are listed: (1) Mitigate the fears linked to the stress of evaluations, allowing the students to access the center a comfortable number of times; (2) To free the teacher, who teaches in the classroom, of the burden of summative evaluations and to facilitate their establishment as the master-friend who helps the disciple to overcome the evaluation in the center. Thus, ceasing their state of opposition and becoming in communion. (3) Freeing the learning system from a strong “predominant examination pedagogy” and inaugurating the predominance of a PWb-CR; (4) Facilitate student progression by evaluating shorter modules of the syllabus; (5) Mitigate interpersonal comparison by increasing exam opportunities, further trivializing the grades and creating the student’s private area, where all data is stored and access reserved. The grades are no longer published or displayed in public places; (6) Mitigate the intense interpersonal comparison, since the evaluations are not done at the same time in the class where results are also disclosed, tending to make this whole process more personal and diffuse; (7) Mitigate the interpersonal comparison, in the long term, by facilitating a future new school organization that will put an end to the permanent and exclusive connection of a student to the same class during the school, allowing the enrollment of the student in several modules of the various subjects, in different classes. This will facilitate the progressive implementation of a system that, in parallel with a compulsory curriculum, will open more to an offer of options for free choice by students, referred to a credit system. It is intended to facilitate the disappearance or reduction of incidence of the class-tribe, creating situations of greater socialization of students, because it is more diversified and makes even more diffuse comparisons of grades and other skills. (8) Facilitate the possibility of different degrees of progression of pupils according to their abilities, by allowing them, at their request, to perform exams more quickly and with greater difficulty.

The next section is devoted to the presentation of the ACiSs technological platform.

4 ACiSs Technological Platform

4.1 ACiSs Flow

The ACiSs will be supported on a web platform accessible from any browser, given the proper credentials. The platform will manage users, subjects, subject chapters and CG, questions, exams, grades, etc. In a general description (see Fig. 1), teacher will upload questions to the platform, including the wording, the subject chapter and CG involved in its resolutions (see Sect. 4.2), the resolution time estimation, the degree of difficulty, and one or more answers.

Upon request, exams are generated taking into consideration the subject chapter or the required CG to solve the questions (i.e., users can request an exam from a particular chapter or simply focusing in some particular CG), a maximum solving time estimation, and the degree of difficulty. Then students solve the exam and upload their solutions to the system, answer by answer or bulk, to be classified by a teacher. The teacher will then classify the student’s resolutions and simultaneously mark the CG which where attained by the students.

As IT system should be approved by the users (namely, teachers and students), key elements in the system design, the authors decided to use a mix of Data Flow Diagram (DFD) and Use Cases diagrams to present the system’s specifications, see (Yourdon, 2000; Kulak and Guiney, 2004) and also the arguments of Millet and Nelson (2007). On the other hand, the object/class concept of Unified Modeling Language (UML) was not used for describing the IT system due to its complexity and the fact that many teachers are not used to object oriented concepts. Use Case diagrams is used to describe the interaction of actors (Teachers and Students) with the IT system, see Fig. 1. A context DFD shows the data flow between the IT system and external entities, see Fig. 2 showing most important data flows. The interaction of a teacher with ACiSs system, in an exam creation, is showed in an activity diagram, see Fig. 3. Figure 4 shows the different processes that will be responsible of perform actions, to achieve the goals described in the next 2 paragraphs.

In this context, users management includes personal data, actions log and, in the particular case of the students, information about the taken exams, attained CG and corresponding grades.

Exams generation refers to the definition, automatically or manually, of sets of questions to form an exam. As expected, the automatic version depends on the subject’s chapter/CG, degree of difficulty, and an estimated resolution time. Furthermore, exams’ questions should be diversified within the defined subject chapter or set of CG, as exams are not a collection of questions. If the exam is formative, its generation will also take into consideration student’s data, that is, it will use questions related with the student’s CG difficulties. The exams can latter be retrieved given an identifier, which allows to associate exams to students. In this context, exams can also be semi-automatically generated, e.g., an automatically generated exam can be reformulated (delete and/or add questions) or, on the opposite way, from a fixed set of questions the system can be asked to complete the exam with other questions from the database. As so, beside being used to maintain the users’ grades and assessment, CG attainment are used to generate exams adequate to the student. Other functionalities are also in study, such as automatically sending exams by email to the students to prepare them for scheduled summative examinations.

Out of the scope of this paper is the automatic exam generation algorithm. Nevertheless, the problem has already been studied by some authors in different context and with different complexities. For instance, Nalawade and Ramesh (2016) propose a somehow similar system which automatically generates a question paper from a semantically tagged question repository. This system offers flexibility by supporting four tags (cognitive level, difficulty level, type of question, content/topic for defining a question). However, the system does not generate exams according with the students difficulties, i.e., to prepare them for summative assessments. González and Muñoz (2006) proposed a web-based tool, known as e-status, which allows students in introductory courses to solve exercises in probability and statistical inference. The proposed solution is dynamic in the sense that every problem presents new sample data that is generated randomly, i.e., the student could do the same exercise again but the data (and the solution) would be different. Other examples of similar projects can be seen in (Soler et al. 2012; Rashad et al. 2010). Also out of the scope of this paper is automated grading, already essayed in works such the ones from Ramalingam et al. (2018) where an automated essay assessment system was defined by use of machine learning techniques, by classifying a corpus of textual entities into small number of discrete categories, corresponding to possible grades. Zesch et al. (2015) investigated which features are task-independent and evaluate their transferability on English and German datasets. This approach tries to solve the complicated problem of transferring a system trained on one task to another.

Fig. 1.
figure 1

Main actors in ACiSs system.

Full size image
Fig. 2.
figure 2

Context diagram showing the most important data flows between the ACiSs system, teachers, students and administrator.

Full size image
Fig. 3.
figure 3

Activity diagram of a teacher in an exam creation process.

Full size image

4.2 The Curricular Goals Approach

To be more precise on how CG work, this section considers the set of CG of the “Mathematics A” subject that students from the “Sciences and Technology” and “Socio-economic Sciences” courses must undertake during their Secondary Education in the Portuguese education system (Diário da República 2012; Ministério da Educação 2015). The overall objectives, complemented with more precise descriptors/CG, are organized in each education year, by domains and sub-domains, according to the following structure:  

Domain:

 

Sub-domain:
1.:

General objective

(a) :

Descriptor (CG 1)

(b) :

Descriptor (CG 2)

(c) :

etc.

 

 

Fig. 4.
figure 4

DFD Level 1, interaction of the Teachers and Students with processes in ACiSs system. The data flows descriptions are omitted to simplify the understanding.

Full size image

The different descriptors/CG are written objectively, in strict language to the teacher, and the teacher must select an appropriate teaching strategy. Furthermore, the precise meaning of certain verbs with which some descriptors/CG begin are accurately defined as: Identify/Designate/Refer – the student should correctly use the designation of the concept presented as indicated or equivalent; Recognize – the student must present a coherent argument, even if possibly more informal than the explanation provided by the teacher; Know – the student must know the result, without being required any kind of justification or verification; Prove/Show/Demonstrate – the student must present a mathematical demonstration as accurate as possible; or Justify – the student must justify the wording in a simple way, evoking a property already known. These prerequisites are not detailed in the text, and the teacher must identify them according to the need, the pertinence and the characteristics of each group of students.

The remaining section introduces some examples, liable of being presented in an exam of the 10th Grade for the Portuguese Schools System. The chosen domain is Algebra and the sub-domain is Radicals. For the Radicals sub-domain eleven CG are defined. By way of example, the first three are:  

Algebra:

 

Radicals:
1. :

Defining and performing radicals operations.

a. :

Recognize, given two real numbers a and b and an odd number \(n\in \mathbb {N}\), that if \(a < b\) then \(a^n < b^n\).

b. :

Recognize, given two real numbers a and b and an even number \(n\in \mathbb {N}\), that if \(0 \le a < b\) then \(0 \le a^n < b^n\) and if \(a < b \le 0\) then \(a^n > b^n \ge 0\).

c. :

Know, given a real number a and an odd number \(n\in \mathbb {N}\), that there is a real number b such that \(b^n = a\), prove that it is unique, designate it by root index, and represent it by \(\root n \of {a}\).

 

 

In this context, the wording of two exercises evaluating the CG 1a and 1b can be stated as follow:

Exercise 1

Let a and b be two real number such that \(0 \le a < b\).

  1. (a)

    Prove that \(a^2 < b^2\) and \(a^3 < b^3\).

  2. (b)

    Prove that if \(a^n < b^n\) for a given \(n\in {\mathbb {N}}\) then \(a^{n+1} < b^{n+1}\).

Exercise 2

It is known that if x and y are two real number such that \(0 \le x < y\) and n is a natural number then \(x^n < y^n\). Prove that if \(a< b < 0\) then, \(a^n < b^n\) if n is odd and \(a^n > b^n\) if n is even.

The previous questions should be uploaded to the system separately and the student’s resolution should then be evaluated taking into consideration the attained CG. To upload the questions, the teacher uses a Microsft Word template allowing him/her to upload complexly composed exercises (e.g., containing with tables, figures, mathematical formulas, chemical equations, etc.). Furthermore, the template helps to maintain a congruent design, e.g., in terms of font type and size. Then the ACiSs software will concatenate a header with a set of those word upload documents to generate a final document which can be in several formats, such as HTML, open document formats for office applications (such as, docx or odt), or Portable Document Format (pdf).

5 Conclusion

ACiSs is a pilot research project that suitable to be used in the majority of the subjects present in any level of education. In its core, the system is composed of a dataset of question associated to the CG defined for that subject. Those questions are then used to generate exams to perform formative and summative evaluations. The students resolutions are evaluated in accordance with the attained CG, being this information maintained in the system and latter used to generate appropriate exams. I.e., if a formative evaluation is being performed then, besides CG, resolution time and degree of difficulty, questions will take into consideration the user’s previous answers. The ACiSs platform is in an embryonic stage where simple interfaces were developed and are being tested to their usage.

In the future, the authors intend to study the influence of the ACiSs system by applying it to different classes of the same year while maintaining control classes, i.e., classes using the traditional assessment of exams in the classroom. A comparative analysis of the learning outcomes in both groups will, hopefully, prove the usefulness of the ACiSs. Furthermore, other actions will be taken to measure the stress and concentration of students during the exams using biometric devices, inquiring of their satisfaction, interviewing teachers and school directors.