Latest revision: 2024-07-02

Exam Construction

In our workshops, we have seen that teachers’ experience with constructing exams varies. Therefore, we have compiled a model based on Jonathan Wedman’s book “Prov” (in Swedish only), chapter 7 “Provkonstruktion”.

Which skills should be examined and how? See learning outcomes and assessment methods in the course plan.
Which area should be examined in the exam?
Will the area be divided into different sub-areas/subjects (“sections” in Inspera)? If so, how should the different parts be weighted?
What content should be examined in each sub-area? How should the content be weighted?
Choose the right question type.
Formulate questions and, if necessary, answer options.
If necessary: Choose the right media and resources for individual questions (image, sound, film, text, document).
If necessary: Choose the right resources for the entire exam (documents, websites, support programs).
Determine threshold values and conditions for different grades.

Automatic and manual marking

In the demo exam, you got to know the different question types that are available in Inspera. The overview below will list the advantages and disadvantages of automatic and manual marking. In an Inspera-exam, you can combine question with automatic and manual marking.

Automatic marking

Advantages

saves a lot of time in the whole marking process (Donnelly, 2014)
convenient for large student groups
transparent marking process that’s perceived as objectively (Donnelly, 2014)
easy to examine knowledge according to Bloom's taxonomy which is about remembering, comprehending, and applying
easy to use the data for statistics (Donnelly, 2014) and learning analytics

Disadvantages

it takes a lot of time to develop questions
more difficult to examine certain knowledge according to Bloom's taxonomy (create, evaluate)
there is a lot of best practice in formulating good multiple-choice questions, but little research. "To increase the validity of teacher-made tests, many item-writing rules-of-thumb are available in the literature, but few rules have been tested experimentally." (Frey et al., 2005, p.357)

Manual marking

Advantages

greater freedom to create questions that students answer using their own words
good for questions that request students to develop their own reasoning

Disadvantages

may take time to evaluate long texts
if there isn't any clear marking criteria, the marking may be perceived as subjectively

References

Here are the references that are cited above. If you find more useful research articles, we're glad if you let us know by sending an e-mail to support@mau.se.

Donnelly, Christina. (2014). The Use of Case Based Multiple Choice Questions for Assessing Large Group Teaching: Implications on Student's Learning. Irish Journal of Academic Practice, 3(1), 1-15. https://doi.org/10.21427/D7CX32

Frey, Bruce B., Petersen, Stephanie, Edwards, Lisa M., Pedrotti, Jennifer Teramoto, Peyton, Vicki. (2005). Item-writing rules: Collective wisdom. Teaching and Teacher Education, 21(4), 357-364. https://doi.org/10.1016/j.tate.2005.01.008

Haladyna, Thomas M., Downing, Steven M., & Rodriguez, Michael C. (2002). A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment. Applied Measurement in Education, 15(3), 309-333. https://doi.org/10.1207/S15324818AME1503_5

Martínez, Rafael J., Moreno, Rafael, Martín, Irene, & Trigo, M. Eva (2009). Evaluation of five guidelines for option development in multiple-choice item-writing. Psicothema, 21(2), 326-330.

Wedman, Jonathan (2022). Prov (Andra utgåvan). Umeå: Giraffhuset.

Advanced readings: didactic perspectives

In modul 7 "Didactics (advanced)" you can read more about how to

design multiple-choice questions
use Bloom's taxonomy to design questions
design questions with Universal Design for Learning (UDL) in mind

Useful links