Summary

• Page properties (2 modified, 0 added, 0 removed)

Details

Page properties

Title

...	...	@@ -1,1 +1,1 @@
1		-Oral Examination
	1	+Written Test - Multiple Choice

Content

...	...	@@ -4,8 +4,14 @@
4	4
5	5	= Description =
6	6
7		-The oral test is a method of assessing the candidate's spoken word. Here assessor and candidate enter into a spoken interaction. An oral test can take place as a presentation (see AMD Presentation), interrogation or application. The oral test generally does not assess linguistic competence as such, but rather knowledge, understanding and problem solving ability.
	7	+The multiple-choice test consists of questions where only one answer can be correct. In contrast, the multiple-response tests consists of questions where several answers can be correct. Neither craft skills nor creativity can be measured with either type of test. Nevertheless, it is not only possible to query factual knowledge. Unless otherwise stated, both test types are summarized below under the term "multiple-choice test". Following cognitive performance levels can be tested via multiple-choice tests:
8	8
	9	+* Reproduction of stored knowledge
	10	+* Reorganization: Learned knowledge is processed and arranged independently.
	11	+* Transfer: Basic principles are transferred to new, similar tasks.
	12	+* Problem-solving thinking: Tasks with new questions and aspects are solved.
	13	+
	14	+
9	9	----
10	10
11	11	= Quality Concepts =
...	...	@@ -12,20 +12,15 @@
12	12
13	13	=== Validity ===
14	14
15		-Oral tests could be valid for assessing e.g. understanding, applied problem solving, interpersonal competence, intrapersonal qualities (confidence, self-awareness, professionalism, ethics) and intergrated practice.
16		-Oral tests are valid if the candidate can demonstrate the competencies that are relevant to the validation process. It must therefore be checked whether the test queries what it should query (face validity). To check the content validity, it must be asked whether the discussed content is part of the required competencies, whether it goes beyond it or whether it is too barely discussed. In order to increase the construct validity of the oral test, the tasks must be in line with the theoretical understanding of the skills of event technology.
	21	+The validity is not necessarily present. The main problem is the discrepancy between complex cognitive processes and simply ticking an answer. Multiple-Choice Tests can evaluate a competence to act if they require the quick assessment of different facets of a complex subject.
17	17
18	18	=== Reliability ===
19	19
20		-Reliability can be affected by candidate anxiety, candidate hearing or speech difficulties or assessor bias (e.g. due to gender, ethnicity, language).
21		-In order to standardize the test and increase comparability, a criteria list and an answer key should be created. The fact that the assessor can ask questions can affect reliability. An answer that was only given on request should possibly get fewer points.
22		-In order to increase the reliability, a competence should be tested in different ways.
23		-The use of multiple assessors also increases reliability.
24		-Due to the limited number of questions and the many disruptive factors in the interaction, the oral test is less reliable.
	25	+The objectivity of Multiple-Choice Tests is significantly higher than for classic written exams. The reason for this is that the correction is independent of the correcting person and is often automated. In addition, well-formulated questions go hand in hand with high reliability. If the tests are carried out regularly and with a large number of test persons, they are particularly economical.
25	25
26	26	== Limitations ==
27	27
28		-Since the time and question options are limited, only a limited amount of knowledge can be queried. Due to disturbing factors (excitement, stress, unusual situation) the method should be used in combination with other methods.
	29	+Creativity or craft skills cannot be tested with multiple choice tests.
29	29
30	30	----
31	31
...	...	@@ -33,18 +33,15 @@
33	33
34	34	== Tips ==
35	35
36		-The candidates should be informed in advance about the framework conditions of the oral test: e.g. time period, location, requirements and items that are not included.
37		-A criteria list helps the assessors to objectively assess the results of the oral test. The assessors should be aware that inquiries can give unwanted tips.
38		-Candidates should be given sufficient time to answer.
	37	+To reduce the possibility of scoring by gambling, the alternative answers should be of high quality. According to studies, three answer options are sufficient to make successful gambling unlikely.
39	39
40	40	== Traps ==
41	41
42		-It should not be the presentation and speaking skills that are assessed, but rather the competencies that are queried as part of the assessment process.
43		-If the oral test is not recorded, the results can be objected afterwards.
	41	+Disadvantages of too many alternative answers are a greater effort in the designing, risk of not plausible distractors (distractor = false answer option) and revealing of context hints for other tasks. Depending on the task, five or more answer options can also be useful.
44	44
45	45	== Scoring Tools ==
46	46
47		-The results of the oral test can usually be evaluated immediately afterwards. A marking guide can be used here, which also serves as a framework for feedback to the candidate. Since it can be difficult to take notes and interview the candidate at the same time, it makes sense to use several assessors. In this case, one can record the conversation while the other is conducting the conversation. Audio or video recording of the oral test is also highly recommended.
	45	+A major advantage of Multiple-Choice / response tests (short: MC-tests) is that they can be evaluated easily. In addition, the quality of particular questions can be determined statistically. Bad formulated questions can be recognized as such. As already mentioned, only one answer can be correct for Multiple-Choice questions. If the right answer is marked, one point will be awarded. If a wrong answer or no answer is marked, no point will be awarded. The awarding of points can be different for Multiple-Response tests. Every correct marked answer scores one point (right answers marked, or false answers not marked). Every incorrect marked answer brings one minus point (right answers not marked or false answers marked). The difference determines the total score of the question. If the difference is negative, the score is 0 Points.
48	48
49	49	----
50	50
...	...	@@ -52,34 +52,37 @@
52	52
53	53	== Information for Standard ==
54	54
55		-The created marking guide should be aligned with the competence descriptions of the UNITs. The tasks and questions of the oral test should be precisely formulated in advance and each candidate should be asked the same questions. The candidate's answers are compared with the expected answers from the marking guide. In addition, cues and support that the assessors can provide are defined in advance.
	53	+For a high degree of standardization, the test sheets should be formulated in a clear and understandable way and used without modification. Language barriers should be taken into account.
56	56
57	57	== Development ==
58	58
59		-The development process involves the creation of tasks and questions that are designed to assess the relevant competencies. Expected answers and possible cues are set out in a marking guide.
	57	+When preparing the test, high quality distractors should be chosen that are plausible without fooling the candidate. The test should be focused on the competencies to be measured and not just on smartness. A solution sheet gives the necessary information about the correct answers.
60	60
61	61	== Needs/Set-Up ==
62	62
63		-Depending on the task, a preparation room with writing materials for the candidate, an examination room and technical equipment that is part of the task is required. In addition to the questioning assessor, a minute taker is advisable.
	61	+The test can be performed either with a pen and paper or electronically on a computer.
64	64
65	65	== Requirements for Assessors ==
66	66
67		-The assessor needs basic skills in conducting unbiased dialogue. He needs professional skills in order to be able to deduce appropriate competencies from the candidate's report according to the marking guide. Professional and pedagogical competences are necessary to create the tasks.
	65	+The design of the test requires a high level of technical expertise in order to develop useful distractors. For the evaluation of the test no special skills are required.
68	68
69	69	== Examples ==
70	70
71		-An oral test can be used to test management skills. For example, the candidate should explain in a subtask how to check which materials and equipment are required for a given job.
	69	+For example, a multiple choice test can be used to test knowledge about the use of special effects.
72	72
73	73	== In Combination with ==
74	74
75		-Since the oral test does not test practical skills, a combination with corresponding methods like observation or role play is appropriate.
	73	+Written Test - Open Answers
76	76
77	77	= References/Notes =
78	78
79		-* Joughin, G. (2010): A short guide to oral assessment. Online: [[https:~~/~~/www.qub.ac.uk/directorates/AcademicStudentAffairs/CentreforEducationalDevelopment/FilestoreDONOTDELETE/Filetoupload,213702,en.pdf>>https://www.qub.ac.uk/directorates/AcademicStudentAffairs/CentreforEducationalDevelopment/FilestoreDONOTDELETE/Filetoupload,213702,en.pdf]] (last 21.07.2020).
80		-* Assessment Resource Centre (2014): Types of Assessment Methods. Online: [[https:~~/~~/ar.cetl.hku.hk/am_orals.htm>>https://ar.cetl.hku.hk/am_orals.htm]] (last 21.07.2020).
81		-* Iqbal, I. et al. (2010): The value of oral assessments: A review. Online: [[https:~~/~~/publishing.rcseng.ac.uk/doi/pdf/10.1308/147363510X511030>>https://publishing.rcseng.ac.uk/doi/pdf/10.1308/147363510X511030]] (last 21.07.2020).
82		-* Centre of Expertise for Higher Education / University of Antwerp (n.d.): Oral tests and objective assessment. Online: [[https:~~/~~/www.uantwerpen.be/en/centres/centre-expertise-higher-education/didactic-information/teaching-tips-english/assessing-students/oral-tests-objectivity/>>https://www.uantwerpen.be/en/centres/centre-expertise-higher-education/didactic-information/teaching-tips-english/assessing-students/oral-tests-objectivity/]] (last 21.07.2020).
	77	+* Baghaei, P. / Amrahi, N. (2011): The effects of the number of options on the psychometric characteristics of Multiple-Choice items. In: Psychological Test and Assessment Modeling. 53 (2), p. 192-211.
	78	+* CTL Center for Teaching and Learning / Universität Wien (2015): Erstellen von schriftlichen, mündlichen und Multiple-Choice-Prüfungen. Online: [[https:~~/~~/ctl.univie.ac.at/fileadmin/user_upload/z_ctl/Qualitaet_von_Studien/Qualitaet_von_Pruefungen/170314_Erstellen_von_schriftlichen_muendlichen_MC_Pruefungen.pdf >>https://ctl.univie.ac.at/fileadmin/user_upload/z_ctl/Qualitaet_von_Studien/Qualitaet_von_Pruefungen/170314_Erstellen_von_schriftlichen_muendlichen_MC_Pruefungen.pdf]](last 25.05.2020).
	79	+* Elsa / Leibniz Universität Hannover (n.d.): Erstellen und Bewerten von Multiple-Choice-Aufgaben. Online: [[https:~~/~~/www.zqs.uni-hannover.de/fileadmin/zqs/PDF/E-Learning/elsa_handreichung_zum_erstellen_und_bewerten_von_mc-fragen_2013.pdf >>https://www.zqs.uni-hannover.de/fileadmin/zqs/PDF/E-Learning/elsa_handreichung_zum_erstellen_und_bewerten_von_mc-fragen_2013.pdf]](last: 25.05.2020).
	80	+* Kubinger, K. (2014): Gutachten zur Erstellung “gerichtsfester” Multiple-Choice-Prüfungsaufgaben. In: Psychologische Rundschau. 65 (3), p. 169-178.
	81	+* Rodriguez, M. (2005): Three Options Are Optimal for Multiple-Choice Items: A Meta-Analysis of 80 Years of Research. In: Educational Measurement. Issues and Practice. 24 (2), p. 3-13.
	82	+* Universität Kassel (n.d.): Handreichung für Klausuren mit Aufgaben nach dem Antwort-Wahl-Verfahren (Single-Choice/Multiple-Choice). Online: [[http:~~/~~/www.uni-kassel.de/einrichtungen/fileadmin/datas/einrichtungen/scl/E-Klausuren/Handreichung_Antwort_Wahl_Aufgaben_final.pdf>>http://www.uni-kassel.de/einrichtungen/fileadmin/datas/einrichtungen/scl/E-Klausuren/Handreichung_Antwort_Wahl_Aufgaben_final.pdf]] (last: 25.05.2020).
	83	+* Universität Zürich (n.d.): Hochschuldidaktik A – Z. Multiple-Choice-Prüfungen. Online: [[http:~~/~~/www.hochschuldidaktik.uzh.ch/dam/jcr:ffffffff-9a08-8cca-0000-00002cfe461f/A_Z_Multiple-Choice.pdf >>http://www.hochschuldidaktik.uzh.ch/dam/jcr:ffffffff-9a08-8cca-0000-00002cfe461f/A_Z_Multiple-Choice.pdf]](last: 02.06.2020).
83	83
84	84	== AT ==
85	85