Changes for page Written Test - Open Answers
Last modified by Ivan Kharitonov on 2022/01/30 17:25
Change comment:
There is no comment for this version
Summary
-
Page properties (2 modified, 0 added, 0 removed)
Details
- Page properties
-
- Title
-
... ... @@ -1,1 +1,1 @@ 1 - WrittenTest- Multiple Choice1 +Oral Examination - Content
-
... ... @@ -4,14 +4,8 @@ 4 4 5 5 = Description = 6 6 7 -The multiple-choicetestconsistsofquestionswhereonlyoneanswercan becorrect.In contrast, themultiple-response testsconsists of questionswhereseveralanswers canbe correct.Neither craftskillsnorcreativity canbemeasured witheitherypeoftest. Nevertheless,its not onlypossibletoquery factualknowledge. Unlessotherwise stated,both testtypesaresummarizedbelowundertheterm"multiple-choicetest". Followingcognitiveperformance levels can be tested via multiple-choicetests:7 +The oral test is a method of assessing the candidate's spoken word. Here assessor and candidate enter into a spoken interaction. An oral test can take place as a presentation (see AMD Presentation), interrogation or application. The oral test generally does not assess linguistic competence as such, but rather knowledge, understanding and problem solving ability. 8 8 9 -* Reproduction of stored knowledge 10 -* Reorganization: Learned knowledge is processed and arranged independently. 11 -* Transfer: Basic principles are transferred to new, similar tasks. 12 -* Problem-solving thinking: Tasks with new questions and aspects are solved. 13 - 14 - 15 15 ---- 16 16 17 17 = Quality Concepts = ... ... @@ -18,15 +18,20 @@ 18 18 19 19 === Validity === 20 20 21 -The validity is not necessarily present. The main problem is the discrepancy between complex cognitive processes and simply ticking an answer. Multiple-Choice Tests can evaluate a competence to act if they require the quick assessment of different facets of a complex subject. 15 +Oral tests could be valid for assessing e.g. understanding, applied problem solving, interpersonal competence, intrapersonal qualities (confidence, self-awareness, professionalism, ethics) and intergrated practice. 16 +Oral tests are valid if the candidate can demonstrate the competencies that are relevant to the validation process. It must therefore be checked whether the test queries what it should query (face validity). To check the content validity, it must be asked whether the discussed content is part of the required competencies, whether it goes beyond it or whether it is too barely discussed. In order to increase the construct validity of the oral test, the tasks must be in line with the theoretical understanding of the skills of event technology. 22 22 23 23 === Reliability === 24 24 25 -The objectivity of Multiple-Choice Tests is significantly higher than for classic written exams. The reason for this is that the correction is independent of the correcting person and is often automated. In addition, well-formulated questions go hand in hand with high reliability. If the tests are carried out regularly and with a large number of test persons, they are particularly economical. 20 +Reliability can be affected by candidate anxiety, candidate hearing or speech difficulties or assessor bias (e.g. due to gender, ethnicity, language). 21 +In order to standardize the test and increase comparability, a criteria list and an answer key should be created. The fact that the assessor can ask questions can affect reliability. An answer that was only given on request should possibly get fewer points. 22 +In order to increase the reliability, a competence should be tested in different ways. 23 +The use of multiple assessors also increases reliability. 24 +Due to the limited number of questions and the many disruptive factors in the interaction, the oral test is less reliable. 26 26 27 27 == Limitations == 28 28 29 - Creativityorcraftskillscannotbetestedwithmultiplechoicetests.28 +Since the time and question options are limited, only a limited amount of knowledge can be queried. Due to disturbing factors (excitement, stress, unusual situation) the method should be used in combination with other methods. 30 30 31 31 ---- 32 32 ... ... @@ -34,15 +34,18 @@ 34 34 35 35 == Tips == 36 36 37 -To reduce the possibility of scoring by gambling, the alternative answers should be of high quality. According to studies, three answer options are sufficient to make successful gambling unlikely. 36 +The candidates should be informed in advance about the framework conditions of the oral test: e.g. time period, location, requirements and items that are not included. 37 +A criteria list helps the assessors to objectively assess the results of the oral test. The assessors should be aware that inquiries can give unwanted tips. 38 +Candidates should be given sufficient time to answer. 38 38 39 39 == Traps == 40 40 41 -Disadvantages of too many alternative answers are a greater effort in the designing, risk of not plausible distractors (distractor = false answer option) and revealing of context hints for other tasks. Depending on the task, five or more answer options can also be useful. 42 +It should not be the presentation and speaking skills that are assessed, but rather the competencies that are queried as part of the assessment process. 43 +If the oral test is not recorded, the results can be objected afterwards. 42 42 43 43 == Scoring Tools == 44 44 45 - Amajoradvantageof Multiple-Choice / responsetests(short:MC-tests)is that theycan be evaluatedeasily. In addition, thequalityof particular questions can be determinedtatistically.Badformulatedquestionscan berecognized as such. Asalreadymentioned,only one answer can be correct for Multiple-Choicequestions.If the right answerismarked,one point will beawarded.Ifa wronganswerornoansweris marked,nopointwill be awarded.The awardingof pointscan be different for Multiple-Responsetests. Every correctmarkedanswer scoresonepoint(right answers marked,or falseanswersnotarked).Everyincorrect markedanswer brings oneminuspoint (rightanswers notmarkedor false answersmarked).Thedifferencedeterminesthetotalscoreofthequestion.Ifthedifferenceisnegative,thescore is0 Points.47 +The results of the oral test can usually be evaluated immediately afterwards. A marking guide can be used here, which also serves as a framework for feedback to the candidate. Since it can be difficult to take notes and interview the candidate at the same time, it makes sense to use several assessors. In this case, one can record the conversation while the other is conducting the conversation. Audio or video recording of the oral test is also highly recommended. 46 46 47 47 ---- 48 48 ... ... @@ -50,37 +50,34 @@ 50 50 51 51 == Information for Standard == 52 52 53 - Forahighdegree of standardization,the testets should be formulated in alearand understandablewayandused without modification.Languagebarriersshouldbetaken intoaccount.55 +The created marking guide should be aligned with the competence descriptions of the UNITs. The tasks and questions of the oral test should be precisely formulated in advance and each candidate should be asked the same questions. The candidate's answers are compared with the expected answers from the marking guide. In addition, cues and support that the assessors can provide are defined in advance. 54 54 55 55 == Development == 56 56 57 - Whenpreparingthetest, highqualitydistractorshouldbechosen that areplausiblewithoutfooling the candidate.Thetest should befocused onhecompetencies to bemeasuredandnotjuston smartness.Asolution sheetgivesthenecessaryinformationabout thecorrect answers.59 +The development process involves the creation of tasks and questions that are designed to assess the relevant competencies. Expected answers and possible cues are set out in a marking guide. 58 58 59 59 == Needs/Set-Up == 60 60 61 - The testcanbeperformedeitherwith apen andpaper or electronicallyonacomputer.63 +Depending on the task, a preparation room with writing materials for the candidate, an examination room and technical equipment that is part of the task is required. In addition to the questioning assessor, a minute taker is advisable. 62 62 63 63 == Requirements for Assessors == 64 64 65 -The designofthetestrequiresahighleveloftechnicalexpertisein order todevelopusefuldistractors.For theevaluation ofthetestnospecial skillsarerequired.67 +The assessor needs basic skills in conducting unbiased dialogue. He needs professional skills in order to be able to deduce appropriate competencies from the candidate's report according to the marking guide. Professional and pedagogical competences are necessary to create the tasks. 66 66 67 67 == Examples == 68 68 69 - Forexample,a multiplechoice test can be used to testknowledge aboutthe useofspecial effects.71 +An oral test can be used to test management skills. For example, the candidate should explain in a subtask how to check which materials and equipment are required for a given job. 70 70 71 71 == In Combination with == 72 72 73 - WrittenTest- OpenAnswers75 +Since the oral test does not test practical skills, a combination with corresponding methods like observation or role play is appropriate. 74 74 75 75 = References/Notes = 76 76 77 -* Baghaei, P. / Amrahi, N. (2011): The effects of the number of options on the psychometric characteristics of Multiple-Choice items. In: Psychological Test and Assessment Modeling. 53 (2), p. 192-211. 78 -* CTL Center for Teaching and Learning / Universität Wien (2015): Erstellen von schriftlichen, mündlichen und Multiple-Choice-Prüfungen. Online: [[https:~~/~~/ctl.univie.ac.at/fileadmin/user_upload/z_ctl/Qualitaet_von_Studien/Qualitaet_von_Pruefungen/170314_Erstellen_von_schriftlichen_muendlichen_MC_Pruefungen.pdf >>https://ctl.univie.ac.at/fileadmin/user_upload/z_ctl/Qualitaet_von_Studien/Qualitaet_von_Pruefungen/170314_Erstellen_von_schriftlichen_muendlichen_MC_Pruefungen.pdf]](last 25.05.2020). 79 -* Elsa / Leibniz Universität Hannover (n.d.): Erstellen und Bewerten von Multiple-Choice-Aufgaben. Online: [[https:~~/~~/www.zqs.uni-hannover.de/fileadmin/zqs/PDF/E-Learning/elsa_handreichung_zum_erstellen_und_bewerten_von_mc-fragen_2013.pdf >>https://www.zqs.uni-hannover.de/fileadmin/zqs/PDF/E-Learning/elsa_handreichung_zum_erstellen_und_bewerten_von_mc-fragen_2013.pdf]](last: 25.05.2020). 80 -* Kubinger, K. (2014): Gutachten zur Erstellung “gerichtsfester” Multiple-Choice-Prüfungsaufgaben. In: Psychologische Rundschau. 65 (3), p. 169-178. 81 -* Rodriguez, M. (2005): Three Options Are Optimal for Multiple-Choice Items: A Meta-Analysis of 80 Years of Research. In: Educational Measurement. Issues and Practice. 24 (2), p. 3-13. 82 -* Universität Kassel (n.d.): Handreichung für Klausuren mit Aufgaben nach dem Antwort-Wahl-Verfahren (Single-Choice/Multiple-Choice). Online: [[http:~~/~~/www.uni-kassel.de/einrichtungen/fileadmin/datas/einrichtungen/scl/E-Klausuren/Handreichung_Antwort_Wahl_Aufgaben_final.pdf>>http://www.uni-kassel.de/einrichtungen/fileadmin/datas/einrichtungen/scl/E-Klausuren/Handreichung_Antwort_Wahl_Aufgaben_final.pdf]] (last: 25.05.2020). 83 -* Universität Zürich (n.d.): Hochschuldidaktik A – Z. Multiple-Choice-Prüfungen. Online: [[http:~~/~~/www.hochschuldidaktik.uzh.ch/dam/jcr:ffffffff-9a08-8cca-0000-00002cfe461f/A_Z_Multiple-Choice.pdf >>http://www.hochschuldidaktik.uzh.ch/dam/jcr:ffffffff-9a08-8cca-0000-00002cfe461f/A_Z_Multiple-Choice.pdf]](last: 02.06.2020). 79 +* Joughin, G. (2010): A short guide to oral assessment. Online: [[https:~~/~~/www.qub.ac.uk/directorates/AcademicStudentAffairs/CentreforEducationalDevelopment/FilestoreDONOTDELETE/Filetoupload,213702,en.pdf>>https://www.qub.ac.uk/directorates/AcademicStudentAffairs/CentreforEducationalDevelopment/FilestoreDONOTDELETE/Filetoupload,213702,en.pdf]] (last 21.07.2020). 80 +* Assessment Resource Centre (2014): Types of Assessment Methods. Online: [[https:~~/~~/ar.cetl.hku.hk/am_orals.htm>>https://ar.cetl.hku.hk/am_orals.htm]] (last 21.07.2020). 81 +* Iqbal, I. et al. (2010): The value of oral assessments: A review. Online: [[https:~~/~~/publishing.rcseng.ac.uk/doi/pdf/10.1308/147363510X511030>>https://publishing.rcseng.ac.uk/doi/pdf/10.1308/147363510X511030]] (last 21.07.2020). 82 +* Centre of Expertise for Higher Education / University of Antwerp (n.d.): Oral tests and objective assessment. Online: [[https:~~/~~/www.uantwerpen.be/en/centres/centre-expertise-higher-education/didactic-information/teaching-tips-english/assessing-students/oral-tests-objectivity/>>https://www.uantwerpen.be/en/centres/centre-expertise-higher-education/didactic-information/teaching-tips-english/assessing-students/oral-tests-objectivity/]] (last 21.07.2020). 84 84 85 85 == AT == 86 86