Last modified by Ivan Kharitonov on 2022/01/30 17:25
From version < 12.1 >
edited by Randell Greenlee
on 2021/11/09 16:57
To version < 14.1 >
edited by Randell Greenlee
on 2021/11/09 17:11
< >
Change comment: There is no comment for this version

Summary

Details

Page properties
Title
... ... @@ -1,1 +1,1 @@
1 -Oral Examination
1 +Written Test - Open Answers
Content
... ... @@ -4,8 +4,11 @@
4 4  
5 5  = Description =
6 6  
7 -The oral test is a method of assessing the candidate's spoken word. Here assessor and candidate enter into a spoken interaction. An oral test can take place as a presentation (see AMD Presentation), interrogation or application. The oral test generally does not assess linguistic competence as such, but rather knowledge, understanding and problem solving ability.
7 +Open questions are all questions for which there are no answer options. This means that there is no influence from given answer options. In the written test, the questions are answered in writing. Open questions can be used to query information in text form or numerical information (e.g. number of minutes). Open questions can be used, for example, if the understanding of a situation is to be assessed or if the number of answer options would be too large.
8 8  
9 +Case studies can be recorded as a sub-form of the written test (open answers). They can be used to simulate everyday working life and the tasks associated with it. The tasks to be solved address a problem common to the industry. Here, analytical and organizational competencies, such as the approach to a difficult problem, are tested.
10 +Case studies measure action orientation, entrepreneurial thinking and understanding of complexity.
11 +
9 9  ----
10 10  
11 11  = Quality Concepts =
... ... @@ -12,20 +12,19 @@
12 12  
13 13  === Validity ===
14 14  
15 -Oral tests could be valid for assessing e.g. understanding, applied problem solving, interpersonal competence, intrapersonal qualities (confidence, self-awareness, professionalism, ethics) and intergrated practice.
16 -Oral tests are valid if the candidate can demonstrate the competencies that are relevant to the validation process. It must therefore be checked whether the test queries what it should query (face validity). To check the content validity, it must be asked whether the discussed content is part of the required competencies, whether it goes beyond it or whether it is too barely discussed. In order to increase the construct validity of the oral test, the tasks must be in line with the theoretical understanding of the skills of event technology.
18 +Open questions are used to check knowledge or situational interpretation. The disadvantage is that it checks more the skill to express yourself on paper than it checks the real ability to perform in real life. It proves your know how to act, but not that you are able to act. Answers are checked against a checklist but need interpretation of skilled assessors. 
19 +Open answers are most suitable in situations, where new information should be gained, where respondents should not be primed by the given response-possibilities or where holistic feedback is asked and given answers would significantly limit the informative value of the answer.
17 17  
18 18  === Reliability ===
19 19  
20 -Reliability can be affected by candidate anxiety, candidate hearing or speech difficulties or assessor bias (e.g. due to gender, ethnicity, language).
21 -In order to standardize the test and increase comparability, a criteria list and an answer key should be created. The fact that the assessor can ask questions can affect reliability. An answer that was only given on request should possibly get fewer points.
22 -In order to increase the reliability, a competence should be tested in different ways.
23 -The use of multiple assessors also increases reliability.
24 -Due to the limited number of questions and the many disruptive factors in the interaction, the oral test is less reliable.
23 +Tests can be intimidating for people who have had bad experiences with these types of tests in previous learning contexts.
24 +Research has shown that dissatisfied people give longer answers to express their dissatisfaction. Thus, the respondent's mood influences the length of the response, which limits reliability.
25 +Different field sizes for the answer to the same question affect reliability.
25 25  
26 26  == Limitations ==
27 27  
28 -Since the time and question options are limited, only a limited amount of knowledge can be queried. Due to disturbing factors (excitement, stress, unusual situation) the method should be used in combination with other methods.
29 +Since a high degree of formulation competence is required to answer open questions, a poor result cannot necessarily be inferred from an inadequate result.
30 +Open questions are not suitable for measuring practical skills. They are only of limited help when assessing social skills.
29 29  
30 30  ----
31 31  
... ... @@ -33,18 +33,18 @@
33 33  
34 34  == Tips ==
35 35  
36 -The candidates should be informed in advance about the framework conditions of the oral test: e.g. time period, location, requirements and items that are not included.
37 -A criteria list helps the assessors to objectively assess the results of the oral test. The assessors should be aware that inquiries can give unwanted tips.
38 -Candidates should be given sufficient time to answer.
38 +The questions should be clearly formulated. It must be clear to the candidate what form of answer is expected (e.g. bullet points, small essay, several details).
39 +With extensive case studies, it takes more time to analyze the text and answer the question.
40 +In order to reduce the chance of accidental hits and thus increase reliability, there should be several independent observation options for each requirement dimension.
39 39  
40 40  == Traps ==
41 41  
42 -It should not be the presentation and speaking skills that are assessed, but rather the competencies that are queried as part of the assessment process.
43 -If the oral test is not recorded, the results can be objected afterwards.
44 +The field sizes for the response should be adjusted according to the expected scope. A reasonable amount of time should be calculated to answer the question.
44 44  
45 45  == Scoring Tools ==
46 46  
47 -The results of the oral test can usually be evaluated immediately afterwards. A marking guide can be used here, which also serves as a framework for feedback to the candidate. Since it can be difficult to take notes and interview the candidate at the same time, it makes sense to use several assessors. In this case, one can record the conversation while the other is conducting the conversation. Audio or video recording of the oral test is also highly recommended.
48 +Correction keys can be in place (what the assessors want to see) in order to enable the assessment. 
49 +To assess the case study, the assessor uses a model solution (chronologically based on the items in the case study) and an observation sheet (sorted by competencies). The answers given are compared with the model solution. In order to ensure the evaluation objectivity, especially creative answers do not receive additional points. If the required answer has not been given, the candidate can be asked per item. The ticks are then added and entered on an appropriate scale. Finally, the assessors compare the results of their observations with each other in order to record an overall result.
48 48  
49 49  ----
50 50  
... ... @@ -52,34 +52,34 @@
52 52  
53 53  == Information for Standard ==
54 54  
55 -The created marking guide should be aligned with the competence descriptions of the UNITs. The tasks and questions of the oral test should be precisely formulated in advance and each candidate should be asked the same questions. The candidate's answers are compared with the expected answers from the marking guide. In addition, cues and support that the assessors can provide are defined in advance.
57 +The written test with open answer is difficult to standardize. An evaluation of the answers by several assessors can increase the validity of the results. A marking guide should allow a wide range of possible answers without losing the professionalism of the test.
56 56  
57 57  == Development ==
58 58  
59 -The development process involves the creation of tasks and questions that are designed to assess the relevant competencies. Expected answers and possible cues are set out in a marking guide.
61 +The questions should be designed in such a way that it is transparent what the scope of the answers should be. The predefined text field can provide the candidate with information about the scope of the answer. The questions should be derived from the UNITs of the competences to be measured. In the assessment, care should be taken not to assess linguistic expression.
60 60  
61 61  == Needs/Set-Up ==
62 62  
63 -Depending on the task, a preparation room with writing materials for the candidate, an examination room and technical equipment that is part of the task is required. In addition to the questioning assessor, a minute taker is advisable.
65 +Besides pen and paper, a computer can also be used to answer the questions.
64 64  
65 65  == Requirements for Assessors ==
66 66  
67 -The assessor needs basic skills in conducting unbiased dialogue. He needs professional skills in order to be able to deduce appropriate competencies from the candidate's report according to the marking guide. Professional and pedagogical competences are necessary to create the tasks.
69 +Assessors need comprehensive skills to evaluate complex texts without bias. They must be able to identify content and professional skills despite a lack of articulation. The assessment of answers requires in-depth professional expertise.
68 68  
69 69  == Examples ==
70 70  
71 -An oral test can be used to test management skills. For example, the candidate should explain in a subtask how to check which materials and equipment are required for a given job.
73 +A case study is possible in which the candidate describes how he would react in the event of an accident at work.
72 72  
73 73  == In Combination with ==
74 74  
75 -Since the oral test does not test practical skills, a combination with corresponding methods like observation or role play is appropriate.
77 +Since the test does not test practical skills, it should be combined with Role Play or Observation, for example. The test can be combined with a multiple choice test to test more factual knowledge.
76 76  
77 77  = References/Notes =
78 78  
79 -* Joughin, G. (2010): A short guide to oral assessment. Online: [[https:~~/~~/www.qub.ac.uk/directorates/AcademicStudentAffairs/CentreforEducationalDevelopment/FilestoreDONOTDELETE/Filetoupload,213702,en.pdf>>https://www.qub.ac.uk/directorates/AcademicStudentAffairs/CentreforEducationalDevelopment/FilestoreDONOTDELETE/Filetoupload,213702,en.pdf]] (last 21.07.2020).
80 -* Assessment Resource Centre (2014): Types of Assessment Methods. Online: [[https:~~/~~/ar.cetl.hku.hk/am_orals.htm>>https://ar.cetl.hku.hk/am_orals.htm]] (last 21.07.2020).
81 -* Iqbal, I. et al. (2010): The value of oral assessments: A review. Online: [[https:~~/~~/publishing.rcseng.ac.uk/doi/pdf/10.1308/147363510X511030>>https://publishing.rcseng.ac.uk/doi/pdf/10.1308/147363510X511030]] (last 21.07.2020).
82 -* Centre of Expertise for Higher Education / University of Antwerp (n.d.): Oral tests and objective assessment. Online: [[https:~~/~~/www.uantwerpen.be/en/centres/centre-expertise-higher-education/didactic-information/teaching-tips-english/assessing-students/oral-tests-objectivity/>>https://www.uantwerpen.be/en/centres/centre-expertise-higher-education/didactic-information/teaching-tips-english/assessing-students/oral-tests-objectivity/]] (last 21.07.2020).
81 +* CEDEFOP (2016): Europäische Leitlinien für die Validierung nicht formalen und informellen Lernens. Luxemburg: Amt für Veröffentlichungen der Europäischen Union.
82 +* Eck, C. et al. (2016): Assessment-Center. Entwicklung und Anwendung – mit 57 AC-Aufgaben und Checklisten zum Downloaden und Bearbeiten im Internet. 3. Aufl. Berlin / Heidelberg: Springer.
83 +* Obermann, C. (2018): Assessment Center. Entwicklung, Durchführung, Trends. Mit neuen originalen AC-Übungen. 6., vollständig überarb. u. erw. Aufl. Wiesbaden: Springer Fachmedien.
84 +* Züll, G. (2015): Offene Fragen. Hannover: Leibniz-Institut für Sozialwissenschaften. Online: [[https:~~/~~/www.gesis.org/fileadmin/upload/SDMwiki/Archiv/Offene_Fragen_Zuell_012015_1.0.pdf>>https://www.gesis.org/fileadmin/upload/SDMwiki/Archiv/Offene_Fragen_Zuell_012015_1.0.pdf]].
83 83  
84 84  == AT ==
85 85