Changes for page Post Box Exercise

Last modified by Ivan Kharitonov on 2022/01/30 17:24
From version < 17.1 >
edited by Eric deRuijter
on 2021/11/24 10:24
To version < 13.1 >
edited by Randell Greenlee
on 2021/11/09 16:48
< >
Change comment: There is no comment for this version

Summary

Details

Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.EricDeRuijter
1 +XWiki.RandellGreenlee
Content
... ... @@ -18,9 +18,14 @@
18 18  
19 19  (% style="color:inherit; font-family:inherit; font-size:23px" %)Reliability
20 20  
21 +|(((
21 21  Reliability and evaluation objectivity are special challenges in post box exercises. A meta-analysis based on 16 individual studies showed an interrater reliability of .76 (Whetzel et al. 2014). The assessment of different raters are therefore "good" or "substantial".
22 22  Since chance hits can distort the assessment results, a sufficient number of tasks should be set. Standardization can further increase reliability. In the scenario, an order and a time window for processing should be defined.
23 23  
25 +Reliabilität und Auswertungsobjektivität sind besondere Herausforderungen bei Post Box Exercise. Eine Metaanalyse auf Basis von 16 Einzelstudien ergab eine Interrater-Reliabilität von .76 (Whetzel et al. 2014). Die Urteile unterschiedlicher Bewerter stimmen also "gut" bzw. "substanziell" miteinander überein.
26 +Da chance hits die Assessment Ergebnisse verzerren können, sollten ausreichend viele Aufgaben gestellt werden. Durch eine Standardisierung kann die Reliabilität zusätzlich erhöht werden. So sollte im Szenario eine Reihenfolge und ein Zeitfenster zur Bearbeitung festgelegt werden.
27 +)))
28 +
24 24  == Limitations ==
25 25  
26 26  Due to the low validity, post box exercises should be used in combination with other methods. This method is less suitable for querying factual knowledge. Complex practical skills are more likely to be assessed with an observation in a simulated environment or on site.
... ... @@ -48,7 +48,6 @@
48 48  ~1. The candidate processes the tasks with free formulations. The observers then evaluate the documents. This procedure is less recommendable, since the comparability and evaluation objectivity are endangered.
49 49  2. Open questions are pre-formulated for each process. The candidates answer them in interviews or a presentation.
50 50  3. Pre-formulated questions with given answers are handed out.
51 -
52 52  It should be noted that post box exercise solutions are usually not right or wrong, but more or less successful.
53 53  
54 54  ----