Summary

• Page properties (4 modified, 0 added, 0 removed)

Details

Page properties

Title

...	...	@@ -1,1 +1,1 @@
1		-Observation in simulated environment
	1	+Post Box Exercise

Parent

...	...	@@ -1,1 +1,1 @@
1		-Main.Sectoral Layer.Sectoral Layer Glossary.WebHome
	1	+01 Sectoral Layer.Sectoral Layer Glossary.WebHome

Author

...	...	@@ -1,1 +1,1 @@
1		-XWiki.ChrisVanGoethem
	1	+XWiki.RandellGreenlee

Content

...	...	@@ -4,42 +4,95 @@
4	4
5	5	= Description =
6	6
7		-The simulated environment reflects a real live situation, but is standardised. This makes it possible to build in incentives for behaviour or choices. The situation can be a "copy" of a real live situation, but also a roleplay (for more behaviour skills). The candidate is observed in this simulated situation.
	7	+This method is used for skills that result in a written or drawn result. The candidate gets an assignment on paper and has time to prepare the written result. The result is checked with a prepared checklist of sample solutions. Examples or results could be personnel planning, a light plot, an Email. The advantage is a high certainty of competence, compared with assessing prior work. For more artistic skills, this can be combined with a role play or an interview. This method is highly effective for assessing action orientation and entrepreneurial thinking. The candidate can act independently in a wide frame. Complex information has to be quickly analysed, processed and combined to recommend rational actions.
8	8
9		-This method is used for skills that can be shown in the workspace. The assessment method allows to test very specific competences, as the environment can be controlled. Mainly for practical, observable skills.
	9	+----
10	10
11	11	= Quality Concepts =
12	12
13	13	=== Validity ===
14	14
15		-== Reliability ==
	15	+Due to the high visual validity, the post box exercise is a popular method. It can be used to evaluate action orientation and entrepreneurial thinking well. The method is used, among other things, to evaluate work organization, decision-making ability, organizational skills, conceptual work, planning and decision-making behavior, recognizing relations, forming priorities and ability to delegate. The factors of behavior include preparation and planning for action, work performance (quantity) and the search for precise instructions.
	16	+The validity of Post Box Exercise is limited by the fact that processes can be very interdependent. To solve one problem, you have to solve a prior one.
	17	+Previous studies have shown different correlation coefficients (from -.45 to .76) for the relation between post basket result and job success (see Schippmann et al. 1990). Obermann (2018) uses a value of .18, which corresponds to a very low correlation. However, it should be noted that in the assessment context of validation, job success is only indirectly affected.
16	16
	19	+(% style="color:inherit; font-family:inherit; font-size:23px" %)Reliability
	20	+
	21	+
	22	+Reliability and evaluation objectivity are special challenges in post box exercises. A meta-analysis based on 16 individual studies showed an interrater reliability of .76 (Whetzel et al. 2014). The assessment of different raters are therefore "good" or "substantial".
	23	+Since chance hits can distort the assessment results, a sufficient number of tasks should be set. Standardization can further increase reliability. In the scenario, an order and a time window for processing should be defined.
	24	+
	25	+Reliabilität und Auswertungsobjektivität sind besondere Herausforderungen bei Post Box Exercise. Eine Metaanalyse auf Basis von 16 Einzelstudien ergab eine Interrater-Reliabilität von .76 (Whetzel et al. 2014). Die Urteile unterschiedlicher Bewerter stimmen also "gut" bzw. "substanziell" miteinander überein.
	26	+Da chance hits die Assessment Ergebnisse verzerren können, sollten ausreichend viele Aufgaben gestellt werden. Durch eine Standardisierung kann die Reliabilität zusätzlich erhöht werden. So sollte im Szenario eine Reihenfolge und ein Zeitfenster zur Bearbeitung festgelegt werden.
	27	+
	28	+|(((
	29	+
	30	+)))
	31	+
17	17	== Limitations ==
18	18
	34	+Due to the low validity, post box exercises should be used in combination with other methods. This method is less suitable for querying factual knowledge. Complex practical skills are more likely to be assessed with an observation in a simulated environment or on site.
	35	+
	36	+----
	37	+
19	19	= Considerations =
20	20
21	21	== Tips ==
22	22
	42	+|(((
	43	+The more realistic the presentation of the documents and the settings, the higher the tasks will be accepted.
	44	+After the written processing, an oral discussion should follow. Otherwise, excellent ideas for task processing that deviate from the proposed solutions may not be recognized accordingly. In the debriefing, the candidate can resolve ambiguities and answer questions.
	45	+)))
	46	+
23	23	== Traps ==
24	24
	49	+One of the traps is that the assignments could be too easy or too difficult.
	50	+The solvability of the processes should not depend too much on one another. On the other hand, the documents should be linked to each other in a complex manner, otherwise the analysis level will remain at a low level.
	51	+If the processing time is too short, the competence is overemphasized how quickly texts are recorded.
	52	+
25	25	== Scoring Tools ==
26	26
	55	+There are three options for evaluation:
	56	+~1. The candidate processes the tasks with free formulations. The observers then evaluate the documents. This procedure is less recommendable, since the comparability and evaluation objectivity are endangered.
	57	+2. Open questions are pre-formulated for each process. The candidates answer them in interviews or a presentation.
	58	+3. Pre-formulated questions with given answers are handed out.
	59	+It should be noted that post box exercise solutions are usually not right or wrong, but more or less successful.
	60	+
	61	+----
	62	+
27	27	= Implementation =
28	28
29	29	== Information for Standard ==
30	30
	67	+The tasks and scheduled processing time should not vary compared to other candidates. Possible language barriers should be considered. The answers should be evaluated according to a marking guide.
	68	+
31	31	== Development ==
32	32
	71	+An appropriate task difficulty must be chosen for the development. The planned time for the tasks must not be too short. The main focus during development is on an extensive and diversified creation of tasks that are as close as possible to the daily work environment.
	72	+
33	33	== Needs/Set-Up ==
34	34
	75	+In addition to an examination room with chair and work table, the correspondingly prepared equipment is required. This must be sorted according to the intended order of processing.
	76	+
35	35	== Requirements for Assessors ==
36	36
	79	+Assessors need appropriate pedagogical and technical knowledge to prepare the tasks and evaluate the approaches and results.
	80	+
37	37	== Examples ==
38	38
	83	+A possible Post Box Exercise to test management skills is a combination of scheduling, consultation with technicians and email correspondence.
	84	+
39	39	== In Combination with ==
40	40
	87	+To check the theoretical knowledge, written tests or oral tests should also be conducted.
	88	+
41	41	= References/Notes =
42	42
	91	+* Eck, C. et al. (2016): Assessment-Center. Entwicklung und Anwendung – mit 57 AC-Aufgaben und Checklisten zum Downloaden und Bearbeiten im Internet. 3. Aufl. Berlin / Heidelberg: Springer.
	92	+* Obermann, C. (2018): Assessment Center. Entwicklung, Durchführung, Trends. Mit neuen originalen AC-Übungen. 6., vollständig überarb. u. erw. Aufl. Wiesbaden: Springer Fachmedien.
	93	+* Whetzel DL, Rotenberry PF, McDaniel MA (2014): In-basket Validity: a systematic review. Int J Sel Assess 22(1):62–79.
	94	+* Schippmann, J.S., Prien, E.P. and Katz, J.A. (1990): RELIABILITY AND VALIDITY OF IN‐BASKET PERFORMANCE MEASURES. Personnel Psychology, 43:837-859.
	95	+
43	43	== AT ==
44	44
45	45	== BE ==