Summary

• Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

...	...	@@ -24,15 +24,28 @@
24	24
25	25	=== Reliability ===
26	26
27		-The quality of simulated environment observation depends on the accuracy and repeatability of the test setup.
28		-Simulated environments guarantee equal treatment of candidates, the result should be identical, wherever and by whatever assessors they are conducted. Therefore every candidate is assessed in an identical situation.
29		-One of the elements in this are well trained assessors and a levelling system, this avoids that assessment would be biased by assessors influenced by previous tests or looking outside the competences to for example not occupation related behaviour.
30		-The reliability is increased by the possibility to easily develop exact observable criteria.
	27	+|(((
	28	+The role play should be constructed in such a way that the competences to be measured should not be reflected in a single situation. Otherwise there is a risk that the candidate will accidentally behave correctly. If several situations are to reveal the same competencies, the reliability is increased.
31	31
	30	+The situations and the start of the conversation should be clearly worded. Unclear formulations can lead to the candidate misunderstanding the task and situation, which disrupts reliability. Language barriers should also be taken into account accordingly. The starting position is identical for each candidate if the scenario is well developed. But the path can be different and contain unpredicted situations. The assessor needs to find a balance to steer on one hand, but leave the initiative to the candidate on the other.
	31	+
	32	+Various factors can limit the candidate's concentration and performance:
	33	+
	34	+poor room lighting or ventilation, disturbing sources of noise, lack of equipment, poor health or psychological condition of the candidate (e.g. fear, stress).
	35	+
	36	+The reliability is increased by not changing the order of the methods. In this way, the assessment processes remain comparable.
	37	+
	38	+The assessors / observers should be extensively trained to ensure evaluation and interpretation, to ensure comparability and objectivity. It should be clear which behaviors can be attributed to certain skills. So, reliability is enforced by a good scoring structure.
	39	+
	40	+The behavior of the interlocutors is standardized by means of a written instruction. This is a prerequisite for objectivity, which in turn ensures that different results are due to the performance of the participants and not to variations in the interlocutors.
	41	+)))
	42	+
32	32	== Limitations ==
33	33
34		-Development of a assessment set-up is time consuming.
	45	+Role plays are less suitable for assessing knowledge-based skills. Likewise, no competencies are expressed that are expressed in written and drawn results. The methodology is less usefull for observing physical skills, exept if combined with an observation in a simulated environment.
35	35
	47	+To check how the candidate behaves in a non-simulated environment, real-life observations should be preferred. A good scenario should be able to bring forward knowledge based skills.
	48	+
36	36	----
37	37
38	38	= Considerations =
...	...	@@ -39,25 +39,25 @@
39	39
40	40	== Tips ==
41	41
42		-Organise the test in a way that the candidate feels at ease. If it is a tradition to have a cup of coffee at the start of a working day, include this in the startup of the test.
43		-Give the candidate time to discover the situation.
44		-Do not built in traps or tricky situations that hardly ever occur in real life.
45		-Be clear and open about the role and activities of the observers. Attention points can be:
	55	+|(((
	56	+Educate the assessee that their acting performance is not being tested. He should behave as he would in everyday work. The atmosphere should be as realistic as possible. Assessee and interlocutors (= role players) need prior written information. This gives the other party information on how to react to certain situations and questions. The person watching should not have direct eye contact with the assessee. In advance, it is recommended for observer training to practice role-playing with video analysis. Assessors need to be trained for the specific scenario's, idealy in interaction with their collegues.
46	46
47		-* Observers write also about positive points.
48		-* Observers are silent, because they keep a distance.
49		-* Observers will only stop the test in case of danger or overtime.
	58	+It can help to get the candidate on the right track to use some properties (assessories) to visualise the role of the actor (for example a typical piece of clothing, documents, models, ...). Make clear the candidate is playing his/her own role in the given situation.
	59	+)))
50	50
51	51	== Traps ==
52	52
53		-If the candidate needs support, the assistant must be trained to limit the intervention to what the candidate requires and not (as we would do in reality) to take over the decision-making process or be proactive.
54		-There is a risk that the assessor is biased. That is why assessors should be professionals from the field of competences being assessed. Assessors have to be aware that there are different methods to perform a specific task and should take distance from one prefered method, for as far as the goal is reached.
	63	+The situation characteristics from the requirements analysis cannot be implemented carefully enough. This can have the consequence that the behavior is not relevant for the competence to be tested. Sentences like "In reality I would do it completely differently." are signs of this. There is a risk for socialy expected answers and behavior.
55	55
	65	+If the assessor has no experience with role-playing games, an unpredictable momentum can develop which affects the comparability. Assessors need to be aware of the reasons for all scenario steps and have to balance natural behaviour and following the storyline of the scenario that leads to the visibility of the skills. Errors can be a high level of willingness to compromise or hardness. Principle of local independence: The interview partner must act neutrally in the individual situations, even if the assessee had a poor start in a previous situation. Otherwise only the overall performance can be assessed, but not individual competencies. Avoid that the same assessor is playing different roles for the same candidate.
	66	+
	67	+A short instruction in the role play overemphasizes the situational flexibility. Communicative observations can take place less.
	68	+
	69	+Uncertainties about the setting (e.g. Has the exercise already started? Who ends the conversation?) should be avoided. Other participants should not act as conversation partners as this does not standardize the exercise. An assessor can't be the role player at the same time.
	70	+
56	56	== Scoring Tools ==
57	57
58		-Observing can be done through a list of observable criteria. The criteria should be derived from the sectoral layer skills, in other words, they are a concretisation of the visible, observable result of the skill in a specific situation.
59		-As the situation is always identical, the scoring tool can be very specific and leave little room for interpretation.
60		-The final decision is made based on the link of the criteria with the competence and by comparing the observations of the different assessors.
	73	+There could be used a check List and notes about the behaviour of the candidat with reference to the expected behaviour. The criteria should be derived from the sectoral layer skills, in other words, they are a concretisation of the visible, observable result of the skill in a specific situation. As the situation can differ, the scoring tool will be more general and leave more room for interpretation than other methods.
61	61
62	62	----
63	63