Role Play

Version 10.2 von Randell Greenlee am 2021/11/09 09:49
Contents

Description

The candidate is placed in a situation with an actor as counterpart. The actor steers the situation, based on a predefined scenario, passing specific realistic situations. Observation is done based on a checklist.

The role play is effective for testing communicative and social competences like interviewing, rhetoric, argumentation, empathy, assertiveness, persuasiveness, sensitivity (behavioral observation). It's also useful for assessing operational readiness, goal orientation, frustration tolerance, persistence, problem solving skills, analytic skills, decision making skills. and. It can test the skill of adapting communication and language to different target groups. The setting of the role play can be a difficult collaborator interview, a conflict with a leading person, a critical incident on the workplace, a discussion with a designer or a discussion with a technical director. 

Quality Concepts

Validity

The validity of the method is increased by the tasks being appropriate for the competences to be measured. Above all, interpersonal and artistic skills can be assessed with the role play. The following communicative and social skills can be assessed: interviewing, rhetoric, reasoning, empathy, assertiveness, persuasiveness and sensitivity. In order to increase the validity, the selected role-playing situations should be representative of the event technology. It is important to make the situation challenging and relevant. The situations need to occur regularly in real life. As the scenario only reflects one concrete situation there is a risk that it doesn't reflect all situations in professional practice. However, the method is close to real life and is well controllable.

A particular challenge for the validity of the role play is the requirement on the candidate to be active as an actor. Care should be taken here to make it clear that the acting performance is not assessed. The candidate should be able to develop his role freely and without pressure in order to be able to express his inherent competencies.

According to Obermann (2018), the relationship between job success and the results of role play is .16. This corresponds to a very low correlation. It should be noted, however, that in the assessment context, job success is only indirectly relevant in the context of validation.

Reliability

The quality of simulated environment observation depends on the accuracy and repeatability of the test setup.
Simulated environments guarantee equal treatment of candidates, the result should be identical, wherever and by whatever assessors they are conducted. Therefore every candidate is assessed in an identical situation.
One of the elements in this are well trained assessors and a levelling system, this avoids that assessment would be biased by assessors influenced by previous tests or looking outside the competences to for example not occupation related behaviour.
The reliability is increased by the possibility to easily develop exact observable criteria.

Limitations

Development of a assessment set-up is time consuming.

Considerations

Tips

Organise the test in a way that the candidate feels at ease. If it is a tradition to have a cup of coffee at the start of a working day, include this in the startup of the test.
Give the candidate time to discover the situation.
Do not built in traps or tricky situations that hardly ever occur in real life.
Be clear and open about the role and activities of the observers. Attention points can be:

  • Observers write also about positive points.
  • Observers are silent, because they keep a distance.
  • Observers will only stop the test in case of danger or overtime.

Traps

If the candidate needs support, the assistant must be trained to limit the intervention to what the candidate requires and not (as we would do in reality) to take over the decision-making process or be proactive.
There is a risk that the assessor is biased. That is why assessors should be professionals from the field of competences being assessed. Assessors have to be aware that there are different methods to perform a specific task and should take distance from one prefered method, for as far as the goal is reached.

Scoring Tools

Observing can be done through a list of observable criteria. The criteria should be derived from the sectoral layer skills, in other words, they are a concretisation of the visible, observable result of the skill in a specific situation.
As the situation is always identical, the scoring tool can be very specific and leave little room for interpretation.
The final decision is made based on the link of the criteria with the competence and by comparing the observations of the different assessors.

Implementation

Information for Standard

The standard must describe the specific situations, incentives and expected complexity of the skills to be assessed.

Development

The development of an observation in a simulated environment starts with the analysis of the skills that need to be evaluated. Since not every skill can be tested in all variations, representative situations are chosen to reflect the mastery of the general skill. The skills are built into a well-chosen scenario that reflects a real-life experience, but also integrates behavioural incentives and choices. The candidate is asked to perform a task, but the environment limits or alters the way the task is performed. In this way, the candidate must make his/her own decisions.  
The activities should reflect different contexts. Often a skill or behavior is built in twice to improve reliability and avoid "false positives".
Assessment facilities must be tested and updated before they are used with "real" candidates.

Needs/Set-Up

This is an observation in a “real life” professional setting. It must be organized as a normal day in the life of the candidate (= working day). One assessor could be acting as a “colleague” the other would assess from a distance. There could also be trained “colleagues” (must not have an assessor qualification), who “work with” the candidate in the observation environment. This is only necessary when a colleague is “physically” necessary to assess the competence at hand. One assessor can't oversee all activities, idealy there are at least two assessors, one who is observing from a distance and a second one observing close.
Technical competence is relatively easy to assess. Knowledge behind the action can be assessed in most cases, if the test is prepared in the proper way. Competences are tested in the “group” working environment, as it is in reality. Several competences can almost always be assessed at one time. The proper atmosphere is very important.
The assessments could be done at educational institutions with the necessary equipment.

Requirements for Assessors

Assessors need competences for valid observations, such as those that can be acquired in observer training courses. They should have a basic knowledge of diagnostics, be able to deal with perceptual effects (e.g. errors of observation and assessment) and be able to recognize their own subjectivity. A professional competence is essential for the evaluation of the candidate's performance against the background of the assessment standard. It is also needed to construct a work situation appropriate to the competences to be assessed.

Examples

For the skill "Working on heights" a candidate should perform several activities on ladders, scaffolding, … Based on a checklist, his/her behaviour is observed.

In Combination with

This Method can be combined with a criterion focused interviews to fill the gaps or skills that have not been observed (not negative or positive). It can be combined with a multiple choice or open answer test for knowledge that is not made visible in practice.

References/Notes

AT

BE

DE

IT

NL

Tags:
Erstellt von Randell Greenlee am 2021/11/09 16:43