Changes for page Criterion Based Interview
Last modified by Ivan Kharitonov on 2022/01/30 17:24
Change comment:
There is no comment for this version
Summary
-
Page properties (1 modified, 0 added, 0 removed)
Details
- Page properties
-
- Content
-
... ... @@ -17,20 +17,26 @@ 17 17 === Validity === 18 18 19 19 |((( 20 - In theCriterionBased Interview,thebasicsumptionisthatpastbehaviorallows apredictionforfuture behavior.Thevalidity dependshisbasic assumption. The followingvalue demonstrates the prediction rateforjobsuccessthroughthe assessmentmethod. It shouldbenotedthat in the assessment context,jobsuccess isonlyindirectlyrelevant inthe context ofvalidation.20 +The validity of the method is increased by the tasks being appropriate for the competences to be measured. Above all, interpersonal and artistic skills can be assessed with the role play. The following communicative and social skills can be assessed: interviewing, rhetoric, reasoning, empathy, assertiveness, persuasiveness and sensitivity. In order to increase the validity, the selected role-playing situations should be representative of the event technology. It is important to make the situation challenging and relevant. The situations need to occur regularly in real life. As the scenario only reflects one concrete situation there is a risk that it doesn't reflect all situations in professional practice. However, the method is close to real life and is well controllable. 21 21 22 - Standardized interviews have a validityof.7122 +A particular challenge for the validity of the role play is the requirement on the candidate to be active as an actor. Care should be taken here to make it clear that the acting performance is not assessed. The candidate should be able to develop his role freely and without pressure in order to be able to express his inherent competencies. 23 23 24 -(Obermann 2018: 114). According to Schmidt & Hunter (2000), the prognostic validity of structured interviews is .51. 25 - 26 -The candidate wants to sell himself in the best way, which limits the validity. 24 +According to Obermann (2018), the relationship between job success and the results of role play is .16. This corresponds to a very low correlation. It should be noted, however, that in the assessment context, job success is only indirectly relevant in the context of validation. 27 27 ))) 28 28 29 29 === Reliability === 30 30 31 31 |((( 32 - Forhiring interviews,the interraterreliabilityis.68,which correspondso a goodorsubstantialreliability.Even iftheassessment contextinthevalidationprocessdiffersfromthe recruitmentprocess,it cancertainly be saidthatthere isgoodreliabilitywhenthe criteria-basednterviewsarecarriedout correctly.Thereliabilityisncreased bythe questionsbeing pre-formulatedin advance andthe wordingremainingthe same.This increases comparability with other interviews.To ensure objectivity in the evaluation,theinterviewsshould be recordedand evaluated byseveral assessors.30 +The role play should be constructed in such a way that the competences to be measured should not be reflected in a single situation. Otherwise there is a risk that the candidate will accidentally behave correctly. If several situations are to reveal the same competencies, the reliability is increased. 33 33 32 +The situations and the start of the conversation should be clearly worded. Unclear formulations can lead to the candidate misunderstanding the task and situation, which disrupts reliability. Language barriers should also be taken into account accordingly. The starting position is identical for each candidate if the scenario is well developed. But the path can be different and contain unpredicted situations. The assessor needs to find a balance to steer on one hand, but leave the initiative to the candidate on the other. 33 + 34 +Various factors can limit the candidate's concentration and performance: 35 + 36 +poor room lighting or ventilation, disturbing sources of noise, lack of equipment, poor health or psychological condition of the candidate (e.g. fear, stress). 37 + 38 +The reliability is increased by not changing the order of the methods. In this way, the assessment processes remain comparable. 39 + 34 34 The assessors / observers should be extensively trained to ensure evaluation and interpretation, to ensure comparability and objectivity. It should be clear which behaviors can be attributed to certain skills. So, reliability is enforced by a good scoring structure. 35 35 36 36 The behavior of the interlocutors is standardized by means of a written instruction. This is a prerequisite for objectivity, which in turn ensures that different results are due to the performance of the participants and not to variations in the interlocutors. ... ... @@ -38,9 +38,9 @@ 38 38 39 39 == Limitations == 40 40 41 - The interviewdoes notallowto observe/confirm thecapacityofthe participantto actuallycarryouta task. The method isveryintensiveandtime consumingfor assessors.47 +Role plays are less suitable for assessing knowledge-based skills. Likewise, no competencies are expressed that are expressed in written and drawn results. The methodology is less usefull for observing physical skills, exept if combined with an observation in a simulated environment. 42 42 43 - It canonlybeusedfor amitedset of competences.49 +To check how the candidate behaves in a non-simulated environment, real-life observations should be preferred. A good scenario should be able to bring forward knowledge based skills. 44 44 45 45 ---- 46 46 ... ... @@ -49,34 +49,25 @@ 49 49 == Tips == 50 50 51 51 |((( 52 - The interviewshouldfollowabiographical approach,which helpsthe assessor tounderstandthecareer path of the participant.58 +Educate the assessee that their acting performance is not being tested. He should behave as he would in everyday work. The atmosphere should be as realistic as possible. Assessee and interlocutors (= role players) need prior written information. This gives the other party information on how to react to certain situations and questions. The person watching should not have direct eye contact with the assessee. In advance, it is recommended for observer training to practice role-playing with video analysis. Assessors need to be trained for the specific scenario's, idealy in interaction with their collegues. 53 53 54 -Ensure that all assessors use the same path/structure. The variations should be limited. Continue questioning untill you reach the desired focus. Use simple open questions (e.g. who, when, where, why, how, for how long, …). Don't judge in your answers or sub questions. Listen actively, show interest, listen to what is said in between the lines, ask deeper if needed, but give the candidate the lead. 55 - 56 -Disruptive factors should be minimized by a high degree of standardization. 60 +It can help to get the candidate on the right track to use some properties (assessories) to visualise the role of the actor (for example a typical piece of clothing, documents, models, ...). Make clear the candidate is playing his/her own role in the given situation. 57 57 ))) 58 58 59 59 == Traps == 60 60 61 -|((( 62 -The personal interests and opinions of the interviewer can interfere with the interview process (bias). To avoid this distortion, the interviewer has to make himself aware of his / her perception tendencies. Attractiveness, size, eloquence, humor, clothing style, nervousness and gender 65 +The situation characteristics from the requirements analysis cannot be implemented carefully enough. This can have the consequence that the behavior is not relevant for the competence to be tested. Sentences like "In reality I would do it completely differently." are signs of this. There is a risk for socialy expected answers and behavior. 63 63 64 - of theparticipantcaninfluence the assessment of the assessors,even though they have no information about the tested competence.Thecontextcanalso influence the candidate.67 +If the assessor has no experience with role-playing games, an unpredictable momentum can develop which affects the comparability. Assessors need to be aware of the reasons for all scenario steps and have to balance natural behaviour and following the storyline of the scenario that leads to the visibility of the skills. Errors can be a high level of willingness to compromise or hardness. Principle of local independence: The interview partner must act neutrally in the individual situations, even if the assessee had a poor start in a previous situation. Otherwise only the overall performance can be assessed, but not individual competencies. Avoid that the same assessor is playing different roles for the same candidate. 65 65 66 - Itshould be notedthat this interview shouldnotassumethe roleofanoral test,eventhoughthe boundariesmightbe blurryatsomepoints.69 +A short instruction in the role play overemphasizes the situational flexibility. Communicative observations can take place less. 67 67 68 -There is a risk that, especially in sub questions, the assessor suggests the answers unconsciously. The direction of the interview is depending on the candidate and the sub questions of the assessors. 69 -))) 71 +Uncertainties about the setting (e.g. Has the exercise already started? Who ends the conversation?) should be avoided. Other participants should not act as conversation partners as this does not standardize the exercise. An assessor can't be the role player at the same time. 70 70 71 71 == Scoring Tools == 72 72 73 - Before the interview, the assessor determineswhich75 +There could be used a check List and notes about the behaviour of the candidate with reference to the expected behaviour. The criteria should be derived from the sectoral layer skills, in other words, they are a concretization of the visible, observable result of the skill in a specific situation. As the situation can differ, the scoring tool will be more general and leave more room for interpretation than other methods. 74 74 75 -answers to the questions receive positive and which answers receive negative points. The "positive indicators" and the "negative indicators" can be compared in a table. The "negative indicators" can also be divided into "minor negative indicators" and "decisive negative indicators". The respective extent to which the indicators match with the answers is rated on a scale of 0-4 (0 = no evidence; 1 = poor; 2 = areas of concern; 3 = satisfactory; 4 = good to excellent). A good list of what is expected needs to be used. The list should reflect the skills assessed. 76 - 77 - 78 -If the method is used as backup, scoring lists from other methods can be used. 79 - 80 80 ---- 81 81 82 82 = Implementation = ... ... @@ -83,48 +83,33 @@ 83 83 84 84 == Information for Standard == 85 85 86 - If theinterview is foreseen in the assessment, the context,the focus,the time andcriteriaaredescribed. Ifthe methodis usedas backup, itshould bementionedwith other methods.83 +The method should include a short description and the expected behaviour. 87 87 88 88 == Development == 89 89 90 -|((( 91 -The main development is in the training of the assessors to use the method. But some scheme with questions and sub questions can support the assessors. 87 +The development of an assessment scenario starts from the analysis of the skills that need to be assessed. The scenario reflects a real life situation in which the tested skills are prominently present. Based on the scenario, a list of observable criteria, derived from the competences is created. 92 92 93 -The method is based on an interviewing technique using principles of the STARR method: 94 - 95 -* S(ituation): What was the situation? - description of a past work situation 96 -* T(ask): What was your task? - clarification of the responsibilities of the candidate 97 -* A(ction): What actions did you take, what did you do? - explanation of the performed action 98 -* R(esult): What was the result, what happened? - statement about the results 99 -* R(eflection): What did you learn? - evaluation of the situation from his/her perspective today. 100 -))) 101 -|((( 102 - 103 -))) 104 - 105 105 == Needs/Set-Up == 106 106 107 - The interviewneedsobeconductedbytwoassessorsinordertoavoidbiasf the result(sympathy etc.)or mightevenrequireawrittentranscriptforevaluation.Itneeds a quietroom(withatable,chairs,...) andplenty oftime.91 +In most cases a quiet space with a table and some chairs is all that is needed. Depending on the test, documentation can be needed (light plots, sound documentation, …). 108 108 109 109 == Requirements for Assessors == 110 110 111 - The assessorneeds basicskillsin conducting unbiased interviews.He needsprofessional skills in ordertobeabletodeduce appropriate competenciesfrom thecandidate's report.95 +Assessors must be able to instruct the actors according to the situation. As observers, they need skills in observing, diagnosing and evaluating the candidate's behavior. The assessment of competencies requires professional expertise. 112 112 113 113 == Examples == 114 114 115 -The situationfortheinterview,which isassumed,maybe the observance ofsafetyprecautions duringthe planningofprevious lightsettings.99 +The candidate has a discussion with a designer about a design concept that needs to be translated into a practical plan. In this discussion e.g. conceptual and artistical understanding, technical problem solving and predicting problems and language use can be assessed. 116 116 117 117 == In Combination with == 118 118 119 - |Apost-box exerciseorroleplay wouldberequired inaddition.Oftenacompetencefocussedinterviewisused asbackupifothertestdidn'tshowcertain competences(notpositive ornegative).103 +Role Plays can be used in combination with a post box exercise, where the information needed is not (or limited) put on paper, but the counterpart is presenting the information and the candidate can ask questions about the required results. It can be complemented with a criterium focused interview for skills that didn't become visible. 120 120 121 121 = References/Notes = 122 122 123 123 * Catalogus Assessmentmethodes voor EVC, Agentschap Hoger Onderwijs, volwassenenonderwijs, Kwalificaties en Studietoelagen, Ministery of education and training of the Flemish community (2015). Online: [[http:~~/~~/www.erkennenvancompetenties.be/evc-professionals/evctoolbox/bestanden/catalogus-assessmentmethodes-evc-2015.pdf>>http://www.erkennenvancompetenties.be/evc-professionals/evctoolbox/bestanden/catalogus-assessmentmethodes-evc-2015.pdf]]. (last 17.08.2020) 124 -* ISC Professional (2016): Competency-Based Interviews. Online: [[https:~~/~~/www.interview-skills.co.uk/freeinformation/interview-guide/competency-basedinterviews>>https://www.interview-skills.co.uk/freeinformation/interview-guide/competency-basedinterviews]] (last 27.7.2020) 125 -* Obermann, C. (2018): Assessment Center. Entwicklung, Durchführung, Trends. Mit neuen originalen AC-Übungen. 126 -6., vollständig überarb. u. erw. Aufl. Wiesbaden: Springer Fachmedien. 127 -* Schmidt, F. L. & Hunter, J. E. (2000). Messbare Personenmerkmale:Stabilität, Variabilität und Validität zur Vorhersage zukünftiger Berufsleistung und berufsbezogenen Lernens. In: M. Kleinmann & B. Strauß (Hrsg.), Potentialfeststellung und Personalentwicklung (S. 15–43). Göttingen: Verlag für Angewandte Psychologie. 108 +* Eck, C. et al. (2016): Assessment-Center. Entwicklung und Anwendung – mit 57 AC-Aufgaben und Checklisten zum Downloaden und Bearbeiten im Internet. 3. Aufl. Berlin / Heidelberg: Springer. 109 +* Obermann, C. (2018): Assessment Center. Entwicklung, Durchführung, Trends. Mit neuen originalen AC-Übungen. 6., vollständig überarb. u. erw. Aufl. Wiesbaden: Springer Fachmedien. 128 128 129 129 == AT == 130 130