Home  /  Publications  /  Audits of Recruitment and Selection Activity  /  Appointment Process for the position of Executive Officer in Revenue
 

Commission’s Response to the Allegations

The Commission received four requests for review from candidates following the notification of their results of the online tests.  The allegations of breaches of the Code can be summarised as follows;

(i)            Negative Marking – It is alleged that ‘negative marking’ was used to determine the order of merit on the panel which adversely affected candidates, i.e. candidates did not have prior knowledge that marks they received for answering correct answers would be removed in the event that they answered other questions incorrectly and that the failure to provide them with this key information had an adverse impact upon them.  It is further alleged that Revenue failed to communicate the assessment mechanism in advance and there was a breach of the Code which states that ‘the approach adopted in any appointment process should be clear and evident’. 

(ii)            Insufficient Feedback– It is alleged that the feedback provided to candidates on the psychometric tests was not ‘clear, specific and meaningful’ as the service provider was not in a position to provide information in relation to the test scores awarded and how these were calculated.  It has also been alleged that candidates were not provided with information on individual performance and their specific positioning on the order of merit which is neither fair nor transparent. 

(iii)         Delays in processing review request – It has been alleged that Revenue breached the Code by not adhering to the timeframes outlined in the Code review procedures.

Commission’s Response

(i)         The Commission understands that the Licence Holder outsourced the evaluation of candidates’ performance in the online tests and that Cut-E, the service provider, had been contracted to carry out the assessment process.  It also appreciates that the Licence Holder acted on the advice of the service provider, which has vast experience in this field of online testing, in relation to the information and instructions that were provided to candidates before completing the tests.

While it notes that candidates were advised to work quickly and accurately, the Commission is of the view that the instructions to candidates could have been clearer.   The Commission appreciates that the assessment mechanism employed by the service provider was complex thus making it difficult to provide more detailed information on the evaluation process.  That said the Commission considers that best practice is to provide clear and meaningful information to candidates to assist them in preparing for the psychometric tests    and recommends that, in cases where similar tests are deployed again in the future, the instructions to candidates make it clear that their performance will be assessed based on a number of factors such as the number of correct answers and the level of accuracy involved.

The Commission believes that the provision of more explicit information may have allowed for a better understanding of the scores candidates achieved and may have helped to avoid some of the confusion among candidates in the relation to their scores.    

(ii)        The Commission is satisfied that the feedback information made available to candidates was meaningful.  The feedback not only set out what the tests were designed to measure, it provided candidates with an understanding of their performance at the tests.  The Commission has reviewed the basis for the calculation of the minimum qualifying score and accepts that the service provider cannot provide further meaningful information to candidates without compromising its intellectual property rights.

While candidates were provided with their scores on the number of completed questions and the number answered correctly the feedback did not indicate how candidates performed in comparison to the overall candidate group.  In this regard the Commission recommends that consideration is given to providing additional information to candidates on their performance relative to others which would be helpful in terms of further enhancing the level of transparency involved in the evaluation of candidates.

(iii)       The Commission notes that following notification of the online test results Revenue received a large volume of both informal and formal complaints.  It is noted that this coincided with the alleged breach of security that required immediate action. 

In light of the extenuating circumstances involved that required urgent attention to resolve the matter the Commission accepts that there were unavoidable delays by the Licence Holder in processing the complaints received.  The Commission has received assurances that the Licence Holder will endeavour to adhere to the time timeframes outlined in the Code review procedures in the future.