We are so excited to introduce you to our newest product update - question weighting now varies based on the question type!
Currently, instead of each question being worth 10 points, there is now a sliding scale based on the value and effort required to complete. For example, Simulations, Code Challenges, Documents, File Uploads, and Spreadsheets are now worth a maximum of 10 points, while simple questions (like Multiple Choice) are only worth 3 points.
Question-type weightings will help to improve the way we grade your candidates' responses and, therefore, help you to continue hiring with confidence.
What is a question type and how were they previously weighted?
Question types are all the different ways Vervoe allows you to test candidates. These formats include multiple choice, text, audio, video, file upload, presentation, spreadsheet, code, document, numbers, typing test and the Customer Service Simulator. Historically, each question type in Vervoe has been allocated the same amount of points (10), which means each question is of equal weight to a final score. To learn more about question types and how to add them to your assessment, take a look at this article here.
How will this update help my company hire the best candidate?
Over the years, our question types have improved and increased; with some now being more immersive and requiring a lot more effort from the candidate to complete, which also makes them more insightful. In order to recognize this differentiation between question types, we have introduced question weighting, which will make some question types more valuable to a score and others less. This should help you find better candidates as those that score well on highly-weighted questions will naturally rank higher in the Select stage.
Examples of new weighting:
Please find an outline of the new weightings in the image below:
- In order to trigger the new question weighting in an existing assessment there must be a new completion after the release.
- Existing assessments that do not have new completions will not be recalculated using the new weighting.
- Manual grading always overrides the AI score so candidates that have been manually graded will not recalculate
- We recommend you duplicate any rolling assessments (open assessments designed for multiple hires rather than one hire), so new scoring will be kept consistent and historic candidates do not rank for your active roles. Alternatively, you can sort candidates by completion date as a workaround to this.
An assessment has 6 questions (3 x simulations, 1 x video, 1 x text, and 1 x MC) across 3 different skill groups. In the old weighting example below, each question is worth 10 points. The candidate scored 8/10 across each question, which gives them an overall score of 80%
With the new question weighting in place, if the same candidate has earned the same percentage of points per question (80% of the maximum points, per question), they will still have the same overall percentage of 80%. Although questions are weighted differently, the overall percentage earned is what's calculated in the overall score. This means that questions with less of a value do not automatically drop the overall score:
However, if the same candidate performed better on the simulation questions but worse on the other question types, their score would improve to 84.49%:
And lastly if the same candidate performed worse on the simulations but better on the other question types, their score would be lower overall because the simulations have been weighted to be more important.