When creating or editing skills assessments, understanding the available question settings is crucial. These settings influence how questions are scored, how AI evaluates responses, and the overall candidate experience. Below is a breakdown of each setting and how to use it effectively.
Skill Assessed
Each question is linked to a specific skill, determining how responses contribute to the candidate’s score within that skill group. The skill field is pre-filled but can be updated. To change the skill or grouping, navigate to Assessment Settings > Skill Groups in the top-right corner of the assessment builder. From there, you can rename skills or reassign questions to different groups.
Organising and Customising Your Assessment with Skill Groups
Example of a Correct Answer
Providing example answers for text, audio, or video responses is vital. These examples help train AI on what constitutes a strong response, improving grading accuracy. Without example answers, assessments for these question types may not perform correctly. You can provide one or multiple example answers to accommodate different correct response formats.
Scoring Instructions
For assessments with multiple reviewers, the Scoring Instructions field ensures consistency in grading. This is particularly helpful for open-ended or creative questions where interpretation may vary. Include notes on what to look for in a strong response to guide your team and maintain standardised evaluations.
Scoring instructions are available under each question. These guidelines are key to maintaining consistency in grading across multiple reviewers. Use the rich text editor to detail what constitutes a perfect response (e.g., 0/10 to 10/10). This ensures all reviewers align on how to score responses fairly.
How Scoring Instructions Are Displayed
-
Manual Grading: When reviewing one candidate at a time, scoring instructions are shown below each question.
-
AI Grading: In the AI grading portal, instructions appear in the right-hand panel.
Important Notes
-
Scoring instructions are for guidance only and do not affect AI grading.
-
They are intended solely for manual grading to ensure consistency across reviewers and do not influence the candidate's final score.
Question Settings
These controls shape both the candidate experience and the grading process:
-
Timer: Set a time limit for the question. Be cautious—if a required, timed question remains unanswered, candidates may be blocked from submitting.
-
Required: Marks the question as mandatory before moving on. Avoid using this for timed questions to prevent submission issues.
-
Plagiarism Check: Flags potential plagiarism in text-based responses.
-
Exclude from Grading: Use for questions that don’t need scoring, such as instructions or introductions. These won’t affect the overall score.
-
Manual Grading: Allows for manual review and scoring of the question. AI will not score these responses, and the overall assessment score won’t be available until all manually graded questions are completed.
Previewing Your Assessment
You can preview your assessment at any time by clicking the Preview button in the top-right corner. This lets you see the candidate experience and ensure everything functions correctly.