Uploading Candidate Responses

Summary

As an exam administrator, there may be occasions when you need to upload candidate response data to Practique. For example, if the Exam has been delivered offline using paper and the responses need to be manually uploaded into Practique for validation and marking, or if the results need to be updated or overridden after the Exam has been delivered. These changes can be carried out in bulk using the Upload Responses feature.

User Guide

To access this feature, navigate to Exams and then click view sessions for the required exam. Publish the exam, navigate to the Dashboard and start the session, then stop the session. This will put the exam into Marking state.

In the View Sessions page, click Upload Responses

The upload screen shows several different options:

Select the file from your computer to upload
Specify the format of the data in the file
1. Responses only: This is for use when the dataset contains the raw answer options selected by candidates (e.g. A, B, C etc.)
2. Scores only: Where the file contains scoring values from the validated responses (e.g. 0, 1, 2 etc.)
3. Scores for text questions, responses for others: To be used where tests contain a mixture of question types. Questions that have a defined set of answer options can receive the responses data whereas open-ended questions will receive the scoring information.
Allow empty scores: This option allows you to choose whether Practique should reject the file if it detects that the dataset is incomplete. Please note that this is ignored for “responses only” uploads.
Upload the selected file
Download a template file for population
Exit and return to the previous screen

Uploading responses only

To upload a new set of responses these must be organised into a spreadsheet first. Practique has strict validation rules that must be adhered to if you are to achieve a successful upload (detailed on the upload page), but the simplest way to ensure compliance is to first download a template file.

Formatting data for upload

The upload template is presented with all the registered candidates in Column A and all of the questions (or sub-criteria) for which Practique expects a response in Row 1.

The formatting of the sheet may differ depending on the structure of the questions included in the exam and the data expected to be returned. For example:

Single selectable answer type (e.g. single best answer, assertion-response questions): A single answer key entry in the appropriate cell (e.g. “A”)
Multiple selectable answer type (e.g. multiple acceptable answers, clinical prioritisation questions): Several comma-separated answer keys in the appropriate cell (e.g. “A,B,D”)
Open-ended text input (e.g. short answer questions, very short answer questions): A text string in the appropriate cell (e.g. “This is my answer…”)
Multi-part questions (e.g. Extended matching, multiple true/false, OSCE): Single answer keys spread over several columns corresponding with the number of criteria present in the question.

The below example shows how the spreadsheet might look if you were to update a single SBA item response for candidate 2 on question 2.

The upload process

Once the spreadsheet is ready, use the Upload File button to import the information to Practique.

Practique will validate the file and, where there is no existing response recorded, the system will populate the missing information as appropriate. Where a response does already exist and the information in the uploaded spreadsheet is different, the uploaded information will overwrite the response that was already in Practique.

Once the uploaded information has been processed, and as a final step, Practique will offer you the option to revalidate and reprocess the results to bring them up to date.

Uploading scores only

Uploading scoring data allows you to change the number of marks achieved by candidates on items but without adjusting the actual answer key they selected. For question types with a defined set of selectable responses this could be useful if, for example, you wish to assign additional credit where some ambiguity or error in a question is discovered after the test.

There are other possible applications for the upload scores feature that are more useful for different question types. For example, in open-ended style questions, there are no selectable responses so scores that have been generated via any offline marking activity that might have taken place could be uploaded in this way.

The process and the data organisation are very similar to that described in the previous section, only rather than uploading answer keys, you would instead upload numerical score values in your dataset. Practique will validate the uploaded file to determine whether it matches the marking schema applied to each item and, if validation is successful, will update any changed or missing score values.

Note: Please note that when uploading scoring data, the “Allow empty scores” field also gives you the option to instruct Practique to reject the file if it does not detect a full dataset with a score for every candidate on every question.

Uploading scores and responses for OSCEs

In the case of OSCEs, the process for uploading information post-hoc is slightly different to a written exam. Whereas a written exam tends to generally produce a single dataset, OSCEs have data generated from multiple different stations. This means that any changes in OSCEs must be uploaded and processed on a station-by-station basis, and you will be prompted to select the station you want to upload to during the first step of the upload process.

OSCE stations also differ slightly from some of the previous examples because they tend to have several measurement points contained within a single item. The data structure for upload to an OSCE station will also, therefore, vary depending on its design.

Again, the easiest way to ensure compliance with the validation rules is to use the template offered by Practique. The below shows a reasonably straightforward station template with three scoring domains, a global mark and a free text feedback field. In this example, a complete dataset is being uploaded for all three candidates against all measurement points for that station.

Notes:

As a note of caution, the upload responses feature makes it possible for users to make extensive changes to candidate submissions in relatively few steps. Therefore we advise that the instructions that follow are applied with the utmost care.
When uploading scoring data, the Allow empty scores field also gives you the option to instruct Practique to reject the file if it does not detect a full dataset with a score for every candidate on every question.

Keywords: Upload, responses, results processing