| Theme | Summary | Features | Applicable Exam Types | Schedule |
|---|
| 1 | Examiner Availability Calendar | This would allow users to recruit and book OSCE Examiners for future exams, at any point. | - When scheduling an exam, you would be able to pull in examiner allocations from the calendar
| OSCE |
|
| 2 | Standard Setting | We would like to improve the Standard Setting workflow and add some new features. | - Enhance live standard setting information while and exam is in progress
- Allow the import of Ebel judging from spreadsheets
- Allow Examiner-performance rescalling (hawk/dove
- Have exam blueprint based Ebel grids
- Allow mark re-scaling to university standards
- Ability to sort standard setting screen by all fields PSD-256
- Show passing and failing students (column in the download) PSD-256
- Improvements to the UI of setting exam standard
- Add Rasch modelling
- Show negative -1, -2 SEM in Standard setting
| Written, OSCE |
|
| 3 | Exports/Import | We are going to improve the way that data is imported to and exported from Practique. | - Consolidation of the structure of exam data exports
- Allow filtered candidates and examiners to be exported
- Add the export of candidate and examiner scheduleing
- Separate the import of items permissions
- Allow the export of Item Sets with Item IDs and blueprint tagging
- Simplify CSV field mapping
- Auto detect CSV encoding in upload
- Allow filtered Item bank export
- Add Max score per station, and max score per exam in the CSV export of exams
- Improve naming convention of results download
- Allow export of Standard Setting question scores
- Include Examiner ID in all results exports
| Written, OSCE |
|
| 4 | iPad exam settings | We are going to improve the way that users can install and update the iPad application on their devices. | - Control the settings for the iPad app on a per-exam basis (examiners editing marks, rotation management)
| OSCE |
|
| 5 | OSCE Marksheet Improvements | We are going to improve the way that OSCE marksheets are created and displayed on the iPads | - Improve the way that marksheets are created, by
- Allowing schemas to be searched for, and deleted
- Improve marksheet 'Count As' tagging review process
- Warn users if global criterias are incorrectly set up
- Marking Schema instructions hidden in marking buttons
- Improve the content of the marksheet by
- Allowing photos to be captured on the iPad
- Improve the way that structured feedback can be recorded
- Adding a sliding scale observation criteria
- Allowing custom text to be inserted on the marksheet
- Improve the way that generic feedback statements can be recorded
- Linking from the mark sheet to resources - Ability to embed links in observation criteria that will open specific resources attached to item.
- Allow styling (bold, italic etc) in observation criteria
| OSCE |
|
| 6 | Global Marksheet Improvements | We will be changing the Global Marksheet wokflow to improve versioning and control | - Investigate versioning improvements to the Global Marksheet
| OSCE |
|
| 7 | Custom Marksheet Validation | We will allow custom rules to be set for the marksheet. | - Configure maximum/minimum number of words recorded
- Set rules for mandatory information. For example, if 'Fail' is selected, feedback must be completed.
| OSCE |
|
| 8 | Permissions & Ownership | Improving the way that permissions and roles are managed, and adding more granular permissions. | - Ability to separate exam-control permissions on a exam basis
- Ability to manage only certain modules (ie. written or OSCE)
- Permission based use of the item bank
| Written, OSCE |
|
| 9 | Security Improvements | Ensuring that Practique conforms to latest security standards. | - Improvements to user management
- Streamline account creation process
- Investigate two factor authentication
| Written, OSCE |
|
| 10 | Batch Operations | The abiliy to perform existing actions on one or multiple candidates, examiners, items and exams. | - Approve/reject/submit/retire items en masse
- Master Blueprint management improvements
- Change examiners on a session/block/exam basis
- Changing examiner information en masse
- Start all sessions at the same time
- Advance Exam Days
| Written, OSCE |
|
| 11 | Instant Message | The ability for OSCE examiners to send text messages to floor marshals. | - Text field in the 'request help' field
- Sent/recieved/read confirmation
| OSCE |
|
| 12 | Notification System | Improve the way that system notifications are delivered and handled. | - Notify users where items are obsolete due to blueprinting changes
- Ability to set 'reply to' email notification address and set a custom rejection/bounce back message
| Written, OSCE |
|
| 13 | Blueprinting Improvements | Updating Blueprinting to improve its ease of use and to add some new features. | - Blueprint tagging validations before an item can be submitted
- Graphical representation of the exam blueprint
- Define order of exam blueprints in the list
- Functional dependencies between blueprint dimensions
- Cloning Blueprint Dimensions and having same BP dimension in multiple maps.
- Enhanced Blueprint reporting
- Remove/archive exam blueprints
| Written, OSCE |
|
| 14 | Double Marking | Allowing more than one examiner/marker to assign a score to a candidate's response. | - Enhance the way that double marking is handled in Practique for OSCE stations
- Allow custom aggregation of multiple marks
- Allow large scale peer marking
- Enhanced marking allocation and processing for written exams
| Written, OSCE |
|
| 15 | Exam Preparation | Improving the workflow for scheduling exams. | - Impove the way that error messages are allocated
- Simplification of Exam Preparation Workflow
- Add more checks and balances (eg Candidates/Examiners in impossible allocations)
- Improvements to Double Length Station Rotation
- Implement filtering for candidate select view
- Drag and Drop examiner and candidate
- Sequential testing
| Written, OSCE |
|
| 16 | Paper Output | Being able to do more with the paper output function of Practique | - Enhanced OSCE exam day materials
- Paper notes for rest station
- Track and bundle all exam day documents
- Production of candidate and examiner lables
- Enhanced written papers
- Multiple exam paper templates
- Prepare and export a generic final exam paper
| Written, OSCE |
|
| 17 | Short Answer Question paper delivery. | Being able to scan SAQ papers, and view and mark electronically. | - Scanning of SAQ answer papers
- Mark responses on-screen
| Written, OSCE |
|
| 18 | Exam in progress changes | Being able to make changes to the the candidates/examiners of an exam while the exam is running | - Being able to make ad-hoc changes to OSCE participants
| Written, OSCE |
|
| 19 | Question authoring | Speeding up question authoring for written questions. | - Ability to dynamically add and remove SBA and EMQ options
- Option to automatically set SBA/VSAQ questions to 'Title Case'
- Text formatting (including indenting)
| Written |
|
| 20 | Senior Marshal | A new role for OSCE exams that gives someone an overview of all circuits and locations | | OSCE |
|
| 21 | Extra Time | Ability to set custom time per-candidate as exam goes on. | - Extra Time could be amended when exam is in session
| Written |
|
| 22 | Station iPads | Station specific iPads than can show resources. A substitute to door notes. | - Notes accessible to various candidates
| OSCE |
|
| 23 | Candidate Images | Ability to have a profile image for candidates | - Candidate photos on iPad
- Be able to run exam in anonymous mode so Marshall & Examiner do not see candidate name
| OSCE |
|
| 24 | Exam Control and Comments | Ability to add comments about an exam, and review exam log in details | - Have a way of toggling users that have not logged in at all (written iPad)
- Attach notes to exams
- Attach Audio to sessions
| Written, OSCE |
|
| 25 | Post-Exam SBA changes | Ability to re-define the correct answers after running an SBA paper | - In the unusual case that an SBA question is set up incorrectly, this would allow users to re key the question.
| Written, OSCE |
|
| 26 | iPad Audio Improvements | Improvements to the way that sound clips are recoreded on the iPads | - Improving the UI and adding some additional features
| OSCE |
|
| 27 | Help & Documentation | Adding to the help text and documentation in the system. | - Increasing the amount of help and documentation within the system
| Written, OSCE |
|
| 28 | Custom Written Questions | Ability to create your own question styles, using Practique's existing assessment tools. | - Scenario based custom written question
| Written |
|
| 29 | MCQ Question Type | Adding the MCQ question type in addition to the SBA | - Being able to have multiple correct answers | Written |
|
| 30 | Resources Source File | Ability to upload a source file for resources PDFs | - To allow source files to be uploaded to items
| Written, OSCE |
|
| 31 | Item Authoring notifications and permissions | More control over who sees and approves Items. To-do list style notifcation for Item Authors and approvers. |
This will allow users to streamline question preparation | Written, OSCE |
|
| 32 | Candidate Feedback Report Improvements | More control over content and appearance of Candidate Feedback Report. | - Improvements to candidate feedback report generation
- Allow pass mark in the candidate feedback report to be toggled
- Hide 'pass/fail' in candidate feedback report
- Customisable content from candidate feedback report
- Text display based on score on CFR
- Feedback in CFR that depends on the domain scores ie another column in this table that says 'Feedback'.
- Ability to remove elements : Ability to remove the overall score percentage and pass/fail from candidate reports, so that candidates can just recieve examiner feedback
- Feedback only CFR, without results (ie formative feedback)
- Scheduled release of reports
- Question breakdown - Ideally these should all be toggable, i.e. Your Answer / Correct Answers/
Examiner notes - Hide Result analysis (table & graph with the per station breakdown). At the moment the exam has only one station.
- Keep Station Examiner feedback section.
- Option to generate candidate feedback reports without the Overall Score and Overall Result rows (Added by Robert re PSD-1580)
- Show pass mark on Candidate feedback report (PSD-1597)
- Ability to choose either the overall score OR min stations passed on the candidate report or both
- Toggle Examiner notes (show or not show) Examiner notes should be changed to "Feedback"
| Written, OSCE |
|
| 33 | Reporting | Introducing a suite of new reporting functionality | - Improvements to the way that available reports are presented
- Add analysis of variance (annova) summary by session
- Exam analysis report: Comparison of students mean scores across sessions
- Point biserial report (linking to Ebel)
- Item response model
- Store timestamp of PDF generation & e-mail out operations
- Report distribution should re-use generated PDFs
- Report distribution depends on PDF generation
- Distribution list & status
- Results transcript
- Figure out statistics for different versions
- Correlation statistics across exams
- Reports UX review
- Station Cut score report - graphs for other Standard setting methods
- In Written Exams, items are referred to as stations in exam reports
- Include date and time of generation
| Written, OSCE |
|
| 34 | Mesh Networking | Automatic synchronisation of content through local wireless network, for improved resiliance. | - This will allow synchronisation to marshal when there is no internet access | OSCE |
|
| 35 | Multilingual | Infrastructure for multilingual backend and multilingual delivery of exams | - Ability top store translated items linked in the bank
- Able to use the system in another language
| Written, OSCE |
|
| 36 | Exam Attendance | Ability to control and record attendance of exams | - The ability to scan candidate id card (barcodes) to take attendance.
- Show candidate photos on the iPad
| Written, OSCE |
|
| 37 | Spell Checker | Allowing on-device spell check to be enabled for some exam types | - Ability to toggle spell check for exams
| Written, OSCE |
|
| 38 | Server Licence Infrastructure | Ability to validate Practique licence automatically | - Ability to validate Practique licence automatically
| Written, OSCE |
|
| 39 | Multiple Cases per Station | Ability to mark different items depending on the Candidate in a Station Room | - Assign multiple cases to a particular station
- Candidate can do all cases or select from a range
| OSCE |
|
| 40 | User Terms and Conditions | Ability to upload an organisations Terms and Conditions to cover users of Practique | - Show T&Cs to Examiners before marking
- Show T&Cs to admin team on first log in
- Show T&Cs to written candidates
| OSCE, Written |
|
| 41 | Candidate and Examiner ID changes | Make it easier to change the ID of Candidates and Examiners in Practique | - Allow the IDs to be updated
| OSCE, Written |
|
| 42 | Filtering in Item Sets | Allow filtering of Items within the Item Set | - Similar to main Item Bank filtering
| OSCE, Written |
|
| 43 | Candidate Item Order | Show Item number when viewing Candidate Marks | - Including when delivered randomly
| Written |
|
| 44 | Angoff Standard Setting Workflow | Improvements to the Angoff standard setting process to allow judges to score within Practique | - Support Angoff judging methods
- Allow custom Angoff arbitration
| Written, OSCE |
|
| 45 | Improve allocation of Item Sets to Exams | Add standardised filtering to the Item Set allocation page in exam preparation |
| Written, OSCE |
|
| 46 | Item Change Differences | Ability to see the differences between items that have been changed easily | - Allow approver to review the differences
| Written, OSCE |
|
| 47 | Ebel Judging Improvements | Improvements to the Ebel Scoring process | - CSV export of Ebel Score
- Show question paper number in addition to Item number
- Include question stem with Ebel judging form
- Improve scrolling of large Ebel Sets
| Written |
|
| 48 | Exam Paper Cover Page | Ability to attach a PDF cover page to written papers | - This would suplement the Edit Blueprint Cover template
- Ability to offset automatic page numbering
| Written |
|
| 49 | Filtering Improvements | Ability to be able to multi select review state and type when filtering item bank | - For example, this could be used to filter for draft or approved items
| Written, OSCE |
|
| 50 | Time taken Report | Ability to track and report on the time taken for candidates to complete written questions | - Shown for individual questions
| Written |
|
| 51 | Formatting of Text on server | Update the styling of Practique to make text easier to read | | Written, OSCE |
|
| 52 | Names in approval process | Ability to see te full name of a reviewer and author when reviewing items, as opposed to username |
| Written, OSCE |
|
| 53 | Support Item Set Sections in P4B | Ability to create sections within a written exam | - Instruction text between the sections
- Configure in Item Set page
- Randomisation per section
| Written |
|
| 54 | Easier ordering of Rest Stations | Ability to set order of rest stations as part of Item Set | - This would work similarly to Item Set Sections
| OSCE |
|
| 55 | Item Set Exclusion | Ability to exclude certain Item Sets from Item Set generation | - Easily limit the Items available to be included in an exam
- To be set when generating an Item Set
| OSCE, Written |
|