| Theme | Summary | Features | Applicable Exam Types | Schedule |
---|
1 | Examiner Availability Calendar | This would allow users to recruit and book OSCE Examiners for future exams, at any point. | - When scheduling an exam, you would be able to pull in examiner allocations from the calendar
| OSCE | To be confirmed at PUG |
2 | Editable Item Documents | This would allow item authors to create OSCE station documents within Practique. | - Ability to create and edit basic text documents within the Resources tab
- These resources will be versioned inline with current resources management
| OSCE | To be confirmed at PUGCompleted |
3 | Standard Setting | We would like to improve the Standard Setting workflow and add some new features. | - Enhance live standard setting information while and exam is in progress
- Allow the import of Ebel judging from spreadsheets
- Allow Examiner-performance rescalling (hawk/dove
- Have exam blueprint based Ebel grids
- Allow mark re-scaling to university standards
- Allow in-built Angoff Judging
- Ability to sort standard setting screen by all fields
- Improvements to the UI of setting exam standard
- Add Rasch modelling
- Show negative -1, -2 SEM in Standard setting
| Written, OSCE | Next 12 months |
4 | Exports/Import | We are going to improve the way that data is imported to and exported from Practique. | - Consolidation of the structure of exam data exports
- Allow filtered candidates and examiners to be exported
- Add the export of candidate and examiner scheduleing
- Separate the import of items permissions
- Allow the export of Item Sets with Item IDs and blueprint tagging
- Simplify CSV field mapping
- Auto detect CSV encoding in upload
- Allow filtered Item bank export
- Add Max score per station, and max score per exam in the CSV export of exams
- Improve naming convention of results download
| Written, OSCE | Continual Improvement |
5 | API | We are going to enhance the way that Practique communicates with other systems | - This will allow users to have Practique integrated with other systems they use. Practique will be able to load information from, and push information to these systems
- This will include candidate results and feedback
- This will allow timetable and scheduling information for OSCE exams to be shared
| Written, OSCE | Next six monthsCompleted |
6 | Updating iPad app | We are going to improve the way that users can install and update the iPad application on their devices. | - Allow IPA file to be downloaded directly
- Control the settings for the iPad app on a per-exam basis (examiners editing marks, rotation management)
| OSCE | Postponed |
7 | OSCE Marksheet Improvements | We are going to improve the way that OSCE marksheets are created and displayed on the iPads | - Improve the way that marksheets are created, by
- Allowing schemas to be searched for, and deleted
- Improve marksheet 'Count As' tagging review process
- Warn users if global criterias are incorrectly set up
- Improve the content of the marksheet by
- Allowing photos to be captured on the iPad
- Improve the way that structured feedback can be recorded
- Adding a sliding scale observation criteria
- Allowing custom text to be inserted on the marksheet
- Improve the way that generic feedback statements can be recorded
- Linking from the mark sheet to resources - Ability to embed links in observation criteria that will open specific resources attached to item.
- Allow styling (bold, italic etc) in observation criteria
| OSCE | Postponed |
8 | Global Marksheet Improvements | We will be changing the Global Marksheet wokflow to improve versioning and control | - Investigate versionning improvements to the Global Marksheet
| OSCE | Next 12 months |
9 | Marksheet Validation | We will allow custom rules to be set for the marksheet. | - Configure maximum/minimum number of words recorded
- Set rules for mandatory information. For example, if 'Fail' is selected, feedback must be completed.
| OSCE | Next six months |
10 | Permissions & Ownership | Improving the way that permissions and roles are managed, and adding more granular permissions. | - Ability to separate exam-control permissions on a exam basis
- Ability to manage only certain modules (ie. written or OSCE)
- Permission based use of the item bank
| Written, OSCE | Postponed |
11 | Security Improvements | Ensuring that Practique conforms to latest security standards. | - Improvements to user management
- Streamline account creation process
- Investigate two factor authentication
| Written, OSCE | Next 12 months |
12 | Batch Operations | The abiliy to perform existing actions on one or multiple candidates, examiners, items and exams. | - Approve/reject/submit/retire items en masse
- Master Blueprint management improvements
- Change examiners on a session/block/exam basis
- Changing examiner information en masse
- Start all sessions at the same time
| Written, OSCE | Next three months |
13 | Instant Message | The ability for OSCE examiners to send text messages to floor marshals. | - Text field in the 'request help' field
- Sent/recieved/read confirmation
| OSCE | Postponed |
14 | Notification System | Improve the way that system notifications are delivered and handled. | - Notify users where items are obsolete due to blueprinting changes
- Notify users when reports are finished generating
- Ability to set 'reply to' email notification address and set a custom rejection/bounce back message
| Written, OSCE | Postponed |
15 | Blueprinting Improvements | Updating Blueprinting to improve its ease of use and to add some new features. | - Blueprint tagging validations before an item can be submitted
- Graphical representation of the exam blueprint
- Define order of exam blueprints in the list
- Functional dependencies between blueprint dimensions
- Cloning Blueprint Dimensions and having same BP dimension in multiple maps.
- Enhanced Blueprint reporting
- Remove/archive exam blueprints
| Written, OSCE | Postponed |
16 | Double Marking | Allowing more than one examiner/marker to assign a score to a candidate's response. | - Enhance the way that double marking is handled in Practique for OSCE stations
- Allow custom aggregation of multiple marks
- Allow large scale peer marking
- Enhanced marking allocation and processing for written exams
| Written, OSCE | Postponed |
17 | Exam Preparation | Improving the workflow for scheduling exams. | - Impove the way that error messages are allocated
- Simplification of Exam Preparation Workflow
- Add more checks and balances (eg Candidates/Examiners in impossible allocations)
- Improvements to Double Length Station Rotation
- Implement filtering for candidate select view
- Drag and Drop examiner and candidate
- Sequential testing
| Written, OSCE | Postponed |
18 | Simultaneous Operations | Tasks that are non-instanteous(such as reporting) will run in the background. | - Better communication of complex actions progress
- Increase speed of paper generator
- Improve speed of exports
- Improvements to automatic item set generation
| Written, OSCE | Next three monthsCompleted |
19 | Paper Output | Being able to do more with the paper output function of Practique | - Enhanced OSCE exam day materials
- Paper notes for rest station
- Track and bundle all exam day documents
- Production of candidate and examiner lables
- Enhanced written papers
- Multiple exam paper templates
- Prepare and export a generic final exam paper
| Written, OSCE | Continuous Improvement |
20 | Short Answer Question paper delivery. | Being able to scan SAQ papers, and view and mark electronically. | - Scanning of SAQ answer papers
- Mark responses on-screen
| Written, OSCE | Postponed |
21 | Exam in progress changes | Being able to make changes to the the candidates/examiners of an exam while the exam is running | - Being able to make ad-hoc changes to OSCE participants
| Written, OSCE | Rejected - in contradiction with robust exam workflow |
22 | Question authoring | Speeding up question authoring for written questions. | - Ability to dynamically add and remove SBA and EMQ options
- Option to automatically set SBA/VSAQ questions to 'Title Case'
- Text formatting (including indenting)
| Written | Next three months |
23 | Senior Marshal | A new role for OSCE exams that gives someone an overview of all circuits and locations | | OSCE | Postponed |
24 | Exam Timer | Adding an indicative timer for candidates when taking a written exam | - This would not lock candidates out
| Written | Continuous ImprovementCompleted |
25 | Extra Time | Ability to set custom time per-candidate as exam goes on. | - Extra Time could be amended when exam is in session
| Written | Postponed |
26 | Station iPads | Station specific iPads than can show resources. A substitute to door notes. | - Notes accessible to various candidates
| OSCE | Postponed |
27 | Multiple Filtering | Ability to have multiple filters at the same time when searching | - Add multiple 'AND' filter for same blueprint dimension
| Written, OSCE | Continuous ImprovementCompleted |
28 | Candidate Images | Ability to have a profile image for candidates | - Candidate photos on iPad
- Be able to run exam in anonymous mode so Marshall & Examiner do not see candidate name
| OSCE | Postponed |
29 | Exam Control and Comments | Ability to add comments about an exam, and review exam log in details | - Have a way of toggling users that have not logged in at all (written iPad)
- Attach notes to exams
- Attach Audio to sessions
| Written, OSCE | Postponed |
30 | Post-Exam SBA changes | Ability to re-define the correct answers after running an SBA paper | - In the unusual case that an SBA question is set up incorrectly, this would allow users to re key the question.
| Written, OSCE | Continuous Improvement |
31 | iPad Audio Improvements | Improvements to the way that sound clips are recoreded on the iPads | - Improving the UI and adding some additional features
| OSCE | Postponed |
32 | Help & Documentation | Adding to the help text and documentation in the system. | - Increasing the amount of help and documentation within the system
| Written, OSCE | Continuous Improvement |
33 | EMQ Delivery | Ability to deliver EMQs on iPad and Practique for Browser | - Support EMQ on iPad
- Support EMQ on Practique for Browser
| Images as EMQ options | Written | Next 12 monthsCompleted |
34 | Custom Written Questions | Ability to create your own question styles, using Practique's existing assessment tools. | - Scenario based custom written question
| Written | Next 12 months |
35 | MCQ Question Type | Adding the MCQ question type in addition to the SBA | - Being able to have multiple correct answers | Written | Continuous Improvement |
36 | Marksheet & Item Preview | Ability to preview the marksheet and written questions, as you are editing them. | - Able to generate PDF/word of single item (written)
- Marksheet preview to check how it would look on iPad for examiner
- Export Marksheet with count As tagging so that it can be reviewed pre exam by external examiners etc
- Generate generic paper marksheet as a backup that can be completed in an exam
| Written, OSCE | Next three monthsCompleted |
37 | Item Set Sectionwise Randomisation | Ability to specify which sections of the Item Sets have randomised question order | - Item set randomisation per section
| Written | Postponed |
38 | Blueprint Constraint Satisfaction Report | Be able to see how well your Item Set meets the Exam Blueprint | - Warnings on item set constraint satisfaction
| Written, OSCE | Continuous ImprovementCompleted |
39 | Resources Source File | Ability to upload a source file for resources PDFs | - To allow source files to be uploaded to items
| Written, OSCE | Continuous Improvement |
40 | Item Authoring notifications and permissions | More control over who sees and approves Items. To-do list style notifcation for Item Authors and approvers. |
This will allow users to streamline question preparation | Written, OSCE | Postponed |
41 | Candidate Feedback Report Improvements | More control over content and appearance of Candidate Feedback Report. | - Improvements to candidate feedback report generation
- Allow pass mark in the candidate feedback report to be toggled
- Hide 'pass/fail' in candidate feedback report
- Customisable content from candidate feedback report
- Text display based on score on CFR
- Feedback in CFR that depends on the domain scores ie another column in this table that says 'Feedback'.
- Ability to remove elements : Ability to remove the overall score percentage and pass/fail from candidate reports, so that candidates can just recieve examiner feedback
- Feedback only CFR, without results (ie formative feedback)
- Scheduled release of reports
| Written, OSCE | Continuous Improvement |
42 | Reporting | Introducing a suite of new reporting functionality | - Improvements to the way that available reports are presented
- Add analysis of variance (annova) summary by session
- Exam analysis report: Comparison of students mean scores across sessions
- Point biserial report (linking to Ebel)
- Item response model
- Store timestamp of PDF generation & e-mail out operations
- Report distribution should re-use generated PDFs
- Report distribution depends on PDF generation
- Distribution list & status
- Results transcript
- Figure out statistics for different versions
- Correlation statistics across exams
- Reports UX review
- Station Cut score report - graphs for other Standard setting methods
| Written, OSCE | Next 12 Months |
43 | Mesh Networking | Automatic synchronisation of content through local wireless network, for improved resiliance. | - This will allow synchronisation to marshal when there is no internet access | OSCE | Postponed |
44 | Multilingual | Infrastructure for multilingual backend and multilingual delivery of exams | - Ability top store translated items linked in the bank
- Able to use the system in another language
| Written, OSCE | Postponed |
45 | Exam Attendance | Ability to control and record attendance of exams | - The ability to scan candidate id card (barcodes) to take attendance.
- Show candidate photos on the iPad
| Written, OSCE | Postponed |
46 | Spell Checker | Allowing on-device spell check to be enabled for some exam types | - Ability to toggle spell check for exams
| Written, OSCE | Postponed |
47 | Server Licence Infrastructure | Ability to validate Practique licence automatically | - Ability to validate Practique licence automatically
| Written, OSCE | Postponed |