| Theme | Summary | Features | Applicable Exam Types | Schedule |
---|
1 | Editable Item Documents Examiner Availability Calendar | This would allow item authors to create OSCE station documents within Practique.Ability to create and edit basic text documents within the Resources tabThese resources will be versioned inline with current resources managementusers to recruit and book OSCE Examiners for future exams, at any point. | - When scheduling an exam, you would be able to pull in examiner allocations from the calendar
| OSCE | To be confirmed at PUG |
2 | Examiner Availability CalendarEditable Item Documents | This would allow users to recruit and book OSCE Examiners for future exams, at any point.When scheduling an exam, you would be able to pull in examiner allocations from the calendaritem authors to create OSCE station documents within Practique. | - Ability to create and edit basic text documents within the Resources tab
- These resources will be versioned inline with current resources management
| OSCE | To be confirmed at PUG |
3 | Standard Setting | We would like to improve the Standard Setting workflow and add some new features. | - Enhance live standard setting information while and exam is in progress
- Allow the import of Ebel judging from spreadsheets
- Allow Examiner-performance rescalling (hawk/dove
- Have exam blueprint based Ebel grids
- Allow mark re-scaling to university standards
- Allow in-built Angoff Judging
- Ability to sort standard setting screen by all fields
- Improvements to the UI of setting exam standard
- Add Rasch modelling
- Show negative -1, -2 SEM in Standard setting
| Written, OSCE | Next 12 months |
4 | Exports/Import | We are going to improve the way that data is imported to and exported from Practique. | - Consolidation of the structure of exam data exports
- Allow filtered candidates and examiners to be exported
- Add the export of candidate and examiner scheduleing
- Separate the import of items permissions
- Allow the export of Item Sets with Item IDs and blueprint tagging
- Simplify CSV field mapping
- Auto detect CSV encoding in upload
- Allow filtered Item bank export
- Add Max score per station, and max score per exam in the CSV export of exams
- Improve naming convention of results download
| Written, OSCE | Continual Improvement |
5 | API | We are going to enhance the way that Practique communicates with other systems | - This will allow users to have Practique integrated with other systems they use. Practique will be able to load information from, and push information to these systems
- This will include candidate results and feedback
- This will allow timetable and scheduling information for OSCE exams to be shared
| Written, OSCE | Next six months |
6 | Updating iPad app | We are going to improve the way that users can install and update the iPad application on their devices. | - Allow IPA file to be downloaded directly
- Control the settings for the iPad app on a per-exam basis (examiners editing marks, rotation management)
| OSCE | Postponed |
7 | OSCE Marksheet Improvements | We are going to improve the way that OSCE marksheets are created and displayed on the iPads | - Improve the way that marksheets are created, by
- Allowing schemas to be searched for, and deleted
- Improve marksheet 'Count As' tagging review process
- Warn users if global criterias are incorrectly set up
- Improve the content of the marksheet by
- Allowing photos to be captured on the iPad
- Improve the way that structured feedback can be recorded
- Adding a sliding scale observation criteria
- Allowing custom text to be inserted on the marksheet
- Improve the way that generic feedback statements can be recorded
- Linking from the mark sheet to resources - Ability to embed links in observation criteria that will open specific resources attached to item.
- Allow styling (bold, italic etc) in observation criteria
| OSCE | Postponed |
8 | Global Marksheet Improvements | We will be changing the Global Marksheet wokflow to improve versioning and control | - Investigate versionning improvements to the Global Marksheet
| OSCE | Next 12 months |
9 | Marksheet Validation | We will allow custom rules to be set for the marksheet. | - Configure maximum/minimum number of words recorded
- Set rules for mandatory information. For example, if 'Fail' is selected, feedback must be completed.
| OSCE | Next six months |
10 | Permissions & Ownership | Improving the way that permissions and roles are managed, and adding more granular permissions. | - Ability to separate exam-control permissions on a exam basis
- Ability to manage only certain modules (ie. written or OSCE)
- Permission based use of the item bank
| Written, OSCE | Postponed |
11 | Security Improvements | Ensuring that Practique conforms to latest security standards. | - Improvements to user management
- Streamline account creation process
- Investigate two factor authentication
| Written, OSCE | Next 12 months |
12 | Batch Operations | The abiliy to perform existing actions on one or multiple candidates, examiners, items and exams. | - Approve/reject/submit/retire items en masse
- Master Blueprint management improvements
- Change examiners on a session/block/exam basis
- Changing examiner information en masse
- Start all sessions at the same time
| Written, OSCE | Next three months |
13 | Instant Message | The ability for OSCE examiners to send text messages to floor marshals. | - Text field in the 'request help' field
- Sent/recieved/read confirmation
| OSCE | Postponed |
14 | Notification System | Improve the way that system notifications are delivered and handled. | - Notify users where items are obsolete due to blueprinting changes
- Notify users when reports are finished generating
- Ability to set 'reply to' email notification address and set a custom rejection/bounce back message
| Written, OSCE | Postponed |
15 | Blueprinting Improvements | Updating Blueprinting to improve its ease of use and to add some new features. | - Blueprint tagging validations before an item can be submitted
- Graphical representation of the exam blueprint
- Define order of exam blueprints in the list
- Functional dependencies between blueprint dimensions
- Cloning Blueprint Dimensions and having same BP dimension in multiple maps.
- Enhanced Blueprint reporting
- Remove/archive exam blueprints
| Written, OSCE | Postponed |
16 | Double Marking | Allowing more than one examiner/marker to assign a score to a candidate's response. | - Enhance the way that double marking is handled in Practique for OSCE stations
- Allow custom aggregation of multiple marks
- Allow large scale peer marking
- Enhanced marking allocation and processing for written exams
| Written, OSCE | Postponed |
17 | Exam Preparation | Improving the workflow for scheduling exams. | - Impove the way that error messages are allocated
- Simplification of Exam Preparation Workflow
- Add more checks and balances (eg Candidates/Examiners in impossible allocations)
- Improvements to Double Length Station Rotation
- Implement filtering for candidate select view
- Drag and Drop examiner and candidate
- Sequential testing
| Written, OSCE | Postponed |
18 | Simultaneous Operations | Tasks that are non-instanteous(such as reporting) will run in the background. | - Better communication of complex actions progress
- Increase speed of paper generator
- Improve speed of exports
- Improvements to automatic item set generation
| Written, OSCE | Next three months |
19 | Paper Output | Being able to do more with the paper output function of Practique | - Enhanced OSCE exam day materials
- Paper notes for rest station
- Track and bundle all exam day documents
- Production of candidate and examiner lables
- Enhanced written papers
- Multiple exam paper templates
- Prepare and export a generic final exam paper
| Written, OSCE | Continuous Improvement |
20 | Short Answer Question paper delivery. | Being able to scan SAQ papers, and view and mark electronically. | - Scanning of SAQ answer papers
- Mark responses on-screen
| Written, OSCE | Postponed |
21 | Exam in progress changes | Being able to make changes to the the candidates/examiners of an exam while the exam is running | - Being able to make ad-hoc changes to OSCE participants
| Written, OSCE | Rejected - in contradiction with robust exam workflow |
22 | Question authoring | Speeding up question authoring for written questions. | - Ability to dynamically add and remove SBA and EMQ options
- Option to automatically set SBA/VSAQ questions to 'Title Case'
- Text formatting (including indenting)
| Written | Next three months |
23 | Senior Marshal | A new role for OSCE exams that gives someone an overview of all circuits and locations | | OSCE | Postponed |
24 | Exam Timer | Adding an indicative timer for candidates when taking a written exam | - This would not lock candidates out
| Written | Continuous Improvement |
25 | Extra Time | Ability to set custom time per-candidate as exam goes on. | - Extra Time could be amended when exam is in session
| Written | Postponed |
26 | Station iPads | Station specific iPads than can show resources. A substitute to door notes. | - Notes accessible to various candidates
| OSCE | Postponed |
27 | Multiple Filtering | Ability to have multiple filters at the same time when searching | - Add multiple 'AND' filter for same blueprint dimension
| Written, OSCE | Continuous Improvement |
28 | Candidate Images | Ability to have a profile image for candidates | - Candidate photos on iPad
- Be able to run exam in anonymous mode so Marshall & Examiner do not see candidate name
| OSCE | Postponed |
29 | Exam Control and Comments | Ability to add comments about an exam, and review exam log in details | - Have a way of toggling users that have not logged in at all (written iPad)
- Attach notes to exams
- Attach Audio to sessions
| Written, OSCE | Postponed |
30 | Post-Exam SBA changes | Ability to re-define the correct answers after running an SBA paper | - In the unusual case that an SBA question is set up incorrectly, this would allow users to re key the question.
| Written, OSCE | Continuous Improvement |
31 | iPad Audio Improvements | Improvements to the way that sound clips are recoreded on the iPads | - Improving the UI and adding some additional features
| OSCE | Postponed |
32 | Help & Documentation | Adding to the help text and documentation in the system. | - Increasing the amount of help and documentation within the system
| Written, OSCE | Continuous Improvement |
33 | EMQ Delivery | Ability to deliver EMQs on iPad and Practique for Browser | - Support EMQ on iPad
- Support EMQ on Practique for Browser
- Images as EMQ options
| Written | Next 12 months |
34 | Custom Written Questions | Ability to create your own question styles, using Practique's existing assessment tools. | - Scenario based custom written question
| Written | Next 12 months |
35 | MCQ Question Type | Adding the MCQ question type in addition to the SBA | - Being able to have multiple correct answers | Written | Continuous Improvement |
36 | Marksheet & Item Preview | Ability to preview the marksheet and written questions, as you are editing them. | - Able to generate PDF/word of single item (written)
- Marksheet preview to check how it would look on iPad for examiner
- Export Marksheet with count As tagging so that it can be reviewed pre exam by external examiners etc
- Generate generic paper marksheet as a backup that can be completed in an exam
| Written, OSCE | Next three months |
37 | Item Set Sectionwise Randomisation | Ability to specify which sections of the Item Sets have randomised question order | - Item set randomisation per section
| Written | Postponed |
38 | Blueprint Constraint Satisfaction Report | Be able to see how well your Item Set meets the Exam Blueprint | - Warnings on item set constraint satisfaction
| Written, OSCE | Continuous Improvement |
39 | Resources Source File | Ability to upload a source file for resources PDFs | - To allow source files to be uploaded to items
| Written, OSCE | Continuous Improvement |
40 | Item Authoring notifications and permissions | More control over who sees and approves Items. To-do list style notifcation for Item Authors and approvers. |
This will allow users to streamline question preparation | Written, OSCE | Postponed |
41 | Candidate Feedback Report Improvements | More control over content and appearance of Candidate Feedback Report. | - Improvements to candidate feedback report generation
- Allow pass mark in the candidate feedback report to be toggled
- Hide 'pass/fail' in candidate feedback report
- Customisable content from candidate feedback report
- Text display based on score on CFR
- Feedback in CFR that depends on the domain scores ie another column in this table that says 'Feedback'.
- Ability to remove elements : Ability to remove the overall score percentage and pass/fail from candidate reports, so that candidates can just recieve examiner feedback
- Feedback only CFR, without results (ie formative feedback)
- Scheduled release of reports
| Written, OSCE | Continuous Improvement |
42 | Reporting | Introducing a suite of new reporting functionality | - Improvements to the way that available reports are presented
- Add analysis of variance (annova) summary by session
- Exam analysis report: Comparison of students mean scores across sessions
- Point biserial report (linking to Ebel)
- Item response model
- Store timestamp of PDF generation & e-mail out operations
- Report distribution should re-use generated PDFs
- Report distribution depends on PDF generation
- Distribution list & status
- Results transcript
- Figure out statistics for different versions
- Correlation statistics across exams
- Reports UX review
- Station Cut score report - graphs for other Standard setting methods
| Written, OSCE | Next 12 Months |
43 | Mesh Networking | Automatic synchronisation of content through local wireless network, for improved resiliance. | - This will allow synchronisation to marshal when there is no internet access | OSCE | Postponed |
44 | Multilingual | Infrastructure for multilingual backend and multilingual delivery of exams | - Ability top store translated items linked in the bank
- Able to use the system in another language
| Written, OSCE | Postponed |
45 | Exam Attendance | Ability to control and record attendance of exams | - The ability to scan candidate id card (barcodes) to take attendance.
- Show candidate photos on the iPad
| Written, OSCE | Postponed |
46 | Spell Checker | Allowing on-device spell check to be enabled for some exam types | - Ability to toggle spell check for exams
| Written, OSCE | Postponed |
47 | Server Licence Infrastructure | Ability to validate Practique licence automatically | - Ability to validate Practique licence automatically
| Written, OSCE | Postponed |