Practique Development Roadmap

Below you can see the features we are focussing on and when. We will aim to release improvements under the themes stated here and you will receive an update on what is released in our Practique Server release notes and Practique for iPad release notes. Remember that we regularly review our roadmap and priorities based on user feedback. This should, however, give you a good overview of our plans. You can also see all of the features that we have captured from your feedback. If your feature is not here it may not have gone through our review process yet or we may have included it as part of another feature.

How we plan and focus our work

Next Release - These are large features we feel are really important for our customers and the progression of Practique. These are larger features or blocks of work that we feel are important. We will also allot a proportion of our development time budget to features decided on during the Practique User Group meetings. You should expect to see these features begin to appear in the next release. Generally we think the release should be ready for the next Practique User Group if they are every six months.

Next 12 months - Larger features that we have prioritised from user feedback that we know will make a great difference to Practique and are part of our longer term strategy. We expect these to be the first candidates when planning the work for the next release

Continuous - These are smaller features based on user feedback. We often can include these as part of our ongoing development as they take less time than the larger features. You should expect to see some of these features in the next six months.

Postponed - These are features that we have captured from users' feedback, but that are not seen as a priority for the next 12 months.

Next release

Feature Name	Summary
Examiner Availability Tool	Brand new tool that provides users with a calendar they can use to communicate exam dates to potential examiners. Flexible and configurable to allow users to capture and manage examiner information.
Candidate feedback report improvements	Continued improvements to the customisation of the candidate feedback report.
Batch operations	Re-allocate examiners after the exam has finished Select multiple Items to archive in bulk
Senior Marshal	Addition of a Senior Marshal role for OSCE exams who has an overview of all circuits and stations
P4B calculator	In application calculator that is accessible during written exams
Exam board report	Single report with all the information you need for the exam board post exam. We are taking feedback from our users on what information to include. If you have specific needs please send us an example of your own exam board report.

Feature List

Note: This table is sortable, by clicking the up and down arrows next to the column headings.

	Theme	Summary	Features	Applicable Exam Types
1	Examiner Availability Calendar	This would allow users to recruit and book OSCE Examiners for future exams, at any point.	When scheduling an exam, you would be able to pull in examiner allocations from the calendar	OSCE
2	Standard Setting	We would like to improve the Standard Setting workflow and add some new features.	Enhance live standard setting information while and exam is in progress Allow the import of Ebel judging from spreadsheets Allow Examiner-performance rescalling (hawk/dove Have exam blueprint based Ebel grids Allow mark re-scaling to university standards Ability to sort standard setting screen by all fields PSD-256 Show passing and failing students (column in the download) PSD-256 Improvements to the UI of setting exam standard Add Rasch modelling Show negative -1, -2 SEM in Standard setting	Written, OSCE
3	Exports/Import	We are going to improve the way that data is imported to and exported from Practique.	Consolidation of the structure of exam data exports Allow filtered candidates and examiners to be exported Add the export of candidate and examiner scheduleing Separate the import of items permissions Allow the export of Item Sets with Item IDs and blueprint tagging Simplify CSV field mapping Auto detect CSV encoding in upload Allow filtered Item bank export Add Max score per station, and max score per exam in the CSV export of exams Improve naming convention of results download Allow export of Standard Setting question scores Include Examiner ID in all results exports	Written, OSCE
4	iPad exam settings	We are going to improve the way that users can install and update the iPad application on their devices.	Control the settings for the iPad app on a per-exam basis (examiners editing marks, rotation management)	OSCE
5	OSCE Marksheet Improvements	We are going to improve the way that OSCE marksheets are created and displayed on the iPads	Improve the way that marksheets are created, by Allowing schemas to be searched for, and deleted Improve marksheet 'Count As' tagging review process Warn users if global criterias are incorrectly set up Marking Schema instructions hidden in marking buttons Improve the content of the marksheet by Allowing photos to be captured on the iPad Improve the way that structured feedback can be recorded Adding a sliding scale observation criteria Allowing custom text to be inserted on the marksheet Improve the way that generic feedback statements can be recorded Linking from the mark sheet to resources - Ability to embed links in observation criteria that will open specific resources attached to item. Allow styling (bold, italic etc) in observation criteria	OSCE
6	Global Marksheet Improvements	We will be changing the Global Marksheet wokflow to improve versioning and control	Investigate versioning improvements to the Global Marksheet	OSCE
7	Custom Marksheet Validation	We will allow custom rules to be set for the marksheet.	Configure maximum/minimum number of words recorded Set rules for mandatory information. For example, if 'Fail' is selected, feedback must be completed.	OSCE
8	Permissions & Ownership	Improving the way that permissions and roles are managed, and adding more granular permissions.	Ability to separate exam-control permissions on a exam basis Ability to manage only certain modules (ie. written or OSCE) Permission based use of the item bank	Written, OSCE
9	Security Improvements	Ensuring that Practique conforms to latest security standards.	Improvements to user management Streamline account creation process Investigate two factor authentication	Written, OSCE
10	Batch Operations	The abiliy to perform existing actions on one or multiple candidates, examiners, items and exams.	Approve/reject/submit/retire items en masse Master Blueprint management improvements Change examiners on a session/block/exam basis Changing examiner information en masse Start all sessions at the same time Advance Exam Days	Written, OSCE
11	Instant Message	The ability for OSCE examiners to send text messages to floor marshals.	Text field in the 'request help' field Sent/recieved/read confirmation	OSCE
12	Notification System	Improve the way that system notifications are delivered and handled.	Notify users where items are obsolete due to blueprinting changes Ability to set 'reply to' email notification address and set a custom rejection/bounce back message	Written, OSCE
13	Blueprinting Improvements	Updating Blueprinting to improve its ease of use and to add some new features.	Blueprint tagging validations before an item can be submitted Graphical representation of the exam blueprint Define order of exam blueprints in the list Functional dependencies between blueprint dimensions Cloning Blueprint Dimensions and having same BP dimension in multiple maps. Enhanced Blueprint reporting Remove/archive exam blueprints	Written, OSCE
14	Double Marking	Allowing more than one examiner/marker to assign a score to a candidate's response.	Enhance the way that double marking is handled in Practique for OSCE stations Allow custom aggregation of multiple marks Allow large scale peer marking Enhanced marking allocation and processing for written exams	Written, OSCE
15	Exam Preparation	Improving the workflow for scheduling exams.	Impove the way that error messages are allocated Simplification of Exam Preparation Workflow Add more checks and balances (eg Candidates/Examiners in impossible allocations) Improvements to Double Length Station Rotation Implement filtering for candidate select view Drag and Drop examiner and candidate Sequential testing	Written, OSCE
16	Paper Output	Being able to do more with the paper output function of Practique	Enhanced OSCE exam day materials Paper notes for rest station Track and bundle all exam day documents Production of candidate and examiner lables Enhanced written papers Multiple exam paper templates Prepare and export a generic final exam paper	Written, OSCE
17	Short Answer Question paper delivery.	Being able to scan SAQ papers, and view and mark electronically.	Scanning of SAQ answer papers Mark responses on-screen	Written, OSCE
18	Exam in progress changes	Being able to make changes to the the candidates/examiners of an exam while the exam is running	Being able to make ad-hoc changes to OSCE participants	Written, OSCE
19	Question authoring	Speeding up question authoring for written questions.	Ability to dynamically add and remove SBA and EMQ options Option to automatically set SBA/VSAQ questions to 'Title Case' Text formatting (including indenting)	Written
20	Senior Marshal	A new role for OSCE exams that gives someone an overview of all circuits and locations	Permissions to see specific marksheet submissions Show Candidate not attending on Marshal iPad Communicate from Examiner iPad to Marshal that there are marksheet amendments pending Floor marshal should be able to see all rounds in a session Floor marshal should be able to view station materials To be able to Marshall all circuit and to have multiple users logged in as an observer Start and stop exam and session controls on marshal screen 'request help' to be flagged prominently on marshal screen	OSCE
21	Extra Time	Ability to set custom time per-candidate as exam goes on.	Extra Time could be amended when exam is in session	Written
22	Station iPads	Station specific iPads than can show resources. A substitute to door notes.	Notes accessible to various candidates	OSCE
23	Candidate Images	Ability to have a profile image for candidates	Candidate photos on iPad Be able to run exam in anonymous mode so Marshall & Examiner do not see candidate name	OSCE
24	Exam Control and Comments	Ability to add comments about an exam, and review exam log in details	Have a way of toggling users that have not logged in at all (written iPad) Attach notes to exams Attach Audio to sessions	Written, OSCE
25	Post-Exam SBA changes	Ability to re-define the correct answers after running an SBA paper	In the unusual case that an SBA question is set up incorrectly, this would allow users to re key the question.	Written, OSCE
26	iPad Audio Improvements	Improvements to the way that sound clips are recoreded on the iPads	Improving the UI and adding some additional features	OSCE
27	Help & Documentation	Adding to the help text and documentation in the system.	Increasing the amount of help and documentation within the system	Written, OSCE
28	Custom Written Questions	Ability to create your own question styles, using Practique's existing assessment tools.	Scenario based custom written question	Written
29	MCQ Question Type	Adding the MCQ question type in addition to the SBA	- Being able to have multiple correct answers	Written
30	Resources Source File	Ability to upload a source file for resources PDFs	To allow source files to be uploaded to items	Written, OSCE
31	Item Authoring notifications and permissions	More control over who sees and approves Items. To-do list style notifcation for Item Authors and approvers.	This will allow users to streamline question preparation	Written, OSCE
32	Candidate Feedback Report Improvements	More control over content and appearance of Candidate Feedback Report.	Improvements to candidate feedback report generation Allow pass mark in the candidate feedback report to be toggled Hide 'pass/fail' in candidate feedback report Customisable content from candidate feedback report Text display based on score on CFR Feedback in CFR that depends on the domain scores ie another column in this table that says 'Feedback'. Ability to remove elements : Ability to remove the overall score percentage and pass/fail from candidate reports, so that candidates can just recieve examiner feedback Feedback only CFR, without results (ie formative feedback) Scheduled release of reports Question breakdown - Ideally these should all be toggable, i.e. Your Answer / Correct Answers/ Examiner notes Hide Result analysis (table & graph with the per station breakdown). At the moment the exam has only one station. Keep Station Examiner feedback section. Option to generate candidate feedback reports without the Overall Score and Overall Result rows (Added by Robert re PSD-1580) Show pass mark on Candidate feedback report (PSD-1597) Ability to choose either the overall score OR min stations passed on the candidate report or both Toggle Examiner notes (show or not show) Examiner notes should be changed to "Feedback"	Written, OSCE
33	Reporting	Introducing a suite of new reporting functionality	Improvements to the way that available reports are presented Add analysis of variance (annova) summary by session Exam analysis report: Comparison of students mean scores across sessions Point biserial report (linking to Ebel) Item response model Store timestamp of PDF generation & e-mail out operations Report distribution should re-use generated PDFs Report distribution depends on PDF generation Distribution list & status Results transcript Figure out statistics for different versions Correlation statistics across exams Reports UX review Station Cut score report - graphs for other Standard setting methods In Written Exams, items are referred to as stations in exam reports Include date and time of generation	Written, OSCE
34	Mesh Networking	Automatic synchronisation of content through local wireless network, for improved resiliance.	- This will allow synchronisation to marshal when there is no internet access	OSCE
35	Multilingual	Infrastructure for multilingual backend and multilingual delivery of exams	Ability top store translated items linked in the bank Able to use the system in another language	Written, OSCE
36	Exam Attendance	Ability to control and record attendance of exams	The ability to scan candidate id card (barcodes) to take attendance. Show candidate photos on the iPad	Written, OSCE
37	Spell Checker	Allowing on-device spell check to be enabled for some exam types	Ability to toggle spell check for exams	Written, OSCE
38	Server Licence Infrastructure	Ability to validate Practique licence automatically	Ability to validate Practique licence automatically	Written, OSCE
39	Multiple Cases per Station	Ability to mark different items depending on the Candidate in a Station Room	Assign multiple cases to a particular station Candidate can do all cases or select from a range	OSCE
40	User Terms and Conditions	Ability to upload an organisations Terms and Conditions to cover users of Practique	Show T&Cs to Examiners before marking Show T&Cs to admin team on first log in Show T&Cs to written candidates	OSCE, Written
41	Candidate and Examiner ID changes	Make it easier to change the ID of Candidates and Examiners in Practique	Allow the IDs to be updated	OSCE, Written
42	Filtering in Item Sets	Allow filtering of Items within the Item Set	Similar to main Item Bank filtering	OSCE, Written
43	Candidate Item Order	Show Item number when viewing Candidate Marks	Including when delivered randomly	Written
44	Angoff Standard Setting Workflow	Improvements to the Angoff standard setting process to allow judges to score within Practique	Support Angoff judging methods Allow custom Angoff arbitration	Written, OSCE
45	Improve allocation of Item Sets to Exams	Add standardised filtering to the Item Set allocation page in exam preparation		Written, OSCE
46	Item Change Differences	Ability to see the differences between items that have been changed easily	Allow approver to review the differences	Written, OSCE
47	Ebel Judging Improvements	Improvements to the Ebel Scoring process	CSV export of Ebel Score Show question paper number in addition to Item number Include question stem with Ebel judging form Improve scrolling of large Ebel Sets	Written
48	Exam Paper Cover Page	Ability to attach a PDF cover page to written papers	This would suplement the Edit Blueprint Cover template Ability to offset automatic page numbering	Written
49	Filtering Improvements	Ability to be able to multi select review state and type when filtering item bank	For example, this could be used to filter for draft or approved items	Written, OSCE
50	Time taken Report	Ability to track and report on the time taken for candidates to complete written questions	Shown for individual questions	Written
51	Formatting of Text on server	Update the styling of Practique to make text easier to read	Particularly, left align	Written, OSCE
52	Names in approval process	Ability to see te full name of a reviewer and author when reviewing items, as opposed to username		Written, OSCE
53	Support Item Set Sections in P4B	Ability to create sections within a written exam	Instruction text between the sections Configure in Item Set page Randomisation per section	Written
54	Easier ordering of Rest Stations	Ability to set order of rest stations as part of Item Set	This would work similarly to Item Set Sections	OSCE
55	Item Set Exclusion	Ability to exclude certain Item Sets from Item Set generation	Easily limit the Items available to be included in an exam To be set when generating an Item Set	OSCE, Written

Completed Features

	Theme	Summary	Features	Applicable Exam Types	Release
1	Editable Item Documents	This would allow item authors to create OSCE station documents within Practique.	Ability to create and edit basic text documents within the Resources tab These resources will be versioned inline with current resources management	OSCE	5.4
2	API	We are going to enhance the way that Practique communicates with other systems	This will allow users to have Practique integrated with other systems they use. Practique will be able to load information from, and push information to these systems This will include candidate results and feedback This will allow timetable and scheduling information for OSCE exams to be shared	Written, OSCE	5.3
3	Simultaneous Operations	Tasks that are non-instanteous(such as reporting) will run in the background.	Better communication of complex actions progress Increase speed of paper generator Improve speed of exports Improvements to automatic item set generation	Written, OSCE	5.3
4	Blueprint Constraint Satisfaction Report	Be able to see how well your Item Set meets the Exam Blueprint	Warnings on item set constraint satisfaction	Written, OSCE	5.4
5	EMQ Delivery	Ability to deliver EMQs on iPad and Practique for Browser	Support EMQ on iPad Support EMQ on Practique for Browser	Written	5.3
6	Marksheet & Item Preview	Ability to preview the marksheet and written questions, as you are editing them.	Able to generate PDF/word of single item (written) Marksheet preview to check how it would look on iPad for examiner Export Marksheet with count As tagging so that it can be reviewed pre exam by external examiners etc Generate generic paper marksheet as a backup that can be completed in an exam	Written, OSCE	5.4
7	Exam Timer	Adding an indicative timer for candidates when taking a written exam	This would not lock candidates out	Written	5.3
8	Multiple Filtering	Ability to have multiple filters at the same time when searching	Add multiple 'AND' filter for same blueprint dimension	Written, OSCE	5.3
9	Question Authoring (Tables)	Speeding up question authoring for written questions.		Written	5.3
10	Reporting Refactoring/Investigation	Introducing a suite of new reporting functionality		Written, OSCE	5.3