New assessment tools

EPAs have been developed to map to the Learning Outcomes for each Level of the OST Curriculum 2024 in the Patient Management domain. The purpose of the EPA is to assess the level that a trainee can be entrusted with independent practice in the context of meaningful clinical activity. The EPA allows the NCS to make and document a decision about the degree of independence a doctor can safely work with.  This decision is made about a relatively wide area of practice, which is described by high-level, overarching Learning Outcomes. The EPA requires simultaneous proficiency in multiple competencies and is a more suitable focus for assessment than separate competencies.  The ePortfolio allows the NCS to make summative entrustment decisions and select the degree of entrustment using the following scale:

• Observing
• Needs Direct Supervision
• Needs Indirect Supervision
• Competent to Level

Longitudinal development is captured as the trainee:

• moves up through the entrustment scale within the Level at which they are working.
• reaches competence for the Learning Outcomes of that Level and advances to the next Level.

The EPA form has been designed to include a self-assessment element to facilitate professional insight and reflection on the evidence to be used to reach the judgement. ESs review EPAs and complete the ESR to inform decisions taken by the ARCP panel at the end of each training year or at critical advancement points.  While longitudinal, periodic observation of performance is a key aspect on which to base the entrustment decision, NCSs must consider some mandatory pieces of evidence. Additional evidence may be supplied by the trainee to demonstrate competence across all the requirements for each Level. See the Level Guides (1, 2, 3, 4) prior to preparing the EPA, particularly about the other evidence that trainees may be either required or choose to submit. Elective evidence requesting focused, formative feedback may also be included wherever the NCS has indicated that the trainee is not achieving the expected degree of entrustment.

As well as a confirmation statement about whether the evidence presented corroborates entrustment to practice independently in the area under consideration, it is required to provide a narrative to support that decision and suggested areas for further development. Where the degree of entrustment is below competent, a narrative is required and must include the evidence that is needed to increase the entrustment recommendation.  Space is also given for feedback about what was particularly good and actions that have been agreed for further development.

How to complete an EPA

• Completed examples of EPAs for Competent, Direct or Indirect Supervision are in the Level 1 Guide and Level 3 Guide. Please note that these Word templates will look different in the ePortfolio. Some fields in the EPA will be auto-populated by the ePortfolio.
• The trainee should complete the self-assessment prior to review by the NCS.
• The trainee should link to the evidence they have on the ePortfolio to demonstrate that they have achieved each requirement. It is about quality, not quantity, and one piece of evidence may be used for more than one Learning Outcome.
• If the first example linked to gives a clear indication that it meets the standards required, it is up to the NCS to decide how thoroughly to review links to evidence. A common-sense approach would be for the trainee to link only to good quality example(s), and the NCS could then decide whether to go further. The trainee should not link large numbers of documents, each with weak evidence (e.g. anonymised patient letters), in the hope that it will be acceptable when summated.
• NCSs should note who has supervised and signed off WpBAs and alert ES if there are any concerns regarding the appropriateness of supervisors chosen by the trainee.
• The ‘other mandatory requirements’ listed in the EPA are mandatory in the sense that these are necessary areas of evidence for the NCS to consider when making their decision as to the level of entrustment for the relevant EPA. These requirements are not subject to mandatory quantitative expectations such as set numbers of CbDs or durations of periods of observation.
• Some of the mandatory WpBAs in Level 1 are required to be repeated at higher Levels (e.g. CRS1). It is thus important for the trainee to demonstrate development of the skills required over time, and to be able to apply and tailor them to the SIAs. Other mandatory WpBAs do not need to be repeated and therefore the Level required to be demonstrated is that expected of a competent independent practitioner (i.e. a newly qualified consultant).
• It is important to remember that competence in the EPA should be benchmarked to the Level of the assessment. Where the final entrustment on the scale is below the maximum possible, the assessor will need to add a narrative about the evidence that is needed to increase the entrustment recommendation. Where a trainee has not yet had the experience or opportunity for a higher degree on the scale to be attained, this can be indicated without an exhaustive list of all areas to be developed. However, where a trainee might have been expected to achieve a higher degree on the scale, it is imperative that the narrative is detailed and specific to help the trainee understand what is required.
• NCSs should view any completed MARs (there is no mandated number) as part of the evidence to be used to make the entrustment decision.
• Some competencies may appear in an EPA of more than one SIA (see Table 3). Where a competency has already been signed off in one EPA, this can be taken as evidence of competence and transferred to the other EPA. The competency assessment does not need to be re-assessed. 

Table 3 – Competences that appear in more than one EPA
Level	Competence	SIA
Level 3	Local anaesthesia (theatre)	Cataract, Oculoplastics & Orbit, Cornea, Vitreoretinal
Level 3	Aqueous / vitreous biopsy (clinic or theatre, depending on SIA)	Cataract Surgery, Vitreoretinal Surgery, Urgent Eye Care
Level 3	Botulinum toxin injection (clinic or theatre, depending on SIA)	Oculoplastics, Neuro-ophthalmology, Ocular Motility
Level 3	Interpretation of FFA and ICG (clinic)	Medical Retina, Uveitis
Level 3	Interpretation of orthoptic assessment/examination (clinic)	Neuro-ophthalmology, Ocular Motility, Paediatric Ophthalmology
Level 3	Periocular and intraocular drug delivery (theatre)	Cataract Surgery, Uveitis
Level 3	Corneal gluing (clinic)	Cornea & Ocular Surface, Urgent Eye Care
Level 4	Temporal artery biopsy (theatre)	Neuro-opthalmology, Oculoplastics & Orbit

• Where a WpBA is not completed, it will be important for the trainee to provide other evidence to demonstrate that they have achieved a specific outcome. There are few mandatory DOPS across all SIAs in OST Curriculum 2024; however, the relevant practical skills must be evidenced for the EPA.
• A feedback meeting should be arranged if the trainee and NCS disagree about the level of entrustment. The ePortfolio will only accept an EPA as complete if the chosen degree of entrustment is the same.
• The EPA may need to be repeated If the NCS decides that it is inadequate, and extra time might be needed. This should be highlighted to the ES.
• The trainee is signed off by the NCS as competent to a Level only when all the Learning Outcomes of the SIA for that Level have been achieved.

GSATs have been developed to map to the Learning Outcomes for each Level of the OST Curriculum 2024 in all six generic Domains. This tool is used to make a global professional judgement of a range of different skills and behaviours to make decisions about advancing to the next Level of the training programme and eventually to consultant practice at CCT. Whilst the Learning Outcomes in the Patient Management domain describe activities that meet the criteria for consideration of entrustment, those in the generic Domains are competencies rather than activities.

The GSAT is, first and foremost, a self-assessment by the trainee.  It is used by the ES to make a recommendation and inform the ARCP process. Two assessments should be undertaken in any training year, one at the mid-point and one towards the end of the year, in advance of the ARCP.

Evidence can come from multiple different sources such as: case-based discussions; undertaking mandatory Trust training, e.g. on Health and Safety; passing exam(s); personal audits; departmental audits; attending teaching sessions. Examples of evidence are listed in each Level Guide (1, 2, 3, 4) and are not mandatory. They are simply there to provide guidance. The emphasis should be on the quality of evidence, and it may be that one piece of evidence can demonstrate achievement against more than one Learning Outcome.  It may not be possible at times to provide evidence for every Learning Outcome and a subjective judgement may need to be made.

There is a GSAT for each Level of training that covers the same six non-clinical Domains, and for each Level there are different Learning Outcomes, which are specified on the GSAT forms.  For advice about the expectations for each Level, there is a link on each domain title in the GSAT to the relevant section of the RCOphth Curriculum microsite which contains a helpful list of descriptors.

How to complete a GSAT

• The trainee should pre-populate the GSAT for their current Level with the relevant supporting information.
• The trainee should link to the evidence they have on the ePortfolio to demonstrate that they have achieved each specific outcome. It is about quality, not quantity, and one piece of evidence may be used more for more than one Learning Outcome.
• Unlike the EPA, the GSAT does not mandate any WpBAs. It is up to the trainee to identify the evidence they think will best demonstrate achievement of a learning outcome.
• The trainee can use the same box to comment; for example, if there is no specific evidence but they can demonstrate they have achieved a Learning Outcome or, if they consider they have not, offer a plan as to how they will acquire the evidence.
• The ES reviews the trainee self-assessment, including links to evidence, once the trainee has submitted it for review. They will indicate whether, in their view, the trainee meets or does not meet expectations for each Learning Outcome.
• If the first example linked to gives a clear indication that it meets the standards required, it is up to the ES to decide how thoroughly to review links to evidence. A common-sense approach would be for the trainee to link only to good quality example(s), and the ES could then decide whether to go further. The trainee should not link large numbers of documents, each with weak evidence (e.g. anonymised patient letters), in the hope that it will be acceptable when summated.
• The ES should comment on the Level achieved overall in each domain. Both trainee and ES should agree whether there are areas that need work before advancing to the next Level. Comments in specific boxes, after each domain is considered, are especially important if the trainee does not meet the expected requirements for the Level.
• The GSAT may need to be repeated if the ES decides that it is inadequate, and extra time might be needed.
• All Learning Outcomes must be completed in all Domains across the Level to advance to the next Level.

This summative assessment tool has been designed to reflect the multi-professional working environment and to capture the opinions of other consultants, senior trainees and other colleagues such as orthoptists, optometrists, advanced clinical practitioners and senior technicians, and nursing staff, who have supervised the trainee and are able to comment on important aspects of clinical performance.

The form is additional to the Multi-Source Feedback (MSF) as it is intended to focus specifically on clinical performance.  There are several professional practices, competencies and skills that the contributor(s) is asked to comment on. They should indicate whether the trainee meets or does not meet expectations in each of the areas that are relevant to their experience with the trainee.

The responses contribute to EPAs and a list of appropriate respondents is agreed with the NCS at the outset.  The ES will also view these assessments and reflect them in their ESR.  This tool has been designed to capture the overall impressions and observations of clinicians and healthcare professionals who have supervised the trainee – it is not specific to a particular case or cases, nor is it specific to a Level. It allows them to comment on the clinical knowledge and skills of the trainee and supports the completion of the EPA. There is no requirement to check formal evidence on the ePortfolio.

The NCS reviews the MAR(s) before completing the EPA. Free comments are encouraged to allow the NCS filling in the EPA to gain a better understanding of how the trainee is progressing, as well as the ES to reflect them in their report. The free comments are visible to the trainee and can be useful to receive constructive feedback. The ES will also review the MAR and consider whether there are any issues to follow up or areas of concern.

The MAR can also be used at any time to record any concerns with a trainee. There is no mandatory number of MARs, although normally they will be completed twice a year to support the EPA.

How to complete a MAR

• The NCS determines at the outset the number and likely contributors to the MAR for every rotation on a six-month rolling basis.
• Each contributor should consider whether the trainee meets or does not meet the expectations in each area, from their own experience with that trainee. If the contributor has no evidence or experience in that area, they should choose N/A.
• Comments are encouraged but not mandatory. The more information the contributor is able to offer, the more helpful the MAR will be.
• The ES will review the MAR and consider whether there are any issues to follow up or areas of concern.