Some thoughts on workplace-based assessments (WPBAs)
RCPCH Officer for Assessment
Firstly, why the enthusiasm for WPBAs? Well, assessments such as exams and OSCEs don’t assess performance where it matters most, in other words during clinical work. WPBAs are designed to assess performance based on what doctors actually do and how they behave in real life, on a day-to-day basis.
Just to be picky, I think they should be called work-based, rather than workplace-based. They are not merely assessments carried out in the workplace but they need to assess an individual’s work. So I don’t think a WPBA includes a viva (or tutorial) in the consultant’s office or a departmental case presentation that focusing on points that have no bearing on the trainee’s clinical involvement.
The key thing for me is that the WPBA shouldn’t get in the way of the work – because that is precisely what it is trying to assess! A bit like trying to avoid the Hawthorne Effect in observational research (http://en.wikipedia.org/wiki/Hawthorne_effect)?
Next thing, formative and summative. The main justification for summative assessment is maintaining clinical standards or protecting today’s patients. Formative assessment aims to help trainees learn, by stimulating reflection through feedback and planning how to improve, thereby protecting tomorrow’s patients.
Although we think we know the distinctions, I find that we often forget or blur the distinctions, even if subconsciously. Previously we have asked trainees to do formative WPBAs but then judged their collection of WPBAs in a summative fashion. Also the original CbD and mini-CEX (pre 2013) from the RCPCH had elements of both (scoring and feedback). These have now been revised and refocused as formative WPBAs: Supervised Learning Events (SLEs). Also the LEADER (leadership), ACAT (management of acute take) and HAT (handover) were introduced as additional formative SLEs*.
Background to the revision of WPBAs
There is evidence that trainees are finding the revised SLEs more useful and the uptake has increased year-on-year. Yet, if the new formative assessments are so good from an educational point of view, why aren’t we doing more? That got me thinking about crisps. Crisps are of poor nutritional value and cost money, yet we eat an average of 100 packets a year (some more than others) – how come it is so difficult to do 12-20 SLEs a year, when the latter are free?
Perhaps the answers lie partly in the barriers that face us when trying to do an SLE? If buying crisps were like completing an SLE, it would look something like this (https://plotagon.com/63502) and the manufacturers would have gone out of business a long time ago.
So, perhaps we should learn from the snack food and retail industries? After all, they are excellent at advertising, showing us the value of their product and finding ever easier ways of parting us from our money (point of sale technology, etc.). They have realised that small barriers (such as needing the correct change) can stop people making the financial transaction and thus we have to make similar efforts to help people make the educational transaction of performing an SLE.
How to promote formative feedback and SLEs
- Build them in to the routine; we are doing the clinical bit anyway, the additional time should be for the feedback and planning next steps.
- It is more likely that the assessor will recognize the potential for a “teachable moment”, so one might expect that the majority of SLEs should be initiated by assessors, not trainees.
- Concentrate on the verbal feedback – it needs to be formative, encouraging, safe, challenging, caring and based upon what just happened. This is the quality bit.
- Find easier ways to record that the SLE has been done, along with the written summary of the feedback and action plan – mobile Apps, integrate ASSET and ePortfolio systems (these are planned).
- Measure the quality of the assessments, not merely by their number but how they have resulted in a change in behavior (one definition of education) and how they, together with other evidence (reflective notes, courses, etc.), cover the curriculum.
Try a Buy One Get One Free offer – observe someone doing an SLE for another trainee and give feedback to the assessing trainee on their ability to conduct an assessment – bargain!
Be inventive and have fun.
*Test your knowledge on the acronyms!
YouTube video link