Cross-department Benchmarking Data
March 2018
Introduction
Rotamap provides twice-yearly benchmarks for our client hospital organisations. We review these at our autumn and spring events where we also discuss interesting questions and themes that have emerged from operational data.
In this article:
Cross-department benchmarks
For our spring event held at the Royal Society of Arts in central London on 23 March 2018, Jacob Kirk and Tom Kelly presented the latest edition of our benchmarking data packs for departments using CLWRota and Medirota.
Anaesthetic boxplots
The boxplots described below are examples from the benchmarking data packs, which compare departments of the same or similar specialty in a few key areas of service delivery and highlights the variance of departments' data over the reporting period. The reporting period for all of the graphs in this article is from 02 January 2017 to 31 December 2017.
The first boxplot is showing the percentage of sessions that are delivered by extras and locums by anaesthetic departments on a weekly basis. The 13 departments with the highest average proportion of sessions delivered by extras and locums throughout the reporting period are highlighted.
Boxplot showing anaesthetic departments' extra and locum usage
We can see from this graph that the average percentage of extra and locum usage across over 120 anaesthetics departments is 8.8%.
To further explore this data we can see whether delivering a high number of extra sessions correlates with the departments' use of trainee-led sessions and their ability to deliver their planned activity.
Boxplot showing anaesthetic departments' trainee-led (solo) usage
We can see that in general, these departments are delivering below average in terms of trainee-led sessions as a proportion of total delivery. It may be that a more junior cohort of trainee doctors has led the departments to rely more upon extras, or perhaps mechanisms such as floating consultants are not in place to allow for more trainee-led sessions.
The boxplot below shows the percentage of planned vs achieved activity.
Boxplot showing anaesthetic departments' planned vs achieved activity
A trend is much less discernible when looking at the planned vs achieved boxplots. This looks at the number of sessions a department actually delivers as a percentage of the sessions a department plans to deliver, on a weekly basis. The greater usage of extras doesn't appear to be indicative of departments either struggling to achieve their planned amount of delivery, or achieving above and beyond it. This suggests that these departments are using extra sessions for different reasons.
Inter-specialty coordiation
Another form of analysis explored was a look into the different ways surgical specialties handle unwarranted variation, and how it can be tracked.
We generated dashboards detailing the percentage of extra and locum usage, cancellations and planned vs achieved activity for anaesthetics and a number of surgical specialties to allow for easy comparison. In order to highlight the significance of the variation, as opposed anything else, we stripped down the graphs to exemplify the calmness and/or severity of its variation.
An example of one of these dashboards for a urology department is shown below.
Urology department dashboard, showing the variation of extra and locum usage, cancellations and planned vs achieved activity
The most striking thing present in this data set is the very turbulent rate of cancellations and the effect that has on the planned vs achieved activity. The department appears to pre-empt the incoming loss of activity by increasing their service delivery well above and beyond their planned activity, before having it be greatly reduced. This appears to be a very chaotic way of working.
Another inter-specialty comparison can be made through looking at a boxplot for the planned vs achieved activity for the five departments explored.
Boxplot showing the comparison between anaesthetics and different surgical specialties in the same trust's planned vs achieved activity
Averagely, we can see that the departments deliver 93% of their planned activity. Shown by the size of the boxplots, anaesthetics delivers their activity with the least variation, and ophthalmology delivers their activity with the most.
Anaesthetic trainee activity
The final analysis undertaken for our Spring event was a look into how anaesthetic trainee activity differs with grade and seniority.
The bar graphs below show, by trainee grade and over the course of 2017, the percentage of module related activity and the percentage change between an agreed rota and actuality (i.e. last minute additions/removals of work).
Bar graphs showing trainee module related activity by grade
From this it can be seen that across the departments included in this study, around 30% of trainee activity is related to the modules they are undertaking.
In addition to this, it appears that ST3s and ST7s are the grades that are losing the most of their planned module related activity, and CT1s and ST4s gaining the most at the last minute.
In an attempt to gain a greater insight into why it generally appears that trainees are losing their planned module related activity, we looked into (in the same way) the usage of unsupervised trainee activity, to see if there would be any correlation.
Bar graphs showing unsupervised trainee activity by grade
From these graphs it is clear that the level of unsupervised activity increases by grade. Not only this, but trainees of all grades are added to unsupervised lists in excess of the agreed plan. It is apparent that ST5s are the most affected by this.
It is unclear, however, if this has any positive or negative effect on anaesthetic training.
Benchmarks
The benchmarking data for this season are below.
CLWRota
Medirota
Questions
If you have any questions about the above ideas or would like to know more about how to get reports from your system please contact the Rotamap support team at support@rotamap.net or 020 7631 1555.