Cross-department Benchmarking Data
September 2012
At the 2012 Forum the CLWRota team presented a series of benchmark graphs and charts.
For this series of benchmarks we took the data from a very early version of our Rota Management Console which allows to departments to configure the way their data is counted. We introduced some new statistical measures including the use of P-charts and funnel plots.
Each department's name has been replaced with a random two character code to maintain anonymity.
Department activity
As with previous releases we generated department line graphs showing the levels of different types of activity stacked on top of each other.
The department activity graph is useful for showing how various factors, such as trainee solo sessions or the extra and locum sessions, contribute towards the total DCC session delivery. Variations in leave through the year can also be easily discerned, as can the balance of planned, unplanned and study/professional leave.
Trainee solo activity
This box plot shows Trainee solo sessions (trainee led lists) as a proportion of the full activity of the department. Each box shows the variance in the monthly values across a 12 month period. The mean solo session rate across the departments was 5.2%, with a median of 4.3%.
Departments with more than 6 months but less than 12 months of data are shown with greyed boxplots.
Extra sessions
This box plot shows the extras and locums (paid-for additional activity) as a proportion of all department activity. We have excluded any use of extras facility in CLWRota where the session was used as part of normal department activity.
The mean extra session rate was 5.2% and the median 4.0%.
Study and Professional Leave
The benchmark shows the proportion of study and professional leave as a proportion of normal activity. Normal activity is defined as consultant and NCCG direct clinical care sessions. The mean study and professional leave rate was 4.8% and the median 4.6%.
The study and professional leave boxplots have a large variance (size of box) due to the spread of study and professional leave. For example there is a very little study and professional leave over the summer months.
Funnel plots
An issue with the boxplots is that they don't show the relative sizes of the departments. For example large university teaching hospitals might expect different outcomes to smaller DGHs. The funnel plot shows the annual values for each measure plotted against the total number of sessions for the department.
The outside tram lines show the value at three standard errors from the mean assuming a normal distribution. A significant number of departments fall outside of three standard errors, which is a sign of what is termed 'overdispersion'.
The x-axis shows percentages and the y-axis shows sessions per annum. There is a significant cluster of departments below 10,000 sessions per annum at around the 3% mark.
Further overdispersion can be found when comparing departments with regard to extra & locum activity as a proportion of all DCC activity:
There is one significant outlier which may distort the results for the other departments.
Heat maps
We have introduced a series of heat maps which show concentration of a particular type of activity within a session across a week. The heat maps are made up of 10 squares corresponding with the sessions of the normal working week in the same format as the rota, the columns are the days of the week, with the top row representing 'am' sessions and the bottom row 'pm' sessions.
The stronger the red colour of each box the greater the proportion of sessions of that type. We have generated heat maps for extra sessions usage, solo session usage, planned leave and unplanned leave. A high use of extras or solos in a particular session might reveal that the department is short at that time.
P-charts
A P-chart is an attributes control chart used to examine the proportion of non-conforming values within a sample. They reveal how a process changes over time and can be used to determine whether change is predictable and part of normal variation within a system or there is an outstanding value.
The tram lines on each p-chart show values falling within three standard errors of the mean and as such these values can be considered as normal variance within a system. Therefore a value falling outside of the tramlines and highlighted in red or green shows a special case of variation.
Notes
For the purposes of these graphs 'normal activity' is defined as sessions provided by a consultant or a middle grade where they are working at normal rate. 'All activity' is defined as normal activity, plus trainee's working on solo sessions, plus extra activity, plus locum activity. Each department box is made from the range of values, one for each month, from the period 1st February 2011 to 31st January 2012.
Questions
If you have any questions about the above ideas or would like to know more about how to get reports from your system please contact the CLWRota support team at support@rotamap.net or 020 7631 1555.
Please note that, as with all information on this site, all images and information on this site are copyright © Rotamap Ltd.