Rotamap

Cross-department Benchmarking Data

September 2012

At the 2012 Forum the CLWRota team presented a series of benchmark graphs and charts.

For this series of benchmarks we took the data from a very early version of our Rota Management Console which allows to departments to configure the way their data is counted. We introduced some new statistical measures including the use of P-charts and funnel plots.

Each department's name has been replaced with a random two character code to maintain anonymity.

Department activity

As with previous releases we generated department line graphs showing the levels of different types of activity stacked on top of each other.

The department activity graph is useful for showing how various factors, such as trainee solo sessions or the extra and locum sessions, contribute towards the total DCC session delivery. Variations in leave through the year can also be easily discerned, as can the balance of planned, unplanned and study/professional leave.

Trainee solo activity

This box plot shows Trainee solo sessions (trainee led lists) as a proportion of the full activity of the department. Each box shows the variance in the monthly values across a 12 month period. The mean solo session rate across the departments was 5.2%, with a median of 4.3%.

Solo/total boxplots for the year from August 2012

Departments with more than 6 months but less than 12 months of data are shown with greyed boxplots.

Extra sessions

This box plot shows the extras and locums (paid-for additional activity) as a proportion of all department activity. We have excluded any use of extras facility in CLWRota where the session was used as part of normal department activity.

Extra and locums for the year from August 2012

The mean extra session rate was 5.2% and the median 4.0%.

Study and Professional Leave

The benchmark shows the proportion of study and professional leave as a proportion of normal activity. Normal activity is defined as consultant and NCCG direct clinical care sessions. The mean study and professional leave rate was 4.8% and the median 4.6%.

The study and professional leave boxplots have a large variance (size of box) due to the spread of study and professional leave. For example there is a very little study and professional leave over the summer months.

Funnel plots

An issue with the boxplots is that they don't show the relative sizes of the departments. For example large university teaching hospitals might expect different outcomes to smaller DGHs. The funnel plot shows the annual values for each measure plotted against the total number of sessions for the department.

The outside tram lines show the value at three standard errors from the mean assuming a normal distribution. A significant number of departments fall outside of three standard errors, which is a sign of what is termed 'overdispersion'.

The x-axis shows percentages and the y-axis shows sessions per annum. There is a significant cluster of departments below 10,000 sessions per annum at around the 3% mark.

Further overdispersion can be found when comparing departments with regard to extra & locum activity as a proportion of all DCC activity:

There is one significant outlier which may distort the results for the other departments.

Heat maps

We have introduced a series of heat maps which show concentration of a particular type of activity within a session across a week. The heat maps are made up of 10 squares corresponding with the sessions of the normal working week in the same format as the rota, the columns are the days of the week, with the top row representing 'am' sessions and the bottom row 'pm' sessions.

The stronger the red colour of each box the greater the proportion of sessions of that type. We have generated heat maps for extra sessions usage, solo session usage, planned leave and unplanned leave. A high use of extras or solos in a particular session might reveal that the department is short at that time.

P-charts

A P-chart is an attributes control chart used to examine the proportion of non-conforming values within a sample. They reveal how a process changes over time and can be used to determine whether change is predictable and part of normal variation within a system or there is an outstanding value.

The tram lines on each p-chart show values falling within three standard errors of the mean and as such these values can be considered as normal variance within a system. Therefore a value falling outside of the tramlines and highlighted in red or green shows a special case of variation.

Notes

For the purposes of these graphs 'normal activity' is defined as sessions provided by a consultant or a middle grade where they are working at normal rate. 'All activity' is defined as normal activity, plus trainee's working on solo sessions, plus extra activity, plus locum activity. Each department box is made from the range of values, one for each month, from the period 1st February 2011 to 31st January 2012.

Questions

If you have any questions about the above ideas or would like to know more about how to get reports from your system please contact the CLWRota support team at support@rotamap.net or 020 7631 1555.