Rotamap

Cross-department Benchmarking Data

March 2014

At the 2014 March Training & Management day the Rotamap team presented a series of department statistics and cross-departmental benchmarks.

The data used to generate the departmental benchmarks is generated from the Rota Management Console (RotaMC), part of the CLWRota service. RotaMC is a tool designed to generate reports and statistics for each department and allows fine-grained control over what data is included or included in performance reports. RotaMC is designed to help departments to better understand their month-on-month departmental performance and the achievements of individuals against their job-planned targets.

At every benchmark presentation, each department's name is replaced with a random two character code to maintain the anonymity of the department. Codes are changed for every benchmarking event, which presently happen in March and September each year to coincide with our March Training & Management events and September Forums.

Department activity

We showed, as we have done since the inaugural March 2011 benchmarking event, line graphs showing the levels of different types of activity stacked on top of each other.

The department activity graph shows how much trainee-led, extra and locum sessions contribute towards the total direct clinical care session delivery, as well as fluctuations in activity throughout the year. Variations in leave through the year can also be easily discerned, as can the balance of planned, unplanned and study/professional leave. We intend in future to show weekly activity graphs.

Extra and locum activity

This box plot shows extra-contractual lists and agency locum sessions as a percentage of the full activity of the department. This excludes any use of extra types that represent normal department activity. Each box shows the variance in the monthly values across a 12 month period.

The mean extra and locum session rate across the departments was 8.3% and the median was 6.8%. Departments with more than 6 months but less than 12 months of data are shown in grey.

The extra funnel plot compares extra and locum rates to the number of normal sessions done per annum by each client. Consequently, clients achieving a large number of per annum sessions are shown to the right; those performing more than the mean number of 8.43% extra sessions are shown above the mean line. The tramlines show the value at three standard errors from the mean assuming a normal distribution.

Solo sessions

This box plot shows trainee-led "solo" sessions list as a proportion of all department activity.

The two Trusts at each extreme are specialist centres but on the whole there is a tighter range than previously observed. The mean was 4.1% and the median 3.5%.

The above funnel plot clearly reveals the two Trusts at each extreme and the tighter overall range, however the results continue to show over-dispersion and little correlation between the size of a department and the number of solo sessions it performs.

For this event we included an analysis of trainee usage by seniority. This analysis split the performance of solo sessions by seniority, comparing those at ST4 level and above against their juniors.

The above trainee charts allow departments to evaluate how they are using trainees. There was a median of 50% and mean of 52% for usage of senior trainees to cover lists as a proportion of the total activity done by trainees working solo.

Heat maps

Heat maps help reveal how a factor (such as trainee solo sessions) are balanced through the week, on average. The following graphic shows the spread through the week, averaged over a year, of solo sessions, extra and locum sessions, planned and unplanned leave.

The heat maps of the above department shows that it uses a large number of solo and extra sessions on Monday mornings, where they also suffer from the most unplanned leave. The low amount of planned leave happening on a Monday morning may indicate that there are not as many people job planned to work at that time contributing further to a high usage of solos and extras.

Two-year charts

At the March 2014 Training & Management event we included a number of charts for departments with two or more years of data. The longer analysis period is useful to help discount seasonal influences such as holidays from an appreciation of each department's achievements.

The box plots show a similar performance to the single year analysis period.

Above is a two year P-chart showing the activity of one department showing both the solo usage as a percentage of all activity (top) and extra and locum activity as a percentage of all activity (bottom). Deviations from the expected range (based on the mean) are highlighted. A reduction in usage of solo sessions reveals a correlating increase in the use of extras.

Distribution curves

This is a distribution curve showing the percentage of expected activity achieved by consultant staff. Each point shows a member of staff; the dotted line the median. This shows not only where the average is, but how it is affected by those members of staff achieving significantly more or less than 100%.

Capacity graph

Several of our clients have requested graphs that compare delivered activity to planned sessions. The blue line in both graphs is taken from templated sessions in CLWRota at the time each rota is setup, and the purple line in the first graph sessions that actually went ahead. The second graph measures the different session types from the standard line as a fraction of templated demand, which contextualises the variations that can be observed from previous graphs. Hospital holidays are excluded from the demand count.

Notes

For the purposes of these graphs 'normal activity' is defined as sessions provided by a consultant or a middle grade where they are working at normal rate. 'All activity' is defined as normal activity, plus trainee's working on solo sessions, plus extra activity, plus locum activity.

Questions

If you have any questions about the above ideas or would like to know more about how to get reports from your system please contact the CLWRota support team at support@rotamap.net or 020 7631 1555.