Chris Beeley – The Patient Feedback Challenge

Well, it finally launched, the patient feedback website that we created as part of the Patient Feedback Challenge. The evaluation section was part of a partnership with the Involvement Centre of Nottinghamshire Healthcare NHS Trust and the web design agency Numiko to build a website which can help us to collect feedback data, summarise and report on it and share the actions and learning which have resulted. Go have a look.

I’m going to talk a little bit about some of the thinking behind our approach on the Patient Feedback Challenge. Similar thinking has motivated recent developments within Nottinghamshire Healthcare NHS Trust as well as in the wider NHS but there is a long way to go before the whole NHS nationally embodies the approach being advocated here.

NHS data systems are, quite naturally, built around very strict data management which emphasise the security of data and ensuring accountability of data loss or misuse. This is right and proper. However, very frequently, clinicians, managers, auditors and evaluators do not need to access highly sensitive personal information. In certain cases, information can be shared freely over the internet, as with the data collected on patient experience within the Patient Feedback Challenge (survey responses, Patient Opinion stories, and specific extracts from the PALS database).

The skills and technologies to rapidly analyse and disseminate data of this kind where rights access is not an issue is presently lacking in many NHS settings. Data analytic solutions, where they exist, are often large and complex and are complex and expensive to set up and modify. Where solutions are brought in from private sector providers these tend to be inflexible and it can be difficult for an organisation to achieve the results they want using off the shelf technologies (for example, integrating and resharing different types of data, such as the survey, Patient Opinion, and PALS data involved within the Patient Feedback Challenge). A limitation in the data architecture of many NHS organisations is the tendency for data to exist in silos, with each department maintaining its own separate database with no capacity to combine or reshare any of the data. Often the existence and location of datasets is unknown to others working elsewhere in a Trust.

Another problem endemic in many NHS settings is Spreadsheet addiction (e.g. Burns, 2004). This has been given as a possible culprit in JP Morgan Chase’s having lost 2 billion dollars in May 2012 (from

‘James Kwak, associate professor at the University of Connecticut School of Law. noted some interesting facts in JP Morgan Chase’s post-mortem investigation of the losses. Specifically, that the Value at Risk (VaR) model that underpinned the hedging strategy:

“operated through a series of Excel spreadsheets, which had to be completed manually, by a process of copying and pasting data from one spreadsheet to another”, and “that it should be automated” but never was.’

The process of analysing and presenting Service User and Carer Experience survey data began in a similar fashion, with spreadsheets and macros being used to produce summary graphics, and reports manually cut and pasted together. Fortunately for us, before we lost 2 billion dollars this way there was a mishap and it became obvious that the process needed to be automated in order to meet the reporting timescales. An incremental approach was adopted, and each stage of automation brought more complex and better data summaries, so that within a few years reporting on survey data had gone from a task that was difficult for one person to perform quarterly alongside other tasks to a level of complexity that would demand more than one individual working full-time for the whole of each quarter just to meet each quarterly timescale.

The patient feedback challenge has allowed us (with our friends at Numiko) to clear the final hurdle and to remove the last vestiges of human effort from the regular reporting, which means that quarterly survey reports and custom extracts will be available 24 hours a day, 365 days a year to anyone in the world with an internet connection. Managers will be able to see the latest results from their area and anywhere else in the Trust at any point in the reporting cycle and service users and carers will be able to transparently query and scrutinise all of the patient feedback data that we collect.

All the technologies that were used are simple, free, and open source. Using free software such as R and reporting technologies such as HTML and LaTeX means that not only are there no costs associated with buying or licensing software but also that analysis and reporting technologies are simple to use straight away. In a process known as agile software development (Beck et al., 2001) all of the technologies that have been produced have been put to immediate use. Early on, automation was only possible of graphics and manual effort was required to put together the report and write textual summaries. Later graphics and text were both automated and only formatting and final edits were performed by a human being. With the completion of the Patient Feedback Challenge now the formatting can also be automated. But at each stage the results of the technology were being used live in order to meet the demands of the organisation. This has the twin benefit of giving an immediate payoff to investment in workforce costs developing the software as well as giving the opportunity for the technology to be refined in use. The results of each stage of development were obvious when each reporting cycle was complete and work has been undertaken throughout to better serve both the Involvement team who deliver the survey results as well as the clinicians, managers, service users, and carers who make use of them.

There were two bits of coding work that made up the majority of my contribution, the quarterly report function which summarises feedback at all levels of the organisation in a series of hyperlinked webpages and the custom search function which uses the very simple Shiny package to allow users to query different types of data and search by service type, demographic feature, keyword, etc. The code for both can be found at my GitHub page.

Dr Chris Beeley, Institute of Mental Health


Beck et al. (2001). Manifesto for Agile Software Development. Agile Alliance. Retrieved 20-8-2012

Burns, 2004. Spreadsheet addiction. Retrieved 16-5-2012


Leave a comment

Filed under Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s