"The NHS will last as long as there are folk left with the faith to fight for it"
Aneurin Bevan

Friday 25 March 2011

Fundamentalism

Imagine there is a hypothetical service that we all use and you are asked to create a survey about change in the system. Nothing exists without change. Everything has to change, your survey is to determine the opinion of the public about how much change they think should happen. You have decided that there will be three responses:

Do you:

  1. think the service is almost perfect and only minor changes are needed?
  2. think the service needs some changes, but nothing too drastic?
  3. think the service is so bad that it need completely replacing?
Do you think this is acceptable for the middle response? After all, if the respondent thinks that the service works fine then they will choose #1, and if they think the changes needed are fundamental, they would choose #3. So in this case #2 takes a middle line: changes are needed and improvements can be made. If the respondent wanted drastic, fundamental changes, they would choose #3.

So, now look at the latest survey from Ipsos MORI for the Nuffield Trust (pdf).



The first response says that the NHS needs only minor changes, the last response says that the NHS needs replacing (the drastic option). The middle-of-the-road choice, says that some changes are needed to make it work better. Well, of course that is true, and we can all come up with changes that will make the NHS better. But the significant issue is that Ipsos MORI says that if you choose the middle option you want fundamental changes. Surely fundamental changes means drastic changes? A fundamental change means changing the very basis upon which the NHS is founded. A fundamental change implies that the part of the NHS changed should be rebuilt. This is not middle-of-the-road at all, it is drastic.

Ipsos MORI are clearly leading the respondent into saying something they do not want to say. Further, look at the second response again:

There are some good things in the NHS but some fundamental changes are needed to make it work better.
Now take a look at the summary of the survey. Ipsos MORI say:

Just over half of respondents (56 per cent) thought fundamental changes to the NHS were needed, while a third thought that only minor changes are needed to make it work better
Well, no, the results do not say that. The results said that 91% people said that there were either "some good things in the NHS" or "on the whole the NHS works pretty well". That is quite a ringing endorsement.

Ipsos MORI added the "fundamental changes" bit into the middle response precisely because the intention was to try and get a result that says that the NHS needs fundamental, drastic changes. A better middle response would be this:

There are some good things in the NHS but some changes are needed to make it work better.
This says that the NHS needs changes, buit not drastic changes (that's #3). I reckon that this would still get the 56% vote.

Fundamental is a very important word. It is rather unfair to use it in the question in this way.

You will hear this result ("over half of the public want fundamental changes in the NHS") quoted by government ministers over the next few months. In the absence of any evidence that what they are doing will have any beneficial effect, they will still say: "but the majority of the public want it".

2 comments:

  1. Ipsos Mori state on their website that they have a variety of accreditations http://www.ipsos-mori.com/quality.aspx It might be worth reviewing the criteria for these to see whether they've broken anything as a result of asking this misleading question

    ReplyDelete
  2. This is clearly a 'push-poll' and it is very disappointing to see from Ipsos-MORI who should know better.

    AFZ

    ReplyDelete