Customer Insight is the magazine created by our in-house team of customer research experts. Bringing together features, case studies and the latest thinking on customer experience, it is a must read for anyone interested in using customer insight to improve business performance.

Home > Customer Insight > Customer Experience > Complaint Research Data - True or False?

Complaint Research Data - True or False?

By TLF Research

On 25th October 1946 at 8.30pm in room H3, the Gibbs Building, King’s College, Cambridge, an emotion-laden, and some say violent, encounter took place between the rival philosophers Popper and Wittgenstein . To this day no one can agree what happened between them in a meeting that lasted just 10 minutes. This is all the more astonishing because the events took place in front of an audience of philosopher academics trained in theories of knowledge, understanding and truth. One of the protagonists - Popper - even got the date of the meeting wrong in his own version of events - a version denounced by a follower of Wittgenstein as “false from beginning to end”. Did Wittgenstein really threaten Popper with a poker or was he just waving it about to make a point? (ref 1) Some 60 years later, we know the bare facts of time and place but there still is no account of the event which commands universal acceptance.

Strong feelings present problems. We can all feel bruised and sometimes threatened by confrontation and violent words even if they stop short of action. The senses are overloaded and so it is not surprising then that memories of encounters strongly charged with emotion differ amongst those that were present - each person giving their own version of events.

A role for reason?

Organisational decision making is founded on an assumption that rational men and women can come together to make decisions using careful arguments based on fact. They want to be as objective as possible. In this context, do emotions make any sort of contribution or as in the case of the Wittgenstein and Popper meeting, do they just get in the way of a rational assessment of what went on?

This issue often arises in complainant satisfaction surveys where what can be deep emotional memories are mined to understand the experience of complainants and to define what needs to be done to improve that experience. Can managers trust the data? This article looks at the answers to that question using data from a survey done in 2005 for a hospital complaints department.

Complainant research, as we noted at the beginning, is charged with strong emotion. Indeed complainants need an emotion like anger with escalated complaints where they have had to persist and push to get the hearing they feel they deserve. Also in a hospital, the issue may well arise from a death of a close family member. “I sincerely hope that no member of my family or friends is ever admitted to this hospital. Your negligence killed my mother- and to have to watch for 3 months just how shoddy your hospital is run has opened my eyes to a world that I never thought existed. Disgraceful- is what I think”.

Dangerous territory

Strong feelings are one problem. We now face another technical one. Complainant surveys on matters that have been escalated beyond the front line and the initial attempts to resolve often involve relatively small numbers of complainants. In this case, the sample was drawn from all closed cases in a 17-month period and a sample size of 136 closed cases was achieved with 38 people responding. This is a very small sample.

So now we face two important problems - a small sample and answers given under the influence of strong emotions. This is dangerous territory. A look at the type of questions and the data gathered may provide us with the beginnings of a map through these difficulties. The questionnaire used gathered data on a number of areas - the demographic characteristics; the stimulus for the complaint and the story of the experience; judgements based on feelings about the process gone through and future actions like recommending others to complain.

Acknowledging the limitations of our data

The demographic profile is our equivalent of knowing the date and place of our philosophers’ meeting. People have a choice whether or not to share their personal data and if they choose to do so, we may presume they have no interest in giving false information. The question here is not then about the validity of the data but of deciding what to make of that information. We certainly need to be cautious when using the demographic data. In complainant surveys, ‘representativeness’ can be a red herring. Complainants need not be representative of anyone but themselves. However here it was clear that we were hearing from a very particular part of the population. For example, 91% of those who returned a questionnaire were female. Over half owned their own homes. Most were British-born Christians. This is not a picture of the part of London where the hospital is situated. The findings confirm that the people who chose to take a complaint on up the ladder in any environment and ‘market’ are on the whole amongst the more socially confident and articulate members of our society. We must acknowledge that the information has its limitations and cannot be used to understand the experience of a wider set of patients. It is however ok for our purposes which was to gather some information about users of the escalated complaint system.

Cross checking for validity

Prior to our survey, the management had no complainant-based data at all and so these findings gave them information that they had never had before. However can they be confident that the data represents reality if so few have provided it? Here there was a way to check validity. The hospital had been gathering ‘cause’ data from the complaints handled. These confirmed our findings from the survey. 33% of the people who answered felt that the doctors had made a a mistake in diagnosing or had missed what was wrong. The next most important problem chosen by 21% was the poor attitude of staff (doctors, nurses). The hospital data confirmed that these causes are, sadly, the hardy perennials of health service complaints.

Graph 1

But cross-checking the survey findings against internal information gives an important reassurance that we are on the right track. What the quality manager needs to understand now is whether easier victories are available - the smaller, apparently less important problems that could well be easier to fix?

Multiple Problems in one Complaint

People ticked the boxes for 23 other problems indicating the multi-dimensional problem mix in the complex hospital environment revealed in these comments “In this year (2005) you have had the following continuing problems. Your disabled toilet facilities for outpatients are excellent but those for inpatients are largely of an unacceptable standard. There are still no bath (bathing) facilities for disabled inpatients. 50% of the inpatient toilet/bathrooms are of poor standard + some frankly unhygienic. The refurbished ones are lovely. In February your kitchen ran out of kosher food. Your healthcare assistants do not care enough about whether a patient is fed; can eat unaided; can open food packages; has fresh water or other fluids if recommended by doctor. There are shortages of pillows. Linen comes back from laundry with old dressings and excrement still on it.”

Not definitive but indicative

The numbers of people in our survey are too few for quality managers to be absolutely certain that the problems mentioned by our sample are those that would be picked by all. But they now have a shape to the consumer experience and can start accumulating data on those other problems where progress is possible. The longer catalogue of ‘smaller’ quality failures as perceived by the patient offer richer and potentially more productive pastures for service quality improvement action.

Glimpses of process

What about the process itself? What guidance does the complaint manager get? In contemplating the findings about the narrative of this particular escalated process, we were reminded how little of the process the complainant actually sees or is involved in as opposed to being present as a patient or a frequent visitor to a ward or clinic. They had to guess what was going on and cannot answer questions about parts of the process they have not experienced.

The really useful data from this part of the survey came on that complaint process stage prior to its escalation to the complaints office as a formal written complaint. When asked whether they had talked to anyone about their most serious problem before coming to the Complaints Department, 77% said they had. Some had gone to sources outside the hospital such as a solicitor or the Minister of Health and their MP but for the most part, they went to someone in the hospital. The hospital has a chance of pulling the complainant back from the formal procedure. 19 of the 26 people who answered this question said they had contacted someone at least once and 4 of that number up to three times.

Graph 2

The opportunities for resolution opened up by this data are important and a valuable insight into how hospitals can work with patients to minimise escalation and how much faith patients continue to have in a system that they abandoned reluctantly to take up their complaint formally. Again the numbers are less important than the general guidance on behaviours and the definition of an opportunity to intervene.

We have now seen that feeling strongly about a problem does not impede people from giving a good account of what happened. Did the survey offer a comparable insight into what the complainant felt about the process as a whole?

Don’t believe the promise

The feeling that came through most strongly was the disenchantment and disbelief of the complainant in regard to what the NHS complaint literature and information describe as the central premise and promise of the NHS complaints system - that a complaint would help ensure that things would be improved as a result of complaining. Complainants want to believe this but the system does nothing to reassure them and this arouses strong feelings.

Graph 3

Learning points

Managers need some space to consider the results calmly. A research project done a little time after the event will still uncover as it did here, sad and painful memories but the complainant is not standing over the complaint manager with a poker. So there is some space and time for learning and for improvements to be made.

The learning points are:-

  • Draw on existing hospital data, so validating survey numbers by reference to other information
  • Be honest about the limitations of any one particular survey
  • Confront and incorporate emotions by using complainants’ words to demonstrate the feelings and represent the complainants’ perspective
  • Even when numbers are small, large differences can occur that point to a very clear message. These elements combine to create data that persuades and recruits colleagues to participate in the team-based or departmental-based action that brings lasting improvement. All that remains is to tell complainants that this has happened.


1 .See the book Wittgenstein’s Poker - the story of a ten-minute argument between two great philosophers by David Edmonds and John Eidinow pub Faber and Faber 2001

01484 517575
Taylor Hill Mill, Huddersfield HD4 6JA
Twitter LinkedIn