Surveys, samples and opinion polls

Surveys collect information from a specific group of people, or a random sample of a wider population.

A well-conducted survey can provide useful insights into people’s opinions or behaviour, but there are several things to watch out for that can make the results of a survey unreliable. 

Who commissioned the survey?

Commercial companies can commission good quality surveys, but look out for PR spin. PR agencies will sometimes commission a survey just to get a product or survey into the newspapers.

They are often on trivial subjects designed to create a “news hook”. For instance, 80% of British families will be eating British turkeys this Christmas, says a survey by the UK association of turkey farmers. There are plenty of reputable companies that design and run high-quality surveys, for example IPSOS MORI. 

Who did they interview?

Some surveys involve a group that doesn’t properly represent the wider population. For instance, if we wanted to find out what time most Londoners wake up, interviewing commuters passing through Underground stations at 7am wouldn’t tell us anything useful about Londoners’ sleeping habits as a whole.

How many people did they ask, and how many responded?

Surveys reported in adverts will often include a statement like “78% of people agree that the product did …” But if we don’t know how many people were asked the question, or what proportion of them actually responded, the results may be meaningless. There is no magic number that tells you how large a sample should be to give valid results. But as a rough guide, a political poll in the UK usually involves at least a thousand people. Tens of thousands of respondents would be needed for a survey to assess quality of life in different European countries.

How were the questions asked?

How people respond to a survey can be influenced by the person asking the questions - whether this is deliberate or not. People can be asked leading questions (questions worded or ordered such that they’re more likely to produce a certain answer), or the person being asked the questions may not give an honest reply because they’re embarrassed, don’t remember properly, or feel that the truth would cast them in a negative light.

Read Making Sense of Statistics to learn more about surveys.

(Note: some fields of science commission ‘surveys’ of land, water or the night sky – these are technical surveys which aren’t the kind we mean here).

Tags: evidence

Sign in

×

Sign up

×

Share

×