How Polls Work

How do polls work?

How do you ask only 1,000 people and be able to say what the whole of Northern Ireland (NI) thinks? You hear this question all the time.

Well, you don’t have to eat an entire bowl of soup to know what way it tastes – if it’s properly stirred, one spoonful is enough, as the spoonful is then properly “representative” of the whole soup, and you can then say all of the soup tastes the same way. It’s the same with opinion polling – even though the number can be relatively small, if you ask the right mix of people, they can give you an accurate representation of the views of everyone. For NI polls, that spoonful is the sample that the opinion polls use, and the bowl of soup is the whole of NI!

This works because if we take e.g. a young Unionist male person. around 18-24 years old, living in a working-class area in Fermanagh, it is likely that he will have similar views on a whole range of issues to an 18-24 year-old working-class Unionist Male living in say East Belfast, – even though these two people have never met. The same can be said for obtaining opinions from say 25 to 35 year-old females with 2 children living in rural areas, middle-class Nationalists/ Republicans, the over 65 years age-group, and so on, etc. These groups are known as demographics.

What Poll Companies (like LucidTalk) then do is create a NI opinion sample with representative Nos from the various demographics that make up the total NI population. It will include little groups of all these different types of people e.g. working class and middle-class unionists and nationalists, a set No. of 18-24 year-olds, 25-35 year-olds, 65+ age-group, and people from all areas of NI e.g. West Tyrone, the Newry area, South Belfast, and so on. Most importantly the sample will be designed to have the same proportion of Males and Females, Protestants and Catholics, Unionists and Nationalists, young people to old people, rural based people to urban based people, etc. – as there are in the total Northern Ireland population.

This NI opinion sample (a spoon of the NI soup if you like – properly stirred!) is then an accurate little ‘mini-micro’ NI. The theory then is that if we ask this ‘mini-micro’ NI sample if they e.g. like baked-beans, and it produced a result that 70% do like baked-beans, and 30% don’t – then if the government ran a full NI-Wide referendum on this topic (can you imagine that!) it would produce the same result, or very close to it!

Sample Numbers

Now you can’t do this with just a few hundred people, and the maths say that a sample size of at least 1,080 is required for accurate NI polling. Indeed, LucidTalk usually target NI sample sizes of 2,000 to 3,000 (covering 17 different demographics) to ensure it is fully NI representative.

OK, so there’s a minimum size for a proper opinion sample, but surely an even larger sample will make polls more accurate. Well it doesn’t! – it’s whether the sample itself is representative. Back to the soup – a non-scientific poll or survey is like an unstirred ‘Vat’ of soup, and even if a chef drank a large amount from the top of the ‘Vat’, it would provide a misleading view if some of the ingredients have sunk to the bottom. So just as the trick in checking soup is to stir well, rather than to drink lots, so the essence of a professional scientific poll is to secure a representative sample, rather than a vast one.

How Polls are conducted

Well, nowadays, nearly all opinion polls are done online, usually over 2-3 days, with people answering on their mobile, smartphone, tablet, or laptop computer – as after all, nearly everyone today is accessible via at least one of these devices. Online polls also have the advantage of speed and ease of use, and they provide the comfort of confidentiality that’s inherent in taking a poll in private on your own PC/laptop/smartphone device, compared to an open face-to-face interview. This is why face-to-face and telephone polls usually get a much higher No. of ‘Don’t Knows/Not Sures’, because it’s easier to opt for this non-committal answer as the safe way out of a direct human contact interview situation that you may be uncomfortable with (even sub-consciously).

Plus, compared to online polls, direct interview and telephone surveys have to be carried out over a much longer period of time to obtain the requisite sample Nos – this can be several weeks and perhaps months. The problem with this, is that the world marches on, and events can happen within this long survey period which can impact people’s opinions, and thus you end up with a skewed sample which doesn’t properly represent public opinion. This is why actual elections and referendums are always polled in a 1-day period, to try to obtain a real accurate view of public opinion at that one point in time.

In fact, because online polls more closely match real elections and referendums, they have proved to be the most accurate over the past several years when compared to other methodologies. Indeed, using online polling, LucidTalk forecast the NI results of the UK EU Referendum, the 2017 NI Assembly election, and the 2019 NI Westminster election, all to within 1%.

Weighting

However regardless of how targeted the main poll is, and whether its online, face-to-face, or telephone, it’s impossible to get the initial response sample absolutely 100% precise, and totally NI representative in terms of all the NI demographics e.g. gender, age, community, etc.

This is when the practice of ‘weighting’ is used by opinion pollsters to bring all the targeted demographics totally into line. For example, it maybe that the base sample from the poll ends up with more male than female responses, but this doesn’t match the actual make-up of the NI population which is 51% female, 49% male. What polling Co’s then do is ‘multiply up’ the female responses, and ‘multiply down’ the male responses, in the base poll sample, to bring it precisely into line with the 51% female, 49% male, structure of the NI population (the NI gender demographic).

This is the process that is called ‘weighting’. The same process is then carried out across all other demographics e.g. age, community, residence area, socio-economic group, etc.. – indeed LucidTalk, if required, ‘weight’ their base samples according to 17 NI demographics to ensure it is 100% accurate and fully NI representative. This process of weighting is akin to ‘polishing’, and fine tuning, the main base NI opinion sample from the poll, to get it absolutely 100% NI representative and accurate.

The Limitations of Polls

But there are limits to polls. For one thing, beware of small movements and differences. If, for example, a poll says that 51% support something and 49% are against, we cannot be sure that supporters are more numerous than opponents, because of the small error that applies to all polls.

All that we can safely say is that the public are close to being evenly divided on the subject. A good example of this was the 2016 UK EU Referendum when all the polls showed it was going to be a narrow result either way. Even if some polls showed a slight lead for REMAIN (say e.g. 51% to 49%), that didn’t mean they were wrong, it just meant the poll was showing that the actual referendum result was going to be close – and it was!

Interestingly, the Polling Cos and the media didn’t get this point across strongly enough at the time, and an incorrect image was created that the UK EU Referendum polls were wrong, when they weren’t!

Advantages of Polls

People who sometimes criticise polls perhaps don’t realise that this mathematical sampling theory that is used in polls and surveys, is also used widely in all aspects of science, medicine, and engineering. One good example is that before new drugs are released, they are tested within the population using the exact same mathematical sampling theory as used in professional opinion polling – as obviously you can’t test a new drug on every single person on the planet. Though, this new drug testing (known as clinical trials) has to be carried out over a much longer period than opinion polls (or should be!). This point was particularly relevant in the context of the testing of a vaccine for Coronavirus.

Another major advantage that polls have over elections is that they can be run on a much more frequent basis and research opinion about current issues and topics that elections can’t tell us about e.g. What does NI think about the compulsory wearing of face masks? Plus, another advantage that polls can provide (that elections can’t) is finding out what certain groups feel about an issue. What do DUP voters think about same-sex marriage, abortion, an Irish Language Act? – What do Sinn Fein voters think about that party’s Westminster abstentionist policy? etc. What do Males and Females separately think about a topic? What do the younger age-groups think as compared to older age-groups?, and so on.

Polls are essential in any democracy as they provide a good accurate ‘feel’ for current opinion and inform debate on a wide range of issues. However, they can only give a broad approximate view of what may happen in the future, according to what the current public opinion is today. Polls (like elections) can never be 100% spot-on accurate in terms of representing current public opinion, – but like the weather forecast, they’re never that far off the mark either!