You can’t always expect online polls to be representative.
A research firm like Angus Reid knows that.
Sure, maybe the online population’s love of breakfast cereals matches the general population, but there are certainly areas that we can expect to be skewed by only surveying online users. I suspect that use of technology is one of those areas.
That is why I am reading the latest Rogers Innovation Report with a critical eye. Don’t get me wrong. The report has some very interesting and helpful information to understand how people are using technology. I just don’t think there should be confusion about whether this is supposed to be representative of the Canadian population at large.
The important caveat is found near the bottom of the press release under the heading of “About the survey”:
From January 13th to January 23rd 2012, an online survey was conducted among 1,403 randomly selected adult Canadians that own a smartphone or tablet and use texting, social networking, video calling, email, instant messaging, or BBM. All were Angus Reid Forum panelists. The margin of error—which measures sampling variability—is +/- 2.6%, 19 times out of 20. The results have been statistically weighted according to region and gender. Discrepancies in or between totals are due to rounding.
Let’s parse this.
It is an “online survey”, so we eliminated the 20% of Canadian households that have no home computer. The survey was of “Canadians that own a smartphone or tablet”, which knocks out half of cell-phone users who do not yet have a smartphone, and then further reduce that to eliminate the subset (likely small) who don’t use their smartphone or tablet for the kinds of things we expect them to. The results were weighted for geography and gender, but no other weighting – apparently – for age, intensity of internet use, etc. We also know that the sample was drawn from existing Angus Reid Forum participants – people who like to fill out online surveys in exchange for compensation.
Is that subset of the population representative of Canadians in general or even technology users in general?
The Rogers Innovation Report has some interesting information. Just be careful extrapolating the results.