I’m a numbers guy. An eternity ago, I studied statistics and have a graduate degree in the subject. I helped edit a university level textbook in econometrics.
So I get somewhat frustrated when I see the misinformation from pseudo-statistical studies being circulated with viral velocity, despite flaws that should have been immediately apparent, even to those with just a modicum of numeracy.
For example, last week, an article in the National Post cited figures from Cable.co.uk claiming that for home broadband “Canadians paid $0.94 (U.S.) per megabit per month. The French paid US$0.48, the Americans US$0.20, the Swedes US$0.12.” That same study was discredited last April in a blog post by CWTA. “The questionable quality of the Cable.co.uk methodology is made crystal clear when its 2020 report is compared to its findings in 2019. For 2020, Cable.co.uk concluded that the average fixed broadband monthly cost in Canada was US$76.14 while the average monthly cost in 2019 was $34.86.”
Does anyone really believe that Canadian broadband monthly prices more than doubled between 2019 and 2020? Doesn’t this show that there just might be a problem with the methodology being used by Cable.co.uk? I wrote more detail about those issues last April in “When flawed data leads to flawed conclusions” and a year ago in “Look at the data”.
Price comparisons from Rewheel have merited dubious distinction for systemic problems, referred to as “a careless mish-mash of data points from which no reliable conclusions can be drawn.” As I wrote a year ago, critics have said “The Rewheel story is easy to understand. It is also completely wrong.”
Given the many theoretical and practical flaws and errors contained in the Rewheel study, we find it of no value when comparing prices internationally or establishing the level of competition in a country. A warning label informing readers about the lack of intellectual rigor and the misleading and incorrect nature of the Rewheel study’s results is appropriate and recommended.
For a dozen years, I have been writing about flawed studies being used to support advocacy efforts (see: “Mixing passion and scholarship” from November 2009).
As the year winds down, I will try to put together some general notes on how to read price comparison studies, and what to watch for.