I am saddened to be writing this post, a rare weekend event for me.
On Friday evening, I saw a stream of articles on my Twitter feed promoting a number of localized versions of a Canadian Press wire story (“Test results for Chatham-Kent show many users could not access minimum speeds needed for everyday internet use”) saying research show that in most communities tested, “users could not access minimum speeds needed for everyday internet use”. I found various versions on National Post (with local versions for Chatham, Halifax, Hamilton, Minden Hills, Quebec City, Regina), CBC, Ottawa Citizen, Global TV. I am certain there are others floating out there.
The story is fatally flawed, from the premise through the test architecture and therefore, of course its conclusion.
A Local News Data Hub analysis of tests completed in 53 communities found that in 51 places, including Chatham-Kent, the majority of speed test results did not meet the basic service objectives for both upload and download speeds set by the Canadian Radio-television and Telecommunications Commission (CRTC).
The results of the speed tests were irrelevant, because the tests weren’t measuring the CRTC’s basic service objective. The researchers don’t appear to even understand what the CRTC’s broadband objective is.
The Ryerson Local News Data Hub website describes its research goal:
Our goal was to compare the test data to the basic service objectives set for internet speed by the Canadian Radio-television and Telecommunications Commission (CRTC). The commission says Canadian households should have internet connections with access to broadband speeds of at least 50 megabits per second (mbps) for downloads and 10 mbps for uploads.
Unfortunately, the researchers apparently misunderstood the CRTC’s broadband service objective. The stories state “The commission says Canadian households should have internet connections with access to broadband speeds of at least 50 megabits per second (mbps) for downloads and 10 mbps for uploads.” I will forgive the use of lower case “m” (which refers to “milli”), instead of the proper upper case “M” (for “mega”). The more important flaw, indeed the most fundamental flaw, is that the CRTC’s objective is for Canadians to have the ability to subscribe to a broadband service with 50 Mbps down; 10 Mbps up; and an option for unlimited data. The CRTC didn’t say that this was the minimum speed needed for everyday use.
It is a subtle but important difference. By access, the CRTC was referring to access in the marketplace, not a broadband access line. The CRTC recognized that it was perfectly fine for people to choose a lower speed to meet their needs; the Commission wanted to make sure we had the ability to choose a faster (50/10) connection.
But the second flaw in the research is the use of the deeply flawed CIRA test architecture for its dataset. CIRA’s test does not, and cannot measure the internet access speed for consumers. And it certainly cannot measure whether the CRTC’s broadband service objective is available in an area.
Earlier this year, I wrote about flawed work coming out of Ryerson’s Leadership Lab [see “Mythbusting Canadian telecom”], that also misunderstood the CRTC’s broadband service objective.
The Local News Data Hub has an Ethic Policy that failed to detect these fundamental flaws: a flawed premise; and, a flawed architecture for gathering measurements. Clearly, an opportunity for the Hub to exercise its promise: “We will therefore critically review and update these policies on a regular basis.”
With two significant strikes this year, Ryerson needs to improve the quality of its broadband research. These are some basic flaws impacting the quality of information – indeed, misinformation – being distributed in the public sphere.
Sadly, Ryerson isn’t alone in failing grades for its communications research scholarship. But I will save that discussion for another time.