When I’m reading an article, I can be bothered by missing punctuation, so I am not inclined to let more blatant errors slide by, if I can help it.
I’ve pointed out mix-ups on ‘millions’ versus ‘billions’ more times than I can count – and apparently I can count better than those who confuse the amounts (it really doesn’t take an advanced degree in statistics to understand the difference). A former business reporter blocked me on Twitter years ago after I called him out for writing that capital intensity was on a declining slope, but he was using the inverse of its proper formulation.
More than once, I have challenged headlines saying that wireless subscribers have declined year over year, when the company reports lower (but still positive) net additions.
In the past couple of weeks, I have been especially troubled by three key errors that keep cropping up, and keep getting repeated as the sloppiness gets copied by others.
- Error 1: The mischaracterization of the CRTC’s broadband policy, TRP CRTC 2016-496.
Let us be clear: at no time did the CRTC declare that broadband is a right. Nor was it guaranteed to all Canadians. And the Commission never set 50 Mbps (down) / 10 Mbps (up) as a “basic service standard.”
Further, please tell the Conservative Party of Canada that internet speeds are measured in bits per second, not bytes per second. The symbol used for bits is “b”; the symbol for bytes is “B”. There are 8 bits in a byte (To help remember which is which, think of big B and little b). So when the Connect Canada document says “The CRTC designated broadband as an essential service in 2015, defining broadband to be a 50 megabytes per second (MBPS) download speed and a 10 MBPS upload speed,” that sentence is wrong in at least 4 ways:
- The CRTC never designated broadband as an essential service;
- In its landmark 2016-496 policy, the CRTC defined broadband as: “Broadband is defined as an always-on connection to the Internet that provides a download speed of 1.5 Mbps and above. This connection may be delivered on fixed and mobile wireless networks using a variety of technologies”;
- speeds are generally measured in terms of bits (or megabits) per second, not megabytes (50 MBps would be 400 Mbps)
- and, the relevant decision was issued December 21, 2016, not 2015.
In a document filled with so many errors, 4 in a single sentence is an achievement worth noting.
Here is a simple distillation of what has been garbled by misreading and misrepresenting the Commission’s 2016 policy in what might be described as broken telephone. In 2016, the Commission replaced what used to be an obligation, writing:
Pursuant to its legislative mandate, the Commission is establishing the following universal service objective: Canadians, in urban areas as well as in rural and remote areas, have access to voice services and broadband Internet access services, on both fixed and mobile wireless networks. To measure the successful achievement of this objective, the Commission has established several criteria, including,
- Canadian residential and business fixed broadband Internet access service subscribers should be able to access speeds of at least 50 megabits per second (Mbps) download and 10 Mbps upload, and to subscribe to a service offering with an unlimited data allowance; and
- the latest generally deployed mobile wireless technology should be available not only in Canadian homes and businesses, but on as many major transportation roads as possible in Canada.
Note that there are several aspects to be observed. The objective itself is simple and does not contain any metrics, “Canadians, in urban areas as well as in rural and remote areas, have access to voice services and broadband Internet access services, on both fixed and mobile wireless networks.” The objective applies to voice and data, mobile and fixed.
The objective is not for all Canadians to be connected to these services and technologies, but rather for all Canadians to have access to voice and broadband internet. And as indicated above, the CRTC defines broadband as “an always-on connection to the Internet that provides a download speed of 1.5 Mbps and above.” Make no mistake, the slower speed, is not sufficient to earn a check-mark for attainment of the objective. The CRTC set out how it would measure progress toward the objective, as it wrote in 2016 at paragraph 39: “The Commission will establish criteria to assess progress towards reaching the universal service objective. These criteria will be used to identify which regions do not have the appropriate level of broadband Internet access services and to determine where further infrastructure investment is needed.”
The measurement of 50/10 is one part of the criteria used to determine progress toward attainment of the objective.
- Error 2: How much broadband is there in rural Canada?
I wrote a blog post about this problem a couple weeks ago. It is simply not true that only 40% of rural Canadians have high speed internet. The latest CRTC Communications Monitoring Report says that at at the end of 2018, 90.5% of rural households had access to broadband at speeds higher than 5 Mbps, 84.3% had access to speeds higher than 10 Mbps, and 72.1% had access to speeds of 25 Mbps. The 40.8% figure referred to those in rural communities who already had access to the CRTC’s long term universal service objective of 50 Mbps.
Further, in 2018, the CRTC ruled (in the context of funding projects from its Broadband Fund) “a minimum speed eligibility criterion of 25 Mbps download and 5 Mbps upload would be meaningful and a significant first step towards meeting the universal service objective.”
Recall that CRTC defines broadband as “an always-on connection to the Internet that provides a download speed of 1.5 Mbps and above” but its chart [Figure 9.23] in the Monitoring Report starts at 5 Mbps.
It is also worthwhile noting that the CRTC, like Statistics Canada, defines rural as “areas [that] have populations of less than 1,000, or fewer than 400 people per square kilometre.” Roughly 20% of Canadians live in rural Canada. Other countries define rural differently. For example, the US Census Bureau defines “Rural” as encompassing “all population, housing, and territory not included within an urban area” where urban areas are populations over 2,500. England “defines areas as rural if they fall outside of settlements with more than 10,000 resident population.”
So, when Canada talks about rural areas, we are speaking in terms of much lower population density than many other countries.
- Error 3: Sampling errors
It has been 40 years since I studied graduate level statistics, but it doesn’t take an advanced degree to be able to detect problems in surveys.
There is definitely a divide that separates rural and urban broadband speeds. But as I wrote recently, there will always be such a divide and I am sorry to be the bearer of such news. As such, it was inappropriate for CIRA, an agency that promotes itself as “Canada’s Internet”, to say “we really need to make sure that rural Canadians have access to the same speeds and quality that their urban counterparts do.” No we don’t and for certain we won’t. As I wrote two weeks ago, we need to take a more grounded look at what should be done to provide all Canadians with access to really high-quality digital connectivity, which is distinctly different from trying to provide everyone in remote and rural Canada with all the options available in our cities.
But we are talking about sampling error. CIRA recently published a flawed internet speed study, and submitted it as evidence in the CRTC’s proceeding examining barriers to deploying broadband networks in underserved areas.
Why do I suspect there are problems with the CIRA study? CIRA reported
- “median rural download speeds were measured at 3.78 Mbps, compared to 44.09 Mbps in urban Canada”
- “The data is based on testing data generated between May 2019 and April 2020 from a total of 86,706 urban tests and 31,734 rural tests”
- “Overall, the median download and upload speeds for both rural and urban Canadians combined over the 12 month period were 17.56 Mbps download and 6.69 Mbps upload.”
We know that rural Canada has just under 20% of the population of Canada and has a lower penetration of broadband connections than urban Canada, yet the sample is made up of 27% rural. That kind of over-sampling within a small strata might be done to help reduce sampling error, but these data points are generated by users. One should ask why rural users are more likely to choose to use the CIRA measurement tool.
Contrast the CIRA national results with what Ookla observed for Canada with its Speedtest. For March 2020, Ookla reported Canada’s average download speed was 120.98 Mbps and upload of 52.91 Mbps. The Ookla results tie in more closely to the CRTC’s Monitoring Report that says the weighted average residential download speed was 126.0 Mbps.
Ookla reports that its Q2-Q3 2019 test results are based on nearly 4,000,000 unique devices, with more than 17,000,000 tests. In that partial period, Ookla collected more than 140 times the number of test results from Canada than CIRA did in an entire year. So when Ookla shows a national average speed that is 10 times higher than what CIRA shows, with its sample size, I know which results I am more inclined to believe.
It’s hard to take some advocates seriously when they manipulate data and spread misinformation, like the way Open Media wrote recently: “a whopping 60% of rural households – and many more low income households in cities – don’t have access to high-speed Internet.”
There are serious issues to be explored in improving access to broadband connectivity in Canada. How should we set priorities for funding? Should we maximize the number of homes getting broadband for the first time? Should we work from a basis of minimum subsidy required per household getting improved service? Do funding allocations have to be evenly sprinkled across the country? What timetable can we reasonably work toward?
Let’s start with getting the facts right and demanding accuracy from those participating in – and reporting on – the discussion.
Words matter. Accuracy matters.
I’ll be watching to make sure we get it right.