Canada’s internet registration authority, CIRA, operates an internet performance test that appears to be flawed, likely due to a test architecture that seeks to promote the use of internet exchange points (IXPs) at the expense of providing meaningful test results.
CIRA’s performance data seems to significantly understate the actual internet speeds being delivered to Canadians. It looks like CIRA’s Internet Performance Test itself may not be performing very well.
Yesterday, the CRTC released a report that shows “that all major Canadian ISPs are delivering users with average download speeds that exceed maximum advertised rates and that overprovisioning (providing users with additional throughput) is common.” In its 2019 Communications Monitoring Report, the CRTC said the weighted average internet subscription (by year end 2018) was 126 Mbps down and 51 Mbps up. Yesterday’s report, using measurements from October 2019, shows that Canadians are receiving performance levels exceeding the subscribed rates.
And indeed, Ookla’s Speedtest has been reporting Canadian test speeds closely matching those expected speeds. As I wrote earlier this year (“Words matter. Accuracy matters”), Ookla measured download speeds of 120.98 Mbps and upload of 52.91 Mbps in March. Ookla’s most recent report for Canada (July) shows 139.11 Mbps down and 57.62 Mbps up.
But a CIRA report says “Overall, the median download and upload speeds for both rural and urban Canadians combined over the 12 month period were 17.56 Mbps download and 6.69 Mbps upload.” Its results are off by nearly a full order of magnitude.
It’s unfortunate. Regular readers know that I have been calling for more data to help us build a better understanding of digital economy issues [see for example, “Understanding the digital divide” (March 2018) and “We need more data” (October 2019)].
However, when I called for more data, perhaps I should have been clear. We need more meaningful data.
CIRA’s Internet Performance Test, unfortunately does not live up to its promise of measuring “actual performance of an Internet connection in real network conditions”. Although CIRA claims that locating its test nodes in IXPs is intended to represent the user experience of Canadians, in reality that simply isn’t true. Various groups have shown only a small fraction of Canada’s internet traffic passes through IXPs.
The CRTC’s internet speed tests measured traffic using servers at exchange points but also looked at traffic going to real applications.
If CIRA’s national averages are off by a factor of ten, how can we draw any meaningful conclusions from its reports? How do we know which areas actually fall short of the service objective? How can researchers meaningfully determine funding requirements for under served areas?
CIRA’s testing is not living up to its promise. As such, it fails the test of contributing to a better understanding of issues.
[DELETED] This comment was deleted because the information contained within it was not true.
Perhaps Geo fenced data would be more meaningful… the digital divide occurs in rural areas and urban stats should be excluded from the numbers