Where were you when the lights went out?

The following article appeared first on National Newswatch yesterday.

On November 9, 1965, a relay tripped on a transmission line at the hydro-electric plant in Queenston Ontario, near Niagara Falls, setting in motion a 13-hour power blackout that disrupted electric service for more than 30,000,000 people in Ontario, Connecticut, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, and Vermont. Excess power from that first line migrated to other power lines, that quickly became overloaded, causing their own relays to trip. In a matter of just 10 minutes, the blackout propagated across more than 200,000 square kilometres.

Most telephones (land lines at the time) kept working thanks to emergency generators that were a standard part of central offices.

Following the incident, new monitoring equipment, procedures and systems were introduced, but that didn’t prevent another large scale power grid failure 38 years later, in August, 2003, knocking out service for 55,000,000 in Ontario and 8 US states.

The root cause of the 2003 blackout was determined to be a bug in the alarm system software at FirstEnergy in Akron, Ohio. As a result, operators were unaware of the need to redistribute the power away from overloaded transmission lines. As a result, what should have been at most a manageable local blackout led to another collapse of much of the Northeast electric grid.

I told you those stories to help put last Friday’s Rogers outage into perspective.

We need to be realistic about what happened, and how we can mitigate the consequences of similar network failures. Friday’s network event was unprecedented. I told CHCH-TV News that in over 40 years of my involvement in North American telecom networks, I cannot recall an outage as broad in scale (nation-wide) and scope (spanning mobile and fixed networks, voice and data).

Still, most Canadians did not lose their communications services.

That’s worth repeating. Rogers does not operate a monopoly for any of its services. That should have been self-evident to the people who were tweeting about these so-called “CRTC monopolies” (or, my favourite from a so-called expert on a Twitter Space, “ogilopolies”). We didn’t experience a total communications blackout at any point while Rogers was restoring service.

As was noted on my Twitter stream, access to 9-1-1 services should have kept running. Wireless devices supported by Canadian telephone companies will automatically scan for a different network to complete a 9-1-1 call, if the native network is not available. If that did not work, the device suppliers and telecom carriers need to investigate why.

Some major customer network managers may need to re-examine their communications architectures to ensure sufficient carrier diversity. Some found poetic justice in the CRTC’s tweet that its phone lines were “affected by the Rogers network outage.”

Consumers may decide to re-evaluate the value proposition of bundling, perhaps choosing to pay a little more in order to separate their home connectivity from their mobile service provider.

But, let’s be clear about the overall state of Canada’s telecom competition policy.

If anything, last week’s network failure should serve to reaffirm Canada’s telecom policy promoting facilities-based competition. Customers served by alternate facilities-based providers kept operating. A review of the world’s LTE deployments shows that there are 10 LTE networks operating in Canada compared to 9 in the US, 3 or 4 in most European countries (Russia has 9; Sweden has 6; Denmark has 5).

Wholesale based service providers did not add any measure of network survivability. While one wholesale provider boasted on Friday that its services were 50% down, in reality, half of its customers were 100% out of service. The other half were running on a different facilities-based provider.

Canadians benefit from having robust competition among facilities-based carriers, and policies that encourage investment in diverse infrastructures.

As I observed on Friday, some used the network outage to advocate for structural separation, or for a government-run telecom access network. I couldn’t imagine how anyone would think that the people responsible for Canada’s airport fiascos, passport backlogs, or development of government payroll systems, or delivery of clean drinking water, could be entrusted with our telecommunications infrastructure.

If that hypothetical government network was down, I suggested that a week later, we might see a cabinet-level task force struck to discuss it. How is the government addressing the backlog in processing passports? By buying more chairs for people waiting in line. Seriously.

In mid-August 1966, the New York Times reported a mini-baby boom attributed to that cold blacked-out night, nine months earlier. That turned out to be a false urban legend.

That fake baby boom is another important lesson to apply in the wake of last week’s network event. Much misinformation will continue to circulate while technology professionals determine the root causes of the network outage and develop processes to try to avoid similar events in the future.

As I tweeted at the time, in the fullness of time, we will understand what caused the networks to fail. Industry-wide, carriers will undertake measures to avoid similar events and mitigate the impacts when future outages inevitably occur.

The high cost of low prices

A recent article in “The Connexion” highlights the cost of deep discount mobile service prices. “Internet in rural France: New study shows how bad it really can be” cites a study showing that mobile speeds can be 66% lower in rural areas than in urban areas. Recall that a recent Opensignal study found that Canadian rural speeds were just 10-20% lower than urban speeds. Studies have also found that mobile customers in rural Canada experienced faster connection speeds than the average speeds received in any of the rest of the G7 or Australia.

It is also important to recognize that the French study considered communities of less than 10,000 to be rural; that is a full order of magnitude greater than Statistics Canada’s definition of rural as “the population outside settlements with 1,000 or more population”.

A couple of years ago, “When low prices constrain investment” talked about issues arising in Finland. That article referenced the situation in Israel that I discussed in “Low prices, high cost”.

Consumers want low prices. Of course we do, especially as we face rapidly rising prices for so many essentials.

But, consumers also want quality phone service wherever we are, including rural Canada. Remember the Quality, Coverage and Affordable Prices balance that has been at the heart of Canadian telecom policy for the past few years? The most recent Statistics Canada Consumer Price Index data showed that overall prices increased by 7.7% year over year while mobile services dropped 5.2% over the same period.

Once again, in rural France we see evidence of the high cost of a singular focus on low prices.

Service providers invest billions of dollars in capital to provide the coverage and quality of service that Canadians often take for granted.

The consumer interest is multi-dimensional, needing a balance of quality, coverage, and affordable prices.

Balance.

25% fee increase in one year

Last week, the CRTC announced (in 4 brief paragraphs) that it was raising its fees by 25% year over year.

No explanation. No justification. Just 4 pro-forma paragraphs, matching the same format as in previous years, with the exception that in 2021, the Commission only raised fees by less than 5% and by less than 1% in 2020.

Telecom Order CRTC 2022-177, Telecommunications Fees says

  1. The Telecommunications Fees Regulations, 2010 (Fees Regulations) provide for the payment of annual telecommunications fees by certain telecommunications service providers, as set out in sections 2 and 3 of the Fees Regulations.
  2. Pursuant to subsection 3(4) of the Fees Regulations, the Commission hereby announces that the estimated total telecommunications regulatory costs of the Commission for the 2022-2023 fiscal year (1 April 2022 to 31 March 2023) are $48.072 million. This includes additional funding to secure the necessary incremental increases to the Commission’s resources to address both current and future regulatory operational requirements.
  3. The annual adjustment amount (credit) referred to in subsection 3(5) of the Fees Regulations for the 2021-2022 fiscal year is ($0.552) million.
  4. Taking into account the annual adjustment above, the net billing for the Commission’s telecommunications fees for the 2022-2023 fiscal year is $47.520 million. This estimated net billing represents an increase of 24.17% compared to the amount for the 2021-2022 fiscal year ($38.271 million).

Secretary General

If we reach back to 2019, there was a 20% fee increase, but in that case, the Order included a justification, attributing the increase “to the implementation of the Broadband Fund project management function and to the ratification of collective agreements.” As an aside, that year’s $6M increase provided some visibility in the cost of having the CRTC’s duplicative broadband funding organization, independent of personnel at ISED performing a substantially similar function.

But, this year’s Order calls for increasing fees by another $9M with no explanation, no justification, nothing to allow Canadians to understand why we have to pay more. The CRTC simply says “This includes additional funding to secure the necessary incremental increases to the Commission’s resources to address both current and future regulatory operational requirements.”

What does that mean? “Additional funding to secure incremental increases to the Commission’s resources”? Who writes this way? It strikes me as a little bit redundant (and perhaps repetitive) without providing any meaningful disclosure? More money so we can get more and more? What are those incremental resources going to do?

The CRTC wrote up this fee increase, granting itself nine million dollars more, in just 4 short paragraphs. Contrast that with Telecom Order CRTC 2022-149, that took 22 paragraphs of discussion to award PIAC just over $1,300, 15 months after PIAC’s request, requiring 9 pages of filings, and opportunities for public comment on the appropriateness of the funding.

No public process exists for the CRTC’s unilateral funding order. Just 4 short paragraphs.

These Telecom Fees are part of the cost of regulation for Canadian telecom service providers, costs I have discussed before that ultimately impact the prices paid by consumers. Note that the budget is actually increasing by close to $10.25M, increasing from $37.821M last year to $48.072 this year. The CRTC had a half million dollars left over last year.

What kind of performance improvements will Canadians see as a result of this 27% budget increase? Will the Commission deliver Decisions and Orders that much faster? Where is our money going? Are there measurements of how the CRTC performed against last year’s objectives?

Where is the accountability?

Don’t Canadians deserve more of an explanation than the 4 boilerplate paragraphs in Telecom Order CRTC 2022-177?

Global mobile data

Last week, Ericsson issued the latest update to its Mobility Report [pdf, 3.4MB].

As always, it is a good source of information on the global mobile wireless industry. Global mobile network data traffic has doubled in the last two years, a statistic that in consistent with the 40% increase in one year reported by the CRTC in its latest quarterly update.

Ericsson expects “North America to lead the world in 5G subscription penetration in the next five years with nine-of-every-ten subscriptions in the region expected to be 5G in 2027.”

Around the world, home connections using fixed wireless access (FWA) served by mobile networks are projected to double over the next 5 years. As 5G gets pushed out to more areas, will we see greater use of FWA in Canada?

The report includes more than just network data – there are discussions of various marketing approaches and strategies used by service providers around the world. Articles look at the challenges of securing 5G networks, deploying edge computing, the power of IoT connectivity and more.

With a retrospective look at some of the key trends and events that shaped mobile services around the world over the last year, the report provides Ericsson’s forecasts toward 2027.

Good reading for the upcoming holiday weekend.

Asserting authority over Canada’s internet

Last week’s CRTC decision on botnet-blocking can be viewed as a step toward Canada asserting sovereignty over our piece of the internet, but it also raises questions of duplicated efforts by ISED and the Commission.

In Compliance and Enforcement and Telecom Decision CRTC 2022-170: Development of a network-level blocking framework to limit botnet traffic and strengthen Canadians’ online safety, the Commission determined “regulatory action is necessary to ensure that Canadian carriers that block botnets do so in a way that provides a baseline level of protection to Canadians.”

Last week’s decision establishes guiding principles for a future “network-level botnet-blocking framework” and sets in motion activities for the industry-wide CRTC Interconnection Steering Committee (CISC) to assist in developing technical parameters consistent with these principles within nine months. As an aside, don’t you love nested acronyms like CISC? When we created the technical and operational liaison committees 30 years ago, the first C stood for Canadian.

There had been disagreement over the need for regulatory involvement in addressing botnet traffic; the major ISPs, responsible for 80-90% of consumer connections in Canada, argued in favour of the flexibility enabled through existing channels of collaboration is more adaptable than regulation. The major ISPs are already sharing botnet and malware indicators through the Canadian Security Telecommunications Advisory Committee (CSTAC), under the auspices of ISED. It is noteworthy that “CSTAC” appears 23 times in the Decision.

Is cyber-security another area (like spectrum management) that requires an organizational realignment between the CRTC and ISED in order to rationalize responsibilities, avoid duplication of efforts, and regulatory overlap?

I have been writing about internet blocking for more than 15 years, in the context of the sovereignty of Canada in protecting its own citizens.

A year and a half ago, the CRTC launched this proceeding, “strengthen Canadians’ online safety.” The Commission concluded that botnet traffic “constitutes a significant issue for cyber security, both in terms of volume and severity of harm.” No surprise there. What is the right way to address the issue?

I question the basis for the type of regulatory intervention being prescribed, which the CRTC claims necessary because:

  • Service providers’ current practices are diverse and opaque and lack a practical and consistent mechanism for sharing botnet IOCs;
  • Service providers have a considerable role to play in botnet blocking, consistent with a defence-in-depth strategy toward cyber security;
  • network-level blocking programs are effective and appropriate; and
  • there is confusion among the parties regarding the regulatory basis for the existing botnet blocking conducted by service providers.

Is it notable that a network-level blocking framework may be required as part of Online Harms legislation? To what extent was this influential in the CRTC’s decision?

Some parties suggested the framework should permit consumers to choose whether or not to opt-in to network-level blocking. The CRTC said “CIRA Canadian Shield is used by just 1% of households. This very low figure suggests that opt-in models result in underuse.” I’m not convinced the CRTC’s made accurate use of that statistic. There is a big difference between opting in to protection endorsed by your service provider and having the technical competency to change DNS settings on every device in your home, let alone choosing CIRA’s DNS service. There are competing cyber-security services that offer protections, including protections from the major ISPs themselves.

I am concerned that the centralized model chosen by the CRTC may raise the level of protection afforded to customers of some smaller service providers but could lead to a degradation in responsiveness and security for the majority of Canadians who are served by companies already collaborating.

Security is a competitive service feature offered by service providers and third party firms. As CISC works through the dozen questions asked by the CRTC, it needs to ensure that customers aren’t ultimately losing choices in how their connectivity is safeguarded.

We’ll be hearing back from CISC in 9 months.

Scroll to Top