Measuring success

I wrote a piece a couple weeks ago called “Inconsistent messages; predictable turmoil“.

In the piece, I said:

The lessons for Ottawa: Set clear objectives. Align activities with the achievement of those objectives. Stop doing things that are contrary to the objectives.

Maybe the Industry Minister was thinking of Peter Drucker teachings (“you can’t manage what you can’t measure”) in coming up with his simple formula:

We will continually review the regulations and policies that apply to the wireless telecommunications sector to promote at least four wireless providers in every region of the country so that Canadian consumers benefit from competition.

Is that it? Four wireless providers in every region? On the surface, that seems pretty clear as a measurable objective.

But let’s parse the statement to have a little more clarity on what is meant.

What qualifies as a “wireless provider”? Is it brands, companies or networks? Do affiliates count? For example, are Fido and Rogers counted as one or two? Koodo and TELUS? Does President’s Choice count separately, or is it included with its underlying network provider. If so, why? If not, why not?

What is defined by “a region”? Is it a license area? Is there a reason to expect the service area to expand?

Keep in mind that in the 700 MHz licensing framework, the rural build-out requirement is a subset of the existing network HSPA footprint. There isn’t even an obligation imposed on winners of two blocks of 700 MHz spectrum is to cover their entire existing operating area – only 97% of the population within 7 years. There is no rural build obligation imposed on winners of a single block of 700 MHz spectrum.

Surely the Minister isn’t looking for 4 operators across the entire Canadian geography, when the 700 MHz spectrum auction doesn’t seek to expand wireless coverage at all. Consistency suggests that the objective may be for at least 4 wireless providers to be available for 97% of the current level of HSPA customers. Or for 97% of the total population currently covered by an HSPA network.

That should be a measurable number. What are the “pops” covered by various numbers of service providers?

For bonus points, let’s look at a little more detail. How do you determine success? For example, in Manitoba and Saskatchewan, the so-called Big 3 share about a third of the market, while the so-called new entrants, Sasktel and MTS have about two thirds of the customers. Does it make sense to reassess our definition of new-entrant? Is the presence of 6 network-based providers good or bad for sustainable competition? Such questions may be more a part of some associated subjective analysis.

Objectively, shouldn’t there be a dashboard to monitor success against the objective? As a nation, where are we green, yellow and red? How are we performing on a national and regional basis? What percentage of the population has access to 4 or more wireless providers? What percentage of population has no coverage?

This could be an interesting summer project for a group of student interns.

You can’t manage what you can’t measure. With the right tools, policies and decisions can be expected to include a measurable impact analysis. With each market intervention, shouldn’t we expect more people to be getting coverage by more providers? How will this decision help in the achievement of the policy objective?

Set clear objectives. Align activities with the achievement of those objectives. Stop doing things that are contrary to the objectives.

Let’s start with the clarity in the objectives. Then we need to proceed – and proceed quickly – to start measuring how we are doing.

2 thoughts on “Measuring success”

  1. Jean-François Mezei

    The other worrysome points is the promise to adjust regulations to ensure 4 players.

    Adjusting regulations as each of the new entrant approaches bankrupcy is not the right way to go about it. One must build a strong regulatory foundation to give new entrants a real chance.

    The government pretended to foster competition when those new entrants spent real money in an environment where they have little chance of growing to succes.

    The fact that Shaw, despite having existing land network and plenty of resources abandonned its plans is telling. How many years did it take for Eastlink to startup its wireless network ?

    The fact that the 3 new entrants (the real ones) survived this long is testament to their endurance againsts adverse conditions.

    At the end of the day, the minister’s goal basically means that Wind must survive with the other 2 being “disposable” since they do not contribute to the minimum of 4 in each market.

    In essence, the government’s policy requires one new entrant for Ontario, Alberta and BC since all other markets are now filled with by 4 incumbents. Mission almost accomplished (mostly by lowering the standards to require only 4 carriers, even if all 4 are incumbents who follow each other’s prices).

  2. How to go Mark. Great piece – one of your best. Great analysis and great questions. All bang on!

    Unfortunately, I think the chances of getting answers to your questions are no greater than those of getting a national digital economy strategy. When a government has a strategy or a wireless policy and supporting regulatory framework it can be measured as to how successful it has been in achieving stated goals, objectives, etc. It also means a government is tied to the strategy or policy for the long term.

    Unfortunately, the current federal government doesn’t want to be held accountable for anything other than the Economic Action Plan and so it shies away from strategies and policies that could fail. Rather, it lurches from one crisis to another throwing dollars and other resources at them or, as in this case, new interpretations or takes on existing regulations or new regulations – the perfect recipe for “inconsistent messages, predictable failure”.

    Keep up the great work.

Comments are closed.

Scroll to Top