The Ottawa Citizen ran a sloppily researched and unbalanced story Tuesday night after the CRTC released its first quarterly report on ITMP complaints. It was the subject of my post called Statistical Insignificance. The story looked at the CRTC’s report and concluded there had been a “sharp jump”, a “public outcry” and cited Michael Geist saying the numbers represented a “big spike in complaints.”
Problem was, nobody looked at the numbers. The story’s writer compared the fourth quarter complaints (27) with the total number of complaints over a two year period since the ITMP rules were in place (59) and presumed that this represented a 400% increase, a figure that a Financial Post editor used to justify its headline of “skyrocketing” complaints when re-running the story. Another Financial Post story also referred to the mythical “sharp jump” in complaints.
The story even tried to explain the “big spike”:
In September 2011, the CRTC issued a bulletin announcing improved reporting practices for Internet complaints, which could be responsible for the recent spike in public concern.
None of the reporters, editors or expert commentators considered how the data in the previous period was distributed across the two years. I thought it would be unlikely to be even distribution, but apparently, no one bothered to ask the CRTC what the numbers looked like.
So I did.
The CRTC told me that there were 25 complaints in the period July 1 to September 30.
Twenty five in the third quarter of 2011 versus 27 in the fourth quarter.
Skyrocketing, sharp jump, big spike?