Don’t Make Baby Gauss Cry

Cosma Shalizi writes of “Power-Law Distributions in Empirical Data“:

Because this is, of course, what everyone ought to do with a computational paper, we’ve put our code online, so you can check our calculations, or use these methods on your own data, without having to implement them from scratch. I trust that I will no longer have to referee papers where people use GnuPlot to draw lines on log-log graphs, as though that meant something, and that in five to ten years even science journalists and editors of Wired will begin to get the message.

Mark Liberman is not optimistic (we’ve got a long way to go).

Among several important take-home points, I found the following particularly amusing:

Abusing linear regression makes the baby Gauss cry. Fitting a line to your log-log plot by least squares is a bad idea. It generally doesn’t even give you a probability distribution, and even if your data do follow a power-law distribution, it gives you a bad estimate of the parameters. You cannot use the error estimates your regression software gives you, because those formulas incorporate assumptions which directly contradict the idea that you are seeing samples from a power law. And no, you cannot claim that because the line “explains” (really, describes) a lot of the variance that you must have a power law, because you can get a very high R^2 from other distributions (that test has no “power”). And this is without getting into the additional errors caused by trying to fit a line to binned histograms.

It’s true that fitting lines on log-log graphs is what Pareto did back in the day when he started this whole power-law business, but “the day” was the 1890s. There’s a time and a place for being old school; this isn’t it.

Come to think of it, isn’t fitting a straight line to a log-log plot exactly what Ray Kurzweil did to “predict” the technological singularity?