I appear to have been too generous to Lott’s complaint when I first criticized it. William Ford, a Bigelow teacher at University of Chicago Law, has posted a devastating two-part analysis (via Wright) of the main count of Lott’s complaint. In the first part, he shows that “replication” does not have the “clear and unambiguous” meaning ascribed to it; in the second, he finds examples of Lott himself using “replicate” in meanings other than the definition in his complaint. Lott critic-in-chief Tim Lambert finds some more examples of the use of “replicate” that appear to exonerate Levitt’s phrasing—though Lambert fudges the National Academy of Science report, which says something rather different about replicating Lott’s results than what Levitt said in Freakonomics.
The case won’t necessarily get thrown out of court; Lott still has the second count, an errant e-mail sent by Levitt to a Texas economist accusing Lott of buying articles. But unless he can demonstrate that Levitt repeated that accusation in more than a stray e-mail, or was making other libelous allegations, potential damages look nominal for that count: the big-money claim is in the first count.
9 Comments
Sorry: This parsing of the term “replicate” is a red herring of the first magnitude. (Full disclosure: John Lott and I overlapped in graduate school, and he and I have been friends for many years.) In the context of refereed economics journals, “replicate” has one meaning only: The use of an author’s data and model to ensure that falsification of findings is not an issue. That is the relevant question in the context of Steve Leveatt’s nonreplication assertion about Lott in Freakonomics, and it simply is a fact that Lott’s findings have been replicated repeatedly. The other purported definition of replication”—derivation of the same or similar findings with a different data set and/or model—is called “robustness” (not replication) among economists in the context of peer-reviewed work. Leavitt either lied out of animosity toward Lott, or did not bother to get his facts straight, an amusing approach for someone accusing others of nonreplicated work.
Benjamin, you seem to have missed the point of the post: Lott himself used “replicate” with the more general meaning.
Ted Frank on Lott vs Levitt
More interesting analysis of Lott’s lawsuit from Ted Frank. First, after looking at the examples of the use of “replicate” he concludes: I appear to have been too generous to Lott’s complaint when I first criticized it. Then Ben Zycher,…
Lott v. Levitt IV
David Glenn, in the Chronicle of Higher Education, has the definitive MSM reporting on the affair. (Permanent link here after Apr. 24.) He finds a mixture of scholars who agree and disagree with Lott on…
Dear Tim, With respect to Lott’s use of the term, (in the context of the Civil Rights Commission analysis of the Florida vote in 2000), it is clear that Lott worded that statement badly, in that no CRC statistical analyses/regressions were available publicly at that time; Lott estimated several econometric specifications of his own to see if the CRC findings as reported could be “replicated”; in that context, a far better word would have been “confirmed” or “reproduced.”
In the broader context of peer-reviewed economics publications, “replication” has a very specific meaning: the use of the author’s data and model specifications as a check to ensure that no falsification is present. That is why most economics journals require authors to make their data available to the editors and referees. Different data sets and/or different models are used as tests of robustness.
Again: Lott’s empirical findings on gun ownership and crime in fact have been replicated by several researchers. There have been something on the order of 150 published studies on the guns/crime issue, and not one has found an adverse effect. At the very least—being as charitable to Levitt as possible—his sloppy use of the term “replicate,” which he must know is used in a very specific way by economists, as a vehicle with which to smear Lott, if only by innuendo, is unconscionable.
Again: Lott’s empirical findings on gun ownership and crime in fact have been replicated by several researchers. There have been something on the order of 150 published studies on the guns/crime issue, and not one has found an adverse effect.
Is it my imagination or did you just use the word “replicate” in exactly the way in which you were just saying it would be never used in the context of economics? Or have there been 150 published studies using Lott’s data and model specifications? (One would imagine that people would get tired of reading them after a while.)
‘Again: Lott’s empirical findings on gun ownership and crime in fact have been replicated by several researchers. There have been something on the order of 150 published studies on the guns/crime issue, and not one has found an adverse effect.
Is it my imagination or did you just use the word “replicate” in exactly the way in which you were just saying it would be never used in the context of economics? Or have there been 150 published studies using Lott’s data and model specifications? (One would imagine that people would get tired of reading them after a while.)”
I would suspect that, “several” being quite a bit different from “something on the order of 150”, that there have been “several” researchers who have replicated Lott’s findings, and ADDITIONALLY, there have been “something on the order of 150” studies that are related, none of which undercut Lott’s thesis.
That is, the second sentence you quoted does not seem to refer to replication but to other, generally supportive data (that is, Levitt is questioning Lott’s work, and there is no research to support that criticism).
I should note that you don’t have to get any further than paragraph 10 of the lawsuit to find a reference to Duggan’s paper “More Guns, More Crime”, which does find an adverse effect as the title indicates. I’m surprised that Zycher is unaware of any of the published research that finds that guns have an adverse effect.
Off-topic to Zycher: ask your buddy why he chose a Tobit instead of a Poisson regression for his Rush-Limbaugh-was-right work.