Thursday, June 23, 2011

a recurring story

Five years ago, there was a controversy in the pages of Nature regarding this paper from 1993, the first to claim atomic-resolution chemical analysis via scanning transmission electron microscopy.  At issue was whether or not the data in the paper had been reprocessed (in response to referee concerns) in a legitimate or misrepresentative way, and whether the authors had been honest and forthcoming with the journal and the reviewers about the procedures they'd followed.  The reason that matters came to a head more than 12 years after the original paper was the appearance of a preprint in the arxiv and subsequently submitted to Nature Physics, sharing two of the authors of the original paper, with further questions raised about the handling and analysis of data and images.  This was all discussed clearly and succinctly by ZZ at the time.  Nature allowed the authors to publish a corrigendum, a correction rather than a retraction, regarding the original '93 paper.  This was sufficiently controversial that Nature felt the need to write an editorial explaining their decision.  Oak Ridge did an investigation of the matter, and concluded that there was no fabrication or falsification of data; that report and a response by the authors are linked here.  Judging from the appearance of this on the arxiv last night, it would appear that this isn't quite the end of things.

3 comments:

MstrBee said...

See also:
1) Eugenie Reich's description from 2006: http://www.nasw.org/users/essreich/ornl.pdf
2) Two news articles from Nature, http://www.nature.com/news/2008/080206/full/451618c.html and
http://www.nature.com/news/2007/071128/full/450590a.html

I tried very hard to parse Nature's editorial on this subject from four years ago - after receiving assurances that turned out to be false, the editors then accepted another set of assurances that \begin{quote} the original data, if consistently analysed as intended, would still have supported the central thesis of the paper \end{quote}. . And isn't thirteen years too short of a time to lose source data for such an important experiment?

Douglas Natelson said...

MstrBee - Data retention is a very interesting issue. When surveyed, most researchers would like, ideally, to preserve data "forever". As a practical matter, funding agencies usually require retention for some number of years (3 for NSF, 5 for NIH) beyond the end of a grant. In the modern era, there are real challenges associated with data preservation in the sciences. Migrating data into new formats (e.g., no way to read those old 8" floppies, zip disks, or Jaz cartridges), preserving physical notebooks, boxes of strip chart recordings, physical specimens in the case of biology.... Thirteen years is actually a long time in many ways, in large part because the data we collect now is much more complex than, e.g., glass photographic plates and columns of numbers hand-written in a notebook.

Peter Armitage said...

I think I have ALL my data... if you define "have" liberally enough. So this is a problem....

I have stacks and boxes of small floppies, large floppies, old CDs that are now yellowed and warped, and some archived in false perpetuity on (old) hard drives. In principle responsible stewardship would mean that I would constantly migrate it to the newest formats. (Probably it would be sufficient to put it all on google docs). But limited time and now, no way to retrieve data from the floppies in particular means I haven't done it.