{"id":358443,"date":"2024-12-24T07:36:02","date_gmt":"2024-12-24T06:36:02","guid":{"rendered":"https:\/\/climatescience.press\/?p=358443"},"modified":"2024-12-24T07:36:05","modified_gmt":"2024-12-24T06:36:05","slug":"frankenstein-datasets-and-the-crisis-in-climate-science-integrity","status":"publish","type":"post","link":"https:\/\/climatescience.press\/?p=358443","title":{"rendered":"Frankenstein Datasets and the Crisis in Climate Science Integrity"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

<\/p>\n\n\n\n

From Watts Up With That?<\/a><\/p>\n\n\n\n

Science, at its best, is a self-correcting process\u2014an ongoing pursuit of truth through rigorous evidence, transparent methodology, and open debate. But what happens when errors are introduced, perpetuated, and defended? The tale of the so-called \u201cFrankenstein dataset\u201d in hurricane damage research, brought to light by Roger Pielke Jr., illustrates how flawed data practices can distort both public perception and policy.<\/a><\/p>\n\n\n\n

The Frankenstein dataset, born from undocumented modifications to a rigorous, peer-reviewed dataset, highlights a crisis in scientific integrity. Its story is not just a cautionary tale about bad data but a case study in how the scientific process can fail when institutional accountability is lacking. Let\u2019s unpack how this dataset came to be, why it matters, and what it reveals about the state of climate science.<\/p>\n\n\n\n

The Origins of the Original Dataset<\/h3>\n\n\n\n

The original dataset, as developed through decades of research by Pielke and his colleagues, aimed to normalize hurricane damages by adjusting for inflation, population growth, and other economic factors. This normalization process allowed researchers to compare historical hurricane damages on an apples-to-apples basis, isolating trends in economic loss from changes in wealth or development.<\/p>\n\n\n\n

The dataset, thoroughly documented in studies like Weinkle et al. (2018)<\/em> and Pielke et al. (2008)<\/em>, served as a reliable tool for understanding hurricane impacts. It was grounded in NOAA\u2019s \u201cbest track\u201d data, covering U.S. landfalling hurricanes, and adhered to consistent methodologies\u200b\u200b.<\/p>\n\n\n\n

However, the story took a dark turn when this dataset fell into the hands of ICAT, an insurance company.<\/p>\n\n\n\n

How the Frankenstein Dataset Was Born<\/h3>\n\n\n\n

After Pielke et al. (2008)<\/em> was published, Pielke\u2019s team partnered with ICAT to create the ICAT Damage Estimator, an online tool designed to visualize hurricane damages using the peer-reviewed dataset. Initially, the collaboration worked as intended: the tool increased access to high-quality research for industry stakeholders\u200b.<\/p>\n\n\n\n

But in 2010, ICAT was acquired by another company, and Pielke ceased his involvement. Over the following years, ICAT employees, who lacked expertise in disaster normalization, made undocumented changes to the dataset. These alterations included replacing post-1980 entries with data from NOAA\u2019s Billion-Dollar Disasters (BDD) database, which utilized a completely different methodology\u200b\u200b.<\/p>\n\n\n\n

Key Modifications<\/h4>\n\n\n\n
    \n
  1. Substitution with NOAA\u2019s BDD Data:<\/strong>\u00a0ICAT replaced post-1980 entries with BDD data, which included inland flooding damages (from the National Flood Insurance Program, or NFIP) and broader economic impacts like commodity losses and disaster relief payouts. These additional factors inflated damage estimates post-1980, creating an artificial upward trend\u200b\u200b.<\/li>\n\n\n\n
  2. Additional Events:<\/strong>\u00a0ICAT17, the modified dataset, introduced 61 additional storm damage events, none of which were sourced or documented. Most of these undocumented events occurred after 1980, further skewing the dataset\u200b.<\/li>\n\n\n\n
  3. Methodological Discontinuity:<\/strong>\u00a0NOAA\u2019s BDD methodology, adopted in 2016, was incompatible with the original dataset. For example, NFIP payouts didn\u2019t exist before 1968, making comparisons between pre- and post-1968 damages inherently flawed\u200b\u200b.<\/li>\n\n\n\n
  4. Unsupervised Alterations:<\/strong>\u00a0Beyond substituting BDD data, ICAT17 contained additional undocumented changes to the original dataset. These changes introduced upward biases even before normalization adjustments were applied\u200b.<\/li>\n<\/ol>\n\n\n\n

    Steve McIntyre commented on Pielke Jr.\u2019s post.<\/p>\n\n\n

    \n
    \"\"
    https:\/\/x.com\/ClimateAudit\/status\/1870515744095482144<\/a><\/figcaption><\/figure>\n<\/div>\n\n\n

    By the time ICAT published this Frankenstein dataset online, it had diverged so far from the original peer-reviewed data that it bore no resemblance to a rigorous research product.<\/p>\n\n\n

    \n
    \"\"
    https:\/\/x.com\/RogerPielkeJr\/status\/1870496128304578675<\/a><\/figcaption><\/figure>\n<\/div>\n\n\n

    How the Frankenstein Dataset Was Misused<\/h3>\n\n\n\n

    The ICAT17 dataset, later extended and rebranded as \u201cXCAT\/ICAT 23 in Willoughby et al 2024,\u201d was adopted by researchers who assumed it was a professionally maintained and credible resource. Notably:<\/p>\n\n\n\n