{"id":363473,"date":"2025-01-26T10:28:38","date_gmt":"2025-01-26T09:28:38","guid":{"rendered":"https:\/\/climatescience.press\/?p=363473"},"modified":"2025-01-26T10:28:40","modified_gmt":"2025-01-26T09:28:40","slug":"margins-of-error","status":"publish","type":"post","link":"https:\/\/climatescience.press\/?p=363473","title":{"rendered":"Margins of Error"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"723\" height=\"405\" data-attachment-id=\"363474\" data-permalink=\"https:\/\/climatescience.press\/?attachment_id=363474\" data-orig-file=\"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2025\/01\/0Screenshot-2025-01-26-102032.png?fit=1393%2C779&amp;ssl=1\" data-orig-size=\"1393,779\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"0Screenshot 2025-01-26 102032\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2025\/01\/0Screenshot-2025-01-26-102032.png?fit=723%2C405&amp;ssl=1\" src=\"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2025\/01\/0Screenshot-2025-01-26-102032.png?resize=723%2C405&#038;ssl=1\" alt=\"\" class=\"wp-image-363474\" srcset=\"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2025\/01\/0Screenshot-2025-01-26-102032.png?resize=1024%2C573&amp;ssl=1 1024w, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2025\/01\/0Screenshot-2025-01-26-102032.png?resize=300%2C168&amp;ssl=1 300w, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2025\/01\/0Screenshot-2025-01-26-102032.png?resize=768%2C429&amp;ssl=1 768w, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2025\/01\/0Screenshot-2025-01-26-102032.png?resize=1200%2C671&amp;ssl=1 1200w, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2025\/01\/0Screenshot-2025-01-26-102032.png?w=1393&amp;ssl=1 1393w\" sizes=\"auto, (max-width: 723px) 100vw, 723px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">From <a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/\">Watts Up With That?<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">James D. Agresti | President |&nbsp;<a href=\"http:\/\/james%20d.%20agresti%20%7C%20president%20%7C%20just%20facts\/\">Just Facts<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Ever lost a bet? From the lottery or sporting events to casinos or friendly wagers, you may have risked and lost some money because you hoped to win big.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But let me ask you this: How&nbsp;<em>big<\/em>&nbsp;would the payout have to be and how&nbsp;<em>good<\/em>&nbsp;would the odds need to be to gamble with your&nbsp;<em>life<\/em>&nbsp;or the&nbsp;<em>lives<\/em>&nbsp;of&nbsp;<em>your loved ones<\/em>?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In this lesson from Just Facts Academy about&nbsp;<strong>Margins of Error<\/strong>, we\u2019ll show you how people do that without even realizing it. And more importantly, we\u2019ll give you the tools you need to keep you from falling into this trap.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Ready? C\u2019mon, what have you got to lose?<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-wp-embed is-provider-rumble-com wp-block-embed-rumble-com\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" class=\"wp-embedded-content\" sandbox=\"allow-scripts\" security=\"restricted\" title=\"Margins of Error\" src=\"https:\/\/rumble.com\/embed\/v66pf4a\/#?secret=gKtN2JiRB8\" data-secret=\"gKtN2JiRB8\" width=\"1280\" height=\"720\" frameborder=\"0\"><\/iframe>\n<\/div><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">People often use data from studies, tests, and surveys to make life-or-death decisions, like about what medicines they should take, what kinds of foods they should eat, and what activities should they embrace or avoid.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The problem is such data is that it isn\u2019t always as concrete as the media and certain scholars make it out to be.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Look at it this way. There are four layers to this \u201cmargin of error\u201d cake. Let\u2019s start with the simplest one, like this headline from the Los Angeles Times, which declares, \u201cCalifornia sea levels to rise 5-plus feet this century, study says.\u201d<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn1\">[1]<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That sounds pretty scary, but the study has margins of error, and it actually predicts a sea-level rise of 17 to 66 inches.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn2\">[2]<\/a>&nbsp;In the body of the article, the reporter walks back the headline a little, but he fails to provide even a hint that the \u201c5-plus-feet\u201d is the upper bound of an estimate that extends all the way down to a quarter of this.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn3\">[3]<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Studies often have margins of error, or bounds of uncertainty, so the moment you hear someone summarize a study with a single figure, dig deeper. This is the same principle taught in Just Facts Academy\u2019s lesson on&nbsp;<a href=\"https:\/\/www.justfactsacademy.org\/primarysources\" target=\"_blank\" rel=\"noreferrer noopener\">Primary Sources<\/a>: Don\u2019t rely on secondary sources because they often reflect someone\u2019s interpretation of the facts\u2014instead of the actual facts.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Also, don\u2019t assume that the authors of the primary sources will report the vital margins of error near the top of their studies. In the famed Bangladesh face mask study, for example, the authors lay down 4,000 words before they disclose a range of uncertainty that undercuts their primary finding.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn4\">[4]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn5\">[5]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn6\">[6]<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here are a few more tips to help you critically examine margins of error.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Surveys often present their results like this:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">11.5% \u00b1 0.3<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It\u2019s quite simple. The first number is the nominal or best estimate, technically called the \u201cpoint estimate.\u201d The second number is the margin of error.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In the case of this survey,<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn7\">[7]<\/a>&nbsp;it means that the best estimate of the U.S. poverty rate is 11.5%, but the actual figure may be as low as 11.2% or as high as 11.8%.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Scholarly publications often use a less intuitive convention and present their results like this:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">4.70; 95% CI, 1.77\u201312.52<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Now, don\u2019t let this barrage of digits intimidate you. They\u2019re actually easy to understand once you crack the code.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The first number is the best estimate. In the case of this study,<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn8\">[8]<\/a>&nbsp;it means that bisexual men are roughly 4.7 times \u201cmore likely to report severe psychological distress\u201d than heterosexual men.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The last two digits are the outer bounds of the study\u2019s results after the margins of error are included. They mean that bisexual men are about 1.8 to 12.5 times more likely to report distress than heterosexual men.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn9\">[9]<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That\u2019s a really broad range, especially when compared to the single figure of 4.7. Do you see why margins of error are so essential?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Now, here\u2019s something a lot of people don\u2019t know because journalists rarely explain it or don\u2019t understand it: Reported margins of error and ranges of uncertainty typically account for just&nbsp;<em>one<\/em>&nbsp;type of error known as sampling error.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn10\">[10]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn11\">[11]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn12\">[12]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn13\">[13]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn14\">[14]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn15\">[15]<\/a>&nbsp;This is based purely on the size of the sample used for the study or survey. Generally speaking, the larger the sample, the smaller the margin of sampling error.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn16\"><sup>[16]<\/sup><\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It\u2019s super important to be aware of this, because there are often other layers of uncertainty that aren\u2019t reflected in sampling errors.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn17\">[17]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn18\">[18]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn19\">[19]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn20\">[20]<\/a>&nbsp;Figures like 1.77 to 12.52 sound very specific and solid, but that can be an illusion. If you don\u2019t understand this, you can be easily misled to believe that the results of a study are ironclad when they are&nbsp;<em>not<\/em>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This brings us to the \u201c95% CI.\u201d What does that mean?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It stands for \u201c95% confidence interval,\u201d<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn21\">[21]<\/a>&nbsp;and contrary to what your statistics teacher may have told you,<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn22\">[22]<\/a>&nbsp;it generally means that there\u2019s a 95% chance the upper and lower bounds of the study contain the real figure.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn23\">[23]<\/a>&nbsp;That means there\u2019s a 5% chance they don\u2019t.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">How\u2019s that for gambling? Would you step outside your home today if you knew there was a 1 in 100 chance you wouldn\u2019t make it back alive? Well, even the outer bounds of most study results are less certain than that.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">You see, time, money, and circumstances often limit the sizes of studies, tests, and surveys.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn24\"><sup>[24]<\/sup><\/a>&nbsp;So even if their methodologies are sound, reality may lie outside the bounds of the results due to mere chance.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn25\"><sup>[25]<\/sup><\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">On top of this, some studies measure multiple types of outcomes while failing to account for the fact that each attempt to measure a separate outcome increases the likelihood of getting a seemingly solid result due to pure chance.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn26\"><sup>[26]<\/sup><\/a>&nbsp;Look at it this way: if you roll of pair of dice 12 times, you\u2019re 12 times more likely roll a 2 than if you roll them once.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Even worse, there are scholars who roll those dice behind the scenes by calculating different outcomes until they find one that provides a result they want. And that\u2019s the only one they\u2019ll tell you about.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn27\">[27]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn28\">[28]<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Now, let\u2019s take a step back and look at the layers of the cake:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>First, you have the point estimate.<\/li>\n\n\n\n<li>Then, you have the outer bounds, which commonly account for the margin of sampling error but no other sources of uncertainty.<\/li>\n\n\n\n<li>Then, you have the confidence interval percentage, or the probability that the outer bounds are correct.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">We\u2019ll get to the base layer in a moment, but now is a good time to talk about a concept called \u201cstatistical significance,\u201d because we\u2019ve cut through enough cake to understand it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Study results are typically labeled \u201cstatistically significant\u201d if the margin of sampling error with 95% confidence is entirely positive or entirely negative.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn29\">[29]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn30\">[30]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn31\"><sup>[31]<\/sup><\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn32\">[32]<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, if a medical study finds a treatment is 10% to 30% effective with 95% confidence, this is considered to be a statistically significant outcome. That\u2019s a shorthand way of saying the result probably isn\u2019t due to sampling error.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn33\">[33]<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">And if a study finds that a treatment is \u201310% to 30% effective with 95% confidence, such a result is considered to be \u201cstatistically insignificant\u201d because it crosses past the line of zero effect.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn34\">[34]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn35\">[35]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn36\">[36]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn37\">[37]<\/a>&nbsp;This could mean that the treatment has a positive effect, or no effect, or a negative effect.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn38\">[38]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn39\">[39]<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">One way to sort this out is to look at the size of the study sample. If it\u2019s relatively large, and the results are statistically insignificant, that\u2019s a pretty good indication the effect is trivial.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn40\">[40]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn41\"><sup>[41]<\/sup><\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn42\"><sup>[42]<\/sup><\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn43\"><sup>[43]<\/sup><\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Hundreds of scholars have called for ending the convention of labeling results as \u201cstatistically significant\u201d or \u201cinsignificant.\u201d This is because it can lead people to jump to false conclusions.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn44\">[44]<\/a>&nbsp;Nonetheless, it\u2019s a common practice,<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn45\">[45]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn46\">[46]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn47\"><sup>[47]<\/sup><\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn48\"><sup>[48]<\/sup><\/a>&nbsp;so here are some tips to avoid such risky leaps:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>One, don\u2019t mistake statistical significance for real-world importance. A study\u2019s results can be statistically significant but also tiny or irrelevant.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn49\">[49]<\/a>\u00a0<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn50\">[50]<\/a><\/li>\n\n\n\n<li>Two, don\u2019t assume that a statistically insignificant result means there\u2019s no difference or no effect.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn51\">[51]<\/a>\u00a0Sometimes studies are underpowered, which means their samples are too small to detect statistically significant results.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn52\">[52]<\/a>\u00a0In other words, there\u2019s a major distinction between saying that a study \u201cfound no statistically significant effect\u201d and saying \u201cthere\u2019s no effect.\u201d<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn53\">[53]<\/a><\/li>\n\n\n\n<li>Third and most importantly, don\u2019t fall into the trap of believing that a study is reliable just because the results are statistically significant.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn54\">[54]<\/a>\u00a0That\u2019s the final layer to the cake, and it\u2019s where the riskiest gambling occurs.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Here\u2019s what I mean.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The study on sea level rise we discussed\u2014well, it\u2019s based on a computer model,<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn55\">[55]<\/a>&nbsp;a type of study that is notoriously unreliable.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn56\">[56]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn57\">[57]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn58\">[58]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn59\">[59]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn60\">[60]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn61\">[61]<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">And the study about psychological distress and sexuality\u2014it\u2019s an observational study,<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn62\">[62]<\/a>&nbsp;which can rarely determine cause and effect, even though scholars falsely imply or explicitly claim that they do.<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn63\">[63]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn64\">[64]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn65\">[65]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn66\">[66]<\/a>&nbsp;<a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_edn67\">[67]<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Then there\u2019s all kinds of survey-related errors exposed by Just Facts\u2019 lesson on&nbsp;<a href=\"https:\/\/www.justfactsacademy.org\/polls\" target=\"_blank\" rel=\"noreferrer noopener\">Deconstructing Polls &amp; Surveys<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Bottom line\u2014the \u201cmargins of error\u201d reported by journalists and scholars rarely account for many other sources of error.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Gone are the days when you can blindly trust a study just because it is publicized by your favorite news source, appears in a peer-reviewed journal, was written by a PhD, or is endorsed by a government agency or professional association.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Incompetence and dishonesty are simply far too rampant to outsource major life decisions without critical analysis.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So don\u2019t gamble your life on \u201cexperts\u201d who offer solid bets that \u201cyou can\u2019t lose.\u201d Instead, keep it locked to Just Facts Academy, so you can learn how to research like a genius.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em><a href=\"http:\/\/www.justfacts.com\/\">Just Facts<\/a>&nbsp;is a research and educational institute dedicated to publishing facts about public policies and teaching&nbsp;<a href=\"https:\/\/www.justfactsacademy.org\/\">research skills<\/a>.<\/em><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Endnotes<\/strong><\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref1\">[1]<\/a>&nbsp;Article: \u201cCalifornia Sea Levels to Rise 5-Plus Feet This Century, Study Says.\u201d By Tony Barboza. Los Angeles Times, June 24, 2012. <a href=\"http:\/\/articles.latimes.com\/2012\/jun\/24\/local\/la-me-adv-sea-level-20120625\" rel=\"nofollow\">http:\/\/articles.latimes.com\/2012\/jun\/24\/local\/la-me-adv-sea-level-20120625<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Sea levels along the California coast are expected to rise up to 1 foot in 20 years, 2 feet by 2050 and as much as 5 1\/2 feet by the end of the century, climbing slightly more than the global average and increasing the risk of flooding and storm damage, a new study says. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Coastal California could see serious damage from storms within a few decades, especially in low-lying areas of Southern California and the Bay Area. San Francisco International Airport, for instance, could flood if the sea rises a little more than a foot, a mark expected to be reached in the next few decades. Erosion could cause coastal cliffs to retreat more than 100 feet by 2100, according to the report.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref2\">[2]<\/a>&nbsp;Paper: \u201cSea-Level Rise for the Coasts of California, Oregon, and Washington: Past, Present, and Future.\u201d By the Committee on Sea Level Rise in California, Oregon, and Washington, National Research Council. National Academies Press, 2012. <a href=\"http:\/\/www.nap.edu\/catalog.php?record_id=13389\" rel=\"nofollow\">http:\/\/www.nap.edu\/catalog.php?record_id=13389<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Pages 4\u20136:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For the California coast south of Cape Mendocino, the committee projects that sea level will rise 4\u201330 cm [2\u201312 inches] by 2030 relative to 2000, 12\u201361 cm [5\u201324 inches] by 2050, and 42\u2013167 cm [17\u201366 inches] by 2100.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref3\">[3]<\/a>&nbsp;Article: \u201cCalifornia Sea Levels to Rise 5-Plus Feet This Century, Study Says.\u201d By Tony Barboza. Los Angeles Times, June 24, 2012. <a href=\"http:\/\/articles.latimes.com\/2012\/jun\/24\/local\/la-me-adv-sea-level-20120625\" rel=\"nofollow\">http:\/\/articles.latimes.com\/2012\/jun\/24\/local\/la-me-adv-sea-level-20120625<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Sea levels along the California coast are expected to rise up to 1 foot in 20 years, 2 feet by 2050 and as much as 5 1\/2 feet by the end of the century, climbing slightly more than the global average and increasing the risk of flooding and storm damage, a new study says. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Coastal California could see serious damage from storms within a few decades, especially in low-lying areas of Southern California and the Bay Area. San Francisco International Airport, for instance, could flood if the sea rises a little more than a foot, a mark expected to be reached in the next few decades. Erosion could cause coastal cliffs to retreat more than 100 feet by 2100, according to the report.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref4\">[4]<\/a>&nbsp;Article: \u201cFamed Bangladesh Mask Study Excluded Crucial Data.\u201d By James D. Agresti. Just Facts, April 8, 2022. <a href=\"https:\/\/www.justfactsdaily.com\/famed-bangladesh-mask-study-excluded-crucial-data\" rel=\"nofollow\">https:\/\/www.justfactsdaily.com\/famed-bangladesh-mask-study-excluded-crucial-data<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Beyond excluding the death data, the authors engaged in other actions that reflect poorly on their integrity. One of the worst is touting their findings with far more certainty than warranted by the actual evidence. For example, some of the authors wrote a New York Times&nbsp;<a href=\"https:\/\/www.nytimes.com\/2021\/09\/26\/opinion\/do-masks-work-for-covid-prevention.html\" target=\"_blank\" rel=\"noreferrer noopener\">op-ed<\/a>&nbsp;declaring that \u201cmasks work,\u201d a claim undercut by the following facts from their own study: \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2022 Their&nbsp;<a href=\"https:\/\/www.science.org\/doi\/10.1126\/science.abi9069\" target=\"_blank\" rel=\"noreferrer noopener\">study\u2019s<\/a>&nbsp;\u201cprimary outcome,\u201d a positive blood test for Covid-19 antibodies, found that less than 1% of the participants caught C-19, including 0.68% in villages where people were pressured to wear masks, and 0.76% in villages that were not. This is a total difference of 0.08 percentage points in a study of more than 300,000 people.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2022 Their&nbsp;<a href=\"https:\/\/www.science.org\/doi\/10.1126\/science.abi9069\" target=\"_blank\" rel=\"noreferrer noopener\">paper<\/a>&nbsp;lays down 4,000 words before it&nbsp;<a href=\"https:\/\/www.science.org\/doi\/10.1126\/science.abi9069\" target=\"_blank\" rel=\"noreferrer noopener\">reveals<\/a>&nbsp;the sampling margins of error in the results above, which show with 95% confidence that \u2026 cloth masks reduced the risk of catching symptomatic C-19 by as much as 23% or increased the risk by as much as 8%.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2022 \u201c<a href=\"https:\/\/www.acpjournals.org\/doi\/10.7326\/M20-6817\" target=\"_blank\" rel=\"noreferrer noopener\">Not statistically significant<\/a>\u201d is the common term used to describe study results that aren\u2019t totally positive or totally negative throughout the full margin of error, like the results above. Yet, the authors skip this fact in their&nbsp;<a href=\"https:\/\/www.nytimes.com\/2021\/09\/26\/opinion\/do-masks-work-for-covid-prevention.html\" target=\"_blank\" rel=\"noreferrer noopener\">op-ed<\/a>&nbsp;and bury it in their&nbsp;<a href=\"https:\/\/www.science.org\/doi\/10.1126\/science.abi9069\" target=\"_blank\" rel=\"noreferrer noopener\">paper<\/a>, writing at the end of an unrelated paragraph that it showed \u201cno statistically significant effect for cloth masks.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">NOTE: The next two footnotes document the primary sources.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref5\">[5]<\/a>&nbsp;Paper: \u201cImpact of Community Masking on COVID-19: A Cluster-Randomized Trial in Bangladesh.\u201d By Jason Abaluck and others.&nbsp;<em>Science<\/em>, December 2, 2021. <a href=\"https:\/\/www.science.org\/doi\/10.1126\/science.abi9069\" rel=\"nofollow\">https:\/\/www.science.org\/doi\/10.1126\/science.abi9069<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 3:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We find clear evidence that surgical masks lead to a relative reduction in symptomatic seroprevalence of 11.1% (adjusted prevalence ratio = 0.89 [0.78, 1.00]; control prevalence = 0.81%; treatment prevalence = 0.72%). Although the point estimates for cloth masks suggests that they reduce risk, the confidence limits include both an effect size similar to surgical masks and no effect at all (adjusted prevalence ratio = 0.94 [0.78, 1.10]; control = 0.67%; treatment = 0.61%).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">NOTE: The quote above is buried 4,000 words into the paper. Moreover, the authors misleadingly describe these results. The outer bound of \u201c1.00\u201d for surgical masks actually means no effect at all, but the authors fail to use this term when describing that outcome. Instead, they use the term \u201cno effect at all\u201d to describe the outer bound of \u201c1.10\u201d for cloth masks when this actually means a 10% increase in the risk catching Covid-19.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 4:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We find clear evidence that the intervention reduced symptoms: We estimate a reduction of 11.6% (adjusted prevalence ratio = 0.88 [0.83, 0.93]; control = 8.60%; treatment = 7.63%). Additionally, when we look separately by cloth and surgical masks, we find that the intervention led to a reduction in COVID-19\u2013like symptoms under either mask type (<em>p<\/em>&nbsp;= 0.000 for surgical;&nbsp;<em>p<\/em>&nbsp;= 0.066 for cloth), but the effect size in surgical mask villages was 30 to 80% larger depending on the specification. In table S9, we run the same specifications using the smaller sample used in our symptomatic seroprevalence regression (i.e., those who consented to give blood). In this sample, we continue to find an effect overall and an effect for surgical masks but see no statistically significant effect for cloth masks.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref6\">[6]<\/a>&nbsp;Commentary: \u201cWe Did the Research: Masks Work, and You Should Choose a High Quality Mask if Possible.\u201d By Jason Abaluck, Laura H. Kwong, and Stephen P. Luby. <a href=\"https:\/\/www.nytimes.com\/2021\/09\/26\/opinion\/do-masks-work-for-covid-prevention.html\" rel=\"nofollow\">https:\/\/www.nytimes.com\/2021\/09\/26\/opinion\/do-masks-work-for-covid-prevention.html<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u201cThe bottom line is masks work, and higher quality masks most likely work better at preventing Covid-19.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref7\">[7]<\/a>&nbsp;Report: \u201cPoverty in the United States: 2022.\u201d By Emily A. Shrider and John Creamer. U.S. Census Bureau, September 2023. <a href=\"https:\/\/www.census.gov\/content\/dam\/Census\/library\/publications\/2023\/demo\/p60-280.pdf\" rel=\"nofollow\">https:\/\/www.census.gov\/content\/dam\/Census\/library\/publications\/2023\/demo\/p60-280.pdf<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Pages 20\u201321:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Table A-1. People in Poverty by Selected Characteristics: 2021 and 2022<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">2022 \u2026 Below poverty \u2026 Percent [=] 11.5 \u2026 Margin of error<sup>1<\/sup>&nbsp;(\u00b1) [=] 0.3 \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><sup>1&nbsp;<\/sup>A margin of error (MOE) is a measure of an estimate\u2019s variability. The larger the MOE in relation to the size of the estimate, the less reliable the estimate. This number, when added to and subtracted from the estimate, forms the 90 percent confidence interval. MOEs shown in this table are based on standard errors calculated using replicate weights.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref8\">[8]<\/a>&nbsp;Paper: \u201cComparison of Health and Health Risk Factors Between Lesbian, Gay, and Bisexual Adults and Heterosexual Adults in the United States.\u201d By Gilbert Gonzales, Julia Przedworski, and Carrie Henning-Smith.&nbsp;<em>Journal of the American Medical Association<\/em>, June&nbsp;27, 2016. <a href=\"http:\/\/archinte.jamanetwork.com\/article.aspx?articleid=2530417\" rel=\"nofollow\">http:\/\/archinte.jamanetwork.com\/article.aspx?articleid=2530417<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Data from the nationally representative 2013 and 2014 National Health Interview Survey were used to compare health outcomes among lesbian (n\u2009=\u2009525), gay (n\u2009=\u2009624), and bisexual (n\u2009=\u2009515) adults who were 18 years or older and their heterosexual peers (n\u2009=\u200967\u202f150) using logistic regression. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">After controlling for sociodemographic characteristics \u2026 bisexual men were more likely to report severe psychological distress (OR, 4.70; 95% CI, 1.77-12.52), heavy drinking (OR, 3.15; 95% CI, 1.22-8.16), and heavy smoking (OR, 2.10; 95% CI, 1.08-4.10) than heterosexual men\u2026.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref9\">[9]<\/a>&nbsp;Paper: \u201cComparison of Health and Health Risk Factors Between Lesbian, Gay, and Bisexual Adults and Heterosexual Adults in the United States.\u201d By Gilbert Gonzales, Julia Przedworski, and Carrie Henning-Smith.&nbsp;<em>Journal of the American Medical Association<\/em>, June&nbsp;27, 2016. <a href=\"http:\/\/archinte.jamanetwork.com\/article.aspx?articleid=2530417\" rel=\"nofollow\">http:\/\/archinte.jamanetwork.com\/article.aspx?articleid=2530417<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Data from the nationally representative 2013 and 2014 National Health Interview Survey were used to compare health outcomes among lesbian (n\u2009=\u2009525), gay (n\u2009=\u2009624), and bisexual (n\u2009=\u2009515) adults who were 18 years or older and their heterosexual peers (n\u2009=\u200967\u202f150) using logistic regression. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">After controlling for sociodemographic characteristics \u2026 bisexual men were more likely to report severe psychological distress (OR, 4.70; 95% CI, 1.77-12.52), heavy drinking (OR, 3.15; 95% CI, 1.22-8.16), and heavy smoking (OR, 2.10; 95% CI, 1.08-4.10) than heterosexual men\u2026.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref10\">[10]<\/a>&nbsp;Article: \u201cThe Myth of Margin of Error.\u201d By Jeffrey Henning. Researchscape, October 13, 2017. <a href=\"https:\/\/researchscape.com\/blog\/the-myth-of-margin-of-error\" rel=\"nofollow\">https:\/\/researchscape.com\/blog\/the-myth-of-margin-of-error<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The margin of sampling error is widely reported in public opinion surveys because it is the only error that can be easily calculated. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In fact, many researchers will just \u201cdo the math\u201d to calculate sampling error, ignoring the fact that the assumptions behind the calculation aren\u2019t being met.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref11\">[11]<\/a>&nbsp;Article: \u201cIowa Poll: Kamala Harris Leapfrogs Donald Trump to Take Lead Near Election Day. Here\u2019s How.\u201d By&nbsp; Brianne Pfannenstiel. Des Moines Register, November 2, 20224. Updated November 7, 2024. <a href=\"https:\/\/www.desmoinesregister.com\/story\/news\/politics\/iowa-poll\/2024\/11\/02\/iowa-poll-kamala-harris-leads-donald-trump-2024-presidential-race\/75354033007\/\" rel=\"nofollow\">https:\/\/www.desmoinesregister.com\/story\/news\/politics\/iowa-poll\/2024\/11\/02\/iowa-poll-kamala-harris-leads-donald-trump-2024-presidential-race\/75354033007\/<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A new Des Moines Register\/Mediacom Iowa Poll shows Vice President Harris leading former President Trump 47% to 44% among likely voters just days before a high-stakes election that appears deadlocked in key battleground states. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The poll of 808 likely Iowa voters, which include those who have already voted as well as those who say they definitely plan to vote, was conducted by Selzer &amp; Co. from Oct. 28-31. It has a margin of error of plus or minus 3.4 percentage points. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Questions based on the sample of 808 Iowa likely voters have a maximum margin of error of plus or minus 3.4 percentage points. This means that if this survey were repeated using the same questions and the same methodology, 19 times out of 20, the findings would not vary from the true population value by more than plus or minus 3.4 percentage points.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">NOTES:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Trump won Iowa by 13.2 percentage points,\u00a0<a href=\"https:\/\/www.cnn.com\/election\/2024\/results\/president?election-data-id=2024-PG&amp;election-painting-mode=projection-with-lead&amp;filter-key-races=false&amp;filter-flipped=false&amp;filter-remaining=false\" target=\"_blank\" rel=\"noreferrer noopener\">receiving<\/a>\u00a055.7% of the vote as compared to 42.5% for Harris.<\/li>\n\n\n\n<li>The \u201cmaximum margin of error\u201d reported in this article was only the sampling margin of error, as documented in the footnote above.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref12\">[12]<\/a>&nbsp;Post: \u201cSignificant Marginal Effects but C.I.S for Predicted Margins Overlapping.\u201d By Dr. Clyde Schechter (Albert Einstein College of Medicine). Statalist, October 10, 2017. <a href=\"https:\/\/www.statalist.org\/forums\/forum\/general-stata-discussion\/general\/1290476-significant-marginal-effects-but-c-i-s-for-predicted-margins-overlapping\" rel=\"nofollow\">https:\/\/www.statalist.org\/forums\/forum\/general-stata-discussion\/general\/1290476-significant-marginal-effects-but-c-i-s-for-predicted-margins-overlapping<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If one of the goals is to assess the predicted margins, then they should be presented with confidence intervals because every estimate should always be given with an estimate of the associated uncertainty. (The confidence interval represents a bare minimum estimate of the uncertainty of any estimate in that it accounts only for sampling error, but it is better than nothing.)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref13\">[13]<\/a>&nbsp;Article: \u201cHandling Missing Within-Study Correlations in the Evaluation of Surrogate Endpoints.\u201d By Willem Collier and others.&nbsp;<em>Statistics in Medicine<\/em>, September 3, 2003. <a href=\"https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC10704210\/\" rel=\"nofollow\">https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC10704210\/<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To reduce bias in measures of the performance of the surrogate, the statistical model must account for the sampling error in each trial\u2019s estimated treatment effects and their potential correlation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A weighted least squares (WLS) approach is also frequently used\u2026. The WLS method accounts only for sampling error of estimated effects on the clinical endpoint.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref14\">[14]<\/a>&nbsp;Paper: \u201cMeasuring Coverage in MNCH: Total Survey Error and the Interpretation of Intervention Coverage Estimates from Household Surveys.\u201d&nbsp;<em>PLoS Medicine<\/em>. May 7, 2013. <a href=\"https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC3646211\/\" rel=\"nofollow\">https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC3646211\/<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Nationally representative household surveys are increasingly relied upon to measure maternal, newborn, and child health (MNCH) intervention coverage at the population level in low- and middle-income countries. Surveys are the best tool we have for this purpose and are central to national and global decision making. However, all survey point estimates have a certain level of error (total survey error) comprising sampling and non-sampling error, both of which must be considered when interpreting survey results for decision making. \u2026 Sampling error is usually thought of as the precision of a point estimate and is represented by 95% confidence intervals, which are measurable. \u2026 By contrast, the direction and magnitude of non-sampling error is almost always unmeasurable, and therefore unknown.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref15\">[15]<\/a>&nbsp;Report: \u201c2023 Crime in the United States.\u201d Federal Bureau of Investigation, September 2024. <a href=\"https:\/\/www.justfacts.com\/document\/crime_united_states_2023_fbi.pdf\" rel=\"nofollow\">https:\/\/www.justfacts.com\/document\/crime_united_states_2023_fbi.pdf<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 39 (of the PDF):<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">BJS [Bureau of Justice Statistics] derives the NCVS [National Crime Victimization Survey] estimates from interviewing a sample. The estimates are subject to a margin of error. This error is known and is reflected in the standard error of the estimate.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">NOTE: As documented in the footnote above, the \u201cmargin of error\u201d in this survey only accounts for the sampling margin of error.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref16\">[16]<\/a>&nbsp;Book:&nbsp;<em>Statistics for K\u20138 Educators<\/em>. By Robert Rosenfeld. Routledge, 2013.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 92:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In general, larger random samples will produce smaller margins of error. However, in the real world of research where a study takes time and costs money, at a certain point you just can\u2019t afford to increase the sample size. Your study will take too long or you may decide the increase in precision isn\u2019t worth the expense. For instance, if you increase the sample size from 1,000 to 4,000 the margin of error will drop from about 3% to about 2%, but you might quadruple the cost of your survey.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref17\">[17]<\/a>&nbsp;Paper: \u201cMeasuring Coverage in MNCH: Total Survey Error and the Interpretation of Intervention Coverage Estimates from Household Surveys.\u201d&nbsp;<em>PLoS Medicine<\/em>. May 7, 2013. <a href=\"https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC3646211\/\" rel=\"nofollow\">https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC3646211\/<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Sampling error is usually thought of as the precision of a point estimate and is represented by 95% confidence intervals, which are measurable. \u2026 By contrast, the direction and magnitude of non-sampling error is almost always unmeasurable, and therefore unknown.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref18\">[18]<\/a>&nbsp;Post: \u201cSignificant Marginal Effects but C.I.S for Predicted Margins Overlapping.\u201d By Dr. Clyde Schechter (Albert Einstein College of Medicine). Statalist, October 10, 2017. <a href=\"https:\/\/www.statalist.org\/forums\/forum\/general-stata-discussion\/general\/1290476-significant-marginal-effects-but-c-i-s-for-predicted-margins-overlapping\" rel=\"nofollow\">https:\/\/www.statalist.org\/forums\/forum\/general-stata-discussion\/general\/1290476-significant-marginal-effects-but-c-i-s-for-predicted-margins-overlapping<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If one of the goals is to assess the predicted margins, then they should be presented with confidence intervals because every estimate should always be given with an estimate of the associated uncertainty. (The confidence interval represents a bare minimum estimate of the uncertainty of any estimate in that it accounts only for sampling error, but it is better than nothing.)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref19\">[19]<\/a>&nbsp;Report: \u201cHow Crime in the United States Is Measured.\u201d Congressional Research Service, January 3, 2008. <a href=\"https:\/\/crsreports.congress.gov\/product\/pdf\/RL\/RL34309\" rel=\"nofollow\">https:\/\/crsreports.congress.gov\/product\/pdf\/RL\/RL34309<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Pages 26\u201327:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Because the NCVS [National Crime Victimization Survey] is a sample survey, it is subject to both sampling and non-sampling error, meaning that the estimated victimization rate might not accurately reflect the true victimization rate. Whenever samples are used to represent entire populations, there could be a discrepancy between the sample estimate and the true value of what the sample is trying to estimate. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The NCVS is also subject to non-sampling error. The methodology employed by the NCVS attempts to reduce the effects of non-sampling error as much as possible, but an unquantified amount remains.<sup>242<\/sup><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref20\">[20]<\/a>&nbsp;Report: \u201cEstimating the Incidence of Rape and Sexual Assault.\u201d Edited by Candace Kruttschnitt, William D. Kalsbeek, and Carol C. House. National Academy of Sciences, National Research Council, 2014. <a href=\"https:\/\/nap.nationalacademies.org\/catalog\/18605\/estimating-the-incidence-of-rape-and-sexual-assault\" rel=\"nofollow\">https:\/\/nap.nationalacademies.org\/catalog\/18605\/estimating-the-incidence-of-rape-and-sexual-assault<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 4:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">All surveys are subject to errors, and the NCVS [National Crime Victimization Survey] is no exception. An assessment of the errors and potential errors in a survey is important to understanding the overall quality of the estimates from that survey and to initiate improvements. Total survey error is a concept that involves a holistic view of all potential errors in a survey program, including both sampling error and various forms of nonsampling error.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref21\">[21]<\/a>&nbsp;Report: \u201cHow Crime in the United States Is Measured.\u201d Congressional Research Service, January 3, 2008. <a href=\"https:\/\/crsreports.congress.gov\/product\/pdf\/RL\/RL34309\" rel=\"nofollow\">https:\/\/crsreports.congress.gov\/product\/pdf\/RL\/RL34309<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 26:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Because the NCVS [National Crime Victimization Survey] is a sample survey, it is subject to both sampling and non-sampling error, meaning that the estimated victimization rate might not accurately reflect the true victimization rate. Whenever samples are used to represent entire populations, there could be a discrepancy between the sample estimate and the true value of what the sample is trying to estimate. The NCVS accounts for sampling error by calculating confidence intervals for estimated rates of victimization.<sup>238<\/sup>&nbsp;For example, in 2000, the estimated violent crime victimization rate was 27.9 victimizations per 100,000 people aged 12 and older.<sup>239<\/sup>&nbsp;The calculated 95% confidence interval<sup>240<\/sup>&nbsp;for the estimated violent crime victimization rate was 25.85 to 29.95 victimizations per 100,000 people aged 12 and older.<sup>241<\/sup><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref22\">[22]<\/a>&nbsp;Paper: The Correct Interpretation of Confidence Intervals. By Sze Huey and Say Beng Tan.&nbsp;<em>Proceedings of Singapore Healthcare<\/em>, 2010. <a href=\"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/201010581001900316\" rel=\"nofollow\">https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/201010581001900316<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 277:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A common misunderstanding about CIs is that for say a 95% CI (A to B), there is a 95% probability that the true population mean lies between A and B. This is an incorrect interpretation of 95% CI because the true population mean is a fixed unknown value that is either inside or outside the CI with 100% certainty. As an example, let us assume that we know that the true population mean systolic blood pressure and it is 120mmHg. A study conducted gave us a mean systolic blood pressure of 105mmHg with a 95% CI of (95.5 to 118.9 mmHg). Knowing that the true population mean is 120mmHg it would be incorrect to say that there is a 95% probability that the true population mean lies in the 95% CI of (95.5 to 118.9mmHg) because we are certain that the 95% CI calculated did not contain the true population mean. A 95% CI simply means that if the study is conducted multiple times (multiple sampling from the same population) with corresponding 95% CI for the mean constructed, we expect 95% of these CIs to contain the true population mean<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref23\">[23]<\/a>&nbsp;Article: \u201cWhat Does a Confidence Interval Mean?\u201d By&nbsp;<a href=\"https:\/\/www.allendowney.com\/wp\/\" target=\"_blank\" rel=\"noreferrer noopener\">Allen B. Downey<\/a>&nbsp;(Ph.D.), 2023. <a href=\"https:\/\/allendowney.github.io\/DataQnA\/confidence.html\" rel=\"nofollow\">https:\/\/allendowney.github.io\/DataQnA\/confidence.html<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here\u2019s a&nbsp;<a href=\"https:\/\/www.reddit.com\/r\/statistics\/comments\/1c3iqdd\/q_why_does_a_confidence_interval_not_tell_you\/\" target=\"_blank\" rel=\"noreferrer noopener\">question from the Reddit statistics forum<\/a>&nbsp;(with an edit for clarity):<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Why does a confidence interval not tell you that 90% of the time, [the true value of the population parameter] will be in the interval, or something along those lines?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I understand that the interpretation of confidence intervals is that with repeated samples from the population, 90% of the time the interval would contain the true value of whatever it is you\u2019re estimating. What I don\u2019t understand is why this method doesn\u2019t really tell you anything about what that parameter value is.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is, to put it mildly, a common source of confusion. And here is one of the responses:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">From a frequentist perspective, the true value of the parameter is fixed. Thus, once you have calculated your confidence interval, one if two things are true: either the true parameter value is inside the interval, or it is outside it. So the probability that the interval contains the true value is either 0 or 1, but you can never know which.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This response is the conventional answer to this question\u2014it is what you find in most textbooks and what is taught in most classes. And, in my opinion, it is wrong. To explain why, I\u2019ll start with a story.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Suppose Frank and Betsy visit a factory where 90% of the widgets are good and 10% are defective. Frank chooses a part at random and asks Betsy, \u201cWhat is the probability that this part is good?\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Betsy says, \u201cIf 90% of the parts are good, and you choose one at random, the probability is 90% that it is good.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u201cWrong!\u201d says Frank. \u201cSince the part has already been manufactured, one of two things must be true: either it is good or it is defective. So the probability is either 100% or 0%, but we don\u2019t know which.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Frank\u2019s argument is based on a strict interpretation of frequentism, which is a particular philosophy of probability. But it is not the only interpretation, and it is not a particularly good one. In fact, it suffers from several flaws. This example shows one of them\u2014in many real-world scenarios where it would be meaningful and useful to assign a probability to a proposition, frequentism simply refuses to do so.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Fortunately, Betsy is under no obligation to adopt Frank\u2019s interpretation of probability. She is free to adopt any of several alternatives that are consistent with her commonsense claim that a randomly-chosen part has a 90% probability of being functional. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Suppose that Frank is a statistics teacher and Betsy is one of his students. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Now suppose Frank asks, \u201cWhat is the probability that this CI contains the actual value of \u03bc that I chose?\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Betsy says, \u201cWe have established that 90% of the CIs generated by this process contain \u03bc, so the probability that this CI contains is 90%.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">And of course Frank says \u201cWrong! Now that we have computed the CI, it is unknown whether it contains the true parameter, but it is not random. The probability that it contains \u03bc is either 100% or 0%. We can\u2019t say it has a 90% chance of containing \u03bc.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Once again, Frank is asserting a particular interpretation of probability\u2014one that has the regrettable property of rendering probability nearly useless. Fortunately, Betsy is under no obligation to join Frank\u2019s cult.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Under most reasonable interpretations of probability, you can say that a specific 90% CI has a 90% chance of containing the true parameter. There is no real philosophical problem with that.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref24\">[24]<\/a>&nbsp;Book:&nbsp;<em>Statistics for K\u20138 Educators<\/em>. By Robert Rosenfeld. Routledge, 2013.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 92:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In general, larger random samples will produce smaller margins of error. However, in the real world of research where a study takes time and costs money, at a certain point you just can\u2019t afford to increase the sample size. Your study will take too long or you may decide the increase in precision isn\u2019t worth the expense. For instance, if you increase the sample size from 1,000 to 4,000 the margin of error will drop from about 3% to about 2%, but you might quadruple the cost of your survey.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref25\">[25]<\/a>&nbsp;Report: \u201cDrug Use, Dependence, and Abuse Among State Prisoners and Jail Inmates, 2007\u20132009.\u201d By Jennifer Bronson and others. U.S. Department of Justice, Bureau of Justice Statistics, June 2017. <a href=\"https:\/\/bjs.ojp.gov\/content\/pub\/pdf\/dudaspji0709.pdf\" rel=\"nofollow\">https:\/\/bjs.ojp.gov\/content\/pub\/pdf\/dudaspji0709.pdf<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 19:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Standard errors and tests of significance<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">As with any survey, the NIS [National Inmate Surveys] estimates are subject to error arising from their basis on a sample rather than a complete enumeration of the population of adult inmates in prisons and jails. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A common way to express this sampling variability is to construct a 95% confidence interval around each survey estimate.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref26\">[26]<\/a>&nbsp;Paper: \u201cMultiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects.\u201d By Michael L. Anderson.&nbsp;<em>Journal of the American Statistical Association<\/em>, December 2008. Pages 1481\u20131495. <a href=\"https:\/\/are.berkeley.edu\/~mlanderson\/pdf\/Anderson%202008a.pdf\" rel=\"nofollow\">https:\/\/are.berkeley.edu\/~mlanderson\/pdf\/Anderson%202008a.pdf<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 1481:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This article focuses on the three prominent early intervention experiments: the Abecedarian Project, the Perry Preschool Program, and the Early Training Project. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But serious statistical inference problems affect these studies. The experimental samples are very small, ranging from approximately 60 to 120. Statistical power is therefore limited, and the results of conventional tests based on asymptotic theory may be misleading. More importantly, the large number of measured outcomes raises concerns about multiple inference: Significant coefficients may emerge simply by chance, even if there are no treatment effects. This problem is well known in the theoretical literature \u2026 and the biostatistics field \u2026 but has received limited attention in the policy evaluation literature. These issues\u2014combined with a puzzling pattern of results in which early test score gains disappear within a few years and are followed a decade later by significant effects on adult outcomes\u2014have created serious doubts about the validity of the results\u2026.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 1484:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">[M]ost randomized evaluations in the social sciences test many outcomes but fail to apply any type of multiple inference correction. To gauge the extent of the problem, we conducted a survey of randomized evaluation works published from 2004 to 2006 in the fields of economic or employment policy, education, criminology, political science or public opinion, and child or adolescent welfare. Using the&nbsp;<em>CSA Illumina<\/em>&nbsp;social sciences databases, we identified 44 such articles in peer-reviewed journals. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Nevertheless, only 3 works (7%) implemented any type of multiple-inference correction. \u2026 Although multiple-inference corrections are standard (and often mandatory) in psychological research \u2026 they remain uncommon in other social sciences, perhaps because practitioners in these fields are unfamiliar with the techniques or because they have seen no evidence that they yield more robust conclusions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Pages 1493\u20131494:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">As a final demonstration of the value of correcting for multiple inference, we conduct a stand-alone reanalysis of the Perry Preschool Project, arguably the most influential of the three experiments. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">[A] conventional research design [i.e., one that does not account for multiple inference problems] \u2026 adds eight more significant or marginally significant outcomes: female adult arrests, female employment, male monthly income, female government transfers, female special education rates, male drug use (in the adverse direction), male employment, and female monthly income. Of these eight outcomes, two (male and female monthly income) are not included in the other two studies [Abecedarian and Early Training]. The remaining six fail to replicate in either of the other studies. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">[Previous] researchers have emphasized the subset of unadjusted significant outcomes rather than applying a statistical framework that is robust to problems of multiple inference. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Many studies in this field test dozens of outcomes and focus on the subset of results that achieve significance.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref27\">[27]<\/a>&nbsp;Paper: \u201cHARKing, Cherry-Picking, P-Hacking, Fishing Expeditions, and Data Dredging and Mining as Questionable Research Practices.\u201d&nbsp;<em>Journal of Clinical Psychiatry<\/em>, February 18, 2021. <a href=\"https:\/\/www.psychiatrist.com\/jcp\/harking-cherry-picking-p-hacking-fishing-expeditions-and-data-dredging-and-mining-as-questionable-research-practices\/\" rel=\"nofollow\">https:\/\/www.psychiatrist.com\/jcp\/harking-cherry-picking-p-hacking-fishing-expeditions-and-data-dredging-and-mining-as-questionable-research-practices\/<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em>P<\/em>-hacking is a QRP [questionable research practice] wherein a researcher persistently analyzes the data, in different ways, until a statistically significant outcome is obtained; the purpose is not to test a hypothesis but to obtain a significant result. Thus, the researcher may experiment with different statistical approaches to test a hypothesis; or may include or exclude covariates; or may experiment with different cutoff values; or may split groups or combine groups; or may study different subgroups; and the analysis stops either when a significant result is obtained or when the researcher runs out of options. The researcher then reports only the approach that led to the desired result.<sup>3,8<\/sup><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref28\">[28]<\/a>&nbsp;Paper: \u201cBig Little Lies: A Compendium and Simulation of&nbsp;<em>p<\/em>-Hacking Strategies.\u201d By Angelika M. Stefan and Felix D. Sch\u00f6nbrodt.&nbsp;<em>Royal Society Open Science<\/em>, February 2023. <a href=\"https:\/\/royalsocietypublishing.org\/doi\/10.1098\/rsos.220346\" rel=\"nofollow\">https:\/\/royalsocietypublishing.org\/doi\/10.1098\/rsos.220346<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In an academic system that promotes a \u2018publish or perish\u2019 culture, researchers are incentivized to exploit degrees of freedom in their design, analysis and reporting practices to obtain publishable outcomes [1]. In many empirical research fields, the widespread use of such questionable research practices has damaged the credibility of research results [2\u20135]. Ranging in the grey area between good practice and outright scientific misconduct, questionable research practices are often difficult to detect, and researchers are often not fully aware of their consequences [6\u20138].<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">One of the most prominent questionable research practices is&nbsp;<em>p<\/em>-hacking [4,9]. Researchers engage in&nbsp;<em>p<\/em>-hacking in the context of frequentist hypothesis testing, where the p-value determines the test decision. If the&nbsp;<em>p<\/em>-value is below a certain threshold&nbsp;<em>\u03b1<\/em>, it is labelled \u2018significant\u2019, and the null hypothesis can be rejected. In this paper, we define&nbsp;<em>p<\/em>-hacking broadly as any measure that a researcher applies to render a previously non-significant&nbsp;<em>p<\/em>-value significant.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em>p<\/em>-hacking was first described by De Groot [10] as a problem of multiple testing and selective reporting. The term \u2018<em>p<\/em>-hacking\u2019 appeared shortly after the onset of the replication crisis [9,11], and the practice has since been discussed as one of the driving factors of false-positive results in the social sciences and beyond [12\u201314]. Essentially,&nbsp;<em>p<\/em>-hacking exploits the problem of multiplicity, that is,&nbsp;<em>\u03b1<\/em>-error accumulation due to multiple testing [15]. Specifically, the probability to make at least one false-positive test decision increases as more hypothesis tests are conducted [16,17]. When researchers engage in&nbsp;<em>p<\/em>-hacking, they conduct multiple hypothesis tests without correcting for the&nbsp;<em>\u03b1<\/em>-error accumulation, and report only significant results from the group of tests. This practice dramatically increases the percentage of false-positive results in the published literature [18].<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref29\"><\/a>&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">[29] Article: \u201cIn Research, What Does A \u2018Significant Effect\u2019 Mean?\u201d By Matthew Di Carlo (PhD). Albert Shanker Institute, November 1, 2011. <a href=\"https:\/\/www.shankerinstitute.org\/blog\/research-what-does-significant-effect-mean\" rel=\"nofollow\">https:\/\/www.shankerinstitute.org\/blog\/research-what-does-significant-effect-mean<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Then there\u2019s the term \u201csignificant.\u201d \u201cSignificant\u201d is of course a truncated form of \u201cstatistically significant.\u201d Statistical significance means we can be confident that a given relationship is not zero. That is, the relationship or difference is probably not just random \u201cnoise.\u201d A significant effect can be either positive (we can be confident it\u2019s greater than zero) or negative (we can be confident it\u2019s less than zero). In other words, it is \u201csignificant\u201d insofar as it\u2019s not nothing. The better way to think about it is \u201cdiscernible.\u201d There\u2019s something there.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref30\">[30]<\/a>&nbsp;Paper: \u201cEffectiveness of Adding a Mask Recommendation to Other Public Health Measures to Prevent SARS-CoV-2 Infection in Danish Mask Wearers.\u201d By Henning Bundgaard and others.&nbsp;<em>Annals of Internal Medicine<\/em>, November 18, 2020. <a href=\"https:\/\/www.acpjournals.org\/doi\/10.7326\/M20-6817\" rel=\"nofollow\">https:\/\/www.acpjournals.org\/doi\/10.7326\/M20-6817<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u201cAlthough the difference observed was not statistically significant, the 95% CIs [confidence intervals] are compatible with a 46% reduction to a 23% increase in infection.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref31\">[31]<\/a>&nbsp;Report: \u201cDrug Use, Dependence, and Abuse Among State Prisoners and Jail Inmates, 2007\u20132009.\u201d By Jennifer Bronson and others. U.S. Department of Justice, Bureau of Justice Statistics, June 2017. <a href=\"https:\/\/bjs.ojp.gov\/content\/pub\/pdf\/dudaspji0709.pdf\" rel=\"nofollow\">https:\/\/bjs.ojp.gov\/content\/pub\/pdf\/dudaspji0709.pdf<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 19:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Standard errors and tests of significance<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">As with any survey, the NIS [National Inmate Survey] estimates are subject to error arising from their basis on a sample rather than a complete enumeration of the population of adult inmates in prisons and jails. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A common way to express this sampling variability is to construct a 95% confidence interval around each survey estimate.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref32\">[32]<\/a>&nbsp;Paper: \u201cSchool Vouchers and Student Outcomes: Experimental Evidence from Washington, DC.\u201d By Patrick J. Wolf and others.&nbsp;<em>Journal of Policy Analysis and Management<\/em>, Spring 2013. Pages 246-270. <a href=\"http:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/pam.21691\/abstract\" rel=\"nofollow\">http:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/pam.21691\/abstract<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 258: \u201cResults are described as statistically significant or highly statistically significant if they reach the 95 percent or 99 percent confidence level, respectively.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref33\">[33]<\/a>&nbsp;Article: \u201cStatistical Significance.\u201d By Michael McDonough.&nbsp;<em>Encyclopaedia Britannica<\/em>. Last updated October 15, 2024. <a href=\"https:\/\/www.britannica.com\/topic\/statistical-significance\" rel=\"nofollow\">https:\/\/www.britannica.com\/topic\/statistical-significance<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u201cStatistical significance implies that an observed result is not due to sampling error.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref34\">[34]<\/a>&nbsp;Article: \u201cIn Research, What Does A \u2018Significant Effect\u2019 Mean?\u201d By Matthew Di Carlo (PhD). Albert Shanker Institute, November 1, 2011. <a href=\"https:\/\/www.shankerinstitute.org\/blog\/research-what-does-significant-effect-mean\" rel=\"nofollow\">https:\/\/www.shankerinstitute.org\/blog\/research-what-does-significant-effect-mean<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Then there\u2019s the term \u201csignificant.\u201d \u201cSignificant\u201d is of course a truncated form of \u201cstatistically significant.\u201d Statistical significance means we can be confident that a given relationship is not zero. That is, the relationship or difference is probably not just random \u201cnoise.\u201d A significant effect can be either positive (we can be confident it\u2019s greater than zero) or negative (we can be confident it\u2019s less than zero). In other words, it is \u201csignificant\u201d insofar as it\u2019s not nothing. The better way to think about it is \u201cdiscernible.\u201d There\u2019s something there.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref35\">[35]<\/a>&nbsp;Paper: \u201cRelative Plasma Volume Monitoring During Hemodialysis Aids the Assessment of Dry-Weight.\u201d By Arjun D Sinha, Robert P Light, and Rajiv Agarwal.&nbsp;<em>Hypertension<\/em>, December 28, 2009. <a href=\"https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC2819307\/\" rel=\"nofollow\">https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC2819307\/<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u201cMean changes and their 95% confidence intervals are shown. If the confidence interval crosses zero, the mean is statistically insignificant at the 5% level.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref36\">[36]<\/a>&nbsp;Paper: \u201cInsignificant Effect of Arctic Amplification on the Amplitude of Midlatitude Atmospheric Waves.\u201d By Russell Blackport and James A Screen.&nbsp;<em>Science Advances<\/em>, February 19, 2020. <a href=\"https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC7030927\/\" rel=\"nofollow\">https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC7030927\/<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u201cIn all cases, the spread of the modeled LWA [local wave activity] trends crosses zero, consistent with the statistically insignificant observed multidecadal trends.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref37\">[37]<\/a>&nbsp;Paper: \u201cEffectiveness of Adding a Mask Recommendation to Other Public Health Measures to Prevent SARS-CoV-2 Infection in Danish Mask Wearers.\u201d By Henning Bundgaard and others.&nbsp;<em>Annals of Internal Medicine<\/em>, November 18, 2020. <a href=\"https:\/\/www.acpjournals.org\/doi\/10.7326\/M20-6817\" rel=\"nofollow\">https:\/\/www.acpjournals.org\/doi\/10.7326\/M20-6817<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u201cAlthough the difference observed was not statistically significant, the 95% CIs [confidence intervals] are compatible with a 46% reduction to a 23% increase in infection.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref38\">[38]<\/a>&nbsp;Paper: \u201cEffectiveness of Adding a Mask Recommendation to Other Public Health Measures to Prevent SARS-CoV-2 Infection in Danish Mask Wearers.\u201d By Henning Bundgaard and others.&nbsp;<em>Annals of Internal Medicine<\/em>, November 18, 2020. <a href=\"https:\/\/www.acpjournals.org\/doi\/10.7326\/M20-6817\" rel=\"nofollow\">https:\/\/www.acpjournals.org\/doi\/10.7326\/M20-6817<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u201cAlthough the difference observed was not statistically significant, the 95% CIs [confidence intervals] are compatible with a 46% reduction to a 23% increase in infection.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref39\">[39]<\/a>&nbsp;Paper: \u201cA Review of High Impact Journals Found That Misinterpretation of Non-Statistically Significant Results From Randomized Trials Was Common.\u201d By Karla Hemming, Iqra Javid, and Monica Taljaard.&nbsp;<em>Journal of Clinical Epidemiology<\/em>, May 2022. <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/pii\/S089543562200021X\" rel=\"nofollow\">https:\/\/www.sciencedirect.com\/science\/article\/pii\/S089543562200021X<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The first and most problematic issue is when inconclusive trials are interpreted as providing definitive evidence that the treatment under evaluation is ineffective&nbsp;<sup>[10]<\/sup>. This is referred to as conflating no evidence of a difference with evidence of no difference (i.e., conflating absence of evidence with evidence of absence)&nbsp;<sup>[1]<\/sup>. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Almost all abstracts of RCTs [randomized controlled trials] published in high impact journals with non-statistically significant primary outcomes appropriately report treatment effects and confidence intervals, yet most make definitive conclusions about active treatments being no different to the comparator treatment, despite this being prima facia [at first sight] inconsistent with a non-statistically significant primary outcome result. \u2026 In addition, a large number of studies unhelpfully provide no informative interpretation: in the overall conclusion they simply state that the result is non-statistically significant, despite having reported confidence intervals in the results section. \u2026 Clear statements that the study finding is inconclusive (i.e., when the confidence interval provides support for both benefit and harm) in reports of RCTs in high impact journals are rare. Despite high profile campaigns in 2016 to put a stop to this poor practice&nbsp;<sup>[38]<\/sup>, our review demonstrates that the practice of misinterpretation is still highly prevalent. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Thus, it might be possible that some studies which reported an overall interpretation of no difference between the two treatment arms were correct in this interpretation: some of these associated confidence intervals might well have excluded clinically important differences, although this was not transparent in the abstract&nbsp;<sup>[21]<\/sup>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref40\">[40]<\/a>&nbsp;Article: \u201cThe Most Objective Evidence Shows No Indication That Covid Vaccines Save More Lives Than They Take. By James D. Agresti. Just Facts, March 2, 2022. <a href=\"https:\/\/www.justfactsdaily.com\/most-objective-evidence-covid-vaccines-lives\" rel=\"nofollow\">https:\/\/www.justfactsdaily.com\/most-objective-evidence-covid-vaccines-lives<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In this case, the \u201cintervention\u201d is FDA-approved Covid vaccines, and the \u201coutcome\u201d is death. That vital data was gathered in RCTs involving&nbsp;<a href=\"https:\/\/www.justfacts.com\/reference\/covid-19_mrna_vaccines_mortality.xls\" target=\"_blank\" rel=\"noreferrer noopener\">72,663<\/a>&nbsp;adults and older children for the Moderna and Pfizer vaccines. However, the FDA presented these results in a place and manner likely to be overlooked, and no&nbsp;<a href=\"https:\/\/www.google.com\/search?q=Covid-19+%22Pfizer%22+%2221+deaths%22+%22vaccine%22+%2217%22+%22placebo%22&amp;num=100&amp;lr=&amp;safe=images&amp;hl=en&amp;biw=1024&amp;bih=654&amp;source=lnt&amp;tbs=cdr%3A1%2Ccd_min%3A7%2F1%2F2021%2Ccd_max%3A2%2F22%2F2022&amp;tbm=nws\" target=\"_blank\" rel=\"noreferrer noopener\">major<\/a>&nbsp;media&nbsp;<a href=\"https:\/\/www.google.com\/search?q=Covid-19+%22Moderna%22+%2216+deaths%22+%22vaccine%22+%22placebo%22&amp;num=100&amp;lr=&amp;safe=images&amp;hl=en&amp;biw=1536&amp;bih=1028&amp;tbs=cdr%3A1%2Ccd_min%3A7%2F1%2F2021%2Ccd_max%3A2%2F22%2F2022&amp;tbm=nws&amp;ei=sLYWYu-4BJ6yqtsPw4C_qAY&amp;ved=0ahUKEwjv0OCo8Zb2AhUemWoFHUPAD2UQ4dUDCAw&amp;uact=5&amp;oq=Covid-19+%22Moderna%22+%2216+deaths%22+%22vaccine%22+%22placebo%22&amp;gs_lcp=Cgxnd3Mtd2l6LW5ld3MQAzIFCCEQoAEyBQghEKsCMgUIIRCrAlCYC1iYC2DUDmgAcAB4AIABcIgB3AKSAQMyLjKYAQCgAQKgAQHAAQE&amp;sclient=gws-wiz-news\" target=\"_blank\" rel=\"noreferrer noopener\">outlet<\/a>&nbsp;has covered&nbsp;<a href=\"https:\/\/www.justfacts.com\/reference\/covid-19_mrna_vaccines_mortality.xls\" target=\"_blank\" rel=\"noreferrer noopener\">them<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The results reveal that&nbsp;<a href=\"https:\/\/www.justfacts.com\/reference\/covid-19_mrna_vaccines_mortality.xls\" target=\"_blank\" rel=\"noreferrer noopener\">70<\/a>&nbsp;people died during the Moderna and Pfizer trials, including&nbsp;<a href=\"https:\/\/www.justfacts.com\/reference\/covid-19_mrna_vaccines_mortality.xls\" target=\"_blank\" rel=\"noreferrer noopener\">37<\/a>&nbsp;who received Covid vaccines and&nbsp;<a href=\"https:\/\/www.justfacts.com\/reference\/covid-19_mrna_vaccines_mortality.xls\" target=\"_blank\" rel=\"noreferrer noopener\">33<\/a>&nbsp;who did not. Combined with the fact that&nbsp;<a href=\"https:\/\/www.justfacts.com\/reference\/covid-19_mrna_vaccines_mortality.xls\">half<\/a>&nbsp;of the study participants were given vaccinations and the other half were given placebos, these crucial results provide no indication that the vaccines save more lives than they take.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Accounting for sampling margins of error\u2014as is common for medical journals and uncommon for the media\u2014the results demonstrate with&nbsp;<a href=\"https:\/\/www.justfacts.com\/reference\/covid-19_mrna_vaccines_mortality.xls\" target=\"_blank\" rel=\"noreferrer noopener\">95% confidence<\/a>&nbsp;that:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2022 neither of the vaccines decreased or increased the absolute risk of death by any more than 0.08% over the course of the trials.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2022 the vaccines could prevent up to two deaths or cause up to three deaths per year among every 1,000 people.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref41\">[41]<\/a>&nbsp;Book:&nbsp;<em>Multiple Regression: A Primer<\/em>. By Paul D. Allison. Pine Forge Press, 1998.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Chapter 3: \u201cWhat Can Go Wrong With Multiple Regression?\u201d <a href=\"https:\/\/us.sagepub.com\/sites\/default\/files\/upm-binaries\/2726_allis03.pdf\" rel=\"nofollow\">https:\/\/us.sagepub.com\/sites\/default\/files\/upm-binaries\/2726_allis03.pdf<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Pages 57\u201358:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Sample size has a profound effect on tests of statistical significance. With a sample of 60 people, a correlation has to be at least .25 (in magnitude) to be significantly different from zero (at the .05 level). With a sample of 10,000 people, any correlation larger than .02 will be statistically significant. The reason is simple: There\u2019s very little information in a small sample, so estimates of correlations are very unreliable. If we get a correlation of .20, there may still be a good chance that the true correlation is zero. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Statisticians often describe small samples as having&nbsp;<em>low power<\/em>&nbsp;to test hypotheses. There is another, entirely different problem with small samples that is frequently confused with the issue of power. Most of the test statistics that researchers use (such as t tests, F tests, and chi-square tests) are only approximations. These approximations are usually quite good when the sample is large but may deteriorate markedly when the sample is small. That means that&nbsp;<em>p<\/em>&nbsp;values calculated for small samples may be only rough approximations of the true&nbsp;<em>p<\/em>&nbsp;values. If the calculated&nbsp;<em>p<\/em>&nbsp;value is .02, the true value might be something like .08. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That brings us to the inevitable question: What\u2019s a big sample and what\u2019s a small sample? As you may have guessed, there\u2019s no clear-cut dividing line. Almost anyone would consider a sample less than 60 to be small, and virtually everyone would agree that a sample of 1,000 or more is large. In between, it depends on a lot of factors that are difficult to quantify, at least in practice.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref42\">[42]<\/a>&nbsp;Article: \u201cRegulatory Scientists Are Quiet About EUA, Kids Vax, Paxlovid and Boosters.\u201d By Dr. Vinay Prasad. <a href=\"https:\/\/vinayprasadmdmph.substack.com\/p\/regulatory-scientists-are-quiet-about\" rel=\"nofollow\">https:\/\/vinayprasadmdmph.substack.com\/p\/regulatory-scientists-are-quiet-about<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Many scientists made a career fighting for better regulatory standards. Strangely, when it comes to the regulatory policy around COVID-19, they are dead quiet. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Regulatory experts have told us for year[s] that if outcomes are generally favorable, you need a very large randomized control trial to show a benefit. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2026 Boosting 20-year-olds should not come under the auspices of an EUA [emergency use authorization]. You should do a very large randomized trial to show it has a benefit. And if you can\u2019t run the trial because the sample size is too large that tells you something about how marginal the effect size is.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref43\">[43]<\/a>&nbsp;Article: \u201cFDA Violated Own Safety and Efficacy Standards in Approving Covid-19 Vaccines For Children.\u201d By James D. Agresti. Just Facts, July 14, 2022. <a href=\"https:\/\/www.justfactsdaily.com\/covid-19-vaccines-children-fda-standards-violated\" rel=\"nofollow\">https:\/\/www.justfactsdaily.com\/covid-19-vaccines-children-fda-standards-violated<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That doesn\u2019t mean the vaccine doesn\u2019t work, but there is no way to be sure. This is because the study was&nbsp;<a href=\"https:\/\/medical-dictionary.thefreedictionary.com\/underpowered+clinical+trial\" target=\"_blank\" rel=\"noreferrer noopener\">underpowered<\/a>, a medical term for clinical trials that don\u2019t enroll enough participants to detect important effects. Beyond severe Covid and hospitalizations for it, the&nbsp;<a href=\"https:\/\/www.fda.gov\/media\/159393\/download\" target=\"_blank\" rel=\"noreferrer noopener\">Pfizer<\/a>&nbsp;and&nbsp;<a href=\"https:\/\/www.fda.gov\/media\/144636\/download\" target=\"_blank\" rel=\"noreferrer noopener\">Moderna<\/a>&nbsp;trials were also too underpowered to measure:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2022 overall hospitalizations, which are far more informative than hospitalizations for Covid because they also measure the side effects of the vaccines.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2022 all-cause mortality, which is the&nbsp;<a href=\"https:\/\/www.justfactsdaily.com\/most-objective-evidence-covid-vaccines-lives\" target=\"_blank\" rel=\"noreferrer noopener\">only objective way<\/a>&nbsp;to be certain the vaccines save more lives than they take.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To determine the last of those measures with 95% confidence would require a trial with more than&nbsp;<a href=\"https:\/\/www.justfacts.com\/reference\/covid-19_vaccines_children_fda_standards_violated.xls\" target=\"_blank\" rel=\"noreferrer noopener\">half a billion<\/a>&nbsp;children for a full year. And that assumes the vaccine works flawlessly by preventing all Covid deaths and causing no deaths from side effects. This astronomically large number is needed because deaths from Covid-19 are extremely rare among children, amounting to about one out of every&nbsp;<a href=\"https:\/\/www.justfactsdaily.com\/omicron-delta-naturally-acquired-immunity-vaccines#lockdowns\" target=\"_blank\" rel=\"noreferrer noopener\">500,000<\/a>&nbsp;children in the first year of pandemic. In fact, children are about&nbsp;<a href=\"https:\/\/www.justfactsdaily.com\/omicron-delta-naturally-acquired-immunity-vaccines#lockdowns\" target=\"_blank\" rel=\"noreferrer noopener\">36 times<\/a>&nbsp;more likely to die of accidents than Covid-19.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Microscopically smaller than an adequate study, the Moderna vaccine trials for children aged 6 months to 5 years included a total of&nbsp;<a href=\"https:\/\/www.fda.gov\/media\/144636\/download#page=9\" target=\"_blank\" rel=\"noreferrer noopener\">6,388<\/a>&nbsp;children with a median blinded follow-up time of 68\u201371 days after the second dose. The Pfizer trial was&nbsp;<a href=\"https:\/\/www.fda.gov\/media\/159255\/download#page=10\" target=\"_blank\" rel=\"noreferrer noopener\">similarly sized<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Comparing the data above, the trials that were conducted would need to be about&nbsp;<a href=\"https:\/\/www.justfacts.com\/reference\/covid-19_vaccines_children_fda_standards_violated.xls\" target=\"_blank\" rel=\"noreferrer noopener\">400,000 times<\/a>&nbsp;larger\/longer to&nbsp;<a href=\"https:\/\/www.justfactsdaily.com\/most-objective-evidence-covid-vaccines-lives\" target=\"_blank\" rel=\"noreferrer noopener\">objectively determine<\/a>&nbsp;if the vaccines save more toddlers and preschoolers than they kill.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref44\">[44]<\/a>&nbsp;Commentary: \u201cScientists Rise Up Against Statistical Significance.\u201d By Valentin Amrhein, Sander Greenland, and Blake McShane.&nbsp;<em>Nature<\/em>, March 20, 2019. <a href=\"https:\/\/www.nature.com\/articles\/d41586-019-00857-9\" rel=\"nofollow\">https:\/\/www.nature.com\/articles\/d41586-019-00857-9<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In 2016, the American Statistical Association released a statement in&nbsp;<a href=\"https:\/\/amstat.tandfonline.com\/doi\/abs\/10.1080\/00031305.2016.1154108\" target=\"_blank\" rel=\"noreferrer noopener\">The American Statistician<\/a>&nbsp;warning against the misuse of statistical significance and&nbsp;<em>P<\/em>&nbsp;values. The issue also included many commentaries on the subject. This month, a special issue in the same journal attempts to push these reforms further. It presents more than 40 papers on \u2018Statistical inference in the 21st century: a world beyond&nbsp;<em>P<\/em>&nbsp;&lt; 0.05\u2019. The editors introduce the collection with the caution \u201cdon\u2019t say \u2018statistically significant\u2019\u201d<sup>3<\/sup>. Another article<sup>4<\/sup>&nbsp;with dozens of signatories also calls on authors and journal editors to disavow those terms.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We agree, and call for the entire concept of statistical significance to be abandoned.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We are far from alone. When we invited others to read a draft of this comment and sign their names if they concurred with our message, 250 did so within the first 24 hours. A week later, we had more than 800 signatories\u2014all checked for an academic affiliation or other indication of present or past work in a field that depends on statistical modelling\u2026.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref45\">[45]<\/a>&nbsp;Article: \u201cStatistical Significance.\u201d By Michael McDonough.&nbsp;<em>Encyclopaedia Britannica<\/em>. Last updated October 15, 2024. <a href=\"https:\/\/www.britannica.com\/topic\/statistical-significance\" rel=\"nofollow\">https:\/\/www.britannica.com\/topic\/statistical-significance<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Since its conception in the 18th century, statistical significance has become the gold standard for establishing the validity of a result. Statistical significance does not imply the size, importance, or practicality of an outcome; it simply indicates that the outcome\u2019s difference from a baseline is not due to chance. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A growing number of researchers have voiced concerns over the misinterpretation of, and overreliance on, statistical significance. Often, analysis ends once an observation has been deemed to be statistically significant, and the observation is treated as evidence of an effect.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref46\">[46]<\/a>&nbsp;Paper: \u201cA Review of High Impact Journals Found That Misinterpretation of Non-Statistically Significant Results From Randomized Trials Was Common.\u201d By Karla Hemming, Iqra Javid, and Monica Taljaard.&nbsp;<em>Journal of Clinical Epidemiology<\/em>, May 2022. <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/pii\/S089543562200021X\" rel=\"nofollow\">https:\/\/www.sciencedirect.com\/science\/article\/pii\/S089543562200021X<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The first and most problematic issue is when inconclusive trials are interpreted as providing definitive evidence that the treatment under evaluation is ineffective&nbsp;<sup>[10]<\/sup>. This is referred to as conflating no evidence of a difference with evidence of no difference (i.e., conflating absence of evidence with evidence of absence)&nbsp;<sup>[1]<\/sup>. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Almost all abstracts of RCTs [randomized controlled trials] published in high impact journals with non-statistically significant primary outcomes appropriately report treatment effects and confidence intervals, yet most make definitive conclusions about active treatments being no different to the comparator treatment, despite this being prima facia [at first sight] inconsistent with a non-statistically significant primary outcome result. \u2026 In addition, a large number of studies unhelpfully provide no informative interpretation: in the overall conclusion they simply state that the result is non-statistically significant, despite having reported confidence intervals in the results section. \u2026 Clear statements that the study finding is inconclusive (i.e., when the confidence interval provides support for both benefit and harm) in reports of RCTs in high impact journals are rare. Despite high profile campaigns in 2016 to put a stop to this poor practice&nbsp;<sup>[38]<\/sup>, our review demonstrates that the practice of misinterpretation is still highly prevalent. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Thus, it might be possible that some studies which reported an overall interpretation of no difference between the two treatment arms were correct in this interpretation: some of these associated confidence intervals might well have excluded clinically important differences, although this was not transparent in the abstract&nbsp;<sup>[21]<\/sup>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref47\">[47]<\/a>&nbsp;Textbook:&nbsp;<em>Statistics: Concepts and Controversies<\/em>&nbsp;(6th edition). By David S. Moore and William I. Notz. W. H. Freeman and Company, 2006.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 42: \u201cIt is usual to report the margin of error for 95% confidence. If a news report gives a margin of error but leaves out the confidence level, it\u2019s pretty safe to assume 95% confidence.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref48\">[48]<\/a>&nbsp;Book:&nbsp;<em>Statistics for K\u20138 Educators<\/em>. By Robert Rosenfeld. Routledge, 2013.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 91:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Why 95%? Why not some other percentage? This value gives a level of confidence that has been found convenient and practical for summarizing survey results. There is nothing inherently special about it. If you are willing to change from 95% to some other level of confidence, and consequently change the chances that your poll results are off from the truth, you will therefore change the resulting margin of error. At present, 95% is just the level that is commonly used in a great variety of polls and research projects.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref49\">[49]<\/a>&nbsp;Article: \u201cStatistical Significance.\u201d By Michael McDonough.&nbsp;<em>Encyclopaedia Britannica<\/em>. Last updated October 15, 2024. <a href=\"https:\/\/www.britannica.com\/topic\/statistical-significance\" rel=\"nofollow\">https:\/\/www.britannica.com\/topic\/statistical-significance<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A growing number of researchers have voiced concerns over the misinterpretation of, and overreliance on, statistical significance. Often, analysis ends once an observation has been deemed to be statistically significant, and the observation is treated as evidence of an effect. This tendency is especially problematic given that statistical significance is not equal to clinical significance, a measure of effect size and practical importance. In an experiment, a statistically significant result simply indicates that a difference exists between two groups. This difference might be incredibly small, but, without further testing, its practical impact is unknown.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref50\">[50]<\/a>&nbsp;Paper: \u201cBeyond Statistical Significance: Clinical Interpretation of Rehabilitation Research Literature.\u201d By Phil Page.&nbsp;<em>International Journal of Sports Physical Therapy<\/em>, October 9, 2014. <a href=\"https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC4197528\/\" rel=\"nofollow\">https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC4197528\/<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">While most research focus on statistical significance, clinicians and clinical researchers should focus on clinically significant changes. A study outcome can be statistically significant, but not be clinically significant, and vice\u2010versa. Unfortunately, clinical significance is not well defined or understood, and many research consumers mistakenly relate statistically significant outcomes with clinical relevance. Clinically relevant changes in outcomes are identified (sometimes interchangeably) by several similar terms including \u201cminimal clinically important differences (MCID)\u201d, \u201cclinically meaningful differences (CMD)\u201d, and \u201cminimally important changes (MIC)\u201d.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In general, these terms all refer to the smallest change in an outcome score that is considered \u201cimportant\u201d or \u201cworthwhile\u201d by the practitioner or the patient<sup>8<\/sup>&nbsp;and\/or would result in a change in patient management<sup>9,10<\/sup>. Changes in outcomes exceeding these minimal values are considered clinically relevant. It is important to consider that both harmful changes and beneficial changes may be outcomes of treatment; therefore, the term \u201cclinically\u2010important changes\u201d should be used to identify both minimal and beneficial differences, but also to recognize harmful changes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref51\">[51]<\/a>&nbsp;Commentary: \u201cScientists Rise Up Against Statistical Significance.\u201d By Valentin Amrhein, Sander Greenland, and Blake McShane.&nbsp;<em>Nature<\/em>, March 20, 2019. <a href=\"https:\/\/www.nature.com\/articles\/d41586-019-00857-9\" rel=\"nofollow\">https:\/\/www.nature.com\/articles\/d41586-019-00857-9<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">How do statistics so often lead scientists to deny differences that those not educated in statistics can plainly see? For several generations, researchers have been warned that a statistically non-significant result does not \u2018prove\u2019 the null hypothesis (the hypothesis that there is no difference between groups or no effect of a treatment on some measured outcome)<sup>1<\/sup>. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These and similar errors are widespread. Surveys of hundreds of articles have found that statistically non-significant results are interpreted as indicating \u2018no difference\u2019 or \u2018no effect\u2019 in around half \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2026 Neither should we conclude that two studies conflict because one had a statistically significant result and the other did not. These errors waste research efforts and misinform policy decisions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, consider a series of analyses of unintended effects of anti-inflammatory drugs<sup>2<\/sup>. Because their results were statistically non-significant, one set of researchers concluded that exposure to the drugs was \u201cnot associated\u201d with new-onset atrial fibrillation (the most common disturbance to heart rhythm) and that the results stood in contrast to those from an earlier study with a statistically significant outcome.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Now, let\u2019s look at the actual data. The researchers describing their statistically non-significant results found a risk ratio of 1.2 (that is, a 20% greater risk in exposed patients relative to unexposed ones). They also found a 95% confidence interval that spanned everything from a trifling risk decrease of 3% to a considerable risk increase of 48% (<em>P<\/em>\u2009=\u20090.091; our calculation). The researchers from the earlier, statistically significant, study found the exact same risk ratio of 1.2. That study was simply more precise, with an interval spanning from 9% to 33% greater risk (<em>P<\/em>\u2009=\u20090.0003; our calculation).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It is ludicrous to conclude that the statistically non-significant results showed \u201cno association,\u201d when the interval estimate included serious risk increases; it is equally absurd to claim these results were in contrast with the earlier results showing an identical observed effect.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref52\">[52]<\/a>&nbsp;Entry: \u201cunderpowered clinical trial.\u201d Segen\u2019s Medical Dictionary, 2012. <a href=\"https:\/\/medical-dictionary.thefreedictionary.com\/underpowered+clinical+trial\" rel=\"nofollow\">https:\/\/medical-dictionary.thefreedictionary.com\/underpowered+clinical+trial<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u201cA clinical trial that has so few patients in each arm that the results will fall short of the statistical power needed to provide valid answers.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref53\">[53]<\/a>&nbsp;Paper: \u201cA Review of High Impact Journals Found That Misinterpretation of Non-Statistically Significant Results From Randomized Trials Was Common.\u201d By Karla Hemming, Iqra Javid, and Monica Taljaard.&nbsp;<em>Journal of Clinical Epidemiology<\/em>, May 2022. <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/pii\/S089543562200021X\" rel=\"nofollow\">https:\/\/www.sciencedirect.com\/science\/article\/pii\/S089543562200021X<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The first and most problematic issue is when inconclusive trials are interpreted as providing definitive evidence that the treatment under evaluation is ineffective&nbsp;<sup>[10]<\/sup>. This is referred to as conflating no evidence of a difference with evidence of no difference (i.e., conflating absence of evidence with evidence of absence)&nbsp;<sup>[1]<\/sup>. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Almost all abstracts of RCTs [randomized controlled trials] published in high impact journals with non-statistically significant primary outcomes appropriately report treatment effects and confidence intervals, yet most make definitive conclusions about active treatments being no different to the comparator treatment, despite this being prima facia [at first sight] inconsistent with a non-statistically significant primary outcome result. \u2026 In addition, a large number of studies unhelpfully provide no informative interpretation: in the overall conclusion they simply state that the result is non-statistically significant, despite having reported confidence intervals in the results section. \u2026 Clear statements that the study finding is inconclusive (i.e., when the confidence interval provides support for both benefit and harm) in reports of RCTs in high impact journals are rare. Despite high profile campaigns in 2016 to put a stop to this poor practice&nbsp;<sup>[38]<\/sup>, our review demonstrates that the practice of misinterpretation is still highly prevalent.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref54\">[54]<\/a>&nbsp;Commentary: \u201cScientists Rise Up Against Statistical Significance.\u201d By Valentin Amrhein, Sander Greenland, and Blake McShane.&nbsp;<em>Nature<\/em>, March 20, 2019. <a href=\"https:\/\/www.nature.com\/articles\/d41586-019-00857-9\" rel=\"nofollow\">https:\/\/www.nature.com\/articles\/d41586-019-00857-9<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Nor do statistically significant results \u2018prove\u2019 some other hypothesis. Such misconceptions have famously warped the literature with overstated claims and, less famously, led to claims of conflicts between studies where none exists.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref55\">[55]<\/a>&nbsp;Paper: \u201cSea-Level Rise for the Coasts of California, Oregon, and Washington: Past, Present, and Future.\u201d By the Committee on Sea Level Rise in California, Oregon, and Washington, National Research Council. National Academies Press, 2012. <a href=\"http:\/\/www.nap.edu\/catalog.php?record_id=13389\" rel=\"nofollow\">http:\/\/www.nap.edu\/catalog.php?record_id=13389<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Pages 3\u20134:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Projections of global sea-level rise are generally made using models of the ocean-atmosphere- climate system, extrapolations, or semi-empirical methods. Ocean-atmosphere models are based on knowledge of the physical processes that contribute to sea-level rise, and they predict the response of those processes to different scenarios of future greenhouse gas emissions. These models provide a reasonable estimate of the water density (steric) component of sea-level rise (primarily thermal expansion), but they underestimate the land ice contribution because they do not fully account for rapid changes in the behavior of ice sheets and glaciers as melting occurs (ice dynamics). The IPCC (2007) projections were made using this method, and they are likely too low, even with an added ice dynamics component. Estimates of the total land ice contribution can be made by extrapolating current observations of ice loss rates from glaciers, ice caps, and ice sheets into the future. Extrapolations of future ice melt are most reliable for time frames in which the dynamics controlling behavior are stable, in this case, up to several decades. Semi-empirical methods, exemplified by Vermeer and Rahmstorf (2009), avoid the difficulty of estimating the individual contributions to sea-level rise by simply postulating that sea level rises faster as the Earth gets warmer. This approach reproduces the sea-level rise observed in the past, but reaching the highest projections would require acceleration of glaciological processes to levels not previously observed or understood as realistic. \u2026.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Given the strengths and weaknesses of the different projection methods, as well as the resource constraints of an NRC study, the committee chose a combination of approaches for its projections. The committee projected the steric component of sea-level rise using output from global ocean models under an IPCC (2007) mid-range greenhouse gas emission scenario. The land ice component was extrapolated using the best available compilations of ice mass accumulation and loss (mass balance), which extend from 1960 to 2005 for glaciers and ice caps, and from 1992 to 2010 for the Greenland and Antarctic ice sheets. The contributions were then summed. The committee did not project the land hydrology contribution because available estimates suggested that the sum of groundwater extraction and reservoir storage is near zero, within large uncertainties.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref56\">[56]<\/a>&nbsp;Textbook:&nbsp;<em>Flood Geomorphology<\/em>. By Victor R. Baker and others. Wiley, April 1998.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page ix:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">[T]rue science is concerned with understanding nature no matter what the methodology. In our view, if the wrong equations are programmed because of inadequate understanding of the system, then what the computer will produce, if believed by the analyst, will constitute the opposite of science.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref57\">[57]<\/a>&nbsp;Paper: \u201cThe Use and Misuse of Models for Climate Policy.\u201d By Robert S. Pindyck.&nbsp;<em>Review of Environmental Economics and Policy<\/em>, March 11, 2017. <a href=\"https:\/\/www.journals.uchicago.edu\/doi\/10.1093\/reep\/rew012\" rel=\"nofollow\">https:\/\/www.journals.uchicago.edu\/doi\/10.1093\/reep\/rew012<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In a recent article (Pindyck 2013a), I argued that integrated assessment models (IAMs) \u201chave crucial flaws that make them close to useless as tools for policy analysis\u201d (page 860). In fact, I would argue that the problem goes beyond their \u201ccrucial flaws\u201d: IAM-based analyses of climate policy create a perception of knowledge and precision that is illusory and can fool policymakers into thinking that the forecasts the models generate have some kind of scientific legitimacy. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The argument is sometimes made that we have no choice\u2014that without a model we will end up relying on biased opinions, guesswork, or even worse. Thus we must develop the best models possible and then use them to evaluate alternative policies. In other words, the argument is that working with even a highly imperfect model is better than having no model at all. This might be a valid argument if we were honest and up-front about the limitations of the model. But often we are not.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref58\">[58]<\/a>&nbsp;Report: \u201cFace Coverings in the Community and COVID-19: A Rapid Review.\u201d Public Health England, June 26, 2020. <a href=\"https:\/\/www.justfacts.com\/document\/face_coverings_community_covid-19_public_health_england_june_2020.pdf\" rel=\"nofollow\">https:\/\/www.justfacts.com\/document\/face_coverings_community_covid-19_public_health_england_june_2020.pdf<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 6:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Part of the limitations of modelling studies is that they must make assumptions in cases where the evidence or data are lacking. For example, models used different parameters to define \u2018effectiveness\u2019 of masks, which ranged from an 8% (24) reduction in risk to &gt;95% (29) reduction in risk. The nature of modelling studies also means that simulations are run in controlled environments that may not accurately reflect the behaviours that we observe in real life. Unless controlled for, parameters can be fixed that are usually variable.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Pages 7\u20138:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">[M]odelling and laboratories studies provide only theoretical evidence\u2026. We, therefore, cannot recommend the use of modelling studies alone as evidence to inform or change policy measures.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref59\">[59]<\/a>&nbsp;Commentary: \u201cFive Ways to Ensure That Models Serve Society: A Manifesto.\u201d By Andrea Saltelli and others.&nbsp;<em>Nature<\/em>, June 24, 2020. <a href=\"https:\/\/www.nature.com\/articles\/d41586-020-01812-9\" rel=\"nofollow\">https:\/\/www.nature.com\/articles\/d41586-020-01812-9<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Now, computer modelling is in the limelight, with politicians presenting their policies as dictated by \u2018science\u2019<sup>2<\/sup>. Yet there is no substantial aspect of this pandemic for which any researcher can currently provide precise, reliable numbers. Known unknowns include the prevalence and fatality and reproduction rates of the virus in populations. There are few estimates of the number of asymptomatic infections, and they are highly variable. We know even less about the seasonality of infections and how immunity works, not to mention the impact of social-distancing interventions in diverse, complex societies.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Mathematical models produce highly uncertain numbers that predict future infections, hospitalizations and deaths under various scenarios. Rather than using models to inform their understanding, political rivals often brandish them to support predetermined agendas. To make sure predictions do not become adjuncts to a political cause, modellers, decision makers and citizens need to establish new social norms. Modellers must not be permitted to project more certainty than their models deserve; and politicians must not be allowed to offload accountability to models of their choosing<sup>2,3<\/sup>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref60\">[60]<\/a>&nbsp;Paper: \u201cRisk of Bias in Model-Based Economic Evaluations: The ECOBIAS Checklist. By Charles Christian Adarkwah and others. Expert Review of Pharmacoeconomics &amp; Outcomes Research, November 20, 2015. <a href=\"https:\/\/www.researchgate.net\/profile\/Charles-Adarkwah-2\/publication\/284274465_Risk_of_bias_in_model-based_economic_evaluations_the_ECOBIAS_checklist\/links\/56544ebb08aeafc2aabbb745\/Risk-of-bias-in-model-based-economic-evaluations-the-ECOBIAS-checklist.pdf\" rel=\"nofollow\">https:\/\/www.researchgate.net\/profile\/Charles-Adarkwah-2\/publication\/284274465_Risk_of_bias_in_model-based_economic_evaluations_the_ECOBIAS_checklist\/links\/56544ebb08aeafc2aabbb745\/Risk-of-bias-in-model-based-economic-evaluations-the-ECOBIAS-checklist.pdf<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a><\/a>Page 1:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Economic evaluations are becoming increasingly important in providing policymakers with information for reimbursement decisions. However, in many cases, there is a significant difference between theoretical study results and real-life observations. This can be due to confounding factors or many other variables, which could be significantly affected by bias. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">There are basically two analytical frameworks used to conduct economic evaluations: model-based and trial-based. In a model-based economic evaluation, data from a wide range of sources [e.g., randomized-controlled trials (RCTs)], meta-analyses, observational studies) are combined using a mathematical model to represent the complexity of a healthcare process.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 6:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This study identified several additional biases related to model-based economic evaluation and showed that the impact of these biases could be massive, changing the outcomes from being highly cost-effective to not being cost-effective at all.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref61\">[61]<\/a>&nbsp;Paper: \u201cEconomic Evaluations in Fracture Research an Introduction with Examples of Foot Fractures.\u201d By Noortje Anna Clasina van den Boom and others.&nbsp;<em>Injury<\/em>, March 2022. <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0020138322000146\" rel=\"nofollow\">https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0020138322000146<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The lack of reliable data in the field of economic evaluation fractures could be explained by the lack of reliable literature to base the models on. Since model based studies are the most common design in this field of research, this problem is significant.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref62\">[62]<\/a>&nbsp;Paper: \u201cComparison of Health and Health Risk Factors Between Lesbian, Gay, and Bisexual Adults and Heterosexual Adults in the United States.\u201d By Gilbert Gonzales, Julia Przedworski, and Carrie Henning-Smith.&nbsp;<em>Journal of the American Medical Association<\/em>, June&nbsp;27, 2016. <a href=\"http:\/\/archinte.jamanetwork.com\/article.aspx?articleid=2530417\" rel=\"nofollow\">http:\/\/archinte.jamanetwork.com\/article.aspx?articleid=2530417<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Finally, the NHIS [National Health Interview Survey] is a cross-sectional survey and cannot definitively establish the causal directions of the observed associations because cross-sectional studies are prone to omitted variable bias. Missing and unmeasured variables\u2014such as exposure to discrimination or nondisclosure of sexual orientation to family, friends, and health care professionals\u2014may provide alternative explanations for the association between sexual orientation and health outcomes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">NOTE: See the next footnote, where the lead author of this study makes a causal inference about the study.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref63\">[63]<\/a>&nbsp;Article: \u201cSurvey Finds Excess Health Problems in Lesbians, Gays, Bisexuals.\u201d By Andrew M. Seaman. Reuters, June 28, 2016. <a href=\"https:\/\/ca.news.yahoo.com\/survey-finds-excess-health-problems-lesbians-gays-bisexuals-224741845.html\" rel=\"nofollow\">https:\/\/ca.news.yahoo.com\/survey-finds-excess-health-problems-lesbians-gays-bisexuals-224741845.html<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Gilbert Gonzales of the Vanderbilt University School of Medicine in Nashville and colleagues found that compared to heterosexual women, lesbians were 91 percent more likely to report poor or fair health. Lesbians were 51 percent more likely, and bisexual women were more than twice as likely, to report multiple chronic conditions, compared to straight women. \u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Gonzales told Reuters Health that the health disparities are likely due to the stress of being a minority, which is likely exacerbated among bisexual people, who may not be accepted by lesbian, gay, bisexual and transgender communities.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref64\">[64]<\/a>&nbsp;Paper: \u201cAssociation Is Not Causation: Treatment Effects Cannot Be Estimated From Observational Data in Heart Failure.\u201d By Christopher J Rush and others.&nbsp;<em>European Heart Journal<\/em>, October 2018. <a href=\"https:\/\/academic.oup.com\/eurheartj\/article\/39\/37\/3417\/5063542\" rel=\"nofollow\">https:\/\/academic.oup.com\/eurheartj\/article\/39\/37\/3417\/5063542<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This comprehensive comparison of studies of non-randomized data with the findings of RCTs [randomized controlled trials] in HF [heart failure] shows that it is not possible to make reliable therapeutic inferences from observational associations.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref65\"><\/a>&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">[65] Textbook:&nbsp;<em>Principles and Practice of Clinical Research<\/em>. By John I. Gallin and \u200eFrederick P. Ognibene. Academic Press, 2012.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 226: \u201cWhile consistency in the findings of a large number of observational studies can lead to the belief that the associations are causal, this belief is a fallacy.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref66\">[66]<\/a>&nbsp;Book:&nbsp;<em>Introductory Econometrics: Using Monte Carlo Simulation with Microsoft Excel<\/em>. By Humberto Barreto and Frank M. Howland. Cambridge University Press, 2006.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 491:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Omitted variable bias is a crucial topic because almost every study in econometrics is an observational study as opposed to a controlled experiment. Very often, economists would like to be able to interpret the comparisons they make as if they were the outcomes of controlled experiments. In a properly conducted controlled experiment, the only systematic difference between groups results from the treatment under investigation; all other variation stems from chance. In an observational study, because the participants self-select into groups, it is always possible that varying average outcomes between groups result from systematic difference between groups other than the treatment. We can attempt to control for these systematic differences by explicitly incorporating variables in a regression. Unfortunately, if not all of those differences have been controlled for in the analysis, we are vulnerable to the devastating effects of omitted variable bias.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wattsupwiththat.com\/2025\/01\/24\/margins-of-error\/#_ednref67\">[67]<\/a>&nbsp;Book:&nbsp;<em>Regression With Social Data: Modeling Continuous and Limited Response Variables<\/em>. By Alfred DeMaris. John Wiley &amp; Sons, 2004.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 9:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Regression modeling of nonexperimental data for the purpose of making causal inferences is ubiquitous in the social sciences. Sample regression coefficients are typically thought of as estimates of the causal impacts of explanatory variables on the outcome. Even though researchers may not acknowledge this explicitly, their use of such language as impact or effect to describe a coefficient value often suggest a causal interpretation. This practice is fraught with controversy\u2026.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 12:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Friedman \u2026 is especially critical of drawing causal inferences from observational data, since all that can be \u201cdiscovered,\u201d regardless of the statistical candlepower used, is association. Causation has to be assumed into the structure from the beginning. Or, as Friedman \u2026 says: \u201cIf you want to pull a causal rabbit out of the hat, you have to put the rabbit into the hat.\u201d In my view, this point is well taken; but it does not preclude using regression for causal inference. What it means, instead, is that&nbsp;<em>prior knowledge of the causal status of one\u2019s regressors<\/em>&nbsp;is a prerequisite for endowing regression coefficients with a causal interpretation, as acknowledged by Pearl 1998.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Page 13:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In sum, causal modeling via regression, using nonexperimental data, can be a useful enterprise provided we bear in mind that several strong assumptions are required to sustain it. First, regardless of the sophistication of our methods, statistical techniques only allow us to examine&nbsp;<em>associations<\/em>&nbsp;among variables.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Ever lost a bet? From the lottery or sporting events to casinos or friendly wagers, you may have risked and lost some money because you hoped to win big.<\/p>\n<p>But let me ask you this: How big would the payout have to be and how good would the odds need to be to gamble with your life or the lives of your loved ones?<\/p>\n","protected":false},"author":121246920,"featured_media":363474,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","_crdt_document":"","advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1],"tags":[691832869,691832867,691832868,691832870],"class_list":{"0":"post-363473","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","6":"hentry","7":"category-uncategorized","8":"tag-incompetence-and-dishonesty","9":"tag-just-facts-academy","10":"tag-sampling-error","11":"tag-statistics","13":"fallback-thumbnail"},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2025\/01\/0Screenshot-2025-01-26-102032.png?fit=1393%2C779&ssl=1","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/paxLW1-1wyt","jetpack-related-posts":[{"id":224253,"url":"https:\/\/climatescience.press\/?p=224253","url_meta":{"origin":363473,"position":0},"title":"COP27 \u2014 Hopeless hopeful money will dominate the discussion","author":"uwe.roland.gross","date":"17\/10\/2022","format":false,"excerpt":"Lots of money, many trillions by the wishes, all flowing from the developed world to the \u201cdeveloping\u201d world.","rel":"","context":"Similar post","block_context":{"text":"Similar post","link":""},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2022\/10\/image-803.png?fit=848%2C565&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2022\/10\/image-803.png?fit=848%2C565&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2022\/10\/image-803.png?fit=848%2C565&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2022\/10\/image-803.png?fit=848%2C565&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":311002,"url":"https:\/\/climatescience.press\/?p=311002","url_meta":{"origin":363473,"position":1},"title":"Hertz CEO Steps Down after Catastrophic EV Losses","author":"uwe.roland.gross","date":"21\/03\/2024","format":false,"excerpt":"CEO steps down after being hit with expensive EV repairs and low resale prices following purchase of 100,000 Teslas","rel":"","context":"In \"electric vehicles (EVs)\"","block_context":{"text":"electric vehicles (EVs)","link":"https:\/\/climatescience.press\/?tag=electric-vehicles-evs"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2024\/03\/0TESLAHERTZ.webp?fit=1200%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2024\/03\/0TESLAHERTZ.webp?fit=1200%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2024\/03\/0TESLAHERTZ.webp?fit=1200%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2024\/03\/0TESLAHERTZ.webp?fit=1200%2C800&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2024\/03\/0TESLAHERTZ.webp?fit=1200%2C800&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":274336,"url":"https:\/\/climatescience.press\/?p=274336","url_meta":{"origin":363473,"position":2},"title":"Montana climate decision no big deal","author":"uwe.roland.gross","date":"18\/08\/2023","format":false,"excerpt":"Much ado is being made from the supposed win of a kid\u2019s climate lawsuit in Montana. The alarmists call it a victory, the skeptics a tragedy, but it is neither. What was won is almost funny, while the big ask was in fact denied. The climate kids won a little,\u2026","rel":"","context":"In \"climate alarmist\"","block_context":{"text":"climate alarmist","link":"https:\/\/climatescience.press\/?tag=climate-alarmist"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/08\/070590436007-ap-youth-climate-lawsuit-montana.webp?fit=1200%2C679&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/08\/070590436007-ap-youth-climate-lawsuit-montana.webp?fit=1200%2C679&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/08\/070590436007-ap-youth-climate-lawsuit-montana.webp?fit=1200%2C679&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/08\/070590436007-ap-youth-climate-lawsuit-montana.webp?fit=1200%2C679&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/08\/070590436007-ap-youth-climate-lawsuit-montana.webp?fit=1200%2C679&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":243250,"url":"https:\/\/climatescience.press\/?p=243250","url_meta":{"origin":363473,"position":3},"title":"The Ministry of Climate Truth","author":"uwe.roland.gross","date":"05\/02\/2023","format":false,"excerpt":"What price misinformation?","rel":"","context":"Similar post","block_context":{"text":"Similar post","link":""},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/02\/image-168.png?fit=1200%2C675&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/02\/image-168.png?fit=1200%2C675&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/02\/image-168.png?fit=1200%2C675&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/02\/image-168.png?fit=1200%2C675&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/02\/image-168.png?fit=1200%2C675&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":262587,"url":"https:\/\/climatescience.press\/?p=262587","url_meta":{"origin":363473,"position":4},"title":"Is Climate Change a Black\u00a0Swan?","author":"uwe.roland.gross","date":"17\/06\/2023","format":false,"excerpt":"You can do as much Monte Carlo analysis as you want in order to\u00a0model schedule risk, but the brute fact is that you cannot account for the impact of your ignorance, and so you are likely wasting your time. Another way of putting this is that the risk profile is\u2026","rel":"","context":"In \"Climate change\"","block_context":{"text":"Climate change","link":"https:\/\/climatescience.press\/?tag=climate-change"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/06\/0-Black-Swan.jpg?fit=1200%2C878&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/06\/0-Black-Swan.jpg?fit=1200%2C878&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/06\/0-Black-Swan.jpg?fit=1200%2C878&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/06\/0-Black-Swan.jpg?fit=1200%2C878&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2023\/06\/0-Black-Swan.jpg?fit=1200%2C878&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":417933,"url":"https:\/\/climatescience.press\/?p=417933","url_meta":{"origin":363473,"position":5},"title":"Climate Fundraising Update: Hope amid Doom","author":"uwe.roland.gross","date":"20\/12\/2025","format":false,"excerpt":"It\u2019s a funny, scary time in the climate alarmist camp. Once-favorable economic and political trends are going the other way in the U.S. and increasingly in the world. What to do? Literally thousands of climate-issue employees (grifters?) need to find alternative employment, almost all in net-positive (CO2) industries where value\u2026","rel":"","context":"In \"carbon dioxide (CO\u2082)\"","block_context":{"text":"carbon dioxide (CO\u2082)","link":"https:\/\/climatescience.press\/?tag=carbon-dioxide-co%e2%82%82"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2025\/12\/AQNa2jaLunUspz4plPCTCV_dbvsjyUiiJT-143Bp5CvER8FrjkJ7ZDFzpGguGKgQP4tbsh45bVbmI-kAXeSkBI4KhChr3kUF7YVoxKf0Pzw4dWZjiTCYFIx6SRJOH4KZ3wnsxtyhlMPHsoJpLHnfTKxcUTarTA.jpeg?fit=1200%2C1200&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2025\/12\/AQNa2jaLunUspz4plPCTCV_dbvsjyUiiJT-143Bp5CvER8FrjkJ7ZDFzpGguGKgQP4tbsh45bVbmI-kAXeSkBI4KhChr3kUF7YVoxKf0Pzw4dWZjiTCYFIx6SRJOH4KZ3wnsxtyhlMPHsoJpLHnfTKxcUTarTA.jpeg?fit=1200%2C1200&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2025\/12\/AQNa2jaLunUspz4plPCTCV_dbvsjyUiiJT-143Bp5CvER8FrjkJ7ZDFzpGguGKgQP4tbsh45bVbmI-kAXeSkBI4KhChr3kUF7YVoxKf0Pzw4dWZjiTCYFIx6SRJOH4KZ3wnsxtyhlMPHsoJpLHnfTKxcUTarTA.jpeg?fit=1200%2C1200&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2025\/12\/AQNa2jaLunUspz4plPCTCV_dbvsjyUiiJT-143Bp5CvER8FrjkJ7ZDFzpGguGKgQP4tbsh45bVbmI-kAXeSkBI4KhChr3kUF7YVoxKf0Pzw4dWZjiTCYFIx6SRJOH4KZ3wnsxtyhlMPHsoJpLHnfTKxcUTarTA.jpeg?fit=1200%2C1200&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/climatescience.press\/wp-content\/uploads\/2025\/12\/AQNa2jaLunUspz4plPCTCV_dbvsjyUiiJT-143Bp5CvER8FrjkJ7ZDFzpGguGKgQP4tbsh45bVbmI-kAXeSkBI4KhChr3kUF7YVoxKf0Pzw4dWZjiTCYFIx6SRJOH4KZ3wnsxtyhlMPHsoJpLHnfTKxcUTarTA.jpeg?fit=1200%2C1200&ssl=1&resize=1050%2C600 3x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/climatescience.press\/index.php?rest_route=\/wp\/v2\/posts\/363473","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/climatescience.press\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/climatescience.press\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/climatescience.press\/index.php?rest_route=\/wp\/v2\/users\/121246920"}],"replies":[{"embeddable":true,"href":"https:\/\/climatescience.press\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=363473"}],"version-history":[{"count":4,"href":"https:\/\/climatescience.press\/index.php?rest_route=\/wp\/v2\/posts\/363473\/revisions"}],"predecessor-version":[{"id":363478,"href":"https:\/\/climatescience.press\/index.php?rest_route=\/wp\/v2\/posts\/363473\/revisions\/363478"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/climatescience.press\/index.php?rest_route=\/wp\/v2\/media\/363474"}],"wp:attachment":[{"href":"https:\/\/climatescience.press\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=363473"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/climatescience.press\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=363473"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/climatescience.press\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=363473"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}