Tag: forecasting models

Costly models costing dearly

The reason I don’t pick on the U.S. models is that they are so bad overall. In fact, I hardly even look at them anymore. The GEFS, which is the flagship ensemble model of the U.S. fleet, is last in skill score vs the European, Canadian, and, though not shown here, the UK met. It’s the black line.

A person standing on a snowy landscape at sunrise, surrounded by frosty trees and a misty river.

From CFACT

By Joe Bastardi

A cat wearing a purple scarf in a snowy setting, looking up with snowflakes falling around it.

The reason I don’t pick on the U.S. models is that they are so bad overall. In fact, I hardly even look at them anymore. The GEFS, which is the flagship ensemble model of the U.S. fleet, is last in skill score vs the European, Canadian, and, though not shown here, the UK met.  It’s the black line.

Day 5 – 500 mb forecast:

Line graph displaying the Anomaly Correlation Coefficient of 500 hPa Geopotential Height for the Northern Hemisphere, comparing the performance of three weather models: ECME (red), CMCE (black), and GEFS (blue).

Day 10:

Graph depicting the anomaly correlation coefficient for 500 hPa geopotential height over the Northern Hemisphere (20N-80N) from November 2025 to December 2025, showing various model performances over time.

Day 15:

Line graph displaying the anomaly correlation coefficient for 500 hPa geopotential height over the Northern Hemisphere from November 2, 2025, to December 4, 2025, with data shown for different forecasting models.

This is like having one of the biggest NIL budgets in football and winding up 6-6.

What happens to the coaches of teams when that happens?

On Aug 29, we issued our winter forecast in the face of models forecasting a warm winter with a meteorological consensus that not much was going on.

Here were the bullet points of our forecast, warning — three months before — about the period we are in now!

August 29, 2025

  • The threat of a cold winter looms.
  • Plenty of blocking is expected over the Arctic.
  • A prominent Alaskan ridge should lead to plenty of cold air.
  • Another cold period from Thanksgiving to Christmas is on the table, but this time it may not break.
  • The heart of the winter looks coldest relative to average, but there are questions as to how it starts and ends.

Given the wet front of the summer in Texas that I missed, and the hurricane season red zone being further east, the merit of the winter forecast and the front cold is a plus.  (Again, I am well aware of my mistakes and will acknowledge them. But our methodology gave anyone who wanted to listen a leg up on what is going on now.)

How can I tell there was no urgency about the upcoming winter? Look at the price of natural gas futures on Aug 29 and look what has happened.

Line graph showing the rise in natural gas futures prices from August to November 2025, with a blue line indicating the price level.

This is one of the biggest early winter moves on record — a doubling of the price. It can only occur if there is economic turmoil or adverse weather conditions. The weather has stepped in. For this to happen, it means the market consensus, which is not all meteorologists but a mix of meteorologists, traders, and people who obviously believe they know better, coming up with ideas that every time you see it go up, agree with our ideas from three months before, with counter-punching leading to downturns. The question for the rest of the winter is does this flip to warmer? It’s fascinating, as we have caught the three early-season cold shots now (2022, last year). But the methods I use only work in early winter. Looking at the method and trying to use it in mid-winter, we find an even split between warm and cold. Years when the cold collapsed (1989-90,2005-06). Years when it rolled on into January (1983-84, last year)

So that is what we are focused on for clients, but what I am going to show you is a case study that shows if it’s going to be cold, the models won’t see it.

I am going to pick on the European, which is the acknowledged king of the models. Picking on the U.S. models is like Scarlett O’Hara continuing to whip her beaten mule.

Last winter was a fiasco with models.

In December, the month before January, the Euro had this for temperatures in January.

Temperature anomaly forecast map for January 2025 showing varying temperature changes across the United States, with blues indicating cooler temperatures and oranges indicating warmer temperatures.

This is a bearish pattern for energy. Big markets all warm.

What actually happened was this — almost exactly opposite.

Map showing January 2025 temperature anomalies across the United States compared to the 1991-2020 long-term average.

Five days into the period (released Jan 5), it sees the cold. But that is not a forecast, as it is issued after the start period.

Temperature anomaly map for January 2025, showing varying temperature distributions across the United States, with blue areas indicating colder temperatures and orange areas indicating warmer temperatures.

February was not as bad but still underdone.

The January issuance for February.

Forecast map showing 2m temperature anomalies across the United States for February 2025, with color-coded areas indicating varying temperature deviations.

What happened:

Map displaying NOAA/NCEI climate division temperature anomalies in degrees Fahrenheit for February 2025, compared to the 1991-2020 long-term average.

Again, colder.

February 5 issuance:

Map showing 2-meter temperature anomaly for February 2025 across North America, with contrasting warm and cool areas indicated by color gradients.

Even after the month started, not cold enough.

We are going through it again!

In November, the model had this for December:

Map showing the 2-meter temperature anomaly across the United States for December 2025, with various color gradations indicating temperature differences.

It has flipped to this:

Temperature anomaly map for December 2025, showing varying temperature ranges across North America with areas of colder and warmer temperatures indicated.

That’s more than 8 degrees below normal around Chicago. Chicago, NYC, and Boston all have shots at the top five coldest Decembers in the last 50 years. Simply thinking you can use models is folly. We had seven major physical drivers and analogs we were looking at, none of which the modeling apparently understood mattered.

But here is another aspect, the NWS probability scheme, as I call it, and the fact that most people have no idea what it really means. The Media takes it and simply says they are forecasting this or that, when a probability of an occurrence IS NOT A FORECAST. It’s akin to saying if you have six teams in a division, and you give one of them a one in four chance of winning, you are forecasting them to win. And how much are they going to win by? Imagine trying to collect money from a bookie because you gave him a chance a team would win. He won’t even lay the bet down BECAUSE YOU ARE NOT ACTUALLY SAYING SOMEONE WILL WIN.

If you are doing it in the wrong neighborhood, it could be hazardous to your health.

A forecast has to predict a measurable metric before the start of the period. These probabilities of an occurrence are not truly forecasts, but guidance. There is no such thing as a “probability forecast” as it does not tell you how much above or below it actually is. It is guidance, that is true, but .1 below normal in Chicago, 1 below normal in Chicago, or 10 below normal in Chicago cannot be scored as a probability. And it makes a huge difference. Chicago this month could have a top five coldest December over the past 50 years. They are off to a heck of a start, as is much of the nation, where heating is important.

Here is the December guidance from November 30:

Map depicting the Monthly Temperature Outlook for December 2025, issued November 30, 2025. Areas are color-coded to show probabilities of above normal, equal chances, and below normal temperatures across the United States.

It has Chicago, Boston and NYC leaning below — one day before the start of a month that might be top five coldest in the past 50 years.

Graphic showing probability percentages of temperatures being below normal, with categories labeled 'Leaning Below' and 'Likely Below'.

Yet, if it were a top five coldest, that would be scored as a hit. That’s crazy. It tells you nothing about the real weather. How low is that bar?

The question is why?

Here is what it means.

How to Read the Probability Maps Temperature Example:

  • If a location is shaded with “60% chance of above average temperature”:
    • The historical (climatological) chance of above-average temperature in any given winter is ~33% (because the three categories — above, near, and below — are equal in a third of the time with no climate signals).
    • A 60% probability means the odds have been shifted upward by about 27 percentage points due to La Niña, ENSO, trends, etc.
    • This is considered a modest to moderate lean toward warmer-than-average.
  • Common probability thresholds you’ll see:
    • 33%–40% → Very weak signal (often shown as “EC” or Equal Chances on some maps)
    • 40%–50% → Slight lean
    • 50%–60% → Moderate lean
    • 60%–70% → Strong lean
    • 70% → Very strong signal (rare in seasonal outlooks)

Fact:  The NWS seasonal outlook probabilities do not tell you how far above or below normal the temperatures (or precipitation) will be.

So, does that mean a temperature of 1 below normal would get the same skill score in a probability scheme as 10 below normal?

Grok chimes in: Yes — exactly. In the official NWS/CPC probabilistic seasonal outlook and in the way they compute verification skill scores (like the Heidke Skill Score or Ranked Probability Skill Score), a winter that ends up 1 °F below normal is treated identically to a winter that ends up 10 °F below normal, as long as both finish in the below-normal tercile (the coldest third of the 30-year historical distribution).

An actual temperature forecast (a specific anomaly like “+2.4 °F above normal” or “seasonal average of 44.1 °F”) always gives you far more information than a tercile probability map (“60 % chance of above-normal”).

So basically, you are running away from accountability since the magnitude of the above or below and the amount it takes to be above and below are trying to avoid an exact number, rather than trying to nail it

The official NWS/CPC seasonal outlook is deliberately structured to minimize accountability for magnitude and exact outcomes. By using only tercile probabilities and refusing to put a specific °F or inch number on the public map, the agency protects itself in two big ways:

  1. No magnitude accountability
    +5 °F above normal or +0.5 °F above normal → both count as an identical “hit” if the map said “above-normal.”
    The forecast can be technically correct while being practically useless (or wildly off in intensity).
  2. Very low bar for “success”
    Because the baseline (climatology) is 33 %, any forecast between 34 % and 100 % will be right more often than random guessing.
    A map full of 40–50 % probabilities is almost impossible to beat with a coin flip, so the Heidke skill score is almost always slightly positive, even in years when the winter is a total bust in magnitude

By the way, I am not Monday morning forecasting. We have been very loud about this cold, and you saw me blog on it a couple of weeks ago — in case you were not aware of what was coming.

So, what is the conclusion as heating bills skyrocket?

1. The U.S. models for all the money spent come up woefully short.

2. Even the emperor (European) had no clothes last winter and to start this winter. As cold as it is, not a very good idea.

3. What the NWS puts out is NOT A FORECAST. It’s guidance with a very low bar for verification.

Pretty costly stuff.

Did AccuWeather Write the Project 2025 NOAA Section?

Back in 2005, AccuWeather, possibly along with other commercial weather companies, convinced former senator Rick Santorum to introduce a bill, S.786, 109th Congress, to prevent NOAA and the NWS from sharing data that was available from commercial vendors. The only support came from the weather companies.

From Watts Up With That?

By Julius Sanks

Back in 2005, AccuWeather, possibly along with other commercial weather companies, convinced former senator Rick Santorum to introduce a bill, S.786, 109th Congress, to prevent NOAA and the NWS from sharing data that was available from commercial vendors. The only support came from the weather companies. Most thought it would mean paying for weather data the taxpayers already funded — if they could even buy it from the vendors. The bill did not get a single co-sponsor; nor did a companion bill appear in the House. It died in committee.

Is AccuWeather using Project 2025 to try again?

The Heritage Foundation recently published Project 2025. It is a massive document that addresses, among other things, the administrative state. It steps through the entire federal government, with specific recommendations regarding each organization. One of the organizations it discusses is the National Oceanographic and Atmospheric Administration (NOAA). NOAA’s National Weather Service (NWS) is responsible for weather forecasting. NOAA’s National Environmental Satellite, Data, and Information Service (NESDIS) acquires and operates environmental satellites, among other things. Full disclosure: I spent about a third of my engineering and management career working as a contractor for NOAA. I worked weather forecasting and weather satellite development projects. This analysis focuses only on the National Weather Service and the satellite portion of NOAA’s National Environmental Satellite, Data, and Information Service (NESDIS).

Project 2025 proposes sweeping changes to NOAA. To a large degree, it recommends replacing NOAA’s operations with private-sector commercial efforts. It claims this will result in lower cost and higher quality. But will it? Project 2025 provides virtually no evidence of that. Nor does Project 2025 give a complete view of how the NWS works.

A key flaw in Project 2025 is its emphasis on existing commercial forecasting. It says the NWS provides data to private companies and complains the NWS does not use “commercial partnerships as some other agencies do.” This is supposed to improve weather technology and cost-effectiveness by investing in “different sizes of commercial partners.” The document provides no evidence this is true. Nor does it show how this would improve forecasting accuracy.

Project 2025 greatly oversimplifies what the NWS does, how the NWS operates, and who its customers are. For example, it fails to mention the largest beneficiary of NWS forecasting: the public. A key part of local forecasting Project 2025 misses is watches and warnings. Local offices and national centers issue them. For obvious liability reasons, the commercial companies simply parrot NWS watches and warnings. Would a company such as AccuWeather accept that liability? It seems unlikely. Local offices also interact directly with local government officials, such as county disaster managers, when the situation warrants. Will a commercial company do that?

The NWS is mostly a local organization. Its “front line” is the 122 weather forecasting offices (WFO) and 13 river forecast centers (RFC). Operating 24/7/365, the WFOs are scattered throughout the 50 states, Guam, American Samoa, and Puerto Rico. RFCs, chartered with flood forecasting, normally operate 16 hours a day, 7 days per week, but will extend to 24-hour operation when necessary. Do any commercial companies forecast floods? Would they accept the liability? The WFOs and RFCs have specific areas of responsibility around the country. Their meteorologists and hydrologists are familiar with the local conditions and use that knowledge to supplement information from space and terrestrial sensors, and models, as they build their daily forecasts. About 90% of all weather data comes from satellites; NWS’s supercomputer models depend on satellite data.

This map, found on the NWS home page, frequently updates as forecasters issue and cancel watches and warnings.

In all, the NWS has over 4,000 employees, over 2,000 of which are meteorologists. AccuWeather has 4 – 500, with about 100 meteorologists. It is difficult to believe they can match the NWS’s local knowledge.

At the national level, NWS’s National Centers for Environmental Prediction (NCEP) has nine national centers (NC). Each is responsible for a particular type of forecasting. The centers often trade information to improve forecasting.

•            NCEP Central Operations — The NWS information technology center

•            Aviation Weather Center (AWC) — Aviation forecasts.

•            Climate Prediction Center (CPC) — Issues probabilistic climate forecasts from a week to three months in the future; up to a year for some conditions.

•            Environmental Modeling Center (EMC) — Develop and operate the NWS supercomputer numerical prediction models used for daily forecasts.

•            National Hurricane Center (NHC) — Hurricane forecasts.

•            Ocean Prediction Center OPC) — Provides marine weather and warning forecasts for the North Atlantic and North Pacific oceans.

•            Storm Prediction Center (SPC) — Works with tornadoes, severe thunderstorms, lightning, wildfires, and winter weather.

•            Space Weather Prediction Center(SWPC) — Observes solar activity.

•            Weather Prediction Center (WPC) — Generates nationwide forecasts, such as winter storms and precipitation

After very severe weather events, the NWS puts teams on the ground to assess damage. That is how they determine tornado strength. Sometimes, following a severe event such as a disastrous hurricane or major tornado outbreak, NWS performs service assessments to assess how well they performed. These are available to the public. Are the commercial companies willing to do that?

Another problem with Project 2025’s argument is how the NWS interacts with other government organizations, particularly the military. The NWS shares data with Air Force Global Weather Central, Naval Meteorology and Oceanography Command, and other organizations. It is hard to see how a commercial entity in this interaction would contribute value.

Finally, the NWS interacts with other countries’ forecasting activities.

Weirdly, Project 2025 combines the National Hurricane Center and National Environmental Satellite, Data, and Information Service (NESDIS), complaining that data should be presented neutrally, while ignoring all other national centers. The NHC focuses on forecasting individual hurricanes. Its web site provides statistical information regarding what the USA can expect during hurricane season. Each year, it publishes provides a hurricane season outlook and a mid-season update. While this addresses climatic conditions, the discussion is strictly in terms of how climatic conditions affect hurricane development. It does not address the political climate debate. The CPC, with NHC collaboration, issues NOAA’s seasonal hurricane outlooks. These outlooks use climatic conditions, such as ocean temperature, to establish probable hurricane numbers and strengths. It is important to note as in all NOAA’s climatic forecasts, the hurricane outlook is strictly probabilistic. I.e. the 2024 Atlantic forecast predicted an 85% chance of above normal activity, with 17-25 named storms. NHC’s final 2024 monthly summary lists 18 named storms so far and says the year’s activity was “above normal.”

NESDIS is not part of the NWS. It is a separate NOAA branch. It is puzzling why Project 2025 thinks of them as a single organization. NESDIS acquires and operates weather satellites and serves as NOAA’s data depository. On the satellite side, it figures out how to turn weather satellite data into information useful to the public. It also works with EMC to use satellite data to properly initialize EMC’s forecasting models. Like the NWS forecasters, NESDIS satellite offices work hard to provide the most accurate information they can achieve for public consumption. NESDIS’s National Centers for Environmental Information (NCEI) provide the depository for all NOAA scientific data. Project 2025 is probably referring to NCEI. NCEI has been criticized for adjusting some archival data.

Project 2025 cites a 2020 AccuWeather report to claim studies “have found that the forecasts and warnings provided by the private companies are more reliable than those provided by the NWS.” This is misleading. The study does not even mention the NWS. It compares AccuWeather to five other commercial companies. The claim commercial companies are better than the NWS is therefore, completely unfounded.

I have seen several claims online that in 2019, the American Meteorological Society (AMS) published a study stating for every dollar spent on the NWS, the country receives $73 in value. This claim is actually in the Congressional Record. I have searched exhaustively and doubt it exists. Failing to find it online, I contacted AMS to request a copy. AMS drew a blank but kindly sent me a different, comprehensive, study, published last year. It does not reach the same “value added” result, but concludes the public is quite satisfied with NWS forecasts. It uses a modified contingent valuation method to statistically estimate the public’s “willingness to pay” for NWS forecasts. The basis was a 2022 survey. The basic finding is people who use NWS forecasts are willing to pay about 900 tax dollars per year, with lower bound of about $700/year and upper bound of $1300/year. Multiplying these by the number of people who use NWS forecasts — about 114 million — the result is a total annual value ranging $795 million to $1.48 billion. The NWS FY22 budget request was $1.33 billion. It looks like the public is getting the value it expects from the National Weather Service. How many other federal organizations can say that?

Project 2025’s argument to restructure the NWS is not convincing. If DOGE used Project 2025 as its guidance for analyzing NOAA, it needs to rethink its approach.

Andrew Dessler: The future will be hotter, unless it’s colder.

Andrew Dessler @AndrewDessler Prof of Atmospheric Sciences & climate scientist @ Texas A&M; AGU and AAAS Fellow has been prognosticating on Twitter, talking up a model that predicts another 2C of warming by 2100 (0.25C/decade, twice the rate 2013-2023).

From Tallbloke’s Talkshop

June 23, 2023 by tallbloke

Andrew Dessler @AndrewDessler Prof of Atmospheric Sciences & climate scientist @ Texas A&M; AGU and AAAS Fellow has been prognosticating on Twitter, talking up a model that predicts another 2C of warming by 2100 (0.25C/decade, twice the rate 2013-2023).

When challenged about this alarming rate of warming, he clarified for us.

Of course, if the central estimate is warming of 0.25C/decade, but cooling is instead a possibility, this means that the upper bound for warming is at least 0.5C/decade. I think this tells us all we need to know about the usefulness of climate models (and the sanity of the modellers).

I asked a question, no reply yet.

Think We Can Model the Climate? Clouds Get in the Way!

IPCC forecasters overstate warming because they still somehow really don’t understand clouds at all.

From Watts Up With That?

By Ron Barmby

I’ve looked at climate change from both sides now, and I have found common ground between proponents and skeptics of the belief that climate change is largely caused by humans. When it comes to forecasting global temperatures, distinguished experts in both camps agree a dominant variable cannot be simulated in computer models because clouds get in the way.

Among the proponents is Dr. Bjorn Stevens, a contributing author to the Intergovernmental Panel on Climate Change (IPCC) Assessment Report 5 (2014). Dr. Stevens is also director at the Max Planck Institute for Meteorology, Hamburg, Germany, and a cloud expert. In a recent interview he acknowledged the contribution of clouds to global warming is overestimated in the IPCC’s “Climate Change 2021: The Physical Science Basis.”

“Clouds are tricksters,” he said, referring to their complexity. However, he said, many scientists use oversimplified representations of clouds in modeling “as a guide because they are easier to simulate. This makes the climate models less accurate.”

On the skeptic side is Dr. Richard S. Lindzen, a former lead author for IPCC Assessment Report 3 and now a vocal critic of the IPCC. In a recent podcast the interviewer noted that Lindzen had published sufficient research papers to earn 80 PhDs. (Lindzen humbly declined the praise.)

Lindzen, professor emeritus of Atmospheric Sciences at the Massachusetts Institute of Technology,  points out that IPCC models rely on the assumption that water vapor and clouds amplify the greenhouse gas effects of CO2 in order to achieve forecasts of catastrophic global warming. The IPCC theory is that a warmer atmosphere will have a higher content of water vapor – itself is a greenhouse gas – that adds to the warming caused by CO2. Without this amplifying “positive feedback” effect, the models are still wrong for many reasons, but they no longer project “catastrophic” warming.

Dr. Stevens, who is on record stating that global warming is a “huge problem,” agrees that increased clouds do not amplify global warming: “Water-rich low clouds over the tropical ocean have the greatest cooling effect and low-water ice clouds at high altitudes have the strongest warming effect. Overall, the cooling effect is greater.” That’s called “negative feedback.”

Dr. Lindzen argues that global average temperature is controlled by the polar regions. The temperature at the tropics remains relatively constant over long periods of time while the polar regions have significant variations. In other words, a small change in global average temperature is the result of a big change in polar temperatures. The stability of tropical temperatures indicates that increased clouds provide negative feedback in times of global warming. Rather than exacerbating the global warming effect of CO2, clouds reduce it.

Lindzen proposes the mechanism by which greater negative feedback is produced: High altitude cirrus clouds (Dr. Stevens’ low-water high altitude ice clouds that cause warming) control heat emissions to space. As the air below these clouds warms, the cirrus clouds dissipate and allow more energy to radiate into space. He calls this the Iris Effect.

Nature is dominated by negative feedbacks, not positive ones. French chemist Henry Louis Le Chatelier expresses this in Le Chatelier’s principle: A system in dynamic equilibrium (in this case the climate) is disturbed by changing the conditions (in this case the concentration of CO2), then the equilibrium moves to counteract the change. Negative feedbacks occur because when a variable changes, other variables react in ways that offset the shift.

We can’t model clouds. The IPCC admits this. Nonetheless, its forecasting models rely on assumptions that clouds amplify CO2 warming, without which its temperature forecasts are benign (that’s strike one).

IPCC forecasts based on cloud models have been called out by eminent cloud experts on both sides—proponents and skeptics of the notion of human-caused climate change. A recognized cloud expert and IPCC author (Dr. Stevens) states that clouds cause cooling overall, not warming (strike two).

And, finally, the observed stability of tropical temperatures argues that increased clouds provide even greater negative feedback to CO2, and a cloud expert who has done enough research for 80 Ph.D.s (Dr. Lindzen) has an idea how that happens (strike three).

Clouds do not amplify global warming; they promote global cooling. That’s enough to convince this climate heretic that IPCC forecasts are no “red light” for humanity and to reject the insane drive to eliminate CO2 from our economies.

IPCC forecasters overstate warming because they still somehow really don’t understand clouds at all.

This commentary was first published at Real Clear Markets, April 7, 2023, and can be accessed here.

Ron Barmby, Professional Engineer with a master’s degree majoring in geosciences, had a 40-year career in the energy industry that covered 40 countries and five continents. He is author of “Sunlight on Climate Change: A Heretic’s Guide to Global Climate Hysteria” and is a proud member of the CO2 Coalition, Arlington, Virginia.

Tags: clouds, Richard Lindzen, IPCC, Ron Barmby, Intergovernmental Panel on Climate Change, Bjorn Stevens, Iris Effect, Ronald Barmby