Date: Mon, 08 Sep 2003 11:37:26 -0400
From: "Geoff DiMego" <Geoff.Dimego@noaa.gov>
Organization: DOC/NOAA/NWS/NCEP/EMC

Subject: Notice of Crisis Change to 3DVAR Analysis in Eta/EDAS

Brief Description of Changes:

Surface-land observations of temperature and all off-time surface data are being turned off in the 3DVAR analysis used in the Eta and its Eta Data Assimilation System (EDAS).

Reasons for Changes:

For some time, we have been searching for the reason Eta forecasts initialized off the GFS analysis are superior to the operational ones initialized with our 3DVAR analysis. I sent a note to you all titled "Eta performance" on 8 November 2002 in which I stated we were running a 32 km parallel (Etay) using GFS initial conditions. Stats from those runs were better than the control run using the Eta 3DVAR. There are a rather small number of differences between the Eta and GFS analysis systems since they are both based on the same 3DVAR concepts. These differences include horizontal domain & resolution (global ~55km vs No. America ~12km), vertical domain (model top 0.2 mb vs 25 mb), balance constraints (strong vs weak), use of digital filter (yes vs no), amount of data (more from late data dump vs less from early data dump) and use of surface data (only surface pressure data over land vs all surface data over land). We've spent a ton of effort on the balance issue but to make a long story short, we have very strong evidence that the GFS vs Eta differences are due to the use of surface data and specifically the use of surface temperature data.

The North American Regional Reanalysis (NARR) was instrumental in our discovery. The NARR (a 32 km Eta/EDAS system using a 3DVAR identical to the one used in operations) was being ported from its development platform to our old IBM to make its production runs of 25 year period 1979-2003. During the port it was discovered that surface observations of temperature, moisture and wind had been turned off due to our use of the global reanalysis data files which had these data flagged for non-use by the global reanalysis. On the IBM, this was corrected but the performance of the system became degraded compared to the development system. Numerous runs were made to isolate the source of the degradation. First, all surface data whose obtimes were more than 30 minutes different than the analysis time were excluded. This had a small positive effect (mostly reduced noise). Second, runs were made with only surface temperature, winds or moisture turned on. There was one component that accounted for all of the degradation and it was thesurface temperatures. The first 2 attached gifs show the fit of the assimilation first-guess to 00z and 12z raobs at different levels in the vertical for a system of EDAS runs made 3 hourly for the months of January and July 1993. The runs with surface temperature data show the degradation I spoke of. While the impact is greatest at 850 mb and 700mb (1000 mb level has very few raobs relative to upper levels), it is very important to note that this degradation exists at all levels in the troposphere. Surface wind and moisture obs were found to have no negative impact and all of that data within 30 minutes of analysis time are still used in the NARR.

Upon further review, it is Dave Parrish's belief that the 3DVAR is handicapped by being cast in the step-mountain framework. Because this is NOT terrain following, it is impossible to cleanly limit the vertical influence of surface data as is done in the current GFS
analysis, the RUC 3DVAR and was done in the previous Eta Optimum Interpolation analysis. Anisotropic covariances with vertical stability dependence cast in a terrain following coordinate (at least near the surface) has been our longterm goal, but that is too far away to wait for. Hence, we have decided to turn these data off for the time being, until ways can be found to limit their influence.

Schedule for change:

Final testing : Ongoing since 14 August 2003
Expected implementation : 9-11 September 2003

Description of testing:

Extensive 30-day tests were conducted with the 32 km NARR system. A 12 km parallel EDAS system is running in real-time parallel and we are making periodic 12 km Eta forecasts from 00z from this test cycle. These parallel runs (ETAX) can be viewed at Eric Roger's webpage: http://www.emc.ncep.noaa.gov/mmb/mmbpll/etapll

Results of testing + verifications:

The attached gif file (edas12test.jpg) shows the RMS fit of the 12 km EDAS first guess (ops and parallel) vs. raob temperatures from 14-25 August 2003. The results are very similar to those seen in the 32 km NARR, namely, withholding surface temperatures from the 3DVAR leads to improved fit of the guess to raob temperatures in the lower troposphere.

Statistical verifications of the parallel ETAX forecasts have been made for QPF and at upper-air and surface observations as well. Check out http://wwwt.emc.ncep.noaa.gov/mmb/mmbpll/pll12stats.etx_3dv The impact on QPF scores is universally positive with impact greatest in the eastern CONUS and at 60 hour range. While there is slight negative impact on forecast RH, the RAOB wind, temperature and height fits are neutral to positive and become more positive with longer forecast range. Fits to surface data show surprisingly little impact - even for temperature.

Anticipated impact of forecasts:

The impact of this change on a 60-h Eta forecast is shown in the attached 4-panel chart in "etavsetax.2003082412.gif". The top panels show 60-h forecasts of 500 mb height valid at 12Z 8/24/03 from the ops Eta (left) and the parallel Eta (right) with the changes described above. The parallel-ops 500 mb height difference is shown in the bottom left panel. The feature of interest is the short-wave trough which is located off the coast of New England. The trough is slower and deeper in the parallel Eta (top right) when compared to the ops Eta (top left), with height differences parallel - ops 500 mb difference less than -60 m over New Hampshire and Vermont. The verifying EDAS analysis in the bottom right panel shows that the parallel Eta run had a better prediction of the position and depth of the trough.

A recent case has been pointed out by Phil Schumacher, SOO at Sioux Falls, with too cold initialization of temperatures at 850 mb. I have attached 3 gif files prepared by Eric Rogers. #1 opsedas.850.gif is the 850 temp analysis over the Dakotas from the operational EDAS valid at 12Z 8/30; #2 plledas.850.gif is the same as #1, but for the parallel 12-km EDAS run with the surface temperature data over land excluded from the 3DVAR, and #3 diff.850.gif is the difference = parallel EDAS - ops EDAS. Comparing #1 with #2, note the eastward shift of the 8C isotherm, so that in the parallel EDAS analysis almost all of North Dakota is between 8C and 12Z (ob at Bismarck at 12Z was 9C), while in the ops EDAS all of North Dakota was < 8C at 850 mb. This case shows that excluding surface land temperatures from the 3DVAR analysis will help mitigate part of this problem.

Based on the strength and extent of these impacts, we feel the inability of 3DVAR to properly use surface temperatures over land is a major reason (if not THE "smoking gun") for Eta 3DVAR being inferior to the GFS analysis. All other attempts to find comparability with the GFS have been much less successful than this. Therefore, we expect these changes to result in a) more accurate data assimilation cycle leading to b) more accurate forecasts and c) more consistent performance from run to run.

Field evaluation:

Datasets and webpages from the parallel runs were available for download and review, respectively, but time has been limited.

Points of Contact:

David.Parrish@noaa.gov 301-763-8000 ext 7742
Eric.Rogers@noaa.gov 301-763-8000 ext 7227
Geoff.DiMego@noaa.gov 301-763-8000 ext 7221

Future changes:

New procedures for use of surface temperatures in the 3DVAR are being considered for inclusion in our late Fall Bundle. This bundle of changes will include: a) new shortwave radiation scheme which will eliminate our excess incoming shortwave and high bias in 2m temperatures under clear skies, b) mods to BMJ deep and shallow convection will sharpen the structure of the QPF, c) add GOES 12 radiances and d) modulate analysed precip to compensate for its low bias in precip assimilation and thus to produce unbiased soil moisture. Implementation of these and other changes are planned for the late fall of 2003.