Topic:
Sudden Jump Algorithm in Operations
Date/Time(UTC) Message Issued:
6 May 2009, 1130 UTC
Satellite(s) Involved:
All NOAA POES and MetOp-2/A
Instrument(s) Involved:
AMSU-A
Product(s) Involved/Affected:
All Products/calibration coefficients and flags
Date/Time(UTC) of Initial Implementation:
5 May 2009, Time 1758 UTC
Details/Specifics of Change:
On 5 May 2009 around 1758Z., NOAA implemented the Sudden Jump Algorithm for all AMSU-A level-1B processing. The first orbits processed after the change were:
NSS.HRPT.NK.D09125.S1747.E1800.B5707373.CF
NSS.GHRR.NK.D09125.S1903.E2023.B5707475.WI
NSS.GHRR.NL.D09125.S1427.E1600.B4442930.GC
NSS.HRPT.NL.D09125.S1811.E1822.B4443131.CF
NSS.HRPT.NM.D09125.S1859.E1912.B3567272.GC
NSS.GHRR.NM.D09125.S1726.E1858.B3567172.GC
NSS.HRPT.NN.D09125.S1904.E1917.B2039797.WI
NSS.GHRR.NN.D09125.S1716.E1903.B2039597.WI
NSS.LHRR.NN.D09125.S1848.E1900.B2039697.WI
NSS.FRAC.M2.D09125.S1624.E1805.B1319899.SV
*********************** ORIGINAL MESSAGE ANNOUNCING THE UPDATE AND USER TESTING ***************************
We are testing an algorithm to improve the calibration impacted by noise or sudden jumps in in the space view data used to calculate calibration coefficients. This update has been running since 3 February on our parallel test system and is ready for implementation. There are some calibration quality bits in the level 1B data that will be set more often or set to a non-zero value for the first time and so we believe that users need the opportunity to use this data off-line to compare with their operational products. Our goal is to implement the changes on 5 May 2009. Any comments or concerns should be provided to the level 1B operations contact below. Test data is located on the anonymous ftp server and can be found using the following:
ftp to rigel.nesdis.noaa.gov
download data from pub/ppp/sudden_jump
We will make every effort to replace the data with the day's 24 hours of data each day no earlier than 1200Z (on weekends it may not get updated).
We have noted three changes while testing this update (two more noticeable than the third):
1) There will be more instances of one of the scan's calibration flags being set. The particular flag is the one that indicates the scan was calibrated using fewer than the preferred number of scan lines. In the 1b, this is bit 6 of the "Scan Line Quality Flags [Calibration Problem Code] (byte 31) which states the following: "bit 6: scan line was calibrated using fewer than the preferred number of scan lines because of proximity to start or end of data set or to a data gap". Before this change, there would be fewer than the preferred number of scans only if there were missing/unavailable scans around the one being calibrated. With this change in place, there could also be fewer because the calibration counts on nearby scans were rejected for being much larger/smaller than the counts on the scan being calibrated (i.e., they were too noisy).
2) In addition, if either of the scans immediately adjacent to the one being calibrated contains the much larger/smaller--and therefore rejected--counts, then the "sudden jump" flag is set for the channel in which this was observed. In the 1b, this is bit 8 of each channel's 2-byte word in the "Calibration Quality Flags" (bytes 33-64); i.e. "bit 8: this scan line is either the last one before or the first one after a sudden, anomalous jump (or drop) in calibration counts".
3) Since noisy data is being eliminated from the calibration, the resulting coefficients will be slightly different (and presumably a little better).
These changes affect each satellite differently. That is, the noisy channels are different for each satellite. For instance with NOAA-18, channel 4 will primarily trigger these flags to be set. For Metop, it is channel 7. For NOAA-17, very little noisy data has been seen, so typically nothing different should be observed in the data. Something similar occurs with NOAA-15, although some noisy channel 13 data has been observed. For NOAA-16, it is primarily channel 8, with a few other channels being flagged occasionally. At least 3 occurrences of the Sudden Jump phenomenon have occurred in NOAA-19 channel 7 data since launch,. The data is noisy and infrequently demonstrates the changes.
Following is an illustration using information from an actual NOAA-18 pass: This test data is on rigel.nesdis.noaa.gov at the location given above.
The following table shows the cold calibration readings for channel 4 from scans 314 to 320 of the orbit NSS.AMAX.NN.D09028.S0923.E1109.B1902324.GC. In this particular case, scan 317 (shaded) would be calibrated using a weighted average of these readings.
Channel-4 NSS.AMAX.NN.D09028.S0923.E1109.B1902324.GC
Rec. #
Scan #
Reading 1
Reading 2
Average
315
314
12681
12683
12682.0
316
315
12687
12683
12685.0
317
316
12685
12682
12683.5
318
317
12686
12690
12688.0
Data byte 31, bit 6 set to 1; sudden jump byte 39, bit 8 set to 1
319
318
12707
12708
12707.5
Not used; out of 3-sigma range; sudden jump bit set to 1
320
319
12701
12702
12701.5
321
320
12687
12691
12689.0
Scan being calibrated is highlighted – 3-sigma range is 12670 to 12706
As is shown, the average of the two cold calibration readings for scan 317 is 12688.0. Thus, the average of the other 6 scans' cold calibration readings must fall within a 3? range around this value. The 3? value is assigned based on input from the instrument scientist. For channel 4, this value is 18. Thus, the lower limit of the 3? range is 12670.0 (12688.0 - 18); the upper limit is 12706.0 (12688.0 + 18). The average of the readings for scans 314-317, 319, and 320 all fall within this range. However, the average of the readings for scan 318 is outside this range. Therefore, it is not used in the weighted average of the cold calibration readings for these 7 scans. As such, bit 6 of the 1b field "Scan Line Quality Flags [Calibration Problem Code]" (byte 31) is set to 1 on scan 317. (This bit indicates that the scan line was calibrated using fewer than the preferred number of scan lines.) In addition, since the scan with the out-of-range readings (scan 318) is immediately adjacent to the scan being calibrated (scan 317), the sudden jump flag for this channel is also set to 1. (This is bit 8 of byte 39 within the "Calibration Quality Flags" field (bytes 33-64).)
Contact Person(s) Name/Email/Phone Number for Questions:
Emily Harrod Emily.Harrod@noaa.gov 301-817-3882 level 1B operations
Tsan Mo Tsan.Mo@noaa.gov 301-763-8136 x 137 instrument scientist
Web site(s) that contain other relevant information (where applicable):
http://www.osdpd.noaa.gov/PSB/PPP/PPP.html - History of level 1B notices
http://www.ssd.noaa.gov/PS/SATS/messages.html - All user notices
http://www.oso.noaa.gov/poes/index.htm - SOCC polar satellite status
http://www.oso.noaa.gov/goes/index.htm - SOCC geostationary satellite status
http://www.oso.noaa.gov/daily-news/index.asp - SOCC morning reports
-
*******************************************
*
* Emily D. Harrod
* NOAA/NESDIS/OSDPD/IPD
* Computer Operations Branch
* Pre-Product Processing Group
* E/SP11, NSOF, room 1460
* 4231 Suitland Road
* Suitland, Maryland 20746-4304
* USA
* voice 301-817-3882
* fax 301-817-3902
* email Emily.Harrod@noaa.gov
*
*******************************************