Filtering rules for ingested match-up datasets

Me, Gary Corlett, Owen Embury, Mark Filipiak by phone

We reviewed samples of match-up instances between satellite observations and drifting buoys generated by Gary. Prior to imagery ingestion, some match-up dataset instances can be filtered out because they are essentially duplicates or overly-numerous for our purposes. The aim of the discussion was to agree the rules for this filtering.

For brevity, we used the codes:
A = from the Leicester match-up dataset for any of the ATSR series
M = from the Meteo France MD for Metop
S = from the Meteo France MD for Seviri
G = from the Pathfinder MDs for AVHRR GAC (up to 2003)

We considered an example day carefully (Figure).

Considering these distributions, the agreed filtering rules were:

Within each UTC day:

1. Keep all A+M+S, with A as the primary

2. Keep A+M only if there are no similar A+M+S, with A as the primary

3. Keep A+S only if there are no similar A+M+S, with A as the primary

4. Keep M+S only if there are no similar A+M+S, with M as the primary

5. Keep A (singles) only if no similar A+M+S, A+M or A+S

6. Keep M (singles) only if no similar A+M+S, A+M or M+S

7. Keep no S singles (unnecessary for adequate S coverage)

8. Keep G unless similar to an A (since GAC imagery will be found for all instances in a later step)

"Similar" means a match to the same in situ callsign within 3 hours.

Further points were concluded in conversations with colleagues at Brockmann:
1. For all MM types (A+M+S, etc), only one will be kept in a single day, that being the one with the smallest time difference between primary and in situ.
2. At this stage, restrict in situ to drifting buoys to progress RRDP rapidly
3. We will wait for AMSRE-v7 data and therefore no passive microwave in RRDP distribution

Extraction of data for Round Robin exercise

Me, Gary Corlett (UoL) , Owen Embury & Claire Bulgin (part of discussion)

The result of the consultation on the likely external participation in the Round Robin exercise shows that the only interest is in retrieval algorithms. So, we had a discussion to determine what the optimum data extraction from the Multi-sensor Matchup System should be to support this. Large image extracts are mainly useful for classification and area-comparison work, and most researchers are used to working with small image extracts for algorithm development work. Because large image extracts also have a large overhead in terms of data volume and cost, the best result (in terms of ensuring those wishing to participate can in practice do so) will come from following a similar approach.

Since the product resolution for the long-term climate data record we plan will be 0.05 degrees latitude-longitude, the extracts need to be at least this scale. Allowing for extra leeway to permit local standard deviation calculations (for cloud detection) and pixel shifting for co-location errors, a sensible extraction for ATSR imagery is 11 x 11 pixels centred on the nominal buoy location. For AVHRR, GAC 3 x 5 pixels should cover that area. NWP fields will be extracted to the match centre.

The WPs internal to the project on image classification and on AMSRE uncertainty characterisation will still require large image extracts, with a number of NWP-profiles at selected points across the image area. These will be created by a separate extraction from the MMS.