Jacob Hoyer (DMI), me, Owen Embury and Claire Bulgin (UoE)
Jacob H came to Edinburgh in lieu of attending our third project meeting. We reviewed the work to be done on infra-red SST retrieval and how the high latitudes and marginal ice zones fitted into the wider project framework. It looks like the CASSTA algorithm may be quite specific to the region to which it was tuned, but this will nonetheless be tested in the MMD.
We agreed various points of follow up:
1. For ice/clear/cloud classification in a Bayesian framework, CB needs to use a value for ice surface temperature from ECMWF, and Jacob will review the 2 m air temperature and the skin ST fields and advise on the better to use.
2. Again for classification and for MIZ ST retrieval, the freezing point of the sea water is important to know. CB will define a monthly map of seawater freezing point based on climatological salinity. JH will obtain model surface salinity fields for the Arctic for recent years including areas the were previously perenially frozen, and send to CB for comparison.
3. JH will verify that high latitude M-AERI radiometric observations and drifting buoys in MIZ areas will be included in the MMD.
Thursday 28 April 2011
Tuesday 26 April 2011
Interest in "Round Robin" exercise
The responses are in from the recent post on algorithm selection (accompanied by email circulation to GHRSST etc). It is good to have prompted additional interest in the exercise.
On SST retrieval algorithms, we previously had interest expressed by four groups covering traditional coefficient based algorithms and optimal estimation. As a result of the solicitation of interest, have been approached by a further four groups or individuals, whose interests span additionally neural net approaches. On classification, no external groups have come forward to participate formally in a Round Robin framework. Nonetheless, we have started discussions with a US group expert in AVHRR on like-for-like comparisons of cloud detection, and have submitted our cloud detection results to a comparison within the Cloud-CCI. On time-depth adjustment, again no formal takers, but one conversation has opened up which may lead to useful co-operation.
Therefore, the Round Robin exercise will focus on SST retrieval. The ice-cloud-clear data set referred to in the previous post will also be available should any interested parties emerge later.
On SST retrieval algorithms, we previously had interest expressed by four groups covering traditional coefficient based algorithms and optimal estimation. As a result of the solicitation of interest, have been approached by a further four groups or individuals, whose interests span additionally neural net approaches. On classification, no external groups have come forward to participate formally in a Round Robin framework. Nonetheless, we have started discussions with a US group expert in AVHRR on like-for-like comparisons of cloud detection, and have submitted our cloud detection results to a comparison within the Cloud-CCI. On time-depth adjustment, again no formal takers, but one conversation has opened up which may lead to useful co-operation.
Therefore, the Round Robin exercise will focus on SST retrieval. The ice-cloud-clear data set referred to in the previous post will also be available should any interested parties emerge later.
Thursday 21 April 2011
Progress on Multi-sensor Match-up System
Me, Owen Embury, Mark Filipiak (UoE), Gary Gorlett (UoL), Norman Fomferra, Ralf Quast and Thomas Storm (BC)
The hard work to create the Multi-sensor Match-up System continues. It is key to the project, we need to get it right, and it is significantly more effortful than expected. We are getting close, which is necessary for the time lines of the project to stay on track.
RQ reported several activities on the MMS: problems in reading microwave data are resolved; space for +/-12 h of in situ observations at 0.5 h resolution has been created as agreed at PM3; progress on convergence of data names, types, units, etc, between data coming from different sources, which can be completed on GC provides the final list of all such issues.
TS reported that process is integrated and tested for: AVHRR geolocation information, matching of pixel to in situ location and then extracting the desired fields from AVHHR into the MMS. Processing is too slow on the Eddie cluster virtual machines, however. A possible cause has been identified, and a test will assess whether it solves the issue. It needs to be >5 times faster than at present. Lastly, the BEAM landmask has been implemented. The underlying data have 30 m resolution and in due course the module will be adapted to account for the nominal size of the satellite pixel at the satellite zenith angle in determining whether the pixel overlaps land. This should be a distinct improvement over current approaches based on the pixel centre and coarser underlying data.
GC has got almost all data to Eddie. ECMWF surface data files prior to 2008 were corrupted between ECMWF and the BADC archive, and a second attempt at transferring these is not yet complete; but all ECMWF profile data have been obtained now. In terms of the ATSR MD that will be ingested into the MMS, data to support a gross cloud screening and also additional ship metadata have been added. Gary has also been looking at the matchup statistics, and determined that there is no need to retain SEVIRI-only matches, and is defining a subselection for SEVIRI-Metop matches. The pre-2003 AVHRR MDs from RSMAS required for ingestion have arrived and the spatiotemporal locations can be extracted to give the match-up list for the MMS. GC agreed that the master list of all MMD fields is now critical and he will work (over the holiday weekend!) to deliver it early next week.
MF has modified the list of required AVHRR fields in the extraction to the MMS to include the CLAVR-X Bayesian cloud probability, which is a good idea (not just the yes/no mask). The CLAVR-X processing on Eddie requires about 1.5 hour per satellite-year.
We all reviewed the schedule and it seems that a start to full MMDB population is still credible for end of May. How much data will be available by the launch of the Round Robin exercise at GHRSST will partly depend on the Eddie VM performance.
CM explained that, following the earlier post on interest in the Round Robin elements, there has been interest in SST retrieval comparisons, but no takers for cloud classification work. We have submitted, however, some cloud mask fields into the Cloud-cci comparison, and are talking with RSMAS (Bob Evans) to try to find a way to do a direct CLAVR-X/Pathfinder cloud mask comparison. Carol-Anne Clayson and CM will discuss co-operation on time-depth adjustments; it is not clear yet whether this could happen within the Round Robin.
There is a huge amount going on. The team is working hard, and we are getting fantastic co-operation from scientists round the world. It is all very much appreciated.
CM
The hard work to create the Multi-sensor Match-up System continues. It is key to the project, we need to get it right, and it is significantly more effortful than expected. We are getting close, which is necessary for the time lines of the project to stay on track.
RQ reported several activities on the MMS: problems in reading microwave data are resolved; space for +/-12 h of in situ observations at 0.5 h resolution has been created as agreed at PM3; progress on convergence of data names, types, units, etc, between data coming from different sources, which can be completed on GC provides the final list of all such issues.
TS reported that process is integrated and tested for: AVHRR geolocation information, matching of pixel to in situ location and then extracting the desired fields from AVHHR into the MMS. Processing is too slow on the Eddie cluster virtual machines, however. A possible cause has been identified, and a test will assess whether it solves the issue. It needs to be >5 times faster than at present. Lastly, the BEAM landmask has been implemented. The underlying data have 30 m resolution and in due course the module will be adapted to account for the nominal size of the satellite pixel at the satellite zenith angle in determining whether the pixel overlaps land. This should be a distinct improvement over current approaches based on the pixel centre and coarser underlying data.
GC has got almost all data to Eddie. ECMWF surface data files prior to 2008 were corrupted between ECMWF and the BADC archive, and a second attempt at transferring these is not yet complete; but all ECMWF profile data have been obtained now. In terms of the ATSR MD that will be ingested into the MMS, data to support a gross cloud screening and also additional ship metadata have been added. Gary has also been looking at the matchup statistics, and determined that there is no need to retain SEVIRI-only matches, and is defining a subselection for SEVIRI-Metop matches. The pre-2003 AVHRR MDs from RSMAS required for ingestion have arrived and the spatiotemporal locations can be extracted to give the match-up list for the MMS. GC agreed that the master list of all MMD fields is now critical and he will work (over the holiday weekend!) to deliver it early next week.
MF has modified the list of required AVHRR fields in the extraction to the MMS to include the CLAVR-X Bayesian cloud probability, which is a good idea (not just the yes/no mask). The CLAVR-X processing on Eddie requires about 1.5 hour per satellite-year.
We all reviewed the schedule and it seems that a start to full MMDB population is still credible for end of May. How much data will be available by the launch of the Round Robin exercise at GHRSST will partly depend on the Eddie VM performance.
CM explained that, following the earlier post on interest in the Round Robin elements, there has been interest in SST retrieval comparisons, but no takers for cloud classification work. We have submitted, however, some cloud mask fields into the Cloud-cci comparison, and are talking with RSMAS (Bob Evans) to try to find a way to do a direct CLAVR-X/Pathfinder cloud mask comparison. Carol-Anne Clayson and CM will discuss co-operation on time-depth adjustments; it is not clear yet whether this could happen within the Round Robin.
There is a huge amount going on. The team is working hard, and we are getting fantastic co-operation from scientists round the world. It is all very much appreciated.
CM
Friday 15 April 2011
Preparing for "Round Robin"
As part of this SST CCI project, we will be running a "Round Robin" comparison of algorithms (more or less, an algorithm competition). This exercise will be open from July 2011 to end of October 2011.
Currently, we need to gauge the level of interest among potential volunteer contributors to this exercise, to contribute results for different categories of algorithms.
For those of you who decide this is relevant and you need to read on ... my apologies about the length of this post!
BACKGROUND INFO:
Broadly, to make the Algorithm Comparison and Selection objective, the exercise will proceed as follows:
(i) we distribute a data package containing satellite data, in situ data, and auxiliary information
(ii) participants apply their algorithms to these data (a training subset will be identified for any algorithm tuning that participants want to perform, along with a test subset)
(iii) we distribute a further set of data -- identical in format to the previous data package, but without any in situ observations -- this is the "blind" subset*
(iv) participants apply their final algorithms to the blind subset and submit the results (for all subsets, in a defined format) to the CCI team
(v) objective metrics are calculated for all submitted algorithms, and we compare and select algorithms for implementation in the CCI project
*the algorithm development team within the CCI project also don't have access to the blind subset prior to its distribution
Please note:
(a) The comparative performance of algorithms on the blind test subset will be the basis for selection of algorithms to be applied within the CCI project. If you submit your algorithm results, but, for whatever reason, would not want your algorithm implemented by the project, please make that clear upfront.
(b) Regional algorithms are encouraged as points of comparison, as are algorithms that work only for day or night. Since the project goal is a global, consistent SST data set for climate, algorithms of wide applicability are generally more likely to be selected.
(c) As a minimum, we are looking for application of any given algorithm to at least all of the relevant blind, test and train data for a given sensor (eg, for all years of the AVHRR on NOAA 15)
(d) We'll also need you to provide publicly available references for the algorithm or a 1 page algorithm description
[1] CATEGORIES OF ALGORITHM DEFINITELY TO BE INCLUDED
1. SST retrieval from clear-sky observations for ATSR-series and AVHRR-series, with uncertainty estimates
As well as clear-sky observations, the data package will also include matched NWP fields (ECMWF) for use with algorithms that involve forward modelling.
The project team are developing the following algorithms:
(i) ATSR-series radiative-transfer-based linear retrieval coefficients (updating ARC coefficients**)
(ii) Buoy-referenced optimal estimation (as previously tested on Metop**)
(iii) Buoy-independent optimal estimation (new)
**References:
http://xweb.geos.ed.ac.uk/~chris/Publications/ARC-New-Retrieval.pdf (accepted for RSE) and
Merchant C J, P Le Borgne, A Marsouin and H Roquet (2008), Optimal estimation of sea surface temperature from split-window observations, Rem. Sens. Env., 112 (5), 2469-2484. doi:10.1016/j.rse.2007.11.011
Thanks to several of you who have already indicated interest in participating in this category of algorithms.
Note that we are looking for uncertainty estimates to be attached to retrieved SSTs. We consider these as an essential part of the SST algorithm, since improved uncertainty estimation is a core project requirement.
2. High latitude sea-ice/clear-sea/cloud classification for ATSR-series and AVHRR-series
The project team are developing their Bayesian classification techniques in this regard.
[2] CATEGORIES OF ALGORITHM DEVELOPMENT UNDER CONSIDERATION
3. Cloud/clear discrimination for ATSR-series and AVHRR-series at all latitudes
Within the project we will compare / use:
(i) for ATSR-series: SADIST operational masking and ARC-based Bayesian detection
(ii) for AVHRR-series: CLAVR-X bayesian additionally informed using the cost parameter from optimal estimation
4. Depth and time adjustments for ATSR-series and AVHRR-series
To provide a means to avoid aliasing of diurnal signals and to compare directly with the in situ climate record, CCI SSTs will be provided with an estimate of how to adjust the raw skin SST (the primary product) to a standardised depth-SST at a standardised local time (10.30 for day observations, 22.30 for night observations). This involves modelling the temporal evolution of the skin effect and thermal stratification down to drifting-buoy depth, between the satellite observation time and the standardised local time.
Within the project, we will compare two options for modelling: the approach used in ARC (Fairall skin effect + a Kantha-Clayson model) and use of the model POSH.
It is clear that a single algorithm must be used globally for this purpose for all sensors; therefore, in this case, we need results for all years/sensors and globally.
If you intend to submit algorithms under section [1] above (the definites) it would be useful to hear from you (if we have not already discussed this with me).
**If you want/intend to submit algorithms under section [2] above (under consideration), please e-mail me by 25 April.**
Please use: science.leader@esa-SST-cci.org
Thanks, Chris
Currently, we need to gauge the level of interest among potential volunteer contributors to this exercise, to contribute results for different categories of algorithms.
For those of you who decide this is relevant and you need to read on ... my apologies about the length of this post!
BACKGROUND INFO:
Broadly, to make the Algorithm Comparison and Selection objective, the exercise will proceed as follows:
(i) we distribute a data package containing satellite data, in situ data, and auxiliary information
(ii) participants apply their algorithms to these data (a training subset will be identified for any algorithm tuning that participants want to perform, along with a test subset)
(iii) we distribute a further set of data -- identical in format to the previous data package, but without any in situ observations -- this is the "blind" subset*
(iv) participants apply their final algorithms to the blind subset and submit the results (for all subsets, in a defined format) to the CCI team
(v) objective metrics are calculated for all submitted algorithms, and we compare and select algorithms for implementation in the CCI project
*the algorithm development team within the CCI project also don't have access to the blind subset prior to its distribution
Please note:
(a) The comparative performance of algorithms on the blind test subset will be the basis for selection of algorithms to be applied within the CCI project. If you submit your algorithm results, but, for whatever reason, would not want your algorithm implemented by the project, please make that clear upfront.
(b) Regional algorithms are encouraged as points of comparison, as are algorithms that work only for day or night. Since the project goal is a global, consistent SST data set for climate, algorithms of wide applicability are generally more likely to be selected.
(c) As a minimum, we are looking for application of any given algorithm to at least all of the relevant blind, test and train data for a given sensor (eg, for all years of the AVHRR on NOAA 15)
(d) We'll also need you to provide publicly available references for the algorithm or a 1 page algorithm description
[1] CATEGORIES OF ALGORITHM DEFINITELY TO BE INCLUDED
1. SST retrieval from clear-sky observations for ATSR-series and AVHRR-series, with uncertainty estimates
As well as clear-sky observations, the data package will also include matched NWP fields (ECMWF) for use with algorithms that involve forward modelling.
The project team are developing the following algorithms:
(i) ATSR-series radiative-transfer-based linear retrieval coefficients (updating ARC coefficients**)
(ii) Buoy-referenced optimal estimation (as previously tested on Metop**)
(iii) Buoy-independent optimal estimation (new)
**References:
http://xweb.geos.ed.ac.uk/~chris/Publications/ARC-New-Retrieval.pdf (accepted for RSE) and
Merchant C J, P Le Borgne, A Marsouin and H Roquet (2008), Optimal estimation of sea surface temperature from split-window observations, Rem. Sens. Env., 112 (5), 2469-2484. doi:10.1016/j.rse.2007.11.011
Thanks to several of you who have already indicated interest in participating in this category of algorithms.
Note that we are looking for uncertainty estimates to be attached to retrieved SSTs. We consider these as an essential part of the SST algorithm, since improved uncertainty estimation is a core project requirement.
2. High latitude sea-ice/clear-sea/cloud classification for ATSR-series and AVHRR-series
The project team are developing their Bayesian classification techniques in this regard.
[2] CATEGORIES OF ALGORITHM DEVELOPMENT UNDER CONSIDERATION
3. Cloud/clear discrimination for ATSR-series and AVHRR-series at all latitudes
Within the project we will compare / use:
(i) for ATSR-series: SADIST operational masking and ARC-based Bayesian detection
(ii) for AVHRR-series: CLAVR-X bayesian additionally informed using the cost parameter from optimal estimation
4. Depth and time adjustments for ATSR-series and AVHRR-series
To provide a means to avoid aliasing of diurnal signals and to compare directly with the in situ climate record, CCI SSTs will be provided with an estimate of how to adjust the raw skin SST (the primary product) to a standardised depth-SST at a standardised local time (10.30 for day observations, 22.30 for night observations). This involves modelling the temporal evolution of the skin effect and thermal stratification down to drifting-buoy depth, between the satellite observation time and the standardised local time.
Within the project, we will compare two options for modelling: the approach used in ARC (Fairall skin effect + a Kantha-Clayson model) and use of the model POSH.
It is clear that a single algorithm must be used globally for this purpose for all sensors; therefore, in this case, we need results for all years/sensors and globally.
If you intend to submit algorithms under section [1] above (the definites) it would be useful to hear from you (if we have not already discussed this with me).
**If you want/intend to submit algorithms under section [2] above (under consideration), please e-mail me by 25 April.**
Please use: science.leader@esa-SST-cci.org
Thanks, Chris
Subscribe to:
Posts (Atom)