Many algorithms have been developed to estimate concentration from passive microwave brightness temperatures (TBs; for summaries and evaluations of several algorithms, see, e.g., Comiso et al. 1997; Partington 2000; Meier 2005; Andersen et al. 2007). The empirically based algorithms differ in their inputs (combinations of different passive microwave frequencies and polarizations) and methodologies, but generally use the same source data: passive microwave TBs from the National Aeronautics and Space Administration (NASA) Nimbus-7 Scanning Multichannel Microwave Radiometer (SMMR) for 1978–1987 and a series of US Department of Defense Meteorological Satellite Program (DMSP) Special Sensor Microwave Imagers (SSM/I) and Special Sensor Microwave Imager and Sounders (SSMIS) for 1987–present. Gridded SSM/I and SSMIS TBs (Maslanik & Stroeve 2004) are the source for the production of the concentration CDR. The TBs are gridded on a 25-km resolution polar stereographic projection from swath fields obtained from Remote Sensing Systems Inc., in Santa Rosa, CA (Wentz 1997, 2010). While the algorithms yield different absolute sea-ice concentration, area and extent values (e.g., Kattsov et al. 2010), and the magnitude of the differences varies during the annual cycle, they are reasonably consistent through the interannual time series for a given time of year. This means that trend and variability estimates from different algorithms agree well.

Most of the algorithms are well-validated, mature and suitable for tracking trends and variability. Two widely used products are from the NASA Team (NT; Cavalieri et al. 1984) and Bootstrap (BT; Comiso 1986) algorithms. These provide the basis for the CDR and will therefore be discussed in more detail. Both algorithms were developed at NASA's Goddard Space Flight Center (GSFC) and are distributed by the NSIDC (Cavalieri et al. 1996; Comiso 2000). They have been validated through numerous intercomparisons with other passive microwave products as well as co-located visible, infrared and radar imagery (e.g., Cavalieri et al. 1991; Steffen et al. 1992; Partington 2000; Kwok 2002; Meier 2005; Andersen et al. 2007). Both products include intersensor calibration adjustments to the algorithms and sensor calibrations to assure consistency in extent and concentration estimates across the multiple sensors of the record (Cavalieri et al. 1999; Comiso & Nishio 2008; Cavalieri et al. 2012).

In addition to the basic algorithms and intersensor calibration, a number of quality control steps are applied (Cavalieri et al. 1999; Comiso & Nishio 2008). First, weather filters based on a threshold of TB ratios are used to eliminate low concentration values over open water areas due to wind roughening of the ocean surface or atmospheric moisture. Second, a land-spillover correction is applied to remove spurious ice along coastlines due to mixed ocean–land grid cells; the TB signature from such grid cells is similar to low concentration ice within the algorithms. Third, interpolation is used to fill missing data regions. For isolated missing grid cells, a bilinear interpolation is used to fill missing values in the input TBs. For larger missing regions (often due to missing swaths), a temporal interpolation is done on the resultant concentration fields, filling in the missing values with an average of concentrations from the day before and the day after.

Finally, the fields undergo a manual visual inspection for regions of erroneous concentration data and such data are removed via interactive analysis and replaced with interpolated concentration values. Unfortunately, provenance of these manual corrections is not available, precluding the ability to fully recreate the production of the fields, which is a prerequisite for CDR data sets. Trackable automated methods to replace the manual corrections (such as use of temporal filters) will be investigated for future revisions of the data set.

A well-accepted definition of a CDR is a “timeseries of measurements of sufficient length, consistency, and continuity to determine climate variability and change” (NRC 2004: 1). In this sense, the GSFC products are well-suited CDR products. However, the NOAA CDR programme imposes additional criteria in the interest of documentation standards, transparency and reproducibility. These criteria include: (1) self-describing file format (NetCDF4), (2) ISO 19115-2 compliant collection-level metadata, (3) Climate and Forecast (CF) compliant file-level metadata, (4) grid-cell level metadata (data quality fields), (5) fully automated and reproducible processing and (6) open online access to full documentation with version control, including source code and an algorithm theoretical basic document. The NOAA CDR concentration product presented here has been specifically created to meet these requirements.

The approach to creating the sea-ice concentration CDR is to build upon the legacy of the GSFC algorithm products, enhancing them to meet the NOAA CDR criteria. The CDR was developed by first implementing the NT and BT algorithms at NSIDC. This allows for version control, provenance tracking and full documentation of processing.

To create the CDR concentration parameter, the NT and BT concentration estimates are combined into a single CDR value. This is done by using the BT ice edge (based on a 10% concentration threshold) as the CDR ice edge. Within the ice cover, at each grid cell concentration is estimated from both algorithms and the higher concentration estimate is selected as the CDR concentration value. The rationale for this approach is that passive microwave algorithms tend to underestimate concentration, particularly during the summer melt season with greater underestimation typical in the NT algorithm.

For example, Andersen et al. (2007) found that NT underestimated ice cover by 4.4% in the central Arctic during winter, when passive microwave algorithms are considered most reliable. Agnew & Howell (2003) and Partington et al. (2003) found similar behaviour in comparisons with operational ice charts, which are produced by expert ice analysts combining available imagery. The BT algorithm is dependent, to some extent, on physical temperature and under extreme cold temperatures (such as during winter near the Antarctic coast and in the high Arctic) it tends to slightly underestimate concentration (e.g., Comiso et al. 1997). Concentrations estimated from synthetic aperture radar imagery have been found to generally be near 100% during winter throughout the central Arctic (Kwok 2002), so higher passive microwave values will tend to have lower biases.

During summer, when melt is prevalent, low biases have been found in both algorithms, though generally of larger magnitude in NT (Comiso et al. 1997; Meier 2005). So again, the higher concentration value is likelier to be more accurate. Near the ice edge there is greater ambiguity. The ice edge region is characterized by melt during spring and summer and by thin ice, which has also been found to generally be underestimated (e.g., Cavalieri 1994) during fall and winter. However, due to the low spatial resolution of the passive microwave sensors, the ice edge can be “smeared” out by the algorithms, resulting in detection of ice beyond the true ice edge. Thus, relative performance of the algorithms is quite variable near the ice edge (Meier 2005). Because of this ambiguity and potential inconsistencies between how the edge is detected by NT and BT, the CDR uses only the BT algorithm (10% concentration threshold) to define the ice edge.

We note that the selection of the higher concentration is somewhat arbitrary and it does not address any existing limitations associated with each algorithm. In fact, such a rule may result in overestimation of concentration in some situations (e.g., Meier 2005). However, as noted in the studies cited above (and several others), the higher concentration will typically be more accurate. Thus, by selecting the higher concentration, the low biases in both NT and BT are ameliorated, and the CDR methodology results in a concentration product with a lower average bias that is based upon two algorithms with a long and well-validated heritage. The CDR product does include as ancillary parameters the original NT and BT concentration for users’ convenience; also, the source (NT or BT) for the CDR concentration is noted in a quality assessment flag (see below).

We also note that an enhancement to the NT, commonly referred to as NASA Team 2 (NT2), was developed at the GSFC (Markus & Cavalieri 2000), which ameliorates many of the limitations of the original NT. NT2 compares much more closely to BT, even during melt conditions. However, it uses higher frequency TBs not available on all sensors in the record. It was therefore not chosen as a component for the CDR, although this may be considered in the future if consistency can be assured throughout the record.

The GSFC-produced NT and BT products have a long heritage and have undergone numerous validation studies. Their concentration estimates have been widely disseminated and have been used is numerous studies, including assessments by the Intergovernmental Panel on Climate Change. There is high confidence in the quality of the GSFC products within the scientific community. This was one factor in the selection of the GSFC products for use as the foundation of the NOAA CDR product. Nonetheless, as mentioned above, the GSFC products are lacking in some features that, while they do not affect the quality of concentration retrievals, do not meet NOAA CDR standards. Below, we outline these limitations and how the new concentration product has been designed to meet these standards.

The GSFC-produced NT and BT products contain no grid cell level uncertainty or data quality information, both essential for meeting CDR criteria. Therefore, the CDR product includes two data quality fields. The first is a spatial standard deviation field of both BT and NT concentrations based on a 3×3 grid cell neighbourhood for each grid cell. The local spatial standard deviation is calculated from up to 18 values (nine grid cells each of NT and BT sea-ice concentrations; see figure 3 in Peng et al. [2013] for the schematic diagram) with a minimum of six grid cells with valid values as a threshold for a valid standard deviation. We note that this is not explicitly a quantitative error estimate. However, it serves as an indication of relative data quality between different grid cells. Higher standard deviation values are found when there is larger disagreement between NT and BT concentrations and when there are high spatial gradients in concentration, as illustrated by examples shown in Figs. 1 and 2. Both regimes are indicative of potentially higher uncertainty. Large differences between NT and BT may indicate less confidence in the algorithms and are particularly common to regions where melt occurs. Liquid water or melt ponds on the ice surface contaminates the emission from the ice, reducing the efficacy of the algorithms’ ability to calculate concentration. High spatial gradients in concentration occur near the ice edge where sparse and thin ice result in more ambiguous concentrations. It also reflects the low effective resolution of the sensor (ca. 70×45 km for some passive microwave frequencies used in the algorithms), which limits the precision of the ice edge location. The standard deviation parameter (Figs. 1b, 2b) has low values (2–4%) in the interior, where errors are expected to be small and are consistent with concentration errors in cold, consolidated ice conditions found from previous validation studies (e.g., Kwok 2002; Andersen et al. 2007). Near the ice edge, where thin ice and/or melt dominates, the standard deviation parameter has values of 30–40%, which is in range of high errors seen in other validation studies (e.g., Partington et al. 2003; Meier 2005). While not specifically an error estimate, the spatial patterns of this standard deviation field also agree well with other error maps that have been developed for the Enhanced NT2 algorithm (Brucker et al. 2014) and the European Ocean and Sea Ice Application Satellite Application Facility product (Eastwood et al. 2011).

Thus, the standard deviation field provides a quantitative indication of relative uncertainty. A second quality assessment field contains flags that note salient conditions for each grid cell, including melt state, proximity to coast (indicating potential land-spillover contamination not removed by the automated filter) and low concentration grid cells (which typically have higher errors). These two fields allow users to assess the quality of each CDR concentration estimate. While a quantitative error estimate is desirable, it is difficult for sea ice because of the spatial and temporal sampling difference between the passive microwave data and available validation data, as well as uncertainties in the validation sources themselves (e.g., other satellite imagery). As noted above, the quality assessment field also denotes which algorithm's concentration was used in the CDR parameter for a given grid cell.

Another issue with the GSFC products is that manual quality control is used in their production. It is clear that the automated processing results in some spurious results and that manual quality control by expert scientists knowledgeable about the algorithms and concentration fields result in improved fields. However, as beneficial as the manual corrections may be, such an approach violates the CDR requirement for full transparency and reproducibility. Therefore for the CDR concentration, all processing is fully automated and the source code is made available through the NOAA CDR programme.

Finally, while the GSFC NT and BT fields have little or no file-level metadata and are distributed in a flat-binary format, the CDR is produced in the self-describing NetCDF4 format with fully compliant CF metadata. Such metadata and self-describing file formats are becoming more and more important as the data are being used by a broader community (i.e., beyond sea-ice experts) and provenance and future accessibility are becoming more of a focus in the international community.

Version 1 of the sea ice concentration CDR was initially produced for the SSM/I period, 1987–2007 and it is this version that is evaluated here. It has since been reprocessed to Version 2 and will be updated quarterly in the future. Version 2 updates the time series through 2012 using SSMIS, along with minor changes to metadata and the format of data quality fields. No changes were made to the CDR concentration processing. The verification presented here is therefore equally valid for Version 1 and Version 2. In addition to the concentration CDR and associated data quality fields, the GSFC BT and NT fields are included along with a combined concentration estimate that is produced from the GSFC concentrations using the same method to produce the CDR concentration (Table 1). This combined GSFC parameter is the main point of comparison for the CDR estimates in this paper and is hereafter referred to simply as “GSFC.” Another feature of the GSFC parameter is that it extends the sea ice record through the SMMR period. For the initial version of the CDR, the SMMR period is not included due lack of provenance in the initial processing of SMMR by GSFC and concerns over data quality that in the initial processing required significant manual correction (Parkinson, pers. comm.). We plan to investigate extending the processing of the CDR concentration parameter for the SMMR period in the future.

Specifics of the CDR product and processing methods are provided in a companion paper (Peng et al. 2013). The data are available from the NSIDC (http://nsidc.org/g02202.html) along with accompanying documentation and an Algorithm Theoretical Basis Document (Meier 2012). The ATBD also includes further details of the NT and BT algorithms and discussion of data quality and error sources.

The algorithms included in the sea ice concentration CDR have been widely validated in numerous studies (e.g., Cavalieri et al. 1991; Steffen et al. 1992; Comiso et al. 1997; Partington 2000; Kwok 2002; Meier 2005; Andersen et al. 2007). Here we verify the CDR estimates via comparison with the combined GSFC estimates created from the original NT and BT algorithm products (in the same manner as the CDR concentration, that is, the highest value of NT and BT). As noted above, the primary differences in the CDR and the GSFC products are: (1) missing grid cells are not filled with interpolation, (2) no manual quality control is performed and (3) processing is done at NSIDC (Table 1). A final difference is that NSIDC uses the most recent version of the input TBs available at the time of initial processing (Remote Sensing Systems versions 4 or 6) while GSFC has not reprocessed the NT product with newer TB versions (BT has been reprocessed). The input data therefore have some differences for part of the period. However, these TB version differences are generally quite small.

We assess here the effect of these differences on estimates of concentration, area and extent. “Concentration” here is defined as the percent coverage of a grid cell or region. “Area” is the total surface area of sea ice, summed from the concentration-weighted area of each grid cell. “Extent” is the total surface covered by ice of at least 15% (i.e., not weighted by concentration, just summing the area of all grid cells with ≥15% ice).