Data Integrity

Information regarding data quality and integrity are essential considerations when selecting a suitable dataset and Earth Observation (EO) and climate model data is no different. Data Integrity (DI) is central to the uptake and use of this data across science and commercial applications and research and operational usage scenarios.

Data Integrity Definition

Following consultation with CEMS stakeholders, data integrity was defined in the context of CEMS as:

  • “A measure of confidence in the data, arrived at by characterising and monitoring quality at specific points along the production chain.”
  • To provide  metadata alongside the original data set or via a helpdesk or consultancy service to arrive at a judgement on the data’s fitness for purpose:
    • Data quality – based on the results of routine Quality Control activities;
    • Data accuracy – based on more complex data evaluation and comparison exercises;
    • Data availability – considering the length, coverage and operational scenario of each mission and the performance of data production within the ground segment;
    • Data accessibility – considering data policy issues and the practical implications of accessing data;
    • Data provenance – considering the origin, evolution and status of the algorithms and processes applied to the data.

Data Integrity also includes the analysis of:

  • Quantitative error/uncertainty – derived through error modelling and ideally provided on a pixel by pixel basis, as an integral part of the data set.

In addition, CEMS hosts a number of applications which can be used to derive useful information about a dataset. In particular, CEMS provides a series of tools to identify co-located in situ data and associated datasets to enable comparative accuracy and error characteristics of their data. Such collocation tools include GECA, an operational ESA-funded application and demonstrators such as OceanDIVA, which compares observed and modelled ocean data and Storm TRACK, which tracks storms events and finds simultaneous EO datasets.

Finally, CEMS provides access to experts in a range of themes who can undertake bespoke DI commissions and service development.

Download the CEMS Acceptable Use Policy (Academic Policies are available from CEDA)

Cookies on Catapult explained

To comply with EU directives we now provide detailed information about the cookies we use. To find out more about cookies on this site, what they do and how to remove them, see our information about cookies. Click OK to continue using this site.