Reliable and user-friendly handling of earth observation uncertainty data with CoMet Toolkit
A year on from its initial public release, the NPL-developed CoMet toolkit has been used in several earth observation projects, including MetEOC, and recognised for the value it offers to the earth observation community.
Environmental observations made by satellites and in-situ measurement networks provide data that forms the basis of scientific understanding of the state of the climate. Such datasets need to contain information about the associated uncertainties to be accepted as credible and reliable — the handling of which can be a complex and, potentially, error-prone process.
Moreover, as multiple measurements are typically combined through processing chains, reliably handling this information can seem overly time-consuming for non-specialists.
A simpler way to store and propagate uncertainty and error-correlation information
Metrological approaches defined within QA4EO are broadly accepted by the earth observation community as necessary for developing quantitative characterisations of uncertainty for earth observation data. However, practically implementing these methods requires an understanding of error covariances in data. For example, appreciating that random uncertainties do not combine in the same way as systematic uncertainties.
To simplify the process, NPL developed CoMet Toolkit (Community Metrology Toolkit); open-source software designed to provide a user-friendly means to store and propagate uncertainty and error-correlation information.
CoMet Toolkit offers tools built with quality-assured code specifically developed for the job, that lowers barriers to entry for reliable handling of uncertainty data.
The software implements a method devised in the Horizon 2020 FIDUCEO and GAIA-CLIM projects. The software was developed, and continues to be developed, largely by NPL with the support of ESA and EMPIR MetEOC projects, supplemented by UK national funding from the National Measurement System (NMS) of the Department for Science, Innovation and Technology (DSIT).
Following its release in April 2022, researchers can freely use the tool to gain insights into error-covariance matrices to analyse and quantify uncertainty associated with measurement data. This information can then be used to make informed decisions about the accuracy and reliability of data and improve the quality of measurements.
Hosted on github with packages installable via Python Package Index, CoMet Toolkit provides a simple interface for tools for data analysis, error analysis, and uncertainty propagation. A key feature is the ability to handle and process dataset error-covariance information.
Tools in the CoMet Toolkit
Software tools currently offered are:
- CoMet_maths: a python module with useful mathematical algorithms (including interpolation with uncertainties) for general use as well as for use in the other tools in the CoMet toolkit.
- obsarray: an extension for defining, storing, and interfacing with uncertainty and measurement error-covariance information.
- punpy: ‘Propagation of UNcertainties in Python’ allows users to propagate obsarray dataset uncertainties through any given measurement function, using either the Monte Carlo (MC) method or the law of propagation of uncertainty, as defined in the Guide to the expression of Uncertainty in Measurement (GUM).
CoMet Toolkit in action
Notable examples of projects in which the CoMet toolkit was developed, tested, and used operationally include:
MetEOC
In MetEOC-4 researchers are applying the CoMet Toolkit to provide a preliminary FIDUCEO-style uncertainty analysis for early-stage development of the European Space Agency and European Union Copernicus Imaging Microwave Radiometer, CIMR Mission.
This research is establishing methods to provide evidence of traceability of current and future Copernicus satellite sensors with the added benefit of accessibility.
The CIMR satellite will carry a wide-swath conically scanning multi-frequency microwave radiometer to observe the essential climate variables: sea-surface temperature, sea-ice concentration and sea-surface salinity. It will also observe a wide range of other sea-ice parameters.
Building in metrological best practices from its early stages is an effective and efficient way for the European Space Agency to ensure high-quality, traceable measurements during future Copernicus Sentinel satellite missions.
FRM4SOC Phase-2
State-of-the-art uncertainty analysis was provided by NPL using the CoMet toolkit in the field of Satellite-derived Ocean Colour in the EUMETSAT-funded FRM4SOC Phase-2, or Fiducial Reference Measurements for Satellite Ocean Colour project (Phase 2).
FRM4SOC Phase 2 was launched by EUMETSAT in April 2021 and builds on the first FRM4SOC study.
The main goal of FRM4SOC Phase-2 is to expand the Fiducial Reference Measurements (FRM) capabilities within the Copernicus program and encourage the adoption of FRM principles across the ocean colour community.
Results from the inter-comparison exercise were incorporated into the Ocean component of MetEOC4.
H2020 HYPERNETS Calibration
The CoMet toolkit was instrumental for the H2020 HYPERNETS network processing chain – propagating uncertainties from lab calibration data to Level 2A hyper-spectral surface reflectances (SR) products measured in the field.
The toolkit was also used to derive uncertainty-quantified Top of Atmosphere in-band radiances for comparison to satellite data. The CoMet toolkit provided the functionality and flexibility to handle this complex error-correlation information.
Inaugural Ada Lovelace award for educating the international EO community
In March 2023, Dr. Pieter De Vis, a Higher Research Scientist at NPL, was awarded the NPL Ada Lovelace prize in Data and Computer Science in recognition of his leading role in developing CoMet Toolkit.
The prize was one of several Scientific Awards that recognise and promote scientific, engineering and metrological excellence at NPL. The judging panel cited the toolkit as ‘an example of how a digital NMI can educate a whole international community’.
After receiving the award certificate from NPL’s Chief Executive Officer Dr. Peter Thompson, Pieter De Vis commented, “We hope with the CoMet Toolkit we can enable user-friendly handling of complex uncertainty information within the earth observation community and beyond.”
“It’s exciting to see how our practical implementation of metrological approaches is starting to have a real impact in various projects such as MetEOC.”
Resources and tools
Further information, news, documentation, and training material is available at the CoMet Toolkit website.