Outcome of project Phase 1

From CryoClim


User requirements

The type of products and service discussed with key users have been limited to those inline with the vision of the project, i.e., to that set by intentions and policy of the service providers behind this project. User requirements was in general retrieved from “umbrella organisations” representing groups of users and that already have a broad and clear overview of user needs in general. Examples of such organisations are World Meteorological Organisation (WMO) and European Environment Agency (EEA). There are also other key organisations setting requirements, in particular Group on Earth Observations (GEO).

CryoClim is envisioned to be a contribution to the Global Earth Observation System of Systems (GEOSS). One of the main ideas behind GEOSS is that operational global monitoring of the Earth system is all too large an endeavour for any single organisation or nation. The solution is to assemble a system of components from interested national and international parties worldwide in a concerted way. One of the main tasks is to let these components – individual monitoring systems for parts of the Earth system – be able to communicate with each other and share data. With common communication languages and data standards, the individual systems can be coupled into a network. Users can be provided access to this network through web services that searches and retrieves data without bothering the user with where and how the data is stored. When established, such a system will be extremely powerful and flexible.
Hence, a contribution to a global system of systems needs to be a set of “nodes” that can be networked to other systems. The nodes can also be integrated with other portals, e.g., for national applications and services. This means that the nodes can be components in several networks.

The initial CryoClim service is envisioned to be composed of four cryospheric database nodes hosted by the three mandated organisations: Norwegian Meteorological Institute (METNO), Norwegian Polar Instititute (NPI), and Norwegian Water Resources and Energy Directorate (NVE). The nodes represent product production chains and product storage systems for sea ice, snow and glacier products. Later, the set of nodes could be expanded with more European nodes (contributing organisations).

The envisioned main functionality and requirements for the system are:

  • The system functionality will be provided through web services.
  • The web service should follow state-of-the-art one-stop-shop principles for spatial data.
  • The service offered by the system will be free of charge.
  • The product production chains and the corresponding databases should be distributed and hosted by mandated organisations.
  • The databases should be integrated over the Internet in a seamless and scalable network, which is open for inclusion of other databases/sub-services in the future.
  • The system tools should be state-of-the-art open solutions following international standards.

The users did not have very specific requirements for the web service other than that it should be based on the upfront envisioned principles being a web service. The requirements for the service is then merely of technical character such that the system is 1) compatible with the operational Information and Communication Technology (ICT) principles for the host organisations and 2) that it is in line with requirements set by international standardisation groups,in particular for groups related to web services in general and web services for geospatial data in particular.

The standards and approaches to follow are given in the following. Some of the basic functionality for the system and service is then implicitly given.

The graphical user interface functionality was clarified in the design work<span style="text-decoration: underline;" /> (see System design) and will be revised under the course of the project through a dialogue with the key users.

Meta-information on the products should be listed in a catalogue using a PostGRES database. The metadata should be compliant with WMO Information System (WIS) and GEOSS specifications. The catalogue should provide both a human (interactive portal) and a machine interface (RESTful, i.e., systems following Representational state transfer (REST) principles,  web services). The catalogue should be able to list products by topic category, parameter/variable area, time and institution/project source. The interactive catalogue should offer map search functionality. Selected products should be available for Web Map Service (WMS) presentation where appropriate.

For online processing, the user should be able to specify the required file format, map projection, sub area, etc., wanted within certain limits prior to downloading the product.

Products should be available for download through Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), Open-source Project for a Network Data Access Protocol (OpeNDAP) and Web Coverage Service (WCS). Download can be set up both in push and pull mode. Push mode implies subscription on a regular basis where products are delivered through various channels.

System design

The CryoClim system is a distributed system involving production chains located at several institutions. As such, the CryoClim system will be a system of systems, and the focal point of the system architecture is to set up interoperability principles that support the distributed idea of CryoClim as well as the interoperability of the CryoClim system within a global environment as defined by e.g., GEOSS, WIS and Infrastructure for Spatial Information in the European Community (INSPIRE) principles.

Within the CryoClim system, several production chains will be used. One of these production chains already exist in a prototype version (sea ice). One of the main challenges is to interface the production chains to the rest of the system. CryoClim will in principle not interfere with the production chains, but will need to pose some requirements upon the output and interfaces to them. How this is integrated into the individual production chains within the CryoClim system is determined by the production chain owner. The CryoClim production chains have to implement common standards in the following areas: metadata, quality information, file formats, map projections, and data access interfaces. To ensure compatibility with upcoming systems/requirements (e.g., GEOSS, WIS and INSPIRE) standard interfaces should be utilised. This implies that each production chain within the system should publish data and products using OpenGeoSpatial Consortium (OGC) interfaces.

An overall design of the CryoClim web service portal is provided in the report. First, the main requirements for the portal are described and then follow a description of the functionality of the user interface.

A part of the system design descriptioDescriptions of the algorithms planned for the retrieval of the geophysical variables in each product of the CryoClim service. The sea ice products are based upon the Ocean and Sea Ice Satellite Application Facility (OSI SAF) (the re-analysis for historical data). Retrieval of snow variables will be based on results from other projects, in particular state-of-the-art algorithms for retrieving snow variables from passive microwave radiometers, algorithms for retrieving snow variables from optical data, and multi-sensor time-series algorithms for analysis and retrieval from optical and microwave data simultaneously. Glacier products from mainland Norway are based on semi-automated algorithms and manual procedures retrieving variables from Landsat TM/ETM+ imagery (or other image sources with similar spatial resolution and spectral contents) using image analysis and GIS techniques. The glacier products for Svalbard will be produced using automated image analysis techniques on C-band SAR images. Historical satellite data available in C-band is from ERS-1/2, Envisat ASAR and Radasat-1/2.

Sevice validation protocol

ICT system validation covers validation of the ICT infrastructure applied in the project. In validating the whole system, the following is tested: Processing (production of expected results), performance (processing time to perform typical types of access), and production chains (to check that data is accurately processed throughout the production chain).
The algorithm and model validation is intended to ‘prove’ that the algorithms and models are working according to the expectations. Both system validation criteria and algorithms and model output ‘expectation’ criteria were defined in the validation plan.

A series of quantitative measures are specified which enables the system quality to be assessed objectively. As part of the system validation, these measures will cover system performance (both technically and from an end user’s perspective), accuracy and timeliness of the data provided. As part of the system evaluation, the measures will include ratings of the acceptability and usefulness of the system from the key users, and user performance measures in a usability trial

User evaluation will provide on-going inputs to the CryoClim development team and report on the evaluation of the system to be performed for each version of the system. Evaluation will be based on two principal activities: Demonstrator review (the key users will monitor the development of the system and give feedback) and user panel (user representatives will test and provide comments on the developing system prototype).

The ICT system validation is based upon functional requirements (what the system will do) and non-functional requirements (qualities such as performance, security, interface operation, etc.). User cases have been defined to ensure that all these requirements are met.

Algorithm and product validation includes testing that the algorithm actually works sufficiently well under all expected natural variability and testing that the cryospheric variables retrieved from the satellite data and other data and delivered as products in the operational environment actually stick to the quality criteria specified. The cryospheric algorithms have to undergo rigorous validation before they are taken into operational use. The algorithm validation is to take place before the algorithm is taken into operational use, in contrast to the running product validation that is taking place during operational service provision in order to check that products delivered are of sufficient quality.

Demonstration products

The Demonstration samples of climate products have a design and content equivalent to real climate products to be delivered by the operational system, but their content is not necessarily representing the real world as the retrieval algorithms and processing system are not yet developed. As an exception, full grade climate products have been developed for sea ice.

The file format of the products is netCDF, and the metadata follows the CF-1.0 convention with metadata elements implemented as global attributes.

Demonstration products for sea ice have been made for monthly Sea Ice Concentration (SIC) for the northern hemisphere covering the period from 1988 to 2007. The monthly products are based on daily products from the ongoing European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT) OSI SAF reprocessing project for historical Scanning Multichannel Microwave Radiometer (SMMR) and SpecSSM/I data. The reanalysis produce a consistent data set for sea ice concentration and ice edge, both swath data and daily accumulated data, going back to 1978. The resolution is 10 km.

Demonstration samples for snow products have been made for Snow Cover Extent (SCE), Fractional Snow Cover (FSC), Snow Water Equivalent (SWE), and Snow Depth (SD). The global products have been separated into three different regions: Northern region (northern part of the northern hemisphere, including USA, the mountains of North Africa and the Himalayas); Equator region (from north of Africa to south of South America) and Southern region (Antarctica). The demonstration products have been made from AMSR-E (Advanced Microwave Scanning Radiometer - Earth Observing System) and MODIS snow products.

For mainland Norway, demonstration sample products have been made for Glacier Area Outline (GAO), Glacier Snow Line (GSL), Glacier Firn Line (GFL), Glacier-dammed Lake Outline (GLO), and Glacier Periodic Photo series (GPP). The glacier products for mainland Norway are derived using Landsat images of 30 m resolution.
For glaciers in Svalbard, demonstration sample products have been made for Glacier Surface Type (GST) and Glacier Balance Area (GBA). The glacier products for Svalbard are derived using SAR images of 10-75 m pixel spacing.

Service demonstrator

The aim of developing a demonstrator was to set the project in a position to clarify functionality of the web service and contents of the products at an early stage. The actual web service and products could then be developed more rapidly with fewer trial-and-error iterations with the users, leading to the final web service and products.

The future operational system will offer searching, previewing and ordering of cryospheric data. The portal will allow users who have logged on to preview cryospheric data in a map window. Corresponding metadata are displayed textually. In order to locate data for previewing, search queries may be formulated by selecting from lists specific values for individual search criteria. Search results are collected in a list, and may be expanded, allowing individual products to be selected for preview or download.

In order to present our vision of the future portal, a flash demo has been created in Adobe Flash. The demonstrator shows how the desired functionality may be integrated in one single portal. However, our vision goes beyond the portal. Cryospheric data is but only one of several datasets that one may wish to combine to analyse climate change. This vision becomes true when the cryospheric data infrastructure becomes an integrated part of GEOSS.

Since the demonstrator is focused on a few limited examples, and with no backend, an additional pilot portal has been set up. The pilot portal builds on two backend servers at METNO. One server demonstrates searching for data in a metadata catalogue. The other server demonstrates previewing of cryospheric data in a map of the northern hemisphere. The two servers are not integrated at the moment. Still, the pilot portal demonstrates the usefulness of some existing technologies that could be used in the future portal.

Promotion and outreach activities

The project work on promotion and outreach carried out promotional activities as well as established and ran a dialogue with key organisations crucial to the success of the service to be developed.

Contact has been established with a carefully selected group of key organisations – key users either representing a larger group of end users, ‘high impact’ users and other organisations central in climate monitoring. The key organisations are WMO, Global Climate Observing System (GCOS), World Climate Research Programme (WCRP), EEA (ETC/ACC), Max Planck Institute for Meteorology (MPI), UK Met Office Hadley Centre, GEO, Global Monitoring for Environment and Security (GMES) Bureau, Committee of Earth Observation Satellites (CEOS) and EUMETSAT. The contact with the GEO secretariat resulted in collaboration based on GEOSS Call for Participation in the Architecture Implementation Pilot, Phase 2 (AIP-2). The idea is to integrate the CryoClim service with GEOSS at an early stage as a pilot (test) service. This will ease the full integration of CryoClim with GEOSS when the service goes operational. Also, the project has through its partners linked to CEOS Climate Actions from the Priority-1 actions in the CEOS’ Response to the GCOS Implementation Plan.

Contact has been established with a group of key projects toward which the project needs to harmonise and coordinate work, hopefully creating synergy effects. The projects are OSI SAF (EUMETSAT), Global Land Ice Measurements from Space (GLIMS), EuroClim (EC), GLACIODYN (RCN), DAMOCLES (EC), GlobGlacier (ESA) and GlobSnow (ESA).
The project has developed two service demonstrators setting the project in a position to clarify functionality of the web service, the web portal and the contents of the products in a dialogue with the key users at an early stage. These demonstrators represent a key promotion and outreach opportunity as well.

As a part of the promotion and outreach activities, the current project web page has been created in order to provide information about the existence of the project, the main activities, the products and the services being developed. The web page also gives access to contact information for all the involved partners.

The project and the envisioned service have been presented in six meetings and conferences in Europe. The project has also been presented in three media events. These resulted in 11 other presentations at various news sites on the Internet.
A project summary report is currently being written and will be distributed to a broad group of potential users and other key organisations. The report will also be distributed in relevant meetings and conferences.

Advices from STAG

The progress review meetings with the Scientific and Technical Advisory Group (STAG) provided very valuable input in Phase 1. The overall conclusions and recommendations from STAG were as follows: There are very many challenges in climate monitoring, and CryoClim cannot cover them all. Main emphasis should be put on a few, important issues. STAG’s consensus for main issues was:

  • Make the CryoClim service unique, as a clear added value to existing and other upcoming services.
  • Exploit existing expertise and skills.
  • Give highest priority to make time series of high quality, including making quality information available.
  • Long-term time series of high quality for snow is missing and would represent a significant contribution from the project.