Page Comparison

1

Managing provision of high-volume Core data sets 

(Jeremy) Summary:

1. We have technical mechanisms to (a) limit the cache size, (b) inform data providers when data is not cached, and (c) allow data providers to assert what Core data shouldn’t be cached.

2. We need the “big” data providers to be “reasonable” and share the burden of distributing their data. The key question from my perspective is what being “reasonable” means. We can iterate towards this by:

3. Indicating in the Guide the daily data volume that GC operators are expected to cache (for UK/USA this is 100GB)

4. Asking [“big”] data providers to prioritise which of their Core data should be distributed via the Global Cache and which they will serve directly themselves – perhaps based on their understanding of which data products are most widely used but ultimately their decision; with the aim of limiting the GC volume to the agreed daily amount (e.g., 100GB) We would expect some on-going discussion between big data providers and GC operators to get this right.

5. should be written in the Guide somewhere. Suggest this is in the section for DCPCs?

(Remy) In Notification Message, data producers in property (True or false) to cache or not. TB data should be cached. Cache operators, for those risks posing impact the system, should them cache the data? Proposal is to opt more agile approach, that is recommended data (not need to cache) to use the notification message to inform the data is not cached.

(Tom) Data not cached, NM still still points to the originating nodes.

(Simon) from data provider’s perspective, this mechanism (recommended data not being cached) will make the wis2 less attractive. This will make satellite data similar to the current GTS operation, very little satellite available.

(Jeremy) Current GTS, no data exchanged, no visible at all. But WIS2, recommended data will be visible from GB in Notification Messages. Achievable levels, big data producer, service operators to have a dialogue about the mechanism .

(Remy) data provider, data users to collaborate to decide what data is useful. It is important to decouple the roles of the GC and data providers.

(Simon) Community like CSMS can work with the end users to decide what other data can be shared in WIS2.

---

(Xiaoxia) actions: to draft name convention for the centre ids.

(Tom) delineate the Global services, one for NC and one for DCPC

(Jeremy) proposal for overseas territories, such as Saint Helena (SH), the center id is “sh-metoffice”

Another case: Two Met Office sites in UK, Exeter and Aberdeen, using the same center id “metoffice”. There is a need to distinguish them further in the center id.

for remote territories, name convention?

particular cases, such as USA and UK joint Global Service, the name?

islands, French New Calendonia, make a difference between France, overseas France and “half” France.

(Simon) timetable for transition is from 1 December, four EUMETSAT, the fourth of December will start the new centre id.

2

Checkpoint on writing sections of Guide and Transition document - any outstanding questions

(Tom) Requests a link for the reference to responsibility matrix for each chapter in Guide to WIS.

Images in the publication

• (Tom) suggested placing raw and editable images in the GitHub Guide Repository under the “image” folder https://github.com/wmo-im/wis2-guide/tree/main/guide/images

• (Rémy) asked for guidance for the use of SVG format

• (Enrico) Secretariat will handle the images to ensure a consistent style for publication. For technical specifications, including WCMP2, Notification Message and Topic Hierarchy, they could be added as separate appendices to Manual on WIS (WMO-No.1060, Vol-II).

3

Discuss outstanding issues pertinent to upcoming Satellite and Cryosphere workshops

Satellite workshop

(Enrico) Will present some slides at the workshop. One question from this community is: Does data should be sent to WIS2 or not? According to Manual on WIGOS: data shall be exchanged on WIS. It is an obligation, at least for Core Data.

(Simon) EUMETSAT delegation at the workshop is to ensure understanding of data policy-what core data and recommended data are. It is more on legal part, not technical discussions.

Cryosphere workshop

(Tom) For Cryosphere, Portal, federation, ESGF, Oystein is not in favor of using MQTT protocol but for WIS1 metadata harvesting. A meeting to discuss how we federate with other communities is needed.

(Enrico) There are other components, e.g. huge data (snow data), sending data to ECMWF, could be easily to transmit to WIS2.

4

AoB

(Simon) Shared some takeaways from the MQTT training course last week.
- Decoupling the message from data surprise the MQTT community.

- MQTT 5 feature: shared subscription, may have potential applications.

-(Rémy): (1) Benefits: redundancy + scalability.

(2) Challenge: Additional Topic Hierarchy (TH) is needed above the current TH and needs authentication.

(Simon) transport community was surprise to learn that we did the JSON standardization.