1 | Managing provision of high-volume Core data sets | | High-volume core data sets mechanism (Jeremy) Summary: We have technical mechanisms to (a) limit the cache size, (b) inform data providers when data is not cached, and (c) allow data providers to assert what Core data shouldn’t be cached. We need the “big” data providers to be “reasonable” and share the burden of distributing their data. The key question from my perspective is what being “reasonable” means. We can iterate towards this by: Indicating in the Guide the daily data volume that Global Cache (GC) operators are expected to cache (for UK/USA this is 100GB) Asking [“big”] data providers to prioritise which of their Core data should be distributed via the Global Cache and which they will serve directly themselves – perhaps based on their understanding of which data products are most widely used but ultimately their decision; with the aim of limiting the GC volume to the agreed daily amount (e.g., 100GB) We would expect some on-going discussion between big data providers and GC operators to get this right.
should be written in the Guide somewhere. Suggest this is in the section for DCPCs? Global Cache operators can cache more data than the current volume on GTS. Communities like CGMS, can work together with end users to determine what data to be cached in Global Cache.
Topic Hierarchy (TH) of merging country and centre-id timeline (Rémy) As discussed in the ET-W2AT in Washington (13-15 November), there should be a window for all the WIS2 nodes to be able to migrate from the current TH to the new one. Three Global Cache Operators (DWD, JMA and SYNOPTIC) confirmed that they are able to handle both THs. Proposal: to send an email to all WIS2 node operators in Pilot Phase informing this migration to the new TH starts from 4th December and window is set to be 3 months. For WIS2 node operators using wis2box (new release set to be on 12 December), Action: Secretariat (Xiaoxia) to draft a WIS2 node naming convention
WIS2 node naming convention (Tom) For WIS2 node naming, it is good to delineate the centre ID for Global Services rather than using NC or DCPC. (Jeremy) for overseas/remote territories, using TLD, such as Saint Helena (SH), the good practice is the center id being “sh-metoffice” for particular cases, such as USA and UK jointly providing Global Service, the name? (Rémy) Not really matter as GC name is used internally.
(Rémy) (Simon) confirmed that EUMESAT will start using the new TH from 4th December
|
2 | Checkpoint on writing sections of Guide and Transition document - any outstanding questions | | (Tom) Requests a link for the reference to responsibility matrix for each chapter in Guide to WIS. Images in the publication (Tom) suggested placing raw and editable images in the GitHub Guide Repository under the “image” folder https://github.com/wmo-im/wis2-guide/tree/main/guide/images (Rémy) asked for guidance for the use of SVG format (Enrico) Secretariat will handle the images to ensure a consistent style for publication. For technical specifications, including WCMP2, Notification Message and Topic Hierarchy, they could be added as separate appendices to Manual on WIS (WMO-No.1060, Vol-II).
|
3 | Discuss outstanding issues pertinent to upcoming Satellite and Cryosphere workshops | | Satellite workshop (Enrico) Will present some slides at the workshop. One question from this community is: Does data should be sent to WIS2 or not? According to Manual on WIGOS: data shall be exchanged on WIS. It is an obligation, at least for Core Data. (Simon) EUMETSAT delegation at the workshop is to ensure understanding of data policy-what core data and recommended data are. It is more on legal part, not technical discussions. Cryosphere workshop (Tom) For Cryosphere, Portal, federation, ESGF, Oystein is not in favor of using MQTT protocol but for WIS1 metadata harvesting. A meeting to discuss how we federate with other communities is needed. (Enrico) There are other components, e.g. huge data (snow data), are sending data to ECMWF, which could be easily to transmit to WIS2. |
4 | AoB | | (Simon) Shared some takeaways from the MQTT training course last week.
- Decoupling the message from data surprise the MQTT community. - MQTT 5 feature: shared subscription, may have potential applications. -(Rémy): (1) Benefits: redundancy + scalability. (2) Challenge: Additional Topic Hierarchy (TH) is needed above the current TH and needs authentication. (Simon) transport community was surprise surprised to learn that we did the JSON standardization. |