2022-02-02 ET-W2AT Meeting

1. Opening and Discussion on MQP Protocol

Jeremy

Rémy

MQP Protocol

Which MQP Protocol

• (Jeremy) To specify two protocols: MQTT3.1 vs MQTT 5

• (Henning) MQTT on extremely small messages; any limitation on MQTT

• (Remy) No limitation.

• (Baudouin) Are two MQTT interoperables?

Action: to put the alignment of granularity between topics and metadata on the agenda to discuss

(Kai) GTS: headers in the filenames; metadata topics and real data

(Tom) discovery metadata, pub/sub, providing notification of new files

(Jeremy) data exchange will be done messages with points where data can be downloaded.

(Enrico) WIS2 topic principles is under discussion

(Rémy) : we need to review Global file name conventions in WIS2

2. Discussion on Shared Service

ALL

Shared Service

/wiki/spaces/WIS2/pages/306970677

(Rémy)The previous meetings discuss the concept of shared service and this meeting aims to reach agreement of the concept and discuss the list of core shared service

The team agrees the concept of shared service in WIS2.0

• Global Broker

• Global Catalogue

• Global Cache

(Baudouin) GC abrreviation for both Global Catalogue and Global Cache

(Li Xiang) Suggest using Global Discovery Catalogue

(Baudouin) Who is responsible for the copy messages, GISC or DCPC ?

(Remy) Who will run the shared service is not yet decided. a small number of NCs, download be done by global cache keeping data available and announce the availabilty by providing the URL to the data.

(Tom) TT-WISMetadata is working on the draft document of WMCP2.0 in 2-3 weeks. OGC API records is served as baseline for the catalog protocol and metadata standard

Action: to discuss the metadata protocol at next Monday’s meeting

Action: Tom to share the link for the OGC API records and wmcp 2.0

(Hassan) To test them out on the pilot projects and add the deployments of the WIS2node in a box. To involve the TT and ET at the very beginning to develop the architecture.

3. Discussion on NC/DCPC connection with the shared service

ALL

How many instances are there for the shared service?

Do you agree that NC/DCPC will connect to more than one instance of a shared service

(Rémy) NCs: data to broker, NC should publish its data and message twice at least. Message data flow is not yet decided. It is important to decouple NC and GISC from the data flow aspect.

(Jeremy) WIS Centers, NC, DCPC to connect to more than one instance of the shared service

NC to talk to GISC, or 2 shared service instances,

(Rémy) No cache synchronization. Brokers should see all the messages.

(Henning) No hard limitation for the instances of shared services.

(Rémy) Global Broker is the key

(Hassan) Use the Audit and Certification to limit the number.

(Peiliang) Monitoring will be more effective than audits.

(Jeremy) Audit is important.

(Peter) Audit Effectiveness is important.

(Henning) Rather than taking political issue into account, but to use technical solution for data usage

(Kai) Service registry (Global control center), solution is to have the connections automatically.

(Peiliang) Algorithm to optimize the connectivity sounds great. What we need to consider is to have the monitoring system in place to monitor the daily performance. Then there will be a report at the end of year to present, indicating the global infrastructure situation.

(Kai) data or service monitoring? it should be distinguished.

(Rémy) We need to define various metrics using the same appraoch.

(Timo) The use of standards and a service oriented architecture make it technically easy to replace one components by another. example, a NC plugs into another GC and GB, whose addresses they obtain from a registry. Only components that are working (as per monitoring) are listed in the registry. Since we have standardized the MQP, the message schema and possibly the download schema, GB and GC are interchangeable

Decision: NC/DCPC MUST connect to at least one instance of a shared-service, and SHOULD connect to two or more instances of a shared-service

Decision: NC/DCPC connects directly to the shared-service instance(s) - not via their GISC.

Decision: No requirement for all-to-all fully meshed connection

Decision: There must be more than one instance of each shared service

Decision: there needs to be automated service monitoring of the WIS2 system from day 1

4. Discussion on Global Cache connectivity

ALL

(Rémy) No need for Global caches be connected to each other. Synchronization is only for global brokers and not for global cache.

(Kai) We should not forget about the structure of the connections (not only the number).. We need to avoid having two halves of the brokers which are only connected by one link

(Peiliang) need to ensure that centres are connected at least with a "spanning tree"