Opening

Jeremy

• Reminder of the time-line for completing preparation of the test plans / suites.

• EC-78 (June 2024) request that INFCOM to assess the capability of existing WIS 2.0 Global Services

Update on progress developing test plans / suites for each Global Service type

(Max) any meeting between GB and GC?

(Jeremy) use today’s meeting as a coordination platform

• Global Broker (Remy)

Rémy

Documentation: tests description + Preparing GB for performance test

• Configuration of the WIS2 Nodes subscription

• Preparing the Global Broker for the tests

• Putting back the Global Broker for WIS2 operations

(Max) overhead of changing the password of GB subscribers and re-login? (Rémy) Low impact. Not a big issue.

(Action) Steve to complete the test setup in the global-broker.adoc

To-do-list: mechanism of start/stop of the process

no standard logs but metrics as common languages among GB operators.

(Maaike) will the results be based on metrics collected by the test Global Monitor? (Max) it is an individual component test. (Rémy) the metrics can be collected by 1 primitive instances

(Antje) promethus to be included is a good idea.

(Rémy) functional test, option 1 to check if the GB subscribes to WIS2 Nodes; option 2: to allow mosquito connection as a client direct to the GB. We need to balance, considering more work required by automation work and quick work the manual ones.

For each test (or batch of tests) we need define how to observe test results:

• check that metrics of the Global Service report numbers expected based on test

• dedicated client to subscribe to WIS2 Network during test and check client receives metadata and data published during tests (minus the invalid test data)

• Global Cache (Jeremy)

Jeremy/Max

• test specification of test suits

• test tools

• data bundle

• how to connect

• how to clean and reconnect

(Action) Update the Users by Test clients or Global Broker in Global-Cache.adoc (to be discussed by Jeremy and Max)

(Rémy) It is debatable of having automation test.

(Antje) Agrees with scripts for automation test. And proposes the template of wrong messages.

(Maaike) Where possible tests should be automated using predefined scripts to ensure tests are reproducible. When not possible, we clearly define the manual actions needed to create the starting condition for the tests.  

• Global Monitor

Lei

Two GM operators to agree on the dashboard based on the metrics.

With only one alert defined, the test is only to verify the dashboard

(Action) Lei and Chems to get started working to define alerts.

(Rémy) Fake alerts can be created for test.

Summary: missing parts: tools, data bundle, connection need to be added.

Update and progress on preparing the wis2dev.io test environment (Secretariat)

10VM provided by WMO between the mid August and mid November (3 months).

(Rémy) a permanent test environment with at least a GB, a GC, a GDC, and a GM, pending decision

Update and progress on provisioning tooling for the tests (Remy+)

Discussion

Date for the next meeting: 8th July