Topic hierarchy and filename

Owned by Enrico Fucile
Last updated: Feb 17, 2022
2 min read

Principles

  1. A topic hierarchy provides a taxonomy structure to logically group concepts in support of lowering the barrier to finding data at varying granularity. Benefits include:

    1. Easy aggregation across topic hierarchy levels

    2. Easy to understand relationships (broad/narrow)

    3. Fine grained filtering for search

    4. Fine grained access control as required

  2. Message queuing protocols and metadata benefit from topic hierarchy and organization

  3. A single topic hierarchy which is interchangeable between discovery metadata, catalogues and message queuing protocols topic allows for consistent resource discovery, subscription to data and event driven notifications/workflow

  4. Topic hierarchy built from GTS legacy will be temporary and used only to make GTS data available on WIS2

  5. WIS2 hierarchy will be built gradually following the requirements of WMO domains

    1. Observations

    2. NWP

    3. …

WIS2 topic hierarchy and filename

Core hierarchy

 

Level

Name

Description

Value(s)

Example

 

Level

Name

Description

Value(s)

Example

1

scope

Scope

Scope of topic hierarchy (fixed to WIS2)

wis2

wis2

2

resource-type

Resource type

Type of resource being identified

  • data

  • discovery-metadata

  • monitoring-report

data

3

data-policy

Data Policy

WMO Unified Data Policy category, provides distinction in access control/permissions

  • core

  • recommended

core

4

data-type

Data Type

Top level data type

  • observations

  • nwp

  • …

observations

Examples:

wis2.data.core.observations

wis2.discovery-metadata.recommended.nwp

wis2.data.other.observations

wis2.monitoring-report.gts

Data Type Hierarchies

Data Type hierarchies vary significantly across data types (e.g. different for observations and NWP), and will be built gradually following the requirements of WMO domains.

Each WMO domain will also define filename conventions (basename, extension)

Observation Data Type

 

Level

Name

Description

Value(s)

Example

 

Level

Name

Description

Value(s)

Example

1

country

Country

ISO 3166-2 code of country

https://en.wikipedia.org/wiki/ISO_3166-2

ca

2

station-type

Station type

Station/platform type

WMDR FacilityType

landFixed

3

message-type

Message type

The type of message sent by the observation station

  • surface

  • upperAir

  • aircraft

landFixed.surface

airMobile.aircraft

 

 

Examples:

wis2.data.core.observations.ca.landFixed.surface

wis2.data.core.observations.gr.landMobile.??

wis2.data.core.observations.it.seaFixed.??

wis2.monitoring-reports.wis2.observations.de.??

 

Alternative Proposal

This alternative proposal has the country code and center name higher up in the hierarchie. Could be extended with data policy

 

Filename conventions

Observations

Filename: WIGOS_{wsi}_{datetime}.{extension}

where:

Name

Description

Value(s)

Example(s)

Name

Description

Value(s)

Example(s)

wsi

WIGOS station identifier

  • OSCAR/Surface Catalogue

0-454-2-AWSNAMITAMBO

datetime

Datetime

  • 2000-11-11

  • 20001111

  • 2000-11-11T0932

  • 2000-11-11T09:32:11Z

extension

Filename extension

  • bufr4

  • geojson

  • nc

  • tac

  • xml

geojson

Examples:

WIGOS_0-454-2-AWSNAMITAMBO_20001111.bufr4

NWP

TBD

  1. data format: grib2, nc

 

GTS topic hierarchy and filename

This is the current output of the TT-Protocols team

https://github.com/wmo-im/GTStoWIS2

Question to be answered.

Given that GTS will be dismissed does it make sense to develop a topic hierarchy based on GTS headers?

Shall we make a topic structure directly with GTS headers (T/T/A/A/ii/CCCC )?

Related discussions