add identification of Global Caches to WNM

[tomkralidis]

Note: sections marked with * are required.

Overview of proposal*

Update Global Cache technical considerations, adding a provision to emit their centre-id in WNMs when data is cached.

Amendment details*

See WNM amendment in wmo-im/wis2-notification-message#156, which covers encoding rules.

Requestor(s)*

ET-W2IT

Consultations

(Individuals, groups,or organizations that are consulted. Also cite publications or other documents that as references.)

Comments

[golfvert]

The proposed change is not doing exactly this: "Update Global Cache technical considerations, adding a provision to emit their centre-id in WNMs when data is cached.".
The PR, at the moment, is only for cache: false situation.
We have the cache: true that can be either "accepted" (the data is effectively cached) or "ignored" (when the GC considers the data shouldn't be in the cache for whatever reason: size, experimental,...).
If we decide to add the Global Cache signature in the WNM, I'd suggest to always do it for all core data.
It also means that a GC will have to remove the signature of another GC, if it is caching a file received using a WNM on cache/a/wis2/... and then, adds its own.

[kaiwirt]

I would go one step further: The property global-cache with the centre-id of the cache should be added (or modified) to the WNM when the cache cached the data (i.e. cache: true). The intent was to allow a cache or other user to identify where a data set was downloaded from (origin centre-id, WNM or Global Cache).

So to me in cache: false situations or when data is not cached this property should not be added or updated.

[amilan17]

for W2IT to discuss @hhaddouch

[6a6d74]

Adding notes from WIS2 Gateway testing meeting where this issue was first discussed:

The problem with using “dataserver” as a metric label

• A WIS2 Node might use several dataservers (e.g., ca-eccc-msc) which adds complexity to the metrics.
  We’re not really interested in differentiating between a WIS2 Node’s dataservers - only the behaviour of the WIS2 Node as a whole.
• Furthermore, if a WIS2 Node changes its dataserver(s), previous metrics referencing a retired dataserver will remain forever (or until reset).
• For the _dataserver_status_flag metric, this means that it will always look like the server is connected (or always failed) because the metrics are only changed when an attempt to use that dataserver was made.

Proposal

• Instead of using the host-name as the “dataserver”, we propose to use the centre-id for where the data is hosted - either a WIS2 Node or a Global Cache.
• Where the data is hosted by the origin WIS2 Node it is simple to use the centre-id provided in the notification message.
• However, Global Caches don’t currently identify themselves - except through the download URL.
• Consequently, to help identify when data is hosted in a Global Cache, a Global Cache should add
• properties.cache-id = « centre-id »
• to every notification message they publish - effectively “signing” the message.