R1005 breaks uses cases for partial subscribers by conflating message sequencing and MDIB versioning

[PaulMartinsen]

Section Number

2:A.4.2 MDIB Report Retrofit, R1005:

R1005: A SOMDS Consumer should reconnect or go into a fail-safe mode when it receives a report with an MDIB version that is either lower than the last received version or more than one version higher than the last received version.

Priority

High: Important issue where there is major issue to be resolved. Requires discussion and debate.
Important, I think, because it may have long reaching impact on SDC interoperability. A major issue as it may impact existing implementations and prototypes.

Issue

R1005 provides a straight-forward mechanism for (some) consumers to detect missed reports, independently of network protocol guarantees. Although this requirement isn't a mandate for consumers, any provider that does not comply risks incompatibility weakening SDC interoperability. Moreover, R1005 describes a mechanism that can be used only by consumers that subscribe to every report, including those that may be added in the future.

Partial subscribers have no straight-forward mechanism (beyond network protocol behaviours) to ensure reports aren't missed. During early adoption it may be common for consumers to subscribe to all reports, however partial subscription may become more common as the ecosystem evolves to, for example, reduce network traffic or participant processing load. A consumer that doesn't use remote operations may have no interest in the operation invoked report, an alarm panel may not wish to process waveform streams, and even now, many consumers don't subscribe to periodic reports.

Finally, the proper order for StateEventService/SystemErrorReport, SetService/OperationInvokedReport, WaveformService/ObservedValueStream, periodic and episodic reports is not yet clear (R1004, R1006, R1007).

Note

11073-10207, §5.2.5.6 requires the MDIB version to increase monotonically; only SDPi requires MDIB version to increase by exactly one for each report.

Proposed Change

To resolve these problems, I propose replacing R1005 with a mechanism that separates lost message detection from MDIB versioning and can work for all consumers. R1007 already requires MdibVersion to be strictly increasing so R1005 can focus on message sequencing. For example, a report sequence could, like the MDIB version [11073-10207, §5.2.5.6] follow the WS-Discovery's AppSequence concept:

---

R1005 (proposed)
A SOMDS provider MUST include a d:AppSequence element in every report header. The normative outline for the application sequence header block is:

<s:Envelope ...>
  <s:Header ...>
    <d:AppSequence InstanceId="xs:unsignedInt"
                  [SequenceId="xs:anyURI"]?
                   MessageNumber="xs:unsignedInt"
                   ... />
  </s:Header>
  <s:Body ...> ... </s:Body>
</s:Envelope>

In keeping with the normative constraints from ws-discovery:

• /s:Envelope/s:Header/d:AppSequence/@InstanceId
  MUST be incremented by a positive value (>= 1) each time the subscription to which the report applies ends and MUST NOT be incremented otherwise (to enable detection of lost messages). Means to set this value include, but are not limited to:
  • a counter that is incremented each time the provider starts any new subscription,
  • the time that the subscription begins.
• /s:Envelope/s:Header/d:AppSequence/@SequenceId
  Optional; has no specific meaning for SOMDS provider reports, however, in keeping with the constraints from ws-discovery the sequence id, if included:
  • MUST be unique within @InstanceId,
  • MUST be compared per RFC 3986, §6.2.1 simple string comparison
• /s:Envelope/s:Header/d:AppSequence/@MessageNumber
  Identifies a message within the context of a subscription, InstanceId and SequenceId. It
  • MUST be incremented by exactly 1 for every report acknowledged by a consumer with an HTTP response code and not otherwise (e.g., only acknowledged reports matter).
  • MUST start at 1 for the first message sent in a subscription (e.g., so consumer may, optionally, identify missing first message).
  • MUST never decrease (e.g., so consumer may, optionally, identify missing messages).
  • Transport-level retransmission MUST preserve this value (e.g., for proxy transport between networks).

A SOMDS consumer should reconnect or go into a fail-safe mode when it receives a report with an

• d:AppSequence/@MessageNumber that is either lower than the last received version or more than one version higher than the last received version (e.g., report inexplicably lost).
• d:AppSequence/@InstanceId that is not the same as the InstanceId received when the MessageNumber was 1 (e.g., doesn't meet constraints of AppSequence element),
• d:AppSequence/@MessageNumber that is not 1 for the first message received from a new subscription (e.g., missed the first message in a subscription).

A SOMDS consumer that receives a second copy of a report with a MessageNumber and InstanceId equal to the previous message expected (e.g., communications were interrupted between report and response delivery leaving provider and consumer out of sync):

• Should acknowledge receipt with an HTTP response code (e.g., to support efficient fault recovery without creating a new subscription) or terminate the subscription and/or enter fail-safe mode (e.g., for recovery by rebuilding subscription/ history service).
• Should discard the report (e.g., duplicate processing is not required).

A SOMDS consumer that receives two copies of a report with the same MessageNumber and InstanceId may assume those messages are identical in every respect without additional consideration (e.g., to simplify report processing by consumer).

---

This problem was presented and discussed in a session at plug-a-thon 16. A survey following the presentation found most survey participants wanted more information to know if it was worthwhile to separate message versioning and sequencing and only 25% thought it wouldn't be worth the effort for consumers. There was greater support for a custom sequence element in the soap header than other options. However, the AppSequence element from ws-discovery provides a ready-made solution, with no disadvantages identified so far, that may be easier to incorporate into SDPi than a custom extension.

[ToddCooper]

SDPi Friday 2024.10.11 Review - Added Assignee & Milestone 2.0.

[ToddCooper]

SDPi Workshop #4 -

@d-gregorczyk - Consideration for proposal? Is this 2.0 or later?

[d-gregorczyk]

This is something that needs approval by the community as it is a breaking change to all existing implementations.

[PaulMartinsen]

Note: the AppSequence header has some precedence for this use case in SOAP-over-UDP multicast stream binding (20702:§8.5.4, R0027).