Export of Delay Performance Metrics in IP Flow Information eXport (IPFIX)

Introduction Network operators usually maintain statistical views of delay accross their networks to support diagnostics and performance analysis. These views assist in identifying the location, extent, and potential causes of abnormal delay affecting specific customer traffic or services. To achieve this, delay-related metrics need to be reported from devices covering both data and control planes. Further, in order to understand which customers are affected, delay-related metrics need to be reported in the context of the customer data-plane. This correlation enables the detection of changes in forwarding paths, such as updated intermediate hops or interfaces, and the resulting impact on delay experienced by customer traffic. Delay measurements in the network are computed using an On-Path Telemetry protocol, which inserts metadata into the data-plane packet when entering the monitored domain . To compute delay measurements, the On-Path Telemetry protocol inserts a timestamp reference when entering the OAM encapsulating node. Implementations examples are In Situ Operations, Administration, and Maintenance (IOAM) or Enhanced Alternate Marking . Two modes of On-Path Telemetry are generally recognized: passport mode, in which only the OAM header decapsulating node of the OAM domain reports metrics; and postcard mode, in which OAM header transit nodes also export On-Path Telemetry data. Both modes enable exposure of per-hop performance metrics, including delay accumulation. The approach defined in this document is primarily applicable to postcard mode. To enable the export of the delay-related metrics via IPFIX , this document defines four new IPFIX Information Elements (IEs), exposing the On-Path delay on OAM header transit and decapsulating nodes, following the postcard mode principles. This enables the computation of delay metrics (minimum, maximum, and mean) directly on the OAM header transit and decapsulating node, allowing aggregation within the Flow Record. As these IEs represent performance metrics, they are also registered in the IANA "Performance Metrics" registry in accordance with .

Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. This document defines the following terms:

OAM Header Encapsulating Node: Receives the IP packets, encapsulates the packets with an OAM header and adds the timestamp into the OAM header.
OAM Header Transit Node: Receives the IP packets, measures the delay between the timestamp in the packet and the timestamp when the packet was received.
OAM Header Decapsulating Node: Receives the IP packets, computes the delay between the timestamp in the packet and the timestamp when the packet was received and removes the OAM header from the packet.

This document makes use of the terms defined in , and . The following terms are used as defined in :

Collector
Exporter
Flow
Flow Record
IPFIX
IPFIX Information Elements (IEs)
Observation Point

The following terms are used as defined in :

Performance Metric
Performance Metrics Registry
Registered Performance Metric

The following term is used as defined in :

Hybrid Type I

Solution In line with the guidelines for Registered Performance Metric Requesters and Reviewers , each metric is specified with its required characteristics (e.g., Identifier, Name, URI, Status, Requester, Revision, Description) to ensure comparability of measurement results across implementations and environments. These characteristics are registered in the IANA "Performance Metrics" registry. Metric naming follows the "MetricType_Method_SubTypeMethod_... Spec_Units_Output" convention defined in . This document defines the following performance metrics and IPFIX Information Elements: Assuming time synchronization on devices, the delay is measured by calculating the difference between the timestamp imposed with On-Path Telemetry in the packet at an OAM header encapsulating node and the timestamp exported in the IPFIX flow record from the OAM header transit and OAM header decapsulating nodes. The lowest, highest, mean, and the sum of measured path delay can be exported, thanks to the different IPFIX IE specifications.

Delay use case. Packets flow from host 1 to host 2. . . . . D2 . . x--------------------> . . . . D3 . . x----------------------------------> . . . (H1) -----> (R0) ------> (R1) ------> (R2) -------> (R3) -----> (H2) Host 1 Encapsulating Transit Transit Decapsulating Host 2 Node Node 1 Node 2 Node . . . . ......................................... ]]> In the use case shown in using On-path Telemetry to export the delay metrics, the node R1 exports the delay D1, the node R2 exports the delay D2 and the OAM header decapsulating node R3 exports the total delay D3 for the same flow using IPFIX. This solution enables the computation of delay metrics (minimum, maximum, and mean) directly on the OAM header transit and decapsulating node, allowing aggregation within the Flow Record. This reduces both export bandwidth and processing requirements on the Collector. To compute these metrics locally, the Exporter's Metering Process must perform per-packet caching and processing, particularly when computing mean delay under Flow Aggregation . A less computationally intensive alternative is to export the sum of delays, allowing the Collector to compute the mean via a simple division using the packet count. In contrast, if no delay processing occurs on the OAM header transit or decapsulating node, each packet must be exported as an individual Flow Record, including timestamp information, as specified in . The Collector must then compute the delay metrics and reconstruct the aggregated Flow Record accordingly.

Performance Metrics This section defines four new performance metrics following the template defined in . IANA Note (to be removed): RFC 8192 Section 4 was taken a guiding example.

IP One-Way Delay Hybrid Type I Performance Metrics This section specifies four performance metrics for the Hybrid Type I assessment of IP One-Way Delay, to be registered in the IANA "Performance Metrics" registry. All column entries besides the Identifier, Name, URI, Description, Reference Description (Output only) categories are the same; thus, this section defines four closely related performance metrics. As a result, IANA has assigned corresponding URIs to each of the four registered performance metrics.

Summary This category includes multiple indexes of the registered performance metrics: the element Identifier and Metric Name.

ID (Identifier) IANA has allocated the numeric Identifiers TBD1, TBD2, TBD3, and TBD4 for the four Named Metric Entries in the following section. RFC EDITOR NOTE: please replace TBD1, TBD2, TBD3, and TBD4 with allocated IPFIX entity numbers.

Name TBD1: OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Mean TBD2: OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Min TBD3: OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Max TBD4: OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Sum RFC EDITOR NOTE: please replace [RFC-to-be] with the allocated RFC document number.

URI URI: URI: URI: URI: RFC EDITOR NOTE: please replace [RFC-to-be] with the allocated RFC document number.

Description

OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Mean: This metric assesses the mean of one-way delays of all successfully forwarded IP packets constituting a single Flow. The measurement of one-way delay based on a single Observation Point [RFC7011] somewhere in the network.
OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Min: This metric assesses the minimum of one-way delays of all successfully forwarded IP packets constituting a single Flow. The measurement of one-way delay based on a single Observation Point [RFC7011] somewhere in the network.
OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Max: This metric assesses the maximum of one-way delays of all successfully forwarded IP packets constituting a single Flow. The measurement of one-way delay based on a single Observation Point [RFC7011] somewhere in the network.
OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Sum: This metric assesses the sum of one-way delays of all successfully forwarded IP packets constituting a single Flow. The measurement of one-way delay based on a single Observation Point [RFC7011] somewhere in the network.

RFC EDITOR NOTE: please replace [RFC-to-be] with the allocated RFC document number.

Reference [RFC-to-be] RFC EDITOR NOTE: please replace [RFC-to-be] with the allocated RFC document number.

Change Controller IETF

Version of Registry Format 1.0

Metric Definition This category includes columns to prompt the entry of all necessary details related to the metric definition, including the immutable document reference and values of input factors, called "Fixed Parameters".

Reference Definition Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, Ed., "A One-Way Delay Metric for IP Performance Metrics (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January 2016, <https://www.rfc-editor.org/info/rfc7679>. Morton, A. and E. Stephan, "Spatial Composition of Metrics" , RFC 6049, DOI 10.17487/RFC6049, January 2011, <https://www.rfc-editor.org/info/rfc6049>. provides the reference definition of the singleton (single value) one-way delay metric. provides the reference definition expanded to cover a multi-value sample. Note that terms such as "singleton" and "sample" are defined in . With the Observation Point typically located between the hosts participating in the IP Flow, the one-way delay metric requires one individual measurement between the Observation Point and sourcing host, such that the Spatial Composition of the measurements yields a one-way delay singleton. This document specifies how to export the performance metric using IPFIX.

Fixed Parameters None

Method of Measurement This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous method for implementations.

Reference Methods The foundational methodology for this metric is defined in using the Timestamps option with modifications that allow application at a mid-path Observation Point .

Packet Stream Generation The time when the packet is being received at the OAM header encapsulating node. The timestamp format depends on the On-Path Telemetry implementation. For IOAM, describes the supported timestamps. Sections 4.4.2.3 and 4.4.2.4 describe where the timestamp is being inserted. For the Enhanced Alternate Marking Method, and defines the supported timestamp encodings and granularity.

Traffic Filtering (Observation) Details Runtime Parameters (in the following sections) may be used for Traffic Filtering.

Sampling Distribution This metric requires a partial sample of all packets that qualify according to the Traffic Filter criteria.

Runtime Parameters and Data Format Runtime Parameters are input factors that must be determined, configured into a measurement system, and reported with the results for the context to be complete. The Hybrid Type I metering parameters must be reported to provide the complete measurement context. As an example, if the IPFIX Metering Process is used, then the IPFIX Metering Process parameters (IPFIX Template Record, potential traffic filters, and potential sampling method and parameters) that generate the Flow Records must be reported to provide the complete measurement context. At a minimum, the following fields are required:

Src:: The IP address of the host in the host A Role (format ipv4‑address-no-zone value for IPv4 or ipv6-address-no-zone value for IPv6; see ).
Dst:: The IP address of the host in the host B Role (format ipv4‑address-no-zone value for IPv4 or ipv6-address-no-zone value for IPv6; see ).
T0:: T time, the start of a measurement interval (format "date/time" as specified in ; see also "date-and-time" in ). The UTC Time Zone is required by . When T0 is "all-zeros", a start time is unspecified and Tf is to be interpreted as the duration of the measurement interval. The start time is controlled through other means.
Tf:: A time, the end of a measurement interval (format "date/time" as specified in ; see also "date-and-time" in ). The UTC Time Zone is required by . When T0 is "all-zeros", an ending time and date is ignored and Tf is interpreted as the duration of the measurement interval.

Roles

host A:: Launches an IP packet to start the Flow.
host B:: Receives the IP packet to start the Flow.
OAM Header Encapsulating Node:: Receives the IP packets, encapsulates the packets with the OAM header and adds the timestamp into the OAM header.
OAM Header Transit Node:: Receives the IP packets, measures the delay between the timestamp in the packet and the timestamp when the packet was received.
OAM Header Decapsulating Node:: Receives the IP packets, computes the delay between the timestamp in the packet and the timestamp when the packet was received and removes the OAM header from the packet.

Output This category specifies all details of the output of measurements using the metric.

Type OWDelay Types are discussed in the subsections below.

Reference Definition For all output types:

OWDelay_HybridType1_IP:: The one-way delay of one IP packet is a Singleton

For each <statistic> Singleton one of the following subsections applies.

OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Mean Similar to , the mean SHALL be calculated using the conditional distribution of all packets with a finite value of one-way delay (undefined delays are excluded) -- a single value, as follows: See for details on the conditional distribution to exclude undefined values of delay, and see for background on this analysis choice. See for details on calculating this statistic; see also .

Mean:: The time value of the result is expressed in units of microseconds, as a positive value of type decimal64 with fraction digits = 9 (similar to the decimal64 in YANG, ) with a resolution of 0.001 microseconds (1.0 ns), and with lossless conversion to/from the 64-bit NTP timestamp as per .

OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Min Similar to , the minimum SHALL be calculated using the conditional distribution of all packets with a finite value of one-way delay (undefined delays are excluded) -- a single value, as follows: See for details on the conditional distribution to exclude undefined values of delay, and see for background on this analysis choice. See for details on calculating this statistic; see also .

Min:: The time value of the result is expressed in units of microseconds, as a positive value of type decimal64 with fraction digits = 9 (similar to the decimal64 in YANG, ) with a resolution of 0.001 microseconds (1.0 ns), and with lossless conversion to/from the 64-bit NTP timestamp as per .

OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Max Similar to , the maximum SHALL be calculated using the conditional distribution of all packets with a finite value of one-way delay (undefined delays are excluded) -- a single value, as follows: See for details on the conditional distribution to exclude undefined values of delay, and see for background on this analysis choice. See for a closely related method for calculating this statistic; see also . The formula is as follows: = FiniteDelay[n] for all n ]]> where all packets n = 1 through N have finite singleton delays.

Max:: The time value of the result is expressed in units of microseconds, as a positive value of type decimal64 with fraction digits = 9 (similar to the decimal64 in YANG, ) with a resolution of 0.001 microseconds (1.0 ns), and with lossless conversion to/from the 64-bit NTP timestamp as per .

OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Sum The sum SHALL be calculated using the conditional distribution of all packets with a finite value of one-way delay (undefined delays are excluded) -- a single value, as follows: See for details on the conditional distribution to exclude undefined values of delay, and see for background on this analysis choice. See for details on calculating this statistic, however in this case FiniteDelay or MaxDelay MAY be used.

Sum:: The time value of the result is expressed in units of microseconds, as a positive value of type decimal64 with fraction digits = 9 (similar to the decimal64 in YANG, ) with a resolution of 0.001 microseconds (1.0 ns), and with lossless conversion to/from the 64-bit NTP timestamp as per .

Metric Units

Mean
Min
Max
Sum

The one-way delay of the IP Flow singleton is expressed in microseconds.

Calibration A clock synchronization between the nodes of the monitored OAM domain is needed to compute representative delay measurements at the OAM header transit and decapsulating nodes. NTP, as defined in , can be used for synchronizing the clocks of the monitored nodes.

Administrative Items

Status Current

Requester RFC[RFC-to-be] RFC EDITOR NOTE: please replace [RFC-to-be] with the allocated RFC document number and [RFC-date] with the date when the RFC has been published.

Revision 1.0

Revision Date RFC[RFC-date]

Comments and Remarks none

Use Cases The measured On-Path delay can be aggregated with Flow Aggregation as defined in to the following device and control-plane dimensions to determine:

With node id and egressInterface(14), on which node which logical egress interfaces have been contributing to how much delay.
With node id and egressPhysicalInterface(253), on which node which physical egress interfaces have been contributing to how much delay.
With ipNextHopIPv4Address(15) or ipNextHopIPv6Address(62), the forwarding path to which next-hop IP contributed to how much delay.
With mplsTopLabelIPv4Address(47) or destinationIPv6Address and srhActiveSegmentIPv6(495), the forwarding path to which MPLS top label IPv4 address or IPv6 destination address and SRv6 active segment contributed to how much delay.
BGP communities are often used for setting a path priority or service selection. With bgpDestinationExtendedCommunityList(488) or bgpDestinationCommunityList(485) or bgpDestinationLargeCommunityList(491) which group of prefixes accumulated at which node how much delay.
With destinationIPv4Address(13), destinationTransportPort(11), protocolIdentifier (4) and sourceIPv4Address(8), or equivalent IPFIX IEs for IPv6, the forwarding path delay on each node from each IPv4 source address to a specific application in the network.

Let us consider the example depicted in Figure 1 from Section 1 as topology example. Below example table shows the aggregated delay per each node, ingressInterface,(10) egressInterface(14), destinationIPv6Address(28) and srhActiveSegmentIPv6(495) measured at ingress.

IANA Considerations

Performance Metrics This document requests IANA to add four new performance metrics under the "Performance Metrics" registry with the four templates defined in Section 3.

IPFIX Entities This document requests IANA to register new IPFIX IEs (see table 3) under the "IPFIX Information Elements" registry under the "IP Flow Information Export (IPFIX) Entities" registry group and assign the following initial code points. Note to the RFC-Editor:

Please replace TBD5 - TBD8 with the values allocated by IANA
Please replace all instances of [RFC-to-be] in this section with the RFC number assigned to this document

pathDelayMeanDeltaMicroseconds

Name:: pathDelayMeanDeltaMicroseconds

ElementID:: TBD5

Description:: This Information Element identifies the mean path delay of all packets in the Flow, in microseconds, between an OAM header encapsulating node and the local node with the OAM domain (either an OAM header transit node or an OAM header decapsulating node), according to OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Mean in the IANA Performance Metric Registry.

Abstract Data Type:: unsigned32

Data Type Semantics:: deltaCounter

Reference:: [RFC-to-be]

Additional Information:: OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Mean in the IANA Performance Metric Registry.

pathDelayMinDeltaMicroseconds

Name:: pathDelayMinDeltaMicroseconds

ElementID:: TBD6

Description:: This Information Element identifies the lowest path delay of all packets in the Flow, in microseconds, between an OAM header encapsulating node and the local node with the OAM domain (either an OAM header transit node or an OAM header decapsulating node), according to the OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Min in the IANA Performance Metric Registry.

Abstract Data Type:: unsigned32

Data Type Semantics:: deltaCounter

Reference:: [RFC-to-be]

Additional Information:: OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Min in the IANA Performance Metric Registry.

pathDelayMaxDeltaMicroseconds

Name:: pathDelayMaxDeltaMicroseconds

ElementID:: TBD7

Description:: This Information Element identifies the highest path delay of all packets in the Flow, in microseconds, between an OAM header encapsulating node and the local node with the OAM domain (either an OAM header transit node or an OAM header decapsulating node), according to OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Max in the IANA Performance Metric Registry.

Abstract Data Type:: unsigned32

Data Type Semantics:: deltaCounter

Reference:: [RFC-to-be]

Additional Information:: OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Max in the IANA Performance Metric Registry.

pathDelaySumDeltaMicroseconds

Name:: pathDelaySumDeltaMicroseconds

ElementID:: TBD8

Description:: This Information Element identifies the sum of the path delay of all packets in the Flow, in microseconds, between an OAM header encapsulating node and the local node with the OAM domain (either an OAM header transit node or an OAM header decapsulating node), according to OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Sum in the IANA Performance Metric Registry.

Abstract Data Type:: unsigned64

Data Type Semantics:: deltaCounter

Reference:: [RFC-to-be]

Additional Information:: OWDelay_HybridType1_IP_RFC[RFC-to-be]_Seconds_Sum in the IANA Performance Metric Registry.

Operational Considerations

Time Accuracy The same recommendation as defined in for IPFIX applies in terms of clock precision to this document as well.

Mean Delay The mean (average) path delay can be calculated by dividing the pathDelaySumDeltaMicroseconds(TBD8) by the packetDeltaCount(2) at the IPFIX data collection in order to offload the IPFIX Exporter from calculating the mean for every Flow at export time.

Reduced-size encoding Unsigned64 has been chosen as type for pathDelaySumDeltaMicroseconds to support cases with large delay numbers and where many packets are being accounted. As an example, a specific Flow Record with path delay of 100 milliseconds cannot observe more than 42949 packets without overflowing the unsigned32 counter. The procedure described in may be applied to reduce network bandwidth between the IPFIX Exporter and Collector if unsigned32 would be large enough without wrapping around.

Measurement Interval The delay metrics are computed for the Flow Record life time by comparing the OAM timestamps in each received packet with the timestamp when they were received. For long-running Flow, the IPFIX Metering Process might miss the temporal distribution of the delay (for example, a longer delay only at the beginning of Flow). If this is an operational problem, the IPFIX Metering Process might be configured with a smaller expiration timeout (see Section 5.1.1. Flow Expiration ).

In-Packet OAM Application Multiple methods can be used to compute the delay performance metrics defined in this document. Some examples of such methods are IOAM and Enhanced Alternate Marking . For IOAM, these performance metrics can be computed using the Edge-to-Edge and the Direct Exporting Option-Type. IOAM Edge-to-Edge Option-Type, as described in , can use bits 2 and 3. In this case, timestamps are encoded as defined in Sections 4.4.2.3 and 4.4.2.4 of . This timestamp can be used to compute the delay between an OAM header encapsulating node and the decapsulating node. IOAM Direct Exporting Option-Type, as described in , can use the Extension-Flag defined in to insert a timestamp in the OAM header encapsulating node. The timestamp is encoded as defined in Sections 4.4.2.3 and 4.4.2.4 of . This timestamp can be used to compute the delay between the inserted timestamp and the OAM header transit and decapsulating node. For the Enhanced Alternate Marking Method, and defines that, within the metaInfo, a nanosecond timestamp can be encoded in an OAM header encapsulating node and be read at the OAM header intermediate and decapsulating node to calculate the on-path delay. defines how this can be applied to the IPv6 extensions header and defines how this can be applied to the SRv6 Segment Routing Header . Given that the delay measurements are computed with the timestamp introduced on the OAM header encapsulating node, regardless of the approach, implementations should document at which point of the forwarding plane this timestamp is introduced (e.g. the time at which the packet was received by the node, the time at which the packet was transmitted by the node, etc.). Based on this information, different actions can be taken.

Security Considerations The IPFIX Information Elements introduced in this document do not directly introduce security issues. Rather, they define a set of performance metrics that may, for privacy or business issues, be considered sensitive information. For example, exporting delay metrics may make attacks possible for the receiver of this information; this would otherwise only be possible for direct observers of the reported Flows along the data path. IPFIX collectors MUST ensure that IPFIX data originates from trusted sources. Accepting IPFIX data from unauthenticated sources could lead to data spoofing, policy misapplication, or denial of service. The underlying protocol used to exchange the information described here must therefore apply appropriate procedures to guarantee the integrity and confidentiality of the exported information. These protocols are defined in separate documents, specifically the IPFIX security considerations . Implementations SHOULD also refer to for additional details.

Implementation Status Note to the RFC-Editor: Please remove this section before publishing.

FD.io VPP INSA Lyon implemented the following IEs as part of a prototype in the FD.io VPP (Vector Packet Processing) platform:

pathDelayMeanDeltaMicroseconds
pathDelayMaxDeltaMicroseconds
pathDelayMinDeltaMicroseconds
pathDelaySumDeltaMicroseconds

The open source code can be obtained here: and was validated at the IETF 116 hackathon.

Huawei VRP Huawei implemented the following IEs as part of a production implementation in the VRP platform:

pathDelayMeanDeltaMicroseconds
pathDelayMaxDeltaMicroseconds
pathDelayMinDeltaMicroseconds
pathDelaySumDeltaMicroseconds

The implementation was validated at the IETF 116 hackathon.

Fluvia NTT Com implemented the following IEs in the Fluvia Exporter:

pathDelayMeanDeltaMicroseconds
pathDelayMaxDeltaMicroseconds
pathDelayMinDeltaMicroseconds
pathDelaySumDeltaMicroseconds

The open source code can be obtained here: and was validated at the IETF 118 hackathon.

Pmacct Data Collection Paolo Lucente implemented the IE pathDelayMeanDeltaMicroseconds by dividing IE pathDelaySumDeltaMicroseconds by IE packetDeltaCount in the open source Network Telemetry data collection project pmacct. The source code can be obtained here: and was validated at the IETF 116 hackathon.

Acknowledgements The authors would like to thank Al Morton (Rest in Peace, Al), Justin Iurman, Giuseppe Fioccola, Yannick Buchs, Menachem Dodge Martin Duke, Behcet Sarikaya, Mahesh Jethanandani, Linda Dunbar, Deb Cooley, Mike Bishop, Tim Wicinski, Gunter Van de Velde and Éric Vyncke's for their review and valuable comments. Special thanks to Paul Aitken (as IPFIX Designated Expert), Greg Mirsky (as IP Performance Metrics Designated Expert), and to Med Boucadair for their very detailed feedback.

References Normative References Informative References IANA Performance Metric Registry IANA IP Flow Information Export (IPFIX) Entities Registry INSA Lyon, FD.io VPP implementation NTT Com, Fluvia Exporter Paolo Lucente, Pmacct open source Network Telemetry Data Collection

IPFIX Encoding Examples This appendix represents two different encodings for the newly introduced IEs. Taking Figure 1 from Section 1 as topology example. Below example Table 4 shows the aggregated delay with ingressInterface, egressInterface, destinationIPv6Address and srhActiveSegmentIPv6.

Aggregated On-Path Delay Examples

Template Record and Data Set with Mean Delta With encoding in Figure 2, the mean (average) path delay is calculated on the exporting node.

Ingress interface => ingressInterface
Egress interface => egressInterface
IPv6 destination address => destinationIPv6Address
Active SRv6 Segment => srhActiveSegmentIPv6
Packet Delta Count => packetDeltaCount
Minimum One-Way Delay => pathDelayMinDeltaMicroseconds (TBD6)
Maximum One-Way Delay => pathDelayMaxDeltaMicroseconds (TBD7)
Mean One-Way Delay => pathDelayMeanDeltaMicroseconds (TBD5)

Template Record for pathDelayMeanDeltaMicroseconds. The data set is represented as follows:

Data Set Encoding for pathDelayMeanDeltaMicroseconds.

Template Record and Data Set with Sum Delta With encoding in , the mean (average) path delay is calculated on the IPFIX data collection.

Ingress interface => ingressInterface
Egress interface => egressInterface
IPv6 destination address => destinationIPv6Address
Active SRv6 Segment => srhActiveSegmentIPv6
Packet Delta Count => packetDeltaCount
Minimum One-Way Delay => pathDelayMinDeltaMicroseconds (TBD6)
Maximum One-Way Delay => pathDelayMaxDeltaMicroseconds (TBD7)
Sum of One-Way Delay => pathDelaySumDeltaMicroseconds (TBD8)

Template Record for pathDelaySumDeltaMicroseconds. The data set is represented as follows:

Data Set Encoding for pathDelaySumDeltaMicroseconds