rfc9912.original   rfc9912.txt 
DetNet P. Thubert, Ed. Internet Engineering Task Force (IETF) P. Thubert, Ed.
Internet-Draft Without Affiliation Request for Comments: 9912 February 2026
Intended status: Informational 25 July 2025 Category: Informational
Expires: 26 January 2026 ISSN: 2070-1721
Reliable and Available Wireless Architecture Reliable and Available Wireless (RAW) Architecture
draft-ietf-raw-architecture-30
Abstract Abstract
Reliable and Available Wireless (RAW) extends the reliability and Reliable and Available Wireless (RAW) extends the reliability and
availability of DetNet to networks composed of any combination of availability of Deterministic Networking (DetNet) to networks
wired and wireless segments. The RAW Architecture leverages and composed of any combination of wired and wireless segments. The RAW
extends RFC 8655, the Deterministic Networking Architecture, to adapt architecture leverages and extends RFC 8655 ("Deterministic
to challenges that affect prominently the wireless medium, notably Networking Architecture") to adapt to challenges that prominently
intermittent transmission loss. This document defines a network affect the wireless medium, notably intermittent transmission loss.
control loop that optimizes the use of constrained bandwidth and This document defines a network control loop that optimizes the use
energy while assuring the expected DetNet services. The loop of constrained bandwidth and energy while ensuring the expected
involves a new Point of Local Repair (PLR) function in the DetNet DetNet services. The loop involves a new Point of Local Repair (PLR)
Service sub-layer that dynamically selects the DetNet path(s) for function in the DetNet Service sub-layer that dynamically selects the
packets to route around local connectivity degradation. DetNet path(s) for packets to route around local connectivity
degradation.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This document is not an Internet Standards Track specification; it is
provisions of BCP 78 and BCP 79. published for informational purposes.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months This document is a product of the Internet Engineering Task Force
and may be updated, replaced, or obsoleted by other documents at any (IETF). It represents the consensus of the IETF community. It has
time. It is inappropriate to use Internet-Drafts as reference received public review and has been approved for publication by the
material or to cite them other than as "work in progress." Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are candidates for any level of Internet
Standard; see Section 2 of RFC 7841.
This Internet-Draft will expire on 26 January 2026. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
https://www.rfc-editor.org/info/rfc9912.
Copyright Notice Copyright Notice
Copyright (c) 2025 IETF Trust and the persons identified as the Copyright (c) 2026 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/ Provisions Relating to IETF Documents
license-info) in effect on the date of publication of this document. (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
Please review these documents carefully, as they describe your rights carefully, as they describe your rights and restrictions with respect
and restrictions with respect to this document. Code Components to this document. Code Components extracted from this document must
extracted from this document must include Revised BSD License text as include Revised BSD License text as described in Section 4.e of the
described in Section 4.e of the Trust Legal Provisions and are Trust Legal Provisions and are provided without warranty as described
provided without warranty as described in the Revised BSD License. in the Revised BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction
2. The RAW problem . . . . . . . . . . . . . . . . . . . . . . . 4 2. The RAW Problem
3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 7 3. Terminology
3.1. Acronyms . . . . . . . . . . . . . . . . . . . . . . . . 8 3.1. Abbreviations
3.1.1. ARQ . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.1.1. ARQ
3.1.2. FEC . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.1.2. FEC
3.1.3. HARQ . . . . . . . . . . . . . . . . . . . . . . . . 9 3.1.3. HARQ
3.1.4. ETX . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.1.4. ETX
3.1.5. ISM . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.1.5. ISM
3.1.6. PER and PDR . . . . . . . . . . . . . . . . . . . . . 9 3.1.6. PER
3.1.7. RSSI . . . . . . . . . . . . . . . . . . . . . . . . 10 3.1.7. PDR
3.1.8. LQI . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.1.8. RSSI
3.1.9. OAM . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.1.9. LQI
3.1.10. OODA . . . . . . . . . . . . . . . . . . . . . . . . 10 3.1.10. OAM
3.1.11. SNR . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.1.11. OODA
3.2. Link and Direction . . . . . . . . . . . . . . . . . . . 10 3.1.12. SNR
3.2.1. Flapping . . . . . . . . . . . . . . . . . . . . . . 11 3.2. Link and Direction
3.2.2. Uplink . . . . . . . . . . . . . . . . . . . . . . . 11 3.2.1. Flapping
3.2.3. Downlink . . . . . . . . . . . . . . . . . . . . . . 11 3.2.2. Uplink
3.2.4. Downstream . . . . . . . . . . . . . . . . . . . . . 11 3.2.3. Downlink
3.2.5. Upstream . . . . . . . . . . . . . . . . . . . . . . 11 3.2.4. Downstream
3.3. Path and Recovery Graphs . . . . . . . . . . . . . . . . 11 3.2.5. Upstream
3.3.1. Path . . . . . . . . . . . . . . . . . . . . . . . . 11 3.3. Path and Recovery Graphs
3.3.2. Recovery Graph . . . . . . . . . . . . . . . . . . . 12 3.3.1. Path
3.3.3. Forward and Crossing . . . . . . . . . . . . . . . . 15 3.3.2. Recovery Graph
3.3.4. Protection Path . . . . . . . . . . . . . . . . . . . 15 3.3.3. Forward and Crossing
3.3.5. Segment . . . . . . . . . . . . . . . . . . . . . . . 15 3.3.4. Protection Path
3.4. Deterministic Networking . . . . . . . . . . . . . . . . 15 3.3.5. Segment
3.4.1. The DetNet Planes . . . . . . . . . . . . . . . . . . 15 3.4. Deterministic Networking
3.4.2. Flow . . . . . . . . . . . . . . . . . . . . . . . . 16 3.4.1. The DetNet Planes
3.4.3. Residence Time . . . . . . . . . . . . . . . . . . . 16 3.4.2. Flow
3.4.4. L3 Deterministic Flow Identifier . . . . . . . . . . 16 3.4.3. Residence Time
3.4.5. TSN . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.4.4. L3 Deterministic Flow Identifier
3.4.6. Lower-Layer API . . . . . . . . . . . . . . . . . . . 16 3.4.5. Time-Sensitive Networking
3.5. Reliability and Availability . . . . . . . . . . . . . . 17 3.4.6. Lower-Layer API
3.5.1. Service Level Agreement . . . . . . . . . . . . . . . 17 3.5. Reliability and Availability
3.5.2. Service Level Objective . . . . . . . . . . . . . . . 17 3.5.1. Service Level Agreement
3.5.3. Service Level Indicator . . . . . . . . . . . . . . . 17 3.5.2. Service Level Objective
3.5.4. Precision Availability Metrics . . . . . . . . . . . 17 3.5.3. Service Level Indicator
3.5.5. Reliability . . . . . . . . . . . . . . . . . . . . . 17 3.5.4. Precision Availability Metrics
3.5.6. Availability . . . . . . . . . . . . . . . . . . . . 18 3.5.5. Reliability
4. Reliable and Available Wireless . . . . . . . . . . . . . . . 18 3.5.6. Availability
4.1. High Availability Engineering Principles . . . . . . . . 18 4. Reliable and Available Wireless
4.1.1. Elimination of Single Points of Failure . . . . . . . 18 4.1. High Availability Engineering Principles
4.1.2. Reliable Crossover . . . . . . . . . . . . . . . . . 19 4.1.1. Elimination of Single Points of Failure
4.1.3. Prompt Notification of Failures . . . . . . . . . . . 20 4.1.2. Reliable Crossover
4.2. Applying Reliability Concepts to Networking . . . . . . . 20 4.1.3. Prompt Notification of Failures
4.3. Wireless Effects Affecting Reliability . . . . . . . . . 21 4.2. Applying Reliability Concepts to Networking
5. The RAW Conceptual Model . . . . . . . . . . . . . . . . . . 23 4.3. Wireless Effects Affecting Reliability
5.1. The RAW Planes . . . . . . . . . . . . . . . . . . . . . 23 5. The RAW Conceptual Model
5.2. RAW vs. Upper and Lower Layers . . . . . . . . . . . . . 25 5.1. The RAW Planes
5.3. RAW and DetNet . . . . . . . . . . . . . . . . . . . . . 26 5.2. RAW Versus Upper and Lower Layers
6. The RAW Control Loop . . . . . . . . . . . . . . . . . . . . 30 5.3. RAW and DetNet
6.1. Routing Time-Scale vs. Forwarding Time-Scale . . . . . . 31 6. The RAW Control Loop
6.2. OODA Loop . . . . . . . . . . . . . . . . . . . . . . . . 33 6.1. Routing Timescale Versus Forwarding Timescale
6.3. Observe: The RAW OAM . . . . . . . . . . . . . . . . . . 34 6.2. OODA Loop
6.4. Orient: The RAW-extended DetNet Operational Plane . . . . 36 6.3. Observe: RAW OAM
6.5. Decide: The Point of Local Repair . . . . . . . . . . . . 36 6.4. Orient: The RAW-Extended DetNet Operational Plane
6.6. Act: DetNet Path Selection and Reliability Functions . . 38 6.5. Decide: The Point of Local Repair
7. Security Considerations . . . . . . . . . . . . . . . . . . . 39 6.6. Act: DetNet Path Selection and Reliability Functions
7.1. Collocated Denial of Service Attacks . . . . . . . . . . 39 7. Security Considerations
7.2. Layer-2 encryption . . . . . . . . . . . . . . . . . . . 39 7.1. Collocated Denial-of-Service Attacks
7.3. Forced Access . . . . . . . . . . . . . . . . . . . . . . 40 7.2. Layer 2 Encryption
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 40 7.3. Forced Access
9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 40 8. IANA Considerations
10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 40 9. References
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 41 9.1. Normative References
11.1. Normative References . . . . . . . . . . . . . . . . . . 41 9.2. Informative References
11.2. Informative References . . . . . . . . . . . . . . . . . 42 Acknowledgments
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 45 Contributors
Author's Address
1. Introduction 1. Introduction
Deterministic Networking aims at providing bounded latency and Deterministic Networking (DetNet) aims to provide bounded latency and
eliminating congestion loss, even when co-existing with best-effort eliminate congestion loss, even when coexisting with best-effort
traffic. It provides the ability to carry specified unicast or traffic. It provides the ability to carry specified unicast or
multicast data flows for real-time applications with extremely low multicast data flows for real-time applications with extremely low
packet loss rates and assured maximum end-to-end delivery latency. A packet loss rates and ensures maximum end-to-end delivery latency. A
description of the general background and concepts of DetNet can be description of the general background and concepts of DetNet can be
found in [RFC8655]. found in [DetNet-ARCHI].
DetNet and the related IEEE 802.1 Time-Sensitive networking (TSN) DetNet and the related IEEE 802.1 Time-Sensitive Networking (TSN)
[TSN] initially focused on wired infrastructure, which provides a [TSN] initially focused on wired infrastructure, which provides a
more stable communication channel than wireless networks. Wireless more stable communication channel than wireless networks. Wireless
networks operate on a shared medium where uncontrolled interference, networks operate on a shared medium where uncontrolled interference,
including the self-induced multipath fading, may cause intermittent including self-induced multipath fading, may cause intermittent
transmission losses. Fixed and mobile obstacles and reflectors may transmission losses. Fixed and mobile obstacles and reflectors may
block or alter the signal, causing transient and unpredictable block or alter the signal, causing transient and unpredictable
variations of the throughput and packet delivery ratio (PDR) of a variations of the throughput and Packet Delivery Ratio (PDR) of a
wireless link. This adds new dimensions to the statistical effects wireless link. This adds new dimensions to the statistical effects
that affect the quality and reliability of the link. that affect the quality and reliability of the link.
Nevertheless, deterministic capabilities are required in a number of Nevertheless, deterministic capabilities are required in a number of
wireless use cases as well [RAW-USE-CASES]. With scheduled radios wireless use cases as well [RAW-USE-CASES]. With scheduled radios
such as Time Slotted Channel Hopping (TSCH) and Orthogonal Frequency such as Time-Slotted Channel Hopping (TSCH) and Orthogonal Frequency-
Division Multiple Access (OFDMA) (see [RAW-TECHNOS] for more on both Division Multiple Access (OFDMA) being developed to provide
of these and other technologies as well) being developed to provide
determinism over wireless links at the lower layers, providing DetNet determinism over wireless links at the lower layers, providing DetNet
capabilities has become possible. capabilities has become possible. See [RAW-TECHNOS] for more on
TSCH, OFDMA, and other technologies.
Reliable and Available Wireless (RAW) takes up the challenge of Reliable and Available Wireless (RAW) takes up the challenge of
providing highly available and reliable end-to-end performances in a providing highly available and reliable end-to-end performances in a
DetNet network that may include wireless segments. To achieve this, DetNet network that may include wireless segments. To achieve this,
RAW leverages all the possible transmission diversity and redundancy RAW leverages all possible transmission diversity and redundancy to
to assure packet delivery, while optimizing the use of the shared ensure packet delivery, while optimizing the use of the shared
spectrum to preserve bandwidth and save energy. To that effect, RAW spectrum to preserve bandwidth and save energy. To that effect, RAW
defines Protection Paths can be activated dynamically upon failures defines protection paths that can be activated dynamically upon
and a control loop that dynamically controls the activation and failures and a control loop that dynamically controls the activation
deactivation of the feasible Protection Paths to react quickly to and deactivation of the feasible protection paths to react quickly to
intermittent losses. intermittent losses.
The intent of RAW is to meet Service Level Objectives (SLO) in terms The intent of RAW is to meet Service Level Objectives (SLOs) in terms
of packet delivery ratio (PDR), maximum contiguous losses or latency of PDR, maximum contiguous losses, or latency boundaries for DetNet
boundaries for DetNet flows over mixes of wired and wireless flows over mixes of wired and wireless networks, including wireless
networks, including wireless access and meshes (see Section 2 for access and meshes (see Section 2 for more on the RAW problem). This
more on the RAW problem). This document introduces and/or leverages document introduces and/or leverages terminology (see Section 3),
terminology (see Section 3), principles (see Section 4), and concepts principles (see Section 4), and concepts such as protection paths and
such as protection path and recovery graph, to put together a recovery graphs to put together a conceptual model for RAW (see
conceptual model for RAW (see Section 5), and, based on that model, Section 5). Based on that model, this document elaborates on an in-
elaborate on an in-network optimization control loop (see Section 6). network optimization control loop (see Section 6).
2. The RAW problem 2. The RAW Problem
While the generic "Deterministic Networking Problem Statement" While the generic "Deterministic Networking Problem Statement"
[RFC8557] applies to both the wired and the wireless media, the [RFC8557] applies to both wired and wireless media, the
"Deterministic Networking Architecture" [DetNet-ARCHI] must be "Deterministic Networking Architecture" [DetNet-ARCHI] must be
extended to address less consistent transmissions, energy extended to address less consistent transmissions, energy
conservation, and shared spectrum efficiency. conservation, and shared spectrum efficiency.
Operating at Layer-3, RAW does not change the wireless technology at Operating at Layer 3, RAW does not change the wireless technology at
the lower layers. OTOH, it can further increase diversity in the the lower layers. On the other hand, it can further increase
spatial, time, code, and frequency domains by enabling multiple link- diversity in the spatial, time, code, and frequency domains by
layer wired and wireless technologies in parallel or sequentially, enabling multiple link-layer wired and wireless technologies in
for a higher resilience and a wider applicability. RAW can also parallel or sequentially, for a higher resilience and a wider
provide homogeneous services to critical applications beyond the applicability. RAW can also provide homogeneous services to critical
boundaries of a single subnetwork, e.g., using diverse radio access applications beyond the boundaries of a single subnetwork, e.g.,
technologies to optimize the end-to-end application experience. using diverse radio access technologies to optimize the end-to-end
application experience.
RAW extends the DetNet services by providing elements that are RAW extends the DetNet services by providing elements that are
specialized for transporting IP flows over deterministic radio specialized for transporting IP flows over deterministic radio
technologies such as listed in [RAW-TECHNOS]. Conceptually, RAW is technologies such as those listed in [RAW-TECHNOS]. Conceptually,
agnostic to the lower layer, though the capability to control latency RAW is agnostic to the lower layer, though the capability to control
is assumed to assure the DetNet services that RAW extends. How the latency is assumed to ensure the DetNet services that RAW extends.
lower layers are operated to do so, and, e.g., whether a radio How the lower layers are operated to do so (and whether a radio
network is single-hop or meshed, are opaque to the IP layer and not network is single hop or meshed, for example) are opaque to the IP
part of the RAW abstraction. Nevertheless, cross-layer optimizations layer and not part of the RAW abstraction. Nevertheless, cross-layer
may take place to ensure proper link awareness (think, link quality) optimizations may take place to ensure proper link awareness (such as
and packet handling (think, scheduling). link quality) and packet handling (such as scheduling).
The RAW Architecture extends the DetNet Network Plane, to accommodate The RAW architecture extends the DetNet Network Plane to accommodate
one or multiple hops of homogeneous or heterogeneous wired and one or multiple hops of homogeneous or heterogeneous wired and
wireless technologies. RAW adds reactivity to the DetNet Forwarding wireless technologies. RAW adds reactivity to the DetNet Forwarding
sub-layer to compensate the dynamics for the radio links in terms of sub-layer to compensate the dynamics for the radio links in terms of
lossiness and bandwidth. This may apply, for instance, to mesh lossiness and bandwidth. This may apply, for instance, to mesh
networks as illustrated in Figure 4, or diverse radio access networks networks as illustrated in Figure 4 or diverse radio access networks
as illustrated in Figure 10. as illustrated in Figure 10.
As opposed to wired links, the availability and performance of an As opposed to wired links, the availability and performance of an
individual wireless link cannot be trusted over the long term; it individual wireless link cannot be trusted over the long term; it
varies with transient service discontinuity, and any path that varies with transient service discontinuity, and any path that
includes wireless hops is bound to face short periods of high loss. includes wireless hops is bound to face short periods of high loss.
On the other hand, being broadcast in nature, the wireless medium On the other hand, being broadcast in nature, the wireless medium
provides capabilities that are atypical on modern wired links and provides capabilities that are atypical on modern wired links and
that the RAW Architecture can leverage opportunistically to improve that the RAW architecture can leverage opportunistically to improve
the end-to-end reliability over a collection of paths. the end-to-end reliability over a collection of paths.
Those capabilities include: Those capabilities include:
Promiscuous Overhearing: Some wired and wireless technologies allow Promiscuous overhearing: Some wired and wireless technologies allow
for multiple lower-layer attached nodes to receive the same packet for multiple lower-layer attached nodes to receive the same packet
sent by another node. This differs from a lower-layer network sent by another node. This differs from a lower-layer network
that is physically point-to-point like a wire. With overhearing, that is physically point-to-point, like a wire. With overhearing,
more than one node in the forward direction of the packet may hear more than one node in the forward direction of the packet may hear
or overhear a transmission, and the reception by one may or overhear a transmission, and the reception by one may
compensate the loss by another. The concept of path can be compensate the loss by another. The concept of path can be
revisited in favor of multipoint to multipoint progress in the revisited in favor of multipoint-to-multipoint progress in the
forward direction and statistical chances of successful reception forward direction and statistical chances of successful reception
of any of the transmissions by any of the receivers. of any of the transmissions by any of the receivers.
L2-aware routing: As the quality and speed of a link varies over L2-aware routing: As the quality and speed of a link varies over
time, the concept of metric must also be revisited. Shortest-path time, the concept of metric must also be revisited. Shortest-path
cost loses its absolute value, and hop count turns into a bad idea cost loses its absolute value, and hop count turns into a bad idea
as the link budget drops with the physical distance. Routing over as the link budget drops with the physical distance. Routing over
radio requires both 1) a new and more dynamic sense of link radio requires both:
metrics, with new protocols such as DLEP and L2-trigger to keep L3
up to date with the link quality and availability, and 2) an 1. a new and more dynamic sense of link metrics, with new
approach to multipath routing, where multiple link metrics are protocols such as the Dynamic Link Exchange Protocol (DLEP)
considered since simple shortest-path cost loses its meaning with and Layer 2 (L2) triggers to keep Layer 3 (L3) up to date with
the instability of the metrics. the link quality and availability, and
2. an approach to multipath routing, where multiple link metrics
are considered since simple shortest-path cost loses its
meaning with the instability of the metrics.
Redundant transmissions: Though feasible on any technology, Redundant transmissions: Though feasible on any technology,
proactive (forward) and reactive (ack-based) error correction are proactive (forward) and reactive (ack-based) error correction are
typical to the wireless media. Bounded latency can still be typical for wireless media. Bounded latency can still be obtained
obtained on a wireless link while operating those technologies, on a wireless link while operating those technologies, provided
provided that link latency used in path selection allows for the that link latency used in path selection allows for the extra
extra transmission, or that the introduced delay is compensated transmission or the introduced delay is compensated along the
along the path. In the case of coded fragments and retries, it path. In the case of coded fragments and retries, it makes sense
makes sense to vary all the possible physical properties of the to vary all the possible physical properties of the transmission
transmission to reduce the chances that the same effect causes the to reduce the chances that the same effect causes the loss of both
loss of both original and redundant transmissions. original and redundant transmissions.
Relay Coordination and constructive interference: Though it can be Relay coordination and constructive interference: Though it can be
difficult to achieve at high speed, a fine time synchronization difficult to achieve at high speed, a fine time synchronization
and a precise sense of phase allows the energy from multiple and a precise sense of phase allows the energy from multiple
coordinated senders to add up at the receiver and actually improve coordinated senders to add up at the receiver and actually improve
the signal quality, compensating for either distance or physical the signal quality, compensating for either distance or physical
objects in the Fresnel zone that would reduce the link budget. objects in the Fresnel zone that would reduce the link budget.
From a DetNet perspective, this may translate taking into account From a DetNet perspective, this may translate taking into account
how transmission from one node may interfere with the transmission how transmission from one node may interfere with the transmission
of another node attached to the same wireless sub-layer network. of another node attached to the same wireless sub-layer network.
RAW and DetNet enable application flows that require a special RAW and DetNet enable application flows that require a special
treatment along paths that can provide that treatment. This may be treatment along paths that can provide that treatment. This may be
seen as a form of Path Aware Networking and may be subject to seen as a form of Path Aware networking and may be subject to
impediments documented in [RFC9049]. impediments documented in [RFC9049].
The mechanisms used to establish a path is not unique to, or The mechanism used to establish a path is not unique to, or
necessarily impacted by, RAW. It is expected to be the product of necessarily impacted by, RAW. It is expected to be the product of
the DetNet Controller Plane the DetNet Controller Plane [DetNet-PLANE]; it may use a Path
[I-D.ietf-detnet-controller-plane-framework], and may use a Path Computation Element (PCE) [RFC4655] or the DetNet YANG data model
computation Element (PCE) [RFC4655] or the DetNet Yang Data Model [RFC9633], or it may be computed in a distributed fashion ala the
[RFC9633], or may be computed in a distributed fashion ala Resource Resource ReSerVation Protocol (RSVP) [RFC2205]. Either way, the
ReSerVation Protocol (RSVP) [RFC2205]. Either way, the assumption is assumption is that it is slow relative to local forwarding operations
that it is slow relative to local forwarding operations along the along the path. To react fast enough to transient changes in the
path. To react fast enough to transient changes in the radio radio transmissions, RAW leverages DetNet Network Plane enhancements
transmissions, RAW leverages DetNet Network Plane enhancements to to optimize the use of the paths and match the quality of the
optimize the use of the paths and match the quality of the
transmissions over time. transmissions over time.
As opposed to wired networks, the action of installing a path over a As opposed to wired networks, the action of installing a path over a
set of wireless links may be very slow relative to the speed at which set of wireless links may be very slow relative to the speed at which
the radio conditions vary, and it makes sense in the wireless case to the radio conditions vary; thus, in the wireless case, it makes sense
provide redundant forwarding solutions along a alternate paths (see to provide redundant forwarding solutions along alternate paths (see
Section 3.3) and to leave it to the Network Plane to select which of Section 3.3) and to leave it to the Network Plane to select which of
those forwarding solutions are to be used for a given packet based on those forwarding solutions are to be used for a given packet based on
the current conditions. The RAW Network Plane operations happen the current conditions. The RAW Network Plane operations happen
within the scope of a recovery graph (see Section 3.3.2) that is pre- within the scope of a recovery graph (see Section 3.3.2) that is pre-
established and installed by means outside of the scope of RAW. A established and installed by means outside of the scope of RAW. A
recovery graph may be strict or loose depending on whether each or recovery graph may be strict or loose depending on whether each hop
just a subset of the hops are observed and controlled by RAW. or just a subset of the hops is observed and controlled by RAW.
RAW distinguishes the longer time-scale at which routes are computed RAW distinguishes the longer timescale at which routes are computed
from the shorter time-scale where forwarding decisions are made (see from the shorter timescale where forwarding decisions are made (see
Section 6.1). The RAW Network Plane operations happen at a time- Section 6.1). The RAW Network Plane operations happen at a timescale
scale that sits timewise between the routing and the forwarding time- that sits timewise between the routing and the forwarding timescales.
scales. Their goal is to select dynamically, within the resources Within the resources delineated by a recovery graph, their goal is to
delineated by a recovery graph, the protection path(s) that the dynamically select the protection path(s) that the upcoming packets
upcoming packets of a DetNet flow shall follow. As they influence of a DetNet flow shall follow. As they influence the path for the
the path for entire or portion of flows, the RAW Network Plane entirety of the flows or a portion of them, the RAW Network Plane
operations may affect the metrics used in their rerouting decision, operations may affect the metrics used in their rerouting decisions,
which could potentially lead to oscillations; such effects must be which could potentially lead to oscillations; such effects must be
avoided or dampened. avoided or dampened.
3. Terminology 3. Terminology
RAW reuses terminology defined for DetNet in the "Deterministic RAW reuses terminology defined for DetNet in "Deterministic
Networking Architecture" [DetNet-ARCHI], e.g., PREOF for Packet Networking Architecture" [DetNet-ARCHI], e.g., "PREOF" to stand for
Replication, Elimination and Ordering Functions. RAW inherits and "Packet Replication, Elimination, and Ordering Functions". RAW
augments the IETF art of Protection as seen in DetNet and Traffic inherits and augments the IETF art of protection as seen in DetNet
Engineering. and Traffic Engineering.
RAW reuses terminology defined for Operations, Administration, and RAW also reuses terminology defined for Operations, Administration,
Maintenance (OAM) protocols in Section 1.1 of the "Framework of OAM and Maintenance (OAM) protocols in Section 1.1 of "Framework of
for DetNet" [DetNet-OAM] and "Active and Passive Metrics and Methods Operations, Administration, and Maintenance (OAM) for Deterministic
(with Hybrid Types In-Between)" [RFC7799]. Networking (DetNet)" [DetNet-OAM] and in "Active and Passive Metrics
and Methods (with Hybrid Types In-Between)" [RFC7799].
RAW also reuses terminology defined for MPLS in [RFC4427] such as the RAW also reuses terminology defined for MPLS in [RFC4427], such as
term recovery as covering both Protection and Restoration, a number the term "recovery" to cover both protection and restoration for a
of recovery types. That document defines a number of concepts such number of recovery types. That document defines a number of
as recovery domain that are used in the RAW mechanisms, and defines concepts, such as the recovery domain, that are used in RAW
the new term recovery graph. A recovery graph associates a mechanisms and defines the new term "recovery graph". A recovery
topological graph with usage metadata that represents how the paths graph associates a topological graph with usage metadata that
are built and used within the recovery graph. The recovery graph represents how the paths are built and used within the recovery
provides excess bandwidth for the intended traffic over alternate graph. The recovery graph provides excess bandwidth for the intended
potential paths, and the use of that bandwidth is optimized traffic over alternate potential paths, and the use of that bandwidth
dynamically. is optimized dynamically.
RAW also reuses terminology defined for RSVP-TE in [RFC4090] such as RAW also reuses terminology defined for RSVP-TE in [RFC4090], such as
the Point of Local Repair (PLR). The concept of backup path is the "Point of Local Repair (PLR)". The concept of a backup path is
generalized with protection path, which is the term mostly found in generalized with protection path, which is the term mostly found in
recent standards and used in this document. recent standards and used in this document.
RAW also reuses terminology defined for 6TiSCH in [6TiSCH-ARCHI] and RAW also reuses terminology defined for 6TiSCH in [6TiSCH-ARCHI] and
equates the 6TiSCH concept of a Track with that of a recovery graph. equates the 6TiSCH concept of a Track with that of a recovery graph.
The concept of recovery graph is agnostic to the underlying The concept of a recovery graph is agnostic to the underlying
technology and applies but is not limited to any full or partial technology and applies, but is not limited to, any full or partial
wireless mesh. RAW specifies strict and loose recovery graphs wireless mesh. RAW specifies strict and loose recovery graphs
depending on whether the path is fully controlled by RAW or traverses depending on whether the path is fully controlled by RAW or traverses
an opaque network where RAW cannot observe and control the individual an opaque network where RAW cannot observe and control the individual
hops. hops.
RAW uses the following terminology and acronyms: 3.1. Abbreviations
3.1. Acronyms RAW uses the following abbreviations.
3.1.1. ARQ 3.1.1. ARQ
Automatic Repeat Request, a well-known mechanism, enabling an Automatic Repeat Request. A well-known mechanism that enables an
acknowledged transmission with retries to mitigate errors and loss. acknowledged transmission with retries to mitigate errors and loss.
ARQ may be implemented at various layers in a network. ARQ is ARQ may be implemented at various layers in a network. ARQ is
typically implemented at Layer-2, per hop and not end-to-end in typically implemented per hop (not end to end) at Layer 2 in wireless
wireless networks. ARQ improves delivery on lossy wireless. networks. ARQ improves delivery on lossy wireless. Additionally,
Additionally, ARQ retransmission may be further limited by a bounded ARQ retransmission may be further limited by a bounded time to meet
time to meet end-to-end packet latency constraints. Additional end-to-end packet latency constraints. Additional details and
details and considerations for ARQ are detailed in [RFC3366]. considerations for ARQ are detailed in [RFC3366].
3.1.2. FEC 3.1.2. FEC
Forward Error Correction, adding redundant data to protect against a Forward Error Correction. Adding redundant data to protect against a
partial loss without retries. partial loss without retries.
3.1.3. HARQ 3.1.3. HARQ
Hybrid ARQ, combining FEC and ARQ. Hybrid ARQ. A combination of FEC and ARQ.
3.1.4. ETX 3.1.4. ETX
Expected Transmission Count: a statistical metric that represents the Expected Transmission Count. A statistical metric that represents
expected total number of packet transmissions (including the expected total number of packet transmissions (including
retransmissions) required to successfully deliver a packet along a retransmissions) required to successfully deliver a packet along a
path, used by 6TiSCH [RFC6551]. path, used by 6TiSCH [RFC6551].
3.1.5. ISM 3.1.5. ISM
The industrial, scientific, and medical (ISM) radio band refers to a Industrial, Scientific, and Medical. Refers to a group of radio
group of radio bands or parts of the radio spectrum (e.g., 2.4 GHz bands or parts of the radio spectrum (e.g., 2.4 GHz and 5 GHz) that
and 5 GHz) that are internationally reserved for the use of radio are internationally reserved for the use of radio frequency (RF)
frequency (RF) energy intended for scientific, medical, and energy intended for industrial, scientific, and medical requirements
industrial requirements, e.g., by microwaves, depth radars, and (e.g., by microwaves, depth radars, and medical diathermy machines).
medical diathermy machines. Cordless phones, Bluetooth and LoWPAN Cordless phones, Bluetooth and Low-Power Wireless Personal Area
devices, near-field communication (NFC) devices, garage door openers, Network (LoWPAN) devices, near-field communication (NFC) devices,
baby monitors, and Wi-Fi networks may all use the ISM frequencies, garage door openers, baby monitors, and Wi-Fi networks may all use
although these low-power transmitters are not considered to be ISM the ISM frequencies, although these low-power transmitters are not
devices. In general, communications equipment operating in ISM bands considered to be ISM devices. In general, communications equipment
must tolerate any interference generated by ISM applications, and operating in ISM bands must tolerate any interference generated by
users have no regulatory protection from ISM device operation in ISM applications, and users have no regulatory protection from ISM
these bands. device operation in these bands.
3.1.6. PER and PDR 3.1.6. PER
The Packet Error Rate (PER) is defined as the ratio of the number of Packet Error Rate. The ratio of the number of packets received in
packets received in error to the total number of transmitted packets. error to the total number of transmitted packets. A packet is
A packet is considered to be in error if even a single bit within the considered to be in error if even a single bit within the packet is
packet is received incorrectly. In contrast, the Packet Delivery received incorrectly.
Ratio (PDR) indicates the ratio of the number successful delivery of
data packets to the total number of transmitted packets from the
sender to the receiver.
3.1.7. RSSI 3.1.7. PDR
Received Signal Strength Indication (a.k.a. Energy Detection Level): Packet Delivery Ratio (PDR). The ratio of the number of successfully
a measure of incoherent (raw) RF power in a channel. The RF power delivered data packets to the total number of packets transmitted
can come from any source: other transmitters using the same from the sender to the receiver.
technology, other radio technology using the same band, or background
radiation. For a single-hop, RSSI may be used for LQI.
3.1.8. LQI 3.1.8. RSSI
The link quality indicator (LQI) is an indication of the quality of Received Signal Strength Indication. Also known as "Energy Detection
the data packets received by the receiver. It is typically derived Level". A measure of the incoherent (raw) RF power in a channel.
from packet error statistics, the exact method depending on the The RF power can come from any source: other transmitters using the
same technology, other radio technology using the same band, or
background radiation. For a single hop, RSSI may be used for LQI.
3.1.9. LQI
Link Quality Indicator. An indication of the quality of the data
packets received by the receiver. It is typically derived from
packet error statistics, with the exact method depending on the
network stack being used. LQI values may be exposed to the network stack being used. LQI values may be exposed to the
controller plane for each individual hop or cumulated along segments. controller plane for each individual hop or cumulated along segments.
Outgoing LQI values can be calculated from coherent (demodulated) Outgoing LQI values can be calculated from coherent (demodulated)
PER, RSSI and incoming LQI values. PER, RSSI, and incoming LQI values.
3.1.9. OAM 3.1.10. OAM
OAM stands for Operations, Administration, and Maintenance, and Operations, Administration, and Maintenance. Covers the processes,
covers the processes, activities, tools, and standards involved with activities, tools, and standards involved with operating,
operating, administering, managing, and maintaining any system. This administering, managing, and maintaining any system. This document
document uses the terms Operations, Administration, and Maintenance, uses the term in conformance with "Guidelines for the Use of the
in conformance with the 'Guidelines for the Use of the "OAM" Acronym 'OAM' Acronym in the IETF" [RFC6291], and the system observed by the
in the IETF' [RFC6291] and the system observed by the RAW OAM is the RAW OAM is the recovery graph.
recovery graph.
3.1.10. OODA 3.1.11. OODA
OODA (Observe, Orient, Decide, Act) is a generic formalism to Observe, Orient, Decide, Act. A generic formalism to represent the
represent the operational steps in a Control Loop. In the context of operational steps in a Control Loop. In the context of RAW, OODA is
RAW, OODA is applied to network control and convergence, more in applied to network control and convergence; see Section 6.2 for more.
Section 6.2.
3.1.11. SNR 3.1.12. SNR
Signal-Noise Ratio (a.k.a. S/N): a measure used in science and Signal-to-Noise Ratio. Also known as "S/N Ratio". A measure used in
engineering that compares the level of a desired signal to the level science and engineering that compares the level of a desired signal
of background noise. SNR is defined as the ratio of signal power to to the level of background noise. SNR is defined as the ratio of
noise power, often expressed in decibels. signal power to noise power, often expressed in decibels.
3.2. Link and Direction 3.2. Link and Direction
3.2.1. Flapping 3.2.1. Flapping
In the context of RAW, a link flaps when the reliability of the In the context of RAW, a link flaps when the reliability of the
wireless connectivity drops abruptly for a short period of time, wireless connectivity drops abruptly for a short period of time,
typically of a subsecond to seconds duration. typically a duration of a subsecond to seconds.
3.2.2. Uplink 3.2.2. Uplink
Connection from end-devices to data communication equipment. In the An uplink is the connection from end devices to data communication
context of wireless, uplink refers to the connection between a equipment. In the context of wireless, uplink refers to the
station (STA) and a controller (AP) or a User Equipment (UE) to a connection between a station (STA) and a controller (AP) or a User
Base Station (BS) such as a 3GPP 5G gNodeB (gNb). Equipment (UE) and a Base Station (BS) such as a 3GPP 5G gNodeB
(gNB).
3.2.3. Downlink 3.2.3. Downlink
The reverse direction from uplink. A downlink is the reverse direction from uplink.
3.2.4. Downstream 3.2.4. Downstream
Following the direction of the flow data path along a recovery graph. Downstream refers to the following the direction of the flow data
path along a recovery graph.
3.2.5. Upstream 3.2.5. Upstream
Against the direction of the flow data path along a recovery graph. Upstream refers to going against the direction of the flow data path
along a recovery graph.
3.3. Path and Recovery Graphs 3.3. Path and Recovery Graphs
3.3.1. Path 3.3.1. Path
Quoting section 1.1.3 of [INT-ARCHI]: Section 1.3.3 of [INT-ARCHI] provides a definition of Path:
| At a given moment, all the IP datagrams from a particular source | At a given moment, all the IP datagrams from a particular source
| host to a particular destination host typically traverse the same | host to a particular destination host will typically traverse the
| sequence of gateways. We use the term "path" for this sequence. | same sequence of gateways. We use the term "path" for this
| Note that a path is unidirectional; it is not unusual to have | sequence. Note that a path is uni-directional; it is not unusual
| different paths in the two directions between a given host pair. | to have different paths in the two directions between a given host
| pair.
Section 2 of [RFC9473] points to a longer, more modern definition of Section 2 of [RFC9473] points to a longer, more modern definition of
path, which begins as follows: path, which begins as follows:
| A sequence of adjacent path elements over which a packet can be | A sequence of adjacent path elements over which a packet can be
| transmitted, starting and ending with a node. | transmitted, starting and ending with a node.
| |
| Paths are unidirectional and time-dependent, i.e., there can be a | Paths are unidirectional and time dependent, i.e., there can be a
| variety of paths from one node to another, and the path over which | variety of paths from one node to another, and the path over which
| packets are transmitted may change. A path definition can be | packets are transmitted may change. A path definition can be
| fixed (i.e., the exact sequence of path elements remains the same) | strict (i.e., the exact sequence of path elements remains the
| or mutable (i.e., the start and end node remain the same, but the | same) or loose (i.e., the start and end node remain the same, but
| path elements between them may vary over time). | the path elements between them may vary over time).
| |
| The representation of a path and its properties may depend on the | The representation of a path and its properties may depend on the
| entity considering the path. On the one hand, the representation | entity considering the path. On the one hand, the representation
| may differ due to entities having partial visibility of path | may differ due to entities having partial visibility of path
| elements comprising a path or their visibility changing over time. | elements comprising a path or their visibility changing over time.
It follows that the general acceptance of a path is a linear sequence It follows that the general acceptance of a path is a linear sequence
of links and nodes, as opposed to a multi-dimensional graph, defined of links and nodes, as opposed to a multi-dimensional graph, defined
by the experience of the packet that went from a node A to a node B. by the experience of the packet that went from a node A to a node B.
In the context of this document, a path is observed by following one In the context of this document, a path is observed by following one
copy or one fragment of a packet that conserves its uniqueness and copy or one fragment of a packet that conserves its uniqueness and
integrity. For instance, if C replicates to E and F and D eliminates integrity. For instance, if C replicates to E and F and if D
duplicates, a packet from A to B can experience 2 paths, eliminates duplicates, a packet from A to B can experience two paths:
A->C->E->D->B and A->C->F->D->B. Those paths are called protection A->C->E->D->B and A->C->F->D->B. Those paths are called protection
paths. Protection paths may be fully non-congruent, and paths. Protection paths may be fully non-congruent; alternatively,
alternatively may intersect at replication or elimination points. they may intersect at replication or elimination points.
With DetNet and RAW, a packet may be duplicated, fragmented, and With DetNet and RAW, a packet may be duplicated, fragmented, and
network-coded, and the various byproducts may travel different paths network coded, and the various byproducts may travel different paths
that are not necessarily end-to-end between A and B; we refer to that that are not necessarily end to end between A and B. We refer to
complex scenario as a DetNet path. As such, the DetNet path extends this complex scenario as a DetNet path. As such, the DetNet path
the above description of a path, but it still matches the experience extends the above description of a path, but it still matches the
of a packet that traverses the network. experience of a packet that traverses the network.
With RAW, the path experienced by a packet is subject to change from With RAW, the path experienced by a packet is subject to change from
one packet to the next, but all the possible experiences are all one packet to the next, but all the possible experiences are all
contained within a finite set. Therefore, we introduce below the contained within a finite set. Therefore, we introduce the term
term of a recovery graph that coalesces that set and covers the "recovery graph" (see the next section) that coalesces that set and
overall topology where the possible DetNet paths are all contained. covers the overall topology where the possible DetNet paths are all
As such, the recovery graph coalesces all the possible paths a flow contained. As such, the recovery graph coalesces all the possible
may experience, each with its own statistical probability to be used. paths a flow may experience, each with its own statistical
probability to be used.
3.3.2. Recovery Graph 3.3.2. Recovery Graph
A networking graph that can be followed to transport packets with A recovery graph is a networking graph that can be followed to
equivalent treatment, associated with usage metadata; as opposed to transport packets with equivalent treatment and is associated with
the definition of a path above, a recovery graph represents not an usage metadata. In contrast to the definition of a path above, a
actual but a potential, it is not necessarily a linear sequence like recovery graph represents not an actual but a potential, is not
a simple path, and is not necessarily fully traversed (flooded) by necessarily a linear sequence like a simple path, and is not
all packets of a flow like a DetNet Path. Still, and as a necessarily fully traversed (flooded) by all packets of a flow like a
simplification, the casual reader may consider that a recovery graph DetNet Path. Still, and as a simplification, the casual reader may
is very much like a DetNet path, aggregating multiple paths that may consider that a recovery graph is very much like a DetNet path,
overlap, fork and rejoin, for instance to enable a protection service aggregating multiple paths that may overlap or fork and then rejoin,
by the PREOF operations. for instance, to enable a protection service by the PREOF operations.
_________ _________
| | | |
| IoT G/W | | IoT G/W |
|_________| |_________|
EGRESS <<=== Elimination at Egress EGRESS <<=== Elimination at Egress
| | | |
---+--------+--+--------+-------- ---+--------+--+--------+--------
| Backbone | | Backbone |
__|__ __|__ __|__ __|__
skipping to change at page 13, line 27 skipping to change at line 576
|__ __| Router |__ __| Router |__ __| Router |__ __| Router
| # | | # |
# \ # / <-- protection path # \ # / <-- protection path
# # #-------# # # #-------#
\ # / # ( Low-power ) \ # / # ( Low-power )
# # \ / # ( Lossy Network) # # \ / # ( Lossy Network)
\ / \ /
# INGRESS <<=== Replication at recovery graph Ingress # INGRESS <<=== Replication at recovery graph Ingress
| |
# <-- source device # <-- source device
#: Low-power device
#: Low-power device
Figure 1: Example IoT Recovery Graph to an IoT Gateway with 1+1 Figure 1: Example IoT Recovery Graph to an IoT Gateway with 1+1
Redundancy Redundancy
Refining further, a recovery graph is defined as the coalescence of Refining further, a recovery graph is defined as the coalescence of
the collection of all the feasible DetNet Paths that a packet for the collection of all the feasible DetNet Paths that a packet for
which a flow is assigned to the recovery graph may be forwarded which a flow is assigned to the recovery graph may be forwarded
along. A packet that is assigned to the recovery graph experiences along. A packet that is assigned to the recovery graph experiences
one of the feasible DetNet Paths based on the current selection by one of the feasible DetNet Paths based on the current selection by
the PLR at the time the packet traverses the network. the PLR at the time the packet traverses the network.
Refining even further, the feasible DetNet Paths within the recovery Refining even further, the feasible DetNet Paths within the recovery
graph may or may not be computed in advance, but decided upon the graph may or may not be computed in advance; instead, they may be
detection of a change from a clean slate. Furthermore, the PLR decided upon the detection of a change from a clean slate.
decision may be distributed, which yields a large combination of Furthermore, the PLR decision may be distributed, which yields a
possible and dependent decisions, with no node in the network capable large combination of possible and dependent decisions, with no node
of reporting which is the current DetNet Path within the recovery in the network capable of reporting which is the current DetNet Path
graph. within the recovery graph.
In DetNet [DetNet-ARCHI] terms, a recovery graph has the following In DetNet [DetNet-ARCHI] terms, a recovery graph has the following
properties: properties:
* A recovery graph is a Layer-3 abstraction built upon IP links * A recovery graph is a Layer 3 abstraction built upon IP links
between routers. A router may form multiple IP links over a between routers. A router may form multiple IP links over a
single radio interface. single radio interface.
* A recovery graph has one Ingress and one Egress node, which * A recovery graph has one Ingress and one Egress node, which
operate as DetNet Edge nodes. operate as DetNet Edge nodes.
* The graph of a recovery graph is reversible, meaning that packets * The graph of a recovery graph is reversible, meaning that packets
can be routed against the flow of data packets, e.g., to carry OAM can be routed against the flow of data packets, e.g., to carry OAM
measurements or control messages back to the Ingress. measurements or control messages back to the Ingress.
* The vertices of that graph are DetNet Relay Nodes that operate at * The vertices of that graph are DetNet Relay Nodes that operate at
the DetNet Service sub-layer and provide the PREOF functions. the DetNet Service sub-layer and provide the PREOF functions.
* The topological edges of the graph are strict sequences of DetNet * The topological edges of the graph are strict sequences of DetNet
Transit nodes that operate at the DetNet Forwarding sub-layer. Transit nodes that operate at the DetNet Forwarding sub-layer.
Figure 2 illustrates the generic concept of a recovery graph, between Figure 2 illustrates the generic concept of a recovery graph, between
an Ingress Node and an Egress Node. The recovery graph is composed an Ingress Node and an Egress Node. The recovery graph is composed
of forward protection paths and forward or crossing Segments (see the of forward protection paths, forward Segments, and crossing Segments
definition for those terms in the next sections). The recovery graph (see the definitions of those terms in the next sections). The
contains at least 2 protection paths as a main path and a backup recovery graph contains at least two protection paths: a main path
path. and a backup path.
------------------- forward direction ----------------------> ------------------- forward direction ---------------------->
a ==> b ==> C -=- F ==> G ==> h T1 I: Ingress a ==> b ==> C -=- F ==> G ==> h T1
/ \ / | \ / E: Egress / \ / | \ /
I o n E -=- T2 T1, T2, T3: I o n E -=- T2
\ / \ | / \ External \ / \ | / \
p ==> q ==> R -=- T ==> U ==> v T3 Targets p ==> q ==> R -=- T ==> U ==> v T3
Uppercase: DetNet Relay Nodes I: Ingress
Lowercase: DetNet Transit nodes E: Egress
T1, T2, T3: external targets
Uppercase: DetNet Relay Nodes
Lowercase: DetNet Transit nodes
I ==> a ==> b ==> C : A forward Segment to targets F and o Figure 2: A Recovery Graph and Its Components
C ==> o ==> T: A forward Segment to target T (and/or U)
G | n | U : A crossing Segment to targets G or U
I -> F -> E : A forward Protection Path to targets T1, T2, and T3
I, a, b, C, F, G, h, E : a path to T1, T2, and/or T3 Of note:
I, p, q, R, o, F, G, h, E : segment-crossing protection path
Figure 2: A Recovery Graph and its Components I ==> a ==> b ==> C: A forward Segment to targets F and o
C ==> o ==> T: A forward Segment to target T (and/or U)
G | n | U: A crossing Segment to targets G or U
I -> F -> E: A forward protection path to targets T1, T2, and T3
I, a, b, C, F, G, h, E: A path to T1, T2, and/or T3
I, p, q, R, o, F, G, h, E: A segment-crossing protection path
3.3.3. Forward and Crossing 3.3.3. Forward and Crossing
Forward refers to progress towards the recovery graph Egress. Forward refers to progress towards the Egress of the recovery graph.
Forward links are directional, and packets that are forwarded along Forward links are directional, and packets that are forwarded along
the recovery graph can only be transmitted along the link direction. the recovery graph can only be transmitted along the link direction.
Crossing links are bidirectional, meaning that they can be used in Crossing links are bidirectional, meaning that they can be used in
both directions, though a given packet may use the link in one both directions, though a given packet may use the link in one
direction only. A Segment can be forward, in which case it is direction only. A Segment can be forward, in which case it is
composed of forward links only, or crossing, in which case it is composed of forward links only, or it can be crossing, in which case
composed of crossing links only. A Protection Path is always it is composed of crossing links only. A protection path is always
forward, meaning that it is composed of forward links and Segments. forward, meaning that it is composed of forward links and Segments.
3.3.4. Protection Path 3.3.4. Protection Path
An end-to-end forward path between the Ingress and Egress Nodes of a A protection path is an end-to-end forward path between the Ingress
recovery graph. A protection path in a recovery graph is expressed and Egress Nodes of a recovery graph. A protection path in a
as a strict sequence of DetNet Relay Nodes or as a loose sequence of recovery graph is expressed as a strict sequence of DetNet Relay
DetNet Relay Nodes that are joined by recovery graph Segments. Nodes or as a loose sequence of DetNet Relay Nodes that are joined by
Background information on the concepts related to protection paths Segments in the recovery graph. Background information on the
can be found in [RFC4427] and [RFC6378] concepts related to protection paths can be found in [RFC4427] and
[RFC6378].
3.3.5. Segment 3.3.5. Segment
A strict sequence of DetNet Transit nodes between 2 DetNet Relay A Segment is a strict sequence of DetNet Transit nodes between two
Nodes; a Segment of a recovery graph is composed topologically of two DetNet Relay Nodes; a Segment of a recovery graph is composed
vertices of the recovery graph and one edge of the recovery graph topologically of two vertices of the recovery graph and one edge of
between those vertices. the recovery graph between those vertices.
3.4. Deterministic Networking 3.4. Deterministic Networking
This document reuses the terminology in section 2 of [RFC8557] and This document reuses the terminology in Section 2 of [RFC8557] and
section 4.1.2 of [DetNet-ARCHI] for deterministic networking and Section 4.1.2 of [DetNet-ARCHI] for deterministic networking and
deterministic networks. deterministic networks. This documents also uses the following
terms.
3.4.1. The DetNet Planes 3.4.1. The DetNet Planes
[DetNet-ARCHI] defines three planes: the Application (User) Plane, [DetNet-ARCHI] defines three planes: the Application (User) Plane,
the Controller Plane, and the Network Plane. The DetNet Network the Controller Plane, and the Network Plane. The DetNet Network
Plane is composed of a Data Plane (packet forwarding) and an Plane is composed of a Data Plane (packet forwarding) and an
Operational Plane where OAM operations take place. In the Network Operational Plane where OAM operations take place. In the Network
Plane, the DetNet Service sub-layer focuses on flow protection (e.g., Plane, the DetNet Service sub-layer focuses on flow protection (e.g.,
using redundancy) and can be fully operated at Layer-3, while the using redundancy) and can be fully operated at Layer 3, while the
DetNet forwarding sub-layer establishes the paths, associates the DetNet forwarding sub-layer establishes the paths, associates the
flows to the paths, and ensures the availability of the necessary flows to the paths, ensures the availability of the necessary
resources, leverages Layer-2 functionalities for timely delivery to resources, and leverages Layer 2 functionalities for timely delivery
the next DetNet system, more in Section 2. to the next DetNet system. For more information, see Section 2.
3.4.2. Flow 3.4.2. Flow
A collection of consecutive IP packets defined by the upper layers A flow is a collection of consecutive IP packets defined by the upper
and signaled by the same 5 or 6-tuple (see section 5.1 of [RFC8939]). layers and signaled by the same 5-tuple or 6-tuple (see Section 5.1
Packets of the same flow must be placed on the same recovery graph to of [RFC8939]). Packets of the same flow must be placed on the same
receive an equivalent treatment from Ingress to Egress within the recovery graph to receive an equivalent treatment from Ingress to
recovery graph. Multiple flows may be transported along the same Egress within the recovery graph. Multiple flows may be transported
recovery graph. The DetNet Path that is selected for the flow may along the same recovery graph. The DetNet Path that is selected for
change over time under the control of the PLR. the flow may change over time under the control of the PLR.
3.4.3. Residence Time 3.4.3. Residence Time
A residence time (RT) is defined as the time interval between when A residence time (RT) is defined as the time interval between when
the reception of a packet starts and the transmission of the packet the reception of a packet starts and the transmission of the packet
begins. In the context of RAW, RT is useful for a transit node, not begins. In the context of RAW, RT is useful for a transit nodes, not
ingress or egress. ingress or egress nodes.
3.4.4. L3 Deterministic Flow Identifier 3.4.4. L3 Deterministic Flow Identifier
See section 3.3 of [DetNet-DP]. The classic IP 5-tuple that The classic IP 5-tuple that identifies a flow comprises the source
identifies a flow comprises the source IP, destination IP, source IP, destination IP, source port, destination port, and the Upper-
port, destination port, and the upper layer protocol (ULP). DetNet Layer Protocol (ULP). DetNet uses a 6-tuple where the extra field is
uses a 6-tuple where the extra field is the DSCP field in the packet. the Differentiated Services Code Point (DSCP) field in the packet
The IPv6 flow label is not used for that purpose. (see Section 3.3 of [DetNet-DP]). The IPv6 flow label is not used
for that purpose.
3.4.5. TSN 3.4.5. Time-Sensitive Networking
TSN stands for Time-Sensitive Networking and denotes the efforts at Time-Sensitive Networking (TSN) denotes the efforts at IEEE 802 for
IEEE 802 for deterministic networking, originally for use on deterministic networking, originally for use on Ethernet. Wireless
Ethernet. Wireless TSN (WTSN) denotes extensions of the TSN work on TSN (WTSN) denotes extensions of the TSN work on wireless media such
wireless media such as the selected RAW technologies [RAW-TECHNOS]. as the selected RAW technologies [RAW-TECHNOS].
3.4.6. Lower-Layer API 3.4.6. Lower-Layer API
In addition, RAW includes the concept of a lower-layer API (LL API), RAW includes the concept of a lower-layer API (LL API) that provides
that provides an interface between the lower layer (e.g., wireless) an interface between the lower-layer (e.g., wireless) technology and
technology and the DetNet layers. The LL API is technology dependent the DetNet layers. The LL API is technology dependent as what the
as what the lower layers expose towards the DetNet layers may vary. lower layers expose towards the DetNet layers may vary. Furthermore,
Furthermore, the different RAW technologies are equipped with different RAW technologies are equipped with different reliability
different reliability features, e.g., short range broadcast, features (e.g., short-range broadcast, Multiple User - Multiple Input
Multiple-User, Multiple-Input, and Multiple-Output (MUMIMO), PHY rate Multiple Output (MU-MIMO), PHY rate and other Modulation Coding
and other Modulation Coding Scheme (MCS) adaptation, coding and Scheme (MCS) adaptation, coding and retransmissions methods, and
retransmissions methods, constructive interference and overhearing, constructive interference and overhearing; see [RAW-TECHNOS] for more
see [RAW-TECHNOS] for details. The LL API enables interactions details). The LL API enables interactions between the reliability
between the reliability functions provided by the lower layer and the functions provided by the lower layer and the reliability functions
reliability functions provided by DetNet. That is, the LL API makes provided by DetNet. That is, the LL API makes cross-layer
cross-layer optimization possible for the reliability functions of optimization possible for the reliability functions of different
different layers depending on the actual exposure provided via the LL layers depending on the actual exposure provided via the LL API by
API by the given RAW technology. The Dynamic Link Exchange Protocol the given RAW technology. The Dynamic Link Exchange Protocol (DLEP)
(DLEP) [DLEP] is an example protocol that can be used to implement [DLEP] is an example of a protocol that can be used to implement the
the LL API. LL API.
3.5. Reliability and Availability 3.5. Reliability and Availability
In the context of the RAW work, Reliability and Availability are This document uses the following terms relating to reliability and
defined as follows: availability in the context of the RAW work.
3.5.1. Service Level Agreement 3.5.1. Service Level Agreement
In the context of RAW, an SLA (service level agreement) is a contract In the context of RAW, a Service Level Agreement (SLA) is a contract
between a provider (the network) and a client, the application flow, between a provider (the network) and a client, the application flow,
defining measurable metrics such as latency boundaries, consecutive defining measurable metrics such as latency boundaries, consecutive
losses, and packet delivery ratio (PDR). losses, and Packet Delivery Ratio (PDR).
3.5.2. Service Level Objective 3.5.2. Service Level Objective
A service level objective (SLO) is one term in the SLA, for which A Service Level Objective (SLO) is one term in the SLA, for which
specific network setting and operations are implemented. For specific network setting and operations are implemented. For
instance, a dynamic tuning of the packet redundancy addresses an SLO instance, a dynamic tuning of packet redundancy addresses an SLO of
of consecutive losses in a row by augmenting the chances of delivery consecutive losses in a row by augmenting the chances of delivery of
of a packet that follows a loss. a packet that follows a loss.
3.5.3. Service Level Indicator 3.5.3. Service Level Indicator
A service level indicator (SLI) measures the compliance of an SLO to A Service Level Indicator (SLI) measures the compliance of an SLO to
the terms of the contract. It can be for instance, the statistics of the terms of the contract. For instance, it can be the statistics of
individual losses and losses in a row as time series. individual losses and losses in a row as time series.
3.5.4. Precision Availability Metrics 3.5.4. Precision Availability Metrics
Precision Availability Metrics (PAMs) [RFC9544] aim at capturing Precision Availability Metrics (PAMs) [RFC9544] aim to capture
service levels for a flow, specifically the degree to which the flow service levels for a flow, specifically the degree to which the flow
complies with the SLOs that are in effect. complies with the SLOs that are in effect.
3.5.5. Reliability 3.5.5. Reliability
Reliability is a measure of the probability that an item (e.g., Reliability is a measure of the probability that an item (e.g.,
system, network) will perform its intended function with no failure system or network) will perform its intended function with no failure
for a stated period of time (or a stated number of demands or load) for a stated period of time (or for a stated number of demands or
under stated environmental conditions. In other words, reliability load) under stated environmental conditions. In other words,
is the probability that an item will be in an uptime state (i.e., reliability is the probability that an item will be in an uptime
fully operational or ready to perform) for a stated mission, e.g., to state (i.e., fully operational or ready to perform) for a stated
provide an SLA. See more in [NASA1]. mission (e.g., to provide an SLA). See more in [NASA1].
3.5.6. Availability 3.5.6. Availability
Availability is the probability of an item’s (e.g., a network’s) Availability is the probability of an item's (e.g., a network's)
mission readiness (e.g., to provide an SLA), an uptime state with the mission readiness (e.g., to provide an SLA), an uptime state with the
likelihood of a recoverable downtime state. Availability is likelihood of a recoverable downtime state. Availability is
expressed as (uptime)/(uptime+downtime). Note that it is expressed as (uptime)/(uptime+downtime). Note that it is
availability that addresses downtime (including time for maintenance, availability that addresses downtime (including time for maintenance,
repair, and replacement activities) and not reliability. See more in repair, and replacement activities) and not reliability. See more in
[NASA2]. [NASA2].
4. Reliable and Available Wireless 4. Reliable and Available Wireless
4.1. High Availability Engineering Principles 4.1. High Availability Engineering Principles
The reliability criteria of a critical system pervade through its The reliability criteria of a critical system pervade its elements,
elements, and if the system comprises a data network and then the and if the system comprises a data network, then the data network is
data network is also subject to the inherited reliability and also subject to the inherited reliability and availability criteria.
availability criteria. It is only natural to consider the art of It is only natural to consider the art of high availability
high availability engineering and apply it to wireless communications engineering and apply it to wireless communications in the context of
in the context of RAW. RAW.
There are three principles (pillars) of high availability There are three principles (pillars) of high availability
engineering: engineering:
1. elimination of each single point of failure 1. elimination of each single point of failure
2. reliable crossover 2. reliable crossover
3. prompt detection of failures as they occur 3. prompt detection of failures as they occur
These principles are common to all high availability systems, not These principles are common to all high availability systems, not
just ones with Internet technology at the center. Examples of both just ones with Internet technology at the center. Both non-Internet
non-Internet and Internet are included. and Internet examples are included.
4.1.1. Elimination of Single Points of Failure 4.1.1. Elimination of Single Points of Failure
Physical and logical components in a system happen to fail, either as Physical and logical components in a system happen to fail, either as
the effect of wear and tear, when used beyond acceptable limits, or the effect of wear and tear, when used beyond acceptable limits, or
due to a software bug. It is necessary to decouple component failure due to a software bug. It is necessary to decouple component failure
from system failure to avoid the latter. This allows failed from system failure to avoid the latter. This allows failed
components to be restored while the rest of the system continues to components to be restored while the rest of the system continues to
function. function.
IP Routers leverage routing protocols to reroute to alternate routes IP routers leverage routing protocols to reroute to alternate routes
in case of a failure. When links are cabled through the same in case of a failure. When links are cabled through the same
conduit, they form a shared risk link group (SRLG), and share the conduit, they form a Shared Risk Link Group (SRLG) and share the same
same fate if the conduit is cut, making the reroute operation fate if the conduit is cut, making the reroute operation ineffective.
ineffective. The same effect can happen with virtual links that end The same effect can happen with virtual links that end up in the same
up in a same physical transport through the intricacies of nested physical transport through the intricacies of nested encapsulation.
encapsulation. In a same fashion, an interferer or an obstacle may In the same fashion, an interferer or an obstacle may affect multiple
affect multiple wireless transmissions at the same time, even between wireless transmissions at the same time, even between different sets
different sets of peers. of peers.
Intermediate network Nodes such as routers, switches and APs, wire Intermediate network nodes (such as routers, switches and APs, wire
bundles, and the air medium itself can become single points of bundles, and the air medium itself) can become single points of
failure. For High Availability, it is thus required to use failure. Thus, for high availability, the use of physically link-
physically link-disjoint and Node-disjoint paths; in the wireless disjoint and node-disjoint paths is required; in the wireless space,
space, it is also required to use the highest possible degree of the use of the highest possible degree of diversity (time, space,
diversity (time, space, code, frequency, channel width) in the code, frequency, and channel width) in the transmissions over the air
transmissions over the air to combat the additional causes of is also required to combat the additional causes of transmission
transmission loss. loss.
From an economics standpoint, executing this principle properly From an economics standpoint, executing this principle properly
generally increases capital expense because of the redundant generally increases capital expense because of the redundant
equipment. In a constrained network where the waste of energy and equipment. In a constrained network where the waste of energy and
bandwidth should be minimized, an excessive use of redundant links bandwidth should be minimized, an excessive use of redundant links
must be avoided; for RAW this means that the extra bandwidth must be must be avoided; for RAW, this means that the extra bandwidth must be
used wisely and efficiently. used wisely and efficiently.
4.1.2. Reliable Crossover 4.1.2. Reliable Crossover
Having backup equipment has a limited value unless it can be reliably Backup equipment has limited value unless it can be reliably switched
switched into use within the down-time parameters. IP Routers into use within the downtime parameters. IP routers execute reliable
execute reliable crossover continuously because the routers use any crossover continuously because the routers use any alternate routes
alternate routes that are available [RFC0791]. This is due to the that are available [RFC0791]. This is due to the stateless nature of
stateless nature of IP datagrams and the dissociation of the IP datagrams and the dissociation of the datagrams from the
datagrams from the forwarding routes they take. The "IP Fast Reroute forwarding routes they take. "IP Fast Reroute Framework" [FRR]
Framework" [FRR] analyzes mechanisms for fast failure detection and analyzes mechanisms for fast failure detection and path repair for IP
path repair for IP Fast-Reroute (FRR), and discusses the case of Fast Reroute (FRR) and discusses the case of multiple failures and
multiple failures and SRLG. Examples of FRR techniques include SRLG. Examples of FRR techniques include Remote Loop-Free Alternate
Remote Loop-Free Alternate [RLFA-FRR] and backup label-switched path [RLFA-FRR] and backup Label Switched Path (LSP) tunnels for the local
(LSP) tunnels for the local repair of LSP tunnels using RSVP-TE repair of LSP tunnels using RSVP-TE [RFC4090].
[RFC4090].
Deterministic flows, on the contrary, are attached to specific paths Deterministic flows, on the contrary, are attached to specific paths
where dedicated resources are reserved for each flow. Therefore, where dedicated resources are reserved for each flow. Therefore,
each DetNet path must inherently provide sufficient redundancy to each DetNet path must inherently provide sufficient redundancy to
provide the assured SLOs at all times. The DetNet PREOF typically provide the assured SLOs at all times. The DetNet PREOF typically
leverages 1+1 redundancy whereby a packet is sent twice, over non- leverages 1+1 redundancy whereby a packet is sent twice, over non-
congruent paths. This avoids the gap during the fast reroute congruent paths. This avoids the gap during the FRR operation but
operation, but doubles the traffic in the network. doubles the traffic in the network.
In the case of RAW, the expectation is that multiple transient faults In the case of RAW, the expectation is that multiple transient faults
may happen in overlapping time windows, in which case the 1+1 may happen in overlapping time windows, in which case the 1+1
redundancy with delayed reestablishment of the second path does not redundancy with delayed reestablishment of the second path does not
provide the required guarantees. The Data Plane must be configured provide the required guarantees. The Data Plane must be configured
with a sufficient degree of redundancy to select an alternate with a sufficient degree of redundancy to select an alternate
redundant path immediately upon a fault, without the need for a slow redundant path immediately upon a fault, without the need for a slow
intervention from the Controller Plane. intervention from the Controller Plane.
4.1.3. Prompt Notification of Failures 4.1.3. Prompt Notification of Failures
The execution of the two above principles is likely to render a The execution of the two above principles is likely to render a
system where the end user rarely sees a failure. But a failure that system where the end user rarely sees a failure. However, a failure
occurs must still be detected in order to direct maintenance. that occurs must still be detected in order to direct maintenance.
There are many reasons for system monitoring (FCAPS for fault, There are many reasons for system monitoring (Fault, Configuration,
configuration, accounting, performance, security is a handy mental Accounting, Performance, and Security (FCAPS) is a handy mental
checklist) but fault monitoring is sufficient reason. checklist), but fault monitoring is a sufficient reason.
"Overview and Principles of Internet Traffic Engineering" [TE] "Overview and Principles of Internet Traffic Engineering" [TE]
discusses the importance of measurement for network protection, and discusses the importance of measurement for network protection and
provides an abstract method for network survivability with the provides an abstract method for network survivability with the
analysis of a traffic matrix as observed via a network management analysis of a traffic matrix as observed via a network management
YANG data model, probing techniques, file transfers, IGP link state YANG data model, probing techniques, file transfers, IGP link state
advertisements, and more. advertisements, and more.
Those measurements are needed in the context of RAW to inform the Those measurements are needed in the context of RAW to inform the
controller and make the long-term reactive decision to rebuild a controller and make the long-term reactive decision to rebuild a
recovery graph based on statistical and aggregated information. RAW recovery graph based on statistical and aggregated information. RAW
itself operates in the DetNet Network Plane at a faster time-scale itself operates in the DetNet Network Plane at a faster timescale
with live information on speed, state, etc. This live information with live information on speed, state, etc. This live information
can be obtained directly from the lower layer, e.g., using L2 can be obtained directly from the lower layer (e.g., using L2
triggers, read from a protocol such as DLEP, or transported over triggers), read from a protocol such as DLEP, or transported over
multiple hops using OAM and reverse OAM, as illustrated in Figure 11. multiple hops using OAM and reverse OAM, as illustrated in Figure 11.
4.2. Applying Reliability Concepts to Networking 4.2. Applying Reliability Concepts to Networking
The terms Reliability and Availability are defined for use in RAW in The terms "reliability" and "availability" are defined for use in RAW
Section 3 and the reader is invited to read [NASA1] and [NASA2] for in Section 3, and the reader is invited to read [NASA1] and [NASA2]
more details on the general definition of Reliability. Practically for more details on the general definition of reliability.
speaking, a number of nines is often used to indicate the reliability Practically speaking, a number of nines is often used to indicate the
of a data link, e.g., 5 nines indicate a Packet Delivery Ratio (PDR) reliability of a data link (e.g., 5 nines indicate a Packet Delivery
of 99.999%. Ratio (PDR) of 99.999%).
This number is typical in a wired environment where the loss is due This number is typical in a wired environment where the loss is due
to a random event such as a solar particle that affects the to a random event such as a solar particle that affects the
transmission of a particular packet, but does not affect the previous transmission of a particular packet but does not affect the previous
or next packet, nor packets transmitted on other links. Note that packet, the next packet, or packets transmitted on other links. Note
the QoS requirements in RAW may include a bounded latency, and a that the QoS requirements in RAW may include a bounded latency, and a
packet that arrives too late is a fault and not considered as packet that arrives too late is a fault and not considered as
delivered. delivered.
For a periodic networking pattern such as an automation control loop, For a periodic networking pattern such as an automation control loop,
this number is proportional to the Mean Time Between Failures (MTBF). this number is proportional to the Mean Time Between Failures (MTBF).
When a single fault can have dramatic consequences, the MTBF When a single fault can have dramatic consequences, the MTBF
expresses the chances that the unwanted fault event occurs. In data expresses the chances that the unwanted fault event occurs. In data
networks, this is rarely the case. Packet loss cannot be fully networks, this is rarely the case. Packet loss cannot be fully
avoided and the systems are built to resist some loss, e.g., using avoided, and the systems are built to resist some loss. This can be
redundancy with Retries (as in HARQ), Packet Replication and done by using redundancy with retries (as in HARQ), Packet
Elimination (PRE) FEC, Network Coding (e.g., using FEC with SCHC Replication and Elimination (PRE) FEC, and Network Coding (e.g.,
[RFC8724] fragments), or, in a typical control loop, by linear using FEC with Static Context Header Compression (SCHC) [RFC8724]
interpolation from the previous measurements. fragments). Also, in a typical control loop, linear interpolation
from the previous measurements can be used.
But the linear interpolation method cannot resist multiple However, the linear interpolation method cannot resist multiple
consecutive losses, and a high MTBF is desired as a guarantee that consecutive losses, and a high MTBF is desired as a guarantee that
this does not happen, in other words that the number of losses-in- this does not happen, in other words, that the number of losses in a
a-row can be bounded. In that case, what is really desired is a row can be bounded. In this case, what is really desired is a
Maximum Consecutive Loss (MCL). (See also section 5.9.5 in [DLEP].) Maximum Consecutive Loss (MCL). (See also Section 5.9.5 of [DLEP].)
If the number of losses in a row passes the MCL, the control loop has If the number of losses in a row passes the MCL, the control loop has
to abort and the system, e.g., the production line, may need to enter to abort, and the system (e.g., the production line) may need to
an emergency stop condition. enter an emergency stop condition.
Engineers that build automated processes may use the network Engineers that build automated processes may use the network
reliability expressed in nines as an MTBF as a proxy to indicate an reliability expressed in nines as an MTBF as a proxy to indicate an
MCL, e.g., as described in section 7.4 of the "Deterministic MCL, e.g., as described in Section 7.4 of "Deterministic Networking
Networking Use Cases" [RFC8578]. Use Cases" [RFC8578].
4.3. Wireless Effects Affecting Reliability 4.3. Wireless Effects Affecting Reliability
In contrast with wired networks, errors in transmission are the In contrast with wired networks, errors in transmission are the
predominant source of packet loss in wireless networks. predominant source of packet loss in wireless networks.
The root cause for the loss may be of multiple origins, calling for The root cause for the loss may be of multiple origins, calling for
the use of different forms of diversity: the use of different forms of diversity:
Multipath Fading: A destructive interference by a reflection of the Multipath fading: A destructive interference by a reflection of the
original signal. original signal.
A radio signal may be received directly (line-of-sight) and/or as A radio signal may be received directly (line-of-sight) and/or as
a reflection on a physical structure (echo). The reflections take a reflection on a physical structure (echo). The reflections take
a longer path and are delayed by the extra distance divided by the a longer path and are delayed by the extra distance divided by the
speed of light in the medium. Depending on the frequency, the speed of light in the medium. Depending on the frequency, the
echo lands with a different phase which may add up to echo lands with a different phase, which may either add up to
(constructive interference) or cancel (destructive interference) (constructive interference) or cancel (destructive interference)
the direct signal. the direct signal.
The affected frequencies depend on the relative position of the The affected frequencies depend on the relative position of the
sender, the receiver, and all the reflecting objects in the sender, the receiver, and all the reflecting objects in the
environment. A given hop suffers from multipath fading for environment. A given hop suffers from multipath fading for
multiple packets in a row till a physical movement changes the multiple packets in a row until a physical movement changes the
reflection patterns. reflection patterns.
Co-channel Interference: Energy in the spectrum used for the Co-channel interference: Energy in the spectrum used for the
transmission confuses the receiver. transmission confuses the receiver.
The wireless medium itself is a Shared Risk Link Group (SRLG) for The wireless medium itself is a Shared Risk Link Group (SRLG) for
nearby users of the same spectrum, as an interference may affect nearby users of the same spectrum, as an interference may affect
multiple co-channel transmissions between different peers within multiple co-channel transmissions between different peers within
the interference domain of the interferer, possibly even when they the interference domain of the interferer, possibly even when they
use different technologies. use different technologies.
Obstacle in Fresnel Zone: The Fresnel zone is an elliptical region Obstacle in Fresnel zone: The Fresnel zone is an elliptical region
of space between and around the transmit and receive antennas in a of space between and around the transmit and receive antennas in a
point-to-point wireless communication. The optimal transmission point-to-point wireless communication. The optimal transmission
happens when it is free of obstacles. happens when it is free of obstacles.
In an environment that is rich in metallic structures and mobile In an environment that is rich in metallic structures and mobile
objects, a single radio link provides a fuzzy service, meaning that objects, a single radio link provides a fuzzy service, meaning that
it cannot be trusted to transport the traffic reliably over a long it cannot be trusted to transport the traffic reliably over a long
period of time. period of time.
Transmission losses are typically not independent, and their nature Transmission losses are typically not independent, and their nature
and duration are unpredictable; as long as a physical object (e.g., a and duration are unpredictable; as long as a physical object (e.g., a
metallic trolley between peers) that affects the transmission is not metallic trolley between peers) that affects the transmission is not
removed, or as long as the interferer (e.g., a radar in the ISM band) removed, or as long as the interferer (e.g., a radar in the ISM band)
keeps transmitting, a continuous stream of packets are affected. keeps transmitting, a continuous stream of packets are affected.
The key technique to combat those unpredictable losses is diversity. The key technique to combat those unpredictable losses is diversity.
Different forms of diversity are necessary to combat different causes Different forms of diversity are necessary to combat different causes
of loss and the use of diversity must be maximized to optimize the of loss, and the use of diversity must be maximized to optimize the
PDR. PDR.
A single packet may be sent at different times (time diversity) over A single packet may be sent at different times (time diversity) over
diverse paths (spatial diversity) that rely on diverse radio channels diverse paths (spatial diversity) that rely on diverse radio channels
(frequency diversity) and diverse PHY technologies, e.g., narrowband (frequency diversity) and diverse PHY technologies (e.g., narrowband
vs. spread spectrum, or diverse codes. Using time diversity defeats versus spread spectrum), or diverse codes. Using time diversity
short-term interferences; spatial diversity combats very local causes defeats short-term interferences; spatial diversity combats very
of interference such as multipath fading; narrowband and spread local causes of interference such as multipath fading; narrowband and
spectrum are relatively innocuous to one another and can be used for spread spectrum are relatively innocuous to one another and can be
diversity in the presence of the other. used for diversity in the presence of the other.
5. The RAW Conceptual Model 5. The RAW Conceptual Model
RAW extends the conceptual model described in section 4 of the DetNet RAW extends the conceptual model described in Section 4 of
Architecture [DetNet-ARCHI] with the PLR at the Service sub-layer, as "Deterministic Networking Architecture" [DetNet-ARCHI] with the PLR
illustrated in Figure 3. The PLR (see Section 6.5) is a point of at the Service sub-layer, as illustrated in Figure 3. The PLR (see
local reaction to provide additional agility against transmission Section 6.5) is a point of local reaction to provide additional
loss. The PLR can act, e.g., based on indications from the lower agility against transmission loss. For example, the PLR can act
layer or based on OAM. based on indications from the lower layer or based on OAM.
| packets going | ^ packets coming ^ | packets going | ^ packets coming ^
v down the stack v | up the stack | v down the stack v | up the stack |
+-----------------------+ +-----------------------+ +-----------------------+ +-----------------------+
| Source | | Destination | | Source | | Destination |
+-----------------------+ +-----------------------+ +-----------------------+ +-----------------------+
| Service sub-layer: | | Service sub-layer: | | Service sub-layer: | | Service sub-layer: |
| Packet sequencing | | Duplicate elimination | | Packet sequencing | | Duplicate elimination |
| Flow replication | | Flow merging | | Flow replication | | Flow merging |
| Packet encoding | | Packet decoding | | Packet encoding | | Packet decoding |
skipping to change at page 23, line 34 skipping to change at line 1058
+-----------------------+ +-----------------------+ +-----------------------+ +-----------------------+
| Forwarding sub-layer: | | Forwarding sub-layer: | | Forwarding sub-layer: | | Forwarding sub-layer: |
| Resource allocation | | Resource allocation | | Resource allocation | | Resource allocation |
| Explicit routes | | Explicit routes | | Explicit routes | | Explicit routes |
+-----------------------+ +-----------------------+ +-----------------------+ +-----------------------+
| Lower layers | | Lower layers | | Lower layers | | Lower layers |
+-----------------------+ +-----------------------+ +-----------------------+ +-----------------------+
v ^ v ^
\_________________________/ \_________________________/
Figure 3: Extended DetNet Data-Plane Protocol Stack Figure 3: Extended DetNet Data Plane Protocol Stack
5.1. The RAW Planes 5.1. The RAW Planes
The RAW Nodes are DetNet Relay Nodes that operate in the RAW Network The RAW Nodes are DetNet Relay Nodes that operate in the RAW Network
Plane and are capable of additional diversity mechanisms and Plane and are capable of additional diversity mechanisms and
measurement functions related to the radio interface. RAW leverages measurement functions related to the radio interface. RAW leverages
an Operational Plane orientation function (that typically operates an Operational Plane orientation function (that typically operates
inside the Ingress Edge Nodes) to dynamically adapt the path of the inside the Ingress Edge Nodes) to dynamically adapt the path of the
packets and optimizes the resource usage. packets and optimize the resource usage.
In the case of centralized routing operations, the RAW Controller In the case of centralized routing operations, the RAW Controller
Plane Function (CPF) interacts with RAW Nodes over a Southbound API. Plane Function (CPF) interacts with RAW Nodes over a Southbound API.
It consumes data and information from the network and generates It consumes data and information from the network and generates
knowledge and wisdom to help steer the traffic optimally inside a knowledge and wisdom to help steer the traffic optimally inside a
recovery graph. recovery graph.
DetNet Routing DetNet Routing
CPF CPF CPF CPF CPF CPF CPF CPF
skipping to change at page 24, line 30 skipping to change at line 1100
Figure 4: RAW Nodes (Centralized Routing Case) Figure 4: RAW Nodes (Centralized Routing Case)
When a new flow is defined, the routing function uses its current When a new flow is defined, the routing function uses its current
knowledge of the network to build a new recovery graph between an knowledge of the network to build a new recovery graph between an
Ingress End System and an Egress End System for that flow; it Ingress End System and an Egress End System for that flow; it
indicates to the RAW Nodes where the PREOF and/or radio diversity and indicates to the RAW Nodes where the PREOF and/or radio diversity and
reliability operations may be actioned in the Network Plane. reliability operations may be actioned in the Network Plane.
* The recovery graph may be strict, meaning that the DetNet * The recovery graph may be strict, meaning that the DetNet
forwarding sub-layer operations are enforced end-to-end forwarding sub-layer operations are enforced end to end.
* The recovery graph may be expressed loosely to enable traversing a * The recovery graph may be expressed loosely to enable traversing a
non-RAW subnetwork as in Figure 7. In that case, RAW cannot non-RAW subnetwork as in Figure 7. In that case, RAW cannot
leverage end-to-end DetNet and cannot provide latency guarantees. leverage end-to-end DetNet and cannot provide latency guarantees.
The information that the orientation function reports to the routing The information that the orientation function reports to the routing
function includes may be a time-aggregated, e.g., statistical function includes may be a time-aggregated, e.g., statistical
fashion, to match the longer-term operation of the routing function. fashion, to match the longer-term operation of the routing function.
Example information includes Link-Layer metrics such as Link Example information includes link-layer metrics such as link
bandwidth (the medium speed depends dynamically on the mode of the bandwidth (the medium speed depends dynamically on the mode of the
physical (PHY) layer), number of flows (bandwidth that can be PHY layer), number of flows (bandwidth that can be reserved for a
reserved for a flow depends on the number and size of flows sharing flow depends on the number and size of flows sharing the spectrum),
the spectrum) and average and mean squared deviation of availability and the average and mean squared deviation of availability and
and reliability metrics, such as Packet Delivery Ratio (PDR) over reliability metrics (such as PDR) over long periods of time. It may
long periods of time. It may also report an aggregated expected also report an aggregated Expected Transmission Count (ETX) or a
transmission count (ETX), or a variation of it. variation of it.
Based on those metrics, the routing function installs the recovery Based on those metrics, the routing function installs the recovery
graph with enough redundant forwarding solutions to ensure that the graph with enough redundant forwarding solutions to ensure that the
Network Plane can reliably deliver the packets within an SLA Network Plane can reliably deliver the packets within an SLA
associated with the flows that it transports. The SLA defines end- associated with the flows that it transports. The SLA defines end-
to-end reliability and availability requirements, in which to-end reliability and availability requirements, in which
reliability may be expressed as a successful delivery in-order and reliability may be expressed as a successful delivery in order and
within a bounded delay of at least one copy of a packet. within a bounded delay of at least one copy of a packet.
Depending on the use case and the SLA, the recovery graph may Depending on the use case and the SLA, the recovery graph may
comprise non-RAW segments, either interleaved inside the recovery comprise non-RAW segments, either interleaved inside the recovery
graph (e.g. over tunnels), or all the way to the Egress End Node graph (e.g., over tunnels) or all the way to the Egress End Node
(e.g., a server in the local wired domain). RAW observes the Lower- (e.g., a server in the local wired domain). RAW observes the lower-
Layer Links between RAW nodes (typically, radio links) and the end- layer links between RAW nodes (typically radio links) and the end-to-
to-end Network Layer operation to decide at all times which of the end network-layer operation to decide at all times which of the
diversity schemes is actioned by which RAW Nodes. diversity schemes is actioned by which RAW Nodes.
Once a recovery graph is established, per-segment and end-to-end Once a recovery graph is established, per-segment and end-to-end
reliability and availability statistics are periodically reported to reliability and availability statistics are periodically reported to
the routing function to ensure that the SLA can be met or if not, the routing function to ensure that the SLA can be met; if not, then
then have the recovery graph recomputed. the recovery graph is recomputed.
5.2. RAW vs. Upper and Lower Layers 5.2. RAW Versus Upper and Lower Layers
RAW builds on DetNet-provided features such as scheduling and RAW builds on DetNet-provided features such as scheduling and
shaping. In particular, RAW inherits the DetNet guarantees on end- shaping. In particular, RAW inherits the DetNet guarantees on end-
to-end latency, which can be tuned to ensure that DetNet and RAW to-end latency, which can be tuned to ensure that DetNet and RAW
reliability mechanisms have no side effect on upper layers, e.g., on reliability mechanisms have no side effect on upper layers, e.g., on
transport-layer packet recovery. RAW operations include possible transport-layer packet recovery. RAW operations include possible
rerouting, which in turn may affect the ordering of a burst of rerouting, which in turn may affect the ordering of a burst of
packets. RAW also inherits PREOF from DetNet, which can be used to packets. RAW also inherits PREOF from DetNet, which can be used to
reorder packets before delivery to the upper layers. As a result, reorder packets before delivery to the upper layers. As a result,
DetNet in general and RAW in particular offer a smoother transport DetNet in general and RAW in particular offer a smoother transport
experience for the upper layers than the Internet at large with experience for the upper layers than the Internet at large, with
ultra-low jitter and loss. ultra-low jitter and loss.
RAW improves the reliability of transmissions and the availability of RAW improves the reliability of transmissions and the availability of
the communication resources, and should be seen as a dynamic communication resources, and should be seen as a dynamic optimization
optimization of the use of redundancy to maintain it within certain of the use of redundancy to maintain it within certain boundaries.
boundaries. For instance, ARQ, which provides 1-hop reliability For instance, ARQ (which provides one-hop reliability through
through acknowledgements and retries, and FEC codes such as turbo acknowledgements and retries) and FEC codes (such as turbo codes
codes which reduce the PER, are typically operated at Layer-2 and which reduce the PER) are typically operated at Layer 2 and Layer 1,
Layer-1 respectively. In both cases, redundant transmissions improve respectively. In both cases, redundant transmissions improve the
the 1-hop reliability at the expense of energy and latency, which are one-hop reliability at the expense of energy and latency, which are
the resources that RAW must control. In order to achieve its goals, the resources that RAW must control. In order to achieve its goals,
RAW may leverage the lower-layer operations by abstracting the RAW may leverage the lower-layer operations by abstracting the
concept and providing hints to the lower layers on the desired concept and providing hints to the lower layers on the desired
outcome, e.g., in terms of reliability and timeliness, as opposed to outcome (e.g., in terms of reliability and timeliness), as opposed to
performing the actual operations at Layer-3. performing the actual operations at Layer 3.
Guarantees such as bounded latency depend on the upper layers Guarantees such as bounded latency depend on the upper layers
(Transport or Application) to provide the payload in volumes and at (transport or application) to provide the payload in volumes and at
times that match the contract with the DetNet sub-layers and the times that match the contract with the DetNet sub-layers and the
layers below. Excess of incoming traffic at the DetNet Ingress may layers below. An excess of incoming traffic at the DetNet Ingress
result in dropping or queueing of packets, and can entail loss, may result in dropping or queueing of packets and can entail loss,
latency, or jitter, and therefore, violate the guarantees that are latency, or jitter; this violates the guarantees that are provided
provided inside the DetNet Network. inside the DetNet Network.
When the traffic from upper layers matches the expectation of the When the traffic from upper layers matches the expectation of the
lower layers, RAW still depends on DetNet mechanisms and the lower lower layers, RAW still depends on DetNet mechanisms and the lower
layers to provide the timing and physical resource guarantees that layers to provide the timing and physical resource guarantees that
are needed to match the traffic SLA. When the availability of the are needed to match the traffic SLA. When the availability of the
physical resource varies, RAW acts on the distribution of the traffic physical resource varies, RAW acts on the distribution of the traffic
to leverage alternates within a finite set of potential resources. to leverage alternates within a finite set of potential resources.
The Operational Plane elements (Routing and OAM control) may gather The Operational Plane elements (routing and OAM control) may gather
aggregated information from lower layers about e.g., link quality, aggregated information from lower layers (e.g., information about
either via measurement or communication with the lower layer. This link quality), via measurement or communication with the lower layer.
information may be obtained from inside the device using specialized This information may be obtained from inside the device using
APIs (e.g., L2 triggers), via monitoring and measurement protocols specialized APIs (e.g., L2 triggers) via monitoring and measurement
such as BFD [RFC5880] and STAMP [RFC8762], respectively, or via a protocols such as Bidirectional Forwarding Detection (BFD) [RFC5880]
control protocol exchange with the lower layer via, e.g., DLEP and Simple Two-way Active Measurement Protocol (STAMP) [RFC8762],
[DLEP]. It may then be processed and exported through OAM messaging respectively, or via a control protocol exchange with the lower layer
or via a YANG data model, and exposed to the Controller Plane. (e.g., DLEP [DLEP]). It may then be processed and exported through
OAM messaging or via a YANG data model and exposed to the Controller
Plane.
5.3. RAW and DetNet 5.3. RAW and DetNet
RAW leverages the DetNet Forwarding sub-layer and requires the RAW leverages the DetNet Forwarding sub-layer and requires the
support of OAM in DetNet Transit Nodes (see Figure 3 of support of OAM in DetNet Transit Nodes (see Figure 3 of
[DetNet-ARCHI]) for the dynamic acquisition of link capacity and [DetNet-ARCHI]) for the dynamic acquisition of link capacity and
state to maintain a strict RAW service, end-to-end, over a DetNet state to maintain a strict RAW service end to end over a DetNet
Network. In turn, DetNet and thus RAW may benefit from / leverage Network. In turn, DetNet and thus RAW may benefit from or leverage
functionality such as provided by TSN at the lower layers. functionality such as that provided by TSN at the lower layers.
RAW extends DetNet to improve the protection against link errors such RAW extends DetNet to improve the protection against link errors such
as transient flapping that are far more common in wireless links. as transient flapping that are far more common in wireless links.
Nevertheless, the RAW methods are for the most part applicable to Nevertheless, for the most part, the RAW methods are applicable to
wired links as well, e.g., when energy savings are desirable and the wired links as well, e.g., when energy savings are desirable and the
available path diversity exceeds 1+1 linear redundancy. available path diversity exceeds 1+1 linear redundancy.
RAW adds sub-layer functions that operate in the DetNet Operational RAW adds sub-layer functions that operate in the DetNet Operational
Plane, which is part of the Network Plane. The RAW orientation Plane, which is part of the Network Plane. The RAW orientation
function may run only in the DetNet Edge Nodes (Ingress Edge Node or function may run only in the DetNet Edge Nodes (Ingress Edge Node or
End System), or it also run in DetNet Relay Nodes when the RAW End System), or it can also run in DetNet Relay Nodes when the RAW
operations are distributed along the recovery graph. The RAW Service operations are distributed along the recovery graph. The RAW Service
sub-layer includes the PLR, which decides the DetNet Path for the sub-layer includes the PLR, which decides the DetNet Path for the
future packets of a flow along the DetNet Path, Maintenance End future packets of a flow along the DetNet Path, Maintenance End
Points (MEPs) on edge nodes, and Maintenance Intermediate Points Points (MEPs) on edge nodes, and Maintenance Intermediate Points
(MIPs) within. The MEPs trigger, and learn from, OAM observations, (MIPs) within. The MEPs trigger, and learn from, OAM observations
and feed the PLR for its next decision. and feed the PLR for its next decision.
As illustrated in Figure 5, RAW extends the DetNet Stack (see As illustrated in Figure 5, RAW extends the DetNet Stack (see
Figure 4 of [DetNet-ARCHI] and Figure 3) with additional Figure 4 of [DetNet-ARCHI] and Figure 3) with additional
functionality at the DetNet Service sub-layer for the actuation of functionality at the DetNet Service sub-layer for the actuation of
PREOF based on the PLR decision. DetNet operates at Layer-3, PREOF based on the PLR decision. DetNet operates at Layer 3,
leveraging abstractions of the lower layers and APIs that control leveraging abstractions of the lower layers and APIs that control
those abstractions. For instance, DetNet already leverages lower those abstractions. For instance, DetNet already leverages lower
layers for time-sensitive operations such as time synchronization and layers for time-sensitive operations such as time synchronization and
traffic shapers. As the performances of the radio layers are subject traffic shapers. As the performances of the radio layers are subject
to rapid changes, RAW needs more dynamic gauges and knobs. To that to rapid changes, RAW needs more dynamic gauges and knobs. To that
effect, the LL API provides an abstraction to the DetNet layer that effect, the LL API provides an abstraction to the DetNet layer that
can be used to push reliability and timing hints like suggest X can be used to push reliability and timing hints, like suggesting X
retries (min, max) within a time window, or send unicast (one next retries (min, max) within a time window or sending unicast (one next
hop) or multicast (for overhearing). In the other direction up the hop) or multicast (for overhearing). In the other direction up the
stack, the RAW PLR needs hints about the radio conditions such as L2 stack, the RAW PLR needs hints about the radio conditions such as L2
triggers (e.g., RSSI, LQI, or ETX) over all the wireless hops. triggers (e.g., RSSI, LQI, or ETX) over all the wireless hops.
RAW uses various OAM functionalities at the different layers. For RAW uses various OAM functionalities at the different layers. For
instance, the OAM function in the DetNet Service sub-layer may instance, the OAM function in the DetNet Service sub-layer may
perform Active and/or Hybrid OAM to estimate the link and path perform Active and/or Hybrid OAM to estimate the link and path
availability, end-to-end or limited to a Segment. The RAW functions availability, either end to end or limited to a Segment. The RAW
may be present in the Service sub-layer in DetNet Edge and Relay functions may be present in the Service sub-layer in DetNet Edge and
Nodes. Relay Nodes.
+-----------------+ +-------------------+ +-----------------+ +-------------------+
| Routing | | OAM Control | | Routing | | OAM Control |
+-----------------+ +-------------------+ +-----------------+ +-------------------+
Controller Plane Controller Plane
+-+-+-+-+-+-+-+-+ Southbound Interface -+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ Southbound Interface -+-+-+-+-+-+-+-+-+-+-+-+
Network Plane Network Plane
| |
Operational Plane . Data Plane Operational Plane . Data Plane
| |
+-----------------+ . +-----------------+ .
| Orientation | | | Orientation | |
+-----------------+ . +-----------------+ .
| |
+-----------------+ +-------------------+ . +-----------------+ +-------------------+ .
| Point of | | OAM Maintenance | | | Point of Local | | OAM Maintenance | |
| local Repair | | End Point (MEP) | . | Repair (PLR) | | End Point (MEP) | .
+-----------------+ +-------------------+ | +-----------------+ +-------------------+ |
. .
| |
Figure 5: RAW function placement (Centralized Routing Case) Figure 5: RAW Function Placement (Centralized Routing Case)
There are two main proposed models to deploy RAW and DetNet. In the There are two main proposed models to deploy RAW and DetNet: strict
first model (strict) (illustrated in Figure 6), RAW operates over a (Figure 6) and loose (Figure 7). In the strict model, illustrated in
continuous DetNet Service end-to-end between the Ingress and the Figure 6, RAW operates over a continuous DetNet service end to end
Egress Edge Nodes or End Systems. between the Ingress and the Egress Edge Nodes or End Systems.
sIn the second model (loose), RAW may traverse a section of the In the loose model, illustrated in Figure 7, RAW may traverse a
network that is not serviced by DetNet. RAW / OAM may observe the section of the network that is not serviced by DetNet. RAW / OAM may
end-to-end traffic and make the best of the available resources, but observe the end-to-end traffic and make the best of the available
it may not expect the DetNet guarantees over all paths. For resources, but it may not expect the DetNet guarantees over all
instance, the packets between two wireless entities may be relayed paths. For instance, the packets between two wireless entities may
over a wired infrastructure, in which case RAW observes and controls be relayed over a wired infrastructure, in which case RAW observes
the transmission over the wireless first and last hops, as well as and controls the transmission over the wireless first and last hops,
end-to-end metrics such as latency, jitter, and delivery ratio. This as well as end-to-end metrics such as latency, jitter, and delivery
operation is loose since the structure and properties of the wired ratio. This operation is loose since the structure and properties of
infrastructure are ignored, and may be either controlled by other the wired infrastructure are ignored and may be either controlled by
means such as DetNet/TSN, or neglected in the face of the wireless other means such as DetNet/TSN or neglected in the face of the
hops. wireless hops.
A minimal Forwarding sub-layer service is provided at all DetNet A minimal Forwarding sub-layer service is provided at all DetNet
Nodes to ensure that the OAM information flows. DetNet Relay Nodes Nodes to ensure that the OAM information flows. DetNet Relay Nodes
may or may not support RAW services, whereas the DetNet Edge Nodes may or may not support RAW services, whereas the DetNet Edge Nodes
are required to support RAW in any case. DetNet guarantees, such as are required to support RAW in any case. DetNet guarantees, such as
bounded latency, are provided end-to-end. RAW extends the DetNet bounded latency, are provided end to end. RAW extends the DetNet
Service sub-layer to optimize the use of resources. Service sub-layer to optimize the use of resources.
--------------------Flow Direction----------------------------------> --------------------Flow Direction---------------------------------->
+---------+ +---------+
| RAW | | RAW |
| Control | | Control |
+---------+ +---------+ +---------+ +---------+ +---------+ +---------+
| RAW + | | RAW + | | RAW + | | RAW + | | RAW + | | RAW + |
| DetNet | | DetNet | | DetNet | | DetNet | | DetNet | | DetNet |
skipping to change at page 29, line 32 skipping to change at line 1312
| DetNet | | DetNet |
| Forwarding | | Forwarding |
+------------------------------------------------------------------+ +------------------------------------------------------------------+
Ingress Transit Relay Egress Ingress Transit Relay Egress
Edge ... Nodes ... Nodes ... Edge Edge ... Nodes ... Nodes ... Edge
Node Node Node Node
<------------------End-to-End DetNet Service-----------------------> <------------------End-to-End DetNet Service----------------------->
Figure 6: (Strict) RAW over DetNet Figure 6: RAW over DetNet (Strict Model)
In the second model (loose), illustrated in Figure 7, RAW operates In the loose model (illustrated in Figure 7), RAW operates over a
over a partial DetNet Service where typically only the Ingress and partial DetNet service where typically only the Ingress and the
the Egress End Systems support RAW. The DetNet Domain may extend Egress End Systems support RAW. The DetNet domain may extend beyond
beyond the Ingress Node, or there may be a DetNet domain starting at the Ingress Node, or there may be a DetNet domain starting at an
an Ingress Edge Node at the first hop after the End System. Ingress Edge Node at the first hop after the End System.
In the loose model, RAW cannot observe the hops in the network, and In the loose model, RAW cannot observe the hops in the network, and
the path beyond the first hop is opaque; RAW can still observe the the path beyond the first hop is opaque; RAW can still observe the
end-to-end behavior and use Layer-3 measurements to decide whether to end-to-end behavior and use Layer 3 measurements to decide whether to
replicate a packet and select the first-hop interface(s). replicate a packet and select the first-hop interface(s).
--------------------Flow Direction----------------------------------> --------------------Flow Direction---------------------------------->
+---------+ +---------+
| RAW | | RAW |
| Control | | Control |
+---------+ +---------+ +---------+ +---------+ +---------+ +---------+
| RAW + | | DetNet | | RAW + | | RAW + | | DetNet | | RAW + |
| DetNet | | Only | | DetNet | | DetNet | | Only | | DetNet |
skipping to change at page 30, line 25 skipping to change at line 1345
| DetNet |_______________| DetNet | | DetNet |_______________| DetNet |
| Forwarding _______________ Forwarding | | Forwarding _______________ Forwarding |
+------------------------------------+ +-------------+ +------------------------------------+ +-------------+
Ingress Transit Relay Tunnel Egress Ingress Transit Relay Tunnel Egress
End ... Nodes ... Nodes ... ... End End ... Nodes ... Nodes ... ... End
System System System System
<---------------Partitioned DetNet Service-------------------------> <---------------Partitioned DetNet Service------------------------->
Figure 7: Loose RAW Figure 7: RAW over DetNet (Loose Model)
6. The RAW Control Loop 6. The RAW Control Loop
The RAW Architecture is based on an abstract OODA Loop that controls The RAW architecture is based on an abstract OODA Loop that controls
the operation of a Recovery Graph. The generic concept involves: the operation of a recovery graph. The generic concept involves the
following:
1. Operational Plane measurement protocols for OAM to observe (like 1. Operational Plane measurement protocols for OAM to observe (like
the first O in OODA) some or all hops along a recovery graph as the first "O" in "OODA") some or all hops along a recovery graph
well as the end-to-end packet delivery. as well as the end-to-end packet delivery.
2. The DetNet Controller Plane establish primary and protection 2. The DetNet Controller Plane establishes primary and protection
paths for use by the RAW Network Plane. The orientation function paths for use by the RAW Network Plane. The orientation function
reports data and information such as link statistics to be used reports data and information such as link statistics to be used
by the routing function to compute, install, and maintain the by the routing function to compute, install, and maintain the
recovery graphs. The routing function may also generate recovery graphs. The routing function may also generate
intelligence such as a trained model for link quality prediction, intelligence such as a trained model for link quality prediction,
which in turn can be used by the orientation function (like the which in turn can be used by the orientation function (like the
second O in OODA) to influence the Path selection by the PLR second "O" in "OODA") to influence the Path selection by the PLR
within the RAW OODA loop. within the RAW OODA loop.
3. A PLR operates at the DetNet Service sub-layer and hosts the 3. A PLR operates at the DetNet Service sub-layer and hosts the
decision function (like the D in OODA) of which DetNet Paths to decision function (like the "D" in "OODA"). The decision
use for the future packets that are routed within the recovery function determines which DetNet Paths will be used for future
graph. packets that are routed within the recovery graph.
4. Service protection actions that are actuated or triggered over 4. Service protection actions that are actuated or triggered over
the LL API by the PLR to increase the reliability of the end-to- the LL API by the PLR to increase the reliability of the end-to-
end transmissions. The RAW architecture also covers in-situ end transmissions. The RAW architecture also covers in-situ
signaling that is embedded within live user traffic [RFC9378], signaling that is embedded within live user traffic [RFC9378]
e.g., via OAM, when the decision is acted (like the A in OODA) (e.g., via OAM) when the decision is acted (like the "A" in
upon by a node that is downstream in the recovery graph from the "OODA") upon by a node that is downstream in the recovery graph
PLR. from the PLR.
The overall OODA Loop optimizes the use of redundancy to achieve the The overall OODA Loop optimizes the use of redundancy to achieve the
required reliability and availability SLO(s) while minimizing the use required reliability and availability SLO(s) while minimizing the use
of constrained resources such as spectrum and battery. of constrained resources such as spectrum and battery.
6.1. Routing Time-Scale vs. Forwarding Time-Scale 6.1. Routing Timescale Versus Forwarding Timescale
With DetNet, the Controller Plane Function handles the routing With DetNet, the Controller Plane Function (CPF) handles the routing
computation and maintenance. With RAW, the routing operation is computation and maintenance. With RAW, the routing operation is
segregated from the RAW Control Loop, so it may reside in the segregated from the RAW Control Loop, so it may reside in the
Controller Plane whereas the control loop itself happens in the Controller Plane, whereas the control loop itself happens in the
Network Plane. To achieve RAW capabilities, the routing operation is Network Plane. To achieve RAW capabilities, the routing operation is
extended to generate the information required by the orientation extended to generate the information required by the orientation
function in the loop. The routing function may, e.g., propose DetNet function in the loop. For example, the routing function may propose
Paths to be used as a reflex action in response to network events, or DetNet Paths to be used as a reflex action in response to network
provide an aggregated history that the orientation function can use events or provide an aggregated history that the orientation function
to make a decision. can use to make a decision.
In a wireless mesh, the path to a routing function located in the In a wireless mesh, the path to a routing function located in the
controller plane can be expensive and slow, possibly going across the controller plane can be expensive and slow, possibly going across the
whole mesh and back. Reaching to the Controller Plane can also be whole mesh and back. Reaching the Controller Plane can also be slow
slow in regards to the speed of events that affect the forwarding in regard to the speed of events that affect the forwarding operation
operation in the Network Plane at the radio layer. Note that a in the Network Plane at the radio layer. Note that a distributed
distributed routing protocol may also take time and consume excessive routing protocol may also take time and consume excessive wireless
wireless resources to reconverge to a new optimized state. resources to reconverge to a new optimized state.
As a result, the DetNet routing function is not expected to be aware As a result, the DetNet routing function is not expected to be aware
of and to react to very transient changes. The abstraction of a link of and react to very transient changes. The abstraction of a link at
at the routing level is expected to use statistical metrics that the routing level is expected to use statistical metrics that
aggregate the behavior of a link over long periods of time, and aggregate the behavior of a link over long periods of time and
represent its properties as shades of gray as opposed to numerical represent its properties as shades of gray as opposed to numerical
values such as a link quality indicator, or a Boolean value for values such as a link quality indicator or a Boolean value for either
either up or down. up or down.
The interaction between the network nodes and the routing function is The interaction between the network nodes and the routing function is
handled by the orientation function, which builds reports to the handled by the orientation function, which builds reports to the
routing function and sends control information in a digested form routing function and sends control information in a digested form
back to the RAW node, to be used inside a forwarding control loop for back to the RAW node to be used inside a forwarding control loop for
traffic steering. traffic steering.
Figure 8 illustrates a Network Plane recovery graph with links P-Q Figure 8 illustrates a Network Plane recovery graph with links P-Q
and N-E flapping, possibly in a transient fashion due to a short-term and N-E flapping, possibly in a transient fashion due to short-term
interferences, and possibly for a longer time, e.g., due to obstacles interferences and possibly for a longer time (e.g., due to obstacles
between the sender and the receiver or hardware failures. In order between the sender and the receiver or hardware failures). In order
to maintain a received redundancy around a value of, say, 2, RAW may to maintain a received redundancy around a value of 2 (for instance),
leverage a higher ARQ on these hops if the overall latency permits RAW may leverage a higher ARQ on these hops if the overall latency
the extra delay, or enable alternate paths between ingress I and permits the extra delay or enable alternate paths between ingress I
egress E. For instance, RAW may enable protection path I ==> F ==> N and egress E. For instance, RAW may enable protection path I ==> F
==> Q ==> M ==> R ==> E that routes around both issues and provides ==> N ==> Q ==> M ==> R ==> E that routes around both issues and
some degree of spatial diversity with protection path I ==> A ==> B provides some degree of spatial diversity with protection path I ==>
==> C ==> D ==> E. A ==> B ==> C ==> D ==> E.
+----------------+ +----------------+
| DetNet | | DetNet |
| Routing | | Routing |
+----------------+ +----------------+
^ ^
| |
Slow Slow
| Controller Plane | Controller Plane
_-._-._-._-._-._-. | ._-._-._-._-._-._-._-._-._-._-._-._- _-._-._-._-._-._-. | ._-._-._-._-._-._-._-._-._-._-._-._-
skipping to change at page 32, line 43 skipping to change at line 1457
( A--------B---C----D ) ( A--------B---C----D )
_ - / \ / \ --._ _ - / \ / \ --._
( I---F--------N--***-- E - ( I---F--------N--***-- E -
-_ \ / / ) -_ \ / / )
( P--***---Q----M---R . ( P--***---Q----M---R .
_ )- ._ _ )- ._
- <------ Fast -------> ) - <------ Fast -------> )
( -._ .- ( -._ .-
(_.___.._____________.____.._ __-____) (_.___.._____________.____.._ __-____)
*** = flapping at this time *** = flapping at this time
Figure 8: Time-Scales Figure 8: Timescales
In the case of wireless, the changes that affect the forwarding In the case of wireless, the changes that affect the forwarding
decision can happen frequently and often for short durations, e.g., a decision can happen frequently and often for short durations. An
mobile object moves between a transmitter and a receiver, and cancels example of this is a mobile object that moves between a transmitter
the line of sight transmission for a few seconds, or, a radar and a receiver and cancels the line-of-sight transmission for a few
measures the depth of a pool using the ISM band, and interferes on a seconds. Another example is radar that measures the depth of a pool
particular channel for a split second. using the ISM band and interferes on a particular channel for a split
second.
There is thus a desire to separate the long-term computation of the Thus, there is a desire to separate the long-term computation of the
route and the short-term forwarding decision. In that model, the route and the short-term forwarding decision. In that model, the
routing operation computes a recovery graph that enables multiple routing operation computes a recovery graph that enables multiple
Unequal Cost Multi-Path (UCMP) forwarding solutions along so-called Unequal-Cost Multipath (UCMP) forwarding solutions along so-called
protection paths, and leaves it to the Network Plane to make the protection paths and leaves it to the Network Plane to make the
short-term decision of which of these possibilities should be used short-term decision of which of these possibilities should be used
for which upcoming packets / flows. for which upcoming packets and flows.
In the context of Traffic Engineering (TE), an alternate path can be In the context of Traffic Engineering (TE), an alternate path can be
used upon the detection of a failure in the main path, e.g., using used upon the detection of a failure in the main path, e.g., using
OAM in Multiprotocol Label Switching - Transport Profile (MPLS-TP) or OAM in Multiprotocol Label Switching - Transport Profile (MPLS-TP) or
BFD over a collection of Software-Defined Wide Area Network (SD-WAN) BFD over a collection of Software-Defined Wide Area Network (SD-WAN)
tunnels. tunnels.
RAW formalizes a forwarding time-scale that may be order(s) of RAW formalizes a forwarding timescale that may be order(s) of
magnitude shorter than the Controller Plane routing time-scale, and magnitude shorter than the Controller Plane routing timescale and
separates the protocols and metrics that are used at both scales. separates the protocols and metrics that are used at both scales.
Routing can operate on long-term statistics such as delivery ratio Routing can operate on long-term statistics such as delivery ratio
over minutes to hours, but as a first approximation can ignore the over minutes to hours, but as a first approximation, it can ignore
cause of transient losses. On the other hand, the RAW forwarding the cause of transient losses. On the other hand, the RAW forwarding
decision is made at the scale of a burst of packets, and uses decision is made at the scale of a burst of packets and uses
information that must be pertinent at the present time for the information that must be pertinent at the present time for the
current transmission(s). current transmission(s).
6.2. OODA Loop 6.2. OODA Loop
The RAW Architecture applies the generic OODA model to continuously The RAW architecture applies the generic OODA model to continuously
optimize the spectrum and energy used to forward packets within a optimize the spectrum and energy used to forward packets within a
recovery graph, instantiating the OODA steps as follows: recovery graph, instantiating the OODA steps as follows:
Observe: Network Plane measurements, including protocols for OAM, to Observe: Network Plane measurements, including protocols for OAM,
Observe the local state of the links and some or all hops along a observe the local state of the links and some or all hops along a
recovery graph as well as the end-to-end packet delivery (see more recovery graph as well as the end-to-end packet delivery (see more
in Section 6.3). Information can also be provided by lower-layer in Section 6.3). Information can also be provided by lower-layer
interfaces such as DLEP; interfaces such as DLEP.
Orient: The orientation function, which reports data and information Orient: The orientation function reports data and information such
such as the link statistics, and leverages offline-computed wisdom as the link statistics and leverages offline-computed wisdom and
and knowledge to Orient the PLR for its forwarding decision (see knowledge to orient the PLR for its forwarding decision (see more
more in Section 6.4); in Section 6.4).
Decide: A local PLR that decides which DetNet Path to use for the Decide: A local PLR decides which DetNet Path to use for future
future packet(s) that are routed along the recovery graph (see packet(s) that are routed along the recovery graph (see more in
more in Section 6.5); Section 6.5).
Act: PREOF Data Plane actions are controlled by the PLR over the LL Act: PREOF Data Plane actions are controlled by the PLR over the LL
API to increase the reliability of the end-to-end transmission. API to increase the reliability of the end-to-end transmission.
The RAW architecture also covers in-situ signaling when the The RAW architecture also covers in-situ signaling when the
decision is Acted by a node that is down the recovery graph from decision is acted by a node that is down the recovery graph from
the PLR (see more in Section 6.6). the PLR (see more in Section 6.6).
+-------> Orientation ---------+ +-------> Orientation ---------+
| reflex actions | | reflex actions |
| pre-trained model | | pre-trained model |
| | | |
...................................... ......................................
| | | |
| Service sub-layer | | Service sub-layer |
| v | v
skipping to change at page 34, line 36 skipping to change at line 1541
| | | |
+------- Act (LL API) <--------+ +------- Act (LL API) <--------+
Figure 9: The RAW OODA Loop Figure 9: The RAW OODA Loop
The overall OODA Loop optimizes the use of redundancy to achieve the The overall OODA Loop optimizes the use of redundancy to achieve the
required reliability and availability Service Level Agreement (SLA) required reliability and availability Service Level Agreement (SLA)
while minimizing the use of constrained resources such as spectrum while minimizing the use of constrained resources such as spectrum
and battery. and battery.
6.3. Observe: The RAW OAM 6.3. Observe: RAW OAM
RAW In-situ OAM operation in the Network Plane may observe either a The RAW in-situ OAM operation in the Network Plane may observe either
full recovery graph or the DetNet Path that is being used at this a full recovery graph or the DetNet Path that is being used at this
time. As packets may be load balanced, replicated, eliminated, and / time. As packets may be load balanced, replicated, eliminated, and/
or fragmented for Network Coding FEC, the RAW In-situ operation needs or fragmented for Network Coding FEC, the RAW in-situ operation needs
to be able to signal which operation occurred to an individual to be able to signal which operation occurred to an individual
packet. packet.
Active RAW OAM may be needed to observe the unused segments and Active RAW OAM may be needed to observe the unused segments and
evaluate the desirability of a rerouting decision. evaluate the desirability of a rerouting decision.
Finally, the RAW Service sub-layer Assurance may observe the Finally, the RAW Service sub-layer Assurance may observe the
individual PREOF operation of a DetNet Relay Node to ensure that it individual PREOF operation of a DetNet Relay Node to ensure that it
is conforming; this might require injecting an OAM packet at an is conforming; this might require injecting an OAM packet at an
upstream point inside the recovery graph and extracting that packet upstream point inside the recovery graph and extracting that packet
at another point downstream before it reaches the egress. at another point downstream before it reaches the egress.
This observation feeds the RAW PLR that makes the decision on which This observation feeds the RAW PLR that makes the decision on which
path is used at which RAW Node, for one packet or a small continuous path is used at which RAW Node, for one packet or a small continuous
series of packets. series of packets.
In the case of End-to-End Protection in a Wireless Mesh, the recovery In the case of end-to-end protection in a wireless mesh, the recovery
graph is strict and congruent with the path so all links are graph is strict and congruent with the path so all links are
observed. observed.
Conversely, in the case of Radio Access Protection, illustrated in Conversely, in the case of Radio Access Protection, illustrated in
Figure 10, the recovery graph is Loose and only the first hop is Figure 10, the recovery graph is loose and only the first hop is
observed; the rest of the path is abstracted and considered observed; the rest of the path is abstracted and considered
infinitely reliable. The loss of a packet is attributed to the infinitely reliable. The loss of a packet is attributed to the
first-hop Radio Access Network (RAN), even if a particular loss first-hop Radio Access Network (RAN), even if a particular loss
effectively happens farther down the path. In that case, RAW enables effectively happens farther down the path. In that case, RAW enables
technology diversity (e.g., Wi-Fi and 5G), which in turn improves the technology diversity (e.g., Wi-Fi and 5G), which in turn improves the
diversity in spectrum usage. diversity in spectrum usage.
Opaque to OAM Opaque to OAM
<----------------------------> <---------------------------->
.- .. - .. .- .. - ..
skipping to change at page 35, line 40 skipping to change at line 1594
+-------+ \ ( ). +------+ +-------+ \ ( ). +------+
RAN n ----( ) RAN n ----( )
(_______...___.__...____....__..) (_______...___.__...____....__..)
<-------L2------> <-------L2------>
Observed by OAM Observed by OAM
<----------------------L3-----------------------> <----------------------L3----------------------->
Figure 10: Observed Links in Radio Access Protection Figure 10: Observed Links in Radio Access Protection
The Links that are not observed by OAM are opaque to it, meaning that The links that are not observed by OAM are opaque to it, meaning that
the OAM information is carried across and possibly echoed as data, the OAM information is carried across and possibly echoed as data,
but there is no information captured in intermediate nodes. In the but there is no information captured in intermediate nodes. In the
example above, the Tunnel underlay is opaque and not controlled by example above, the tunnel underlay is opaque and not controlled by
RAW; still the RAW OAM measures the end-to-end latency and delivery RAW; still, RAW OAM measures the end-to-end latency and delivery
ratio for packets sent via RAN 1, RAN 2, and RAN 3, and determines ratio for packets sent via RAN 1, RAN 2, and RAN 3, and determines
whether a packet should be sent over either or a collection of those whether a packet should be sent over either or a collection of those
access links. access links.
6.4. Orient: The RAW-extended DetNet Operational Plane 6.4. Orient: The RAW-Extended DetNet Operational Plane
RAW separates the long time-scale at which a recovery graph is RAW separates the long timescale at which a recovery graph is
computed and installed, from the short time-scale at which the computed and installed from the short timescale at which the
forwarding decision is taken for one or for a few packets (see forwarding decision is taken for one or a few packets (see
Section 6.1) that experience the same path until the network Section 6.1) that experience the same path until the network
conditions evolve and another path is selected within the same conditions evolve and another path is selected within the same
recovery graph. recovery graph.
The recovery graph computation is out of scope, but RAW expects that The recovery graph computation is out of scope, but RAW expects that
the CPF that installs the recovery graph also provides related the CPF that installs the recovery graph also provides related
knowledge in the form of metadata about the links, segments, and knowledge in the form of metadata about the links, segments, and
possible DetNet Paths. That metadata can be a pre-digested possible DetNet Paths. That metadata can be a pre-digested
statistical model, and may include prediction of future flaps and statistical model and may include prediction of future flaps and
packet loss, as well as recommended actions when that happens. packet loss, as well as recommended actions when that happens.
The metadata may include: The metadata may include:
* A set of Pre-Determined DetNet Paths that are prepared to match * A set of pre-determined DetNet Paths that are prepared to match
expected link-degradation profiles, so the DetNet Relay Nodes can expected link-degradation profiles, so the DetNet Relay Nodes can
take reflex rerouting actions when facing a degradation that take reflex rerouting actions when facing a degradation that
matches one such profile; matches one such profile; and
* Link-Quality Statistics history and pre-trained models, e.g., to * Link-quality statistics history and pre-trained models (e.g., to
predict the short-term variation of quality of the links in a predict the short-term variation of quality of the links in a
recovery graph. recovery graph).
The recovery graph is installed with measurable objectives that are The recovery graph is installed with measurable objectives that are
computed by the CPF to achieve the RAW SLA. The objectives can be computed by the CPF to achieve the RAW SLA. The objectives can be
expressed as any of the maximum number of packets lost in a row, expressed as any of the maximum number of packets lost in a row,
bounded latency, maximal jitter, maximum number of interleaved out- bounded latency, maximal jitter, maximum number of interleaved out-
of-order packets, average number of copies received at the of-order packets, average number of copies received at the
elimination point, and maximal delay between the first and the last elimination point, and maximal delay between the first and the last
received copy of the same packet. received copy of the same packet.
6.5. Decide: The Point of Local Repair 6.5. Decide: The Point of Local Repair
The RAW OODA Loop operates at the path selection time-scale to The RAW OODA Loop operates at the path selection timescale to provide
provide agility vs. the brute-force approach of flooding the whole agility versus the brute-force approach of flooding the whole
recovery graph. The OODA Loop controls, within the redundant recovery graph. The OODA Loop controls, within the redundant
solutions that are proposed by the routing function, which is used solutions that are proposed by the routing function, which is used
for each packet to provide a Reliable and Available service while for each packet to provide a reliable and available service while
minimizing the waste of constrained resources. minimizing the waste of constrained resources.
To that effect, RAW defines the Point of Local Repair (PLR), which To that effect, RAW defines the Point of Local Repair (PLR), which
performs rapid local adjustments of the forwarding tables within the performs rapid local adjustments of the forwarding tables within the
path diversity that is available in that in the recovery graph. The path diversity that is available in that in the recovery graph. The
PLR enables exploitation of the richer forwarding capabilities at a PLR enables exploitation of the richer forwarding capabilities at a
faster time-scale over a portion of the recovery graph, in either a faster timescale over a portion of the recovery graph, in either a
loose or a strict fashion. loose or a strict fashion.
The PLR operates on metrics that evolve faster, but that need to be The PLR operates on metrics that evolve quickly and need to be
advertised at a fast rate but only locally, within the recovery advertised at a fast rate (but only locally, within the recovery
graph, and reacts on the metric updates by changing the DetNet path graph), and the PLR reacts on the metric updates by changing the
in use for the affected flows. DetNet path in use for the affected flows.
The rapid changes in the forwarding decisions are made and contained The rapid changes in the forwarding decisions are made and contained
within the scope of a recovery graph and the actions of the PLR are within the scope of a recovery graph, and the actions of the PLR are
not signaled outside the recovery graph. This is as opposed to the not signaled outside the recovery graph. This is as opposed to the
routing function that must observe the whole network and optimize all routing function that must observe the whole network and optimize all
the recovery graphs globally, which can only be done at a slow pace the recovery graphs globally, which can only be done at a slow pace
and using long-term statistical metrics, as presented in Table 1. and with long-term statistical metrics, as presented in Table 1.
+===============+=========================+=====================+ +===============+=========================+=====================+
| | Controller Plane | PLR | | | Controller Plane | PLR |
+===============+=========================+=====================+ +===============+=========================+=====================+
| Communication | Slow, distributed | Fast, local | | Communication | Slow, distributed | Fast, local |
+---------------+-------------------------+---------------------+ +===============+-------------------------+---------------------+
| Time-Scale | Path computation + | Lookup + protection | | Timescale | Path computation + | Lookup + protection |
| (order) | round trip, | switch, micro to | | (order) | round trip, | switch, micro to |
| | milliseconds to seconds | milliseconds | | | milliseconds to seconds | milliseconds |
+---------------+-------------------------+---------------------+ +===============+-------------------------+---------------------+
| Network Size | Large, many recovery | Small, limited set | | Network Size | Large, many recovery | Small, limited set |
| | graphs to optimize | of protection paths | | | graphs to optimize | of protection paths |
| | globally | | | | globally | |
+---------------+-------------------------+---------------------+ +===============+-------------------------+---------------------+
| Considered | Averaged, statistical, | Instantaneous | | Considered | Averaged, statistical, | Instantaneous |
| Metrics | shade of grey | values / boolean | | Metrics | shade of grey | values / boolean |
| | | condition | | | | condition |
+---------------+-------------------------+---------------------+ +===============+-------------------------+---------------------+
Table 1: Centralized Decision vs. PLR Table 1: Centralized Decision Versus PLR
The PLR sits in the DetNet Forwarding sub-layer of Edge and Relay The PLR sits in the DetNet Forwarding sub-layer of Edge and Relay
Nodes. The PLR operates on the packet flow, learning the recovery Nodes. The PLR operates on the packet flow, learning the recovery
graph and path-selection information from the packet, possibly making graph and path-selection information from the packet and possibly
a local decision and retagging the packet to indicate so. On the making a local decision and retagging the packet to indicate so. On
other hand, the PLR interacts with the lower layers (through triggers the other hand, the PLR interacts with the lower layers (through
and DLEP) and with its peers (through OAM) to obtain up-to-date triggers and DLEP) and with its peers (through OAM) to obtain up-to-
information about its links and the quality of the overall recovery date information about its links and the quality of the overall
graph, respectively, as illustrated in Figure 11. recovery graph, respectively, as illustrated in Figure 11.
| |
packet | going Packet | going
down the | stack down the | stack
+==========v==========+=====================+===================+ +==========v==========+=====================+===================+
|(In-situ OAM + iCTRL)| (L2 Triggers, DLEP) | (Hybrid OAM) | |(In-situ OAM + iCTRL)| (L2 triggers, DLEP) | (Hybrid OAM) |
+==========v==========+=====================+===================+ +==========v==========+=====================+===================+
| Learn from | | Learn from | | Learn from | | Learn from |
| packet tagging > Maintain < end-to-end | | packet tagging > Maintain < end-to-end |
+----------v----------+ Forwarding | OAM packets | +----------v----------+ Forwarding | OAM packets |
| Forwarding decision < State +---------^---------| | Forwarding decision < State +---------^---------|
+----------v----------+ | Enrich or | +----------v----------+ | Enrich or |
+ Retag Packet | Learn abstracted > Regenerate | + Retag packet | Learn abstracted > regenerate |
| and Forward | metrics about Links | OAM packets | | and forward | metrics about links | OAM packets |
+..........v..........+..........^..........+........^.v........+ +..........v..........+..........^..........+........^.v........+
| Lower layers | | Lower layers |
+..........v.....................^...................^.v........+ +..........v.....................^...................^.v........+
frame | sent Frame | L2 Ack Active | | OAM Frame | sent Frame | L2 ack Active | | OAM
over | wireless In | In and | | out over | wireless in | in and | | out
v | | v v | | v
Figure 11: PLR Conceptual Interfaces Figure 11: PLR Conceptual Interfaces
6.6. Act: DetNet Path Selection and Reliability Functions 6.6. Act: DetNet Path Selection and Reliability Functions
The main action by the PLR is the swapping of the DetNet Path within The main action by the PLR is the swapping of the DetNet Path within
the recovery graph for the future packets. The candidate DetNet the recovery graph for the future packets. The candidate DetNet
Paths represent different energy and spectrum profiles, and provide Paths represent different energy and spectrum profiles and provide
protection against different failures. protection against different failures.
The LL API enriches the DetNet protection services (PREOF) with The LL API enriches the DetNet protection services (PREOF) with the
potential possibility to interact with lower-layer one-hop possibility to interact with lower-layer, one-hop reliability
reliability functions that are more typical to wireless than wired, functions that are more typical to wireless than wired, including
including ARQ, FEC, and other techniques such as overhearing and ARQ, FEC, and other techniques such as overhearing and constructive
constructive interferences. Because RAW may be leveraged on wired interferences. Because RAW may be leveraged on wired links (e.g., to
links, e.g., to save power, it is not expected that all lower layers save power), it is not expected that all lower layers support all
support all those capabilities. those capabilities.
RAW provides hints to the lower-layer services on the desired RAW provides hints to the lower-layer services on the desired
outcome, and the lower layer acts on those hints to provide the best outcome, and the lower layer acts on those hints to provide the best
approximation of that outcome, e.g., a level of reliability for one- approximation of that outcome, e.g., a level of reliability for one-
hop transmission within a bounded budget of time and/or energy. hop transmission within a bounded budget of time and/or energy.
Thus, the LL API makes possible cross-layer optimization for Thus, the LL API makes possible cross-layer optimization for
reliability depending on the actual abstraction provided. That is, reliability depending on the actual abstraction provided. That is,
some reliability functions are controlled from Layer-3 using an some reliability functions are controlled from Layer 3 using an
abstract interface, while they are really operated at the lower abstract interface, while they are really operated at the lower
layers. layers.
The RAW Path Selection can be implemented in both centralized and The RAW Path Selection can be implemented in both centralized and
distributed approaches. In the centralized approach, the PLR may distributed approaches. In the centralized approach, the PLR may
obtain a set of pre-computed DetNet paths matching a set of expected obtain a set of pre-computed DetNet paths matching a set of expected
failures, and apply the appropriate DetNet paths for the current failures and apply the appropriate DetNet paths for the current state
state of the wireless links. In the distributed approach, the of the wireless links. In the distributed approach, the signaling in
signaling in the packet may be more abstract than an explicit Path, the packet may be more abstract than an explicit Path, and the PLR
and the PLR decision might be revised along the selected DetNet Path decision might be revised along the selected DetNet Path based on a
based on a better knowledge of the rest of the way. better knowledge of the rest of the way.
The dynamic DetNet Path selection in RAW avoids the waste of critical The dynamic DetNet Path selection in RAW avoids the waste of critical
resources such as spectrum and energy while providing for the assured resources such as spectrum and energy while providing for the assured
SLA, e.g., by rerouting and/or adding redundancy only when a loss SLA, e.g., by rerouting and/or adding redundancy only when a loss
spike is observed. spike is observed.
7. Security Considerations 7. Security Considerations
7.1. Collocated Denial of Service Attacks 7.1. Collocated Denial-of-Service Attacks
RAW leverages diversity (e.g., spatial and time diversity, coding RAW leverages diversity (e.g., spatial and time diversity, coding
diversity, and frequency diversity), possibly using heterogeneous diversity, and frequency diversity), possibly using heterogeneous
wired and wireless networking technologies over different physical wired and wireless networking technologies over different physical
paths, to increase the reliability and availability in the face of paths, to increase reliability and availability in the face of
unpredictable conditions. While this is not done specifically to unpredictable conditions. While this is not done specifically to
defeat an attacker, the amount of diversity used in RAW defeats defeat an attacker, the amount of diversity used in RAW defeats
possible attacks that would impact a particular technology or a possible attacks that would impact a particular technology or a
specific path. specific path.
Physical actions by a collocated attacker such as a radio Physical actions by a collocated attacker such as a radio
interference may still lower the reliability of an end-to-end RAW interference may still lower the reliability of an end-to-end RAW
transmission by blocking one segment or one possible path. But if an transmission by blocking one segment or one possible path. However,
alternate path with diverse frequency, location, and/or technology, if an alternate path with diverse frequency, location, and/or
is available, then RAW adapts by rerouting the impacted traffic over technology is available, then RAW adapts by rerouting the impacted
the preferred alternates, which defeats the attack after a limited traffic over the preferred alternates, which defeats the attack after
period of lower reliability. Then again, the security benefit is a a limited period of lower reliability. Then again, the security
side-effect of an action that is taken regardless of whether the benefit is a side effect of an action that is taken regardless of
source of the issue is voluntary (an attack) or not. whether or not the source of the issue is voluntary (an attack).
7.2. Layer-2 encryption 7.2. Layer 2 Encryption
Radio networks typically encrypt at the MAC layer to protect the Radio networks typically encrypt at the Media Access Control (MAC)
transmission. If the encryption is per-pair of peers, then certain layer to protect the transmission. If the encryption is per pair of
RAW operations like promiscuous overhearing become impractical. peers, then certain RAW operations like promiscuous overhearing
become impractical.
7.3. Forced Access 7.3. Forced Access
A RAW policy may typically select the cheapest collection of links A RAW policy may typically select the cheapest collection of links
that matches the requested SLA, e.g., use free Wi-Fi vs. paid 3GPP that matches the requested SLA, e.g., use free Wi-Fi versus paid 3GPP
access. By defeating the cheap connectivity (e.g., PHY-layer access. By defeating the cheap connectivity (e.g., PHY-layer
interference) the attacker can force an End System to use the paid interference) the attacker can force an End System to use the paid
access and increase the cost of the transmission for the user. access and increase the cost of the transmission for the user.
Similar attacks may also be used to deplete resources in lower-power Similar attacks may also be used to deplete resources in lower-power
nodes by forcing additional transmissions for FEC and ARQ, and attack nodes by forcing additional transmissions for FEC and ARQ, and attack
metrics such as battery life of the nodes. By affecting the metrics such as battery life of the nodes. By affecting the
transmissions and the associated routing metrics in one area, an transmissions and the associated routing metrics in one area, an
attacker may force the traffic and cause congestion along a remote attacker may force the traffic and cause congestion along a remote
path, thus reducing the overall throughput of the network. path, thus reducing the overall throughput of the network.
8. IANA Considerations 8. IANA Considerations
This document has no IANA actions. This document has no IANA actions.
9. Contributors 9. References
The editor wishes to thank the following individuals for their
contributions to the text and ideas exposed in this document:
Lou Berger: LabN Consulting, L.L.C, lberger@labn.net
Xavi Vilajosana: Wireless Networks Research Lab, Universitat Oberta
de Catalunya, xvilajosana@gmail.com
Geogios Papadopolous: IMT Atlantique , georgios.papadopoulos@imt-
atlantique.fr
Remous-Aris Koutsiamanis: IMT Atlantique, remous-
aris.koutsiamanis@imt-atlantique.fr
Rex Buddenberg: retired, buddenbergr@gmail.com
Greg Mirsky: Ericsson, gregimirsky@gmail.com
10. Acknowledgments
This architecture could never have been completed without the support
and recommendations from the DetNet Chairs Janos Farkas and Lou
Berger, and Dave Black, the DetNet Tech Advisor. Many thanks to all
of you.
The authors wish to thank Ketan Talaulikar, as well as Balazs Varga,
Dave Cavalcanti, Don Fedyk, Nicolas Montavont, and Fabrice Theoleyre
for their in-depth reviews during the development of this document.
The authors wish to thank Acee Lindem, Eva Schooler, Rich Salz,
Wesley Eddy, Behcet Sarikaya, Brian Haberman, Gorry Fairhurst, Eric
Vyncke, Erik Kline, Roman Danyliw, and Dave Thaler, for their reviews
and comments during the IETF Last Call / IESG review cycle.
Special thanks for Mohamed Boucadair, Giuseppe Fioccola, and Benoit
Claise, for their help dealing with OAM technologies.
11. References
11.1. Normative References 9.1. Normative References
[RAW-TECHNOS] [RAW-TECHNOS]
Thubert, P., Cavalcanti, D., Vilajosana, X., Schmitt, C., Thubert, P., Ed., Cavalcanti, D., Vilajosana, X., Schmitt,
and J. Farkas, "Reliable and Available Wireless (RAW) C., and J. Farkas, "Reliable and Available Wireless (RAW)
Technologies", Work in Progress, Internet-Draft, draft- Technologies", RFC 9913, DOI 10.17487/RFC9913, February
ietf-raw-technologies-17, 15 April 2025, 2026, <https://www.rfc-editor.org/info/rfc9913>.
<https://datatracker.ietf.org/doc/html/draft-ietf-raw-
technologies-17>.
[TSN] IEEE, "Time-Sensitive Networking (TSN)", [TSN] IEEE, "Time-Sensitive Networking (TSN)",
<https://1.ieee802.org/tsn/>. <https://1.ieee802.org/tsn/>.
[6TiSCH-ARCHI]
Thubert, P., Ed., "An Architecture for IPv6 over the Time-
Slotted Channel Hopping Mode of IEEE 802.15.4 (6TiSCH)",
RFC 9030, DOI 10.17487/RFC9030, May 2021,
<https://www.rfc-editor.org/info/rfc9030>.
[RFC4427] Mannie, E., Ed. and D. Papadimitriou, Ed., "Recovery [RFC4427] Mannie, E., Ed. and D. Papadimitriou, Ed., "Recovery
(Protection and Restoration) Terminology for Generalized (Protection and Restoration) Terminology for Generalized
Multi-Protocol Label Switching (GMPLS)", RFC 4427, Multi-Protocol Label Switching (GMPLS)", RFC 4427,
DOI 10.17487/RFC4427, March 2006, DOI 10.17487/RFC4427, March 2006,
<https://www.rfc-editor.org/info/rfc4427>. <https://www.rfc-editor.org/info/rfc4427>.
[RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu, [RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu,
D., and S. Mansfield, "Guidelines for the Use of the "OAM" D., and S. Mansfield, "Guidelines for the Use of the "OAM"
Acronym in the IETF", BCP 161, RFC 6291, Acronym in the IETF", BCP 161, RFC 6291,
DOI 10.17487/RFC6291, June 2011, DOI 10.17487/RFC6291, June 2011,
skipping to change at page 42, line 22 skipping to change at line 1851
DOI 10.17487/RFC8655, October 2019, DOI 10.17487/RFC8655, October 2019,
<https://www.rfc-editor.org/info/rfc8655>. <https://www.rfc-editor.org/info/rfc8655>.
[DetNet-OAM] [DetNet-OAM]
Mirsky, G., Theoleyre, F., Papadopoulos, G., Bernardos, Mirsky, G., Theoleyre, F., Papadopoulos, G., Bernardos,
CJ., Varga, B., and J. Farkas, "Framework of Operations, CJ., Varga, B., and J. Farkas, "Framework of Operations,
Administration, and Maintenance (OAM) for Deterministic Administration, and Maintenance (OAM) for Deterministic
Networking (DetNet)", RFC 9551, DOI 10.17487/RFC9551, Networking (DetNet)", RFC 9551, DOI 10.17487/RFC9551,
March 2024, <https://www.rfc-editor.org/info/rfc9551>. March 2024, <https://www.rfc-editor.org/info/rfc9551>.
11.2. Informative References 9.2. Informative References
[6TiSCH-ARCHI]
Thubert, P., Ed., "An Architecture for IPv6 over the Time-
Slotted Channel Hopping Mode of IEEE 802.15.4 (6TiSCH)",
RFC 9030, DOI 10.17487/RFC9030, May 2021,
<https://www.rfc-editor.org/info/rfc9030>.
[RFC9049] Dawkins, S., Ed., "Path Aware Networking: Obstacles to [RFC9049] Dawkins, S., Ed., "Path Aware Networking: Obstacles to
Deployment (A Bestiary of Roads Not Taken)", RFC 9049, Deployment (A Bestiary of Roads Not Taken)", RFC 9049,
DOI 10.17487/RFC9049, June 2021, DOI 10.17487/RFC9049, June 2021,
<https://www.rfc-editor.org/info/rfc9049>. <https://www.rfc-editor.org/info/rfc9049>.
[INT-ARCHI] [INT-ARCHI]
Braden, R., Ed., "Requirements for Internet Hosts - Braden, R., Ed., "Requirements for Internet Hosts -
Communication Layers", STD 3, RFC 1122, Communication Layers", STD 3, RFC 1122,
DOI 10.17487/RFC1122, October 1989, DOI 10.17487/RFC1122, October 1989,
skipping to change at page 42, line 45 skipping to change at line 1880
[RFC8939] Varga, B., Ed., Farkas, J., Berger, L., Fedyk, D., and S. [RFC8939] Varga, B., Ed., Farkas, J., Berger, L., Fedyk, D., and S.
Bryant, "Deterministic Networking (DetNet) Data Plane: Bryant, "Deterministic Networking (DetNet) Data Plane:
IP", RFC 8939, DOI 10.17487/RFC8939, November 2020, IP", RFC 8939, DOI 10.17487/RFC8939, November 2020,
<https://www.rfc-editor.org/info/rfc8939>. <https://www.rfc-editor.org/info/rfc8939>.
[RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases", [RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases",
RFC 8578, DOI 10.17487/RFC8578, May 2019, RFC 8578, DOI 10.17487/RFC8578, May 2019,
<https://www.rfc-editor.org/info/rfc8578>. <https://www.rfc-editor.org/info/rfc8578>.
[RAW-USE-CASES] [RAW-USE-CASES]
Bernardos, C. J., Papadopoulos, G. Z., Thubert, P., and F. Bernardos, CJ., Ed., Papadopoulos, G., Thubert, P., and F.
Theoleyre, "RAW Use-Cases", Work in Progress, Internet- Theoleyre, "Reliable and Available Wireless (RAW) Use
Draft, draft-ietf-raw-use-cases-11, 17 April 2023, Cases", RFC 9450, DOI 10.17487/RFC9450, August 2023,
<https://datatracker.ietf.org/doc/html/draft-ietf-raw-use- <https://www.rfc-editor.org/info/rfc9450>.
cases-11>.
[RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791,
DOI 10.17487/RFC0791, September 1981, DOI 10.17487/RFC0791, September 1981,
<https://www.rfc-editor.org/info/rfc791>. <https://www.rfc-editor.org/info/rfc791>.
[RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.
Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
Functional Specification", RFC 2205, DOI 10.17487/RFC2205, Functional Specification", RFC 2205, DOI 10.17487/RFC2205,
September 1997, <https://www.rfc-editor.org/info/rfc2205>. September 1997, <https://www.rfc-editor.org/info/rfc2205>.
skipping to change at page 45, line 5 skipping to change at line 1980
[RFC9473] Enghardt, R. and C. Krähenbühl, "A Vocabulary of Path [RFC9473] Enghardt, R. and C. Krähenbühl, "A Vocabulary of Path
Properties", RFC 9473, DOI 10.17487/RFC9473, September Properties", RFC 9473, DOI 10.17487/RFC9473, September
2023, <https://www.rfc-editor.org/info/rfc9473>. 2023, <https://www.rfc-editor.org/info/rfc9473>.
[RFC9633] Geng, X., Ryoo, Y., Fedyk, D., Rahman, R., and Z. Li, [RFC9633] Geng, X., Ryoo, Y., Fedyk, D., Rahman, R., and Z. Li,
"Deterministic Networking (DetNet) YANG Data Model", "Deterministic Networking (DetNet) YANG Data Model",
RFC 9633, DOI 10.17487/RFC9633, October 2024, RFC 9633, DOI 10.17487/RFC9633, October 2024,
<https://www.rfc-editor.org/info/rfc9633>. <https://www.rfc-editor.org/info/rfc9633>.
[I-D.ietf-detnet-controller-plane-framework] [DetNet-PLANE]
Malis, A. G., Geng, X., Chen, M., Varga, B., and C. J. Malis, A. G., Geng, X., Ed., Chen, M., Varga, B., and C.
Bernardos, "Deterministic Networking (DetNet) Controller J. Bernardos, "A Framework for Deterministic Networking
Plane Framework", Work in Progress, Internet-Draft, draft- (DetNet) Controller Plane", Work in Progress, Internet-
ietf-detnet-controller-plane-framework-12, 27 June 2025, Draft, draft-ietf-detnet-controller-plane-framework-14, 9
<https://datatracker.ietf.org/doc/html/draft-ietf-detnet- September 2025, <https://datatracker.ietf.org/doc/html/
controller-plane-framework-12>. draft-ietf-detnet-controller-plane-framework-14>.
[NASA1] Adams, T., "RELIABILITY: Definition & Quantitative [NASA1] Adams, T., "RELIABILITY: Definition & Quantitative
Illustration", <https://extapps.ksc.nasa.gov/Reliability/ Illustration", <https://extapps.ksc.nasa.gov/Reliability/
Documents/150814-3bWhatIsReliability.pdf>. Documents/150814-3bWhatIsReliability.pdf>.
[NASA2] Adams, T., "Availability", [NASA2] Adams, T., "Availability",
<https://extapps.ksc.nasa.gov/Reliability/ <https://extapps.ksc.nasa.gov/Reliability/
Documents/160727.1_Availability_What_is_it.pdf>. Documents/160727.1_Availability_What_is_it.pdf>.
Acknowledgments
This architecture could never have been completed without the support
and recommendations from the DetNet chairs Janos Farkas and Lou
Berger, and from Dave Black, the DetNet Tech Advisor. Many thanks to
all of you.
The authors wish to thank Ketan Talaulikar, as well as Balazs Varga,
Dave Cavalcanti, Don Fedyk, Nicolas Montavont, and Fabrice Theoleyre
for their in-depth reviews during the development of this document.
The authors wish to thank Acee Lindem, Eva Schooler, Rich Salz,
Wesley Eddy, Behcet Sarikaya, Brian Haberman, Gorry Fairhurst, Éric
Vyncke, Erik Kline, Roman Danyliw, and Dave Thaler for their reviews
and comments during the IETF Last Call and IESG review cycle.
Special thanks for Mohamed Boucadair, Giuseppe Fioccola, and Benoit
Claise for their help dealing with OAM technologies.
Contributors
The editor wishes to thank the following individuals for their
contributions to the text and the ideas discussed in this document:
Lou Berger
LabN Consulting, L.L.C
Email: lberger@labn.net
Xavi Vilajosana
Wireless Networks Research Lab, Universitat Oberta de Catalunya
Email: xvilajosana@gmail.com
Geogios Papadopolous
IMT Atlantique
Email: georgios.papadopoulos@imt-atlantique.fr
Remous-Aris Koutsiamanis
IMT Atlantique
Email: remous-aris.koutsiamanis@imt-atlantique.fr
Rex Buddenberg
Retired
Email: buddenbergr@gmail.com
Greg Mirsky
Ericsson
Email: gregimirsky@gmail.com
Author's Address Author's Address
Pascal Thubert (editor) Pascal Thubert (editor)
Without Affiliation
06330 Roquefort-les-Pins 06330 Roquefort-les-Pins
France France
Email: pascal.thubert@gmail.com Email: pascal.thubert@gmail.com
 End of changes. 248 change blocks. 
815 lines changed or deleted 851 lines changed or added

This html diff was produced by rfcdiff 1.48.