| rfc9993.original.md | rfc9993.md | |||
|---|---|---|---|---|
| --- | --- | |||
| title: "RTP Payload Format for Haptics" | title: RTP Payload Format for Haptics | |||
| abbrev: RTP-Payload-Haptic | abbrev: RTP Payload Format for Haptics | |||
| docname: draft-ietf-avtcore-rtp-haptics-latest | docname: draft-ietf-avtcore-rtp-haptics-latest | |||
| date: {DATE} | number: 9993 | |||
| stream: IETF | date: 2026-05 | |||
| submissiontype: IETF | ||||
| category: std | category: std | |||
| consensus: true | ||||
| ipr: trust200902 | ipr: trust200902 | |||
| area: Transport | area: WIT | |||
| workgroup: avtcore | workgroup: avtcore | |||
| updates: 9695 | updates: 9695 | |||
| stand_alone: yes | obsoletes: | |||
| pi: [toc, sortrefs, symrefs, docmapping] | v: 3 | |||
| lang: en | ||||
| pi: [toc, sortrefs, symrefs] | ||||
| keyword: | ||||
| author: | author: | |||
| - | - | |||
| ins: HS. Yang | ins: HS. Yang | |||
| name: Hyunsik Yang | name: Hyunsik Yang | |||
| org: InterDigital | org: InterDigital | |||
| email: hyunsik.yang@interdigital.com | email: hyunsik.yang@interdigital.com | |||
| country: USA | country: United States of America | |||
| - | - | |||
| ins: X. de Foy | ins: X. de Foy | |||
| name: Xavier de Foy | name: Xavier de Foy | |||
| org: InterDigital | org: InterDigital | |||
| email: xavier.defoy@interdigital.com | email: xavier.defoy@interdigital.com | |||
| country: Canada | country: Canada | |||
| normative: | normative: | |||
| ISO.IEC.23090-31: | ISO.IEC.23090-31: | |||
| title: "Information technology - Coded representation of immersive media" | title: "Information technology - Coded representation of immersive media - Part 3 1: Haptics coding" | |||
| author: | author: | |||
| org: "ISO/IEC" | org: "ISO/IEC" | |||
| date: 2025 | date: 2025 | |||
| seriesinfo: | seriesinfo: | |||
| ISO/IEC: 23090-31:2025 | ISO/IEC: 23090-31:2025 | |||
| target: https://www.iso.org/standard/86122.html | target: https://www.iso.org/standard/86122.html | |||
| RFC2119: | RFC2119: | |||
| RFC8174: | RFC8174: | |||
| RFC3550: | RFC3550: | |||
| skipping to change at line 62 ¶ | skipping to change at line 67 ¶ | |||
| RFC4303: | RFC4303: | |||
| RFC9325: | RFC9325: | |||
| RFC5124: | RFC5124: | |||
| RFC3711: | RFC3711: | |||
| RFC3551: | RFC3551: | |||
| RFC4585: | RFC4585: | |||
| RFC5104: | RFC5104: | |||
| --- abstract | --- abstract | |||
| This memo specifies an RTP payload format for the MPEG-I haptic data. A haptic media | <!--[rfced] FYI: We updated the short title that spans the header of | |||
| stream is composed of MIHS units including a MIHS (MPEG-I Haptic Stream) unit header | the PDF file as shown below (full title fits the space). | |||
| and zero or more MIHS packets. The RTP payload header format allows for packetization | ||||
| of a MIHS unit in an RTP packet payload as well as fragmentation of a MIHS unit into | Original: | |||
| multiple RTP packets. The original subtype registration for haptics/hmpg, registered | RTP-Payload-Haptic | |||
| with IANA in RFC9695, did not include any required or optional parameters. This memo | ||||
| updates RFC9695 and the haptics/hmpg registration to add optional parameters. It als | Current: | |||
| o provides SDP usage information for the haptics media type. | RTP Payload Format for Haptics | |||
| --> | ||||
| <!-- [rfced] Please insert any keywords (beyond those that appear in | ||||
| the title) for use on https://www.rfc-editor.org/search. | ||||
| --> | ||||
| This memo specifies an RTP payload format for MPEG-I haptic data. A haptic media stre | ||||
| am is composed of MPEG-I Haptic Stream (MIHS) units including a MIHS unit header and | ||||
| zero or more MIHS packets. The RTP payload header format allows for packetization of | ||||
| a MIHS unit in an RTP packet payload as well as fragmentation of a MIHS unit into mul | ||||
| tiple RTP packets. The original subtype registration for 'haptics/hmpg' (RFC 9695) di | ||||
| d not include any required or optional parameters. This memo updates RFC 9695 and the | ||||
| 'haptics/hmpg' registration to add optional parameters. It also provides Session Des | ||||
| cription Protocol (SDP) usage information for the 'haptics' media type. | ||||
| --- middle | --- middle | |||
| # Introduction | # Introduction | |||
| Haptics provides users with tactile effects in addition to audio and video, allowing them to experience sensory immersion. Haptic data is mainly transmitted to devices th at act as actuators and provides them with information to operate according to the va lues defined in haptic effects. The IETF registered haptics as a primary media type a kin to audio and video [RFC9695]. | Haptics provides users with tactile effects in addition to audio and video, allowing them to experience sensory immersion. Haptic data is mainly transmitted to devices th at act as actuators, providing them with information to operate according to the valu es defined in haptic effects. The IETF registered 'haptics' as a primary media type, akin to 'audio' and 'video' {{RFC9695}}. | |||
| The MPEG Haptics Coding standard [ISO.IEC.23090-31] defines the data formats, metadat a, and codec architecture to encode, decode, synthesize and transmit haptic signals. Within this MPEG standard, a haptic media stream is composed of MIHS units including a MIHS unit header and zero or more MIHS packets. The MIHS unit is a unit of packetiz ation suitable for streaming, and similar in essence to the NAL (Network Abstraction Layer) unit defined in some video specifications. This document specifies how haptic data (MIHS units) can be transmitted using the RTP protocol. This document follows re commendations in [RFC8088] and [RFC2736] for RTP payload format writers. This documen t does not specify synchronization (lip sync) mechanisms between haptics and audio/vi deo components. In addition, this document specifies the associated SDP parameters a nd SDP Offer/Answer considerations for the haptics media type. | The MPEG Haptics Coding standard {{ISO.IEC.23090-31}} defines the data formats, metad ata, and codec architecture to encode, decode, synthesize, and transmit haptic signal s. Within this MPEG standard, a haptic media stream is composed of MIHS units includi ng a MIHS unit header and zero or more MIHS packets. The MIHS unit is a unit of packe tization suitable for streaming and is similar in essence to the Network Abstraction Layer (NAL) unit defined in some video specifications. This document specifies how ha ptic data (MIHS units) can be transmitted using the RTP protocol. This document follo ws recommendations in {{RFC8088}} and {{RFC2736}} for RTP payload format writers. Thi s document does not specify synchronization (lip sync) mechanisms between haptics and audio/video components. In addition, this document specifies the associated SDP par ameters and SDP offer/answer considerations for the 'haptics' media type. | |||
| # Conventions | # Conventions | |||
| The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are t o be interpreted as described in BCP 14 {{RFC2119}} {{RFC8174}} when, and only when, they appear in all capitals, as shown here. | {::boilerplate bcp14-tagged} | |||
| # Definition | # Definitions | |||
| This document uses the definitions of the MPEG Haptics Coding standard {{ISO.IEC.2309 0-31}}. Some of these terms are provided here for convenience. | This document uses the definitions of the MPEG Haptics Coding standard {{ISO.IEC.2309 0-31}}. Some of these terms are provided here for convenience. | |||
| Actuator: component of a device for rendering haptic sensations. | Actuator: | |||
| : Component of a device for rendering haptic sensations. | ||||
| Avatar: body (or part of body) representation. | Avatar: | |||
| : Body (or part of body) representation. | ||||
| Band: component in a channel for containing effects for a specific range of frequenci | Band: | |||
| es. | : Component in a channel for containing effects for a specific range of frequencies. | |||
| Channel: component in a perception containing one or more bands rendered on a device | Channel: | |||
| at a specific body location. | : Component in a perception containing one or more bands rendered on a device at a sp | |||
| ecific body location. | ||||
| Device: physical system having one or more actuators configured to render a haptic se | Device: | |||
| nsation corresponding with a given signal. | : Physical system having one or more actuators configured to render a haptic sensatio | |||
| n corresponding with a given signal. | ||||
| Effect: component of a band for defining a signal, consisting of a haptic waveform or | Effect: | |||
| one or more haptic keyframes. | : Component of a band for defining a signal, consisting of a haptic waveform or one o | |||
| r more haptic keyframes. | ||||
| Experience: top level haptic component containing perceptions and metadata. | Experience: | |||
| : Top-level haptic component containing perceptions and metadata. | ||||
| Haptics: tactile sensations. | Haptics: | |||
| : Tactile sensations. | ||||
| Keyframe: component of an effect mapping a position in time or space to an effect par | Keyframe: | |||
| ameter such as amplitude or frequency. | : Component of an effect mapping a position in time or space to an effect parameter s | |||
| uch as amplitude or frequency. | ||||
| Metadata: global information about an experience, perception, channel, or band. | Metadata: | |||
| : Global information about an experience, perception, channel, or band. | ||||
| MIHS unit: unit of packetization of the MPEG-I Haptic Stream format, which is used as | MIHS unit: | |||
| unit of payload in the format described in this memo. See {{haptic-format-descriptio | : Unit of packetization of the MPEG-I Haptic Stream format, which is used as unit of | |||
| n}} for details. | payload in the format described in this memo. See {{haptic-format-description}} for d | |||
| etails. | ||||
| Modality: type of haptics, such as vibration, force, pressure, position, velocity, or | Modality: | |||
| temperature. | : Type of haptics, such as vibration, force, pressure, position, velocity, or tempera | |||
| ture. | ||||
| Perception: haptic perception containing channels of a specific modality. | Perception: | |||
| : Haptic perception containing channels of a specific modality. | ||||
| Signal: representation of the haptics associated with a specific modality to be rende | Signal: | |||
| red on a device. | : Representation of the haptics associated with a specific modality to be rendered on | |||
| a device. | ||||
| Hmpg format: hmpg is a binary compressed format for haptics data. Information is stor | Hmpg format: | |||
| ed in a binary form and data compression is applied on data at the band level. The ha | : A binary compressed format for haptics data. Information is stored in a binary form | |||
| ptics/hmpg media subtype is registered in {{RFC9695}} and updated by this memo. | , and data compression is applied on data at the band level. The 'haptics/hmpg' media | |||
| subtype is registered in {{RFC9695}} and updated by this memo. | ||||
| Independent unit: a MIHS unit is independent if it can be decoded independently from | Independent unit: | |||
| earlier units. Independent units contain timing information and are also called "sync | : A MIHS unit is independent if it can be decoded independently from earlier units. I | |||
| units" in {{ISO.IEC.23090-31}}. | ndependent units contain timing information and are also called "sync units" in {{ISO | |||
| .IEC.23090-31}}. | ||||
| Dependent unit: a MIHS unit is dependent if it requires earlier units for decoding. D | Dependent unit: | |||
| ependent units do not contain timing information and are also called "non-sync units" | : A MIHS unit is dependent if it requires earlier units for decoding. Dependent units | |||
| in {{ISO.IEC.23090-31}}. | do not contain timing information and are also called "non-sync units" in {{ISO.IEC. | |||
| 23090-31}}. | ||||
| Time-independent effect: a haptic effect that occurs regardless of time. The tactile | Time-independent effect: | |||
| feedback of a texture is a representative example. Time-independent effects are encod | : A haptic effect that occurs regardless of time. The tactile feedback of a texture i | |||
| ed in spatial MIHS units, defined in {{MIHS-format}}. | s a representative example. Time-independent effects are encoded in spatial MIHS unit | |||
| s, as defined in {{MIHS-format}}. | ||||
| Time-dependent effect: a haptic effect that varies over time. For example, tactile fe | Time-dependent effect: | |||
| edback for vibration and force are time-dependent effects, and are encoded in tempor | : A haptic effect that varies over time. For example, tactile feedback for vibration | |||
| al MIHS units, defined in {{MIHS-format}}. | and force are time-dependent effects and are encoded in temporal MIHS units, as defin | |||
| ed in {{MIHS-format}}. | ||||
| # Haptic Format Description {#haptic-format-description} | # Haptic Format Description {#haptic-format-description} | |||
| ## Overview of Haptic Coding | ## Overview of Haptic Coding | |||
| The MPEG Haptics Coding standard specifies methods for efficient transmission and ren dering of haptic signals, to enable immersive experiences. It supports multiple types of perceptions, including the most common vibrotactile (sense of touch that perceive s vibrations) and kinesthetic perceptions (tactile resistance or force), but also oth er, less common perceptions, including for example the sense of temperature or textur e. It also supports two approaches for encoding haptic signals: a "quantized" approac h based on samples of measured data, and a "descriptive" approach where the signal is synthesized using a combination of functions. Both quantized and descriptive data ca n be encoded in a text-based exchange format based on JSON (.hjif), or in a binary pa cketized format for distribution and streaming (.hmpg). This last format is referred to as the MIHS format and is a base for the RTP payload format described in this docu ment. | The MPEG Haptics Coding standard specifies methods for efficient transmission and the rendering of haptic signals, to enable immersive experiences. It supports multiple t ypes of perceptions, including the most common vibrotactile (sense of touch that perc eives vibrations) and kinesthetic perceptions (tactile resistance or force), and also other less common perceptions, such as the sense of temperature or texture, for exam ple. It also supports two approaches for encoding haptic signals: a "quantized" appro ach based on samples of measured data and a "descriptive" approach where the signal i s synthesized using a combination of functions. Both quantized and descriptive data c an be encoded in a text-based exchange format based on JSON (.hjif) or in a binary pa cketized format for distribution and streaming (.hmpg). This last format is referred to as the MIHS format and is a base for the RTP payload format described in this docu ment. | |||
| ## MIHS format {#MIHS-format} | ## MIHS Format {#MIHS-format} | |||
| MIHS is a stream format used to transport haptic data. Haptic data including haptic e ffects is packetized according to the MIHS format, and delivered to actuators, which operate according to the provided effects. The MIHS format has two levels of packetiz ation, MIHS units and MIHS packets. | MIHS is a stream format used to transport haptic data. Haptic data, including haptic effects, is packetized according to the MIHS format and delivered to actuators, which operate according to the provided effects. The MIHS format has two levels of packeti zation: MIHS units and MIHS packets. | |||
| MIHS units are composed of a MIHS unit header and zero or more MIHS packets. Four typ | MIHS units are composed of a MIHS unit header and zero or more MIHS packets. Four typ | |||
| es of MIHS units are defined. An initialization MIHS unit contains MIHS packets carry | es of MIHS units are defined. An initialization MIHS unit contains MIHS packets carry | |||
| ing metadata necessary to reset and initialize a haptic decoder, including a timestam | ing metadata necessary to reset and initialize a haptic decoder, including a timestam | |||
| p. A temporal MIHS unit contains one or more MIHS packets defining time-dependent eff | p. A temporal MIHS unit contains one or more MIHS packets defining time-dependent eff | |||
| ects and providing modalities such as pressure, velocity, and acceleration. The durat | ects and provides modalities such as pressure, velocity, and acceleration. The durati | |||
| ion of a temporal unit is a positive number. A spatial MIHS unit contains one or more | on of a temporal unit is a positive number. A spatial MIHS unit contains one or more | |||
| MIHS packets providing time-independent effects, such as vibrotactile texture, stiff | MIHS packets providing time-independent effects, such as vibrotactile texture, stiffn | |||
| ness, and friction. The duration of a spatial unit is always zero. | ess, and friction. The duration of a spatial unit is always zero. | |||
| A silent MIHS unit indicates that there is no effect during a time interval and its d | A silent MIHS unit indicates that there is no effect during a time interval, and its | |||
| uration is a positive number. | duration is a positive number. | |||
| A MIHS unit can be marked as independent or dependent. When a decoder processes an in | A MIHS unit can be marked as independent or dependent. When a decoder processes an in | |||
| dependent unit, it resets the previous effects and therefore provides a haptic experi | dependent unit, it resets the previous effects and therefore provides a haptic experi | |||
| ence independent from any previous MIHS unit. A dependent unit is the continuation o | ence independent from any previous MIHS unit. A dependent unit is the continuation o | |||
| f previous MIHS units and cannot be independently decoded and rendered without having | f previous MIHS units and cannot be independently decoded and rendered without having | |||
| decoded previous MIHS unit(s). Initialization and spatial MIHS units are always inde | decoded a previous MIHS unit(s). Initialization and spatial MIHS units are always in | |||
| pendent units. Temporal and silent MIHS units can be dependent or independent units. | dependent units. Temporal and silent MIHS units can be dependent or independent units | |||
| . | ||||
| <!--[rfced] FYI: In Figure 1, we adjusted the spacing of the middle | ||||
| box to include "Temporal Unit" on one line. Please let us know of | ||||
| any objections. | ||||
| --> | ||||
| {{figure-stream}} illustrates a succession of MIHS units in a MIHS stream. | {{figure-stream}} illustrates a succession of MIHS units in a MIHS stream. | |||
| ~~~~~~~~~~ aasvg | ~~~~~~~~~~ aasvg | |||
| +--------+ +-------+ +------------+ +-------------+ +-----------+ | +--------+ +-------+ +-------------+ +-------------+ +-----------+ | |||
| |Initial*| |Spatial| | Temporal | |Temporal Unit| |Silent Unit| | |Initial*| |Spatial| |Temporal Unit| |Temporal Unit| |Silent Unit| | |||
| | Unit |-| Unit |-|Unit(indep.)|-| (dependent) |-| (indep.) | | | Unit |-| Unit |-| (indep.) |-| (dependent) |-| (indep.) | | |||
| +--------+ +-------+ +------------+ +-------------+ +-----------+ | +--------+ +-------+ +-------------+ +-------------+ +-----------+ | |||
| *Initialization | *Initialization | |||
| --------+ +-------+ <span class="insert">+-------------+</span> +-------------+ +---- -------+ | ||||
| ~~~~~~~~~~ | ~~~~~~~~~~ | |||
| {: #figure-stream title="Example of MIHS stream"} | {: #figure-stream title="Example of MIHS Stream"} | |||
| # Payload Format For Haptics | # Payload Format for Haptics | |||
| ## RTP Header Usage {#rtp-usage} | ## RTP Header Usage {#rtp-usage} | |||
| The RTP header is defined in {{RFC3550}} and represented in {{figure-rtpheader}}. Unl | The RTP header is defined in {{RFC3550}} and represented in {{figure-rtpheader}}. Unl | |||
| ess contextualized below, the meaning of the fields depicted in {{figure-rtpheader}} | ess contextualized below, the meaning of the fields depicted in {{figure-rtpheader}} | |||
| is the same as in {{rtp-usage}} of {{RFC3550}}. | is the same as in {{Section 5.1 of RFC3550}}. | |||
| <!--[rfced] In Figure 2, should "(TS)" be added after "Timestamp" to | ||||
| correspond with the entry for timestamp and for consistency with the | ||||
| two terms that appear below it in the figure? | ||||
| Original: | ||||
| Timestamp | ||||
| Perhaps: | ||||
| Timestamp (TS) | ||||
| --> | ||||
| ~~~~~~~~~~ aasvg | ~~~~~~~~~~ aasvg | |||
| 0 1 2 3 | 0 1 2 3 | |||
| 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| |V=2|P|X| CC |M| PT | Sequence Number | | |V=2|P|X| CC |M| PT | Sequence Number | | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | Timestamp | | | Timestamp | | |||
| +---------------------------------------------------------------+ | +---------------------------------------------------------------+ | |||
| | Synchronization Source (SSRC) Identifier | | | Synchronization Source (SSRC) Identifier | | |||
| +---------------------------------------------------------------+ | +---------------------------------------------------------------+ | |||
| | Contributing Source (CSRC) Identifiers | | | Contributing Source (CSRC) Identifiers | | |||
| | .... | | | .... | | |||
| +---------------------------------------------------------------+ | +---------------------------------------------------------------+ | |||
| ~~~~~~~~~~ | ~~~~~~~~~~ | |||
| {: #figure-rtpheader title="RTP header for Haptic."} | {: #figure-rtpheader title="RTP Header for Haptic"} | |||
| Marker bit (M): 1 bit. The marker bit SHOULD be set to one in the first non-silent RT | <!--[rfced] We note different list styles in Sections 5.1, 5.2, and | |||
| P packet after a period of haptic silence. This enables jitter buffer adaptation and | 5.3.2. May we make these lists consistent by using the format shown | |||
| haptics device washout (i.e., reset to a neutral position) prior to the beginning of | below? (See the text for more examples.) | |||
| the burst with minimal impact on the quality of experience for the end user. The mark | ||||
| er bit in all other packets MUST be set to zero. | ||||
| Timestamp (TS): 32 bits. A timestamp representing the sampling time of the first samp | Original: | |||
| le of the MIHS unit in the RTP payload. The clock frequency MUST be set to the sample | (Section 5.1) | |||
| rate of the encoded haptic data and is conveyed out-of-band (e.g., as an SDP paramet | Marker bit (M): 1 bit. The marker bit ... | |||
| er). | Timestamp (TS): 32 bits. A timestamp ... | |||
| (Section 5.2) | ||||
| D (Dependency, 1 bit): This field indicates ... | ||||
| UT (Unit Type, 3 bits): This field indicates ... | ||||
| Perhaps: | ||||
| (Section 5.1) | ||||
| M (Marker bit): 1 bit. The marker bit ... | ||||
| TS (Timestamp): 32 bits. A timestamp ... | ||||
| (Section 5.2) | ||||
| D (Dependency): 1 bit. This field indicates ... | ||||
| UT (Unit Type): 3 bits. This field indicates ... | ||||
| --> | ||||
| Marker bit (M): | ||||
| : 1 bit. The marker bit SHOULD be set to one in the first non-silent RTP packet after | ||||
| a period of haptic silence. This enables jitter buffer adaptation and haptics device | ||||
| washout (i.e., reset to a neutral position) prior to the beginning of the burst with | ||||
| minimal impact on the quality of experience for the end user. The marker bit in all | ||||
| other packets MUST be set to zero. | ||||
| Timestamp (TS): | ||||
| : 32 bits. A timestamp representing the sampling time of the first sample of the MIHS | ||||
| unit in the RTP payload. The clock frequency MUST be set to the sample rate of the e | ||||
| ncoded haptic data and is conveyed out of band (e.g., as an SDP parameter). | ||||
| ## Payload Header {#payload-header} | ## Payload Header {#payload-header} | |||
| <!--[rfced] We note two uppercase forms of "Haptic" in Sections 5.2 and | ||||
| 5.4. Are these correct as is, or should they be made lowercase for | ||||
| consistency? | ||||
| Current: | ||||
| The RTP payload header follows the RTP header. Figure 3 describes | ||||
| the RTP payload header for Haptic. | ||||
| In some multimedia conference scenarios using an RTP video mixer | ||||
| (e.g., when adding or selecting a new source), it is recommended to | ||||
| use Full Intra Request (FIR) feedback messages [RFC5104] with Haptics. | ||||
| --> | ||||
| The RTP payload header follows the RTP header. {{figure-payloadheader}} describes the RTP payload header for Haptic. | The RTP payload header follows the RTP header. {{figure-payloadheader}} describes the RTP payload header for Haptic. | |||
| ~~~~~~~~~~ aasvg | ~~~~~~~~~~ aasvg | |||
| +-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+ | |||
| |0|1|2|3|4|5|6|7| | |0|1|2|3|4|5|6|7| | |||
| +-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+ | |||
| |D| UT | L | | |D| UT | L | | |||
| +-+-----+-------+ | +-+-----+-------+ | |||
| ~~~~~~~~~~ | ~~~~~~~~~~ | |||
| {: #figure-payloadheader title="RTP Payload Header for Haptic."} | {: #figure-payloadheader title="RTP Payload Header for Haptic"} | |||
| D (Dependency, 1 bit): this field is used to indicate whether the MIHS unit included | D (Dependency, 1 bit): | |||
| in the RTP payload is, when its value is one, dependent or, when its value is zero, i | : This field indicates whether the MIHS unit included in the RTP payload is dependent | |||
| ndependent. | (when its value is one) or independent (when its value is zero). | |||
| UT (Unit Type, 3 bits): this field indicates the type of the MIHS unit included in th | UT (Unit Type, 3 bits): | |||
| e RTP payload. UT field values are listed in {{figure-transmission-type}}. | : This field indicates the type of the MIHS unit included in the RTP payload. UT fiel | |||
| d values are listed in {{figure-transmission-type}}. | ||||
| L (MIHS Layer, 4 bits): this field is an integer value which indicates the priority o | L (MIHS Layer, 4 bits): | |||
| rder of the MIHS unit included in the RTP payload, as determined by the haptic sender | : This field is an integer value that indicates the priority order of the MIHS unit i | |||
| (e.g., by the haptic codec), based on application-specific needs. For example, the s | ncluded in the RTP payload, as determined by the haptic sender (e.g., by the haptic c | |||
| ender may use the MIHS layer to prioritize perceptions with the largest impact on the | odec), based on application-specific needs. For example, the sender may use the MIHS | |||
| end-user experience. Zero corresponds to the highest priority. The semantic of indiv | layer to prioritize perceptions with the largest impact on the end-user experience. Z | |||
| idual MIHS layers are not specified and left for the application to assign. In cases | ero corresponds to the highest priority. The semantic of individual MIHS layers are n | |||
| where the sender does not use the L field to indicate the priority order of the MIHS | ot specified and are left for the application to assign. In cases where the sender do | |||
| unit, L value is '0'. | es not use the L field to indicate the priority order of the MIHS unit, the L value i | |||
| s '0'. | ||||
| ## Payload Structures | ## Payload Structures | |||
| Three different types of RTP packet payload structures are specified. A single unit p acket contains a single MIHS unit in the payload. A fragmentation unit contains a su bset of a MIHS unit. An aggregation packet contains multiple MIHS units in the payloa d. The unit type (UT) field of the RTP payload header, as shown in {{figure-transmis sion-type}}, identifies both the payload structure and, in the case of a single-unit structure, also identifies the type of MIHS unit present in the payload. | Three different types of RTP packet payload structures are specified. A single unit p acket contains a single MIHS unit in the payload. A fragmentation unit contains a su bset of a MIHS unit. An aggregation packet contains multiple MIHS units in the payloa d. The unit type (UT) field of the RTP payload header, as shown in {{figure-transmis sion-type}}, identifies both the payload structure and, in the case of a single-unit structure, the type of MIHS unit present in the payload. | |||
| ~~~~~~~~~~ aasvg | | Unit Type | Payload Structure | Packet Type Name | | |||
| Unit Payload Packet Type Name | |:----------------------------------------------------------:| | |||
| Type Structure | | 0 | N/A | Unassigned | | |||
| 0 N/A Unassigned | | 1 | Single | Initialization MIHS Unit | | |||
| 1 Single Initialization MIHS Unit | | 2 | Single | Temporal MIHS Unit | | |||
| 2 Single Temporal MIHS Unit | | 3 | Single | Spatial MIHS Unit | | |||
| 3 Single Spatial MIHS Unit | | 4 | Single | Silent MIHS Unit | | |||
| 4 Single Silent MIHS Unit | | 5 | Aggr | Single-Time Aggregation Packet (STAP) | | |||
| 5 Aggr Single-Time Aggregation Packet (STAP) | | 6 | Aggr | Multi-Time Aggregation Packet (MTAP) | | |||
| 6 Aggr Multi-Time Aggregation Packet (MTAP) | | 7 | Frag | Fragmentation Unit | | |||
| 7 Frag Fragmentation Unit | {: #figure-transmission-type title="Payload Structure Type for Haptic"} | |||
| ~~~~~~~~~~ | ||||
| {: #figure-transmission-type title="Payload structure type for haptic"} | ||||
| The payload structures are represented in {{figure-transmission-style}}. The single unit payload structure is specified in {{single}}. The fragmented unit payload struct ure is specified in {{fragmented}}. The aggregation packet payload structure is speci fied in {{aggregated}}. The padding in the figures of these section refers to the RT P padding defined in {{RFC3550}}. | The payload structures are represented in {{figure-transmission-style}}. The single unit payload structure is specified in {{single}}. The fragmented unit payload struct ure is specified in {{fragmented}}. The aggregation packet payload structure is speci fied in {{aggregated}}. The padding in the figures of these sections refers to the R TP padding defined in {{RFC3550}}. | |||
| ~~~~~~~~~~ aasvg | ~~~~~~~~~~ aasvg | |||
| +-------------------+ | +-------------------+ | |||
| | RTP Header | | | RTP Header | | |||
| +-------------------+ | +-------------------+ | |||
| | RTP Payload Header| | | RTP Payload Header| | |||
| +-------------------+ | (UT = Aggr) | | +-------------------+ | (UT = Aggr) | | |||
| | RTP Header | +-------------------+ | | RTP Header | +-------------------+ | |||
| +-------------------+ +-------------------+ | MIHS unit 1 Size | | +-------------------+ +-------------------+ | MIHS Unit 1 Size | | |||
| | RTP Header | | RTP Payload Header| +-------------------+ | | RTP Header | | RTP Payload Header| +-------------------+ | |||
| +-------------------+ | (UT = Frag) | | MIHS Unit 1 | | +-------------------+ | (UT = Frag) | | MIHS Unit 1 | | |||
| | RTP Payload Header| +-------------------+ +-------------------+ | | RTP Payload Header| +-------------------+ +-------------------+ | |||
| +-------------------+ | FU Header | | MIHS unit 2 Size | | +-------------------+ | FU Header | | MIHS Unit 2 Size | | |||
| | RTP Payload | +-------------------+ +-------------------+ | | RTP Payload | +-------------------+ +-------------------+ | |||
| | (Single MIHS unit)| | RTP Payload | | ... | | | (Single MIHS unit)| | RTP Payload | | ... | | |||
| +-------------------+ +-------------------+ +-------------------+ | +-------------------+ +-------------------+ +-------------------+ | |||
| (a) single unit (b)fragmentation unit (c) aggregation packet | (a) single unit (b)fragmentation unit (c) aggregation packet | |||
| ~~~~~~~~~~ | ~~~~~~~~~~ | |||
| {: #figure-transmission-style title="RTP Transmission modes"} | {: #figure-transmission-style title="RTP Transmission Modes"} | |||
| ### Single Unit Payload Structure {#single} | ### Single Unit Payload Structure {#single} | |||
| In a single unit payload structure, as described in {{figure-transmission-single}}, t he RTP packet contains the RTP header, followed by the payload header and one single MIHS unit. The payload header follows the structure described in {{payload-header}}. The payload contains a MIHS unit as defined in {{ISO.IEC.23090-31}}. | In a single unit payload structure, as described in {{figure-transmission-single}}, t he RTP packet contains the RTP header, followed by the payload header and one single MIHS unit. The payload header follows the structure described in {{payload-header}}. The payload contains a MIHS unit as defined in {{ISO.IEC.23090-31}}. | |||
| ~~~~~~~~~~ aasvg | ~~~~~~~~~~ aasvg | |||
| 0 1 2 3 | 0 1 2 3 | |||
| 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | RTP Header | | | RTP Header | | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| |Payload Header | | | |Payload Header | | | |||
| +---------------+ | | +---------------+ | | |||
| | MIHS Unit Data | | | MIHS Unit Data | | |||
| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | |...OPTIONAL RTP padding | | | |...OPTIONAL RTP Padding | | |||
| +-------------------------------+-------------------------------+ | +-------------------------------+-------------------------------+ | |||
| ~~~~~~~~~~ | ~~~~~~~~~~ | |||
| {: #figure-transmission-single title="Single Unit Payload Structure"} | {: #figure-transmission-single title="Single Unit Payload Structure"} | |||
| ### Fragmented Unit Payload Structure {#fragmented} | ### Fragmented Unit Payload Structure {#fragmented} | |||
| In a fragmented unit payload structure, as described in {{figure-fragment-structure}} , the RTP packet contains the RTP header, followed by the payload header, a Fragmente d Unit (FU) header, and a MIHS unit fragment. The payload header follows the structur e described in {{payload-header}}. The value of the UT field of the payload header is 7. The FU header follows the structure described in {{figure-fragment-header}}. In t he case of fragmentation, all RTP payload header fields MUST remain unchanged across all fragments. | In a fragmented unit payload structure, as described in {{figure-fragment-structure}} , the RTP packet contains the RTP header, followed by the payload header, a Fragmente d Unit (FU) header, and a MIHS unit fragment. The payload header follows the structur e described in {{payload-header}}. The value of the UT field of the payload header is 7. The FU header follows the structure described in {{figure-fragment-header}}. In t he case of fragmentation, all RTP payload header fields MUST remain unchanged across all fragments. | |||
| ~~~~~~~~~~ aasvg | ~~~~~~~~~~ aasvg | |||
| 0 1 2 3 | 0 1 2 3 | |||
| skipping to change at line 271 ¶ | skipping to change at line 363 ¶ | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | |||
| | MIHS Unit Fragment | | | MIHS Unit Fragment | | |||
| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | |...OPTIONAL RTP Padding | | | |...OPTIONAL RTP Padding | | |||
| +-------------------------------+-------------------------------+ | +-------------------------------+-------------------------------+ | |||
| ~~~~~~~~~~ | ~~~~~~~~~~ | |||
| {: #figure-fragment-structure title="Fragmentation Unit Payload Structure"} | {: #figure-fragment-structure title="Fragmentation Unit Payload Structure"} | |||
| FU headers are used to enable fragmenting a single MIHS unit into multiple RTP packet s. Fragments of the same MIHS unit MUST be sent in consecutive order with ascending R TP sequence numbers (with no other RTP packets within the same RTP stream being sent between the first and last fragment). FUs MUST NOT be nested, i.e., an FU MUST NOT co ntain a subset of another FU. | FU headers are used to enable fragmenting a single MIHS unit into multiple RTP packet s. Fragments of the same MIHS unit MUST be sent in consecutive order with ascending R TP sequence numbers (with no other RTP packets within the same RTP stream being sent between the first and last fragment). FUs MUST NOT be nested, i.e., an FU MUST NOT co ntain a subset of another FU. | |||
| {{figure-fragment-header}} describes a FU header, including the following fields: | {{figure-fragment-header}} describes an FU header, including the following fields: | |||
| ~~~~~~~~~~ aasvg | ~~~~~~~~~~ aasvg | |||
| +---+---+---+---+---+---+---+---+ | +---+---+---+---+---+---+---+---+ | |||
| | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | | | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | | |||
| +---+---+---+---+---+---+---+---+ | +---+---+---+---+---+---+---+---+ | |||
| |FUS|FUE| RSV | UT | | |FUS|FUE| RSV | UT | | |||
| +---+---+-----------+-----------+ | +---+---+-----------+-----------+ | |||
| ~~~~~~~~~~ | ~~~~~~~~~~ | |||
| {: #figure-fragment-header title="Fragmentation unit header"} | {: #figure-fragment-header title="Fragmentation Unit Header"} | |||
| FUS (Fragmented Unit Start, 1 bit): this field MUST be set to 1 for the first fragmen | FUS (Fragmented Unit Start, 1 bit): | |||
| t, and 0 for the other fragments. | : This field MUST be set to 1 for the first fragment and 0 for the other fragments. | |||
| FUE (Fragmented Unit End, 1 bit): this field MUST be set to 1 for the last fragment, | FUE (Fragmented Unit End, 1 bit): | |||
| and 0 for the other fragments. | : This field MUST be set to 1 for the last fragment and 0 for the other fragments. | |||
| The combination FUS=1 and FUE=1 MUST NOT occur; such packets are invalid. | The combination FUS=1 and FUE=1 MUST NOT occur; such packets are invalid. | |||
| RSV (Reserved, 3 bits): these bits MUST be set to 0 by the sender and ignored by the | RSV (Reserved, 3 bits): | |||
| receiver. | : These bits MUST be set to 0 by the sender and ignored by the receiver. | |||
| UT (Unit Type, 3 bits): this field indicates the type of the MIHS unit this fragment | UT (Unit Type, 3 bits): | |||
| belongs to, using values defined in {{figure-transmission-type}}. | : This field indicates the type of the MIHS unit this fragment belongs to, using valu | |||
| es defined in {{figure-transmission-type}}. | ||||
| The use of MIHS unit fragmentation in RTP means that a media receiver can receive som e fragments, but not other fragments. The missing fragments will typically not be ret ransmitted by RTP. This results in partially received MIHS units, which can be either dropped or used by the decoding application, based on implementation. In cases where consecutive fragments with FUE and FUS are lost, the receiver may in some cases be a ble to detect that surrounding fragments belong to a different partially received MIH S unit (e.g., if the UT field holds a different value). | The use of MIHS unit fragmentation in RTP means that a media receiver can receive som e fragments, but not other fragments. The missing fragments will typically not be ret ransmitted by RTP. This results in partially received MIHS units, which can be either dropped or used by the decoding application, based on implementation. In cases where consecutive fragments with FUE and FUS are lost, the receiver may be able to detect that surrounding fragments belong to a different partially received MIHS unit (e.g., if the UT field holds a different value). | |||
| ### Aggregation Packet Payload Structure {#aggregated} | ### Aggregation Packet Payload Structure {#aggregated} | |||
| In an aggregation packet, as described in {{figure-aggre-structure}}, the RTP packet contains an RTP header, followed by a payload header, and, for each aggregated MIHS U nit, a MIHS unit size followed by the MIHS unit. The payload header follows the struc ture described in {{payload-header}}. | In an aggregation packet, as described in {{figure-aggre-structure}}, the RTP packet contains an RTP header, followed by a payload header, and (for each aggregated MIHS u nit) a MIHS unit size followed by the MIHS unit. The payload header follows the struc ture described in {{payload-header}}. | |||
| ~~~~~~~~~~ aasvg | ~~~~~~~~~~ aasvg | |||
| 0 1 2 3 | 0 1 2 3 | |||
| 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | RTP Header | | | RTP Header | | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | RTP Payload Header | MIHS Unit 1 Size | | | RTP Payload Header | MIHS Unit 1 Size | | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | MIHS Unit 1 | | | MIHS Unit 1 | | |||
| | | | | | | |||
| : : | : : | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | MIHS Unit 2 Size | | | | MIHS Unit 2 Size | | | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | |||
| | MIHS Unit 2 | | | MIHS Unit 2 | | |||
| | | | | | | |||
| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | |...OPTIONAL RTP padding | | | |...OPTIONAL RTP Padding | | |||
| +-------------------------------+-------------------------------+ | +-------------------------------+-------------------------------+ | |||
| ~~~~~~~~~~ | ~~~~~~~~~~ | |||
| {: #figure-aggre-structure title="Single-Time Aggregation Packet"} | {: #figure-aggre-structure title="Single-Time Aggregation Packet"} | |||
| {{figure-aggre-structure}} shows a Single-Time Aggregation Packet (STAP), which can b e used to transmit multiple MIHS units that correspond to the same timestamp. For exa mple, if two frequencies are used for the same content, they can be transmitted at on ce in a STAP. Multiple spatial units can also be sent together in a STAP, since this type of haptics data is time independent. The MIHS unit length field (16 bits) holds the length of the MIHS unit following it, in bytes. The value of the UT field of the payload header is 5. | {{figure-aggre-structure}} shows a Single-Time Aggregation Packet (STAP), which can b e used to transmit multiple MIHS units that correspond to the same timestamp. For exa mple, if two frequencies are used for the same content, they can be transmitted at on ce in a STAP. Multiple spatial units can also be sent together in a STAP, since this type of haptics data is time independent. The MIHS unit length field (16 bits) holds the length of the MIHS unit following it, in bytes. The value of the UT field of the payload header is 5. | |||
| <!--[rfced] FYI: In Figure 9, we moved the bit ruler over one space | ||||
| to the right to get the correct alignment (this is consistent with | ||||
| the other figures that contain bit rulers). Please let us know of | ||||
| any objections. | ||||
| --> | ||||
| ~~~~~~~~~~ aasvg | ~~~~~~~~~~ aasvg | |||
| 0 1 2 3 | 0 1 2 3 | |||
| 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | RTP Header | | | RTP Header | | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | RTP Payload Header | MIHS Unit 1 Size | | | RTP Payload Header | MIHS Unit 1 Size | | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | TS Offset | | | | TS Offset | | | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | |||
| | MIHS Unit 1 | | | MIHS Unit 1 | | |||
| | | | | | | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | MIHS Unit 2 Size | TS Offset | | | MIHS Unit 2 Size | TS Offset | | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | TS offset | | | | TS Offset | | | |||
| |-+-+-+-+-+-+-+-+ | | |-+-+-+-+-+-+-+-+ | | |||
| | MIHS Unit 2 | | | MIHS Unit 2 | | |||
| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | |...OPTIONAL RTP padding | | | |...OPTIONAL RTP Padding | | |||
| +-------------------------------+-------------------------------+ | +-------------------------------+-------------------------------+ | |||
| ~~~~~~~~~~ | ~~~~~~~~~~ | |||
| {: #figure-aggremtap-structure title="Multiple-time aggregation packet"} | {: #figure-aggremtap-structure title="Multiple-Time Aggregation Packet"} | |||
| {{figure-aggremtap-structure}} shows a multi-time aggregation packet. It is used to t ransmit multiple MIHS units with different timestamps, in one RTP packet. Multi-time aggregation can help reduce the number of packets, in environments where some delay i s acceptable. The value of the UT field of the Payload Header is 6. The MIHS unit len gth field (16 bits) holds the length of the MIHS unit following it, in bytes. The tim estamp offset field (TS offset, 16 bits) is present in the MTAP case, and MUST be set to the value of (time of the MIHS unit - RTP timestamp of the packet). The timestam p offset of the earliest aggregation unit MUST always be zero. Therefore, the RTP tim estamp of the MTAP is identical to the earliest MIHS unit time. | {{figure-aggremtap-structure}} shows a Multi-Time Aggregation Packet (MTAP). It is us ed to transmit multiple MIHS units with different timestamps, in one RTP packet. Mult i-time aggregation can help reduce the number of packets in environments where some d elay is acceptable. The value of the UT field of the payload header is 6. The MIHS un it length field (16 bits) holds the length of the MIHS unit following it, in bytes. T he timestamp offset field (TS offset, 16 bits) is present in the MTAP case and MUST b e set to the value of (time of the MIHS unit - RTP timestamp of the packet). The tim estamp offset of the earliest aggregation unit MUST always be zero. Therefore, the RT P timestamp of the MTAP is identical to the earliest MIHS unit time. | |||
| ## MIHS Units Transmission and Reception Considerations {#mihs-trans} | ## MIHS Units Transmission and Reception Considerations {#mihs-trans} | |||
| The following considerations apply for the streaming of MIHS units over RTP: | The following considerations apply for the streaming of MIHS units over RTP. | |||
| The MIHS format enables variable duration units and uses initialization MIHS units to declare the duration of subsequent non-zero duration MIHS units, as well as the maxi mum variation of this duration. A sender SHOULD set constant or low-variability (e.g. , lower than the playout buffer) durations in initialization MIHS units, for RTP stre aming. This enables the receiver to determine early (e.g., using a timer) when a unit has been lost and make the decoder more robust to RTP packet loss. If a sender sends MIHS units with high duration variations, the receiver MAY need to wait for a long p eriod of time (e.g., the upper bound of the duration variation), to determine if a MI HS unit was lost in transmission. Whether this behavior is acceptable or not is appli cation dependent, and the application can configure the encoder to generate MIHS unit of lengths with the appropriate variation. | The MIHS format enables variable duration units and uses initialization MIHS units to declare the duration of subsequent non-zero duration MIHS units, as well as the maxi mum variation of this duration. A sender SHOULD set constant or low-variability (e.g. , lower than the playout buffer) durations in initialization MIHS units, for RTP stre aming. This enables the receiver to determine early (e.g., using a timer) when a unit has been lost and to make the decoder more robust to RTP packet loss. If a sender se nds MIHS units with high duration variations, the receiver MAY need to wait for a lon g period of time (e.g., the upper bound of the duration variation) to determine if a MIHS unit was lost in transmission. Whether this behavior is acceptable or not is app lication dependent, and the application can configure the encoder to generate MIHS un it of lengths with the appropriate variation. | |||
| The MIHS format uses silent MIHS units to signal haptic silence. A sender MAY decide not to send silent units, to save network resources. Since, from a receiver standpoin t, a missed MIHS unit may originate from a not-sent silent unit, or a lost packet, a sender MAY send one, or a few, MIHS silent units at the beginning of a haptic silence . If a media receiver receives a MIHS silent unit, the receiver SHOULD assume that si lence is intended until the reception of a non-silent MIHS unit. This can reduce the number of false detections of lost RTP packets by the decoder. | The MIHS format uses silent MIHS units to signal haptic silence. A sender MAY decide not to send silent units, to save network resources. Since, from a receiver standpoin t, a missed MIHS unit may originate from a not-sent silent unit or a lost packet, a s ender MAY send one, or a few, MIHS silent units at the beginning of a haptic silence. If a media receiver receives a MIHS silent unit, the receiver SHOULD assume that sil ence is intended until the reception of a non-silent MIHS unit. This can reduce the n umber of false detections of lost RTP packets by the decoder. | |||
| In some multimedia conference scenarios using an RTP video mixer (e.g., when adding o r selecting a new source), it is recommended to use Full Intra Request (FIR) feedback messages with Haptics {{RFC5104}}. The purpose of the FIR message is to cause an enc oder to send a decoder refresh point at the earliest opportunity. In the context of h aptics, an appropriate decoder refresh point is an initialization MIHS unit. The init ialization MIHS unit point enables a decoder to be reset to a known state and be able decode all MIHS units following it. | In some multimedia conference scenarios using an RTP video mixer (e.g., when adding o r selecting a new source), it is recommended to use Full Intra Request (FIR) feedback messages {{RFC5104}} with Haptics. The purpose of the FIR message is to cause an enc oder to send a decoder refresh point at the earliest opportunity. In the context of h aptics, an appropriate decoder refresh point is an initialization MIHS unit. The init ialization MIHS unit point enables a decoder to be reset to a known state and to deco de all MIHS units following it. | |||
| # Payload Format Parameters | # Payload Format Parameters | |||
| This section describes payload format parameters. {{optional-param}} specifies new op tional parameters and {{sdp-registration}} further registers a new token in the media sub-registry of the Session Description Protocols (SDP) Parameters registry. | This section describes payload format parameters. {{optional-param}} specifies new op tional parameters, and {{sdp-registration}} further registers a new token in the medi a subregistry of the "Session Description Protocol (SDP) Parameters" registry group. | |||
| ## Optional Parameters Definition {#optional-param} | ## Optional Parameters Definition {#optional-param} | |||
| It is optional to include the SDP parameters in this section. Some parameters have a | It is optional to include the SDP parameters in this section. Some parameters have a | |||
| default value which MUST be inferred if the parameter is not present in the SDP, unle | default value that MUST be inferred if the parameter is not present in the SDP, unles | |||
| ss an out-of-band agreement indicates a different value, as described in {{sdp-cons}} | s an out-of-band agreement indicates a different value, as described in {{sdp-cons}}. | |||
| . The values of the SDP parameters indicated in this section are based on the current | The values of the SDP parameters indicated in this section are based on the current | |||
| version of the MPEG Haptics Coding standard (ISO/IEC 23090-31:2025) and may be diffe | version of the MPEG Haptics Coding standard (ISO/IEC 23090-31:2025) and may be differ | |||
| rent in future versions of {{ISO.IEC.23090-31}}. | ent in future versions of {{ISO.IEC.23090-31}}. | |||
| <!--[rfced] In the list in section 6.1, please clarify "may hold values among" (3 ins | ||||
| tances). Does this | ||||
| mean "may contain the values"? Also, is "or" (before "Custom") | ||||
| correct, or should it be "and/or"? | ||||
| One example | ||||
| Original: | ||||
| The avatar type is defined in [ISO.IEC.23090-31]: | ||||
| MPEG_haptics.avatar object.type is a string which may hold values | ||||
| among "Vibration", "Pressure", "Temperature", "Custom". | ||||
| Perhaps: | ||||
| The avatar type is defined in [ISO.IEC.23090-31]: | ||||
| MPEG_haptics.avatar object.type is a string that may contain the | ||||
| values "Vibration", "Pressure", "Temperature", or "Custom". | ||||
| --> | ||||
| ver: | ver: | |||
| This parameter provides the year of the edition and amendment of ISO/IEC 23090-31 tha t this file conforms to, as defined in {{ISO.IEC.23090-31}}: MPEG_haptics object.vers ion is a string which may hold values such as XXXX or XXXX-Y where XXXX is the year o f publication and Y is the amendment number, if any. For the initial (and current) ve rsion of the MPEG Haptics Coding standard (ISO/IEC 23090-31:2025) , the value is "202 5". When ver is not present, a default value of "2025" SHOULD be inferred. | This parameter provides the year of the edition and amendment of ISO/IEC 23090-31 tha t this file conforms to, as defined in {{ISO.IEC.23090-31}}: MPEG_haptics object.vers ion is a string that may hold values such as XXXX or XXXX-Y where XXXX is the year of publication and Y is the amendment number, if any. For the initial (and current) ver sion of the MPEG Haptics Coding standard (ISO/IEC 23090-31:2025), the value is "2025" . When ver is not present, a default value of "2025" SHOULD be inferred. | |||
| profile: | profile: | |||
| : This parameter indicates the profile used to generate the encoded stream, as define | ||||
| This parameter indicates the profile used to generate the encoded stream as defined i | d in {{ISO.IEC.23090-31}}: MPEG_haptics object.profile is a string that may hold the | |||
| n {{ISO.IEC.23090-31}}: MPEG_haptics object.profile is a string which may hold the va | values "simple-parametric" or "main". When profile is not present, the default value | |||
| lues "simple-parametric" or "main". When profile is not present, the default value "m | "main" SHOULD be inferred. | |||
| ain" SHOULD be inferred. | ||||
| lvl: | lvl: | |||
| : This parameter indicates the level used to generate the encoded stream, as defined | ||||
| This parameter indicates the level used to generate the encoded stream as defined in | in {{ISO.IEC.23090-31}}: MPEG_haptics object.level is an integer that may hold the va | |||
| {{ISO.IEC.23090-31}}: MPEG_haptics object.level is an integer which may hold the valu | lues 1 or 2. When lvl is not present, the default value 2 SHOULD be inferred. | |||
| es 1 or 2. When lvl is not present, the default value 2 SHOULD be inferred. | ||||
| maxlod: | maxlod: | |||
| : This parameter indicates the maximum level of details (LODs) to use for the avatar( | ||||
| This parameter indicates the maximum level of details to use for the avatar(s). The a | s). The avatar LOD is defined in {{ISO.IEC.23090-31}}: MPEG_haptics.avatar object.lod | |||
| vatar level of detail (LOD) is defined in {{ISO.IEC.23090-31}}: MPEG_haptics.avatar o | is an integer that may hold the value 0 or a positive integer. | |||
| bject.lod is an integer which may hold the value 0 or a positive integer. | ||||
| avtypes: | avtypes: | |||
| : This parameter indicates, using a comma-separated list, the types of haptic percept | ||||
| This parameter indicates, using a comma-separated list, types of haptic perception re | ion represented by the avatar(s). The avatar type is defined in {{ISO.IEC.23090-31}}: | |||
| presented by the avatar(s). The avatar type is defined in {{ISO.IEC.23090-31}}: MPEG_ | MPEG_haptics.avatar object.type is a string that may hold values among "Vibration", | |||
| haptics.avatar object.type is a string which may hold values among "Vibration", "Pres | "Pressure", "Temperature", or "Custom". | |||
| sure", "Temperature", "Custom". | ||||
| modalities: | modalities: | |||
| : This parameter indicates, using a comma-separated list, haptic perception modalitie | ||||
| This parameter indicates, using a comma-separated list, haptic perception modalities | s (e.g., pressure, acceleration, velocity, position, temperature, etc.). The percepti | |||
| (e.g., pressure, acceleration, velocity, position, temperature, etc.). The perception | on modality is defined in {{ISO.IEC.23090-31}}: MPEG_haptics.perception object.percep | |||
| modality is defined in {{ISO.IEC.23090-31}}: MPEG_haptics.perception object.percepti | tion_modality is a string that may hold values among "Pressure", "Acceleration", "Vel | |||
| on_modality is a string which may hold values among "Pressure", "Acceleration", "Velo | ocity", "Position", "Temperature", "Vibrotactile", "Water", "Wind", "Force", "Electro | |||
| city", "Position", "Temperature", "Vibrotactile", "Water", "Wind", "Force", "Electrot | tactile", "Vibrotactile Texture", "Stiffness", "Friction", "Humidity", "User-defined | |||
| actile", "Vibrotactile Texture", "Stiffness", "Friction", "Humidity", "User-defined T | Temporal", "User-defined Spatial", or "Other". | |||
| emporal", "User-defined Spatial", "Other". | ||||
| bodypartmask: | bodypartmask: | |||
| : This parameter is an integer that indicates, using a bitmask, the location of the d | ||||
| This parameter is an integer which indicates, using a bitmask, the location of the de | evices or actuators on the body. The body part mask is defined in {{ISO.IEC.23090-31} | |||
| vices or actuators on the body. The body part mask is defined in {{ISO.IEC.23090-31}} | }: MPEG_haptics.reference_device object.body_part_mask is a 32-bit integer that may h | |||
| : MPEG_haptics.reference_device object.body_part_mask is a 32-bit integer which may h | old a bit mask using bit positions defined in Table 7 of {{ISO.IEC.23090-31}}. | |||
| old a bit mask using bit positions defined in table 7 of {{ISO.IEC.23090-31}}. | ||||
| maxfreq: | maxfreq: | |||
| : This parameter is an integer that indicates the maximum frequency of haptic data fo | ||||
| This parameter is an integer which indicates the maximum frequency of haptic data for | r vibrotactile perceptions (Hz). Maximum frequency is defined in {{ISO.IEC.23090-31}} | |||
| vibrotactile perceptions (Hz). Maximum frequency is defined in {{ISO.IEC.23090-31}}: | : MPEG_haptics.reference_device object.maximum_frequency. | |||
| MPEG_haptics.reference_device object.maximum_frequency. | ||||
| minfreq: | minfreq: | |||
| : This parameter is an integer that indicates the minimum frequency of haptic data fo | ||||
| This parameter is an integer which indicates the minimum frequency of haptic data for | r vibrotactile perceptions (Hz). Minimum frequency is defined in {{ISO.IEC.23090-31}} | |||
| vibrotactile perceptions (Hz). Minimum frequency is defined in {{ISO.IEC.23090-31}}: | : MPEG_haptics.reference_device object.minimum_frequency. | |||
| MPEG_haptics.reference_device object.minimum_frequency. | ||||
| dvctypes: | dvctypes: | |||
| : This parameter is an integer that indicates, using a comma-separated list, the type | ||||
| This parameter indicates, using a comma-separated list, the types of actuators. The d | s of actuators. The device type is defined in {{ISO.IEC.23090-31}}: MPEG_haptics.refe | |||
| evice type is defined in {{ISO.IEC.23090-31}}: MPEG_haptics.reference_device object.t | rence_device object.type is a string that may hold values among "LRA", "VCA", "ERM", | |||
| ype is a string which may hold values among "LRA", "VCA", "ERM", "Piezo" or "Unknown" | "Piezo", or "Unknown". | |||
| . | ||||
| silencesupp: | silencesupp: | |||
| : This parameter is an integer that indicates whether silence suppression should be u | ||||
| This parameter is an integer which indicates whether silence suppression should be us | sed (value 1) or not (value 0). When silencesupp is not present, the default value 0 | |||
| ed (1) or not (0). When silencesupp is not present, the default value 0 SHOULD be inf | SHOULD be inferred. | |||
| erred. | ||||
| ## SDP Parameter Registration {#sdp-registration} | ## SDP Parameter Registration {#sdp-registration} | |||
| This memo registers a 'haptics' token in the media sub-registry of the Session Descri ption Protocols (SDP) Parameters registry. This registration contains the required in formation elements outlined in the SDP registration procedure defined in section 8.2 of {{RFC8866}}. | This memo registers a 'haptics' token in the media subregistry of the "Session Descri ption Protocol (SDP) Parameters" registry group. This registration contains the requi red information elements outlined in the SDP registration procedure defined in {{Sect ion 8.2 of RFC8866}}. | |||
| (1) Contact Information: | <!--[rfced] Please note that we updated the registration | |||
| template in Section 6.2 as follows. Please let us know of any objections. | ||||
| Name: Hyunsik Yang | Original: | |||
| Email: hyunsik.yang@interdigital.com | (1) Contact Information: | |||
| (2) Name being registered (as it will appear in SDP): haptics | Name: Hyunsik Yang | |||
| Email: hyunsik.yang@interdigital.com | ||||
| (3) Long-form name in English: haptics | (2) Name being registered (as it will appear in SDP): haptics | |||
| (4) Type of name ('media', 'proto', 'fmt', 'bwtype', 'nettype', or | (3) Long-form name in English: haptics | |||
| 'addrtype'): media | ||||
| (5) Purpose of the registered name: | (4) Type of name ('media', 'proto', 'fmt', 'bwtype', 'nettype', | |||
| or 'addrtype'): media | ||||
| The 'haptics' media type for the Session Description Protocol | (5) Purpose of the registered name: | |||
| is used to describe a media stream whose content can be | ||||
| rendered as touch-related sensations. | ||||
| The media subtype further describes the specific | ||||
| format of the haptics stream. The 'haptics' media type for | ||||
| SDP is used to establish haptics media streams. | ||||
| (6) Specification for the registered name: RFC XXXX | The 'haptics' media type for the Session Description Protocol | |||
| is used to describe a media stream whose content can be | ||||
| rendered as touch-related sensations. | ||||
| The media subtype further describes the specific | ||||
| format of the haptics stream. The 'haptics' media type for | ||||
| SDP is used to establish haptics media streams. | ||||
| RFC Editor Note: Replace RFC XXXX with the published RFC number. | (6) Specification for the registered name: RFC XXXX | |||
| Current: | ||||
| Contact name: Hyunsik Yang | ||||
| Contact email address: hyunsik.yang@interdigital.com | ||||
| Name being defined (as it will appear in SDP): haptics | ||||
| Type of name: media | ||||
| Description: The 'haptics' media type for the Session Description | ||||
| Protocol is used to describe a media stream whose content can be | ||||
| rendered as touch-related sensations. The media subtype further | ||||
| describes the specific format of the haptics stream. The | ||||
| 'haptics' media type for SDP is used to establish haptics media | ||||
| streams. | ||||
| Reference: RFC 9993 | ||||
| --> | ||||
| Contact name: | ||||
| : Hyunsik Yang | ||||
| Contact email address: | ||||
| : hyunsik.yang@interdigital.com | ||||
| Name being defined (as it will appear in SDP): | ||||
| : haptics | ||||
| Type of name: | ||||
| : media | ||||
| Description: | ||||
| : The 'haptics' media type for the Session Description Protocol | ||||
| is used to describe a media stream whose content can be rendered | ||||
| as touch-related sensations. The media subtype further describes | ||||
| the specific format of the haptics stream. The 'haptics' media | ||||
| type for SDP is used to establish haptics media streams. | ||||
| Reference: | ||||
| : RFC 9993 | ||||
| # SDP Considerations | # SDP Considerations | |||
| The mapping of above defined payload format media type to the corresponding fields in the Session Description Protocol (SDP) is done according to {{RFC8866}}. | The mapping of the above-defined payload format media type to the corresponding field s in SDP is done according to {{RFC8866}}. | |||
| The media name in the "m=" line of SDP MUST be haptics. | The media name in the "m=" line of SDP MUST be haptics. | |||
| The encoding name in the "a=rtpmap" line of SDP MUST be hmpg | The encoding name in the "a=rtpmap" line of SDP MUST be hmpg. | |||
| The clock rate in the "a=rtpmap" line may be any sampling rate, typically 8000. | The clock rate in the "a=rtpmap" line may be any sampling rate, typically 8000. | |||
| The optional parameters (defined in {{optional-param}}), when present, MUST be includ ed in the "a=fmtp" line of SDP. This is expressed as a media type string, in the form of a semicolon-separated list of parameter=value pairs. Parameter values, including string values, MUST be written without quotation marks ("") in SDP. Parameter values which are strings are not case sensitive and SHOULD be written in lowercase. | The optional parameters (defined in {{optional-param}}), when present, MUST be includ ed in the "a=fmtp" line of SDP. This is expressed as a media type string, in the form of a semicolon-separated list of parameter=value pairs. Parameter values, including string values, MUST be written without quotation marks ("") in SDP. Parameter values that are strings are not case sensitive and SHOULD be written in lowercase. | |||
| An example of media representation corresponding to the hmpg RTP payload in SDP is as follows: | An example of media representation corresponding to the hmpg RTP payload in SDP is as follows: | |||
| <!--[rfced] The figure in Section 7 is marked as artwork. Please | ||||
| confirm if this is correct or if it should be marked as sourcecode | ||||
| instead. | ||||
| If it is marked as sourcecode, please consider whether the "type" | ||||
| attribute should be set. | ||||
| The current list of preferred values for "type" is available at | ||||
| https://www.rfc-editor.org/materials/sourcecode-types.txt. If the current | ||||
| list does not contain an applicable type, feel free to suggest additions | ||||
| for consideration. Note that it is also acceptable to leave the "type" | ||||
| attribute not set. | ||||
| --> | ||||
| ~~~ | ||||
| m=haptics 43291 UDP/TLS/RTP/SAVPF 115 | m=haptics 43291 UDP/TLS/RTP/SAVPF 115 | |||
| a=rtpmap:115 hmpg/8000 | a=rtpmap:115 hmpg/8000 | |||
| a=fmtp:115 profile=main;lvl=1;ver=2025 | a=fmtp:115 profile=main;lvl=1;ver=2025 | |||
| ~~~ | ||||
| ## SDP Offer/Answer Considerations {#sdp-cons} | ## SDP Offer/Answer Considerations {#sdp-cons} | |||
| When using the offer/answer procedure described in {{RFC3264}} to negotiate the use o f haptic, the following considerations apply: | When using the offer/answer procedure described in {{RFC3264}} to negotiate the use o f haptic, the following considerations apply: | |||
| When used for a unidirectional stream, the SDP parameters represent the properties of the sender (on the sending side) and of the receiver (on the receiving side). When u sed for a sendrecv stream, the SDP parameters represent the properties of the receive r. | When used for a unidirectional stream, the SDP parameters represent the properties of the sender (on the sending side) and of the receiver (on the receiving side). When u sed for a sendrecv stream, the SDP parameters represent the properties of the receive r. | |||
| The receiver properties expressed using the SDP parameters 'ver', 'profile' and 'lvl' correspond to implementation capabilities. The ver, profile, lvl parameters MUST be used symmetrically in SDP offer and answer. That is, their values in the answer MUST match those in the offer, either explicitly signaled or implicitly inferred. In the s ame session, ver, profile, and lvl MUST NOT be changed in subsequent offers or answer s. | The receiver properties expressed using the SDP parameters 'ver', 'profile', and 'lvl ' correspond to implementation capabilities. The ver, profile, and lvl parameters MUS T be used symmetrically in SDP offer and answer. That is, their values in the answer MUST match those in the offer, either explicitly signaled or implicitly inferred. In the same session, ver, profile, and lvl MUST NOT be changed in subsequent offers or a nswers. | |||
| The properties expressed using SDP parameters other than 'ver', 'profile' and 'lvl' a re provided as recommendations for efficient data transmission and are not binding, m eaning that a sender is encouraged but not required to conform to the parameters spec ified by the receiver. These properties MAY be set to different values in offers and answers. These properties MAY be updated in subsequent offers or answers. | The properties expressed using SDP parameters other than 'ver', 'profile', and 'lvl' are provided as recommendations for efficient data transmission and are not binding, meaning that a sender is encouraged but not required to conform to the parameters spe cified by the receiver. These properties MAY be set to different values in offers and answers. These properties MAY be updated in subsequent offers or answers. | |||
| Any receiver compliant with {{ISO.IEC.23090-31}} MUST be capable of decoding any stre am with a compatible version, profile, and level. A receiver supporting a more genera l profile will accept a stream corresponding to a same or less general profile (e.g., "main" is more general than "simple-parametric"). A receiver supporting a given leve l will accept a stream corresponding to a same or lower level. A receiver supporting a given version will accept a stream corresponding to the same version and MAY accept other versions. A receiver MAY ignore any part of a received stream, e.g., that it d oes not have support for rendering. | Any receiver compliant with {{ISO.IEC.23090-31}} MUST be capable of decoding any stre am with a compatible version, profile, and level. A receiver supporting a more genera l profile will accept a stream corresponding to the same or a less general profile (e .g., "main" is more general than "simple-parametric"). A receiver supporting a given level will accept a stream corresponding to the same or a lower level. A receiver sup porting a given version will accept a stream corresponding to the same version and MA Y accept other versions. A receiver MAY ignore any part of a received stream, e.g., t hat it does not have support for rendering. | |||
| The haptic signal can be sampled at different rates. The MPEG Haptics Coding standard does not mandate a specific frequency. A typical sample rate is 8000Hz. | The haptic signal can be sampled at different rates. The MPEG Haptics Coding standard does not mandate a specific frequency. A typical sample rate is 8000Hz. | |||
| The parameter 'ver' indicates the version of the haptic standard specification. If it | <!--[rfced] May we remove the second sentence below? It appears to be | |||
| is not specified, the The parameter 'ver' indicates the version of the haptic standa | duplicate information from the first sentence. | |||
| rd specification. If it is not specified, the value "2025" indicating the MPEG Haptic | ||||
| s Coding standard ISO/IEC 23090-31:2025 {{ ISO.IEC.23090-31}} SHOULD be inferred, al | ||||
| though the sender and receiver MAY use a specific value based on an out-of-band agree | ||||
| ment. The parameter 'profile' is used to restrict the number of tools used (e.g., the | ||||
| simple-parametric profile fits enable simpler implementations than the main profile) | ||||
| . If it is not specified, the most general profile "main" SHOULD be inferred, althoug | ||||
| h the sender and receiver MAY use a specific value based on an out-of-band agreement. | ||||
| The parameter 'lvl' is used to further characterize implementations within a given p | ||||
| rofile, e.g., according to the maximum supported number of channels, bands, and perce | ||||
| ptions. If it is not specified, the most general level "2" SHOULD be inferred, althou | ||||
| gh the sender and receiver MAY use a specific version based on an out-of-band agreeme | ||||
| nt. | ||||
| Other parameters can be used to indicate bitstream properties as well as receiver cap | Original: | |||
| abilities. The parameters 'maxlod', 'avtypes', 'bodypartmask', 'maxfreq', 'minfreq', | The parameter 'ver' indicates the version of the haptic standard | |||
| 'dvctypes', and 'modalities' can be sent by a sender to reflect the characteristics o | specification. If it is not specified, the The parameter 'ver' | |||
| f bitstreams and can be set by a receiver to reflect the nature and capabilities of l | indicates the version of the haptic standard specification. If it is | |||
| ocal actuator devices, or a preferred set of bitstream properties. For example, diffe | not specified, the value "2025" indicating the MPEG Haptics Coding | |||
| rent receivers MAY have different sets of local actuators, in which case these parame | standard ISO/IEC 23090-31:2025 [ISO.IEC.23090-31] SHOULD be inferred, | |||
| ters can be used to select a stream adapted to the receiver. In some other cases, som | although the sender and receiver MAY use a specific value based on an | |||
| e receivers MAY indicate a preference for a set of bitstream properties such as perce | out-of-band agreement. | |||
| ptions, min/max frequency, or body-part-mask, which contribute the most to the user e | ||||
| xperience for a given application, in which case these parameters can be used to sele | Perhaps: | |||
| ct a stream which include and possibly prioritizes those properties. For example, if | The parameter 'ver' indicates the version of the haptic standard | |||
| the haptic stream server provides more information than the body mask specified by th | specification. If it is not specified, the value "2025" indicating | |||
| e receiver, the additional information can be either integrated into a single effect | the MPEG Haptics Coding standard ISO/IEC 23090-31:2025 | |||
| or ignored by the receiver. | [ISO.IEC.23090-31] SHOULD be inferred, although the sender and | |||
| receiver MAY use a specific value based on an out-of-band | ||||
| agreement. | ||||
| --> | ||||
| The parameter 'ver' indicates the version of the haptic standard specification. If it | ||||
| is not specified, the parameter 'ver' indicates the version of the haptic standard s | ||||
| pecification. If it is not specified, the value "2025" indicating the MPEG Haptics Co | ||||
| ding standard ISO/IEC 23090-31:2025 {{ISO.IEC.23090-31}} SHOULD be inferred, althoug | ||||
| h the sender and receiver MAY use a specific value based on an out-of-band agreement. | ||||
| The parameter 'profile' is used to restrict the number of tools used (e.g., the simp | ||||
| le-parametric profile enables simpler implementations than the main profile). If it i | ||||
| s not specified, the most general profile "main" SHOULD be inferred, although the sen | ||||
| der and receiver MAY use a specific value based on an out-of-band agreement. The para | ||||
| meter 'lvl' is used to further characterize implementations within a given profile, e | ||||
| .g., according to the maximum supported number of channels, bands, and perceptions. I | ||||
| f it is not specified, the most general level "2" SHOULD be inferred, although the se | ||||
| nder and receiver MAY use a specific version based on an out-of-band agreement. | ||||
| <!--[rfced] We note "body-part-mask" as well as "body part mask" | ||||
| (as defined in [ISO.IEC.23090-31]). Should the hyphenated form be | ||||
| updated as shown below for consistency? | ||||
| Original: | ||||
| In some other cases, some receivers MAY indicate a preference for a | ||||
| set of bitstream properties such as perceptions, min/max frequency, | ||||
| or body-part-mask, which contribute ... | ||||
| Perhaps: | ||||
| In some other cases, some receivers MAY indicate a preference for a | ||||
| set of bitstream properties such as perceptions, min/max frequency, | ||||
| or body part mask, which contribute ... | ||||
| --> | ||||
| Other parameters can be used to indicate bitstream properties as well as receiver cap | ||||
| abilities. The parameters 'maxlod', 'avtypes', 'bodypartmask', 'maxfreq', 'minfreq', | ||||
| 'dvctypes', and 'modalities' can be sent by a sender to reflect the characteristics o | ||||
| f bitstreams and can be set by a receiver to reflect the nature and capabilities of l | ||||
| ocal actuator devices or a preferred set of bitstream properties. For example, differ | ||||
| ent receivers MAY have different sets of local actuators, in which case these paramet | ||||
| ers can be used to select a stream adapted to the receiver. In some other cases, some | ||||
| receivers MAY indicate a preference for a set of bitstream properties such as percep | ||||
| tions, min/max frequency, or body-part-mask, which contribute the most to the user ex | ||||
| perience for a given application, in which case these parameters can be used to selec | ||||
| t a stream that includes and possibly prioritizes those properties. For example, if t | ||||
| he haptic stream server provides more information than the body mask specified by the | ||||
| receiver, the additional information can be either integrated into a single effect o | ||||
| r ignored by the receiver. | ||||
| The parameter 'silencesupp' can be used to indicate sender and receiver capabilities or preferences. This parameter indicates whether silence suppression should be used, as described in {{mihs-trans}}. If it is not specified, the value "0", indicating no silence suppression, SHOULD be inferred, although the sender and receiver MAY use sil ence suppression based on an out-of-band agreement. | The parameter 'silencesupp' can be used to indicate sender and receiver capabilities or preferences. This parameter indicates whether silence suppression should be used, as described in {{mihs-trans}}. If it is not specified, the value "0", indicating no silence suppression, SHOULD be inferred, although the sender and receiver MAY use sil ence suppression based on an out-of-band agreement. | |||
| ## Declarative SDP Considerations | ## Declarative SDP Considerations | |||
| When haptic content over RTP is offered with SDP in a declarative style, the paramete rs capable of indicating both bitstream properties as well as receiver capabilities a re used to indicate only bitstream properties. For example, in this case, the parame ters maxlod, bodypartmask, maxfreq, minfreq, dvctypes, and modalities declare the val ues used by the bitstream, not the capabilities for receiving bitstreams. A receiver of the SDP is required to support all parameters and values of the parameters provide d; otherwise, the receiver MUST reject or not participate in the session. It falls o n the creator of the session to use values that are expected to be supported by the r eceiving application. | When haptic content over RTP is offered with SDP in a declarative style, the paramete rs capable of indicating both bitstream properties as well as receiver capabilities a re used to indicate only bitstream properties. For example, in this case, the parame ters 'maxlod', 'bodypartmask', 'maxfreq', 'minfreq', 'dvctypes', and 'modalities' dec lare the values used by the bitstream, not the capabilities for receiving bitstreams. A receiver of the SDP is required to support all parameters and values of the parame ters provided; otherwise, the receiver MUST reject or not participate in the session. It falls on the creator of the session to use values that are expected to be suppor ted by the receiving application. | |||
| # Congestion Control Considerations | # Congestion Control Considerations | |||
| <!--[rfced] We note only one instance of "HMPG" - is this correct, or | ||||
| should it be lowercase? If it should remain as uppercase, how may we | ||||
| expand it? | ||||
| Original: | ||||
| The general congestion control considerations for transporting RTP | ||||
| data apply to HMPG haptics over RTP as well [RFC3550] | ||||
| --> | ||||
| The general congestion control considerations for transporting RTP data apply to HMPG haptics over RTP as well {{RFC3550}}. | The general congestion control considerations for transporting RTP data apply to HMPG haptics over RTP as well {{RFC3550}}. | |||
| It is possible to adapt network bandwidth usage by adjusting either the encoder bit r ate or by adjusting the stream content (e.g., level of detail, body parts, actuator f requency range, target device types, modalities). The considerations in this section are applicable to best-effort networks and controlled environments. | It is possible to adapt network bandwidth usage by adjusting either the encoder bit r ate or the stream content (e.g., the LOD, body parts, actuator frequency range, targe t device types, and modalities). The considerations in this section are applicable to best-effort networks and controlled environments. | |||
| In case of congestion, a receiver or intermediate node MAY prioritize independent pac | <!--[rfced] May we revise this text to be two sentences to improve | |||
| kets over dependent ones, since the non-reception of an independent MIHS unit can pre | readability? | |||
| vent the decoding of multiple subsequent dependent MIHS units. In case of congestion, | ||||
| a receiver or intermediate node MAY prioritize initialization MIHS units over other | ||||
| units, since initialization MIHS units contain metadata used to re-initialize the dec | ||||
| oder, and MAY drop silent MIHS units before other types of MIHS units, since a receiv | ||||
| er MAY interpret a missing MIHS unit as a silence. It is also possible, using the lay | ||||
| er field of the RTP payload header, to allocate MIHS units to different layers based | ||||
| on their content, to prioritize haptic data contributing the most to the user experie | ||||
| nce. In case of congestion, intermediate nodes and receivers SHOULD use the MIHS laye | ||||
| r value to determine the relative importance of haptic RTP packets. | ||||
| Receivers should monitor timestamps and treat gaps as loss of the corresponding MIHS | Original: | |||
| units. MIHS units, as defined in {{ISO.IEC.23090-31}}, should be checked for structur | In case of congestion, a receiver or intermediate node MAY prioritize | |||
| al integrity according to their type. When CRC16 or CRC32 information is present in M | initialization MIHS units over other units, since initialization MIHS | |||
| IHS units, receivers must validate data integrity, and units failing CRC checks shoul | units contain metadata used to re-initialize the decoder, and MAY drop | |||
| d be treated as lost. Receivers should further monitor indicators of service degradat | silent MIHS units before other types of MIHS units, since a receiver | |||
| ion such as unexpected silent gaps, repeated decoder reinitializations, or decoding f | MAY interpret a missing MIHS unit as a silence. | |||
| ailures. Receivers should report packet loss to the sender using RTCP Receiver Report | ||||
| s {{RFC3550}} and, when available, may report detailed loss and jitter metrics using | Perhaps: | |||
| mechanisms described in {{RFC4585}}. | In case of congestion, a receiver or intermediate node MAY prioritize | |||
| initialization MIHS units over other units, as these contain metadata | ||||
| that is used to reinitialize the decoder. Additionally, a receiver or | ||||
| intermediate node MAY drop silent units before other types, as a | ||||
| receiver MAY interpret a missing unit as silence. | ||||
| --> | ||||
| In case of congestion, a receiver or intermediate node MAY prioritize independent pac | ||||
| kets over dependent ones, since the non-reception of an independent MIHS unit can pre | ||||
| vent the decoding of multiple subsequent dependent MIHS units. In case of congestion, | ||||
| a receiver or intermediate node MAY prioritize initialization MIHS units over other | ||||
| units, since initialization MIHS units contain metadata used to reinitialize the deco | ||||
| der, and MAY drop silent MIHS units before other types of MIHS units, since a receive | ||||
| r MAY interpret a missing MIHS unit as a silence. It is also possible, using the laye | ||||
| r field of the RTP payload header, to allocate MIHS units to different layers based o | ||||
| n their content to prioritize haptic data that contributes the most to the user exper | ||||
| ience. In case of congestion, intermediate nodes and receivers SHOULD use the MIHS la | ||||
| yer value to determine the relative importance of haptic RTP packets. | ||||
| Receivers should monitor timestamps and treat gaps as loss of the corresponding MIHS | ||||
| units. MIHS units, as defined in {{ISO.IEC.23090-31}}, should be checked for structur | ||||
| al integrity according to their type. When CRC16 or CRC32 information is present in M | ||||
| IHS units, receivers must validate data integrity, and units failing Cyclic Redundanc | ||||
| y Checks (CRCs) should be treated as lost. Receivers should further monitor indicator | ||||
| s of service degradation such as unexpected silent gaps, repeated decoder reinitializ | ||||
| ations, or decoding failures. Receivers should report packet loss to the sender using | ||||
| RTCP Receiver Reports {{RFC3550}} and, when available, may report detailed loss and | ||||
| jitter metrics using mechanisms described in {{RFC4585}}. | ||||
| # Security Considerations | # Security Considerations | |||
| This RTP payload format is subject to security threats commonly associated with RTP p | The RTP payload format is subject to security threats commonly associated with RTP pa | |||
| ayload formats, as well as threats specific to the interaction of haptic devices with | yload formats, as well as threats specific to the interaction of haptic devices with | |||
| the physical world, and threats associated with the use of compression by the codec. | the physical world and threats associated with the use of compression by the codec. | |||
| Security consideration for threats commonly associated with RTP payload formats are o | Security considerations for threats commonly associated with RTP payload formats are | |||
| utlined in {{RFC3550}}, as well as in RTP profiles such as RTP/AVP {{RFC3551}}), RTP/ | outlined in {{RFC3550}}, as well as in RTP profiles such as RTP/AVP {{RFC3551}}, RTP/ | |||
| AVPF {{RFC4585}}, RTP/SAVP {{RFC3711}}, or RTP/SAVPF {{RFC5124}}. | AVPF {{RFC4585}}, RTP/SAVP {{RFC3711}}, and RTP/SAVPF {{RFC5124}}. | |||
| Haptic sensors and actuators operate within the physical environment. This introduces the potential for information leakage through sensors, or damage to actuators due to data tampering. Additionally, misusing the functionalities of actuators (such as for ce, position, temperature, vibration, electro-tactile, etc.) may pose a risk of harm to the user, for example by setting keyframe parameters (e.g., amplitude, position, f requency) or channel gain to a value that surpasses a permissible range. While indivi dual devices can implement security measures to reduce or eliminate those risks on a per-device basis, in some cases harm can be inflicted by setting values which are per missible for the individual device. For example, causing contact with the physical en vironment or triggering unexpected force feedback can potentially harm the user. Each haptic system should therefore implement system-dependent security measures, which a re more error prone. To limit the risk that attackers exploit weaknesses in haptic sy stems, it is important that haptic transmission should be protected against malicious traffic injection or tampering. | Haptic sensors and actuators operate within the physical environment. This introduces the potential for information leakage through sensors or damage to actuators due to data tampering. Additionally, misusing the functionalities of actuators (such as forc e, position, temperature, vibration, electrotactile, etc.) may pose a risk of harm to the user, for example, by setting keyframe parameters (e.g., amplitude, position, an d frequency) or channel gain to a value that surpasses a permissible range. While ind ividual devices can implement security measures to reduce or eliminate those risks on a per-device basis, in some cases, harm can be inflicted by setting values that are permissible for the individual device. For example, causing contact with the physical environment or triggering unexpected force feedback can potentially harm the user. E ach haptic system should therefore implement system-dependent security measures, whic h are more error prone. To limit the risk that attackers exploit weaknesses in haptic systems, it is important that haptic transmission be protected against malicious tra ffic injection or tampering. | |||
| However, as "Securing the RTP Framework: Why RTP Does Not Mandate a Single Media Secu rity Solution" {{RFC7202}} discusses, it is not an RTP payload format's responsibilit y to discuss or mandate what solutions are used to meet the basic security goals like confidentiality, integrity, and source authenticity for RTP in general. The responsi bility for implementing security mechanisms lies with the application developer. They can find guidance on available security mechanisms and important considerations in " Options for Securing RTP Sessions" {{RFC7201}}, although {{RFC7201}} is now considere d dated and several mechanisms described therein have since evolved. | However, as "Securing the RTP Framework: Why RTP Does Not Mandate a Single Media Secu rity Solution" {{RFC7202}} discusses, it is not an RTP payload format's responsibilit y to discuss or mandate what solutions are used to meet the basic security goals like confidentiality, integrity, and source authenticity for RTP in general. The responsi bility for implementing security mechanisms lies with the application developer. They can find guidance on available security mechanisms and important considerations in " Options for Securing RTP Sessions" {{RFC7201}}, although {{RFC7201}} is now considere d dated and several mechanisms described therein have since evolved. | |||
| Applications SHOULD use appropriate and current strong security mechanisms. For moder n best practices, applications can consider the following options: | Applications SHOULD use appropriate and current strong security mechanisms. For moder n best practices, applications can consider the following options: | |||
| <!--[rfced] BCP 195 contains RFCs 8996 and 9325. Should there be a | ||||
| reference to BCP 195 in this sentence (with a corresponding entry under | ||||
| the Informative References section), or is the reference to [RFC9325] | ||||
| sufficient? | ||||
| Current: | ||||
| (D)TLS-based protection: For guidance on using TLS 1.3 and DTLS, | ||||
| applications should refer to BCP 195, including [RFC9325], which | ||||
| provides up-to-date recommendations. | ||||
| --> | ||||
| - (D)TLS-based protection: | - (D)TLS-based protection: | |||
| For guidance on using TLS 1.3 and DTLS, applications should refer to BCP 195, | For guidance on using TLS 1.3 and DTLS, applications should refer to BCP 195, | |||
| including {{RFC9325}}, which provides up-to-date recommendations. | including {{RFC9325}}, which provides up-to-date recommendations. | |||
| - IPsec-based protection: | - IPsec-based protection: | |||
| Relevant and current protocol specifications include {{RFC4303}} (ESP) and {{RFC7296} } (IKEv2). | Relevant and current protocol specifications include {{RFC4303}} ("IP Encapsulating S ecurity Payload (ESP)") and {{RFC7296}} ("Internet Key Exchange Protocol Version 2 (I KEv2)"). | |||
| This document does not mandate a specific security mechanism. Instead, applications a re responsible for selecting mechanisms that follow current best practices for confid entiality, integrity, and source authentication, and that reflect the evolving securi ty landscape beyond what is covered in {{RFC7201}}. | This document does not mandate a specific security mechanism. Instead, applications a re responsible for selecting mechanisms that follow current best practices for confid entiality, integrity, and source authentication and that reflect the evolving securit y landscape beyond what is covered in {{RFC7201}}. | |||
| The haptic codec used with this payload format uses a compression algorithm (see sect ions 8.2.8.5 and 8.3.3.2 in {{ISO.IEC.23090-31}}). An attacker may inject pathologica l datagrams into the stream which are complex to decode and cause the receiver to be overloaded, similarly to {{RFC3551}}. | The haptic codec used with this payload format uses a compression algorithm (see Sect ions 8.2.8.5 and 8.3.3.2 in {{ISO.IEC.23090-31}}). An attacker may inject pathologica l datagrams into the stream that are complex to decode and cause the receiver to be o verloaded, similarly to {{RFC3551}}. | |||
| End-to-end security with authentication, integrity, or confidentiality protection wil l prevent a Media-Aware Network Element (MANE) from performing media-aware operations other than discarding complete packets. In the case of confidentiality protection, i t will even be prevented from discarding packets in a media-aware way. To be allowed to perform such operations, a MANE is required to be a trusted entity that is include d in the security context establishment. | End-to-end security with authentication, integrity, or confidentiality protection wil l prevent a Media-Aware Network Element (MANE) from performing media-aware operations other than discarding complete packets. In the case of confidentiality protection, i t will even be prevented from discarding packets in a media-aware way. To be allowed to perform such operations, a MANE is required to be a trusted entity that is include d in the security context establishment. | |||
| # IANA Considerations | # IANA Considerations | |||
| ## Media Type Registration Update | ## Media Type Registration Update | |||
| This memo updates the 'hmpg' haptic subtype defined in {{RFC9695}} for use with the M PEG-I haptics streamable binary coding format described in ISO/IEC 23090-31: Haptics coding {{ISO.IEC.23090-31}}. This memo especially defines optional parameters for thi s type in {{optional-param}}. The original subtype registration for haptics/hmpg, reg istered with IANA in {{RFC9695}}, did not include any required or optional parameters . This document introduces optional parameters to enable extended functionality while maintaining backward compatibility. | This memo updates the 'hmpg' haptic subtype defined in {{RFC9695}} for use with the M PEG-I haptics streamable binary coding format described in ISO/IEC 23090-31: Haptics coding {{ISO.IEC.23090-31}}. This memo defines optional parameters for this type in { {optional-param}}. The original subtype registration for 'haptics/hmpg', registered w ith IANA in {{RFC9695}}, did not include any required or optional parameters. This do cument introduces optional parameters to enable extended functionality while maintain ing backward compatibility. | |||
| A mapping of the parameters into the Session Description Protocol (SDP) {{RFC8866}} i | A mapping of the parameters into SDP {{RFC8866}} is also provided for applications th | |||
| s also provided for applications that use SDP. Equivalent parameters could be defined | at use SDP. Equivalent parameters could be defined elsewhere for use with control pro | |||
| elsewhere for use with control protocols that do not use SDP. | tocols that do not use SDP. The receiver MUST ignore any parameter unspecified in thi | |||
| The receiver MUST ignore any parameter unspecified in this memo. | s memo. | |||
| This document requests an SDP parameters registration for the haptic media type, as d escribed in {{sdp-registration}}. | IANA has updated the registration for 'haptics', described in {{sdp-registration}}, i n the "haptics" registry within the "Media Types" registry group and listed document as an additional reference. | |||
| The following entries identify the media type being updated: | The following entries identify the updates to the 'media/haptics' registration: | |||
| Type name: haptics | Type name: | |||
| : haptics | ||||
| Subtype name: hmpg | Subtype name: | |||
| : hmpg | ||||
| The following entries are replaced by this memo: | The following entries are replaced by this memo: | |||
| Optional parameters: see section 6.2 of RFC XXX (note to RFC editor: replace with thi | Optional parameters: | |||
| s RFC's number). | : See {{sdp-registration}} of RFC 9993 | |||
| Person & email address to contact for further information: Yeshwant Muthusamy (yeshwa | Person & email address to contact for further information: | |||
| nt@yeshvik.com) and Hyunsik Yang (hyunsik.yang@interdigital.com) | : {{{Yeshwant Muthusamy}}} (yeshwant@yeshvik.com) and {{{Hyunsik Yang}}} (hyunsik.yan | |||
| g@interdigital.com) | ||||
| <!-- [rfced] We had trouble matching up the text in the IANA Considerations with the | ||||
| IANA registrations. We have attempted to make the text more clear. This includes ad | ||||
| ding a section 10.2 to specify the new SDP parameters media registration. | ||||
| Please review carefully and let us know if any updates are needed. | ||||
| --> | ||||
| ## New SDP Parameters Media Registration | ||||
| IANA has registered the following in the "media" registry within the "Session Descrip | ||||
| tion Protocol (SDP) Parameters" registration group. | ||||
| | Type | SDP Name | Reference | | ||||
| |:---------------------------:| | ||||
| |media | haptics | RFC 9993 | | ||||
| --- back | ||||
| # Acknowledgments | # Acknowledgments | |||
| {:numbered="false"} | ||||
| Thanks to Philippe Guillotel, Quentin Galvane, Jonathan Lennox, Marius Kleidl and Ste | Thanks to {{{Philippe Guillotel}}}, {{{Quentin Galvane}}}, {{{Jonathan Lennox}}}, {{{ | |||
| phan Wenger for the comments and discussions about this draft. | Marius Kleidl}}}, and {{{Stephan Wenger}}} for the comments and discussions about thi | |||
| s document. | ||||
| <!--[rfced] FYI: We have added an expansion for the following | ||||
| abbreviation per Section 3.6 of RFC 7322 ("RFC Style Guide"). | ||||
| Please review this as well as each expansion in the document | ||||
| carefully to ensure correctness. | ||||
| Cyclic Redundancy Checks (CRCs) | ||||
| --> | ||||
| <!--[rfced] Please review the "Inclusive Language" portion of the online | ||||
| Style Guide <https://www.rfc-editor.org/styleguide/part2/#inclusive_language> | ||||
| and let us know if any changes are needed. Updates of this nature typically | ||||
| result in more precise language, which is helpful for readers. | ||||
| Note that our script did not flag any words in particular, but this should | ||||
| still be reviewed as a best practice. | ||||
| --> | ||||
| End of changes. 129 change blocks. | ||||
| 305 lines changed or deleted | 566 lines changed or added | |||
This html diff was produced by rfcdiff 1.48. | ||||