FIX_1419_SPATIAL_UMX: Spatial upmix for mono/stereo rendering

Add a configurable spatial upmix for mono/stereo rendering via the renderer config.

Default rendering for mono/stereo output is dry (no spatial upmix), exactly as on the main branch.

Description of the change

Spatial functionality added under FIX_1419_SPATIAL_UMX.

  • To enable a spatial upmix, a renderer configuration file must be supplied specifying channel positions (chapter name MSUPMIX):
    • This file contains 3 parameters: AZIMUTH[], ELEVATION[] and RADIUS[].
    • Radius 0 will be interpreted as omni/non-spatial
    • Any other values are used as a spatial position for rendering (1 value required for mono, 2 for stereo)
  • Technical details:
    • Existing renderers are reused for implementation of this functionality. Overall the changes in this MR are really only "plumbing" to get things to work.
    • In case of binaural rendering the precedent set by high bitrate multichannel is followed:
      • TD object renderer is used for everything except BRIRs, with channel positions set and propagated through hTransSetup.
      • CRend is used for BINAURAL_ROOM_IR; if headrotation is enabled rotation of input sources is performed using EFAP on the pseudo 7.1+4 layout prior to rendering

Affected operating points

  • Decoding a mono bitstream when specifying output format with spatial render config
  • Decoding a stereo bitstream to binaural variants or ambisonics with spatial render config

Overview of rendering paths

Existing paths in main are in italics (last modifications were added in !2346 (merged))

New paths supported by this MR are in bold (requires an explicit render config to be supplied)

Modified in this MR? Input Format Output Format Render config supplied? Rendering path Behaviour on main
Mono Mono N/A Not changed in this MR (direct decoding) N/A
Mono Stereo N/A Not changed in this MR (Non-diegetic upmix) N/A
Mono Multichannel 1 N/A Not changed in this MR (Mixing matrices / ivas_ls_setup_conversion()) N/A
Mono Ambisonics 2 N/A Not changed in this MR (passthrough to channel index 0 (W/Omni)) N/A
Mono Binaural 3 NO Not changed in this MR (Non-diegetic upmix) N/A
Mono Binaural 3 YES TD Object renderer, position specified via renderer config Render config is silently ignored for BINAURAL, render config error (unknown chapter) for BINAURAL_ROOM_REVERB
Mono Binaural (ROOM_IR) NO Not changed in this MR (Non-diegetic upmix) N/A
Mono Binaural (ROOM_IR) YES CRend, position specified by render config but only 0,0 is supported Render config is silently ignored
Mono Split Binaural 4 NO IVAS_ERR_INVALID_RENDER_CONFIG, needs a spatial upmix rendering config IVAS_ERR_WRONG_MODE
Mono Split Binaural 4 YES TD Object renderer, position specified via renderer config IVAS_ERR_WRONG_MODE
Stereo Mono N/A Not changed in this MR (Directly handled by CPE decoding) N/A
Stereo Stereo N/A Not changed in this MR (passthrough/direct decoding) N/A
Stereo Multichannel 1 N/A Not changed in this MR (ivas_ls_setup_conversion()) N/A
Stereo Ambisonics 2 N/A Not changed in this MR (M/S routing to W and Y (W = \frac{L+R}{2}; Y/side = \frac{L-R}{2}) N/A
Stereo Binaural 3 NO Not changed in this MR (Passthrough as Stereo) N/A
Stereo Binaural 3 YES TD Object renderer, positions specified via render config Render config is silently ignored for BINAURAL, render config error (unknown chapter) for BINAURAL_ROOM_REVERB
Stereo Binaural (ROOM_IR) NO Not changed in this MR (Passthrough as Stereo) N/A
Stereo Binaural (ROOM_IR) YES CRend, positions specified by render config but will snap to ±30 or ±90 azimuth with zero elevation IVAS_ERR_INVALID_OUTPUT_FORMAT
Stereo Split Binaural 4 NO IVAS_ERR_INVALID_RENDER_CONFIG, needs a spatial upmix rendering config IVAS_ERR_INVALID_OUTPUT_FORMAT
Stereo Split Binaural 4 YES TD Object renderer, positions specified via render config IVAS_ERR_INVALID_OUTPUT_FORMAT

Commandlines for testing and review (in zip file below).

Zip file with scripts and input render config files. Unzip in IVAS root or adjust paths accordingly.

  1. 5.1, 7.1, 5.1+2, 5.1+4, 7.1+4 2

  2. FOA, HOA2, HOA3 2

  3. BINAURAL, BINAURAL_ROOM_REVERB 2 3 4

  4. BINAURAL_SPLIT_CODED, BINAURAL_SPLIT_PCM 2 3 4

Edited by Archit Tamarapu

Merge request reports

Loading