FIX_1419_SPATIAL_UMX: Spatial upmix for mono/stereo rendering
Add a configurable spatial upmix for mono/stereo rendering via the renderer config.
Default rendering for mono/stereo output is dry (no spatial upmix), exactly as on the main branch.
Description of the change
Spatial functionality added under FIX_1419_SPATIAL_UMX.
- To enable a spatial upmix, a renderer configuration file must be supplied specifying channel positions (chapter name
MSUPMIX):- This file contains 3 parameters:
AZIMUTH[],ELEVATION[]andRADIUS[]. - Radius 0 will be interpreted as omni/non-spatial
- Any other values are used as a spatial position for rendering (1 value required for mono, 2 for stereo)
- This file contains 3 parameters:
- Technical details:
- Existing renderers are reused for implementation of this functionality. Overall the changes in this MR are really only "plumbing" to get things to work.
- In case of binaural rendering the precedent set by high bitrate multichannel is followed:
- TD object renderer is used for everything except BRIRs, with channel positions set and propagated through
hTransSetup. - CRend is used for
BINAURAL_ROOM_IR; if headrotation is enabled rotation of input sources is performed using EFAP on the pseudo 7.1+4 layout prior to rendering
- TD object renderer is used for everything except BRIRs, with channel positions set and propagated through
Affected operating points
- Decoding a mono bitstream when specifying output format with spatial render config
- Decoding a stereo bitstream to binaural variants or ambisonics with spatial render config
Overview of rendering paths
Existing paths in main are in italics (last modifications were added in !2346 (merged))
New paths supported by this MR are in bold (requires an explicit render config to be supplied)
| Modified in this MR? | Input Format | Output Format | Render config supplied? | Rendering path | Behaviour on main |
|---|---|---|---|---|---|
| Mono | Mono | N/A | Not changed in this MR (direct decoding) | N/A | |
| Mono | Stereo | N/A | Not changed in this MR (Non-diegetic upmix) | N/A | |
| Mono | Multichannel 1 | N/A | Not changed in this MR (Mixing matrices / ivas_ls_setup_conversion()) |
N/A | |
| Mono | Ambisonics 2 | N/A | Not changed in this MR (passthrough to channel index 0 (W/Omni)) | N/A | |
| Mono | Binaural 3 | NO | Not changed in this MR (Non-diegetic upmix) | N/A | |
| Mono | Binaural 3 | YES | TD Object renderer, position specified via renderer config | Render config is silently ignored for BINAURAL, render config error (unknown chapter) for BINAURAL_ROOM_REVERB | |
| Mono | Binaural (ROOM_IR) | NO | Not changed in this MR (Non-diegetic upmix) | N/A | |
| Mono | Binaural (ROOM_IR) | YES | CRend, position specified by render config but only 0,0 is supported | Render config is silently ignored | |
| Mono | Split Binaural 4 | NO | IVAS_ERR_INVALID_RENDER_CONFIG, needs a spatial upmix rendering config |
IVAS_ERR_WRONG_MODE |
|
| Mono | Split Binaural 4 | YES | TD Object renderer, position specified via renderer config | IVAS_ERR_WRONG_MODE |
|
| Stereo | Mono | N/A | Not changed in this MR (Directly handled by CPE decoding) | N/A | |
| Stereo | Stereo | N/A | Not changed in this MR (passthrough/direct decoding) | N/A | |
| Stereo | Multichannel 1 | N/A | Not changed in this MR (ivas_ls_setup_conversion()) |
N/A | |
| Stereo | Ambisonics 2 | N/A | Not changed in this MR (M/S routing to W and Y (W = \frac{L+R}{2}; Y/side = \frac{L-R}{2}) | N/A | |
| Stereo | Binaural 3 | NO | Not changed in this MR (Passthrough as Stereo) | N/A | |
| Stereo | Binaural 3 | YES | TD Object renderer, positions specified via render config | Render config is silently ignored for BINAURAL, render config error (unknown chapter) for BINAURAL_ROOM_REVERB | |
| Stereo | Binaural (ROOM_IR) | NO | Not changed in this MR (Passthrough as Stereo) | N/A | |
| Stereo | Binaural (ROOM_IR) | YES | CRend, positions specified by render config but will snap to ±30 or ±90 azimuth with zero elevation | IVAS_ERR_INVALID_OUTPUT_FORMAT |
|
| Stereo | Split Binaural 4 | NO | IVAS_ERR_INVALID_RENDER_CONFIG, needs a spatial upmix rendering config |
IVAS_ERR_INVALID_OUTPUT_FORMAT |
|
| Stereo | Split Binaural 4 | YES | TD Object renderer, positions specified via render config | IVAS_ERR_INVALID_OUTPUT_FORMAT |
Commandlines for testing and review (in zip file below).
Zip file with scripts and input render config files. Unzip in IVAS root or adjust paths accordingly.
Edited by Archit Tamarapu