Resolve "Enable rendering to all output formats for EVS mono and IVAS Stereo bitstreams"
- Related issues: #1419 (closed)
- Requested reviewers:
Reason why this change is needed
- Currently the decoder crashes with a segfault if an IVAS commandline with
OutputConfis specified for EVS Mono bitstreams. For IVAS Stereo, rendering to ambisonics outputs and binaural flavours returns an Invalid output format error. - Decoder format switching between formats does not allow preserving output configuration, since not all output formats are supported for all input (bitstream) formats
Description of the change
- Rendering is enabled for:
- Mono to all output formats 1
- Stereo to all output formats (i.e. extends now to ambisonics and binaural variants)
- By default, the upmix for mono/stereo is non-spatial; this means:
- Mono/[Stereo] to ambisonics route to W/[W±Y]
- Binaural rendering uses passive upmix (non-diegetic panning to center)/[passthrough]
Spatial functionality requires changes under FIX_1419_SPATIAL_UMX.
- To enable a spatial upmix, a renderer configuration file must be supplied specifying channel positions (chapter name
MSUPMIX):- This file contains 3 parameters:
AZIMUTH[],ELEVATION[]andRADIUS[]. - Radius 0 will be interpreted as omni/non-spatial
- Any other values are used as a spatial position for rendering (1 value required for mono, 2 for stereo)
- This file contains 3 parameters:
- Technical details:
- Existing renderers are reused for implementation of this functionality. Overall the changes in this MR are really only "plumbing" to get things to work.
- LS setup conversion renderer is used for multichannel outputs
- Ambisonics spherical response (
ivas_mc2sba()or passive upmix for ambisonics rendering) - In case of binaural rendering the precedent set by high bitrate multichannel is followed:
- TD object renderer is used for everything except BRIRs, with channel positions set and propagated through
hTransSetup. - CRend is used for
BINAURAL_ROOM_IR; if headrotation is enabled rotation of input sources is performed using EFAP on the pseudo 7.1+4 layout prior to rendering
- TD object renderer is used for everything except BRIRs, with channel positions set and propagated through
Affected operating points
- Decoding a mono bitstream when specifying output format
- Decoding a stereo bitstream to binaural variants or ambisonics
Overview of rendering paths (when both switches are enabled)
| Input Format | Output Format | Render config supplied? | Rendering path |
|---|---|---|---|
| Mono | Mono | N/A | Not changed in this MR (direct decoding) |
| Mono | Stereo | N/A | Not changed in this MR (Non-diegetic upmix) |
| Mono | Multichannel 2 | N/A | Mixing matrices (ivas_ls_setup_conversion()) |
| Mono | Ambisonics 3 | N/A | Passthrough to channel index 0 (W/Omni) |
| Mono | Binaural 4 | NO | Non-diegetic upmix |
| Mono | Binaural (ROOM_IR) | NO | Non-diegetic upmix |
| Mono | Binaural 4 | YES | TD Object renderer, position specified via renderer config |
| Mono | Binaural (ROOM_IR) | YES | CRend, position specified by render config but only 0,0 is supported |
| Stereo | Mono | N/A | Not changed in this MR (Directly handled by CPE decoding) |
| Stereo | Stereo | N/A | Not changed in this MR (passthrough/direct decoding) |
| Stereo | Multichannel 2 | N/A | Not changed in this MR (ivas_ls_setup_conversion()) |
| Stereo | Ambisonics 3 | N/A | M/S routing to W and Y (W/mid = \frac{L+R}{2}; Y/side = \frac{L-R}{2}) |
| Stereo | Binaural 4 | NO | Passthrough as Stereo |
| Stereo | Binaural (ROOM_IR) | NO | Passthrough as Stereo |
| Stereo | Binaural 4 | YES | TD Object renderer, positions specified via render config |
| Stereo | Binaural (ROOM_IR) | YES | CRend, positions specified by render config but will snap to ±30 or ±90 azimuth with zero elevation |
Commandlines for testing and review (in zip file below).
Zip file with scripts and input render config files. Unzip in IVAS root or adjust paths accordingly.
Edited by Archit Tamarapu