Waveform and MD desynchronized in external renderer
There is a problem with the synchronization of metadata and waveform in the case of using an external renderer. (This is a follow-up issue to #356 (closed), a related discussion can be found also in #350 (closed), brought to a broader audience.)
In MASA and ISM, metadata and waveforms are time-synchronized at the encoder input. The encoder and then the decoder processing introduces a delay to the waveform while the delay is 12 or 17 ms depending on the renderer used. The metadata are not delayed by the coding process so they are artificially delayed at the decoder by two sub-frames (i.e. by 10 ms, the closest value to 12 ms giving the renderers resolution of 5 ms).
When using an external renderer, the (non-delayed) decoded MD are written to an external file together with the decoded synthesis in a pass-through configuration. Next, they form an input to the external renderer but the renderer is not aware of any time differences between the MD and decoded synthesis. Consequently,
- the MD and waveform are desynchronized
- the output between the decoder with the internal renderer and the decoder followed by an external renderer is very different while it should be theoretically bit-exact
A solution could be to introduce a delay synchronization parameter, e.g. -synchro_delay
to the external renderer which will represent the delay by which the MD should be delayed. Then, there would be basically two cases
- original audio and MD will be rendered using
-synchro_delay 0
and there will be no MD delay - decoded audio and MD will be rendered using
-synchro_delay X
, where X is 12 in case of IVAS, and MD are delayed
Note: the described procedure expects that -no_delay_comp
option is used in the codec, which I believe is the default processing for real usage. When IVAS uses the delay compensation, i.e. without -no_delay_comp
option, there are more options how to solve this issue but I do not find them general enough.