[non-BE][split-non-BE][allow regression] Resolve "Assert (Overflow) in ivas_compute_core_buffers_fx()"

Manuel Janderrequested to merge
basop-2569-assert-overflow-in-ivas_compute_core_buffers_fx into main
  • Related issues:
  • Requested reviewers:

Reason why this change is needed

  • A few LTV test cases showed crashes due to shl() overflows of 16 KHz pre-emphasis state
  • The fix of the observed crash produced some regressions, which are also fixed/mitigated with a few improvements.

Description of the change

  • Q_new is adapted in pre_proc_front_ivas_fx() to avoid the saturation crash.
  • A few calls to norm_s() were changed to norm_arr( , 1 ) to avoid small Q_new values or ineffective normalization if some memories are exactly zero. This reduces regressions.
  • st->q_mem_decim16k_fx was removed from Q_new calculations after resampling did already happen. This improves Q_new and reduces regressions.

Affected operating points

The following operating points do not crash anymore:

./IVAS_cod -q -ism_masa 3 1 ../ltv/ltv48_OMASA_3ISM_1TC_ISM1.csv ../ltv/ltv48_OMASA_3ISM_1TC_ISM2.csv ../ltv/ltv48_OMASA_3ISM_1TC_ISM3.csv ../ltv/ltv48_OMASA_3ISM_1TC.met scripts/switchPaths/sw_13k2_512k_2fr_start_48k_omasatechs_3ism.bin 48 ../ltv/ltv48_OMASA_3ISM_1TC.wav out.192

./IVAS_cod -q -ism_masa 3 2 ../ltv/ltv48_OMASA_3ISM_2TC_ISM1.csv ../ltv/ltv48_OMASA_3ISM_2TC_ISM2.csv ../ltv/ltv48_OMASA_3ISM_2TC_ISM3.csv ../ltv/ltv48_OMASA_3ISM_2TC.met scripts/switchPaths/sw_13k2_512k_2fr_start_160k_omasatechs_3ism.bin 48 ../ltv/ltv48_OMASA_3ISM_2TC.wav out.192

./IVAS_cod -q -ism_masa 3 2 ../ltv/ltv48_OMASA_3ISM_2TC_ISM1.csv ../ltv/ltv48_OMASA_3ISM_2TC_ISM2.csv ../ltv/ltv48_OMASA_3ISM_2TC_ISM3.csv ../ltv/ltv48_OMASA_3ISM_2TC.met scripts/switchPaths/sw_13k2_512k_2fr_start_160k_omasatechs_3ism.bin 48 ../ltv/ltv48_OMASA_3ISM_2TC+10dB.wav out.192

For any other operating point small changes may occur, because the calculation of Q_new (encoder preprocessing) changes.

The MLD difference:

  1. -10 dB :
  • Regressions average 0.4, max 0.68
  • Improvements average 0.83, max 3.03
  1. 0 dB :
  • Regressions average 0.31, max 0.55
  • Improvements average 0.80, max 3.9
  1. +10 dB :
  • Regressions average 0.39, max 1.18
  • Improvements average 0.35, max 1.08

Conclusion: at least for -10dB and 0dB the majority of the changes are improvements, for +10dB it's a tie.

Closes #2569 (closed)

Edited by Manuel Jander

Merge request reports