HQ core harmonization issue: precision in fine_gain_pred_fx
The conversion of the fine_gain_pred_fx in IVAS was done using calculations where the EVS BASOP used a lookup table for the square roots of the bandwidths. The calculations used truncation while the table used rounding, which resulted in a small difference. To keep BE for IVAS, a temporary fix was added to the code. ``` #ifdef HARM_HQ_CORE_KEEP_BE if ( EQ_16( bw_idx, 0 ) ) { tmp1--; /* Lookup table uses rounding while calculations use truncation */ } #endif ``` The rounding error should actually impact more bands, but perhaps it is not triggered in the current test set. The below MATLAB code illustrates the issue. ``` sfms =[ 8, 16, 24, 32, 40, 48, 64, 80, 96 ]; floor(sqrt(sfms)*2^11) - round(sqrt(sfms)*2^11) ans = -1 0 0 0 -1 -1 0 -1 0 ``` Removing the adaptation for IVAS would be non-BE, but should improve the performance slightly since the rounding is more accurate.
task