[Non-BE][Split Non-BE][allow regression]Remove HARM_HQ_CORE_KEEP_BE in fine_gain_pred_fx

norvellrequested to merge
2486-hq-core-harmonization-issue-precision-in-fine_gain_pred_fx into main

Reason why this change is needed

  • The conversion of the fine_gain_pred_fx in IVAS was done using calculations where the EVS BASOP used a lookup table for the square roots of the bandwidths. The calculations used truncation while the table used rounding, which resulted in a small difference. To keep BE for IVAS, a temporary fix was added to the code.
  • This would remove the last HARM_HQ_CORE_KEEP_BE that was added during HQ core harmonization in !2837 (merged).

Description of the change

  • Use table lookup for square root of bandwidths, like in EVS BASOP. Remove patch that would correct between the rounding used in the table, and the truncation from the calculations in the IVAS code. This piece of code is removed:
#ifdef HARM_HQ_CORE_KEEP_BE 
if ( EQ_16( bw_idx, 0 ) )
{ 
   tmp1--; /* Lookup table uses rounding while calculations use truncation */ 
}
#endif

Affected operating points

  • Small differences (improvements) expected for HQ core usage, which means wide-spread differences will occur.

Closes #2486 (closed)

Edited by norvell

Merge request reports