Skip to content

Saturation when moving data from 32b to 16b

In the IVAS fix point code, there is this function

Copy_Scale_sig_32_16()

which moves data from 32 bits to 16 bits vector. By default, this function allows for saturation. I went into it and checked is saturation could happen there and to what level it could.

Here is a very small subset of what I found :

Value (x[i]) Left shift Multiple of MAX_16 File Line Frame
x[2] = -1508 tmp = 31 1508.05 x MAX_16 OVF in acelp_core_dec_ivas_fx at Ln 2274 at frame 5746
x[0] = -5692 tmp = 29 1423.04 x MAX_16 OVF in acelp_core_dec_ivas_fx at Ln 2274 at frame 6492
x[0] = -5261 tmp = 29 1315.29 x MAX_16 OVF in acelp_core_dec_ivas_fx at Ln 2274 at frame 6486
x[10] = 1163 tmp = 31 1163.04 x MAX_16 OVF in acelp_core_dec_ivas_fx at Ln 2274 at frame 5736
x[4] = 4514 tmp = 29 1128.53 x MAX_16 OVF in acelp_core_dec_ivas_fx at Ln 2274 at frame 6485
x[4] = 4299 tmp = 29 1074.78 x MAX_16 OVF in acelp_core_dec_ivas_fx at Ln 2274 at frame 6553
x[10] = -4291 tmp = 29 1072.78 x MAX_16 OVF in acelp_core_dec_ivas_fx at Ln 2274 at frame 6528
x[10] = 1070 tmp = 31 1070.03 x MAX_16 OVF in acelp_core_dec_ivas_fx at Ln 2274 at frame 6399
x[6] = 4052 tmp = 29 1013.03 x MAX_16 OVF in acelp_core_dec_ivas_fx at Ln 2274 at frame 6509
x[10] = 3968 tmp = 29 992.03 x MAX_16 OVF in acelp_core_dec_ivas_fx at Ln 2274 at frame 6527
x[0] = 72136448 tmp = 5 1.07 x MAX_16 OVF in ivas_core_dec_fx at Ln 933 at frame 5733
x[208] = -1234 tmp = 21 1.21 x MAX_16 OVF in ivas_core_dec_fx at Ln 1215 at frame 2126
x[11] = 371169539 tmp = 3 1.38 x MAX_16 OVF in ivas_hq_core_dec_fx at Ln 927 at frame 1789

When the Multiple of is close to 1.0, it could be normal clipping, but when it is above 1.1 (meaning the value is 10% bigger than MAX_16) it already looks like the scaling is wrong and here we got some vectors that are way above normal clipping

these overflow were found with ltv_STEREO_32k_-stereo_-dtx_32000_32.zip

Edited by vaillancour