Saturation when moving data from 32b to 16b
In the IVAS fix point code, there is this function
Copy_Scale_sig_32_16()
which moves data from 32 bits to 16 bits vector. By default, this function allows for saturation. I went into it and checked is saturation could happen there and to what level it could.
Here is a very small subset of what I found :
| Value (x[i]) | Left shift | Multiple of | MAX_16 | File | Line | Frame | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| x[2] = | -1508 | tmp = | 31 | 1508.05 | x MAX_16 | OVF in | acelp_core_dec_ivas_fx | at Ln | 2274 | at frame | 5746 |
| x[0] = | -5692 | tmp = | 29 | 1423.04 | x MAX_16 | OVF in | acelp_core_dec_ivas_fx | at Ln | 2274 | at frame | 6492 |
| x[0] = | -5261 | tmp = | 29 | 1315.29 | x MAX_16 | OVF in | acelp_core_dec_ivas_fx | at Ln | 2274 | at frame | 6486 |
| x[10] = | 1163 | tmp = | 31 | 1163.04 | x MAX_16 | OVF in | acelp_core_dec_ivas_fx | at Ln | 2274 | at frame | 5736 |
| x[4] = | 4514 | tmp = | 29 | 1128.53 | x MAX_16 | OVF in | acelp_core_dec_ivas_fx | at Ln | 2274 | at frame | 6485 |
| x[4] = | 4299 | tmp = | 29 | 1074.78 | x MAX_16 | OVF in | acelp_core_dec_ivas_fx | at Ln | 2274 | at frame | 6553 |
| x[10] = | -4291 | tmp = | 29 | 1072.78 | x MAX_16 | OVF in | acelp_core_dec_ivas_fx | at Ln | 2274 | at frame | 6528 |
| x[10] = | 1070 | tmp = | 31 | 1070.03 | x MAX_16 | OVF in | acelp_core_dec_ivas_fx | at Ln | 2274 | at frame | 6399 |
| x[6] = | 4052 | tmp = | 29 | 1013.03 | x MAX_16 | OVF in | acelp_core_dec_ivas_fx | at Ln | 2274 | at frame | 6509 |
| x[10] = | 3968 | tmp = | 29 | 992.03 | x MAX_16 | OVF in | acelp_core_dec_ivas_fx | at Ln | 2274 | at frame | 6527 |
| x[0] = | 72136448 | tmp = | 5 | 1.07 | x MAX_16 | OVF in | ivas_core_dec_fx | at Ln | 933 | at frame | 5733 |
| x[208] = | -1234 | tmp = | 21 | 1.21 | x MAX_16 | OVF in | ivas_core_dec_fx | at Ln | 1215 | at frame | 2126 |
| x[11] = | 371169539 | tmp = | 3 | 1.38 | x MAX_16 | OVF in | ivas_hq_core_dec_fx | at Ln | 927 | at frame | 1789 |
When the Multiple of is close to 1.0, it could be normal clipping, but when it is above 1.1 (meaning the value is 10% bigger than MAX_16) it already looks like the scaling is wrong and here we got some vectors that are way above normal clipping
these overflow were found with ltv_STEREO_32k_-stereo_-dtx_32000_32.zip
Edited by vaillancour