Skip to content

maximum_exp_fx bug : biggest value in vector may not be found

Basic info

6cf87ab2

maximum_exp_fx bug : biggest value in vector may not be found

Bug description

maximum_exp_fx() is located in lib_com/tools/fx.c

if tmp is ever 0x7FFFFFFF, all other mantissas which saturate when shifted, wont be recognized as bigger:

FOR( j = 1; j < lvec_fx; j++ )
{
    IF( LT_16( tmp, shr_sat( vec_fx[j], sub( exp, exp_vec[j] ) ) ) )
    {
        ind = j;
        move16();
#ifdef FIX_1981_MAXIMUM_EXP_FX //<< This is the proposed fix
        scale = sub( norm_s( vec_fx[ind] ), 1 );
        tmp = shl( vec_fx[ind], scale );
        exp = sub( exp_vec[ind], scale );
#else 
        tmp = vec_fx[j];
        move16(); 
        exp = exp_vec[j]; 
        move16(); 
#endif 
    } 
}

Ways to reproduce

simply set vec_fx[0] to 32767 and vec_fx[1] to 32000 while setting exp_vec[0] to 0 and exp_vec[1] to 10 while debugging:

Its easy to see that the second value is way bigger 'cause of its exponent. Though, it wont be recognized as the bigger one.

Edited by Fabian Bauer