noise floor in SBA speech scene

In an SBA scene where a single speaker is panned around the listener, the binaural output contains a strong noise floor at all frequencies. The command lines are

./ivas-basop/IVAS_cod -sba 3 -max_band fb 32000 48  input.wav bit
./ivas-basop/IVAS_dec BINAURAL 48 bit dec.wav

The commit tested is aa0d22c1 . This is possibly related to #685 (closed), but the noise spans all frequencies here. The noise is audible and the speaker sometimes appears to come from a slightly different location. The problem also appears when the bitstream is generated with the floating-point code and decoded with the fixed-point.