[non-be][split-non-be][allow regression] Resolve "Undefined behaviour in BASOP implementation causes assert in Stereo DMX EVS encoder with optimized build"

Manuel Janderrequested to merge
basop-2600-undefined-behaviour-in-basop-implementation-causes-assert-in-stereo-dmx-evs-encoder-with into main

Reason why this change is needed

  • Undefined behaviour errors shall not happen, to make sure that Basop is bit exact on different hardware/compilers, but it was discovered that this is not the case.
  • Overflow detection of some operators does not work, depending on compiler optimizer behaviour, resulting in undefined behaviour.
  • Some shift operators limit the shift range to nbits instead of nbits-1, or do not check/limit it at all, and leads to undefined behaviour.
  • Some bugs were uncovered while trying to fix undefined behaviour problems, resulting in non bit exact result.

Description of the change

  • Fix overflow detection so that it will not be removed by any optimizer.
  • Fix shift range checks/limits
  • Fix bugs encountered while avoiding undefined behaviour.

Affected operating points

  • Any operating point.

Closes #2600 (closed)

Edited by Manuel Jander

Merge request reports