Undefined behaviour in BASOP implementation causes assert in Stereo DMX EVS encoder with optimized build
# Basic info <!--- Add commit SHA used to reproduce--> - Fixed point: 73dd51941 # Bug description When building the basop codec with an optimization level \>0, one LTV testcase for the EVS compatible stereo downmix crashes. It seems that the input value to the square root gets negative in frame 9956: ![Screenshot 2026-05-21 at 17.34.11.png](/uploads/66599a889dc5bf44dc8957a468b57792/Screenshot_2026-05-21_at_17.34.11.png){width="900" height="352"} This was found on Linux (conformance reference platform Ubuntu 24.04 + clang 18) as well as on Mac (clang 18). ## Analysis TL;DR: the `W_round48_L_o` BASOP implementation relies on c undefined behaviour for checking overflow and applying saturation. The compiler optimizes parts of that away in -O2. `-fwrapv` build flag can be used to prevent that, but it would be better to not rely on C undef behaviour in the BASOPs. ### Longer version: Mpy_32_32_r(INT32_MIN, INT32_MIN) should saturate to INT32_MAX (= 0x7FFFFFFF). When the codec is built with -O0 this works correctly. But when built with -O2 the same input produces INT32_MIN (= 0x80000000), a sign-flipped, wrapped-around result. The wrong value propagates through the IVAS stereo downmix path: Nr\*Nr becomes negative, so L_tmp1 = Nr² + Ni² becomes negative, the divide that follows produces a negative quotient, and Sqrt32 then triggers the assert. The actual bug is inside `W_round48_L_o` (called from Mpy_32_32_r via W_shr → W_round48_L). It detects 64-bit signed overflow like this: ```c L64_var_out = L64_var1 + L64_var2; if (((L64_var1 ^ L64_var2) & L64_MIN) == 0) { // same sign? if ((L64_var_out ^ L64_var1) & L64_MIN) { // result flipped sign? L64_var_out = (L64_var1 < 0) ? L64_MIN : L64_MAX; // saturate set_overflow(Overflow); } } ``` Under the C standard, signed integer overflow is undefined behavior. At -O2 the compiler is allowed to assume this never occurs in L64_var1 + L64_var2 and thus assumes (L64_var_out ^ L64_var1) & L64_MIN is always zero. The compiler deletes the saturation check, the addition wraps at runtime, and W_extract_h returns the wrong upper half (0x80000000). ### Why -fwrapv fixes it `-fwrapv` tells the compiler that signed integer overflow has well-defined two's-complement wrap-around semantics. With that guarantee, the compiler can no longer assume the overflow check is always false, so the saturation branch survives optimization and W_round48_L_o correctly clamps the result to L64_MAX. Mpy_32_32_r then returns 0x7FFFFFFF and the downstream math stays in valid range. A cleaner long-term fix would be to rewrite the overflow detection without relying on undef behaviour so the code is correct at any optimization level without needing -fwrapv. # Ways to reproduce Here is a test program for the root cause: [mpy_repro.c](/uploads/19b485e677505576c9382d3d08ae55f1/mpy_repro.c) Build like this: ```bash cc -O0 mpy_repro.c -o mpy_O0 cc -O2 mpy_repro.c -o mpy_O2 cc -O2 -fwrapv mpy_repro.c -o mpy_O2_wrap ``` Command line for encoder: <!--Commandline or script--> ```bash make clean OPTIM=2 make -j ./IVAS_cod -stereo_dmx_evs 24400 48 ltv48_STEREO.wav bit ``` <!--- Below are labels that will be added but are not shown in description. This is a template to help fill them. Add further information to the first row and remove and add labels as necessary.-->
issue