Apple Clang 16 Release vs Debug differences in ISM

Commit SHA: 32c65a1d (probably)

This issue shows differences in Release vs. Debug build run of short self-test on Mac M4, OS version 15.6.1, Apple Clang 16. This was discussed in IVAS conformance call on Monday September 22nd.

This is the issue for ISM format. Maximum ABS diff in these cases seems to be 807 so not too serious.

Test showed these results

FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-1 ISM with extended metadata at 64 kbps, 48 kHz in, 48 kHz out, BINAURAL out, rendconf dir w id] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 17)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-1 ISM with metadata at 13.2 kbps, 48 kHz in, 48 kHz out, DTX on, BINAURAL out, random FER at 5%] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 90)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-1 ISM with metadata at 48 kbps, 48 kHz in, 48 kHz out, BINAURAL ROOM IR out, HR, exo, random FER at 5%] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 95)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-1 ISM with metadata at 48 kbps, 48 kHz in, 48 kHz out, BINAURAL ROOM IR out, HR, random FER at 5%] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 220)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-1 ISM with metadata at 48 kbps, 48 kHz in, 48 kHz out, BINAURAL out, HR, random FER at 5%] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 1)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-1 ISM with metadata at 80 kbps, 48 kHz in, 16 kHz out, BINAURAL out (Model from file), HR, exo, random FER at 5%] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 807)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-2 ISM with metadata at 128 kbps, 48 kHz in, 32 kHz out, BINAURAL out (Model from file), HR, exo] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 1)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-2 ISM with metadata at 128 kbps, 48 kHz in, 32 kHz out, BINAURAL out (Model from file), HR] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 1)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-2 ISM with metadata at 16.4 kbps, 48 kHz in, 48 kHz out, BINAURAL out] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 252)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-2 ISM with metadata at 160 kbps, 48 kHz in, 32 kHz out, BINAURAL out] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 84)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-3 ISM with metadata at 192 kbps, 48 kHz in, 48 kHz out, BINAURAL out (Model from file)] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 620)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-3 ISM with metadata at 192 kbps, 48 kHz in, 48 kHz out, BINAURAL out, HR, random FER at 5%] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 1)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-4 ISM with extended metadata and non diegetic pan object switching bitrate 256 kbps, 48 kHz in, 48 kHz out, DTX on, BINAURAL out] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 118)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-4 ISM with extended metadata at 128 kbps, 48 kHz in, 48 kHz out, BINAURAL_ROOM_REVERB out, rendconf dir w id] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 109)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-4 ISM with extended metadata at 128 kbps, 48 kHz in, 48 kHz out, BINAURAL_ROOM_REVERB out] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 115)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-4 ISM with extended metadata at 256 kbps, 48 kHz in, 48 kHz out, BINAURAL out, HR, directivity configuration, random FER at 5%] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 1)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-4 ISM with extended metadata at 256 kbps, 48 kHz in, 48 kHz out, BINAURAL out, HR, exo, directivity configuration, random FER at 5%] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 1)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-4 ISM with metadata at 128 kbps, 48 kHz in, 48 kHz out, BINAURAL out, object editing, DISC, FER at 5%] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 28)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-4 ISM with metadata at 256 kbps, 48 kHz in, 48 kHz out, BINAURAL out, HR, exo] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 1)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-4 ISM with metadata at 256 kbps, 48 kHz in, 48 kHz out, BINAURAL out, HR] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 1)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-4 ISM with metadata at 256 kbps, 48 kHz in, 48 kHz out, BINAURAL out] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 698)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-4 ISM with metadata bitrate switching from 48 kbps to 32 kbps, 48 kHz in, 48 kHz out, BINAURAL out, object editing, JBM Prof 5] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 42)
FAILED tests/codec_be_on_mr_nonselection/test_param_file.py::test_param_file_tests[stv-4 ISm with metadata at 256 kbps, 48 kHz in, 48 kHz out, TD BINAURAL out, HR, exo, OT] - Failed: Difference between ref and dut in output only (Non-BE - MAX_ABS_DIFF: 1)

Build and tests were done with

cmake . -Bbuild-debug
cmake --build build-debug -- -j
cmake . -Bbuild-release -DCMAKE_BUILD_TYPE=Release
cmake --build build-release -- -j
pytest tests/codec_be_on_mr_nonselection -v --update_ref 1 --ref_encoder_path=./build-debug/IVAS_cod --ref_decoder_path=./build-debug/IVAS_dec
pytest tests/codec_be_on_mr_nonselection -v --update_ref 0 --dut_encoder_path=./build-release/IVAS_cod --dut_decoder_path=./build-release/IVAS_dec