Track potential regressions from MR 2553
This issue is to track potential regressions from !2553 (merged). An LTV regression check has been performed in https://forge.3gpp.org/rep/sa4/audio/ivas-basop/-/pipelines/75337
Encoder 0dB
MLD Regressions > 5
| Status | Testcase | MLD-curr | MLD-prev | MLD-diff | enc_cmd | dec_dmc |
|---|---|---|---|---|---|---|
| test_param_file_tests[ltv_basop_encoder-SBA FOA bitrate switching from 13.2 kbps to 192 kbps, 32kHz in, 32kHz out, DTX on, FOA out] | 15.178274 | 9.50429 | 5.673984000000001 | IVAS_cod_dut -dtx -sba 1 scripts/switchPaths/sw_13k2_192k_50fr.bin 32 scripts/testv/ltv32_FOA.wav ltv32_FOA.wav_SBA_FOA_bitrate_switching_from_13_2_kbps_to_192_kbps_32kHz_in_32kHz_out_DTX_on_FOA_out.192 | IVAS_dec_merge_source_float_ref -fr 20 FOA 32 ltv32_FOA.wav_SBA_FOA_bitrate_switching_from_13_2_kbps_to_192_kbps_32kHz_in_32kHz_out_DTX_on_FOA_out.192 ltv32_FOA.wav_SBA_FOA_bitrate_switching_from_13_2_kbps_to_192_kbps_32kHz_in_32kHz_out_DTX_on_FOA_out.dec.wav | |
| test_param_file_tests[ltv_basop_encoder-Multi-channel 5_1_2 bitrate switching from 24.4 kbps to 256 kbps, 48kHz in, 48kHz out, 5_1_2 out] | 14.584008 | 7.828079 | 6.755929000000001 | IVAS_cod_dut -mc 5_1_2 scripts/switchPaths/sw_24k4_256k.bin 48 scripts/testv/ltv48_MC512.wav ltv48_MC512.wav_Multi_channel_5_1_2_bitrate_switching_from_24_4_kbps_to_256_kbps_48kHz_in_48kHz_out_5_1_2_out.192 | IVAS_dec_merge_source_float_ref -fr 20 5_1_2 48 ltv48_MC512.wav_Multi_channel_5_1_2_bitrate_switching_from_24_4_kbps_to_256_kbps_48kHz_in_48kHz_out_5_1_2_out.192 ltv48_MC512.wav_Multi_channel_5_1_2_bitrate_switching_from_24_4_kbps_to_256_kbps_48kHz_in_48kHz_out_5_1_2_out.dec.wav | |
| test_param_file_tests[ltv_basop_encoder-SBA planar 3OA bitrate switching from 13.2 kbps to 512 kbps, 48kHz in, 48kHz out, HOA3 out] | 17.95856 | 8.235016 | 9.723543999999999 | IVAS_cod_dut -sba -3 scripts/switchPaths/sw_13k2_512k.bin 48 scripts/testv/ltv48_HOA3.wav ltv48_HOA3.wav_SBA_planar_3OA_bitrate_switching_from_13_2_kbps_to_512_kbps_48kHz_in_48kHz_out_HOA3_out.192 | IVAS_dec_merge_source_float_ref -fr 20 HOA3 48 ltv48_HOA3.wav_SBA_planar_3OA_bitrate_switching_from_13_2_kbps_to_512_kbps_48kHz_in_48kHz_out_HOA3_out.192 ltv48_HOA3.wav_SBA_planar_3OA_bitrate_switching_from_13_2_kbps_to_512_kbps_48kHz_in_48kHz_out_HOA3_out.dec.wav | |
| See #2518 (closed) | test_param_file_tests[ltv_basop_encoder-SBA 3OA bitrate switching from 13.2 kbps to 512 kbps, 48kHz in, 48kHz out, HOA3 out] | 19.732393 | 8.217885 | 11.514507999999998 | IVAS_cod_dut -sba 3 scripts/switchPaths/sw_13k2_512k.bin 48 scripts/testv/ltv48_HOA3.wav ltv48_HOA3.wav_SBA_3OA_bitrate_switching_from_13_2_kbps_to_512_kbps_48kHz_in_48kHz_out_HOA3_out.192 | IVAS_dec_merge_source_float_ref -fr 20 HOA3 48 ltv48_HOA3.wav_SBA_3OA_bitrate_switching_from_13_2_kbps_to_512_kbps_48kHz_in_48kHz_out_HOA3_out.192 ltv48_HOA3.wav_SBA_3OA_bitrate_switching_from_13_2_kbps_to_512_kbps_48kHz_in_48kHz_out_HOA3_out.dec.wav |
| See #2540 (closed) | test_param_file_tests[ltv_basop_encoder-OSBA 3OA 4ISM bitrate switching 13.2 to 512, 48kHz in, 48kHz out, EXT out] | 24.54567 | 17.128004 | 7.4176660000000005 | IVAS_cod_dut -ism_sba 4 3 scripts/testv/ltv48_OSBA_1ISM_HOA3_ISM1.csv scripts/testv/ltv48_OSBA_2ISM_HOA3_ISM2.csv scripts/testv/ltv48_OSBA_3ISM_HOA3_ISM3.csv scripts/testv/ltv48_OSBA_4ISM_HOA3_ISM4.csv scripts/switchPaths/sw_13k2_512k.bin 48 scripts/testv/ltv48_OSBA_4ISM_HOA3.wav ltv48_OSBA_4ISM_HOA3.wav_OSBA_3OA_4ISM_bitrate_switching_13_2_to_512_48kHz_in_48kHz_out_EXT_out.192 | IVAS_dec_merge_source_float_ref -fr 20 EXT 48 ltv48_OSBA_4ISM_HOA3.wav_OSBA_3OA_4ISM_bitrate_switching_13_2_to_512_48kHz_in_48kHz_out_EXT_out.192 ltv48_OSBA_4ISM_HOA3.wav_OSBA_3OA_4ISM_bitrate_switching_13_2_to_512_48kHz_in_48kHz_out_EXT_out.dec.wav |
| test_param_file_tests[ltv_basop_encoder-SBA 2OA bitrate switching from 13.2 kbps to 512 kbps, 32kHz in, 32kHz out, HOA2 out] | 13.804638 | 8.109187 | 5.695451 | IVAS_cod_dut -sba 2 scripts/switchPaths/sw_13k2_512k.bin 32 scripts/testv/ltv32_HOA2.wav ltv32_HOA2.wav_SBA_2OA_bitrate_switching_from_13_2_kbps_to_512_kbps_32kHz_in_32kHz_out_HOA2_out.192 | IVAS_dec_merge_source_float_ref -fr 20 HOA2 32 ltv32_HOA2.wav_SBA_2OA_bitrate_switching_from_13_2_kbps_to_512_kbps_32kHz_in_32kHz_out_HOA2_out.192 ltv32_HOA2.wav_SBA_2OA_bitrate_switching_from_13_2_kbps_to_512_kbps_32kHz_in_32kHz_out_HOA2_out.dec.wav | |
| test_param_file_tests[ltv_basop_encoder-OMASA 2TC 4ISM at br sw techs 13.2 to 512 kbps start 80 kbps, 48kHz in, 48kHz out, EXT out] | 19.504745 | 11.289236 | 8.215508999999999 | IVAS_cod_dut -ism_masa 4 2 NULL scripts/testv/ltv48_OMASA_2ISM_2TC_ISM2.csv scripts/testv/ltv48_OMASA_3ISM_2TC_ISM3.csv scripts/testv/ltv48_OMASA_4ISM_2TC_ISM4.csv scripts/testv/ltv48_OMASA_4ISM_2TC.met scripts/switchPaths/sw_13k2_512k_2fr_start_80k_omasatechs_4ism.bin 48 scripts/testv/ltv48_OMASA_4ISM_2TC.wav ltv48_OMASA_4ISM_2TC.wav_OMASA_2TC_4ISM_at_br_sw_techs_13_2_to_512_kbps_start_80_kbps_48kHz_in_48kHz_out_EXT_out.192 | IVAS_dec_merge_source_float_ref -fr 20 EXT 48 ltv48_OMASA_4ISM_2TC.wav_OMASA_2TC_4ISM_at_br_sw_techs_13_2_to_512_kbps_start_80_kbps_48kHz_in_48kHz_out_EXT_out.192 ltv48_OMASA_4ISM_2TC.wav_OMASA_2TC_4ISM_at_br_sw_techs_13_2_to_512_kbps_start_80_kbps_48kHz_in_48kHz_out_EXT_out.dec.wav |
Encoder +10 dB
MLD Regressions > 5
| Status | Testcase | MLD-curr | MLD-prev | MLD-diff | enc_cmd | dec_cmd |
|---|---|---|---|---|---|---|
| test_param_file_tests[ltv_basop_encoder-OMASA 2TC 3ISM at 32 kbps, 48kHz in, 48kHz out, EXT out] | 14.242584 | 8.422516 | 5.820068000000001 | IVAS_cod_dut -ism_masa 3 2 NULL NULL NULL scripts/testv/ltv48_OMASA_3ISM_2TC.met 32000 48 scripts/testv/ltv48_OMASA_3ISM_2TC.wav ltv48_OMASA_3ISM_2TC.wav_OMASA_2TC_3ISM_at_32_kbps_48kHz_in_48kHz_out_EXT_out.192 | IVAS_dec_merge_source_float_ref -fr 20 EXT 48 ltv48_OMASA_3ISM_2TC.wav_OMASA_2TC_3ISM_at_32_kbps_48kHz_in_48kHz_out_EXT_out.192 ltv48_OMASA_3ISM_2TC.wav_OMASA_2TC_3ISM_at_32_kbps_48kHz_in_48kHz_out_EXT_out.dec.wav | |
| See #2517 | test_param_file_tests[ltv_basop_encoder-OMASA 2TC 3ISM at 32 kbps, 48kHz in, 16kHz out, EXT out] | 30.626423 | 9.926726 | 20.699697 | IVAS_cod_dut -ism_masa 3 2 scripts/testv/ltv48_OMASA_1ISM_2TC_ISM1.csv scripts/testv/ltv48_OMASA_2ISM_2TC_ISM2.csv scripts/testv/ltv48_OMASA_3ISM_2TC_ISM3.csv scripts/testv/ltv48_OMASA_3ISM_2TC.met 32000 48 scripts/testv/ltv48_OMASA_3ISM_2TC.wav ltv48_OMASA_3ISM_2TC.wav_OMASA_2TC_3ISM_at_32_kbps_48kHz_in_16kHz_out_EXT_out.192 | IVAS_dec_merge_source_float_ref -fr 20 EXT 16 ltv48_OMASA_3ISM_2TC.wav_OMASA_2TC_3ISM_at_32_kbps_48kHz_in_16kHz_out_EXT_out.192 ltv48_OMASA_3ISM_2TC.wav_OMASA_2TC_3ISM_at_32_kbps_48kHz_in_16kHz_out_EXT_out.dec.wav |
| test_param_file_tests[ltv_basop_encoder-OMASA 1TC 4ISM at br sw techs 13.2 to 512 kbps start 32 kbps, 48kHz in, 48kHz out, EXT out] | 16.890337 | 10.430279 | 6.460057999999998 | IVAS_cod_dut -ism_masa 4 1 scripts/testv/ltv48_OMASA_1ISM_1TC_ISM1.csv scripts/testv/ltv48_OMASA_2ISM_1TC_ISM2.csv scripts/testv/ltv48_OMASA_3ISM_1TC_ISM3.csv scripts/testv/ltv48_OMASA_4ISM_1TC_ISM4.csv scripts/testv/ltv48_OMASA_4ISM_1TC.met scripts/switchPaths/sw_13k2_512k_2fr_start_32k_omasatechs_4ism.bin 48 scripts/testv/ltv48_OMASA_4ISM_1TC.wav ltv48_OMASA_4ISM_1TC.wav_OMASA_1TC_4ISM_at_br_sw_techs_13_2_to_512_kbps_start_32_kbps_48kHz_in_48kHz_out_EXT_out.192 | IVAS_dec_merge_source_float_ref -fr 20 EXT 48 ltv48_OMASA_4ISM_1TC.wav_OMASA_1TC_4ISM_at_br_sw_techs_13_2_to_512_kbps_start_32_kbps_48kHz_in_48kHz_out_EXT_out.192 ltv48_OMASA_4ISM_1TC.wav_OMASA_1TC_4ISM_at_br_sw_techs_13_2_to_512_kbps_start_32_kbps_48kHz_in_48kHz_out_EXT_out.dec.wav | |
| test_param_file_tests[ltv_basop_encoder-OSBA 3ISM 2OA at bitrate switching 13.2 to 512 kbps, 48kHz in, 32kHz out, EXT out] | 14.576198 | 9.573644 | 5.002554 | IVAS_cod_dut -ism_sba 3 2 scripts/testv/ltv48_OSBA_1ISM_HOA2_ISM1.csv scripts/testv/ltv48_OSBA_2ISM_HOA2_ISM2.csv scripts/testv/ltv48_OSBA_3ISM_HOA2_ISM3.csv scripts/switchPaths/sw_13k2_512k.bin 48 scripts/testv/ltv48_OSBA_3ISM_HOA2.wav ltv48_OSBA_3ISM_HOA2.wav_OSBA_3ISM_2OA_at_bitrate_switching_13_2_to_512_kbps_48kHz_in_32kHz_out_EXT_out.192 | IVAS_dec_merge_source_float_ref -fr 20 EXT 32 ltv48_OSBA_3ISM_HOA2.wav_OSBA_3ISM_2OA_at_bitrate_switching_13_2_to_512_kbps_48kHz_in_32kHz_out_EXT_out.192 ltv48_OSBA_3ISM_HOA2.wav_OSBA_3ISM_2OA_at_bitrate_switching_13_2_to_512_kbps_48kHz_in_32kHz_out_EXT_out.dec.wav |
Encoder -10 dB
MLD Regressions > 5
| Status | Testcase | MLD-curr | MLD-prev | MLD-diff | enc_cmd | dec_cmd |
|---|---|---|---|---|---|---|
| test_param_file_tests[ltv_basop_encoder-OMASA 2TC 4ISM at br sw techs 13.2 to 512 kbps start 80 kbps, 48kHz in, 48kHz out, EXT out] | 20.441902 | 10.522297 | 9.919604999999999 | IVAS_cod_dut -ism_masa 4 2 NULL scripts/testv/ltv48_OMASA_2ISM_2TC_ISM2.csv scripts/testv/ltv48_OMASA_3ISM_2TC_ISM3.csv scripts/testv/ltv48_OMASA_4ISM_2TC_ISM4.csv scripts/testv/ltv48_OMASA_4ISM_2TC.met scripts/switchPaths/sw_13k2_512k_2fr_start_80k_omasatechs_4ism.bin 48 scripts/testv/ltv48_OMASA_4ISM_2TC.wav ltv48_OMASA_4ISM_2TC.wav_OMASA_2TC_4ISM_at_br_sw_techs_13_2_to_512_kbps_start_80_kbps_48kHz_in_48kHz_out_EXT_out.192 | IVAS_dec_merge_source_float_ref -fr 20 EXT 48 ltv48_OMASA_4ISM_2TC.wav_OMASA_2TC_4ISM_at_br_sw_techs_13_2_to_512_kbps_start_80_kbps_48kHz_in_48kHz_out_EXT_out.192 ltv48_OMASA_4ISM_2TC.wav_OMASA_2TC_4ISM_at_br_sw_techs_13_2_to_512_kbps_start_80_kbps_48kHz_in_48kHz_out_EXT_out.dec.wav | |
| test_param_file_tests[ltv_basop_encoder-SBA at 13.2 kbps, 48kHz in, 48kHz out, FOA out] | 23.569818 | 13.771723 | 9.798095000000002 | IVAS_cod_dut -sba 1 13200 48 scripts/testv/ltv48_FOA.wav ltv48_FOA.wav_SBA_at_13_2_kbps_48kHz_in_48kHz_out_FOA_out.192 | IVAS_dec_merge_source_float_ref -fr 20 FOA 48 ltv48_FOA.wav_SBA_at_13_2_kbps_48kHz_in_48kHz_out_FOA_out.192 ltv48_FOA.wav_SBA_at_13_2_kbps_48kHz_in_48kHz_out_FOA_out.dec.wav | |
| test_param_file_tests[ltv_basop_encoder-stereo at 32 kbps, 32kHz in, 32kHz out, DTX on, STEREO out] | 12.386349 | 7.242153 | 5.144195999999999 | IVAS_cod_dut -stereo -dtx 32000 32 scripts/testv/ltv32_STEREO.wav ltv32_STEREO.wav_stereo_at_32_kbps_32kHz_in_32kHz_out_DTX_on_STEREO_out.192 | IVAS_dec_merge_source_float_ref -fr 20 STEREO 32 ltv32_STEREO.wav_stereo_at_32_kbps_32kHz_in_32kHz_out_DTX_on_STEREO_out.192 ltv32_STEREO.wav_stereo_at_32_kbps_32kHz_in_32kHz_out_DTX_on_STEREO_out.dec.wav | |
| test_param_file_tests[ltv_basop_encoder-Multi-channel 7_1_4 bitrate switching from 13.2 kbps to 512 kbps, 48kHz in, 48kHz out, 7_1_4 out] | 22.67244 | 14.700307 | 7.972133000000001 | IVAS_cod_dut -mc 7_1_4 scripts/switchPaths/sw_mctech_5fr.bin 48 scripts/testv/ltv48_MC714.wav ltv48_MC714.wav_Multi_channel_7_1_4_bitrate_switching_from_13_2_kbps_to_512_kbps_48kHz_in_48kHz_out_7_1_4_out.192 | IVAS_dec_merge_source_float_ref -fr 20 7_1_4 48 ltv48_MC714.wav_Multi_channel_7_1_4_bitrate_switching_from_13_2_kbps_to_512_kbps_48kHz_in_48kHz_out_7_1_4_out.192 ltv48_MC714.wav_Multi_channel_7_1_4_bitrate_switching_from_13_2_kbps_to_512_kbps_48kHz_in_48kHz_out_7_1_4_out.dec.wav | |
| See #2519 | test_param_file_tests[ltv_basop_encoder-MASA 2TC at 32 kbps, 48kHz in, 48kHz out, EXT out] | 22.359663 | 10.962725 | 11.396938 | IVAS_cod_dut -masa 2 scripts/testv/ltv48_MASA2TC.met 32000 48 scripts/testv/ltv48_MASA2TC.wav ltv48_MASA2TC.wav_MASA_2TC_at_32_kbps_48kHz_in_48kHz_out_EXT_out.192 | IVAS_dec_merge_source_float_ref -fr 20 EXT 48 ltv48_MASA2TC.wav_MASA_2TC_at_32_kbps_48kHz_in_48kHz_out_EXT_out.192 ltv48_MASA2TC.wav_MASA_2TC_at_32_kbps_48kHz_in_48kHz_out_EXT_out.dec.wav |
| test_param_file_tests[ltv_basop_encoder-OMASA 1TC 3ISM at br sw techs 13.2 to 512 kbps start 48 kbps, 48kHz in, 32kHz out, EXT out] | 22.245468 | 15.531543 | 6.713925 | IVAS_cod_dut -ism_masa 3 1 scripts/testv/ltv48_OMASA_1ISM_1TC_ISM1.csv scripts/testv/ltv48_OMASA_2ISM_1TC_ISM2.csv scripts/testv/ltv48_OMASA_3ISM_1TC_ISM3.csv scripts/testv/ltv48_OMASA_3ISM_1TC.met scripts/switchPaths/sw_13k2_512k_2fr_start_48k_omasatechs_3ism.bin 48 scripts/testv/ltv48_OMASA_3ISM_1TC.wav ltv48_OMASA_3ISM_1TC.wav_OMASA_1TC_3ISM_at_br_sw_techs_13_2_to_512_kbps_start_48_kbps_48kHz_in_32kHz_out_EXT_out.192 | IVAS_dec_merge_source_float_ref -fr 20 EXT 32 ltv48_OMASA_3ISM_1TC.wav_OMASA_1TC_3ISM_at_br_sw_techs_13_2_to_512_kbps_start_48_kbps_48kHz_in_32kHz_out_EXT_out.192 ltv48_OMASA_3ISM_1TC.wav_OMASA_1TC_3ISM_at_br_sw_techs_13_2_to_512_kbps_start_48_kbps_48kHz_in_32kHz_out_EXT_out.dec.wav |
Attached are also the merged CSV-files for the LTV comparison:
Edited by Manuel Jander