Possible failure of the automatic memory re-allocation mechanism when ind_list[] buffer is depleted in MASA mode with 2 TC
On behalf of @paulusj:
The automatic memory re-allocation mechanism doesn't seem to work in the MASA mode with with 2 TC in case the maximum memory of the ind_list[] buffer is manually decreased. The following hypothetical case illustrates the issue.
In bitstream.c, adjust the initial index buffer size for 160kbps smaller: line 389 => return 500; /* was 850; */ -- Note: Using an initial buffer size of 600 (= 500 + STEP_MAX_NUM_INDICES), which should be the same as after the reallocation, no index exhaustion or crash is observed.
- Encoding with an item from ivas-pc-testfiles/samples/MASA_2TC/: IVAS_cod -masa 2 nok_three-males-arboretum2_MASA2TC_2Dir.met 160000 48 nok_three-males-arboretum2_MASA2TC.pcm masa.bs
This should display (when complied with clang sanitizers ASAN or MSAN):
MASA inp. metadata file: nok_three-males-arboretum2_MASA2TC_2Dir.met
Input audio file: nok_three-males-arboretum2_MASA2TC.pcm
Output bitstream file: masa.bs
Max. encoded bandwidth: FB
Input sampling rate: 48000 Hz
Bitrate: 160.00 kbps
IVAS format: MASA format
------ Running the encoder ------
Frames processed: 127 Warning: The maximum number of indices 500 has been exceeded in frame 170! Increase the limits in get_ivas_max_num_indices() or get_max_num_indices_metadata().
The address sanitizer reports the following:
bitstream.c
Heap-use-after-free on address 0x61f000000082 at pc 0x000000a94c9a bp 0x7ffdbdc19f30 sp 0x7ffdbdc19f28
READ of size 2 at 0x61f000000082 thread T0
0xa94c99 write_indices_to_stream bitstream.c:1494
0xa89eeb write_indices_element bitstream.c:1656
0xa88e30 write_indices_ivas bitstream.c:1787
0x509859 IVAS_ENC_EncodeFrameToSerial lib_enc.c:1232
0x4f6519 main encoder.c:745
0x7f2e2296e082 __libc_start_main libc-start.c:308
0x44257d _start
Freed by thread T0 here:
0x4beac2 free
0xa82363 ind_list_realloc bitstream.c:207
0xa83cb5 check_ind_list_limits bitstream.c:869
0xa86649 push_next_bits bitstream.c:1133
0x7ca93d RCcontextMapping_encode2_no_mem_s17_LCS ACcontextMapping_enc.c:1017
0x867628 writeTCXparam enc_prm.c:559
0x611ca1 ivas_mdct_quant_coder ivas_mdct_core_enc.c:1353
0x9a8517 stereo_mdct_core_enc ivas_stereo_mdct_core_enc.c:465
0x8d17a9 ivas_core_enc ivas_core_enc.c:282
0x8fa67c ivas_cpe_enc ivas_cpe_enc.c:620
0x5816ca ivas_enc ivas_enc.c:256
0x509825 IVAS_ENC_EncodeFrameToSerial lib_enc.c:1225
0x4f6519 main encoder.c:745
0x7f2e2296e082 __libc_start_main libc-start.c:308
Previously allocated by thread T0 here:
0x4bed2d __interceptor_malloc
0x5855a1 ivas_init_encoder ivas_init_enc.c:370
0x502c31 configureEncoder lib_enc.c:902
0x505e80 IVAS_ENC_ConfigureForMasa lib_enc.c:546
0x4f386c main encoder.c:417
0x7f2e2296e082 __libc_start_main libc-start.c:308
And the memory sanitizer finds the same issue:
bitstream.c
Use-of-uninitialized-value
0xda54c5 write_indices_to_stream bitstream.c:1497
0xd92fee write_indices_element bitstream.c:1656
0xd913eb write_indices_ivas bitstream.c:1787
0x4e3fcb IVAS_ENC_EncodeFrameToSerial lib_enc.c:1232
0x4c74f1 main encoder.c:745
0x7fc39a782082 __libc_start_main libc-start.c:308
0x44251d _startEdited by Ghost User