Skip to content

Metadata file comparison for EXT output not working in self test comparison

Basic info

  • Commit SHA: main
  • Platform: all

Bug description

The current test system for comparing metadata files with EXT output depends on reading specific output lines from the decoder printout, which is starting with Output metadata file. The problem is that for OMASA, there are actually two variants Output MASA metadata file and Output ISM metadata file. Moreover, OSBA was not printing anything (when output worked). Once we fix metadata output for combined formats, we should also change this to catch the different variants of output.

Another problem is that the current check for metadata files is done based on the tested decoder and not the reference decoder. This basically means that if the new code completely removes the metadata files (which has now happened in main), the test will pass successfully as there was no files to compare. On the other side, if metadata files would be added, then the test would always fail as there was no comparison point but I guess that would be correct performance.

Checking for different outputs is relatively easy by adding a list of cases. However, it is prone to errors if changes are introduced to the naming. More important would be to check if there is a difference in number of metadata files.