[CI] Rendering test updates
- Related issues: #247 (closed) / !1189 (merged) is currently blocked or would need a workaround without this MR
- Requested reviewers: @kiene, @vaclav, @pihlajakuja, @kinuthia, @millsa + other interested CI TF participants
Reason why this change is needed
- The renderer pytest is currently not very flexible when a change in commandline arguments is attempted. Additionally the
ref-using-main
tag support was rather limited. - Also see comment: !1189 (comment 30801)
Description of the change
- The renderer tests are now more similar in behaviour to the
ivas-pytest
with REF and CUT creation as two different steps -
test_renderer_be_comparison.py
is removed, less duplication and no need to update tests in two places anymore. -
ref-using-main
behaviour should be in line with theivas-pytest
- OTR tests were moved to a separate location in the file - these tests were only relevant for smoke testing and the outputs were actually never compared between commits in the BE comparison. Thus it makes sense to only run them for smoke testing, and any BE change should be detected by the other binaural rendering tests.
Affected operating points
- BE, only affects how the CI tests are run
- Similar updates for the split rendering tests will follow shortly on this MR (could be split into a second one if desired)
TODOs before MR is ready
-
Verify that all tests are preserved (need to restore testcases only present in the _be_comparison.py
variant) -
Verify ref-using-main
behaves as intended -
Perform identical updates for the split rendering testsuite
Edited by Archit Tamarapu