Create manual test for codec BE check
As discussed in CI task force meeting, we will together develop a test for verifying BE state with further merge requests to main against the latest accepted release. This will be implemented as a manual test pipeline which can be invoked for any branch and will run against the release binaries on Windows runners. At least following tasks are required for this:
-
Make necessary CI changes to allow running a manual pipeline for the purpose (@pihlajakuja looking at this) -
Create CI job for running the task -> (@kinuthia, @demirkanik to start) -
Decide and obtain the test sample test for the purpose. Preferably put them in a branch in the test files repository. See here for ltv testvector mode coverage assessment. -
Map and decide all the modes that should be covered -
Create the test system (probably in pytest) -> @kiene to start -
Create bitrate and bandwidth switching files (ideally covering all possibilities for all subsets of bitrates) -
Create error patterns for PLC tests. Basic one with high frameloss rate + special situations (which can maybe included into the general pattern)
-
Edited by Jan Kiene