Commit 3dd1c714 authored by Vladimir Malenovsky's avatar Vladimir Malenovsky
Browse files

clarifications

parent 1c495962
Loading
Loading
Loading
Loading
Loading
+7 −10
Original line number Diff line number Diff line
@@ -243,9 +243,7 @@ Use `--filter TOKEN` to control conformance level, test groups, output formats,

### LEVEL1, LEVEL2 and LEVEL3 behavior

With no `--filter`, all test groups are selected: `ENC`, `DEC`, `REND`, `ISAR_ENC`, and `ISAR`.

When `--filter LEVEL1` is used, the default test-group baseline is `ENC` + `DEC`.
For all levels, the default test-group baseline is `ENC` + `DEC`.
`REND`, `ISAR`, and `ISAR_ENC` are optional and are only included if explicitly selected as plain test-group tokens or added via `+REND`, `+ISAR` and `+ISAR_ENC`.

When `--filter LEVEL1` is specified, the following default tests are run:
@@ -267,8 +265,8 @@ When `--filter LEVEL1` is specified, the following default tests are run:
  - Add `+REND`, `+ISAR`, and/or `+ISAR_ENC` in `--filter` to include them.
  - Use `+ISAR*` if you want wildcard expansion across all `ISAR*`-prefixed test-group tokens.

When `--filter LEVEL2` is specified, there are no default tag restrictions,
and the bitrate cap is set to 192 kbps for level-limited checks:
When `--filter LEVEL2` is specified, all selection rules above remain the same,
except the bitrate cap is set to 192 kbps:

- Encoder (`ENC`) tests: only tests with bitrate up to 192 kbps (inclusive).
- Decoder (`DEC`) tests:
@@ -276,21 +274,20 @@ and the bitrate cap is set to 192 kbps for level-limited checks:
  - `MONO` output format: all bitrates.
  - `STEREO` output format: all bitrates.

When `--filter LEVEL3` is specified, there are no restrictions on the bitrate or output formats,
and no default tag restrictions.
When `--filter LEVEL3` is specified, there are no restrictions on the bitrate or output formats.

Examples (non-BE):

- Default behavior (no filter): run all test groups
- Default behavior (same as LEVEL3 baseline): run only ENC and DEC test groups

  ```shell
  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse
  ```

- LEVEL3 explicit (equivalent tag scope to default; no bitrate/output restrictions)
- LEVEL3 plus renderer and ISAR test groups

  ```shell
  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter LEVEL3
  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter LEVEL3 +REND +ISAR +ISAR_ENC
  ```

- LEVEL1 baseline (ENC+DEC with LEVEL1 restrictions)