add .exe to Win executables

To run CUT binaries on the targeted platform, it is necessary to have the correct setup for python and dependency packages.

- Create virtual environment for Python 3.13 and install requirements

- Create virtual environment for Python 3.13 and install requirements from the root folder of the IVAS codebase

  ```shell

  python3.13 -m venv pyConformance

  source pyConformance/bin/activate

  cd ivas-codec

  python -m pip install -r scripts/ivas_conformance/requirements.txt

  ```

## Executing specific tests only

Use `--filter TOKEN` to control conformance level, test groups, output formats, and substring matching.

Use `--filter TOKEN` to select test groups, output formats, and apply substring matching.

- Token types:

  - `LEVEL1`, `LEVEL2`, `LEVEL3`: conformance levels. `LEVEL3` is the default.

  - `ENC`, `DEC`, `REND`, `ISAR`, `ISAR_ENC`: test groups.

  - `MONO`, `STEREO`, `EXT`, `HOA`, `SBA`, `MC`: output-format tokens.

    - `HOA` expands to `HOA2`, `HOA3`.

  - `-TOKEN`: subtractive token. Removes matching tests from the final selection.

  - `TOKEN*`: wildcard token. Matches all known tokens starting with the given prefix.

  - `+TOKEN*`, `-TOKEN*`: wildcard token with add/remove behavior.

### LEVEL1, LEVEL2 and LEVEL3 behavior

For all levels, the default test-group baseline is `ENC` + `DEC`.

`REND`, `ISAR`, and `ISAR_ENC` are optional and are only included if explicitly selected as plain test-group tokens or added via `+REND`, `+ISAR` and `+ISAR_ENC`.

When `--filter LEVEL1` is specified, the following default tests are run:

- Encoder (`ENC`) tests: only tests with bitrate up to 80 kbps (inclusive).

- Decoder (`DEC`) tests:

  - `EXT` output format: only bitrate up to 80 kbps (inclusive).

  - `MONO` output format: all bitrates.

  - `STEREO` output format: all bitrates.

- The default `LEVEL1` tests may be restricted by adding more tokens (acting as logical AND).

  - Example: `--filter LEVEL1 DEC MONO` keeps only `MONO` tests from the LEVEL1-eligible DEC set.

  - Example: `--filter LEVEL1 JBM` keeps all LEVEL1-eligible ENC tests but only JBM tests from the LEVEL1-eligible DEC tests.

- `+TOKEN` adds tests to the final LEVEL1 selection, even if they would otherwise be restricted.

  - Example: `--filter LEVEL1 DEC JBM +BINAURAL*` runs JBM-matching LEVEL1 DEC tests and additionally includes DEC tests with `BINAURAL` output formats, i.e. `BINAURAL`, `BINAURAL_IR`, `BINAURAL_ROOM_IR`, `BINAURAL_REVERB`.

- `-TOKEN` removes matching tests from the final LEVEL1 selection (including tests added via `+TOKEN`).

  - Example: `--filter LEVEL1 DEC +JBM -VOIP` adds JBM-matching tests to LEVEL1 DEC tests and excludes DEC `VOIP` tests.

- Renderer and ISAR tests are not run by default in `LEVEL1`.

  - Add `+REND`, `+ISAR`, and/or `+ISAR_ENC` in `--filter` to include them.

  - Use `+ISAR*` if you want wildcard expansion across all `ISAR*`-prefixed test-group tokens.

When `--filter LEVEL2` is specified, all selection rules above remain the same,

except the bitrate cap is set to 192 kbps:

- Encoder (`ENC`) tests: only tests with bitrate up to 192 kbps (inclusive).

- Decoder (`DEC`) tests:

  - `EXT` output format: only bitrate up to 192 kbps (inclusive).

  - `MONO` output format: all bitrates.

  - `STEREO` output format: all bitrates.

When `--filter LEVEL3` is specified, there are no restrictions on the bitrate or output formats.

  - **Note**: wildcards must be quoted in shell (e.g. `'ISAR*'`) to prevent the shell from expanding them as filename globs before the script receives them.

Examples (non-BE):

- Default behavior (same as LEVEL3 baseline): run only ENC and DEC test groups

- Default behavior: run all test groups (ENC, DEC, REND, ISAR, ISAR_ENC)

  ```shell

  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse

  ```

- LEVEL3 plus renderer and ISAR test groups

  ```shell

  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter LEVEL3 +REND +ISAR +ISAR_ENC

  ```

- LEVEL1 baseline (ENC+DEC with LEVEL1 restrictions)

  ```shell

  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter LEVEL1

  ```

- LEVEL1 plus renderer and ISAR test groups

  ```shell

  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter LEVEL1 +REND +ISAR +ISAR_ENC

  ```

- LEVEL1 with additional case-insensitive command substring filtering

- Run only REND, ISAR, and ISAR_ENC test groups

  ```shell

  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter LEVEL1 DEC voip

  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter REND ISAR ISAR_ENC

  ```

- LEVEL1 with additive BINAURAL decoder matching

- Run DEC tests with HOA output formats

  ```shell

  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter LEVEL1 +BINAURAL

  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter DEC HOA

  ```

- LEVEL1 with restrictive and additive terms together

- Run DEC tests excluding those matching 'voip'

  ```shell

  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter LEVEL1 DEC JBM +BINAURAL

  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter DEC -voip

  ```

- LEVEL2 baseline (ENC+DEC with LEVEL2 restrictions)

- Run DEC tests with any BINAURAL output format

  ```shell

  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter LEVEL2

  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter DEC 'BINAURAL*'

  ```

- LEVEL2 plus renderer and ISAR test groups

- Run ISAR and ISAR_ENC groups, excluding 'voip' tests

  ```shell

  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter LEVEL2 +REND +ISAR +ISAR_ENC

  PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --ref_build_path=testvec/bin --analyse --filter 'ISAR*' -voip

  ```

    "ISAR": "ISAR_post_rend",

}

is_windows = platform.system() == "Windows"

# Automatically append .exe on Windows

if platform.system() == "Windows":

if is_windows:

    IVAS_Bins = {k: f"{v}.exe" for k, v in IVAS_Bins.items()}

DECODER_OUTPUT_FORMATS = {

    missing = []

    for tag, binary in IVAS_Bins.items():

        candidate = os.path.join(abs_build_path, binary)

        candidate_exe = f"{candidate}.exe"

        exists = os.path.isfile(candidate) or (

            is_windows and os.path.isfile(candidate_exe)

        )

        if not exists:

            shown = candidate_exe if is_windows else candidate

            missing.append(f"{tag}: {shown}")

        if not os.path.isfile(candidate):

            missing.append(f"{tag}: {candidate}")

    if missing:

        parser.error(f"Missing {build_label} binaries:\n  - " + "\n  - ".join(missing))

        text = rawCmdline.lower()

        return any(term.lower() in text for term in terms)

    def _matchesLevel1(self, tag: str, rawCmdline: str) -> bool:

        if tag == "ENC":

            return self._isBitrateAtMost80(rawCmdline)

        if tag == "DEC":

            formats = self._outputFormatsInCommand(rawCmdline)

            requested_formats = set(getattr(self.args, "filter_decoder_formats", []))

            ext_ok = "EXT" in formats and self._isBitrateAtMost80(rawCmdline)

            mono_ok = "MONO" in formats

            stereo_ok = "STEREO" in formats

            default_level1_dec_ok = ext_ok or mono_ok or stereo_ok

            if requested_formats:

                # Plain decoder format tokens are restrictive under LEVEL1.

                return default_level1_dec_ok and bool(formats.intersection(requested_formats))

            return default_level1_dec_ok

        # For REND/ISAR/ISAR_ENC under LEVEL1, tag-level inclusion is decided at testTags parsing.

        return True

    def _matchesLevel2(self, tag: str, rawCmdline: str) -> bool:

        if tag == "ENC":

            return self._isBitrateAtMost192(rawCmdline)

        if tag == "DEC":

            formats = self._outputFormatsInCommand(rawCmdline)

            requested_formats = set(getattr(self.args, "filter_decoder_formats", []))

            ext_ok = "EXT" in formats and self._isBitrateAtMost192(rawCmdline)

            mono_ok = "MONO" in formats

            stereo_ok = "STEREO" in formats

            default_level2_dec_ok = ext_ok or mono_ok or stereo_ok

            if requested_formats:

                # Plain decoder format tokens are restrictive under LEVEL2.

                return default_level2_dec_ok and bool(formats.intersection(requested_formats))

            return default_level2_dec_ok

        # For REND/ISAR/ISAR_ENC under LEVEL2, tag-level inclusion is decided at testTags parsing.

        return True

    def _testPassesFilter(self, tag: str, rawCmdline: str) -> bool:

        level = getattr(self.args, "filter_level", "LEVEL3")

        restrictive_terms = getattr(self.args, "filter_restrictive_terms", [])

        additive_terms = getattr(self.args, "filter_add_terms", [])

        subtractive_terms = getattr(self.args, "filter_remove_terms", [])

        if subtractive_terms and self._matchesAnyTerm(rawCmdline, subtractive_terms):

            return False

        passes_level = True

        if level == "LEVEL1":

            passes_level = self._matchesLevel1(tag, rawCmdline)

        elif level == "LEVEL2":

            passes_level = self._matchesLevel2(tag, rawCmdline)

        passes_requested_formats = True

        if requested_formats:

            cmd_formats = self._outputFormatsInCommand(rawCmdline)

            passes_requested_formats = bool(cmd_formats.intersection(requested_formats))

        passes_restrictive_terms = self._matchesAllTerms(rawCmdline, restrictive_terms)

        base_selected = passes_level and passes_restrictive_terms and passes_requested_formats

        base_selected = passes_restrictive_terms and passes_requested_formats

        if base_selected:

            return True

            "Select which tests to run. Default baseline: ENC+DEC tests (REND/ISAR optional).\n"

            "\n"

            "Token types:\n"

            "  LEVEL1, LEVEL2, LEVEL3              — Conformance level. LEVEL1: ≤80 kbps; LEVEL2: ≤192 kbps; LEVEL3: unlimited (default).\n"

            "  ENC, DEC, REND, ISAR, ISAR_ENC      — Test groups.\n"

            "  MONO, STEREO, EXT, HOA, SBA, MC     — Output formats. Aliases: HOA→{HOA2,HOA3}, SBA→{FOA,HOA2,HOA3}, MC→{5_1,7_1,5_1_4,5_1_2,7_1_4}.\n"

            "  (any other)                         — Substring match (case-insensitive). Multiple terms combine with AND.\n"

            "  +TOKEN*, -TOKEN*                    — Wildcard with add/remove modifiers (e.g., +ISAR*, -BINAURAL*).\n"

            "\n"

            "Examples:\n"

            "  --filter LEVEL1                     Run LEVEL1 ENC+DEC (≤80 kbps).\n"

            "  --filter LEVEL2                     Run LEVEL2 ENC+DEC (≤192 kbps).\n"

            "  --filter LEVEL1 DEC MONO            Restrict to DEC MONO tests at LEVEL1.\n"

            "  --filter LEVEL1 +REND +ISAR         Add REND and ISAR to LEVEL1 baseline ENC+DEC.\n"

            "  --filter DEC HOA                    Run DEC tests with HOA2/HOA3 outputs.\n"

            "  --filter DEC -voip                  Run DEC tests except those matching 'voip'.\n"

            "  --filter ISAR*                      Add ISAR and ISAR_ENC (wildcard expansion).\n"

            "  --filter DEC +BINAURAL*             Run LEVEL3 DEC + all BINAURAL output formats.\n"

            "  --filter DEC +BINAURAL*             Run DEC + all BINAURAL output formats.\n"

            "  --filter +ISAR* -voip               Add ISAR/ISAR_ENC groups then remove 'voip' tests.\n"

        ),

    )

    if args.cut_build_path:

        validate_build_binaries(parser, args.cut_build_path, "CUT")

    # Parse --filter into level + optional tag selection + optional format/substring filters.

    # Parse --filter into optional tag selection + optional format/substring filters.

    raw_filter = " ".join(args.filter) if args.filter else ""

    filter_tokens = [tok for tok in re.split(r"[\s,]+", raw_filter.strip()) if tok]

    valid_tags = set(IVAS_Bins.keys())

    valid_levels = {"LEVEL1", "LEVEL2", "LEVEL3"}

    valid_decoder_formats = set(DECODER_OUTPUT_FORMATS).union(

        DECODER_OUTPUT_FORMAT_ALIASES.keys()

    )

    level_tokens = []

    tag_tokens = []

    tag_add_tokens = []

    tag_remove_tokens = []

            continue

        if upper_tok in valid_levels:

            if sign:

                parser.error(f"Level token '{tok}' cannot be prefixed with '+' or '-'.")

            level_tokens.append(upper_tok)

        elif upper_tok in valid_tags:

        if upper_tok in valid_tags:

            if sign == "+":

                tag_add_tokens.append(upper_tok)

            elif sign == "-":

            else:

                restrictive_terms.append(base_tok)

    if len(set(level_tokens)) > 1:

        parser.error("Multiple filter levels specified. Use only one of LEVEL1, LEVEL2, LEVEL3.")

    filter_level = level_tokens[0] if level_tokens else "LEVEL3"

    # Preserve order while removing duplicates.

    tag_tokens = list(dict.fromkeys(tag_tokens))

    tag_add_tokens = list(dict.fromkeys(tag_add_tokens))

    decoder_format_tokens = list(dict.fromkeys(decoder_format_tokens))

    # Default with no --filter: run all tags.

    # LEVEL1 default baseline: ENC + DEC (REND/ISAR optional unless explicitly selected/added).

    # For other filter cases, plain tag_tokens restrict selected tags and +tag_tokens add tags.

    # Plain tag_tokens restrict selected tags; +tag_tokens add tags.

    if tag_tokens:

        selected_tag_set = set(tag_tokens)

    elif filter_tokens and filter_level == "LEVEL1":

        selected_tag_set = {"ENC", "DEC"}

    else:

        selected_tag_set = set(IVAS_Bins.keys())

    for tag in tag_add_tokens:

    args.filter_add_terms = additive_terms

    args.filter_remove_terms = subtractive_terms

    args.filter_decoder_formats = decoder_format_tokens

    args.filter_level = filter_level

    conformance = MLDConformance(args)

    conformance.accumulateCommands()