Merge branch 'main' into ericsson/add-bs1534-20-processing-script

*.bs

*.192

mc.double

experiments/selection/*/proc_input*/cat*/*.wav

experiments/selection/*/proc_input*/cat*/*.csv

experiments/selection/*/proc_input*/cat*/*.pcm

experiments/selection/*/proc_input*/FOA*/*.wav

experiments/selection/*/proc_input*/HOA2*/*.wav

experiments/selection/*/background_noise/*.wav

experiments/selection/*/background_noise/*.txt

experiments/selection/*/proc_input*/*.wav

experiments/selection/*/proc_input*/*.pcm

experiments/selection/*/proc_input*/*.csv

experiments/selection/*/proc_input*/*.log

experiments/selection/*/proc_input*/*.yml

experiments/selection/*/proc_output*/

experiments/selection/*/config/*-lab_*.yml

experiments/selection/*/gen_input/IRs/*.wav

experiments/selection/*/gen_input/items_mono/*.wav

experiments/*/*/proc_input*/cat*/*.wav

experiments/*/*/proc_input*/cat*/*.csv

experiments/*/*/proc_input*/cat*/*.met

experiments/*/*/proc_input*/cat*/*.pcm

experiments/*/*/proc_input*/FOA*/*.wav

experiments/*/*/proc_input*/HOA2*/*.wav

experiments/*/*/background_noise/*.wav

experiments/*/*/background_noise/*.csv

experiments/*/*/background_noise/*.txt

experiments/*/*/proc_input*/*.wav

experiments/*/*/proc_input*/*.pcm

experiments/*/*/proc_input*/*.csv

experiments/*/*/proc_input*/*.log

experiments/*/*/proc_input*/*.yml

experiments/*/*/proc_output*/

experiments/*/*/config/*-lab_*.yml

experiments/*/*/gen_input/IRs/*.wav

experiments/*/*/gen_input/items_mono/*.wav

*~

tests/temp_output_*

tests/cut

****<!---

\*\*\*\*<!---

(C) 2022-2025 IVAS codec Public Collaboration with portions copyright Dolby International AB, Ericsson AB,

Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V., Huawei Technologies Co. LTD.,

---

# Running characterization experiments

For an overview of the experiments, see [characterization_tests.md](characterization_tests.md)

## Prerequisites

- A working python environment (minimum version >= 3.9) with the required packages installed.

- The required additional binaries from ITU STL or the respective 3GPP contributions:

  - The binaries must be placed in ivas_processing_scripts/bin OR

  - a path specified in [binary_paths.yml](ivas_processing_scripts/binary_paths.yml) configuration file (please read the instructions carefully if using).

  - Please note the required patch for networkSimulator_g192 that must be manually applied.

- EVS fixed point binaries

  - These should be placed in the bin folder and named EVS_cod_fx and EVS_dec_fx

- Compiled versions of the codec from the BASOP repository:

  - Binaries compiled with ivas_float_update should be placed in the bin folder and named IVAS_cod_flt_ref and IVAS_dec_flt_ref

  - Binaries compiled with main (BASOP) should be placed in the bin folder and named IVAS_cod_fx and IVAS_dec_fx

- IVAS_rend from the floating-point repository:

  - This should be placed in the bin folder and named named IVAS_rend

- A local clone of this repository

## Setup

- Ensure you are on the latest version of the main branch of the processing scripts repo (`git pull`)

- Navigate to the box.com folder “Inbox_selected” for the experiment you wish to run.

- Download the experiment folder, e.g., “Inbox_selected/P.800/P.800-1”

- Copy/move the downloaded items into the corresponding folder in experiments/characterization in your copy of the processing scripts repo.

- e.g., `cp -r ~/Downloads/P.800-1/\* experiments/characterization/P800-1/`

## Processing

Run the processing from the root directory of the repository by invoking the python module with the experiment configuration YAML file:

```sh

python3 -m ivas_processing_scripts experiments/characterization/P800-1/config/P800-1_cat1_STEREO-16LKFS.yml

```

Repeat the above step for all configuration files to process all categories and levels.

Notes:

- The python module does NOT accept a list of configs; they must be run individually.

- This can be somewhat worked around with a shell loop, but please note this can make finding an error more difficult:

```sh

for config in $(ls experiments/characterization/P800-1/config/*.yml); do python3 -m ivas_processing_scripts $config; done

```

## Follow up tasks

After the processing is finished, the outputs will be present in the respective `proc_output` directories.

- For P.800 tests, these outputs need to be collected according to the test plan to assemble the final outputs required for P.800 test software.

- For MUSHRA tests, the outputs may be used directly, or in case of multi-input format experiments again need to be manually assembled.

At this point of time there is no script to automatically perform this.

## Directory structure

### P.800 experiments

```bash

.

└── P800-1

    ├── background_noise

    │   └── <background noise files.wav>

    ├── config

    │   └── <configurations from git.yaml>

    ├── proc_input

    │   ├── cat1

    │   │   ├── <item_name.wav>

    │   │   ├── <item_name.wav.{0,1,2,3}.csv> for ISM metadata

    │   │   └── <item_name.wav.met> for MASA metadata

    │   ├── cat2...

    │   └── ...cat6

    └── proc_output

        ├── cat1

        │   ├── out_-16LKFS

        │   │   ├── c01

        │   │   │   └── <item_name.c01.wav>

        │   │   ├── c02...

        │   │   └── ...cXX

        │   ├── out_-26LKFS

        │   └── out_-36LKFS

        ├── cat2...

        └── ...cat6

```

### BS.1534 experiments

```bash

.

└── BS1534-1

    ├── config

    │   └── <configurations from git.yaml>

    ├── proc_input[_FMT] # for multiple input formats, suffix is present

    │   ├── <item_name.wav>

    │   ├── <item_name.wav.{0,1,2,3}.csv> for ISM metadata

    │   └── <item_name.wav.met> for MASA metadata

    └── proc_output[_FMT] # for multiple input formats, suffix is present

        ├── c01

        │   └── <item_name.c01.wav>

        ├── c02...

        └── ...cXX

```

# Quick guide: How to set up a listening test

This section describes step by step how to set up a listening test according to the Processing Plan (IVAS-7) and Test Plan (IVAS-8).

- `input`: list of mono `.wav` files

- `azimuth` / `elevation`: spatial placement (°)

- `level`: loudness in dB

- `shift`: timing offsets in seconds

Dynamic positioning (e.g., `"-20:1.0:360"`) means the source will move over time, stepping every 20 ms.

- `shift`: signal offset/overlap in seconds

- `background`: background noise file (applicable to STEREO and SBA only)

- `background_level`: level of the background noise (applicable to STEREO and SBA only)

The total duration of the output signal can be controlled using the `duration` field. The output signal may optionally be rendered to the BINAURAL format by specifying the `binaural_output` field.

Dynamic positioning (e.g., `"-20:1.0:360"`) means the source will move over time, stepping every 20 ms. The maximum total duration of the output signal can be controlled using the `duration` field. The output signal may optionally be rendered to the BINAURAL format by specifying the `binaural_output` field. The `shift` parameter ensures time adjustment (offset) of the input signal (positive value delays the signal). Aternatively, the notation `X(i_ref)` generates overlap by `X` seconds from the reference signal `i_ref` (0-based index) (positive value creates gap).

Start by running a single scene to verify settings. Output includes both audio and optional metadata files. You can enable multiprocessing by setting `multiprocessing: true`.

### Item processing

The addition of custom background noise at specific level is supported for the STEREO and SBA formats only. For ISMs it's not applicable. For OMASA and OSBA formats, it is expected that the backround noise is provided in the FOA/HOA2/HOA3 format as the first item in the `input` list.

### Item processing (selection)

The input has to be in the folder `experiments/selection/P800-{X}/proc_input_{l}`. If item generation is performed previous to this step, the corresponding files are already in the right folder.

If this step is performed independently of the previous one the input files have to be copied to the respective folder sorted by category.

The set up for the MUSHRA test only consists of the item processing.

### Item processing

### Item processing (Selection)

To process a BS1534-{X}{x} (X = 1, 2, ...7, x = a, b) listening test, the input files have to be placed in the folder `experiments/selection/BS1534-{X}{x}/proc_input_{l}` and the command

`python generate_test.py BS1534-{X}{x},{l}` has to be run from the root of the repository. 'l' stands for the listening lab designator: a (Force Technology), b (HEAD acoustics), c (MQ University), d (Mesaqin.com).

      # fs: 48000

      ### Additional commandline options; default = null

      # opts: ["-q", "-no_delay_cmp"]

      ### Per-item renderer configuration. Set to true to search for a file with suffix .cfg; default = false

      # render_config: true

      ### Head-tracking trajectory file for binaural output OR 'true' which will search for a file with the suffix .ht.csv next to the input; default = null

      ###   NOTE: this automatically configures the '-T' argument to the decoder, so may conflict if also specified in `opts`

      # trajectory: "path/to/file"

      ### Limit the trajectory to 3DoF via truncation; default = false

      # only_3dof: false

  ### IVAS condition ###############################

  c07:

  # bin_lfe_gain: 1

  ### Flag whether output should be limited to avoid clipping (can alter target loudness); default = true

  # limit: false

  ### Head-tracking trajectory file for binaural output; default = null

  ### Head-tracking trajectory file for binaural output OR 'true' which will search for a file with the suffix .ht.csv in the input dir; default = null

  # trajectory: "path/to/file"

  ### Limit the trajectory to 3DoF via truncation; default = false

  # only_3dof: false

```

</details>

## Additional Executables

The following additional executables are needed for the different processing steps:

| Processing step                                                                             | Executable                 | Where to find                                                                                                   |

| ------------------------------------------------------------------------------------------- | -------------------------- | --------------------------------------------------------------------------------------------------------------- |

| Loudness measurement and adjustment                                                         | bs1770demo                 | <https://github.com/ErikNorvell-Ericsson/STL> (Note branch)                                                     |

| MNRU                                                                                        | p50fbmnru                  | <https://github.com/openitu/STL>                                                                                |

| ESDRU                                                                                       | esdru                      | <https://github.com/openitu/STL>                                                                                |

| Frame error pattern application                                                             | eid-xor                    | <https://github.com/openitu/STL>                                                                                |

| Error pattern generation                                                                    | gen-patt                   | <https://www.itu.int/rec/T-REC-G.191-201003-S/en> (Note: Version in <https://github.com/openitu/STL> is buggy!) |

| Reverberation module                                                                        | reverb                     | <https://github.com/openitu/STL>                                                                                |

| Filtering, Resampling                                                                       | filter                     | <https://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_76/docs/S4-131277.zip>                                         |

| Random offset/seed generation (necessary for background noise and FER bitstream processing) | random                     | <https://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_76/docs/S4-131277.zip>                                         |

| JBM network simulator                                                                       | networkSimulator_g192      | <https://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_76/docs/S4-131277.zip>                                         |

| MASA rendering (also used in loudness measurement of MASA items)                            | masaRenderer, masaAnalyzer | <https://www.3gpp.org/ftp/TSG_SA/WG4_CODEC/TSGS4_122_Athens/Docs/S4-230221.zip>                                 |

| EVS reference conditions                                                                    | EVS_cod, EVS_dec           | <https://www.3gpp.org/ftp/Specs/archive/26_series/26.443/26443-h00.zip>                                         |

Additional executables are needed for the different processing steps, please see the dedicated [README](ivas_processing_scripts/bin/README.md).

The necessary binaries have to be either placed in the [ivas_processing_scripts/bin](./ivas_processing_scripts/bin) folder or the path has to be specified in

[ivas_processing_scripts/binary_paths.yml](./ivas_processing_scripts/binary_paths.yml).

# Characterization experiments

Each link in the table leads to the config directory of the respective experiment.

## P.800

| Experiment                                              | Format               |

| ------------------------------------------------------- | -------------------- |

| [P800-1](experiments/characterization/P800-1/config/)   | Stereo               |

| [P800-2](experiments/characterization/P800-2/config/)   | Stereo               |

| [P800-3](experiments/characterization/P800-3/config/)   | FOA                  |

| [P800-4](experiments/characterization/P800-4/config/)   | HOA2                 |

| [P800-5](experiments/characterization/P800-5/config/)   | HOA3                 |

| [P800-6](experiments/characterization/P800-6/config/)   | MC 5.1, 7.1          |

| [P800-7](experiments/characterization/P800-7/config/)   | MC 5.1+4, 7.1+4      |

| [P800-8](experiments/characterization/P800-8/config/)   | MC - Mixed CICP      |

| [P800-9](experiments/characterization/P800-9/config/)   | 1-2 Objects          |

| [P800-10](experiments/characterization/P800-10/config/) | 3-4 Objects          |

| [P800-11](experiments/characterization/P800-11/config/) | 1-4 Objects          |

| [P800-12](experiments/characterization/P800-12/config/) | MASA 1 TC            |

| [P800-13](experiments/characterization/P800-13/config/) | MASA 2 TCs           |

| [P800-14](experiments/characterization/P800-14/config/) | MASA 1-2 TC          |

| [P800-15](experiments/characterization/P800-15/config/) | OSBA (1-2 objects)   |

| [P800-16](experiments/characterization/P800-16/config/) | OSBA (3-4 objects)   |

| [P800-17](experiments/characterization/P800-17/config/) | OSBA (1-4 objects)   |

| [P800-18](experiments/characterization/P800-18/config/) | OMASA (1-2 objects)  |

| [P800-19](experiments/characterization/P800-19/config/) | OMASA (3-4 objects)  |

| [P800-20](experiments/characterization/P800-20/config/) | OMASA (1-4 objects)  |

| [P800-21](experiments/characterization/P800-21/config/) | JBM with Stereo      |

| [P800-22](experiments/characterization/P800-22/config/) | JBM with 1-2 objects |

| [P800-23](experiments/characterization/P800-23/config/) | JBM with FOA         |

## BS.1534

| Experiment                                                  | Format                     |

| ----------------------------------------------------------- | -------------------------- |

| [BS1534-1](experiments/characterization/BS1534-1/config/)   | Stereo                     |

| [BS1534-2](experiments/characterization/BS1534-2/config/)   | Stereo                     |

| [BS1534-3](experiments/characterization/BS1534-3/config/)   | FOA                        |

| [BS1534-4](experiments/characterization/BS1534-4/config/)   | FOA                        |

| [BS1534-5](experiments/characterization/BS1534-5/config/)   | HOA3                       |

| [BS1534-6](experiments/characterization/BS1534-6/config/)   | Multichannel 5.1           |

| [BS1534-7](experiments/characterization/BS1534-7/config/)   | Multi-channel 5.1, 7.1     |

| [BS1534-8](experiments/characterization/BS1534-8/config/)   | Multi-channel 5.1+2, 5.1+4 |

| [BS1534-9](experiments/characterization/BS1534-9/config/)   | Multi-channel 7.1+4        |

| [BS1534-10](experiments/characterization/BS1534-10/config/) | ISM 1-2                    |

| [BS1534-11](experiments/characterization/BS1534-11/config/) | ISM 3-4                    |

| [BS1534-12](experiments/characterization/BS1534-12/config/) | ISM 3-4                    |

| [BS1534-13](experiments/characterization/BS1534-13/config/) | MASA (1TC)                 |

| [BS1534-14](experiments/characterization/BS1534-14/config/) | MASA (1TC)                 |

| [BS1534-15](experiments/characterization/BS1534-15/config/) | MASA (2TC)                 |

| [BS1534-16](experiments/characterization/BS1534-16/config/) | OSBA (1-4 obj.)            |

| [BS1534-17](experiments/characterization/BS1534-17/config/) | OSBA (1-4 obj.)            |

| [BS1534-18](experiments/characterization/BS1534-18/config/) | OMASA (1-4 obj.)           |

| [BS1534-19](experiments/characterization/BS1534-19/config/) | Stereo downmix for EVS     |

| [BS1534-20](experiments/characterization/BS1534-20/config/) | ISM 6 DoF (4 objects)      |

add_low_level_random_noise: true

### Process with parallel streams

multiprocessing: False

multiprocessing: false

################################################

### Item generation - Filename conventions

###   input:       input filename(s)

###   azimuth:     azimuth in the range [-180,180]; positive values point to the left

###   elevation:   elevation in the range [-90,90]; positive values indicate up

###   shift:       time adjustment of the input signal (negative value delays the signal)

###   shift:       time adjustment of the input signal (positive value delays the signal)

###                alternatively, the notation X(i_ref) generates overlap by X seconds from the reference signal i_ref (0-based index) (positive value creates gap)

###   background:  background noise filename (if used, the 'add_low_level_random_noise' parameter is ignored)

###   background_level:  normalized background noise loudness to X dB LKFS

###

### Note 0: you can use relative paths in filenames (the program assumes that the root directory is the parent directory of the ivas_processing_scripts subfolder)

### Note 1: use brackets [val1, val2, ...] when specifying multiple values