diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 81e3d0a3d601d6dbf451343a34f97d208cd5e424..01a59853c14928147954966b1181e7dca9bc9620 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,7 +1,7 @@
variables:
# note: GitLab cannot reference variables defined by users in the include ref:, we need to use a YAML anchor for this
# see https://docs.gitlab.com/ci/yaml/includes/#use-variables-with-include for more information
- IVAS_CODEC_CI_REF: &IVAS_CODEC_CI_REF main
+ IVAS_CODEC_CI_REF: &IVAS_CODEC_CI_REF deduplicate-renderer-tests
# If you need to set some config variable only in a local branch, then add an overwrite here
# One example is DISABLE_HRTF - this will be set on a branch which is about to be merged and will be removed in a subsequent second MR
# this is more easily done directly here in the child repo
diff --git a/scripts/ivas_conformance/README.md b/scripts/ivas_conformance/README.md
index d56974ef0a113d6a44b660f5d66f6403dc0e0db5..884611e0a7728f741a502e0c3e41effd2a5ae9fa 100644
--- a/scripts/ivas_conformance/README.md
+++ b/scripts/ivas_conformance/README.md
@@ -42,12 +42,11 @@ It might be required to set Clang-18 as the default clang on the machine
sudo apt install python3.13 python3.13-venv
```
-
## Run CUT tests on Target platform
Running the conformance tests requires around 30 gb of disk space and around 6 gb of RAM space.
-To run CUT binaries on the targeted platform, it is necessary to have the correct setup for python and dependency packages.
+To run CUT binaries on the targeted platform, it is necessary to have the correct setup for python and dependency packages.
- Create virtual environment for Python 3.13 and install requirements
@@ -70,7 +69,7 @@ PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDi
Example Output of CUT execution
Accumulating commands from Readme_IVAS_dec.txt
-Accumulating commands from Readme_IVAS_rend.txt
+Accumulating commands from Readme_IVAS_rend.txt
Accumulating commands from Readme_IVAS_enc.txt
Accumulating commands from Readme_IVAS_ISAR_post_rend.txt
Accumulating commands from Readme_IVAS_ISAR_dec.txt
@@ -105,8 +104,7 @@ CUT_OUTPUTS
If CUT test execution is done on a different platform, the scripts/CUT_OUTPUTS must be copied and provided in the reference platform's scripts/CUT_OUTPUTS. Then the BE analysis or non-BE analysis procedure below should be followed. It is recommended to perform first analysis with BE comparison and then analysis with non-BE comparison if non-BE outputs were found. Note, non-BE conformance applies currently only if metadata file output is BE and there are non-BE results only in wave-file output.
-
-### Perform the BE comparison on the CUT outputs on reference platform
+### Perform the BE comparison on the CUT outputs on reference platform
The BE comparison is performed to the CUT outputs using the command below. Encoded outputs will be decoded using the reference decoder executables as part of the process. The BE comparison is then performed between the CUT and reference decoded outputs. This includes comparison of ".wav"-files, and ".csv" and ".met" metadata files. If any non-BE results are observed, this is reported on the command-line and link to an analysis ".csv" file is given. The analysis file shows which exact files were non-BE. An example passing output is shown below. If all test sets print `PASSED BE TEST`, then CUT outputs are BE-conformant.
@@ -142,7 +140,6 @@ Analysing tests for ISAR (1252 tests)
-
### Perform the MLD based non-BE analysis on the CUT outputs on reference platform (Ubuntu 24.04)
The MLD-based non-BE analysis is performed to the CUT outputs with the command below. Encoded outputs will be decoded using the reference decoder executables as part of the process. The MLD analysis is then performed between the CUT and reference decoded outputs (only ".wav" files are compared). Comparison to MLD corridor is also done as part of this process. An example passing output is shown below. If all test sets print `MLD Corridor passed for...` and there were no non-BE metadata comparisons in BE-test, then CUT outputs are Non-BE conformant.
@@ -244,7 +241,6 @@ MLD Corridor passed for ISAR with max MLD diff of 0.0
-
## Executing specific tests only
All CUT tests can be run specifically for IVAS Encoder,IVAS Decoder,IVAS Renderer, ISAR Encoder and ISAR Decoder only. The commandline allows for ```-test-mode=``` for this functionality, examples:
@@ -291,13 +287,13 @@ All CUT tests can be run specifically for IVAS Encoder,IVAS Decoder,IVAS Rendere
PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --cut_build_path=CUT_BIN_DIR --test-mode=REND
```
-- Analyse BE conformance for CUT Renderer Outputs Only
+- Analyse BE conformance for CUT Renderer Outputs Only
```shell
PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --test-mode=REND --analyse --be-test
```
-- Analyse NON-BE conformance CUT Renderer Outputs Only
+- Analyse NON-BE conformance CUT Renderer Outputs Only
```shell
PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --test-mode=REND --analyse
@@ -333,7 +329,7 @@ All CUT tests can be run specifically for IVAS Encoder,IVAS Decoder,IVAS Rendere
PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --test-mode=ISAR --analyse --be-test
```
-- Analyse NON-BE conformance CUT ISAR Decoder Outputs Only
+- Analyse NON-BE conformance CUT ISAR Decoder Outputs Only
```shell
PYTHONPATH=scripts python scripts/ivas_conformance/runConformance.py --testvecDir $PWD/testvec --test-mode=ISAR --analyse
diff --git a/scripts/ivas_conformance/ivas_be_conf_test_gen.sh b/scripts/ivas_conformance/ivas_be_conf_test_gen.sh
index b41e3ab7247d0ca84cf1bd536b9cb1cabe90b01f..6e7f2bb55c5e5a426afff58270e4c21b1f171be0 100644
--- a/scripts/ivas_conformance/ivas_be_conf_test_gen.sh
+++ b/scripts/ivas_conformance/ivas_be_conf_test_gen.sh
@@ -8,15 +8,15 @@ cp IVAS_cod IVAS_cod_ref
cp IVAS_dec IVAS_dec_ref
cp IVAS_rend IVAS_rend_ref
cp ISAR_post_rend ISAR_post_rend_ref
-python3 -m pytest -q tests/codec_be_on_mr_nonselection tests/renderer_short/test_renderer.py tests/split_rendering/test_split_rendering.py -v -n auto --update_ref 1 --create_ref --keep_files --html=report_cmd.html --self-contained-html
-python3 scripts/parse_commands.py report_cmd.html Readme_IVAS.txt
+python3 -m pytest -q tests/codec_be_on_mr_nonselection tests/renderer/test_renderer_short.py tests/split_rendering/test_split_rendering.py -v -n auto --update_ref 1 --create_ref --keep_files --html=report_cmd.html --self-contained-html
+python3 scripts/parse_commands.py report_cmd.html Readme_IVAS.txt
rm -rf testvec
mkdir testvec
mkdir testvec/binauralRenderer_interface
mkdir testvec/testv
mkdir testvec/testv/renderer_short
-mkdir testvec/testv/split_rendering
-mkdir testvec/bin
+mkdir testvec/testv/split_rendering
+mkdir testvec/bin
cp -r scripts/testv/* testvec/testv
cp -r scripts/ls_layouts testvec
cp -r scripts/object_edit testvec
@@ -24,7 +24,7 @@ cp -r scripts/switchPaths testvec
cp -r scripts/trajectories testvec
cp -r scripts/binauralRenderer_interface/binaural_renderers_hrtf_data testvec/binauralRenderer_interface
cp -r tests/ref testvec/testv/ref
-cp -r tests/renderer_short/ref testvec/testv/renderer_short/ref
+cp -r tests/renderer/ref testvec/testv/renderer_short/ref
cp -r tests/split_rendering/ref testvec/testv/split_rendering/ref
cp -r tests/split_rendering/renderer_configs testvec/testv/split_rendering/renderer_configs
cp -r tests/split_rendering/error_patterns testvec/testv/split_rendering/error_patterns
diff --git a/scripts/ivas_conformance/runConformance.py b/scripts/ivas_conformance/runConformance.py
index 50f851db9eb6f0d1908c0602bd02a9e0c8c02861..3fe69b1805d2a9a3cd9f97704aa5bd716185c65c 100644
--- a/scripts/ivas_conformance/runConformance.py
+++ b/scripts/ivas_conformance/runConformance.py
@@ -211,15 +211,16 @@ def validate_build_binaries(parser, build_path: str, build_label: str) -> None:
for tag, binary in IVAS_Bins.items():
candidate = os.path.join(abs_build_path, binary)
candidate_exe = f"{candidate}.exe"
- exists = os.path.isfile(candidate) or (is_windows and os.path.isfile(candidate_exe))
+ exists = os.path.isfile(candidate) or (
+ is_windows and os.path.isfile(candidate_exe)
+ )
if not exists:
shown = candidate_exe if is_windows else candidate
missing.append(f"{tag}: {shown}")
if missing:
- parser.error(
- f"Missing {build_label} binaries:\n - " + "\n - ".join(missing)
- )
+ parser.error(f"Missing {build_label} binaries:\n - " + "\n - ".join(missing))
+
ReferenceMldFiles = {
"ENC": "mld_ref_ENC.csv",
@@ -253,7 +254,9 @@ class MLDConformance:
with open(self.failedCmdsFile, "r") as f:
return sum(1 for line in f if line.strip())
- def appendRunlog(self, command: str = "", output: str = "", context: str = "") -> None:
+ def appendRunlog(
+ self, command: str = "", output: str = "", context: str = ""
+ ) -> None:
if not getattr(self, "logFile", None):
return
with open(self.logFile, "a") as fd:
@@ -266,7 +269,9 @@ class MLDConformance:
if not output.endswith("\n"):
fd.write("\n")
- def appendFailed(self, command: str = "", output: str = "", context: str = "") -> None:
+ def appendFailed(
+ self, command: str = "", output: str = "", context: str = ""
+ ) -> None:
if not getattr(self, "failedCmdsFile", None):
return
with open(self.failedCmdsFile, "a") as fd:
@@ -297,7 +302,7 @@ class MLDConformance:
if self.args.clean_output_dir and os.path.exists(self.outputDir):
shutil.rmtree(self.outputDir, ignore_errors=False)
os.makedirs(self.outputDir, exist_ok=True)
- subdirs = ["enc", "dec", "renderer_short", "split_rendering"]
+ subdirs = ["enc", "dec", "renderer", "split_rendering"]
for odir in subdirs:
os.makedirs(os.path.join(self.testvDir, odir), exist_ok=True)
os.makedirs(os.path.join(self.outputDir, odir), exist_ok=True)
@@ -359,8 +364,8 @@ class MLDConformance:
self.wavdiffbin = os.path.join(self.toolsdir, exe_platform, "wav-diff")
if not os.path.exists(self.wavdiffbin):
if exe_platform == "Windows":
- exe_platform = "Win32"
- self.wavdiffbin = os.path.join(self.toolsdir, exe_platform, "wav-diff")
+ exe_platform = "Win32"
+ self.wavdiffbin = os.path.join(self.toolsdir, exe_platform, "wav-diff")
self.CutBins = dict()
self.mldcsv = dict()
self.BEcsv = dict()
@@ -643,14 +648,16 @@ class MLDConformance:
)
return (non_be, None, None, None)
else:
- if not os.path.exists(testDesc.refOutput) or not os.path.exists(testDesc.dutOutput):
- msg = (
- f"Missing file for compare: ref={testDesc.refOutput}, dut={testDesc.dutOutput}"
- )
+ if not os.path.exists(testDesc.refOutput) or not os.path.exists(
+ testDesc.dutOutput
+ ):
+ msg = f"Missing file for compare: ref={testDesc.refOutput}, dut={testDesc.dutOutput}"
self.appendFailed(context=f"[{tag}:{dutPytestTag}] {msg}")
return (None, None, (msg, ""), None)
- validate_err = self.validateAudioPairHeader(testDesc.refOutput, testDesc.dutOutput)
+ validate_err = self.validateAudioPairHeader(
+ testDesc.refOutput, testDesc.dutOutput
+ )
if validate_err:
self.appendFailed(context=f"[{tag}:{dutPytestTag}] {validate_err}")
return (None, None, (validate_err, ""), None)
@@ -705,17 +712,23 @@ class MLDConformance:
)
if rc != 0:
return (None, None, (dutDecCmd, err_output), dutDecCmd)
- if not os.path.exists(refDecOutputFile) or not os.path.exists(dutDecOutputFile):
+ if not os.path.exists(refDecOutputFile) or not os.path.exists(
+ dutDecOutputFile
+ ):
msg = f"Missing file for compare: ref={refDecOutputFile}, dut={dutDecOutputFile}"
self.appendFailed(context=f"[{tag}:{encPytestTag}] {msg}")
return (None, None, (msg, ""), dutDecCmd)
- validate_err = self.validateAudioPairHeader(refDecOutputFile, dutDecOutputFile)
+ validate_err = self.validateAudioPairHeader(
+ refDecOutputFile, dutDecOutputFile
+ )
if validate_err:
self.appendFailed(context=f"[{tag}:{encPytestTag}] {validate_err}")
return (None, None, (validate_err, ""), dutDecCmd)
- non_be = int(not filecmp.cmp(refDecOutputFile, dutDecOutputFile, shallow=False))
+ non_be = int(
+ not filecmp.cmp(refDecOutputFile, dutDecOutputFile, shallow=False)
+ )
max_mld, mld_error = self.mld(
tag, encPytestTag, refFile=refDecOutputFile, dutFile=dutDecOutputFile
)
@@ -736,7 +749,9 @@ class MLDConformance:
return (non_be, None, None, None)
else:
refDecOutputFile = testDesc.refOutput.replace(".splt.bit", ".wav")
- dutDecOutputFile = testDesc.dutOutput.replace(".splt.bit", "_CUT_REFDECODED.wav")
+ dutDecOutputFile = testDesc.dutOutput.replace(
+ ".splt.bit", "_CUT_REFDECODED.wav"
+ )
# Decode the encoded output with Reference ISAR decoder
dutDecCmd = testDesc.refDecCmdline.split()
for idx, cmd in enumerate(dutDecCmd):
@@ -754,17 +769,23 @@ class MLDConformance:
)
if rc != 0:
return (None, None, (dutDecCmd, err_output), dutDecCmd)
- if not os.path.exists(refDecOutputFile) or not os.path.exists(dutDecOutputFile):
+ if not os.path.exists(refDecOutputFile) or not os.path.exists(
+ dutDecOutputFile
+ ):
msg = f"Missing file for compare: ref={refDecOutputFile}, dut={dutDecOutputFile}"
self.appendFailed(context=f"[{tag}:{pytestTag}] {msg}")
return (None, None, (msg, ""), dutDecCmd)
- validate_err = self.validateAudioPairHeader(refDecOutputFile, dutDecOutputFile)
+ validate_err = self.validateAudioPairHeader(
+ refDecOutputFile, dutDecOutputFile
+ )
if validate_err:
self.appendFailed(context=f"[{tag}:{pytestTag}] {validate_err}")
return (None, None, (validate_err, ""), dutDecCmd)
- non_be = int(not filecmp.cmp(refDecOutputFile, dutDecOutputFile, shallow=False))
+ non_be = int(
+ not filecmp.cmp(refDecOutputFile, dutDecOutputFile, shallow=False)
+ )
max_mld, mld_error = self.mld(
tag, pytestTag, refFile=refDecOutputFile, dutFile=dutDecOutputFile
)
@@ -857,7 +878,9 @@ class MLDConformance:
):
# Run CUT Cmdline
testPrefix = f"[{tag} {testIndex}/{totalTests}]"
- self.appendRunlog(context=self.formatTestHeader(testPrefix, "Running test", pyTestsTag))
+ self.appendRunlog(
+ context=self.formatTestHeader(testPrefix, "Running test", pyTestsTag)
+ )
testDesc = self.TestDesc[tag][pyTestsTag]
rc, err_output = self.process(
command=testDesc.dutCmdline,
@@ -884,28 +907,49 @@ class MLDConformance:
errorDetails = None
executedCommand = None
if tag == "ENC":
- non_be, max_mld, errorDetails, executedCommand = self.analyseOneEncoderTest(tag, pyTestsTag)
+ non_be, max_mld, errorDetails, executedCommand = self.analyseOneEncoderTest(
+ tag, pyTestsTag
+ )
elif tag == "DEC":
- non_be, max_mld, errorDetails, executedCommand = self.analyseWavOutputTest(tag, pyTestsTag)
+ non_be, max_mld, errorDetails, executedCommand = self.analyseWavOutputTest(
+ tag, pyTestsTag
+ )
elif tag == "REND":
- non_be, max_mld, errorDetails, executedCommand = self.analyseWavOutputTest(tag, pyTestsTag)
+ non_be, max_mld, errorDetails, executedCommand = self.analyseWavOutputTest(
+ tag, pyTestsTag
+ )
elif tag == "ISAR_ENC":
- non_be, max_mld, errorDetails, executedCommand = self.analyseOneIsarEncoderTest(tag, pyTestsTag)
+ non_be, max_mld, errorDetails, executedCommand = (
+ self.analyseOneIsarEncoderTest(tag, pyTestsTag)
+ )
elif tag == "ISAR":
- non_be, max_mld, errorDetails, executedCommand = self.analyseWavOutputTest(tag, pyTestsTag)
+ non_be, max_mld, errorDetails, executedCommand = self.analyseWavOutputTest(
+ tag, pyTestsTag
+ )
else:
assert False, f"Un-implemented Tag {tag}"
if errorDetails is not None:
if errorDetails:
cmd, err_output = errorDetails
- self.appendFailed(context=header, command=cmd, output=(err_output or ""))
+ self.appendFailed(
+ context=header, command=cmd, output=(err_output or "")
+ )
elif executedCommand:
self.appendFailed(context=header, command=executedCommand)
else:
self.appendFailed(context=header)
self.stats()
- return (testPrefix, pyTestsTag, "ERROR", None, errorDetails, executedCommand, None, None)
+ return (
+ testPrefix,
+ pyTestsTag,
+ "ERROR",
+ None,
+ errorDetails,
+ executedCommand,
+ None,
+ None,
+ )
if self.args.be_test:
verdict = "NON-BE" if non_be else "BE"
@@ -918,7 +962,16 @@ class MLDConformance:
result_text = f"{verdict}, MLD_MAX={max_mld}"
self.stats()
- return (testPrefix, pyTestsTag, "OK", result_text, None, executedCommand, verdict, max_mld)
+ return (
+ testPrefix,
+ pyTestsTag,
+ "OK",
+ result_text,
+ None,
+ executedCommand,
+ verdict,
+ max_mld,
+ )
def analyseOneCommandFromTuple(self, args):
return self.analyseOneCommand(*args)
@@ -944,9 +997,13 @@ class MLDConformance:
(tag, pyTestsTag, idx, self.totalTests)
for idx, pyTestsTag in enumerate(selectedTests, start=1)
]
- for testPrefix, pyTestsTag, rc, command, err_output in pool.imap_unordered(
- self.runOneCommandFromTuple, args
- ):
+ for (
+ testPrefix,
+ pyTestsTag,
+ rc,
+ command,
+ err_output,
+ ) in pool.imap_unordered(self.runOneCommandFromTuple, args):
status = "OK" if rc == 0 else "ERROR"
print(
f"{testPrefix} Running test: {pyTestsTag} ... {status}",
@@ -1037,7 +1094,12 @@ class MLDConformance:
verdict,
test_max_mld,
):
- nonlocal command_fail_count, be_count, non_be_count, failure_count, worst_failure
+ nonlocal \
+ command_fail_count, \
+ be_count, \
+ non_be_count, \
+ failure_count, \
+ worst_failure
if runStatus != "OK":
command_fail_count += 1
@@ -1047,7 +1109,9 @@ class MLDConformance:
non_be_count += 1
if test_max_mld is not None and test_max_mld > corridor_threshold:
failure_count += 1
- fail_header = self.formatTestHeader(testPrefix, "Analyzing test", pyTestsTag)
+ fail_header = self.formatTestHeader(
+ testPrefix, "Analyzing test", pyTestsTag
+ )
self.appendFailed(
context=(
fail_header
@@ -1099,7 +1163,9 @@ class MLDConformance:
(tag, pyTestsTag, idx, self.totalTests)
for idx, pyTestsTag in enumerate(selectedTests, start=1)
]
- for result in pool.imap_unordered(self.analyseOneCommandFromTuple, args):
+ for result in pool.imap_unordered(
+ self.analyseOneCommandFromTuple, args
+ ):
handle_test_result(*result)
else:
for idx, pyTestsTag in enumerate(selectedTests, start=1):
@@ -1190,7 +1256,11 @@ class MLDConformance:
if emitConsole:
print(f"{prefix}Failed command: {command}", flush=True)
if c.stdout:
- print(c.stdout, end="" if c.stdout.endswith("\n") else "\n", flush=True)
+ print(
+ c.stdout,
+ end="" if c.stdout.endswith("\n") else "\n",
+ flush=True,
+ )
if returnOutput:
return c.returncode, (c.stdout or "")
@@ -1320,7 +1390,10 @@ class MLDConformance:
wavdiff_log_lines = []
wavdiff_rows_omitted = 0
for line in wavdiff_output.splitlines():
- if re.match(r"^\s*[-+]?\d+(?:\.\d+)?;[-+]?\d+(?:\.\d+)?;[-+]?\d+(?:\.\d+)?\s*$", line):
+ if re.match(
+ r"^\s*[-+]?\d+(?:\.\d+)?;[-+]?\d+(?:\.\d+)?;[-+]?\d+(?:\.\d+)?\s*$",
+ line,
+ ):
wavdiff_rows_omitted += 1
else:
wavdiff_log_lines.append(line)
@@ -1372,7 +1445,9 @@ class MLDConformance:
mldWithTags = np.column_stack(
(
mldThisFile,
- np.array([f"{pytestTag}-FRM{x:05d}" for x in range(mldThisFile.size)]),
+ np.array(
+ [f"{pytestTag}-FRM{x:05d}" for x in range(mldThisFile.size)]
+ ),
)
)
with open(self.mldcsv[tag], "ab") as f:
@@ -1463,9 +1538,7 @@ class MLDConformance:
maxDiff = float(diff.max()) if diff.size else 0.0
corridor_failed = maxDiff > threshold
if corridor_failed:
- msg = (
- f"[{tag}] MLD corridor failed: max(dut-ref)={maxDiff} exceeds threshold={threshold}"
- )
+ msg = f"[{tag}] MLD corridor failed: max(dut-ref)={maxDiff} exceeds threshold={threshold}"
self.appendRunlog(context=msg)
self.appendFailed(context=msg)
else:
@@ -1577,7 +1650,9 @@ class MLDConformance:
all_ok = all_ok and corridor_ok
corridor_fail_count += int(not corridor_ok)
else:
- missing_msg = f"Missing reference MLD file for {tag} : {refMldFile}"
+ missing_msg = (
+ f"Missing reference MLD file for {tag} : {refMldFile}"
+ )
print(f"\033[91m{missing_msg} \033[00m")
self.appendRunlog(context=missing_msg)
self.appendFailed(context=missing_msg)
@@ -1727,4 +1802,3 @@ if __name__ == "__main__":
for tag in testTags:
tag_status = "OK" if tag_results.get(tag, False) else "FAILED"
print(f"[{tag}] {tag_status}")
-
diff --git a/scripts/parse_commands.py b/scripts/parse_commands.py
index 7de24cb28ad9053b137a7cb0dbeb1c93813d078f..66b1f9613d432524ebb6765d5973e0d28339d3cd 100644
--- a/scripts/parse_commands.py
+++ b/scripts/parse_commands.py
@@ -7,67 +7,72 @@ from pathlib import Path
import glob
# Main routine
-if __name__ == '__main__':
- parser = argparse.ArgumentParser(description='Parse HTML reports to extract commands')
- parser.add_argument('input',type=str,help='Path to HTML report or directory')
- parser.add_argument('txt_file',type=str,help='Output txt file, e.g. output.txt')
+if __name__ == "__main__":
+ parser = argparse.ArgumentParser(
+ description="Parse HTML reports to extract commands"
+ )
+ parser.add_argument("input", type=str, help="Path to HTML report or directory")
+ parser.add_argument("txt_file", type=str, help="Output txt file, e.g. output.txt")
args = parser.parse_args()
input = args.input
txt_file = args.txt_file
- TESTV_PATH='$TESTV_PATH'
- REF_PATH='$REF_PATH'
- CUT_PATH='$CUT_PATH'
-
+ TESTV_PATH = "$TESTV_PATH"
+ REF_PATH = "$REF_PATH"
+ CUT_PATH = "$CUT_PATH"
+
here = Path(__file__).parent.resolve()
- cmds_enc=[]
- cmds_dec=[]
- cmds_rend=[]
- cmds_isar_post_rend=[]
+ cmds_enc = []
+ cmds_dec = []
+ cmds_rend = []
+ cmds_isar_post_rend = []
all_args = set()
if path.isdir(input):
- input = Path(input).rglob('*.html')
+ input = Path(input).rglob("*.html")
else:
input = [input]
for html_report in input:
-
- with open(html_report,'r') as infile:
+ with open(html_report, "r") as infile:
lines = infile.readlines()
for idx, line in enumerate(lines):
- # parse encoder and decoder commands
- cmds_enc.extend(re.findall(r"REF encoder command:\\n\\t(.*?)\\n", line))
- cmds_dec.extend(re.findall(r"REF decoder command:\\n\\t(.*?)\\n", line))
+ # parse encoder and decoder commands
+ cmds_enc.extend(re.findall(r"REF encoder command:\\n\\t(.*?)\\n", line))
+ cmds_dec.extend(re.findall(r"REF decoder command:\\n\\t(.*?)\\n", line))
- # parse renderer commands, including ISAR EXT REND commands (the next line contains IVAS_rend or IVAS_rend_ref)
- cmds_rend.extend(re.findall(r"Running command\\n(.*?)\\n", line))
- next_line = lines[idx + 1] if idx + 1 < len(lines) else ""
- if (
- "Running ISAR EXT REND command" in line
- and ("IVAS_rend" in next_line or "IVAS_rend_ref" in next_line)
- ):
- cmds_rend.extend(re.findall(r"Running ISAR EXT REND command\\n(.*?)\\n", line))
+ # parse renderer commands, including ISAR EXT REND commands (the next line contains IVAS_rend or IVAS_rend_ref)
+ cmds_rend.extend(re.findall(r"Running command\\n(.*?)\\n", line))
+ next_line = lines[idx + 1] if idx + 1 < len(lines) else ""
+ if "Running ISAR EXT REND command" in line and (
+ "IVAS_rend" in next_line or "IVAS_rend_ref" in next_line
+ ):
+ cmds_rend.extend(
+ re.findall(r"Running ISAR EXT REND command\\n(.*?)\\n", line)
+ )
- # parse ISAR post renderer commands, including ISAR EXT REND commands (the next line contains ISAR_post_rend or ISAR_post_rend_ref)
- cmds_isar_post_rend.extend(re.findall(r"Running ISAR post renderer command\\n(.*?)\\n", line))
- if (
- "Running ISAR EXT REND command" in line
- and ("ISAR_post_rend" in next_line or "ISAR_post_rend_ref" in next_line)
- ):
- cmds_isar_post_rend.extend(re.findall(r"Running ISAR EXT REND command\\n(.*?)\\n", line))
+ # parse ISAR post renderer commands, including ISAR EXT REND commands (the next line contains ISAR_post_rend or ISAR_post_rend_ref)
+ cmds_isar_post_rend.extend(
+ re.findall(r"Running ISAR post renderer command\\n(.*?)\\n", line)
+ )
+ if "Running ISAR EXT REND command" in line and (
+ "ISAR_post_rend" in next_line or "ISAR_post_rend_ref" in next_line
+ ):
+ cmds_isar_post_rend.extend(
+ re.findall(r"Running ISAR EXT REND command\\n(.*?)\\n", line)
+ )
# If pytest-html < v4 is used, the parsing will fail and render empty lists. This is a work-around in case that happens.
if all(not x for x in [cmds_enc, cmds_dec, cmds_rend, cmds_isar_post_rend]):
for html_report in input:
- with open(html_report,'r') as infile:
+ with open(html_report, "r") as infile:
enc_cmd = False
dec_cmd = False
rend_cmd = False
isar_post_rend_cmd = False
for line in infile.readlines():
- line_trim = line.split("
")[0] # Remove trailing html tags
+ line_trim = line.split("
")[0] # Remove trailing html tags
if enc_cmd:
cmds_enc.append(line_trim)
enc_cmd = False
@@ -76,21 +81,21 @@ if __name__ == '__main__':
dec_cmd = False
elif rend_cmd:
cmds_rend.append(line_trim)
- rend_cmd = False
+ rend_cmd = False
elif isar_post_rend_cmd:
cmds_isar_post_rend.append(line_trim)
- isar_post_rend_cmd = False
+ isar_post_rend_cmd = False
else:
if "REF encoder command" in line:
enc_cmd = True
elif "REF decoder command" in line:
dec_cmd = True
elif "Running command" in line:
- rend_cmd = True
+ rend_cmd = True
elif "Running ISAR post renderer command" in line:
- isar_post_rend_cmd = True
+ isar_post_rend_cmd = True
elif "Running ISAR EXT REND command" in line:
- isar_post_rend_cmd = True
+ isar_post_rend_cmd = True
# Sort lists to keep deterministic order between runs
cmds_enc.sort()
@@ -102,43 +107,53 @@ if __name__ == '__main__':
# Relies on the list being sorted
i = 0
while i + 1 < len(cmds_enc):
- if cmds_enc[i].split()[:-1] == cmds_enc[i+1].split()[:-1]:
- del cmds_enc[i+1]
+ if cmds_enc[i].split()[:-1] == cmds_enc[i + 1].split()[:-1]:
+ del cmds_enc[i + 1]
else:
i = i + 1
# Filter out networkSimulator_g192 commands from cmds_rend
cmds_rend = [cmd for cmd in cmds_rend if not "networkSimulator_g192" in cmd]
- with open(txt_file.replace('.','_enc.'),'w', newline='\n') as outfile:
- with open('scripts/enc_header.txt','r') as header:
+ with open(txt_file.replace(".", "_enc."), "w", newline="\n") as outfile:
+ with open("scripts/enc_header.txt", "r") as header:
outfile.write(header.read())
for cmd in cmds_enc:
args = []
for arg in cmd.split():
# Adjust file arguments, pass other arguments as they are
if path.exists(arg):
- arg = path.relpath(arg).replace('\\','/')
- arg = re.sub('IVAS_cod_ref(.exe)?', '$CUT_ENC_BIN', arg)
- arg = re.sub('scripts', TESTV_PATH, arg)
- arg = re.sub('tests', CUT_PATH, arg)
+ arg = path.relpath(arg).replace("\\", "/")
+ arg = re.sub("IVAS_cod_ref(.exe)?", "$CUT_ENC_BIN", arg)
+ arg = re.sub("scripts", TESTV_PATH, arg)
+ arg = re.sub("tests", CUT_PATH, arg)
args.append(arg)
- cmd = ' '.join(args)
- outfile.write(cmd+'\n')
+ cmd = " ".join(args)
+ outfile.write(cmd + "\n")
bts = re.search(r"\s(([\S]+)(.bts|.192|.pkt|.fer))$", cmd)
if bts:
- outfile.write('$DIFF_BIN '+bts.group(1).replace(CUT_PATH, REF_PATH)+' '+bts.group(1)+' >> $LOG_FILE 2>&1\n')
- outfile.write('\n')
- with open('scripts/script_footer.txt','r') as footer:
- outfile.write(footer.read())
+ outfile.write(
+ "$DIFF_BIN "
+ + bts.group(1).replace(CUT_PATH, REF_PATH)
+ + " "
+ + bts.group(1)
+ + " >> $LOG_FILE 2>&1\n"
+ )
+ outfile.write("\n")
+ with open("scripts/script_footer.txt", "r") as footer:
+ outfile.write(footer.read())
- with open(txt_file.replace('.','_dec.'),'w', newline='\n') as outfile_dec, open(txt_file.replace('.','_JBM_dec.'),'w', newline='\n') as outfile_jbm, open(txt_file.replace('.','_ISAR_dec.'),'w', newline='\n') as outfile_isar:
- with open('scripts/dec_header.txt','r') as header:
+ with (
+ open(txt_file.replace(".", "_dec."), "w", newline="\n") as outfile_dec,
+ open(txt_file.replace(".", "_JBM_dec."), "w", newline="\n") as outfile_jbm,
+ open(txt_file.replace(".", "_ISAR_dec."), "w", newline="\n") as outfile_isar,
+ ):
+ with open("scripts/dec_header.txt", "r") as header:
outfile_dec.write(header.read())
- with open('scripts/jbm_header.txt','r') as header:
+ with open("scripts/jbm_header.txt", "r") as header:
outfile_jbm.write(header.read())
- with open('scripts/dec_isar_header.txt','r') as header:
- outfile_isar.write(header.read())
+ with open("scripts/dec_isar_header.txt", "r") as header:
+ outfile_isar.write(header.read())
for cmd in cmds_dec:
absolute_out = re.search(r"\s(([\S]+)(.wav))$", cmd)
@@ -147,111 +162,155 @@ if __name__ == '__main__':
for arg in cmd.split():
# Adjust file arguments, pass other arguments as they are
if path.exists(arg):
- arg = path.relpath(arg).replace('\\','/')
- arg = re.sub('IVAS_dec_ref(.exe)?', '$CUT_DEC_BIN', arg)
- arg = re.sub('scripts', TESTV_PATH, arg)
+ arg = path.relpath(arg).replace("\\", "/")
+ arg = re.sub("IVAS_dec_ref(.exe)?", "$CUT_DEC_BIN", arg)
+ arg = re.sub("scripts", TESTV_PATH, arg)
# Identify input files
- if '.txt' in arg:
- arg = re.sub('tests', REF_PATH, arg)
+ if ".txt" in arg:
+ arg = re.sub("tests", REF_PATH, arg)
# Identify special cases of output files: dectrace, spltmd.bit
- if 'spltmd' in arg or 'dectrace' in arg:
- arg = re.sub('tests', CUT_PATH, arg)
- if 'tests' in arg:
+ if "spltmd" in arg or "dectrace" in arg:
+ arg = re.sub("tests", CUT_PATH, arg)
+ if "tests" in arg:
if not input_set:
- arg = re.sub('tests', REF_PATH, arg) # First occurence of tests/* is the input file
+ arg = re.sub(
+ "tests", REF_PATH, arg
+ ) # First occurence of tests/* is the input file
input_set = True
else:
- arg = re.sub('tests', CUT_PATH, arg) # Remaining occurences of tests/* are output files
+ arg = re.sub(
+ "tests", CUT_PATH, arg
+ ) # Remaining occurences of tests/* are output files
args.append(arg)
- cmd = ' '.join(args)
-
- if 'VOIP' in cmd:
+ cmd = " ".join(args)
+
+ if "VOIP" in cmd:
outfile = outfile_jbm
- elif 'BINAURAL_SPLIT' in cmd:
+ elif "BINAURAL_SPLIT" in cmd:
outfile = outfile_isar
else:
outfile = outfile_dec
-
- outfile.write(cmd+'\n')
+
+ outfile.write(cmd + "\n")
out = re.search(r"\s(([\S]+)(.wav))$", cmd)
isar_out = re.search(r"\s(([\S]+)(.bit))$", cmd)
isar_md_out = re.search(r"-om\s(([\S]+)(.bit))", cmd)
if out:
- diff_cmds=[]
- for output in glob.glob(absolute_out.group(1) + '*'):
- output = path.relpath(output).replace('\\','/')
- output = re.sub('tests', CUT_PATH, output)
- diff_cmds.append('$DIFF_BIN '+output.replace(CUT_PATH, REF_PATH)+' '+output+' >> $LOG_FILE 2>&1\n')
- outfile.write(('; ').join(diff_cmds))
+ diff_cmds = []
+ for output in glob.glob(absolute_out.group(1) + "*"):
+ output = path.relpath(output).replace("\\", "/")
+ output = re.sub("tests", CUT_PATH, output)
+ diff_cmds.append(
+ "$DIFF_BIN "
+ + output.replace(CUT_PATH, REF_PATH)
+ + " "
+ + output
+ + " >> $LOG_FILE 2>&1\n"
+ )
+ outfile.write(("; ").join(diff_cmds))
if isar_out and "ref" in isar_out.group(1):
- outfile.write('$DIFF_BIN '+isar_out.group(1).replace(CUT_PATH, REF_PATH)+' '+isar_out.group(1)+' >> $LOG_FILE 2>&1\n')
+ outfile.write(
+ "$DIFF_BIN "
+ + isar_out.group(1).replace(CUT_PATH, REF_PATH)
+ + " "
+ + isar_out.group(1)
+ + " >> $LOG_FILE 2>&1\n"
+ )
if isar_md_out and "ref" in isar_md_out.group(1):
- outfile.write('; $DIFF_BIN '+isar_md_out.group(1).replace(CUT_PATH, REF_PATH)+' '+isar_md_out.group(1)+' >> $LOG_FILE 2>&1\n')
- outfile.write('\n')
- with open('scripts/script_footer.txt','r') as footer:
- outfile_dec.write(footer.read())
+ outfile.write(
+ "; $DIFF_BIN "
+ + isar_md_out.group(1).replace(CUT_PATH, REF_PATH)
+ + " "
+ + isar_md_out.group(1)
+ + " >> $LOG_FILE 2>&1\n"
+ )
+ outfile.write("\n")
+ with open("scripts/script_footer.txt", "r") as footer:
+ outfile_dec.write(footer.read())
footer.seek(0)
outfile_jbm.write(footer.read())
footer.seek(0)
- outfile_isar.write(footer.read())
+ outfile_isar.write(footer.read())
- with open(txt_file.replace('.','_rend.'),'w', newline='\n') as outfile:
- with open('scripts/rend_header.txt','r') as header:
+ with open(txt_file.replace(".", "_rend."), "w", newline="\n") as outfile:
+ with open("scripts/rend_header.txt", "r") as header:
outfile.write(header.read())
for cmd in cmds_rend:
absolute_out = re.search(r"-o ([^>]+\.wav)", cmd)
-
+
args = []
for arg in cmd.split():
# Adjust file arguments, pass other arguments as they are
if path.exists(arg):
- arg = path.relpath(arg).replace('\\','/')
- arg = re.sub('IVAS_rend_ref(.exe)?', '$CUT_REND_BIN', arg)
- arg = re.sub('scripts', TESTV_PATH, arg)
- arg = re.sub('tests/renderer_short/data', TESTV_PATH + r'/renderer_short/data/', arg)
- arg = re.sub('tests', CUT_PATH, arg)
+ arg = path.relpath(arg).replace("\\", "/")
+ arg = re.sub("IVAS_rend_ref(.exe)?", "$CUT_REND_BIN", arg)
+ arg = re.sub("scripts", TESTV_PATH, arg)
+ arg = re.sub(
+ "tests/renderer/data", TESTV_PATH + r"/renderer/data/", arg
+ )
+ arg = re.sub("tests", CUT_PATH, arg)
args.append(arg)
- cmd = ' '.join(args)
+ cmd = " ".join(args)
if "ref" in cmd:
- outfile.write(cmd+'\n')
+ outfile.write(cmd + "\n")
out = re.search(r"-o\s(([\S]+)(.wav|.raw|.pcm))", cmd)
if out and "ref" in out.group(1):
- diff_cmds=[]
- for output in glob.glob(absolute_out.group(1) + '*'):
- output = path.relpath(output).replace('\\','/')
- output = re.sub('tests', CUT_PATH, output)
- diff_cmds.append('$DIFF_BIN '+output.replace(CUT_PATH + r'/renderer_short/ref',REF_PATH + r'/renderer_short/ref')+' '+ output +' >> $LOG_FILE 2>&1\n')
- outfile.write(('; ').join(diff_cmds))
- outfile.write('\n')
- with open('scripts/script_footer.txt','r') as footer:
- outfile.write(footer.read())
-
- with open(txt_file.replace('.','_ISAR_post_rend.'),'w', newline='\n') as outfile:
- with open('scripts/isar_post_rend_header.txt','r') as header:
+ diff_cmds = []
+ for output in glob.glob(absolute_out.group(1) + "*"):
+ output = path.relpath(output).replace("\\", "/")
+ output = re.sub("tests", CUT_PATH, output)
+ diff_cmds.append(
+ "$DIFF_BIN "
+ + output.replace(
+ CUT_PATH + r"/renderer/ref",
+ REF_PATH + r"/renderer/ref",
+ )
+ + " "
+ + output
+ + " >> $LOG_FILE 2>&1\n"
+ )
+ outfile.write(("; ").join(diff_cmds))
+ outfile.write("\n")
+ with open("scripts/script_footer.txt", "r") as footer:
+ outfile.write(footer.read())
+
+ with open(txt_file.replace(".", "_ISAR_post_rend."), "w", newline="\n") as outfile:
+ with open("scripts/isar_post_rend_header.txt", "r") as header:
outfile.write(header.read())
for cmd in cmds_isar_post_rend:
args = []
for arg in cmd.split():
# Adjust file arguments, pass other arguments as they are
if path.exists(arg):
- arg = path.relpath(arg).replace('\\','/')
- arg = re.sub('ISAR_post_rend_ref(.exe)?', '$CUT_ISAR_POST_REND_BIN', arg)
- arg = re.sub('scripts', TESTV_PATH, arg)
- arg = re.sub('tests/split_rendering/error_patterns', TESTV_PATH + r'/testv/split_rendering/error_patterns', arg)
- if re.search("^tests.*bit$",arg):
- arg = re.sub('tests', REF_PATH, arg)
- arg = re.sub('tests', CUT_PATH, arg)
+ arg = path.relpath(arg).replace("\\", "/")
+ arg = re.sub(
+ "ISAR_post_rend_ref(.exe)?", "$CUT_ISAR_POST_REND_BIN", arg
+ )
+ arg = re.sub("scripts", TESTV_PATH, arg)
+ arg = re.sub(
+ "tests/split_rendering/error_patterns",
+ TESTV_PATH + r"/testv/split_rendering/error_patterns",
+ arg,
+ )
+ if re.search("^tests.*bit$", arg):
+ arg = re.sub("tests", REF_PATH, arg)
+ arg = re.sub("tests", CUT_PATH, arg)
args.append(arg)
- cmd = ' '.join(args)
+ cmd = " ".join(args)
if "ref" in cmd:
- outfile.write(cmd+'\n')
+ outfile.write(cmd + "\n")
out = re.search(r"-o\s(([\S]+)(.wav|.raw|.pcm))", cmd)
if out and "ref" in out.group(1):
- outfile.write('$DIFF_BIN '+out.group(1).replace(CUT_PATH, REF_PATH)+' '+out.group(1)+' >> $LOG_FILE 2>&1\n')
- outfile.write('\n')
- with open('scripts/script_footer.txt','r') as footer:
- outfile.write(footer.read())
-
+ outfile.write(
+ "$DIFF_BIN "
+ + out.group(1).replace(CUT_PATH, REF_PATH)
+ + " "
+ + out.group(1)
+ + " >> $LOG_FILE 2>&1\n"
+ )
+ outfile.write("\n")
+ with open("scripts/script_footer.txt", "r") as footer:
+ outfile.write(footer.read())
diff --git a/tests/renderer_short/test_renderer.py b/tests/renderer/test_renderer_short.py
similarity index 100%
rename from tests/renderer_short/test_renderer.py
rename to tests/renderer/test_renderer_short.py
diff --git a/tests/renderer_short/README.md b/tests/renderer_short/README.md
deleted file mode 100644
index 713821c5d389adaaf89de09b0ec60d4aee84dacf..0000000000000000000000000000000000000000
--- a/tests/renderer_short/README.md
+++ /dev/null
@@ -1,44 +0,0 @@
-========================================
-THIS FOLDER WILL NOT BE PART OF DELIVERY
-========================================
-
-# External Renderer Tests
-
-See also the [contribution page](https://forge.3gpp.org/rep/ivas-codec-pc/ivas-codec/-/wikis/Contributions/2-external-renderer) for related presentations.
-
-### Run tests with:
-
-## Smoke test:
-
-```bash
-python3 -m pytest -q -n auto tests/renderer/test_renderer.py
-```
-
-## Comparison test:
-
-```bash
-python3 -m pytest -q -n auto tests/renderer/test_renderer.py --create_ref # requires IVAS_rend_ref in root!
-python3 -m pytest -q -n auto tests/renderer/test_renderer.py --create_cut
-```
-
-### Important flags (see [pytest docs](https://docs.pytest.org/en/7.2.x/) for more information):
-
-- `-k` flag can filter test cases, e.g. `-k "test_ism_binaural_static"`
-- `-rA` reports ALL (pass, xpass, xfail, fail) instead of the default behaviour of reporting only failed tests\
- this option will also report captured logs, **required for obtaining the commandline of testcases that pass or xfail**
-- `--last-failed` re-runs only the cases that failed in the last test run
-- `--collect-only` is useful when adding new testcases to check if argument parametrization is working correctly
-
-### Directory tree
-
-```
-.
-├── compare_audio.py -> Python implementation of CompAudio, used for comparisons in tests
-├── conftest.py -> Pytest configuration (enable commandline argument ingestion)
-├── constants.py -> Important paths, formats, metadata files and commandline templates
-├── cut -> Default location for output files for test conditions
-├── data -> Input test vectors
-├── ref -> Default location for output files for reference conditions
-├── test_renderer.py -> Runs the renderer for all modes
-└── utils.py -> Wrapper functions for executables for use in testcases
-```
diff --git a/tests/renderer_short/__init__.py b/tests/renderer_short/__init__.py
deleted file mode 100644
index e856f0584f9bb5f9f0f491c15dfac84732fd78f2..0000000000000000000000000000000000000000
--- a/tests/renderer_short/__init__.py
+++ /dev/null
@@ -1,31 +0,0 @@
-#!/usr/bin/env python3
-
-"""
- (C) 2022-2026 IVAS codec Public Collaboration with portions copyright Dolby International AB, Ericsson AB,
- Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V., Huawei Technologies Co. LTD.,
- Koninklijke Philips N.V., Nippon Telegraph and Telephone Corporation, Nokia Technologies Oy, Orange,
- Panasonic Holdings Corporation, Qualcomm Technologies, Inc., VoiceAge Corporation, and other
- contributors to this repository. All Rights Reserved.
-
- This software is protected by copyright law and by international treaties.
- The IVAS codec Public Collaboration consisting of Dolby International AB, Ericsson AB,
- Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V., Huawei Technologies Co. LTD.,
- Koninklijke Philips N.V., Nippon Telegraph and Telephone Corporation, Nokia Technologies Oy, Orange,
- Panasonic Holdings Corporation, Qualcomm Technologies, Inc., VoiceAge Corporation, and other
- contributors to this repository retain full ownership rights in their respective contributions in
- the software. This notice grants no license of any kind, including but not limited to patent
- license, nor is any license granted by implication, estoppel or otherwise.
-
- Contributors are required to enter into the IVAS codec Public Collaboration agreement before making
- contributions.
-
- This software is provided "AS IS", without any express or implied warranties. The software is in the
- development stage. It is intended exclusively for experts who have experience with such software and
- solely for the purpose of inspection. All implied warranties of non-infringement, merchantability
- and fitness for a particular purpose are hereby disclaimed and excluded.
-
- Any dispute, controversy or claim arising under or in relation to providing this software shall be
- submitted to and settled by the final, binding jurisdiction of the courts of Munich, Germany in
- accordance with the laws of the Federal Republic of Germany excluding its conflict of law rules and
- the United Nations Convention on Contracts on the International Sales of Goods.
-"""
diff --git a/tests/renderer_short/compare_audio.py b/tests/renderer_short/compare_audio.py
deleted file mode 100644
index 3d59025e453ad69779cdb5358837b1b2cb9e2d4d..0000000000000000000000000000000000000000
--- a/tests/renderer_short/compare_audio.py
+++ /dev/null
@@ -1,105 +0,0 @@
-#!/usr/bin/env python3
-
-"""
- (C) 2022-2026 IVAS codec Public Collaboration with portions copyright Dolby International AB, Ericsson AB,
- Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V., Huawei Technologies Co. LTD.,
- Koninklijke Philips N.V., Nippon Telegraph and Telephone Corporation, Nokia Technologies Oy, Orange,
- Panasonic Holdings Corporation, Qualcomm Technologies, Inc., VoiceAge Corporation, and other
- contributors to this repository. All Rights Reserved.
-
- This software is protected by copyright law and by international treaties.
- The IVAS codec Public Collaboration consisting of Dolby International AB, Ericsson AB,
- Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V., Huawei Technologies Co. LTD.,
- Koninklijke Philips N.V., Nippon Telegraph and Telephone Corporation, Nokia Technologies Oy, Orange,
- Panasonic Holdings Corporation, Qualcomm Technologies, Inc., VoiceAge Corporation, and other
- contributors to this repository retain full ownership rights in their respective contributions in
- the software. This notice grants no license of any kind, including but not limited to patent
- license, nor is any license granted by implication, estoppel or otherwise.
-
- Contributors are required to enter into the IVAS codec Public Collaboration agreement before making
- contributions.
-
- This software is provided "AS IS", without any express or implied warranties. The software is in the
- development stage. It is intended exclusively for experts who have experience with such software and
- solely for the purpose of inspection. All implied warranties of non-infringement, merchantability
- and fitness for a particular purpose are hereby disclaimed and excluded.
-
- Any dispute, controversy or claim arising under or in relation to providing this software shall be
- submitted to and settled by the final, binding jurisdiction of the courts of Munich, Germany in
- accordance with the laws of the Federal Republic of Germany excluding its conflict of law rules and
- the United Nations Convention on Contracts on the International Sales of Goods.
-"""
-
-import sys
-import warnings
-from typing import Tuple
-
-import numpy as np
-
-from .constants import SCRIPTS_DIR
-
-sys.path.append(str(SCRIPTS_DIR))
-from pyaudio3dtools.audioarray import getdelay
-
-
-def compare_audio_arrays(
- left: np.ndarray, left_fs: int, right: np.ndarray, right_fs: int
-) -> Tuple[float, float]:
- if left_fs != right_fs:
- return ValueError(f"Differing samplerates: {left_fs} vs {right_fs}!")
-
- if left.shape[1] != right.shape[1]:
- cmp_ch = min(left.shape[1], right.shape[1])
- warnings.warn(
- f"Differing number of channels: {left.shape[1]} vs {right.shape[1]}! Comparing first {cmp_ch} channel(s)",
- category=RuntimeWarning,
- )
- left = left[:, :cmp_ch]
- right = right[:, :cmp_ch]
-
- if left.shape[0] != right.shape[0]:
- cmp_smp = min(left.shape[0], right.shape[0])
- warnings.warn(
- f"Warning - different durations: {left.shape[0] / left_fs:.2f}s vs {right.shape[0] / right_fs:.2f}s! Comparing first {cmp_smp / left_fs : .2f} sample(s)",
- category=RuntimeWarning,
- )
- left = left[:cmp_smp, :]
- right = right[:cmp_smp, :]
-
- if not np.array_equal(left, right):
- delay = getdelay(left, right)
- delay_abs = np.abs(delay)
- # getdelay can return large values if signals are quite different
- # limit any delay compensation to 20 ms
- if delay != 0 and (delay_abs < left_fs / 50):
- warnings.warn(
- f"File B is delayed by {delay} samples ({delay*1000 / left_fs : .2f}ms)!",
- category=RuntimeWarning,
- )
-
- # shift array
- left = np.roll(left, delay, axis=0)
-
- # zero shifted out samples
- if delay < 0:
- left[-np.abs(delay) :, :] = 0
- elif delay > 0:
- left[: np.abs(delay), :] = 0
- """
- http://www-mmsp.ece.mcgill.ca/Documents/Software/Packages/AFsp/AFsp/CompAudio.html
- """
- num = np.sum(left * right)
- den = np.sqrt(np.sum(left**2) * np.sum(right**2))
- if den > 0:
- r = num / den
- else:
- r = np.inf
- snr = 10 * np.log10(1 / (1 - (r**2)))
- gain_b = num / np.sum(right**2)
- max_diff = np.abs(np.max(left - right))
- else:
- snr = np.inf
- gain_b = 1
- max_diff = 0
-
- return snr, gain_b, max_diff
diff --git a/tests/renderer_short/constants.py b/tests/renderer_short/constants.py
deleted file mode 100644
index 1c8763fe97ec8d17f677b6827fc3ef6cb0c74be2..0000000000000000000000000000000000000000
--- a/tests/renderer_short/constants.py
+++ /dev/null
@@ -1,443 +0,0 @@
-#!/usr/bin/env python3
-
-"""
-(C) 2022-2026 IVAS codec Public Collaboration with portions copyright Dolby International AB, Ericsson AB,
-Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V., Huawei Technologies Co. LTD.,
-Koninklijke Philips N.V., Nippon Telegraph and Telephone Corporation, Nokia Technologies Oy, Orange,
-Panasonic Holdings Corporation, Qualcomm Technologies, Inc., VoiceAge Corporation, and other
-contributors to this repository. All Rights Reserved.
-
-This software is protected by copyright law and by international treaties.
-The IVAS codec Public Collaboration consisting of Dolby International AB, Ericsson AB,
-Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V., Huawei Technologies Co. LTD.,
-Koninklijke Philips N.V., Nippon Telegraph and Telephone Corporation, Nokia Technologies Oy, Orange,
-Panasonic Holdings Corporation, Qualcomm Technologies, Inc., VoiceAge Corporation, and other
-contributors to this repository retain full ownership rights in their respective contributions in
-the software. This notice grants no license of any kind, including but not limited to patent
-license, nor is any license granted by implication, estoppel or otherwise.
-
-Contributors are required to enter into the IVAS codec Public Collaboration agreement before making
-contributions.
-
-This software is provided "AS IS", without any express or implied warranties. The software is in the
-development stage. It is intended exclusively for experts who have experience with such software and
-solely for the purpose of inspection. All implied warranties of non-infringement, merchantability
-and fitness for a particular purpose are hereby disclaimed and excluded.
-
-Any dispute, controversy or claim arising under or in relation to providing this software shall be
-submitted to and settled by the final, binding jurisdiction of the courts of Munich, Germany in
-accordance with the laws of the Federal Republic of Germany excluding its conflict of law rules and
-the United Nations Convention on Contracts on the International Sales of Goods.
-"""
-
-import platform
-from pathlib import Path
-
-""" Set up paths """
-TESTS_DIR = Path(__file__).parent
-SCRIPTS_DIR = TESTS_DIR.parents[1].joinpath("scripts").resolve()
-TEST_VECTOR_DIR = SCRIPTS_DIR.joinpath("testv")
-
-OUTPUT_PATH_REF = TESTS_DIR.joinpath("ref")
-OUTPUT_PATH_CUT = TESTS_DIR.joinpath("cut")
-
-CUSTOM_LAYOUT_DIR = SCRIPTS_DIR.joinpath("ls_layouts")
-HR_TRAJECTORY_DIR = SCRIPTS_DIR.joinpath("trajectories")
-TESTV_DIR = SCRIPTS_DIR.joinpath("testv")
-LTV_DIR = TESTV_DIR
-
-BIN_SUFFIX_MERGETARGET = "_ref"
-
-if platform.system() == "Windows":
- EXE_SUFFIX = ".exe"
-elif platform.system() in ["Linux", "Darwin"]:
- EXE_SUFFIX = ""
-else:
- assert False, f"Unsupported platform {platform.system()}"
-
-SAMPLING_RATES = ["48kHz", "32kHz", "16kHz"]
-
-""" Renderer commandline template """
-RENDERER_CMD = [
- str(TESTS_DIR.parent.parent.joinpath("IVAS_rend")),
- "-i",
- "", # 2 -> input file
- "-if",
- "", # 4 -> input format
- "-o",
- "/dev/null", # 6 -> output file
- "-of",
- "", # 8 -> output format
- "-fs",
- "48", # 10 -> input fs
- "-no_delay_cmp",
- # "-ndl",
- "-q",
-]
-
-
-""" Format to file mappings """
-NCHAN_TO_FILE = {
- 1: TEST_VECTOR_DIR.joinpath("spectral_test_1ch_48kHz.wav"),
- 2: TEST_VECTOR_DIR.joinpath("spectral_test_2ch_48kHz.wav"),
- 3: TEST_VECTOR_DIR.joinpath("spectral_test_3ch_48kHz.wav"),
- 4: TEST_VECTOR_DIR.joinpath("spectral_test_4ch_48kHz.wav"),
- 5: TEST_VECTOR_DIR.joinpath("spectral_test_5ch_48kHz.wav"),
- 6: TEST_VECTOR_DIR.joinpath("spectral_test_6ch_48kHz.wav"),
- 7: TEST_VECTOR_DIR.joinpath("spectral_test_7ch_48kHz.wav"),
- 8: TEST_VECTOR_DIR.joinpath("spectral_test_8ch_48kHz.wav"),
- 9: TEST_VECTOR_DIR.joinpath("spectral_test_9ch_48kHz.wav"),
- 10: TEST_VECTOR_DIR.joinpath("spectral_test_10ch_48kHz.wav"),
- 11: TEST_VECTOR_DIR.joinpath("spectral_test_11ch_48kHz.wav"),
- 12: TEST_VECTOR_DIR.joinpath("spectral_test_12ch_48kHz.wav"),
- 13: TEST_VECTOR_DIR.joinpath("spectral_test_13ch_48kHz.wav"),
- 15: TEST_VECTOR_DIR.joinpath("spectral_test_15ch_48kHz.wav"),
- 16: TEST_VECTOR_DIR.joinpath("spectral_test_16ch_48kHz.wav"),
- 17: TEST_VECTOR_DIR.joinpath("spectral_test_17ch_48kHz.wav"),
- 18: TEST_VECTOR_DIR.joinpath("spectral_test_18ch_48kHz.wav"),
- 19: TEST_VECTOR_DIR.joinpath("spectral_test_19ch_48kHz.wav"),
- 20: TEST_VECTOR_DIR.joinpath("spectral_test_20ch_48kHz.wav"),
-}
-
-FORMAT_TO_FILE_SMOKETEST = {
- "MONO": NCHAN_TO_FILE[1],
- "STEREO": NCHAN_TO_FILE[2],
- "5_1": NCHAN_TO_FILE[6],
- "7_1": NCHAN_TO_FILE[8],
- "5_1_2": NCHAN_TO_FILE[8],
- "5_1_4": NCHAN_TO_FILE[10],
- "7_1_4": NCHAN_TO_FILE[12],
- "FOA": NCHAN_TO_FILE[4],
- "HOA2": NCHAN_TO_FILE[9],
- "HOA3": NCHAN_TO_FILE[16],
- "ISM1": NCHAN_TO_FILE[1],
- "ISM2": NCHAN_TO_FILE[2],
- "ISM3": NCHAN_TO_FILE[3],
- "ISM4": NCHAN_TO_FILE[4],
- "NDP_ISM4": NCHAN_TO_FILE[4],
- "MASA1": NCHAN_TO_FILE[1],
- "MASA2": NCHAN_TO_FILE[2],
- "ISM1MASA1": NCHAN_TO_FILE[2],
- "ISM2MASA1": NCHAN_TO_FILE[3],
- "ISM3MASA1": NCHAN_TO_FILE[4],
- "ISM4MASA1": NCHAN_TO_FILE[5],
- "ISM1MASA2": NCHAN_TO_FILE[3],
- "ISM2MASA2": NCHAN_TO_FILE[4],
- "ISM3MASA2": NCHAN_TO_FILE[5],
- "ISM4MASA2": NCHAN_TO_FILE[6],
- "ISM1SBA1": NCHAN_TO_FILE[5],
- "ISM2SBA1": NCHAN_TO_FILE[6],
- "ISM3SBA1": NCHAN_TO_FILE[7],
- "ISM4SBA1": NCHAN_TO_FILE[8],
- "ISM1SBA2": NCHAN_TO_FILE[10],
- "ISM2SBA2": NCHAN_TO_FILE[11],
- "ISM3SBA2": NCHAN_TO_FILE[12],
- "ISM4SBA2": NCHAN_TO_FILE[13],
- "ISM1SBA3": NCHAN_TO_FILE[17],
- "ISM2SBA3": NCHAN_TO_FILE[18],
- "ISM3SBA3": NCHAN_TO_FILE[19],
- "ISM4SBA3": NCHAN_TO_FILE[20],
- "META": TEST_VECTOR_DIR.joinpath("mixed_scene_48.txt"),
- "16ch_8+4+4": NCHAN_TO_FILE[16],
- "t_design_4": NCHAN_TO_FILE[12],
-}
-
-FORMAT_TO_FILE_COMPARETEST = {
- "MONO": TESTV_DIR.joinpath("stv48c.wav"),
- "STEREO": TESTV_DIR.joinpath("stvST48c.wav"),
- "5_1": TESTV_DIR.joinpath("stv51MC48c.wav"),
- "7_1": TESTV_DIR.joinpath("stv71MC48c.wav"),
- "5_1_2": TESTV_DIR.joinpath("stv512MC48c.wav"),
- "5_1_4": TESTV_DIR.joinpath("stv514MC48c.wav"),
- "7_1_4": TESTV_DIR.joinpath("stv714MC48c.wav"),
- "FOA": TESTV_DIR.joinpath("stvFOA48c.wav"),
- "HOA2": TESTV_DIR.joinpath("stv2OA48c.wav"),
- "HOA3": TESTV_DIR.joinpath("stv3OA48c.wav"),
- "ISM1": TESTV_DIR.joinpath("stv1ISM48s.wav"),
- "ISM2": TESTV_DIR.joinpath("stv2ISM48s.wav"),
- "ISM3": TESTV_DIR.joinpath("stv3ISM48s.wav"),
- "ISM4": TESTV_DIR.joinpath("stv4ISM48s.wav"),
- "MASA1": TESTV_DIR.joinpath("stv1MASA1TC48c.wav"),
- "MASA2": TESTV_DIR.joinpath("stv2MASA2TC48c.wav"),
- "ISM1MASA1": TESTV_DIR.joinpath("stvOMASA_1ISM_1MASA1TC48c.wav"),
- "ISM2MASA1": TESTV_DIR.joinpath("stvOMASA_2ISM_2MASA1TC48c.wav"),
- "ISM3MASA1": TESTV_DIR.joinpath("stvOMASA_3ISM_1MASA1TC48c.wav"),
- "ISM4MASA1": TESTV_DIR.joinpath("stvOMASA_4ISM_2MASA1TC48c.wav"),
- "ISM1MASA2": TESTV_DIR.joinpath("stvOMASA_1ISM_1MASA2TC48c.wav"),
- "ISM2MASA2": TESTV_DIR.joinpath("stvOMASA_2ISM_2MASA2TC48c.wav"),
- "ISM3MASA2": TESTV_DIR.joinpath("stvOMASA_3ISM_1MASA2TC48c.wav"),
- "ISM4MASA2": TESTV_DIR.joinpath("stvOMASA_4ISM_2MASA2TC48c.wav"),
- "ISM1SBA1": TESTV_DIR.joinpath("stvOSBA_1ISM_FOA48c.wav"),
- "ISM1SBA2": TESTV_DIR.joinpath("stvOSBA_1ISM_2OA48c.wav"),
- "ISM1SBA3": TESTV_DIR.joinpath("stvOSBA_1ISM_3OA48c.wav"),
- "ISM2SBA1": TESTV_DIR.joinpath("stvOSBA_2ISM_FOA48c.wav"),
- "ISM2SBA2": TESTV_DIR.joinpath("stvOSBA_2ISM_2OA48c.wav"),
- "ISM2SBA3": TESTV_DIR.joinpath("stvOSBA_2ISM_3OA48c.wav"),
- "ISM3SBA1": TESTV_DIR.joinpath("stvOSBA_3ISM_FOA48c.wav"),
- "ISM3SBA2": TESTV_DIR.joinpath("stvOSBA_3ISM_2OA48c.wav"),
- "ISM3SBA3": TESTV_DIR.joinpath("stvOSBA_3ISM_3OA48c.wav"),
- "ISM4SBA1": TESTV_DIR.joinpath("stvOSBA_4ISM_FOA48c.wav"),
- "ISM4SBA2": TESTV_DIR.joinpath("stvOSBA_4ISM_2OA48c.wav"),
- "ISM4SBA3": TESTV_DIR.joinpath("stvOSBA_4ISM_3OA48c.wav"),
- "META": TEST_VECTOR_DIR.joinpath("mixed_scene_48.txt"),
- "16ch_8+4+4": TESTV_DIR.joinpath("stv3OA48c.wav"),
- "t_design_4": TESTV_DIR.joinpath("stv714MC48c.wav"),
-}
-
-FORMAT_TO_FILE_LTV = {
- "MONO": LTV_DIR.joinpath("ltv48_MONO.wav"),
- "STEREO": LTV_DIR.joinpath("ltv48_STEREO.wav"),
- "5_1": LTV_DIR.joinpath("ltv48_MC51.wav"),
- "7_1": LTV_DIR.joinpath("ltv48_MC71.wav"),
- "5_1_2": LTV_DIR.joinpath("ltv48_MC512.wav"),
- "5_1_4": LTV_DIR.joinpath("ltv48_MC514.wav"),
- "7_1_4": LTV_DIR.joinpath("ltv48_MC714.wav"),
- "FOA": LTV_DIR.joinpath("ltv48_FOA.wav"),
- "HOA2": LTV_DIR.joinpath("ltv48_HOA2.wav"),
- "HOA3": LTV_DIR.joinpath("ltv48_HOA3.wav"),
- "ISM1": LTV_DIR.joinpath("ltv48_1ISM.wav"),
- "ISM2": LTV_DIR.joinpath("ltv48_2ISM.wav"),
- "ISM3": LTV_DIR.joinpath("ltv48_3ISM.wav"),
- "ISM4": LTV_DIR.joinpath("ltv48_4ISM.wav"),
- "MASA1": LTV_DIR.joinpath("ltv48_MASA1TC.wav"),
- "MASA2": LTV_DIR.joinpath("ltv48_MASA2TC.wav"),
- "ISM1MASA1": LTV_DIR.joinpath("ltv48_OMASA_1ISM_1TC.wav"),
- "ISM2MASA1": LTV_DIR.joinpath("ltv48_OMASA_2ISM_1TC.wav"),
- "ISM3MASA1": LTV_DIR.joinpath("ltv48_OMASA_3ISM_1TC.wav"),
- "ISM4MASA1": LTV_DIR.joinpath("ltv48_OMASA_4ISM_1TC.wav"),
- "ISM1MASA2": LTV_DIR.joinpath("ltv48_OMASA_1ISM_2TC.wav"),
- "ISM2MASA2": LTV_DIR.joinpath("ltv48_OMASA_2ISM_2TC.wav"),
- "ISM3MASA2": LTV_DIR.joinpath("ltv48_OMASA_3ISM_2TC.wav"),
- "ISM4MASA2": LTV_DIR.joinpath("ltv48_OMASA_4ISM_2TC.wav"),
- "ISM1SBA1": LTV_DIR.joinpath("ltv48_OSBA_1ISM_FOA.wav"),
- "ISM1SBA2": LTV_DIR.joinpath("ltv48_OSBA_1ISM_HOA2.wav"),
- "ISM1SBA3": LTV_DIR.joinpath("ltv48_OSBA_1ISM_HOA3.wav"),
- "ISM2SBA1": LTV_DIR.joinpath("ltv48_OSBA_2ISM_FOA.wav"),
- "ISM2SBA2": LTV_DIR.joinpath("ltv48_OSBA_2ISM_HOA2.wav"),
- "ISM2SBA3": LTV_DIR.joinpath("ltv48_OSBA_2ISM_HOA3.wav"),
- "ISM3SBA1": LTV_DIR.joinpath("ltv48_OSBA_3ISM_FOA.wav"),
- "ISM3SBA2": LTV_DIR.joinpath("ltv48_OSBA_3ISM_HOA2.wav"),
- "ISM3SBA3": LTV_DIR.joinpath("ltv48_OSBA_3ISM_HOA3.wav"),
- "ISM4SBA1": LTV_DIR.joinpath("ltv48_OSBA_4ISM_FOA.wav"),
- "ISM4SBA2": LTV_DIR.joinpath("ltv48_OSBA_4ISM_HOA2.wav"),
- "ISM4SBA3": LTV_DIR.joinpath("ltv48_OSBA_4ISM_HOA3.wav"),
- "META": TEST_VECTOR_DIR.joinpath("mixed_scene_48.txt"),
- "16ch_8+4+4": LTV_DIR.joinpath("ltv48_HOA3.wav"),
- "t_design_4": LTV_DIR.joinpath("ltv48_MC714.wav"),
-}
-
-FORMAT_TO_METADATA_FILES = {
- "ISM1": [str(TESTV_DIR.joinpath("stvISM1.csv"))],
- "ISM2": [
- str(TESTV_DIR.joinpath("stvISM1.csv")),
- str(TESTV_DIR.joinpath("stvISM2.csv")),
- ],
- "ISM3": [
- str(TESTV_DIR.joinpath("stvISM1.csv")),
- str(TESTV_DIR.joinpath("stvISM2.csv")),
- str(TESTV_DIR.joinpath("stvISM3.csv")),
- ],
- "ISM4": [
- str(TESTV_DIR.joinpath("stvISM1.csv")),
- str(TESTV_DIR.joinpath("stvISM2.csv")),
- str(TESTV_DIR.joinpath("stvISM3.csv")),
- str(TESTV_DIR.joinpath("stvISM4.csv")),
- ],
- "NDP_ISM4": [
- str(TESTV_DIR.joinpath("stvISM1.csv")),
- str(TESTV_DIR.joinpath("stvISM2_non-diegetic-pan.csv")),
- str(TESTV_DIR.joinpath("stvISM3.csv")),
- str(TESTV_DIR.joinpath("stvISM4.csv")),
- ],
- "MASA1": [str(TESTV_DIR.joinpath("stv1MASA1TC48c.met"))],
- "MASA2": [str(TESTV_DIR.joinpath("stv2MASA2TC48c.met"))],
- "ISM1MASA1": [
- str(TESTV_DIR.joinpath("stvISM1.csv")),
- str(TESTV_DIR.joinpath("stvOMASA_1ISM_1MASA1TC48c.met")),
- ],
- "ISM2MASA1": [
- str(TESTV_DIR.joinpath("stvISM1.csv")),
- str(TESTV_DIR.joinpath("stvISM2.csv")),
- str(TESTV_DIR.joinpath("stvOMASA_2ISM_2MASA1TC48c.met")),
- ],
- "ISM3MASA1": [
- str(TESTV_DIR.joinpath("stvISM1.csv")),
- str(TESTV_DIR.joinpath("stvISM2.csv")),
- str(TESTV_DIR.joinpath("stvISM3.csv")),
- str(TESTV_DIR.joinpath("stvOMASA_3ISM_1MASA1TC48c.met")),
- ],
- "ISM4MASA1": [
- str(TESTV_DIR.joinpath("stvISM1.csv")),
- str(TESTV_DIR.joinpath("stvISM2.csv")),
- str(TESTV_DIR.joinpath("stvISM3.csv")),
- str(TESTV_DIR.joinpath("stvISM4.csv")),
- str(TESTV_DIR.joinpath("stvOMASA_4ISM_2MASA1TC48c.met")),
- ],
- "ISM1MASA2": [
- str(TESTV_DIR.joinpath("stvISM1.csv")),
- str(TESTV_DIR.joinpath("stvOMASA_1ISM_1MASA2TC48c.met")),
- ],
- "ISM2MASA2": [
- str(TESTV_DIR.joinpath("stvISM1.csv")),
- str(TESTV_DIR.joinpath("stvISM2.csv")),
- str(TESTV_DIR.joinpath("stvOMASA_2ISM_2MASA2TC48c.met")),
- ],
- "ISM3MASA2": [
- str(TESTV_DIR.joinpath("stvISM1.csv")),
- str(TESTV_DIR.joinpath("stvISM2.csv")),
- str(TESTV_DIR.joinpath("stvISM3.csv")),
- str(TESTV_DIR.joinpath("stvOMASA_3ISM_1MASA2TC48c.met")),
- ],
- "ISM4MASA2": [
- str(TESTV_DIR.joinpath("stvISM1.csv")),
- str(TESTV_DIR.joinpath("stvISM2.csv")),
- str(TESTV_DIR.joinpath("stvISM3.csv")),
- str(TESTV_DIR.joinpath("stvISM4.csv")),
- str(TESTV_DIR.joinpath("stvOMASA_4ISM_2MASA2TC48c.met")),
- ],
-}
-
-FORMAT_TO_METADATA_FILES_LTV = {
- "ISM1": [str(LTV_DIR.joinpath("ltvISM1.csv"))],
- "ISM2": [
- str(LTV_DIR.joinpath("ltvISM1.csv")),
- str(LTV_DIR.joinpath("ltvISM2.csv")),
- ],
- "ISM3": [
- str(LTV_DIR.joinpath("ltvISM1.csv")),
- str(LTV_DIR.joinpath("ltvISM2.csv")),
- str(LTV_DIR.joinpath("ltvISM3.csv")),
- ],
- "ISM4": [
- str(LTV_DIR.joinpath("ltvISM1.csv")),
- str(LTV_DIR.joinpath("ltvISM2.csv")),
- str(LTV_DIR.joinpath("ltvISM3.csv")),
- str(LTV_DIR.joinpath("ltvISM4.csv")),
- ],
- "NDP_ISM4": [ # Should not be needed, because it is included in all ISM metadata files.
- str(LTV_DIR.joinpath("ltvISM1.csv")),
- str(LTV_DIR.joinpath("ltvISM2.csv")),
- str(LTV_DIR.joinpath("ltvISM3.csv")),
- str(LTV_DIR.joinpath("ltvISM4.csv")),
- ],
- "MASA1": [str(LTV_DIR.joinpath("ltv48_MASA1TC.met"))],
- "MASA2": [str(LTV_DIR.joinpath("ltv48_MASA2TC.met"))],
- "ISM1MASA1": [
- str(LTV_DIR.joinpath("ltvISM1.csv")),
- str(LTV_DIR.joinpath("ltv48_OMASA_1ISM_1TC.met")),
- ],
- "ISM2MASA1": [
- str(LTV_DIR.joinpath("ltvISM1.csv")),
- str(LTV_DIR.joinpath("ltvISM2.csv")),
- str(LTV_DIR.joinpath("ltv48_OMASA_2ISM_1TC.met")),
- ],
- "ISM3MASA1": [
- str(LTV_DIR.joinpath("ltvISM1.csv")),
- str(LTV_DIR.joinpath("ltvISM2.csv")),
- str(LTV_DIR.joinpath("ltvISM3.csv")),
- str(LTV_DIR.joinpath("ltv48_OMASA_3ISM_1TC.met")),
- ],
- "ISM4MASA1": [
- str(LTV_DIR.joinpath("ltvISM1.csv")),
- str(LTV_DIR.joinpath("ltvISM2.csv")),
- str(LTV_DIR.joinpath("ltvISM3.csv")),
- str(LTV_DIR.joinpath("ltvISM4.csv")),
- str(LTV_DIR.joinpath("ltv48_OMASA_4ISM_1TC.met")),
- ],
- "ISM1MASA2": [
- str(LTV_DIR.joinpath("ltvISM1.csv")),
- str(LTV_DIR.joinpath("ltv48_OMASA_1ISM_2TC.met")),
- ],
- "ISM2MASA2": [
- str(LTV_DIR.joinpath("ltvISM1.csv")),
- str(LTV_DIR.joinpath("ltvISM2.csv")),
- str(LTV_DIR.joinpath("ltv48_OMASA_2ISM_2TC.met")),
- ],
- "ISM3MASA2": [
- str(LTV_DIR.joinpath("ltvISM1.csv")),
- str(LTV_DIR.joinpath("ltvISM2.csv")),
- str(LTV_DIR.joinpath("ltvISM3.csv")),
- str(LTV_DIR.joinpath("ltv48_OMASA_3ISM_2TC.met")),
- ],
- "ISM4MASA2": [
- str(LTV_DIR.joinpath("ltvISM1.csv")),
- str(LTV_DIR.joinpath("ltvISM2.csv")),
- str(LTV_DIR.joinpath("ltvISM3.csv")),
- str(LTV_DIR.joinpath("ltvISM4.csv")),
- str(LTV_DIR.joinpath("ltv48_OMASA_4ISM_2TC.met")),
- ],
-}
-
-""" Input formats """
-INPUT_FORMATS_AMBI = ["FOA", "HOA2", "HOA3"]
-INPUT_FORMATS_MC = ["MONO", "STEREO", "5_1", "5_1_2", "5_1_4", "7_1", "7_1_4"]
-INPUT_FORMATS_ISM = ["ISM1", "ISM2", "ISM3", "ISM4"]
-INPUT_FORMATS_MASA = ["MASA1", "MASA2"]
-INPUT_FORMATS_OMASA = [
- "ISM1MASA1",
- "ISM2MASA1",
- "ISM3MASA1",
- "ISM4MASA1",
- "ISM1MASA2",
- "ISM2MASA2",
- "ISM3MASA2",
- "ISM4MASA2",
-]
-INPUT_FORMATS_OSBA = [
- "ISM1SBA1",
- "ISM1SBA2",
- "ISM1SBA3",
- "ISM2SBA1",
- "ISM2SBA2",
- "ISM2SBA3",
- "ISM3SBA1",
- "ISM3SBA2",
- "ISM3SBA3",
- "ISM4SBA1",
- "ISM4SBA2",
- "ISM4SBA3",
-]
-
-
-""" Non binaural / parametric output formats """
-OUTPUT_FORMATS = [
- "MONO",
- "STEREO",
- "5_1",
- "5_1_2",
- "5_1_4",
- "7_1",
- "7_1_4",
- "FOA",
- "HOA2",
- "HOA3",
-]
-
-""" Custom loudspeaker input/output """
-CUSTOM_LS_TO_TEST = [
- "t_design_4",
- "16ch_8+4+4",
-]
-
-""" Mixed scene ( metadata ) rendering """
-METADATA_SCENES_TO_TEST = ["mixed_scene", "mixed_scene_simple"]
-METADATA_SCENES_TO_TEST_NO_BE = ["masa_scene"]
-METADATA_SCENES_TO_TEST_MASA_PREREND = ["mixed_10sec_mc714_foa_masa1_ism4", "mixed_10sec_mc714_foa_masa2_ism4"]
-
-""" Binaural rendering """
-OUTPUT_FORMATS_BINAURAL = ["BINAURAL", "BINAURAL_ROOM_IR", "BINAURAL_ROOM_REVERB"]
-HR_TRAJECTORIES_TO_TEST = [
- "full_circle_in_15s",
- # "rotate_yaw_pitch_roll1",
-]
-
-""" Frame Size """
-FRAMING_TO_TEST = ["5ms", "20ms"]
-
-PEAQ_SUPPORTED_FMT = [
- "MONO",
- "STEREO",
- "BINAURAL",
- "BINAURAL_ROOM_IR",
- "BINAURAL_ROOM_REVERB",
-]
diff --git a/tests/renderer_short/cut/.gitignore b/tests/renderer_short/cut/.gitignore
deleted file mode 100644
index f935021a8f8a7bd22f9d6703cafa5134bb6a57f8..0000000000000000000000000000000000000000
--- a/tests/renderer_short/cut/.gitignore
+++ /dev/null
@@ -1 +0,0 @@
-!.gitignore
diff --git a/tests/renderer_short/ref/.gitignore b/tests/renderer_short/ref/.gitignore
deleted file mode 100644
index f935021a8f8a7bd22f9d6703cafa5134bb6a57f8..0000000000000000000000000000000000000000
--- a/tests/renderer_short/ref/.gitignore
+++ /dev/null
@@ -1 +0,0 @@
-!.gitignore
diff --git a/tests/renderer_short/utils.py b/tests/renderer_short/utils.py
deleted file mode 100644
index eb5a1f5bc163cbe812f700e812154ea2d5cc731b..0000000000000000000000000000000000000000
--- a/tests/renderer_short/utils.py
+++ /dev/null
@@ -1,801 +0,0 @@
-#!/usr/bin/env python3
-
-"""
-(C) 2022-2026 IVAS codec Public Collaboration with portions copyright Dolby International AB, Ericsson AB,
-Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V., Huawei Technologies Co. LTD.,
-Koninklijke Philips N.V., Nippon Telegraph and Telephone Corporation, Nokia Technologies Oy, Orange,
-Panasonic Holdings Corporation, Qualcomm Technologies, Inc., VoiceAge Corporation, and other
-contributors to this repository. All Rights Reserved.
-
-This software is protected by copyright law and by international treaties.
-The IVAS codec Public Collaboration consisting of Dolby International AB, Ericsson AB,
-Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V., Huawei Technologies Co. LTD.,
-Koninklijke Philips N.V., Nippon Telegraph and Telephone Corporation, Nokia Technologies Oy, Orange,
-Panasonic Holdings Corporation, Qualcomm Technologies, Inc., VoiceAge Corporation, and other
-contributors to this repository retain full ownership rights in their respective contributions in
-the software. This notice grants no license of any kind, including but not limited to patent
-license, nor is any license granted by implication, estoppel or otherwise.
-
-Contributors are required to enter into the IVAS codec Public Collaboration agreement before making
-contributions.
-
-This software is provided "AS IS", without any express or implied warranties. The software is in the
-development stage. It is intended exclusively for experts who have experience with such software and
-solely for the purpose of inspection. All implied warranties of non-infringement, merchantability
-and fitness for a particular purpose are hereby disclaimed and excluded.
-
-Any dispute, controversy or claim arising under or in relation to providing this software shall be
-submitted to and settled by the final, binding jurisdiction of the courts of Munich, Germany in
-accordance with the laws of the Federal Republic of Germany excluding its conflict of law rules and
-the United Nations Convention on Contracts on the International Sales of Goods.
-"""
-
-import errno
-import filecmp
-import logging
-import os
-import re
-import subprocess as sp
-import sys
-import tempfile
-from pathlib import Path
-from typing import Dict, Optional, Union
-
-import numpy as np
-import pytest
-
-from ..constants import CAT_NORMAL
-from .compare_audio import compare_audio_arrays
-from .constants import (
- BIN_SUFFIX_MERGETARGET,
- FORMAT_TO_FILE_COMPARETEST,
- FORMAT_TO_FILE_LTV,
- FORMAT_TO_FILE_SMOKETEST,
- FORMAT_TO_METADATA_FILES,
- FORMAT_TO_METADATA_FILES_LTV,
- LTV_DIR,
- OUTPUT_PATH_CUT,
- OUTPUT_PATH_REF,
- PEAQ_SUPPORTED_FMT,
- RENDERER_CMD,
- SCRIPTS_DIR,
-)
-
-sys.path.append(SCRIPTS_DIR)
-from pyaudio3dtools.audiofile import readfile
-
-from ..cmp_pcm import cmp_pcm
-from ..conftest import get_split_idx, parse_properties
-
-
-def _run_cmd(cmd, test_info=None, env=None):
- """
- Helper function for running some command.
- Raises a SystemError if either the return code is non-zero or a USAN printout is detected
- """
- proc = sp.run(cmd, capture_output=True, text=True, env=env)
- stdout = proc.stdout + proc.stderr
-
- # check for USAN error first
- if "UndefinedBehaviorSanitizer" in stdout:
- error = f"USAN error detected in stdout of command: {' '.join(cmd)}\n{stdout}"
- if test_info is not None:
- test_info.error = error
- raise SystemError(error)
-
- # then handle possible crash
- try:
- proc.check_returncode()
- except sp.CalledProcessError as e:
- error = f"Command returned non-zero exit status ({e.returncode}): {' '.join(e.cmd)}\n{e.stderr}\n{e.stdout}"
- if test_info is not None:
- test_info.error = error
- raise SystemError(error)
-
-
-def run_cmd(cmd, test_info=None, env=None):
- logging.info(f"\nRunning command\n{' '.join(cmd)}\n")
- _run_cmd(cmd, test_info=test_info, env=env)
-
-
-def run_isar_ext_rend_cmd(cmd, test_info=None, env=None):
- logging.info(f"\nRunning ISAR EXT REND command\n{' '.join(cmd)}\n")
- _run_cmd(cmd, test_info=test_info, env=env)
-
-
-def run_ivas_isar_enc_cmd(cmd, test_info=None, env=None):
- logging.info(f"\nRunning IVAS ISAR encoder command\n{' '.join(cmd)}\n")
- _run_cmd(cmd, test_info=test_info, env=env)
-
-
-def run_ivas_isar_dec_cmd(cmd, test_info=None, env=None):
- if BIN_SUFFIX_MERGETARGET in cmd[0]:
- logging.info(f"\nREF decoder command:\n\t{' '.join(cmd)}\n")
- else:
- logging.info(f"\nDUT decoder command:\n\t{' '.join(cmd)}\n")
- _run_cmd(cmd, test_info=test_info, env=env)
-
-
-def run_isar_post_rend_cmd(cmd, test_info=None, env=None):
- logging.info(f"\nRunning ISAR post renderer command\n{' '.join(cmd)}\n")
- _run_cmd(cmd, test_info=test_info, env=env)
-
-
-def check_BE(
- test_info,
- ref: np.ndarray,
- ref_fs: int,
- cut: np.ndarray,
- cut_fs: int,
- atol: int = 2,
-) -> tuple:
- if ref is None or np.array_equal(ref, np.zeros_like(ref)):
- pytest.fail("REF signal does not exist or is zero!")
-
- if cut is None or np.array_equal(cut, np.zeros_like(cut)):
- pytest.fail("CuT signal does not exist or is zero!")
-
- snr, gain_b, max_diff = compare_audio_arrays(ref, ref_fs, cut, cut_fs)
-
- if np.isnan(snr) or gain_b == 0:
- pytest.fail("Invalid comparison result, check your signals!")
-
- if ref.shape[0] < cut.shape[0]:
- ref = np.pad(ref, [(0, cut.shape[0] - ref.shape[0]), (0, 0)])
- elif ref.shape[0] > cut.shape[0]:
- cut = np.pad(cut, [(0, ref.shape[0] - cut.shape[0]), (0, 0)])
-
- # check max_diff as well, since compare_audio_arrays will try to adjust for small delay differences
- diff_found = not np.allclose(ref, cut, rtol=0, atol=atol) and max_diff > atol
-
- return diff_found, snr, gain_b, max_diff
-
-
-def run_renderer(
- record_property,
- props_to_record,
- test_info,
- in_fmt: str,
- out_fmt: str,
- metadata_input: Optional[str] = None,
- in_meta_files: Optional[list] = None,
- trj_file: Optional[str] = None,
- non_diegetic_pan: Optional[str] = None,
- name_extension: Optional[str] = None,
- refrot_file: Optional[str] = None,
- refvec_file: Optional[str] = None,
- refveclev_file: Optional[str] = None,
- config_file: Optional[str] = None,
- binary_suffix: str = "",
- frame_size: Optional[str] = "20ms",
- hrtf_file: Optional[str] = None,
- get_mld=False,
- mld_lim=0,
- get_mld_lim=0,
- abs_tol=0,
- get_ssnr=False,
- get_odg=False,
- get_odg_bin=False,
- aeid: Optional[Union[Path, int]] = None,
- in_file=None,
- out_file=None,
- sr="48kHz",
- render_for_peaq=False,
- split_comparison=False,
-) -> str:
- # prepare arguments and filepaths
- if trj_file is not None:
- trj_name = f"_{trj_file.stem}"
- else:
- trj_name = ""
-
- if refrot_file is not None:
- refrot_name = f"_{refrot_file.stem}"
- else:
- refrot_name = ""
-
- if refvec_file is not None:
- refvec_name = f"_{refvec_file.stem}"
- else:
- refvec_name = ""
-
- if refveclev_file is not None:
- refveclev_name = f"_{refveclev_file.stem}"
- else:
- refveclev_name = ""
-
- if config_file is not None:
- config_name = f"_{config_file.stem}"
- else:
- config_name = ""
-
- if frame_size:
- framing_name = f"_{frame_size}"
- else:
- framing_name = ""
-
- if aeid is not None:
- if isinstance(aeid, Path):
- aeid_name = f"_{aeid.stem}"
- else:
- aeid_name = aeid
- else:
- aeid_name = ""
-
- if not isinstance(out_fmt, str):
- out_name = f"{out_fmt.stem}"
- else:
- out_name = out_fmt
-
- if hrtf_file is not None:
- hrtf_file_name = f"_{hrtf_file.stem}"
- else:
- hrtf_file_name = ""
-
- if test_info.config.option.create_ref:
- output_path_base = OUTPUT_PATH_REF
- else:
- output_path_base = OUTPUT_PATH_CUT
-
- # if in REF or CUT creation mode use the comparetestv
- if test_info.config.option.create_ref or test_info.config.option.create_cut:
- format_to_file = FORMAT_TO_FILE_COMPARETEST
- else:
- format_to_file = FORMAT_TO_FILE_SMOKETEST
-
- format_to_metadata_files = FORMAT_TO_METADATA_FILES
-
- if test_info.config.option.use_ltv:
- if test_info.config.option.ltv_dir:
- format_to_file = dict()
- format_to_metadata_files = dict()
- for k, v in FORMAT_TO_FILE_LTV.items():
- format_to_file[k] = str(v).replace(
- str(LTV_DIR), str(test_info.config.option.ltv_dir)
- )
- for k, v in FORMAT_TO_METADATA_FILES_LTV.items():
- format_to_file[k] = str(v).replace(
- str(LTV_DIR), str(test_info.config.option.ltv_dir)
- )
- else:
- format_to_file = FORMAT_TO_FILE_LTV
- format_to_metadata_files = FORMAT_TO_METADATA_FILES_LTV
-
- if in_file is None:
- if metadata_input is not None:
- in_file = metadata_input
- in_name = metadata_input.stem
- elif not isinstance(in_fmt, str):
- in_file = format_to_file[in_fmt.stem]
- in_name = in_fmt.stem
- else:
- in_file = format_to_file[in_fmt]
- in_name = in_fmt
- in_file = str(in_file).replace("48", sr[:2])
-
- if in_meta_files is None and in_fmt in format_to_metadata_files:
- in_meta_files = format_to_metadata_files[in_fmt]
-
- # If metadata not given with ISM input, use default NULL
- if in_meta_files is None and isinstance(in_fmt, str) and "ism" in in_fmt.lower():
- match = re.search(r"ism(\d)", in_fmt.lower())
- assert match is not None
- num_obj = int(match[1])
- in_meta_files = ["NULL"] * num_obj
-
- if out_file is None:
- out_file_stem = f"{in_name}_to_{out_name}{trj_name}{non_diegetic_pan}{refrot_name}{refvec_name}{refveclev_name}{config_name}{framing_name}{hrtf_file_name}{name_extension}{aeid_name}_{sr}.wav"
- out_file = str(output_path_base.joinpath(out_file_stem))
-
- cmd = RENDERER_CMD[:]
- cmd[2] = str(in_file)
- cmd[4] = str(in_fmt)
- cmd[6] = str(out_file)
- cmd[8] = str(out_fmt)
- cmd[10] = str(sr[:2])
-
- if test_info.config.option.create_ref:
- cmd[0] += BIN_SUFFIX_MERGETARGET
- cmd[0] += binary_suffix
-
- if in_meta_files is not None:
- cmd.extend(["-im", *in_meta_files])
-
- if trj_file is not None:
- cmd.extend(["-T", str(trj_file)])
-
- if hrtf_file is not None:
- cmd.extend(["-hrtf", str(hrtf_file)])
-
- if non_diegetic_pan is not None:
- cmd.extend(["-non_diegetic_pan", str(non_diegetic_pan)])
- if refrot_file is not None:
- cmd.extend(["-rf", str(refrot_file)])
- cmd.extend(["-otr", "ref"])
-
- if refvec_file is not None:
- cmd.extend(["-rvf", str(refvec_file)])
- cmd.extend(["-otr", "ref_vec"])
-
- if refveclev_file is not None:
- cmd.extend(["-rvf", str(refveclev_file)])
- cmd.extend(["-otr", "ref_vec_lev"])
-
- if config_file is not None:
- cmd.extend(["-render_config", str(config_file)])
-
- if frame_size:
- cmd.extend(["-fr", str(frame_size.replace("ms", ""))])
-
- if aeid is not None:
- cmd.extend(["-aeid", str(aeid)])
-
- # Set env variables for UBSAN
- env = os.environ.copy()
- if test_info.node.name and "UBSAN_OPTIONS" in env.keys():
- env["UBSAN_OPTIONS"] = (
- env["UBSAN_OPTIONS"] + f",log_path=usan_log_{test_info.node.name}"
- )
-
- testcase_props = {
- "format": "Renderer",
- "category": CAT_NORMAL,
- }
-
- if record_property is not None:
- for k, v in testcase_props.items():
- record_property(k, v)
-
- # run the renderer
- run_cmd(cmd, test_info, env)
-
- if test_info.config.option.create_cut and not render_for_peaq:
- # CUT creation mode will run a comparison with REF
- out_file_ref = str(OUTPUT_PATH_REF.joinpath(out_file_stem))
-
- # Check if we need to render to mono, stereo or binaural for PEAQ comparison
- odg_input = None
- odg_test = None
- odg_ref = None
- if get_odg_bin:
- odg_input = out_file_ref[0:-4] + ".INPUT.BINAURAL.wav"
- odg_test = str(out_file)[0:-4] + ".BINAURAL.wav"
- odg_ref = out_file_ref[0:-4] + ".BINAURAL.wav"
-
- if out_fmt not in PEAQ_SUPPORTED_FMT:
- if in_fmt in PEAQ_SUPPORTED_FMT:
- new_fmt = in_fmt # MONO or STEREO
- else:
- # If input is META which contains stereo, new_fmt needs to be STEREO.
- if in_fmt == "META":
- with open(in_file, "r") as scene:
- if "STEREO" in scene.read():
- new_fmt = "STEREO"
- else:
- new_fmt = "BINAURAL"
- else:
- new_fmt = "BINAURAL"
-
- # Render test to PEAQ supported format (MONO, STEREO or BINAURAL)
- cmd2 = RENDERER_CMD[:]
- cmd2[2] = str(out_file) # in_file
- cmd2[4] = str(out_fmt) # in_fmt
- cmd2[6] = odg_test # out_file
- cmd2[8] = new_fmt # out_fmt
- cmd2[10] = str(sr[:2])
- cmd2[0] += BIN_SUFFIX_MERGETARGET # Use IVAS_rend_ref for re-rendering
- cmd2[0] += binary_suffix
- if "MASA" in str(out_fmt):
- cmd2.extend(["-im", out_file + ".met"])
- run_cmd(cmd2, test_info, env)
-
- # Render ref to BINAURAL with same settings as test
- cmd2[2] = str(out_file_ref) # in_file
- cmd2[6] = odg_ref # out_file
- run_cmd(cmd2, test_info, env)
- out_fmt_bin = new_fmt
- else:
- out_fmt_bin = out_fmt
- odg_test = out_file
- odg_ref = out_file_ref
-
- if out_fmt_bin != in_fmt:
- # Render input to match out_fmt_bin using same config as input, but with IVAS_rend_ref
- cmd[0] += BIN_SUFFIX_MERGETARGET
- cmd[0] += binary_suffix
- cmd[6] = odg_input # out_file
- cmd[8] = out_fmt_bin # out_fmt
- run_cmd(cmd, test_info, env)
- else:
- odg_input = in_file
-
- ### run the comparison tools
- split_idx = np.empty(0)
- prop_suffix = [""]
-
- # 1. run comparison on whole files - this is done always, regardless of the presence of --split_comparison
- ref_fs = int(cmd[10]) * 1000
- output_differs_parts, reason_parts = cmp_pcm(
- out_file_ref,
- out_file,
- out_fmt,
- ref_fs,
- get_mld=get_mld,
- mld_lim=get_mld_lim,
- abs_tol=abs_tol,
- get_ssnr=get_ssnr,
- get_odg=get_odg,
- get_odg_bin=get_odg_bin,
- odg_input=odg_input,
- odg_test=odg_test,
- odg_ref=odg_ref,
- mld_playback_level=test_info.config.option.mld_playback_level,
- split_idx=split_idx,
- )
-
- # 2. run comparison on split files if --split_comparison is given
- # for JBM cases, comparison will fail because of length mismatch beetween split wav files and tracefiles
- # -> skip split comparison for these cases
- if split_comparison:
- split_idx = get_split_idx(str(Path(in_file).stem), ref_fs // 1000)
-
- # this extra if takes care of cases where no splits are found, e.g. the "NOOP" case in the self_test_ltv prm file
- # if this would not be there, then the comparison of the whole file would run twice
- if len(split_idx) > 0:
- output_differs_splits, reason_splits = cmp_pcm(
- out_file_ref,
- out_file,
- out_fmt,
- ref_fs,
- get_mld=get_mld,
- mld_lim=get_mld_lim,
- abs_tol=abs_tol,
- get_ssnr=get_ssnr,
- get_odg=get_odg,
- get_odg_bin=get_odg_bin,
- odg_input=odg_input,
- odg_test=odg_test,
- odg_ref=odg_ref,
- mld_playback_level=test_info.config.option.mld_playback_level,
- split_idx=split_idx,
- )
- output_differs_parts += output_differs_splits
- reason_parts += reason_splits
-
- if split_comparison:
- prop_suffix = ["_whole"] + [
- f"_split{i:03d}" for i in range(1, len(split_idx) + 1)
- ]
-
- for output_differs, reason, suffix in zip(
- output_differs_parts, reason_parts, prop_suffix
- ):
- result_props = parse_properties(
- reason, output_differs, props_to_record, suffix
- )
- for k, v in result_props.items():
- record_property(k, v)
-
- if output_differs_parts[0]:
- logging.error(f"Command line was: {' '.join(cmd)}")
- pytest.fail(f"Output differs: ({reason_parts[0]})")
-
- # compare metadata files in case of MASA prerendering
- if "MASA" in str(out_fmt):
- meta_file_ref = out_file_ref + ".met"
- meta_file_cut = out_file + ".met"
- if not filecmp.cmp(meta_file_cut, meta_file_ref):
- pytest.fail("Metadata file differs from reference")
-
- return out_file
-
-
-def compare_renderer_args(
- record_property,
- props_to_record,
- test_info,
- in_fmt,
- out_fmt,
- ref_kwargs: Dict,
- cut_kwargs: Dict,
- split_comparison=False,
- atol=2,
-):
- out_file_ref = run_renderer(
- record_property,
- props_to_record,
- test_info,
- in_fmt,
- out_fmt,
- **ref_kwargs,
- split_comparison=split_comparison,
- )
- ref, ref_fs = readfile(out_file_ref)
- out_file_cut = run_renderer(
- record_property,
- props_to_record,
- test_info,
- in_fmt,
- out_fmt,
- **cut_kwargs,
- split_comparison=split_comparison,
- )
- cut, cut_fs = readfile(out_file_cut)
- [diff_found, snr, gain_b, max_diff] = check_BE(
- test_info, ref, ref_fs, cut, cut_fs, atol
- )
- if diff_found:
- pytest.fail(
- f"CuT not BE to REF! SNR : {snr:3.2f} dB, Gain CuT: {gain_b:1.3f}, Max Diff = {int(max_diff)}"
- )
-
-
-def binauralize_input_and_output(
- record_property,
- props_to_record,
- test_info,
- input_file,
- dut_output_file,
- ref_output_file,
- in_fmt,
- output_config,
- enc_opts,
- dec_opts,
- in_sr,
- out_sr,
-):
- # Use current folder as location for temporary directory, since scene description does not handle spaces in path
- with tempfile.TemporaryDirectory(dir=".") as tmp_dir:
- tmp_dir = Path(tmp_dir)
- scene_dut = str(tmp_dir.joinpath("scene_dut.txt"))
- scene_ref = str(tmp_dir.joinpath("scene_ref.txt"))
- scene_in = str(tmp_dir.joinpath("scene_in.txt"))
-
- # File names for binauralized signals, if needed
- ref_input_file_binaural = ref_output_file[0:-4] + ".INPUT.BINAURAL.wav"
- dut_output_file_binaural = dut_output_file[0:-4] + ".BINAURAL.wav"
- ref_output_file_binaural = ref_output_file[0:-4] + ".BINAURAL.wav"
-
- # Identify metadata
- in_meta_files = [
- str(SCRIPTS_DIR.joinpath(m)) if m != "NULL" else m
- for m in re.findall(r"\b\S+\.csv|NULL\b", enc_opts)
- ] # All .csv or NULL files in enc_opts are ISM metadata files.
- n_obj = len(in_meta_files)
-
- # If extended metadata is not used, strip the metadata for the external renderer
- extended_md_used = (
- re.search(r"-ism\s?\+[1-4]", enc_opts)
- and "OMASA" not in in_fmt
- and "OSBA" not in in_fmt
- )
- if not extended_md_used and n_obj > 0:
- truncated_meta_files = []
- for md in in_meta_files:
- if md != "NULL":
- md_out_file = str(tmp_dir.joinpath(os.path.basename(md)))
- with open(md_out_file, "w") as fp_out, open(md, "r") as fp_in:
- for line in fp_in:
- fp_out.write(
- ",".join(line.split(",")[:2]) + "\n"
- ) # Keep only first two elements: azim, elev
- else:
- md_out_file = "NULL" # Cannot truncate NULL, just insert it without modification
- truncated_meta_files.append(md_out_file)
- in_meta_files = truncated_meta_files
-
- in_meta_files = in_meta_files + [
- str(SCRIPTS_DIR.joinpath(m)) for m in re.findall(r"\b\S+\.met\b", enc_opts)
- ] # All .met files in enc_opts are MASA metadata files.
- out_meta_files = None
- if output_config == "EXT":
- out_meta_files = []
- if n_obj > 0:
- out_meta_files = out_meta_files + [
- f"{dut_output_file}.{i}.csv" for i in range(0, n_obj)
- ]
- if "MASA" in in_fmt:
- out_meta_files = out_meta_files + [f"{dut_output_file}.met"]
- if output_config == "EXT":
- output_config = in_fmt
- if output_config == "":
- output_config = "MONO" # EVS mono
- metadata_input = None
-
- if "OSBA" in in_fmt or "OMASA" in in_fmt:
- scene_description_file(in_fmt, scene_in, n_obj, input_file, in_meta_files)
- input_file = scene_in
- in_meta_files = None
- in_fmt = "META"
-
- if "OSBA" in output_config or "OMASA" in output_config:
- if "OSBA" in output_config:
- output_config = (
- output_config[:-1] + "3"
- ) # Temporary fix to handle than IVAS_dec produces HOA3 for all OSBA configs. Needs to be removed when this fix is ported to BASOP.
- scene_description_file(
- output_config, scene_dut, n_obj, dut_output_file, out_meta_files
- )
- dut_output_file = scene_dut
- scene_description_file(
- output_config, scene_ref, n_obj, ref_output_file, out_meta_files
- )
- ref_output_file = scene_ref
- out_meta_files = None
- output_config = "META"
-
- # Identify headtracking and orientation trajectories
- trj_file = findstr(r"-t\s+(\S+)", dec_opts)
- non_diegetic_pan = findstr(r"-non_diegetic_pan\s+(\S+)", dec_opts)
- if non_diegetic_pan is not None:
- output_config = "STEREO"
- name_extension = None
- refrot_file = findstr(r"-rf\s+(\S+)", dec_opts)
- rot_tmp_file = findstr(r"-rvf\s+(\S+)", dec_opts)
- refveclev_file = None
- refvec_file = None
- if "-otr ref_vec_lev".upper() in dec_opts.upper():
- refveclev_file = rot_tmp_file
- else:
- if "-otr ref_vec".upper() in dec_opts.upper():
- refvec_file = rot_tmp_file
-
- # Rendering configuration
- config_file = findstr(r"-render_config\s+(\S+)", dec_opts)
- binary_suffix = "_ref"
- frame_size = findstr(r"-fr\s+(\S+)", dec_opts)
- # hrtf_file = findstr(r'-hrtf\s+(\S+)', dec_opts)
- hrtf_file = None # Default HRTFs used for binaural rendering of output
-
- aeid = findstr(r"-aeid\s+(\S+)", dec_opts)
-
- if output_config.upper() not in PEAQ_SUPPORTED_FMT:
- # Render output to BINAURAL
- output_reformat = "BINAURAL"
-
- check_and_makedir(str(Path(dut_output_file_binaural).parent))
-
- run_renderer(
- record_property,
- props_to_record,
- test_info,
- output_config,
- output_reformat,
- metadata_input,
- out_meta_files,
- trj_file,
- non_diegetic_pan,
- name_extension,
- refrot_file,
- refvec_file,
- refveclev_file,
- config_file,
- binary_suffix,
- frame_size,
- hrtf_file,
- aeid,
- in_file=dut_output_file,
- out_file=dut_output_file_binaural,
- sr=out_sr,
- render_for_peaq=True,
- )
-
- check_and_makedir(str(Path(ref_output_file_binaural).parent))
-
- run_renderer(
- record_property,
- props_to_record,
- test_info,
- output_config,
- output_reformat,
- metadata_input,
- out_meta_files,
- trj_file,
- non_diegetic_pan,
- name_extension,
- refrot_file,
- refvec_file,
- refveclev_file,
- config_file,
- binary_suffix,
- frame_size,
- hrtf_file,
- aeid,
- in_file=ref_output_file,
- out_file=ref_output_file_binaural,
- sr=out_sr,
- render_for_peaq=True,
- )
-
- # Update output_config to rendered format
- output_config = output_reformat
- else:
- # Signal already mono, stereo or binaural
- dut_output_file_binaural = dut_output_file
- ref_output_file_binaural = ref_output_file
-
- if in_fmt.upper() != output_config.upper():
- # Render input to match output_config
- out_fmt = output_config
-
- check_and_makedir(str(Path(ref_input_file_binaural).parent))
-
- run_renderer(
- record_property,
- props_to_record,
- test_info,
- in_fmt,
- out_fmt,
- metadata_input,
- in_meta_files,
- trj_file,
- non_diegetic_pan,
- name_extension,
- refrot_file,
- refvec_file,
- refveclev_file,
- config_file,
- binary_suffix,
- frame_size,
- hrtf_file,
- aeid,
- in_file=input_file,
- out_file=ref_input_file_binaural,
- sr=in_sr,
- render_for_peaq=True,
- )
- else:
- ref_input_file_binaural = input_file
- return (
- ref_input_file_binaural,
- dut_output_file_binaural,
- ref_output_file_binaural,
- )
-
-
-def findstr(exp, s, one_element=True):
- result = [SCRIPTS_DIR.joinpath(x) for x in re.findall(exp, s)]
- if len(result) == 0:
- return None
- if one_element:
- return result[0]
- return result
-
-
-def check_and_makedir(dir_path):
- if not os.path.exists(dir_path):
- try:
- os.makedirs(dir_path)
- except OSError as e:
- if e.errno != errno.EEXIST:
- raise # raises the error again
-
-
-def scene_description_file(in_fmt, metadata_tmp, n_obj, input_file, in_meta_files):
- with open(metadata_tmp, "w") as fp_meta:
- currdir = Path(
- metadata_tmp
- ).parent # File names must be relative to config file location
- fp_meta.write(f"{os.path.relpath(input_file, currdir)}\n") # Input file
- fp_meta.write(f"{n_obj + 1}\n") # Number of sources
- for n in range(0, n_obj):
- if in_meta_files[n] == "NULL":
- md_file = "1\n1,0,0" # NULL metadata position: azim=0,elev=0 for 1 frame, looped throughout all frames.
- else:
- md_file = os.path.relpath(in_meta_files[n], currdir)
- fp_meta.write(f"ISM\n{n + 1}\n{md_file}\n") # ISM metadata
- if "OSBA" in in_fmt:
- fp_meta.write(
- "gain_dB:-6\n"
- ) # Set -6 dB on all components for OSBA to match IVAS_dec
- fp_meta.write(f"{in_fmt.split('_')[0][1:]}\n") # SBA or MASA
- fp_meta.write(f"{n_obj + 1}\n")
- fp_meta.write(f"{in_fmt.split('_')[-1]}\n") # SBA or MASA parameter
- if "OMASA" in in_fmt:
- fp_meta.write(
- f"{os.path.relpath(in_meta_files[n_obj], currdir)}\n"
- ) # MASA metadata
- if "OSBA" in in_fmt:
- fp_meta.write(
- "gain_dB:-6\n"
- ) # Set -6 dB on all components for OSBA to match IVAS_dec