Refer to the [IVAS Wiki](https://forge.3gpp.org/rep/ivas-codec-pc/ivas-codec/-/wikis/home) for more info on the CI setup.
This repository is mainly meant to share scripts and configs between the two repositories and (for BASOP) between differently configured branches.
This repository is mainly meant to share scripts and configs between the two repositories and between differently configured branches.
## Notes about usage of the snippets
## Structure
``` bash
.
├── includes # folder for (possibly shared) yaml files which are included in the main ones
│ ├── default-variables-basop.yml
│ ├── default-variables.yml
│ ├── job-templates.yml
│ ├── rules-basop.yml
│ └── rules.yml
├── main-basop.yml # CI config for ivas-basop repo
├── main.yml # shared CI config
├── ...
└── snippets # Folder for scripts used in CI only. Shared ones go in the folder directly. Can be small ones or bigger scripts which implement a whole job
├── basop
│ ├── ...
├── float
│ └── ...
└── ...
```
## Usage
For both repositories (ivas-codec and ivas-basop), there should be one main yml file in here which includes stuff from `includes`.
Ideally, there is no other CI code in the respective repositories except a minimal `.gitlab-ci.yml` file which includes from here (currently only achieved for ivas-basop), e.g. (from ivas-basop):
``` yaml
variables:
# note: GitLab cannot reference variables defined by users in the include ref:, we need to use a YAML anchor for this
# see https://docs.gitlab.com/ci/yaml/includes/#use-variables-with-include for more information
IVAS_CODEC_CI_REF:&IVAS_CODEC_CI_REFmain
# all CI code and config is included from https://forge.3gpp.org/rep/ivas-codec-pc/ivas-codec-ci
include:
-project:ivas-codec-pc/ivas-codec-ci
ref:*IVAS_CODEC_CI_REF
file:main-basop.yml
```
This way, any CI change can be developed centrally in this repository and does not need to be merged/ported to different repositories/branches (e.g. keeping ivas-basop main and ivas-float-update in synch).
### Adding new CI code
1. Implement your new CI code on a branch in this repo, e.g. `add-awesome-test` and ideally add a merge request for easier review.
2. Test it by opening a branch in the respective ivas-* repository and changing IVAS_CODEC_CI_REF to your WIP branch, so the yaml there should read:
3. If your change affects multiple repos/branches, you might need to use multiple test branches
4. Once you are done and review is complete, merge `add-awesome-test` to `main` here. The test branches in the other repos are not needed and SHOULD NOT BE MERGED.
### Overwriting CI variables in the ivas-* repos' CI configs
Usually, everything should be done in the files here, but if there is the need to still overwrite some configuration/control CI variables - e.g. to switch off something temporarily or in a branch-specific manner - this should be done by in the variables: section at the top of the `.gitlab-ci.yml` file.
Example: The variable `DISABLE_HRTF` is used in ivas-basop to disable all testcase that use external HRTF files.
The relevant part in the `.gitlab-ci.yml` file in ivas-basop looks like given below.
In practice, one would uncomment the last three lines to switch off the HRTF testcase in the respective branch only.
``` yaml
variables:
# note: GitLab cannot reference variables defined by users in the include ref:, we need to use a YAML anchor for this
# see https://docs.gitlab.com/ci/yaml/includes/#use-variables-with-include for more information
IVAS_CODEC_CI_REF:&IVAS_CODEC_CI_REFmain
# If you need to set some config variable only in a local branch, then add an overwrite here
# One example is DISABLE_HRTF - this will be set on a branch which is about to be merged and will be removed in a subsequent second MR
# this is more easily done directly here in the child repo
# these lines are suposed to stay commented out to serve as an example
# # set this to true to skip the external HRTF testcases in pytest calls
# DISABLE_HRTF: "true"
```
Useful configuration variables:
-`DISABLE_HRTF` - adds `-k not model` to `PYTEST_ADDOPTS` which skips the testcases with -hrtf in the selftest. Probably more for historic reasons.
-`BASOP_CI_SCRIPTS_BRANCH` - used to change the branch from which testscripts are pulled, see [here](https://forge.3gpp.org/rep/ivas-codec-pc/ivas-codec/-/wikis/Software-development/Test-Setup-For-BASOP-Code#how-do-i-check-that-my-new-testcase-does-work-for-basop) for more info.
### Some guidelines/hints
- use only one include section at the top of the main yml file to keep things clear, remember that later stuff overwrites earlier stuff
- in general prefer adding scripts here over using yaml anchors in a CI config file
- if your script shall modify some environment variable, you need to run it with `source` instead of calling it.
- the merge CI config in the ivas-* repos can be viewed in the pipeline editor under "full configuration" - this is especially useful when making changes which should not functionally affect the CI, then it is helpful to copy the full merged config with and without the changes and compare the files directly or as parsed yaml dict locally.
