@@ -12,14 +12,9 @@ This repository is mainly meant to share scripts and configs between the two rep
``` bash
.
├── includes # folder for (possibly shared) yaml files which are included in the main ones
│ ├── default-variables-basop.yml
│ ├── default-variables.yml
│ ├── job-templates.yml
│ ├── rules-basop.yml
│ └── rules.yml
├── includes # folder for shared yaml files which are included in the main ones
├── main-basop.yml # CI config for ivas-basop repo
├── main.yml # shared CI config
├── main-float.yml # CI config for ivas-basop repo
├── ...
└── snippets # Folder for scripts used in CI only. Shared ones go in the folder directly. Can be small ones or bigger scripts which implement a whole job
├── basop
@@ -32,7 +27,7 @@ This repository is mainly meant to share scripts and configs between the two rep
## Usage
For both repositories (ivas-codec and ivas-basop), there should be one main yml file in here which includes stuff from `includes`.
Ideally, there is no other CI code in the respective repositories except a minimal `.gitlab-ci.yml` file which includes from here (currently only achieved for ivas-basop), e.g. (from ivas-basop):
There is no other CI code in the respective repositories except a minimal `.gitlab-ci.yml` file which includes from here, e.g. (from ivas-basop):
``` yaml
variables:
@@ -47,11 +42,37 @@ include:
file:main-basop.yml
```
This way, any CI change can be developed centrally in this repository and does not need to be merged/ported to different repositories/branches (e.g. keeping ivas-basop main and ivas-float-update in synch).
This way, any CI change can be developed centrally in this repository and does not need to be merged/ported to different repositories/branches.
### Adding new CI code
### Adding a new job
In general, job definitions should go into `main-float.yml` or `main-basop-yml` if they are specific to one of the repos.
Shared jobs should go into `includes/`.
It might also be helpful for bundling stuff together to put repo-specific code into `includes/` as well.
The workflow and rules section for each CI config are located at the top of the respective `main-*.yml` file.
After implementing your job, adapt or add there so that your job runs when desired.
Prefer adding scripts in `snippets/` over yaml anchors.
Also check the [guidelines](#some-guidelines-hints) below.
For shared jobs, it might be needed to check in which project the CI is running.
You can do that by checking the gitlab-predefined `CI_PROJECT_ID` variable against `PROJECT_ID_FLOAT` and/or `PROJECT_ID_BASOP`, e.g.:
``` yaml
script:
-|
if [ "$CI_PROJECT_ID" == "$PROJECT_ID_BASOP" ]; then
1. Implement your new CI code on a branch in this repo, e.g. `add-awesome-test` and ideally add a merge request for easier review.
- Note: your job needs to inherit the `before_script:` section from `.test-job-linux` or `.test-job-windows`, respectively. Without that, the job will not clone this repo (i.e. ivas-codec-ci) and the snippets will be missing. If your job does not make use of any snippet, in theory you can do without this, but as a general rule, you should inherit from one of these. If your job uses a before_script: section as well, see [guidelines section below](#some-guidelines-hints).
2. Test it by opening a branch in the respective ivas-* repository and changing IVAS_CODEC_CI_REF to your WIP branch, so the yaml there should read:
``` yaml
@@ -61,6 +82,21 @@ This way, any CI change can be developed centrally in this repository and does n
3. If your change affects multiple repos/branches, you might need to use multiple test branches
4. Once you are done and review is complete, merge `add-awesome-test` to `main` here. The test branches in the other repos are not needed and SHOULD NOT BE MERGED.
### Some guidelines/hints
- if your new job needs to overwrite the before_script: section, one can use this construct to still get the ci repo cloning:
```yaml
before_script:
-!reference[.test-job-linux,before_script]
-...
```
- use only one include section at the top of the main yml file to keep things clear, remember that later stuff overwrites earlier stuff
- in general prefer adding scripts here over using yaml anchors in a CI config file
- if your script shall modify some environment variable, you need to run it with `source` instead of calling it.
- the merge CI config in the ivas-* repos can be viewed in the pipeline editor under "full configuration" - this is especially useful when making changes which should not functionally affect the CI, then it is helpful to copy the full merged config with and without the changes and compare the files directly or as parsed yaml dict locally.
### Overwriting CI variables in the ivas-* repos' CI configs
Usually, everything should be done in the files here, but if there is the need to still overwrite some configuration/control CI variables - e.g. to switch off something temporarily or in a branch-specific manner - this should be done by in the variables: section at the top of the `.gitlab-ci.yml` file.
-`DISABLE_HRTF` - adds `-k not model` to `PYTEST_ADDOPTS` which skips the testcases with -hrtf in the selftest. Probably more for historic reasons.
-`BASOP_CI_SCRIPTS_BRANCH` - used to change the branch from which testscripts are pulled, see [here](https://forge.3gpp.org/rep/ivas-codec-pc/ivas-codec/-/wikis/Software-development/Test-Setup-For-BASOP-Code#how-do-i-check-that-my-new-testcase-does-work-for-basop) for more info.
### Some guidelines/hints
- use only one include section at the top of the main yml file to keep things clear, remember that later stuff overwrites earlier stuff
- in general prefer adding scripts here over using yaml anchors in a CI config file
- if your script shall modify some environment variable, you need to run it with `source` instead of calling it.
- the merge CI config in the ivas-* repos can be viewed in the pipeline editor under "full configuration" - this is especially useful when making changes which should not functionally affect the CI, then it is helpful to copy the full merged config with and without the changes and compare the files directly or as parsed yaml dict locally.
