docs: add documentation on how hw-ci integration works in this repo
[ci:skip-vercheck] gitlab-ci.yml.j2 files changed Signed-off-by: Pablo Correa Gómez <pabloyoyoista@postmarketos.org> Part-of: <https://gitlab.postmarketos.org/postmarketOS/pmaports/-/merge_requests/7881>
This commit is contained in:
Pablo Correa Gómez
The Friendly Meow (merge) Bot
parent
cb182b59ca
commit
2e12f24b15
2 changed files with 141 additions and 0 deletions
140
docs/hardware-ci.md
Normal file
140
docs/hardware-ci.md
Normal file
|
|
@ -0,0 +1,140 @@
|
|||
# Hardware CI
|
||||
|
||||
Hardware CI includes a lot of moving parts, and as debugging is a lot harder
|
||||
than development, it can be hard to understand which part of the whole system is
|
||||
resulting in failures. This aims to explain the CI setup in pmaports to make it
|
||||
easier to understand the different moving parts.
|
||||
|
||||
## Overall architecture
|
||||
|
||||
The hardware CI consists of the following building blocks:
|
||||
|
||||
* Dynamic generation of jobs: the `.ci/lib/generate_build_jobs.py` is run on CI
|
||||
to detect what changed from the target branch. If there are packages to be
|
||||
built, or changes that should prompt testing on hardware, a complete GitLab
|
||||
pipeline definition is created based on the input template. This generated
|
||||
pipeline is consumed by GitLab to create a [downstream
|
||||
pipeline](https://docs.gitlab.com/ci/pipelines/downstream_pipelines/#dynamic-child-pipelines)
|
||||
and contains all the details about what to build, and the specific process to
|
||||
test each of the relevant devices. If there is nothing to build or test, a
|
||||
dummy pipeline is created instead (see
|
||||
<https://gitlab.com/gitlab-org/gitlab/-/issues/368248>)
|
||||
* Building changed packages: those packages that are changed are built by CI,
|
||||
together with a repo that allows to use `mrtest add` and `mrtest
|
||||
upgrade`. This is independent of any work related to hardware.
|
||||
* Preparation of test artifacts: if any of the packages that were built is a
|
||||
dependency of a DUT, then CI proceeds to create boot artifacts for the DUT
|
||||
using those newly-built packages. This step often makes use of helper
|
||||
functions in
|
||||
[ci-common](https://gitlab.postmarketos.org/postmarketOS/ci-common/-/blob/ci-tron-integration/ci-tron/common.yml).
|
||||
* Definition of tests: how and what to tests for a device is defined by a
|
||||
`gitlab-ci.yml.j2` jinja template under the device package folder. Those
|
||||
templates often include jobs from other places, since tests are usually
|
||||
generic and shared across multiple devices.
|
||||
|
||||
## Add a new device to pmaports
|
||||
|
||||
Adding a new device for testing under pmaports can be done by adding a
|
||||
`gitlab-ci.yml.j2` file in your device package. This file should define all the
|
||||
test jobs for your device and will get included when generating the final
|
||||
pipeline by `generate_build_jobs.py`.
|
||||
|
||||
Your jobs are expected to set all the necessary attributes to run jobs
|
||||
on a CI-tron-provided runner, but the following template variables and
|
||||
hidden jobs are provided to you as helpers:
|
||||
|
||||
- Jinja template variables (defined in `.ci/lib/generate_build_jobs.py`):
|
||||
- `device`: Instance of the `Device` class which represents the
|
||||
device under test.
|
||||
- Hidden jobs (defined in `.ci/build-jobs.yaml.j2`):
|
||||
- `.test-{{ device.name }}`: Defines all the common parameters for
|
||||
testing this device (;
|
||||
- `.device-boot-flow-separate-artifacts`: Sets the kernel
|
||||
and initramfs URLs when the prepare job generates separate kernel and
|
||||
initramfs files as artifacts.
|
||||
- `.device-boot-flow-fastboot-image`: Sets the full fastboot boot image when
|
||||
the prepare job generates a boot image ready for booting into the DUT.
|
||||
|
||||
:::{note}
|
||||
Rather than duplicating the `gitlab-ci.yml.j2` file for all the devices that can
|
||||
make use of it, you may save the file in `.gitlab-ci/packages/` then link to it
|
||||
in the device folder.
|
||||
:::
|
||||
|
||||
## Tests
|
||||
|
||||
There are 3 kind of tests in the postmarketOS hardware CI setup:
|
||||
|
||||
### Kernel tests
|
||||
|
||||
One important part of the hardware CI is to be able to test all the low-level
|
||||
integration and logic. That includes, first and foremost, the kernel and some
|
||||
initramfs functionality.
|
||||
|
||||
The package `postmarketos-mkinitfs-hook-ci` takes care of that. If installed, it
|
||||
will:
|
||||
|
||||
* Run in the initramfs any scripts installed under
|
||||
`/usr/libexec/pmos-tests-initramfs`.
|
||||
* Prints a success or fail so it can be detected by whatever is orchestrating
|
||||
the tests.
|
||||
* Print a message that the execution is done, and halt the boot.
|
||||
|
||||
This allows for any package to provide arbitrary scripts in that location, and
|
||||
have them execute during hardware-CI testing on the initramfs.
|
||||
|
||||
However, most test routines are pretty generic. So instead of every device
|
||||
creating there own, most of them are aggregated as subpackages of the
|
||||
`postmarketos-test` package. Then, if a DUT wants to enable a specific initramfs
|
||||
test, everything it needs is a subpackage that depends on the corresponding
|
||||
tests from `postmarketos-test`, and that gets installed if
|
||||
`postmarketos-mkinitfs-hook-ci` is also installed. This subpackage is, as a
|
||||
convention, named `pmtest`. This is how device package's APKBUILD that enables
|
||||
tests for unl0kr and suspend would look like:
|
||||
|
||||
```sh
|
||||
subpackages="$pkgname-pmtest"
|
||||
|
||||
pmtest() {
|
||||
install_if="$pkgname=$pkgver-r$pkgrel postmarketos-mkinitfs-hook-ci"
|
||||
depends="postmarketos-test-suspend postmarketos-test-unl0kr"
|
||||
|
||||
mkdir -p "$subpkgdir"
|
||||
}
|
||||
```
|
||||
|
||||
### Boot tests
|
||||
|
||||
:::{note}
|
||||
The infrastructure for these tests has not been developed yet.
|
||||
:::
|
||||
|
||||
Boot tests are expected to test the boot sequence. For that, they should run the
|
||||
exact same boot sequence that the DUTs they run on, including an unmodified
|
||||
initramfs. These tests have not yet been developed. These tests could test
|
||||
things like checking that the partition detection is correct, or that FDE runs
|
||||
successfully.
|
||||
|
||||
### System tests
|
||||
|
||||
:::{note}
|
||||
The infrastructure for these tests has not been developed yet.
|
||||
:::
|
||||
|
||||
Anything that requires complex user-space integration or deamons should be done
|
||||
on top of the rootfs, not the initramfs. These tests could cover things like
|
||||
checking WiFi or Bluetooth connectivity, or making calls.
|
||||
|
||||
## Debugging
|
||||
|
||||
### Dynamic pipeline generation
|
||||
|
||||
Testing pipelines in pmaports are generated dynamically, so that
|
||||
changing an armv7 devices only builds for that architecture, and does
|
||||
not run ci-tron tests for any other device. The pipeline is generated by
|
||||
the `.ci/lib/generate_build_jobs.py` script. The script requires jinja2
|
||||
and pmbootstrap to be installed, but once they are it can be run passing
|
||||
a jinja template (unless doing changes, it should be
|
||||
`.ci/build-jobs.yaml.j2`) as a first argument, and the output file as a
|
||||
second argument. This way, it is possible to run the code generation
|
||||
locally, and inspect the resulting pipeline.
|
||||
|
|
@ -20,6 +20,7 @@ in pmaports and the processes around device categorization.
|
|||
firmware
|
||||
generic-device-packages
|
||||
generic-kernels
|
||||
hardware-ci
|
||||
kconfig-fragments
|
||||
kconfigcheck
|
||||
kernel-cmdline
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue