Structured tagging (cf. mesa/mesa!33421) captures a checksum of the
thing we think we're building, and verifies this through the chain.
When we run container builds, we check that the tag we've captured in
the CI variables matches the calculated checksum, to make sure the
declared tags are consistent and we always have traceability.
When we run tests, we check the tags again between what was declared in
the CI variables and what we're actually running from the test
container. This makes sure that we're always testing what we think we're
testing.
As a side advantage, the rule inheritance we need to make this work
means that we can start doing more optional downloads via overlays,
instead of pulling a whole container full of stuff we might not ever
use.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34539>
This affects:
- generic jobs (sanity, rustfmt, shader-db, docs, etc.)
- linux image builds
- linux mesa builds
- software renderer tests
- android tests
- virgl & venus tests
Marge pipelines get high priority, nightly pipelines get low priority,
and everything else is in between.
(Hardware test farms have their own mechanisms.)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34264>
That tag was supposed to allow these jobs to run faster, but these
runners are currently having disk issues, and the normal runners look
like they're plenty fast enough (at least right now since almost nobody
runs ci jobs ^^).
We might revert this later, but for now let's merge this to unblock CI.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34120>
Now that every ANGLE use is covered by tag consistency checks
(structured tagging), we don't need the USE_ANGLE flag anymore, because
if we have ANGLE_TAG set, it means that ANGLE is required in this job.
In detail, it means that the test job has inherited ANGLE_TAG from
`.container-builds-angle`.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33421>
Symlinking $CARGO_HOME to /usr/local/bin made rustup uninstaller delete
the entire folder, causing mysterious build errors, so let's do the
traditional .cargo/env sourcing to make rustup available to the rest of
the build scripts.
Also make sure that required scripts run the shell's rcfile to be able
to setup the PATH correctly.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33519>
By having the DUTs download and extract from a CI-Tron artifact, we
deduplicate the downloading of the build artifact across all DUTs from
a CI farm, leading to quicker and more reliable jobs, and lower
bandwidth usage on both FD.o and the CI gateway.
Inside the CI-Tron infra, this should also drastically reduce the job
submission time by removing needless copies (executorctl -> executor,
executor -> S3, S3 -> B2C, and even B2C -> NBD when applicable).
As an additional bonus, the size of install.tar is reduced by virtue of
zstd providing better compression than zip/deflate.
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32927>
debian-testing is the critical path: the shortest possible job to build
exactly what we need to execute on hardware, and nothing else.
debian-build-testing exists to give us better coverage at the expense of
running longer.
Since the only jobs using r300 and Nine, and the only jobs using NVK,
are in post-merge stages which are manually triggered, move these builds
to debian-build-testing. This makes the critical path to those a little
longer, but we do get to make it shorter for everyone else just running
regular Marge jobs.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33287>
Clarify that artifacts from the Android guest are downloaded explicitly
and put them under /mesa-android directory so that they don't clash with
the Linux host artifacts downloaded by gitlab under
"${CI_PROJECT_DIR}/install".
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Since run-pytest.sh uses the debian/x86_64_pyutils container, it's not
necessary to add an additional layer of isolation by creating a virtual
environment for run-pytest.sh.
So stop creating a venv when run-pytest is run in a container, but keep
the option of using a venv to run-pytest.sh locally.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32602>
Currently the python-test job runs pytest and then creates python
related artifacts for use by other jobs. But the python-test job isn't
triggered by changes to the python scripts and libraries. Instead, changes
to python scripts and libraries can currently be merged without running
pytest and, if they do generate errors, they will only appear in ci
later when trying to generate the python artifacts.
Add pytest to the existing linting jobs and expand the rules so that it
will be triggered by changes to python scripts and libraries. Remove
pytest from the job that creates artifacts. This will ensure that the
artifacts are still valid while catching script/library errors before they
are merged.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32340>
Currently the yaml-toml-shell-test extends the python-test job to make
use of the python packages available there. But the python-test also
generates artifacts which yaml-toml-shell-test does not need to do.
Instead of extending the python-test job, avoid extra work by just using
the pyutils container directly.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32340>
Since $RESULTS_DIR is now centrally defined in setup-test-env.sh it's no
longer necessary to manually add a hard-coded results directory for the
b2b-test job results.
This keeps the results directory consistent between b2c-test jobs and lava.
Fixes: 9b6d14aed1 ("ci: Always create results dir from init")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32051>
The `.baremetal-arm64-asan-test` job extends `.baremetal-test` and
`.use-debian/baremetal_arm64_test` but doing this is unnecessary because
all of the arm64-asan jobs also inherit the `.baremetal-test-arm64` job
which does the same thing.
Furthermore when `.baremetal-arm64-asan-test` appears last in the list of
extended jobs, as in the case of the `a630-gles-asan`, the general rules
inherited by `.baremetal-arm64-asan-test` override the more specific
driver rules and prevent the driver job from appearing in the merge
pipeline.
Simply bumping the `.baremetal-arm64-asan-test` job back earlier in the
list of extended jobs would allow the driver rules to take precedence but
then the S3_ARTIFACT_NAME, provided by `.baremetal-arm64-asan-test`, which
is specific for asan builds, is overridden.
By removing the inherited jobs from the `.baremetal-arm64-asan-test`,
this job can continue to provide the asan artifact without interfering
with the driver jobs appearing in the merge pipeline.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31916>
The `.baremetal-arm32-asan-test` job extends `.baremetal-test` and
`.use-debian/baremetal_arm32_test` but doing this is unnecessary because
the only job that extends `.baremetal-arm32-asan-test`, i.e.
`gc2000-gles2-asan`, also indirectly inherits `.baremetal-test-arm32`
which does the same thing.
Duplicating the extended jobs in `.baremetal-arm32-asan-test` can
unnecessarily cause rule overrides and unexpectedly remove jobs from
pipelines.
Remove the inherited jobs from the `.baremetal-arm32-asan-test` job so
that (when the necessary farm is enabled) `gc2000-gles2-asan` will appear
in pipelines as expected.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31916>
