With the workflow keyword, we can have more control over how pipelines
are created.
One of the features is to set a variable for the entire pipeline
depending on the rule. These variables would be available for all jobs
manifest and can be used inside job rules, for example.
We can use that to set a variable to enable performance jobs in the
pipeline, both at the YAML and script levels.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21492>
Since the Collabora LAVA update related to the downtime from
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21119, the
LAVA logs from Collabora continued to use the hack for older versions
which digested some control characters, such as carriage returns acting
as newlines, which made it necessary to recover from split lines to make
Gitlab sections work in job logs as expected.
Collabora's LAVA instance now gives a more raw log output. It is
necessary to pay attention to newlines at the end of each log message,
which may cause double newlines when printed with Python built-in
`print` function. I decided to remove the repeating `\n` from the
received log messages to make them transparent to LogFollower users.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8242
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21325>
LavaFarm is a class created to handle the different types of LAVA farms
and their tags in Mesa CI. Since specific jobs may require different
types of LAVA farms to run on, it is essential to determine which farm
the runner is running on to configure the job correctly.
LavaFarm provides an easy-to-use interface for checking the runner tag
and returning the corresponding LAVA farm, making it simple for Mesa CI
to configure jobs appropriately. By adding tests for LavaFarm, the team
can ensure that this class is functioning as expected, allowing for the
smooth execution of Mesa CI jobs on the correct LAVA farm.
The tests ensure that get_lava_farm returns the correct LavaFarm value
when given invalid or valid tags and that it returns LavaFarm.UNKNOWN
when no tag is provided. The tests use Hypothesis strategies to generate
various labels and farms for testing.
Example of use:
```
from lava.utils.lava_farm import LavaFarm, get_lava_farm
lava_farm = get_lava_farm()
if lava_farm == LavaFarm.DUMMY:
# Configure the job for the DUMMY farm
...
elif lava_farm == LavaFarm.COLLABORA:
# Configure the job for the COLLABORA farm
...
elif lava_farm == LavaFarm.KERNELCI:
# Configure the job for the KERNELCI farm
...
else:
# Handle the case where the LAVA farm is unknown
...
```
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21325>
Use requirements.txt and requirements-test.txt to organize better Python
dependencies related to LAVA.
Now LAVA tooling can use recent and fixed library versions.
And test-related libs will not trigger container rebuilding anymore.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21325>
They got accidentally disabled entirely, so they didn't block merge, but
once they re-enable then they'll block us again. The problem was that I
moved allow_failure to a .performance-rules section, but we only ever
inherit the rules from that location, not the rest of yml.
This is basically a revert of 67547a04b6 ("ci: Move the performance
jobs' allow_failure:true to the gl rules."), though I still keep the
allow_failure in a more common location with comments, since perf jobs are
a huge trap.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21002>
meson tests sharing a binary (and deviating in their env/args) will
produce temporary logs to the same directory, which is assumed to exist
only for the duration of a single test. This is problematic when running
tests in parallel, as one test may remove the directory before the
other(s) finish, causing a test flake.
This appends the each test's pid to the output directory to enforce
uniqueness and avoid the race.
Signed-off-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20779>
We have a common pain point with fractional CTS coverage, where the test
list changes on a CTS uprev or board load rebalancing, so you get a
different subset of tests run. The dev updates the list of xfails (a
pain), but also we end up with xfails left behind that aren't tested any
more and don't reflect reality.
For some drivers (tu, freedreno, zink-anv) we have manual jobs available
for curious devs to look at the current state of the CTS, but without
anyone having to keep the full xfails updated during uprevs, you don't
necessarily know what to do with the results you get on your MR.
So, let's introduce nightly testing for the tests that aren't guaranteed
green by Marge. With that, Someone (possibly me? sigh) can review the
nightly results and push up updates for full-run xfails so everyone can be
on the same page other than a day or so of delay. We also have some hope
for automated tooling to do this thanks to what Collabora has been working
on for automated CI uprev MR generation.
Reviewed-by: Martin Roukala <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20950>
If you're only affecting one or a couple of drivers, it would be nice if
your pipeline buttons on the web UI weren't full of manual run buttons for
all the other drivers.
This is a bunch of duplicated lines, but less than it could have been now
that we have !references.
In some of these cases (i915g, nouveau, etnaviv), we have no non-manual
jobs for those drivers, so I could have just rewritten the original
"driver-rules" to "driver-manual-rules". I decided to keep things
consistent between drivers, though, because this is all esoteric enough to
readers already without making different drivers' rules look different.
Fixes: #4891
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17445>
David Heidelberg