Commit Graph

388 Commits

Author SHA1 Message Date
Eric Engestrom b13473efe9 bin/pick-ui: use venv wrapper
Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24367>
2023-08-03 23:21:32 +00:00
Eric Engestrom 2eca558dd4 bin/ci/update_traces_checksum: use venv wrapper
Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24367>
2023-08-03 23:21:32 +00:00
Eric Engestrom ce49034bbb bin/ci/gitlab_gql: use venv wrapper
Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24367>
2023-08-03 23:21:32 +00:00
Eric Engestrom 6e4843bb04 bin/ci/ci_run_n_monitor: use venv wrapper
Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24367>
2023-08-03 23:21:31 +00:00
Eric Engestrom 75f44bd79a bin: add wrapper to run scripts in a python venv
This isolates the script environment from the rest of the machine,
avoiding missing/incompatible dependencies and avoiding polluting the
rest of the machine with python packages.

Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24367>
2023-08-03 23:21:31 +00:00
David Heidelberg ced762a8d3 bin/ci: handle errors more gracefully in update_traces_checksum script
UnicodeDecodeError is present very often, since GitLab logs are scary.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24444>
2023-08-03 21:58:13 +00:00
Eric Engestrom 42021b4d13 bin/ci_run_n_monitor: get git sha from pipeline if specified, instead of requiring --rev to match
Fixes: 9ce717ab31 ("ci_run_n_monitor: add ability to specify the pipeline to use, instead of auto-detecting it")
Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24366>
2023-07-28 13:10:41 +00:00
Guilherme Gallo 701b035179 bin/ci: Ensure that all jobs have nodes in DAG
Some automatic jobs, such as 'rustfmt' and 'clang-format', are skipped
during the graph sweep because their parents are already included in the
node set.

This commit ensures all visited jobs are in DAG and fixes iteration
modification using deepcopy.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9376

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24176>
2023-07-17 15:34:11 +00:00
Dylan Baker bea9111749 bin/pick-ui: use asyncio.new_event_loop
Instead of .get_event_loop, which is deprecated when there isn't a
running loop (like in our case).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23472>
2023-06-15 22:37:31 +00:00
Dylan Baker 0630a599fc bin/pick: Add support for adding notes on patches
This is pretty useful for keeping track of why a patch isn't landed, or
who I'm waiting on feedback from.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23472>
2023-06-15 22:37:31 +00:00
Dylan Baker 41394187aa bin/pick: use lineboxes to make the UI clearer
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23472>
2023-06-15 22:37:31 +00:00
Dylan Baker dd760f8846 bin/pick: fix issue where None for nomination_type could fail
We have an assumption it's never None, so use a special value in the
Enum instead.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23472>
2023-06-15 22:37:31 +00:00
Daniel Stone b3c6cd0886 ci: Respect $HTTP_PROXY for ci_run_n_monitor
We need to explicitly tell AIOHTTP that it's OK to take the HTTP proxy
from the environment.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23341>
2023-06-02 02:45:36 +00:00
David Heidelberg b3b883ca00 bin/ci: mention requirements.txt
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22078>
2023-05-28 15:19:42 +02:00
Eric Engestrom 51482b2c6a bin/ci: fix mistakenly hardcoded repo name in get_gitlab_project()
It didn't matter so far because all callers always pass "mesa".

Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23223>
2023-05-26 10:13:09 +00:00
Eric Engestrom 9ce717ab31 ci_run_n_monitor: add ability to specify the pipeline to use, instead of auto-detecting it
The auto-detection code currently looks for a repo called "mesa" in the
current user's fork (ie. the user providing the api token), which is great for
the common use case, but sometimes needs to be able to be overridden, such as
when running a pipeline in another fork than one's own, when working with
someone else in their fork.

Signed-off-by: Eric Engestrom <eric@igalia.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23230>
2023-05-26 00:12:03 +00:00
Eric Engestrom bcdd5b5e74 ci: bump bin/ci/ deps to support python 3.11
Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22943>
2023-05-10 14:32:02 +00:00
Guilherme Gallo eb1945c3d9 bin/ci: Add StructuredLogger to improve log handling
This commit introduces the StructuredLogger module, which provides a
robust and flexible logging utility supporting multiple data formats
(CSV, JSON, and YAML). By incorporating this module into our CI system,
we enhance our log management capabilities, making it easier to:

1. Monitor and analyze logs: The StructuredLogger is a dict-like data
   abstraction which autosaves into a structured data file, whenever it
   is updated. With this file, one can easily know specifics of the job
   execution without having to grep it in the traces logs or exploring
   the job artifacts. The autosave feature makes it useful even when the
   CI job fails unexpectedly, since the partial dict is always written
   back to the disk.
2. Maintain data integrity: The module includes context managers for
   file locking and editing log data, ensuring data integrity and
   preventing race conditions.
3. Support multiple formats: With built-in support for CSV, JSON, and
   YAML formats, this module caters to a wide range of use cases and
   user preferences.
4. Increase maintainability: The modular design of the StructuredLogger
   and its corresponding strategies simplifies maintenance and allows
   for seamless integration of additional formats in the future.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22500>
2023-04-19 14:36:37 +00:00
Harri Nieminen fd767a4517 bin: Fix typos
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22189>
2023-03-30 21:37:00 +00:00
Erik Faye-Lund ba6336ce3e docs: use version-number as toctree-title for relnotes
Using the full titles makes most of these take up two lines in the
toctree. The version number is really the only thing we should care
about, though.

Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21956>
2023-03-30 10:09:01 +00:00
Pino Toscano 9d42e31b54 symbols-check: support OSes based on GNU toolchain
Some of the symbols listed in PLATFORM_SYMBOLS are not only specific
to Linux, but rather specific to the GNU toolchain. Hence, use them
when inspecting ELF binaries produced by a GNU toolchain: this means
on Hurd ('GNU'), and on e.g. kFreeBSD ('GNU/kFreeBSD').

Signed-off-by: Pino Toscano <toscano.pino@tiscali.it>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21825>
2023-03-10 13:25:48 +00:00
David Heidelberg 26dc5b3737 ci/ci_run_n_monitor: while we usually disable many jobs, print them inline
Saving scrolling time...

Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21713>
2023-03-06 01:51:59 +01:00
Eric Engestrom 22c1f4be8c docs: mention meson configure and drop broken workaround script
The script is broken, and nobody noticed so it wasn't used much.

Meson has had support for printing the options by pointing to the source
dir for a while (not sure the exact version though) so I think we can
just recommend users do that.

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21469>
2023-02-23 08:50:40 +00:00
Tomeu Vizoso b1bd6aa21a android: Make libbacktrace optional again
For the same reason why we used to have USE_LIBBACKTRACE with the old
Android makefiles, allow to build Mesa without linking to it.

In recent VNDK versions, libbacktrace isn't available.

When building without linking libbacktrace, for some reason some symbols
related to C++ exception handling are exposed. Allow them in the symbols
check script.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Signed-off-by: Helen Koike <helen.koike@collabora.com>
Reviewed-by: Sergi Blanch Torné <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20019>
2023-01-25 08:58:25 -03:00
Dylan Baker a79e6457d8 meson: use builtin support for reading version from a file
In meson 0.57 support was added for reading a version from a file to
meson natively, so we don't need this workaround anymore.

Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20752>
2023-01-19 23:06:07 +00:00
Eric Engestrom c672b593b9 bin/ci: add gitlab_gql.py.cache to the .gitignore
On my machine, the filecache doesn't have the extra `.db` suffix, just `.cache`.

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20722>
2023-01-16 15:27:38 +00:00
Eric Engestrom b3f517b988 gen_release_notes: include links in relnotes.rst when generating the new release note
This is required to allow the docs to build, which in turn is required
if we want to allow merge requests against release (staging) branches.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20460>
2023-01-12 14:34:59 +00:00
Eric Engestrom 13af997567 gen_release_notes: allow using the script from another checkout
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20459>
2023-01-12 14:32:10 +00:00
Eric Engestrom 4a2635153c gen_release_notes: avoid crashing when none of the commits mention closing an issue
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20459>
2023-01-12 14:32:10 +00:00
Eric Engestrom b114debffb gen_release_notes: stop the script if we can't generate the release note
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20459>
2023-01-12 14:32:10 +00:00
Eric Engestrom a6e487e367 ci_run_n_monitor.py: automatically retry on transient errors
This morning a GitLab update was deployed, leading to a few moments
where the API returned an error.

python-gitlab is actually able to handle a number of those and retry
automatically, so let's enable that:
https://python-gitlab.readthedocs.io/en/stable/api-usage-advanced.html#transient-errors

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20094>
2023-01-06 19:28:55 +00:00
Konstantin Kharlamov 05ca7d114e bin/gen_release_notes.py: read Closes/Fixes tags case-insensitively
Cc: mesa-stable
Signed-off-by: Konstantin Kharlamov <Hi-Angel@yandex.ru>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20241>
2022-12-30 03:39:38 +03:00
Konstantin Kharlamov e67578a264 bin/gen_release_notes.py: parse "Fixes" tags as well as "Closes"
Some commits refer to bugs being fixed with "Fixes" tag. Example of one:

    e13d53e1fdb 'Revert "glx/dri: Fix DRI drawable release at MakeCurrent time"'

Parse this tag as well.

Cc: mesa-stable
Signed-off-by: Konstantin Kharlamov <Hi-Angel@yandex.ru>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20241>
2022-12-30 03:39:38 +03:00
Konstantin Kharlamov 52cd87ea16 bin/gen_release_notes.py: don't fail if "Closes" refers to an MR
Sometimes a tag "Closes:" in a commit may refer to a merge request
instead of an issue. Examples of such commits:

    34319c7d84 "ci/freedreno: disable antichambers trace"
    998122d9c2 "mesa: fix GL_INVALID_OPERATION in glEGLImageTargetTexStorageEXT"

Avoid failing on these by explicitly checking that the URL refers to an
issue

Cc: mesa-stable
Signed-off-by: Konstantin Kharlamov <Hi-Angel@yandex.ru>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20241>
2022-12-30 03:39:38 +03:00
Konstantin Kharlamov 334123a908 bin/gen_release_notes.py: do not fail on confidential features
A commit may refer to an issue marked as confidential. That will look
like a 404 page for outside users. One example of such commit is:

    369c12e5be "anv: clear descriptorsets if AllocateDescriptorSets fails"

Let's handle that case.

Cc: mesa-stable
Signed-off-by: Konstantin Kharlamov <Hi-Angel@yandex.ru>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20241>
2022-12-30 03:39:38 +03:00
Konstantin Kharlamov bd807eecd1 bin/gen_release_notes.py: do not end "features" with "None"
Currently, the "New features" list unconditionally ends with a "None"
point, which makes no sense. The original author probably meant to check
whether the file is empty, so remove the else clause, and add the check
for emptiness.

Cc: mesa-stable
Signed-off-by: Konstantin Kharlamov <Hi-Angel@yandex.ru>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20241>
2022-12-30 03:39:37 +03:00
Eric Engestrom 980d6a91ee gen_release_notes: strip second newline in new features
Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20132>
2022-12-29 19:20:07 +00:00
David Heidelberg a04720dc1c ci: move the CI handling scripts into bin/ci
Any change in .gitlab-ci/ directory triggers an entire CI run,
which is unnecessary for just changing the developers' scripts.

Added symlink to `.gitlab-ci/bin` to keep original invocation compatibility.

Reviewed-by: Martin Roukala <martin.roukala@mupuf.org>
Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20371>
2022-12-19 16:36:19 +01:00
Eric Engestrom cf841cdd0b commit_in_branch.py: variables cleanup
Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19988>
2022-11-30 17:08:48 +00:00
Eric Engestrom 707015891f commit_in_branch.py: add support for checking staging branches
Or any branch that contains a `/` slash.

Cc: mesa-stable
Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19988>
2022-11-30 17:08:48 +00:00
Eric Engestrom 3b07d57ff9 commit_in_branch.py: fix tests
Apparently these were backported since I wrote these tests ^^

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19988>
2022-11-30 17:08:48 +00:00
Danylo Piliaiev 703d85485f meson: do not use source_root() when possible
source_root() function is deprecated in Meson version 0.56.0 because
it returns the source root of the parent project if called from a
subproject.

Why would anyone need Mesa as a meson subproject?
It would be used as subproject in a project generated by command buffer
"decompiler" for Freedreno.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19901>
2022-11-22 06:11:07 +00:00
Jason Ekstrand 4b6c5781dc scripts: Fix khronos-update.py to use main for OpenCL headers
Also, cl.hpp no longer exists in OpenCL-Headers to fix it at the last
commit where it did exist.

Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19816>
2022-11-18 22:22:58 +00:00
Yonggang Luo ab20ec9da7 meson: fixes mingw-clang32 building
Cc: mesa-stable

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19610>
2022-11-11 05:38:19 +00:00
Yonggang Luo f03421702c meson: Refactoring shared gen_vs_module_defs_normal_command out
Cc: mesa-stable

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19610>
2022-11-11 05:38:19 +00:00
Vinson Lee 87182ba977 bin/perf-annotate-jit.py: Update to Python 3.
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12926>
2022-08-21 22:14:43 -07:00
Yonggang Luo f5bb9dd738 egl: Remove eglextchromium.h and import eglext_angle.h
The upstream eglextchromium.h in https://chromium.googlesource.com/chromium/src/+/refs/heads/master/ui/gl/EGL/eglextchromium.h
are broken, so remove it and using eglext_angle.h instead.

By running "python bin/khronos-update.py egl" to update EGL files,
then discard changes other than eglext_angle.h

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17815>
2022-08-09 21:02:50 +00:00
Eric Engestrom 446d2039cb bin/gen_release_notes.py: bump advertised vulkan version to 1.3
Fixes: df8ac77af8 ("anv: Advertise Vulkan 1.3")
Fixes: 08c6f437cf ("radv: advertise Vulkan 1.3")
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17759>
2022-08-03 19:12:25 +00:00
Jesse Natalie 0bcb5f919e bin: Add a script for filtering/sorting refcount logs
Reviewed-by: Sil Vilerino <sivileri@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16182>
2022-05-02 16:58:20 +00:00
Yonggang Luo 4ead2f6579 win32: Fixes 32 bits visual studio module definition files by add script gen_vs_module_defs.py
Getting opengl32*.def consistence with Windows SDK.
Getting osmesa.mingw.def's gl* functions consistence with Windows SDK.
stw_* functions are cdecl, not stdcall, so there is no need mangling the symbol.
Fixes egl.def for x86
d3d10sw: Move the place of d3d10_sw.def to d3d10_sw.def.in
Fixes vulkan_lvp.def for x86
Fixes #5552
Remove stdcall-fixup

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14041>
2022-04-19 19:38:47 +00:00