ff900ceffe
asahi: Enable scratch debugging
...
Via ASAHI_MESA_DEBUG=scratch. This will assert if enabled and the
scratch workgroup allocation count turns out not to be sufficient (that
is, there were failed allocations), to help debug the max occupancy
calculation.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
8845702f8c
asahi: Allocate scratch for shaders
...
Finally, now that scratch is available, tell the hardware how much we
need and allocate the scratch buffers as necessary.
We only do the max subgroup calculation for compute right now, and it's
probably conservative (we could probably calculate a reduced occupancy
depending on register pressure). Getting this too low won't break, the
GPU will just be less efficient if there is not enough scratch
allocated. Vertex and fragment just get the max 96 for now.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
bb5277aa3d
asahi: Hook up scratch
...
Instantiate our 3 scratch managers for each shader type, and pass them
to the hardware if a batch uses scratch.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
fa475c1b56
asahi: scratch: Add feature to debug core IDs
...
Define SCRATCH_DEBUG_CORES to a raw core count to bypass the mapping
machinery and allocate memory for many cores, so we can work out how the
core IDs are mapped.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
494399c65c
asahi: Implement scratch allocation
...
Implement our helper program to map scratch blocks into stack memory,
and the driver side that allocates these blocks as necessary.
Alloction is grow-only right now. Drivers are expected to instantiate
scratch memory managers for each shader type (VS, FS, CS) and the same
buffers are reused across commands for each one, growing as necessary.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
da9677f14b
asahi: cmdbuf: Fix scratch bucket offset/size
...
This is 4 bits. It is an opaque value that gets passed to the
helper, except 0 means no scratch.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
d6cc820d1a
asahi: libagx: Move PACKED and GLOBAL macros to libagx.h
...
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
eec7c4beb8
agx: compiler: Enable stack_adjust
...
We'll need this once we start using scratch.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
1f1751d258
agx: compiler: Export scratch size to the driver
...
So it can configure the pipeline correctly, and allocate the required
scratch space.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
b89da92a5e
agx: compiler: Add fence_helper_exit_agx barrier
...
This is used by the helper program on exit.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
c406ce793b
agx: Add scaffolding to build the helper shader at device init
...
Add the scaffolding to compile our helper program, and load it at device
init time.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
05c8b59f5b
agx: Hook up helper intrinsics into CL
...
So we can use these intrinsicts from libagx.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
30862d7189
agx: Hook up AGX helper NIR intrinsics
...
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
b07dbf7b0f
nir: Add AGX-specific helper opcodes
...
These opcodes are used by the helper program to fetch the current
operation info and core ID.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
5d0f1ca77f
agx: Rename some SRs
...
I think these better represent what these SRs actually do.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
28d34f6352
asahi: libagx: introduce AGX_STATIC_ASSERT
...
Using the array size trick, this works in both OpenCL and C.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
8661006ef0
agx: add some more bitop tests
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
923767a968
agx: Add a bitop optimizer pass
...
Signed-off-by: Mary Guillemard <mary@mary.zone >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
2b89eb979e
agx: Fuse not into and/or/xor
...
Signed-off-by: Mary Guillemard <mary@mary.zone >
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
ba508fe854
agx: Remove and/or/xor pseudo ops
...
Switch back to bitop while keeping the aliases on agx_print_instr.
Also add all variants of 2 args of agx_bitop_table.
Signed-off-by: Mary Guillemard <mary@mary.zone >
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
a5b6ff3ccc
agx: Add more bitops in agx_bitop_table
...
Signed-off-by: Mary Guillemard <mary@mary.zone >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
62001e175f
asahi: fix UB in qbo's
...
upper bits may be uninitialized. fixes
KHR-GL46.transform_feedback_overflow_query_ARB when built with gcc (passed by
chance with clang).
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
a06b51559b
asahi: plug geometry heap leak
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
877c180677
asahi: plug glsl type leak
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
06cf222830
asahi: plug so target leak
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
e2c5a93358
asahi: plug early_serialized_nir leak
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
ffac4e7179
asahi: plug pre-gs leak
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
7b44d15e5f
asahi: plug geometry shader leaks
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
1e973dd5a9
asahi: rework meta shader infra
...
this fixes the leaks, and is also a lot more pleasant to work with.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
7eda3da983
asahi: plug passthrough tcs leak
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
e91c0b6423
asahi: delete bogus assertion
...
dEQP-GLES3.functional.fragment_ops.scissor.outside_render_line at certain render
sizes.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
5261962186
asahi: fix prim restart unrolling with indirects
...
need to account for draw->start.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
c0d3bf3608
agx: optimize vote_eq
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
0de7018879
agx: optimize first_invocation
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
fb81201a34
agx: implement active_subgroup_invocation_agx
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
311070f7af
nir: add active_subgroup_invocation_agx sysval
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
da924e2f03
agx: fuse ballot+cmp
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
def00f7bf0
agx: introduce ballot pseudo
...
for more opts
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
92dcf75fcd
agx: lower more subgroups
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
f52e0bd70b
agx: implement load_subgroup_invocation
...
There's an sr.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
6f7511b522
asahi: decode uniform_high records
...
print them like low uniforms.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
1ee9b8b668
asahi: optimize more when linking libagx
...
Otherwise we get spurious scratch from lower_vars_to_explicit_types on dead
vars.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
9287bec892
asahi: fix unbound ssbos
...
fixes null ptr dereference in arb_shader_storage_buffer_object-array-ssbo-binding
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
3a6083dea3
asahi: enable robustness
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
591ce607e0
asahi: implement reset queries
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
5dc0f5ccba
asahi: implement VBO robustness
...
GL semantics. GLES (weaker) and VK (stronger) semantics are left as a todo, with
explanations given. Enabled always to deal with null VBOs, this should be
optimized once we have soft fault.
This necessitates a rework of VBO keys, but hopefully for the best.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
4aadf67523
asahi: fix metadata for images with VS lowered to GS
...
KHR-GL46.shader_image_load_store.basic-allTargets-atomicVS
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:28 +00:00
9753cd44f7
asahi: Implement skeleton for tessellation
...
This implements a rough skeleton of what's needed for tessellation. It contains
the relevant lowerings to merge the VS and TCS, running them as a compute
kernel, and to lower the TES to a new VS (possibly merged in with a subsequent
GS). This is sufficient for both standalone tessellation and tess + geom/xfb
together. It does not yet contain a GPU accellerated tessellator, simply falling
back to the CPU for that for now. Nevertheless the data structures are
engineered with that end goal in mind, in particular to be able to tessellate
all patches in parallel without needing any prefix sums etc (using simple
watermark allocation for the heap).
Work on fleshing out the skeleton continues in parallel. For now, this does pass
the tests and lets the harder stuff get regression tested more easily. And
merging early will ease rebase.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:28 +00:00
2d37d1b704
asahi: lower poly stipple
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:28 +00:00
2d913892b2
asahi: gut macOS related code
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:28 +00:00
Asahi Lina