Commit Graph

193069 Commits

Author SHA1 Message Date
Zan Dobersek 3ee81ffe14 tu: fix memory leaks in tu_shader
When tu_shader object is destroyed through vk_pipeline_cache, the relevant
destroy callback should relay to the general tu_shader_destroy function
that will also clean up owned resources.

During shader creation, the ir3_shader object should be destroyed once the
shader variants are retrieved. Since those variants are owned by tu_shader
they should be freed up in tu_shader_destroy.

Signed-off-by: Zan Dobersek <zdobersek@igalia.com>
Fixes: a03525d8db ("tu: Split program draw state into per-shader states")
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27847>
2024-03-30 08:56:03 +01:00
Alyssa Rosenzweig fcf1a8062b asahi: switch to VS/FS prolog/epilog system
With the exception of some variants for framebuffer fetch (to be addressed in a
follow up MR, this is big enough as it is) -- this switches us to a shader
precompile path for VS & FS. VS prologs let us implement vertex buffer fetch
with dynamic formats, FS prologs let us implement misc emulation like API sample
masking and cull distance, while FS epilogs handle blending and tilebuffer
stores. This should cut down shader recompile jank significantly in the GL
driver. It also prepares us with most of what we need for big ticket Vulkan
extensions like ESO, GPL, and EDS3.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:20 +00:00
Alyssa Rosenzweig 742a842811 asahi/clc: stop padding binaries
we're going to switch away from the dynarray soon.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 3a2d13f59e asahi: add fast linker
to stitch together programs from separate parts

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 66862fa160 agx: add main_size info
so we can fastlink when there are preambles

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 0298c5d905 agx: add agx_shader_part data structure
for fastlinking.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig e6105cdf0c asahi: static assert blend key size
need to make sure it's padded because C

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig b0698b796e agx: drop shader stage assertion
we'll reuse these for SW VS too which is HW compute

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 57fa9a2b8e nir: add intrinsics for non-monolithic agx shaders
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 99a4d0fcad asahi: don't allocate tib space for gaps
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig eadf4cfe1c asahi: constify agx_build_tilebuffer_layout
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 5d375e6143 asahi: add agx_usc_push_packed helper
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 9974f68bb9 agx: document non-monolithic ABI
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig a9ccd72265 agx: implement exports
translate export/load_exported instructions into moves to/from the requested
GPRs at shader part boundaries, with coalescing in RA for perf.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig c6839cfd69 agx: generalize preloaded cache
we'll need it for more regs

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 61a4414bf6 agx: wire up samples_log2 sr
details determined by trial & error.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig e536b4973f nir: add export/load_exported_agx intrinsics
for lowering non-monolithic ABI

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig df8e52a795 nir: add samples_log2_agx sysval
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig fbd6182549 asahi: fix bit sizes in point sprite lower
not sure how this wasn't caught before. ati_fragment_shader-error05-passes

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig caad4703aa agx: always reserve sampler #0 for txf
this is a lot simpler for non-monolithic programs and is unlikely to be a
meaningful performance problem.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig be81812a4b agx: inline sampler states
to avoid regresisng code gen next commit

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 781dcc8ef8 agx: optimize out wait_pix in some cases
noticed in prop driver asm

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig c955845891 asahi: don't set writes_memory for tib spilling
we don't want the zs_emit dance emitted, all we care about is making sure tag
writes are enabled as if we had a regular tib store

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 70395c1ac1 asahi: delete layer id code
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 6a63fffeaa asahi: fix _packed USC structs
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig b3fdede75b asahi: use ht derive more
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig a108b21d4c asahi: drop dead linked_so code
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig ad2f096a8b agx/lower_vbo: dce as we go
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig acc1fd3a8a asahi: stop using GLSL indirect lowering
we typically don't need it, and where we do, NIR does better.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 5c7ce24896 asahi: make point size replacement dynamic
I'm not measuring a significant perf difference in
-bshading:shading=phong:model=bunny -bideas -brefract so this seems Good Enough
For Me.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 7595dc9036 asahi: drop now-empty base key
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig f0e1ccc8d4 asahi: rewrite varying linking
Lower store_output to store_uvs_agx + math. Link UVS indices at draw-time
instead of compile-time to get efficient separate shaders. Also picks up varying
compaction along the way.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 3764adbef1 agx: inline imm into st_vary
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig a25e8de180 agx: pack indirect st_vary
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 78ede839f7 agx: rm unnecessary iter hack
I added this hack to avoid WaW hazards with iter instructions. Now that we know
about the iter elide bit and are not setting it improperly, we can drop the
complexity and just allow the hazard.

total instructions in shared programs: 2039480 -> 2038792 (-0.03%)
instructions in affected programs: 123441 -> 122753 (-0.56%)
helped: 811
HURT: 124
Instructions are helped.

total bytes in shared programs: 13983802 -> 13977870 (-0.04%)
bytes in affected programs: 806882 -> 800950 (-0.74%)
helped: 823
HURT: 117
Bytes are helped.

total regs in shared programs: 590670 -> 592862 (0.37%)
regs in affected programs: 8585 -> 10777 (25.53%)
helped: 29
HURT: 398
Regs are HURT.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 0d9ab6f7c7 asahi: advertise indirect fs inputs
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig a17b350a3d agx: handle indirect varyings
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 87551ba3eb agx: pack indirect CF
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig ea9ddf1b96 agx: explicitly assign coeff registers
prereq to doing FS indirect inputs nicely

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 70277f8ff7 asahi: extract agx_cf_binding
for convenience

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig a2265ff588 asahi: drop =varyings debug
hasn't been relevant.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 9c2df9e814 asahi: add agx_push_packed
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 499d091208 nir: add intrinsics for lowered VS outputs
handling VS indirects will require some driver cooperation.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 1773eb329c nir: add offset to load_coefficients_agx
for indirect varyings

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig a1f6e2818d docs/asahi: document UVS
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 43613314ab agx/opt_preamble: improve rewrite cost est
this keeps us from hoisting piles of iadd for no benefit withthe new vertex
path. results on shaderdb without HW VS:

total bytes in shared programs: 13975632 -> 13975666 (<.01%)
bytes in affected programs: 3298 -> 3332 (1.03%)
helped: 0
HURT: 3

total uniforms in shared programs: 1516540 -> 1516522 (<.01%)
uniforms in affected programs: 234 -> 216 (-7.69%)
helped: 3
HURT: 0

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig fd76caf5d3 agx/opt_preamble: preamble cycle estimates
total instructions in shared programs: 2038338 -> 2038217 (<.01%)
instructions in affected programs: 32658 -> 32537 (-0.37%)
helped: 9
HURT: 4
Instructions are helped.

total alu in shared programs: 1593474 -> 1593094 (-0.02%)
alu in affected programs: 110828 -> 110448 (-0.34%)
helped: 315
HURT: 9
Alu are helped.

total fscib in shared programs: 1589634 -> 1589254 (-0.02%)
fscib in affected programs: 110828 -> 110448 (-0.34%)
helped: 315
HURT: 9
Fscib are helped.

total ic in shared programs: 477960 -> 477948 (<.01%)
ic in affected programs: 12 -> 0
helped: 3
HURT: 0

total bytes in shared programs: 13975162 -> 13975632 (<.01%)
bytes in affected programs: 978988 -> 979458 (0.05%)
helped: 14
HURT: 313
Inconclusive result (value mean confidence interval includes 0).

total uniforms in shared programs: 1516534 -> 1516540 (<.01%)
uniforms in affected programs: 4278 -> 4284 (0.14%)
helped: 5
HURT: 6
Inconclusive result (value mean confidence interval includes 0).

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig 4c77f59e8a agx/opt_preamble: restrain ourselves
hoisting EVERYTHING is counterproductive now that we can remat constants.

total instructions in shared programs: 2038818 -> 2038338 (-0.02%)
instructions in affected programs: 33002 -> 32522 (-1.45%)
helped: 9
HURT: 3
Instructions are helped.

total alu in shared programs: 1594051 -> 1593474 (-0.04%)
alu in affected programs: 22012 -> 21435 (-2.62%)
helped: 12
HURT: 0
Alu are helped.

total fscib in shared programs: 1590211 -> 1589634 (-0.04%)
fscib in affected programs: 22012 -> 21435 (-2.62%)
helped: 12
HURT: 0
Fscib are helped.

total bytes in shared programs: 13978022 -> 13975162 (-0.02%)
bytes in affected programs: 219746 -> 216886 (-1.30%)
helped: 12
HURT: 0
Bytes are helped.

total regs in shared programs: 592854 -> 592758 (-0.02%)
regs in affected programs: 954 -> 858 (-10.06%)
helped: 6
HURT: 0
Regs are helped.

total uniforms in shared programs: 1516579 -> 1516534 (<.01%)
uniforms in affected programs: 4605 -> 4560 (-0.98%)
helped: 9
HURT: 0
Uniforms are helped.

total threads in shared programs: 20329216 -> 20329600 (<.01%)
threads in affected programs: 3840 -> 4224 (10.00%)
helped: 6
HURT: 0
Threads are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig cae914978b agx/opt_preamble: improve preamble cost function
save lots of uniforms by no longer pushing silly things.

total instructions in shared programs: 2039473 -> 2039480 (<.01%)
instructions in affected programs: 28335 -> 28342 (0.02%)
helped: 51
HURT: 47
Inconclusive result (value mean confidence interval includes 0).

total bytes in shared programs: 13983778 -> 13983802 (<.01%)
bytes in affected programs: 451678 -> 451702 (<.01%)
helped: 151
HURT: 79
Inconclusive result (value mean confidence interval includes 0).

total regs in shared programs: 590373 -> 590670 (0.05%)
regs in affected programs: 2354 -> 2651 (12.62%)
helped: 13
HURT: 65
Regs are HURT.

total uniforms in shared programs: 1532271 -> 1516549 (-1.03%)
uniforms in affected programs: 295709 -> 279987 (-5.32%)
helped: 2302
HURT: 0
Uniforms are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig b9a359e9bd agx: start a crude cycle model
based on notes by Dougall Johnson and Philip Turner.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00