Commit Graph

190776 Commits

Author SHA1 Message Date
Alyssa Rosenzweig
aad82f4265 agx: optimize txf with lod 0
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
5c898e56b2 agx: reserve scratch registers for mem<-->mem swaps
Trying to spill/fill a temp on the fly has pathologically bad performance in bad
cases, reserving a scratch reg is much more well-behaved. Fixes a fragment
shader timeout in dEQP-VK.graphicsfuzz.spv-stable-maze-flatten-copy-composite

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
f7ff0041c1 agx: document another sample_mask restriction
hit this with force early fragment tests.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
840394c169 agx: expose agx_link_libagx
for deduplication.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
5a3ac73747 agx: report uses_txf
so driver can optimize usc binding.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
54ec9512ef agx: flesh out subgroup lowering
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
659db5049c agx: lower 8-bit subgroups
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
00cfe84fd9 agx: forbid uniforms on ballots
needs more investigation..

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
c2c49b261c agx: add missing b2b16 implementation
will show up with a subgroup lowering.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
a3cb0cbfda agx: handle quad swaps
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
f4a5ac3554 agx: handle quad_broadcast
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
7b33c549b9 agx: handle non-immediate shuffles in divergent CF
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
589c69a646 agx: lower shuffle
per dougallj's notes in applegpu.py

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
fbcd9a83e4 agx: model more subgroup ops
additional shuffles, scans, reduces.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
e718a536ac agx: delete unreachable blocks
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
3efa723b63 agx: lower nir_intrinsic_load_num_subgroups
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
765bf9a25b agx: switch to demote internally
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
35d6f4a394 agx: fix spilling inside sample loop
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
bdd200a202 agx: handle subgroup barriers
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
d183b76fd4 agx: fix frag sidefx with sample shading
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
6269a1474d agx: fix load_helper_invocation with sample shading
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
94f0209fb2 agx: fix phi translation corruption
we can't stomp over srcs[], where we allocated our space for sources. unclear
how this worked before but it definitely breaks once you have a phi with 7
sources.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
f21dbfe5ae agx: allow 8-bit bcsel
can be generated from our lowerings but it just works with the implicit
conversion semantics we have.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
a948244058 agx: handle cross-workgroup memory barriers
there's no prior art for this, but experimentally this seems to do the right
thing.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
c22ce3cab9 agx: fix some ms texture packing
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
ec47f325f8 agx: fix query LOD of array
need to ignore the layer

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
8df39ac49b agx: enable more lowering
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
69d7063ec0 agx: optimize and/or with booleans
Beneficial so we can fuse the comparison.

total instructions in shared programs: 2188179 -> 2185535 (-0.12%)
instructions in affected programs: 392512 -> 389868 (-0.67%)
helped: 894
HURT: 9
Instructions are helped.

total alu in shared programs: 1706063 -> 1703445 (-0.15%)
alu in affected programs: 275063 -> 272445 (-0.95%)
helped: 880
HURT: 9
Alu are helped.

total fscib in shared programs: 1702385 -> 1699743 (-0.16%)
fscib in affected programs: 276199 -> 273557 (-0.96%)
helped: 894
HURT: 9
Fscib are helped.

total ic in shared programs: 462494 -> 462490 (<.01%)
ic in affected programs: 124 -> 120 (-3.23%)
helped: 1
HURT: 0

total bytes in shared programs: 14476964 -> 14464512 (-0.09%)
bytes in affected programs: 2870824 -> 2858372 (-0.43%)
helped: 888
HURT: 155
Bytes are helped.

total regs in shared programs: 662444 -> 662461 (<.01%)
regs in affected programs: 1025 -> 1042 (1.66%)
helped: 14
HURT: 12
Inconclusive result (value mean confidence interval includes 0).

total uniforms in shared programs: 1638301 -> 1638374 (<.01%)
uniforms in affected programs: 17778 -> 17851 (0.41%)
helped: 22
HURT: 54
Uniforms are HURT.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
c43413f729 compiler: add ACCESS_IN_BOUNDS_AGX
useful for internal shaders on agx.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
eb5f82d221 nir,agx: fix load_active_subgroup_index
It can't be reordered globally, since its value is control-flow dependent.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
7fb60c4c81 nir,agx: add depth=never workaround
There seems to be a hardware issue where fragment shaders with side effects get
skipped if depth testing with NEVER. Add a workaround for this case where we
discard programmatically instead.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
9d824bd123 nir: add quad_ballot_agx intrinsic
to lower quad votes in nir.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:24 +00:00
Alyssa Rosenzweig
2912f531a7 nir: add texops for AGX border colour emulation
AGX has limited border colour hardware. To support full
customBorderColorWithoutFormat semantics, we're forced to emulate in shaders at
a substantial performance penalty. Actually, that's needed just to pass CTS
because of other hardware issues stacking on top of each others... Hooray!

Add the texops we need to facilitate efficient custom border colour lowering.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:24 +00:00
Alyssa Rosenzweig
8b9ed851ec nir: add is_first_fan_agx sysval
needed for correct flatshading with fans, without falling back on software input
assembly.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:24 +00:00
Faith Ekstrand
8bc694223e zink: Set workarounds.can_do_invalid_linear_modifier for NVK
This fixes most of the egl_image_dma_buf* piglit tests.  The remaining
fails are YCbCr tests which are likely unrelated to core dma-buf
import/export.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Faith Ekstrand
e6f77defec nvk/wsi: Advertise modifier support
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Faith Ekstrand
28342a581f vulkan/wsi: Bind memory planes, not YCbCr planes.
Reviewed-by: Joshua Ashton <joshua@froggi.es>
Fixes: f5433e4d6c ("vulkan/wsi: Add modifiers support to wsi_create_native_image")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10176
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Faith Ekstrand
cd428e01d7 nvk: Advertise VK_EXT_image_drm_format_modifier
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9636
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9480
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Faith Ekstrand
d8e200c0d9 nvk: Advertise VK_EXT_queue_family_foreign
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Mohamed Ahmed
bca2f13dd8 nvk: enable rendering to DRM_FORMAT_MOD_LINEAR images
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Faith Ekstrand
224d9a514a nvk: Implement DRM format modifier queries
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Faith Ekstrand
4ad79bfef4 nvk: Set tile mode and PTE kind on dedicated dma-buf BOs
This is our compromise to make NVK and nouveau GL play nice when it
comes to modifiers.  The old GL driver depends heavily on the PTE kind
and tile mode, even for images with modifiers.  While it correctly
encodes the PTE kind and tile mode in the modifiers it advertises, it
may ignore the modifier and just trust what's set on the BO when it
imports a dma-buf image.  This is partly because it doesn't support
VM_BIND and partly because of preexisting bugs in the modifiers
implementation.  In either case, we can't fix it retroactively.

To work around this, NVK also sets the PTE kind and tile mode on the BO
when it's a dedicated allocation created for a DRM format modifiers
image.  If DRM format modifiers are used without dedicated allocations,
things may still break but that's getting into vanishingly unlikely
scenarios.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Faith Ekstrand
f1fdffa1b2 nvk: Support image creation with modifiers
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Faith Ekstrand
3bb531d245 nouveau/winsys: Add back nouveau_ws_bo_new_tiled()
This reverts commit ce1cccea98.  In this
new version, we also add a query for whether or not tiled BOs are
supported by nouveau.ko.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Faith Ekstrand
03c4a46fe5 drm-uapi: Sync nouveau_drm.h
Taken from drm-misc-next-fixes:

    commit 959314c438caf1b62d787f02d54a193efda38880
    Author: Mohamed Ahmed <mohamedahmedegypt2001@gmail.com>
    Date:   Thu May 9 23:43:52 2024 +0300

        drm/nouveau: use tile_mode and pte_kind for VM_BIND bo allocations

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Faith Ekstrand
8cce121da4 nvk: Allow VK_IMAGE_ASPECT_MEMORY_PLANE_0_BIT
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Mohamed Ahmed
6063f96c61 nil: Support creating images with DRM modifiers
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Mohamed Ahmed
e1bd4127f3 nil: Add some helpers for DRM format modifiers
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Faith Ekstrand
b7773f96f9 nil: Default to NV_MMU_PTE_KIND_GENERIC_MEMORY on Turing+
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00
Faith Ekstrand
603389f7a3 nvk: Set color/Z compression based on nil_image::compressed
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24795>
2024-05-14 04:04:34 +00:00