Commit Graph

10822 Commits

Author SHA1 Message Date
Alyssa Rosenzweig 4f1bafa6d5 nir: drop load_sample_id_no_per_sample
unused now.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36429>
2025-07-30 22:13:23 +00:00
Alyssa Rosenzweig 04be5a8910 nir/lower_system_values: simplify load_helper_invocation lowering
The two backends (ir3, dxil) using the lowering have
info->fs.uses_sample_shading matching set when sample shading is used -- the VK
drivers pass the rasterization state flag into the compiler, while
freedreno and d3d12 have caps->force_persample_interp so the frontend
creates a shader variant with info->fs.uses_sample_shading set.

This means that we can drop the sample-id SHL/AND in the pixel-rate
shading case, which in turn means that drivers don't need to have a
load_sample_id() that doesn't trigger sample-rate shading (which Adreno
doesn't have).

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36429>
2025-07-30 22:13:23 +00:00
Caio Oliveira dde6fa5728 spirv: Implement SPV_KHR_untyped_pointers
The untyped pointer types only have a storage class associated, and the
operations using them would carry the necessary "data type" information.

Untyped pointers themselves are identified by "vtn_type::pointed" being
NULL.  For the NIR lowering the operations will have explicit casts
before them when applicable and the nir_derefs representing untyped
pointers will use the "void" glsl_type.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36427>
2025-07-30 15:35:33 +00:00
Caio Oliveira 8eaf1dced0 spirv: Load block descriptors as soon as we hit them
With changes suggested by Faith.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36427>
2025-07-30 15:35:33 +00:00
Faith Ekstrand d1be63c355 spirv: Assert that vtn_pointer_to_deref() doesn't return NULL
There are a few things that could cause us to return NULL here and we
should just fail to parse the SPIR-V if that ever happens instead of
crashing when someone tries to access it.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36427>
2025-07-30 15:35:32 +00:00
Faith Ekstrand 8a993cf900 spirv: Simplify pointer_to/from_ssa a bit
This adds a vtn_pointer_ssa_is_desc_index() helper and makes both
to/from_ssa work in terms of it, ensuring we never screw up the
condition and turn a deref into a block index or vice versa.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36427>
2025-07-30 15:35:31 +00:00
Faith Ekstrand df8ee49bc6 spirv: Drop block_index/offset pointers
These haven't existed in a while but the vtn_pointer fields stayed
around.  Drop offset since no one uses it and rename block_index to
desc_idx with a better comment.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36427>
2025-07-30 15:35:31 +00:00
Faith Ekstrand 46d02bd244 spirv: Assert !ptr_as_array for blocks and acceleration structures
As of ea9deafff4 ("spirv: Add more restrictions around Blocks"), this
case can't happen.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36427>
2025-07-30 15:35:30 +00:00
Georg Lehmann e43ef6533b nir/opt_algebraic: remove 8bit roundtrip when vectorizing i2i16(unpack_4x8(a).zw)
Explicit 16bit instructions are nicer to vectorize.

Helps FSR4 on GFX11 marginally.

Foz-DB Navi31:
Totals from 10 out of 14 FSR4 shaders:
Instrs: 59781 -> 58518 (-2.11%)
CodeSize: 413428 -> 404156 (-2.24%)
Latency: 193770 -> 190768 (-1.55%)
InvThroughput: 226274 -> 221628 (-2.05%)
VClause: 796 -> 793 (-0.38%); split: -1.01%, +0.63%
Copies: 3342 -> 3008 (-9.99%); split: -11.01%, +1.02%
PreSGPRs: 312 -> 305 (-2.24%)
VALU: 51448 -> 50213 (-2.40%)
SALU: 1074 -> 1048 (-2.42%)
VOPD: 1783 -> 1718 (-3.65%); split: +0.95%, -4.60%

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36117>
2025-07-30 07:25:51 +00:00
Georg Lehmann 92cf02ee23 nir/search: support swizzles on expressions in replacement patterns
Before this wasn't a compile time error, but it also didn't do anything.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36117>
2025-07-30 07:25:51 +00:00
Marek Olšák d61edf079b nir: add nir_move_only_convergent/divergent
This will be needed by nir_opt_move_reorder_loads, which will use
the move flags.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36357>
2025-07-29 16:20:53 -04:00
Marek Olšák 35bbc8405b nir: add more nir_move_options
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36357>
2025-07-29 16:20:51 -04:00
Marek Olšák 44d78c4451 nir: handle load_input_vertex in nir_can_move_instr
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36357>
2025-07-29 16:20:49 -04:00
Marek Olšák 8d3e76c250 nir: split nir_move_load_frag_coord from nir_move_load_input
It's a pure system value on AMD, not an input.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36357>
2025-07-29 16:20:48 -04:00
Marek Olšák 5083769fcb nir: renumber nir_move_options
for future commits

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36357>
2025-07-29 16:20:46 -04:00
Marek Olšák 8d584586f5 nir: handle can_reorder robustly in nir_can_move_instr
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36357>
2025-07-29 16:20:44 -04:00
Marek Olšák c229c93540 nir: change how can_mov_out_of_loop is set for intrinsics in nir_can_move_instr
Set to false first, then set to true when needed.

More intrinsics will set false.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36357>
2025-07-29 16:20:42 -04:00
Marek Olšák ad1cfcc841 nir: mark inverse_ballot & is_subgroup_invocation_lt_amd as CAN_REORDER
nir_can_move_instr already ignores that this flag isn't set.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36357>
2025-07-29 16:20:39 -04:00
Marek Olšák 2eea9b968d nir/group_loads: rename to nir_opt_group_loads
It's meant to be an optimization pass.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36100>
2025-07-29 19:28:59 +00:00
Marek Olšák 3bceb7b622 nir/group_loads: don't use pass_flags to store the indirection level
no change in behavior

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36100>
2025-07-29 19:28:59 +00:00
Marek Olšák 7b9acabe89 nir/group_loads: store our custom instr->index in an array
we'll put more stuff in the new structure

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36100>
2025-07-29 19:28:58 +00:00
Marek Olšák 821dc611c5 nir/group_loads: use nir_instr_next/prev
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36100>
2025-07-29 19:28:58 +00:00
Marek Olšák 318a57c57c nir/group_loads: make is_grouped_load use get_load_resource
and unify get_intrinsic_resource & get_uniform_inst_resource into the new
helper get_load_resource.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36100>
2025-07-29 19:28:57 +00:00
Marek Olšák b9f7dc5d26 nir/group_loads: remove mostly duplicated function is_memory_load
is_grouped_load does the same thing

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36100>
2025-07-29 19:28:57 +00:00
Marek Olšák ef0a5e1a65 nir/group_loads: invert the return value of can_move to reflect its true meaning
The previous commit handles non-reorderable instructions better, so this is
no longer needed.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36100>
2025-07-29 19:28:56 +00:00
Marek Olšák ad8b5325d4 nir/group_loads: group any reorderable intrinsics regardless of barriers
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36100>
2025-07-29 19:28:56 +00:00
Marek Olšák e0d8979dea nir/group_loads: split is_barrier into is_barrier + is_terminate
and handle demote

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36100>
2025-07-29 19:28:56 +00:00
Marek Olšák 7d9aca2235 nir/group_loads: allow moving loads across instructions without defs
It wouldn't group these:
    load (reordeable)
    store
    load (reordeable)

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36100>
2025-07-29 19:28:55 +00:00
Marek Olšák aa732f6f30 nir/group_loads: handle more loads
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36100>
2025-07-29 19:28:55 +00:00
Antonio Ospite 8825bf9b48 nir: fix returning _Bool instead of pointer
When building for C23 the compiler warns about returning a boolean when
a different type is expected instead.

Change the code to return NULL instead of false, fixing the following
error:

-----------------------------------------------------------------------
../src/compiler/nir/nir_lower_io_indirect_loads.c: In function ‘get_load_once_variable’:
../src/compiler/nir/nir_lower_io_indirect_loads.c:87:17: error: incompatible types when returning type ‘_Bool’ but ‘nir_variable **’ {aka ‘struct nir_variable **’} was expected
   87 |          return false;
      |                 ^~~~~
-----------------------------------------------------------------------

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36323>
2025-07-29 14:07:06 +00:00
Rhys Perry 928c9c618d nir/opt_access: support RT/callable shaders
I don't know if this affects any real application.

No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35938>
2025-07-28 09:19:01 +00:00
Rhys Perry a9a1da0264 nir/uub: fix 8/16-bit overflow
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Backport-to: 25.1
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13552
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13553
Tested-by: @LingMan
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36372>
2025-07-28 08:46:51 +00:00
Timothy Arceri dd71263c21 glsl: rename setup_uniform_remap_tables()
Remove the preceeding nir as that is generally reserved for helpers
used across files.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36366>
2025-07-26 07:22:36 +00:00
Timothy Arceri cb558b2b88 glsl: add mark_array_elements_referenced() fast path
Add a fast path for single dimension arrays. This is around 3x
faster on large arrays. For example when testing a shader from
issue 9953 compile time went from ~12 seconds down to ~4 seconds.

Issue: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9953

Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36366>
2025-07-26 07:22:36 +00:00
Timothy Arceri 8579c21798 glsl: move mark_array_elements_referenced() with ubo code
Since we no longer share this with the old glsl ir linker just move
it to where it is called from.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36366>
2025-07-26 07:22:35 +00:00
Georg Lehmann 845961ab77 nir: remove NIR_PASS_V
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10409
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36381>
2025-07-25 23:09:56 +00:00
Pierre-Eric Pelloux-Prayer 718fd47dde nir/lower_io: make sure range is not 0
For samplers the type_size() callback can return 0, which triggers
a NIR validation error.
In this case set range to ~0 which means the range is unknown.

Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36263>
2025-07-25 15:47:20 +02:00
Marek Olšák d9cb26fdf9 nir: handle store_buffer_amd in nir_intrinsic_writes_external_memory
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36099>
2025-07-24 18:41:38 +00:00
Marek Olšák a15c1e1896 nir: silence a warning in nir_opt_shrink_vectors
gcc assumes the array is a string, so it treats it as having 15 elements
instead of 16.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36099>
2025-07-24 18:41:38 +00:00
Marek Olšák b35e8a38e7 nir/opt_move_to_top: check can_reorder & can_speculate
No effect now, but may be needed in the future.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36099>
2025-07-24 18:41:38 +00:00
Marek Olšák 6a85761c4c nir/divergence_analysis: simplify nir_vertex_divergence_analysis
by reusing nir_divergence_analysis_impl.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36099>
2025-07-24 18:41:38 +00:00
Marek Olšák 09e607c385 nir: add access to load_smem_amd (for ACCESS_CAN_SPECULATE)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36099>
2025-07-24 18:41:38 +00:00
Marek Olšák 688a639117 nir: add nir_tex_instr::can_speculate
Set to true everywhere except:
- spirv_to_nir used by Vulkan
- bindless handles in GLSL
- some internal shaders and driver-specific code

Acked-by: Job Noorman <job@noorman.info>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36099>
2025-07-24 18:41:38 +00:00
Marek Olšák b4afe848a1 nir: add nir_instr_can_speculate helper (for LICM)
Acked-by: Job Noorman <job@noorman.info>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36099>
2025-07-24 18:41:37 +00:00
Paolo Bonzini 8ea232a9ae meson: rename Rust subprojects to NAME-SEMVER-rs
Use the convention for Rust subprojects that was adopted by Meson 1.5.0
and newer.

Distros would prefer to avoid vendored crate sources, and instead use
local sources from e.g. /usr/share/cargo/registry.  While Meson does not
support a local registry, it can be emulated with MESON_PACKAGE_CACHE_DIR.

However, because the distro might not be using the exact version of the
package, but only one that has the same semver, packagers need to add
some hacks to rewrite the wrap files.  For example, in Fedora:

    export MESON_PACKAGE_CACHE_DIR="%{cargo_registry}/"
    # So... Meson can't actually find them without tweaks
    %define inst_crate_nameversion() %(basename %{cargo_registry}/%{1}-*)
    %define rewrite_wrap_file() sed -e "/source.*/d" -e "s/%{1}-.*/%{inst_crate_nameversion %{1}}/" -i subprojects/%{1}.wrap
    %rewrite_wrap_file proc-macro2
    %rewrite_wrap_file quote
    %rewrite_wrap_file syn
    %rewrite_wrap_file unicode-ident
    %rewrite_wrap_file paste

Having a common convention for the name of Rust wraps makes it possible
to perform this transformation with a script without listing
the wraps one by one, and to share the script across multiple packages
(which will be useful when QEMU starts using Rust in a similar way to Mesa).

For an example of such a script, see
https://lore.kernel.org/r/20250722083507.678542-1-pbonzini@redhat.com/.

Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36284>
2025-07-24 17:52:34 +00:00
Alyssa Rosenzweig 8a1a410389 treewide: use SWAP macro
Via Coccinelle patch + manual clean up:

    @@
    identifier temporary, a, b;
    type T;
    @@

    -T temporary = a;
    -a = b;
    -b = temporary;
    +SWAP(a, b);

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36297>
2025-07-23 19:49:47 +00:00
Alyssa Rosenzweig 3f795a2b8d nir/divergence_analysis: handle more AGX
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36265>
2025-07-23 14:15:57 +00:00
Alyssa Rosenzweig ebc18de6f5 nir/opt_vectorize_io: allow i/o semantics w/o component
load_uvs_index_agx always implicitly uses component 0.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36265>
2025-07-23 14:15:57 +00:00
Alyssa Rosenzweig 75f25b35f8 nir: handle frag_coord_z/w intrinsics
so we can gather_info later.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36265>
2025-07-23 14:15:57 +00:00
Alyssa Rosenzweig 8716012b21 glsl,nir: factor out nir_opt_varyings_bulk
Correctly/optimally using nir_opt_varyings directly is pretty tricky. For GL, we
have all the right logic in the GLSL linker. for VK, we don't want to duplicate
this dance in every driver. Wrap it all up in a nir_opt_varyings_bulk helper
that operates on an entire pipeline of nir_shader's, following the GLSL linker's
logic. This is suitable for Vulkan drivers.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36265>
2025-07-23 14:15:57 +00:00