be64454710
nir/tests: test opt_loop_peel_initial_break with derefs in header block
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31324 >
2024-10-01 12:24:22 +00:00
0484044b1a
nir/opt_loop: rematerialize header block derefs in their use blocks
...
Otherwise, we could end up with phis of derefs.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Fixes: 6b4b044739https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31324 >
2024-10-01 12:24:22 +00:00
1421319dcf
compiler/rust: Copy MappedInstrs from NAK
...
Rename it to SmallVec, make it more generic and switch NAK
to it.
Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com >
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31409 >
2024-10-01 11:33:35 +00:00
f19f1ec17b
nir/opt_algebraic: Allow two-step lowering of ftrunc@64 to use ffract@64
...
If ftrunc@64 is lowered by nir_lower_doubles it is turned into a
comparable long series of 32 bit operations. If the hardware
supports ffract@64 then nir_opt_algebraic can first lower ftrunc@64
to use some combinations with ffloor@64. They can then be turned
into a combination of fsub@64 and ffract@64 resulting in less
all-over instructions.
Fixes: 5218cff34bgert.wollny@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29281 >
2024-09-30 23:51:02 +00:00
0b34a7aff0
nir: Don't generate single iteration loops to zero-initialize memory
...
If the stride we're adding to our loop counter is larger than the total
amount of shared local memory we're trying to initialize, we know the
loop will run at most one time. So we can skip emitting a loop.
Loop unrolling appears to be unable to detect this currently.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31312 >
2024-09-30 05:27:17 +00:00
bb7e8d51b6
nir: delete nir_opt_reuse_constants
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31031 >
2024-09-27 05:19:16 +00:00
60776f87c3
nir/opt_remove_phis: rematerialize constants
...
Foz-DB Navi31:
Totals from 749 (0.94% of 79395) affected shaders:
Instrs: 1224359 -> 1223722 (-0.05%); split: -0.07%, +0.02%
CodeSize: 6468392 -> 6466296 (-0.03%); split: -0.06%, +0.03%
Latency: 9764410 -> 9766457 (+0.02%); split: -0.01%, +0.03%
InvThroughput: 1017401 -> 1017380 (-0.00%); split: -0.03%, +0.03%
VClause: 19902 -> 19873 (-0.15%); split: -0.16%, +0.02%
SClause: 38441 -> 38424 (-0.04%); split: -0.05%, +0.01%
Copies: 86880 -> 86304 (-0.66%); split: -0.73%, +0.06%
Branches: 34206 -> 34159 (-0.14%); split: -0.14%, +0.01%
PreSGPRs: 45557 -> 45527 (-0.07%); split: -0.08%, +0.01%
PreVGPRs: 32406 -> 32408 (+0.01%)
VALU: 671633 -> 671533 (-0.01%); split: -0.02%, +0.01%
SALU: 155284 -> 154675 (-0.39%); split: -0.40%, +0.00%
VMEM: 27303 -> 27271 (-0.12%)
SMEM: 67490 -> 67455 (-0.05%)
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31031 >
2024-09-27 05:19:16 +00:00
40fc85c15b
nir: make nir_instr_clone usable with load_const and undef
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31031 >
2024-09-27 05:19:16 +00:00
a9f8089240
nir: replace nir_opt_remove_phis_block with a single source version
...
This is what callers actually want, and it simplifies nir_opt_remove_phis
because we can assume dominance meta data is valid.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31031 >
2024-09-27 05:19:16 +00:00
41e82b8b8e
nir: sink is_subgroup_invocation_lt_amd
...
Having it closer to the branches means we can eliminate an exec copy.
Foz-DB Navi31:
Totals from 11615 (14.63% of 79395) affected shaders:
Instrs: 6804372 -> 6804903 (+0.01%); split: -0.04%, +0.05%
CodeSize: 33684672 -> 33680584 (-0.01%); split: -0.07%, +0.05%
VGPRs: 578616 -> 578604 (-0.00%)
SpillSGPRs: 1506 -> 1304 (-13.41%)
Latency: 29817034 -> 29821320 (+0.01%); split: -0.03%, +0.05%
InvThroughput: 3581587 -> 3581217 (-0.01%); split: -0.02%, +0.01%
VClause: 124826 -> 124782 (-0.04%); split: -0.04%, +0.00%
SClause: 187916 -> 187645 (-0.14%); split: -0.27%, +0.13%
Copies: 520969 -> 510027 (-2.10%); split: -2.20%, +0.10%
PreSGPRs: 442584 -> 421344 (-4.80%)
VALU: 3810755 -> 3810267 (-0.01%); split: -0.01%, +0.00%
SALU: 763402 -> 752650 (-1.41%); split: -1.48%, +0.07%
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31184 >
2024-09-26 14:29:14 +00:00
bcfc5c09fa
amd: add offset to is_subgroup_invocation_lt_amd
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31184 >
2024-09-26 14:29:13 +00:00
09e64e3682
nir/opt_shrink_vectors: shrink memory loads, not just IO
...
The problem with radeonsi+ACO is that UBO loads from vec4 uniforms using
only 1 component always load all 4 components. This fixes that.
We are only interested in shrinking UBO and SSBO loads, but I added more
intrinsics because why not.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29384 >
2024-09-26 03:01:38 +00:00
f6e7520b13
glsl: remove now unused linker code
...
This has all be replaced by a nir based linker implementation.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:44 +00:00
cbfc225e2b
glsl: switch to a full nir based linker
...
This commit does 3 things at once (3 squashed commits) as required
to make sure the commit doesn't break things.
1. convert to nir at compile time
2. enable full nir linking
3. switch standalone compiler to nir linker
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:44 +00:00
5108a9a37d
glsl: set blake3 hash in standalone scaffolding
...
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:44 +00:00
1c88ed6194
glsl: add lower_derivatives_without_layout() helper
...
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:44 +00:00
ff4596ae61
spirv: explicitly lower derivatives to zero
...
To allow removal of the existing nir_builder lowering.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:44 +00:00
721d23b8ff
glsl: add intrastage shader linking helpers for nir linker
...
Conversions of the existing glsl ir linking code to nir.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:44 +00:00
fe9b93fc1c
nir: handle wildcard array deref
...
Here we add handling of wildcard array derefs when attempting to mark
an io as partially used rather than hitting an assert.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:44 +00:00
6bb6b0e5ad
nir: add nir_intrinsic_deref_implicit_array_length intrinsic
...
This will be used to handle .length() calls on unsized arrays
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:44 +00:00
083b4ffb78
glsl: add gl_nir_linker_size_arrays()
...
This will size implicitly size arrays during linking, replacing the glsl
ir version.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:44 +00:00
a235da080e
glsl: add gl_nir_link_function_calls()
...
This will link functions combining multiple shaders from the same stage
into a single shader. Unlike alot of the glsl ir linker that has been
converted to NIR the logic here is completely different from the glsl ir
code that linked functions. The existing nir cloning code allows us to
implement this functionality in a much more eligant way than what glsl
ir was doing.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:44 +00:00
60937b5286
nir: add implicit_conversion_prohibited field to nir_parameter
...
Will be used in link time validation in following patches.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:44 +00:00
5645495156
nir: store variable mode in nir_parameter
...
This will be used by the nir glsl linker in following patches.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:44 +00:00
89a2411c54
nir: serialize nir_parameter type
...
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:44 +00:00
6ff3e87e5f
nir: add function in/outs to variable modes
...
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:44 +00:00
f3da074dc3
glsl: move _mesa_glsl_can_implicitly_convert() to linker_util.cpp
...
Makes more sense here as its used by both the compiler and linker.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:43 +00:00
1cb115abd2
nir: add nir_function_impl_clone_remap_globals()
...
This will be use by the glsl nir linker when we are combining
different shaders from the same shader stage that might have multiple
declarations of global variables across the different shaders.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:43 +00:00
7a1061e0dd
nir: add max_ifc_array_access field to vars
...
This will be used in following patches by the nir based glsl
linker code.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:43 +00:00
7c5b21c032
glsl: add support for converting global instructions to NIR
...
NIR doesn't really support global instructions such as global val
initilisation. So here we add functionality to glsl_to_nir() to
put these instructions into a temporary function that will be
later inlined into main.
We give the function a name starting with gl_mesa_tmp_ as functions
starting with gl_ are reserved and will not have any clashes with
user functions, we finish the name with the blake3 of the shader
source to avoid conflicts with multiple shaders attached to a single
stage.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:43 +00:00
1fc9d5223e
glsl: make gl_nir_validate_intrastage_arrays() more flexible
...
This will allow us to use it before shaders from the same stage have
been linked and merged.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:43 +00:00
ffbd763586
glsl: add gl_nir_validate_intrastage_interface_blocks()
...
This is a nir version of the existing glsl ir validation function.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:43 +00:00
90e76d34ff
mesa/glsl: add nir fields to the gl shader structs
...
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137 >
2024-09-25 09:39:43 +00:00
e0bcab953d
nir: add amd shared append/consume
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31075 >
2024-09-19 16:21:47 +00:00
eeb3512498
nir/lower_ssbo: Extend the load_ssbo_address intrinsic to pass an offset
...
On Mali(Valhall), the bounds checking can be done when in hardware, but
for this to work properly, we need to pass the offset to the
nir_load_ssbo_address() intrinsic.
Add an offset source to the intrinsic, and adjust the lowering pass
to conditionally lower the offset addition.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com >
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Acked-by: Eric R. Smith <eric.smith@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31164 >
2024-09-18 13:45:57 +00:00
adadb097a3
nir/lower_ssbo: Add an option to conditionally lower loads
...
On Mali(Valhall), we have a way to load SSBO data without going through
an SSBO index -> global address translation, so let's provide a way
to tell nir_lower_ssbo() when it shouldn't lower loads.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com >
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Acked-by: Eric R. Smith <eric.smith@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31164 >
2024-09-18 13:45:57 +00:00
3b99f9bc0e
mesa: support OVR_multiview2
...
this reuses the same pipe cap to indicate that
both extensions are supported
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31128 >
2024-09-18 13:07:41 +00:00
328c29d600
mesa,glsl,gallium: add GL_OVR_multiview
...
Co-authored-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31128 >
2024-09-18 13:07:41 +00:00
a3d6a770c0
nir/instr_set: fix fp_fast_math
...
We can't just ignore the flags of the match, we need the union.
Fixes: 666647acaefaith.ekstrand@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31195 >
2024-09-17 20:00:03 +00:00
0f6fa4679d
clc: find opencl headers from the installed llvm/clang location
...
A number of people report the headers not being found when running
intel-clc. I've run into the same issue but only on the most recent
Ubuntu version.
Signed-off-by: Lionel Landwerlin <llandwerlin@gmail.com >
Reviewed-by: Karol Herbst <kherbst@redhat.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30711 >
2024-09-17 14:29:19 +03:00
99f1b7d732
compiler/glsl: explicitly delete copy constructor and copy-assign
...
Most of our visitors have custom destructors because they wrap C types,
and need to manually call destructors for those types. Because of this,
the implicitly generated copy constructors and copy-assignment operators
are not safe due copying pointers rather than data. Since we don't need
these features just delete them, so any attempt to use them would be a
compilation error. This doesn't fix any existing issue except stopping
coverity from complaining, but it does prevent new issues in the future.
Acked-by: Caio Oliveira <caio.oliveira@intel.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29662 >
2024-09-16 21:08:04 +00:00
6a09d33549
nir: Add a pass to generate BFI instructions from logical operations
...
Inspired by a commit message in !30934 , I set about optimizing the code
generated for nir_copysign. It would be possible to just implement an
opt_algebraic pattern for the specific values used by nir_copysign, but
this casts a slightly larger net.
As noted in a comment in the code, there may be variations of the
pattern that this pass misses. The opt_algebraic pattern would miss them
too.
v2: Use nir_def_replace. Suggested by Alyssa. Allow more "root"
instruction types. Suggested by Georg.
v3: Treat extract_u16(x, 0) as (x & 0x0000ffff), and treat extract_u8(x,
0) as (x & 0x000000ff).
v4: Use nir_scalar. Suggested by Georg.
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31006 >
2024-09-13 00:21:00 +00:00
057c7c9f53
nir/algebraic: Recognize open-coded bitfield_reverse in XCOM 2
...
The XCOM 2 shaders in my shader-db use iadd instead of ior.
No fossil-db changes on any Intel platform.
shader-db:
All Intel platforms had similar results. (Meteor Lake shown)
total instructions in shared programs: 19787210 -> 19787034 (<.01%)
instructions in affected programs: 1187 -> 1011 (-14.83%)
helped: 6 / HURT: 0
total cycles in shared programs: 906024436 -> 906012612 (<.01%)
cycles in affected programs: 72978 -> 61154 (-16.20%)
helped: 6 / HURT: 0
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31006 >
2024-09-13 00:21:00 +00:00
97f4250a7c
nir: skip opt_loop_peel_initial_break if continue block only has phis
...
Doing that optimization wouldn't do anything useful in this case.
nir_block_has_non_copy() is used by opt_loop_peel_initial_break().
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31002 >
2024-09-12 23:36:58 +00:00
8410b4cdd6
nir/tests: add some loop peeling tests
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31002 >
2024-09-12 23:36:58 +00:00
64ac601049
nir/opt_loop: skip peeling if the loop ends with any kind of jump
...
Any kind of jump prevents us from moving it to the top of the loop, not
just breaks.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Fixes: 6b4b044739https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31002 >
2024-09-12 23:36:58 +00:00
af3b099e0a
nir/opt_loop: skip peeling if the break is non-trivial
...
If this nir_if contains continues or other breaks, we can't move it
outside the loop.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Fixes: 6b4b044739https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31002 >
2024-09-12 23:36:57 +00:00
4f44a944bb
nir/opt_if: fix fighting between split_alu_of_phi and peel_initial_break
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Fixes: 6b4b044739https://gitlab.freedesktop.org/mesa/mesa/-/issues/11822
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31002 >
2024-09-12 23:36:57 +00:00
7fa7812219
nir: merge out of loop decision with nir_can_move_instr logic
...
One place to modify instead of two when adding new intrinsics here.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30906 >
2024-09-12 21:49:34 +00:00
91f8e32a85
nir/opt_sink: do not sink inverse_ballot out of loops
...
Inverse_ballot result is undefined if the input is not dynamically uniform.
And sinking out of loops might make the input divergent.
Fixes: 18a0ff137fdaniel@schuermann.dev >
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30906 >
2024-09-12 21:49:34 +00:00
Rhys Perry