0210b554d6
treewide: Replace the include of nir_types.h with glsl_types.h
...
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com >
Acked-by: Caio Oliveira <caio.oliveira@intel.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26753 >
2023-12-30 15:08:11 +00:00
da6a5e1f63
nir: Add pass for clearing memory at the end of a shader.
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26679 >
2023-12-20 09:15:45 +00:00
bc99b73d70
nir: Add nir_static_workgroup_size helper.
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26679 >
2023-12-20 09:15:45 +00:00
3e042173e4
nir/lower_doubles: Add lowering for fmin/fmax/fsat
...
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26587 >
2023-12-20 02:40:25 +00:00
6cad2fc230
nir: add helper to create cursor after all @decl_regs
...
@decl_reg intrinsics must be in the first block so it's convenient to be
able to create an insertion point after all @decl_regs when the first
block needs to be split.
Signed-off-by: Job Noorman <jnoorman@igalia.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26737 >
2023-12-18 14:52:02 +00:00
6e7a61df4c
nir: add _safe variants of nir_foreach_reg_load/store
...
Signed-off-by: Job Noorman <jnoorman@igalia.com >
Reviewed-by: Rob Clark <robdclark@chromium.org >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26175 >
2023-12-15 17:19:28 +00:00
aac1e3f595
nir: Add a new has_fmulz_no_denorms flag
...
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26569 >
2023-12-11 15:29:17 +00:00
6979a1aa07
nir/opt_preamble: make load_workgroup_size handling optional
...
not all drivers support it being in the preamble, e.g. asahi.
Signed-off-by: Karol Herbst <git@karolherbst.de >
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26614 >
2023-12-09 10:56:37 -04:00
c468aa6059
nir: add nir_fixup_deref_types()
...
This will be used to fix up types after arrays have been resized.
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26534 >
2023-12-07 03:05:09 +00:00
09fc5e1c4d
nir: Split has_[su]dot_4x8 bits into regular and _sat versions
...
Reviewed-by: Karol Herbst <kherbst@redhat.com >
Reviewed-by: Jesse Natalie <jenatali@microsoft.com >
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26533 >
2023-12-06 23:15:33 +00:00
629af540ca
spirv: Plumb variable alignments through to NIR
...
Reviewed-by: Karol Herbst <kherbst@redhat.com >
Reviewed-by: Jesse Natalie <jenatali@microsoft.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26522 >
2023-12-06 18:37:36 +00:00
e14633fa7d
nir/lower_tex: Add 1D lowering
...
From amd/common.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26377 >
2023-11-28 20:32:03 +00:00
27a9ddad28
nir: return progress from nir_remove_sysval_output
...
Changing IO semantics doesn't affect the SSA structure.
Same as nir_remove_varying.
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26274 >
2023-11-24 15:37:24 +00:00
32ee6376ad
nir: add lowering from FS LAYER input to LAYER_ID sysval
...
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26274 >
2023-11-24 15:37:24 +00:00
110887de2b
glsl/nir: implement a nir based lower distance pass
...
This is based off the original GLSL IR pass but it is much much
simpler as it doesn't need to do all of the hackery required in
GLSL IR to achieve the lowering.
Acked-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25860 >
2023-11-21 00:45:21 +00:00
7e4aac46ad
nir: add force_f2f16_rtz option to lower f2f16 to f2f16_rtz
...
Used by OpenGL driver like radeonsi which has undefined rounding mode.
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25990 >
2023-11-20 02:20:17 +00:00
df4fc66649
nir: Add a flag to opt_if to prevent fighting with splitting 64bit phis
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Bill Kristiansen <billkris@microsoft.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26210 >
2023-11-17 19:19:11 +00:00
6d73f62e42
nir: Simplify nir_alu_instr_channel_used definition
...
Deduplicates the "get # of channels" logic which was the same between the
helpers.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26084 >
2023-11-17 09:36:08 -04:00
1dab2c5bd2
nir/subgroups: Add option to lower Boolean subgroup reductions
...
This will be useful for AMD, and probably Intel as well.
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/218 >
2023-11-17 09:45:40 +00:00
288e9db053
nir/lower_fp16_casts: add option to split fp64 casts
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Ivan Briano <ivan.briano@intel.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25566 >
2023-11-16 11:02:30 +00:00
41f814df6f
nir: allow vec derefs on system values
...
There is no real reason to prevent this as far as I know. And some of the
SPIR-V generated by DPCPP is running into this.
Signed-off-by: Karol Herbst <kherbst@redhat.com >
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25701 >
2023-11-15 08:34:57 +00:00
cc3f20ca6c
nir: Also gather decomposed primitive count
...
Simple extension.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056 >
2023-11-07 00:05:54 +00:00
b65636ca40
nir/lower_gs_intrinsics: Count decomposed primitives too
...
We need both: decomposed primitives for transform feedback and regular
primitives for the sizing the index buffer.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056 >
2023-11-07 00:05:54 +00:00
0a35aa3a2b
nir/lower_gs_intrinsics: Append EndPrimitive
...
This is simpler for generic GS lowering.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056 >
2023-11-07 00:05:54 +00:00
1793adbd3a
nir/validate: Allow array derefs on vectors on function/shader_temp
...
This is required by OpenCL.
Reviewed-by: Karol Herbst <kherbst@redhat.com >
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22580 >
2023-11-02 20:28:46 +00:00
23bea25207
nir: Add nir_remove_non_exported
...
For libraries.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com >
Reviewed-by: Karol Herbst <kherbst@redhat.com >
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25498 >
2023-11-02 11:37:46 +00:00
6014f745d5
nir,vtn: Add exported bool to nir_function
...
For optimizing libraries.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com >
Reviewed-by: Karol Herbst <kherbst@redhat.com >
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25498 >
2023-11-02 11:37:46 +00:00
debddca134
nir: add helpers to skip idempotent passes
...
For example, in the loop:
while (more_late_algebraic) {
more_late_algebraic = false;
NIR_PASS(more_late_algebraic, nir, nir_opt_algebraic_late);
NIR_PASS(_, nir, nir_opt_constant_folding);
NIR_PASS(_, nir, nir_copy_prop);
NIR_PASS(_, nir, nir_opt_dce);
NIR_PASS(_, nir, nir_opt_cse);
}
if nir_opt_algebraic_late makes no progress, later passes might be
skippable depending on which ones made progress in the previous iteration.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24197 >
2023-11-01 14:16:37 +00:00
d5c310899a
nir: Split nir_lower_subgroup_options::lower_vote_eq into two bits
...
On NVIDIA, we can do a vote_ieq on bool in one hardware op so we don't
want that lowered. We do want to lower vote_feq and other vote_ieq,
though.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25894 >
2023-10-26 23:05:44 +00:00
16664b74a2
nir: Add a lower_read_first_invocation option to lower_subgroups
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25894 >
2023-10-26 23:05:44 +00:00
3d027cca1e
nir: Add a lower_first_invocation_to_ballot option to lower_subgroups
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25894 >
2023-10-26 23:05:44 +00:00
eabd93bba8
docs/nir: use hawkmoth instead of doxygen
...
Use the hawkmoth c:auto* directives to incorporate nir documentation.
Convert @param style parameter descriptions to rst info field lists.
Add static stubs for generated headers. Fix a lot of references, in
particular the symbols are now in the Sphinx C domain, not C++
domain. Tweak syntax here and there.
Based on the earlier work by Erik Faye-Lund <kusmabite@gmail.com >
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24507 >
2023-10-26 16:13:26 +00:00
d407cd8216
nir: drop **< style documentation comments
...
Prepare for using Hawkmoth.
Hawkmoth does not support trailing comments using /**< ... */
syntax. Replace with regular documentation comments.
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24507 >
2023-10-26 16:13:25 +00:00
4024d0c196
nir: add names to some typedef'd structs/enums
...
In order to document "typedef struct { ... } T;" as a struct in
hawkmoth, the structs need to have names. Similar for enums.
Note: This is no longer required with Hawkmoth 0.16.0+ and Clang 16 and
later. With the next Hawkmoth release, this should be fixed also for
Clang 15 and earlier.
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24507 >
2023-10-26 16:13:25 +00:00
1fa7c37a36
nir: Add NVIDIA-specific I/O intrinsics
...
NVIDIA hardware doesn't take a vertex index for per-vertex I/O.
Instead, it takes an offset into the primitive. This has to be fetched
using a combination of SR_INVOCATION_INFO and the ISBERD instruction.
To keep things simple and allow for maximum CSE, we do the lowering in
NIR and patch the load/store_per_vertex_input/output intrinsic.
Reviewed-by: Emma Anholt <emma@anholt.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000 >
2023-10-24 22:21:18 +00:00
8188842fdc
nir: Add a range to most I/O intrinsics
...
Reviewed-by: Emma Anholt <emma@anholt.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000 >
2023-10-24 22:21:18 +00:00
1a2e8290ab
nir: Add NV-specific texture opcodes
...
These are for implementing various texture queries.
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Emma Anholt <emma@anholt.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000 >
2023-10-24 22:21:18 +00:00
5984265d45
nir: Add a load_sysval_nv intrinsic
...
Reviewed-by: Emma Anholt <emma@anholt.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000 >
2023-10-24 22:21:18 +00:00
abf3175161
nir/lower_tex: Add a lower_txd_clamp option
...
Some of us want to lower all TXD with min_lod regardless of whether or
not it's shadow or cube or whatever.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Emma Anholt <emma@anholt.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000 >
2023-10-24 22:21:17 +00:00
0680330cf7
nir: Add a nir_ssa_def_all_uses_are_fsat() helper
...
Extracted from nir_lower_to_source_mods, this is useful for back-ends
which don't want to rely on NIR source and destination modifiers.
Reviewed-by: Emma Anholt <emma@anholt.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000 >
2023-10-24 22:21:17 +00:00
b416248cb5
nir: Add nir_lower_dsign as 64-bit fsign lowering.
...
Right now some drivers are doing dsign lowering in GLSL and haven't had to
have a NIR path due to there not being a corresponding vulkan driver. We
want this in NIR now so that we can retire that batch of GLSL lowering
code.
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25777 >
2023-10-24 00:16:30 +00:00
f3886e9c02
nir: split FLOAT_CONTROLS_SIGNED_ZERO_INF_NAN_PRESERVE_FP* flags
...
GLSL doesn't preserve NaNs, but it optionally preserves Infs.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25392 >
2023-10-17 17:27:12 +00:00
335cf5f22f
nir: Use a tagged pointer for nir_src parents
...
This allows us to pack the is_if boolean into the bottom bit of the parent
pointer, eliminating the boolean and hence shrinking the nir_src by 8 bytes (due
to the extra 63 bits of padding incurred in the old layout).
Because all access is forced through helpers now, this is a local change.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24671 >
2023-10-10 04:58:05 -04:00
316af8c965
nir: Assert the nir_src union is used safely
...
It is undefined behaviour in C to read a different member of a union than was
written. Nothing in-tree should be using this behaviour with the nir_src union:
nir_if should never be read as nir_instr and vice versa. Assert this.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24671 >
2023-10-10 04:58:05 -04:00
c39896b17b
nir: Use getters for nir_src::parent_*
...
First, we need to give the parent_instr field a unique name to be able to
replace with a helper. We have parent_instr fields for both nir_src and
nir_def, so let's rename nir_src::parent_instr in preparation for rework.
This was done with a combination of sed and manual fix-ups.
Then we use semantic patches plus manual fixups:
@@
expression s;
@@
-s->renamed_parent_instr
+nir_src_parent_instr(s)
@@
expression s;
@@
-s.renamed_parent_instr
+nir_src_parent_instr(&s)
@@
expression s;
@@
-s->parent_if
+nir_src_parent_if(s)
@@
expression s;
@@
-s.renamed_parent_if
+nir_src_parent_if(&s)
@@
expression s;
@@
-s->is_if
+nir_src_is_if(s)
@@
expression s;
@@
-s.is_if
+nir_src_is_if(&s)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24671 >
2023-10-10 04:58:05 -04:00
19f8e0e3aa
nir: Add trivial nir_src_* getters
...
These will become nontrivial later in the series. For now these have no smarts
in them, in order to make the conversion completely mechanical.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24671 >
2023-10-10 04:58:04 -04:00
348eee9c97
nir: handle nir_var_mem_ubo in nir_clone_uniform_variable
...
for UBOs
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25394 >
2023-10-07 11:18:40 +00:00
b47b8d16d9
nir: expose reusable linking helpers for cloning uniform loads
...
for the new varying optimizer
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25394 >
2023-10-07 11:18:40 +00:00
0f2491cbdd
nir: add dual-slot input information into load_input intrinsics
...
This is necessary to allow optimizing VS inputs after nir_lower_io, which
is currently impossible because the loss of dual-slot information in NIR
would break VS inputs. With this, driver locations can be recomputed by
calling nir_recompute_io_bases. It's a prerequisite for optimizing varyings
with lowered IO.
When this is used, we will be able to eliminate unused dual-slot VS inputs
as well as unused low and high halves of dual-slot VS inputs for the first
time, which can happen due to optimizations of varyings. Without this,
st/mesa binds vertex buffers for dual-slot inputs that are fully or
partially unused in the shader.
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25394 >
2023-10-07 11:18:40 +00:00
f37e32b78b
nir: sort variables by location in nir_lower_io_passes to work around a bug
...
I don't know why this is necessary, but it unblocks the work on varying
optimizations.
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25394 >
2023-10-07 11:18:40 +00:00
Yonggang Luo