2e833b16bc
nir/lower_amul: Use num_ubos/ssbos instead of recomputing it.
...
Now that num_ubos is correctly maintained, we can just trust it. Fixes an
assertion failure in freedreno I triggered on
dEQP-GLES31.functional.ubo.random.all_per_block_buffers.1 for reasons I
don't really understand.
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com >
Reviewed-by: Rob Clark <robdclark@chromium.org >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6179 >
2020-08-05 04:35:05 +00:00
611f654fcf
nir/deref: Don't try to compare derefs containing casts
...
One day, we may want copy_prop_vars or other passes to be able to see
through certain types of casts such as when someone casts a uint64_t to
a uvec2. However, for now we should just avoid casts all together.
Fixes: d8e3edb784jenatali@microsoft.com >
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6072 >
2020-08-03 21:49:25 +00:00
ee2f21b10d
nir: Remove the old nir_opt_shrink_load.
...
The old pass only handled intrinsic load_constant, while the new
nir_opt_shrink_vectors handles ALU ops, nir load_consts, along with all
the load intrinsics.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6050 >
2020-08-03 21:26:45 +00:00
1c9906d5ff
nir: Add a pass to cut the trailing ends of vectors.
...
Ideally we'd also handle unused middles of vectors and reswizzle ALU-only
uses of it so we could write fewer channels, but that's future work/
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6050 >
2020-08-03 21:26:45 +00:00
d8fdb8dab4
nir: Add colorspace support to YUV lowering pass
...
This change adds support for BT709 and BT2020 colorspace to the YUV
lowering pass. The default remains BT601.
This change also fixes minor imprecision in the last digits of the BT601
offsets due to computation from rounded values when the math was
simplified.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com >
Reviewed-by: Rob Clark <robdclark@chromium.org >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6122 >
2020-07-31 07:27:03 +00:00
e1b114868a
nir: Get rid of __[u]int64_to_fp32() and __fp32_to_[u]int64()
...
Those are now handled by nir_lower_int64() which has native NIR
implementations.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com >
Reviewed-by: Matt Turner <mattst88@gmail.com >
Acked-by: Jason Ekstrand <jason@jlekstrand.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5588 >
2020-07-30 16:54:24 +00:00
936c58c8fc
nir: Extend nir_lower_int64() to support i2f/f2i lowering
...
That's an attempt at replacing the complex __int64_to_float() and
__float_to_int64() implementations found in float64.glsl by a simpler
native NIR equivalent.
Thanks to that, we can have lower those conversion without having to
compile a GLSL shader, which would be quite annoying for OpenCL
kernels.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com >
Reviewed-by: Matt Turner <mattst88@gmail.com >
Acked-by: Jason Ekstrand <jason@jlekstrand.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5588 >
2020-07-30 16:54:24 +00:00
bfee35b45c
nir: Stop passing an options arg to nir_lower_int64()
...
This information is exposed through shader->options->lower_int64_options.
Removing the extra arg forces drivers to initialize this field correctly.
This also allows us to check the int64 lowering options from each int64
lowering helper and decide if we should lower the instructions we
introduce.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com >
Reviewed-by: Matt Turner <mattst88@gmail.com >
Reviewed-by: Rob Clark <robdclark@chromium.org >
Acked-by: Jason Ekstrand <jason@jlekstrand.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5588 >
2020-07-30 16:54:24 +00:00
dd155aef44
nir: Allow casts in nir_deref_instr_get[_const]_offset()
...
Allow casts in a deref chain so we can calculate an offset from
a base pointer dereference or have pointer type casts in the
middle of the chain (both are pretty common in CL).
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com >
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5682 >
2020-07-30 17:11:43 +02:00
6a1382399c
nir: Use a switch in build_deref_offset()/deref_instr_get_const_offset()
...
We are about to add support for casts when calculating offset, but let's
first turn the if()/else if()/else block into a switch() statement to
ease addition of new cases.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com >
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5682 >
2020-07-30 17:11:35 +02:00
6b99cf6064
nir/load_store_vectorize: fix indentation
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5980 >
2020-07-29 17:57:13 +00:00
1a42e7dae9
nir: Filter modes of scoped memory barrier in nir_opt_load_store_vectorize
...
Otherwise a scoped memory barrier containing nir_var_mem_ubo (which
memoryBarrier() does lower to) would incorrectly prevent the
optimization to happen in UBOs.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5980 >
2020-07-29 17:57:13 +00:00
5c5555a862
nir: Add a find_variable_with_[driver_]location helper
...
We've hand-rolled this loop 10 places and those are just the ones I
found easily.
Reviewed-by: Eric Anholt <eric@anholt.net >
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
789ec95dcd
nir/split_per_member_structs: Inline split_variables_in_list
...
This lets us do one list walk instead of three.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
d70fff99c5
nir: Use a single list for all shader variables
...
Instead of having separate lists of variables, roughly sorted by mode,
use a single list for all shader-level NIR variables. This makes a few
list walks a bit longer here and there but list walks aren't a very
common thing in NIR at all. On the other hand, it makes a lot of things
like validation, printing, etc. way simpler. Also, there are a number
of cases where we move variables from inputs/outputs to globals and this
makes it way easier because we no longer have to move them between
lists. We only have to deal with that if moving them from the shader to
a nir_function_impl.
Reviewed-by: Rob Clark <robdclark@chromium.org >
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com >
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
c256cd900e
nir/lower_variable_initializers: Restrict the modes we lower
...
This is not a functional change because these are the only modes we
handle. All others get silently ignored.
Reviewed-by: Dave Airlie <airlied@redhat.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
4d7e064623
nir/split_per_member_structs: Use nir_variable_with_modes_safe
...
Reviewed-by: Dave Airlie <airlied@redhat.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
90cf4709d9
nir/lower_uniforms_to_ubo: Use nir_foreach_variable_with_modes
...
Reviewed-by: Dave Airlie <airlied@redhat.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
d0d5ef6139
nir/lower_two_sided_color: Use nir_variable_create
...
Reviewed-by: Dave Airlie <airlied@redhat.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
bb3994cfe7
nir/io_to_vector: Use nir_foreach_variable_with_modes
...
Reviewed-by: Dave Airlie <airlied@redhat.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
0a77c67442
nir/lower_io_to_temporaries: Use a separate list for new inputs
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
cd66005b23
nir: Use a nir_shader and mode in lower_clip_cull_distance_arrays
...
Reviewed-by: Gert Wollny <gert.wollny@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
d5d15c301e
nir/lower_amul: Add a variable mode check
...
This loop should only apply to UBOs and SSBOs because max_slot is never
used for normal uniforms.
Reviewed-by: Gert Wollny <gert.wollny@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
3be0be7d54
nir: Split nir_index_vars into two functions
...
We also very slightly change the semantics. It no longer is one index
per list for global variables and is a single index over-all.
Reviewed-by: Gert Wollny <gert.wollny@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
86c9303814
nir/split_vars: Add mode checks to list walks
...
Reviewed-by: Gert Wollny <gert.wollny@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
4c3a30393d
nir/linking: Rework some internal helpers
...
Instead of taking a variable list, take a nir_shader and mode.
Reviewed-by: Gert Wollny <gert.wollny@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
caab46c1e9
nir: Take a shader and variable mode in nir_assign_io_var_locations
...
Reviewed-by: Gert Wollny <gert.wollny@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
964c1c4b87
nir: Take a nir_shader and variable mode in assign_var_locations
...
Reviewed-by: Gert Wollny <gert.wollny@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
6f6f7a34c5
nir: Add and use a nir_variable_list_for_mode helper
...
We also add a new list iterator which takes a modes bitfield and
automatically figures out which list to use. In the future, this
iterator will work for multiple modes but today it assumes a single mode
thanks to the behavior of nir_variable_list_for_mode. This also doesn't
work for function_temp variables.
Reviewed-by: Gert Wollny <gert.wollny@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
feb32f898c
nir: Add a nir_foreach_uniform_variable helper
...
This one's a bit more complex because it filters off only those
variables with mode == nir_var_uniform. As such, it's not exactly a
drop-in replacement for nir_foreach_variable(var, &nir->uniforms).
Reviewed-by: Gert Wollny <gert.wollny@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
92dcda5ce9
nir: Add a nir_foreach_function_temp_variable helper
...
Reviewed-by: Gert Wollny <gert.wollny@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:58 +00:00
2956d53400
nir: Add nir_foreach_shader_in/out_variable helpers
...
Reviewed-by: Jose Maria Casanova Crespo <jmcasanova@igalia.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:57 +00:00
9bf8572222
nir/dead_variables: Respect the modes passed to remove_dead_vars
...
For the most part, this doesn't actually matter today. We already only
call remove_dead_vars on the lists that are specified in the modes. The
only functional change here is for the uniform, mem_ubo, and mem_ssbo
modes because they share a list. If nir_remove_dead_variables is called
with a mode of nir_var_uniform, it will no longer remove UBOs or SSBOs,
for instance.
Reviewed-by: Gert Wollny <gert.wollny@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:57 +00:00
5746af4446
nir: Take a mode in remove_unused_io_vars
...
Reviewed-by: Jose Maria Casanova Crespo <jmcasanova@igalia.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966 >
2020-07-29 17:38:57 +00:00
2adb337256
nir,radv/aco: add and use pass to lower make available/visible barriers
...
Lower them to ACCESS_COHERENT to simplify the backend and
probably give better performance than invalidating or writing back the
entire L0/L1 cache.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4905 >
2020-07-28 16:56:34 +00:00
17fd191eed
nir/lower_clip: make the pass compatible with Vulkan semantics
...
Vulkan allows fragment shaders to read gl_ClipDistance[], in which
case the SPIR-V compiler inserts a single compact array variable for
VARYING_SLOW_CLIP_DIST0 and the lowering should not try to inject
its own variables, but instead work in terms of the existing one.
Vulkan drivers are expected to call this with use_clipdist_array set
to true to be consistent with this setup.
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6022 >
2020-07-27 08:25:57 +02:00
e2e89fb137
nir/lower_io: assert that offsets are used for shader_in
...
Signed-off-by: Karol Herbst <kherbst@redhat.com >
Reviewed-by: Jesse Natalie <jenatali@microsoft.com >
Acked-by: Jason Ekstrand <jason@jlekstrand.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6059 >
2020-07-25 08:51:48 +00:00
56846a2b68
nir/schedule: Add an option for a fallback scheduling algorithm
...
The current scheduling algorithm favors parallelism a bit too
aggressively and sometimes generates shaders that fail register
allocation. This happens even if the threshold is set to zero to force
it to always use the CSR instruction choosing algorithm.
This patch adds an option to use an even more aggressive fallback that
just always picks the instruction with the shortest maximum delay in the
hope that that will generate the least register pressure. The intention
is to use this as a last resort after register allocation fails in order
to at least have a working shader.
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com >
Acked-by: Iago Toral Quiroga <itoral@igalia.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953 >
2020-07-24 12:26:26 +02:00
bafd259177
nir/schedule: Add a callback for backend-specific dependencies
...
Adds a callback function to nir_schedule_options to give the backend a
chance to add custom dependencies between certain intrinsics. The
callback can assign a class number to the intrinsic and then set a read
or write dependency on that class.
v2: Use a linked-list of schedule nodes for the dependency classes
instead of a fixed-sized array.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com >
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953 >
2020-07-24 09:21:11 +02:00
260a8f759a
nir/schedule: Store a pointer to the options struct in scoreboard
...
Instead of copying the individual members of nir_schedule_options into
the scoreboard, it now just keeps a pointer to the options. This avoids
the duplicated comments and makes it easier to add more options later.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com >
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953 >
2020-07-24 09:21:11 +02:00
7665398e6c
nir/scheduler: Move nir_scheduler to its own header
...
nir_schedule already has a struct for options which makes it more than
just a function declaration. Later patches intend to add more structs to
complement these options. In order to make the code easier to manage,
this moves the nir_scheduler-related parts out of nir.h to their own
header.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com >
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953 >
2020-07-24 09:21:11 +02:00
953b7a3603
spirv: Use nir_bany/ball for OpAny/All
...
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5278 >
2020-07-23 22:43:20 -05:00
456edf0b30
nir: Support 8 and 16 component vectors for reduceable intrinsics
...
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com >
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com >
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6030 >
2020-07-23 18:23:20 -07:00
d572f4dfd9
nir: Support algebraic opts on vectors larger than 4
...
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com >
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com >
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6030 >
2020-07-23 18:23:17 -07:00
aa581fcc69
nir: Support vec8/vec16 in nir_lower_bit_size
...
Reviewed-by: Gert Wollny <gert.wollny@collabora.com >
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com >
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6030 >
2020-07-23 18:23:07 -07:00
ce4064fe2f
nir/ lower_int_to_float: Handle umax and umin
...
8e1b75b3anarsoul@gmail.com >
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net >
Signed-off-by: Andreas Baierl <ichgeh@imkreisrum.de >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6043 >
2020-07-23 14:45:32 +00:00
772ed657a2
nir_ allow nir_lower_clip_halfz to run in tess eval shader
...
Reviewed-by: Eric Anholt <eric@anholt.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6027 >
2020-07-23 08:41:17 +00:00
09ecfd95ee
nir: allow lower_psiz_mov to run in tessellation stages
...
Reviewed-by: Eric Anholt <eric@anholt.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6027 >
2020-07-23 08:41:17 +00:00
c30824adc0
nir/lower_io: Add support for global scratch addressing
...
This provides an alternate lowering for scratch in which it uses global
reads/writes and bases scratch addresses on a base pointer.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com >
Reviewed-by: Eric Anholt <eric@anholt.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5927 >
2020-07-22 23:43:35 +00:00
4815ae51d7
nir/lower_io: Use b2b for shader and function temporaries
...
This way we can avoid some unnecessary conversions because there's no
need to sanitize to 0/1 for scratch.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com >
Reviewed-by: Eric Anholt <eric@anholt.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5927 >
2020-07-22 23:43:35 +00:00
Eric Anholt