b75b9d6324
ac/llvm: set invariant.load on image loads with ACCESS_CAN_REORDER
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146 >
2022-12-06 13:27:16 +00:00
de3fcc77c7
ac/llvm: inline ac_get_load_intr_attribs
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146 >
2022-12-06 13:27:16 +00:00
ce860953a6
ac/llvm: rename attribute enums
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146 >
2022-12-06 13:27:16 +00:00
ce0a83efb8
ac/llvm: use invariant.load metadata on load intrinsics instead of readnone
...
readnone is illegal now and we should be using invariant.load since LLVM 15.
(suggested by Nicolai)
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146 >
2022-12-06 13:27:16 +00:00
d3a03d24fc
ac/llvm: don't set "readnone" on non-memory intrinsics
...
It's illegal and LLVM always knows which intrinsics don't read memory.
This started failing IR validation with LLVM 16.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146 >
2022-12-06 13:27:16 +00:00
50bece9322
ac/llvm: don't set "convergent" on intrinsics where it's automatic
...
LLVM adds the flag automatically.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146 >
2022-12-06 13:27:16 +00:00
2b5edf96ff
ac/llvm: simplify how function attributes are set
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146 >
2022-12-06 13:27:16 +00:00
9b4142ae9f
ac/llvm: simplify how call attributes are set
...
set them directly in ac_build_intrinsic, the only place that sets them
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146 >
2022-12-06 13:27:16 +00:00
e94b040add
ac/llvm: remove illegal function attributes READONLY and WRITEONLY
...
They are only legal on parameters.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146 >
2022-12-06 13:27:16 +00:00
5104f4d496
ac/llvm: remove unused AC_FUNC_ATTR_LEGACY
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146 >
2022-12-06 13:27:16 +00:00
834d59a83e
ac/llvm: remove AC_FUNC_ATTR_INACCESSIBLE_MEM_ONLY because LLVM 16 removed it
...
We didn't use it reasonably anyway.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146 >
2022-12-06 13:27:15 +00:00
6c44d92362
ac/llvm,radeonsi: lower attribute ring intrinsics in nir
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010 >
2022-12-02 07:34:32 +00:00
daaa8ddb8e
ac/llvm,radeonsi: lower nir primitive counter add intrinsics
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010 >
2022-12-02 07:34:31 +00:00
bb837bf6ef
nir,ac/llvm: add nir_buffer_atomic_add_amd
...
Used by radeonsi for lower nir_atomic_add_gen/xfb_prim_count_amd.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010 >
2022-12-02 07:34:31 +00:00
7cec2e7520
ac/llvm,radeonsi: lower nir_load_streamout_buffer_amd
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010 >
2022-12-02 07:34:31 +00:00
daf5d30b59
ac/llvm,radeonsi: lower nir_load_user_clip_plane in abi
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010 >
2022-12-02 07:34:31 +00:00
84abc307a5
ac/llvm: remove lowered abi->intrinsic_load() intrinsics
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010 >
2022-12-02 07:34:31 +00:00
8030fbcf16
nir,ac/llvm: add nir_load_smem_buffer_amd
...
Used by radeonsi to load const buffer.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010 >
2022-12-02 07:34:31 +00:00
73ea7d651a
ac/llvm: nir_load_smem_amd support 32bit base address
...
For radeonsi which use 32bit address in ac_build_load_to_sgpr().
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010 >
2022-12-02 07:34:31 +00:00
9b2ec290c4
ac/llvm: remove unused llvm cull
...
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17109 >
2022-12-02 04:37:23 +00:00
7e1b804992
radeonsi: implement two lds base load intrinsics
...
LDS will be accessed starting from esgs_ring which has offset 0.
So ngg_scratch and ngg_emit base address is just the offset from
the esgs_ring base.
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17109 >
2022-12-02 04:37:23 +00:00
3c1ebebeae
radeonsi: use nir_lower_gs_intrinsics
...
Replace some llvm code.
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17109 >
2022-12-02 04:37:23 +00:00
9b19a93319
ac/nir: combine shift translation
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Mihai Preda <mhpreda@gmail.com >
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19966 >
2022-11-25 14:40:22 +00:00
064336d359
ac/nir: mask shift operands
...
NIR shifts are defined to truncate the shift amount to the number of bits
needed to represent the bit-size of the value shifted. LLVM treats large
shifts as poison. This fix achieves NIR semantics for shifts.
As an example, a|(b << 32), where "a" is 32bits, should produce a|b
according to NIR (because 32&31 == 0).
This caused LLVM to incorrectly optimize "(a >> c) | (b << (32 - c))" to a
u2u32(pack_64_2x32(a, b) >> c) (v_alignbit_b32), when the original NIR
should have returned "a | b" if c==0.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Mihai Preda <mhpreda@gmail.com >
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19966 >
2022-11-25 14:40:22 +00:00
e3b1f26a2b
ac/llvm: fix 16bit varying llvm compile error
...
Found when 16bit vec3 varying with llvm14 (not found
when llvm15), one 32bit vec4 slot is filled like this:
vec3[0] | undef
vec3[1] | undef
vec3[2] | undef
undef | undef
LLVM error is for the elements with undef:
%287 = insertelement float %280, half %279, i64 0
After this change, we get:
%287 = insertelement <2 x half> %280, half %279, i64 0
Fixes: 279eea5bdapierre-eric.pelloux-prayer@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19848 >
2022-11-21 01:06:56 +00:00
88b1bb326d
ac/llvm: fix gfx11 fs input load for 16bit varying
...
Otherwise we get empty output.
Fixes: b07204d780pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19637 >
2022-11-15 05:43:37 +00:00
3aaf7c103e
ac/llvm: implement intrinsic for gfx11 param export
...
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19429 >
2022-11-11 04:22:20 +00:00
533b39bfcb
nir,ac/llvm,radeonsi: add nir_load_clamp_vertex_color_amd
...
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19429 >
2022-11-11 04:22:20 +00:00
24e8a6f17c
ac/llvm: don't use the mbcnt workaround for LLVM 16 and set range metadata
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19477 >
2022-11-06 18:16:48 -05:00
8956682810
amd: rename enums ARCTURUS -> MI100, ALDEBARAN -> MI200
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19477 >
2022-11-06 17:20:39 -05:00
c4104482c8
ac/llvm: remove unnecessary gfx11 condition for nir_intrinsic_store_buffer_amd
...
ac_build_buffer_store_dword is responsible for this.
Reviewed-by: Mihai Preda <mhpreda@gmail.com >
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19419 >
2022-11-04 00:42:07 +00:00
e6d26cb288
nir,ac/nir,aco,radv: replace has_input_*_amd with more general intrinsics
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19228 >
2022-10-31 14:33:43 +00:00
0ac37b595a
nir: add nir_intrinsic_optimization_barrier_vgpr_amd for LLVM
...
We need this for the MSAA resolve shader.
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Reviewed-by: Mihai Preda <mhpreda@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19243 >
2022-10-29 18:38:33 +00:00
13fb7f8f2c
ac/nir/ngg,ac/llvm,aco: save nogs ngg culling one lds dword
...
TES rel patch id is <256, so we can use an existing unused LDS
byte instead of extra dword.
To ease the programing, change the index of repacked_arg_vars
for these variables.
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18832 >
2022-10-27 07:35:01 +00:00
238eeeacb2
ac/llvm: get back intrinsics used by NGG
...
Will be used by radeonsi.
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18832 >
2022-10-27 07:35:01 +00:00
3a3df9acda
ac/llvm: add support for device clock on GFX11
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19267 >
2022-10-25 20:22:48 +02:00
382831c986
radv,nir: add intrinsics for streamout and GS copy shaders
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19302 >
2022-10-25 17:35:08 +00:00
cf74cf3901
radeonsi: implement nir shader query enabled intrinsics
...
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17457 >
2022-10-25 12:58:43 +00:00
a119a6464f
nir,ac,radv: add primitive count add intrinsics
...
radeonsi use shader buffer, but radv use gds for the query
result storage.
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Signed-off-by: Qiang Yu <yuq825@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17457 >
2022-10-25 12:58:43 +00:00
cea6723243
ac/llvm: Implement signed idot on GFX11.
...
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19114 >
2022-10-24 19:07:16 +00:00
434e74e658
radeonsi: zero init using the correct components count
...
This fixes the replay of the trace attached to
https://gitlab.freedesktop.org/mesa/mesa/-/issues/7530 which
would otherwise fail with an LLVM error:
"All operands to PHI node must be the same type as the PHI node!"
Fixes: ae4379d81emarek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19189 >
2022-10-24 09:33:48 +02:00
c5bf63753d
ac/nir,radv/llvm: add support for the attribute ring
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19216 >
2022-10-24 07:55:06 +02:00
049b60cf4b
ac/llvm: Implement [ui]find_msb_rev.
...
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com >
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18951 >
2022-10-22 11:57:06 +02:00
8c716525cf
ac/llvm: ignore g16 if the image opcode doesn't use derivatives
...
g16 is irrelevant in this case.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19198 >
2022-10-21 19:09:49 +00:00
c492779924
ac/llvm: fix mixing non-uniform/uniform sampler/texture descriptors
...
Non-uniform descriptors are not loaded in NIR and instead the pointer is
passed to abi->load_sampler_desc. It can be the case that only the texture
or sampler uses this path.
For the descriptor which doesn't use the path, we would previously pass
NULL to abi->load_sampler_desc.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19198 >
2022-10-21 19:09:49 +00:00
19b6c889be
Revert "ac: use LLVMContextSetOpaquePointers if available"
...
This reverts commit 940734630dmhpreda@gmail.com >
Reviewed-by: Dave Airlie <airlied@redhat.com >
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19035 >
2022-10-21 07:56:38 +00:00
f5abb7ce4c
ac/llvm: pass ac_llvm_ptr instead of separate type/ptr
...
This simplifies the API.
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19035 >
2022-10-21 07:56:38 +00:00
efa5c52822
ac/llvm: add ac_get_ptr_arg
...
For argument of type AC_AR_..._PTR, this returns a ac_llvm_pointer.
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19035 >
2022-10-21 07:56:38 +00:00
b31affddf4
ac/llvm: remove gep_2 and others temporary functions
...
Reviewed-by: Mihai Preda <mhpreda@gmail.com >
Reviewed-by: Dave Airlie <airlied@redhat.com >
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19035 >
2022-10-21 07:56:38 +00:00
f12adf2cc9
ac/llvm: remove unneeded casts
...
The only useful information in pointer types is the address space and these
casts don't change it.
The type information is explicitely passed by the caller of GEP2 and similar
functions.
Reviewed-by: Mihai Preda <mhpreda@gmail.com >
Reviewed-by: Dave Airlie <airlied@redhat.com >
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19035 >
2022-10-21 07:56:38 +00:00
Marek Olšák