AlexIndustrial/mesa
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
185,394 Commits 1 Branch 0 Tags
9bfccc1935340c581cbdc760e476d7e58808f587
Commit Graph

1102 Commits

Author SHA1 Message Date
Bas Nieuwenhuizen
c7b2ac3377 radv: Remove ray_launch_size_addr_amd system value. 
Not used anymore, so clean it up.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27664>
 2024-02-17 11:08:16 +00:00
Daniel Schürmann
2649717a36 aco: enable WQM if demote is used with maximal reconvergence 
If otherwise no helper lanes are required by the shader, then demote
behaves like discard and immediately terminates the invocations.
With maximal reconvergence, however, we need to ensure that helper lanes
are not terminated unless the entire quad was demoted.
In order to fix this, generally enable helper lanes in this unlikely
corner case and avoid a major refactor.

Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27277>
 2024-02-09 05:32:35 +00:00
Marek Olšák
72948d9ff9 radeonsi,aco: remove the VS prolog 
The upside is that this removes 600 lines of code. The downside is
that if instance divisors are used, we will compile the VS on demand.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27120>
 2024-02-07 09:50:53 +00:00
Georg Lehmann
4c74077b62 aco: implement rotate 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27118>
 2024-01-24 16:38:40 +00:00
Georg Lehmann
0a03cf5b3c aco: remove boolean shuffle isel 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27116>
 2024-01-19 20:13:34 +00:00
Georg Lehmann
6b031daf16 aco: implement as_uniform and ballot_relaxed 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27116>
 2024-01-19 20:13:34 +00:00
Rhys Perry
ae54cbeb3f nir: remove sad_u8x4 
All uses of this can be replaced with msad_4x8.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26907>
 2024-01-05 18:55:22 +00:00
Rhys Perry
1410735a62 aco: implement msad_4x8 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26907>
 2024-01-05 18:55:22 +00:00
Friedrich Vock
1e3541728b radv,aco: Convert 1D ray launches to 2D 
Because we use unaligned dispatches, 1D launches only use 8 threads per
wave. Converting to 2D and fixing up launch IDs in the prolog
significantly increases occupancy.

Gives ~30% uplift in Ghostwire Tokyo.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26105>
 2024-01-05 17:08:05 +00:00
Georg Lehmann
71edf4de5e aco/gfx12: implement broadcast dmask shrink behavior 
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26897>
 2024-01-05 12:03:54 +00:00
Georg Lehmann
4a6ee2c483 aco: shrink buffer stores with undef/zero components 
Buffer stores store 0 like image stores for unspecified components.

Foz-DB Navi21:
Totals from 91 (0.11% of 79330) affected shaders:
Instrs: 63327 -> 63121 (-0.33%)
CodeSize: 315312 -> 314440 (-0.28%); split: -0.28%, +0.00%
VGPRs: 3144 -> 3120 (-0.76%)
Latency: 441424 -> 441300 (-0.03%); split: -0.03%, +0.00%
InvThroughput: 65501 -> 65130 (-0.57%)
Copies: 6197 -> 5999 (-3.20%)
PreVGPRs: 2197 -> 2182 (-0.68%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26897>
 2024-01-05 12:03:54 +00:00
Georg Lehmann
9ecfd7919b aco: optimize 32bit fsign by using fmulz with Inf 
2 instruction fsign with the power of cursed DX9 floating point rules.

Foz-DB Navi31:
Totals from 3803 (4.86% of 78196) affected shaders:
Instrs: 8436366 -> 8412549 (-0.28%); split: -0.29%, +0.00%
CodeSize: 43174284 -> 43114676 (-0.14%); split: -0.14%, +0.01%
SpillSGPRs: 3241 -> 3247 (+0.19%)
Latency: 66333841 -> 66287361 (-0.07%); split: -0.08%, +0.01%
InvThroughput: 10331902 -> 10316916 (-0.15%); split: -0.15%, +0.01%
VClause: 165455 -> 165472 (+0.01%); split: -0.01%, +0.02%
SClause: 242352 -> 242335 (-0.01%); split: -0.02%, +0.01%
Copies: 604086 -> 605781 (+0.28%); split: -0.04%, +0.32%
Branches: 214017 -> 214013 (-0.00%)
PreSGPRs: 209413 -> 209726 (+0.15%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26765>
 2024-01-02 13:07:30 +01:00
Samuel Pitoiset
81eeb157f8 aco: export depth/stencil/samplemask in create_fs_jump_to_epilog() 
This currently has no effects because the store_output instructions
are removed earlier (in ac_nir_lower_ps). Though, this will be needed
for exporting MRTZ from PS epilogs for alpha to coverage on RDNA3.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26413>
 2023-12-06 11:49:31 +00:00
Qiang Yu
7656251294 aco: fix set_wqm segfault when ps prolog 
ps prolog does not have nir shader.

Fixes: 3b10547e67 ("aco: enable helper lanes if shader->info.fs.require_full_quads")
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26512>
 2023-12-06 05:34:30 +00:00
Rhys Perry
e110eac171 aco: insert p_end_wqm before p_jump_to_epilog 
Otherwise, we can transition to exact before p_jump_to_epilog, then
transition to WQM again and then back to exact:
p_jump_to_epilog //transitions to exact
p_logical_end //transitions to wqm
p_end_wqm //transitions to exact

We rely on ssa elimination to clean most of this up.

fossil-db (navi21):
Totals from 1 (0.00% of 79330) affected shaders:
Instrs: 111 -> 110 (-0.90%)
CodeSize: 572 -> 568 (-0.70%)
Copies: 16 -> 15 (-6.25%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25440>
 2023-12-05 21:02:04 +00:00
Rhys Perry
468ee8b80c aco: implement 16-bit fsat on GFX8 
GFX8 doesn't have v_med3_f16.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26445>
 2023-12-05 16:56:58 +00:00
Rhys Perry
de51a21e26 aco: implement 16-bit derivatives 
These are used by radeonsi.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26445>
 2023-12-05 16:56:58 +00:00
Rhys Perry
997a0884a5 aco: implement 16-bit fsign on GFX8 
GFX8 doesn't have v_med3_i16.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26445>
 2023-12-05 16:56:58 +00:00
Rhys Perry
b7725b072b aco: flush denormals for 16-bit fmin/fmax on GFX8 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26445>
 2023-12-05 16:56:57 +00:00
Daniel Schürmann
3b10547e67 aco: enable helper lanes if shader->info.fs.require_full_quads 
This enables helper invocations also for lowered quad group operations.

Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26026>
 2023-11-22 11:32:53 +01:00
Samuel Pitoiset
e1345c5295 aco: rename color_exports to exports in create_fs_jump_to_epilog() 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26231>
 2023-11-21 08:47:50 +00:00
Qiang Yu
1fabf535fa aco: handle GL_TEXTURE_RECTANGLE in tg4_integer_workarounds 
Ported from LLVM side lower_gather4_integer().

Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26244>
 2023-11-20 02:59:23 +00:00
Qiang Yu
695fc67baa aco: set MIMG unrm for GL_TEXTURE_RECTANGLE 
This fixes VDPAU compositor shaders compiled by ACO.

Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26244>
 2023-11-20 02:59:23 +00:00
Daniel Schürmann
f2bb7b185d aco: delete instruction selection for boolean subgroup operations 
These are now lowered in NIR.

Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/218>
 2023-11-17 09:45:40 +00:00
Daniel Schürmann
88afbbba11 nir: optimize open-coded quadVote* directly to new nir_quad intrinsics 
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/218>
 2023-11-17 09:45:40 +00:00
Connor Abbott
387e698bde amd: Implement quad_vote intrinsics 
Co-authored-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/218>
 2023-11-17 09:45:40 +00:00
Rhys Perry
ae30edd2a7 aco: remove f16<->f64 conversions 
radeonsi and RADV now use nir_lower_fp16_casts.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25566>
 2023-11-16 11:02:31 +00:00
Georg Lehmann
d9c3ba3b90 aco: use correct operand size for int tg4 wa 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26163>
 2023-11-15 12:35:32 +00:00
Georg Lehmann
509ce19643 aco: add missing scc def for SALU quad broadcast 
Cc: mesa-stable

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26163>
 2023-11-15 12:35:32 +00:00
Rhys Perry
b70c235e4a aco: skip LS VGPR initialization bug workaround if the prolog exists 
Otherwise, we would do this twice.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26111>
 2023-11-13 12:09:55 +00:00
Rhys Perry
967c52097e aco: workaround LS VGPR initialization bug in RADV prologs 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26111>
 2023-11-13 12:09:53 +00:00
Tatsuyuki Ishi
55d21f2f12 radv, aco: Inline struct aco_vs_input_state. 
Now that we no longer use the radv_vs_input_state pointer, we can simply
inline all the state-related fields.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26023>
 2023-11-13 11:47:42 +00:00
Tatsuyuki Ishi
3fc3a94bce radv, aco: Rework VS prolog key handling. 
The main change is to use struct radv_vs_prolog_key directly instead of
the compressed representation to simplify an upcoming rework in prolog /
epilog caching. In doing so the state struct pointer was replaced with
an inline struct.

Care was also taken to pre-mask all the states with the active attribute
mask and other masks when it makes sense; this ensures that we don't
accidentally use information not hashed into the key during compilation.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26023>
 2023-11-13 11:47:42 +00:00
Tatsuyuki Ishi
d8a5b76307 aco: Replace aco_vs_input_state.divisors with bitfields. 
Instead of concrete divisor value, we only pass down the information
whether the divisor is zero or nontrivial (>1).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26023>
 2023-11-13 11:47:41 +00:00
Georg Lehmann
04956d54ce aco: force uniform result for LDS load with uniform address if it can be non uniform 
Because a LDS load is 2 separate loads on gfx10+ with wave64, a different wave
can write LDS in between and cause a non uniform result. Use v_readfirst_lane
instead of p_as_uniform because it cannot be copy propagated.

Fixes a OpenCL CTS test with zink+rusticl.

Totals from 136 (0.17% of 78196) affected shaders:
MaxWaves: 3236 -> 3244 (+0.25%)
Instrs: 130069 -> 131221 (+0.89%)
CodeSize: 698048 -> 703436 (+0.77%)
VGPRs: 5464 -> 5440 (-0.44%)
SpillSGPRs: 94 -> 96 (+2.13%)
Latency: 5361017 -> 5363781 (+0.05%); split: -0.00%, +0.05%
InvThroughput: 883010 -> 884100 (+0.12%)
SClause: 3822 -> 3821 (-0.03%); split: -0.05%, +0.03%
Copies: 14220 -> 14314 (+0.66%); split: -0.01%, +0.68%
Branches: 4549 -> 4551 (+0.04%)
PreSGPRs: 4934 -> 4940 (+0.12%)
PreVGPRs: 4666 -> 4655 (-0.24%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25973>
 2023-11-06 22:43:33 +00:00
Georg Lehmann
ab87831ae8 aco, radv: vectorize f2f16 if rounding mode is rtz 
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25952>
 2023-11-06 21:05:34 +00:00
Samuel Pitoiset
0477346c0b aco: remove dead code in nir_intrinsic_xfb_counter_{add,sub}_amd 
This code path is only used by GFX11 now.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25903>
 2023-10-30 08:51:51 +00:00
Qiang Yu
9c63138ae3 aco: stop emit s_endpgm for first stage of merged shader 
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25631>
 2023-10-26 02:01:49 +00:00
Qiang Yu
14022a3a0e aco: move end program handling to select_shader 
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25631>
 2023-10-26 02:01:49 +00:00
Qiang Yu
e2af0b0b3f aco: add create_end_for_merged_shader 
For radeonsi merged shader LS/ES part to pass args to next
stage.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25631>
 2023-10-26 02:01:49 +00:00
Qiang Yu
71fd3c2a35 aco: do not fix_exports when separately compiled ngg vs or es 
For radeonsi not abort when this case, as it does not jump at
last.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25631>
 2023-10-26 02:01:49 +00:00
Rhys Perry
4c3677094e aco,nir: add export_row_amd intrinsic 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
 2023-10-24 21:36:06 +00:00
Bas Nieuwenhuizen
5e7c828c0e aco: Add WMMA instructions. 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
 2023-10-24 13:24:18 +00:00
Samuel Pitoiset
f0abdaea9f amd/llvm,aco,radv: implement NGG streamout with GDS_STRMOUT registers on GFX11 
According to RadeonSI, this is required for preemption, user queues,
and we only have to wait for VS after streamout which should be more
performant.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25284>
 2023-10-12 16:58:22 +00:00
Qiang Yu
6eb0910d45 aco: do not fix_exports when program has epilog 
PS with epilog does not need to fix_exports. And radeonsi use
p_end_with_regs so does not have jump instruction at last.

radeonsi may also have exec restore instruction, so may break
before reach to p_end_with_regs.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24973>
 2023-10-10 02:36:34 +00:00
Qiang Yu
c9c18d3da5 aco: compact ps expilog color export for radeonsi 
radeonsi need to compact color export for ps epilog while radv does not.
radv will fill empty color slot, so won't affected by this change.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24973>
 2023-10-10 02:36:34 +00:00
Qiang Yu
1517aa7a8a aco,radv: add radeonsi spec ps epilog code 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24973>
 2023-10-10 02:36:34 +00:00
Qiang Yu
9972a385fb aco: simplify export_fs_mrt_color 
It's now used by ps epilog only.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24973>
 2023-10-10 02:36:34 +00:00
Qiang Yu
77d5966661 aco,radv: rename ps epilog info inputs to colors 
Will add other mrtz args for radeonsi.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24973>
 2023-10-10 02:36:34 +00:00
Qiang Yu
57b0f19582 aco: add create_fs_end_for_epilog for radeonsi 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24973>
 2023-10-10 02:36:33 +00:00