AlexIndustrial/mesa
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
197,421 Commits 1 Branch 0 Tags
d1d41be43fa7ed135b8e3bb2e0174964ced11d83
Commit Graph

1236 Commits

Author SHA1 Message Date
Samuel Pitoiset d1d41be43f aco: declare phys regs for tba_hi/tma_hi 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32026>
 2024-11-08 14:00:15 +00:00
Samuel Pitoiset 13bab450a2 aco: fix storing SQ_WAVE_STATUS in the trap handler shader 
SQ_WAVE_STATUS can change inside the trap because of SCC.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32026>
 2024-11-08 14:00:14 +00:00
Samuel Pitoiset 494050d2ea aco: add a helper to dump SGPR to memory for the trap handler 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32026>
 2024-11-08 14:00:14 +00:00
Samuel Pitoiset 8c6f2fef1b aco: use scalar buffer stores for dumping SGPRS from the trap on GFX8 
This avoids using any VGPRs on GFX8.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32026>
 2024-11-08 14:00:14 +00:00
Samuel Pitoiset 17f6b4e51e aco: save/restore SCC in the trap handler shader 
SCC is only updated on GFX9+ but let's do it by default because the
trap handler shader is likely going to be more and more complex over
time.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32026>
 2024-11-08 14:00:14 +00:00
Samuel Pitoiset 7b4386facd aco: cleanup using fixed registers in the trap handler shader 
It's easier to read and potentially less error prone.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32026>
 2024-11-08 14:00:14 +00:00
Rhys Perry b318fe47e9 aco: don't byte align global VMEM loads if it might be unsafe 
Using the byte align path can be unsafe even when 12 byte loads are
supported.

fossil-db (navi21):
Totals from 185 (0.23% of 79395) affected shaders:
Instrs: 391501 -> 391575 (+0.02%); split: -0.03%, +0.05%
CodeSize: 2147336 -> 2147672 (+0.02%); split: -0.03%, +0.05%
Latency: 3762613 -> 3860941 (+2.61%); split: -0.01%, +2.62%
InvThroughput: 871429 -> 888013 (+1.90%); split: -0.08%, +1.98%
VClause: 9712 -> 10210 (+5.13%)
Copies: 53775 -> 53010 (-1.42%); split: -1.46%, +0.04%
VALU: 254009 -> 252146 (-0.73%)
SALU: 56698 -> 56699 (+0.00%); split: -0.00%, +0.00%
VMEM: 18503 -> 19601 (+5.93%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Fixes: 391bf3ea30 ("aco: don't expand smem/mubuf global loads")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31807>
 2024-11-06 19:07:16 +00:00
Georg Lehmann 2cd8a9fef7 amd: lower gl_FragCoord.w rcp in NIR 
This allows NIR to remove the rcps if the application uses rcp(gl_FragCoord.w).
D3D provides w, not 1/w like GL/VK in the shader, so this is commonly used.

Foz-DB Navi21:
Totals from 2068 (2.61% of 79206) affected shaders:
MaxWaves: 45636 -> 45652 (+0.04%)
Instrs: 2173444 -> 2169671 (-0.17%); split: -0.18%, +0.00%
CodeSize: 11881304 -> 11867208 (-0.12%); split: -0.12%, +0.01%
VGPRs: 118000 -> 117968 (-0.03%)
Latency: 35689676 -> 35675909 (-0.04%); split: -0.06%, +0.02%
InvThroughput: 9167199 -> 9159801 (-0.08%); split: -0.08%, +0.00%
VClause: 45076 -> 45078 (+0.00%); split: -0.01%, +0.02%
SClause: 92503 -> 92366 (-0.15%); split: -0.31%, +0.17%
Copies: 140282 -> 140303 (+0.01%); split: -0.13%, +0.14%
Branches: 53347 -> 53346 (-0.00%); split: -0.01%, +0.00%
PreVGPRs: 96495 -> 96465 (-0.03%)
VALU: 1522980 -> 1519252 (-0.24%); split: -0.25%, +0.01%
SALU: 213451 -> 213460 (+0.00%); split: -0.02%, +0.02%

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31967>
 2024-11-06 12:57:08 +00:00
Samuel Pitoiset 32a537b25b aco: use inlined constant offsets for storing SGPRs in the trap handler 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31976>
 2024-11-05 11:55:24 +00:00
Samuel Pitoiset 9bcf17ef5a aco: add support for the trap handler shader on GFX11 
This has been verified on navi31.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31960>
 2024-11-05 07:58:38 +00:00
Samuel Pitoiset 6d5a2ae928 aco: clear the current wave exception in the trap handler 
This is required to re-enable VALU instructions in this wave, only
float exception seem to be affected.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31960>
 2024-11-05 07:58:38 +00:00
Samuel Pitoiset 81f4670ed6 radv,aco: dump all SGPRS from the trap handler 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31960>
 2024-11-05 07:58:38 +00:00
Georg Lehmann a58d2b59e9 aco: implement load_pixel_coord 
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31864>
 2024-11-04 12:34:30 +00:00
Samuel Pitoiset 1fa0fe1e0c aco: add support for the trap handler shader on GFX9-GFX10.3 
This has been tested on navi21.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31926>
 2024-11-04 10:48:52 +00:00
Samuel Pitoiset 281eb14df8 aco: fix reading registers from the trap handler shader 
It should read 32-bit values, otherwise some MSB are 0 and it's missing
some information.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31926>
 2024-11-04 10:48:52 +00:00
Samuel Pitoiset 49682fc0cb radv,aco: save SQ_WAVE_GPR_ALLOC from the trap handler 
This would be used to dump SGPRs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31934>
 2024-11-01 15:40:25 +00:00
Daniel Schürmann 10958d04d5 aco: Respect addressible SGPR limit in VS prologs 
On Tonga, the effective SGPR limit is 96, including VCC.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31859>
 2024-10-28 11:29:06 +00:00
Daniel Schürmann c8348139fd nir: change signature of nir_src_is_divergent() 
Now, it takes nir_src * instead of nir_src.
Also move the implementation to nir_divergence_analysis.c.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30787>
 2024-10-24 10:06:17 +00:00
Daniel Schürmann e2705a9d85 aco: set Precolored flag before register allocation 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31387>
 2024-10-22 12:29:18 +00:00
Georg Lehmann 10951bb11a aco: fix 64bit extract_i8/extract_i16 
The old code only sign extended to 32bit, with a zero hi half.

Fixes: 1f2518ef9f ("aco: implement nir_op_extract/nir_op_insert")
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31734>
 2024-10-21 07:13:57 +00:00
Rhys Perry 33eb2d7fe4 aco: skip uniformization of certain merge phis 
If a source is a VGPR, then skip if it's safe. This fixes the regressions
from the previous commit.

fossil-db (navi31):
Totals from 5118 (6.45% of 79395) affected shaders:
MaxWaves: 159560 -> 159520 (-0.03%); split: +0.01%, -0.03%
Instrs: 2165351 -> 2138456 (-1.24%); split: -1.26%, +0.02%
CodeSize: 11260340 -> 11152460 (-0.96%); split: -0.98%, +0.02%
VGPRs: 218124 -> 225144 (+3.22%); split: -0.13%, +3.35%
Latency: 11059208 -> 11116102 (+0.51%); split: -0.18%, +0.69%
InvThroughput: 1252148 -> 1230193 (-1.75%); split: -1.77%, +0.01%
VClause: 39513 -> 39518 (+0.01%); split: -0.48%, +0.49%
SClause: 59434 -> 59378 (-0.09%); split: -0.11%, +0.02%
Copies: 165997 -> 156172 (-5.92%); split: -6.68%, +0.76%
PreSGPRs: 181203 -> 181094 (-0.06%)
PreVGPRs: 139393 -> 139731 (+0.24%)
VALU: 1244301 -> 1220769 (-1.89%); split: -1.91%, +0.02%
SALU: 200240 -> 199567 (-0.34%); split: -0.34%, +0.00%

fossil-db (navi21):
Totals from 35520 (44.74% of 79395) affected shaders:
MaxWaves: 951870 -> 951830 (-0.00%)
Instrs: 20229388 -> 20227776 (-0.01%); split: -0.01%, +0.00%
CodeSize: 105379916 -> 105513740 (+0.13%); split: -0.01%, +0.13%
VGPRs: 1375232 -> 1375400 (+0.01%)
Latency: 81046435 -> 81013986 (-0.04%); split: -0.04%, +0.00%
InvThroughput: 15269166 -> 15273295 (+0.03%); split: -0.01%, +0.04%
VClause: 354314 -> 354310 (-0.00%); split: -0.00%, +0.00%
SClause: 417049 -> 417047 (-0.00%); split: -0.00%, +0.00%
Copies: 1699445 -> 1699488 (+0.00%); split: -0.01%, +0.01%
Branches: 591274 -> 591269 (-0.00%); split: -0.00%, +0.00%
PreSGPRs: 1371062 -> 1370567 (-0.04%)
PreVGPRs: 1100716 -> 1100953 (+0.02%)
VALU: 11076189 -> 11075167 (-0.01%); split: -0.01%, +0.00%
SALU: 3648002 -> 3647378 (-0.02%); split: -0.02%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30211>
 2024-10-10 14:59:27 +00:00
Daniel Schürmann 30e7644e5f aco: simplify Definition constructors 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31362>
 2024-10-07 07:00:19 +00:00
Georg Lehmann 07032102e9 aco: use s_pack_lh for bitfield_select(0xffff) 
Foz-DB Navi31
Totals from 13 (0.02% of 79206) affected shaders:
Instrs: 44871 -> 44838 (-0.07%)
CodeSize: 223804 -> 223608 (-0.09%)
Latency: 220186 -> 220191 (+0.00%); split: -0.01%, +0.02%
InvThroughput: 54169 -> 54186 (+0.03%); split: -0.00%, +0.03%
SALU: 5048 -> 5023 (-0.50%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31509>
 2024-10-05 17:55:08 +00:00
Georg Lehmann a6f82cf16d aco: use s_pack_hl for shfr16 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31509>
 2024-10-05 17:55:08 +00:00
Georg Lehmann 78b8ec9c93 aco: optimize lanecount_to_mask 
s_bfe uses 7 bits for the size, so when we extract from -1,
we can get all possible lane masks in one instruction.

Foz-DB Navi31:
Totals from 38601 (48.62% of 79395) affected shaders:
Instrs: 13670163 -> 13509738 (-1.17%)
CodeSize: 68011644 -> 67368308 (-0.95%)
Latency: 61203404 -> 61065419 (-0.23%); split: -0.23%, +0.00%
InvThroughput: 6897028 -> 6894634 (-0.03%); split: -0.05%, +0.01%
SALU: 1491291 -> 1342553 (-9.97%)

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31184>
 2024-09-26 14:29:14 +00:00
Georg Lehmann bcfc5c09fa amd: add offset to is_subgroup_invocation_lt_amd 
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31184>
 2024-09-26 14:29:13 +00:00
Georg Lehmann 0e21cd9e15 aco/gfx10+: work around non uniform ds_append wave64 result 
In wave64 for hw with native wave32, ds_append seems to be split in a load for
the low half and an atomic for the high half, and other LDS instructions can
be scheduled between the two.
Which means the result of the low half is unusable because it might be out of date.

I was only able to reproduce this issue in WGP mode, but be conservative and
apply the workaround in CU mode too.

Foz-DB Navi31:
Totals from 13 (0.02% of 79395) affected shaders:
Instrs: 7599 -> 7656 (+0.75%)
CodeSize: 39708 -> 39972 (+0.66%)
Latency: 83174 -> 83572 (+0.48%)
InvThroughput: 8271 -> 8357 (+1.04%)
Copies: 718 -> 717 (-0.14%)
VALU: 3689 -> 3703 (+0.38%)
SALU: 935 -> 965 (+3.21%)

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/11921
Fixes: 45e935800a ("aco: implement nir_shared_append/consume_amd")

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31301>
 2024-09-23 13:17:58 +00:00
Georg Lehmann 96b9f695d4 aco/isel: use upper bound for v_mul_u32_u24 
The optimizer can use this.

Foz-DB Navi31:
Totals from 577 (0.73% of 79395) affected shaders:
Instrs: 4209237 -> 4206859 (-0.06%); split: -0.06%, +0.00%
CodeSize: 21511192 -> 21511984 (+0.00%); split: -0.02%, +0.02%
SpillSGPRs: 679 -> 671 (-1.18%)
Latency: 28448559 -> 28443863 (-0.02%); split: -0.04%, +0.03%
InvThroughput: 5221932 -> 5218443 (-0.07%); split: -0.09%, +0.02%
Copies: 297965 -> 298076 (+0.04%); split: -0.01%, +0.05%
VALU: 2385304 -> 2383500 (-0.08%)
SALU: 485553 -> 485533 (-0.00%); split: -0.01%, +0.00%

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31245>
 2024-09-19 17:08:47 +00:00
Georg Lehmann 45e935800a aco: implement nir_shared_append/consume_amd 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31075>
 2024-09-19 16:21:48 +00:00
Georg Lehmann 27cf11dc8a aco: remove per block inf/nan/sz control 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31172>
 2024-09-18 20:46:17 +00:00
Georg Lehmann 9850f759dd aco/isel: set per instruction float control modes 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31172>
 2024-09-18 20:46:17 +00:00
Georg Lehmann fc4b23130c aco/isel: add function to create builder for alu 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31172>
 2024-09-18 20:46:17 +00:00
Samuel Pitoiset 15b1790a1e radv,aco: fix legacy vertex attributes when offset >= stride on GFX6-7 
The indexing needs to be adjusted and the best solution seems to
use soffset instead of const_offset, it's simpler and generate less
prologs than passing the vertex binding strides to the prolog.

Fixes dEQP-VK.pipeline.*.vertex_input.legacy_vertex_attributes.*stride_1*.

Fixes: 38cbc3c605 ("radv: advertise VK_EXT_legacy_vertex_attributes")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31209>
 2024-09-18 07:21:28 +00:00
Samuel Pitoiset 26d8f1a306 aco,radv,radeonsi: move has_epilog to the fragment shader info 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31150>
 2024-09-16 07:53:00 +00:00
Georg Lehmann 9bb10b58f3 aco: use v_cvt_pk_u8_f32 for f2u8 
Foz-DB Navi31:
Totals from 42 (0.05% of 79395) affected shaders:
Instrs: 3253747 -> 3248867 (-0.15%); split: -0.15%, +0.00%
CodeSize: 16690136 -> 16661772 (-0.17%); split: -0.17%, +0.00%
VGPRs: 4176 -> 4128 (-1.15%)
Latency: 18485157 -> 18479752 (-0.03%); split: -0.03%, +0.00%
InvThroughput: 3659404 -> 3658222 (-0.03%); split: -0.03%, +0.00%
Copies: 231891 -> 228145 (-1.62%)
VALU: 1785800 -> 1782054 (-0.21%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29532>
 2024-09-02 19:49:20 +00:00
Samuel Pitoiset aad503ecfa aco: fix bogus assert in RT prolog on GFX11+ 
in_scratch_offset isn't defined on GFX11+ and only useful on < GFX9.

Fixes: bd525f4282 ("aco: Fix 1D->2D dispatch conversion on <gfx9")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30717>
 2024-08-19 21:07:31 +00:00
Friedrich Vock bd525f4282 aco: Fix 1D->2D dispatch conversion on <gfx9 
out_args->scratch_offset and in_wg_id_x will alias on <gfx9.
To avoid the conversion code reading a garbage WG ID, move the
scratch/ring offset writing to the very end.

Fixes: 1e354172 ("radv,aco: Convert 1D ray launches to 2D")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30707>
 2024-08-17 17:01:10 +00:00
Rhys Perry b934255510 aco: split selection_control_remove into rarely_taken and never_taken 
No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Backport-to: 24.1
Backport-to: 24.2
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30321>
 2024-08-15 16:00:18 +00:00
Georg Lehmann dedfff9dbf aco: only set latekill in live_var_analysis 
Cleaner to have this all in one place, in my opinion.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30368>
 2024-08-12 10:31:09 +00:00
Georg Lehmann df16f47036 aco: optimize dd[xy]_fine if it's only used by abs 
If we can ignore the sign of the derivative, we can swap the lanes
instead of broadcasting per direction. abs(a - b) = abs(b - a).
Shamelessly copied from bifrost.

Foz-DB Navi31:
Totals from 5 (0.01% of 79206) affected shaders:
Instrs: 6191 -> 6184 (-0.11%)
CodeSize: 31960 -> 31920 (-0.13%)
Latency: 111961 -> 111926 (-0.03%)
InvThroughput: 10390 -> 10372 (-0.17%)
VALU: 3286 -> 3279 (-0.21%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30557>
 2024-08-08 17:39:55 +00:00
Alyssa Rosenzweig daa97bb41a amd: switch to derivative intrinsics 
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30565>
 2024-08-08 15:26:07 +00:00
Georg Lehmann f36fccabf5 aco: optimize 64bit find_lsb/find_msb 
No Foz-DB changes, but this should be better, especially for gfx6-7 where
uadd_sat is 2 instructions.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30549>
 2024-08-08 11:15:55 +00:00
Georg Lehmann bee487df48 aco/gfx11.5+: use vinterp for fddx/fddy 
Since GFX11.5 VINTERP can be dual issued, DPP cannot.

Foz-DB GFX11.5:
Totals from 8401 (10.58% of 79395) affected shaders:
MaxWaves: 247880 -> 247848 (-0.01%)
Instrs: 6802675 -> 6815061 (+0.18%); split: -0.08%, +0.26%
CodeSize: 36539444 -> 36500948 (-0.11%); split: -0.22%, +0.11%
VGPRs: 444324 -> 445932 (+0.36%); split: -0.01%, +0.37%
SpillSGPRs: 1350 -> 1346 (-0.30%)
Latency: 63628380 -> 63523687 (-0.16%); split: -0.20%, +0.04%
InvThroughput: 10566750 -> 10486009 (-0.76%); split: -0.77%, +0.01%
VClause: 100171 -> 100248 (+0.08%); split: -0.08%, +0.16%
SClause: 175467 -> 176208 (+0.42%); split: -0.05%, +0.47%
Copies: 356817 -> 356935 (+0.03%); split: -0.17%, +0.20%
PreVGPRs: 283403 -> 283898 (+0.17%); split: -0.02%, +0.20%
VALU: 4217969 -> 4229831 (+0.28%); split: -0.03%, +0.31%
SALU: 479367 -> 479428 (+0.01%); split: -0.00%, +0.01%

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30350>
 2024-07-30 15:25:19 +00:00
Georg Lehmann 8c6e299141 aco: reorder dpp for ddx/ddy 
Having the mov last allows us to fuse it with the use instruction.

Foz-DB Navi31:
Totals from 9400 (11.84% of 79395) affected shaders:
MaxWaves: 273998 -> 274030 (+0.01%)
Instrs: 8303778 -> 8282997 (-0.25%); split: -0.29%, +0.04%
CodeSize: 44428088 -> 44464860 (+0.08%); split: -0.09%, +0.18%
VGPRs: 506616 -> 504492 (-0.42%)
SpillSGPRs: 1389 -> 1393 (+0.29%)
Latency: 76923466 -> 76983332 (+0.08%); split: -0.06%, +0.14%
InvThroughput: 12386888 -> 12391262 (+0.04%); split: -0.04%, +0.07%
VClause: 125136 -> 125059 (-0.06%); split: -0.13%, +0.07%
SClause: 227361 -> 226615 (-0.33%); split: -0.43%, +0.10%
Copies: 440787 -> 440749 (-0.01%); split: -0.17%, +0.16%
PreVGPRs: 339783 -> 333343 (-1.90%); split: -1.92%, +0.02%
VALU: 5088362 -> 5069737 (-0.37%); split: -0.37%, +0.01%
SALU: 606596 -> 606609 (+0.00%); split: -0.01%, +0.01%

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30350>
 2024-07-30 15:25:19 +00:00
Rhys Perry a8a15dc5b5 aco: add struct and helpers for exec potentially empty 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30320>
 2024-07-26 11:08:27 +00:00
Rhys Perry 39264a90c3 aco: consider exec empty after divergent continue then divergent break 
For:
loop {
   if (divergent)
      continue
   if (divergent)
      break
   //exec is potentially empty here
   loop {
      if (divergent)
         break
   }
}
If a subset of invocations take the continue and then the rest take the
break, then exec will be empty after the break.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30320>
 2024-07-26 11:08:27 +00:00
Marek Olšák b2d32ae246 nir: add nir_intrinsic_load_per_primitive_input, split from io_semantics flag 
Instead of having 1 bit in nir_io_semantics indicating a per-primitive
FS input, add a dedicated intrinsic for it.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29895>
 2024-07-23 16:13:16 +00:00
Rhys Perry cccfbe6141 aco: move s_setprio to before NGG exec initialization 
fossil-db (gfx1150):
Totals from 32 (0.04% of 79395) affected shaders:
Instrs: 17397 -> 17365 (-0.18%)
CodeSize: 83700 -> 83580 (-0.14%)
Latency: 59006 -> 58974 (-0.05%)

fossil-db (navi21):
Totals from 4 (0.01% of 79395) affected shaders:

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30241>
 2024-07-23 13:14:52 +00:00
Rhys Perry 0919ce1ac4 aco/gfx11.5: workaround export priority issue 
https://github.com/llvm/llvm-project/pull/99273

fossil-db (gfx1150):
Totals from 73996 (93.20% of 79395) affected shaders:
Instrs: 36015357 -> 36807177 (+2.20%)
CodeSize: 189072544 -> 192238748 (+1.67%)
Latency: 245845181 -> 246790550 (+0.38%); split: -0.00%, +0.38%
InvThroughput: 45068018 -> 45116177 (+0.11%); split: -0.00%, +0.11%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Backport-to: 24.2
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30241>
 2024-07-23 13:14:51 +00:00
Georg Lehmann efb9258814 aco: handle clustered uniform reductions correctly 
Alternatively we could just trust divergence analysis to do the right thing.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30235>
 2024-07-19 07:24:34 +00:00