eecdb45d61
aco: consider s_cbranch_exec* instructions in needs_exec_mask()
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32477 >
2025-01-23 00:11:06 +00:00
5375d77488
aco: wait for scratch stores to complete before dealloc_vgprs
...
fossil-db (navi31):
Totals from 392 (0.49% of 79395) affected shaders:
Instrs: 5052043 -> 5054100 (+0.04%)
CodeSize: 26701200 -> 26709428 (+0.03%)
Latency: 43614861 -> 43615368 (+0.00%)
InvThroughput: 7353147 -> 7353216 (+0.00%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24884 >
2024-11-06 09:58:05 +00:00
295b7d606f
aco: insert NOP before dealloc_vgpr in the insert_NOPs pass
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24884 >
2024-11-06 09:58:05 +00:00
0ad713ca9f
aco: add waitcnt build helper
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24884 >
2024-11-06 09:58:04 +00:00
e2705a9d85
aco: set Precolored flag before register allocation
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31387 >
2024-10-22 12:29:18 +00:00
977f435f4c
aco/ir: add function to parse depctr waits
...
No Foz-DB changes on Navi31.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31132 >
2024-10-17 11:16:16 +00:00
27cf11dc8a
aco: remove per block inf/nan/sz control
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31172 >
2024-09-18 20:46:17 +00:00
9bb10b58f3
aco: use v_cvt_pk_u8_f32 for f2u8
...
Foz-DB Navi31:
Totals from 42 (0.05% of 79395) affected shaders:
Instrs: 3253747 -> 3248867 (-0.15%); split: -0.15%, +0.00%
CodeSize: 16690136 -> 16661772 (-0.17%); split: -0.17%, +0.00%
VGPRs: 4176 -> 4128 (-1.15%)
Latency: 18485157 -> 18479752 (-0.03%); split: -0.03%, +0.00%
InvThroughput: 3659404 -> 3658222 (-0.03%); split: -0.03%, +0.00%
Copies: 231891 -> 228145 (-1.62%)
VALU: 1785800 -> 1782054 (-0.21%)
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29532 >
2024-09-02 19:49:20 +00:00
7b92e11e16
aco: forget valu delays after certain s_waitcnt_depctr/LDSDIR
...
fossil-db (navi31):
Totals from 55242 (69.58% of 79395) affected shaders:
Instrs: 40507666 -> 40138006 (-0.91%); split: -0.91%, +0.00%
CodeSize: 212516104 -> 211025880 (-0.70%); split: -0.70%, +0.00%
Latency: 281643258 -> 281628053 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 46370668 -> 46369637 (-0.00%); split: -0.00%, +0.00%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Acked-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23337 >
2024-08-22 13:57:01 +00:00
b0c8c5e42e
aco: implement aco::validate_live_vars()
...
This is intended for passes which manually update live variables
and RegisterDemand, like e.g. the scheduler, and can be enabled
with ACO_DEBUG=validate-livevars.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30182 >
2024-08-14 08:11:48 +00:00
62fa5b9d6f
aco/gfx11+: apply neg to vinterp
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30350 >
2024-07-30 15:25:19 +00:00
08a4853ffd
aco: add export instructions to should_form_clause
...
fossil-db (gfx1150):
Totals from 1 (0.00% of 79395) affected shaders:
Instrs: 906 -> 901 (-0.55%)
CodeSize: 4692 -> 4672 (-0.43%)
Latency: 1582 -> 1561 (-1.33%)
fossil-db (navi21):
Totals from 9917 (12.49% of 79395) affected shaders:
Instrs: 6120530 -> 6116252 (-0.07%)
CodeSize: 31808916 -> 31791804 (-0.05%)
Latency: 23854892 -> 23866438 (+0.05%); split: -0.01%, +0.06%
InvThroughput: 4655278 -> 4655490 (+0.00%); split: -0.00%, +0.01%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30241 >
2024-07-23 13:14:52 +00:00
3b732340ec
aco/gfx11.5: skip dealloc_vgprs for stages with exports
...
fossil-db (gfx1150):
Totals from 72997 (91.94% of 79395) affected shaders:
Instrs: 35047397 -> 34679136 (-1.05%)
CodeSize: 182414292 -> 180934440 (-0.81%)
Latency: 225452700 -> 225045411 (-0.18%)
InvThroughput: 40416910 -> 40395608 (-0.05%); split: -0.06%, +0.00%
Branches: 604968 -> 601660 (-0.55%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30241 >
2024-07-23 13:14:52 +00:00
6affd916b5
aco/gfx11.5: fix s_fmac acc to definition
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29245 >
2024-07-18 08:36:14 +00:00
4399c7bac3
aco: add aco_opcode::p_s_cvt_f16_f32_rtne
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29245 >
2024-07-18 08:36:14 +00:00
d58d0274a8
aco/gfx12: use trans s_delay_alu for pseudo scalar
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29245 >
2024-07-18 08:36:14 +00:00
7fc8ad2ddd
aco/ir: remove unused vopc helpers
...
And rename get_swapped and get_inverse to show that they should only be used for VOPC.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29467 >
2024-06-27 08:12:30 +00:00
17f2ebe8d2
aco: use 1.5x vgprs for gfx1151 and gfx12
...
From LLVM.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29839 >
2024-06-24 18:39:40 +00:00
c3c398d56d
aco: make local functions static in files without anonymous namespace
...
I don't think adding an anonymous namespace in these files is worth it
given the amount of global functions
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29740 >
2024-06-18 17:53:07 +00:00
15f2c9c553
aco: Limit rt stages to 128 vgprs
...
Totals from 35472 (7.40% of 479373) affected shaders:
MaxWaves: 206239 -> 283776 (+37.60%)
Instrs: 193922210 -> 202721106 (+4.54%)
CodeSize: 1056819972 -> 1110833680 (+5.11%); split: -0.00%, +5.11%
VGPRs: 6026704 -> 4540416 (-24.66%)
SpillSGPRs: 23742 -> 25754 (+8.47%)
SpillVGPRs: 118897 -> 2295118 (+1830.34%)
Scratch: 7201792 -> 152752128 (+2021.03%)
Latency: 2713432565 -> 3194796286 (+17.74%); split: -0.20%, +17.94%
InvThroughput: 1052131232 -> 935049835 (-11.13%); split: -16.59%, +5.46%
VClause: 6972784 -> 8716721 (+25.01%); split: -0.02%, +25.03%
SClause: 4879313 -> 4852452 (-0.55%); split: -0.88%, +0.33%
Copies: 32782141 -> 35223995 (+7.45%)
Branches: 11075847 -> 11094087 (+0.16%); split: -0.00%, +0.17%
VALU: 118525960 -> 120929058 (+2.03%)
SALU: 33924572 -> 33973293 (+0.14%); split: -0.03%, +0.17%
VMEM: 12419116 -> 17104582 (+37.73%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29593 >
2024-06-10 19:39:52 +00:00
2a4424425a
aco/gfx12: fix s_wait_event immediate
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Acked-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29466 >
2024-06-06 14:26:52 +00:00
12b4bdc134
aco/gfx12: decrease max_nsa_vgprs for VSAMPLE
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29330 >
2024-05-28 10:52:11 +00:00
c8123b67e0
aco/gfx12: don't create v_fmac_legacy_f32
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29330 >
2024-05-28 10:52:11 +00:00
cc404d45ff
aco: remove perfwarn
...
This didn't do anything useful.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29270 >
2024-05-21 13:31:23 +00:00
f01cac835f
aco/stats: support GFX12 in collect_preasm_stats()
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29225 >
2024-05-20 10:45:39 +00:00
cadce0f3b7
aco: add GFX12 wait counters
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29225 >
2024-05-20 10:45:38 +00:00
e9a25151fa
aco/tests: support GFX12
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Acked-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29162 >
2024-05-14 20:50:27 +00:00
80b8bbf0c5
aco/gfx11: use v_swap_b16
...
I tested that v_swap_b16 can be encoded as VOP3, because the ISA doc doesn't list
it as a possible VOP3 opcode. VOP3 is nessecary to access v128+.
Foz-DB Navi31:
Totals from 32 (0.04% of 79395) affected shaders:
Instrs: 201865 -> 195168 (-3.32%)
CodeSize: 1082220 -> 1031228 (-4.71%); split: -4.71%, +0.00%
Latency: 2258198 -> 2238586 (-0.87%)
InvThroughput: 796731 -> 788934 (-0.98%)
Copies: 34514 -> 29220 (-15.34%)
VALU: 122457 -> 117163 (-4.32%)
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29143 >
2024-05-13 18:42:19 +00:00
75532d8687
aco: add wait_imm::unpack and wait_imm::max
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28981 >
2024-05-10 11:53:08 +00:00
20b4e30e25
aco: make wait_imm indexable
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28981 >
2024-05-10 11:53:08 +00:00
c2a467dd4b
aco: remove occupancy check in dealloc_vgprs()
...
This didn't consider that there might be different programs using the same
SIMD.
fossil-db (navi31):
Totals from 68129 (85.81% of 79395) affected shaders:
Instrs: 23230924 -> 23388315 (+0.68%)
CodeSize: 120636544 -> 121272888 (+0.53%)
Latency: 115645106 -> 115683965 (+0.03%)
InvThroughput: 18804076 -> 18806912 (+0.02%); split: -0.00%, +0.02%
Branches: 404644 -> 407945 (+0.82%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28633 >
2024-04-11 18:30:47 +00:00
e2cb9c57a2
aco: use v_interp_p2_f16 opsel
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28435 >
2024-04-10 07:49:27 +00:00
d15ca421c4
aco/gfx9: all non legacy opsel instructions only write 16bits
...
This affects 16bit VOP3 instructions that were new on gfx9, like max3.
No Foz-DB changes on vega10.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28435 >
2024-04-10 07:49:27 +00:00
7a69d78ba2
aco: use SPDX-License-Identifier
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28622 >
2024-04-08 15:49:25 +00:00
be93e38152
aco/vn: remove instruction hash templates
...
This reuses the instruction data size helper used for creating instruction.
It also removes the hashing of the aco::spans used for operands and definitions,
which was redundant with hash finalization.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28527 >
2024-04-05 10:23:16 +00:00
db29984c25
aco: move create_instruction() to aco_ir.cpp
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370 >
2024-03-28 11:25:43 +00:00
a863c7951e
aco: remove create_instruction() template parameter
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370 >
2024-03-28 11:25:43 +00:00
4cb73cb5f3
aco: defer instruction size from aco::Format in create_instruction()
...
rather than using the template argument.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370 >
2024-03-28 11:25:43 +00:00
1187189235
aco: unify different SALU types into single struct SALU_instruction
...
This removes
- SOP1_instruction
- SOP2_instruction
- SOPC_instruction
- SOPK_instruction
- SOPP_instruction
and their corresponding methods.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370 >
2024-03-28 11:25:43 +00:00
5d265257a0
aco: remove SOPP_instruction::block member
...
Re-use SOPP_instruction::imm instead.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370 >
2024-03-28 11:25:43 +00:00
e927c5004f
aco/gfx11+: disable v_pk_fmac_f16_dpp
...
Public docs are apparently wrong: https://github.com/llvm/llvm-project/pull/79598#issuecomment-1933988048
Cc: mesa-stable
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27533 >
2024-02-09 21:36:52 +00:00
48461c0d9e
aco: enable VOPD scheduler
...
fossil-db (navi31, wave32):
Totals from 75237 (94.95% of 79242) affected shaders:
Instrs: 40383337 -> 36522907 (-9.56%); split: -9.56%, +0.00%
CodeSize: 209282312 -> 209883000 (+0.29%); split: -0.18%, +0.46%
Latency: 256898944 -> 257834204 (+0.36%); split: -0.23%, +0.60%
InvThroughput: 59105835 -> 52725948 (-10.79%); split: -10.90%, +0.11%
VClause: 759487 -> 758151 (-0.18%); split: -0.18%, +0.01%
SClause: 1264895 -> 1263230 (-0.13%); split: -0.14%, +0.01%
VALU: 22886416 -> 18837085 (-17.69%)
VOPD: 0 -> 4049331 (+inf%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23367 >
2024-02-05 10:51:15 +00:00
023e78b4d7
aco: add new post-RA scheduler for ILP
...
Totals from 77247 (97.37% of 79330) affected shaders: (GFX11)
Instrs: 44371374 -> 43215723 (-2.60%); split: -2.64%, +0.03%
CodeSize: 227819532 -> 223188224 (-2.03%); split: -2.06%, +0.03%
Latency: 301016823 -> 290147626 (-3.61%); split: -3.70%, +0.09%
InvThroughput: 48551749 -> 47646212 (-1.87%); split: -1.88%, +0.01%
VClause: 870581 -> 834655 (-4.13%); split: -4.13%, +0.00%
SClause: 1487061https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25676 >
2024-01-06 11:30:42 +00:00
72a5c659d4
aco: form clauses for LDS instructions
...
No fossil-db changes.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25676 >
2024-01-06 11:30:42 +00:00
8f16745821
aco: fix should_form_clause() for memory instructions without operands
...
In particular, this applies to s_memtime and s_memrealtime.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25676 >
2024-01-06 11:30:41 +00:00
dd7b6898e6
radv: fix number of physical SGPRs on GFX10+
...
This change has no effect.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26521 >
2023-12-11 10:39:51 +00:00
5ebba87772
aco: rename max_wave64_per_simd -> max_waves_per_simd
...
and update usage. Changes are because the scheduler targets
a different number of waves.
Totals from 195 (0.25% of 79330) affected shaders: (GFX11)
MaxWaves: 3120 -> 3108 (-0.38%)
Instrs: 71202 -> 71070 (-0.19%); split: -0.27%, +0.09%
CodeSize: 383272 -> 382828 (-0.12%); split: -0.21%, +0.10%
VGPRs: 7392 -> 7752 (+4.87%)
Latency: 2280141 -> 2262487 (-0.77%); split: -0.79%, +0.02%
InvThroughput: 4759022 -> 5725442 (+20.31%); split: -0.01%, +20.32%
VClause: 1737 -> 1741 (+0.23%); split: -3.11%, +3.34%
SClause: 2385 -> 2376 (-0.38%); split: -0.80%, +0.42%
Copies: 5257 -> 5274 (+0.32%); split: -0.25%, +0.57%
Branches: 1213 -> 1212 (-0.08%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26521 >
2023-12-11 10:39:50 +00:00
0e79f76aa5
aco: add fetch_inactive field to DPP instructions
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25525 >
2023-10-04 18:53:43 +00:00
26fce534b5
aco: shrink DPP8_instruction
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25525 >
2023-10-04 18:53:43 +00:00
73605d46dd
aco: assume newer generation will use GFX11 wait_imm packing
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25522 >
2023-10-04 08:35:48 +00:00
Daniel Schürmann