d6cb45dbb0
aco/spill: Allow spilling live-through operands
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29730 >
2025-03-26 19:18:30 +00:00
ca7ce1fb33
aco/spill: Invert reloads map
...
So we can quickly look up if an operand was reloaded without having to
check renames.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29730 >
2025-03-26 19:18:30 +00:00
52253da783
aco: unify get_addr_sgpr_from_waves() and get_addr_vgpr_from_waves() into one function
...
which returns the limit as RegisterDemand.
Also remove the unused get_extra_sgprs() from aco_ir.h.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33644 >
2025-02-21 13:49:41 +00:00
e8472d484f
aco/spill: use float division for score() calculation rather than integers
...
This was the original intention and should result in more fine-grained
and thus better decisions.
Totals from 63 (0.08% of 79395) affected shaders: (Navi31)
Instrs: 3173500 -> 3174012 (+0.02%); split: -0.01%, +0.02%
CodeSize: 16345348 -> 16349288 (+0.02%); split: -0.01%, +0.03%
Latency: 18528036 -> 18526082 (-0.01%); split: -0.02%, +0.01%
InvThroughput: 3619125 -> 3618709 (-0.01%); split: -0.02%, +0.01%
VClause: 82654 -> 82648 (-0.01%)
SClause: 61256 -> 61257 (+0.00%); split: -0.00%, +0.01%
Copies: 250037 -> 250158 (+0.05%); split: -0.06%, +0.11%
Branches: 101302 -> 101303 (+0.00%)
VALU: 1791447 -> 1791435 (-0.00%); split: -0.00%, +0.00%
SALU: 401898 -> 402007 (+0.03%); split: -0.03%, +0.06%
VOPD: 730 -> 741 (+1.51%)
Totals from 40 (0.06% of 63053) affected shaders: (Vega10)
Instrs: 161584 -> 161567 (-0.01%); split: -0.04%, +0.03%
CodeSize: 891168 -> 891004 (-0.02%); split: -0.04%, +0.03%
Latency: 3550766 -> 3549770 (-0.03%); split: -0.05%, +0.03%
InvThroughput: 2627028 -> 2626484 (-0.02%); split: -0.03%, +0.01%
VClause: 2970 -> 2971 (+0.03%)
SClause: 4203 -> 4205 (+0.05%); split: -0.26%, +0.31%
Copies: 19923 -> 19893 (-0.15%); split: -0.44%, +0.29%
VALU: 116045 -> 116054 (+0.01%); split: -0.01%, +0.02%
SALU: 22100 -> 22066 (-0.15%); split: -0.39%, +0.24%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31769 >
2024-10-23 14:35:29 +00:00
30d85b23ef
aco/spill: fix faulty assertions
...
By unintentionally using integer division for score(), these
assertions were likely to be raised by accident.
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31769 >
2024-10-23 14:35:29 +00:00
541cfb21ba
aco: don't attempt to spill dead phis
...
These don't affect register pressure.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30182 >
2024-08-14 08:11:48 +00:00
dedfff9dbf
aco: only set latekill in live_var_analysis
...
Cleaner to have this all in one place, in my opinion.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30368 >
2024-08-12 10:31:09 +00:00
6723128e94
aco/spill: Don't add phi definitions to live-in variables
...
Changes are because we don't add artificial uses to the phi definitions anymore.
Totals from 13 (0.02% of 79395) affected shaders: (GFX10.3)
Instrs: 230510 -> 230285 (-0.10%); split: -0.10%, +0.00%
CodeSize: 1269916 -> 1268760 (-0.09%); split: -0.10%, +0.01%
SpillSGPRs: 2057 -> 2058 (+0.05%)
Latency: 2729731 -> 2723103 (-0.24%)
InvThroughput: 696888 -> 695286 (-0.23%)
VClause: 5795 -> 5768 (-0.47%)
SClause: 6855 -> 6858 (+0.04%)
Copies: 32336 -> 32275 (-0.19%); split: -0.22%, +0.03%
VALU: 151782 -> 151731 (-0.03%); split: -0.04%, +0.01%
SALU: 30766 -> 30758 (-0.03%); split: -0.03%, +0.01%
VMEM: 12157 -> 12078 (-0.65%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120 >
2024-07-16 14:00:49 +00:00
c146d4b6b6
aco/spill: use live-in variables directly rather than computing them
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120 >
2024-07-16 14:00:49 +00:00
6c6f382d68
aco: add RegisterDemand member to Instruction
...
Since we never need both at the same time, we can use
a union with pass_flags.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29962 >
2024-07-10 12:31:02 +00:00
c4a38c6583
aco/spill: don't remove spilled phis
...
They will be removed during register allocation.
Few changes due to different phi order.
Totals from 14 (0.02% of 79395) affected shaders: (GFX11)
Instrs: 315724 -> 315675 (-0.02%); split: -0.02%, +0.01%
CodeSize: 1673608 -> 1673268 (-0.02%); split: -0.03%, +0.00%
Latency: 3194243 -> 3189025 (-0.16%); split: -0.19%, +0.03%
InvThroughput: 638369 -> 637323 (-0.16%); split: -0.19%, +0.03%
VClause: 5716 -> 5714 (-0.03%)
Copies: 37786 -> 37748 (-0.10%); split: -0.13%, +0.03%
Branches: 10469 -> 10454 (-0.14%); split: -0.16%, +0.02%
VALU: 182498 -> 182454 (-0.02%); split: -0.03%, +0.00%
SALU: 36038 -> 36046 (+0.02%); split: -0.01%, +0.04%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804 >
2024-06-26 09:42:34 +00:00
5cfa5b784b
aco: remove get_demand_before()
...
The register demand before executing an instruction is now included
in the instruction's register demand and this function is unused.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804 >
2024-06-26 09:42:34 +00:00
09f1c40f2e
aco: track and use the live-in register demand per basic block
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804 >
2024-06-26 09:42:34 +00:00
001c8caae0
aco: calculate register demand per instruction as maximum necessary to execute the instruction
...
Previously, the register demand per instruction was calculated as the number of
live variables in the register file after executing an instruction plus additional
temporary registers, necessary during the execution of the instruction.
With this change, now it also includes all variables which are live right before
executing an instruction, i.e. killed Operands.
Care has been taken so that the invariant
register_demand[idx] = register_demand[idx - 1] - get_temp_registers(prev_instr)
+ get_live_changes(instr) + get_temp_registers(instr)
still holds.
Slight changes in scheduling:
Totals from 316 (0.40% of 79395) affected shaders: (GFX11)
Instrs: 301329 -> 300777 (-0.18%); split: -0.31%, +0.12%
CodeSize: 1577976 -> 1576204 (-0.11%); split: -0.21%, +0.10%
SpillSGPRs: 448 -> 447 (-0.22%)
Latency: 1736349 -> 1726182 (-0.59%); split: -2.01%, +1.42%
InvThroughput: 243894 -> 243883 (-0.00%); split: -0.03%, +0.03%
VClause: 6134 -> 6280 (+2.38%); split: -1.04%, +3.42%
SClause: 6142 -> 6137 (-0.08%); split: -0.13%, +0.05%
Copies: 14037 -> 14032 (-0.04%); split: -0.56%, +0.52%
Branches: 3284 -> 3283 (-0.03%)
VALU: 182750 -> 182718 (-0.02%); split: -0.04%, +0.03%
SALU: 18522 -> 18538 (+0.09%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804 >
2024-06-26 09:42:33 +00:00
4c2f231cc0
aco/spill: Unconditionally add 2 SGPRs to live-in demand
...
Due to undefined Operands, it might not be enough to check the
predecessors' register demand.
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804 >
2024-06-26 09:42:33 +00:00
a497d105e3
aco: move live var information into struct Program
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29713 >
2024-06-14 14:32:35 +00:00
ec8512ce85
aco/spill: Don't spill phis with all-undef operands
...
Fixes some crashes when limiting RT stages to 128 VGPRs.
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29593 >
2024-06-10 19:39:52 +00:00
b41f0f6cc1
aco: use ac_hw_cache_flags
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29243 >
2024-06-07 13:22:42 +00:00
3d6957268b
aco: use new common helpers for building buffer descriptors
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29268 >
2024-05-22 08:31:39 +00:00
590ea76104
aco/spill: Insert p_start_linear_vgpr right after p_logical_end
...
If p_start_linear_vgpr allocates a VGPR that is already blocked, RA
will try moving the blocking VGPR somewhere else. If
p_start_linear_vgpr is inserted right before the branch, that move will
be inserted after exec has been overwritten, which might cause the move
to be skipped for some threads.
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28041 >
2024-05-14 13:54:34 +00:00
58a5de5c34
amd: add gfx12 register definitions into the register header generator
...
The generator renamed some definitions to resolve conflicts.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Acked-by: Timur Kristóf <timur.kristof@gmail.com >
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29007 >
2024-05-11 22:14:05 -04:00
7a69d78ba2
aco: use SPDX-License-Identifier
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28622 >
2024-04-08 15:49:25 +00:00
a4231d4f56
aco/spill: use average use distances to spill loop variables
...
Totals from 83 (0.10% of 79395) affected shaders: (GFX11)
Instrs: 3429510 -> 3427917 (-0.05%); split: -0.05%, +0.01%
CodeSize: 17587884 -> 17570224 (-0.10%); split: -0.11%, +0.01%
SpillSGPRs: 4660 -> 4630 (-0.64%); split: -1.07%, +0.43%
Latency: 20054145 -> 20054454 (+0.00%); split: -0.00%, +0.01%
InvThroughput: 4989606 -> 4989740 (+0.00%); split: -0.00%, +0.01%
VClause: 90844 -> 90843 (-0.00%)
SClause: 69534 -> 69535 (+0.00%); split: -0.04%, +0.04%
Copies: 283288 -> 283415 (+0.04%); split: -0.11%, +0.15%
Branches: 113543 -> 113409 (-0.12%); split: -0.12%, +0.01%
VALU: 1888933 -> 1887489 (-0.08%); split: -0.08%, +0.00%
SALU: 423548 -> 423609 (+0.01%); split: -0.07%, +0.09%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799 >
2024-03-28 13:57:36 +00:00
c371882060
aco/spill: use average use distances in init_live_in_vars() for merge blocks
...
Totals from 18 (0.02% of 79395) affected shaders: (GFX11)
Instrs: 726400 -> 726409 (+0.00%); split: -0.00%, +0.00%
CodeSize: 3750396 -> 3750420 (+0.00%); split: -0.00%, +0.00%
Latency: 4363014 -> 4363039 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 1483486 -> 1483485 (-0.00%)
SClause: 13393 -> 13392 (-0.01%); split: -0.01%, +0.01%
Copies: 70236 -> 70254 (+0.03%); split: -0.01%, +0.03%
Branches: 23811 -> 23810 (-0.00%)
VALU: 404250 -> 404247 (-0.00%); split: -0.00%, +0.00%
SALU: 92208 -> 92226 (+0.02%); split: -0.01%, +0.02%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799 >
2024-03-28 13:57:36 +00:00
c2ac4f1917
aco/spill: use average use distances in process_block()
...
Totals from 128 (0.16% of 79395) affected shaders: (GFX11)
Instrs: 672936 -> 672086 (-0.13%); split: -0.40%, +0.28%
CodeSize: 3574396 -> 3565540 (-0.25%); split: -0.55%, +0.30%
SpillSGPRs: 6254 -> 6636 (+6.11%); split: -0.32%, +6.43%
SpillVGPRs: 967 -> 839 (-13.24%)
Latency: 6489362 -> 6469427 (-0.31%); split: -0.36%, +0.05%
InvThroughput: 2216723 -> 2212417 (-0.19%); split: -0.24%, +0.05%
VClause: 11670 -> 11517 (-1.31%); split: -1.50%, +0.19%
SClause: 15711 -> 15754 (+0.27%); split: -0.29%, +0.57%
Copies: 85185 -> 84155 (-1.21%); split: -2.14%, +0.93%
Branches: 18706 -> 18708 (+0.01%); split: -0.05%, +0.06%
VALU: 386892 -> 386204 (-0.18%); split: -0.40%, +0.23%
SALU: 85754 -> 86211 (+0.53%); split: -0.46%, +1.00%
VMEM: 18777 -> 18524 (-1.35%); split: -1.37%, +0.02%
VOPD: 670 -> 628 (-6.27%); split: +0.75%, -7.01%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799 >
2024-03-28 13:57:36 +00:00
9833120e91
aco/spill: gather information about average use distances
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799 >
2024-03-28 13:57:36 +00:00
b6e56ab356
aco/spill: use live variables instead of next_use_distances in add_coupling_code()
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799 >
2024-03-28 13:57:36 +00:00
7b96482ec9
aco/spill: maintain valid live vars at any point
...
In order to do so, we need to process blocks unconditionally.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799 >
2024-03-28 13:57:36 +00:00
fab8757831
aco/spill: compute live-in variables from live-out
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799 >
2024-03-28 13:57:36 +00:00
bfb62fa70d
aco/spill: pass live_vars to spill_ctx
...
We will use them in future commits.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799 >
2024-03-28 13:57:36 +00:00
66616f0862
aco/spill: Fix assertion for nested loops
...
Fixes: 898fd9227ahttps://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799 >
2024-03-28 13:57:36 +00:00
a863c7951e
aco: remove create_instruction() template parameter
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370 >
2024-03-28 11:25:43 +00:00
9b0ebcc39b
aco: change return type of create_instruction() to Instruction*
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370 >
2024-03-28 11:25:43 +00:00
1187189235
aco: unify different SALU types into single struct SALU_instruction
...
This removes
- SOP1_instruction
- SOP2_instruction
- SOPC_instruction
- SOPK_instruction
- SOPP_instruction
and their corresponding methods.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370 >
2024-03-28 11:25:43 +00:00
898fd9227a
aco/spill: keep loop variables spilled during nested loops
...
Totals from 36 (0.05% of 79395) affected shaders: (GFX11)
Instrs: 2775993 -> 2781480 (+0.20%); split: -0.01%, +0.20%
CodeSize: 14184524 -> 14231956 (+0.33%); split: -0.01%, +0.34%
SpillSGPRs: 1441 -> 1207 (-16.24%)
SpillVGPRs: 477 -> 421 (-11.74%)
Scratch: 35840 -> 35072 (-2.14%)
Latency: 15559241 -> 15606047 (+0.30%); split: -0.01%, +0.31%
InvThroughput: 3787994 -> 3803749 (+0.42%); split: -0.02%, +0.44%
VClause: 76422 -> 76411 (-0.01%)
SClause: 55717 -> 55671 (-0.08%); split: -0.10%, +0.01%
Copies: 214877 -> 215185 (+0.14%); split: -0.04%, +0.18%
Branches: 93158 -> 93129 (-0.03%); split: -0.03%, +0.00%
VALU: 1520556 -> 1525425 (+0.32%); split: -0.00%, +0.32%
SALU: 341104 -> 341408 (+0.09%); split: -0.02%, +0.11%
VMEM: 119673 -> 119617 (-0.05%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774 >
2024-03-25 09:21:20 +00:00
80bc658efc
aco/spill: select more loop-carried variables to be spilled
...
This changes the heuristic to also spill live-through variables
if they are used within the loop if the register pressure is too high.
Totals from 68 (0.09% of 79395) affected shaders: (GFX11)
Instrs: 3385477 -> 3386659 (+0.03%); split: -0.11%, +0.14%
CodeSize: 17329668 -> 17353796 (+0.14%); split: -0.12%, +0.26%
SpillSGPRs: 4246 -> 3659 (-13.82%); split: -17.12%, +3.30%
SpillVGPRs: 978 -> 884 (-9.61%); split: -10.02%, +0.41%
Scratch: 58880 -> 59648 (+1.30%)
Latency: 20567445 -> 20753412 (+0.90%); split: -0.08%, +0.98%
InvThroughput: 5091128 -> 5188098 (+1.90%); split: -0.07%, +1.97%
VClause: 90446 -> 90499 (+0.06%); split: -0.05%, +0.11%
SClause: 68270 -> 68072 (-0.29%); split: -0.35%, +0.06%
Copies: 280689 -> 278003 (-0.96%); split: -1.08%, +0.12%
Branches: 112577 -> 112608 (+0.03%); split: -0.10%, +0.13%
VALU: 1863415 -> 1866878 (+0.19%); split: -0.07%, +0.26%
SALU: 415572 -> 413062 (-0.60%); split: -0.63%, +0.03%
VMEM: 140890 -> 141045 (+0.11%); split: -0.04%, +0.15%
VOPD: 650 -> 670 (+3.08%); split: +3.54%, -0.46%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774 >
2024-03-25 09:21:20 +00:00
31f6aebb2c
aco/spill: keep loop-carried variables spilled at merge blocks
...
If loop-carried variables are partially spilled at some predecessor,
keep them spilled without extra instruction.
Also slightly refactors the code for better readability.
Totals from 13 (0.02% of 79395) affected shaders: (GFX11)
Instrs: 259129 -> 258194 (-0.36%)
CodeSize: 1337604 -> 1331768 (-0.44%)
SpillSGPRs: 561 -> 560 (-0.18%)
SpillVGPRs: 146 -> 140 (-4.11%)
Latency: 2410886 -> 2408560 (-0.10%)
InvThroughput: 759229 -> 758416 (-0.11%)
VClause: 4178 -> 4154 (-0.57%)
SClause: 7217 -> 7219 (+0.03%)
Copies: 32411 -> 32174 (-0.73%)
Branches: 9426 -> 9296 (-1.38%)
VALU: 144602 -> 144098 (-0.35%)
SALU: 30582 -> 30405 (-0.58%)
VMEM: 6961 -> 6908 (-0.76%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774 >
2024-03-25 09:21:20 +00:00
971407bd2f
aco/spill: keep loop-carried variables spilled at loop headers
...
Now, that we avoid re-spilling loop-carried variables, we can
keep them spilled at loop-headers.
Totals from 31 (0.04% of 79395) affected shaders: (GFX11)
Instrs: 1539175 -> 1538109 (-0.07%); split: -0.14%, +0.07%
CodeSize: 7877948 -> 7871916 (-0.08%); split: -0.16%, +0.08%
SpillSGPRs: 1936 -> 1523 (-21.33%)
SpillVGPRs: 160 -> 146 (-8.75%)
Scratch: 18176 -> 16896 (-7.04%)
Latency: 9439964 -> 9441197 (+0.01%); split: -0.01%, +0.02%
InvThroughput: 2406253 -> 2406700 (+0.02%); split: -0.01%, +0.03%
VClause: 38903 -> 38922 (+0.05%)
SClause: 33347 -> 33349 (+0.01%); split: -0.01%, +0.02%
Copies: 131420 -> 130974 (-0.34%); split: -0.57%, +0.23%
Branches: 53091 -> 53212 (+0.23%); split: -0.03%, +0.26%
VALU: 844372 -> 843876 (-0.06%); split: -0.12%, +0.06%
SALU: 190329 -> 189891 (-0.23%); split: -0.40%, +0.17%
VMEM: 60491 -> 60510 (+0.03%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774 >
2024-03-25 09:21:20 +00:00
e968a18652
aco/spill: avoid re-spilling loop-carried variables in add_coupling_code()
...
These variables only need to be spilled once at the loop-preheader.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774 >
2024-03-25 09:21:20 +00:00
7726b92707
aco/spill: avoid re-spilling loop-carried variables in process_block()
...
If a variable is spilled at the loop-header of the current loop,
there is no need to spill it redundantly. Instead, re-use the
spill-slot from the loop-header.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774 >
2024-03-25 09:21:20 +00:00
bb3b6a5ff3
aco/spill: add interferences with variables spilled at loop headers
...
We will re-use the spill slots rather than re-spilling the variables.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774 >
2024-03-25 09:21:20 +00:00
09c5414760
aco/spill: don't prefer to spill phis at merge blocks
...
Totals from 44 (0.06% of 79242) affected shaders: (GFX11)
Instrs: 165578 -> 165545 (-0.02%); split: -0.08%, +0.06%
CodeSize: 833528 -> 832988 (-0.06%); split: -0.11%, +0.04%
SpillSGPRs: 346 -> 329 (-4.91%)
Latency: 1176341 -> 1176231 (-0.01%); split: -0.02%, +0.01%
InvThroughput: 235771 -> 235761 (-0.00%); split: -0.01%, +0.01%
SClause: 3287 -> 3289 (+0.06%); split: -0.21%, +0.27%
Copies: 16290 -> 16299 (+0.06%); split: -0.62%, +0.68%
Branches: 3633 -> 3641 (+0.22%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774 >
2024-03-25 09:21:20 +00:00
d234f789b5
aco/spill: keep live-out variables spilled at branch blocks
...
There is no reason to reload early. Leave scheduling to the scheduler.
Totals from 98 (0.12% of 79242) affected shaders: (GFX11)
Instrs: 841944 -> 841995 (+0.01%); split: -0.02%, +0.03%
CodeSize: 4354332 -> 4354544 (+0.00%); split: -0.02%, +0.02%
SpillSGPRs: 2557 -> 2554 (-0.12%)
Latency: 5772080 -> 5772209 (+0.00%); split: -0.00%, +0.01%
InvThroughput: 1694834 -> 1694873 (+0.00%); split: -0.00%, +0.00%
SClause: 16036 -> 16042 (+0.04%); split: -0.07%, +0.11%
Copies: 85117 -> 85125 (+0.01%); split: -0.18%, +0.19%
Branches: 25222 -> 25219 (-0.01%); split: -0.02%, +0.00%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774 >
2024-03-25 09:21:20 +00:00
32882d647e
aco/spill: refactor adding spilled vars into separate function add_to_spills()
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774 >
2024-03-25 09:21:20 +00:00
a5e8f9f879
aco/spill: add spills_entry interferences only when necessary
...
For variables which are already spilled in previous blocks,
the interferences already exist.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774 >
2024-03-25 09:21:20 +00:00
1a1d6aef56
aco/spill: don't allocate extra spill_id for phi operands in add_coupling_code()
...
We can re-use the phi definition's spill_id.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774 >
2024-03-25 09:21:20 +00:00
b1468fc952
aco/spill: refactor SSA repairing
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774 >
2024-03-25 09:21:19 +00:00
9bbb9f1104
aco: use small_vec as Block::edge_vec for predecessors and successors
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27984 >
2024-03-19 13:06:58 +00:00
07eab4ab40
aco: require linear vgpr uses to be late kill
...
This also removes some linear VGPR tests which will be replaced soon
anyway.
fossil-db (navi31):
Totals from 107 (0.14% of 79242) affected shaders:
Instrs: 66203 -> 66211 (+0.01%); split: -0.09%, +0.10%
CodeSize: 354644 -> 354588 (-0.02%); split: -0.08%, +0.07%
VGPRs: 4476 -> 4452 (-0.54%); split: -0.80%, +0.27%
Latency: 513863 -> 513877 (+0.00%); split: -0.08%, +0.08%
InvThroughput: 68871 -> 68870 (-0.00%); split: -0.02%, +0.02%
SClause: 1589 -> 1590 (+0.06%)
PreVGPRs: 3404 -> 3415 (+0.32%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27697 >
2024-03-06 12:55:45 +00:00
e3098bb232
aco: give spiller more room to assign spilled SGPRs to VGPRs
...
On chordal graphs, a greedy coloring can be done in a way that never uses
more colors than are required for the largest clique. However, since we
have vector values and force phi resources into the same spill slots, the
interference graphs are not chordal, and thus, this assumption doesn't hold.
Use twice as many spill slots as upper bound.
Totals from 10 (0.01% of 79242) affected shaders: (GFX11)
MaxWaves: 52 -> 54 (+3.85%)
Instrs: 271386 -> 271779 (+0.14%)
CodeSize: 1362544 -> 1365432 (+0.21%)
VGPRs: 2536 -> 2532 (-0.16%)
SpillVGPRs: 778 -> 818 (+5.14%)
Scratch: 73472 -> 76800 (+4.53%)
Latency: 3331718 -> 3328798 (-0.09%); split: -0.14%, +0.05%
InvThroughput: 1665860 -> 1643350 (-1.35%); split: -1.40%, +0.05%
VClause: 3292 -> 3329 (+1.12%); split: -0.06%, +1.18%
Copies: 46082 -> 46257 (+0.38%)
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27011 >
2024-01-19 14:15:27 +00:00
Natalie Vock