bcf94bb933
aco: properly recognize that s_waitcnt mitigates VMEMtoScalarWriteHazard
...
fossil-db (Navi):
Totals from 555 (0.41% of 135946) affected shaders:
CodeSize: 1005716 -> 1003400 (-0.23%)
Instrs: 195326 -> 194744 (-0.30%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5923 >
2020-07-18 00:14:12 +00:00
2f424c83e0
aco: only break SMEM clauses if XNACK is enabled (mostly APUs)
...
According to LLVM, it seems only required for APUs like RAVEN, but
we still ensure that SMEM stores are in their own clause.
pipeline-db (VEGA10):
Totals from affected shaders:
SGPRS: 1775364 -> 1775364 (0.00 %)
VGPRS: 1287176 -> 1287176 (0.00 %)
Spilled SGPRs: 725 -> 725 (0.00 %)
Spilled VGPRs: 0 -> 0 (0.00 %)
Code Size: 65386620 -> 65107460 (-0.43 %) bytes
Max Waves: 287099 -> 287099 (0.00 %)
pipeline-db (POLARIS10):
Totals from affected shaders:
SGPRS: 1797743 -> 1797743 (0.00 %)
VGPRS: 1271108 -> 1271108 (0.00 %)
Spilled SGPRs: 730 -> 730 (0.00 %)
Spilled VGPRs: 0 -> 0 (0.00 %)
Code Size: 64046244 -> 63782324 (-0.41 %) bytes
Max Waves: 254875 -> 254875 (0.00 %)
This only affects GFX6-GFX9 chips because the compiler uses a
different pass for GFX10.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4349 >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4349 >
2020-04-01 17:50:31 +00:00
c6e0c062da
aco: improve control flow handling in GFX6-9 NOP pass
...
Fixes Detroit: Become Human hang. Also affects World of Warships.
pipeline-db (Tahiti):
Totals from affected shaders:
SGPRS: 0 -> 0 (0.00 %)
VGPRS: 0 -> 0 (0.00 %)
Spilled SGPRs: 0 -> 0 (0.00 %)
Spilled VGPRs: 0 -> 0 (0.00 %)
Scratch size: 0 -> 0 (0.00 %) dwords per thread
Code Size: 0 -> 0 (0.00 %) bytes
LDS: 0 -> 0 (0.00 %) blocks
Max Waves: 0 -> 0 (0.00 %)
pipeline-db (Polaris):
Totals from affected shaders:
SGPRS: 17168 -> 17168 (0.00 %)
VGPRS: 11296 -> 11296 (0.00 %)
Spilled SGPRs: 1870 -> 1870 (0.00 %)
Spilled VGPRs: 0 -> 0 (0.00 %)
Scratch size: 0 -> 0 (0.00 %) dwords per thread
Code Size: 1472628 -> 1473292 (0.05 %) bytes
LDS: 0 -> 0 (0.00 %) blocks
Max Waves: 628 -> 628 (0.00 %)
pipeline-db (Vega):
Totals from affected shaders:
SGPRS: 17168 -> 17168 (0.00 %)
VGPRS: 11296 -> 11296 (0.00 %)
Spilled SGPRs: 1870 -> 1870 (0.00 %)
Spilled VGPRs: 0 -> 0 (0.00 %)
Scratch size: 0 -> 0 (0.00 %) dwords per thread
Code Size: 1409716 -> 1410380 (0.05 %) bytes
LDS: 0 -> 0 (0.00 %) blocks
Max Waves: 0 -> 0 (0.00 %)
Max Waves is lower than it should be because of a null winsys bug.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4004 >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4004 >
2020-03-05 19:37:24 +00:00
47b7f104a0
aco: consider non-hazard writes in handle_raw_hazard_internal
...
I think this helps GFX6 in particular because code like this is common:
s_add_i32 s4, 0x60, s3
s_mov_b32 s5, 0
s_load_dwordx4 s[4:7], s[4:5], 0x0
s_buffer_load_dword s4, s[4:7], 0xcc
pipeline-db (Tahiti):
Totals from affected shaders:
SGPRS: 1923878 -> 1923878 (0.00 %)
VGPRS: 1528964 -> 1528964 (0.00 %)
Spilled SGPRs: 476 -> 476 (0.00 %)
Spilled VGPRs: 0 -> 0 (0.00 %)
Scratch size: 0 -> 0 (0.00 %) dwords per thread
Code Size: 88723604 -> 88528880 (-0.22 %) bytes
LDS: 241 -> 241 (0.00 %) blocks
Max Waves: 145402 -> 145402 (0.00 %)
pipeline-db (Polaris):
Totals from affected shaders:
SGPRS: 428128 -> 428128 (0.00 %)
VGPRS: 353092 -> 353092 (0.00 %)
Spilled SGPRs: 119251 -> 119251 (0.00 %)
Spilled VGPRs: 0 -> 0 (0.00 %)
Scratch size: 0 -> 0 (0.00 %) dwords per thread
Code Size: 57580468 -> 57563964 (-0.03 %) bytes
LDS: 0 -> 0 (0.00 %) blocks
Max Waves: 11631 -> 11631 (0.00 %)
piepline-db (Vega):
Totals from affected shaders:
SGPRS: 425016 -> 425016 (0.00 %)
VGPRS: 349588 -> 349588 (0.00 %)
Spilled SGPRs: 117835 -> 117835 (0.00 %)
Spilled VGPRs: 0 -> 0 (0.00 %)
Scratch size: 0 -> 0 (0.00 %) dwords per thread
Code Size: 54890792 -> 54874432 (-0.03 %) bytes
LDS: 0 -> 0 (0.00 %) blocks
Max Waves: 54 -> 54 (0.00 %)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4004 >
2020-03-05 19:37:24 +00:00
38743577f8
aco: improve get_wait_states()
...
pipeline-db (Tahiti):
Totals from affected shaders:
SGPRS: 21208 -> 21208 (0.00 %)
VGPRS: 22388 -> 22388 (0.00 %)
Spilled SGPRs: 0 -> 0 (0.00 %)
Spilled VGPRs: 0 -> 0 (0.00 %)
Scratch size: 0 -> 0 (0.00 %) dwords per thread
Code Size: 3278596 -> 3277004 (-0.05 %) bytes
LDS: 19 -> 19 (0.00 %) blocks
Max Waves: 238 -> 238 (0.00 %)
pipeline-db (Polaris):
Totals from affected shaders:
SGPRS: 64 -> 64 (0.00 %)
VGPRS: 96 -> 96 (0.00 %)
Spilled SGPRs: 0 -> 0 (0.00 %)
Spilled VGPRs: 0 -> 0 (0.00 %)
Scratch size: 0 -> 0 (0.00 %) dwords per thread
Code Size: 5200 -> 5192 (-0.15 %) bytes
LDS: 0 -> 0 (0.00 %) blocks
Max Waves: 10 -> 10 (0.00 %)
pipeline-db (Vega):
Totals from affected shaders:
SGPRS: 0 -> 0 (0.00 %)
VGPRS: 0 -> 0 (0.00 %)
Spilled SGPRs: 0 -> 0 (0.00 %)
Spilled VGPRs: 0 -> 0 (0.00 %)
Scratch size: 0 -> 0 (0.00 %) dwords per thread
Code Size: 0 -> 0 (0.00 %) bytes
LDS: 0 -> 0 (0.00 %) blocks
Max Waves: 0 -> 0 (0.00 %)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4004 >
2020-03-05 19:37:24 +00:00
7f1b537304
aco: add new NOP insertion pass for GFX6-9
...
This new pass is more similar to the GFX10 pass and should be able to
handle control flow better.
No pipeline-db changes.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4004 >
2020-03-05 19:37:24 +00:00
71440ba0f5
aco: reorder VMEM operands in ACO IR
...
For all VMEM instructions, the resource constant is now
in operands[0]. For MIMG instructions, the sampler shares
operands[1] with write data in case this instruction writes memory.
Moving the VADDR to be the last operand for MIMG is the first step to
support Navi NSA encoding.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3602 >
2020-01-29 18:45:23 +00:00
e6c90e4af9
aco: fix WaR check for >64-bit FLAT/GLOBAL instructions
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Fixes: 5986e0019daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3541 >
2020-01-27 14:50:37 +00:00
1ac49ba908
aco: fix a hazard with v_interp_* and v_{read,readfirst}lane_* on GFX6
...
It's required to insert 1 wait state if the dst VGPR of any v_interp_*
is followed by a read with v_readfirstlane or v_readlane to fix GPU
hangs on GFX6. Note that v_writelane_* is apparently not affected.
This hazard isn't documented anywhere but AMD confirmed it.
This fixes a GPU hang with the texturemipmapgen Sascha demo on GFX6.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3533 >
2020-01-24 18:34:27 +00:00
c787b8d2a1
aco/gfx10: Fix VcmpxExecWARHazard mitigation.
...
The SOPP instruction shouldn't have a definition, and its block
should be set to -1 in order to prevent it from being recognized
as a branch.
Also fix a typo in the readme.
Fixes: d6dfce02d0timur.kristof@gmail.com >
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3552 >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3552 >
2020-01-24 16:21:08 +00:00
70f63c1988
aco: improve support for s_sendmsg
...
In particular, the messages needed for GS.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2421 >
2020-01-24 13:35:07 +00:00
6a586a6006
aco: split read/writelane opcode into VOP2/VOP3 version for SI/CI
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
2019-12-07 11:23:11 +01:00
6fc9ddfef8
aco: recognize SI/CI SMRD hazards
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
2019-12-07 11:23:11 +01:00
5986e00194
aco: improve WAR hazard workaround with >64bit stores
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
2019-11-29 17:46:01 +00:00
a9fc81b098
aco: add v_nop inbetween exec write and VMEM/DS/FLAT
...
LLVM and the proprietary compiler seem to do this
Fixes: b01847bd9pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
2019-11-29 17:46:01 +00:00
8657eede8a
aco: check if SALU instructions are predeceeded by exec when calculating WQM needs
...
Reviewed-By: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
2019-11-14 17:27:10 +01:00
d4684a294b
aco: a couple loop handling fixes for GFX10 hazard pass
...
It was joining from the wrong blocks and block.kind is a bitmask instead
of an enum.
Reviewed-By: Timur Kristóf <timur.kristof@gmail.com >
2019-10-30 18:13:53 +00:00
c580f134ae
aco: Refactor hazard mitigations, separate pass for GFX10.
...
GFX10 hazards require a different approach compared to previous
generations, for example it doesn't need s_nop, and most hazards
can't be solved by adding NOPs at all. Also, they are not
resolved by branch instructions.
This commit reorganizes aco_insert_NOPs so that there is now a
separate pass for GFX10. The new GFX10 pass also respects the
control flow of the shader.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
2019-10-25 10:10:42 +02:00
b01847bd94
aco/gfx10: Fix mitigation of VMEMtoScalarWriteHazard.
...
This commit refines the VMEMtoScalarWriteHazard mitigation, based
upon a closer look at what LLVM does. Also changes the code to
match the structure of the other hazard mitigations.
* The hazard is not only triggered by VMEM, FLAT and GLOBAL
but also SCRATCH and DS instructions.
* The SMEM/SALU instructions only cause a hazard when they
write a register that the VMEM/etc. are reading.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
2019-10-25 10:10:42 +02:00
c037ba1bb7
aco/gfx10: Mitigate LdsBranchVmemWARHazard.
...
There is a hazard caused by there is a branch between a
VMEM/GLOBAL/SCRATCH instruction and a DS instruction.
This commit adds a workaround that avoids the problem.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
2019-10-25 10:10:42 +02:00
09d676d81a
aco/gfx10: Mitigate SMEMtoVectorWriteHazard.
...
There is a hazard that happens when an SMEM instruction
reads an SGPR and then a VALU instruction writes that same SGPR.
This commit adds a workaround that avoids the problem.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
2019-10-25 10:10:42 +02:00
d6dfce02d0
aco/gfx10: Mitigate VcmpxExecWARHazard.
...
There is a hazard when a non-VALU instruction reads the EXEC mask
and then a VALU instruction writes the EXEC mask.
This commit adds a workaround that avoids the problem.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
2019-10-25 10:10:42 +02:00
e5a8616973
aco/gfx10: Mitigate VcmpxPermlaneHazard.
...
Any permlane instruction that follows any VOPC instruction can cause a hazard,
this commit implements a workaround that avoids this causing a problem.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
2019-10-25 10:10:42 +02:00
6a6bef59b0
aco: Initial work to avoid GFX10 hazards.
...
Currently just breaks up SMEM groups and fixes
FeatureVMEMtoScalarWriteHazard (name from LLVM).
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-By: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
2019-10-10 09:57:53 +02:00
93c8ebfa78
aco: Initial commit of independent AMD compiler
...
ACO (short for AMD Compiler) is a new compiler backend with the goal to replace
LLVM for Radeon hardware for the RADV driver.
ACO currently supports only VS, PS and CS on VI and Vega.
There are some optimizations missing because of unmerged NIR changes
which may decrease performance.
Full commit history can be found at
https://github.com/daniel-schuermann/mesa/commits/backend
Co-authored-by: Daniel Schürmann <daniel@schuermann.dev >
Co-authored-by: Rhys Perry <pendingchaos02@gmail.com >
Co-authored-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl >
Co-authored-by: Connor Abbott <cwabbott0@gmail.com >
Co-authored-by: Michael Schellenberger Costa <mschellenbergercosta@googlemail.com >
Co-authored-by: Timur Kristóf <timur.kristof@gmail.com >
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl >
2019-09-19 12:10:00 +02:00
Rhys Perry