e0f4b52559
aco: Add Primitive Ordered Pixel Shading waitcnt rules
...
When letting the overlapping waves enter their ordered sections, there must
be no memory accesses to resources which need primitive-ordered access that
are still pending, or there would be a race between the current wave and
the overlapping waves.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250 >
2023-06-26 15:58:04 +00:00
a87628cd08
aco: Send MSG_ORDERED_PS_DONE where necessary
...
If the wave has set the Primitive Ordered Pixel Shading packer ID hardware
register, it must send MSG_ORDERED_PS_DONE once before the program ends.
It's also safe to send the message if the packer ID register hasn't been
set yet, therefore the message may be sent conservatively. For simplicity,
to ensure that it's sent on all execution paths after setting the packer ID
register, always sending it from a top-level block. This is required for
GFX9-10.3 POPS.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250 >
2023-06-26 15:58:04 +00:00
2194e8bd82
aco: Add Primitive Ordered Pixel Shading scheduling rules
...
Implementing the acquire/release semantics of fragment shader interlock
ordered section in Vulkan, and preventing reordering of memory accesses
requiring primitive ordering out of the ordered section.
Also, the ordered section should be as short as possible, so not reordering
the instructions awaiting overlapped waves upwards, and the exit from the
ordered section downwards.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250 >
2023-06-26 15:58:04 +00:00
6082e126eb
aco: Skip waitcnt insertion in the discard early exit block
...
Waits are needed for early exits from inside a Primitive Ordered Pixel
Shading ordered section, but that code doesn't insert them reliably anyway
because it doesn't obtain the counters for the exact locations of the
jumps, which may be anywhere inside the predecessor blocks.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250 >
2023-06-26 15:58:04 +00:00
f8e744f07f
aco: Add Primitive Ordered Pixel Shading pseudo-instructions
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250 >
2023-06-26 15:58:04 +00:00
94d2888da2
aco: Add s_wait_event argument bit definitions
...
A wait for export_ready (if the corresponding bit is not set in the
instruction) is done to enter the Primitive Ordered Pixel Shading ordered
section on GFX11.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250 >
2023-06-26 15:58:04 +00:00
b892f31e4d
ac: Define POPS collision wave ID argument SGPR
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250 >
2023-06-26 15:58:03 +00:00
308a5ea43a
aco: Support pops_exiting_wave_id PhysReg usage
...
pops_exiting_wave_id is a volatile ALU source operand containing the ID of
the latest wave that hasn't exited yet, for comparing with the newest
overlapped wave ID in overlapping waves.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250 >
2023-06-26 15:58:03 +00:00
90c88f5933
ac/nir: Support Primitive Ordered Pixel Shading in lower_ps
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250 >
2023-06-26 15:58:03 +00:00
82e2802b7d
radv/amdgpu: add a helper to get a new IB
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23727 >
2023-06-26 09:10:10 +00:00
148f42be1d
radv/amdgpu: rename old_ib_buffers to ib_buffers
...
No need to prefix with 'old' actually because this is just an array
of IB buffers.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23727 >
2023-06-26 09:10:10 +00:00
d74de65069
radv/amdgpu: use cs_finalize() when growing a CS
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23727 >
2023-06-26 09:10:10 +00:00
437456b47c
radv/amdgpu: use the array of IB buffers for the chained IB path
...
For executing IB on the compute queue (ie. IB2 isn't supported), we
will need to break chaining, this is a first step towards this.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23727 >
2023-06-26 09:10:10 +00:00
81e308df72
radv/amdgpu: do not set the IB size when ending a CS with RADV_DEBUG=noibs
...
This was only necessary for preambles/postambles, let's clarify this
by determining the IB info from the first IB in the array instead.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23727 >
2023-06-26 09:10:10 +00:00
df0c742543
radv/amdgpu: rework growing a CS with the chained IB path slightly
...
This should allow us to use cs_finalize().
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23727 >
2023-06-26 09:10:09 +00:00
c11a62a7b0
radv/amdgpu: use the correct IB size when growing a CS with RADV_DEBUG=noibs
...
The current IB size is copied when radv_amdgpu_cs_add_old_ib_buffer()
is called, which might not be the real IB size because we might still
pad the CS with NOP packets after.
Found by inspection.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23727 >
2023-06-26 09:10:09 +00:00
d3e5e04a75
amd/drm-shim: use fixed-width types
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Eric Engestrom <eric@igalia.com >
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9221
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23725 >
2023-06-23 18:35:52 +00:00
0f8864e047
radv: adjust alignment of the preprocess buffer with DGC
...
The preprocess buffer is the buffer used to generate the cmdbuf. It
was aligned to 256 bytes but the correct alignment is actually
ac_gpu_info::ib_alignment.
Otherwise, if a DGC IB is executed like a IB1, this hits an assertion
in radv_amdgpu_cs_submit() because the alignment is incorrect.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23764 >
2023-06-23 17:17:08 +00:00
06cdf222a6
radv: only dirty the active push constant stages with DGC
...
It's unnecessary to dirty all stages.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23761 >
2023-06-23 16:56:44 +00:00
3b329e195e
radv: only dirty the index type when necessary with DGC
...
This should only be needed for non-indexed draws and it's already
dirty if the DGC binds an index buffer.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23761 >
2023-06-23 16:56:44 +00:00
2d97cc89fb
radv/amdgpu: dump all cs with RADV_DEBUG=noibs
...
It was only dumping the oldest.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23646 >
2023-06-23 16:35:22 +00:00
8af705a856
radv/amdgpu: fix dumping cs with RADV_DEBUG=noibs
...
The ib_buffer is NULL now.
Fixes: 50e6b16855samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23646 >
2023-06-23 16:35:21 +00:00
ae7721d163
radv: reserve more space in CS for SQTT
...
Otherwise, it can hit an assertion.
Fixes: 7893040f80samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23826 >
2023-06-23 13:51:13 +00:00
bbdbab15fc
aco: Drop NIR parallel copy handling
...
Backends never see these instructions.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Suggested-by: Daniel Schürmann <daniel@schuermann.dev >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23831 >
2023-06-23 13:25:22 +00:00
3b21c59fc3
aco: Remove unneeded stage related info fields.
...
Cleanup of various fields with redundant information.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23597 >
2023-06-23 12:49:05 +00:00
bc971ba2c7
aco: Use aco_shader_info::hw_stage instead of guessing.
...
With this change, ACO is going to rely on the caller to set
the HW stage and will no longer guess it from the input shaders.
This will help enable compiling merged shaders separately,
but that will need further changes in instruction selection.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23597 >
2023-06-23 12:49:05 +00:00
6028c146d5
radv: Set aco_shader_info::hw_stage
...
ACO will rely on this field instead of guessing
the stage internally.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23597 >
2023-06-23 12:49:05 +00:00
0fef6b95ca
aco: Add hw_stage field to aco_shader_info.
...
Unused in this commit, but this is going to replace the shader
stage selection inside ACO after the drivers set it correctly.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23597 >
2023-06-23 12:49:04 +00:00
05928f4200
aco: Use ac_hw_stage instead of aco-specific HWStage.
...
The new ac_hw_stage is going to be used by drivers as well.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23597 >
2023-06-23 12:49:04 +00:00
cc2307008a
ac: Add ac_hw_stage enum.
...
This is going to be shared between RADV, RadeonSI and ACO.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Qiang Yu <yuq825@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23597 >
2023-06-23 12:49:04 +00:00
b69a1b4153
vulkan: Migrate shader module hash to BLAKE3.
...
Shaders are the largest thing we hash now, so they benefit from a faster
hash.
Change the field name from `sha1` to `hash` to avoid tying the definition
to a particular algorithm. This doubles down as a precaution against
callers still assuming a 20-byte hash (in which case the compilation will
error out).
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22571 >
2023-06-23 09:28:04 +00:00
e15a4e6e1a
radv: pre-init surface info
...
this is costly to do at render time, so avoid it when possible
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23770 >
2023-06-22 13:36:13 +00:00
3005c27a92
Revert "amd/ci: temporarily disable some manual jobs that take a long time to run"
...
This reverts commit 4031ed5c8amartin.roukala@mupuf.org >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23798 >
2023-06-22 12:42:08 +00:00
77f5b1cce0
radeonsi: clean up #includes
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23517 >
2023-06-22 08:35:55 +00:00
4031ed5c8a
amd/ci: temporarily disable some manual jobs that take a long time to run
...
We are trying to re-enable the valve CI... but doing so runs all the
jobs, including the manual ones.
Since some can take over an hour to run, let's disable them, and
re-enable them in another MR by reverting this commit.
Sorry for the noise!
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23789 >
2023-06-22 08:10:06 +00:00
df6749ed56
radv: inline radv_can_enable_dual_src()
...
this is unexpectedly heavy
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23601 >
2023-06-21 18:59:07 +00:00
8b73109a93
radv: remove redundant intermediate variable in radv_is_mrt0_dual_src()
...
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23601 >
2023-06-21 18:59:07 +00:00
cfa7eec06c
aco: don't set exec_hi for wave32 scan reductions
...
fossil-db (wave32):
Totals from 21 (0.02% of 133428) affected shaders:
Instrs: 10778 -> 10712 (-0.61%)
CodeSize: 56604 -> 56208 (-0.70%)
Latency: 168293 -> 168251 (-0.02%)
InvThroughput: 25256 -> 25253 (-0.01%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23745 >
2023-06-21 17:58:44 +00:00
9d75795087
radv: Move most of DB_SHADER_CONTROL to PS, more precise GFX11 blend WA
...
Move most of the DB_SHADER_CONTROL fields from the pipeline to the pixel
shader for preparation for shader objects.
Also, the GFX11 export conflict bug workaround doesn't need to be enabled
for non-1x sample counts or if blending is not enabled, so make the
application of DB_SHADER_CONTROL consider the current sample count and
blending state even if they're dynamic.
Having access to the exact sample count in DB_SHADER_CONTROL setup is also
necessary for good performance in SampleInterlock execution modes of
fragment shader interlock, for configuration of POPS_OVERLAP_NUM_SAMPLES
(GFX9-10.3) or OVERRIDE_INTRINSIC_RATE (GFX11), as PixelInterlock is
massively slower with multisampling due to overlap between adjacent
polygons sharing covered pixels among the common edge.
The name of the dynamic state controlling DB_SHADER_CONTROL is now
unambiguous - previously line rasterization mode had effect on attachment
feedback loop state emission.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23474 >
2023-06-21 14:16:24 +00:00
48b32124c6
radv/rt: Store the prolog outside the shaders array
...
Avoids including it in executable statistics queries.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23524 >
2023-06-21 09:13:04 +00:00
ebe4e337fd
radv: Use _mesa_shader_stage_to_string for executable name
...
We can not use it for all stages because RADV has some special stages
and shader-db doesn't handle all stages.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23524 >
2023-06-21 09:13:04 +00:00
334a20ae28
radv: Implement executable properties for ray tracing stages
...
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23524 >
2023-06-21 09:13:04 +00:00
bf5b74c3ba
radv: Use get_shader_from_executable_index for executable properties
...
This will make handling ray tracing pipelines easier.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23524 >
2023-06-21 09:13:04 +00:00
8a5c4b3ebf
radv: Adjust the traversal shader description
...
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23524 >
2023-06-21 09:13:04 +00:00
eeae2fb5c4
radv: Move the shader type to radv_shader_info
...
Since the default value is 0, this removes some boilerplate.
Moving it to the shader info also makes it accessible using
radv_shader.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23524 >
2023-06-21 09:13:04 +00:00
836990b651
radv: enable NV_device_generated_commands on GFX6
...
Now that GFX6 supports IBs, DGC can also be enabled.
Pass all vkd3d-proton tests on Pitcairn.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23689 >
2023-06-21 06:54:03 +00:00
a9c8366261
radv: implement padding cmdbuffer for DGC on GFX6
...
GFX6 only supports PKT2_NOP_PAD.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23689 >
2023-06-21 06:54:03 +00:00
146adbd2f4
radv/video: drop incorrect defines for uapi ones.
...
I missed there's an off by one, and found the kernel defined
the correct bits, so just use that.
Fixes: 9477f117f4dev@lynne.ee >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23757 >
2023-06-21 05:28:01 +00:00
f1f01aaef5
aco/gfx11: schedule for VMEM store clauses
...
fossil-db (gfx1100):
Totals from 49486 (37.09% of 133428) affected shaders:
Instrs: 18376819 -> 18480712 (+0.57%); split: -0.00%, +0.57%
CodeSize: 91810836 -> 92227292 (+0.45%); split: -0.00%, +0.45%
VGPRs: 2031824 -> 2047784 (+0.79%); split: -0.02%, +0.81%
Latency: 104259318 -> 103804792 (-0.44%); split: -0.44%, +0.00%
InvThroughput: 16388760 -> 16399819 (+0.07%); split: -0.13%, +0.19%
VClause: 568844 -> 432401 (-23.99%)
Copies: 1197942 -> 1231202 (+2.78%); split: -0.08%, +2.86%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23505 >
2023-06-20 18:37:34 +00:00
f837fec213
aco/gfx11: use s_clause with stores
...
fossil-db (gfx1100):
Totals from 3535 (2.65% of 133428) affected shaders:
Instrs: 1963996 -> 1968979 (+0.25%)
CodeSize: 10077784 -> 10097716 (+0.20%)
Latency: 13264216 -> 13264741 (+0.00%)
InvThroughput: 2995676 -> 3004083 (+0.28%); split: -0.00%, +0.28%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23505 >
2023-06-20 18:37:33 +00:00
Vitaliy Triang3l Kuzmin