This has been fixed since V3D 4.2.14 (Rpi4), which is the hardware
we are targetting. Our version resolution doesn't allow us to check
for 4.2 versions lower than .14, but that is okay because the
simulator would still validate this in any case.
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8980>
V3D 3.x has V3D_QPU_WADDR_TMU which in V3D 4.x is V3D_QPU_WADDR_UNIFA
(which isn't a TMU write address). This change passes a devinfo to
any functions that need to do these checks so we can account for the
target V3D version correctly.
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8980>
The QPU scheduler allows to move certain TMU instructions around and
since we enabled pipelining, we need to protect against the case where
doing this might break a TMU sequence. For example, this test:
dEQP-VK.rasterization.line_continuity.line-strip
Was generating this VIR:
mov tmud, t187
mov.pushz null, t176
mov.ifa tmua, t9
nop null; wrtmuc (img[0].p0 | 0x0)
mov tmut, t185
mov tmud, t180
mov.ifa tmusf, t183
nop null; thrsw
where we have a general TMU access (tmud,tmua) followed by an image
access (wrtmuc, tmut, tmud, tmusf), which the QPU scheduler was turning
into:
nop ; nop ; ldunifrf.rf22 (0xffffff00 / -nan)
nop ; nop ; wrtmuc (img[0].p0 | 0x0)
nop ; nop ; ldtmu.r2
add r0, r2, 1 ; nop ; ldtmu.r3
nop ; nop ; ldtmu.r4
nop ; mov tmud, r0
nop ; mov.ifa tmua, rf15
nop ; mov tmut, r4 ; thrsw
nop ; mov tmud, rf22
nop ; mov.ifa tmusf, r3
where it allowed the wrtmuc to move up and before the general TMU access,
leading to an incorrect TMU sequence.
Fix this by flagging TMUA writes (which are the sequence terminators for
general TMU accessess) as writing new TMU configuration, like we do for all
other TMU sequence terminators for textures and images.
Fixes: 197090a3fc ('broadcom/compiler: implement pipelining for general TMU operations')
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8954>
The GL frontend can lower this weird GL feature away for us. This should
fix redeclaration of the gl_Color/SecondaryColor as centroid, since that
case had been missed in the !flat special case here.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8601>
This is trivial because this path relies on our blit_shader interface
which supports this already, so it just needs to pass it along.
I don't think this is ever triggered practice, since we should be
able to handle any case that could require this with the texel buffer
path, but at least it allows us to simplify a bit the code.
Tested by manually disabling the priority paths to ensure we exercise
component swizzles with this path.
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8875>
This path is very memory hungry and batching allows us to reduce
this by allocating memory just once and reuse it for all regions
in the batch instead of allocating once per region.
v2: document return value for this function (apinheiro).
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8875>
We do this by converting them to a compatible color copy and using a
destination color mask as well as a source component swizzle to handle
D24 format semantics according to the V3D hardware requirements,
similar to what we do with our blit shader interface.
This path is faster than the terrible copy_buffer_to_image_blit,
which requires to copy the source buffer to a tiled image first
and should be avoided as much as possible, since it is slow and
can also quickly increase device memory usage.
This fixes occasional OOM errors when loading traces in renderdoc.
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8875>
We have been trying to avoid this by tracking fifo usages in the driver and
flushing all outstanding TMU sequences if we overflowed any of these, however,
this is actually not the most efficient strategy. Instead, we would like to
flush only enough operations to get things going again, which is better for
pipelining. Doing that in the driver would require some additional work, but
thankfully, it is not required, since this seems to be what the hardware does
automatically, so we can just remove overflow tracking for these two fifos
and enjoy the benefits.
This also further improves shader-db stats:
total instructions in shared programs: 8975062 -> 8955145 (-0.22%)
instructions in affected programs: 1637624 -> 1617707 (-1.22%)
helped: 4050
HURT: 2241
Instructions are helped.
total threads in shared programs: 236802 -> 237042 (0.10%)
threads in affected programs: 252 -> 492 (95.24%)
helped: 122
HURT: 2
Threads are helped.
total sfu-stalls in shared programs: 19901 -> 19592 (-1.55%)
sfu-stalls in affected programs: 4744 -> 4435 (-6.51%)
helped: 1248
HURT: 1051
Sfu-stalls are helped.
total inst-and-stalls in shared programs: 8994963 -> 8974737 (-0.22%)
inst-and-stalls in affected programs: 1636184 -> 1615958 (-1.24%)
helped: 4050
HURT: 2239
Inst-and-stalls are helped.
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>
TMU pipelining can severely reduce our capacity to emit TMU spills,
causing us to fail to compile a shader we may otherwise be able to
compile. This is because pipelining extends the liveness of TMU
sequences by posponing the thread switch and LDTMU until a result
is needed, and we can't emit TMU spills while in the middle of a
TMU sequence.
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>
This mostly moves code around to group together the code involved with
actually emitting a TMU sequence. This will make it a bit easier to
then implement pipelining while reusing this code, similar to how we
handled other cases of TMU pipelining.
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>
This follows the same idea as for TMU general instructions of reusing
the existing infrastructure to first count required register writes and
flush outstanding TMU dependencies, and then emit the actual writes, which
requires that we split the code that decides about register writes to
a helper.
We also need to start using a component mask instead of the number
of components that we need to read with a particular TMU operation.
v2: update tmu_writes for V3D_QPU_WADDR_TMUOFF
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>
This creates the basic infrastructure to implement TMU pipelining and
applies it to general TMU. Follow-up patches will expand this
to texture and image/load store operations.
TMU pipelining means that we don't immediately end TMU sequences,
and instead, we postpone the thread switch and LDTMU (for loads)
or TMUWT (for stores) until we really need to do them.
For loads, we may need to flush them if another instruction reads
the result of a load operation. We can detect this because in that
case ntq_get_src() will not find the definition for that ssa/reg
(since we have not emitted the LDTMU instructions for it yet), so
when that happens, we flush all pending TMU operations and then
try again to find the definition for the source.
We also need to flush pending TMU operations when we reach the end
of a control flow block, to prevent the case where we emit a TMU
operation in a block, but then we read the result in another block
possibly under control flow.
It is also required to flush across barriers and discards to honor
their semantics.
Since this change doesn't implement pipelining for texture and
image load/store, we also need to flush outstanding TMU operations
if we ever have to emit one of these. This will be corrected with
follow-up patches.
Finally, the TMU has 3 fifos where it can queue TMU operations.
These fifos have limited capacity, depending on the number of threads
used to compile the shader, so we also need to ensure that we
don't have too many outstanding TMU requests and flush pending
TMU operations if a new TMU operation would overflow any of these
fifos. While overflowing the Input and Config fifos only leads
to stalls (which we want to avoid anyway), overflowing the Output
fifo is incorrect and would end up with a broken shader. This means
that we need to know how many TMU register writes are required
to emit a TMU operation and use that information to decide if we need
to flush pending TMU operations before we emit any register
writes for the new TMU operation.
v2: fix TMU flushing for NIR registers reads (jasuarez)
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>
Follow-up patches will implement support for TMU pipelining in the
compiler, which basically means that we will be able to have more
than one outstanding TMU operation.
Our spilling code currently relies on properly identifying the end
of a TMU sequence (since we can't emit a new TMU sequence for a spill
in the middle of an existing TMU sequence), however, that code expects
that only one TMU sequence may be outstanding, which won't be true
once we implement pipelining.
This change fixes the 'end of TMU sequence' checks to account for this
in preparation for upcoming patches.
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>
Where it is safe to do so, avoid the generation of code to convert a
condition code into a boolean which is then tested to generate a
condition code. This is only done in uniform ifs, and only for condition
values that are SSA and only used once (in that if statement).
shader-db relative to MR 7726:
total instructions in shared programs: 8985667 -> 8974151 (-0.13%)
instructions in affected programs: 390140 -> 378624 (-2.95%)
helped: 810
HURT: 276
helped stats (abs) min: 1 max: 49 x̄: 17.77 x̃: 16
helped stats (rel) min: 0.10% max: 33.63% x̄: 7.97% x̃: 6.45%
HURT stats (abs) min: 1 max: 46 x̄: 10.42 x̃: 10
HURT stats (rel) min: 0.16% max: 21.54% x̄: 2.26% x̃: 2.03%
95% mean confidence interval for instructions value: -11.46 -9.75
95% mean confidence interval for instructions %-change: -5.76% -4.97%
Instructions are helped.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8709>
Now that all drivers are converted over, we can make a few changes.
First off, vk_device_init no longer takes two separate allocators
because we can assume that the parent instance is non-null and it can
pull the instance allocator from that. Second, dispatch tables and the
instance extension table are no longer optional. We leave the device
extension table optional for now because we don't do any verification at
vk_init_physical_device time and some drivers find it more convenient to
set the extensions later in their own physical_device_init for various
reasons.
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8676>
