One advantage here of moving a bunch of stuff to NIR is that we can
now have consistent payload types straight from the NIR conversion to
BRW.
This massively simplifies the BRW lowering code and avoids type errors
that are quite common to make in the backend.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37527>
Use FD20 macro that will account for the implicit LSB zero value and is
already used for sources. For the new macro we need to use the entire
bit-range of the field (55-51), so remove the adjustments we used to
do prior to encoding and decoding.
Fixes assertion in vkpeak (https://github.com/nihui/vkpeak) when running
bf16 tests on BMG. And the code now will correctly apply the subreg_nr
to the destination, e.g. a mad(32) gets splitted into two pieces, the
generation would not fill out the upper-part of the register
```
mad(16) g13<1>BF g10<8,8,1>BF g12<8,8,1>BF g56<1,1,1>F { align1 1H A@5 };
-mad(16) g13<1>BF g10.16<8,8,1>BF g12.16<8,8,1>BF g57<1,1,1>F { align1 2H A@5 };
+mad(16) g13.16<1>BF g10.16<8,8,1>BF g12.16<8,8,1>BF g57<1,1,1>F { align1 2H A@5 };
```
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37236>
The following Vulkan CTS tests that emit massive shaders were
regressing after "intel/brw/xe3+: Select scheduler heuristic with best
trade-off between register pressure and latency.":
dEQP-VK.graphicsfuzz.cov-nested-loops-set-struct-data-verify-in-function
dEQP-VK.graphicsfuzz.cov-dfdx-dfdy-after-nested-loops
The reason is that they have so many nested loops that they cause the
performance analysis utilization estimates to overflow the 32-bit
floating-point variables used to calculate them, which causes our
throughput estimate to underflow and equal zero for those shaders,
which breaks the logic introduced in brw_allocate_registers() to
select the scheduling variant with highest throughput, since none of
the scheduling modes tried has better throughput than the initial
value equal to zero of "best_perf". Instead use -INFINITY as initial
value for "best_perf" so we always select a scheduling mode.
This should have been caught by CI but oddly the tests above are
showing up as "not run" on my last baseline runs, so this wasn't
flagged as a regression for me.
v2: Use -INFINITY instead of previous approach that used NaN (Ian).
Fixes: 531a34c7dd ("intel/brw/xe3+: Select scheduler heuristic with best trade-off between register pressure and latency.")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13884
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13885
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (v1)
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37322>
Move from a single ralloc allocation per instruction to contiguous
blocks of allocations. Still use ralloc for those large blocks.
Each ralloc allocation has at least 5 pointers of overhead, which would
be about a third of the current brw_inst, and get worse as we try to
pack brw_inst better.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36730>
Move all the SEND specific fields from brw_inst into brw_send_inst.
This new instruction kind will contain all variants of SENDs plus the
virtual opcodes that were already relying on those SEND fields.
Use the `as_send()` helper to go from a brw_inst into the brw_send_inst
when applicable. Some of the code was changed to use the brw_send_inst
type directly.
Until other kinds are added, all the instructions are allocated the same
amount of space as brw_send_inst. This ensures that all
brw_transform_inst() calls are still valid. This will change after
a few patches so that BASE instructions can use less memory.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36730>
Prepare code for supporting subclasses of brw_inst for certain
specialized kinds of instructions. This will allow
- Move certain fields from brw_inst to the specialized one,
reducing its size and making it easy to understand what applies
to which instruction;
- Move certain control sources into the specialized inst type, which
currently take a full brw_reg to encode small integers. Reducing
the overall sources we walk and care also might help the code
in general.
Next commits will add the new instruction kinds.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36730>
Flatten all the work being done into brw_new_inst() and
brw_clone_inst() and allocate both the instruction and
the sources in one swoop.
For now we still keep a pointer to the array instead of
declaring an array as last element to still allow growing
the array -- which is used by the compiler in a few places.
This commit removes the constructors for brw_inst, the idea
is that the instructions are managed by the brw_shader, so
we always go through it for new ones.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36730>
In the few cases we have to _increase_ the number of sources, the new
code will not attempt to recollect the memory, i.e. it delays freeing
the old smaller one source array. For the instructions that may need
this (when making a SEND into a SEND_GATHER), this is not expected to
happen more than once.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36730>
These dependency hints were primarily useful for the vec4 backend, where
it was common to write subsets of a vec4's components across multiple
instructions. In the scalar backend, we rarely used them. They also no
longer exist on Tigerlake and later in favor of software scoreboarding.
Dropping this allows us to clean up the IR a bit.
We still use the hardware hints in the generator in a couple places:
- Gfx9-12.0 scratch headers
- Quad swizzles
- Indirect MOV lowering
In theory we might want them back if we moved that lowering to the IR.
For scratch at least, I suspect it won't have a huge impact, as we're
already incurring the cost of spills/fills. The others are fairly rare
as well, so it may not be worth keeping.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36730>
This disables for now the "optimistic" SIMD heuristic that was
implemented for xe3+ and makes it dependent on a debugging option,
instead use the static analysis-based codepath that was used in
previous generations and was extended by previous commits in this MR
to model the xe3 trade-off between register use and thread
parallelism.
The reason is that the main assumption of the optimistic SIMD
heuristic didn't hold up with reality: Real-world testing on PTL shows
that there are many cases where SIMD32 shows performance degradation
relative to SIMD16 despite the ability of xe3 hardware to scale the
GRF file of a thread on demand, unfortunately that scenario seems to
be more pervasive than hoped when the optimistic SIMD heuristic was
implemented pre-silicon.
In many cases what seems to be going on is that even when the register
file is able to scale with the increased register use of SIMD32, the
thread parallelism of the EU is scaled down by a similar factor, so at
the bottom line SIMD32 (depending on the actual ratio of register use
between both variants) may not buy us anything, and it frequently
encounters constraints (like SIMD lowering and less effective
scheduling) that lead to worse codegen than SIMD16, easily tipping the
balance in favor of SIMD16. The extension of the performance analysis
pass that was done in a previous commit allows the original SIMD32
heuristic to take into account quantitatively this effect, and that
seems pretty effective at disabling SIMD32 shaders that underperform
judging from the statistically significant improvement of most Traci
test-cases that run on my PTL system (4 iterations, 5% significance),
no statistically significant regressions were observed:
Nba2K23-trace-dx11-2160p-ultra: 10.16% ±0.34%
Superposition-trace-dx11-2160p-extreme: 4.06% ±0.50%
TotalWarWarhammer3-trace-dx11-1080p-high: 3.52% ±0.76%
Payday3-trace-dx11-1440p-ultra: 2.41% ±0.81%
MetroExodus-trace-dx11-2160p-ultra: 2.28% ±0.78%
Borderlands3-trace-dx11-2160p-ultra: 1.89% ±0.65%
MountAndBlade2-trace-dx11-1440p-veryhigh: 1.81% ±0.40%
Blackops3-trace-dx11-1080p-high: 1.66% ±0.29%
HogwartsLegacy-trace-dx12-1080p-ultra: 1.53% ±0.22%
TotalWarPharaoh-trace-dx11-1440p-ultra: 1.44% ±0.31%
Fortnite-trace-dx11-2160p-epix: 1.44% ±0.27%
Naraka-trace-dx11-1440p-highest: 1.39% ±0.27%
PubG-trace-dx11-1440p-ultra: 1.30% ±0.49%
Destiny2-trace-dx11-1440p-highest: 1.10% ±0.23%
Factorio-trace-1080p-high: 1.10% ±1.77%
TerminatorResistance-trace-dx11-2160p-ultra: 1.08% ±0.31%
Ghostrunner2-trace-dx11-1440p-ultra: 1.05% ±0.15%
ShadowTombRaider-trace-dx11-2160p-ultra: 0.98% ±0.19%
CitiesSkylines2-trace-dx11-1440p-high: 0.67% ±0.19%
Palworld-trace-dx11-1080p-med: 0.44% ±0.22%
The downside is that this will reverse the large reduction in
compile-time we gained from the optimistic SIMD heuristic -- The
run-time of both shader-db and fossil-db jump back up by nearly 20%
with this change. I'm working on a better compromise based on
run-time feedback that will hopefully allow us to preserve the
compile-time benefit of the optimistic heuristic without the reduction
in run-time performance, but in the meantime it seems like the
run-time performance gap from SIMD32 is the more urgent issue to
address since it has an impact on titles across the board. Despite
the reversal of that compile-time improvement xe3 still achieves
slightly lower compile time on the average than previous generations
as a result of VRT, so this doesn't seem terribly tragic.
v2: Add bit to brw_get_compiler_config_value() (Lionel).
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36618>
Lionel Landwerlin