595f03957c
nak/nir: Several FS output fixes
...
1. Lower arrays to elements
2. Multiply locations by 4 to get GPR numbers
3. Fix dual-src blend
4. Sample mask and depth both consume slots if either is used
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
ea750f1ffd
nak: Handle empty OpFSOut
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
ecf5c4c13b
nak: Properly handle OpFSOut in RA and liveness
...
That one's a bit magic because it exists to ensure that each output GPR
ends up in the right register at the end of the shader. We tried to
handle it as a simple lowering before but it's easier and safer to
handle it directly in RA and liveness.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
c47488341e
nak: Use load_interpolated_input for frag_coord
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
0e327167a2
nak: do perspective divide for interp none as well
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
ab80e1189f
nak: Add support for centroid and sample interp modes
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
850668c942
nak: make ipa encoding match the order in codegen gv100
...
78 has interp mode
76 has sample mode
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
dadc79d96f
nak: Handle flat FS inputs
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
ea6057deb4
nvk: Use sysvals for fragcoord etc. with NAK
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
073b2257dd
nak: Emit FS_OUT before EXIT
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
a07aa29f61
nak: Re-work Instr::get_latency()
...
We re-order and re-arrange the whole thing by instruction type. Also,
instead of returning an Option<u32>, have a has_fixed_latench() method
to check the instruction and then get_dst_latench() to get the latency
from instruction launch to the given destination index being available.
This lets us handle predicates properly which have a different number of
cycles for some reason. Oh, it's now just as correct as the estimates
in nv50_ir_target_gm107.cpp.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
3ad3f1376c
nak: Rework calc_delay.rs
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
2288b33a48
nak: Rework the barrier insert pass
...
This is mostly a refactor but it does tuck in a three functional
changes:
1. We no lonter insert read barriers for sources that we immediately
overwrite in the instruction.
2. We track read and write barriers separately so we don't wait for
read-after-read hazards but do for write-after-read.
3. Wait on all barriers in branch instructions
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
91ea23103a
nak: Use the right number of predicates in RegTracker
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
57dde7f217
nak: Implement Index[Mut] for RegTracker
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
405a9ccca8
nak: Make PRMT selection a Src
...
On SM50, because of immediate form limitations to 20 bits, we need to
use a register to use PRMT.
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
c38c456270
nak: sm75: Fix panic when encoding MUFU with SQRT and TANH
...
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
1b869d2db5
nak: Rename OpBFind to OpFlo
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
eb10a778fd
nak: Fix instruction ordering in nak_ir.rs
...
I'm trying to stick to the ordering in the NVIDIA docs.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
3e8344a4c8
nak: Remap GLSL_SAMPLER_DIM_SUBPASS and SUBPASS_MS to 2D and MS
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
80230731ec
nak: IMAD also has a destination predicate
...
Also use set_pred_dst(.., Dst::None) for IMAD64
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
ab7a4b531d
nak: Disable lower_image_size_to_txs for NAK
...
We don't need it for NAK and it actually screws up cubes.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
86c7bcc9e3
nak/legalize: Fix too many IADD3 source modifiers
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
0222107699
nak: Move all the IADD3 insanity to a new OpIAdd3X opcode
...
Because of its crazy behavior around overflow, we don't want the full
IADD3 opcode to support any sort of source modifier propagation. This
makes us a new OpIAdd3X opcode which contains all the crazy and lets
IAdd3 remain the usual 32-bit integer thing everyone knows and loves.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
c5b9d42ac2
nak: Add an alloc_vec() to SSAValueAllocator
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
52acb12f75
nak: Document spilling and RA
...
This isn't full documentation but it at least sprinkles paper citations
all over as well as descriptions of where we diverge from said papers.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
a0bf406057
nak/spill: Tweak the construction of S sets
...
Instead of just taking live-in \ W, consider anything previously spilled
to be spilled. This lets us avoid a bunch of redundant spills because
we now allow spills to persist across blocks even if the value is in W.
In the loop header case, however, we still need to add in live-in \ W or
else we can end up in cases where a value is neither in W nor S.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
ea0ae395a9
nak/ra: Improve coalescing
...
We now delete OpParCopy src/dst pairs when they get coalesced. We
can also coalesce OpCopy which is potentially useful given they are
generated by spilling.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
46d489fa4d
nak: Implement shader clock
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
84d1765690
nak: Add a debug flag to test spilling
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
07b9465b5d
nak: Spill registers before RA
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
aa93578915
nak: Use the correct number of GPRs on Turing+
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
bcad2add47
nak: Add a spilling pass
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
a502a994e8
nak: Add more NextUseLiveness helpers
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
72a4721412
nak: Add a RegFileSet filter to NextUseLiveness::for_function()
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
002ee9e40e
nak: Expose LiveSet for incremental liveness tracking
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:12 +00:00
8c04737a6a
nak/ra: Don't free killed for OpPhiSrcs
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:11 +00:00
efee3264b2
nak/ra: Handle parallel copies as a special case
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:11 +00:00
4d5bf265b0
nak/ra: Drop the pointless AssignRegs struct
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:11 +00:00
57a7932bb5
nak: Union find
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:11 +00:00
4a08105922
nak: Add an SSA repair pass
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:11 +00:00
214b7e4b88
nak: Add a to-CSSA pass
...
Conventional SSA (also called CSSA) requires phi nodes be isolated by
parallel copies such that there is no interference between SSA values.
This is required for many out-of-SSA algorithms and, in our case, a
prerequisite for spilling.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:11 +00:00
d574d29102
nak: Add helpers to BasicBlock to get phis
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:11 +00:00
2f9565e725
nak: Add a dominance check to CFG
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:11 +00:00
bef6c1095e
nak: Add interference helpers
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:11 +00:00
0fd3510166
nak: Restructure liveness construction
...
Drop the for_block() stuff and instead have the parent add defs and uses
one at a time.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:11 +00:00
2c3eb239f8
nak: Allow DCE on functions
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:11 +00:00
002022a0b1
nak: Handle RegFile::Mem in parallel copy lowering
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:11 +00:00
4f05308ad9
nak: Add a Mem register file
...
For spilling, we want to be able to treat TLS as if it were a register
file. It unifies and makes everything easier. Also add support to
OpCopy to for copying between GPR and Mem. We cannot, however copy
directly from Mem to Mem.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:11 +00:00
fe7ac83a94
nak: Use more OpCopy
...
Instead of using OpMov or OpParCopy, use OpCopy directly. This reserves
OpParCopy for RA type things and OpMOv for actual codegen. We can also
drop OpMov from copy propagation now.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:11 +00:00
Faith Ekstrand