ir3: Perform vectorization on ldg/stg just like other memory access.

shader-db effect comes from (perhaps older vkd3d?) DX12 fossils.  It sure
cleans up BDA CTS testcases, too.

Totals:
MaxWaves: 22151808 -> 22485222 (+1.51%); split: +1.51%, -0.06%
Instrs: 397754007 -> 396485642 (-0.32%); split: -0.41%, +0.10%
CodeSize: 990135656 -> 916286528 (-7.46%); split: -7.64%, +0.00%
STPs: 234296 -> 218861 (-6.59%); split: -6.62%, +0.03%
LDPs: 171664 -> 155386 (-9.48%); split: -9.53%, +0.04%

Totals from 130513 (8.36% of 1560788) affected shaders:
MaxWaves: 1376826 -> 1710240 (+24.22%); split: +24.30%, -0.90%
Instrs: 57789492 -> 56521127 (-2.19%); split: -2.84%, +0.66%
CodeSize: 256430400 -> 182581272 (-28.80%); split: -29.51%, +0.01%
NOPs: 7619148 -> 8015627 (+5.20%); split: -2.42%, +7.79%
MOVs: 4971200 -> 3968352 (-20.17%); split: -21.62%, +1.37%
COVs: 683166 -> 661427 (-3.18%); split: -3.24%, +0.04%
Full: 2422659 -> 1886993 (-22.11%); split: -22.98%, +0.03%
(ss): 1410724 -> 1479394 (+4.87%); split: -3.24%, +8.43%
(sy): 611391 -> 674588 (+10.34%); split: -1.14%, +11.66%
(ss)-stall: 4930898 -> 5265129 (+6.78%); split: -3.37%, +10.29%
(sy)-stall: 24138107 -> 29357000 (+21.62%); split: -1.82%, +23.71%
STPs: 112159 -> 96724 (-13.76%); split: -13.83%, +0.07%
LDPs: 113355 -> 97077 (-14.36%); split: -14.43%, +0.07%
Preamble Instrs: 83269073 -> 46493442 (-44.16%); split: -45.32%, +0.01%
Early Preamble: 79 -> 87 (+10.13%)
Cat0: 8516132 -> 8878083 (+4.25%); split: -2.28%, +6.68%
Cat1: 5994766 -> 4973985 (-17.03%); split: -19.02%, +1.94%
Cat2: 22297064 -> 22240177 (-0.26%); split: -0.30%, +0.05%
Cat3: 13748284 -> 13748279 (-0.00%)
Cat6: 1337948 -> 780266 (-41.68%); split: -42.19%, +0.27%
Cat7: 3247729 -> 3252768 (+0.16%); split: -1.48%, +1.71%
Subgroup size: 9280960 -> 9524224 (+2.62%); split: +2.63%, -0.01%

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38704>

src/freedreno/ir3/ir3_nir.c

@@ -723,7 +723,8 @@ ir3_finalize_nir(struct ir3_compiler *compiler,
     * descriptor, even when it doesn't.
     */
    nir_load_store_vectorize_options vectorize_opts = {
-      .modes = nir_var_mem_ubo | nir_var_mem_ssbo | nir_var_mem_shared | nir_var_uniform,
+      .modes = nir_var_mem_ubo | nir_var_mem_ssbo | nir_var_mem_shared |
+               nir_var_uniform | nir_var_mem_global,
       .callback = ir3_nir_should_vectorize_mem,
       .robust_modes = options->robust_modes,
       .cb_data = compiler,