This thing is entirely opt-in wrt caring about it when writing to
a file anyway. Since we also lock the two at the same time and they
have an 1-1 relation we can just lock one of the two files. Saves
some syscalls.
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11485>
This avoids all locks for reads and using lock only while actually
writing.
This is enabled by doing two things:
1) Reading the index incrementally. This way we get new entries
written by other processes and do not write duplicate entries.
2) Taking the lock only during writes, and applying the incremental
read while holding the lock so we always append to the actual end of the file.
Fixes: eca6bb9540 ("util/fossilize_db: add basic fossilize db util to read/write shader caches")
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11485>
Often disassemblers and things in our drivers want to be able to
incrementally printf together a line, but that gets in the way of
Android's logging that wants to see a whole line all at once. Make a
little wrapper to do the ralloc_asprintf_rewrite_tail() and flushing lines
as they appear.
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9262>
Instead of spawning 4 threads when the cache is created,
spawn 1 and let u_queue grow the number of threads if
needed.
I wrote this patch because when running piglit's quick_shader
profile I had lots of samples in disk cache threads - mostly
in native_queued_spin_lock_slowpath kernel function.
Since these tests shouldn't really stress the cache, I assumed
it was caused only by thread creations.
After writing the patch and redoing the measurement, I got an
improvement but I still more hits in the same function for
shader_runner:$disk0 thread so something was wrong.
After digging more, I found out that my shader cache index was
corrupted: the on-disk size was 29MB but the index reported it
was way more than 1GB. So each disk cache thread was spending
a lot of time trying to evict files. Given that my cache had
a really low count of files, the LRU method based on randomly
generating subfolder names failed, so evicting was very slow.
Now that my cache index is fixed, the disk cache threads are
mostly idle but I still think it makes sense to grow the
number of threads instead of spawning 4 at the program start.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11296>
This flag allow to create a single thread initially, but set
max_thread to the request thread count.
If the queue is full and num_threads is lower than max_threads,
we spawn a new thread to help process the queue faster.
This avoid creating N threads at queue creation time.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11296>
This prevents problems when some CPUs are offline. In a four CPU
system, if CPUs 1 and 2 are offline, the cache topology code would
only examine CPUs 0 and 1... giving incorrect information.
The types are changed to int16_t so that the offset of num_L3_caches
does not change. This triggered a STATIC_ASSERT failure:
STATIC_ASSERT(offsetof(struct util_cpu_caps_t, num_L3_caches) == 5 * sizeof(uint32_t));
I'm assuming there's some assembly code or something that depends on
this offset, and I don't feel like messing with it.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>
In the current code, this prevents a very unlikely corner case. More
importantly, it should prevent the next commit from breaking the
universe.
Imagine a system with 64 CPUs configured, but first 32 CPUs are offline.
_SC_NPROCESSORS_CONF will return 32. All of the surrounding code will
interpret this as meaning CPUs 0 through 31, but all of those CPUs are
offline. Nothing good can happen then.
The problem cases require systems with more than 32 CPUs because
util_cpu_caps.num_cpu_mask_bits is always rounded up to a multiple of
32.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>
Debugging fd mix-ups (ie. where, possibly via close()ing the original
fd, etc, you end up with something that is a valid fd but not a valid
*fence* fd) can be difficult. Fortunately we can use the FILE_INFO
ioctl, which will return an error if the fd is not a fence fd.
For android, we instead use the libsync API, which does a similar thing
on modern kernels, but has a fallback path for older android kernels.
Note that the FILE_INFO ioctl has existed upstream since at least prior
to destaging of sync_file.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11202>
Now that we have an idea of how many regs the conflicting allocation uses,
we can just skip to the next one and save repeated tests to find the same
conflicting neighbor again.
shadowrun-returns shader-db time on skl -1.62821% +/- 1.58079% (n=679),
now there's no statistically significant change from the start of the series
(n=420)
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9437>
In the fully general case of register classes, to expose an allocation
class of unaligned 2-contiguous-regs allocations, for example, you'd have
your base individual regs (128 on intel), and another set of 127 regs that
each conflicted with the corresponding pair of the base regs. Single-reg
nodes would allocate in the 128, and double-reg nodes would allocate in
the 127 and the user would remap from the 127 down to the base regs with
some irritating table.
If you need many different contiguous allocation sizes (16 is a pretty
common number across drivers), your number of regs explodes, wasting
memory and making the q computation expensive at startup.
If all the user has is contiguous-reg classes, we can easily compute the q
value up front (as found in the intel driver and nouveau, for example),
and we only have to change a couple of places in the conflict-checking
logic so the contiguous-reg classes can use the base registers.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9437>
It notably didn't fit the pattern of RGB5_A1_UNORM, and violated the
general pattern for bitmask format BE channels (channels are ordered
right-to-left in the BE columns in the CSV due to the parser walking them
in that order for historical reasons).
Acked-by: Adam Jackson <ajax@redhat.com>
Acked-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10505>
These tests passed for LE, and the BE channel ordering specified obviously
didn't fit the pattern of the other BE formats (channels are listed
right-to-left in the BE columns for historical reasons).
Note that we can't write pure-integer format tests in u_format_tests.c
currently.
Acked-by: Adam Jackson <ajax@redhat.com>
Acked-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10505>
Android 29 introduced general-dynamic TLS variable support ("quick
function call to look up the location of the dynamically allocated
storage"), while Mesa on normal Linux has used initial-exec ("use some of
the startup-time fixed slots, hope for the best!"). Both would be better
options than falling all the way back to pthread_getspecific(), which is
the alternative we have pre-29.
Reviewed-by: Roman Stratiienko <r.stratiienko@gmail.com>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10389>
