Otherwise we end up removing refcount even when we don't have a color
surface already for applications going from SRGB_NONLINEAR to
PASS_THROUGH dring runtime.
To reproduce the bug, start mpv with "--target-colorspace-hint=yes" then
set it to "no" during runtime with a keybind
Fixes: 789507c99c ("vulkan/wsi: implement the Wayland color management protocol")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34708>
llyyr
033ce1bae1