graphics/vulkan-loader: unbreak build on non-x86 archs after e99b707ea9df

CMake Error at loader/CMakeLists.txt:311 (target_compile_definitions):

Cannot specify compile definitions for target "asm_offset" which is not
built by this project.

This reverts commit e99b707ea9df7966f9f95e2871a053715318f00a.

PR: 259718
Reported by: mikael (aarch64)