arm64/memcpy.S: Make memcpy more compatible with device memory

[dan.kotowski_a9development.com]

Summary

For normal non-cacheable memory ACE supports 4x128 bit r/w WRAP
transfers or 1x128 bit r/w INCR transfers. By re-ordering the
stp's in memcpy / memmove we can accomodate this better without
impacting the existing code.

This fixes an issue seen on multiple Cortex-A72 SOCs when writing
directly to a PCIe memmapped frame-buffer, which resulted in
corruption.

Originally by Jon Nettleton <[e-mail]>

https://gist.github.com/jnettlet/80f8d09d01c0dc0ffc0122f36ed78de6

https://gist.github.com/jnettlet/f6f8b49bb7c731255c46f541f875f436

[andrew]

What is calling memcpy on the device memory? The kernel should be using bus_space to write to a device.

[dan.kotowski_a9development.com]

At least in Linux, the drm drivers create a fb0 device and things like mesa may talk directly to that via memcpy.

[andrew]

If it's for mesa this change is unlikely to help as this is a kernel only memcpy while, as far as I know, mesa is userspace only so will use one of the memcpy implementations from the Arm Optimized Routines in contrib/arm-optimized-routines/string/aarch64/mempy*

[dan.kotowski_a9development.com]

Added the same reorderings to sys/arm64/arm64/memcpy.S

[andrew]

Can you send the Arm Optimized Routine change upstream [1]. I'd prefer to not maintain a local patch that is likely to get clobbered when new releases are imported.

[1] https://github.com/ARM-software/optimized-routines