a10_ehci: Remove the passby code

It doesn't seems to be needed anymore and this make ehci working again
on the Pine64.
Thanks to jmcneill@ for the help.

Tested on: Pine64 (A64), OrangePi One (H3), BananapiM2 (A31s)