I just ran a test without the timeout in the loop waiting for a connection
attempt and the time it took for a failover was only 1minute more than
with that patch.

The problem is that, without the "timeout" it takes too long for the msleep() to wake up.
The "timeout" or whatever you prefer to call it is meant to set the upper bound on the
time that this loop waits for a TCP connection to complete to approximately
tv.tv_sec (the same timeout as set for SO_SNDTIMEO).

Added comments as kib@ suggested and used tvtohz(), so that tv_usec component is included in timeout.
(I have emailed kib@ one that uses the 1sec timeout and I put that here if he prefers that one.)

I commented inline. I'll admit I don't see spurious wakeup()s will be a problem.
I can (actually already have for the patch I have) include the tv_usec field in
the timeout calculation.
(I'll admit I get nervous fooling with things like time_uptime, which can go
negative in 2038 and assorted things related to overflow.)

Explained inline, but basically timeouts at both places count as one retry
and the RPC attempt fails when the limit on these retries is reached,
so it seems to me that they should be the same amount of time?

I added an inline comment which was basically that I didn't think a timeout of a fraction
of 1sec or setting a timeout of greater than 1sec resolution would be useful.
(My assumption is that normal operation against a slow server could result
in delays of 1sec.)