For pthread_mutex_trylock() call on owned error-check or non-portable
adaptive mutex, return EDEADLK as required by POSIX. The
pthread_mutex_lock() is already compliant.

Tested by: Guy Yur <guyyur@gmail.com>
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Approved by: re (gjb)