btree/bt_seq.c: Fix two NULL pointer dereferences

This change fixes two NULL pointer dereferences caused by the
__bt_first function.

The first was caused by returning 0 (i.e., RET_SUCCESS) when a key
was not found, causing the caller to dereference an uninitalized
or NULL pointer. The second one was caused by an if statment clobbering
a local variable with a function call result that might be NULL.

Reported by: clang-tidy
Sponsored by: Klara, Inc.
Reviewed by: markj
Obtained from: https://github.com/apple-oss-distributions/libc (partially)
Differential Revision: https://reviews.freebsd.org/D54905