db/hash.c: Do not return an error when opening a missing database with O_CREAT

dbm_open currently returns an error when opening a missing database
with O_CREAT but creates the request database file. This is caused
by a buggy check in __hash_open which attempts to detect a new
(or empty) database file, but fails to take the O_CREAT flag into
account.

Fix this by expanding the check to include O_CREAT.

Fixes: edcdc752ecdd
Sponsored by: Klara, Inc.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D51491