Fix pjfstest issue tests/rename/19.t

The rename call with args like:
"./dir0/dir1/.." "./dir2" will cause MPASS failure.
The tmpfs_dir_lookup() does not accept names like
'.' and '..' for lookup. Move the '.' and '..' entry
check before tmpfs_dir_lookup() call.

Reviewed by: kib
MFC after: 2 week
Differential revision: https://reviews.freebsd.org/D38051