Remove lib32 versions of libl.

Reviewed by: imp, emaste
Fixes: eb61f7bdf266 Stop building libl and liby
Sponsored by: The University of Cambridge, Google Inc.
Differential Revision: https://reviews.freebsd.org/D33851

(cherry picked from commit cc8af33eb340d5ee0c755fb3eb9dc07f0819afdc)