MFC 353369: Remove adapters from t4_list earlier during detach.

This ensures the clip task won't race with t4_destroy_clip_table.

While here, make some mutex destroys unconditional since attach always
initializes them.

Sponsored by: Chelsio Communications