cam: Assert we have a reference when freeing sim

Before we decrement refcount to sleep on the sim, assert that the
refcount >= 1. If it were 0 here, we'd never wake up.