In retrospect I don't know why i didn't do this in the first place. Rather than putting a slab in vtoslab() just so we can figure out what size the thing was, stuff the size and a special bit in the 'slab' pointer in the page. We can drop the slab allocation entirely.
I can do something similar for large zone sizes later.