Its not clear that this change is necessary. Why is it necessary?

This function should return true or false, not 0 or 2.

Have you seen that this change causes some compiler to generate better assembly language for this function? Could you show us that?

vm_page_in_laundry() would otherwise return true if m->queue == PQ_NONE.

We're testing queue == 2 || queue == 3, which can be optimized to (queue | 1) == 3. I don't think this kind of micro-optimization is really worth changing the definition of PQ_NONE.

Copy the below adjusted code to https://godbolt.org/ and see the assembly code.
Unfortunately, clang isn't yet supported and gcc 9.2 didn't like bool nor inline; but it is 4 + 7 = 11 compare to 4 instructions.

#define PQ_NONE 255
#define PQ_INACTIVE 0
#define PQ_ACTIVE 1
#define PQ_LAUNDRY 2
#define PQ_UNSWAPPABLE 3
#define PQ_COUNT 4

struct vm_page {

char queue;

};

typedef struct vm_page *vm_page_t;

static inline char
vm_page_queue(vm_page_t m)
{
return (m->queue);
}

static char
vm_page_in_laundry(vm_page_t m)
{
char queue;

queue = vm_page_queue(m);
return (queue == PQ_LAUNDRY || queue == PQ_UNSWAPPABLE);
}

#undef PQ_NONE
#define PQ_NONE (255-0x2)

static char vm_page_in_laundry2(vm_page_t m)
{

return (vm_page_queue(m) & PQ_LAUNDRY);
}