Implement 64MB memory limit for test to ensure that it fails reliably in
600 seconds; it would previously fail inconsistently when run in some virtual
machine configurations

This patch might need to be reverted or revisited later (see the attached PR
for more details)

PR: 169302

Submitted by: pho
Sponsored by: EMC / Isilon Storage Division