Page MenuHomeFreeBSD

rm(1): Formalize non-functional status of -P flag

Authored by cem on Nov 8 2018, 9:22 AM.



-P was introduced in 4.4BSD-Lite2 around 1994. It overwrote file contents
with a pass of 0xff, 0x00, then 0xff, in a low effort attempt to "really
delete" files.

It has no user-visible effect; at the end of the day, the file is unlinked via
the filesystem. Furthermore, the utility of overwriting files with patterned
data is extremely limited due to caveats at every layer of the stack[0] and
therefore mostly futile. At the least, three passes is likely wasteful on
modern hardware[1]. It could also be seen as a violation of the "Unix
Philosophy" to do one thing per tiny, composable program.

Since 1994, FreeBSD has left it alone; OpenBSD replaced it with a single
pass of arc4random(3) output in 2012[2]; and NetBSD implemented partial, but
explicitly incomplete support for U.S. DoD 5220.22-M, "National Industrial
Security Program Operating Manual" in 2004[3].

NetBSD's enhanced comment above rm_overwrite makes a strong case for removing
the flag entirely:

This is an expensive way to keep people from recovering files from your
non-snapshotted FFS filesystems using fsdb(8). Really. No more.
It is impossible to actually conform to the exact procedure given in
[NISPOM] if one is overwriting a file, not an entire disk, because the
procedure requires examination and comparison of the disk's defect lists.
Any program that claims to securely erase *files* while conforming to the
standard, then, is not correct.
Furthermore, the presence of track caches, disk and controller write
caches, and so forth make it extremely difficult to ensure that data have
actually been written to the disk, particularly when one tries to repeatedly
overwrite the same sectors in quick succession. We call fsync(), but
controllers with nonvolatile cache, as well as IDE disks that just plain lie
about the stable storage of data, will defeat this.
[NISPOM] requires physical media destruction, rather than any technique of
the sort attempted here, for secret data.

As a first step towards evental removal, make it a placebo. It's not like
it was serving any security function with more code.

If you are security conscious and need to erase your disks, use a woodchipper.

If you are cheap and your secrets aren't worth spending any money on, use a
single iteration of gshred (part of GNU coreutils) (or just dd if=/dev/zero).


Test Plan

No problem with POSIX -- only -ifR are required.

Diff Detail

rS FreeBSD src repository
Automatic diff as part of commit; lint not applicable.
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

cem created this revision.Nov 8 2018, 9:22 AM

LGTM although I wonder about adding a warning if it's used.

This revision is now accepted and ready to land.Nov 8 2018, 9:30 AM
markj accepted this revision.Nov 8 2018, 5:45 PM
markj added inline comments.
235 ↗(On Diff #50167)

I'd put this under COMPATIBILITY.

130 ↗(On Diff #50167)

It'd probably be worth printing a warning to stderr if this option is specified.

cem marked 2 inline comments as done.Nov 8 2018, 6:57 PM
cem added inline comments.
235 ↗(On Diff #50167)

Oh sure, that seems reasonable. Will do.

130 ↗(On Diff #50167)

(@Daniel as well)

Hm, my concern with printing any additional warning is that it is a functional (observable) change in behavior as far as scripting is concerned. Would something like a syslog message be a good compromise, or do you think that's a worse idea?

Is there prior art in this space? I know some commands just ignore obsolete options and don't print anything. Do we have a good example of the warning case? Thanks.

cem updated this revision to Diff 50187.Nov 8 2018, 11:49 PM
cem marked an inline comment as done.

Move compatibility note to COMPATIBILITY section, per markj

This revision now requires review to proceed.Nov 8 2018, 11:49 PM
cem marked an inline comment as done.Nov 8 2018, 11:57 PM
cem added inline comments.
130 ↗(On Diff #50167)

Some examples:

xz(1): --fast/--best "provided only for backwards compatibility. Avoid using these options." No warning printed when used.

look(1): "The -a and ... flags are ignored for compatibility." No warning is printed when used.

as(1): "-D Ignored. This option is accepted for ... compatibility." No warning is printed.

diff(1): "--speed-large-files: Stub option for compatibility with GNU diff". No warning is printed.

I'm leaning towards no warning. Are there some good counter examples? Maybe UPDATING / relnotes: yes?

markj accepted this revision.Nov 9 2018, 12:05 AM
markj added inline comments.
130 ↗(On Diff #50167)

Fair enough. I tried to think of some examples and couldn't come up with any. You're right that it'd probably break scripts.

This revision is now accepted and ready to land.Nov 9 2018, 12:05 AM
This revision was automatically updated to reflect the committed changes.

This is a massive POLA violation. What the hell. You took a flag that was intended to make <some effort> to overwrite data and made it SILENTLY DO NOTHING. That's insane.

  • Not all files are written to more than once; the file may have been written once and overwriting its blocks may actually be sufficient.
  • Not all filesystems are ZFS
  • Not all users have root to wipe a disk
  • Not all users want or need to WIPE AN ENTIRE DISK to wipe 1 file
  • Not all files warrant a full cleanse; a half hearted effort is still better than nothing.
  • Not all users have the quota to fill the disk with dd
  • Not all filesystems have snapshots
  • You mentioned what OpenBSD and NetBSD did (to keep it) but ended with basically "fuck it let's remove it".

Yes the multi-pass was pointless. But 1 pass isn't entirely pointless. In the end this flag was _better than nothing_ and now it's a silent lie.

This flag was not advertised as some DoD or NISPOM standard-abiding feature. The note about it not being a security feature was added in 2010 with explanations on caveats.

While FreeBSD is about backwards-compatibility and POLA, this silent change may actually expose data for people where unexpected. That failure case is worth actually making this PRINT SOMETHING or ERROR with an invalid flag. Or better yet just be left alone. As I mentioned the case of a once-written file would be perfectly fine with this feature no? Ok sure there's the cache issues but that's more limited in its capacity but NOW THERES THE MAIN DATA BLOCKS ISSUES which is far greater in its exposure capacity.