Page MenuHomeFreeBSD

grep: periodic timer-based fflush instead of unconditional per-line flush
ClosedPublic

Authored by bapt on Sat, Jun 13, 6:28 PM.

Details

Summary

Replace the unconditional fflush(stdout) in grep_printline and
procmatches with a periodic timer that flushes at most once every
100ms. This preserves interactive responsiveness (grep | tee,
grep | tail -f) while avoiding 1M+ write(2) syscalls when
processing large inputs.

The flush interval is tracked via clock_gettime(CLOCK_MONOTONIC)
and a static timespec. --line-buffered continues to flush
immediately via setlinebuf(3), as before.

Benchmark on 1M lines (37MB output to file):

unconditional fflush:  1.90s  (sys 1.22s)
periodic 100ms timer:   0.49s  (sys 0.007s)

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 73864
Build 70747: arc lint + arc unit

Event Timeline

bapt requested review of this revision.Sat, Jun 13, 6:28 PM
usr.bin/grep/util.c
810

This specifically seems to only be relevant to the last caller below- the other one is after an \n output that will have already flushed if it's line buffered. I'd probably just move this branch down there and call this from grep_printline as well.

fix missed initialization of printline_last_flush

simplify per @kevans comment

bapt marked an inline comment as done.Sun, Jun 14, 7:21 AM

I don't see either of these comments as blockers, fine with this or any change in between going in. Thanks!

usr.bin/grep/util.c
778

Looking at what we ended up with, it'd be worth an early return if lbfflag && fileeol == '\n'. In that case, any progress has been flushed and we can save the extra clock_gettime() call for the caller on line 904 below. On FreeBSD, this likely wouldn't save much because clock_gettime() takes advantage of the VDSO to avoid the syscall overhead, but it's nice for the portability aspect.

794

I was going to note that this could be lbflag && fileeol != '\n', but fflush in the \n case is going to just be a littlle bit of uncontended FILE locking as putchar() just flushed it and that's probably not going to show up on any kind of benchmarking as significant.

This revision is now accepted and ready to land.Sun, Jun 14, 2:09 PM