support for DIRECT access via ggated and ggatec

[david_crossfamilyweb.com]

Summary

Support DIRECT access via ggated and ggatec.

ggatec supports specifying -o direct in addition to the typical rw/wo/ro access specifier to request DIRECT access from ggated.

ggated supports adding ,DIRECT and ,NODIRECT on export lines. Specfying DIRECT will force any accesses to have DIRECT set, setting NODIRECT will preclude any a client from choosing DIRECT. Not specificed allows the client to request (or not)

Test Plan

ggated exports of:
host RO /path/ggated/ro
host RW /path/ggated/rw
host WO /path/ggated/wo
host RO,DIRECT /path/ggated/rod
host RW,DIRECT /path/ggated/rwd
host WO,DIRECT /path/ggated/wod
host RO,NODIRECT /path/ggated/rond
host RW,NODIRECT /path/ggated/rwnd
host WO,NODIRECT /path/ggated/wond

Client tested with:
./ggatec create -o ro server /path/ggated/ro
./ggatec create -o rw server /path/ggated/ro
./ggatec create -o wo server /path/ggated/ro

./ggatec create -o ro -o direct server /path/ggated/ro
./ggatec create -o wo -o direct server /path/ggated/ro
./ggatec create -o wo -o direct server /path/ggated/ro

./ggatec create -o ro server /path/ggated/wo
./ggatec create -o rw server /path/ggated/wo
./ggatec create -o wo server /path/ggated/wo

./ggatec create -o ro -o direct server /path/ggated/wo
./ggatec create -o rw -o direct server /path/ggated/wo
./ggatec create -o wo -o direct server /path/ggated/wo

./ggatec create -o ro server /path/ggated/rw
./ggatec create -o wo server /path/ggated/rw
./ggatec create -o rw server /path/ggated/rw

./ggatec create -o ro -o direct server /path/ggated/rw
./ggatec create -o wo -o direct server /path/ggated/rw
./ggatec create -o rw -o direct server /path/ggated/rw

./ggatec create -o ro server /path/ggated/rod
./ggatec create -o wo server /path/ggated/rod
./ggatec create -o rw server /path/ggated/rod

./ggatec create -o ro -o direct server /path/ggated/rod
./ggatec create -o wo -o direct server /path/ggated/rod
./ggatec create -o rw -o direct server /path/ggated/rod

./ggatec create -o ro server /path/ggated/wod
./ggatec create -o wo server /path/ggated/wod
./ggatec create -o rw server /path/ggated/wod

./ggatec create -o ro -o direct server /path/ggated/wod
./ggatec create -o wo -o direct server /path/ggated/wod
./ggatec create -o rw -o direct server /path/ggated/wod

./ggatec create -o ro server /path/ggated/rwd
./ggatec create -o wo server /path/ggated/rwd
./ggatec create -o rw server /path/ggated/rwd

./ggatec create -o ro -o direct server /path/ggated/rwd
./ggatec create -o wo -o direct server /path/ggated/rwd
./ggatec create -o rw -o direct server /path/ggated/rwd

./ggatec create -o ro server /path/ggated/rond
./ggatec create -o wo server /path/ggated/rond
./ggatec create -o rw server /path/ggated/rond

./ggatec create -o ro -o direct server /path/ggated/rond
./ggatec create -o wo -o direct server /path/ggated/rond
./ggatec create -o rw -o direct server /path/ggated/rond

./ggatec create -o ro server /path/ggated/wond
./ggatec create -o wo server /path/ggated/wond
./ggatec create -o rw server /path/ggated/wond

./ggatec create -o ro -o direct server /path/ggated/wond
./ggatec create -o wo -o direct server /path/ggated/wond
./ggatec create -o rw -o direct server /path/ggated/wond

./ggatec create -o ro server /path/ggated/rwnd
./ggatec create -o wo server /path/ggated/rwnd
./ggatec create -o rw server /path/ggated/rwnd

./ggatec create -o ro -o direct server /path/ggated/rwnd
./ggatec create -o wo -o direct server /path/ggated/rwnd
./ggatec create -o rw -o direct server /path/ggated/rwnd

Validated via a (removed) debugging logline that showed the open(2) flags. All flags passed correctly.

[cy]

Can you please regenerate a full patch using -U99999?

[david_crossfamilyweb.com]

Updated for -U99999 as requested.

Additionally:

• fixed manpage for ggatec, removed an old way I had for specifying direct access that I had already removed from code but missed in the documentation
• updated ggatel to also support O_DIRECT.

*WARNING* I haven't tested AT ALL (not even compiled) ggatel with these changes (but they were just imports from ggatec).. I will be doing that now, but I was hoping to kickstart the review process while I can test those

[zlei]

sbin/ggate/ggatec/ggatec.c

81–82

IIUC, the original options <ro|wo|rw> are mutually exclusive, but they can be combined with direct, then it seems [-o <ro|wo|rw, direct>] or [-o <ro|wo|rw]> [-o <direct>] is better than option = {ro, wo, rw, direct}.

547

This blank is intentional as per style(9) .

sbin/ggate/ggated/ggated.8

92

Shall client have the option [-o nodirect] ?

[david_crossfamilyweb.com]

sbin/ggate/ggated/ggated.8

92

The intent here is no, DIRECT has implications on the server. In my case I want to avoid these images thrashing the cache on my server from accesses, but after I did that I noticed that it had a profound impact (predictably) on the rest of the IO on that device and I realized that the operator may want to be able to have say over how this impacts their server.

If the server operator does not care, they can allow the client to pick DIRECT or not (default is not)

If the server operator DOES care they can choose to enforce DIRECT or NODIRECT at their discretion,

If they've chosen to pick DIRECT then the client should be forced to DIRECT, if they pick NODIRECT then they should be forced off.

If the client doesn't want direct, it should just not specify; but what the server specifies rules.

I chose NOT to outright reject the request since from the client's perspective it doesn't change the contract of the data operations any, it is just cache optimization on the server.

[david_crossfamilyweb.com]

Updated with PR comments