Page MenuHomeFreeBSD

merge projects/ipsec into head/
ClosedPublic

Authored by ae on Jan 26 2017, 4:11 PM.
Tags
None
Referenced Files
Unknown Object (File)
Fri, Nov 29, 12:59 PM
Unknown Object (File)
Wed, Nov 27, 11:44 PM
Unknown Object (File)
Tue, Nov 26, 5:16 PM
Unknown Object (File)
Tue, Nov 26, 1:30 AM
Unknown Object (File)
Tue, Nov 26, 1:30 AM
Unknown Object (File)
Tue, Nov 26, 1:30 AM
Unknown Object (File)
Tue, Nov 26, 1:30 AM
Unknown Object (File)
Tue, Nov 26, 1:30 AM

Details

Summary

A small summary


o Almost all IPsec releated code was moved into sys/netipsec.
o New kernel modules added: ipsec.ko and tcpmd5.ko. New kernel
  option IPSEC_SUPPORT added. It enables support for loading
  and unloading of ipsec.ko and tcpmd5.ko kernel modules.
o IPSEC_NAT_T option was removed. Now NAT-T support is enabled by
  default. The UDP_ENCAP_ESPINUDP_NON_IKE encapsulation type
  support was removed. Added TCP/UDP checksum handling for
  inbound packets that were decapsulated by transport mode SAs.
o New network pseudo interface if_ipsec(4) added. For now it is
  build as part of ipsec.ko module (or with IPSEC kernel).
  It implements IPsec virtual tunnels to create route-based VPNs.
o The network stack now invokes IPsec functions using special
  methods. The only one header file <netipsec/ipsec_support.h>
  should be included to declare all the needed things to work
  with IPsec.
o All IPsec protocols handlers (ESP/AH/IPCOMP protosw) were removed.
  Now this implemened via IPsec methods.
o TCP_SIGNATURE support is reworked to be more close to RFC.
o PF_KEY SADB was reworked:
  - now all security associations stored in the single SPI namespace,
    and all SAs MUST have unique SPI.
  - several hash tables added to speedup lookups in SADB.
  - SADB now uses rmlock to protect access, and concurrent threads
    can do SA lookups in the same time.
  - many PF_KEY messages handlers were reworked to reflect changes
    in SADB.
o ipsecrequest and secpolicy structures were cardinally changed to
  avoid locking protection for ipsecrequest. Also now we support
  only limited number (4) of bundled SAs, but they are supported
  for both INET and INET6.
o INPCB security policy cache was introduced. Each PCB now caches
  the used security policies to avoid SP lookup for each packet.
o For inbound security policies added the mode, when the kernel does
  check for full history of applied IPsec transforms.
o References counting rules for security policies and security
  associations were changed. Also proper SA locking added into xform
  code.
o xform code also was changed. Now it is possible to unregister xforms.
  tdb_xxx structures were changed and renamed to reflect changes in
  SADB/SPDB, and changed rules for locking and refcounting.
Test Plan

We use most of these changes in production at Yandex.
Also several people did some tests and I made some changes using feedback from them.
I have several scripts to do simple test with manual configuration. I'll add them a bit later.

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
ae updated this object.
ae edited the test plan for this revision. (Show Details)
ae set the repository for this revision to rS FreeBSD src repository - subversion.
ae edited edge metadata.

How did you create this diff? I'm wondering why we ended up with all of these directories added

Oh, @allanjude mentioned on IRC, SVN mergeinfo. Was this done through arc or by uploading the diff? (If the latter we'll want to update the wiki instructions)

Oh, @allanjude mentioned on IRC, SVN mergeinfo. Was this done through arc or by uploading the diff? (If the latter we'll want to update the wiki instructions)

This diff is the result of svn merge as described here https://svnweb.freebsd.org/base/projects/GUIDELINES.txt?revision=235275&view=markup
It was don by arc diff --preview. I'm looking for a way to create more readable diff, probably from git.

In D9352#193109, @ae wrote:

This diff is the result of svn merge as described here https://svnweb.freebsd.org/base/projects/GUIDELINES.txt?revision=235275&view=markup
It was don by arc diff --preview. I'm looking for a way to create more readable diff, probably from git.

In this case, it might be worth using: the --less-context flag in arc. normally we ask for the full context, but this change is very large.

Or maybe just generating the diff with svn like the above, but with: --ignore-properties --no-diff-deleted
To exclude mergeinfo and files that are being deleted entirely.

Some language fixes and .Dd bumps.

share/man/man4/ipsec.4
298 ↗(On Diff #24478)

I would rephrase this sentence to:
... policies is to make sure that .... (no comma, s/just/to/)

300 ↗(On Diff #24478)

This sentence sounds strange, maybe there is a word missing or at the wrong position?
Is this what you meant:

... value, each packet handled by IPsec will be checked ...

share/man/man4/tcp.4
37 ↗(On Diff #24478)

You need to bump this document date, too.

share/man/man4/udp.4
31 ↗(On Diff #24478)

This date needs to be bumped.

In this case, it might be worth using: the --less-context flag in arc. normally we ask for the full context, but this change is very large.

I'm not sure that would help - the problem isn't the context in the files that did change, it's all of the files/directories that are included but did not change (other than mergeinfo) that bloat the review.

This diff is the result of command

svn diff --ignore-properties --diff-cmd=diff -x -U999999 ^/head@312817 ^/projects/ipsec@312817
usr.bin/netstat/inet.c
861–873 ↗(On Diff #24478)

I think this can be committed independently right now?

share/man/man4/ipsec.4
300 ↗(On Diff #24478)

I try to describe what I mean:

if (check_policy_history != 0) { // <- this variable has non-zero value

IPsec has handled a packet. This packet must have history, i.e. list of operations that IPsec did.
Each entry in the list contains the following information: protocol, mode, SA address:
struct xform_history {
        union sockaddr_union    dst;            /* destination address */
        uint32_t                spi;            /* Security Parameters Index */
        uint8_t                 proto;          /* IPPROTO_ESP or IPPROTO_AH */
        uint8_t                 mode;           /* transport or tunnel */
};
Security policy has a list of operation, that administrator requires, e.g.
ipsec esp/tunnel/SRC-DST/require ah/transport//require;

Now we compare each entry in the history with the entries in the policy.
If they don't match, the kernel will drop packet.

} else {

Each packet has flags, that some sort of IPsec operation was done.
The kernel checks that some encryption was used and some AH SA was used. 
But the kernel doesn't check SA addresses.

}

Some tests for manual IPsec checking:


You need to have two hosts, configure host.conf on them, and run each ipsecN.sh script, then do what script suggests.

textproc/igor can be run on source and other text files, locating this and other potential problems like common misspellings. Try it with igor -R myfile.c | less -RS. Get yours today, while supplies last.

Also see https://www.freebsd.org/doc/en_US.ISO8859-1/books/fdp-primer/editor-config-nano.html to set up nano with syntax highlighting that identifies whitespace problems. It is not necessary to use it as an everyday editor, but can be useful for checking files.

share/man/man4/if_ipsec.4
36 ↗(On Diff #24481)

Needs an article: s/of/of the/

Also, igor says there is trailing whitespace on this line.

39 ↗(On Diff #24481)

s/the following/this/

45 ↗(On Diff #24481)
It can also be loaded as part of the
.Cm ipsec
kernel module if the kernel was compiled with
.Bd -ragged -offset indent
​.Cd "options IPSEC_SUPPORT"
​.Ed
55 ↗(On Diff #24481)

IPv[46] looks ugly. What it means is clear if the reader expects filename-type wildcards, but is it normally stated this way?

Also, ESP is not defined.

Is this clearer?

It can tunnel IPv4 and IPv6 traffic over either IPv4 or IPv6 and secure it with ESP.
64 ↗(On Diff #24481)

s/needs to/must/

66 ↗(On Diff #24481)

s/endpoints/endpoint/

68 ↗(On Diff #24481)

s/also can/can also/

75 ↗(On Diff #24481)

Needs an article: s/When/When the/

Also, igor says there is trailing whitespace on this line.

77 ↗(On Diff #24481)

Break this sentence here:

interface is configured, it automatically creates special security policies.
These policies can be used to acquire security associations from the IKE daemon,
which are needed for establishing an IPsec tunnel.
80 ↗(On Diff #24481)
It is also possible to create needed security associations manually with the
86 ↗(On Diff #24481)

Needs an article: s/has/has an/

90 ↗(On Diff #24481)

s/used/is used/

95 ↗(On Diff #24481)

s/creating/creation/

96 ↗(On Diff #24481)

s/it is//

97 ↗(On Diff #24481)

s/of//

100 ↗(On Diff #24481)

s/of//

102 ↗(On Diff #24481)
The example below shows manual configuration of an IPsec tunnel
103 ↗(On Diff #24481)

s/Assuming host/Host/

129 ↗(On Diff #24481)

Should be used? Or really *must* be used?

Also, igor says there is trailing whitespace on this line.

share/man/man4/ipsec.4
155 ↗(On Diff #24481)

s/from/on/

158 ↗(On Diff #24481)

s/The/A/

Some would say "properly-formed" should have a hyphen.

161 ↗(On Diff #24481)

Needs article: s/for/for the/

287 ↗(On Diff #24481)

s/This variable controls/Control/

289 ↗(On Diff #24481)

Needs article: s/non-zero/a non-zero/

Also, igor says there is trailing whitespace on this line.

291 ↗(On Diff #24481)

igor says there is trailing whitespace on this line.

293 ↗(On Diff #24481)

s/will incrementally recompute/incrementally recomputes/

295 ↗(On Diff #24481)

Avoid contractions: s/weren't/were not/

Passive->active: s/will be/are/

297 ↗(On Diff #24481)

s/This variable enables/Enables/

300 ↗(On Diff #24478)

Hmm. How about:

Default inbound security policies check that packets handled by IPsec
have been decrypted and authenticated.
If this variable is set to a non-zero value, each packet handled by IPsec
is checked against the history of IPsec security associations.
The IPsec security protocol, mode, and SA addresses must match.
share/man/man4/tcp.4
276 ↗(On Diff #24481)

Not sure about this, but maybe

When this option is enabled on a socket, all outgoing TCP segments
278 ↗(On Diff #24481)

s/also must/must also/

294 ↗(On Diff #24481)

Passive->active:
s/will not/does not/
s/will drop/drops/

297 ↗(On Diff #24481)

Passive->active: s/will be/is/

share/man/man4/udp.4
109 ↗(On Diff #24481)
Only one value is supported for this option:
111 ↗(On Diff #24481)

s/3948/3948,/

sys/conf/NOTES
633 ↗(On Diff #24481)

Avoid contractions: s/doesn't/does not/

634 ↗(On Diff #24481)

Needs article: s/as/as a/

1032 ↗(On Diff #24481)

s/one of/either/

sys/conf/files
603 ↗(On Diff #24481)

Trailing whitespace on this line.

sys/net/if_ipsec.c
882 ↗(On Diff #24481)

s/can not/cannot/

sys/netipsec/ipsec.c
206 ↗(On Diff #24481)
"If set, filter packets from an IPsec tunnel.");
sys/netipsec/ipsec.h
115 ↗(On Diff #24481)
By default, policies are set to NULL. This means that they have ENTRUST type.
116 ↗(On Diff #24481)

s/flags/the flags/

117 ↗(On Diff #24481)

s/also/is also/
s/In case/When/

118 ↗(On Diff #24481)

s/speedup/speed up/

Fixed panic in r313046.
And applied requested changes to manual pages.

ae marked 47 inline comments as done.Feb 3 2017, 1:28 AM

Consider all comments and suggestions.

usr.bin/netstat/inet.c
861–873 ↗(On Diff #24478)

I prefer to keep commit all changes together to do not introduce merge conflicts.

share/man/man4/if_ipsec.4
131 ↗(On Diff #24671)

s/values/value/ (I think).

86 ↗(On Diff #24481)

Still needs an article: s/has/has an/

96 ↗(On Diff #24481)

s/will be/is/

share/man/man4/tcp.4
278 ↗(On Diff #24481)

Looking at this sentence, can it just be:

When this option is enabled on a socket, all inbound and outgoing
TCP segments must be signed with MD5 digests.

So, does anyone want to do a review for the code?

gnn edited edge metadata.
gnn added inline comments.
sys/net/if_ipsec.c
87 ↗(On Diff #24671)

I am confused by the statically defined arrays for the security policies passed in this code. Can you explain why you chose to do it this way?

213 ↗(On Diff #24671)

Why not define a macro, like the other IPSEC_XX_LOCK() macros, for this as well.

799 ↗(On Diff #24671)

Please change the magic number to a #define.

sys/netipsec/ipsec.c
482 ↗(On Diff #24671)

What happened to this statistic?

This revision is now accepted and ready to land.Feb 5 2017, 6:01 PM
sys/net/if_ipsec.c
87 ↗(On Diff #24671)

Each if_ipsec(4) interface has own security policies that should match all traffic.
Due to the constrains in the code that does policy comparison, we need to explicitly specify the address family for a policy. This is similar to setkey's:

spdadd -6 ...  -P out ...
spdadd -6 ... -P in ...
spdadd -4 ... -P out ...
spdadd -4 ... -P in ...

To match both IPv4 and IPv6 we need 4 policies (inbound and outbound).

213 ↗(On Diff #24671)

There are only a few uses of this lock, I used it without macro like in gif(4) and gre(4). From my point of view, the reason why we use macro instead of direct call for locking is ability to change the lock definition and avoid changing in all places of the code. E.g. change mutex to rwlock, or rwlock to rmlock. In this case I don't see suitable candidates for migration.

sys/netipsec/ipsec.c
482 ↗(On Diff #24671)

ipsec_checkpolicy() is generic function for both IPv4 and IPv6. Since we have separate statistics for INET and INET6 ipsec, I moved statistic accounting in AF-specific functions, i.e. ipsec4_checkpolicy() and ipsec6_checkpolicy().

This revision was automatically updated to reflect the committed changes.