MFC libnv to 10-STABLE

[kbowling]

Summary

libnv is needed to MFC SR-IOV and possibly other interesting things.

Test Plan

1. buildworld WITH_TESTS=yes
2. kyua test -k /usr/tests/lib/libnv/Kyuafile
3. Test with D4232

[oshogbo]

I don't plan any more work on libnv.
So we can integrate it, will you do it or should I do it?

[kbowling]

This brings back some work that happened in r279437, r284107, r286796 to make things coexist with opensolaris compat's nvpair.

[kbowling]

@oshogbo I am not a committer, if it includes all the necessary changes and does not break -STABLE can you integrate?

[kbowling]

lib/libnv/Makefile needs s/src.opts.mk/bsd.own.mk/

[kbowling]

Fixed libnv makefile to use older option handling. This is ready to go.

[kbowling]

Two fixes to the lib/libnv/tests/Makefile to undo some mk changes from -CURRENT.. this now installs the tests correctly and they pass in Kyua.

[kbowling]

ping

[oshogbo]

WIP. Plis give me a little more time :)

[oshogbo]

There is only this small change :)
I build 10 with your path.
It's look good for me.

If nobody will have any objections I will commit it.

nvpair.h

26

This file should be in sys/cddl/compat/opensolaris/sys/nvpair.h not in nvpair.h.

[kbowling]

[sbruno]

[kbowling]

Will you commit it? I'd like to start poking people on SR-IOV so it makes the 10.3 cycle.

[ngie]

Kevin: please give me the list of commits that need to be MFCed and I'll take care of it.

Thanks!

[kbowling]

Hi Ngie, this diff has been tested and I would prefer it applied directly to -STABLE (there are changes vs HEAD). The patch file needs one tweak, /nvpair.h should be moved to sys/cddl/compat/opensolaris/sys/nvpair.h before commit.

[ngie]

In D4249#93840, @kevin.bowling_kev009.com wrote:

Hi Ngie, this diff has been tested and I would prefer it applied directly to -STABLE (there are changes vs HEAD). The patch file needs one tweak, /nvpair.h should be moved to sys/cddl/compat/opensolaris/sys/nvpair.h before commit.

Hi Kevin,

The reason for the commits that need MFCing is that I want to make sure the SVN metadata is correct/in sync. Unfortunately that information isn't present in your diff, so tracking the changes (and the reasons for them) would get extremely hairy, i.e. svn log won't be correct :(.

I can use this patch as a basis though. If everything's done right, then the patches will line up.

Thanks!
-NGie

[ngie]

I'm almost done with compiling the list of items that need to be MFCed:

$ svn diff ^/stable/10 ^/user/ngie/stable-10-libnv | diffstat
 contrib/mdocml/lib.in                   |    1 
 etc/mtree/BSD.tests.dist                |    3 
 lib/Makefile                            |    1 
 lib/libnv/Makefile                      |   92 +
 lib/libnv/common_impl.h                 |   37 
 lib/libnv/msgio.c                       |  469 ++++++++++
 lib/libnv/msgio.h                       |   50 +
 lib/libnv/nv.3                          |  653 ++++++++++++++
 lib/libnv/tests/Makefile                |   21 
 lib/libnv/tests/dnv_tests.cc            |  567 ++++++++++++
 lib/libnv/tests/nv_tests.cc             | 1240 ++++++++++++++++++++++++++
 lib/libnv/tests/nvlist_add_test.c       |  196 ++++
 lib/libnv/tests/nvlist_exists_test.c    |  321 ++++++
 lib/libnv/tests/nvlist_free_test.c      |  221 ++++
 lib/libnv/tests/nvlist_get_test.c       |  182 +++
 lib/libnv/tests/nvlist_move_test.c      |  161 +++
 lib/libnv/tests/nvlist_send_recv_test.c |  342 +++++++
 share/mk/bsd.libnames.mk                |    1 
 sys/conf/files                          |    3 
 sys/kern/subr_dnvlist.c                 |  128 ++
 sys/kern/subr_nvlist.c                  | 1475 ++++++++++++++++++++++++++++++++
 sys/kern/subr_nvpair.c                  | 1111 ++++++++++++++++++++++++
 sys/sys/dnv.h                           |  116 ++
 sys/sys/nv.h                            |  200 ++++
 sys/sys/nv_impl.h                       |  131 ++
 sys/sys/nvlist_impl.h                   |   49 +
 sys/sys/nvpair_impl.h                   |   94 ++
 27 files changed, 7864 insertions(+), 1 deletion(-)

It compiles on amd64 at least; need to handle some more bits to get parity with the diff you've provided here.

Here are the revisions I've merged thus far:

r258065 r258594 r259430 r260222 r261407 r261408 r263479 r264021 r266351 r269603 r271026 r271027 r271028 r271241 r271578 r271579 r271847 r272102 r272843 r273752 r277920 r277921 r277925 r277926 r277927 r279421 r279422 r279423 r279424 r279425 r279426 r279427 r279428 r279429 r279430 r279431 r279432 r279434 r279435 r279436 r279438 r279439 r279440 r279760 r282254 r282257 r282304 r282312

I haven't MFCed these items because it was starting to get hairy with pci(4), but there might be some items here that I can easily cherrypick:

r282346 r282347 r282348 r285063 r285129 r285139 r286796

[ngie]

Makefile.inc1

1655

This isn't needed because capsicum isn't being backported. I'm skipping this part of the diff.

lib/libnv/Makefile.depend

1

This isn't needed because meta-mode's not being backported.

nvpair.h

26

I'm not touching this file. This isn't present in either ^/head or ^/stable/10

share/man/man9/Makefile

191

TODO: still need to work out nv(9)

sys/conf/files

479

TODO: this needs to be handled as part of the nv(9) import.

[kbowling]

lib/libnv/Makefile.depend

1

sounds good

nvpair.h

26

It started from r279437 and allows for things to coexist with cddl nvpair. Looks like r284107 r286796 both touch it after the move.

[ngie]

In D4249#98955, @ngie wrote:

I'm almost done with compiling the list of items that need to be MFCed:

...

I haven't MFCed these items because it was starting to get hairy with pci(4), but there might be some items here that I can easily cherrypick:

r282346 r282347 r282348 r285063 r285129 r285139 r286796

These revisions all conflict. Once iovctl and the pci(4) bits have been merged in, the rest of the revisions should fall like dominoes. I'll double-check, but I think the MFC list I have so far is "good enough" for merging the initial support to stable/10.. I'll work on figuring out the other bits ASAP.

make tinderbox passed, minus ia64 on ref10-amd64.freebsd.org. I'll run it again (see if I can repro the issue). If not, then I'll commit the first MFC to stable/10.

I'm debating about stable/9 because of the tests related changes. I can move the tests back to tools/regression/lib/libnv though in the MFstable/10, but I'd prefer doing it _once_ all of the MFCs to stable/10 have been done.

[ngie]

In D4249#100103, @ngie wrote:

In D4249#98955, @ngie wrote:

I'm almost done with compiling the list of items that need to be MFCed:

...

I haven't MFCed these items because it was starting to get hairy with pci(4), but there might be some items here that I can easily cherrypick:

r282346 r282347 r282348 r285063 r285129 r285139 r286796

These revisions all conflict. Once iovctl and the pci(4) bits have been merged in, the rest of the revisions should fall like dominoes. I'll double-check, but I think the MFC list I have so far is "good enough" for merging the initial support to stable/10.. I'll work on figuring out the other bits ASAP.

make tinderbox passed, minus ia64 on ref10-amd64.freebsd.org. I'll run it again (see if I can repro the issue). If not, then I'll commit the first MFC to stable/10.

I'm debating about stable/9 because of the tests related changes. I can move the tests back to tools/regression/lib/libnv though in the MFstable/10, but I'd prefer doing it _once_ all of the MFCs to stable/10 have been done.

The ia64 issue reproed. When I used make buildenv, then cd to rescue/rescue, gcc on stable/10 complains about a lot of -Wunused variables, related to #ifndef lint >_> (copyright, etc).

I don't want to break ia64, but it's basically a dead architecture *sigh*...

[ngie]

In D4249#100204, @ngie wrote:

...

The ia64 issue reproed. When I used make buildenv, then cd to rescue/rescue, gcc on stable/10 complains about a lot of -Wunused variables, related to #ifndef lint >_> (copyright, etc).

I don't want to break ia64, but it's basically a dead architecture *sigh*...

emaste tracked down why the build failure is occurring -- it's unrelated to this change:

12:28 < emaste> ngie: aha
12:28 < emaste> ELFNAMEEND(check)(int fd, const char *fn)
12:28 < emaste> has no EM_IA64 case :(
12:28 < ngie> crap
12:28 < emaste> :q
12:28 < ngie> ok :/...
12:29 < ngie> whoops!
12:29 < emaste> ngie: this might be a mismerge on my part
12:29 < emaste> ngie: r292421 :(
12:29  * ngie shrugs... it is a dead architecture!
12:30 < ngie> ah, whoops :(
12:31 < emaste> yeah. I took care to make sure EM_IA_64 stuck around when I tried the merge in git, but must have mixed it up when redoing with svn (to get mergeinfo)

[emaste]

emaste tracked down why the build failure is occurring -- it's unrelated to this change:

Fixed by rS292885

[bdrewery]

In D4249#93849, @ngie wrote:

In D4249#93840, @kevin.bowling_kev009.com wrote:

Hi Ngie, this diff has been tested and I would prefer it applied directly to -STABLE (there are changes vs HEAD). The patch file needs one tweak, /nvpair.h should be moved to sys/cddl/compat/opensolaris/sys/nvpair.h before commit.

Hi Kevin,

The reason for the commits that need MFCing is that I want to make sure the SVN metadata is correct/in sync. Unfortunately that information isn't present in your diff, so tracking the changes (and the reasons for them) would get extremely hairy, i.e. svn log won't be correct :(.

I can use this patch as a basis though. If everything's done right, then the patches will line up.

Thanks!
-NGie

You can just apply the patch and then set the mergeinfo for all merged revisions. It's simple for stable/10.

svn merge --record-only

[ngie]

In D4249#102040, @bdrewery wrote:

...

You can just apply the patch and then set the mergeinfo for all merged revisions. It's simple for stable/10.

svn merge --record-only

Logistically, yes. The problem is that the patch included files that ultimately weren't needed in order to backport nv(3)/nv(9), and there was some breakage in the pci(4) space as well with interfaces, etc...

[sbruno]

At this point, nobody is super excited about getting this MFC'd back to stable/10.

Is anyone still working on this review?