Page MenuHomeFreeBSD

setsockopt: Improve SO_SETFIB handling
Changes PlannedPublic

Authored by markj on Nov 1 2024, 1:36 PM.
Tags
None
Referenced Files
Unknown Object (File)
Wed, Dec 18, 11:20 PM
Unknown Object (File)
Sun, Dec 8, 11:09 PM
Unknown Object (File)
Sun, Dec 8, 8:33 AM
Unknown Object (File)
Nov 24 2024, 2:22 AM
Unknown Object (File)
Nov 19 2024, 2:40 AM
Unknown Object (File)
Nov 14 2024, 12:30 PM
Unknown Object (File)
Nov 11 2024, 3:41 PM
Unknown Object (File)
Nov 5 2024, 2:16 PM

Details

Reviewers
zlei
Group Reviewers
network
Summary
  • Serialize updates to so_fibnum with the socket lock.
  • Make sure we can't change the FIB of a listening socket.

My goal here is to permit FIB-local listening sockets, wherein
multiple listening sockets listen on a port (using SO_REUSEPORT),
but each listening socket only accepts connections that come from
its FIB.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 60326
Build 57210: arc lint + arc unit

Event Timeline

markj requested review of this revision.Nov 1 2024, 1:36 PM

In fact, this probably does not go far enough. I'm not sure when it's useful to change the fibnum of a socket after creation time, but it's dangerous in general since the fibnum is also inherited by the inpcb.

In fact, this probably does not go far enough. I'm not sure when it's useful to change the fibnum of a socket after creation time, but it's dangerous in general since the fibnum is also inherited by the inpcb.

What about multi-fib-aware applications? For example, nginx allows to specify a specific fib for each listening socket.

In fact, this probably does not go far enough. I'm not sure when it's useful to change the fibnum of a socket after creation time, but it's dangerous in general since the fibnum is also inherited by the inpcb.

What about multi-fib-aware applications? For example, nginx allows to specify a specific fib for each listening socket.

Do you know offhand if it sets the FIB after calling listen()? I would hope/expect not.

In fact, this probably does not go far enough. I'm not sure when it's useful to change the fibnum of a socket after creation time, but it's dangerous in general since the fibnum is also inherited by the inpcb.

What about multi-fib-aware applications? For example, nginx allows to specify a specific fib for each listening socket.

From nginx doc https://nginx.org/en/docs/http/ngx_http_core_module.html,

setfib=number
this parameter (0.8.44) sets the associated routing table, FIB (the SO_SETFIB option) for the listening socket. This currently works only on FreeBSD.

and the usage: https://github.com/nginx/nginx/blob/master/src/core/ngx_connection.c#L817

nginx set SO_SETFIB option before listening ( the socket ).

For example, nginx allows to specify a specific fib for each listening socket.

Do you hint that one listening socket can have multiple / change the fib number ? That sounds not possible. Well it is absolutely OK to have multiple fibs for multiple different listening socket.

In fact, this probably does not go far enough. I'm not sure when it's useful to change the fibnum of a socket after creation time, but it's dangerous in general since the fibnum is also inherited by the inpcb.

What about multi-fib-aware applications? For example, nginx allows to specify a specific fib for each listening socket.

From nginx doc https://nginx.org/en/docs/http/ngx_http_core_module.html,

setfib=number
this parameter (0.8.44) sets the associated routing table, FIB (the SO_SETFIB option) for the listening socket. This currently works only on FreeBSD.

and the usage: https://github.com/nginx/nginx/blob/master/src/core/ngx_connection.c#L817

nginx set SO_SETFIB option before listening ( the socket ).

Thanks. I think this means that my patch will not break nginx, though it should probably allow setting the FIB if the number isn't changing.

The problem I'm trying to deal with is that this SO_FIBNUM handler does not update the fibnum stored in the associated inpcb.

For example, nginx allows to specify a specific fib for each listening socket.

Do you hint that one listening socket can have multiple / change the fib number ? That sounds not possible. Well it is absolutely OK to have multiple fibs for multiple different listening socket.

One limitation here is that the fibnum is not used when deciding which listening socket to pass a new connection to. I would like to have a mode (disabled by default to avoid breaking compatibility) where a listening socket only accepts connections from the FIB it's associated with. This is the larger goal that I'm working towards with some of these patches.

Do you hint that one listening socket can have multiple / change the fib number ? That sounds not possible

Well, what if we allow changing the fib number of listening socket, so filtering new connections based on the new fib number ?

@markj The first part Serialize updates to so_fibnum with the socket lock. looks good to me.

As for the second part Make sure we can't change the FIB of a listening socket, I do not know if future plan will do the opposite. Maybe it should be discussed via net mailing list.

markj planned changes to this revision.Nov 8 2024, 2:15 PM

Do you hint that one listening socket can have multiple / change the fib number ? That sounds not possible

Well, what if we allow changing the fib number of listening socket, so filtering new connections based on the new fib number ?

Yes, we could try to support that. I will look into it further.

@markj The first part Serialize updates to so_fibnum with the socket lock. looks good to me.

As for the second part Make sure we can't change the FIB of a listening socket, I do not know if future plan will do the opposite. Maybe it should be discussed via net mailing list.

Thank you for the feedback. I'll work on this some more and start a discussion.