In D2986#58711, @eri wrote:

After more testing, with help of Olivier, keep the workaround of testing if any SP is present to not impact general forwarding in non-IPSEC usage case.

Here is the forwarding bench results (value in paquet-per-second) without any SP configured:

x forwarding-NOIPSEC-in-kernel (head r285000)
+ forwarding-IPSEC-in-kernel-but-not-enabled (head r285000)
* forwarding-D2986-IPSEC-in-kernel-but-not-enabled (head r285000 with D2986) 
+--------------------------------------------------------------------------+
|*                    * *  x*     +      +  xxx+ *+          +            x|
|                             |______________M_A________________|          |
|                                   |__________A_________|                 |
|       |_______________MA________________|                                |
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5       1180885       1310062       1231130       1236671     46402.755
+   5       1200343       1273193       1236747     1234650.6       27525.7
No difference proven at 95.0% confidence
*   5       1109849       1241128       1172876     1175047.6     46705.073
No difference proven at 95.0% confidence

First time SPD is looked for INPUT policy and you have removed this functionality.

Andrey what is the use case for that?

Regarding of the use case. I think such INBOUND policy can be used like a firewall rule. You can add a DISCARD policy for some traffic flows. We don't use them, but someone can use.

So you are saying someone will only create a IN SP for just discarding packets without any ipsec policy in place?
Sounds like a very weird scenario when the user can just load a module in previous versions of FreeBSD for firewalling!

The above patch supports still the inbound filtering/discarding but relies on the user to have some ipsec policy already in place to protect packets.
Also the same consistency/check is not maintained on applications using IPSEC activated/encapsulated sockets.

I also understand that this code path also has the purpose of trying to check if the full policy has been applied to the packet but it skips some locking and its not correct, just that the issue is not hit often since SPDs/ISR on IPSEC do not get modified very often as they are read-mostly properties.

I just noticed, that your change breaks some of the functionality. I personally don't use it :)

In D2986#59234, @eri wrote:

I also understand that this code path also has the purpose of trying to check if the full policy has been applied to the packet but it skips some locking and its not correct, just that the issue is not hit often since SPDs/ISR on IPSEC do not get modified very often as they are read-mostly properties.

Each consumer of SPD entry holds reference to SPD, and there is no way to modify SPD entry. You only can unlink existing entry from the chain and add new entry. So, I don't see the problem here. The main problem is with locking of SAD and ISRs.

In D2986#59244, @ae wrote:

In D2986#59234, @eri wrote:

I also understand that this code path also has the purpose of trying to check if the full policy has been applied to the packet but it skips some locking and its not correct, just that the issue is not hit often since SPDs/ISR on IPSEC do not get modified very often as they are read-mostly properties.

Each consumer of SPD entry holds reference to SPD, and there is no way to modify SPD entry. You only can unlink existing entry from the chain and add new entry. So, I don't see the problem here. The main problem is with locking of SAD and ISRs.

Yes, that is correct that ISR and SAD are the problematic ones especially ISR needs more thought.

This looks stuck. Ermal, do you want to take another try on this one?