Question about keep state and TCP state table timing

[Kevin Hoffman]

Hello,

We're using IPfilter v4.1.9 on Solaris 10 amd64 (installed as part of the Solaris OS package for ipfilter). IPfilter has been working really well for us, it's fast and easy to use!

We have a rule setup like:

pass in quick proto tcp from 172.25.0.0/16 to any port = 2049 keep state

TCP port 2049 is the NFSv4 port. Under normal circumstances, this passes traffic just fine. We have some Linux computers (3.2.x kernel) connecting to the Solaris box mounting some NFSv4 shares hosted on the Solaris box.

If the TCP connection between the Linux machine and the Solaris machine gets interrupted, the Linux machine will try to reconnect to the Solaris box by opening a new TCP connection with destination port 2049.

However, here's the interesting part: Linux will re-use the same TCP *source* port when trying to reconnect. So what we see is that the Solaris box running ipfilter will have the existing TCP connection timeout. The linux box will then attempt to re-connect. However, because Linux is re-using the same source port the entry in the TCP state table for the prior connection has not yet expired. ipfilter thus ignores the new SYN packet from the Linux machine, and the new TCP connection does not get re-established (we can see the TCP connection on linux in the "SYN_SENT" state, and no record of a new TCP connection on the Solaris box, yet establishing a new TCP connection with telnet to 2049 from Linux works just fine (because the source port is different).

From what we understand (we've read the man/howto), we could remove the "keep state" rule option and it would pass all traffic to port 2049 (including SYN packets sent from the same source port as a recently closed TCP connection)? (is this the case?) However, keep state does have some nice security properties to not allow meddling with established TCP connections in progress. Also it has been mentioned that performance can be better when you're keeping state because it avoids a check on the inbound/outbound rule table

Is there any way to adjust the timeouts of the TCP state table so that it forgets the previously established connection more quickly? Ideally we could adjust the timeout only for connections that match this rule (e.g., something like "keep state lifetime-timeout 60"). If not, unless we can tell Linux to always use a different source port for the TCP connection, we probably will just have to avoid using "keep state" for that rule?

Thanks,
Kevin

[Kevin Hoffman]

Hi Darren,

Which Solaris 10 Update are you using?

We're using Solaris 10 update 10.

Not as such.
What should happen is that the first SYN packet should prompt ipfilter to discard the old state information quickly and in addition, drop that packet. When the SYN gets retransmitted, state should get created.

In most cases, this seems to happen. But there was a few cases where we saw the SYN packet get transmitted several times by the Linux client over a period of several minutes (e.g., 30 minutes) and the TCP connection would not setup until we disabled the firewall service and then things proceeded normally. Unfortunately we didn't dump the TCP state tables before we shutdown the ipfilter service. Once we get a test environment and can replicate this on, we'll try dumping the TCP state tables as well as logging rejected packets to try and get more information. Is there anything else we should gather in our diagnosis?

Thanks a lot,
--Kevin

[Kevin Hoffman]

Unfortunately I've helped about as far as I can with this and to go further, you'll need a support contract with Oracle. This is because the source code is closed and I cannot begin to guess what it does without looking at it.
To make any further meaningful progress, you would need to download version 5.1.1 from http://coombs.anu.edu.au/~avalon/ and get thtat running.

No problem, we've got an active support contract with Oracle and will proceed there. We thought we would try the mailing list first in case others had seen it before. We did get a direct response from Eric Behr indicating that they had seen a similar scenario (Novell client mounting SMB shares with same port, ipfilter was rejecting new connection with TCP out-of-window), and they were able to get around the issue with a rule that had the option "with oow" -- we will also be trying this in test environment.

I appreciate your time and the responses.

Best,
Kevin