Notes Come in Handy Dropped SSH Connections Over IPSEC

March 5th, 2008

I just got FIOS at my office and I'm trying to debug an ssh problem with a VPN I've setup between my office and my home.

Soekris net4501 m0n0wall error

"It is happening again." (Fast forward to 4:33)

Darn it. I need to get to the bottom of this before it drives me crazy. Here's a recap of what happens. I login to a shell through an IPSEC connection, type dmesg, and the connection dies. I connect through another machine through IPSEC, then connect through another IPSEC tunnel to the same machine as the first try, type dmesg, and it works fine.

Clear DF Bit

I'm trying to set the clear DF big instead of dropping it option in pfSense advanced.

Workaround for operating systems that generate fragmented packets with the don't fragment (DF) bit set. Linux NFS is known to do this. This will cause the filter to not drop such packets but instead clear the don't fragment bit. The filter will also randomize the IP identification field of outgoing packets with this option on, to compensate for operating systems that set the DF bit but set a zero IP identification header field.

The link I provided at first describes my attempts to fix this under m0n0wall, where I believe the problem was caused by my allowing fragmented ipsec packets. This option isn't available in pfSense, so I'm trying some new techniques. Nope, that didn't work.


I tried this:

sysctl -a | grep ipsec

to see if that would shed some light on the matter but not much:

$ sysctl -a | grep ipsec  ipsecpolicy    64    16K       -     5578  256 ipsecrequest     4     1K       -       20  128   ipsec-misc    24     1K       -      132  32    ipsec-saq     0     0K       -        6  128    ipsec-reg     3     1K       -        6  16
net.inet.ipsec.def_policy: 1
net.inet.ipsec.esp_trans_deflev: 1
net.inet.ipsec.esp_net_deflev: 1
net.inet.ipsec.ah_trans_deflev: 1
net.inet.ipsec.ah_net_deflev: 1
net.inet.ipsec.ah_cleartos: 1
net.inet.ipsec.ah_offsetmask: 0
net.inet.ipsec.dfbit: 0
net.inet.ipsec.ecn: 0
net.inet.ipsec.debug: 0
net.inet.ipsec.esp_randpad: -1
net.inet.ipsec.crypto_support: 0
net.inet6.ipsec6.def_policy: 1
net.inet6.ipsec6.esp_trans_deflev: 1
net.inet6.ipsec6.esp_net_deflev: 1
net.inet6.ipsec6.ah_trans_deflev: 1
net.inet6.ipsec6.ah_net_deflev: 1
net.inet6.ipsec6.ecn: 0
net.inet6.ipsec6.debug: 0
net.inet6.ipsec6.esp_randpad: -1

Both machines have the same settings. Hmmm.

Aha! I just remembered I had some wacky tcp settings on the machine I was connecting to, I just commented them out of the sysctl.conf file, maybe that will fix it? Rebooting now...

#net.ipv4.tcp_fin_timeout = 30
#net.ipv4.tcp_timestamps = 0
#net.ipv4.tcp_keepalive_time = 1800
#net.ipv4.tcp_max_tw_buckets = 1440000
#net.ipv4.tcp_max_syn_backlog = 1024
#net.ipv4.tcp_syncookies = 1
#net.core.rmem_max = 16777216
#net.core.wmem_max = 16777216
#net.ipv4.tcp_mem = 4096 65536 16777216
#net.ipv4.tcp_rmem = 4096 87380 16777216
#net.ipv4.tcp_wmem = 4096 65536 16777216
#net.ipv4.tcp_no_metrics_save = 1

Nope, still happens.

Maximum Transmission Unit (MTU)

I just found this document about FreeSWAN, fragmented packets, and MTU and I was reminded of the advice shared by Chris B. and the pfSense / m0n0wall folks when I first ran into this problem. They recommended reducing the MTU, so I just tried doing that now, and it worked! In fact for whatever reason, by setting it to 1500 on both firewalls, the problem has gone away. Cool. Actually no I have to take that back, after changing to 1500 and re-logging in, the problem persisted, however I just found this on Verizon's network:

MTU (Maximum Transmission Units) - The MTU defines the largest single unit of data that can be transmitted over your connection. The FiOS network requires an MTU of 1492 bytes.

So in a nutshell, I believe that the 1492 MTU minus the IPSEC headers would equal the MTU I need to set as the WAN device connected to FIOS. I don't know what size those headers are, and I believe they vary depending upon the encryption type and IPSEC configuration, so I'm going to go with 1400 as a safe bet.


Yearly Indexes: 2003 2004 2006 2007 2008 2009 2010 2011 2012 2013 2015 2019 2020 2022