blog.crox.net

tcpdump and wireshark are the tools that usually come to mind when you have to capture network traffic. But in some situations where you have to record a large amount of data and you want to avoid losing packets, tcpdump has some limitations. When I was hit myself by the tcpdump packet loss problem, I quickly found out that I was not alone and that a number of people had already researched the topic and/or provided alternatives.*

In particular, I found two different tools to perform the task: Corey Satten's gulp (http://corey.elsewhere.org/gulp/) and lindump from HP Labs (http://tesla.hpl.hp.com/opensource/)

I also found two interesting papers about capturing high volumes of traffic: http://www.usenix.org/events/fast09/tech/full_papers/anderson/anderson_html/ and http://docs.di.fc.ul.pt/jspui/bitstream/10455/3299/1/thesis-nhenriqu.pdf (the second quotes the first one among others, and also contains useful info to optimally spread the load among different cores)

After some tests I quickly became a happy gulp user, and thanks to the software being open source I was able to add features to it that I missed from the latest tcpdump versions:

-n - allows to change the default filename template
-t - allows to add a timestamp to the filename
-G - rotate pcap file every n seconds
-F - allows to skip the check for an ethernet interface
-Z - allows to specify a command to post-process each capture file

I've sent a patch to Corey Satten, who intends to setup a repository to hold the various contributions he gets for gulp. In the meanwhile, you can find my changes in the attached file (02-gulp-ntGFZ.patch.gz). For your convenience and for completeness, I also provide here the patch from Guy Harris that fixes issues on 64 bit systems (see http://seclists.org/wireshark/2009/Oct/105, apply that one first).

Updates:
2012-08-22: new version of my patch to fix the issue reported by SgtMalicious
2017-02-03: long-standing bug fixed

Downloads:
01-gulp-amd64.patch.gz: fix issues with 64-bit systems
02-gulp-ntGFZ.patch.gz: additional functionality as described above
gulp-1.58-crox.tgz: source with both patches applied

* other people have reported a performance drop with libpcap version 1.0 compared to previous builds, see http://thread.gmane.org/gmane.network.tcpdump.devel/4629 or http://seclists.org/tcpdump/2010/q3/index.html#11

1. create and enable a "regular" swap partition (fdisk / mkswap / swapon)

2. install ecryptfs-utils and run ecryptfs-setup-swap

sudo apt-get install ecryptfs-utils
sudo ecryptfs-setup-swap



                

For years the default behaviour in Thunderbird had been to forward e-mails as attachments, but at some point it was changed to "inline". You can still manually choose how you'd like to transfer a message by going to "Message" -> "Forward as", but I couldn't find a way to set the default in the preferences.

There is, however, a way to change it without messing with manual edit of config files. Go to "Preferences" -> "Advanced" -> "Config Editor...", and look for "mail.forward_message_mode". 0 is for "attached", 2 is for "inline".

When performing tests you may sometimes want to send specially crafted icmp packets. hping is a handy tool for that.

However, the default behavior is to refuse to send "unsupported" Type/Code combinations. eg

hping3 -c 1 --icmp -C 33 -K 0 192.168.70.1
HPING 192.168.70.1 (wlan0 192.168.70.10): icmp mode set, 28 headers + 0 data bytes
[send_icmp] Unsupported icmp type!

Fortunately, there is an (undocumented) --force-icmp option that you can add to bypass the check:

hping3 -c 1 --icmp --force-icmp -C 33 -K 0 192.168.70.1
HPING 192.168.70.1 (wlan0 192.168.70.10): icmp mode set, 28 headers + 0 data bytes

--- 192.168.70.1 hping statistic ---
1 packets transmitted, 0 packets received, 100% packet loss
round-trip min/avg/max = 0.0/0.0/0.0 ms

After a fresh pfSense install, I found out that traffic from specific hosts was being dropped when it should have been allowed based on the firewall rules I defined.

It turned out that the option "block bogon networks" was activated on the WAN interface, and that fresh pfSense images come with a slightly outdated bogon list.

If you are facing this problem, you have three options:

1. disable the "Block bogon networks" option at the bottom of the WAN interface page

2. after at most one week, the list will be updated automatically as long as the box is online (there is a cron entry, grep your config file for bogon)

3. if you don't want 1. and can't wait for 2, you can trigger the update process manually by running:

/etc/rc.update_bogons.sh 0

Check the output from the Status -> System Logs -> System page (I ran it from a serial console, but it should work fine by ssh or from the exec.php page too)