gulp - tcpdump alternative for lossless capture on Linux
tcpdump and wireshark are the tools that usually come to mind when you have to capture network traffic. But in some situations where you have to record a large amount of data and you want to avoid losing packets, tcpdump has some limitations. When I was hit myself by the tcpdump packet loss problem, I quickly found out that I was not alone and that a number of people had already researched the topic and/or provided alternatives.*
In particular, I found two different tools to perform the task: Corey Satten's gulp (http://corey.elsewhere.org/gulp/) and lindump from HP Labs (http://tesla.hpl.hp.com/opensource/)
I also found two interesting papers about capturing high volumes of traffic: http://www.usenix.org/events/fast09/tech/full_papers/anderson/anderson_html/ and http://docs.di.fc.ul.pt/jspui/bitstream/10455/3299/1/thesis-nhenriqu.pdf (the second quotes the first one among others, and also contains useful info to optimally spread the load among different cores)
After some tests I quickly became a happy gulp user, and thanks to the software being open source I was able to add features to it that I missed from the latest tcpdump versions:
-n - allows to change the default filename template
-t - allows to add a timestamp to the filename
-G - rotate pcap file every n seconds
-F - allows to skip the check for an ethernet interface
-Z - allows to specify a command to post-process each capture file
I've sent a patch to Corey Satten, who intends to setup a repository to hold the various contributions he gets for gulp. In the meanwhile, you can find my changes in the attached file (02-gulp-ntGFZ.patch.gz). For your convenience and for completeness, I also provide here the patch from Guy Harris that fixes issues on 64 bit systems (see http://seclists.org/wireshark/2009/Oct/105, apply that one first).
Updates:
2012-08-22: new version of my patch to fix the issue reported by SgtMalicious
2017-02-03: long-standing bug fixed
Downloads:
01-gulp-amd64.patch.gz: fix issues with 64-bit systems
02-gulp-ntGFZ.patch.gz: additional functionality as described above
gulp-1.58-crox.tgz: source with both patches applied
* other people have reported a performance drop with libpcap version 1.0 compared to previous builds, see http://thread.gmane.org/gmane.network.tcpdump.devel/4629 or http://seclists.org/tcpdump/2010/q3/index.html#11
In particular, I found two different tools to perform the task: Corey Satten's gulp (http://corey.elsewhere.org/gulp/) and lindump from HP Labs (http://tesla.hpl.hp.com/opensource/)
I also found two interesting papers about capturing high volumes of traffic: http://www.usenix.org/events/fast09/tech/full_papers/anderson/anderson_html/ and http://docs.di.fc.ul.pt/jspui/bitstream/10455/3299/1/thesis-nhenriqu.pdf (the second quotes the first one among others, and also contains useful info to optimally spread the load among different cores)
After some tests I quickly became a happy gulp user, and thanks to the software being open source I was able to add features to it that I missed from the latest tcpdump versions:
-n - allows to change the default filename template
-t - allows to add a timestamp to the filename
-G - rotate pcap file every n seconds
-F - allows to skip the check for an ethernet interface
-Z - allows to specify a command to post-process each capture file
I've sent a patch to Corey Satten, who intends to setup a repository to hold the various contributions he gets for gulp. In the meanwhile, you can find my changes in the attached file (02-gulp-ntGFZ.patch.gz). For your convenience and for completeness, I also provide here the patch from Guy Harris that fixes issues on 64 bit systems (see http://seclists.org/wireshark/2009/Oct/105, apply that one first).
Updates:
2012-08-22: new version of my patch to fix the issue reported by SgtMalicious
2017-02-03: long-standing bug fixed
Downloads:
01-gulp-amd64.patch.gz: fix issues with 64-bit systems
02-gulp-ntGFZ.patch.gz: additional functionality as described above
gulp-1.58-crox.tgz: source with both patches applied
* other people have reported a performance drop with libpcap version 1.0 compared to previous builds, see http://thread.gmane.org/gmane.network.tcpdump.devel/4629 or http://seclists.org/tcpdump/2010/q3/index.html#11
Trackbacks
No Trackbacks
Comments
Display comments as Linear | Threaded
Don on :
gulp -i eth1 > pcapfile
which appears to be capturing and writing to a file called pcapfile but when I attempt to read:
tcpdump -r pcapfile
reading from file pcapfile, link-type EN10MB (Ethernet)
tcpdump: pcap_loop: bogus savefile header
Any ideas of what I could be doing wrong?
Thanks
crox on :
Don on :
SgtMalicious on :
crox on :
Khi on :
Can you please send me your working version? Thanks.
crox on :
Which version of libpcap are you using?
This is the original source from Corey with patches applied: http://blog.crox.net/uploads/gulp-1.58-crox.tgz
ahat on :
root@moat:/tmp# ./gulp -i eth0 -r 1024 > /var/log/magi/local.pcap
^C
422051 packets captured
422106 packets received by filter
55 packets dropped by kernel
ring buffer use: 0.0% of 1024 MB
crox on :
ghaleb on :
sorry for this question but how to run patch file??
crox on :
You can also download the full source with patches applied, see comment #2.1.1.1.
roetherb on :
crox on :
crox on :
kk on :
how can i install gulp?
I applied 64bit patch by "patch -i 01-gulp-amd64.patch" then "make" but getting error.
ubuntu:~/gulp$make
cc -g -0 gulp.c -o gulp -lpthread -lpcap
gulp.c:79:18 fatal error: pcap.h: no such file or directory
#include
^
complilation terminated.
make ***[gulp} Error 1
crox on :
$ sudo apt-get install libpcap-dev
tpz on :
crox on :
snprintf(ofile, sizeof(ofile), "%s/%s%lld.%03d", dir, oname, (long long int)time(NULL), num);
tpz on :
crox on :
tpz on :
crox on :
ogd on :
Steveno on :
This is how I launched gulp to run continuously:
nohup ./gulp -i bond0 -f "net 10.30.205.0/24" -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t &
ps -ef | grep gulp
root 1871 1 1 Oct18 ? 01:07:55 ./gulp -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 1874 1871 0 Oct18 ? 00:14:53 ./gulp -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 3282 1871 0 Oct19 ? 00:05:24 ./gulp -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 3762 1871 0 09:37 ? 00:01:23 ./gulp -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 3778 1871 0 10:19 ? 00:01:14 ./gulp -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 3789 1871 0 10:50 ? 00:01:06 ./gulp -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 3802 1871 0 11:20 ? 00:00:59 ./gulp -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 3825 1871 0 12:35 ? 00:00:41 ./gulp -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 3841 1871 0 13:19 ? 00:00:30 ./gulp -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 3852 1871 0 13:46 ? 00:00:25 ./gulp -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 3861 1871 0 14:08 ? 00:00:19 ./gulp -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 3874 1871 0 14:32 ? 00:00:13 ./gulp -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 3883 1871 0 15:03 ? 00:00:06 ./gulp -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
n3tus3r 3996 3974 0 15:27 pts/0 00:00:00 grep --color=auto gulp
Steveno on :
I'm seeing odd behavior when running gulp with rotating packet captures in that once in awhile a file is created with minimal traffic but is saved as a 500MB file per my following command execution. For instance, when it works right, the pcap files contain millions of packets but when it flakes out it has between 200 - 300 packets. Has anyone experienced this and is there a fix? I'm running the patched gulp version on ubuntu 14.0.x. Any help would be very much appreciated.
Here's how I execute the gulp process. You'll notice I'm using nohup and the "&" to run my captures while away from my computer in order to investigate a problem we're experiencing with VoIP.
nohup ./gulp -x -i bond0 -f "net 10.30.205.0/24" -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t &
Not sure if it's related but when I check on the gulp process running, I see multiple instances.
Example:
ps -ef | grep gulp
root 7458 1 2 Oct22 ? 00:27:20 ./gulp -x -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 7461 7458 0 Oct22 ? 00:05:37 ./gulp -x -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 8270 7458 0 08:28 ? 00:00:58 ./gulp -x -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 8285 7458 0 09:15 ? 00:00:47 ./gulp -x -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 8302 7458 0 10:03 ? 00:00:36 ./gulp -x -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 8317 7458 0 10:36 ? 00:00:28 ./gulp -x -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 8330 7458 0 11:14 ? 00:00:19 ./gulp -x -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
root 8348 7458 0 12:03 ? 00:00:08 ./gulp -x -i bond0 -f net 10.30.205.0/24 -r 500 -C 1 -W 500 -o captures/ -n 200LS- -t
crox on :
[1] https://www.gnu.org/software/screen/
crox on :
Jishnu on :
I'm tryng to install GULP.I have installed libpcap:1386 but still make gives me the following error:
cc -O -m32 check64bit.c -o check64bit
./check64bit; rm -f check64bit
cc -g -O -m32 gulp.c -o gulp -lpthread -lpcap
gulp.c: In function ‘Reader’:
gulp.c:400:22: warning: ignoring return value of ‘read’, declared with attribute warn_unused_result [-Wunused-result]
if (tmpfd >= 0) read(tmpfd, (char )&fh, sizeof(fh));
^
gulp.c:436:24: warning: ignoring return value of ‘fscanf’, declared with attribute warn_unused_result [-Wunused-result]
if (procf) {fscanf(procf, "%d", &rmem_def); fclose(procf);}
^
gulp.c:438:24: warning: ignoring return value of ‘fscanf’, declared with attribute warn_unused_result [-Wunused-result]
if (procf) {fscanf(procf, "%d", &rmem_max); fclose(procf);}
^
gulp.c: In function ‘newoutfile’:
gulp.c:493:11: warning: ignoring return value of ‘fchown’, declared with attribute warn_unused_result [-Wunused-result]
fchown(tmpfd, getuid(), -1); / in case running setuid /
^
gulp.c: In function ‘Writer’:
gulp.c:542:9: warning: ignoring return value of ‘seteuid’, declared with attribute warn_unused_result [-Wunused-result]
seteuid(getuid()); / drop setuid privilege /
^
/usr/bin/ld: cannot find -lpcap
collect2: error: ld returned 1 exit status
make: ** [gulp] Error 1
Could you please help me
crox on :
I assume you are using a 64-bit distribution? You should not need 32-bit libraries in that case. But you may need to install libpcap-dev or the equivalent package for your distribution.
Try to unpack the patched source from comment #2.1.1.1 and then just run "make".
Jishnu on :
It gave me the same error with the patch. But when I manually complie using gcc (with pcap) i get a gulp executable.
I hope that will work for me. COuld you also please help me to use gulp to save 100 pcap files with 20MB size with 1GB as buffer size. It seems gulp wont let me use a smaller file size with huge buffer size
regards
crox on :
gulp -i eth0 -r 1024 | gulp -i - -r 20 -C 1 -o /path/
Jishnu on :
Thanks for reply, I used your code but it give me
Gulp : fatal write error: Broken pipe
I also have a doubt, my intention is to use Gulp to capture at 200Mbps, but as the buffer cannot be increased above 1024MB how will it clear the buffer and not report "Buffer full"
crox on :
This works for me:
root@laptop:~# mkdir /tmp/gulp
root@laptop:~# gulp -i wlan0 -r 1024 | gulp -i - -r 1024 | gulp -i - -r 20 -C 1 -o /tmp/gulp/
It creates 20 MB files named pcap000, pcap001 etc. in /tmp/gulp
qwerty on :
cc -O check64bit.c -o check64bit
./check64bit; rm -f check64bit
If output from Gulp is not compatible with tcpdump or wireshark,
Please see: http://staff.washington.edu/corey/gulp/gulp.html#64bit
cc -g -O gulp.c -o gulp -lpthread -lpcap
Please help
crox on :
I recommend you compile from the source with both patches applied: http://blog.crox.net/uploads/gulp-1.58-crox.tgz
qwerty on :
could you please give a detailed explanation of how to install gulp from the patch file in #2.1.1.1.I did not understand the installation
crox on :
Once you've successfully compiled it, just copy gulp to e.g. /usr/local/bin/ or /usr/local/sbin/
qwerty on :
crox on :
qwerty on :
crox on :
-r sets the size of the ring buffer in MB (default is 100 MB) - check the man page (or gulpman.pdf) for details.
Chris on :
Chris on :
Carsten on :
your collection solves my issues with the initial version of gulp on a current 64bit Linux. Thereby I want to share my feedback with the current version.
In process_savefile() a final exit() is missing. It's necessary if the zflag is not set or execlp() fails. In both cases the child process would unexpectedly return to the main loop and probably run in an infinite loop.
The second change is triggered by the error message "gzip: No such file or directory":
$ ./gulp -i eth0 -o . -Z gzip
gzip: No such file or directory
^C (Ctrl-C pressed after afew seconds)
3405 packets captured
3409 packets received by filter
0 packets dropped by kernel
ring buffer use: 0.1% of 100 MB
This message from gzip is caused by the first call of process_savefile() with an empty filename. The message itself is just ugly and confusing with gzip, but may cause serious trouble with other post processing programs.
This is the call stack for the situation if the error message is triggered by an empty filename:
Breakpoint 1, process_savefile (filename=0x60a400 "") at /home/carsten/gulp-1.58-crox/gulp.c:491
491 if (fork())
(gdb) bt
#0 process_savefile (filename=0x60a400 "") at /home/carsten/gulp-1.58-crox/gulp.c:491
#1 0x0000000000403686 in newoutfile (dir=0x7fffffffec92 ".", num=0) at /home/carsten/gulp-1.58-crox/gulp.c:550
#2 0x00000000004038ad in Writer (arg=0x0) at /home/carsten/gulp-1.58-crox/gulp.c:604
#3 0x00007ffff7bc6e25 in start_thread () from /lib64/libpthread.so.0
#4 0x00007ffff76b334d in clone () from /lib64/libc.so.6
May you execute "execlp(zcmd,...)" only the filename is set ("strlen(filename) > 0").
Thank you for your great extension of gulp.
Regards,
Carsten
Jernej on :
crox on :
Chris Glover on :
Gulp works very nicely for capturing SMPTE 2022-6 video streams at 1.5Gbps from a 100G NIC. Set the ringbuffer to maximum and had no dropped packets.
Makes my life so much easier!