fgt2eth Script

Packet capture for only certain TCP flags (for use with Fortigate 'sniffer' command):Information extracted from the Man pages of 'tcpdump' command:# Updated Jan 2007 for Fortinet KC by Stephen Dennis. There are 8 bits in the control bits section of the TCP header: Let's assume that we want to watch packets used in establishing a TCP
connection. Recall that TCP uses a 3-way handshake protocol when it initial-
izes a new connection; the connection sequence with regard to the TCP control
bits is 1) Caller sends SYN
2) Recipient responds with SYN, ACK
3) Caller sends ACK Now we're interested in capturing packets that have only the SYN bit set (Step
1). Note that we don't want packets from step 2 (SYN-ACK), just a plain ini-
tial SYN. What we need is a correct filter expression for tcpdump. Recall the structure of a TCP header without options: 0 15 31
-----------------------------------------------------------------
| source port | destination port |
-----------------------------------------------------------------
| sequence number |
-----------------------------------------------------------------
| acknowledgment number |
-----------------------------------------------------------------
| HL | rsvd |C|E|U|A|P|R|S|F| window size |
-----------------------------------------------------------------
| TCP checksum | urgent pointer |
----------------------------------------------------------------- A TCP header usually holds 20 octets of data, unless options are present. The
first line of the graph contains octets 0 - 3, the second line shows octets 4
- 7 etc. Starting to count with 0, the relevant TCP control bits are contained in octet
13: 0 7| 15| 23| 31
----------------|---------------|---------------|----------------
| HL | rsvd |C|E|U|A|P|R|S|F| window size |
----------------|---------------|---------------|----------------
| | 14th octet | | | Let's have a closer look at octet no. 13: | |
|---------------|
|C|E|U|A|P|R|S|F| <- TCP (and ECN) flags
|---------------|
|7 5 3 0| <- bit position These are the TCP control bits we are interested in. We have numbered the
bits in this octet from 0 to 7, right to left, so the PSH bit is bit number 3
(4th bit from right), while the URG bit is number 5 (6th bit from right). Recall that we want to capture packets with only SYN set. Let's see what hap-
pens to octet 13 if a TCP datagram arrives with the SYN bit set in its header: |C|E|U|A|P|R|S|F| <- TCP and ECN flags
|---------------|
|0 0 0 0 0 0 1 0| <- bit value
|---------------|
|7 6 5 4 3 2 1 0| <- bit position Looking at the control bits section we see that only bit number 1 (SYN) is
set. Assuming that octet number 13 is an 8-bit unsigned integer in network byte
order, the binary value of this octet is 00000010 and its decimal representation is: | bit | dec. | bit | | TCP
| pos. | value | value | total | Flag
____|______|________|_______|_______|_____
| | | | |
2 ^ 7 = 128, x 0 = 0 | CWR (ECN Congestion Window Reduced)
2 ^ 6 = 64, x 0 = 0 | ECE (ECN Capable Echo)
2 ^ 5 = 32, x 0 = 0 | URG
2 ^ 4 = 16, x 0 = 0 | ACK
2 ^ 3 = 8, x 0 = 0 | PSH
2 ^ 2 = 4, x 0 = 0 | RST
2 ^ 1 = 2, x 1 = 2 | SYN*
2 ^ 0 = 1, x 0 = 0 | FIN
____|______|________|_______|_______|_____
sum: 2
==========================================
('2 ^ 7 = 128' means '2 to the power of 7 equals 128') We're almost done, because now we know that if only SYN is set, the value of
the 14th octet in the TCP header, when interpreted as an 8-bit unsigned integer
in network byte order, must be exactly 2. This relationship can be expressed as
'tcp[13]==2' The expression says "match all TCP datagrams where octet 13 (i.e. the 14th octet)
is equal to the decimal value 2", which is exactly what we want. But now, let's assume that we need to capture ALL packets with the SYN bit set,
regardless of whether any other TCP control bits (such as ACK, URG, PSH etc.) are
set. Let's see what happens to octet 13 when a TCP datagram arrives with both the
SYN and ACK bits set: |C|E|U|A|P|R|S|F| <- TCP (and ECN) flags
|---------------|
|0 0 0 1 0 0 1 0| <- bit value
|---------------|
|7 6 5 4 3 2 1 0| <- bit position Now bits 4 and 1 are set in the 14th octet. The binary value of octet 13 is now: 00010010 which translates to a decimal value of: | bit | dec. | bit | | TCP
| pos. | value | value | total | Flag
____|______|________|_______|____________
| | | | |
2 ^ 7 = 128, x 0 = 0 | CWR (ECN Congestion Window Reduced)
2 ^ 6 = 64, x 0 = 0 | ECE (ECN Capable Echo)
2 ^ 5 = 32, x 0 = 0 | URG
2 ^ 4 = 16, x 1 = 16 | ACK*
2 ^ 3 = 8, x 0 = 0 | PSH
2 ^ 2 = 4, x 0 = 0 | RST
2 ^ 1 = 2, x 1 = 2 | SYN*
2 ^ 0 = 1, x 0 = 0 | FIN
____|______|________|_______|_______|_____
sum: 18
========================================== We can't just use 'tcp[13]==18' in the tcpdump filter expression, because that
would match only those packets where both SYN and ACK are set, but not packets
with only SYN set. Remember that we don't care if ACK or any other control bit
is set, as long as SYN is set. In order to achieve our goal, we need to logically 'AND' the binary value of
octet 13 with some other value to preserve the SYN bit. We know that we want
SYN to be set in any case, so we'll logically AND the value in the 14th octet
with the binary value of a SYN: v v
00010010 SYN-ACK 00000010 SYN
AND 00000010 (we want SYN) AND 00000010 (we want SYN)
-------- --------
= 00000010 = 00000010
^ ^
We see that this AND operation delivers the same result regardless of whether
any other TCP control bits are set or not. The decimal representation of the
AND value as well as the result of this operation is 2 (binary 00000010), so we
know that for packets with SYN set the following relation must hold true: ( ( value of octet 13 ) AND ( 2 ) ) == ( 2 ) This relationship can be expressed as: 'tcp[13]&2==2' The following list of expressions expand on this idea: 'tcp[13]==1' matches packets with only the FIN bit set to 1;
'tcp[13]&4==4' matches all packet with the RST bit set to 1;
'tcp[13]&8==8' matches all packet with the PSH bit set to 1;
'tcp[13]==16' matches packets with only the ACK bit set to 1;
'tcp[13]&32==32' matches all packets with the URG bit set to 1;
'tcp[13]&64==64' matches all packets with the ECE bit set to 1;
'tcp[13]&128==128' matches all packets with the CWR bit set to 1;
'tcp[13]==24' matches packets with only the PSH and ACK bits set to 1;Fortigate examples:
--------------------Example to sniff for packets with ONLY the TCP SYN set (no other flags set):# diag sniff packet wan1 'tcp[13]==2'Example to sniff for ANY packets with the TCP SYN set (regardless of other flags):# diag sniff packet wan1 'tcp[13]&2==2'Example to sniff for any packets on port 80 (HTTP) with the TCP SYN set:# diag sniff packet wan1 'port 80 and tcp[13]&2==2'Example to sniff for any packets on port 80 (HTTP) or 443 (HTTPS) with the TCP SYN set:# diag sniffer packet internal 'port 80 or 443 and tcp[13]&2==2'

#!/usr/bin/perl# This work (and included software, documentation or other related items) is
# being provided by the copyright holders under the following license. By
# obtaining, using and/or copying this work, you (the licensee) agree that you
# have read, understood, and will comply with the following terms and conditions
#
# Permission to copy, modify, and distribute this software and its documentation
# with or without modification, for any purpose and without fee or royalty is
# hereby granted, provided that you include the following on ALL copies of the
# software and documentation or portions thereof, including modifications:
#
# 1. The full text of this NOTICE in a location viewable to users of the
# redistributed or derivative work.
# 2. Notice of any changes or modifications to the files, including the date
# changes were made.
#
#
# THIS SOFTWARE AND DOCUMENTATION IS PROVIDED "AS IS," AND COPYRIGHT HOLDERS
# MAKE NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT
# LIMITED TO, WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR
# PURPOSE OR THAT THE USE OF THE SOFTWARE OR DOCUMENTATION WILL NOT INFRINGE ANY
# THIRD PARTY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS.
#
# COPYRIGHT HOLDERS WILL NOT BE LIABLE FOR ANY DIRECT, INDIRECT, SPECIAL OR
# CONSEQUENTIAL DAMAGES ARISING OUT OF ANY USE OF THE SOFTWARE OR DOCUMENTATION.
#
# Title to copyright in this software and any associated documentation will at
# all times remain with copyright holders.
#
# Copyright: Fortinet Inc - 2005
## Fortinet EMEA Support
# This script converts a Fortigate verbose 3 sniffer output file to a file that
# can be opened by Ethereal. It uses the text2pcap binary included in the Ethereal package.
# It is supplied as is and no technical support will be provided for it. my $version = "Dec 19 2014";# ------------------ don't edit after this line -------------------------------
use strict;
use Getopt::Long;
use FileHandle;
use Config;
use vars qw ($debug $help $vers $in $out $lines $demux $childProcess);
use Data::Dumper;# Autoflush
$| = ;# Global variables
my $line_count = ;
my ($fh_in, $fh_out);
my @fields = {};
my @subhexa = {};
my ($offset,$hexa,$garbage) = "";
my %outfileList;
my %outfilenameList; # Get commandLine arguments
getArgs(); # In order to support real-time display in wireshark/ethereal, we need to pipe
# our stdout into wireshark stdin, which is not allowed by the OS...
# The trick consists in creating a child process with the appropriate anonymous
# pipes already in place and delegate the work to the child.
spawnPipedProcess(); open(fh_in, '<', $in) or die "Cannot open file ".$in." for reading\n";# Convert
if( $debug ) {
print STDERR "Conversion of file ".$in." phase 1 (FGT verbose 3 conversion)\n";
print STDERR "Output written to ".$out.".\n";
} my @packetArray = (); #Parse the entire source file
my $DuplicateESP = ;
my $eth0 = ;
my $skipPacket = ;followMode:
while (<fh_in>) {
s/^\d{}(\d{})/0x$/;
#and build an array from the current packet
if( isTimeStamp() ) {
$skipPacket = ;
if( not $demux and /eth0/ ) {
$eth0++;
$skipPacket = ;
} # Select the appropriate output file for the interface.
$fh_out = getOutputFileHandler() if defined $demux;
$skipPacket |= convertTimeStamp();
} elsif ( isHexData() and not $skipPacket ) {
buildPacketArray();
adjustPacket();
_startConvert();
}
} if( $out eq "-" ) {
# no more incoming data. Wait 2 seconds and try again
sleep ;
goto followMode;
} print "** Skipped $eth0 packets captured on eth0\n" if $eth0;# Close files and start text2pcap if needed
close (fh_in) or die "Cannot close file ".$in."\n";
my $text2pcap = getText2PcapCmd();
foreach my $intf( keys %outfileList ) {
close $outfileList{ $intf };
my $filename_in = $outfilenameList{$intf};
my $filename_out = $filename_in;
$filename_out =~ s/\.tmp$/\.pcap/;
system("$text2pcap $filename_in $filename_out");
unlink($filename_in);
} if( $debug ) {
print STDERR "Output file to load in Ethereal is \'".$out."\'\n";
print STDERR "End of script\n";
}sub isHexData { /^(0x[-9a-f]+[ \t\xa0]+)/ }
sub isTimeStamp { /^[-]+[\.\-][-]+/ }
# /Dummy comment to workaround Komodo parsingsub buildPacketArray {
my $line = ;
@packetArray = (); do {
# Format offset from 0x0000 to 000000 (text2pcap requirement)
s/^0x([\da-f]{})/$/;
if ( s/^([\da-f]{})\s+// ) {
# Remove ASCII translation at the end of the line
s/\s\s+.*//;
my @bytes = /([\da-f]{})\s?([\da-f]{})?/g;
$#bytes = 15;
push @packetArray => @bytes;
}
$_ = <fh_in>;
} until ( /^\s*$/ );
}sub convertTimeStamp {
# Keep timestamps.
return if /truncated\-ip \- [-]+ bytes missing!/ ;
if ( /^([-]+)\.([-]+) / )
{
my $packet = ;
my $time = $; # Extract days
my $nbDays = int($time / );
my $day = sprintf("%0.2d", +$nbDays);
$time = $time % ; # Extract hours
my $hour = int($time / );
$time = $time % ; # Extract minutes
my $minute = int( $time / );
$time = $time % ; # and remaining seconds
my $sec = $time; _print("01/$day/2005 " . $hour . ":" . $minute . ":" . $sec . ".$2\n");
} elsif ( /^(\d+-\d+-\d+ \d+:\d+:\d+\.\d+) / ) {
# absolute timestamp
my $timestamp = $;
$timestamp =~ s/(\d+)-(\d+)-(\d+)/$\/$\/$/;
_print("$timestamp\n");
}
# Check if line is a duplicate ESP packet (FGT display bug)
return ;}sub getOutputFileHandler
{
my ($currIntf) = $_ =~ / (\S+) (?:out|in) /;
$currIntf = "[noIntf]" if $currIntf eq "";
if( not defined( $outfileList{$currIntf} )) {
my $filename = $out ? $out : $in;
$filename =~ s/\.zip$//g;
my $suffix = ".$currIntf.tmp";
$suffix =~ s/\//-/g; # Escape '/' char in interface name
$filename .= $suffix;
open( $outfileList{$currIntf}, "> $filename");
$outfilenameList{$currIntf} = $filename;
}
return $outfileList{$currIntf};
}#----------
# name : adjustPacket
# description:
# Applies changes to the current packetArray to make it convertible into
# ethereal format.
# - Removes internal Fortigate tag when capture interface is any.
#
sub adjustPacket { next unless $packetArray[][] =~ /0x0000/;
stripBytes( , ) if $packetArray[][] =~ //;
stripBytes( , ) if $packetArray[][] =~ //;
addHdrMAC() if $packetArray[][] =~ /[0c]/;
$packetArray[][] =~ s/|// if $packetArray[][] =~ /0x0000/;
}sub addHdrMAC
{
my $nbRows = scalar @packetArray; # And populate 0x0000 line
unshift @packetArray => qw( );
# left shift the IP ver+IHL (4500)
stripBytes(,);
}sub stripBytes
{
my $start = shift;
my $nbBytes = shift; my @subArray = @packetArray[$start..$#packetArray-1];
shift @subArray for ..$nbBytes;
@packetArray = (@packetArray[..$start-],@subArray);
}sub _startConvert {
LINE:
#Initialisation
my $hexa = "";
my $garbage = ""; my $offset = ;
foreach my $byte (@packetArray) {
_print( sprintf( "%0.6x ", $offset )) unless $offset % ;
_print( " ". $byte);
$offset ++;
_print("\n") unless $offset % ;
}
_print( "\n"); $line_count++;
if (defined($lines)) {
if ($line_count >= $lines) {
print STDERR "Reached max number of lines to write in output file\n";
last LINE;
}
}
}sub spawnPipedProcess
{
# Spawn a new process to pipe ourselves with text2pcap
my( $fgt2eth, $text2pcap, $wireshark ); # Are we already in the pipelined session ?
return if $childProcess or $demux; $fgt2eth = $ . " -in \"$in\" -out \"$out\" -childProcess";
$text2pcap = getText2PcapCmd(). "- ";
$wireshark = getWiresharkCmd(); my $cmd = "$fgt2eth | $text2pcap"; # Prepare output filename
if( $out eq "-" ) {
$cmd .= "- | $wireshark";
} else {
$out = $in . ".pcap" unless $out;
$cmd .= " $out";
} print STDERR $cmd if $debug; open( CMD, "$cmd |" );
while( <CMD> ) {
_print($_);
} close CMD; # Make sure we don't go further in the parent process
exit;
}sub getText2PcapCmd
{
my $cmd; my @windowsPath = qw(
c:\\Progra~\\Ethereal
c:\\Progra~\\Wireshark
); if ($ENV{'OS'} =~ /windows/i) {
if( $Config{osname} =~ /cygwin/i ) {
#OS is Windows running Cygwin
$cmd = "'/cygdrive/c/Program Files/Wireshark/text2pcap'";
} else {
# OS is windows
my $dir;
for $dir( @windowsPath ) {
$cmd = "$dir\\text2pcap.exe" if -e "$dir\\text2pcap.exe";
}
}
} else {
# OS is linux :-)
$cmd = "text2pcap";
} # Sanity
die "Text2Pcap could not be found\n" unless $cmd; $cmd .= " -q -t \"%d/%m/%Y %H:%M:%S.\" "; return $cmd;
}sub getWiresharkCmd
{
my $cmd; my @windowsPath = qw(
c:\\Progra~\\Wireshark
); if ($ENV{'OS'} =~ /windows/i) {
if( $Config{osname} =~ /cygwin/i ) {
# OS is Windows running Cygwin
$cmd = "'/cygdrive/c/Program Files/Wireshark/wireshark'";
} else {
# OS is windows
my $dir;
for $dir( @windowsPath ) {
$cmd = "$dir\\wireshark.exe" if -e "$dir\\wireshark.exe";
}
}
} else {
# OS is linux :-)
$cmd = "wireshark";
} # Sanity
die "wireshark could not be found\n" unless $cmd; $cmd .= " -k -i -"; return $cmd;
}sub _print{ my $msg = shift; if( defined $fh_out ) {
print $fh_out $msg;
} else {
print $msg;
}
}sub getArgs
{ # Control command line options
GetOptions(
"debug" => \$debug, # use -debug to turn on debug
"version" => \$vers, # use -version to display version
"help" => \$help, # use -help to display help page
"in=s" => \$in, # use -in <filename> to specify an input file
"out=s" => \$out, # use -out <filename> to specify an output file
"lines=i" => \$lines, # use -lines <number> to stop after <number> lines written
"demux" => \$demux, # use -demux to create one pcap per intf
"childProcess" => \$childProcess,
); if ($help) {
Print_help();
exit;
} if ($vers) {
Print_version();
exit;
} # Sanity checks
if (not(defined($in))) {
Print_usage();
exit;
}
}#------------------------------------------------------------------------------
sub Print_usage { print <<EOT;
Version : $version
Usage : fgt2eth.pl -in <input_file_name>Mandatory argument are :
-in <input_file> Specify the file to convert (FGT verbose text file)Optional arguments are :
-help Display help only
-version Display script version and date
-out <output_file> Specify the output file (Ethereal readable)
By default <input_file>.pcap is used
- will start wireshark for realtime follow-up
-lines <lines> Only convert the first <lines> lines
-demux Create one pcap file per interface (verbose only)
-debug Turns on debug modeEOT
}#------------------------------------------------------------------------------
sub Print_help { print <<EOT;
This script permits to sniff packets on the fortigate with built-in sniffer
diag sniff interface <interface> verbose filters '....'
and to be able to open the captured packets with Ethereal free sniffer.* What do I need to know about this script ?
- It can be sent to customers, but it is given as is.
- No support is available for it as it is not an 'offical' fortinet product.
- It should run on windows and linux as long as perl is installed.
- To install perl on windows system,
http://www.activestate.com/Products/ActivePerl/
- All lines from the source file that do not begin with '0x' are ignored.
- Do not add garbage characters to the file during the capture
- If possible do not hit the keyboard during capture.;Remarks concerning this script can be sent to eu_support\@fortinet.com
Thanks to Claudio for this great idea and Ellery from Vancouver Team for the timestampsCedricEOT}
#------------------------------------------------------------------------------sub Print_version {
print "\nVersion : ".$version."\n\n";
}sub myDump {
my $object = shift; my $dumper = new Data::Dumper([$object]);
$dumper->Maxdepth();
print STDERR $dumper->Dump;
}

> fgt2eth.02.2008.exe /?
Version : 1.22 (Feb 21 2008)Assuming OS: windowsUsage : fgt2eth.pl -in <input_file_name>Mandatory argument are : -in <input_file> Specify the file to convert (FGT verbose 3 text file)Optional arguments are : -help Display help only
-version Display script version and date
-out <output_file> Specify the output file (Ethereal readable)
By default '<input_file>.eth' is used
-lines <lines> Only convert the first <lines> lines
-system <system> Can be either linux or windows
-debug Turns on debug mode

> fgt2eth.02.2008.exe /?
Version : 1.22 (Feb 21 2008)Assuming OS: windowsUsage : fgt2eth.pl -in <input_file_name>Mandatory argument are : -in <input_file> Specify the file to convert (FGT verbose 3 text file)Optional arguments are : -help Display help only
-version Display script version and date
-out <output_file> Specify the output file (Ethereal readable)
By default '<input_file>.eth' is used
-lines <lines> Only convert the first <lines> lines
-system <system> Can be either linux or windows
-debug Turns on debug mode
PS C:\Users\lsgxt\Downloads> .\fgt2eth.12.2014.exe /?
Version : Dec 19 2014
Usage : fgt2eth.pl -in <input_file_name>Mandatory argument are :
-in <input_file> Specify the file to convert (FGT verbose 3 text file)Optional arguments are :
-help Display help only
-version Display script version and date
-out <output_file> Specify the output file (Ethereal readable)
By default <input_file>.pcap is used
- will start wireshark for realtime follow-up
-lines <lines> Only convert the first <lines> lines
-demux Create one pcap file per interface (verbose 6 only)
-debug Turns on debug mode

Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 2013/11/30 9:08 84 convert.bat
-a---- 2013/11/30 9:08 104 convertopen.bat
-a---- 2010/11/17 4:19 1808230 fgt2eth.exe
-a---- 2012/10/29 3:57 1149982 libglib-2.0-0.dll
-a---- 2012/10/29 3:57 46371 libgmodule-2.0-0.dll
-a---- 2012/9/4 4:30 130181 libintl-8.dll
-a---- 2013/11/2 2:07 33200 libwsutil.dll
-a---- 2011/2/20 14:03 211280 msvcp100.dll
-a---- 2011/2/19 15:40 356176 msvcr100.dll
-a---- 2013/12/3 2:14 1102 README.txt
-a---- 2013/11/2 2:07 338864 text2pcap.exe
-a---- 2018/11/30 10:28 48968 uninst.exe