Copyright © 2003, 2004 Jeremy Kerr, Rusty Russell
$LastChangedDate: 2005-11-03 04:49:05 +1100 (Thu, 03 Nov 2005) $, released under GPL.
Abstract
This document provides an overview of the Netfilter Simulator system, used for testing kernel netfilter code.
Table of Contents
The netfilter simulator provides an environment to run kernel netfilter code, by providing a kernel-like 'environment'. The simulator consists of:
Provide generic kernel structures and functions, such as
spinlocks. This code is in the kernelenv/
directory.
Provides the protocol independent netfilter structures and
functions, which are used by the protocol module and the netfilter code
under test. The core functionality is provided by
core/*.c.
Allows userspace programs to talk to the simulated
kernel using the {s,g}etsockopt() function. This
allows tools such as iptables to be used without
modification.
Provides all the protocol-specific functions for the current
protocol - at present there is only an IPv4 protocol module. This provides
the routing code, address formats, etc, and is where the netfilter hooks
are called from. The protocol module is in the
core/<protocol>/ directory.
The netfilter code under test. This is imported from an external
kernel tree into the netfilter/ directory. Also, the
.config and Makefile files are
copied to allow building.
These tools add functionality to the user interface. Each tool
provides one or more commands, such as ifconfig or
route. Tools can be added by adding new files into
the tools/ directory.
First, the netfilter code needs to be configured - the source directory
is specified to ./configure, using the
[--kerneldir] flag. By default, it is set to the current
kernel's build directory:
/lib/modules/$(uname -r)/build
If you are reusing your source tree, it is recommended you do a
make distclean before running ./configure.
|
Note |
|---|---|
|
At present, the imported .config file is not used - instead, the
packaged file |
Next, the simulator can be compiled with make.
Because the simulator is in development, the kernel environment may not yet provide all functionality required to the imported netfilter code. If there are any errors during compilation, email me.
|
Note |
|---|---|
|
Each build of the simulator is dependent on the netfilter code under test; the binary is not 'installed' as would be the norm with other applications. |
Run the simulator with ./simulator.
The simulator will start, initialising the following items (in order):
simulator core
external message interface
user interface
logging system
user interface tools
kernel environment
protocol module
netfilter testing code
During initialisation (or at any other time), calls to
printk() will be shown, as would occur on boot or
kernel module loading.
By default, the simulator is started with the three interfaces (eth0, eth1 and lo) interface configuration:
lo addr: 127.0.0.1 mask: 255.0.0.0 bcast: 127.255.255.255 RX packets: 0 bytes: 0 TX packets: 0 bytes: 0 eth0 addr: 192.168.0.1 mask: 255.255.255.0 bcast: 192.168.0.255 RX packets: 0 bytes: 0 TX packets: 0 bytes: 0 eth1 addr: 192.168.1.1 mask: 255.255.255.0 bcast: 192.168.1.255 RX packets: 0 bytes: 0 TX packets: 0 bytes: 0
In short - two ethernet interfaces on different networks, and a loopback interface.
Routing tables are initialised also, with the appropriate network available from each device:
network iface 127.0.0.0/8 lo 192.168.0.0/24 eth0 192.168.1.0/24 eth1
The --failtest option runs the script,
but deliberately inserts failures (currently allocation failures
and user-copying failures). At each potential failure, both
failure and success will be simulated: where we fail, the script
will no doubt fail later which is OK, but we ensure that the
entire system doesn't fall over or leak memory. If an error is
found, the path of successes/failures will be printed out,
which can be replayed using --failpath
Given a failure path, (from --failtest), this will
replay the sequence of sucesses/failures, allowing debugging. The input
should be the same as the original which caused the failure.
This testing can be slow, but allows for testing of failure paths which would otherwise be very difficult to test automatically.
Sometimes code deliberately ignores the failures of a certain function. This suppresses complaints about that for any functions containing this name.
nfsim will echo each command before it is executed. Useful when commands are read from a file
Causes nfsim to reduce its output to the minimum possible - no prompt is displayed, and most warning messages are suppressed
If --exit is specified, nfsim will exit (with a
non-zero error code) on the first script error it encounters (eg an
expect command does not match). This is useful when nfsim is invoked as a
non-interactive script
Usually all netfilter modules are loaded on startup, this option will suppress this behaviour. To load modules after invoking nfsim with this argument, use the insmod command
Causes nfsim to print its command line arguments and then exit
Prior to 2.6.10, there were many bugs involving nonlinear packets. Suppressing them allows testing for other bugs.
Inline help is available on most commands, and is reproduced here.
Displays general help, or help for a specified command
help [command]
With no arguments, help will show general system help, and list the available commands. If an argument is specified, then help will show help for that command, if available.
Exit the simulator
exit
quit
The exit and quit commands are synonomous. They both exit the simulator.
Manage logging settings
log [ = | + | - ] {type, ...}
Each log message is classified into one of the following types:
Kernel messages (including printk() calls)
Messages from the user interface (deprecated?)
Routing information
Information from the protocol (IPv4) layer
Information about packet movements (including netfilter hook results)
The log command allows you to select which messages are displayed. By default, all messages will be shown.
If the types argument is prefixed with a
+, - or = character, those types will be added, removed or set as the
current types of messages to be logged (repectively). If none of these
characters is specified, = is assumed (the types are set to only those
specified)
Messages generated as a result of user input are always logged.
Catch logging information for automated testing
expect
expect [!] {command} {pattern}
expect will set up a set of patterns to expect in logging messages for a particular command. If that command finishes without matching the specified pattern, the simulator will exit with a non-zero return value. After the command is run, all expectations on that command are deleted.
expect with no arguments will print out the current list of expectations, as a command and a pattern.
expect command pattern
will expect the specified pattern to occur the next time
command is invoked. If an '!' appears before
the command, then the expectation is negated - if the pattern appears in
the output, then the simulator will exit with an error
The pattern itself is similar to a simple shell wildcard, except whitespace is loosely matched. The * character will match any a string of any length.
Run the iptables command on the simulator
iptables [options]
The external iptables binary will be
invoked, with its operations redirected to the simulator. If the
NFSIM_IPTABLES_PREFIX environment variable is set,
the command in that directory will be executed (useful for
testing specific variants). Otherwise, the current
PATH is searched, then
/usr/local/sbin, /sbin,
/usr/sbin.
You do not need to be root to use iptables in this way. If the iptables command fails, that will be reported as
iptables: command failed
; with the [-e] to nfsim, nfsim will abort.
Manipulate the IPv4 routing table.
route [print]route add {
(1) network
} {interface}route del {
(1) network
}
(1) {{
ip_addr /
netmask_bits
} | {
ip_addr mask
netmask
}}
route with no arguments, or a single argument "print" will display the current routing table.
route del network will
delete the route to the sepecified network.
route add network interface
will add a route to the sepecified network, via the specified
interface.
Query or advance the kernel time
time
time {time} {+advance}
time {+infinity}
time with no arguments will show the current kernel time (in seconds).
time time will set the
kernel time to time - which must be greater
than the current time value. time
+advance will increment the time by
the value of advance, which must be positive.
When the kernel time is incremented, kernel timers may be triggered. If so, the simulator will show logging messages:
> time +30 running timer to ip_conntrack_core.c:init_conntrack() >
The time
+infinity command will
forward time past all the currently pending timers, leaving a
clean slate.
Insert simple netfilter hooks
hook
hook {hookname} {priority} {verdict}
The hook command allows new hooks to be inserted. These hooks will
simply return the specified verdict for all
packets.
All hooks are cleared after the next gen_ip command.
The hookname argument specifies which list
to insert the hook into. One of
IP_PRE_ROUTING, IP_INPUT, IP_FORWARD, IP_OUTPUT, IP_POST_ROUTING
The priority argument specifies the
priority of the hook, from INT_MIN to
INT_MAX.
The verdict argument specifies the
verdict to return from the hook itself:
ACCEPT, DROP, QUEUE, STOLEN, REPEAT
Generate IP packets and send through the protocol layers
gen_ip [IF=interface] [FRAG=] [DF] [MF] [CE] [MAC=ethaddr] [TOS=diffserv field] [TTL=ttl] {src} {dest} {datalen} {protocol} {protocol-opts}
gen_ip [options] {src} {dest} {datalen}
TCP
{srcpt} {dstpt} {flags} [SEQ=seqnum] [ACK=seqnum] [WIN=wsize] [OPT=options] [DATA args]
gen_ip [options] {src} {dest} {datalen}
UDP
{srcpt} {dstpt}
gen_ip [options] {src} {dest} {datalen}
ICMP
{type} {code} [id] [sequence]
gen_ip will construct an IP packet and send it on the specified interface, or if no interface is specified, the packet will be sent from as if from the local host.
If [protocol] is TCP, then the ports and TCP flags must be specified. The format of the [flags] is 'SYN/ACK' etc. The format of the [OPT=] option is a comma-separated list of numbers, and there must be a multiple of four of them, such as:
1,2,3,4
. If DATA is specified, then the data of the packet will be populated with the rest of the arguments, with escapes for \n, \r, \t and \0.
If [protocol] is UDP then the ports must be specified.
If [protocol] is ICMP then the type and code must be specified: the id and sequence fields are optional.
Displays and set /proc filesystem information
proc
proc {cat} {file}
proc {write} {file} {args...}
proc with no arguments will print the heirachy of the currently-registered proc files.
proc cat file will
display the contents of the proc file specified.
proc write file args.. will write args to the proc file.
Generate an ICMP error packet and send it through the protocol layers
gen_err [IF=interface] [LEN=length] {srcip} {icmp-type} {icmp-code} {gen-ip-options}
gen_err will use gen_ip to construct an IP packet, generate an ICMP error for that packet, as would be generated by a host at [srcip], then send it on the specified interface. If no interface is specified, the packet will be sent from as if from the local host. The [length] is the number of bytes to include from the original packet, default 28.
See gen_ip for how to specify the options for the initial packet.
Print out the kernel configuration used to build nfsim
config
This command is mainly used for scripts to determine what modules and/or features are available.
Report information about all system calls and user-kernel copies
strace
strace {off}
strace will report all system calls from iptables, and all copy_to_user and copy_from_user results. This is useful for debugging iptables, and checking that the kernel is returning the right error values.
Tracing will continue until the [off] argument is given.
Administer the simulated network interfaces. Interfaces can be brought up or down, or reconfigured while up.
At present, interfaces can be assigned only one interface (this limitiation is due solely to the ifconfig tool; the simulated kernel environment can handle multiple interface addresses)
ifconfig [interface]
ifconfig {interface} {
(1) if_options
}
ifconfig {interface} {
(1) if_options
} up
ifconfig {interface} down
(1) {{
ip_addr
/
netmask_bits
} | {
ip_addr mask
netmask
}} {broadcast_addr}
If ifconfig is invoked with no arguments, it will show the configuration of all devices. If a single argument is specified, it is assumed to be a device name, and the configuration for that device is displayed.
To bring a new interface up, specify the interface name and a network specification:
ifconfig eth2 10.0.0.1 8 10.255.255.255 up
Alternatively, the netmask can be specified in dotted-decimal notation. This command is equivalent to the previous:
ifconfig eth2 10.0.0.1 mask 255.0.0.0 10.255.255.255 up
Interfaces can be brought down with the command
ifconfig {interface} down
To reconfigure an interface, use the syntax
ifconfig {device} [ip_addr
[netmask_bits
[broadcast_addr]
]
]
Manage the userspace packet queue
queuequeue {verdict} [id]
When a netfitler hook returns NF_QUEUE, it is
stored within the simulator's own packet queue. When a packet enters this
queue, a logging message is printed:
> gen_ip IF=eth0 192.168.0.2 192.168.0.1 50 udp 10 20
rcv:eth0
queue:added {IPv4 192.168.0.2 192.168.0.1 50 17 10 20}
The packet is assigned a unique id number (monotonically increasing over the life of the simulator session), which can be used to reinject packets in any order
The contents of the packet queue can be printed with the queue command with no arguments. Each packet in the queue is prefixed with its id:
> queue
0: {IPv4 192.168.0.2 192.168.0.1 50 17 10 20}
1: {IPv4 192.168.1.10 192.168.0.1 50 6 1025 80 SYN}
To reinject a packet from the queue, specify a verdict (one of the
netfilter hook return values, for example NF_ACCEPT),
and optionally, the id of the packet. If no id is specified, the packet
at the front of the queue is reinjected.
Displays version information about the kernel simulator.
version
version kernel
The version displays information about the nfsim version and the version of the kernel it's simulating. With the [kernel] argument, it only displays the version of the kernel (useful for testsuites).
List modules (both available and loaded)
lsmod
lsmod lists all the loaded modules, one to a line.
Insert a module
insmod [modulename]
insmod {-a}
insmod loads a module, or all modules. The caller must ensure that any other module this one expects are already loaded.
Remove a module
rmmod
insmod {-a}
rmmod removes a loaded module. The caller must ensure that any module which is using this is already unloaded.
Generate TCP session
tcpsession {OPEN} {src} {dest} {srcpt} {dstpt}
tcpsession {OPEN} {src} {dest} {srcpt} {dstpt} {reply-src} {reply-dest} {reply-srcpt} {reply-dstpt}
tcpsession {DATA} {direction} {args}
tcpsession {LENCHANGE} {number}
tcpsession {CLOSE} {direction}
tcpsession {RESET} {direction}
tcpsession {ABANDON}
tcpsession is shorthand for creating a TCP session using multiple gen_ip commands. The four-argument [OPEN] form creates a simple connection (no NAT expected), the eight-argument form creates a connection which might be NATted. You can only have one open connection at a time: this is used for all operations. Sequence numbers will be 1001 for the sender, and 2001 for the recipient. Initial packet comes in eth0, replies come in eth1.
The [DATA] form sends data on the current connection, in a single packet (and sends an ACK in reply). [direction] is either 'original' or 'reply'.
The [LENCHANGE] command sets expected change in length of the next packet. It is used to tell tcpsession that the next packet is going to expected to be shortened or lengthened.
The [CLOSE] form sends the required FIN and ACK packets to close the connection: [direction] specifies who initiates the close.
The [ABANDON] argument causes the connection to simply be forgotten, so you can open a new one.
Displays information about the kernel simulator.
info {subject}
The subject argument to
info must be one of the following:
displays the currently registered netfilter hooks
displays the currently registered netfilter sockopts
shows the current kernel timers - when each is set to expire, as well as the function where the timer was registered.
the contents of the kernel routing cache
show how many failure points have been passed: with --failtest, each of these alternatives would be attempted.