Commit 52371caf authored by Grigorii Demidov's avatar Grigorii Demidov

Project description were added

parent efa7b942
Deckard
=======
**Introduction**
Deckard is intended for dns software testing in your own environment to show
that it has desired behavior. Main part of the project is the python script.
Script runs given binary as a subprocess, then sends him some prescripted queries,
compares answers with referenced data and decides whether test is successful or not.
At the moment only UDP transport is supported. All network communications are
redirected over UNIX sockets, so you don't need to get test network because dns
binaries runs locally at a kind of sandbox. When it is required by testing context,
dns binary can be forced to get faked system time. Test process flowing is guided
by scenario. Scenario is a simple text-based file, which contains queries, expected
answers and additional data. Also is possible to include raw dns packets in it.
**Requirements**
Deckard requires next software to be installed :
- Python (tested on Python 2.7)
- `DNS toolkit`_ for Python
- Jinja2_ template engine for Python
- `socket wrapper`_ library
also it depends on
- Libfaketime_ library; the libfaketime is included in ``contrib/libfaketime``
as it depends on rather latest version of it, it is automatically synchronised
with ``make``
**Compatibility**
Project has been tested with `Knot DNS Resolver`_ on Linux and MacOS
and `PowerDNS Recursor`_ on Linux only.
**Usage**
Using of Deckard consists of :
a) **Setting up socket wrapper library (cwrap)**
Detailed instructions on using cwrap you can read here_
Generally, explicit environment setup for cwrap is not required.
When cwrap environment is absent, default values will be used :
- **SOCKET_WRAPPER_DEFAULT_IFACE** = 10
- **SOCKET_WRAPPER_DIR** will be created in default temporary directory with
randomly generated name, prefixed by tmp
- **SOCKET_WRAPPER_DEBUGLEVEL** will not be set
**SOCKET_WRAPPER_DIR** also used as work directory for binary under test. When test
failed, work directory can contain useful information to analyze. For debugging
purposes sometimes might be better to use well-known location rather then
temporary directory with randomly generated name. In this case you can explicitly
set **SOCKET_WRAPPER_DIR** to any eligible value.
b) **Writing your own scenario**
See `scenario guide`_
c) **Setting up configuration for binary, intended to test.**
Generally server software can be configured by using configuration file.
It is a very convenient way, but in our case some configuration values can be
only known after the test started. To resolve this problem jinja2 templating
engine is used. You can prepare jinja2 template of configuration file. It will
be processed before test really started, then actual configuration file
will be generated at working dir.
You can use next template variables:
- **ROOT_ADDR** - address of root server. It is a IP4 address looks like 127.0.0.XXX,
where XXX is **SOCKET_WRAPPER_DEFAULT_IFACE** environment variable value. It must
be used as a entry of root hints list. When root hints resides in separated file,
this file must be edited manually. Port is not set and assumed to be equal to 53.
- **SELF_ADDR** - address, to which binary under test must be bounded. It is a IP4
address looks like 127.0.0.XXX, where XXX is **KRESD_WRAPPER_DEFAULT_IFACE** value.
Port is not set and assumed to be equal to 53.
- **NO_MINIMIZE** - 'true' of 'false', enables or disables query minimization respectively.
- **WORKING_DIR** - working directory, it is a value of **SOCKET_WRAPPER_DIR**
environment variable.
d) **Setting up your environment**
You can alter test process flow by using next environment variables :
- **TESTS** - path to scenario files; default value is **sets/resolver**
- **DAEMON** - path to binary have to be tested; default value is **kresd**
- **TEMPLATE** - jinja2 template file to generate configuration file; default value is **kresd.j2**
- **CONFIG** - name of configuration file to be generated; default value is **config**
- **ADDITIONAL** - additional parameters for binary, intended to test; not set by default
Note, that default values intended to be used with Knot DNS Resolver.
Also, **KRESD_WRAPPER_DEFAULT_IFACE** environment variable is used to set up default socket
wrapper interface for the binary under test. If not set explicitly, this default value will
be used : **KRESD_WRAPPER_DEFAULT_IFACE** = **SOCKET_WRAPPER_DEFAULT_IFACE** + 1.
Generally there are no reasons to set up this value explicitly, except better readability
of configuration script.
e) **Running.**
Execute the tests by running **make** utility.
.. code-block:: bash
make TESTS=sets/resolver DAEMON=/usr/local/bin/kresd TEMPLATE=kresd.j2 CONFIG=config
As said above, default values are set for using with Knot DNS resolver.
If Knot DNS resolver is properly installed, you should not set any parameters.
Below is a example of script, which explicitly sets environment variables and
runs tests for Knot DNS Resolver daemon.
.. code-block:: bash
#!/bin/bash
# Python fake root server will be located at 127.0.0.2:53
SOCKET_WRAPPER_DEFAULT_IFACE=2
# KRESD_WRAPPER_DEFAULT_IFACE is not set,
# so Knot DNS Resolver daemon
# will bind to 127.0.0.3:53
# directory which contains socket_wrapper UNIX sockets
# also - working directory
SOCKET_WRAPPER_DIR=~/work/kresd/wrapsock
# Path to scenario files
TESTS=sets/resolver
# Path to daemon
DAEMON=/usr/local/bin/kresd
# Template file name
TEMPLATE=kresd.j2
# Config file name
CONFIG=config
export SOCKET_WRAPPER_DEFAULT_IFACE SOCKET_WRAPPER_DIR TESTS DAEMON TEMPLATE CONFIG
make
configuration template example
::
net.listen('{{SELF_ADDR}}',53)
cache.size = 1*MB
modules = {'stats', 'block', 'hints'}
hints.root({['k.root-servers.net'] = '{{ROOT_ADDR}}'})
option('NO_MINIMIZE', {{NO_MINIMIZE}})
option('ALLOW_LOCAL', true)
-- Self-checks on globals
assert(help() ~= nil)
assert(worker.id ~= nil)
-- Self-checks on facilities
assert(cache.count() == 0)
assert(cache.stats() ~= nil)
assert(cache.backends() ~= nil)
assert(worker.stats() ~= nil)
assert(net.interfaces() ~= nil)
-- Self-checks on loaded stuff
assert(net.list()['{{SELF_ADDR}}'])
assert(#modules.list() > 0)
-- Self-check timers
ev = event.recurrent(1 * sec, function (ev) return 1 end)
event.cancel(ev)
ev = event.after(0, function (ev) return 1 end)
Below is a example of script, which tests Power DNS Recursor
.. code-block:: bash
#!/bin/bash
# Python fake root server will be located at 127.0.0.2:53
SOCKET_WRAPPER_DEFAULT_IFACE=2
# Knot DNS Resolver daemon will be bound to 127.0.0.10:53
KRESD_WRAPPER_DEFAULT_IFACE=10
# directory which contains socket_wrapper UNIX sockets
# also - working directory
SOCKET_WRAPPER_DIR=~/work/pdns/wrapsock
# Path to scenario files
TESTS=sets/pdns
# Path to daemon
DAEMON=pdns_recursor
# Template file name
TEMPLATE=recursor.j2
# Config file name
CONFIG=recursor.conf
# Additional parameter for pdns_recursor
# it means that configuration file can be found in working directory
ADDITIONAL=--config-dir=./
export SOCKET_WRAPPER_DEFAULT_IFACE SOCKET_WRAPPER_DIR TESTS DAEMON TEMPLATE CONFIG ADDITIONAL
make
configuration template example, shown only changed lines of original recursor.conf
::
...
#################################
# config-dir Location of configuration directory (recursor.conf)
#
config-dir={{WORKING_DIR}}
...
#################################
# local-address IP addresses to listen on, separated by spaces or commas. Also accepts ports.
#
local-address={{SELF_ADDR}}
...
#################################
# socket-dir Where the controlsocket will live
#
socket-dir={{WORKING_DIR}}
...
.. _`DNS toolkit`: http://www.dnspython.org/
.. _Jinja2: http://jinja.pocoo.org/
.. _`socket wrapper`: https://cwrap.org/socket_wrapper.html
.. _Libfaketime: https://github.com/wolfcw/libfaketime
.. _`Knot DNS Resolver`: https://gitlab.labs.nic.cz/knot/resolver/blob/master/README.md
.. _`PowerDNS Recursor`: https://doc.powerdns.com/md/recursor/
.. _here: https://git.samba.org/?p=socket_wrapper.git;a=blob;f=doc/socket_wrapper.1.txt;hb=HEAD
.. _`scenario guide` : https://gitlab.labs.nic.cz/knot/deckard/blob/master/SCENARIO_GUIDE.rst
DAEMON ?= $(abspath kresd)
TEMPLATE ?= kresd.j2
# Defaults
TESTS ?= sets/resolver
DAEMON ?= kresd
TEMPLATE ?= kresd.j2
CONFIG ?= config
LIBEXT := .so
PLATFORM := $(shell uname -s)
ifeq ($(PLATFORM),Darwin)
......@@ -25,7 +28,7 @@ endif
# Targets
ifeq ($(HAS_socket_wrapper), yes)
all: depend
$(preload_syms) ./deckard.py $(TESTS) $(DAEMON) $(TEMPLATE) config
$(preload_syms) ./deckard.py $(TESTS) $(DAEMON) $(TEMPLATE) $(CONFIG) $(ADDITIONAL)
depend: $(libfaketime) $(libcwrap)
else
$(error missing required socket_wrapper)
......
DNS software test harness
=========================
DNS software test harness (Deckard)
===================================
The tests depend on cwrap's `socket_wrapper`_, libfaketime_ and Python.
The libfaketime is included in ``contrib/libfaketime`` as it depends on rather latest version of it,
it is automatically synchronised with ``make``.
Project is intended for dns software testing in your own environment to show
that it has desired behavior. All network communications are redirected over
UNIX sockets, so you don't need to get test network because dns binaries runs
locally at a kind of sandbox.
Execute the tests by:
See detailed `description`_
.. code-block:: bash
.. _`description` : https://gitlab.labs.nic.cz/knot/deckard/blob/master/DECKARD.rst
$ make DAEMON=/usr/local/bin/kresd
.. todo:: Writing tests.
.. _cmocka: https://cmocka.org/
.. _`socket_wrapper`: https://cwrap.org/socket_wrapper.html
.. _libfaketime: https://cwrap.org/socket_wrapper.html
Scenario example
=================
iter_ns_badaa.rpl
::
; config options
stub-addr: 193.0.14.129 # K.ROOT-SERVERS.NET.
CONFIG_END
SCENARIO_BEGIN Test iterator with NS falsely declaring referral answer as authoritative.
; K.ROOT-SERVERS.NET.
RANGE_BEGIN 0 100
ADDRESS 193.0.14.129
ENTRY_BEGIN
MATCH opcode qtype qname
ADJUST copy_id
REPLY QR NOERROR
SECTION QUESTION
. IN NS
SECTION ANSWER
. IN NS K.ROOT-SERVERS.NET.
SECTION ADDITIONAL
K.ROOT-SERVERS.NET. IN A 193.0.14.129
ENTRY_END
ENTRY_BEGIN
MATCH opcode subdomain
ADJUST copy_id copy_query
; False declaration here
REPLY QR AA NOERROR
SECTION QUESTION
MORECOWBELL. IN A
SECTION AUTHORITY
MORECOWBELL. IN NS a.gtld-servers.net.
SECTION ADDITIONAL
a.gtld-servers.net. IN A 192.5.6.30
ENTRY_END
ENTRY_BEGIN
MATCH opcode qtype qname
ADJUST copy_id copy_query
REPLY QR NOERROR
SECTION QUESTION
a.gtld-servers.net. IN A
SECTION ANSWER
a.gtld-servers.net. IN A 192.5.6.30
ENTRY_END
ENTRY_BEGIN
MATCH opcode qtype qname
ADJUST copy_id copy_query
REPLY QR NOERROR
SECTION QUESTION
a.gtld-servers.net. IN AAAA
SECTION AUTHORITY
. SOA bla bla 1 2 3 4 5
ENTRY_END
RANGE_END
; a.gtld-servers.net.
RANGE_BEGIN 0 100
ADDRESS 192.5.6.30
ENTRY_BEGIN
MATCH opcode qtype qname
ADJUST copy_id copy_query
REPLY QR AA NOERROR
SECTION QUESTION
CATALYST.MORECOWBELL. IN A
SECTION ANSWER
CATALYST.MORECOWBELL. IN A 10.20.30.40
SECTION AUTHORITY
CATALYST.MORECOWBELL. IN NS a.gtld-servers.net.
ENTRY_END
RANGE_END
STEP 1 QUERY
ENTRY_BEGIN
REPLY RD
SECTION QUESTION
catalyst.morecowbell. IN A
ENTRY_END
; recursion happens here.
STEP 10 CHECK_ANSWER
ENTRY_BEGIN
MATCH all
REPLY QR RD RA NOERROR
SECTION QUESTION
catalyst.morecowbell. IN A
SECTION ANSWER
catalyst.morecowbell. IN A 10.20.30.40
ENTRY_END
SCENARIO_END
Execution flow :
First of all IP4 addresses, found in the script when parsing will be translated
to 127.0.0.XXX range :
- 193.0.14.129 <==> 127.0.0.2
- 192.5.6.30 <==> 127.0.0.4
- 10.20.30.40 <==> 127.0.0.5
At this example SOCKET_WRAPPED_DEFAULT_IFACE set to 2 and
KRESD_WRAPPER_DEFAULT_IFACE not set, so 127.0.0.3 occupied by Resolver. At next
steps only local addresses will be used.
Next, STEP 1 QUERY will be performed.
Python sends query to Resolver
::
id 31296
opcode QUERY
rcode NOERROR
flags RD
edns 1
eflags
payload 1280
;QUESTION
catalyst.morecowbell. IN A
;ANSWER
;AUTHORITY
;ADDITIONAL
Resolver have been configured to use address 127.0.0.2 as a root server.
So it sends query to Python fake server which listen at address 127.0.0.2
::
id 7367
opcode QUERY
rcode NOERROR
flags
edns 0
payload 1452
;QUESTION
CAtaLyST.MOreCOWBeLL. IN A
;ANSWER
;AUTHORITY
;ADDITIONAL
Python fake server starts range analyzing to make answer.
Let's look at first range
::
RANGE_BEGIN 0 100
ADDRESS 193.0.14.129
STEP ID is equal 1, so it matches the condition n1 <= step id <= n2
Next, 193.0.14.129 is mapped to 127.0.0.2.
Since query was directed to 127.0.0.2, this range will be used.
Next, Python walks through list of entries to choose eligible entry.
First entry at this range requires comparison of "opcode qtype qname" field list.
Since opcode is QUERY, first comparison is true.
Next, qtype field at question section is equal NS.
But qtype field at question section of incoming query is A.
So this comparison failed and this entry will be rejected.
Next entry requires comparison of opcode and subdomain fields.
As we seen, opcode matches.
Let's look at domain names.
ENTRY datablock:
::
SECTION QUESTION
MORECOWBELL. IN A
Incoming query :
::
;QUESTION
caTALysT.moReCoWbEll. IN A
So, subdomain matches and second entry of first range used as answer pattern.
Python fake server sends answer to Resolver :
::
id 7367
opcode QUERY
rcode NOERROR
flags QR AA
edns 0
payload 1280
;QUESTION
CAtaLyST.MOreCOWBeLL. IN A
;ANSWER
;AUTHORITY
MORECOWBELL. 3600 IN NS a.gtld-servers.net.
;ADDITIONAL
a.gtld-servers.net. 3600 IN A 127.0.0.4
Note that additional section contains local IP. Because new address is found,
Python fake server immediately starts listening on this address.
Resolver sends next query to 127.0.0.4:
::
id 58167
opcode QUERY
rcode NOERROR
flags
edns 0
payload 1452
;QUESTION
cAtaLyst.MoRECowBEll. IN A
;ANSWER
;AUTHORITY
;ADDITIONAL
Since query is directed to 127.0.0.4 which mapped to 192.5.6.30,
this range will be analyzed :
::
; a.gtld-servers.net.
RANGE_BEGIN 0 100
ADDRESS 192.5.6.30
It has a single entry, which requires "opcode qtype qname" field list to be compared.
Opcode and qtype fields are the same as fields in incoming query.
Let's compare qname.
ENTRY datablock :
::
SECTION QUESTION
CATALYST.MORECOWBELL. IN A
Incoming query :
::
;QUESTION
cAtaLyst.MoRECowBEll. IN A
So, qname also the same. All fields matches and Python server sends answer
derived from this entry :
::
id 58167
opcode QUERY
rcode NOERROR
flags QR AA
edns 0
payload 1280
;QUESTION
cAtaLyst.MoRECowBEll. IN A
;ANSWER
CATALYST.MORECOWBELL. 3600 IN A 127.0.0.5
;AUTHORITY
CATALYST.MORECOWBELL. 3600 IN NS a.gtld-servers.net.
;ADDITIONAL
Here Python found new address 127.0.0.5 and starts listening.
Next queries and answers :
query; Resolver ---> Python (127.0.0.2)
::
id 13810
opcode QUERY
rcode NOERROR
flags
edns 0
payload 1452
;QUESTION
A.gTld-serveRS.NET. IN AAAA
;ANSWER
;AUTHORITY
;ADDITIONAL
answer; Python ---> Resolver
::
id 13810
opcode QUERY
rcode NOERROR
flags QR
edns 0
payload 1280
;QUESTION
A.gTld-serveRS.NET. IN AAAA
;ANSWER
;AUTHORITY
. 3600 IN SOA bla. bla. 1 2 3 4 5
;ADDITIONAL
query; Resolver ---> Python (127.0.0.2)
::
id 20735
opcode QUERY
rcode NOERROR
flags
edns 0
payload 1452
;QUESTION
A.gTLd-seRVeRs.nEt. IN A
;ANSWER
;AUTHORITY
;ADDITIONAL
answer; Python ---> Resolver
::
id 20735
opcode QUERY
rcode NOERROR
flags QR
edns 0
payload 1280
;QUESTION
A.gTLd-seRVeRs.nEt. IN A
;ANSWER
a.gtld-servers.net. 3600 IN A 127.0.0.4
;AUTHORITY
;ADDITIONAL
query; Resolver ---> Python (127.0.0.4)
::
id 10288
opcode QUERY
rcode NOERROR
flags
edns 0
payload 1452
;QUESTION
CAtalYst.MORECowBeLl. IN A
;ANSWER
;AUTHORITY
;ADDITIONAL
answer; Python ---> Resolver
::
id 10288
opcode QUERY
rcode NOERROR
flags QR AA
edns 0
payload 1280
;QUESTION
CAtalYst.MORECowBeLl. IN A
;ANSWER
CATALYST.MORECOWBELL. 3600 IN A 127.0.0.5
;AUTHORITY
CATALYST.MORECOWBELL. 3600 IN NS a.gtld-servers.net.
;ADDITIONAL
at this point Resolver returns answer to query from STEP 1 QUERY.
::
opcode QUERY
rcode NOERROR
flags QR RD RA
edns 0
payload 1452
;QUESTION
catalyst.morecowbell. IN A
;ANSWER
catalyst.morecowbell. 3600 IN A 127.0.0.5
;AUTHORITY
;ADDITIONAL
Now STEP 10 will be performed. Is has a single entry which contains
**MATCH all** clause. **MATCH all** means set of dns flags must be equal and
all sections presented in ENTRY must be equal to ones in answer.
Incoming answer has next flags were set: **QR RD AA**. ENTRY datablock contains
**REPLY QR RD RA NOERROR** clause. As we see, flags set is equal. If we remember
that address 10.20.30.40 is mapped to 127.0.0.5, we easily can see equality of
question and answer sections of both dns messages.
So, Python got expected answer and test is passed.
Writing scenarios
=================
**Rewiev**
Scenario is a text file which must have the extension .rpl.
It consists of two sequential parts - configuration and scenario.
Configuration, which contains data with global scope, is always placed first.
Scenario follows Configuration. This part consists of sequence of datablocks
of two different types - RANGE and STEP. Generally RANGE datablock contains
data, used Python fake dns server to make answers to binary's under test
queries. And STEP datablock defines action will be taken - send next query
to binary under test, send reply to binary under test, set faked system time,
check the last answer. Each datablock must contain at least one ENTRY block
and may contain some extra data. Each ENTRY block contains header data and
at least one SECTION or RAW block. SECTION block contains Record Resource
Sets like a dns message sections. RAW contains single-line data which will be
interpreted as raw dns message. Lines started with semicolon (;) are ignored
and can be used as comments.
Notice that configuration values, header data or SECTION blocks can contain
IP addresses. Despite the fact that socket wrapper can only deal with a range
of 127.0.0.2-127.0.0.254 (or fd00::5357:5f02 - fd00::5357:5ffE) it's not
necessary to use only these ranges. Arbitrary addresses can be used. They
will be automatically translated to local addresses.
**Configuration**
Configuration part is a list of "key : value" pairs, one pair per line.
Configuration have no explicit start, it's assumed it starts immediately at
scenario file begin. It must be explicitly ended with CONFIG_END statement.
Next keys can be used
- query-minimization : on
value "on" means query minimization algorithm will be used; any other value
means query minimization algorithm will not be used.
- stub-addr : ipv4-addr
address will be translated to default local address, which will be listened
by Python fake dns server immediately after startup.
Example
::
; config options
query-minimization: on
stub-addr: 193.0.14.129 # K.ROOT-SERVERS.NET.
CONFIG_END
**Scenario**
Scenario part starts with SCENARIO_BEGIN and ends with SCENARIO_END statements.
SCENARIO_BEGIN can be followed by scenario description.
Example
::
SCENARIO_BEGIN Test basic query minimization www.example.com.
...
SCENARIO_END
**RANGE datablock**
RANGE datablock starts with RANGE_BEGIN and ends with RANGE_END statements.
This datablock contains data, used Python fake dns server to make answers to
binary's under test queries.
Format:
::
RANGE_BEGIN n1 n2
ADDRESS addr
...
RANGE_END
- n1 and n2 respectively minimal or maximal step ids (see below) to which this
range can be applied.
- addr - IP address for which RANGE datablock is prepared; this statement can be omitted.
Datablock will be used for fetching reply to query only for these steps, whose identificators greater then or equal n1 and
lesser then or equal n2. Also one of the next condition must be met :
- addr is not set
- addr is set, and query is directed to this addr
- address, to which query is directed, can not be found within the range addresses list for whole scenario
**STEP datablock**
STEP datablock starts with STEP statement and continues until to next STEP,
RANGE or END_SCENARIO statement. This datablock defines action will be taken by
testing environment - send next query to binary under test, send reply to binary
under test, set faked system time or check the last answer.
Format
::
STEP id type [additional data]
- id - step identificator, positive integer value; all steps must have
different id's. This value used within RANGE datablock, see above.
- type - step type; can be QUERY | REPLY | CHECK_ANSWER | TIME_PASSES ELAPSE <TIMESTAMP>
- QUERY - at this step new query must be sent
- REPLY - send answer to last query; steps of this type fired when eligible
RANGE datablock can not be found
- CHECK_ANSWER - last received answer must be checked
- TIME_PASSES - new time must be passed to binary under test;
TIMESTAMP - POSIX timemestamp.
**ENTRY**
ENTRY is an basic informational block, it has a DNS-message based structure.
It contains all necessary data to perform action for which it was intended.
Block starts with ENTRY_BEGIN and ends with ENTRY_END statements.
Format
::
ENTRY_BEGIN
MATCH <field list>
ADJUST <field list>
REPLY <flags>
SECTION <type>
...
RAW
...
ENTRY_END
- MATCH <field list> - space-separated list of ENTRY block elements to be compared
with elements of incoming query (answer); when all elements matches, this entry
block will be used, otherwise next entry will be analyzed.
<field list> can contain values :
- opcode - check if the incominq query is a standard query (OPCODE is 0)
- qtype - check if QTYPE fields of both question sections are equal
- qname - check if domain name (QNAME) fields of question sections are equal
- subdomain - check if domain from question section of incoming query (answer)
is a subdomain of domain from question section of this ENTRY block.
- flags - check if set of dns flags (QR AA TC RD RA) is equal
- question,
- answer,
- authority,
- additional - check if lists of RR sets for question,answer,authority and
additional section respectively is equal
- all - check if set of dns flags is equal and all sections presented
in entry are equal to ones in incoming query (answer); incoming query
(answer) can contain some extra sections which will not be compared
- ADJUST <field list> - when ENTRY block is used as a pattern to prepare answer