SCENARIO_GUIDE.rst 10.5 KB
Newer Older
1 2 3
Writing scenarios
=================

4 5 6 7 8
A scenario has three parts - a configuration in the header, description of the environment and sequential steps.
The environment is described as a set of mock answers valid within a time range using **RANGE** keyword.
A mocked DNS message is represented by an **ENTRY** block and may contain patterns for matching,
DNS header and records in appropriate sections, it looks similar to a dig(1) output which makes
generating test cases easier.
9

10 11
Each **STEP** keyword then describes a single step in the test scenario. 
Lines started with semicolon or hash are treated as comments.
12 13 14 15 16

**Configuration**

Configuration part is a list of "key : value" pairs, one pair per line.
Configuration have no explicit start, it's assumed it starts immediately at
17
scenario file begin.
18

19 20 21 22
- **do-not-query-localhost:** : on

  value "on" means that queries cannot be sent to 127.0.0.1/8 or ::1/128 addresses.

23
- **query-minimization** : on
24 25 26

  value "on" means query minimization algorithm will be used; any other value
  means query minimization algorithm will not be used.
27

28
- **stub-addr** : ipv4-addr
29

30 31 32
  address, which will be listened by Python fake dns server immediately after startup.
  When configuration files will be generated by Jinja2 engine, ``ROOT_ADDR`` template 
  variable will be replaced by this address.
Grigorii Demidov's avatar
Grigorii Demidov committed
33 34
- **thrust-anchor** : ``string value``

Grigorii Demidov's avatar
Grigorii Demidov committed
35 36
  Trust anchor, can be used for DNSSEC-related scenarios
- **val-override-timestamp** : ``string value``
Grigorii Demidov's avatar
Grigorii Demidov committed
37 38

  POSIX timestamp; the system time will be reported to binary under the test
Grigorii Demidov's avatar
Grigorii Demidov committed
39 40 41 42
- **val-override-date** : ``string value``

  the system time to be reported to binary under the test; ``string value`` has format
  ``YYYYMMDDHHMMSS``, so ``20120420235959`` means ``Fri Apr 20 23:59:59 2012``
Grigorii Demidov's avatar
Grigorii Demidov committed
43

44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
- **features** : jinja2_var1=v1;jinja2_var2=v2;...;jinja2_varN=vN

  semicolon-separated list of key=value pairs, all keys and values are strings;
  contains user-defined jinja2 template variables which can be used for improving
  flexibility of configuration generation process; when variable value is not needed,
  it can be omitted as well as equal sign; **features** configuration key can be
  repeated, so

  ::

      **features** : jinja2_var1=v1;jinja2_var2=v2;...;jinja2_varN=vN

  equal to

  ::

      **features** : jinja2_var1=v1
      **features** : jinja2_var2=v2
      ...
      **features** : jinja2_varN=vN

Grigorii Demidov's avatar
Grigorii Demidov committed
65 66 67 68 69 70 71 72 73 74 75
- **feature-list** : jinja_var=value

  single key=value pair; key and value are strings; similar to **features**, it contains
  user-defined jinja2 template variable, but difference is that jinja_var behaves as list 
  of values; each **feature-list** row appends ``v`` to the list named ``jinja_var``; 
  you can use loops in your jinja template to iterate through it

  ::
  
      feature-list: policy=policy:add(policy.suffix(policy.DENY, {todname('example.com')}))
      feature-list: policy=policy:add(policy.all(policy.FORWARD('2001:DB8::1')))
76 77 78

Example
::
79

80 81 82
    ; config options
	    query-minimization: on
	    stub-addr: 193.0.14.129 	# K.ROOT-SERVERS.NET.
Grigorii Demidov's avatar
Grigorii Demidov committed
83 84
	    trust-anchor: ". 3600 IN DS 17272 13 4 B87AD8C76DC2244E7AA57285057BF533F2E248CC8D7E1A071D8A3837A711A5EA705C4707E6E8911DA653BE1AE019927B"
	    val-override-date: "1442323400"
85 86
            features: dns64
            features: dns64_prefix = fe80::21b:aabb:0:0
87 88 89 90
    CONFIG_END

**Scenario**

91 92
Scenario part starts with **SCENARIO_BEGIN** and ends with **SCENARIO_END** statements.
**SCENARIO_BEGIN** can be followed by scenario description.
93 94 95

Example
::
96

97 98 99 100 101 102
    SCENARIO_BEGIN Test basic query minimization www.example.com.
    ...
    SCENARIO_END

**RANGE datablock**

103
**RANGE** datablock starts with **RANGE_BEGIN** and ends with **RANGE_END** statements.
104 105 106 107 108
This datablock contains data, used Python fake dns server to make answers to 
binary's under test queries. 

Format: 
::
109

110
    RANGE_BEGIN n1 n2
111 112 113
        ADDRESS addr1
        ADDRESS addr2
        ADDRESS addr3
114 115 116
    ...
    RANGE_END

117
- **n1** and **n2** respectively minimal or maximal step ids (see below) to which this  
118
  range can be applied. 
119
- **addr** - IP address for which **RANGE** datablock is prepared; this statement can be omitted or repeated multiple times.
120 121 122 123 124

Datablock will be used for fetching reply to query only for these steps, whose identificators greater then or equal n1 and
lesser then or equal n2. Also one of the next condition must be met : 

- addr is not set
125 126
- addr is set, and query is directed to one of listed addresses
- address, to which query is directed, can not be found within the range addresses lists for whole scenario
127 128 129

**STEP datablock**

130 131
**STEP** datablock starts with **STEP** statement and continues until to next **STEP**,
**RANGE** or **END_SCENARIO** statement. This datablock defines action will be taken by 
132 133 134 135 136
testing environment - send next query to binary under test, send reply to binary
under test, set faked system time or check the last answer. 

Format
::
137

138 139
   STEP id type [additional data]

140
- **id** - step identificator, positive integer value; all steps must have 
141
  different id's. This value used within RANGE datablock, see above.
142
- **type** - step type; can be **QUERY** | **REPLY** | **CHECK_ANSWER** | **TIME_PASSES ELAPSE** <**TIMESTAMP**>
143
  
144 145 146
  - **QUERY** - at this step new query must be sent
  - **REPLY** - send answer to last query; steps of this type fired when eligible 
    **RANGE** datablock can not be found
147 148
  - **CHECK_ANSWER** - last received answer must be checked; this step can have additional fields **RETRY** = `integer value` **PAUSE** = `float value` **NEXT** = `integer value`. This additional values are intended to ensure error recovery possibility. When answer checking failed, is possible to take    step with predefined step id. For example, **STEP CHECK_ANSWER RETRY** = `3` **PAUSE** = `0.5` **NEXT** = `10` means that when current step fails, then step with id = 10 must be taken after pause. Pause duration is 0.5 seconds. Maximal number of retries is 3. When maximal number of retries is reached, scenario fails.
  - **TIME_PASSES ELAPSE** - new time must be set for binary under test; **TIMESTAMP** - POSIX timemestamp, it will be added to current system time.
149 150 151 152


**ENTRY**

153
**ENTRY** is an basic informational block, it has a DNS-message based structure. 
154
It contains all necessary data to perform action for which it was intended.
155
Block starts with **ENTRY_BEGIN** and ends with **ENTRY_END** statements.
156 157 158

Format
::
159

160 161 162 163 164 165 166 167 168 169
    ENTRY_BEGIN
    MATCH <field list>
    ADJUST <field list>
    REPLY <flags>
    SECTION <type>
       ...
    RAW
       ...
    ENTRY_END

170
- **MATCH** <field list> - space-separated list of **ENTRY** block elements to be compared
171 172 173 174
  with elements of incoming query (answer); when all elements matches, this entry 
  block will be used, otherwise next entry will be analyzed.
  <field list> can contain values :
  
175 176
  - **opcode**     - check if the incominq query is a standard query (**OPCODE** is 0) 
  - **qtype**      - check if **QTYPE** fields of both question sections are equal
177 178
  - **qname**      - check if domain name (**QNAME**) fields of question sections are equal (case insensitive)
  - **qcase**      - check if domain name (**QNAME**) fields of question sections are equal (case sensitive)
179
  - **subdomain**  - check if domain from question section of incoming query (answer) 
180
    is a subdomain of domain from question section of this **ENTRY** block (case insensitive)
181 182
  - **flags**      - check if sets of dns flags (**QR** **AA** **TC** **RD** **RA**) are equal
  - **rcode**      - check if response codes are equal
183 184 185 186
  - **question**,
  - **answer**,
  - **authority**,
  - **additional** - check if lists of RR sets for question,answer,authority and 
187
    additional section respectively is equal
188
  - **all**        - check if sets of dns flags are equal, rcodes and all sections presented
189 190 191
    in entry are equal to ones in incoming query (answer); incoming query 
    (answer) can contain some extra sections which will not be compared
    
192
- **ADJUST** <field list> - when **ENTRY** block is used as a pattern to prepare answer
193 194 195
  to incoming query, it must be preprocessed; values in <field list> defines
  actions will be taken:

196
  - **copy_id**    - query id and domain name (question section QNAME field) only 
197
    will be copied from incoming message
198
  - **copy_query** - whole question section will be copied from incoming message
199

200
- **REPLY** <flags> - space-separated list of flags will be set in reply values
201 202
  can be used:

203 204 205 206
  - **QR**, **AA**, **TC**, **RD**, **RA** - i.e. standard dns flags
  - **NOERROR**, **FORMERR**, **SERVFAIL**, **NXDOMAIN**, **NOTIMP**, **REFUSED**, **YXDOMAIN**, **YXRRSET**, 
    **NXRRSET**, **NOTAUTH**, **NOTZONE**, **BADVERS** - standard rcodes
  - **DO** - enable 'DNSSEC desired' flag
207
              
208 209
- **SECTION** <type> - defines section of dns message, so <type> can be equal to 
  **QUESTION**, **ANSWER**, **AUTHORITY** or **ADDITIONAL** each section contains rr sets like 
210
  standard dns message
211

212 213
Example
::
214

215 216 217 218 219 220 221 222 223
  SECTION QUESTION
  www.example.com.	IN A
  SECTION ANSWER
  www.example.com.	IN A	10.20.30.40
  SECTION AUTHORITY
  example.com.	IN NS	ns.example.com.
  SECTION ADDITIONAL
  ns.example.com.	IN A	1.2.3.4

224
- **RAW** - section used to sending raw dns messages. Contains a single-line data 
225 226 227 228 229
  interpreted as hexadecimal string. This string will be sent to binary under 
  test without any changes. Raw messages used to sending badly formed queries
  to binary under test. Queries assumed not be answered, so no waiting for answer
  is performed.Main goal of this behavior is to check if binary under test is 
  able to serve valid queries after getting of series badly formed packets. 
230 231 232 233
  So using **RAW** section in conjunction of other sections  is meaningless. 
  All sections other than **RAW** will be ignored. Also, **ENTRY** datablock can contain 
  only one **RAW** section.

234 235
Example
::
236

237 238 239
  RAW
      b5c9ca3d50104320f4120000000000000000

240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255
Writing load tests 
------------------

The test harness support a simple benchmarking steps for writing load tests. Note that this is to
test how the subject behaves under load, not for comparative benchmarking.

::

	STEP REPLAY [query_count]
	<qname> [qclass] <qtype>
	...

This replays the list of queries described below replay step repetitively to the subject until ``query_count`` is reached.
Following example replays 1000 queries (500 times each).

::
Štěpán Balážik's avatar
Štěpán Balážik committed
256

257 258 259 260
	STEP REPLAY 1000
	example.com A
	www.example.com AAAA

261
.. tip:: Define ``VERBOSE=20`` environment variable to see benchmarking results (queries sent, received and response rate).
262

263 264 265 266
`SCRIPT EXAMPLE`_

.. _`SCRIPT EXAMPLE`: https://gitlab.labs.nic.cz/knot/deckard/blob/master/SCENARIO_EXAMPLE.rst