deckard issueshttps://gitlab.nic.cz/knot/deckard/-/issues2022-08-29T16:32:09+02:00https://gitlab.nic.cz/knot/deckard/-/issues/74Add missing dependecy to readme.md2022-08-29T16:32:09+02:00AureliaAdd missing dependecy to readme.mdOn Ubuntu 22.04.1 LTS I needed to install Ubuntu [dpkt](https://dpkt.readthedocs.io/en/latest/)
I can open the related MR and update the readme.On Ubuntu 22.04.1 LTS I needed to install Ubuntu [dpkt](https://dpkt.readthedocs.io/en/latest/)
I can open the related MR and update the readme.https://gitlab.nic.cz/knot/deckard/-/issues/73Update Readme.md with faketime link2022-08-29T15:56:29+02:00AureliaUpdate Readme.md with faketime linkIn the readme, [here](https://gitlab.nic.cz/knot/deckard/-/blob/master/README.rst?plain=1#L37), the actual link to faketime is missing at the end of the file.In the readme, [here](https://gitlab.nic.cz/knot/deckard/-/blob/master/README.rst?plain=1#L37), the actual link to faketime is missing at the end of the file.https://gitlab.nic.cz/knot/deckard/-/issues/72bind9 - many test cases end with 'unsupported as of yet'2022-03-24T11:11:45+01:00perlangbind9 - many test cases end with 'unsupported as of yet'I have tried it with the newest version BIND-9.18.0.
The BIND-9.18.0 is compiled with '--enable-full-report' '--enable-dnstap' '--without-jemalloc',
by gcc 8.5.0.
I get the followed summary result,
`====== 251 failed, 30 passed, 99 sk...I have tried it with the newest version BIND-9.18.0.
The BIND-9.18.0 is compiled with '--enable-full-report' '--enable-dnstap' '--without-jemalloc',
by gcc 8.5.0.
I get the followed summary result,
`====== 251 failed, 30 passed, 99 skipped, 1 warning in 377.15s (0:06:17) =======`
about 171 failed cases have the wrong syntax configure file named.conf which
include a line like "unsupported as of yet" on line 20.
my environment is RockyLinux 8.5,
the libfaketime is from epel, others is from rocky's repo,
and the python modules are installed via
`# pip3 install -r requirements`
python version is 3.9.6 (from rocky).https://gitlab.nic.cz/knot/deckard/-/issues/69Introduce a new format to describe (not only) Deckard scenarios2021-03-18T12:06:32+01:00Štěpán BalážikIntroduce a new format to describe (not only) Deckard scenariosI'll summarize reasons, caveats and discussion points of this in separate threads in this issue.I'll summarize reasons, caveats and discussion points of this in separate threads in this issue.Štěpán BalážikŠtěpán Balážikhttps://gitlab.nic.cz/knot/deckard/-/issues/68investigate random fails in knot-resolver CI2020-12-18T14:08:47+01:00Štěpán Balážikinvestigate random fails in knot-resolver CIThe `deckard` job often fails in the CI [like this](https://gitlab.nic.cz/knot/knot-resolver/-/jobs/472294) with `E RuntimeError: Server took too long to respond` and other random fail which in the current setup will trigger a rerun of...The `deckard` job often fails in the CI [like this](https://gitlab.nic.cz/knot/knot-resolver/-/jobs/472294) with `E RuntimeError: Server took too long to respond` and other random fail which in the current setup will trigger a rerun of the whole test suite.
Investigate the cause of the random errors, if that becomes too hard, consider adding some logic to rerun only the randomly failed tests (there is some retry logic in Deckard already, so maybe use that). Alternatively, making the timeout for queries longer might help as well.https://gitlab.nic.cz/knot/deckard/-/issues/67extension for NSEC(3) range handling in scenarios2022-02-07T18:19:54+01:00Petr Špačekextension for NSEC(3) range handling in scenariosCurrently RPL format allows only exact matches on qname and/or qtype or subdomain match. This leads to significant duplication of answers with NSEC/NSEC3 replies because we have to define several exact match entries which ultimately prov...Currently RPL format allows only exact matches on qname and/or qtype or subdomain match. This leads to significant duplication of answers with NSEC/NSEC3 replies because we have to define several exact match entries which ultimately provide the same set of NSEC/NSEC3 records.
Maybe we could implement something like `MATCH nsec` or `MATCH nsec3` and put a NSEC(3) record into QUESTION section on the entry (or elsewhere). Then Deckard would use the entry only if it belongs to the given range.https://gitlab.nic.cz/knot/deckard/-/issues/66testserver does not respect EDNS buffer size2020-10-21T21:29:06+02:00Petr Špačektestserver does not respect EDNS buffer sizeTestserver currently hardcodes EDNS version 0 buffer size 4096 and ignores EDNS parameters sent by client. This leads to situation where testserver might send back oversized UDP response which is later refused by client.
We need to disc...Testserver currently hardcodes EDNS version 0 buffer size 4096 and ignores EDNS parameters sent by client. This leads to situation where testserver might send back oversized UDP response which is later refused by client.
We need to discuss what to do with that. Current behavior breaks tests if resolver under test is using different value than 4096 B.
Options:
a) Add ability to specify alternative ENTRY when first matching ENTRY does not fit into EDNS buffer. This is probably too verbose because in vast majority of cases we want to send back empty response with TC bit and let client fall back to TCP.
b) Detect that current message size if bigger than min(client, Deckard) values and automatically generate response with TC=1. Do we need ability to override this automatic behavior?
TODO: Check if queries over TCP actually work with answers bigger than EDNS buffer size.https://gitlab.nic.cz/knot/deckard/-/issues/65all recently imported tests from testbound might be broken (i.e. not test any...2021-01-25T20:06:33+01:00Štěpán Balážikall recently imported tests from testbound might be broken (i.e. not test anything)The following discussion from !153 should be addressed:
> Procedure for generating seems flawed:
>
> Upstream [black_prime.rpl](https://github.com/NLnetLabs/unbound/blob/68c57314c4e02079569ed697b9273fa9faaafeb6/testdata/black_prime.rpl...The following discussion from !153 should be addressed:
> Procedure for generating seems flawed:
>
> Upstream [black_prime.rpl](https://github.com/NLnetLabs/unbound/blob/68c57314c4e02079569ed697b9273fa9faaafeb6/testdata/black_prime.rpl) has intentionally expired RRSIGs for the whole `example.com` in `RANGE` with `ADDRESS 1.2.3.4` but the generator puts valid signatures in the resulting [file](https://gitlab.nic.cz/knot/deckard/-/blob/98bd7a71f24e76cc278593a42bfe07b3fac7b6d5/sets/resolver/black_prime.rpl).
>
> This means that any of the tests merged in !73, !141, !146 and !148 might actually not test anything.
>
> I'll go through the `black_*` tests manually as there the mistakes are obvious; others will have to wait for a fix for the generator.https://gitlab.nic.cz/knot/deckard/-/issues/57find smarter way of describing expected DNS answers2021-02-02T11:10:21+01:00Petr Špačekfind smarter way of describing expected DNS answersProblem
=======
Currently `CHECK_ANSWER` step takes DNS answer message and matches specified fields against a hardcoded DNS message in RPL file. This is especially problem for answers in border cases where exact content is not 100 % spec...Problem
=======
Currently `CHECK_ANSWER` step takes DNS answer message and matches specified fields against a hardcoded DNS message in RPL file. This is especially problem for answers in border cases where exact content is not 100 % specified in the DNS protocol or depends on content of DNS cache etc. E.g. server might add just subset of glue addresses into additional section, or it might add NS records to authority section (i.e. non-minimal answers) etc. This is impossible to represent in the current RPL format.
Idea
====
Mark RRs in `RANGE` sections with "permissibility" for different sections in answers. E.g.:
- bogus RR would be marked "no section unless CD bit is on"
- NS records would be marked "optional in authority section unless QTYPE=NS"
etc.
This should lead to tests which are not too sensitive to random RR selection or minor changes in selection on the responder side.https://gitlab.nic.cz/knot/deckard/-/issues/56randomize order of steps in idempotent tests2020-08-07T11:07:59+02:00Petr Špačekrandomize order of steps in idempotent testsProblem
=======
Current RPL tests are quite static and often do not test various combinations of cache state.
Idea
====
Tests which use one static set of answers (i.e. all QUERY/CHECK_ANSWER steps belong to single numeric range which co...Problem
=======
Current RPL tests are quite static and often do not test various combinations of cache state.
Idea
====
Tests which use one static set of answers (i.e. all QUERY/CHECK_ANSWER steps belong to single numeric range which covers all RANGE_BEGIN entries) should produce the same DNS answers even if QUERY steps are run in different order.
For these "idempotent" tests Deckard could:
- [ ] randomize ordering of QUERY+CHECK pairs
- [ ] randomize delay between QUERY steps (based on TTLs in the test?) or maybe in combination with randomizing TTLs
- [ ] continue testing after a failed step? run the scenario again and skip the failed test?
- [ ] run steps in isolation?https://gitlab.nic.cz/knot/deckard/-/issues/55investigate warning: unknown key log_print in deckard_pytest.ini2020-08-06T13:09:59+02:00Petr Špačekinvestigate warning: unknown key log_print in deckard_pytest.iniCommit 5708002a29f3824c618fde02224edba89aa63c2f by @sbalazik introduced key `log_print` into `deckard_pytest.ini` but apparently it is not implemented in py.test 6.0.0.
Please investigate if the key is needed or if it can be safely remo...Commit 5708002a29f3824c618fde02224edba89aa63c2f by @sbalazik introduced key `log_print` into `deckard_pytest.ini` but apparently it is not implemented in py.test 6.0.0.
Please investigate if the key is needed or if it can be safely removed.Štěpán BalážikŠtěpán Balážikhttps://gitlab.nic.cz/knot/deckard/-/issues/53investigate dumpcap problems/replacement2020-08-05T15:11:24+02:00Petr Špačekinvestigate dumpcap problems/replacementProblem: `dumpcap` on my Arch machine sometimes truncates PCAP file and I do not see how it could happen.
`tcpdump --immediate-mode` on my machine works well so I wrote branch https://gitlab.nic.cz/knot/deckard/-/tree/dumpcap_replacemen...Problem: `dumpcap` on my Arch machine sometimes truncates PCAP file and I do not see how it could happen.
`tcpdump --immediate-mode` on my machine works well so I wrote branch https://gitlab.nic.cz/knot/deckard/-/tree/dumpcap_replacement to replace dumpcap with tcpdump ... but somehow magically broke inside Docker *only on CI machines*.
On Ubuntu CI machines the tcpdump gets stuck and does not create the output file. On my Arch machine it works *even with the same Docker image*.
WTF?
dumpcap versions which cause problem on my Arch machine:
- wireshark-cli 3.2.5-1
- linux 5.7.12.arch1-1
- libpcap 1.9.1-2
tcpdump versions which work on my Arch machine:
- linux 5.7.12.arch1-1
- docker 1:19.03.12-2
- tcpdump inside docker:
- tcpdump 4.9.3-1~deb10u1
- libpcap0.8:amd64 1.8.1-6
tcpdump versions which do not work on my Debian CI machines:
- linux 5.4.0-42-generic #46~18.04.1-Ubuntu SMP
- docker-ce 5:19.03.12~3-0~u
- tcpdump inside docker: **same as in the case where it works**https://gitlab.nic.cz/knot/deckard/-/issues/52investigate random socket binding failures2020-07-28T13:54:00+02:00Petr Špačekinvestigate random socket binding failurespydnstest/testserver.py has workaround around line `for i in range(self.RETRIES_ON_BIND):` which attempts to work around seemingly random errors:
```
pydnstest/testserver.py:213: in start_srv
sock.bind(address)
E OSError: [Errno 99...pydnstest/testserver.py has workaround around line `for i in range(self.RETRIES_ON_BIND):` which attempts to work around seemingly random errors:
```
pydnstest/testserver.py:213: in start_srv
sock.bind(address)
E OSError: [Errno 99] Cannot assign requested address
```
As far as I can tell it happens only for IPv6, and surprisingly failing addresses were not in use before.
networking.py _add_address() already disables IPv6 Duplicate Address Detection so it should not be a problem.
I've ran out of ideas so the I'm leaving the workaround there for the time being.https://gitlab.nic.cz/knot/deckard/-/issues/46support multi-message answers (for AXFR/IXFR)2021-02-02T11:10:21+01:00Petr Špačeksupport multi-message answers (for AXFR/IXFR)https://gitlab.nic.cz/knot/deckard/-/issues/43add support for different EDNS buffer sizes2021-02-02T11:44:15+01:00Petr Špačekadd support for different EDNS buffer sizesRight now file `pydnstest/scenario.py` hardcodes EDNS buffer size to 4096 B:
```
class Entry:
def __init__(self, node):
self.message.use_edns(edns=0, payload=4096)
```
We need to make this configurable so we can write tests...Right now file `pydnstest/scenario.py` hardcodes EDNS buffer size to 4096 B:
```
class Entry:
def __init__(self, node):
self.message.use_edns(edns=0, payload=4096)
```
We need to make this configurable so we can write tests with big RRsets etc.https://gitlab.nic.cz/knot/deckard/-/issues/41error insection (chaos mode)2020-08-11T10:27:22+02:00Petr Špačekerror insection (chaos mode)Motivation: Resolver must handle random timeouts and some SERVFAILs.
Idea: Deckard might have additional mode which introduces random timeouts and/or SERVFAILs.
Details: TBDMotivation: Resolver must handle random timeouts and some SERVFAILs.
Idea: Deckard might have additional mode which introduces random timeouts and/or SERVFAILs.
Details: TBDhttps://gitlab.nic.cz/knot/deckard/-/issues/39Test suggestions2019-01-24T11:02:47+01:00Jan PavlinecTest suggestions* qname minimization - test with forwarding when ISP is messing DNS (inspiration from https://gitlab.labs.nic.cz/turris/openwrt/issues/236 )
* test TCP blocking on port 53
* Test max packet length - related to ENDS (some firewalls assume...* qname minimization - test with forwarding when ISP is messing DNS (inspiration from https://gitlab.labs.nic.cz/turris/openwrt/issues/236 )
* test TCP blocking on port 53
* Test max packet length - related to ENDS (some firewalls assume max DNS packet length 512 bytes)
* Test DNSSEC validation of wildcard record on NSEC3 signed domain (it was a bug in _Bind_ < _9.9.0_)
* Test DNS-over-TLS - It's new protocol but it sounds reasonable (from ISP as we know them :-) ) to block that. Test could be some equivalent of openssl s_client -connect ‘1.1.1.1:853’
* redirection of all packets on port 53 to recursive resolver - https://gitlab.labs.nic.cz/knot/deckard/issues/39#note_95910 Štěpán BalážikŠtěpán Balážikhttps://gitlab.nic.cz/knot/deckard/-/issues/36Possible `rplint` improvements2018-12-04T08:24:32+01:00Štěpán BalážikPossible `rplint` improvements* [ ] detect IP addresses in A/AAAA/config without corresponding RANGE
* [ ] find a way to selectively turn off checks for some RPLs
* [ ] detect files without `CHECK_ANSWER`
* [ ] detect _orphan_ `RANGE`s (not being pointed to by an...* [ ] detect IP addresses in A/AAAA/config without corresponding RANGE
* [ ] find a way to selectively turn off checks for some RPLs
* [ ] detect files without `CHECK_ANSWER`
* [ ] detect _orphan_ `RANGE`s (not being pointed to by any NS record)
* [ ] detect `ENTRY` in `RANGE` or `STEP CHECK_ANSWER` without a `MATCH` rule
* [ ] detect `ENTRY` in `RANGE` without `copy_id`
* [ ] detect more than one record in `SECTION QUESTION` (use `matchpart.check_question` for that)
* [ ] #27
* [ ] `ADJUST do_not_answer` should not be combined with other `ADJUST` parameters
* [ ] `STEP REPLY` should be flagged as non-portable (to different DNS implementations)
I will happily accept further suggestions. :top:Štěpán BalážikŠtěpán Balážikhttps://gitlab.nic.cz/knot/deckard/-/issues/34rplint: improve error message about overlaps2018-07-27T19:15:07+02:00Petr Špačekrplint: improve error message about overlapsCurrent rplint version prints error message like this:
```
line 6438 RANGE has common IPs with some previous overlapping RANGE
```
This is hard to debug for user. Please print line numbers of other overlaping RANGEs.Current rplint version prints error message like this:
```
line 6438 RANGE has common IPs with some previous overlapping RANGE
```
This is hard to debug for user. Please print line numbers of other overlaping RANGEs.https://gitlab.nic.cz/knot/deckard/-/issues/30rpl scenarios: when defaults are removed, certain tests start to fail2018-12-18T16:15:03+01:00Tomas Krizekrpl scenarios: when defaults are removed, certain tests start to failWhen defaults values are removed from scenarios, these tests start to fail:
- val_nsec3_cnametocnamewctoposwc.rpl (try to revert 7ad288a7b232b45be9dbc59eb64ea75d34aa19ab )
- iter_hint_lame.rpl (try to revert b7a7dd97bf620047d009373bab56...When defaults values are removed from scenarios, these tests start to fail:
- val_nsec3_cnametocnamewctoposwc.rpl (try to revert 7ad288a7b232b45be9dbc59eb64ea75d34aa19ab )
- iter_hint_lame.rpl (try to revert b7a7dd97bf620047d009373bab56743b7cfc2e19 )
Note: The following `sed` commands were executed on all `*.rpl` files to remove the default values:
- `/REPLY QR NOERROR$/d`
- `/REPLY NOERROR QR$/d`
- `/MATCH opcode qname qtype$/d`
- `/MATCH opcode qtype qname$/d`
- `/ADJUST copy_id$/d`
See [remove-defaults-from-sets](https://gitlab.labs.nic.cz/knot/deckard/tree/remove-defaults-from-sets) branch for commits.