README.rst 2.88 KB
Newer Older
1 2
Respdiff
========
3

4 5 6
Respdiff is an abbreviation from "response differences" used as name for set of
tools allowing to gather answers to DNS queries from DNS servers and compare
them based on specified criteria.
7

8
Installation
9 10
------------

11
.. code-block:: console
12

13 14 15
   $ git clone https://gitlab.labs.nic.cz/knot/respdiff.git
   $ cd respdiff
   $ pip3 install -r requirements.txt
16

17 18 19 20
Respdiff toolchain requires **Python 3.5.2+**. There are also some Python package
dependencies that are listed in ``requirements.txt``. If you'd prefer to use
your distribution packages, please install them manually instead of using
``pip3``.
21

22 23 24
Usage
-----

25
Please note that this is very basic use-case where nothing is prepared beforehand.
26 27 28 29

.. code-block:: console

   $ ./qprep.py "${DIR}" < list_of_queries_in_text_format
30 31 32
   $ ./orchestrator.py "${DIR}"
   $ ./msgdiff.py "${DIR}"
   $ ./diffsum.py "${DIR}" > "${DIR}"/"${DIR}".txt
33

34
You can also re-run ``msgdiff.py`` and ``diffsum.py`` using different configuration.
35

36 37
Advanced usage
~~~~~~~~~~~~~~
38

39 40 41 42 43 44
- Customize ``respdiff.cfg`` (see instructions in the file).
- All executable scripts in the project's repository are tools for with a
  specific purpose. You can find their documentation in `doc/ <doc/>`__ and ``--help``.
- Queries and answeres can be gathered with a different tool
  (e.g. `dnsjit <https://github.com/DNS-OARC/dnsjit>`__) and read from LMDB
  (see `doc/lmdb_format.rst <doc/lmdb_format.rst>`__ for description of the used binary format.
45

46 47
Overview
--------
48

49
Respdiff is conceptually chain of independent tools:
50

51 52 53 54 55 56 57 58 59
1. `qprep <doc/qprep.rst>`__: generate queries in wire format and store to LMDB
2. `orchestrator <doc/orchestrator.rst>`__: send pre-generated wire format to
   servers and gather answers to LMDB
3. `msgdiff <doc/msgdiff.rst>`__: compare DNS answers
4. `diffrepro <doc/diffrepro.rst>`__: attempt to reproduce the differences
5. `diffsum <doc/diffsum.rst>`__: summarize differences into textual and
   machine readable report
6. `histogram <doc/histogram.rst>`__: plot graph of answer latencies
7. `sumcmp <doc/sumcmp.rst>`__: compare a new diffsum report to a reference one
60
8. `sumstat <doc/sumstat.rst>`__: generate statistics from multiple diffsum reports
Tomas Krizek's avatar
Tomas Krizek committed
61
9. `statcmp <doc/statcmp.rst>`__: plot and compare statistics and reports
62

63

64 65
Changelog
---------
66

67 68 69
- 2018-08-30: JSON datafile format changed: ``other_disagreements`` now
  includes a list of all affected queries. Older reports will incorrectly report
  0 for this field. (Can be fixed by re-creating reports with ``msgdiff.py``).
70 71 72
- 2018-08-21: ``qname``, ``qcase`` and ``qtype`` field comparisons were removed.
  ``question`` comparison now performs all these check in a case sensitive manner.
  Remove these checks from ``criteria`` and ``field_weights`` in ``respdiff.cfg``.
73
- 2018-08-16: new dependency ``scipy`` added to ``requirements.txt``
74
- 2018-06-01: reorganized tools and created a new git repo with cleaned-up history