Commit 93fa182b authored by Tomas Krizek's avatar Tomas Krizek

README: document LMDB binary format

parent 00cdfaec
......@@ -139,3 +139,70 @@ Plot a logarithmic percentile graph of answer latencies.
$ ./histogram.py -o histogram.svg "${DIR}"
You can also re-run ``msgdiff.py`` and ``diffsum.py`` using different configuration.
LMDB Binary Format
------------------
If the data was gathered using tools other than ``orchestartor.py``, e.g.
`dnsjit <https://github.com/DNS-OARC/dnsjit>`__, the following LMDB database
environment can be used to achieve compatibility with the rest of respdiff
toolchain.
All numbers represented in binary format defined below use the **little endian** byte order.
Database ``queries``
~~~~~~~~~~~~~~~~~~~~
``queries`` database is used to store the wire format of queries that were sent
to the servers. Each query has a unique integer identifier, ``<QID>``.
+-----------+-----------------+-----------------------------+------------------+
| Key | Key Type | Value Description | Value Type |
+===========+=================+=============================+==================+
| ``<QID>`` | 4B unsigned int | DNS query sent to server(s) | DNS wire format |
+-----------+-----------------+-----------------------------+------------------+
Database ``answers``
~~~~~~~~~~~~~~~~~~~~
``answers`` database stores the binary reponses from the queried servers.
If there are multiple servers, their reponses are stored within a single
``<QID>`` key. Multiple responses (in their binary form ``time | length | wire``
as described below) are simply concatenated within the value. Please
note the order of responses is significant and must correspond with the server
definition in the ``meta`` database.
+-----------+-----------------+-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| Key | Key Type | Value Description | Value Type |
+===========+=================+===============================+=====================================================================================================================================+
| ``<QID>`` | 4B unsigned int | DNS reponse(s) from server(s) | One or more of |
| | | | ``time | length | wire`` |
| | | | - ``time``: 4B unsigned int; time to receive the answer in microseconds; ``4294967295`` (max value) means *timeout* |
| | | | - ``length``: 2B unsigned short; the length (in bytes) of the DNS message that follows (``length`` is present in case of *timeout*) |
| | | | - ``wire``: ``length``-byte DNS wire format with the response from server |
+-----------+-----------------+-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------+
Database ``meta``
~~~~~~~~~~~~~~~~~
``meta`` database stores additional information used for further processing of the data.
+----------------+----------+-------------------------------------------------------------------+-----------------+
| Key | Key Type | Value Description | Value Type |
+================+==========+===================================================================+=================+
| ``version`` | ASCII | respdiff binary format version (current: ``2018-05-21``) | ASCII |
+----------------+----------+-------------------------------------------------------------------+-----------------+
| ``servers`` | ASCII | number of servers reponses are collected from | 4B unsigned int |
+----------------+----------+-------------------------------------------------------------------+-----------------+
| ``name0`` | ASCII | name identifier of the first server (same as in ``respdiff.cfg``) | ASCII |
+----------------+----------+-------------------------------------------------------------------+-----------------+
| ``name1`` | ASCII | name identifier of the second server | ASCII |
+----------------+----------+-------------------------------------------------------------------+-----------------+
| ``name<N>`` | ASCII | name identifier of the ``N+1``-th server | ASCII |
+----------------+----------+-------------------------------------------------------------------+-----------------+
| ``start_time`` | ASCII | (*optional*) unix timestamp of the start of data collection | 4B unsigned int |
+----------------+----------+-------------------------------------------------------------------+-----------------+
| ``end_time`` | ASCII | (*optional*) unix timestamp of the end of data collection | 4B unsigned int |
+----------------+----------+-------------------------------------------------------------------+-----------------+
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment