Commit 9e4963b2 authored by Marek Vavruša's avatar Marek Vavruša

Merge branch 'config_documentation'

parents 5b21d8ac a747baff
# Knot DNS Resolver
[![Build Status](https://travis-ci.org/CZ-NIC/knot-resolver.svg?branch=master)](https://travis-ci.org/CZ-NIC/knot-resolver)
[![Coverage Status](https://coveralls.io/repos/CZ-NIC/knot-resolver/badge.svg)](https://coveralls.io/r/CZ-NIC/knot-resolver)
[![Coverity](https://scan.coverity.com/projects/3912/badge.svg)](https://scan.coverity.com/projects/3912)
[![Build Status](https://img.shields.io/travis/CZ-NIC/knot-resolver.svg)](https://travis-ci.org/CZ-NIC/knot-resolver)
[![Coverage Status](https://img.shields.io/coveralls/CZ-NIC/knot-resolver.svg)](https://coveralls.io/r/CZ-NIC/knot-resolver)
[![Coverity](https://img.shields.io/coverity/scan/3912.svg)](https://scan.coverity.com/projects/3912)
The Knot DNS Resolver is a minimalistic caching resolver implementation. The project provides both a resolver
......
************************
Knot DNS Resolver daemon
************************
Requirements
============
* libuv_ 1.0+ (a multi-platform support library with a focus on asynchronous I/O)
* Lua_ 5.1+ (embeddable scripting language, LuaJIT_ is preferred)
Running
=======
There is a separate resolver library in the `lib` directory, and a minimalistic daemon in
the `daemon` directory.
.. code-block:: bash
$ ./daemon/kresolved -h
Interacting with the daemon
---------------------------
The daemon features a CLI interface if launched interactively, type ``help`` to see the list of available commands.
You can load modules this way and use their properties to get information about statistics and such.
.. code-block:: bash
$ kresolved /var/run/knot-resolver
[system] started in interactive mode, type 'help()'
> cache.count()
53
.. role:: lua(code)
:language: lua
Configuration
=============
.. contents::
:depth: 2
:local:
In it's simplest form it requires just a working directory in which it can set up persistent files like
cache and the process state. If you don't provide the working directory by parameter, it is going to make itself
comfortable in the current working directory.
.. code-block:: sh
$ kresolved /var/run/kresolved
And you're good to go for most use cases! If you want to use modules or configure daemon behavior, read on.
There are several choices on how you can configure the daemon, a RPC interface a CLI and a configuration file.
Fortunately all share common syntax and are transparent to each other, e.g. changes made during the runtime are kept
in the redo log and are immediately visible.
.. warning:: Redo log is not yet implemented, changes are visible during the process lifetime only.
Configuration example
---------------------
.. code-block:: lua
-- 10MB cache
cache.open(10*MB)
-- static hints
modules = {
hints = true,
cachectl = true
}
-- interfaces
net.listen('127.0.0.1')
Configuration syntax
--------------------
The configuration is kept in the ``config`` file in the daemon working directory, and it's going to get loaded automatically.
If there isn't one, the daemon is going to start with sane defaults, listening on `localhost`.
The syntax for options is like follows: ``group.option = value`` or ``group.action(parameters)``.
You can also comment using a ``--`` prefix.
A simple example would be to load static hints.
.. code-block:: lua
modules = {
cachectl = true -- no configuration
}
If the module accepts accepts configuration, you can provide a table.
The syntax for table is ``{ key1 = value, key2 = value }``, and it represents the unpacked `JSON-encoded`_ string, that
the modules use as the :ref:`input configuration <mod-properties>`.
.. code-block:: lua
modules = {
cachectl = true,
hints = {
file = '/etc/hosts'
}
}
The possible simple data types are strings, integers or floats and boolean.
.. tip:: The configuration and CLI syntax is Lua language, with which you may already be familiar with.
If not, you can read the `Learn Lua in 15 minutes`_ for a syntax overview. Spending just a few minutes
will allow you to break from static configuration, write more efficient configuration with iteration, and
leverage events and hooks. Lua is heavily used for scripting in applications ranging from embedded to game engines,
but in DNS world notably in `PowerDNS Recursor`_. Knot DNS Resolver does not simply use Lua modules, but it is
the heart of the daemon for everything from configuration, internal events and user interaction.
Dynamic configuration
^^^^^^^^^^^^^^^^^^^^^
Knowing that the the configuration is a valid Lua script enables you to write dynamic rules, and also avoid
additional configuration templating. One example is to differentiate between internal and external
interfaces based on environment variable.
.. code-block:: lua
if hostname() == 'hidden' then
net.listen(net.eth0)
else
net.listen(net.eth1.addr[1])
end
Another example would show how it is possible to bind to all interfaces, using iteration.
.. code-block:: lua
for name, addr_list in pairs(net.interfaces()) do
net.listen(addr_list)
end
You can also use third-party packages (available for example through LuaRocks_) as on this example
to download cache from parent, to avoid cold-cache start.
.. code-block:: lua
local http = require('socket.http')
local ltn12 = require('ltn12')
if cache.count() == 0 then
-- download cache from parent
http.request {
url = 'http://parent/cache.mdb',
sink = ltn12.sink.file(io.open('cache.mdb', 'w'))
}
-- reopen cache with 100M limit
cache.open('.', 100*MB)
end
Events and services
^^^^^^^^^^^^^^^^^^^
The Lua supports a concept called closures, this is extremely useful for scripting actions upon various events.
.. note:: Work in progress, come back later!
* Timers and events
* File watchers
* Data I/O
Configuration reference
-----------------------
This is a reference for variables and functions available to both configuration file and CLI.
Environment
^^^^^^^^^^^
.. envvar:: env (table)
Return environment variable.
.. code-block:: lua
env.USER -- equivalent to $USER in shell
.. function:: hostname()
:return: Machine hostname.
Network configuration
^^^^^^^^^^^^^^^^^^^^^
.. function:: net.listen(address, [port = 53])
:return: boolean
Listen on address, port is optional.
.. function:: net.listen({address1, ...}, [port = 53])
:return: boolean
Listen on list of addresses.
.. function:: net.listen(interface, [port = 53])
:return: boolean
Listen on all addresses belonging to an interface.
Example:
.. code-block:: lua
net.listen(net.eth0) -- listen on eth0
.. function:: net.close(address, [port = 53])
:return: boolean
Close opened address/port pair, noop if not listening.
.. function:: net.list()
:return: Table of bound interfaces.
Example output:
.. code-block:: lua
[127.0.0.1] => {
[port] => 53
[tcp] => true
[udp] => true
}
.. function:: net.interfaces()
:return: Table of available interfaces and their addresses.
Example output:
.. code-block:: lua
[lo0] => {
[addr] => {
[1] => ::1
[2] => 127.0.0.1
}
[mac] => 00:00:00:00:00:00
}
[eth0] => {
[addr] => {
[1] => 192.168.0.1
}
[mac] => de:ad:be:ef:aa:bb
}
.. tip:: You can use ``net.<iface>`` as a shortcut for specific interface, e.g. ``net.eth0``
Modules configuration
^^^^^^^^^^^^^^^^^^^^^
The daemon provides an interface for dynamic loading of :ref:`daemon modules <modules-implemented>`.
.. tip:: Use syntactic sugar for module loading. Declaring a variable ``modules`` equals to loading a table of modules.
.. code-block:: lua
modules = { hints = {file = '/etc/hosts'} }
Equals to:
.. code-block:: lua
modules.load('cachectl')
cachectl.config({file = '/etc/hosts'})
.. function:: modules.list()
:return: List of loaded modules.
.. function:: modules.load(name)
:param string name: Module name, e.g. "hints"
:return: boolean
Load a module by name.
.. function:: modules.unload(name)
:param string name: Module name
:return: boolean
Unload a module by name.
Cache configuration
^^^^^^^^^^^^^^^^^^^
The cache in Knot DNS Resolver is persistent with LMDB backend, this means that the daemon doesn't lose
the cached data on restart or crash to avoid cold-starts. Interestingly the cache may be reused between cache
daemons or manipulated from other processes, making for example synchronisation between load-balanced recursors possible.
.. function:: cache.open(max_size)
:param number max_size: Maximum cache size in bytes.
:return: boolean
Open cache with size limit. The cache will be reopened if already open.
Note that the max_size cannot be lowered, only increased due to how cache is implemented.
.. tip:: Use ``kB, MB, GB`` constants as a multiplier, e.g. ``100*MB``.
.. function:: cache.count()
:return: Number of entries in the cache.
.. function:: cache.close()
:return: boolean
Close the cache.
.. _`JSON-encoded`: http://json.org/example
.. _`Learn Lua in 15 minutes`: http://tylerneylon.com/a/learn-lua/
.. _`PowerDNS Recursor`: https://doc.powerdns.com/md/recursor/scripting/
.. _LuaRocks: https://rocks.moonscript.org/
.. _libuv: https://github.com/libuv/libuv
.. _Lua: http://www.lua.org/about.html
.. _LuaJIT: http://luajit.org/luajit.html
\ No newline at end of file
......@@ -126,20 +126,61 @@ static int net_list(lua_State *L)
return 1;
}
/** Listen on interface address list. */
static int net_listen_iface(lua_State *L, int port)
{
/* Expand 'addr' key if exists */
lua_getfield(L, 1, "addr");
if (lua_isnil(L, -1)) {
lua_pop(L, 1);
lua_pushvalue(L, 1);
}
/* Bind to address list */
struct engine *engine = engine_luaget(L);
size_t count = lua_rawlen(L, -1);
for (size_t i = 0; i < count; ++i) {
lua_rawgeti(L, -1, i + 1);
int ret = network_listen(&engine->net, lua_tostring(L, -1),
port, NET_TCP|NET_UDP);
if (ret != 0) {
lua_pushstring(L, kr_strerror(ret));
lua_error(L);
}
lua_pop(L, 1);
}
lua_pushboolean(L, true);
return 1;
}
/** Listen on endpoint. */
static int net_listen(lua_State *L)
{
/* Check parameters */
int n = lua_gettop(L);
if (n < 2) {
lua_pushstring(L, "expected (string addr, int port)");
int port = KR_DNS_PORT;
if (n > 1 && lua_isnumber(L, 2)) {
port = lua_tointeger(L, 2);
}
/* Process interface or (address, port) pair. */
if (lua_istable(L, 1)) {
return net_listen_iface(L, port);
} else if (n < 1 || !lua_isstring(L, 1)) {
lua_pushstring(L, "expected (string addr, int port = 53)");
lua_error(L);
}
/* Open resolution context cache */
struct engine *engine = engine_luaget(L);
int ret = network_listen(&engine->net, lua_tostring(L, 1), lua_tointeger(L, 2), NET_TCP|NET_UDP);
lua_pushboolean(L, ret == 0);
int ret = network_listen(&engine->net, lua_tostring(L, 1), port, NET_TCP|NET_UDP);
if (ret != 0) {
lua_pushstring(L, kr_strerror(ret));
lua_error(L);
}
lua_pushboolean(L, true);
return 1;
}
......@@ -160,8 +201,7 @@ static int net_close(lua_State *L)
return 1;
}
/** List available interfaces.
*/
/** List available interfaces. */
static int net_interfaces(lua_State *L)
{
/* Retrieve interface list */
......@@ -227,23 +267,45 @@ int lib_net(lua_State *L)
return 1;
}
/** Return number of cached records. */
static int cache_count(lua_State *L)
{
struct engine *engine = engine_luaget(L);
const namedb_api_t *storage = kr_cache_storage();
/* Fetch item count */
namedb_txn_t txn;
int ret = kr_cache_txn_begin(engine->resolver.cache, &txn, NAMEDB_RDONLY);
if (ret != 0) {
lua_pushstring(L, kr_strerror(ret));
lua_error(L);
}
lua_pushinteger(L, storage->count(&txn));
kr_cache_txn_abort(&txn);
return 1;
}
/** Open cache */
static int cache_open(lua_State *L)
{
/* Check parameters */
int n = lua_gettop(L);
if (n < 2) {
lua_pushstring(L, "expected (string path, int size)");
if (n < 1) {
lua_pushstring(L, "expected (number max_size)");
lua_error(L);
}
/* Open resolution context cache */
/* Close if already open */
struct engine *engine = engine_luaget(L);
engine->resolver.cache = kr_cache_open(lua_tostring(L, 1), engine->pool, lua_tointeger(L, 2));
if (engine->resolver.cache != NULL) {
kr_cache_close(engine->resolver.cache);
}
/* Open resolution context cache */
engine->resolver.cache = kr_cache_open(".", engine->pool, lua_tointeger(L, 1));
if (engine->resolver.cache == NULL) {
lua_pushstring(L, "invalid cache directory: ");
lua_pushstring(L, lua_tostring(L, 1));
lua_concat(L, 2);
lua_pushstring(L, "can't open cache in rundir");
lua_error(L);
}
......@@ -255,8 +317,9 @@ static int cache_close(lua_State *L)
{
struct engine *engine = engine_luaget(L);
if (engine->resolver.cache != NULL) {
kr_cache_close(engine->resolver.cache);
struct kr_cache *cache = engine->resolver.cache;
engine->resolver.cache = NULL;
kr_cache_close(cache);
}
lua_pushboolean(L, 1);
......@@ -266,6 +329,7 @@ static int cache_close(lua_State *L)
int lib_cache(lua_State *L)
{
static const luaL_Reg lib[] = {
{ "count", cache_count },
{ "open", cache_open },
{ "close", cache_close },
{ NULL, NULL }
......
-- Default configuration
cache.open('.', 10485760)
cache.open(10*MB)
-- Listen on localhost
if not next(net.list()) then
if not pcall(net.listen, '127.0.0.1') then
error('failed to bind to localhost#53')
end
end
\ No newline at end of file
-- Units
kB = 1024
MB = 1024*1024
GB = 1024*1024
-- Function aliases
-- `env.VAR returns os.getenv(VAR)`
env = {}
setmetatable(env, {
__index = function (t, k) return os.getenv(k) end
})
-- Quick access to interfaces
-- `net.<iface>` => `net.interfaces()[iface]`
setmetatable(net, {
__index = function (t, k)
local v = rawget(t, k)
if v then return v
else return net.interfaces()[k]
end
end
})
-- Syntactic sugar for module loading
-- `modules.<name> = <config>`
setmetatable(modules, {
......
......@@ -49,8 +49,8 @@ There are also *optional* packages that enable specific functionality in Knot DN
.. [#] Requires C99, ``__attribute__((cleanup))`` and ``-MMD -MP`` for dependency file generation. GCC, Clang and ICC are supported.
.. [#] You can use variables ``<dependency>_CFLAGS`` and ``<dependency>_LIBS`` to configure dependencies manually (i.e. ``libknot_CFLAGS`` and ``libknot_LIBS``).
Docker image
~~~~~~~~~~~~
Getting Docker image
--------------------
Docker images require only either Linux or a Linux VM (see boot2docker_ on OS X).
......@@ -65,8 +65,10 @@ You can hack on the container by changing the container entrypoint to shell like
$ docker run -it --entrypoint=/bin/bash cznic/knot-resolver
.. tip:: You can build the Docker image yourself with ``docker build -t knot-resolver scripts``.
Building from sources
~~~~~~~~~~~~~~~~~~~~~
---------------------
The Knot DNS Resolver depends on the development version of the Knot DNS library, and a reasonably recent version of `libuv`.
Several dependencies may not be in the packages yet, the script pulls and installs all dependencies in a chroot.
......@@ -93,18 +95,20 @@ Usually you only really need to rebuild `libknot`.
$ make check libknot_CFLAGS="-I/opt/include" libknot_LIBS="-L/opt/lib -lknot -lknot-int -ldnssec"
.. note:: If the dependencies lie outside of library search path, you need to add them somehow.
Try ``LD_LIBRARY_PATH`` on Linux/BSD, and ``DYLD_FALLBACK_LIBRARY_PATH`` on OS X. Otherwise you might
need to add the locations to the linker search path.
.. warning:: If the dependencies lie outside of library search path, you need to add them somehow.
Try ``LD_LIBRARY_PATH`` on Linux/BSD, and ``DYLD_FALLBACK_LIBRARY_PATH`` on OS X.
Otherwise you need to add the locations to linker search path.
When you have all the dependencies ready, you can build, test and install.
.. code-block:: bash
$ make
$ make PREFIX="/usr/local"
$ make check
$ make install
.. note:: Always build with ``PREFIX`` if you want to install, as it is hardcoded in the executable for module search path.
Alternatively you can build only specific parts of the project, i.e. ``library``.
.. code-block:: bash
......
......@@ -49,6 +49,8 @@ html_use_smartypants = True
# Output file base name for HTML help builder.
htmlhelp_basename = 'apidoc'
# Theme
html_theme = 'sphinx_rtd_theme'
# -- Options for LaTeX output --------------------------------------------------
......
Daemon configuration
--------------------
The Knot DNS Resolver daemon has no traditional concept of static configuration.
In it's simplest form it requires just a working directory in which it can set up persistent files like
cache and the process state.
.. code-block:: sh
$ kresolved /var/run/kresolved
And you're good to go!
Introduction
~~~~~~~~~~~~
There are several choices on how you can configure the daemon, a RPC interface a CLI or a configuration file,
but fortunately all share a common syntax and are transparent to each other, e.g. if you change a knob, you're going to
see it projected to other interfaces as well.
.. note:: Expect this page to change a lot, as it's still just a proof of concept implementation.
Configuration 101
~~~~~~~~~~~~~~~~~
If there is a `config` file in the daemon working directory, it's going to get loaded automatically, if there isn't one
the daemon is going to start with sane defaults and listening on `localhost`. The syntax for options is like follows: ``group.option = value``
or ``group.action(parameters)``. You can also comment using a ``--`` prefix.
A simple example would be to increase the cache size.
.. code-block:: lua
-- increase the cache to 100MB
cache.open(".", 100*1024*1024)
Dynamic configuration
~~~~~~~~~~~~~~~~~~~~~
Packages and services
~~~~~~~~~~~~~~~~~~~~~
The Lua supports a concept called closures, this is extremely useful for scripting actions upon various events.
.. note:: TODO, come back later!
* Timers and events
* File watchers
* Serialization
* Data I/O
Knot DNS Resolver daemon
========================
Requirements
------------
* libuv_ 1.0+ (a multi-platform support library with a focus on asynchronous I/O)
Starting the daemon
-------------------
There is a separate resolver library in the `lib` directory, and a minimalistic daemon in
the `daemon` directory. The daemon accepts a few CLI parameters, and there's no support for configuration
right now.
.. code-block:: bash
$ ./daemon/kresolved -h
$ ./daemon/kresolved -a 127.0.0.1#53
.. _libuv: https://github.com/libuv/libuv
Interacting with the daemon
---------------------------
The daemon features a CLI interface if launched interactively, type ``help`` to see the list of available commands.
You can load modules this way and use their properties to get information about statistics and such.
.. code-block:: bash
$ kresolved /var/run/knot-resolver
...
[system] started in interactive mode, type 'help()'
> modules.load('cachectl')
> return cachectl.size()
{ "size": 53 }
\ No newline at end of file
.. include:: ../daemon/README.rst
\ No newline at end of file
#################
Knot DNS Resolver
=================
#################
The Knot DNS Resolver is a minimalistic caching resolver implementation.
The project provides both a resolver library and a small daemon.
Modular architecture of the library keeps the core tiny and efficient, and provides a state-machine like API for extensions.
.. toctree::
:maxdepth: 2
:maxdepth: 4
build
lib
daemon
config
modules
......
......@@ -2,43 +2,8 @@
.. include:: ../lib/README.rst
Library layout
--------------
The library as described provides basic services for name resolution, which should cover the usage.
The following part is for those who are planning to hack on the library or develop modules, to give
you an idea about the API and the library layout.
Name resolution
~~~~~~~~~~~~~~~
.. _lib_rplan:
Resolution plan
~~~~~~~~~~~~~~~
.. _lib_cache:
Cache
~~~~~
.. _lib_nameservers:
Nameservers
~~~~~~~~~~~
.. _lib_modules:
Modules
~~~~~~~
Utilities
~~~~~~~~~
.. _lib_api:
API reference
-------------
=============
.. doxygengroup:: resolution
:project: libkresolve
......@@ -54,6 +19,8 @@ API reference
.. doxygengroup:: nameservers
:project: libkresolve
.. _lib_api_modules:
.. doxygengroup:: modules
:project: libkresolve
......
.. include:: ../modules/README.rst
.. _modules-implemented:
Implemented modules
-------------------
===================
.. contents::
:depth: 1
:local:
.. include:: ../modules/hints/README.rst
.. include:: ../modules/cachectl/README.rst
.. include:: ../modules/cachectl/README.rst
\ No newline at end of file
*************************
Knot DNS Resolver library
=========================
*************************
Requirements
------------
============
* libknot_ 2.0 (Knot DNS high-performance DNS library.)
Overview
--------
Library layout
==============
.. contents::