Huge policy blacklist draws kresd unusable
Hello,
I was trying to implement blacklist in order to blacklist all domains specified in my domains file. I used the policy policy.DENY
. My blacklist is compiled from several advertisement host files available on the internet and it's final size is around 7 MiB. The method works very well for smaller blacklist. Should I use different approach to block specific domains? Or is it not possible.
Thank you in advance for looking into this.
Version info:
Target: Knot DNS Resolver 2.2.0-POSIX
Compiler: clang-7 -g -O3 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Wdate-time -D_FORTIFY_SOURCE=2 -std=c99 -D_GNU_SOURCE -Wno-unused -Wtype-limits -Wformat -Wformat-security -Wall -I/opt/build-env/kresd -I/opt/build-env/kresd/lib/generic -I/opt/build-env/kresd/contrib -DPACKAGE_VERSION="\"2.2.0\"" -DPREFIX="\"/usr/local\"" -DMODULEDIR="\"/usr/local/lib/kdns_modules\"" -fvisibility=hidden -I/usr/include/p11-kit-1 -I/usr/include/luajit-2.1 -I/usr/include/p11-kit-1 -Icontrib/ccan/compiler -Icontrib/ccan/ilog -Icontrib/ccan/isaac -Icontrib/ccan/json -Icontrib/ccan/asprintf -Icontrib/murmurhash3 -DENABLE_COOKIES
Linker: clang-7 -Wl,-z,relro -Wl,-z,now -fPIC -pthread -lm -Wl,--export-dynamic -Wl,-z,relro,-z,now -ldl
> lua -v
Lua 5.2.4 Copyright (C) 1994-2015 Lua.org, PUC-Rio
luajit
2.1.0~beta3+dfsg-5.1
> uname -r
4.15.0-1-amd64
I implemented it this way:
function trim(s)
return (string.gsub(s, "^%s*(.-)%s*$", "%1"))
end
local function isEmpty(s)
return s == nil or s == ""
end
function loadFileToBlacklist(filename)
local file, err = io.open(filename, "r")
for line in file:lines() do
local tLine = trim(line)
if not isEmpty(tLine) then
policy.add(policy.suffix(policy.DENY, { todname(tLine) }))
end
end
io.close(file)
end
-- load blacklist
loadFileToBlacklist("/etc/knot-resolver/hosts/blacklist.conf")
sample of blacklist:
101com.com
101order.com
123found.com
123freeavatars.com
180hits.de
180searchassistant.com
207.net
20a840a14a0ef7d6.com
247media.com
24log.com
24log.de
24pm-affiliation.com
...
full compiled file is attached here blacklist.conf.xz
the blacklist currently consists of these reformatted blacklists
https://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext
https://adzhosts.fr/hosts/adzhosts-mac-linux.txt
https://adblock.mahakala.is
Note: items in blacklist do not occur more then once