updater issueshttps://gitlab.nic.cz/turris/updater/updater/-/issues2020-11-12T03:15:39+01:00https://gitlab.nic.cz/turris/updater/updater/-/issues/179Packages miss reported as having multiple candidates2020-11-12T03:15:39+01:00Karel KociPackages miss reported as having multiple candidatesSometimes packages are reported as having multiple candidates. But in repository
there is only one. There seems to be bug and it might even cause more damage than
this if they are sorted invalidly.Sometimes packages are reported as having multiple candidates. But in repository
there is only one. There seems to be bug and it might even cause more damage than
this if they are sorted invalidly.Turris OS 3.6https://gitlab.nic.cz/turris/updater/updater/-/issues/177Check if we timeout if postinst script hangs2020-11-12T03:15:43+01:00Karel KociCheck if we timeout if postinst script hangsWe encountered problem with watchcat post install script hanging update. Investigate.We encountered problem with watchcat post install script hanging update. Investigate.Turris OS 3.6https://gitlab.nic.cz/turris/updater/updater/-/issues/173Run `pkgupdate` even if migration went through2020-11-12T03:15:47+01:00Ghost UserRun `pkgupdate` even if migration went throughEven when `auto.lua` contains the line the migration went through, run `pkgupdate` to update things and potentially finish up everything that didn't work before.Even when `auto.lua` contains the line the migration went through, run `pkgupdate` to update things and potentially finish up everything that didn't work before.Turris OS 3.6https://gitlab.nic.cz/turris/updater/updater/-/issues/172Migrator: cleanup before running2020-11-12T03:15:35+01:00Ghost UserMigrator: cleanup before runningDue to issues with the old update infrastructure, the `auto.lua` is a bit broken when the migrator is run.
- [x] Do some clean up before running (eg. make sure the updater and probably other packages are not mandated and packages are no...Due to issues with the old update infrastructure, the `auto.lua` is a bit broken when the migrator is run.
- [x] Do some clean up before running (eg. make sure the updater and probably other packages are not mandated and packages are not linked)
- [x] Blacklist (eg `Uninstall`) updater and similar from the list, so even if it is required from somewhere, it is not really installed.Turris OS 3.6https://gitlab.nic.cz/turris/updater/updater/-/issues/190Allow updater.sh to be executed with verbose output2020-11-12T03:21:20+01:00Karel KociAllow updater.sh to be executed with verbose outputRunning pkgupdate optionally with DBG or TRACE output should help us in
diagnostics and might be usable even to users.
TODO add this also to diagnosticsRunning pkgupdate optionally with DBG or TRACE output should help us in
diagnostics and might be usable even to users.
TODO add this also to diagnosticsTurris OS 3.7https://gitlab.nic.cz/turris/updater/updater/-/issues/182Updater: [string "planner"]:62: assertion failed!2020-11-12T03:20:44+01:00Michal ČihařUpdater: [string "planner"]:62: assertion failed!When I run updater, I get following error:
```
# updater.sh
WARN:Script revision-specific not found, but ignoring its absence as requested
WARN:Script serial-specific not found, but ignoring its absence as requested
DIE:
[string "plann...When I run updater, I get following error:
```
# updater.sh
WARN:Script revision-specific not found, but ignoring its absence as requested
WARN:Script serial-specific not found, but ignoring its absence as requested
DIE:
[string "planner"]:62: assertion failed!
```
It really does not tell what went wrong.
I can workaround that by removing following lines from `/etc/updater/auto.lua`:
```
Package "knot-libdnssec" { content = "file:///usr/share/updater/local-pkgs/knot-libdnssec_2.4.0-2_mvebu.ipk" }
Install "knot-libdnssec"
Package "knot-libknot" { content = "file:///usr/share/updater/local-pkgs/knot-libknot_2.4.0-2_mvebu.ipk" }
Install "knot-libknot"
Package "knot-libzscanner" { content = "file:///usr/share/updater/local-pkgs/knot-libzscanner_2.4.0-2_mvebu.ipk" }
Install "knot-libzscanner"
Package "knot-resolver" { content = "file:///usr/share/updater/local-pkgs/knot-resolver_1.2.0-4_mvebu.ipk" }
Install "knot-resolver"
```
I've added these when testing fixes for https://gitlab.labs.nic.cz/knot/resolver/issues/132 and https://gitlab.labs.nic.cz/knot/resolver/issues/137, but the error started to show up after upgrading to Turris 3.5.2.Turris OS 3.7https://gitlab.nic.cz/turris/updater/updater/-/issues/181Automatic migration to new updater2020-11-12T03:20:40+01:00Ghost UserAutomatic migration to new updaterAfter releasing 3.6 we want to start the work on automatically migrating to the new updater on all routers. Therefore we want to enable the automatic mode and test what goes wrong.After releasing 3.6 we want to start the work on automatically migrating to the new updater on all routers. Therefore we want to enable the automatic mode and test what goes wrong.Turris OS 3.7https://gitlab.nic.cz/turris/updater/updater/-/issues/170Support system certificates2020-11-12T03:20:50+01:00Karel KociSupport system certificatesUpdater prefers certificates to be given explicitly, but that is unexpected by
users. More expected would be if url is configured as that it should check
certificate and if ca extra option is nil, than we would use system certificates.Updater prefers certificates to be given explicitly, but that is unexpected by
users. More expected would be if url is configured as that it should check
certificate and if ca extra option is nil, than we would use system certificates.Turris OS 3.7https://gitlab.nic.cz/turris/updater/updater/-/issues/169Read /etc/opkg/customfeeds.conf in entry.lua2020-11-12T03:21:12+01:00Karel KociRead /etc/opkg/customfeeds.conf in entry.luaOpkg can be configured with any additional repositories, in such case opkg can
install software that updater doesn't know. This shouldn't be happening. For this
reason we should read opkg custom feeds in entry script.Opkg can be configured with any additional repositories, in such case opkg can
install software that updater doesn't know. This shouldn't be happening. For this
reason we should read opkg custom feeds in entry script.Turris OS 3.7https://gitlab.nic.cz/turris/updater/updater/-/issues/166Updated packages can leave untracked files behind2020-11-12T03:21:26+01:00Karel KociUpdated packages can leave untracked files behindWhen some package stops using some package (it disappears from .list file), then
we won't remove such file. (At least it seems that way by some test I conducted)
This can be solved together with #159When some package stops using some package (it disappears from .list file), then
we won't remove such file. (At least it seems that way by some test I conducted)
This can be solved together with #159Turris OS 3.7https://gitlab.nic.cz/turris/updater/updater/-/issues/197Approvals notifications should contain list of changes to be approved2020-11-12T03:27:29+01:00Karel KociApprovals notifications should contain list of changes to be approvedNotifications about waiting approvals says nothing about changes to be approved. It's like a small suprise that have to be unpacked in Foris.Notifications about waiting approvals says nothing about changes to be approved. It's like a small suprise that have to be unpacked in Foris.Turris OS 3.8https://gitlab.nic.cz/turris/updater/updater/-/issues/186If package is provided by some other it shouldn't be selected2020-11-12T03:27:14+01:00Karel KociIf package is provided by some other it shouldn't be selectedCurrently if package is provided by some other package. Both are selected as part
of process resolution, because for provided package it is candidate. This breaks
common idiom in packages:
```
Provides: pkg
Conflicts: pkg
```Currently if package is provided by some other package. Both are selected as part
of process resolution, because for provided package it is candidate. This breaks
common idiom in packages:
```
Provides: pkg
Conflicts: pkg
```Turris OS 3.8https://gitlab.nic.cz/turris/updater/updater/-/issues/184Assertion failure if user list added multiple times2020-11-12T03:27:20+01:00Ghost UserAssertion failure if user list added multiple timesIt's the version now in rc: 49.1.
```
Stack Traceback
===============
(2) Lua function '?' at line 64 of chunk '"logging"]'
Local variables:
err = [string "backend"]:1246: assertion failed! {msg:[string "backend"]:1246: assertion fa...It's the version now in rc: 49.1.
```
Stack Traceback
===============
(2) Lua function '?' at line 64 of chunk '"logging"]'
Local variables:
err = [string "backend"]:1246: assertion failed! {msg:[string "backend"]:1246: assertion failed!, tp:error, reason:runtime (more...)}
err2string = Lua function '?' (defined at line 45 of chunk "logging"])
msg = string: "\
runtime: [string \"backend\"]:1246: assertion failed!"
(*temporary) = table: 0x10d97cb8 {msg:
runtime: [string "backend"]:1246: assertion failed!}
(3) C function 'function: 0x10420908'
(4) upvalue C function 'error'
(5) Lua global 'required_pkgs' at line 53 of chunk '"updater"]'
Local variables:
entrypoint = string: "internal:entry_lua"
tlc = table: 0x104100c8 {env:table: 0x10434590, level_check:function: 0x10520500, tp:context, sec_level:Full (more...)}
ep_uri = table: 0x105c3be0 {ok:function: 0x10509be8, tp:uri, cback:function: 0x10611ae8, callbacks:table: 0x105c3c80 (more...)}
ok = boolean: true
tls = string: "-- Do not edit this file. It is the entry point. Edit the user.lua file.\
\
local branch = \"\"\
local lists\
if uci then\
-- If something is really broken, we could be unable to load the uci dynamic module. Try to do some recovery in such case and hope it works well the next time.\
local cursor = uci.cursor()\
branch = cursor:get(\"updater\", \"override\", \"branch\")\
if branch then\
WARN(\"Branch overriden to \" .. branch)\
branch = branch .. \"/\"\
else\
branch = \"\"\
end\
lists = cursor:get(\"updater\", \"pkglists\", \"lists\")\
else\
ERROR(\"UCI library is not available. Not processing user lists.\")\
end\
\
-- Guess what board this is.\
local base_model = \"\"\
if model then\
if model:match(\"[Oo]mnia\") then\
base_model = \"omnia/\"\
elseif model:match(\"[Tt]urris\") then\
base_model = \"turris/\"\
end\
end\
\
local base_url\
if base_model then\
base_url = \"https://api.turris.cz/updater-defs/\" .. turris_version .. \"/\" .. base_model .. branch\
end\
\
-- Reused options for remotely fetched scripts\
local script_options = {\
security = \"Remote\",\
ca = \"file:///etc/ssl/updater.pem\",\
crl = \"file:///tmp/crl.pem\",\
pubkey = {\
\"file:///etc/updater/keys/release.pub\",\
\"file:///etc/updater/keys/standby.pub\",\
\"file:///etc/updater/keys/test.pub\" -- It is normal for this one to not be present in production systems\
}\
}\
\
if base_url then\
-- The distribution script. It contains the repository and bunch of basic packages. The URI is computed based on the branch and the guessed board\
Script(\"base\", base_url .. \"base.lua\", script_options)\
-- Ask for a script specific for our serial. But ignore if it's not there (which is common).\
if serial then\
local script_options_ignore = {}\
for n, v in pairs(script_options) do\
script_options_ignore[n] = v\
end\
script_options_ignore.ignore = {\"missing\"}\
-- First try just the revision\
Script(\"revision-specific\", base_url .. \"revision/\" .. serial:sub(1, 8) .. \".lua\", script_options_ignore)\
Script(\"serial-specific\", base_url .. \"serial/\" .. serial .. \".lua\", script_options_ignore)\
end\
end\
\
-- Some provided by the user\
Script \"user-src\" \"file:///etc/updater/user.lua\" { security = \"Local\" }\
-- Some auto-generated by command line\
Script \"auto-src\" \"file:///etc/updater/auto.lua\" { security = \"Local\" }\
\
if uci and base_url then\
-- Go through user lists and pull them in.\
if type(lists) == \"string\" then\
lists = {lists}\
end\
if type(lists) == \"table\" then\
for _, l in ipairs(lists) do\
-- TODO: Make restricted security work\
Script(\"userlist-\" .. l, base_url .. \"userlists/\" .. l .. \".lua\", script_options)\
end\
end\
end\
"
err = [string "backend"]:1246: assertion failed! {msg:[string "backend"]:1246: assertion failed!, tp:error, reason:runtime (more...)}
(6) Lua function '?' at line 61 of chunk '"updater"]'
Local variables:
entrypoint = string: "internal:entry_lua"
```Turris OS 3.8https://gitlab.nic.cz/turris/updater/updater/-/issues/178Support 'Conflicts'2020-11-12T03:27:24+01:00Karel KociSupport 'Conflicts'Packages can have field 'Conflicts', we are ignoring this field for now. But
support for it should be added.Packages can have field 'Conflicts', we are ignoring this field for now. But
support for it should be added.Turris OS 3.8https://gitlab.nic.cz/turris/updater/updater/-/issues/262Add mode in which pkgupdate reinstall all packages2019-09-27T12:51:52+02:00Karel KociAdd mode in which pkgupdate reinstall all packagesThis should "solve" problem when we switch between branches. In single branch we have script that bumps packages and forces updater to reinstall them but between branches we have nothing like that. The good enough hack is to reinstall ev...This should "solve" problem when we switch between branches. In single branch we have script that bumps packages and forces updater to reinstall them but between branches we have nothing like that. The good enough hack is to reinstall everything every time. This can be archived easily just by not filtering out installed packages (causing reinstall instead). So this is option to skip single check in code.Turris OS 4.0https://gitlab.nic.cz/turris/updater/updater/-/issues/256Change user agent to contain additional information2019-05-06T17:46:46+02:00Karel KociChange user agent to contain additional informationWe should add additional information to user agent. Currently we only report updater version but it is common to also report the OS version, kernel version and network software version. This should help us to discover problematic clients...We should add additional information to user agent. Currently we only report updater version but it is common to also report the OS version, kernel version and network software version. This should help us to discover problematic clients and detect if some combination of software is not failing because of our server configuration.
Current user agent contains only updater version in following format: `"Turris Updater/" UPDATER_VERSION`
I suggest to append additional information in standard form: `"Turris Updater/" UPDATER_VERSION (ADD1; ADD2)`
Additional information should be:
* Version and name of distribution
* Architecture and kernel version and name
* Version of libcurlTurris OS 4.0https://gitlab.nic.cz/turris/updater/updater/-/issues/252Refactor URI2019-04-18T17:40:59+02:00Karel KociRefactor URIUpdater contains complicated uri handling and almost separate downloading framework (that is integrated with events). We should instead reimplement it from ground up in C (just as lua bindings) with dedicated downloading system. Idea is ...Updater contains complicated uri handling and almost separate downloading framework (that is integrated with events). We should instead reimplement it from ground up in C (just as lua bindings) with dedicated downloading system. Idea is to separate current downloader implementation from events and then use that to implement uri abstraction.
There should also be small feature that to simplify uri usage we should also accept strings without `URI://` format to be local paths (`file://` uri).
* [x] Separate downloader from events
* [x] Implement uri as a lua bindingsTurris OS 4.0https://gitlab.nic.cz/turris/updater/updater/-/issues/227Add support for relative uri2019-05-06T17:46:46+02:00Karel KociAdd support for relative uriIf we are in script and using new uri then it makes sense that we shouldn't care where this script is really running and we should be able to use uris relative to ours.
Also it would be nice to have current uri in some variable in scrip...If we are in script and using new uri then it makes sense that we shouldn't care where this script is really running and we should be able to use uris relative to ours.
Also it would be nice to have current uri in some variable in script it self or maybe even call trace.Turris OS 4.0https://gitlab.nic.cz/turris/updater/updater/-/issues/160Memory consumption2019-05-06T17:46:47+02:00Ghost UserMemory consumptionWhile the consumption isn't catastrophic, it is surprisingly high. And it isn't eaten just during the dependency computation, but during the whole run, which is a bit strange. During an update is is over 100MB.While the consumption isn't catastrophic, it is surprisingly high. And it isn't eaten just during the dependency computation, but during the whole run, which is a bit strange. During an update is is over 100MB.Turris OS 4.0https://gitlab.nic.cz/turris/updater/updater/-/issues/151Repository integrity error handling2019-05-06T17:47:26+02:00Karel KociRepository integrity error handlingWe currently not handling broken integrity of repository. We report it as missing
instead. We should fix this and add support for `ignore="integrity"`.We currently not handling broken integrity of repository. We report it as missing
instead. We should fix this and add support for `ignore="integrity"`.Turris OS 4.0