Is the default Turris default configuration set to download history every time the client runs? Could we please turn it off? Is it really needed?
I've seen requests to /RMBTControlServer/history
that took whopping 89 seconds to complete. And they are not the exception, but rather a rule. And I can't fix this on the server without breaking any other client.
In case the history file is really needed, couldn't we just assemble it in the client ourselves? The data are there and it's just a matter of appending them to a file.
We should add more fields to the --debug
output to improve debugging of (not only) server-side issues.
Among others, we should add these fields:
Current debug output looks like this:
Checking uuid on the control server...
Sending the following request to https://control.netmetr.cz/RMBTControlServer/settings?skip_history=true
{
"uuid": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"language": "en_US",
"name": "RMBT",
"terms_and_conditions_accepted": "true",
"type": "DESKTOP",
"version_name": "Python netmetr client v1.6.0"
}
response:
{
"settings": [
{
"urls": {
"url_ipv6_check": "https://ipv6.control.netmetr.cz/RMBTControlServer/ip",
"control_ipv4_only": "ipv4.control.netmetr.cz",
"url_ipv4_check": "https://ipv4.control.netmetr.cz/RMBTControlServer/ip",
"control_ipv6_only": "ipv6.control.netmetr.cz",
"statistics": "https://www.netmetr.cz/cs/statistiky_app.html"
},
"qostesttype_desc": [
{
"name": "DNS",
"test_type": "DNS"
},
{
"name": "Non Transparent Proxy",
"test_type": "NON_TRANSPARENT_PROXY"
},
{
"name": "HTTP Proxy",
"test_type": "HTTP_PROXY"
},
{
"name": "Website",
"test_type": "WEBSITE"
},
{
"name": "UDP",
"test_type": "UDP"
},
{
"name": "TCP",
"test_type": "TCP"
},
{
"name": "JITTER",
"test_type": "JITTER"
}
],
"versions": {
"control_server_version": "master_2.0.83-CZ_0.9.28-354-gc1fca50"
}
}
]
}
Requesting test config from the control server...
Sending the following request to https://control.netmetr.cz/RMBTControlServer/testRequest
{
"client": "HW-PROBE",
"language": "en_US",
"time": 1624869256000,
"timezone": "CEST",
"type": "DESKTOP",
"uuid": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"version": "0.1"
}
response:
{
"user_server_selection": false,
"test_server_address": "ipv4.speed.netmetr.cz",
"test_server_port": 8081,
"test_server_name": "CZ.NIC (Prague)",
"test_server_encryption": false,
"test_server_type": "RMBT",
"test_duration": "5",
"test_numthreads": "3",
"test_numpings": "10",
"client_remote_ip": "94.199.96.205",
"result_url": "https://control.netmetr.cz/RMBTControlServer/result",
"result_qos_url": "https://control.netmetr.cz/RMBTControlServer/resultQoS",
"test_token": "yyyyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy_yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy",
"test_uuid": "yyyyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy",
"test_id": 12345678,
"test_wait": 0,
"error": []
}
Starting ping test...
ping_1_msec = 5.33
ping_2_msec = 4.65
ping_3_msec = 4.58
ping_4_msec = 4.84
ping_5_msec = 4.57
ping_6_msec = 4.69
ping_7_msec = 4.70
ping_8_msec = 6.01
ping_9_msec = 4.67
ping_10_msec = 4.98
Starting speed test...
==== rmbt 217f17f31 ====
connecting...
connected with 3 flow(s) for dl; 3 flow(s) for ul
pretest downlink start... (min 1s)
pretest downlink end.
rtt_tcp_payload start... (11 times)
rtt_tcp_payload end.
downlink test start... (5s)
downlink test end.
pretest uplink start... (min 1s)
pretest uplink end.
uplink test start... (5s)
uplink test end.
disconnecting.
dl_throughput_mbps = 311.058123
ul_throughput_mbps = 163.598457
Exiting.
Speed test result:
{
"cnf_file_flows":"\/tmp\/tmppoty0vv4.xz"
}
{
"res_id_test":"yyyyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy",
"res_time_start_s":1624869261,
"res_time_end_s":1624869275,
"res_status":"success",
"res_status_msg":null,
"res_version_client":"217f17f31",
"res_version_server":"RMBTv0.3",
"res_server_ip":"217.31.202.97",
"res_server_port":8081,
"res_encrypt":false,
"res_chunksize":4096,
"res_tcp_congestion":"cubic",
"res_total_bytes_dl":219333470,
"res_total_bytes_ul":115688700,
"res_uname_sysname":"Linux",
"res_uname_nodename":"turris",
"res_uname_release":"4.14.236",
"res_uname_version":"#0 SMP Wed Jun 16 00:15:12 2021",
"res_uname_machine":"armv7l",
"res_rtt_tcp_payload_num":11,
"res_rtt_tcp_payload_client_ns":5068863,
"res_rtt_tcp_payload_server_ns":4689316,
"res_dl_num_flows":3,
"res_dl_time_ns":5088070543,
"res_dl_bytes":197835709,
"res_dl_throughput_kbps":311058.12284332549,
"res_ul_num_flows":3,
"res_ul_time_ns":5037242320,
"res_ul_bytes":103010634,
"res_ul_throughput_kbps":163598.45718123007
}
Sending the following request to https://control.netmetr.cz/RMBTControlServer/result
(speed detail omitted)
{
"client_language": "en_US",
"client_name": "HW-PROBE",
"client_version": "0.1",
"client_software_version": "Python netmetr client v1.6.0",
"geoLocations": [],
"model": "Turris Omnia",
"network_type": 98,
"product": "os: 5.2.2 hw: cznic,turris-omnia",
"test_bytes_download": 649290658,
"test_bytes_upload": 232981900,
"test_nsec_download": 5066639628,
"test_nsec_upload": 5061379096,
"test_num_threads": 3,
"test_ping_shortest": 2250000,
"num_threads_ul": 3,
"test_speed_download": 940827.3755364075,
"test_speed_upload": 288109.34022951126,
"test_token": "yyyyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy_yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy",
"pings": []
}
response:
{
"error": []
}
Download sync code from the control server.
Sending the following request to https://control.netmetr.cz/RMBTControlServer/sync
{
"language": "en_US",
"timezone": "CEST",
"uuid": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}
response:
{
"sync": [
{
"sync_code": "xxxxxxxxx"
}
],
"error": []
}
Your Sync code is: xxxxxxxxx
The server sends explanatory messages with every error it returns. It would be really helpful for debugging, if we could print this message and log it somehow. For example, the messages could be used as messages in exceptions.
Netmetr has been superseded by LibreSpeed and is no longer maintained. I'll close this issue.
Netmetr has been superseded by LibreSpeed and is no longer maintained. I'll close this issue.
Netmetr has been superseded by LibreSpeed and is no longer maintained. I'll close this issue.
Netmetr has been superseded by LibreSpeed and is no longer maintained. I'll close this issue.
I installed netmetr through reforis, didn't enable autostart in the netmetr tab in reforis, waited 12 hours and when I checked back, netmetr ran 7 hours ago.
Something similar was reported here: https://forum.turris.cz/t/turris-os-6-0-is-now-in-hbt-testing-branch/17776/22, but that user had netmetr installed before updating from TOS5 to TOS6 and it happens every 2 hours for them.
For example with this configuration:
config settings 'settings'
option control_server 'control.netmetr.cz'
option max_history_logs '10'
option client 'HW-PROBE'
option uuid 'xYxxYYxY-xYYx-YxYx-xxYY-xYYxYYxxxYYx'
list hours_to_run '1'
list hours_to_run '3'
list hours_to_run '5'
list hours_to_run '7'
list hours_to_run '9'
list hours_to_run '11'
list hours_to_run '12'
list hours_to_run '14'
list hours_to_run '16'
list hours_to_run '18'
list hours_to_run '20'
list hours_to_run '22'
list hours_to_run '23'
option autostart_enabled '0'
option sync_code 'xYxYYxYxxYxx'
Netmetr has been superseded by LibreSpeed. Thus I think that this issue should be closed.
Since it is possible to measure both IPv4 and IPv6 from version 2.0.0, the raw text output is not so predictable and would be harder to parse. Therefore, JSON output would come handy for parsers like foris controller. This task consists of two subtasks:
@mmatejek will you please take care of the first subtask?
Netmetr integration with Turris OS will be dropped in upcoming Turris OS releases, therefore any updates for netmetr 2.0 will no longer be needed.
Therefore, I am closing this issue.
I have the same problem: root@turris:~# cat /etc/config/netmetr
config settings 'settings' option control_server 'control.netmetr.cz' option max_history_logs '30' option uuid '35XXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX' option client 'HW-PROBE' option autostart_delay '1392' option protocol_mode 'prefer_6' option autostart_enabled '0' list hours_to_run '4' option sync_code 'YYYYYYYYYYYY'
I tried to remove the following lines, but it did not help: option autostart_delay '1392' option autostart_enabled '0' list hours_to_run '4'
I installed netmetr through reforis, didn't enable autostart in the netmetr tab in reforis, waited 12 hours and when I checked back, netmetr ran 7 hours ago.
Something similar was reported here: https://forum.turris.cz/t/turris-os-6-0-is-now-in-hbt-testing-branch/17776/22, but that user had netmetr installed before updating from TOS5 to TOS6 and it happens every 2 hours for them.
For example with this configuration:
config settings 'settings'
option control_server 'control.netmetr.cz'
option max_history_logs '10'
option client 'HW-PROBE'
option uuid 'xYxxYYxY-xYYx-YxYx-xxYY-xYYxYYxxxYYx'
list hours_to_run '1'
list hours_to_run '3'
list hours_to_run '5'
list hours_to_run '7'
list hours_to_run '9'
list hours_to_run '11'
list hours_to_run '12'
list hours_to_run '14'
list hours_to_run '16'
list hours_to_run '18'
list hours_to_run '20'
list hours_to_run '22'
list hours_to_run '23'
option autostart_enabled '0'
option sync_code 'xYxYYxYxxYxx'
It seems that the current version of netmeter (1.6.0) unsuccessfully attempts to use ping to IPv6 address.
root@turris:~# netmetr
Checking uuid on the control server...
Requesting test config from the control server...
Starting ping test...
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
Starting speed test...
==== rmbt 027abd174 ====
connecting...
connected with 3 flow(s) for dl; 3 flow(s) for ul
pretest downlink start... (min 1s)
pretest downlink end.
rtt_tcp_payload start... (11 times)
rtt_tcp_payload end.
downlink test start... (5s)
downlink test end.
pretest uplink start... (min 1s)
pretest uplink end.
uplink test start... (5s)
uplink test end.
disconnecting.
dl_throughput_mbps = 50.419186
ul_throughput_mbps = 21.650615
Exiting.
Your Sync code is: e2660bf6aaf0
root@turris:~#
At the same time, ping6 on the same address works correctly
root@turris:~# ping6 ipv6.speed.netmetr.cz
PING ipv6.speed.netmetr.cz(speed.netmetr.cz) 56 data bytes
64 bytes from speed.netmetr.cz: icmp_seq=1 ttl=60 time=4.06 ms
64 bytes from speed.netmetr.cz: icmp_seq=2 ttl=60 time=4.47 ms
64 bytes from speed.netmetr.cz: icmp_seq=3 ttl=60 time=4.88 ms
64 bytes from speed.netmetr.cz: icmp_seq=4 ttl=60 time=4.34 ms
64 bytes from speed.netmetr.cz: icmp_seq=5 ttl=60 time=8.55 ms
64 bytes from speed.netmetr.cz: icmp_seq=6 ttl=60 time=4.39 ms
64 bytes from speed.netmetr.cz: icmp_seq=7 ttl=60 time=5.21 ms
64 bytes from speed.netmetr.cz: icmp_seq=8 ttl=60 time=4.03 ms
64 bytes from speed.netmetr.cz: icmp_seq=9 ttl=60 time=4.03 ms
64 bytes from speed.netmetr.cz: icmp_seq=10 ttl=60 time=8.40 ms
64 bytes from speed.netmetr.cz: icmp_seq=11 ttl=60 time=4.52 ms
original IPv4 ping does not work
root@turris:~# ping ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
root@turris:~#
At the same time, reForis overview shows that the IPv6 connection works correctly.
It seems that the current version of netmeter (1.6.0) unsuccessfully attempts to use ping to IPv6 address.
root@turris:~# netmetr
Checking uuid on the control server...
Requesting test config from the control server...
Starting ping test...
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
Starting speed test...
==== rmbt 027abd174 ====
connecting...
connected with 3 flow(s) for dl; 3 flow(s) for ul
pretest downlink start... (min 1s)
pretest downlink end.
rtt_tcp_payload start... (11 times)
rtt_tcp_payload end.
downlink test start... (5s)
downlink test end.
pretest uplink start... (min 1s)
pretest uplink end.
uplink test start... (5s)
uplink test end.
disconnecting.
dl_throughput_mbps = 50.419186
ul_throughput_mbps = 21.650615
Exiting.
Your Sync code is: e2660bf6aaf0
root@turris:~#
At the same time, ping6 on the same address works correctly
root@turris:~# ping6 ipv6.speed.netmetr.cz
PING ipv6.speed.netmetr.cz(speed.netmetr.cz) 56 data bytes
64 bytes from speed.netmetr.cz: icmp_seq=1 ttl=60 time=4.06 ms
64 bytes from speed.netmetr.cz: icmp_seq=2 ttl=60 time=4.47 ms
64 bytes from speed.netmetr.cz: icmp_seq=3 ttl=60 time=4.88 ms
64 bytes from speed.netmetr.cz: icmp_seq=4 ttl=60 time=4.34 ms
64 bytes from speed.netmetr.cz: icmp_seq=5 ttl=60 time=8.55 ms
64 bytes from speed.netmetr.cz: icmp_seq=6 ttl=60 time=4.39 ms
64 bytes from speed.netmetr.cz: icmp_seq=7 ttl=60 time=5.21 ms
64 bytes from speed.netmetr.cz: icmp_seq=8 ttl=60 time=4.03 ms
64 bytes from speed.netmetr.cz: icmp_seq=9 ttl=60 time=4.03 ms
64 bytes from speed.netmetr.cz: icmp_seq=10 ttl=60 time=8.40 ms
64 bytes from speed.netmetr.cz: icmp_seq=11 ttl=60 time=4.52 ms
original IPv4 ping does not work
root@turris:~# ping ipv6.speed.netmetr.cz
ping: unknown host ipv6.speed.netmetr.cz
root@turris:~#
At the same time, reForis overview shows that the IPv6 connection works correctly.
Not completely the same thing, now the problem seems to be the other way around. See labs/netmetr#54 (moved) for details.
cc @kaklik
The server sends explanatory messages with every error it returns. It would be really helpful for debugging, if we could print this message and log it somehow. For example, the messages could be used as messages in exceptions.
We should add more fields to the --debug
output to improve debugging of (not only) server-side issues.
Among others, we should add these fields:
Current debug output looks like this:
Checking uuid on the control server...
Sending the following request to https://control.netmetr.cz/RMBTControlServer/settings?skip_history=true
{
"uuid": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"language": "en_US",
"name": "RMBT",
"terms_and_conditions_accepted": "true",
"type": "DESKTOP",
"version_name": "Python netmetr client v1.6.0"
}
response:
{
"settings": [
{
"urls": {
"url_ipv6_check": "https://ipv6.control.netmetr.cz/RMBTControlServer/ip",
"control_ipv4_only": "ipv4.control.netmetr.cz",
"url_ipv4_check": "https://ipv4.control.netmetr.cz/RMBTControlServer/ip",
"control_ipv6_only": "ipv6.control.netmetr.cz",
"statistics": "https://www.netmetr.cz/cs/statistiky_app.html"
},
"qostesttype_desc": [
{
"name": "DNS",
"test_type": "DNS"
},
{
"name": "Non Transparent Proxy",
"test_type": "NON_TRANSPARENT_PROXY"
},
{
"name": "HTTP Proxy",
"test_type": "HTTP_PROXY"
},
{
"name": "Website",
"test_type": "WEBSITE"
},
{
"name": "UDP",
"test_type": "UDP"
},
{
"name": "TCP",
"test_type": "TCP"
},
{
"name": "JITTER",
"test_type": "JITTER"
}
],
"versions": {
"control_server_version": "master_2.0.83-CZ_0.9.28-354-gc1fca50"
}
}
]
}
Requesting test config from the control server...
Sending the following request to https://control.netmetr.cz/RMBTControlServer/testRequest
{
"client": "HW-PROBE",
"language": "en_US",
"time": 1624869256000,
"timezone": "CEST",
"type": "DESKTOP",
"uuid": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"version": "0.1"
}
response:
{
"user_server_selection": false,
"test_server_address": "ipv4.speed.netmetr.cz",
"test_server_port": 8081,
"test_server_name": "CZ.NIC (Prague)",
"test_server_encryption": false,
"test_server_type": "RMBT",
"test_duration": "5",
"test_numthreads": "3",
"test_numpings": "10",
"client_remote_ip": "94.199.96.205",
"result_url": "https://control.netmetr.cz/RMBTControlServer/result",
"result_qos_url": "https://control.netmetr.cz/RMBTControlServer/resultQoS",
"test_token": "yyyyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy_yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy",
"test_uuid": "yyyyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy",
"test_id": 12345678,
"test_wait": 0,
"error": []
}
Starting ping test...
ping_1_msec = 5.33
ping_2_msec = 4.65
ping_3_msec = 4.58
ping_4_msec = 4.84
ping_5_msec = 4.57
ping_6_msec = 4.69
ping_7_msec = 4.70
ping_8_msec = 6.01
ping_9_msec = 4.67
ping_10_msec = 4.98
Starting speed test...
==== rmbt 217f17f31 ====
connecting...
connected with 3 flow(s) for dl; 3 flow(s) for ul
pretest downlink start... (min 1s)
pretest downlink end.
rtt_tcp_payload start... (11 times)
rtt_tcp_payload end.
downlink test start... (5s)
downlink test end.
pretest uplink start... (min 1s)
pretest uplink end.
uplink test start... (5s)
uplink test end.
disconnecting.
dl_throughput_mbps = 311.058123
ul_throughput_mbps = 163.598457
Exiting.
Speed test result:
{
"cnf_file_flows":"\/tmp\/tmppoty0vv4.xz"
}
{
"res_id_test":"yyyyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy",
"res_time_start_s":1624869261,
"res_time_end_s":1624869275,
"res_status":"success",
"res_status_msg":null,
"res_version_client":"217f17f31",
"res_version_server":"RMBTv0.3",
"res_server_ip":"217.31.202.97",
"res_server_port":8081,
"res_encrypt":false,
"res_chunksize":4096,
"res_tcp_congestion":"cubic",
"res_total_bytes_dl":219333470,
"res_total_bytes_ul":115688700,
"res_uname_sysname":"Linux",
"res_uname_nodename":"turris",
"res_uname_release":"4.14.236",
"res_uname_version":"#0 SMP Wed Jun 16 00:15:12 2021",
"res_uname_machine":"armv7l",
"res_rtt_tcp_payload_num":11,
"res_rtt_tcp_payload_client_ns":5068863,
"res_rtt_tcp_payload_server_ns":4689316,
"res_dl_num_flows":3,
"res_dl_time_ns":5088070543,
"res_dl_bytes":197835709,
"res_dl_throughput_kbps":311058.12284332549,
"res_ul_num_flows":3,
"res_ul_time_ns":5037242320,
"res_ul_bytes":103010634,
"res_ul_throughput_kbps":163598.45718123007
}
Sending the following request to https://control.netmetr.cz/RMBTControlServer/result
(speed detail omitted)
{
"client_language": "en_US",
"client_name": "HW-PROBE",
"client_version": "0.1",
"client_software_version": "Python netmetr client v1.6.0",
"geoLocations": [],
"model": "Turris Omnia",
"network_type": 98,
"product": "os: 5.2.2 hw: cznic,turris-omnia",
"test_bytes_download": 649290658,
"test_bytes_upload": 232981900,
"test_nsec_download": 5066639628,
"test_nsec_upload": 5061379096,
"test_num_threads": 3,
"test_ping_shortest": 2250000,
"num_threads_ul": 3,
"test_speed_download": 940827.3755364075,
"test_speed_upload": 288109.34022951126,
"test_token": "yyyyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy_yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy",
"pings": []
}
response:
{
"error": []
}
Download sync code from the control server.
Sending the following request to https://control.netmetr.cz/RMBTControlServer/sync
{
"language": "en_US",
"timezone": "CEST",
"uuid": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}
response:
{
"sync": [
{
"sync_code": "xxxxxxxxx"
}
],
"error": []
}
Your Sync code is: xxxxxxxxx