mirror of
https://github.com/markqvist/Reticulum.git
synced 2026-11-22 16:06:25 -05:00
Compare commits
1084 commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f55c0c6ea6 | ||
|
|
840eb3699b | ||
|
|
cff4d03fdb | ||
|
|
528986836b | ||
|
|
01ffb98ee5 | ||
|
|
eb00622fef | ||
|
|
e8b3b14b73 | ||
|
|
ef3e80868e | ||
|
|
48ba76df9c | ||
|
|
41260ea163 | ||
|
|
1f7cd14513 | ||
|
|
b9903c53a8 | ||
|
|
1a71073cc0 | ||
|
|
f7b54ccc11 | ||
|
|
3f55e9ebeb | ||
|
|
c90a5d5887 | ||
|
|
9568b70458 | ||
|
|
8e7dc8b776 | ||
|
|
4a058544a5 | ||
|
|
8c73e203b1 | ||
|
|
70eb417d90 | ||
|
|
5bd93ad979 | ||
|
|
d72cfdd7f0 | ||
|
|
833f6cf2aa | ||
|
|
05ac4bba8e | ||
|
|
0123133e71 | ||
|
|
f876ed8dd9 | ||
|
|
48c7706188 | ||
|
|
4916017aee | ||
|
|
12b2b8c013 | ||
|
|
cc5d6dd065 | ||
|
|
25cce0a847 | ||
|
|
2b1a984c91 | ||
|
|
338df09607 | ||
|
|
aecc1547da | ||
|
|
da368e05e7 | ||
|
|
884f109b2b | ||
|
|
fd36a97220 | ||
|
|
51fa841327 | ||
|
|
b7a93b9b6b | ||
|
|
13b98a6bc2 | ||
|
|
e35bd08f18 | ||
|
|
715707b1bf | ||
|
|
d077d1ab00 | ||
|
|
bb63dd4406 | ||
|
|
e96c81c3f2 | ||
|
|
29ce7ca568 | ||
|
|
ad6c709e80 | ||
|
|
f361a354ba | ||
|
|
096e6726b4 | ||
|
|
a450be5d7e | ||
|
|
85d3697938 | ||
|
|
29dc449af8 | ||
|
|
c850cdac6c | ||
|
|
350b7e2614 | ||
|
|
56b23c4810 | ||
|
|
464e75986e | ||
|
|
3281e28315 | ||
|
|
daec67a116 | ||
|
|
819076c378 | ||
|
|
d183791956 | ||
|
|
01e7ed8443 | ||
|
|
d5f1e014db | ||
|
|
6852971d3c | ||
|
|
04585ff2a8 | ||
|
|
54159d1d93 | ||
|
|
134dc6777c | ||
|
|
20e25a9f0c | ||
|
|
9b6b724b08 | ||
|
|
f96cef7ed9 | ||
|
|
db67adea0a | ||
|
|
3c4b271f41 | ||
|
|
5f2dbc169d | ||
|
|
9ef6c90ea6 | ||
|
|
0bc5c37c39 | ||
|
|
1855f9927f | ||
|
|
70b9d58f15 | ||
|
|
7852f53a4c | ||
|
|
3ac3236b2f | ||
|
|
023849ed29 | ||
|
|
50b6733a1c | ||
|
|
16621a78e3 | ||
|
|
132ec70f75 | ||
|
|
aaf08e6107 | ||
|
|
2c7cf8272b | ||
|
|
c5d205ffea | ||
|
|
e0da9c381e | ||
|
|
d4af18fd6f | ||
|
|
cb1260891d | ||
|
|
2960e6a41d | ||
|
|
baccc6d8a2 | ||
|
|
1fee7af7f2 | ||
|
|
dad1302b1e | ||
|
|
9d67be9acb | ||
|
|
50e939ab0e | ||
|
|
29e5a997ac | ||
|
|
01aef5bc2a | ||
|
|
65a7208a9d | ||
|
|
acb38a8a1c | ||
|
|
c97e2158f5 | ||
|
|
426673e974 | ||
|
|
f15b34870a | ||
|
|
eb826cde90 | ||
|
|
e9d82e698d | ||
|
|
3305fc1cbd | ||
|
|
53dc223a03 | ||
|
|
9b6678b317 | ||
|
|
ceac4a28bd | ||
|
|
8a09c9b550 | ||
|
|
e49fafecce | ||
|
|
ecd677ba7f | ||
|
|
cda4d72bb8 | ||
|
|
d5891fde14 | ||
|
|
fa683e6c21 | ||
|
|
d3874ddedb | ||
|
|
825c2e1993 | ||
|
|
ade030ada9 | ||
|
|
0cc6f5aef3 | ||
|
|
263dc6bae5 | ||
|
|
6927c3c589 | ||
|
|
22bf35d59d | ||
|
|
e64ce99565 | ||
|
|
af4504d675 | ||
|
|
2117087710 | ||
|
|
c4ed59be2c | ||
|
|
1d9067c3bd | ||
|
|
f1fc51ba09 | ||
|
|
fe2fe94dd8 | ||
|
|
d45a5860fd | ||
|
|
eab2983b95 | ||
|
|
1ee89ad8d6 | ||
|
|
2ef855c892 | ||
|
|
81b6ba364c | ||
|
|
9152bc1319 | ||
|
|
c147201470 | ||
|
|
5b2b8afbdf | ||
|
|
1a6e26ac2b | ||
|
|
9b46438233 | ||
|
|
2e6963c388 | ||
|
|
da027e9c0f | ||
|
|
f57fa2655c | ||
|
|
796ef2d44b | ||
|
|
75465efad8 | ||
|
|
34bb4ac88c | ||
|
|
2c142973bf | ||
|
|
a8cc6cb239 | ||
|
|
5f46fc6b48 | ||
|
|
78fb90ea6f | ||
|
|
c0fe08c08d | ||
|
|
d7749b0b06 | ||
|
|
5d38ce3072 | ||
|
|
025f6d609f | ||
|
|
ba3c42459b | ||
|
|
134db17c81 | ||
|
|
4867e889e2 | ||
|
|
ca6451a02a | ||
|
|
ba8c2d76db | ||
|
|
c37b5f7f44 | ||
|
|
9955e1bf67 | ||
|
|
30c6428e0b | ||
|
|
c3af19893f | ||
|
|
8d5871915a | ||
|
|
7c1b648607 | ||
|
|
c68066edff | ||
|
|
51be88fbd8 | ||
|
|
7ec975fc88 | ||
|
|
156c7323a5 | ||
|
|
d3ec00cd51 | ||
|
|
d2dd730801 | ||
|
|
d1da5c3d67 | ||
|
|
9a01a20ce6 | ||
|
|
4ebd4dcc5b | ||
|
|
3e83fcc18b | ||
|
|
156290733e | ||
|
|
044d4c9adc | ||
|
|
af8d05819d | ||
|
|
e24d207312 | ||
|
|
f3ebec9cb1 | ||
|
|
ee82fa75a7 | ||
|
|
ae67448e88 | ||
|
|
a29bfb51c4 | ||
|
|
318b78b416 | ||
|
|
2578750bf7 | ||
|
|
cadee01f9e | ||
|
|
f0875ac44e | ||
|
|
49ae71e06c | ||
|
|
1e6ebd3fa0 | ||
|
|
2bae9ff0dc | ||
|
|
d1a7e0c8a0 | ||
|
|
fca509ff04 | ||
|
|
e40191b3d1 | ||
|
|
7f2b3b9b52 | ||
|
|
0a25e1a9c8 | ||
|
|
ddeb44b1de | ||
|
|
39a888c1e2 | ||
|
|
1f74990752 | ||
|
|
a714c20043 | ||
|
|
942434b288 | ||
|
|
47318034dd | ||
|
|
bb1320913a | ||
|
|
fc95caf519 | ||
|
|
6d1409be1c | ||
|
|
0a0e661beb | ||
|
|
26aec0040e | ||
|
|
135e941fa2 | ||
|
|
1d9ebe8c38 | ||
|
|
4f79110995 | ||
|
|
5cdff28703 | ||
|
|
06dc4ad550 | ||
|
|
6c9d717cd6 | ||
|
|
f25b31702b | ||
|
|
73c60e6a49 | ||
|
|
cce96d38c6 | ||
|
|
71583c5c2d | ||
|
|
d5962d14eb | ||
|
|
d3153bd778 | ||
|
|
69c425d9ae | ||
|
|
a93c6ba5be | ||
|
|
1cf176f6e5 | ||
|
|
b899389956 | ||
|
|
3ad70c63ea | ||
|
|
3b7429149e | ||
|
|
e2ba876ebf | ||
|
|
6ecda49394 | ||
|
|
d23261c8d9 | ||
|
|
84709ccf09 | ||
|
|
7283cb417a | ||
|
|
2c30a88e85 | ||
|
|
24b1ac521e | ||
|
|
e68f4ff118 | ||
|
|
c95fd8e6ca | ||
|
|
3f95b47282 | ||
|
|
aeccf69fc7 | ||
|
|
8a7ad40d64 | ||
|
|
c4be063650 | ||
|
|
87a75c2553 | ||
|
|
c963581a6f | ||
|
|
1f7dbe6dde | ||
|
|
0af42a6885 | ||
|
|
ff4e04099c | ||
|
|
780b7e6bfc | ||
|
|
192898864c | ||
|
|
c7be675739 | ||
|
|
21a2365045 | ||
|
|
a22de54b4a | ||
|
|
99de23c040 | ||
|
|
395241b190 | ||
|
|
15e6dd4a88 | ||
|
|
34b36362ab | ||
|
|
ddd55d50cc | ||
|
|
cb66b3d10f | ||
|
|
8743ddcbe5 | ||
|
|
23101e45f6 | ||
|
|
2289a8effa | ||
|
|
3c479ddb65 | ||
|
|
9d85931c9d | ||
|
|
58881af307 | ||
|
|
7373f81349 | ||
|
|
29f6bc8652 | ||
|
|
abf355b4d7 | ||
|
|
484cf2d1a5 | ||
|
|
e699bb23f1 | ||
|
|
badd850088 | ||
|
|
ffad91431a | ||
|
|
d0231f6ee3 | ||
|
|
b4c6deee78 | ||
|
|
900b4c607c | ||
|
|
9397114f56 | ||
|
|
16ba292bd0 | ||
|
|
032e14384f | ||
|
|
b79f6e0b1c | ||
|
|
3b0425c8a8 | ||
|
|
b043414add | ||
|
|
1e7432af39 | ||
|
|
b67872d946 | ||
|
|
99bfde84bf | ||
|
|
0bc132d7df | ||
|
|
28eba69900 | ||
|
|
1565126ffd | ||
|
|
0dbc9e90a3 | ||
|
|
9199cc9e4b | ||
|
|
7785fd277a | ||
|
|
851c18bc27 | ||
|
|
996d822393 | ||
|
|
0c97853554 | ||
|
|
fae64abf05 | ||
|
|
7330c20c6d | ||
|
|
e7a7c48c0a | ||
|
|
f231fdcb4d | ||
|
|
87823c091f | ||
|
|
36a62e7b1a | ||
|
|
859dc81805 | ||
|
|
409380a458 | ||
|
|
b0933d5c6f | ||
|
|
0bb41bf948 | ||
|
|
27910f25a1 | ||
|
|
602d52f178 | ||
|
|
7396e3994e | ||
|
|
8a82a50fc9 | ||
|
|
5c1141d1ce | ||
|
|
6b6dd73bc8 | ||
|
|
1a7e9e31a1 | ||
|
|
2f29b56e96 | ||
|
|
3bc149e3d5 | ||
|
|
eb8414bee9 | ||
|
|
ea98db4f53 | ||
|
|
ad3c06636b | ||
|
|
9ce4502961 | ||
|
|
a3cd84111f | ||
|
|
d5b2fc5609 | ||
|
|
83a30b187a | ||
|
|
943771a3f9 | ||
|
|
5b4117dadc | ||
|
|
6bc0481cdf | ||
|
|
4eebf0b685 | ||
|
|
149e415109 | ||
|
|
53ed9c1ed3 | ||
|
|
ccc5468bcc | ||
|
|
c1d7c12b52 | ||
|
|
9878b18377 | ||
|
|
6e8c976421 | ||
|
|
b28f5ebf7a | ||
|
|
77c8256a1a | ||
|
|
dc9cf0c2f4 | ||
|
|
fa07c8c87c | ||
|
|
b7fe01237d | ||
|
|
3543dc9802 | ||
|
|
5b04a88026 | ||
|
|
5fdd661081 | ||
|
|
ae0191578f | ||
|
|
e714c559cc | ||
|
|
75cbc73f64 | ||
|
|
d32ba8c1a1 | ||
|
|
ff3a72209e | ||
|
|
803b548951 | ||
|
|
67a45ac5e9 | ||
|
|
e71c01959f | ||
|
|
0a1b9453b2 | ||
|
|
79cad39e78 | ||
|
|
de0dac695b | ||
|
|
281c47f3e0 | ||
|
|
99c428a9f5 | ||
|
|
cb9071b478 | ||
|
|
10848b7cff | ||
|
|
89e73db94a | ||
|
|
0ec84c7491 | ||
|
|
ac9130f01f | ||
|
|
54a919f045 | ||
|
|
bafc09d3a6 | ||
|
|
419956bada | ||
|
|
1a16fdb727 | ||
|
|
3b7cfc0666 | ||
|
|
de76b8939f | ||
|
|
d80245b62c | ||
|
|
dcbc7638d0 | ||
|
|
0536b972d7 | ||
|
|
47add6381d | ||
|
|
be2ba7c2f3 | ||
|
|
dd204e11ce | ||
|
|
be4ee32908 | ||
|
|
9f66b5a6a3 | ||
|
|
d38a8de571 | ||
|
|
dea0124c57 | ||
|
|
aba8d606dd | ||
|
|
8221f82dc0 | ||
|
|
38e9d1cdd4 | ||
|
|
17e980ff79 | ||
|
|
f111709902 | ||
|
|
2d2167140d | ||
|
|
d044db2931 | ||
|
|
516cb106c1 | ||
|
|
7e197542e5 | ||
|
|
77f7632584 | ||
|
|
5e013464da | ||
|
|
b9278ce352 | ||
|
|
d81421dad3 | ||
|
|
629e4fde2d | ||
|
|
7311dc8544 | ||
|
|
0e070aacf6 | ||
|
|
39899651ee | ||
|
|
1034d78828 | ||
|
|
9c2f424aaa | ||
|
|
dca5b9639e | ||
|
|
40281f91da | ||
|
|
cf5d6a796e | ||
|
|
9302415f9e | ||
|
|
9da6664976 | ||
|
|
1694a17a75 | ||
|
|
3fdfe93ec7 | ||
|
|
a9538e9fd1 | ||
|
|
956d688e10 | ||
|
|
1c83e732ad | ||
|
|
5da0870e2a | ||
|
|
171868c6cb | ||
|
|
cfddb9abe6 | ||
|
|
929aba0282 | ||
|
|
7347034f9a | ||
|
|
aef9e5b416 | ||
|
|
e806ae5837 | ||
|
|
602085a1cd | ||
|
|
84597f3186 | ||
|
|
88c629e3ad | ||
|
|
2058596dd1 | ||
|
|
b123a756b0 | ||
|
|
bfab2964b6 | ||
|
|
1bad7f5807 | ||
|
|
26e3ca4f2b | ||
|
|
d25ea38c84 | ||
|
|
e32d4df754 | ||
|
|
561e2f23e1 | ||
|
|
b397870c97 | ||
|
|
6f6751d6b6 | ||
|
|
2aed542e61 | ||
|
|
091e021d0e | ||
|
|
fc69f323a8 | ||
|
|
6b2c5bcf0b | ||
|
|
3885d23293 | ||
|
|
b1aa527861 | ||
|
|
4c69b376b2 | ||
|
|
2b1b9589f1 | ||
|
|
378840a6bf | ||
|
|
7b8923b61f | ||
|
|
b3ef214e72 | ||
|
|
cab513fa31 | ||
|
|
60eb0509f2 | ||
|
|
42f6d64b9c | ||
|
|
6ba7cc8cb4 | ||
|
|
880db0a7b7 | ||
|
|
5f2f4438d0 | ||
|
|
7488336985 | ||
|
|
614e7bd834 | ||
|
|
386ef1f370 | ||
|
|
4ab0755d0a | ||
|
|
d478e380c9 | ||
|
|
05e6717d21 | ||
|
|
9ae3db169e | ||
|
|
68cda4a855 | ||
|
|
4b914fb9a4 | ||
|
|
bde5611a0d | ||
|
|
49073fcca5 | ||
|
|
6738db5437 | ||
|
|
49918c7e1d | ||
|
|
954567c581 | ||
|
|
77a1bb9b19 | ||
|
|
df80181006 | ||
|
|
0c0034f1ae | ||
|
|
2e9443ffe2 | ||
|
|
65222e0de8 | ||
|
|
db7daa4d94 | ||
|
|
c95bb551ab | ||
|
|
acecc1f490 | ||
|
|
39e3854dac | ||
|
|
6cd5be18ec | ||
|
|
0e2041c837 | ||
|
|
338173802d | ||
|
|
40a862acbd | ||
|
|
d825e39379 | ||
|
|
731a63b01b | ||
|
|
7c912d0936 | ||
|
|
503bd6c87b | ||
|
|
1a70bd3cd1 | ||
|
|
636012bbe5 | ||
|
|
e1b7bc316c | ||
|
|
1c488947bd | ||
|
|
1809461a07 | ||
|
|
01a78bee2a | ||
|
|
8c08b9ce09 | ||
|
|
efb8c8500d | ||
|
|
f366dd9cf2 | ||
|
|
0d15cfc134 | ||
|
|
e81532f541 | ||
|
|
d41afd2d81 | ||
|
|
7a1291d530 | ||
|
|
fc0f84f23e | ||
|
|
c2eac12ff5 | ||
|
|
e46496032c | ||
|
|
2d7f858a54 | ||
|
|
9ebcb55cb5 | ||
|
|
bbc1a0d06b | ||
|
|
31298e5eda | ||
|
|
e31c570d2b | ||
|
|
9b4947ef46 | ||
|
|
0d9bf5b862 | ||
|
|
c6f9ef1047 | ||
|
|
90ac62620d | ||
|
|
0c41027765 | ||
|
|
691211bf0d | ||
|
|
0ec51cce67 | ||
|
|
55755c6b3c | ||
|
|
72ba27d63c | ||
|
|
6c1182c2c2 | ||
|
|
ce81b339dc | ||
|
|
3db15dae44 | ||
|
|
38a73f9508 | ||
|
|
ebd9b862e9 | ||
|
|
4fc8e03d65 | ||
|
|
b48b96e616 | ||
|
|
529a9fd460 | ||
|
|
64fee86ebc | ||
|
|
e3f1a5e7cd | ||
|
|
0416c41956 | ||
|
|
4760103aa6 | ||
|
|
b2188ce9a7 | ||
|
|
4631d78bea | ||
|
|
0d16e2305e | ||
|
|
e5d37355b8 | ||
|
|
224124aac7 | ||
|
|
e29b839429 | ||
|
|
12c21d4e25 | ||
|
|
6a761a768c | ||
|
|
1ecf845cea | ||
|
|
a29a08716f | ||
|
|
e46b012e95 | ||
|
|
0f33d71966 | ||
|
|
00d57a22ee | ||
|
|
c25b56db9a | ||
|
|
a4b6129875 | ||
|
|
bce5f859da | ||
|
|
93526c17f5 | ||
|
|
6d4523c27e | ||
|
|
c4297d318f | ||
|
|
90e1dbb912 | ||
|
|
cbf50460c2 | ||
|
|
889c5ff55f | ||
|
|
566aa68fc0 | ||
|
|
3ca71527d9 | ||
|
|
5577e781fb | ||
|
|
7611fca6bf | ||
|
|
c10c465c02 | ||
|
|
9bc73819f6 | ||
|
|
bebf211b01 | ||
|
|
1af173e8a5 | ||
|
|
b051e76d00 | ||
|
|
4838875691 | ||
|
|
122f17fad6 | ||
|
|
be36abd857 | ||
|
|
fa4d4c6716 | ||
|
|
a5728be421 | ||
|
|
032b1aa3b2 | ||
|
|
fb7479a6f6 | ||
|
|
f81b267509 | ||
|
|
6527423526 | ||
|
|
d8bc20d4cb | ||
|
|
6c6238ce29 | ||
|
|
cbba3502b7 | ||
|
|
2b79db03c0 | ||
|
|
2c9edc43f1 | ||
|
|
ef9244f79f | ||
|
|
2d811dc398 | ||
|
|
e64d815033 | ||
|
|
bfade97051 | ||
|
|
3c4ef622cc | ||
|
|
3898d63689 | ||
|
|
ec8d43a548 | ||
|
|
5081255317 | ||
|
|
13a538168f | ||
|
|
a5ed0a4321 | ||
|
|
d93f9798ae | ||
|
|
88833f170b | ||
|
|
cd6911ed53 | ||
|
|
b95c51b96e | ||
|
|
3a36c367fe | ||
|
|
cf6010da59 | ||
|
|
bb2897445f | ||
|
|
406b141370 | ||
|
|
00b2f81a82 | ||
|
|
9adf045a31 | ||
|
|
de0f399a16 | ||
|
|
dca2a92829 | ||
|
|
b1b3ff7143 | ||
|
|
a0f0f31807 | ||
|
|
72db6e0ef2 | ||
|
|
b70688883a | ||
|
|
dd3ddb9d8a | ||
|
|
4ba1750c01 | ||
|
|
eacff56fe2 | ||
|
|
8768e163ae | ||
|
|
c0bc19d998 | ||
|
|
c99ec922aa | ||
|
|
0a6f76e41b | ||
|
|
156ce9cd34 | ||
|
|
176092ebf9 | ||
|
|
98817e3f9d | ||
|
|
276b7bb92f | ||
|
|
d1813e1ff0 | ||
|
|
5355a9bb20 | ||
|
|
48d17a8616 | ||
|
|
7e603f9606 | ||
|
|
ac6f0199f5 | ||
|
|
71f6f721fd | ||
|
|
51b7dc7cd4 | ||
|
|
03ddb08248 | ||
|
|
422dc05549 | ||
|
|
50c0a354c9 | ||
|
|
e98487c1cf | ||
|
|
50e03a24e8 | ||
|
|
0cc42568c7 | ||
|
|
41790ca707 | ||
|
|
20b1bfd01e | ||
|
|
1be5e10abe | ||
|
|
13a9ebed83 | ||
|
|
aee8c3be0c | ||
|
|
ef8ccf67ed | ||
|
|
91d531b053 | ||
|
|
0de169836e | ||
|
|
a2ef978208 | ||
|
|
5b3bb050e7 | ||
|
|
9b9efe5fac | ||
|
|
482d5ac4cb | ||
|
|
804a7ddbe1 | ||
|
|
739fdd387e | ||
|
|
f3517d2e4b | ||
|
|
c59f1e3dd6 | ||
|
|
2cf07099e3 | ||
|
|
0292465666 | ||
|
|
b4e15503c1 | ||
|
|
a5ca1ee41e | ||
|
|
f9c786fa28 | ||
|
|
bcf35030bc | ||
|
|
237eada209 | ||
|
|
afb984d3d4 | ||
|
|
675a25c90e | ||
|
|
f6d6314842 | ||
|
|
22ab5c29bd | ||
|
|
4931c6a54c | ||
|
|
d3fcc2a38c | ||
|
|
e9609b7f25 | ||
|
|
70cd51f0fa | ||
|
|
fa7699a37d | ||
|
|
2c9b794182 | ||
|
|
eae84ed8ba | ||
|
|
31539c5a0e | ||
|
|
abca32bca4 | ||
|
|
1d7cfe7c20 | ||
|
|
a516960e6f | ||
|
|
870bee16b1 | ||
|
|
3814102936 | ||
|
|
aaadff547d | ||
|
|
f0a3fadcd1 | ||
|
|
7cbce84cbd | ||
|
|
fe334c0d7c | ||
|
|
33d5a8e2a8 | ||
|
|
e80bf471ec | ||
|
|
7dfdea2395 | ||
|
|
74b61aebd2 | ||
|
|
ce9071e2d3 | ||
|
|
d6cf59dcc8 | ||
|
|
de61652d37 | ||
|
|
6181f62d93 | ||
|
|
ed66b4873e | ||
|
|
26869941a4 | ||
|
|
ee7b4e7ae5 | ||
|
|
7866484453 | ||
|
|
817b3b1a12 | ||
|
|
17f6968467 | ||
|
|
a96a1d6692 | ||
|
|
c1081fa9a4 | ||
|
|
dd3104094b | ||
|
|
dc68eea313 | ||
|
|
794e437f6d | ||
|
|
ebf544d335 | ||
|
|
939f30fef2 | ||
|
|
4549bbfdb9 | ||
|
|
6c989eb38e | ||
|
|
137d73ad0d | ||
|
|
58d4162f6d | ||
|
|
67625395fe | ||
|
|
f62512381a | ||
|
|
888e3102de | ||
|
|
f83435c697 | ||
|
|
5243d646f0 | ||
|
|
bdb284ce5d | ||
|
|
1b34820601 | ||
|
|
01010dd599 | ||
|
|
da32709f7c | ||
|
|
cdc6159a15 | ||
|
|
eb2f7ae455 | ||
|
|
a74f1bd89f | ||
|
|
603f709139 | ||
|
|
0dd063a32e | ||
|
|
9885a70a88 | ||
|
|
e4a85de089 | ||
|
|
ca0d2dffbe | ||
|
|
511d169c77 | ||
|
|
19bc8ef85c | ||
|
|
1e33d3eebb | ||
|
|
d18f434583 | ||
|
|
64749b4d18 | ||
|
|
875d8ef7eb | ||
|
|
20283f1536 | ||
|
|
c4af328802 | ||
|
|
a2193b9ffd | ||
|
|
d2542fd49b | ||
|
|
0333884877 | ||
|
|
63947ed69a | ||
|
|
d6d18ce29c | ||
|
|
2ef58d8b59 | ||
|
|
15f2d1635e | ||
|
|
c83b71f49a | ||
|
|
f0824fd71e | ||
|
|
8dde60658f | ||
|
|
9437648ae5 | ||
|
|
71b19aca2c | ||
|
|
7d320f8cd5 | ||
|
|
340d0883a7 | ||
|
|
66096acc29 | ||
|
|
e35100d865 | ||
|
|
128455ef01 | ||
|
|
10156cc90e | ||
|
|
4f5482f2ae | ||
|
|
2b9fdae74b | ||
|
|
7506caa0da | ||
|
|
b1f522277c | ||
|
|
af6e0c9ecf | ||
|
|
176567e3f1 | ||
|
|
15cd4268ac | ||
|
|
9307db16c4 | ||
|
|
0f29ab629a | ||
|
|
b2a4ceb853 | ||
|
|
6c7f1d068b | ||
|
|
b76beb602d | ||
|
|
0c68f6491a | ||
|
|
038981474a | ||
|
|
df0b4a5165 | ||
|
|
db7359f56d | ||
|
|
12e45b6483 | ||
|
|
ba8fca6f87 | ||
|
|
9b99b72f61 | ||
|
|
03cfbc2eb6 | ||
|
|
c92872a81b | ||
|
|
f3f4d9bca3 | ||
|
|
e7a317f0a0 | ||
|
|
d5b64a4af3 | ||
|
|
5667a0bbac | ||
|
|
7e46422c16 | ||
|
|
869a803149 | ||
|
|
f744e4d9a3 | ||
|
|
1a7607cba3 | ||
|
|
d881c111f6 | ||
|
|
bdac57ec0b | ||
|
|
c15f566cfa | ||
|
|
bdc79b9097 | ||
|
|
102eccb77d | ||
|
|
e8b236c7d8 | ||
|
|
d69491eb80 | ||
|
|
256a4d0b92 | ||
|
|
c5add012c1 | ||
|
|
6ecc8933b4 | ||
|
|
42b5661979 | ||
|
|
6333fb39bf | ||
|
|
ea27a8b8a7 | ||
|
|
358f9c3b0c | ||
|
|
cb3ef69072 | ||
|
|
eee9354657 | ||
|
|
ff86a1d7e6 | ||
|
|
e49f31322c | ||
|
|
95502e2c21 | ||
|
|
3dd4145e62 | ||
|
|
1d7ddc3f8a | ||
|
|
d731b4396c | ||
|
|
c186a1f6b0 | ||
|
|
a049ec8b7b | ||
|
|
4c93f6c7f4 | ||
|
|
35c7a89b19 | ||
|
|
c86b9c9703 | ||
|
|
64ebdd0ee3 | ||
|
|
9179b914d5 | ||
|
|
eb5d46b20b | ||
|
|
54c36f515b | ||
|
|
5c5668a4fc | ||
|
|
eeefb60c89 | ||
|
|
018df10a26 | ||
|
|
93ead77435 | ||
|
|
bd0e1ad0ca | ||
|
|
d0ceeacb37 | ||
|
|
7d5fb6a13f | ||
|
|
855ef7bfd1 | ||
|
|
323890021a | ||
|
|
e004e7592b | ||
|
|
0ebec014e5 | ||
|
|
1b624cc0e2 | ||
|
|
e8d161c0d5 | ||
|
|
e5c7dd7ec7 | ||
|
|
7d6ed59e6e | ||
|
|
11e4e7953a | ||
|
|
a5b292ee81 | ||
|
|
d619bafb8d | ||
|
|
0119a589dc | ||
|
|
b7346bed4d | ||
|
|
fcea57cb8e | ||
|
|
8d8af5e60a | ||
|
|
1a732ac1c1 | ||
|
|
f827d945be | ||
|
|
e03c4ee455 | ||
|
|
35e7ccb773 | ||
|
|
a932a10492 | ||
|
|
c5108c3a19 | ||
|
|
767782e425 | ||
|
|
60c440a3b6 | ||
|
|
6551a25877 | ||
|
|
70db2c5369 | ||
|
|
8ed31d0dc8 | ||
|
|
ef1ecb35e1 | ||
|
|
6768f10631 | ||
|
|
fee6a53473 | ||
|
|
bbfa3b0aa0 | ||
|
|
325ae654ef | ||
|
|
8655a4fb37 | ||
|
|
b30d272ee6 | ||
|
|
cc90ac2853 | ||
|
|
55473f39cb | ||
|
|
6d73881b07 | ||
|
|
d107cd4b42 | ||
|
|
33247e21b2 | ||
|
|
6bdc769af3 | ||
|
|
e923ccbf1b | ||
|
|
d402ee33a2 | ||
|
|
d8d420745f | ||
|
|
524f2068cd | ||
|
|
5db089ff19 | ||
|
|
08d6780c73 | ||
|
|
ca3f0bba6d | ||
|
|
830327e4a2 | ||
|
|
f96409dfa9 | ||
|
|
18e2da7d2b | ||
|
|
dfd046afb6 | ||
|
|
63d7f1e295 | ||
|
|
9d076d6a19 | ||
|
|
c6fa33a8aa | ||
|
|
37fa4392a5 | ||
|
|
90c88ade00 | ||
|
|
bb08f63a9f | ||
|
|
bdfad57d3f | ||
|
|
7ceb2d2078 | ||
|
|
304acdd0c1 | ||
|
|
8b6609c588 | ||
|
|
8a1d3aedd4 | ||
|
|
d49f100edd | ||
|
|
83d9ee1c5f | ||
|
|
b527c59735 | ||
|
|
23498a7a0a | ||
|
|
ac2cf79451 | ||
|
|
42b7426ed8 | ||
|
|
928c02099b | ||
|
|
c0ae63e27a | ||
|
|
62532e1c54 | ||
|
|
3136b53277 | ||
|
|
9352cff870 | ||
|
|
9e5fd0f079 | ||
|
|
1d37ba4780 | ||
|
|
134c1fb6ac | ||
|
|
24df04f304 | ||
|
|
26595bb25a | ||
|
|
5ee7dcf5a3 | ||
|
|
8b2ba9907f | ||
|
|
d1c59ef3b6 | ||
|
|
2dd23b15a8 | ||
|
|
93ad11f193 | ||
|
|
ec27d8bfde | ||
|
|
4d6e164d62 | ||
|
|
d82ffce504 | ||
|
|
7ecd435911 | ||
|
|
49f56e7d0d | ||
|
|
b8d6a14599 | ||
|
|
9c166936ad | ||
|
|
69db87cc24 | ||
|
|
5d86232fbe | ||
|
|
607e80bc82 | ||
|
|
f9625b2b88 | ||
|
|
3d8079c02b | ||
|
|
5c05a7fa58 | ||
|
|
2fa959a560 | ||
|
|
c39494d9fa | ||
|
|
a3cd1ea83d | ||
|
|
d4ddf6bb13 | ||
|
|
8661a3886b | ||
|
|
2ddbef70fe | ||
|
|
bb051e5a11 | ||
|
|
080085e813 | ||
|
|
85454b1f25 | ||
|
|
3f5653f650 | ||
|
|
b1357eb146 | ||
|
|
7731e799f4 | ||
|
|
15320e4d2c | ||
|
|
78596b687a | ||
|
|
729dc8dc11 | ||
|
|
3c08eb8122 | ||
|
|
9d12c86ac8 | ||
|
|
3bd573688c | ||
|
|
07ff87974e | ||
|
|
e8fa92950d | ||
|
|
ab6532742e | ||
|
|
4e583770e5 | ||
|
|
f9b6dc2ab8 | ||
|
|
1c2bc0c7b8 | ||
|
|
05760f914c | ||
|
|
3f6e8605af | ||
|
|
b6bfd1655c | ||
|
|
8cbd0e22ff | ||
|
|
15ec64e974 | ||
|
|
3de16e085e | ||
|
|
4cbd4ed60c | ||
|
|
b8fbd616e5 | ||
|
|
f8a79d2f51 | ||
|
|
0218ff4e26 | ||
|
|
1f3ce7e78f | ||
|
|
9009e1d232 | ||
|
|
cc73b2c2b9 | ||
|
|
dbf19ed054 | ||
|
|
a1cff4e8ab | ||
|
|
c9822968c8 | ||
|
|
8acabd95b5 | ||
|
|
49f6a6924d | ||
|
|
8d73265cf4 | ||
|
|
fceb7d18d7 | ||
|
|
337007cf70 | ||
|
|
4733d6d75a | ||
|
|
c8235544e8 | ||
|
|
3d1111ff02 | ||
|
|
83c9f2b10a | ||
|
|
734eb53aa7 | ||
|
|
6d39cb8e7c | ||
|
|
3c3f38b239 | ||
|
|
86d52d3884 | ||
|
|
6782672cb8 | ||
|
|
7fada7e5ab | ||
|
|
4380026a4e | ||
|
|
5143ea3d02 | ||
|
|
4802bcd829 | ||
|
|
6038096b95 | ||
|
|
2acfc31350 | ||
|
|
2742e5253f | ||
|
|
46f2e994b9 | ||
|
|
2c97a20c12 | ||
|
|
9be10ebd47 | ||
|
|
93cbfe7f7e | ||
|
|
4589de2115 | ||
|
|
662054ae25 | ||
|
|
3cf186f3cb | ||
|
|
7a91c82e4b | ||
|
|
72aace40d3 | ||
|
|
0c9a65b5f1 | ||
|
|
ea749499c3 | ||
|
|
828cbe7f20 | ||
|
|
1d8d547872 | ||
|
|
16c53221e3 | ||
|
|
74936010c4 | ||
|
|
f3245e1d65 | ||
|
|
1f74570ed9 | ||
|
|
88d1b7d2d1 | ||
|
|
fb5dcf0631 | ||
|
|
a23086d3fc | ||
|
|
a4cbcbca97 | ||
|
|
9dd008d42b | ||
|
|
76fa07cb90 | ||
|
|
35d72f27ed | ||
|
|
852891c779 | ||
|
|
f4aa7dc389 | ||
|
|
d7c3859f61 | ||
|
|
85d77c10a1 | ||
|
|
95222c7793 | ||
|
|
0a18b47e8c | ||
|
|
70f5126499 | ||
|
|
b60eab0fcf | ||
|
|
17310fc294 | ||
|
|
9c892dc1a4 | ||
|
|
c596dab806 | ||
|
|
fcb590e661 | ||
|
|
328017cca0 | ||
|
|
63dba562ae | ||
|
|
cf20f26098 | ||
|
|
e1e6063d17 | ||
|
|
ccbbe6f2f8 | ||
|
|
55c95bf59a | ||
|
|
043a5dc4e7 | ||
|
|
32a1cdf494 | ||
|
|
f924086198 | ||
|
|
6abb31e469 | ||
|
|
3eee369704 | ||
|
|
695d4d8684 | ||
|
|
015692d51e | ||
|
|
86004a89e5 | ||
|
|
86031ef3f8 | ||
|
|
034239daf3 | ||
|
|
a7b0f9924e | ||
|
|
a1d35b34b9 | ||
|
|
8d7e337dff | ||
|
|
de7e0996ce | ||
|
|
7377b69144 | ||
|
|
c933cfdaa3 | ||
|
|
726185cee2 | ||
|
|
de1000bfda | ||
|
|
555e8c0376 | ||
|
|
d836de3fe7 | ||
|
|
6ade1269ea | ||
|
|
a8b519e06e | ||
|
|
7d502306ea | ||
|
|
e9fa57c660 | ||
|
|
7d4ab17f0d | ||
|
|
d532902320 | ||
|
|
e592244443 | ||
|
|
c1def5da19 | ||
|
|
6a7f081f12 | ||
|
|
11555198eb | ||
|
|
6c77e27a50 | ||
|
|
17e8159fd8 | ||
|
|
c71f5d8c5e | ||
|
|
31cc9fc7d1 | ||
|
|
1d2421b0af | ||
|
|
a5df765951 | ||
|
|
622019ee06 | ||
|
|
45e12cc668 | ||
|
|
a21024a57e | ||
|
|
c175491bb0 | ||
|
|
09b0469faf | ||
|
|
3d63bbf4bf | ||
|
|
56d5d01497 | ||
|
|
a70bd44426 | ||
|
|
8c082b2fcc | ||
|
|
1732cac806 | ||
|
|
e1340e87eb | ||
|
|
e9bfef2131 | ||
|
|
b408699e65 | ||
|
|
3d1c508868 | ||
|
|
84e0746c9c | ||
|
|
b5658c4865 | ||
|
|
d413a4bc53 | ||
|
|
ce5ab902b6 | ||
|
|
294408b0bb | ||
|
|
53372fbe4c | ||
|
|
7fdac2118b | ||
|
|
1dbf78ed71 | ||
|
|
c9101a0c21 | ||
|
|
2e6264c04b | ||
|
|
e0aa46ba22 | ||
|
|
8093c3cd2c | ||
|
|
c6778e4e29 | ||
|
|
c77548d299 | ||
|
|
26d435ea64 | ||
|
|
c3f0d98e41 | ||
|
|
3c50f4aee9 | ||
|
|
4a930ba82a | ||
|
|
866e63f0fe | ||
|
|
d461cfa8ce | ||
|
|
18708636fb | ||
|
|
1901cca2f3 | ||
|
|
344019f108 | ||
|
|
e22a8021d3 | ||
|
|
111c9c0ed0 | ||
|
|
2445d18149 | ||
|
|
739523d559 | ||
|
|
23c0a493b1 | ||
|
|
fa353fb0b3 | ||
|
|
9f817bd918 | ||
|
|
2e5480a6bd | ||
|
|
1b50b7f446 | ||
|
|
ecc413ee01 | ||
|
|
0b1bf13b84 | ||
|
|
1fc6e68f3f | ||
|
|
1bee46ed81 | ||
|
|
a7772ffcd9 | ||
|
|
1263444b2b | ||
|
|
286a78ef8c | ||
|
|
0accff3e18 | ||
|
|
5f62481e62 | ||
|
|
82b8e1f79a | ||
|
|
85e2ca96bc | ||
|
|
fdbf287fee | ||
|
|
fa4b69181f | ||
|
|
a32641d9f4 | ||
|
|
44d8db043e | ||
|
|
be89b12c96 | ||
|
|
fd954589b5 | ||
|
|
a2f44668b6 | ||
|
|
ab2ab37844 | ||
|
|
b280a734a2 | ||
|
|
5c1bd15639 | ||
|
|
24fc67f242 | ||
|
|
642e0fc87e | ||
|
|
1528c09049 | ||
|
|
0f4617e9c4 | ||
|
|
a496e22ad1 | ||
|
|
a420565ded | ||
|
|
b3f0a479c2 |
245 changed files with 66857 additions and 8309 deletions
1
.gitignore
vendored
1
.gitignore
vendored
|
|
@ -14,3 +14,4 @@ tests/rnsconfig/logfile*
|
|||
*.data
|
||||
*.result
|
||||
.buildinfo.bak
|
||||
docs/Reticulum Manual.*
|
||||
|
|
|
|||
276
CRNS/__init__.py
276
CRNS/__init__.py
|
|
@ -1,12 +1,278 @@
|
|||
compiling = False
|
||||
noticed = False
|
||||
notice_delay = 0.3
|
||||
import time
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import threading
|
||||
from importlib.util import find_spec
|
||||
if find_spec("pyximport") and find_spec("cython"):
|
||||
import pyximport; pyxloader = pyximport.install(pyimport=True, language_level=3)[1]
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
from importlib.abc import MetaPathFinder
|
||||
from importlib.machinery import ExtensionFileLoader, SourceFileLoader
|
||||
from importlib.machinery import EXTENSION_SUFFIXES
|
||||
from importlib.util import spec_from_file_location
|
||||
from distutils.extension import Extension
|
||||
from pyximport import pyxbuild
|
||||
|
||||
COMPILE_DIRECTIVES = { "annotation_typing": False }
|
||||
|
||||
_WORKER_SCRIPT = r"""
|
||||
import sys, os, json
|
||||
from distutils.extension import Extension
|
||||
from pyximport import pyxbuild
|
||||
def main():
|
||||
tasks_path = sys.argv[1]
|
||||
with open(tasks_path, "r") as fh: tasks = json.load(fh)
|
||||
results = []
|
||||
for task in tasks:
|
||||
try:
|
||||
os.makedirs(task["scratch"], exist_ok=True)
|
||||
extension = Extension(name=task["basename"], sources=[task["source"]])
|
||||
extension.cython_directives = task["directives"]
|
||||
so_path = pyxbuild.pyx_to_dll(task["source"], ext=extension, build_in_temp=True, pyxbuild_dir=task["scratch"])
|
||||
os.makedirs(os.path.dirname(task["target"]), exist_ok=True)
|
||||
os.replace(so_path, task["target"])
|
||||
results.append({"fullname": task["fullname"], "ok": True, "error": None})
|
||||
except Exception as e:
|
||||
results.append({"fullname": task["fullname"], "ok": False,
|
||||
"error": "{}: {}".format(type(e).__name__, e)})
|
||||
with open(tasks_path + ".results", "w") as fh: json.dump(results, fh)
|
||||
if __name__ == "__main__": main()
|
||||
"""
|
||||
|
||||
def _compile_jobs():
|
||||
jobs = os.environ.get("CRNS_JOBS")
|
||||
if jobs: return max(1, int(jobs))
|
||||
return max(1, min(os.cpu_count() or 1, 16))
|
||||
|
||||
class _ImportFinder(MetaPathFinder):
|
||||
|
||||
def __init__(self):
|
||||
self.build_dir = os.path.join(os.path.expanduser("~"), ".pyxbld")
|
||||
self.mirror_root = os.path.join(self.build_dir, "rnslib")
|
||||
self.source_root = self._find_source_root()
|
||||
self.abi_suffix = EXTENSION_SUFFIXES[0]
|
||||
self.unbuildable = {}
|
||||
self.unbuildable_path = os.path.join(self.build_dir, "unbuildable.json")
|
||||
self._verify_mirror_directives()
|
||||
try:
|
||||
with open(self.unbuildable_path, "r") as fh: self.unbuildable = json.load(fh)
|
||||
except Exception: self.unbuildable = {}
|
||||
|
||||
def _verify_mirror_directives(self):
|
||||
# If the compiler directives have changed since the mirror was
|
||||
# built, the entire mirror is invalidated and rebuilt.
|
||||
stamp_path = os.path.join(self.build_dir, "rnslib.stamp")
|
||||
directives = json.dumps(COMPILE_DIRECTIVES, sort_keys=True)
|
||||
stamp = directives + "\n" + self.abi_suffix
|
||||
previous = None
|
||||
try:
|
||||
with open(stamp_path, "r") as fh: previous = fh.read()
|
||||
except Exception: pass
|
||||
if previous != stamp:
|
||||
shutil.rmtree(self.mirror_root, ignore_errors=True)
|
||||
os.makedirs(self.mirror_root, exist_ok=True)
|
||||
try:
|
||||
os.makedirs(self.build_dir, exist_ok=True)
|
||||
with open(stamp_path, "w") as fh: fh.write(stamp)
|
||||
except Exception: pass
|
||||
|
||||
def _save_unbuildable(self):
|
||||
try:
|
||||
os.makedirs(os.path.dirname(self.unbuildable_path), exist_ok=True)
|
||||
with open(self.unbuildable_path, "w") as fh: json.dump(self.unbuildable, fh, indent=1)
|
||||
except Exception: pass
|
||||
|
||||
def _mark_unbuildable(self, fullname, source):
|
||||
try:
|
||||
self.unbuildable[fullname] = int(os.path.getmtime(source))
|
||||
self._save_unbuildable()
|
||||
except Exception: pass
|
||||
|
||||
def _unmark_buildable(self, fullname):
|
||||
if fullname in self.unbuildable:
|
||||
del self.unbuildable[fullname]
|
||||
self._save_unbuildable()
|
||||
|
||||
def _is_unbuildable(self, fullname, source):
|
||||
if fullname not in self.unbuildable: return False
|
||||
try:
|
||||
if int(os.path.getmtime(source)) == self.unbuildable[fullname]: return True
|
||||
except Exception: pass
|
||||
self.unbuildable.pop(fullname, None)
|
||||
return False
|
||||
|
||||
def _find_source_root(self):
|
||||
here = os.path.dirname(os.path.abspath(__file__))
|
||||
candidate = os.path.abspath(os.path.join(here, "..", "RNS", "__init__.py"))
|
||||
if os.path.exists(candidate): return os.path.dirname(os.path.dirname(candidate))
|
||||
for entry in [os.getcwd()] + list(sys.path):
|
||||
if not entry: continue
|
||||
candidate = os.path.join(entry, "RNS", "__init__.py")
|
||||
if os.path.exists(candidate): return os.path.dirname(candidate)
|
||||
raise RuntimeError("CRNS could not locate the RNS source tree")
|
||||
|
||||
def _all_source_modules(self):
|
||||
rns_root = os.path.join(self.source_root, "RNS")
|
||||
for root, dirs, files in os.walk(rns_root):
|
||||
dirs[:] = [d for d in dirs if d not in ("__pycache__", "i2plib")]
|
||||
rel = os.path.relpath(root, self.source_root).replace(os.sep, ".")
|
||||
for fn in sorted(files):
|
||||
if not fn.endswith(".py"): continue
|
||||
if fn == "__init__.py": fullname = rel
|
||||
else: fullname = rel + "." + fn[:-3]
|
||||
yield fullname
|
||||
|
||||
def _mirror_target(self, fullname, is_package):
|
||||
rel = fullname.split(".")
|
||||
basename = rel[-1]
|
||||
if is_package: return os.path.join(self.mirror_root, *rel, "__init__" + self.abi_suffix)
|
||||
return os.path.join(self.mirror_root, *rel[:-1], basename + self.abi_suffix)
|
||||
|
||||
def _source(self, fullname):
|
||||
rel = fullname.split(".")
|
||||
package_source = os.path.join(self.source_root, *rel, "__init__.py")
|
||||
if os.path.exists(package_source): return package_source, True
|
||||
module_source = os.path.join(self.source_root, *rel) + ".py"
|
||||
if os.path.exists(module_source): return module_source, False
|
||||
return None, None
|
||||
|
||||
def _compile(self, fullname, source):
|
||||
basename = fullname.rsplit(".", 1)[-1]
|
||||
extension = Extension(name=basename, sources=[source])
|
||||
extension.cython_directives = COMPILE_DIRECTIVES
|
||||
so_path = pyxbuild.pyx_to_dll(source, ext=extension, build_in_temp=True, pyxbuild_dir=self.build_dir)
|
||||
abi_suffix = os.path.basename(so_path)[len(basename):]
|
||||
return so_path, abi_suffix
|
||||
|
||||
def _is_current(self, fullname, source, is_package):
|
||||
target = self._mirror_target(fullname, is_package)
|
||||
if not os.path.exists(target): return False
|
||||
try: return os.path.getmtime(target) >= os.path.getmtime(source)
|
||||
except Exception: return False
|
||||
|
||||
def _ensure_compiled(self, fullname, source, is_package):
|
||||
target = self._mirror_target(fullname, is_package)
|
||||
if os.path.exists(target) and os.path.getmtime(target) >= os.path.getmtime(source): return target
|
||||
so_path, abi_suffix = self._compile(fullname, source)
|
||||
os.makedirs(os.path.dirname(target), exist_ok=True)
|
||||
os.replace(so_path, target)
|
||||
return target
|
||||
|
||||
def precompile(self):
|
||||
if not os.path.isdir(self.mirror_root):
|
||||
os.makedirs(self.mirror_root, exist_ok=True)
|
||||
|
||||
pending = []
|
||||
for fullname in self._all_source_modules():
|
||||
source, is_package = self._source(fullname)
|
||||
if source is None:
|
||||
continue
|
||||
if self._is_current(fullname, source, is_package):
|
||||
self._unmark_buildable(fullname)
|
||||
continue
|
||||
if self._is_unbuildable(fullname, source):
|
||||
continue
|
||||
|
||||
pending.append((fullname, source, is_package))
|
||||
|
||||
if not pending: return
|
||||
|
||||
jobs = min(_compile_jobs(), len(pending))
|
||||
if jobs > 1:
|
||||
# Round-robin assignment spreads the large modules evenly
|
||||
# across workers.
|
||||
chunks = [[] for _ in range(jobs)]
|
||||
for i, item in enumerate(pending): chunks[i % jobs].append(item)
|
||||
|
||||
workers_dir = os.path.join(self.build_dir, "workers")
|
||||
os.makedirs(workers_dir, exist_ok=True)
|
||||
procs = []
|
||||
for w in range(jobs):
|
||||
tasks = []
|
||||
for fullname, source, is_package in chunks[w]:
|
||||
tasks.append({ "fullname": fullname,
|
||||
"basename": fullname.rsplit(".", 1)[-1],
|
||||
"source": source,
|
||||
"target": self._mirror_target(fullname, is_package),
|
||||
"scratch": os.path.join(workers_dir, "w{}".format(w)),
|
||||
"directives": COMPILE_DIRECTIVES })
|
||||
task_path = os.path.join(workers_dir, "tasks{}.json".format(w))
|
||||
with open(task_path, "w") as fh: json.dump(tasks, fh)
|
||||
procs.append((w, task_path, subprocess.Popen([sys.executable, "-c", _WORKER_SCRIPT, task_path],
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE)))
|
||||
|
||||
for w, task_path, proc in procs:
|
||||
stdout, stderr = proc.communicate()
|
||||
if proc.returncode != 0:
|
||||
for fullname, _, _ in chunks[w]:
|
||||
source, _ = self._source(fullname)
|
||||
self._mark_unbuildable(fullname, source)
|
||||
print(f"CRNS: compile worker failed for {fullname}, using interpreted module", file=sys.stderr)
|
||||
if stderr:
|
||||
tail = stderr.decode(errors="replace").strip().splitlines()[-5:]
|
||||
print("CRNS: worker output:\n" + "\n".join(tail), file=sys.stderr)
|
||||
continue
|
||||
|
||||
results = {}
|
||||
try:
|
||||
with open(task_path + ".results", "r") as fh:
|
||||
for entry in json.load(fh): results[entry["fullname"]] = entry
|
||||
except Exception: results = {}
|
||||
|
||||
for fullname, _, _ in chunks[w]:
|
||||
entry = results.get(fullname)
|
||||
if entry is not None and entry.get("ok"): self._unmark_buildable(fullname)
|
||||
else:
|
||||
error = "" if entry is None else entry.get("error", "")
|
||||
source, _ = self._source(fullname)
|
||||
self._mark_unbuildable(fullname, source)
|
||||
print(f"Could not compile {fullname}, using interpreted module: {error}", file=sys.stderr)
|
||||
else:
|
||||
for fullname, source, is_package in pending:
|
||||
try:
|
||||
self._ensure_compiled(fullname, source, is_package)
|
||||
self._unmark_buildable(fullname)
|
||||
except Exception as e:
|
||||
self._mark_unbuildable(fullname, source)
|
||||
print(f"Could not compile {fullname}, using interpreted module: {e}", file=sys.stderr)
|
||||
|
||||
def _spec(self, fullname, source, is_package, compile_attempted):
|
||||
if fullname == "RNS.vendor.i2plib" or fullname.startswith("RNS.vendor.i2plib."):
|
||||
rel = fullname.split(".")
|
||||
locations = [os.path.join(self.source_root, *rel)] if is_package else None
|
||||
return spec_from_file_location(fullname, source, loader=SourceFileLoader(fullname, source), submodule_search_locations=locations)
|
||||
|
||||
if compile_attempted:
|
||||
try:
|
||||
target = self._ensure_compiled(fullname, source, is_package)
|
||||
self._unmark_buildable(fullname)
|
||||
if is_package: locations = [os.path.dirname(target)]
|
||||
else: locations = None
|
||||
return spec_from_file_location(fullname, target, loader=ExtensionFileLoader(fullname, target),
|
||||
submodule_search_locations=locations)
|
||||
except Exception as e:
|
||||
self._mark_unbuildable(fullname, source)
|
||||
print(f"Could not compile {fullname}, using interpreted module: {e}", file=sys.stderr)
|
||||
|
||||
rel = fullname.split(".")
|
||||
locations = [os.path.join(self.source_root, *rel)] if is_package else None
|
||||
return spec_from_file_location(fullname, source,loader=SourceFileLoader(fullname, source),submodule_search_locations=locations)
|
||||
|
||||
def find_spec(self, fullname, path, target=None):
|
||||
if not isinstance(fullname, str): return None
|
||||
if not (fullname == "RNS" or fullname.startswith("RNS.")): return None
|
||||
|
||||
source, is_package = self._source(fullname)
|
||||
if source is None: return None
|
||||
|
||||
return self._spec(fullname, source, is_package, compile_attempted=not self._is_unbuildable(fullname, source))
|
||||
|
||||
_importfinder = _ImportFinder()
|
||||
sys.meta_path.insert(0, _importfinder)
|
||||
|
||||
def notice_job():
|
||||
global noticed
|
||||
|
|
@ -14,13 +280,13 @@ def notice_job():
|
|||
while compiling:
|
||||
if time.time() > started+notice_delay and compiling:
|
||||
noticed = True
|
||||
print("Compiling RNS object code... ", end="")
|
||||
print("Compiling RNS object code...")
|
||||
sys.stdout.flush()
|
||||
break
|
||||
time.sleep(0.1)
|
||||
|
||||
|
||||
compiling = True
|
||||
threading.Thread(target=notice_job, daemon=True).start()
|
||||
_importfinder.precompile()
|
||||
import RNS; compiling = False
|
||||
if noticed: print("Done."); sys.stdout.flush()
|
||||
if noticed: print("Compilation done"); sys.stdout.flush()
|
||||
|
|
|
|||
865
Changelog.md
865
Changelog.md
|
|
@ -1,3 +1,868 @@
|
|||
### RNS 1.6.1
|
||||
|
||||
This release brings yet another round of major efficiency, performance and filtering improvements to RNS, and continues the transition towards full network stamp functionality.
|
||||
|
||||
All network stamp types will now be generated and emitted by default. Expect some CPU load once your applications start up for the first time on `1.6.1`, as destination stamps are generated.
|
||||
|
||||
**Changes**
|
||||
- Enabled network stamp generation by default
|
||||
- Added ability to filter destination types by hash
|
||||
- Added hop limiter options both globally and per-interface
|
||||
- Added super fast announce ingress pre-filtering
|
||||
- Added filtered traffic stats to `rnstatus`
|
||||
- Added link traffic stats to `rnstatus`
|
||||
- Added display of remote transport IDs per client interface in `rnstatus`.
|
||||
- Ability to inspect filtered traffic, announces, et cetera in `rnstatus` per interface
|
||||
- Ability to inspect tunnel properties, stamp costs, remote implementation info and more in `rnstatus`
|
||||
- Added ability to set `client_limit` on `BackboneInterface`, defaults to 500
|
||||
- Added ability to configure the maximum amount of stamp workers and worker niceness
|
||||
- Added ability to place a `blocked_ips` file in your config directory to permanently block IPs
|
||||
- Added ability to directly blackhole remote transport nodes
|
||||
- Added ability to configure stamp values and costs for `rngit`
|
||||
- Added asynchronous, interleaved held announce queue cleaning across all interfaces based on potential announce usability
|
||||
- Added bidirectional tunnel synthesis on non shared-medium interfaces, with stamp cost and version signalling
|
||||
- Added announce rate table persistence between instance restarts
|
||||
- Added default caps for configured auto-connect interface counts
|
||||
- Added path state info to API path table getters
|
||||
- Significantly optimized and improved inbound announce filtering and deduplication
|
||||
- Significantly optimized and reworked the filtering subsystem for scalability across hundreds of interfaces
|
||||
- Improved `rnpath` display and filtering options
|
||||
- Improved traffic counting
|
||||
- Improved path responsiveness tracking
|
||||
- Improved protocol violation tracking and logging
|
||||
- Improved path pool and tunnel table culling
|
||||
- Tuned announce validation caches and other parameters
|
||||
- Fixed a regression introduced in `1.6.0`, that could cause path table culling to temporarily remove an instance's own local destination paths
|
||||
- Fixed dataplane egress control state not resetting on `BackboneInterface` client connection recovery
|
||||
- Fixed a potential logging deadlock
|
||||
- Fixed a potential exit handler deadlock if a signal arrives mid RPC call
|
||||
- Fixed missing wanted destination type for interface discovery when local announce filtering was enabled
|
||||
- Fixed split-instance initial known-destinations load not being available on Android
|
||||
- Added mitigation for a CPython process fork deadlock dug in stdin handling
|
||||
|
||||
**Verified Retrieval**
|
||||
You can retrieve and verify this release over Reticulum using the built-in `rngit release` utility. To retrieve only the installation `.whl` package, and the release manifest for future updates, you can use:
|
||||
|
||||
```sh
|
||||
rngit release rns://7649a50d84610232d1416b41d2896aff/reticulum/reticulum fetch "latest:rns-*.whl" --signer bc7291552be7a58f361522990465165c
|
||||
```
|
||||
|
||||
To download all artifacts, including the documentation and source archive, you can use the following command:
|
||||
|
||||
```sh
|
||||
rngit release rns://7649a50d84610232d1416b41d2896aff/reticulum/reticulum fetch latest:all --signer bc7291552be7a58f361522990465165c
|
||||
```
|
||||
|
||||
**Release Signatures**
|
||||
Release artifacts include a signed `rsm` release manifest and `rsg` signature files that can be validated against the RNS release signing identity `<bc7291552be7a58f361522990465165c>` using `rngit` or `rnid`. To perform an offline verification of all release artifacts using a manifest:
|
||||
|
||||
```sh
|
||||
rngit release rns_*.rsm verify --signer bc7291552be7a58f361522990465165c
|
||||
```
|
||||
|
||||
To verify release artifacts using individual `rsg` files, while also verifying the manifest itself, download the `rsm` and `rsg` signatures, make sure they are in the same folder as the release artifact, and run `rnid` signature verification with the release identity as the required signer:
|
||||
|
||||
```sh
|
||||
rnid -i bc7291552be7a58f361522990465165c -V rns_*.rsm *.rsg
|
||||
```
|
||||
|
||||
The `rnid` utility will then verify the signatures, and display whether they are valid. If the signature cannot be verified, the release has been tampered with and should be discarded.
|
||||
|
||||
### RNS 1.6.0
|
||||
|
||||
This major release of Reticulum brings significant improvements to resource consumption, performance, stability and resilience, especially for constrained devices and nodes.
|
||||
|
||||
**Memory & Performance Improvements**
|
||||
|
||||
A new, SQLite based storage backend has been implemented, and is now the default. All legacy file-based storage will be auto-migrated to the database-backed store on the first start. If you are using an esoteric system that does not support the `sqlite3` module, Reticulum will automatically fall back to using the file-based storage system.
|
||||
|
||||
The memory management system has been completely reworked, and significantly improved. For long-running transport nodes, this cuts memory consumption by approximately 50%, and for non-transport instances, standalone applications and appliance devices by up to 80%.
|
||||
|
||||
**Path Table Sharding**
|
||||
|
||||
This release implements the initial transport awareness and mechanisms for emergent path table sharding. Transport instances will no longer attempt to hold full inter-network path tables in memory, but keep only paths relevant to the nodes they are transporting traffic for. Unused paths are now kept for much shorter periods of time, and used as a rolling pool that potential new traffic corridors can form into. The system is entirely passive, and adds no inter-node coordination or additional protocol mechanics. The current release brings the foundation, and the upcoming 1.6.x releases will tune, improve it and add the remaining pieces of the functionality.
|
||||
|
||||
**Network Stamps**
|
||||
|
||||
The stamp system, first introduced in LXMF, where it was tested and refined over the last two years, has now been brought into the core of Reticulum to help protect against spam and network abuse. All the relevant core primtives now support stamp generation and verification:
|
||||
|
||||
- Identities can be created with an embedded stamp value. This adds no additional bytes or identity format changes. Generation and verification is completely transparent, and carried within the Reticulum Identity keyset.
|
||||
- Destinations can now be stamped, and will embed stamps and required link request cost in their announces. The change is backwards-compatible, so even older versions of Reticulum carry stamped announces transparently. Destination stamps are one-time generations, stored and persisted on success, so these have relatively high default target values. Expect some CPU usage when an unstamped destination has its stamp generated for the first time on 1.6.0+.
|
||||
- Path requests can now be stamped, and are also backwards-compatible. A stamped path request will be carried transparently by non stamp-aware transport nodes.
|
||||
- Link requests are stamped according to the announced link request cost of the destination. As with all the other primitives, these are also completely transparent and backwards-compatible. The stamp is carried embedded in the ephemeral link request keyset.
|
||||
|
||||
The stamp functionality and requirements will be deployed in a phased transition, so both network operators and application developers have time to update their systems, and any potential bugs and issues can be worked out before the full deployment:
|
||||
|
||||
- Version `1.6.0` implements all the core functionality and primitives, and starts excercising the verification mechanics. By default, no destination and path request stamps are generated yet in this release, and while stamps are checked and verified, they are not *enforced*. However, both stamp emission and enforcement can be enabled in the configuration for testing or early opt-in. Link request stamps *will* be generated, if the destination has announced a cost.
|
||||
- Version `1.6.1` will enable stamp emission for all network primitives by default. Old, unstamped destinations will have their stamps generated automatically on their first announce, and will start announcing link request costs as configured. Enforcement in this phase will still be disabled by default, but can be enabled by configuration.
|
||||
- Version `1.6.2` will complete the transition to network stamps. All functionality will now be enabled by default, including enforcement. Destination announces, path requests and link requests that do not meet stamp requirements will be dropped. Any old transport nodes that have not updated to `1.6.x` will not be able to communicate with stamp-enforcing nodes, but will still *carry traffic for* stamp-capable systems.
|
||||
|
||||
The `1.6.0` and `1.6.1` releases will be an important time for observing the on-network behaviour, functionality and stability of the system, and to make any potential adjustments or fix problems before the final transition to `1.6.2`. If you're operating a transport node, please help out by reporting any problems or issues you discover.
|
||||
|
||||
**For application developers**, there is no immediate refactoring or changes required application-side. Simply updating RNS to `1.6.0` is sufficient to transparently enable all the new functionality and improvements. It is very much recommended, however, to familiarize yourself with the new API methods for specifying link request costs and destination values, as these are configurable on a per-destination basis, and might warrant specific configuration for your intended use-case.
|
||||
|
||||
**Changes**
|
||||
- Added `Identity` stamps
|
||||
- Added destination and announce stamps
|
||||
- Added path request stamps
|
||||
- Added link request stamps
|
||||
- Added link request cost signalling to announces
|
||||
- Added fractional stamp value support
|
||||
- Added path usage tracking
|
||||
- Added automatic path table culling based on traffic and usage
|
||||
- Added path retainment prioritization for slow interfaces
|
||||
- Added unused path pool monitoring and handling
|
||||
- Added managed storage backend using SQLite
|
||||
- Added fallback to legacy file-based storage if SQLite is unavailable
|
||||
- Added managed storage for known destinations and ratchets
|
||||
- Added managed storage for cached announces
|
||||
- Added memory-density optimized announce rate table implementation
|
||||
- Added memory-density optimized packet hashlist implementation
|
||||
- Added heap-tracking memory management
|
||||
- Added more comprehensive metrics to status output structure
|
||||
- Added improved filtering backend capabilities
|
||||
- Added new `[filter]` configuration section
|
||||
- Added `hop_limiter` configuration option
|
||||
- Added allow and ignore lists for destination types
|
||||
- Added ability for applications to receive required announces only
|
||||
- Added cost-aware interface discovery and auto-connects
|
||||
- Added cost and value display to `rnstatus` and `rnpath` utilities
|
||||
- Added `app_flag` signalling to resource advertisements
|
||||
- Added link MTU aware resource advertisements
|
||||
- Significantly improved inbound announce handling performance
|
||||
- Significantly improved stamper performance on Android
|
||||
- Significantly improved announce handler callback performance
|
||||
- Optimized and improved stamper implementation
|
||||
- Optimized announce rate table handling
|
||||
- Optimized packet filter hashlist handling
|
||||
- Improved early filtering and malformed packet detection
|
||||
- Improved path request protocol violation handling
|
||||
- Fixed `RNodeInterface` phy stats regression on Android
|
||||
- Fixed missing interface parameter inheritances
|
||||
- Fixed `rngit` default config path typo for `~/.config/rngit` case
|
||||
- Fixed invalid RPC error handling in some cases
|
||||
- Fixed various minor bugs and inconsistencies
|
||||
- Updated documentation
|
||||
|
||||
### 2026-10-05: RNS 1.5.7
|
||||
|
||||
This maintenance release fixes a bug in the dataplane egress control handling.
|
||||
|
||||
**Changes**
|
||||
- Fixed egress control TX stall state deadlock on BackboneInterface initiator instances
|
||||
|
||||
### 2026-10-02: RNS 1.5.6
|
||||
|
||||
This maintenance release improves interface discovery handling.
|
||||
|
||||
**Changes**
|
||||
- Fixed ephemeral transport identity being possible when discoverable interfaces are present on non-transport instances
|
||||
- Ensure only transport-enabled interfaces are autoconnected at discovery time
|
||||
|
||||
### 2026-09-29: RNS 1.5.5
|
||||
|
||||
This release adds live interface attach, detach and reload, as well as significant improvements to interface discovery. Auto-connecting interfaces is now also supported on Windows and macOS.
|
||||
|
||||
It also brings improvements to `rngit` and fixes discovered `I2PInterface` configuration snippets, along with a number of other bugs.
|
||||
|
||||
**Changes**
|
||||
- Added ability to `--attach`, `--detach` and `--reload` interfaces to the `rnstatus` utility
|
||||
- Added interface discovery auto-connect support on Windows and macOS
|
||||
- Added ability to download micron-converted versions of markdown files to the `rngit` page node
|
||||
- Added workdoc counts to filter scope links to the `rngit` page node
|
||||
- Added version info to `rnstatus` discovered interfaces output
|
||||
- Added `--show-stale` and `--show-unknown` options to `rnstatus`
|
||||
- Added sequential naming of auto-connected interfaces on announced name collisions
|
||||
- Added auto-connect implementation and version criteria filtering
|
||||
- Added configuration option for allowing auto-connects to discovered interfaces with missing version info
|
||||
- Improved sanitization of nonsensical interface discovery IFAC announce data
|
||||
- Fixed `I2PInterface` discovery config snippets not including `.b32.i2p`
|
||||
- Fixed timeout and response rejects not being processed for single-packet requests, by **JRG**
|
||||
- Fixed missing configurable property propagations on spawned interfaces, by **JRG**
|
||||
- Fixed ingress control producer comprehension at mid-watermark using incorrect variable name for interface type check, by **JRG**
|
||||
- Fixed instances of `announce_cap` property ensurance logic not storing announce cap value as a fraction, by **JRG**
|
||||
- Fixed a potential race condition in `LocalInterface` initialization.
|
||||
- Updated documentation and manual
|
||||
|
||||
### 2026-09-11: RNS 1.5.4
|
||||
|
||||
This release improves RNode BLE connectivity reliability on desktop operating systems.
|
||||
|
||||
**Changes**
|
||||
- Fixed RNode BLE device address acquisition on windows, by **Nickie Deuxyeux**
|
||||
- Fixed RNode BLE re-connection deadlock on desktop
|
||||
- Improved RNode BLE reconnect reliability
|
||||
|
||||
### 2026-09-10: RNS 1.5.3
|
||||
|
||||
This maintenance release adds functionality to, and fixes bugs in `rngit`, and includes a few other optimizations and improvements.
|
||||
|
||||
**Changes**
|
||||
- Added media request handler to `rngit`, for serving images to `nomadnet` clients
|
||||
- Added media conversion handler to `rngit`, auto-converting images to webp
|
||||
- Added importable helper method for using the `rngit` media converter in other programs
|
||||
- Added a `no_ident` template to the `rngit` page node
|
||||
- Added immediate activation of changed permissions with `rngit perms`
|
||||
- Added optimized HDLC framer
|
||||
- Fixed `rngit` repo admins not being able activate/complete workdocs from other users, by **Bergie**
|
||||
- Fixed a typo, I'm positively certain this is the last one
|
||||
|
||||
### 2026-08-28: RNS 1.5.2
|
||||
|
||||
This maintenance release fixes a regression in resource transfers that was introduced in 1.5.1, an `I2PInterface` bug, and sets dataplane control parameters to default values that are actually sensible. If you installed `1.5.1`, it's update time again, baby.
|
||||
|
||||
**Changes**
|
||||
- Tuned dataplane control parameters
|
||||
- Added example for blocking unidentified page node peers to `rngit` default config
|
||||
- Fixed regression in resource transfers causing some `rngit` file downloads to fail
|
||||
- Fixed keepalive frames being passed to transport core on `I2PInterface`
|
||||
|
||||
### 2026-08-28: RNS 1.5.1
|
||||
|
||||
This release focuses on dataplane control, memory bounding and transport throughput, with adaptive ingress and egress control for interfaces, a new (and very efficient) zero-copy coalescing transmit buffer, an optimized HDLC deframer, and a range of other improvements to the inbound packet processing paths. It also introduces support for live profiling, and various new diagnostics output in `rnstatus`, along with a number of bug fixes and general improvements.
|
||||
|
||||
**Changes**
|
||||
- Added adaptive dataplane egress control
|
||||
- Added adaptive dataplane ingress control
|
||||
- Added zero-copy coalescing transmit buffers to interfaces using the `BackboneInterface` backend
|
||||
- Added early protocol violation checks for invalid frames
|
||||
- Added transport implementation name and version to discovery information requirements
|
||||
- Added full live profiling to the built-in `Profiler` primitive, by **K8**
|
||||
- Added the `@RNS.Profiler.profile` decorator, by **K8**
|
||||
- Added support for indefinitely running, reentrant profilers with bounded capture, by **K8**
|
||||
- Added live profiling results output to `rnstatus`, by **K8**
|
||||
- Added PPS statistics to `rnstatus`
|
||||
- Added interface MTU display to `rnstatus`
|
||||
- Added interface TX drop statistics to `rnstatus`
|
||||
- Added interface TX buffer size output to `rnstatus`
|
||||
- Added throughput benchmarker to the test suite
|
||||
- Added support for natively compiled module builds, and the ability to load compiled modules when available
|
||||
- Added module compilation status reporting from in-wheel build information
|
||||
- Added parity tests for HDLC, IFAC and HKDF against the legacy implementations
|
||||
- Added shared medium hints to interfaces
|
||||
- Improved memory and CPU consumption under high traffic loads
|
||||
- Improved traffic class handling
|
||||
- Optimized HKDF to ~5.7x performance, if this worries you, **good**; then see the full parity and standard vector test suite and read the code
|
||||
- Optimized inbound and outbound IFAC handling to ~90x performance on large frames
|
||||
- Optimized inbound packet processing by reducing lock acquisitions and path table contention
|
||||
- Optimized lookups for pending and active links using hash maps for lookup operations
|
||||
- Optimized announce validation by caching signature validation results, significantly lowering announce storm CPU load
|
||||
- Optimized the HDLC deframer
|
||||
- Reduced redundant packet hashing in inbound processing
|
||||
- Tuned default inbound queue lengths and announce queuing
|
||||
- Tuned automatic interface MTU configuration
|
||||
- Fixed `Resource` transfers failing when initialized from stream-based data sources that fell outside `MAX_EFFICIENT_SIZE`
|
||||
- Fixed an invalid prefix stripping bug in the `rngit` page server (`.something` files not viewable)
|
||||
- Fixed RSSI/SNR reporting regression
|
||||
- Fixed keepalive handling on non-epoll backends
|
||||
- Fixed a bug in `rngit` page navigation content initialization order
|
||||
- Fixed the `rngit` page node not being able to serve in-tree downloads for large files
|
||||
- Fixed `rnstatus` blocked IP listing including IPs that were actually not yet blocked
|
||||
- Fixed `rnstatus` traffic totals counting the local shared instance inter-app transit in totals
|
||||
- Fixed various minor bugs in `rnsh`, `rnir`, identity handling
|
||||
- Fixed a latent bug in the AES module, where an exception would not resolve it's exception description correctly, and instead raise another exception.
|
||||
- Removed dead Python 2 code from `umsgpack`
|
||||
|
||||
### 2026-08-22: RNS 1.5.0
|
||||
|
||||
This release significantly improves the core Transport handling of RNS, with a priority-based ingress queue backend, substantially improved efficiency of both data and management traffic handling, more efficient ingress/egress limiting, and a long list of other improvements and bugfixes.
|
||||
|
||||
**Changes**
|
||||
- Added ability to include operator LXMF address in interface discovery information
|
||||
- Added prioritized inbound traffic processing to the transport core
|
||||
- Added configurable inbound queue lengths for data, announce, path request and ingress limited traffic
|
||||
- Added early filtering for inbound traffic and improved filtering performance
|
||||
- Added per-interface protocol violation tracking
|
||||
- Added in-flight path requests tracking
|
||||
- Added request and response batching for in-flight path requests
|
||||
- Added ability to signal blackholed status in announce validation return to the API
|
||||
- Added full link MDU utilization for `Channel` and `Buffer`
|
||||
- Added inbound queue pressure and queue drop statistics to `rnstatus`
|
||||
- Added detailed announce and path request traffic flow statistics per interface to `rnstatus`
|
||||
- Added total announce and PR count/frequency stats per interface to `rnstatus`
|
||||
- Added data flow speed and composition stats to `rnstatus`
|
||||
- Added protocol violation stats to `rnstatus`
|
||||
- Added active link statistics to `rnstatus`
|
||||
- Added blocked IP listings to `rnstatus`
|
||||
- Added medium bitrate based timeout calculation helpers and RPC functions, by **Zenith**
|
||||
- Added extra timeout for discovery path requests when slow interfaces are online, by **Zenith**
|
||||
- Added adaptive timeouts to `rncp`, `rnpath`, `rnprobe` and `rnx`, by **Zenith**
|
||||
- Added adaptive timeout calculation to `rngit`, by **Zenith**
|
||||
- Improved overall Transport inbound processing speed
|
||||
- Improved path request handling significantly
|
||||
- Improved path request ingress limiting and accounting
|
||||
- Improved egress limiting responsiveness under high incoming path request load
|
||||
- Improved transport background job processing
|
||||
- Improved early rejection of packets with excessive hop counts
|
||||
- Fixed `BackboneInterface` EPOLL receive starvation and ingress control timestamp handling, by **JRG**
|
||||
- Fixed potential transport deadlock on receipts lock when callbacks sends packets, by **JRG**
|
||||
- Fixed transport state handling edge cases in path request, announce queue and pending link processing, by **JRG**
|
||||
- Fixed link watchdog not resetting on receive exceptions, by **JRG**
|
||||
- Fixed `Resource` cancellation on multi-segment resources, by **JRG**
|
||||
- Fixed `Resource` transfer part index alignment and rebinding, by **JRG**
|
||||
- Fixed stale BLE device reference in `RNodeInterface`, by **JRG**
|
||||
- Fixed ratchet cleaning retained preservation, by **JRG**
|
||||
- Fixed invalid `rnstatus` statistics handling, by **JRG**
|
||||
- Fixed various bugs in packet, link and interface handling, by **JRG**
|
||||
- Fixed per-interface burst count inconsistencies in `rnstatus` output
|
||||
- Fixed `rngit` file resource operations failing on Windows
|
||||
- Fixed `rnodeconf` config summary incorrectly displaying WiFi mode
|
||||
- Fixed speedtest example aborting transfers on stale link status
|
||||
- Updated documentation, including manual sections on queue tuning and interface discovery options
|
||||
|
||||
### 2026-07-26: RNS 1.4.2
|
||||
|
||||
This maintenance release fixes bugs in blackholed identity filtering and recursive path request handling for RNode interfaces.
|
||||
|
||||
**Changes**
|
||||
- Fixed a bug where recursive path requests would cause a division by zero error on uninitialized RNode interfaces.
|
||||
- Fixed slow blackholed filtering for discovered interfaces on Android in some cases.
|
||||
|
||||
### 2026-07-24: RNS 1.4.1
|
||||
|
||||
This release significantly improves path convergence and stability with the new path re-balancing functionality. It also introduces interface gravity configuration, new API functionality and fixes a range of bugs and inefficiencies.
|
||||
|
||||
**Changes**
|
||||
- Added dynamic path re-balancing
|
||||
- Added `set_max_request_size` to `Destination` API
|
||||
- Added `max_response_size` to request API
|
||||
- Added interface gravity handling
|
||||
- Added `autoconnect_interface_mode` option
|
||||
- Added `autoconnect_announces_to_internal` option
|
||||
- Added `autoconnect_interface_gravity` option
|
||||
- Added `default_gravity` option
|
||||
- Added `announces_to_internal` interface option
|
||||
- Added `gravity` interface option
|
||||
- Added gravity display and sorting option to `rnstatus`
|
||||
- Added boundary -> boundary and boundary -> gateway path requests
|
||||
- Fixed `I2PInterface` tasks getting garbage-collected prematurely, thanks to **welo**
|
||||
- Fixed various minor bugs in `I2PInterface`
|
||||
- Fixed ingress control burst active flag deadlocking until new announces arrived under burst timing patterns
|
||||
- Fixed various memory inefficiencies
|
||||
- Fixed loglevel `LOG_EXTREME` not being usable in some cases
|
||||
- Fixed historical interface discoveries not being cleaned according to blackholed identities
|
||||
|
||||
### 2026-07-20: RNS 1.4.0
|
||||
|
||||
This release focuses on performance, stability and reliability improvements, particularly with resource-constrained systems in mind. It also fixes several bugs, reduces logging noise, and makes information about blocked `BackboneInterface` clients available in `ifstats`.
|
||||
|
||||
**Changes**
|
||||
- Optimized transport data persistence to avoid CPU spikes on low-powered systems
|
||||
- Optimized interface hashes calculation and access to reduce overall processing load
|
||||
- Implemented known destinations cleaning as background priority to avoid lock contention and reduce load on CPU-constrained systems
|
||||
- Deprecated known destination on-disk recombination on background data persist to alleviate lock contention on CPU-constrained systems
|
||||
- Implemented caching of invalid discovery stamps to reduce CPU load on low-powered systems
|
||||
- Implemented valid discovery announce caching and sequential validation lock to reduce processing load on CPU-constrained systems
|
||||
- Fixed link stale teardown erroneously occurring due to missing keepalive from initiator side when destination continously sends packets but initiator stays silent
|
||||
- Fixed race condition in link watchdog timing
|
||||
- Fixed missing None-check in `BackboneInterface` fast-flap detection
|
||||
- Fixed invalid exception log handler in `BackboneInterface`
|
||||
- Increased default discovery stamp value to 16
|
||||
- Added blocked IPs list to ifstats
|
||||
- Improved backbone interface logging
|
||||
- Reduced overall logging noise
|
||||
|
||||
### 2026-07-19: RNS 1.3.9
|
||||
|
||||
**Critical Security Update**: This release fixes a severe security flaw in `rnsh`. Due to the nature of the issue, I will not disclose any further details for the time being. Once operators have had time to update, I will provide a full report for transparency. If you use `rnsh`, update **right now**.
|
||||
|
||||
**Important**: The new version of `rnsh` changes default identity file locations, and these will now be sourced from `~/.rnsh/identity` (initiator) and `~/.rnsh/identity.default` (listener). Make sure you copy your old files to this directory, or specify a custom identity path using the command line arguments. The `--config` argument has also been renamed to `--rnsconfig`, and the `--config` argument will now specify the `rnsh` configuration directory instead of the RNS configuration directory, bringing the behavior into alignment with other RNS utilities.
|
||||
|
||||
Additionally, this release includes automated blocking of fast-flapping clients on `BackboneInterface` listeners (see the Interfaces chapter of the manual for details), and a number of improvements to resource handling. The logging system has also been improved, and path and destination information moved to a new `LOG_PATHING` loglevel, to decrease log noise.
|
||||
|
||||
**Changes**
|
||||
- Fixed a critical security issue in `rnsh`
|
||||
- Added automated blocking of fast-flapping clients to `BackboneInterface`
|
||||
- Added new `LOG_PATHING` loglevel, improved logging
|
||||
- Added ability to make internal-mode interfaces discoverable
|
||||
- Added ability to get discoverable interface location from external script
|
||||
- Added `RESOURCE_RCL` signal on resource receiver cancel
|
||||
- Improved resource handling and reliability
|
||||
- Updated `rnsh` config args to work similarly to other RNS utilities
|
||||
|
||||
### 2026-07-10: RNS 1.3.8
|
||||
|
||||
This release fixes various inconsistencies in link and hop-count related APIs.
|
||||
|
||||
**Changes**
|
||||
- Fixed inconsistent link traffic stats calculation
|
||||
- Fixed link hop-count metric only being available on initiator side
|
||||
- Fixed potential hop-count serialization error on transport
|
||||
- Updated `WeaveInterface` to support latest Weave firmware
|
||||
|
||||
### 2026-07-03: RNS 1.3.7
|
||||
|
||||
This maintenance release improves announces propagation logic, and adds additional options for configuring announce propagation and interface behavior in transport mode.
|
||||
|
||||
**Changes**
|
||||
- Added `internal` interface mode
|
||||
- Added `recursive_prs` interface option
|
||||
- Added `announces_from_internal` interface option
|
||||
- Added new options to the manual
|
||||
- Improved and cleaned up announce propagation logic
|
||||
|
||||
### 2026-07-03: RNS 1.3.6
|
||||
|
||||
This release contained a bug in the local instance transport handling, and was superseded by version `1.3.7`.
|
||||
|
||||
### 2026-06-01: RNS 1.3.5
|
||||
|
||||
This maintenance release contains an important fix for `AutoInterface` reliability when roaming between different physical networks.
|
||||
|
||||
**Changes**
|
||||
- Fixed UDP listener replacement deadlocking inbound AutoInterface traffic when fast-roaming between physical interfaces or WiFi APs
|
||||
- Fixed some paths never resolving when using other interfaces at the same time as a deadlocked AutoInterface
|
||||
|
||||
### 2026-05-29: RNS 1.3.4
|
||||
|
||||
This release fixes a regression that could cause sub-optimal path selection under conditions where the same announce was received within a very short timespan on different interfaces, as well as a few other bugs and inefficiencies.
|
||||
|
||||
**Changes**
|
||||
- Fixed regression in inbound announce de-duplication
|
||||
- Fixed missing check for shared instance type configuration conflict
|
||||
- Fixed superfluous path state configuration on new announce from new destinations
|
||||
- Improved cleanup of stale known destinations
|
||||
- Improved shared instance RPC handling
|
||||
|
||||
### 2026-05-28: RNS 1.3.3
|
||||
|
||||
This release fixes a regression in persistence of known destination on Windows.
|
||||
|
||||
**Changes**
|
||||
- Fixed regression in known destinations persist on Windows
|
||||
|
||||
### 2026-05-28: RNS 1.3.2
|
||||
|
||||
This release adds commit signing and validation support to the `rngit` system, as well as improvements to the blackhole functionality.
|
||||
|
||||
**Changes**
|
||||
- Extended blackhole functionality to immediately terminate links from blackholed identities
|
||||
- Added commit signing and validation to `rngit`
|
||||
- Added commit hash inclusion in generated release manifest to `rngit`
|
||||
- Added local `verify` operation shorthand to `rngit release`
|
||||
- Added option to configure blackhole update interval
|
||||
- Added configuration option to log without timestamps
|
||||
|
||||
### 2026-05-22: RNS 1.3.1
|
||||
|
||||
This maintenance release fixes a single bug.
|
||||
|
||||
**Changes**
|
||||
- Fixed regression in request response transfer size accumulator
|
||||
|
||||
### 2026-05-21: RNS 1.3.0
|
||||
|
||||
This maintenance release fixes a number of bugs.
|
||||
|
||||
**Changes**
|
||||
- Added ability to use wildcards and pattern matches in `rngit` artifact fetch targets
|
||||
- Fixed channel outlet sequence holes and ghost envelopes on dying outlets by **neutral**
|
||||
- Fixed known destination iteration races by **neutral**
|
||||
- Fixed timeout deadlock in `rnsh` by **neutral**
|
||||
- Fixed commit message rendering in `rngit`
|
||||
- Fixed various minor bugs and output inconsistencies in `rngit`
|
||||
- Adjusted timeouts for remote operations in `rngit`
|
||||
- Updated documentation
|
||||
|
||||
### 2026-05-19: RNS 1.2.9
|
||||
|
||||
This release completes the operational functionality of the `rngit` system, which now has full release creation, fetch and verified update support using the `rngit release` command. Additionally, two chapters have been added to the manual should cover all the things that `rngit` is currently capable of.
|
||||
|
||||
**Changes**
|
||||
- Added full `rngit` documentation to the manual
|
||||
- Added offline `.rsm` release manifest verification
|
||||
- Added the ability to fetch release updates directly from `.rsm` manifests
|
||||
- Added canonical `.rsm` release structure validator to `rnid` for import
|
||||
- Added `.rsm` manifest saving when using `rngit release fetch`
|
||||
- Added remote `HEAD` tracking for forks and mirros to `rngit`
|
||||
- Improved known destinations persist reliability
|
||||
- Improved page node ref link handling in `rngit`
|
||||
- Improved logging in various locations
|
||||
|
||||
### 2026-05-18: RNS 1.2.8
|
||||
|
||||
This release improves the `rngit` system with signed release manifest generation and automatic artifact signing. It also includes several additions to `rnid` and various minor fixes and improvements to the `rngit` system.
|
||||
|
||||
**Changes**
|
||||
- Added signed release manifest generation to `rngit release`
|
||||
- Added verified release fetching to `rngit release`
|
||||
- Added automatic artifact signing to `rngit release`
|
||||
- Added signed message creation from file to `rnid`
|
||||
- Added signed message metadata output option to `rnid`
|
||||
- Added `rsm` metadata embedding and spec validation to `rnid`
|
||||
- Added identity and destination aliases to `rngit`
|
||||
- Added blocked identities option to `rngit`
|
||||
- Added ability to render raw micron in markdown files to `rngit`
|
||||
- Added fork and mirror last sync time to repository page in `rngit`
|
||||
- Better handling of silly links in `rngit`
|
||||
- Fixed markdown table cell truncation not closing micron tags
|
||||
- Fixed various minor bugs and inconsistencies in `rngit`
|
||||
- Dropped `note` metadata field requirement from `rsg` structure
|
||||
|
||||
### 2026-05-17: RNS 1.2.7
|
||||
|
||||
This release significantly improves the `rngit` system with fork, mirroring and empty repository creation functionality, a new work document proposals feature, improvements to the transport core reliability and efficiency and various other tweaks and improvements.
|
||||
|
||||
**Changes**
|
||||
- Added work document proposals functionality to `rngit`
|
||||
- Added fork and mirroring support to `rngit`
|
||||
- Added ability to create new repositories remotely to `rngit`
|
||||
- Added latest release management to `rngit`
|
||||
- Added download stats to `rngit`
|
||||
- Improved shared instance RPC error handling
|
||||
- Improved announce cache cleaning
|
||||
- Improved `rngit` page node link handling
|
||||
- Improved stats pages `rngit`
|
||||
- Improved transfer completed feedback in `rncp`, thanks to **neutral**
|
||||
- Improved interface transport insertion and removal
|
||||
|
||||
### 2026-05-14: RNS 1.2.6
|
||||
|
||||
This release adds further improvements to the `rnid` and `rngit` utilities, and includes several bugfixes and other improvements.
|
||||
|
||||
**Changes**
|
||||
- Added embedded message signing, validation and viewing to `rnid`
|
||||
- Added file encryption for multiple file path inputs and shell expansions to `rnid`
|
||||
- Added file decryption for multiple file path inputs and shell expansions to `rnid`
|
||||
- Added signature creation for multiple file path inputs and shell expansions to `rnid`
|
||||
- Added signature validation of multiple file path inputs and shell expansions to `rnid`
|
||||
- Added workdoc signing and validation to `rngit`
|
||||
- Added ability to edit workdoc titles to `rngit`
|
||||
- Added ability to download workdocs via the `nomadnet` interface to `rngit`
|
||||
- Added local URL resolution to the `rngit` repository frontpage markdown readme renderer
|
||||
- Improved `rnstatus` remote monitor loop
|
||||
- Improved `rngit` workdoc page handling
|
||||
- Improved `rngit` release page rendering
|
||||
- Fixed missing none check in interface discovery sanitizer thanks to PAzter1101
|
||||
- Fixed potential race condition in interface discovery
|
||||
- Fixed `rngit` remote helper hanging on startup if no client config had been created previously, and RNS loglevel was configured at debug or higher
|
||||
|
||||
### 2026-05-09: RNS 1.2.5
|
||||
|
||||
This release brings substantial improvements to path request handling, and should significantly reduce overall network and local transport node processing loads. Path requests are now automatically ingress and egress limited per interface and sub-interface. Although the defaults are effective and sane, and should work right out of the box bring an end to practically all the PR and announce spam going on lately, the backend is fully configurable for both defaults and per interface, if you want to fiddle with the settings.
|
||||
|
||||
People who have written (ahem... *prompted into existence*) strange applications, that believed sending 25 random path requests every 10 seconds to try and punch holes through announce limiting, will now most likely find any potential users of such applications complaining that they are losing the ability to resolve paths alltogether, which is (entirely) by design, of course. Seriously, don't do crap like that.
|
||||
|
||||
You can read more about how the new ingress and egress controls work in the updated manual sections, in the Interfaces chapter.
|
||||
|
||||
For all node ops out there, I'd recomment updating to this at some sort of semi-expedient, but of course not un-leisurely pace, so peace and order on the networks can be restored.
|
||||
|
||||
**Changes**
|
||||
- Added path request ingress and egress control with sane defaults for transport nodes
|
||||
- Added full configurability of ingress and egress controls per interface and for instance-wide defaults
|
||||
- Significantly improved transport logic for path request and announce handling
|
||||
- Added path request frequency display to `rnstatus`
|
||||
- Added AutoInterface per-peer announe rate display to `rnstatus`
|
||||
- Added abilit to filter interfaces by burst state to `rnstatus`
|
||||
- Added hex/base32/base64 ASCII-wrapped output to `rnid` signature generator
|
||||
- Tuned default ingress control parameters
|
||||
- Fixed regression in link close handling in `rnstatus` and `rnpath` remote management handling
|
||||
- Fixed invalid handling of corrupted interface discovery files
|
||||
- Fixed announce processing edge case handling if path was cleaned while waiting for rebroadcast
|
||||
- Improved `rngit` error logging
|
||||
- Improved transport background jobs error handling
|
||||
- Fixed various edge-cases and inconsistencies in markdown rendering in `rngit`
|
||||
- Ensured canonical validation functions in `rngit`
|
||||
- Lots of other small fixes and stability improvements to `rngit`
|
||||
|
||||
### 2026-05-07: RNS 1.2.4
|
||||
|
||||
This release brings a complete rewrite and update to the `rnid` utility, which is now a lot more useful, and better at finding and saving identities. It also includes a bunch of other improvements, such as expanded `rngit` functionality, better transport performance and a few bugfixes. Enjoy!
|
||||
|
||||
Unless something really crazy happens, this will probably be the last release that is also published to GitHub, since everything can now run over Reticulum itself. Updates to `pip` will continue at least until `rnpkg` is complete, and RNS is completely self-hosting.
|
||||
|
||||
**Changes**
|
||||
- Completely rewrote the `rnid` utility, **much** better now
|
||||
- Added ability to query network for raw identities to `rnid`
|
||||
- Added new, much more useful `rsg` file signature format
|
||||
- Added auto-retain functionality for used identities to `rnid`
|
||||
- Added outbound announce frequency per-client display to `rnstatus`
|
||||
- Added announce rate control settings display to `rnstatus`
|
||||
- Added announce rate control defaults configuration options
|
||||
- Added saner default announce rate settings for transport nodes
|
||||
- Added detection of Yggdrasil addresses to auto-connect handler
|
||||
- Added work document permissions resolver to `rngit`
|
||||
- Added ability to create updates and comments on `rngit` work documents
|
||||
- Added work document permissions control logic and CLI interaction to `rngit`
|
||||
- Added support for node-local URL-scoping in `rngit` markdown converter
|
||||
- Added API functionality for retaining identity data
|
||||
- Added the manual in markdown format
|
||||
- Improved `rngit` releases page rendering
|
||||
- Improved auto-connect logging
|
||||
- Improved transport performance
|
||||
- Improved logging performance
|
||||
- Improved shutdown handling
|
||||
- Improved workdoc sorting
|
||||
- Fixed time formatting being unintuitive sometimes
|
||||
- Fixed markdown-to-micron formatting and syntax highlighting being weird sometimes
|
||||
|
||||
**Release Hashes**
|
||||
```
|
||||
e821a0b6a18d6b3263bbcdde880d0388fb4dd0c07c7eb2f83cb0dbc30eda5965 rns-1.2.4-py3-none-any.whl
|
||||
618e823cec0bd368f2f211431dfb78efef75e59132bad93d3101dacbe7deb7a6 rnspure-1.2.4-py3-none-any.whl
|
||||
```
|
||||
|
||||
**Release Signatures**
|
||||
Release artifacts include `rsg` signature files that can be validated against the RNS release signing identity `<bc7291552be7a58f361522990465165c>` using `rnid`. To verify files, download the `rsg` signatures, make sure they are in the same folder as the release artifact, and run `rnid` signature verification with the release identity as the required signer:
|
||||
|
||||
```sh
|
||||
rnid -i bc7291552be7a58f361522990465165c -V rns-1.2.4-py3-none-any.whl
|
||||
```
|
||||
|
||||
The `rnid` utility will then verify the signatures, and display whether it is valid. If the signature cannot be verified, the file has been tampered with and should be thrown very far away in a jiffy.
|
||||
|
||||
This is the first release using the new `rsg` signature format, and you will need this latest version of RNS to verify them. Ironic, I know, but that's how it is. Since release file hashes are now embbeded in the `rsg` signatures, this is the last release that will explicitly post the raw release hashes. Verifying with `rnid` is much more effective, since it ensures all data was signed by the release identity.
|
||||
|
||||
### 2026-05-05: RNS 1.2.3
|
||||
|
||||
This release adds Work Document and update/commenting support to `rngit`.
|
||||
|
||||
**Changes**
|
||||
- Added Work Document management to `rngit`.
|
||||
- Added Work pages to the page node of `rngit`.
|
||||
- Added `interact` permission type to `rngit`.
|
||||
- Added `admin` permission type to `rngit`.
|
||||
- Added markdown blockquote support to the `rngit` markdown-to-micron converter.
|
||||
- Improved markdown-to-micron conversion and syntax highlighting accuracy in `rngit`.
|
||||
|
||||
**Release Hashes**
|
||||
```
|
||||
8562130f297a6b33be9d72c449bbe6ae83cad41e1530e0fa112f5fa545a3f364 rns-1.2.3-py3-none-any.whl
|
||||
0862f46a08e610add1bcac0916c6554f3e79590ab2765900178d5e1f1f0c7026 rnspure-1.2.3-py3-none-any.whl
|
||||
```
|
||||
|
||||
**Release Signatures**
|
||||
Release artifacts include `rsg` signature files that can be validated against the RNS release signing identity `<bc7291552be7a58f361522990465165c>` using `rnid`:
|
||||
|
||||
```sh
|
||||
rnid -i bc7291552be7a58f361522990465165c -V rns-1.2.3-py3-none-any.whl.rsg
|
||||
```
|
||||
|
||||
### 2026-05-05: RNS 1.2.2
|
||||
|
||||
This release adds release management workflows to the `rngit` utility. Downloading files and release artifacts from `rngit` will require the latest version of Nomad Network. Other nomadnet clients *may* have to update their file download link handling, if they don't already support passing query parameters for file download links.
|
||||
|
||||
**Changes**
|
||||
- Added release management to `rngit`.
|
||||
- Added release pages to the page node of `rngit`.
|
||||
- Added file downloads in the tree browser of `rngit`.
|
||||
|
||||
**Release Hashes**
|
||||
```
|
||||
4bf0a376a9778de8a91b9ec8a5bc4b929be928eede8784b20022c7fe52bbce62 rns-1.2.2-py3-none-any.whl
|
||||
d85f8b765dcf718d284388b249ca0e48e785f250bb41773a83e159e46c5bcf70 rnspure-1.2.2-py3-none-any.whl
|
||||
```
|
||||
|
||||
**Release Signatures**
|
||||
Release artifacts include `rsg` signature files that can be validated against the RNS release signing identity `<bc7291552be7a58f361522990465165c>` using `rnid`:
|
||||
|
||||
```sh
|
||||
rnid -i bc7291552be7a58f361522990465165c -V rns-1.2.2-py3-none-any.whl.rsg
|
||||
```
|
||||
|
||||
### 2026-05-04: RNS 1.2.1
|
||||
|
||||
This release adds a nomadnet Git page node to the `rngit` utility.
|
||||
|
||||
**Changes**
|
||||
- Added nomadnet page node to `rngit`.
|
||||
|
||||
**Release Hashes**
|
||||
```
|
||||
5ccbfc31b528133c4dd06c132034c2151e4eed74bc2dcf40af52385094492c9e rns-1.2.1-py3-none-any.whl
|
||||
cda45994a58f18bf25244a1f396c9197240bc012dd85c86bffc2e73dcf0607de rnspure-1.2.1-py3-none-any.whl
|
||||
```
|
||||
|
||||
**Release Signatures**
|
||||
Release artifacts include `rsg` signature files that can be validated against the RNS release signing identity `<bc7291552be7a58f361522990465165c>` using `rnid`:
|
||||
|
||||
```sh
|
||||
rnid -i bc7291552be7a58f361522990465165c -V rns-1.2.1-py3-none-any.whl.rsg
|
||||
```
|
||||
|
||||
### 2026-04-28: RNS 1.2.0
|
||||
|
||||
This release brings the ability to use Git natively over Reticulum networks, adds the `rnsh` program as part of the included utilities, and additionally includes several improvements and performance optimizations.
|
||||
|
||||
**Changes**
|
||||
- Added Reticulum Git Repositories Node utility as part of included utility programs.
|
||||
- Added git remote helper to interact with git repositories over Reticulum.
|
||||
- Added the `rnsh` program to the included utilities.
|
||||
- Added LocalInterface client TX hold on client app sleep on Android.
|
||||
- Added AutoInterface filters for `rmnet` interfaces on Android.
|
||||
- Added inbound packet wait during transport core initialization.
|
||||
- Added the ability to set logfile destination before RNS initialization.
|
||||
- Added automatic active link teardown on instance shutdown.
|
||||
- Improved link teardown on SIGINT/SIGTERM.
|
||||
- Improved ratchet cleaning.
|
||||
|
||||
**Release Hashes**
|
||||
```
|
||||
b58e97332241755ed32e309d46e09615a123490430ae85fcbdec9318c9e26154 rns-1.2.0-py3-none-any.whl
|
||||
9813a6c2236edba18af7d3a072a6226bc65ae384d23b1f41467cb3617d65fdae rnspure-1.2.0-py3-none-any.whl
|
||||
```
|
||||
|
||||
**Release Signatures**
|
||||
Release artifacts include `rsg` signature files that can be validated against the RNS release signing identity `<bc7291552be7a58f361522990465165c>` using `rnid`:
|
||||
|
||||
```sh
|
||||
rnid -i bc7291552be7a58f361522990465165c -V rns-1.2.0-py3-none-any.whl.rsg
|
||||
```
|
||||
|
||||
### 2026-04-22: RNS 1.1.9
|
||||
|
||||
This maintenance release fixes a critical security issue, that would allow an attacker to craft a BZ2 decompression bomb via Resource transfers or Buffer StreamDataMessage, causing an out-of-memory condition and crashing the receiving process via OOM killer.
|
||||
|
||||
Big thanks to @defidude (github.com/ratspeak) for discovering and reporting this vulnerability!
|
||||
|
||||
**Changes**
|
||||
- Fixed bz2 decompression bomb vulnerability in Resource transfer assembly and Buffer StreamDataMessage unpacking.
|
||||
|
||||
**Release Hashes**
|
||||
```
|
||||
39a131aeb5d76fd73bfc67f68135f49ab0cf8628af154e04096a05c208ce77b6 rns-1.1.9-py3-none-any.whl
|
||||
aab7bfc8c65514c9bdf4c22f00d288faf6c9e1777fc002dbe3eb29c286e67128 rnspure-1.1.9-py3-none-any.whl
|
||||
```
|
||||
|
||||
**Release Signatures**
|
||||
Release artifacts include `rsg` signature files that can be validated against the RNS release signing identity `<bc7291552be7a58f361522990465165c>` using `rnid`:
|
||||
|
||||
```sh
|
||||
rnid -i bc7291552be7a58f361522990465165c -V rns-1.1.9-py3-none-any.whl.rsg
|
||||
```
|
||||
|
||||
### 2026-04-21: RNS 1.1.8
|
||||
|
||||
This maintenance release fixes a critical bug in path state management, that could result in significant path convergence degradation under certain conditions.
|
||||
|
||||
**Changes**
|
||||
- Fixed path state potentially being applied before path table entry exists, causing worse paths to be selected.
|
||||
|
||||
**Release Hashes**
|
||||
```
|
||||
9cf728e9e9a9fe113e4ac14e6b833f7ee65feedf8468e6ab94a261bf205f2632 rns-1.1.8-py3-none-any.whl
|
||||
407dc3975335e9eabaaddb7ed1dc75fc3a1b8d24a7207e740797440c2ad0b3e5 rnspure-1.1.8-py3-none-any.wh
|
||||
```
|
||||
|
||||
**Release Signatures**
|
||||
Release artifacts include `rsg` signature files that can be validated against the RNS release signing identity `<bc7291552be7a58f361522990465165c>` using `rnid`:
|
||||
|
||||
```sh
|
||||
rnid -i bc7291552be7a58f361522990465165c -V rns-1.1.8-py3-none-any.whl.rsg
|
||||
```
|
||||
|
||||
### 2026-04-21: RNS 1.1.7
|
||||
|
||||
**Changes**
|
||||
- Added periodic known destination data cleaning based on local relevance.
|
||||
- Improved resource transfer sequencing timing calculations and reliability.
|
||||
- Improved BackboneInterface error handling on EPOLL errors.
|
||||
- Ensured non-background data persist runs synchronously.
|
||||
|
||||
**Release Hashes**
|
||||
```
|
||||
4d9702c5d9bb8a3c8b94766cb51cccad5afd78d615af9a6b146730347044e6f0 rns-1.1.7-py3-none-any.whl
|
||||
172dede7656b41b85e4319354ed04649b518e58c54586da7e443579c620a0a5b rnspure-1.1.7-py3-none-any.whl
|
||||
```
|
||||
|
||||
**Release Signatures**
|
||||
Release artifacts include `rsg` signature files that can be validated against the RNS release signing identity `<bc7291552be7a58f361522990465165c>` using `rnid`:
|
||||
|
||||
```sh
|
||||
rnid -i bc7291552be7a58f361522990465165c -V rns-1.1.7-py3-none-any.whl.rsg
|
||||
```
|
||||
|
||||
### 2026-04-18: RNS 1.1.6
|
||||
|
||||
**Changes**
|
||||
- Improved transport memory consumption.
|
||||
- Improved transport tunnel handling.
|
||||
- Improved gracious transport data persist handling.
|
||||
- Added ingress control bypass for pending path requests.
|
||||
- Added local destinations lookup map for better transport efficiency to local destinations.
|
||||
- Fixed disk I/O bound thread execution time starvation on cache management jobs.
|
||||
- Fixed invalid EPOLL modification error handler.
|
||||
- Fixed incorrect default IFAC size for autoconnected, discovered interfaces. Thanks @taprootmx!
|
||||
- Ensure loop-originating closures have variables captured at iteration-time. Thanks @taprootmx!
|
||||
|
||||
**Release Hashes**
|
||||
```
|
||||
2ce4451668f8c464295cc269188c232e7805ddd618ec0135550a5e6809df5de0 rns-1.1.6-py3-none-any.whl
|
||||
ba3e541e69a2f4892177383c8ec4e7d172d298546317e08270928c0163865aa3 rnspure-1.1.6-py3-none-any.wh
|
||||
```
|
||||
|
||||
**Release Signatures**
|
||||
Release artifacts include `rsg` signature files that can be validated against the RNS release signing identity `<bc7291552be7a58f361522990465165c>` using `rnid`:
|
||||
|
||||
```sh
|
||||
rnid -i bc7291552be7a58f361522990465165c -V rns-1.1.6-py3-none-any.whl.rsg
|
||||
```
|
||||
|
||||
### 2026-04-13: RNS 1.1.5
|
||||
|
||||
**Changes**
|
||||
- Initial refactoring work for free-threaded transport I/O.
|
||||
- Improved interface discovery validation.
|
||||
- Fixed invalid ingress control burst activation and subsequent path resolution failure due to incorrect announce frequency calculation.
|
||||
- Fixed missing configuration entry generation for discovered I2P interfaces.
|
||||
- Fixed resource transfer cancellation failing on in-flight split resource transfers.
|
||||
- Fixed ingress control configuration not inheriting down to spawned interfaces on some interface types.
|
||||
|
||||
**Release Hashes**
|
||||
```
|
||||
28f39ad97ef307a1e270b91ef19db07d8e1a7bbc8628c478303725894c64deff rns-1.1.5-py3-none-any.whl
|
||||
1a90db16d2cff4ad909b44baf9b4fd0177da2ed545cdb9cfb2c51423707b49e9 rnspure-1.1.5-py3-none-any.whl
|
||||
```
|
||||
|
||||
**Release Signatures**
|
||||
Release artifacts include `rsg` signature files that can be validated against the RNS release signing identity `<bc7291552be7a58f361522990465165c>` using `rnid`:
|
||||
|
||||
```sh
|
||||
rnid -i bc7291552be7a58f361522990465165c -V rns-1.1.5-py3-none-any.whl.rsg
|
||||
```
|
||||
|
||||
#
|
||||
|
||||
### 2026-03-12: RNS 1.1.4
|
||||
|
||||
**Changes**
|
||||
- Fixed invalid application of IP/hostname validation for on non-relevant interfaces. Thanks @joakim!
|
||||
|
||||
**Release Hashes**
|
||||
```
|
||||
b2a175abd64d1581dd058206832793dbf7053a304c819ff8bc143a79c49cb747 rns-1.1.4-py3-none-any.whl
|
||||
16c4ae6722bbd016e8db046e7bdd60eb24f9ec55966ec5723dc39301265d0186 rnspure-1.1.4-py3-none-any.whl
|
||||
```
|
||||
|
||||
**Release Signatures**
|
||||
Release artifacts include `rsg` signature files that can be validated against the RNS release signing identity `<bc7291552be7a58f361522990465165c>` using `rnid`:
|
||||
|
||||
```sh
|
||||
rnid -i bc7291552be7a58f361522990465165c -V rns-1.1.4-py3-none-any.whl.rsg
|
||||
```
|
||||
|
||||
### 2026-01-17: RNS 1.1.3
|
||||
|
||||
**Changes**
|
||||
- Improved discovered interface auto-connect handling
|
||||
- Improved interface discovery handling
|
||||
- Added `discovered_interfaces` API method
|
||||
- Fixed a potential race condition in request timeout handling
|
||||
- Fixed a regression in resource file transfers
|
||||
|
||||
**Release Hashes**
|
||||
```
|
||||
1de9b46c8f24931fa41974664ddbf4251d3fdd069be4de03c64b42a7cf4f8fb4 rns-1.1.3-py3-none-any.whl
|
||||
eac8d223fcb6ce94e1bd3f04730d8542675caf4b22286e11988e9402ea9b69c0 rnspure-1.1.3-py3-none-any.whl
|
||||
```
|
||||
|
||||
**Release Signatures**
|
||||
Release artifacts include `rsg` signature files that can be validated against the RNS release signing identity `<bc7291552be7a58f361522990465165c>` using `rnid`:
|
||||
|
||||
```sh
|
||||
rnid -i bc7291552be7a58f361522990465165c -V rns-1.1.3-py3-none-any.whl.rsg
|
||||
```
|
||||
|
||||
### 2026-01-04: RNS 1.1.0
|
||||
|
||||
Enjoy.
|
||||
|
|
|
|||
|
|
@ -2,57 +2,66 @@
|
|||
|
||||
Welcome, and thank you for your interest in contributing to Reticulum!
|
||||
|
||||
Apart from writing code, there are many ways in which you can contribute. Before interacting with this community, read these short and simple guidelines.
|
||||
Before interacting with this community, read these short and simple guidelines.
|
||||
|
||||
If you didn't shit your pants after reading this document, you may well find that we are all really quite cuddly and nice to be around. Mostly.
|
||||
|
||||
## Expected Conduct
|
||||
|
||||
First and foremost, there is one simple requirement for taking part in this community: While we primarily interact virtually, your actions matter and have real consequences. Therefore: **Act like a responsible, civilized person** - especially in the face of disputes and heated disagreements. Speak your mind here; discussions are welcome. Just do so in the spirit of being face-to-face with everyone else. Thank you.
|
||||
First and foremost, there is one simple requirement for taking part in the development of this technology: While we primarily interact virtually, your actions matter and have real consequences. Therefore: **Act like a responsible, civilized person** - especially in the face of disputes and heated disagreements. Speak your mind here; discussions are welcome. Just do so in the spirit of being face-to-face with everyone else. Thank you.
|
||||
|
||||
In order to keep the discussion forums and issue trackers navigable and useful, the following types of posts will be deleted without notice:
|
||||
In order to keep the discourse navigable and useful, the following types of posts and inquiries will be deleted and ignored:
|
||||
|
||||
- Spam.
|
||||
- Questions that have already been adequately answered elsewhere. Use the search function.
|
||||
- Low-effort posts or comments that contain no actual information or useful content. This is not a tea-house.
|
||||
- Questions that have already been adequately answered elsewhere. Utilize search functions.
|
||||
- Low-effort posts or comments that contain no actual information or useful substance. [This is not a tea-house](./This%20Is%20Not%20a%20Teahouse.md).
|
||||
- Post or comments solely containing personal opinions or beliefs without adding anything to the discussion. Facebook and X exists.
|
||||
- Content that simply waste the developer's / maintainer's time with completely obvious "ideas", "insights" or "recommendations". Yes, we have *at least* 8 neurons ourselves.
|
||||
- Posts that fail to understand that developing a highly complex software project with a very small amount of resources and people takes time. Imagining perfection on our behalf is useless.
|
||||
|
||||
If you're new to the community and start out your engagement with any of the above transgressions, you will simply be banned without notice or explanation, and your post will be deleted.
|
||||
If you're new to the community and start out your engagement with any of the above transgressions, you will simply be banned permanently without notice or explanation, and any further communication from you will be ignored.
|
||||
|
||||
Except for the case of real, well-grounded, clearly documented, human-written security advisories, technical investigations or bug reports, if your inquiry or communication is not attached to a Reticulum Identity, expect it to be ignored.
|
||||
|
||||
If you find this "harsh", "unfair" or "unwelcoming", go somewhere else. This is not social club, but a work environment for the people contributing to the project.
|
||||
|
||||
## Asking Questions
|
||||
## Reporting Bugs
|
||||
|
||||
If you want to ask a question, **do not open an issue**. The issue tracker is used by people *working on Reticulum* to track bugs, issues and improvements. Instead, ask away on the [discussions](https://github.com/markqvist/Reticulum/discussions).
|
||||
|
||||
Do not post feature requests or general ideas on the issue tracker, or in direct messages to the primary developers. You are much more likely to get a response and start a constructive discussion by posting your ideas in the public channels created for these purposes.
|
||||
|
||||
## Reporting Issues
|
||||
|
||||
If you have found a bug or issue in this project, please report it using the [issue tracker](https://github.com/markqvist/Reticulum/issues). Be sure to include details on how to reproduce the bug.
|
||||
|
||||
Anything submitted to the issue tracker that does not follow these guidelines will be closed and removed without comments or explanation.
|
||||
If you have found a bug or issue in this project, please report it to the maintainer over LXMF, or if strictly not possible, over email. Detailed and serious bug reports can also be posted in the various on-network forums and RRC channels that the developers frequent. Due to the realities described in [This Is Not a Teahouse](./This%20Is%20Not%20a%20Teahouse.md), there is no way to report issues or request features over the internet.
|
||||
|
||||
## Writing Code
|
||||
|
||||
If you are interested in contributing code, fixing open issues or adding features, please coordinate the effort with the maintainer or one of the main developers **before** submitting a pull request. Before deciding to contribute, it is also a good idea to ensure your efforts are in alignment with the [Roadmap](./Roadmap.md) and current development focus.
|
||||
Patches and code contributions are *generally* only welcomed from developers already known to the community, with a proven track-record of competence, skill, careful and considerate design and responsible conduct. If we do not know you yet, but you have created something that you believe lives up to such standards, we are eager to hear from you.
|
||||
|
||||
Pull requests have a high chance of being accepted if they are:
|
||||
Current development priorities and work documents are not publicly available on the internet, but shared and coordinated directly between the developers over Reticulum itself, following the principles outlined in [Distributed Development](https://reticulum.network/manual/distributed.html).
|
||||
|
||||
- In alignment with the [Roadmap](./Roadmap.md) or solve an open issue or feature request
|
||||
- Sufficiently tested to work with all API functions, and pass the standard test suite
|
||||
- Functionally and conceptually complete and well-designed
|
||||
- Not simply formatting or code style changes
|
||||
- Well-documented
|
||||
## Commit Signing
|
||||
|
||||
Even new ideas and proposals that have not been approved by a maintainer, or fall outside the established roadmap, are *occasionally* accepted - if they possess the remaining of the above qualities. If not, they will be closed and removed without comments or explanation.
|
||||
All commits in Reticulum and related projects **must** be signed with a Reticulum identity. If you want your author credentials to be included directly in the commit log, ensure that you have [configured commit signing](https://reticulum.network/manual/git.html#commit-signing). Bare patches submitted manually or fetched remote commits without signatures are still very welcome, but we will review, squash merge and sign with the key of the maintainer that approved the changes. Attribution will be provided in the commit message instead (unless you wish to remain anonymous).
|
||||
|
||||
## Generative AI Policy
|
||||
## Machine Assistance & LLMs
|
||||
|
||||
Contributions written using large language models (LLMs) or other generative 'AI' programs are prohibited. LLMs produce errors so frequently and in a way that is so unlike human error that such issues are incredibly time-consuming to spot and fix. This is not a worthwhile tradeoff for Reticulum.
|
||||
For a highly complex, critical piece of cryptographically based networking infrastructure, we should not need to state the following, but alas, experience has proven otherwise. Therefore:
|
||||
|
||||
This applies to all Reticulum-related projects and documentation, as well as all submitted issues and discussion in official channels, except in cases where language translation and/or speech recogntion technologies are required for communication.
|
||||
**Including material generated using LLMs or other "AI" programs in Reticulum is, almost without exception, prohibited.**
|
||||
|
||||
Before even *thinking* about using a generative software system to "create" something for Reticulum, read [Brandolini's Reference](https://reticulum.network/manual/brandolinis.html), in full. If you do not have the capacity for that, you have failed the first requirement for contributing: Basic literacy.
|
||||
|
||||
*If* you are not already a known contributor with a proven record of being able to responsibly handle these tools, a **full** audit trail of all ML-utilizing workflows, including prompts, LLM reasoning traces, tool calls and outputs (that is, the *entire*, unaltered session log) will need to be provided with any proposed contribution. Without that, we will not even *consider* looking at it.
|
||||
|
||||
This applies to all Reticulum-related projects and documentation, as well as all submitted issues, reports and discussion, **except** of course in cases where language translation, speech recogntion, or other assistive technologies are required.
|
||||
|
||||
In **all** circumstances, clear disclosure is **mandatory**. Any piece of material, file, code snippet, method, function or similar that was generated with machine assistance must be prepended with a disclosure header or comment that states the framework, exact model used and reviewer, such as:
|
||||
|
||||
```python
|
||||
# Generated with lc/deepseek_v4_flash_q8_k_xl
|
||||
# Reviewed and corrected by Mark Qvist
|
||||
```
|
||||
|
||||
Additionally, any explanatory comments or doc-strings generated by software must be manually re-written in clear, concise language by the reviewer stated in the disclosure header.
|
||||
|
||||
## Contributor License Agreement
|
||||
|
||||
By contributing code to this project, you agree that copyright for the code is transferred to the Reticulum maintainers and that the code is irrevocably placed under the [Reticulum License](./LICENSE).
|
||||
|
||||
For a more in-depth elaboration on the current CLA position, see [this post](https://rns.recipes/forum/general/brandolinis-reference-or-alternatively-agentsmd?page=1#post-3002).
|
||||
|
|
|
|||
|
|
@ -42,6 +42,9 @@ def server(configpath):
|
|||
"linkexample"
|
||||
)
|
||||
|
||||
# We can optionally configure a link request stamp cost
|
||||
server_destination.set_link_cost(14.2)
|
||||
|
||||
# We configure a function that will get called every time
|
||||
# a new client creates a link to this destination.
|
||||
server_destination.set_link_established_callback(client_connected)
|
||||
|
|
|
|||
|
|
@ -92,6 +92,7 @@ def resource_concluded(resource):
|
|||
if resource.status == RNS.Resource.COMPLETE:
|
||||
RNS.log(f"Resource {resource} received")
|
||||
RNS.log(f"Metadata: {resource.metadata}")
|
||||
RNS.log(f"Application flag is {'set' if resource.app_flag else 'not set'}")
|
||||
RNS.log(f"Data length: {os.stat(resource.data.name).st_size}")
|
||||
RNS.log(f"Data can be read directly from: {resource.data}")
|
||||
RNS.log(f"Data can be moved or copied from: {resource.data.name}")
|
||||
|
|
@ -192,16 +193,22 @@ def client_loop():
|
|||
RNS.log(f"First 32 bytes of data: {RNS.hexrep(data[:32])}")
|
||||
|
||||
# Generate some metadata
|
||||
metadata = {"text": random_text_generator(), "numbers": [1,2,3,4], "blob": os.urandom(16)}
|
||||
metadata = {"text": random_text_generator(),
|
||||
"numbers": [1,2,3,4], "blob": os.urandom(16)}
|
||||
|
||||
# An optional application-specific flag can be set
|
||||
flag_set = True
|
||||
|
||||
# Send the resource
|
||||
resource = RNS.Resource(data, server_link, metadata=metadata, callback=resource_concluded_sending, auto_compress=False)
|
||||
resource = RNS.Resource(data, server_link, metadata=metadata, app_flag=flag_set,
|
||||
callback=resource_concluded_sending, auto_compress=False)
|
||||
|
||||
# Alternatively, you can stream data
|
||||
# directly from an open file descriptor
|
||||
|
||||
# with open("/path/to/file", "rb") as data_file:
|
||||
# resource = RNS.Resource(data_file, server_link, metadata=metadata, callback=resource_concluded_sending, auto_compress=False)
|
||||
# resource = RNS.Resource(data_file, server_link, metadata=metadata,
|
||||
# callback=resource_concluded_sending, auto_compress=False)
|
||||
|
||||
except Exception as e:
|
||||
RNS.log("Error while sending resource over the link: "+str(e))
|
||||
|
|
@ -222,7 +229,7 @@ def link_established(link):
|
|||
|
||||
# Inform the user that the server is
|
||||
# connected
|
||||
RNS.log("Link established with server, hit enter to sand a resource, or type in \"quit\" to quit")
|
||||
RNS.log("Link established with server, hit enter to send a resource, or type in \"quit\" to quit")
|
||||
|
||||
# When a link is closed, we'll inform the
|
||||
# user, and exit the program
|
||||
|
|
|
|||
|
|
@ -239,7 +239,7 @@ def link_established(link):
|
|||
RNS.log("Link established with server, sending...")
|
||||
rd = os.urandom(link.mdu)
|
||||
started = time.time()
|
||||
while link.status == RNS.Link.ACTIVE and data_sent < data_cap*1.25:
|
||||
while link.status != RNS.Link.CLOSED and data_sent < data_cap*1.25:
|
||||
RNS.Packet(server_link, rd, create_receipt=False).send()
|
||||
data_sent += len(rd)
|
||||
|
||||
|
|
|
|||
2
LICENSE
2
LICENSE
|
|
@ -1,6 +1,6 @@
|
|||
Reticulum License
|
||||
|
||||
Copyright (c) 2016-2025 Mark Qvist
|
||||
Copyright (c) 2016-2026 Mark Qvist
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
|
|||
49
Makefile
49
Makefile
|
|
@ -21,12 +21,26 @@ clean:
|
|||
@-rm -rf ./tests/__pycache__
|
||||
@-rm -rf ./tests/rnsconfig/storage
|
||||
@-rm -rf ./*.egg-info
|
||||
@-rm -rf ./RNS/*.c
|
||||
@-rm -rf ./RNS/Cryptography/*.c
|
||||
@-rm -rf ./RNS/Cryptography/aes/*.c
|
||||
@-rm -rf ./RNS/Cryptography/pure25519/*.c
|
||||
@-rm -rf ./RNS/Interfaces/*.c
|
||||
@-rm -rf ./RNS/Interfaces/util/*.c
|
||||
@-rm -rf ./RNS/Interfaces/Android/*.c
|
||||
@-rm -rf ./RNS/Utilities/*.c
|
||||
@-rm -rf ./RNS/Utilities/rngit/*.c
|
||||
@-rm -rf ./RNS/Utilities/rnsh/*.c
|
||||
@-rm -rf ./RNS/vendor/*.c
|
||||
@-rm -rf ./RNS/vendor/i2plib/*.c
|
||||
@-rm -rf
|
||||
@make -C docs clean
|
||||
@echo Done
|
||||
|
||||
purge_docs:
|
||||
@echo Purging documentation build...
|
||||
@-rm -rf ./docs/manual
|
||||
@-rm -rf ./docs/markdown
|
||||
@-rm -rf ./docs/*.pdf
|
||||
@-rm -rf ./docs/*.epub
|
||||
|
||||
|
|
@ -49,21 +63,48 @@ build_wheel:
|
|||
build_pure_wheel:
|
||||
python3 setup.py bdist_wheel --pure
|
||||
|
||||
build_native:
|
||||
python3 setup.py bdist_wheel --native
|
||||
|
||||
documentation:
|
||||
make -C docs html
|
||||
make -C docs html markdown
|
||||
|
||||
manual:
|
||||
make -C docs latexpdf epub
|
||||
|
||||
distcollect:
|
||||
mv docs/Reticulum\ Manual.* dist
|
||||
|
||||
build_spkg: remove_symlinks build_sdist create_symlinks
|
||||
|
||||
release: test remove_symlinks build_sdist build_wheel build_pure_wheel documentation manual create_symlinks
|
||||
release: test remove_symlinks build_sdist build_wheel build_pure_wheel documentation manual distcollect create_symlinks
|
||||
|
||||
debug: remove_symlinks build_wheel build_pure_wheel create_symlinks
|
||||
|
||||
native: remove_symlinks build_native create_symlinks
|
||||
|
||||
local: release sign
|
||||
|
||||
sign:
|
||||
rngit release rns://7649a50d84610232d1416b41d2896aff/reticulum/reticulum create $$(python setup.py --getversion):dist --name rns --local
|
||||
|
||||
upload:
|
||||
@echo Ready to publish release, hit enter to continue
|
||||
@echo Ready to publish release over Reticulum
|
||||
@read VOID
|
||||
rngit release rns://7649a50d84610232d1416b41d2896aff/reticulum/reticulum create $$(python setup.py --getversion):dist --name rns
|
||||
|
||||
upload-pip: upload-rns-pip upload-rnspure-pip
|
||||
|
||||
upload-rns-pip:
|
||||
@echo Ready to publish rns release, hit enter to continue
|
||||
@read VOID
|
||||
@echo Uploading to PyPi...
|
||||
twine upload dist/*
|
||||
twine upload dist/rns-*.whl dist/rns-*.tar.gz
|
||||
@echo Release published
|
||||
|
||||
upload-rnspure-pip:
|
||||
@echo Ready to publish rnspure release, hit enter to continue
|
||||
@read VOID
|
||||
@echo Uploading to PyPi...
|
||||
twine upload dist/rnspure-*.whl
|
||||
@echo Release published
|
||||
329
README.md
329
README.md
|
|
@ -1,43 +1,37 @@
|
|||
Reticulum Network Stack <img align="right" src="https://static.pepy.tech/personalized-badge/rns?period=month&units=international_system&left_color=grey&right_color=blue&left_text=Installs/month" style="padding-left:10px"/><a href="https://github.com/markqvist/Reticulum/actions/workflows/build.yml"><img align="right" src="https://github.com/markqvist/Reticulum/actions/workflows/build.yml/badge.svg"/></a>
|
||||
==========
|
||||
|
||||
> [!WARNING]
|
||||
> A significant number of hastily launched, incorrect, dubious, directly harmful and, in most cases, entirely LLM-generated fakes of Reticulum implementations, "documentation", "guides" and user-facing applications are currently being marketed under polished, but ultimately misleading or outright deceptive claims.
|
||||
>
|
||||
> For more detailed information, see the relevant sections of the [Brandolini's Reference](https://reticulum.network/manual/brandolinis.html) chapter of the manual. Known good community implementations are listed in the [Community Implementations](#community-implementations) section of this readme. For software and projects recognized by the Reticulum community, see the [Programs Using Reticulum](https://reticulum.network/manual/software.html) chapter of the manual.
|
||||
>
|
||||
> Machine-generated content derived from the RNS source code or documentation (such as "implementations", "guides" for sale, et cetera), are in several, well-documented cases violating the open source licenses that granted rights of copying and distribution.
|
||||
>
|
||||
> Any claimed assertion of copyright or grant of license (Apache, GPL, et cetera) that such projects are making are [void grants](https://reticulum.network/manual/brandolinis.html#void-grants-legal-foundations-of-machine-generated-code). Both the violators themselves, and all downstream projects using the infringing derivatives, are **directly** and personally liable for the legal consequences.
|
||||
>
|
||||
> Do **not** use these "projects", they are a hazard to the entire ecosystem we have been carefully building over the last ten years.
|
||||
|
||||
<p align="center"><img width="200" src="https://raw.githubusercontent.com/markqvist/Reticulum/master/docs/source/graphics/rns_logo_512.png"></p>
|
||||
|
||||
*This repository is [a public mirror](./MIRROR.md). All development is happening elsewhere.*
|
||||
|
||||
Reticulum is the cryptography-based networking stack for building local and wide-area
|
||||
networks with readily available hardware. It can operate even with very high latency
|
||||
and extremely low bandwidth. Reticulum allows you to build wide-area networks
|
||||
with off-the-shelf tools, and offers end-to-end encryption and connectivity,
|
||||
initiator anonymity, autoconfiguring cryptographically backed multi-hop
|
||||
transport, efficient addressing, unforgeable delivery acknowledgements and
|
||||
more.
|
||||
To understand the foundational philosophy and goals of this system, read the [Zen of Reticulum](Zen%20of%20Reticulum.md).
|
||||
|
||||
The vision of Reticulum is to allow anyone to be their own network operator,
|
||||
and to make it cheap and easy to cover vast areas with a myriad of independent,
|
||||
inter-connectable and autonomous networks. Reticulum **is not** *one* network.
|
||||
It is **a tool** for building *thousands of networks*. Networks without
|
||||
kill-switches, surveillance, censorship and control. Networks that can freely
|
||||
interoperate, associate and disassociate with each other, and require no
|
||||
central oversight. Networks for human beings. *Networks for the people*.
|
||||
Reticulum is the cryptography-based networking stack for building local and wide-area networks with readily available hardware. It can operate even with very high latency and extremely low bandwidth. Reticulum allows you to build wide-area networks with off-the-shelf tools, and offers end-to-end encryption and connectivity, initiator anonymity, autoconfiguring cryptographically backed multi-hop transport, efficient addressing, unforgeable delivery acknowledgements and more.
|
||||
|
||||
Reticulum is a complete networking stack, and does not rely on IP or higher
|
||||
layers, but it is possible to use IP as the underlying carrier for Reticulum.
|
||||
It is therefore trivial to tunnel Reticulum over the Internet or private IP
|
||||
networks.
|
||||
The vision of Reticulum is to allow anyone to be their own network operator, and to make it cheap and easy to cover vast areas with a myriad of independent, inter-connectable and autonomous networks. Reticulum **is not** *one* network. It is **a tool** for building *thousands of networks*. Networks without kill-switches, surveillance, censorship and control. Networks that can freely interoperate, associate and disassociate with each other, and require no central oversight. Networks for human beings. *Networks for the people*.
|
||||
|
||||
Having no dependencies on traditional networking stacks frees up overhead that
|
||||
has been used to implement a networking stack built directly on cryptographic
|
||||
principles, allowing resilience and stable functionality, even in open and
|
||||
trustless networks.
|
||||
Reticulum is a complete networking stack, and does not rely on IP or higher layers, but it is possible to use IP as the underlying carrier for Reticulum. It is therefore trivial to tunnel Reticulum over the Internet or private IP networks.
|
||||
|
||||
No kernel modules or drivers are required. Reticulum runs completely in
|
||||
userland, and can run on practically any system that runs Python 3.
|
||||
Having no dependencies on traditional networking stacks frees up overhead that has been used to implement a networking stack built directly on cryptographic principles, allowing resilience and stable functionality, even in open and trustless networks.
|
||||
|
||||
No kernel modules or drivers are required. Reticulum runs completely in userland, and can run on practically any system that runs Python 3.
|
||||
|
||||
## Read The Manual
|
||||
The full documentation for Reticulum is available at [markqvist.github.io/Reticulum/manual/](https://markqvist.github.io/Reticulum/manual/).
|
||||
|
||||
You can also download the [Reticulum manual as a PDF](https://github.com/markqvist/Reticulum/raw/master/docs/Reticulum%20Manual.pdf) or [as an e-book in EPUB format](https://github.com/markqvist/Reticulum/raw/master/docs/Reticulum%20Manual.epub).
|
||||
You can also download the [Reticulum manual as a PDF](https://github.com/markqvist/Reticulum/releases/latest/download/Reticulum.Manual.pdf) or [as an e-book in EPUB format](https://github.com/markqvist/Reticulum/releases/latest/download/Reticulum.Manual.epub).
|
||||
|
||||
For more info, see [reticulum.network](https://reticulum.network/) and [the FAQ section of the wiki](https://github.com/markqvist/Reticulum/wiki/Frequently-Asked-Questions).
|
||||
|
||||
|
|
@ -76,66 +70,88 @@ For more info, see [reticulum.network](https://reticulum.network/) and [the FAQ
|
|||
- Low cost of keeping links open at only 0.44 bits per second
|
||||
- Reliable sequential delivery with Channel and Buffer mechanisms
|
||||
|
||||
## Roadmap
|
||||
While Reticulum is already a fully featured and functional networking stack,
|
||||
many improvements and additions are actively being worked on, and planned for the future.
|
||||
## Community Resources
|
||||
|
||||
To learn more about the direction and future of Reticulum, please see the [Development Roadmap](./Roadmap.md).
|
||||
The references listed here are not run, managed or moderated by the Reticulum project or its maintainer. They are provided by dedicated, independent volunteers in the community, for the benefit of anyone interested in learning more about Reticulum.
|
||||
|
||||
- [Reticulum Community Wiki](https://reticulum.miraheze.org/wiki)
|
||||
- [RNS Recipes](https://rns.recipes/)
|
||||
- [RNS Moscow](https://rns.moscow/)
|
||||
- [RMAP.world](https://rmap.world/)
|
||||
|
||||
If you find anything that is no longer accurate, incorrect or simply not quite up to date, do not demand or complain towards the maintainers of these resources. Instead, take the task upon yourself to help out and improve things by working on, and offering the contributions you would like to see, like the creators of the above did themselves.
|
||||
|
||||
## Reference Implementation
|
||||
|
||||
The Python code in this repository is the Reference Implementation of Reticulum. The Reticulum Protocol is defined entirely and authoritatively by this reference implementation, and its associated manual. It is maintained by Mark Qvist, identified by the Reticulum Identity `<bc7291552be7a58f361522990465165c>`.
|
||||
|
||||
No second-order protocol specification derivatives currently do, or will ever exist. You can't get a better specification than one that is executable, functional, testable, deployed and operating 24/7 across the planet.
|
||||
|
||||
There is not, and never will be a separate, derived "formal spec", "RFC" or anything similar for Reticulum:
|
||||
|
||||
**The reference implementation *is* the final, authoritative specification.**
|
||||
|
||||
Compatibility with the Reticulum Protocol is defined as having full interoperability, and sufficient functional parity with this reference implementation. Any specific protocol implementation that achieves this is Reticulum. Any that does not is not Reticulum.
|
||||
|
||||
The reference implementation is licensed under the Reticulum License.
|
||||
|
||||
The Reticulum Protocol was dedicated to the Public Domain in 2016.
|
||||
|
||||
## Community Implementations
|
||||
|
||||
The implementations listed here have demonstrated wire-compatibility, sufficient feature parity, sensible security and development practices, long-term commitment and care for implementation quality. Additionally, they have a track record of at least 18 months of active development.
|
||||
|
||||
Most importantly, their developers and maintainers are recognized by the community to actually know what they are doing, and as having a commitment to improving the overall Reticulum ecosystem.
|
||||
|
||||
- [Chad Atterman's](https://github.com/attermann) [microReticulum](https://github.com/attermann/microReticulum/) is a C++ implementation targeting 32-bit micro-controllers.
|
||||
- [Quad4's](https://quad4.io/) [Reticulum-Go](https://reticulum-go.quad4.io/) is a Go implementation targeting both desktop, mobile and embedded systems.
|
||||
|
||||
*If you are the maintainer of an implementation not listed above, and believe that it deserves inclusion by following the same standards of rigor and commitment, your proposal will be welcomed.*
|
||||
|
||||
---
|
||||
|
||||
**Warning!** A significant number of hastily launched, incorrect, dubious, directly harmful and, in most cases, entirely LLM-generated fakes of Reticulum implementations (and user facing applications) are currently being marketed and pushed under polished, but ultimately misleading or outright deceptive claims. Most of these are either quick attention or money grabs, completely unreviewed, incompatible, without maintenance or human oversight, actively trying to fragment the community, or a combination thereof.
|
||||
|
||||
---
|
||||
|
||||
For more extensive information on this topic, read the [Brandolini's Reference](https://reticulum.network/manual/brandolinis.html) chapter of the manual (for the legal foundations, see [Void Grants](https://reticulum.network/manual/brandolinis.html#void-grants-legal-foundations-of-machine-generated-code) specifically). The chapter is very long, very detailed, and is the polite, measured analysis, intended as an evolving, sectioned reference on these issues.
|
||||
|
||||
If you would rather just have the short, unfiltered and quite angry version, which consists of an enacted response to one particular manifestation of the phenomenon as a whole, used as an example, you can read [Yes, I am Angry](docs/history/2026_09_18_Yes_I_am_Angry.md). It gets the same point across, in a very different language, if you can handle that sort of communication.
|
||||
|
||||
The core Reticulum maintainers and developers **strongly advise against using any such "implementation"**. The [deceitful and self-serving behavior](https://reticulum.network/manual/brandolinis.html#a-persistent-but-fragile-triad) of a small number of either malintented, attention-seeking or incompetent individuals has caused significant [damage and waste of time and resources](https://reticulum.network/manual/brandolinis.html#intent-consequence) for both developers, node operators, users and the community and Reticulum project at large.
|
||||
|
||||
We kindly, but urgently ask all users to excercise caution, common sense and healthy skepticism when [evaluating](https://reticulum.network/manual/brandolinis.html#evaluating-a-reticulum-implementation) the potential use of any implementation not listed above; and to accept that **choosing to do so regardless translates the responsibility for potential damage from an irresponsible or malicious individual onto yourself**.
|
||||
|
||||
## Examples of Reticulum Applications
|
||||
If you want to quickly get an idea of what Reticulum can do, take a look at the
|
||||
following resources.
|
||||
If you want to quickly get an idea of what Reticulum can do, take a look at the [Programs Using Reticulum](https://reticulum.network/manual/software.html) section of the manual, or the following resources:
|
||||
|
||||
- You can use the [rnsh](https://github.com/acehoss/rnsh) program to establish remote shell sessions over Reticulum.
|
||||
- [LXMF](https://github.com/markqvist/lxmf) is a distributed, delay and disruption tolerant message transfer protocol built on Reticulum
|
||||
- The [LXST](https://github.com/markqvist/lxst) protocol and framework provides real-time audio and signals transport over Reticulum. It includes primitives and utilities for building voice-based applications and hardware devices, such as the `rnphone` program, that can be used to build hardware telephones.
|
||||
- For an off-grid, encrypted and resilient mesh communications platform, see [Nomad Network](https://github.com/markqvist/NomadNet)
|
||||
- The Android, Linux, macOS and Windows app [Sideband](https://github.com/markqvist/Sideband) has a graphical interface and many advanced features, such as file transfers, image and voice messages, real-time voice calls, a distributed telemetry system, mapping capabilities and full plugin extensibility.
|
||||
- [MeshChat](https://github.com/liamcottle/reticulum-meshchat) is a user-friendly LXMF client with a web-based interface, that also supports image and voice messages, as well as file transfers. It also includes a built-in page browser for browsing Nomad Network nodes.
|
||||
- [MeshChatX](https://git.quad4.io/RNS-Things/MeshChatX) is a full-featured LXMF client with many built-in tools and functionalities, that also supports image and voice messages, file transfers and voice calls. It also includes a built-in page browser for browsing Nomad Network nodes.
|
||||
- You can use the included [rnsh](https://reticulum.network/manual/using.html#the-rnsh-utility) program to establish remote shell sessions over Reticulum.
|
||||
- The included [rngit](https://reticulum.network/manual/git.html) system provides a full solution for managing, hosting, collaborating on and interacting with Git repositories over Reticulum.
|
||||
|
||||
## Where can Reticulum be used?
|
||||
Over practically any medium that can support at least a half-duplex channel
|
||||
with greater throughput than 5 bits per second, and an MTU of 500 bytes. Data radios,
|
||||
modems, LoRa radios, serial lines, AX.25 TNCs, amateur radio digital modes,
|
||||
WiFi and Ethernet devices, free-space optical links, and similar systems are
|
||||
all examples of the types of physical devices Reticulum can use.
|
||||
Over practically any medium that can support at least a half-duplex channel with greater throughput than 5 bits per second, and an MTU of 500 bytes. Data radios, modems, LoRa radios, serial lines, AX.25 TNCs, amateur radio digital modes, WiFi and Ethernet devices, free-space optical links, and similar systems are all examples of the types of physical devices Reticulum can use.
|
||||
|
||||
An open-source LoRa-based interface called
|
||||
[RNode](https://markqvist.github.io/Reticulum/manual/hardware.html#rnode) has
|
||||
been designed specifically for use with Reticulum. It is possible to build
|
||||
yourself, or it can be purchased as a complete transceiver that just needs a
|
||||
USB connection to the host.
|
||||
An open-source LoRa-based interface called [RNode](https://markqvist.github.io/Reticulum/manual/hardware.html#rnode) has been designed specifically for use with Reticulum. It is possible to build yourself, or it can be purchased as a complete transceiver that just needs a USB, WiFi or Bluetooth connection to the host.
|
||||
|
||||
Reticulum can also be encapsulated over existing IP networks, so there's
|
||||
nothing stopping you from using it over wired Ethernet, your local WiFi network
|
||||
or the Internet, where it'll work just as well. In fact, one of the strengths
|
||||
of Reticulum is how easily it allows you to connect different mediums into a
|
||||
self-configuring, resilient and encrypted mesh, using any available mixture of
|
||||
available infrastructure.
|
||||
Reticulum can also be encapsulated over existing IP networks, so there's nothing stopping you from using it over wired Ethernet, your local WiFi network or the Internet, where it'll work just as well. In fact, one of the strengths of Reticulum is how easily it allows you to connect different mediums into a self-configuring, resilient and encrypted mesh, using any available mixture of available infrastructure.
|
||||
|
||||
As an example, it's possible to set up a Raspberry Pi connected to both a LoRa
|
||||
radio, a packet radio TNC and a WiFi network. Once the interfaces are
|
||||
configured, Reticulum will take care of the rest, and any device on the WiFi
|
||||
network can communicate with nodes on the LoRa and packet radio sides of the
|
||||
network, and vice versa.
|
||||
As an example, it's possible to set up a Raspberry Pi connected to both a LoRa radio, a packet radio TNC and a WiFi network. Once the interfaces are configured, Reticulum will take care of the rest, and any device on the WiFi network can communicate with nodes on the LoRa and packet radio sides of the network, and vice versa.
|
||||
|
||||
## How do I get started?
|
||||
The best way to get started with the Reticulum Network Stack depends on what
|
||||
you want to do. For full details and examples, have a look at the
|
||||
[Getting Started Fast](https://markqvist.github.io/Reticulum/manual/gettingstartedfast.html)
|
||||
section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
|
||||
The best way to get started with the Reticulum Network Stack depends on what you want to do. For full details and examples, have a look at the [Getting Started Fast](https://markqvist.github.io/Reticulum/manual/gettingstartedfast.html) section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
|
||||
|
||||
To simply install Reticulum and related utilities on your system, the easiest way is via `pip`.
|
||||
You can then start any program that uses Reticulum, or start Reticulum as a system service with
|
||||
[the rnsd utility](https://markqvist.github.io/Reticulum/manual/using.html#the-rnsd-utility).
|
||||
To simply install Reticulum and related utilities on your system, the easiest way is via `pip`. You can then start any program that uses Reticulum, or start Reticulum as a system service with [the rnsd utility](https://markqvist.github.io/Reticulum/manual/using.html#the-rnsd-utility).
|
||||
|
||||
```bash
|
||||
pip install rns
|
||||
```
|
||||
|
||||
If you are using an operating system that blocks normal user package installation via `pip`,
|
||||
you can return `pip` to normal behaviour by editing the `~/.config/pip/pip.conf` file,
|
||||
and adding the following directive in the `[global]` section:
|
||||
If you are using an operating system that blocks normal user package installation via `pip`, you can return `pip` to normal behaviour by editing the `~/.config/pip/pip.conf` file, and adding the following directive in the `[global]` section:
|
||||
|
||||
```text
|
||||
[global]
|
||||
|
|
@ -148,23 +164,14 @@ Alternatively, you can use the `pipx` tool to install Reticulum in an isolated e
|
|||
pipx install rns
|
||||
```
|
||||
|
||||
When first started, Reticulum will create a default configuration file,
|
||||
providing basic connectivity to other Reticulum peers that might be locally
|
||||
reachable. The default config file contains a few examples, and references for
|
||||
creating a more complex configuration.
|
||||
When first started, Reticulum will create a default configuration file, providing basic connectivity to other Reticulum peers that might be locally reachable. The default config file contains a few examples, and references for creating a more complex configuration.
|
||||
|
||||
If you have an old version of `pip` on your system, you may need to upgrade it first with `pip install pip --upgrade`. If you no not already have `pip` installed, you can install it using the package manager of your system with `sudo apt install python3-pip` or similar.
|
||||
|
||||
For more detailed examples on how to expand communication over many mediums such
|
||||
as packet radio or LoRa, serial ports, or over fast IP links and the Internet using
|
||||
the UDP and TCP interfaces, take a look at the [Supported Interfaces](https://markqvist.github.io/Reticulum/manual/interfaces.html)
|
||||
section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
|
||||
For more detailed examples on how to expand communication over many mediums such as packet radio or LoRa, serial ports, or over fast IP links and the Internet using the UDP and TCP interfaces, take a look at the [Supported Interfaces](https://markqvist.github.io/Reticulum/manual/interfaces.html) section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
|
||||
|
||||
## Included Utilities
|
||||
Reticulum includes a range of useful utilities for managing your networks,
|
||||
viewing status and information, and other tasks. You can read more about these
|
||||
programs in the [Included Utility Programs](https://markqvist.github.io/Reticulum/manual/using.html#included-utility-programs)
|
||||
section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
|
||||
Reticulum includes a range of useful utilities for managing your networks, viewing status and information, and other tasks. You can read more about these programs in the [Included Utility Programs](https://markqvist.github.io/Reticulum/manual/using.html#included-utility-programs) section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
|
||||
|
||||
- The system daemon `rnsd` for running Reticulum as an always-available service
|
||||
- An interface status utility called `rnstatus`, that displays information about interfaces
|
||||
|
|
@ -172,22 +179,18 @@ section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/)
|
|||
- A diagnostics tool called `rnprobe` for checking connectivity to destinations
|
||||
- A simple file transfer program called `rncp` making it easy to transfer files between systems
|
||||
- The identity management and encryption utility `rnid` let's you manage Identities and encrypt/decrypt files
|
||||
- The remote command execution program `rnx` let's you run commands and
|
||||
programs and retrieve output from remote systems
|
||||
- The `rnsh` program allows you to establish fully interactive shell session with remote systems
|
||||
- The remote command execution program `rnx` let's you run simple commands and programs and retrieve output from remote systems
|
||||
- The `rngit` program provides a full multi-repository Git node for serving repositories over Reticulum
|
||||
- The included `git-remote-rns` helper allows you to interact with Git repositories over Reticulum
|
||||
|
||||
All tools, including `rnx` and `rncp`, work reliably and well even over very
|
||||
low-bandwidth links like LoRa or Packet Radio. For full-featured remote shells
|
||||
over Reticulum, also have a look at the [rnsh](https://github.com/acehoss/rnsh)
|
||||
program.
|
||||
All tools, including `rnx` and `rncp`, work reliably and well even over very low-bandwidth links like LoRa or Packet Radio. For full-featured remote shells over Reticulum, also have a look at the [rnsh](https://github.com/acehoss/rnsh) program.
|
||||
|
||||
## Supported interface types and devices
|
||||
|
||||
Reticulum implements a range of generalised interface types that covers most of
|
||||
the communications hardware that Reticulum can run over. If your hardware is
|
||||
not supported, it's [simple to implement a custom interface module](https://markqvist.github.io/Reticulum/manual/interfaces.html#custom-interfaces).
|
||||
Reticulum implements a range of generalised interface types that covers most of the communications hardware that Reticulum can run over. If your hardware is not supported, it's [simple to implement a custom interface module](https://markqvist.github.io/Reticulum/manual/interfaces.html#custom-interfaces).
|
||||
|
||||
Pull requests for custom interfaces are gratefully accepted, provided they are
|
||||
generally useful and well-tested in real-world usage.
|
||||
Contributions of custom interfaces are gratefully accepted, provided they are generally useful and well-tested in real-world usage.
|
||||
|
||||
Currently, the following built-in interfaces are supported:
|
||||
|
||||
|
|
@ -202,92 +205,48 @@ Currently, the following built-in interfaces are supported:
|
|||
- Custom hardware via stdio or pipes
|
||||
|
||||
## Performance
|
||||
Reticulum targets a *very* wide usable performance envelope, but prioritises
|
||||
functionality and performance on low-bandwidth mediums. The goal is to
|
||||
provide a dynamic performance envelope from 250 bits per second, to 1 gigabit
|
||||
per second on normal hardware.
|
||||
*All performance numbers here assume `rnsd` running in fully interpreted Python mode, limited to a single CPU core.*
|
||||
|
||||
Currently, the usable performance envelope is approximately 150 bits per second
|
||||
to 500 megabits per second, with physical mediums faster than that not being
|
||||
saturated. Performance beyond the current level is intended for future
|
||||
upgrades, but not highly prioritised at this point in time.
|
||||
Reticulum targets a *very* wide usable performance envelope, but prioritises functionality and performance on low-bandwidth mediums. The goal is to provide a dynamic performance envelope from 100 bits per second, to 10 gigabits per second on normal hardware.
|
||||
|
||||
Currently, the on-network, usable performance envelope is approximately 150 bits per second to 1 gigabit per second on modest hardware. Real-world performance will vary widely with the speed of your system, availability of cryptographic hardware acceleration, and other factors.
|
||||
|
||||
Raw routing throughput for a transport node hosted on a Raspberry Pi 5 is approximately 5 gigabits per second. Systems with modern crypto hardware acceleration can achieve transport throughput of 15 gigabits per second and higher.
|
||||
|
||||
While the reference implementation is perfectly capable of reaching 100+ Gbps, by utilizing multiple CPU cores and vectorized transport, performance beyond the current level is not higly prioritised, since RNS is already able to saturate available physical medium bandwidth in almost all real-world situations.
|
||||
|
||||
The primary objectives for RNS are *reliability*, *correctness*, *security*, *proper traffic management* and *actually practical performance*, not hollow on-paper "benchmarks" with little to no bearing on real-world usability.
|
||||
|
||||
All alternative-implementation maintainers are *highly* encouraged to adopt a similar approach, and especially to focus on getting their implementations working properly and reliably before publishing empirically dubious or factually wrong performance claims.
|
||||
|
||||
## Current Status
|
||||
All core protocol features are implemented and functioning, but additions will
|
||||
probably occur as real-world use is explored and understood. The API and wire-format
|
||||
can be considered stable.
|
||||
All core protocol features are implemented and functioning, but additions will probably occur as real-world use is explored and understood. The API and wire-format can be considered stable.
|
||||
|
||||
## Dependencies
|
||||
The installation of the default `rns` package requires the dependencies listed
|
||||
below. Almost all systems and distributions have readily available packages for
|
||||
these dependencies, and when the `rns` package is installed with `pip`, they
|
||||
will be downloaded and installed as well.
|
||||
The installation of the default `rns` package requires only two external dependencies, listed below. Almost all systems and distributions have readily available packages for these dependencies, and when the `rns` package is installed with `pip`, they will be downloaded and installed as well.
|
||||
|
||||
- [PyCA/cryptography](https://github.com/pyca/cryptography)
|
||||
- [pyserial](https://github.com/pyserial/pyserial)
|
||||
|
||||
On more unusual systems, and in some rare cases, it might not be possible to
|
||||
install or even compile one or more of the above modules. In such situations,
|
||||
you can use the `rnspure` package instead, which require no external
|
||||
dependencies for installation. Please note that the contents of the `rns` and
|
||||
`rnspure` packages are *identical*. The only difference is that the `rnspure`
|
||||
package lists no dependencies required for installation.
|
||||
On more unusual systems, and in some rare cases, it might not be possible to install or even compile one or more of the above modules. In such situations, you can use the `rnspure` package instead, which require no external dependencies for installation. Please note that the contents of the `rns` and `rnspure` packages are *identical*. The only difference is that the `rnspure` package lists no dependencies required for installation.
|
||||
|
||||
No matter how Reticulum is installed and started, it will load external
|
||||
dependencies only if they are *needed* and *available*. If for example you want
|
||||
to use Reticulum on a system that cannot support
|
||||
[pyserial](https://github.com/pyserial/pyserial), it is perfectly possible to
|
||||
do so using the `rnspure` package, but Reticulum will not be able to use
|
||||
serial-based interfaces. All other available modules will still be loaded when
|
||||
needed.
|
||||
No matter how Reticulum is installed and started, it will load external dependencies only if they are *needed* and *available*. If for example you want to use Reticulum on a system that cannot support [pyserial](https://github.com/pyserial/pyserial), it is perfectly possible to do so using the `rnspure` package, but Reticulum will not be able to use serial-based interfaces. All other available modules will still be loaded when needed.
|
||||
|
||||
**Please Note!** If you use the `rnspure` package to run Reticulum on systems
|
||||
that do not support [PyCA/cryptography](https://github.com/pyca/cryptography),
|
||||
it is important that you read and understand the [Cryptographic
|
||||
Primitives](#cryptographic-primitives) section of this document.
|
||||
**Please Note!** If you use the `rnspure` package to run Reticulum on systems that do not support [PyCA/cryptography](https://github.com/pyca/cryptography), it is important that you read and understand the [Cryptographic Primitives](#cryptographic-primitives) section of this document.
|
||||
|
||||
## Bootstrapping Connectivity
|
||||
|
||||
Reticulum is not a service you subscribe to, nor is it a single global network you "join". Reticulum provides functionality for discovering available public interfaces over the network itself, and the broader community has provided various directories of publicly available entrypoints to bootstrap connectivity.
|
||||
|
||||
To learn how to establish initial connectivity over Reticulum, read the [Bootstrapping Connectivity](https://reticulum.network/manual/gettingstartedfast.html#bootstrapping-connectivity) section of the manual.
|
||||
|
||||
If you already have a general idea of how this works, you can use community-run sites such as [directory.rns.recipes](https://directory.rns.recipes/) and [rmap.world](https://rmap.world) to find interface definitions for initial connectivity to the global distributed Reticulum backbone.
|
||||
|
||||
## Public Testnet
|
||||
***Important!** As the amount of global Reticulum nodes and entrypoints have grown to a substantial quantity, the public Amsterdam Testnet entrypoint is slated for de-commisioning in the first quarter of 2026. If your own instances rely on this entrypoint for connectivity, it is high time to start configuring alternatives. Reticulum now includes a full on-network interface discovery and connectivity bootstrapping system. Read the [Bootstrapping Connectivity](https://reticulum.network/manual/gettingstartedfast.html#bootstrapping-connectivity) section of the manual for pointers.*
|
||||
|
||||
If you just want to get started experimenting without building any physical
|
||||
networks, you are welcome to join the RNS Development Testnet.
|
||||
|
||||
The testnet is just that, an informal network for testing and experimenting.
|
||||
It will be up most of the time, and anyone can join, but it also means that
|
||||
there's no guarantees for service availability.
|
||||
|
||||
It probably goes without saying, but *don't use the testnet entry-points as
|
||||
hardcoded or default interfaces in any applications you ship to users*. When
|
||||
shipping applications, the best practice is to provide your own default
|
||||
connectivity solutions, if needed and applicable, or in most cases, simply
|
||||
leave it up to the user which networks to connect to, and how.
|
||||
|
||||
The testnet runs the very latest version of Reticulum (often even a short while
|
||||
before it is publicly released). Sometimes experimental versions of Reticulum
|
||||
might be deployed to nodes on the testnet, which means strange behaviour might
|
||||
occur. If none of that scares you, you can join the testnet via either TCP or
|
||||
I2P. Just add one of the following interfaces to your Reticulum configuration
|
||||
file:
|
||||
|
||||
```
|
||||
# TCP/IP interface to the RNS Amsterdam Hub
|
||||
[[RNS Testnet Amsterdam]]
|
||||
type = TCPClientInterface
|
||||
enabled = yes
|
||||
target_host = amsterdam.connect.reticulum.network
|
||||
target_port = 4965
|
||||
|
||||
# TCP/IP interface to the BetweenTheBorders Hub (community-provided)
|
||||
[[RNS Testnet BetweenTheBorders]]
|
||||
type = TCPClientInterface
|
||||
enabled = yes
|
||||
target_host = reticulum.betweentheborders.com
|
||||
target_port = 4242
|
||||
|
||||
```
|
||||
***Important!** Historically, a developer-targeted testnet was made available by the Reticulum project itself. As the amount of global Reticulum nodes and entrypoints have grown to a substantial quantity, this public testnet, including the Amsterdam Testnet entrypoint, has now been decommissioned. If your still have instances that relied on this entrypoint for connectivity, transition to using the distributed backbone instead. Reticulum now includes a full on-network interface discovery and connectivity bootstrapping system. Read the [Bootstrapping Connectivity](https://reticulum.network/manual/gettingstartedfast.html#bootstrapping-connectivity) section of the manual for pointers.*
|
||||
|
||||
## Support Reticulum
|
||||
You can help support the continued development of open, free and private communications systems by donating via one of the following channels:
|
||||
For this to be possible, I need your help. Please support the continued development of open, free and private communications systems by donating via one of the following channels:
|
||||
|
||||
- Monero:
|
||||
```
|
||||
|
|
@ -306,17 +265,11 @@ You can help support the continued development of open, free and private communi
|
|||
- Ko-Fi: https://ko-fi.com/markqvist
|
||||
|
||||
## Cryptographic Primitives
|
||||
Reticulum uses a simple suite of efficient, strong and well-tested cryptographic
|
||||
primitives, with widely available implementations that can be used both on
|
||||
general-purpose CPUs and on microcontrollers.
|
||||
Reticulum uses a simple suite of efficient, strong and well-tested cryptographic primitives, with widely available implementations that can be used both on general-purpose CPUs and on microcontrollers.
|
||||
|
||||
One of the primary considerations for choosing this particular set of primitives is
|
||||
that they can be implemented *safely* with relatively few pitfalls, on practically
|
||||
all current computing platforms.
|
||||
One of the primary considerations for choosing this particular set of primitives is that they can be implemented *safely* with relatively few pitfalls, on practically all current computing platforms.
|
||||
|
||||
The primitives listed here **are authoritative**. Anything claiming to be Reticulum,
|
||||
but not using these exact primitives **is not** Reticulum, and possibly an
|
||||
intentionally compromised or weakened clone. The utilised primitives are:
|
||||
The primitives listed here **are authoritative**. Anything claiming to be Reticulum, but not using these exact primitives **is not** Reticulum, and possibly an intentionally compromised or weakened clone. The utilised primitives are:
|
||||
|
||||
- Reticulum Identity Keys are 512-bit Curve25519 keysets
|
||||
- A 256-bit Ed25519 key for signatures
|
||||
|
|
@ -331,13 +284,7 @@ intentionally compromised or weakened clone. The utilised primitives are:
|
|||
- SHA-256
|
||||
- SHA-512
|
||||
|
||||
In the default installation configuration, the `X25519`, `Ed25519`,
|
||||
and `AES-256-CBC` primitives are provided by [OpenSSL](https://www.openssl.org/)
|
||||
(via the [PyCA/cryptography](https://github.com/pyca/cryptography) package).
|
||||
The hashing functions `SHA-256` and `SHA-512` are provided by the standard
|
||||
Python [hashlib](https://docs.python.org/3/library/hashlib.html). The `HKDF`,
|
||||
`HMAC`, `Token` primitives, and the `PKCS7` padding function are always
|
||||
provided by the following internal implementations:
|
||||
In the default installation configuration, the `X25519`, `Ed25519`, and `AES-256-CBC` primitives are provided by [OpenSSL](https://www.openssl.org/) (via the [PyCA/cryptography](https://github.com/pyca/cryptography) package). The hashing functions `SHA-256` and `SHA-512` are provided by the standard Python [hashlib](https://docs.python.org/3/library/hashlib.html). The `HKDF`, `HMAC`, `Token` primitives, and the `PKCS7` padding function are always provided by the following internal implementations:
|
||||
|
||||
- [HKDF.py](RNS/Cryptography/HKDF.py)
|
||||
- [HMAC.py](RNS/Cryptography/HMAC.py)
|
||||
|
|
@ -345,39 +292,18 @@ provided by the following internal implementations:
|
|||
- [PKCS7.py](RNS/Cryptography/PKCS7.py)
|
||||
|
||||
|
||||
Reticulum also includes a complete implementation of all necessary primitives
|
||||
in pure Python. If OpenSSL and PyCA are not available on the system when
|
||||
Reticulum is started, Reticulum will instead use the internal pure-python
|
||||
primitives. A trivial consequence of this is performance, with the OpenSSL
|
||||
backend being *much* faster. The most important consequence however, is the
|
||||
potential loss of security by using primitives that has not seen the same
|
||||
amount of scrutiny, testing and review as those from OpenSSL.
|
||||
Reticulum also includes a complete implementation of all necessary primitives in pure Python. If OpenSSL and PyCA are not available on the system when Reticulum is started, Reticulum will instead use the internal pure-python primitives. A trivial consequence of this is performance, with the OpenSSL backend being *much* faster. The most important consequence however, is the potential loss of security by using primitives that has not seen the same amount of scrutiny, testing and review as those from OpenSSL.
|
||||
|
||||
Please note that by default, installing Reticulum will **require** OpenSSL and
|
||||
PyCA to also be automatically installed if not already available. It is only
|
||||
possible to use the pure-python primitives if this requirement is specifically
|
||||
overridden by the user, for example by installing the `rnspure` package instead
|
||||
of the normal `rns` package, or by running directly from local source-code.
|
||||
Please note that by default, installing Reticulum will **require** OpenSSL and PyCA to also be automatically installed if not already available. It is only possible to use the pure-python primitives if this requirement is specifically overridden by the user, for example by installing the `rnspure` package instead of the normal `rns` package, or by running directly from local source-code.
|
||||
|
||||
If you want to use the internal pure-python primitives, it is **highly
|
||||
advisable** that you have a good understanding of the risks that this pose, and
|
||||
make an informed decision on whether those risks are acceptable to you.
|
||||
If you want to use the internal pure-python primitives, it is **highly advisable** that you have a good understanding of the risks that this pose, and make an informed decision on whether those risks are acceptable to you.
|
||||
|
||||
Reticulum is relatively young software, and should be considered as such. While
|
||||
it has been built with cryptography best-practices very foremost in mind, it
|
||||
_has not_ been externally security audited, and there could very well be
|
||||
privacy or security breaking bugs. If you want to help out, or help sponsor an
|
||||
audit, please do get in touch.
|
||||
Having been in development for ten, and wider deployment for around five years, Reticulum is relatively young software, and should be considered as such. While it has been carefully built with cryptography best-practices very foremost in mind, there could very well still be privacy or security breaking bugs.
|
||||
|
||||
## Acknowledgements & Credits
|
||||
Reticulum can only exist because of the mountain of Open Source work it was
|
||||
built on top of, the contributions of everyone involved, and everyone that has
|
||||
supported the project through the years. To everyone who has helped, thank you
|
||||
so much.
|
||||
Reticulum can only exist because of the mountain of Open Source work it was built on top of, the contributions of everyone involved, and everyone that has supported the project through the years. To everyone who has helped, thank you so much.
|
||||
|
||||
A number of other modules and projects are either part of, or used by
|
||||
Reticulum. Sincere thanks to the authors and contributors of the following
|
||||
projects:
|
||||
A number of other modules and projects are either part of, or used by Reticulum. Sincere thanks to the authors and contributors of the following projects:
|
||||
|
||||
- [PyCA/cryptography](https://github.com/pyca/cryptography), *BSD License*
|
||||
- [Pure-25519](https://github.com/warner/python-pure25519) by [Brian Warner](https://github.com/warner), *MIT License*
|
||||
|
|
@ -390,4 +316,5 @@ projects:
|
|||
- [Configobj](https://github.com/DiffSK/configobj) by Michael Foord, Nicola Larosa, Rob Dennis & Eli Courtwright, *BSD License*
|
||||
- [ifaddr](https://github.com/pydron/ifaddr) by Stefan C. Mueller, *MIT License*
|
||||
- [Umsgpack.py](https://github.com/vsergeev/u-msgpack-python) by [Ivan A. Sergeev](https://github.com/vsergeev)
|
||||
- [rnsh](https://github.com/acehoss/rnsh) by [Aaron Heise](https://github.com/acehoss)
|
||||
- [Python](https://www.python.org)
|
||||
|
|
|
|||
291
README.mu
Normal file
291
README.mu
Normal file
|
|
@ -0,0 +1,291 @@
|
|||
>> Reticulum Network Stack
|
||||
|
||||
To understand the foundational philosophy and goals of this system, read the `_`!`[Zen of Reticulum`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=Zen+of+Reticulum.md]`!`_.
|
||||
|
||||
Reticulum is the cryptography-based networking stack for building local and wide-area networks with readily available hardware. It can operate even with very high latency and extremely low bandwidth. Reticulum allows you to build wide-area networks with off-the-shelf tools, and offers end-to-end encryption and connectivity, initiator anonymity, autoconfiguring cryptographically backed multi-hop transport, efficient addressing, unforgeable delivery acknowledgements and more.
|
||||
|
||||
`(RNS Logo`w=18`a=c`:/media/reticulum/reticulum/HEAD/docs/source/graphics/rns_logo_256.webp)
|
||||
|
||||
The vision of Reticulum is to allow anyone to be their own network operator, and to make it cheap and easy to cover vast areas with a myriad of independent, inter-connectable and autonomous networks. Reticulum `!is not`! `*one`* network. It is `!a tool`! for building `*thousands of networks`*. Networks without kill-switches, surveillance, censorship and control. Networks that can freely interoperate, associate and disassociate with each other, and require no central oversight. Networks for human beings. `*Networks for the people`*.
|
||||
|
||||
Reticulum is a complete networking stack, and does not rely on IP or higher layers, but it is possible to use IP as the underlying carrier for Reticulum. It is therefore trivial to tunnel Reticulum over the Internet or private IP networks.
|
||||
|
||||
Having no dependencies on traditional networking stacks frees up overhead that has been used to implement a networking stack built directly on cryptographic principles, allowing resilience and stable functionality, even in open and trustless networks.
|
||||
|
||||
No kernel modules or drivers are required. Reticulum runs completely in userland, and can run on practically any system that runs Python.
|
||||
|
||||
>> Read The Manual
|
||||
|
||||
The full documentation for Reticulum is available on `_`!`[this node`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/index.md]`!`_.
|
||||
|
||||
You can also download the `_`!`[Reticulum manual as a PDF`:/file/artifact`g=reticulum|r=reticulum|t=latest|a=Reticulum+Manual.pdf]`!`_ or `_`!`[as an e-book in EPUB format`:/file/artifact`g=reticulum|r=reticulum|t=latest|a=Reticulum+Manual.epub]`!`_.
|
||||
|
||||
>> Notable Features
|
||||
|
||||
• Coordination-less globally unique addressing and identification
|
||||
• Fully self-configuring multi-hop routing over heterogeneous carriers
|
||||
• Flexible scalability over heterogeneous topologies
|
||||
• Reticulum can carry data over any mixture of physical mediums and topologies
|
||||
• Low-bandwidth networks can co-exist and interoperate with large, high-bandwidth networks
|
||||
• Initiator anonymity, communicate without revealing your identity
|
||||
• Reticulum does not include source addresses on any packets
|
||||
• Asymmetric X25519 encryption and Ed25519 signatures as a basis for all communication
|
||||
• The foundational Reticulum Identity Keys are 512-bit Elliptic Curve keysets
|
||||
• Forward Secrecy is available for all communication types, both for single packets and over links
|
||||
• Reticulum uses the following format for encrypted tokens:
|
||||
• Ephemeral per-packet and link keys and derived from an ECDH key exchange on Curve25519
|
||||
• AES-256 in CBC mode with PKCS7 padding
|
||||
• HMAC using SHA256 for authentication
|
||||
• IVs are generated through os.urandom()
|
||||
• Unforgeable packet delivery confirmations
|
||||
• Flexible and extensible interface system
|
||||
• Reticulum includes a large variety of built-in interface types
|
||||
• Ability to load and utilise custom user- or community-supplied interface types
|
||||
• Easily create your own custom interfaces for communicating over anything
|
||||
• Authentication and virtual network segmentation on all supported interface types
|
||||
• An intuitive and easy-to-use API
|
||||
• Simpler and easier to use than sockets APIs, but more powerful
|
||||
• Makes building distributed and decentralised applications much simpler
|
||||
• Reliable and efficient transfer of arbitrary amounts of data
|
||||
• Reticulum can handle a few bytes of data or files of many gigabytes
|
||||
• Sequencing, compression, transfer coordination and checksumming are automatic
|
||||
• The API is very easy to use, and provides transfer progress
|
||||
• Lightweight, flexible and expandable Request/Response mechanism
|
||||
• Efficient link establishment
|
||||
• Total cost of setting up an encrypted and verified link is only 3 packets, totalling 297 bytes
|
||||
• Low cost of keeping links open at only 0.44 bits per second
|
||||
• Reliable sequential delivery with Channel and Buffer mechanisms
|
||||
|
||||
>> Reference Implementation
|
||||
|
||||
The Python code in this repository is the Reference Implementation of Reticulum. The Reticulum Protocol is defined entirely and authoritatively by this reference implementation, and its associated manual. It is maintained by Mark Qvist, identified by the Reticulum Identity `B333<bc7291552be7a58f361522990465165c>`b.
|
||||
|
||||
No second-order protocol specification derivatives currently do, or will ever exist. You can't get a better specification than one that is executable, functional, testable, deployed and operating 24/7 across the planet.
|
||||
|
||||
There is not, and never will be a separate, derived "formal spec", "RFC" or anything similar for Reticulum:
|
||||
|
||||
`!The reference implementation `*is`* the final, authoritative specification.`!
|
||||
|
||||
Compatibility with the Reticulum Protocol is defined as having full interoperability, and sufficient functional parity with this reference implementation. Any specific protocol implementation that achieves this is Reticulum. Any that does not is not Reticulum.
|
||||
|
||||
The reference implementation is licensed under the Reticulum License.
|
||||
|
||||
The Reticulum Protocol was dedicated to the Public Domain in 2016.
|
||||
|
||||
>> Examples of Reticulum Applications
|
||||
|
||||
If you want to quickly get an idea of what Reticulum can do, take a look at the `_`!`[Programs Using Reticulum`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/software.md]`!`_ section of the manual, or the following resources:
|
||||
|
||||
• `_`!`[LXMF`a8d24177d946de4f1f0a0fe1af9a1338:/page/repo.mu`g=reticulum|r=lxmf]`!`_ is a distributed, delay and disruption tolerant message transfer protocol built on Reticulum
|
||||
|
||||
• The `_`!`[LXST`a8d24177d946de4f1f0a0fe1af9a1338:/page/repo.mu`g=reticulum|r=lxst]`!`_ protocol and framework provides real-time audio and signals transport over Reticulum. It
|
||||
includes primitives and utilities for building voice-based applications and hardware devices,
|
||||
such as the `B333rnphone`b program, that can be used to build hardware telephones.
|
||||
|
||||
• For an off-grid, encrypted and resilient mesh communications platform, see `_`!`[Nomad Network`a8d24177d946de4f1f0a0fe1af9a1338:/page/repo.mu`g=reticulum|r=nomadnet]`!`_.
|
||||
|
||||
• The Android, Linux, macOS and Windows app `_`!`[Sideband`a8d24177d946de4f1f0a0fe1af9a1338:/page/repo.mu`g=reticulum|r=sideband]`!`_ has a graphical interface and many advanced
|
||||
features, such as file transfers, image and voice messages, real-time voice calls, a distributed
|
||||
telemetry system, mapping capabilities and full plugin extensibility.
|
||||
|
||||
• `_`!`[MeshChatX`c10d80b1a42fa958c37a6cc30dc04f53]`!`_ (`_`!`[source`5399f5a0212477618821e91e88ce053b:/page/repo.mu`g=quad4|r=MeshChatX]`_`!) is a full-featured LXMF client with many built-in tools and functionalities,
|
||||
that also supports image and voice messages, file transfers and voice calls. It also includes a
|
||||
built-in page browser for browsing Nomad Network nodes.
|
||||
|
||||
• You can use the included `_`!`[rnsh`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/using.md|anchor=the-rnsh-utility]`!`_ program to establish remote shell sessions over Reticulum.
|
||||
|
||||
>> Where can Reticulum be used?
|
||||
|
||||
Over practically any medium that can support at least a half-duplex channel with greater throughput than 5 bits per second, and an MTU of 500 bytes. Data radios, modems, LoRa radios, serial lines, AX.25 TNCs, amateur radio digital modes, WiFi and Ethernet devices, free-space optical links, and similar systems are all examples of the types of physical devices Reticulum can use.
|
||||
|
||||
An open-source LoRa-based interface called `_`!`[RNode`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/hardware.md|anchor=rnode]`!`_ has been designed specifically for use with Reticulum. It is possible to build yourself, or it can be purchased as a complete transceiver that just needs a USB connection to the host.
|
||||
|
||||
Reticulum can also be encapsulated over existing IP networks, so there's nothing stopping you from using it over wired Ethernet, your local WiFi network or the Internet, where it'll work just as well. In fact, one of the strengths of Reticulum is how easily it allows you to connect different mediums into a self-configuring, resilient and encrypted mesh, using any available mixture of available infrastructure.
|
||||
|
||||
As an example, it's possible to set up a Raspberry Pi connected to both a LoRa radio, a packet radio TNC and a WiFi network. Once the interfaces are configured, Reticulum will take care of the rest, and any device on the WiFi network can communicate with nodes on the LoRa and packet radio sides of the network, and vice versa.
|
||||
|
||||
>> How do I get started?
|
||||
|
||||
The best way to get started with the Reticulum Network Stack depends on what you want to do. For full details and examples, have a look at the `_`!`[Getting Started Fast`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/gettingstartedfast.md]`!`_ section of the `_`!`[Reticulum Manual`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/index.md]`!`_.
|
||||
|
||||
To simply install Reticulum and related utilities on your system, the easiest way is via `B333pip`b. You can then start any program that uses Reticulum, or start Reticulum as a system service with `_`!`[the rnsd utility`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/using.md|anchor=the-rnsd-utility]`!`_.
|
||||
|
||||
`B333
|
||||
`=
|
||||
pip install rns
|
||||
`=
|
||||
`b
|
||||
|
||||
If you are using an operating system that blocks normal user package installation via `B333pip`b, you can return `B333pip`b to normal behaviour by editing the `B333~/.config/pip/pip.conf`b file, and adding the following directive in the `B333[global]`b section:
|
||||
|
||||
`B333
|
||||
`=
|
||||
[global]
|
||||
break-system-packages = true
|
||||
`=
|
||||
`b
|
||||
|
||||
Alternatively, you can use the `B333pipx`b tool to install Reticulum in an isolated environment:
|
||||
|
||||
`B333
|
||||
`=
|
||||
pipx install rns
|
||||
`=
|
||||
`b
|
||||
|
||||
When first started, Reticulum will create a default configuration file, providing basic connectivity to other Reticulum peers that might be locally reachable. The default config file contains a few examples, and references for creating a more complex configuration.
|
||||
|
||||
If you have an old version of `B333pip`b on your system, you may need to upgrade it first with `B333pip install pip --upgrade`b. If you no not already have `B333pip`b installed, you can install it using the package manager of your system with `B333sudo apt install python3-pip`b or similar.
|
||||
|
||||
For more detailed examples on how to expand communication over many mediums such as packet radio or LoRa, serial ports, or over fast IP links and the Internet using the UDP and TCP interfaces, take a look at the `_`!`[Supported Interfaces`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/interfaces.md]`!`_ section of the `_`!`[Reticulum Manual`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/index.md]`!`_.
|
||||
|
||||
>> Included Utilities
|
||||
|
||||
Reticulum includes a range of useful utilities for managing your networks, viewing status and information, and other tasks. You can read more about these programs in the `_`!`[Included Utility Programs`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/using.md|anchor=included-utility-programs]`!`_ section of the `_`!`[Reticulum Manual`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/index.md]`!`_.
|
||||
|
||||
• The system daemon `B333rnsd`b for running Reticulum as an always-available service
|
||||
• An interface status utility called `B333rnstatus`b, that displays information about interfaces
|
||||
• The path lookup and management tool `B333rnpath`b letting you view and modify path tables
|
||||
• A diagnostics tool called `B333rnprobe`b for checking connectivity to destinations
|
||||
• A simple file transfer program called `B333rncp`b making it easy to transfer files between systems
|
||||
• The identity management and encryption utility `B333rnid`b let's you manage Identities and encrypt/decrypt files
|
||||
• The `B333rnsh`b program allows you to establish fully interactive shell session with remote systems
|
||||
• The remote command execution program `B333rnx`b let's you run simple commands and programs and retrieve output from remote systems
|
||||
• The `B333rngit`b program provides a full multi-repository Git node for serving repositories over Reticulum
|
||||
• The included `B333git-remote-rns`b helper allows you to interact with Git repositories over Reticulum
|
||||
|
||||
>> Supported interface types and devices
|
||||
|
||||
Reticulum implements a range of generalised interface types that covers most of the communications hardware that Reticulum can run over. If your hardware is not supported, it's `_`!`[simple to implement a custom interface module`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/interfaces.md|anchor=custom-interfaces]`!`_.
|
||||
|
||||
Currently, the following built-in interfaces are supported:
|
||||
|
||||
• Any Ethernet device
|
||||
• LoRa using `_`!`[RNode`a8d24177d946de4f1f0a0fe1af9a1338:/page/repo.mu`g=reticulum|r=rnode_firmware]`!`_
|
||||
• Packet Radio TNCs (with or without AX.25)
|
||||
• KISS-compatible hardware and software modems
|
||||
• Any device with a serial port
|
||||
• TCP over IP networks
|
||||
• UDP over IP networks
|
||||
• External programs via stdio or pipes
|
||||
• Custom hardware via stdio or pipes
|
||||
|
||||
>> Performance
|
||||
|
||||
`*All performance numbers here assume `B333rnsd`b running in fully interpreted Python mode, limited to a single CPU core.`*
|
||||
|
||||
Reticulum targets a `*very`* wide usable performance envelope, but prioritises functionality and performance on low-bandwidth mediums. The goal is to provide a dynamic performance envelope from 100 bits per second, to 10 gigabits per second on normal hardware.
|
||||
|
||||
Currently, the on-network, usable performance envelope is approximately 150 bits per second to 1 gigabit per second on modest hardware. Real-world performance will vary widely with the speed of your system, availability of cryptographic hardware acceleration, and other factors.
|
||||
|
||||
Raw routing throughput for a transport node hosted on a Raspberry Pi 5 is approximately 4 gigabits per second. Systems with modern crypto hardware acceleration can achieve transport throughput of 15 gigabits per second and higher.
|
||||
|
||||
While the reference implementation is perfectly capable of reaching 100+ Gbps, by utilizing multiple CPU cores and vectorized transport, performance beyond the current level is not higly prioritised, since RNS is already able to saturate available physical medium bandwidth in almost all real-world situations.
|
||||
|
||||
The primary objectives for RNS are `*reliability`*, `*correctness`*, `*security`*, `*proper traffic management`* and `*actually practical performance`*, not hollow on-paper "benchmarks" with little to no bearing on real-world usability.
|
||||
|
||||
All alternative-implementation maintainers are `*highly`* encouraged to adopt a similar approach, and especially to focus on getting their implementations working properly before publishing empirically dubious or factually wrong performance claims.
|
||||
|
||||
>> Current Status
|
||||
|
||||
All core protocol features are implemented and functioning, but additions will probably occur as real-world use is explored and understood. The API and wire-format can be considered stable.
|
||||
|
||||
>> Dependencies
|
||||
|
||||
The installation of the default `B333rns`b package requires only two external dependencies, listed below. Almost all systems and distributions have readily available packages for these dependencies, and when the `B333rns`b package is installed with `B333pip`b, they will be downloaded and installed as well.
|
||||
|
||||
• PyCA/cryptography
|
||||
• pyserial
|
||||
|
||||
On more unusual systems, and in some rare cases, it might not be possible to install or even compile one or more of the above modules. In such situations, you can use the `B333rnspure`b package instead, which require no external dependencies for installation. Please note that the contents of the `B333rns`b and `B333rnspure`b packages are `*identical`*. The only difference is that the `B333rnspure`b package lists no dependencies required for installation.
|
||||
|
||||
No matter how Reticulum is installed and started, it will load external dependencies only if they are `*needed`* and `*available`*. If for example you want to use Reticulum on a system that cannot support `B333pyserial`b, it is perfectly possible to do so using the `B333rnspure`b package, but Reticulum will not be able to use serial-based interfaces. All other available modules will still be loaded when needed.
|
||||
|
||||
`!Please Note!`! If you use the `B333rnspure`b package to run Reticulum on systems that do not support PyCA/cryptography, it is important that you read and understand the `!Cryptographic Primitives`! section of this document.
|
||||
|
||||
>> Bootstrapping Connectivity
|
||||
|
||||
Reticulum is not a service you subscribe to, nor is it a single global network you "join". Reticulum provides functionality for discovering available public interfaces over the network itself, and the broader community has provided various directories of publicly available entrypoints to bootstrap connectivity.
|
||||
|
||||
To learn how to establish initial connectivity over Reticulum, read the `_`!`[Bootstrapping Connectivity`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/gettingstartedfast.md|anchor=bootstrapping-connectivity]`!`_ section of the manual.
|
||||
|
||||
If you already have a general idea of how this works, you can use community-run sites such as `_`!`[rns.recipes`9ce92808be498e9e05590ff27cbfdfe4]`!`_ and `_`!`[rmap.world`a4a5e861626ce97c9aa544d9ecdf6d22]`!`_ to find interface definitions for initial connectivity to the global distributed Reticulum backbone.
|
||||
|
||||
>> Public Testnet
|
||||
|
||||
`!`*Important!`! Historically, a developer-targeted testnet was made available by the Reticulum project itself. As the amount of global Reticulum nodes and entrypoints have grown to a substantial quantity, this public testnet, including the Amsterdam Testnet entrypoint, has now been decommissioned. If you still have instances that relied on this entrypoint for connectivity, transition to using the distributed backbone instead. Reticulum now includes a full on-network interface discovery and connectivity bootstrapping system. Read the `_`[Bootstrapping Connectivity`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/gettingstartedfast.md|anchor=bootstrapping-connectivity]`_ section of the manual for pointers.`*
|
||||
|
||||
>> Support Reticulum
|
||||
|
||||
For this to be possible, I need your help. Please support the continued development of open, free and private communications systems by donating via one of the following channels:
|
||||
|
||||
• `!Monero`!
|
||||
84FpY1QbxHcgdseePYNmhTHcrgMX4nFfBYtz2GKYToqHVVhJp8Eaw1Z1EedRnKD19b3B8NiLCGVxzKV17UMmmeEsCrPyA5w
|
||||
|
||||
• `!Bitcoin`!
|
||||
bc1pgqgu8h8xvj4jtafslq396v7ju7hkgymyrzyqft4llfslz5vp99psqfk3a6
|
||||
|
||||
• `!Ethereum`!
|
||||
0x91C421DdfB8a30a49A71d63447ddb54cEBe3465E
|
||||
|
||||
• `!Liberapay`!
|
||||
`[https://liberapay.com/Reticulum/]
|
||||
|
||||
• `!Ko-Fi`!
|
||||
`[https://ko-fi.com/markqvist]
|
||||
|
||||
>> Cryptographic Primitives
|
||||
|
||||
Reticulum uses a simple suite of efficient, strong and well-tested cryptographic primitives, with widely available implementations that can be used both on general-purpose CPUs and on microcontrollers.
|
||||
|
||||
One of the primary considerations for choosing this particular set of primitives is that they can be implemented `*safely`* with relatively few pitfalls, on practically all current computing platforms.
|
||||
|
||||
The primitives listed here `!are authoritative`!. Anything `*claiming`* to be Reticulum, but not using these exact primitives `FTA35050`!is not`!`f Reticulum, and possibly an intentionally compromised or weakened clone. The utilised primitives are:
|
||||
|
||||
• Reticulum Identity Keys are 512-bit Curve25519 keysets
|
||||
• A 256-bit Ed25519 key for signatures
|
||||
• A 256-bit X22519 key for ECDH key exchanges
|
||||
• HKDF for key derivation
|
||||
• Encrypted tokens are based on the `_`!`[Fernet spec`https://github.com/fernet/spec/]`!`_
|
||||
• Ephemeral keys derived from an ECDH key exchange on Curve25519
|
||||
• HMAC using SHA256 for message authentication
|
||||
• IVs must be generated through `B333os.urandom()`b or better
|
||||
• AES-256 in CBC mode with PKCS7 padding
|
||||
• No Fernet version and timestamp metadata fields
|
||||
• SHA-256
|
||||
• SHA-512
|
||||
|
||||
In the default installation configuration, the `B333X25519`b, `B333Ed25519`b, and `B333AES-256-CBC`b primitives are provided by `_`!`[OpenSSL`https://www.openssl.org/]`!`_ (via the `_`!`[PyCA/cryptography`https://github.com/pyca/cryptography]`!`_ package). The hashing functions `B333SHA-256`b and `B333SHA-512`b are provided by the standard Python `_`!`[hashlib`https://docs.python.org/3/library/hashlib.html]`!`_. The `B333HKDF`b, `B333HMAC`b, `B333Token`b primitives, and the `B333PKCS7`b padding function are always provided by the following internal implementations:
|
||||
|
||||
• `_`!`[HKDF.py`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=RNS/Cryptography/HKDF.py]`!`_
|
||||
• `_`!`[HMAC.py`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=RNS/Cryptography/HMAC.py]`!`_
|
||||
• `_`!`[Token.py`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=RNS/Cryptography/Token.py]`!`_
|
||||
• `_`!`[PKCS7.py`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=RNS/Cryptography/PKCS7.py]`!`_
|
||||
|
||||
Reticulum also includes a complete implementation of all necessary primitives in pure Python. If OpenSSL and PyCA are not available on the system when Reticulum is started, Reticulum will instead use the internal pure-python primitives. A trivial consequence of this is performance, with the OpenSSL backend being `*much`* faster. The most important consequence however, is the potential loss of security by using primitives that has not seen the same amount of scrutiny, testing and review as those from OpenSSL.
|
||||
|
||||
Please note that by default, installing Reticulum will `!require`! OpenSSL and PyCA to also be automatically installed if not already available. It is only possible to use the pure-python primitives if this requirement is specifically overridden by the user, for example by installing the `B333rnspure`b package instead of the normal `B333rns`b package, or by running directly from local source-code.
|
||||
|
||||
If you want to use the internal pure-python primitives, it is `!highly advisable`! that you have a good understanding of the risks that this pose, and make an informed decision on whether those risks are acceptable to you.
|
||||
|
||||
Having been in development for ten, and wider deployment for around five years, Reticulum is relatively young software, and should be considered as such. While it has been carefully built with cryptography best-practices very foremost in mind, there could very well still be privacy or security breaking bugs.
|
||||
|
||||
>> Acknowledgements & Credits
|
||||
|
||||
Reticulum can only exist because of the mountain of Open Source work it was built on top of, the contributions of everyone involved, and everyone that has supported the project through the years. To everyone who has helped, thank you so much.
|
||||
|
||||
A number of other modules and projects are either part of, or used by Reticulum. Sincere thanks to the authors and contributors of the following projects:
|
||||
|
||||
• `_`!`[PyCA/cryptography`https://github.com/pyca/cryptography]`!`_, `*BSD License`*
|
||||
• `_`!`[Pure-25519`https://github.com/warner/python-pure25519]`!`_, by `_`!`[Brian Warner`https://github.com/warner]`!`_, `*MIT License`*
|
||||
• `_`!`[Pysha2`https://github.com/thomdixon/pysha2]`!`_ by `_`!`[Thom Dixon`https://github.com/thomdixon]`!`_, `*MIT License`*
|
||||
• `_`!`[Python AES-128`https://github.com/orgurar/python-aes]`!`_ by `_`!`[Or Gur Arie`https://github.com/orgurar]`!`_, `*MIT License`*
|
||||
• `_`!`[Python AES-256`https://github.com/boppreh/aes]`!`_ by `_`!`[BoppreH`https://github.com/boppreh]`!`_, `*MIT License`*
|
||||
• `_`!`[Curve25519.py`https://gist.github.com/nickovs/cc3c22d15f239a2640c185035c06f8a3]`!`_ by `_`!`[Nicko van Someren`https://gist.github.com/nickovs]`!`_, `*Public Domain`*
|
||||
• `_`!`[I2Plib`https://github.com/l-n-s/i2plib]`!`_ by `_`!`[Viktor Villainov`https://github.com/l-n-s]`!`_
|
||||
• `_`!`[PySerial`https://github.com/pyserial/pyserial]`!`_ by Chris Liechti, `*BSD License`*
|
||||
• `_`!`[Configobj`https://github.com/DiffSK/configobj]`!`_ by Michael Foord, Nicola Larosa, Rob Dennis & Eli Courtwright, `*BSD License`*
|
||||
• `_`!`[ifaddr`https://github.com/pydron/ifaddr]`!`_ by Stefan C. Mueller, `*MIT License`*
|
||||
• `_`!`[Umsgpack.py`https://github.com/vsergeev/u-msgpack-python]`!`_ by `_`!`[Ivan A. Sergeev`https://github.com/vsergeev]`!`_
|
||||
• `_`!`[rnsh`https://github.com/acehoss/rnsh]`!`_ by `_`!`[Aaron Heise`https://github.com/acehoss]`!`_
|
||||
• `_`!`[Python`https://www.python.org]`!`_
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -53,7 +53,8 @@ class StreamDataMessage(MessageBase):
|
|||
The stream id is limited to 2 bytes - 2 bit
|
||||
"""
|
||||
|
||||
OVERHEAD = 2 + 6 # 2 for stream data message header, 6 for channel envelope
|
||||
HEADER_LEN = 2 # 2 for stream data message header
|
||||
OVERHEAD = HEADER_LEN + 6 # 6 for channel envelope
|
||||
MAX_DATA_LEN = RNS.Link.MDU - OVERHEAD
|
||||
"""
|
||||
When the Buffer package is imported, this value is
|
||||
|
|
@ -92,7 +93,9 @@ class StreamDataMessage(MessageBase):
|
|||
self.data = raw[2:]
|
||||
|
||||
if self.compressed:
|
||||
self.data = bz2.decompress(self.data)
|
||||
decompressor = bz2.BZ2Decompressor()
|
||||
self.data = decompressor.decompress(self.data, max_length=RawChannelWriter.MAX_CHUNK_LEN)
|
||||
if not decompressor.eof: raise IOError("Decompressed buffer chunk exceeds maximum legitimate size")
|
||||
|
||||
|
||||
class RawChannelReader(RawIOBase, AbstractContextManager):
|
||||
|
|
@ -224,7 +227,7 @@ class RawChannelWriter(RawIOBase, AbstractContextManager):
|
|||
self._stream_id = stream_id
|
||||
self._channel = channel
|
||||
self._eof = False
|
||||
self._mdu = channel.mdu - StreamDataMessage.OVERHEAD
|
||||
self._mdu = channel.mdu - StreamDataMessage.HEADER_LEN
|
||||
|
||||
def write(self, __b: bytes) -> int | None:
|
||||
try:
|
||||
|
|
@ -240,7 +243,7 @@ class RawChannelWriter(RawIOBase, AbstractContextManager):
|
|||
chunk_segment_length = int(chunk_len/comp_try)
|
||||
compressed_chunk = bz2.compress(__b[:chunk_segment_length])
|
||||
compressed_length = len(compressed_chunk)
|
||||
if compressed_length < StreamDataMessage.MAX_DATA_LEN and compressed_length < chunk_segment_length:
|
||||
if compressed_length < self._mdu and compressed_length < chunk_segment_length:
|
||||
comp_success = True
|
||||
break
|
||||
else:
|
||||
|
|
@ -250,7 +253,7 @@ class RawChannelWriter(RawIOBase, AbstractContextManager):
|
|||
chunk = compressed_chunk
|
||||
processed_length = chunk_segment_length
|
||||
else:
|
||||
chunk = bytes(__b[:StreamDataMessage.MAX_DATA_LEN])
|
||||
chunk = bytes(__b[:self._mdu])
|
||||
processed_length = len(chunk)
|
||||
|
||||
message = StreamDataMessage(self._stream_id, chunk, self._eof, comp_success)
|
||||
|
|
|
|||
358
RNS/Channel.py
358
RNS/Channel.py
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -45,63 +45,6 @@ TPacket = TypeVar("TPacket")
|
|||
class SystemMessageTypes(enum.IntEnum):
|
||||
SMT_STREAM_DATA = 0xff00
|
||||
|
||||
class ChannelOutletBase(ABC, Generic[TPacket]):
|
||||
"""
|
||||
An abstract transport layer interface used by Channel.
|
||||
|
||||
DEPRECATED: This was created for testing; eventually
|
||||
Channel will use Link or a LinkBase interface
|
||||
directly.
|
||||
"""
|
||||
@abstractmethod
|
||||
def send(self, raw: bytes) -> TPacket:
|
||||
raise NotImplemented()
|
||||
|
||||
@abstractmethod
|
||||
def resend(self, packet: TPacket) -> TPacket:
|
||||
raise NotImplemented()
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def mdu(self):
|
||||
raise NotImplemented()
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def rtt(self):
|
||||
raise NotImplemented()
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def is_usable(self):
|
||||
raise NotImplemented()
|
||||
|
||||
@abstractmethod
|
||||
def get_packet_state(self, packet: TPacket) -> MessageState:
|
||||
raise NotImplemented()
|
||||
|
||||
@abstractmethod
|
||||
def timed_out(self):
|
||||
raise NotImplemented()
|
||||
|
||||
@abstractmethod
|
||||
def __str__(self):
|
||||
raise NotImplemented()
|
||||
|
||||
@abstractmethod
|
||||
def set_packet_timeout_callback(self, packet: TPacket, callback: Callable[[TPacket], None] | None,
|
||||
timeout: float | None = None):
|
||||
raise NotImplemented()
|
||||
|
||||
@abstractmethod
|
||||
def set_packet_delivered_callback(self, packet: TPacket, callback: Callable[[TPacket], None] | None):
|
||||
raise NotImplemented()
|
||||
|
||||
@abstractmethod
|
||||
def get_packet_id(self, packet: TPacket) -> any:
|
||||
raise NotImplemented()
|
||||
|
||||
|
||||
class CEType(enum.IntEnum):
|
||||
"""
|
||||
ChannelException type codes
|
||||
|
|
@ -113,7 +56,6 @@ class CEType(enum.IntEnum):
|
|||
ME_ALREADY_SENT = 4
|
||||
ME_TOO_BIG = 5
|
||||
|
||||
|
||||
class ChannelException(Exception):
|
||||
"""
|
||||
An exception thrown by Channel, with a type code.
|
||||
|
|
@ -122,7 +64,6 @@ class ChannelException(Exception):
|
|||
super().__init__(args)
|
||||
self.type = ce_type
|
||||
|
||||
|
||||
class MessageState(enum.IntEnum):
|
||||
"""
|
||||
Set of possible states for a Message
|
||||
|
|
@ -132,7 +73,6 @@ class MessageState(enum.IntEnum):
|
|||
MSGSTATE_DELIVERED = 2
|
||||
MSGSTATE_FAILED = 3
|
||||
|
||||
|
||||
class MessageBase(abc.ABC):
|
||||
"""
|
||||
Base type for any messages sent or received on a Channel.
|
||||
|
|
@ -144,7 +84,7 @@ class MessageBase(abc.ABC):
|
|||
MSGTYPE = None
|
||||
"""
|
||||
Defines a unique identifier for a message class.
|
||||
|
||||
|
||||
* Must be unique within all classes registered with a ``Channel``
|
||||
* Must be less than ``0xf000``. Values greater than or equal to ``0xf000`` are reserved.
|
||||
"""
|
||||
|
|
@ -170,7 +110,6 @@ class MessageBase(abc.ABC):
|
|||
|
||||
MessageCallbackType = NewType("MessageCallbackType", Callable[[MessageBase], bool])
|
||||
|
||||
|
||||
class Envelope:
|
||||
"""
|
||||
Internal wrapper used to transport messages over a channel and
|
||||
|
|
@ -180,8 +119,7 @@ class Envelope:
|
|||
msgtype, self.sequence, length = struct.unpack(">HHH", self.raw[:6])
|
||||
raw = self.raw[6:]
|
||||
ctor = message_factories.get(msgtype, None)
|
||||
if ctor is None:
|
||||
raise ChannelException(CEType.ME_NOT_REGISTERED, f"Unable to find constructor for Channel MSGTYPE {hex(msgtype)}")
|
||||
if ctor is None: raise ChannelException(CEType.ME_NOT_REGISTERED, f"Unable to find constructor for Channel MSGTYPE {hex(msgtype)}")
|
||||
message = ctor()
|
||||
message.unpack(raw)
|
||||
self.unpacked = True
|
||||
|
|
@ -190,14 +128,13 @@ class Envelope:
|
|||
return message
|
||||
|
||||
def pack(self) -> bytes:
|
||||
if self.message.__class__.MSGTYPE is None:
|
||||
raise ChannelException(CEType.ME_NO_MSG_TYPE, f"{self.message.__class__} lacks MSGTYPE")
|
||||
if self.message.__class__.MSGTYPE is None: raise ChannelException(CEType.ME_NO_MSG_TYPE, f"{self.message.__class__} lacks MSGTYPE")
|
||||
data = self.message.pack()
|
||||
self.raw = struct.pack(">HHH", self.message.MSGTYPE, self.sequence, len(data)) + data
|
||||
self.packed = True
|
||||
return self.raw
|
||||
|
||||
def __init__(self, outlet: ChannelOutletBase, message: MessageBase = None, raw: bytes = None, sequence: int = None):
|
||||
def __init__(self, outlet: LinkChannelOutlet, message: MessageBase = None, raw: bytes = None, sequence: int = None):
|
||||
self.ts = time.time()
|
||||
self.id = id(self)
|
||||
self.message = message
|
||||
|
|
@ -255,11 +192,11 @@ class Channel(contextlib.AbstractContextManager):
|
|||
|
||||
# The maximum window size for transfers on fast links
|
||||
WINDOW_MAX_FAST = 48
|
||||
|
||||
|
||||
# For calculating maps and guard segments, this
|
||||
# must be set to the global maximum window.
|
||||
WINDOW_MAX = WINDOW_MAX_FAST
|
||||
|
||||
|
||||
# If the fast rate is sustained for this many request
|
||||
# rounds, the fast link window size will be allowed.
|
||||
FAST_RATE_THRESHOLD = 10
|
||||
|
|
@ -278,13 +215,14 @@ class Channel(contextlib.AbstractContextManager):
|
|||
SEQ_MAX = 0xFFFF
|
||||
SEQ_MODULUS = SEQ_MAX+1
|
||||
|
||||
def __init__(self, outlet: ChannelOutletBase):
|
||||
def __init__(self, outlet: LinkChannelOutlet):
|
||||
"""
|
||||
|
||||
@param outlet:
|
||||
"""
|
||||
self._outlet = outlet
|
||||
self._lock = threading.RLock()
|
||||
self._send_lock = threading.Lock()
|
||||
self._tx_ring: collections.deque[Envelope] = collections.deque()
|
||||
self._rx_ring: collections.deque[Envelope] = collections.deque()
|
||||
self._message_callbacks: [MessageCallbackType] = []
|
||||
|
|
@ -306,11 +244,9 @@ class Channel(contextlib.AbstractContextManager):
|
|||
self.window_min = Channel.WINDOW_MIN
|
||||
self.window_flexibility = Channel.WINDOW_FLEXIBILITY
|
||||
|
||||
def __enter__(self) -> Channel:
|
||||
return self
|
||||
def __enter__(self) -> Channel: return self
|
||||
|
||||
def __exit__(self, __exc_type: Type[BaseException] | None, __exc_value: BaseException | None,
|
||||
__traceback: TracebackType | None) -> bool | None:
|
||||
def __exit__(self, __exc_type: Type[BaseException] | None, __exc_value: BaseException | None, __traceback: TracebackType | None) -> bool | None:
|
||||
self._shutdown()
|
||||
return False
|
||||
|
||||
|
|
@ -326,20 +262,13 @@ class Channel(contextlib.AbstractContextManager):
|
|||
|
||||
def _register_message_type(self, message_class: Type[MessageBase], *, is_system_type: bool = False):
|
||||
with self._lock:
|
||||
if not issubclass(message_class, MessageBase):
|
||||
raise ChannelException(CEType.ME_INVALID_MSG_TYPE,
|
||||
f"{message_class} is not a subclass of {MessageBase}.")
|
||||
if message_class.MSGTYPE is None:
|
||||
raise ChannelException(CEType.ME_INVALID_MSG_TYPE,
|
||||
f"{message_class} has invalid MSGTYPE class attribute.")
|
||||
if message_class.MSGTYPE >= 0xf000 and not is_system_type:
|
||||
raise ChannelException(CEType.ME_INVALID_MSG_TYPE,
|
||||
f"{message_class} has system-reserved message type.")
|
||||
try:
|
||||
message_class()
|
||||
if not issubclass(message_class, MessageBase): raise ChannelException(CEType.ME_INVALID_MSG_TYPE, f"{message_class} is not a subclass of {MessageBase}.")
|
||||
if message_class.MSGTYPE is None: raise ChannelException(CEType.ME_INVALID_MSG_TYPE, f"{message_class} has invalid MSGTYPE class attribute.")
|
||||
if message_class.MSGTYPE >= 0xf000 and not is_system_type: raise ChannelException(CEType.ME_INVALID_MSG_TYPE, f"{message_class} has system-reserved message type.")
|
||||
|
||||
try: message_class()
|
||||
except Exception as ex:
|
||||
raise ChannelException(CEType.ME_INVALID_MSG_TYPE,
|
||||
f"{message_class} raised an exception when constructed with no arguments: {ex}")
|
||||
raise ChannelException(CEType.ME_INVALID_MSG_TYPE, f"{message_class} raised an exception when constructed with no arguments: {ex}")
|
||||
|
||||
self._message_factories[message_class.MSGTYPE] = message_class
|
||||
|
||||
|
|
@ -382,27 +311,30 @@ class Channel(contextlib.AbstractContextManager):
|
|||
if envelope.packet is not None:
|
||||
self._outlet.set_packet_timeout_callback(envelope.packet, None)
|
||||
self._outlet.set_packet_delivered_callback(envelope.packet, None)
|
||||
envelope.tracked = False
|
||||
|
||||
for envelope in self._rx_ring:
|
||||
envelope.tracked = False
|
||||
|
||||
self._tx_ring.clear()
|
||||
self._rx_ring.clear()
|
||||
|
||||
def _emplace_envelope(self, envelope: Envelope, ring: collections.deque[Envelope]) -> bool:
|
||||
with self._lock:
|
||||
i = 0
|
||||
|
||||
for existing in ring:
|
||||
|
||||
for existing in ring:
|
||||
if envelope.sequence == existing.sequence:
|
||||
RNS.log(f"Envelope: Emplacement of duplicate envelope with sequence "+str(envelope.sequence), RNS.LOG_EXTREME)
|
||||
RNS.log(f"Envelope: Emplacement of duplicate envelope with sequence "+str(envelope.sequence), RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
|
||||
return False
|
||||
|
||||
|
||||
if envelope.sequence < existing.sequence and not (self._next_rx_sequence - envelope.sequence) > (Channel.SEQ_MAX//2):
|
||||
ring.insert(i, envelope)
|
||||
|
||||
envelope.tracked = True
|
||||
return True
|
||||
|
||||
|
||||
i += 1
|
||||
|
||||
|
||||
envelope.tracked = True
|
||||
ring.append(envelope)
|
||||
|
||||
|
|
@ -413,10 +345,8 @@ class Channel(contextlib.AbstractContextManager):
|
|||
|
||||
for cb in cbs:
|
||||
try:
|
||||
if cb(message):
|
||||
return
|
||||
except Exception as e:
|
||||
RNS.log("Channel "+str(self)+" experienced an error while running a message callback. The contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
if cb(message): return
|
||||
except Exception as e: RNS.log(f"Channel {str(self)} experienced an error while running a message callback. The contained exception was: {e}", RNS.LOG_ERROR)
|
||||
|
||||
def _receive(self, raw: bytes):
|
||||
try:
|
||||
|
|
@ -428,16 +358,20 @@ class Channel(contextlib.AbstractContextManager):
|
|||
window_overflow = (self._next_rx_sequence+Channel.WINDOW_MAX) % Channel.SEQ_MODULUS
|
||||
if window_overflow < self._next_rx_sequence:
|
||||
if envelope.sequence > window_overflow:
|
||||
RNS.log("Invalid packet sequence ("+str(envelope.sequence)+") received on channel "+str(self), RNS.LOG_EXTREME)
|
||||
RNS.log(f"Invalid packet sequence ({envelope.sequence}) received on channel {self}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
return
|
||||
else:
|
||||
RNS.log("Invalid packet sequence ("+str(envelope.sequence)+") received on channel "+str(self), RNS.LOG_EXTREME)
|
||||
RNS.log(f"Invalid packet sequence ({envelope.sequence}) received on channel {self}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
return
|
||||
|
||||
elif envelope.sequence > self._next_rx_sequence + self.WINDOW_MAX:
|
||||
RNS.log(f"Invalid packet sequence ({envelope.sequence}) received on channel {self}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
return
|
||||
|
||||
is_new = self._emplace_envelope(envelope, self._rx_ring)
|
||||
|
||||
if not is_new:
|
||||
RNS.log("Duplicate message received on channel "+str(self), RNS.LOG_EXTREME)
|
||||
RNS.log("Duplicate message received on channel "+str(self), RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
|
||||
return
|
||||
else:
|
||||
with self._lock:
|
||||
|
|
@ -453,11 +387,9 @@ class Channel(contextlib.AbstractContextManager):
|
|||
self._next_rx_sequence = (self._next_rx_sequence + 1) % Channel.SEQ_MODULUS
|
||||
|
||||
for e in contigous:
|
||||
if not e.unpacked:
|
||||
m = e.unpack(self._message_factories)
|
||||
else:
|
||||
m = e.message
|
||||
|
||||
if not e.unpacked: m = e.unpack(self._message_factories)
|
||||
else: m = e.message
|
||||
|
||||
self._rx_ring.remove(e)
|
||||
self._run_callbacks(m)
|
||||
|
||||
|
|
@ -470,67 +402,46 @@ class Channel(contextlib.AbstractContextManager):
|
|||
|
||||
:return: True if ready
|
||||
"""
|
||||
if not self._outlet.is_usable:
|
||||
return False
|
||||
|
||||
if not self._outlet.is_usable: return False
|
||||
with self._lock:
|
||||
outstanding = 0
|
||||
for envelope in self._tx_ring:
|
||||
if envelope.outlet == self._outlet:
|
||||
if envelope.outlet == self._outlet:
|
||||
if not envelope.packet or not self._outlet.get_packet_state(envelope.packet) == MessageState.MSGSTATE_DELIVERED:
|
||||
outstanding += 1
|
||||
|
||||
if outstanding >= self.window:
|
||||
return False
|
||||
if outstanding >= self.window: return False
|
||||
|
||||
return True
|
||||
|
||||
def _packet_tx_op(self, packet: TPacket, op: Callable[[TPacket], bool]):
|
||||
target_id = self._outlet.get_packet_id(packet)
|
||||
with self._lock:
|
||||
envelope = next(filter(lambda e: self._outlet.get_packet_id(e.packet) == self._outlet.get_packet_id(packet),
|
||||
self._tx_ring), None)
|
||||
|
||||
envelope = next(filter(lambda e: e.packet is not None and self._outlet.get_packet_id(e.packet) == target_id, self._tx_ring), None)
|
||||
if envelope and op(envelope):
|
||||
envelope.tracked = False
|
||||
if envelope in self._tx_ring:
|
||||
self._tx_ring.remove(envelope)
|
||||
|
||||
if self.window < self.window_max:
|
||||
self.window += 1
|
||||
|
||||
# TODO: Remove at some point
|
||||
# RNS.log("Increased "+str(self)+" window to "+str(self.window), RNS.LOG_DEBUG)
|
||||
if self.window < self.window_max: self.window += 1
|
||||
|
||||
if self._outlet.rtt != 0:
|
||||
if self._outlet.rtt > Channel.RTT_FAST:
|
||||
self.fast_rate_rounds = 0
|
||||
|
||||
if self._outlet.rtt > Channel.RTT_MEDIUM:
|
||||
self.medium_rate_rounds = 0
|
||||
|
||||
if self._outlet.rtt > Channel.RTT_MEDIUM: self.medium_rate_rounds = 0
|
||||
else:
|
||||
self.medium_rate_rounds += 1
|
||||
if self.window_max < Channel.WINDOW_MAX_MEDIUM and self.medium_rate_rounds == Channel.FAST_RATE_THRESHOLD:
|
||||
self.window_max = Channel.WINDOW_MAX_MEDIUM
|
||||
self.window_min = Channel.WINDOW_MIN_LIMIT_MEDIUM
|
||||
# TODO: Remove at some point
|
||||
# RNS.log("Increased "+str(self)+" max window to "+str(self.window_max), RNS.LOG_DEBUG)
|
||||
# RNS.log("Increased "+str(self)+" min window to "+str(self.window_min), RNS.LOG_DEBUG)
|
||||
|
||||
else:
|
||||
self.fast_rate_rounds += 1
|
||||
if self.window_max < Channel.WINDOW_MAX_FAST and self.fast_rate_rounds == Channel.FAST_RATE_THRESHOLD:
|
||||
self.window_max = Channel.WINDOW_MAX_FAST
|
||||
self.window_min = Channel.WINDOW_MIN_LIMIT_FAST
|
||||
# TODO: Remove at some point
|
||||
# RNS.log("Increased "+str(self)+" max window to "+str(self.window_max), RNS.LOG_DEBUG)
|
||||
# RNS.log("Increased "+str(self)+" min window to "+str(self.window_min), RNS.LOG_DEBUG)
|
||||
|
||||
|
||||
else:
|
||||
RNS.log("Envelope not found in TX ring for "+str(self), RNS.LOG_EXTREME)
|
||||
if not envelope:
|
||||
RNS.log("Spurious message received on "+str(self), RNS.LOG_EXTREME)
|
||||
else: RNS.log("Envelope not found in TX ring for "+str(self), RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
|
||||
if not envelope: RNS.log("Spurious message received on "+str(self), RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
|
||||
|
||||
def _packet_delivered(self, packet: TPacket):
|
||||
self._packet_tx_op(packet, lambda env: True)
|
||||
|
|
@ -539,44 +450,47 @@ class Channel(contextlib.AbstractContextManager):
|
|||
for envelope in self._tx_ring:
|
||||
updated_timeout = self._get_packet_timeout_time(envelope.tries)
|
||||
if envelope.packet and hasattr(envelope.packet, "receipt") and envelope.packet.receipt and envelope.packet.receipt.timeout:
|
||||
if updated_timeout > envelope.packet.receipt.timeout:
|
||||
envelope.packet.receipt.set_timeout(updated_timeout)
|
||||
if updated_timeout > envelope.packet.receipt.timeout: envelope.packet.receipt.set_timeout(updated_timeout)
|
||||
|
||||
def _get_packet_timeout_time(self, tries: int) -> float:
|
||||
to = pow(1.5, tries - 1) * max(self._outlet.rtt*2.5, 0.025) * (len(self._tx_ring)+1.5)
|
||||
return to
|
||||
|
||||
def _packet_timeout(self, packet: TPacket):
|
||||
def retry_envelope(envelope: Envelope) -> bool:
|
||||
if envelope.tries >= self._max_tries:
|
||||
RNS.log("Retry count exceeded on "+str(self)+", tearing down Link.", RNS.LOG_ERROR)
|
||||
self._shutdown() # start on separate thread?
|
||||
self._outlet.timed_out()
|
||||
return True
|
||||
if self._outlet.get_packet_state(packet) == MessageState.MSGSTATE_DELIVERED: return
|
||||
|
||||
envelope.tries += 1
|
||||
self._outlet.resend(envelope.packet)
|
||||
self._outlet.set_packet_delivered_callback(envelope.packet, self._packet_delivered)
|
||||
self._outlet.set_packet_timeout_callback(envelope.packet, self._packet_timeout, self._get_packet_timeout_time(envelope.tries))
|
||||
self._update_packet_timeouts()
|
||||
target_id = self._outlet.get_packet_id(packet)
|
||||
envelope_to_resend: Envelope | None = None
|
||||
should_teardown = False
|
||||
with self._lock:
|
||||
envelope = next(filter(lambda e: e.packet is not None and self._outlet.get_packet_id(e.packet) == target_id, self._tx_ring), None)
|
||||
if envelope is None: return
|
||||
|
||||
if self.window > self.window_min:
|
||||
self.window -= 1
|
||||
# TODO: Remove at some point
|
||||
# RNS.log("Decreased "+str(self)+" window to "+str(self.window), RNS.LOG_DEBUG)
|
||||
if envelope.tries >= self._max_tries: should_teardown = True
|
||||
else:
|
||||
envelope.tries += 1
|
||||
envelope_to_resend = envelope
|
||||
|
||||
if self.window_max > (self.window_min+self.window_flexibility):
|
||||
self.window_max -= 1
|
||||
# TODO: Remove at some point
|
||||
# RNS.log("Decreased "+str(self)+" max window to "+str(self.window_max), RNS.LOG_DEBUG)
|
||||
if self.window > self.window_min:
|
||||
self.window -= 1
|
||||
if self.window_max > (self.window_min+self.window_flexibility):
|
||||
self.window_max -= 1
|
||||
|
||||
# TODO: Remove at some point
|
||||
# RNS.log("Decreased "+str(self)+" window to "+str(self.window), RNS.LOG_EXTREME)
|
||||
if should_teardown:
|
||||
RNS.log("Retry count exceeded on "+str(self)+", tearing down Link.", RNS.LOG_ERROR)
|
||||
self._shutdown()
|
||||
self._outlet.timed_out()
|
||||
return
|
||||
|
||||
return False
|
||||
if envelope_to_resend is not None:
|
||||
self._outlet.resend(envelope_to_resend.packet)
|
||||
with self._lock:
|
||||
self._outlet.set_packet_delivered_callback(envelope_to_resend.packet, self._packet_delivered)
|
||||
self._outlet.set_packet_timeout_callback(envelope_to_resend.packet, self._packet_timeout, self._get_packet_timeout_time(envelope_to_resend.tries))
|
||||
self._update_packet_timeouts()
|
||||
already_delivered = (self._outlet.get_packet_state(envelope_to_resend.packet) == MessageState.MSGSTATE_DELIVERED)
|
||||
|
||||
if self._outlet.get_packet_state(packet) != MessageState.MSGSTATE_DELIVERED:
|
||||
self._packet_tx_op(packet, retry_envelope)
|
||||
if already_delivered: self._packet_delivered(envelope_to_resend.packet)
|
||||
|
||||
def send(self, message: MessageBase) -> Envelope:
|
||||
"""
|
||||
|
|
@ -585,27 +499,33 @@ class Channel(contextlib.AbstractContextManager):
|
|||
|
||||
:param message: an instance of a ``MessageBase`` subclass
|
||||
"""
|
||||
envelope: Envelope | None = None
|
||||
with self._lock:
|
||||
if not self.is_ready_to_send():
|
||||
raise ChannelException(CEType.ME_LINK_NOT_READY, f"Link is not ready")
|
||||
|
||||
envelope = Envelope(self._outlet, message=message, sequence=self._next_sequence)
|
||||
self._next_sequence = (self._next_sequence + 1) % Channel.SEQ_MODULUS
|
||||
self._emplace_envelope(envelope, self._tx_ring)
|
||||
with self._send_lock:
|
||||
with self._lock:
|
||||
if not self.is_ready_to_send(): raise ChannelException(CEType.ME_LINK_NOT_READY, f"Link is not ready")
|
||||
|
||||
if envelope is None:
|
||||
raise BlockingIOError()
|
||||
reserved_sequence = self._next_sequence
|
||||
envelope = Envelope(self._outlet, message=message, sequence=reserved_sequence)
|
||||
envelope.pack()
|
||||
if len(envelope.raw) > self._outlet.mdu:
|
||||
raise ChannelException(CEType.ME_TOO_BIG, f"Packed message too big for packet: {len(envelope.raw)} > {self._outlet.mdu}")
|
||||
self._next_sequence = (reserved_sequence + 1) % Channel.SEQ_MODULUS
|
||||
|
||||
envelope.pack()
|
||||
if len(envelope.raw) > self._outlet.mdu:
|
||||
raise ChannelException(CEType.ME_TOO_BIG, f"Packed message too big for packet: {len(envelope.raw)} > {self._outlet.mdu}")
|
||||
|
||||
envelope.packet = self._outlet.send(envelope.raw)
|
||||
envelope.tries += 1
|
||||
self._outlet.set_packet_delivered_callback(envelope.packet, self._packet_delivered)
|
||||
self._outlet.set_packet_timeout_callback(envelope.packet, self._packet_timeout, self._get_packet_timeout_time(envelope.tries))
|
||||
self._update_packet_timeouts()
|
||||
envelope.packet = self._outlet.send(envelope.raw)
|
||||
|
||||
if envelope.packet is None or getattr(envelope.packet, "raw", None) is None or (hasattr(envelope.packet, "receipt") and envelope.packet.receipt is None):
|
||||
with self._lock: self._next_sequence = reserved_sequence
|
||||
raise ChannelException(CEType.ME_LINK_NOT_READY, "Outlet did not transmit packet")
|
||||
|
||||
with self._lock:
|
||||
self._emplace_envelope(envelope, self._tx_ring)
|
||||
envelope.tries += 1
|
||||
self._outlet.set_packet_delivered_callback(envelope.packet, self._packet_delivered)
|
||||
self._outlet.set_packet_timeout_callback(envelope.packet, self._packet_timeout, self._get_packet_timeout_time(envelope.tries))
|
||||
self._update_packet_timeouts()
|
||||
already_delivered = (self._outlet.get_packet_state(envelope.packet) == MessageState.MSGSTATE_DELIVERED)
|
||||
|
||||
# Prevent _tx_ring envelope leak
|
||||
if already_delivered: self._packet_delivered(envelope.packet)
|
||||
|
||||
return envelope
|
||||
|
||||
|
|
@ -620,14 +540,13 @@ class Channel(contextlib.AbstractContextManager):
|
|||
:return: number of bytes available
|
||||
"""
|
||||
mdu = self._outlet.mdu - 6 # sizeof(msgtype) + sizeof(length) + sizeof(sequence)
|
||||
if mdu > 0xFFFF:
|
||||
mdu = 0xFFFF
|
||||
if mdu > 0xFFFF: mdu = 0xFFFF
|
||||
return mdu
|
||||
|
||||
|
||||
class LinkChannelOutlet(ChannelOutletBase):
|
||||
class LinkChannelOutlet(ABC, Generic[TPacket]):
|
||||
"""
|
||||
An implementation of ChannelOutletBase for RNS.Link.
|
||||
A channel outlet implementation for RNS.Link.
|
||||
Allows Channel to send packets over an RNS Link with
|
||||
Packets.
|
||||
|
||||
|
|
@ -638,68 +557,51 @@ class LinkChannelOutlet(ChannelOutletBase):
|
|||
|
||||
def send(self, raw: bytes) -> RNS.Packet:
|
||||
packet = RNS.Packet(self.link, raw, context=RNS.Packet.CHANNEL)
|
||||
if self.link.status == RNS.Link.ACTIVE:
|
||||
packet.send()
|
||||
if self.link.status == RNS.Link.ACTIVE: packet.send()
|
||||
return packet
|
||||
|
||||
def resend(self, packet: RNS.Packet) -> RNS.Packet:
|
||||
receipt = packet.resend()
|
||||
if not receipt:
|
||||
RNS.log("Failed to resend packet", RNS.LOG_ERROR)
|
||||
if not receipt: RNS.log("Failed to resend packet", RNS.LOG_ERROR)
|
||||
return packet
|
||||
|
||||
@property
|
||||
def mdu(self):
|
||||
return self.link.mdu
|
||||
def mdu(self): return self.link.mdu
|
||||
|
||||
@property
|
||||
def rtt(self):
|
||||
return self.link.rtt
|
||||
def rtt(self): return self.link.rtt
|
||||
|
||||
# TODO: Initial implementation had issues
|
||||
# proxying this to link status. Should
|
||||
# return False when link is not ACTIVE.
|
||||
# Investigate what happened here.
|
||||
@property
|
||||
def is_usable(self):
|
||||
return True # had issues looking at Link.status
|
||||
def is_usable(self): return True
|
||||
|
||||
def get_packet_state(self, packet: TPacket) -> MessageState:
|
||||
if packet.receipt == None:
|
||||
return MessageState.MSGSTATE_FAILED
|
||||
|
||||
status = packet.receipt.get_status()
|
||||
if status == RNS.PacketReceipt.SENT:
|
||||
return MessageState.MSGSTATE_SENT
|
||||
if status == RNS.PacketReceipt.DELIVERED:
|
||||
return MessageState.MSGSTATE_DELIVERED
|
||||
if status == RNS.PacketReceipt.FAILED:
|
||||
return MessageState.MSGSTATE_FAILED
|
||||
else:
|
||||
raise Exception(f"Unexpected receipt state: {status}")
|
||||
if status == RNS.PacketReceipt.SENT: return MessageState.MSGSTATE_SENT
|
||||
if status == RNS.PacketReceipt.DELIVERED: return MessageState.MSGSTATE_DELIVERED
|
||||
if status == RNS.PacketReceipt.FAILED: return MessageState.MSGSTATE_FAILED
|
||||
else: raise Exception(f"Unexpected receipt state: {status}")
|
||||
|
||||
def timed_out(self):
|
||||
self.link.teardown()
|
||||
|
||||
def __str__(self):
|
||||
return f"{self.__class__.__name__}({self.link})"
|
||||
|
||||
def set_packet_timeout_callback(self, packet: RNS.Packet, callback: Callable[[RNS.Packet], None] | None,
|
||||
timeout: float | None = None):
|
||||
if timeout and packet.receipt:
|
||||
packet.receipt.set_timeout(timeout)
|
||||
|
||||
def inner(receipt: RNS.PacketReceipt):
|
||||
callback(packet)
|
||||
|
||||
if packet and packet.receipt:
|
||||
packet.receipt.set_timeout_callback(inner if callback else None)
|
||||
def set_packet_timeout_callback(self, packet: RNS.Packet, callback: Callable[[RNS.Packet], None] | None, timeout: float | None = None):
|
||||
if timeout and packet.receipt: packet.receipt.set_timeout(timeout)
|
||||
def inner(receipt: RNS.PacketReceipt): callback(packet)
|
||||
if packet and packet.receipt: packet.receipt.set_timeout_callback(inner if callback else None)
|
||||
|
||||
def set_packet_delivered_callback(self, packet: RNS.Packet, callback: Callable[[RNS.Packet], None] | None):
|
||||
def inner(receipt: RNS.PacketReceipt):
|
||||
callback(packet)
|
||||
|
||||
if packet and packet.receipt:
|
||||
packet.receipt.set_delivery_callback(inner if callback else None)
|
||||
def inner(receipt: RNS.PacketReceipt): callback(packet)
|
||||
if packet and packet.receipt: packet.receipt.set_delivery_callback(inner if callback else None)
|
||||
|
||||
def get_packet_id(self, packet: RNS.Packet) -> any:
|
||||
if packet and hasattr(packet, "get_hash") and callable(packet.get_hash):
|
||||
if packet and getattr(packet, "raw", None) is not None and hasattr(packet, "get_hash") and callable(packet.get_hash):
|
||||
return packet.get_hash()
|
||||
else:
|
||||
return None
|
||||
else: return None
|
||||
|
||||
def timed_out(self): self.link.teardown()
|
||||
|
||||
def __str__(self): return f"{self.__class__.__name__}({self.link})"
|
||||
|
|
@ -43,7 +43,7 @@ elif cp.PROVIDER == cp.PROVIDER_PYCA:
|
|||
class AES_128_CBC:
|
||||
@staticmethod
|
||||
def encrypt(plaintext, key, iv):
|
||||
if len(key) != 16: raise ValueError(f"Invalid key length {len(key)*8} for {self}")
|
||||
if len(key) != 16: raise ValueError(f"Invalid key length {len(key)*8} for AES_128_CBC")
|
||||
if cp.PROVIDER == cp.PROVIDER_INTERNAL:
|
||||
cipher = AES128(key)
|
||||
return cipher.encrypt(plaintext, iv)
|
||||
|
|
@ -60,7 +60,7 @@ class AES_128_CBC:
|
|||
|
||||
@staticmethod
|
||||
def decrypt(ciphertext, key, iv):
|
||||
if len(key) != 16: raise ValueError(f"Invalid key length {len(key)*8} for {self}")
|
||||
if len(key) != 16: raise ValueError(f"Invalid key length {len(key)*8} for AES_128_CBC")
|
||||
if cp.PROVIDER == cp.PROVIDER_INTERNAL:
|
||||
cipher = AES128(key)
|
||||
return cipher.decrypt(ciphertext, iv)
|
||||
|
|
@ -78,7 +78,7 @@ class AES_128_CBC:
|
|||
class AES_256_CBC:
|
||||
@staticmethod
|
||||
def encrypt(plaintext, key, iv):
|
||||
if len(key) != 32: raise ValueError(f"Invalid key length {len(key)*8} for {self}")
|
||||
if len(key) != 32: raise ValueError(f"Invalid key length {len(key)*8} for AES_256_CBC")
|
||||
if cp.PROVIDER == cp.PROVIDER_INTERNAL:
|
||||
cipher = AES256(key)
|
||||
return cipher.encrypt_cbc(plaintext, iv)
|
||||
|
|
@ -95,7 +95,7 @@ class AES_256_CBC:
|
|||
|
||||
@staticmethod
|
||||
def decrypt(ciphertext, key, iv):
|
||||
if len(key) != 32: raise ValueError(f"Invalid key length {len(key)*8} for {self}")
|
||||
if len(key) != 32: raise ValueError(f"Invalid key length {len(key)*8} for AES_256_CBC")
|
||||
if cp.PROVIDER == cp.PROVIDER_INTERNAL:
|
||||
cipher = AES256(key)
|
||||
return cipher.decrypt_cbc(ciphertext, iv)
|
||||
|
|
|
|||
|
|
@ -35,7 +35,6 @@ class Ed25519PrivateKey:
|
|||
def __init__(self, seed):
|
||||
self.seed = seed
|
||||
self.sk = ed25519.SigningKey(self.seed)
|
||||
#self.vk = self.sk.get_verifying_key()
|
||||
|
||||
@classmethod
|
||||
def generate(cls):
|
||||
|
|
|
|||
|
|
@ -32,8 +32,59 @@ import hashlib
|
|||
from math import ceil
|
||||
from RNS.Cryptography import HMAC
|
||||
|
||||
# Precomputed byte-permutation tables used to key the inner
|
||||
# and outer hashing states of the HMAC construction
|
||||
trans_5C = bytes((x ^ 0x5C) for x in range(256))
|
||||
trans_36 = bytes((x ^ 0x36) for x in range(256))
|
||||
|
||||
# Precomputed HKDF counter bytes
|
||||
_counters = [bytes([(i + 1) % 0x100]) for i in range(0x100)]
|
||||
|
||||
def hkdf(length=None, derive_from=None, salt=None, context=None):
|
||||
hash_len = 32
|
||||
hash_block_size = 64
|
||||
|
||||
if length == None or length < 1: raise ValueError("Invalid output key length")
|
||||
if derive_from == None or derive_from == "": raise ValueError("Cannot derive key from empty input material")
|
||||
|
||||
if salt == None or len(salt) == 0: salt = bytes([0] * hash_len)
|
||||
if context == None: context = b""
|
||||
|
||||
pseudorandom_key = HMAC.new(salt, derive_from).digest()
|
||||
|
||||
# The pseudorandom key is always a 32-byte SHA-256 digest, so the
|
||||
# HMAC key schedule below never needs the long-key hashing branch.
|
||||
key = pseudorandom_key.ljust(hash_block_size, b"\x00")
|
||||
|
||||
# Precompute the keyed inner and outer hashing states of the
|
||||
# HMAC construction once. Each expansion block then only requires
|
||||
# two cheap hash-object copies instead of constructing a complete
|
||||
# HMAC object per 32 bytes of output. This is possible since the
|
||||
# PRK is **always** 32-byte SHA-256 HMAC output.
|
||||
inner_state = hashlib.sha256(key.translate(trans_36))
|
||||
outer_state = hashlib.sha256(key.translate(trans_5C))
|
||||
|
||||
# Expand
|
||||
has_context = len(context) > 0
|
||||
block = b""
|
||||
derived = bytearray()
|
||||
|
||||
for i in range(ceil(length / hash_len)):
|
||||
inner = inner_state.copy()
|
||||
inner.update(block)
|
||||
if has_context: inner.update(context)
|
||||
inner.update(_counters[i & 0xFF])
|
||||
|
||||
outer = outer_state.copy()
|
||||
outer.update(inner.digest())
|
||||
|
||||
block = outer.digest()
|
||||
derived += block
|
||||
|
||||
return bytes(derived[:length])
|
||||
|
||||
def hkdf_legacy(length=None, derive_from=None, salt=None, context=None):
|
||||
hash_len = 32
|
||||
|
||||
def hmac_sha256(key, data):
|
||||
return HMAC.new(key, data).digest()
|
||||
|
|
|
|||
|
|
@ -180,4 +180,9 @@ def digest(key, msg, digest):
|
|||
outer.update(key.translate(trans_5C))
|
||||
inner.update(msg)
|
||||
outer.update(inner.digest())
|
||||
return outer.digest()
|
||||
return outer.digest()
|
||||
|
||||
def _cd_internal(a, b): return a == b
|
||||
try: from hmac import compare_digest as _cd
|
||||
except ImportError: _cd = _cd_internal
|
||||
compare_digest = _cd
|
||||
|
|
@ -62,3 +62,9 @@ def sha512(data):
|
|||
digest.update(data)
|
||||
|
||||
return digest.digest()
|
||||
|
||||
def file_sha256(file):
|
||||
if not hashlib: raise SystemError("The hashlib module is not available on this system")
|
||||
# TODO: Could implement fallback for old snakes here
|
||||
if not hasattr(hashlib, "file_digest"): raise SystemError("The file_digest method is not available on this system. This functionality requires Python 3.11 or later.")
|
||||
else: return hashlib.file_digest(file, "sha256").digest()
|
||||
|
|
|
|||
|
|
@ -80,7 +80,7 @@ class Token():
|
|||
received_hmac = token[-32:]
|
||||
expected_hmac = HMAC.new(self._signing_key, token[:-32]).digest()
|
||||
|
||||
if received_hmac == expected_hmac: return True
|
||||
if HMAC.compare_digest(received_hmac, expected_hmac): return True
|
||||
else: return False
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -52,5 +52,7 @@ elif cp.PROVIDER == cp.PROVIDER_PYCA:
|
|||
|
||||
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
|
||||
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
|
||||
modules = py_modules+pyc_modules
|
||||
__all__ = list(set([os.path.basename(f).replace(".pyc", "").replace(".py", "") for f in modules if not (f.endswith("__init__.py") or f.endswith("__init__.pyc"))]))
|
||||
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
|
||||
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
|
||||
modules = py_modules+pyc_modules+so_modules+pyd_modules
|
||||
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -35,6 +35,8 @@ import threading
|
|||
import RNS
|
||||
|
||||
from RNS.Cryptography import Token
|
||||
from RNS.Stamps import generate_stamp, store_destination_stamp, load_destination_stamp
|
||||
from RNS.Stamps import cost_to_bytes, quantize_cost
|
||||
from .vendor import umsgpack as umsgpack
|
||||
|
||||
class Callbacks:
|
||||
|
|
@ -146,6 +148,14 @@ class Destination:
|
|||
|
||||
return Destination.hash(identity, app_name, *aspects)
|
||||
|
||||
@staticmethod
|
||||
def name_hash(full_name):
|
||||
components = Destination.app_and_aspects_from_name(full_name)
|
||||
app_name = str(components[0])
|
||||
aspects = f".{'.'.join(components[1])}" if components[1] else ""
|
||||
name = f"{app_name}{aspects}".encode("utf-8")
|
||||
return RNS.Identity.full_hash(name)[:(RNS.Identity.NAME_HASH_LENGTH//8)]
|
||||
|
||||
def __init__(self, identity, direction, type, app_name, *aspects):
|
||||
# Check input values and build name string
|
||||
if "." in app_name: raise ValueError("Dots can't be used in app names")
|
||||
|
|
@ -155,9 +165,15 @@ class Destination:
|
|||
self.accept_link_requests = True
|
||||
self.callbacks = Callbacks()
|
||||
self.request_handlers = {}
|
||||
self.max_request_size = None
|
||||
self.type = type
|
||||
self.direction = direction
|
||||
self.proof_strategy = Destination.PROVE_NONE
|
||||
self.stamp = None
|
||||
self.stamp_value = None
|
||||
self.stamp_generation_lock = threading.Lock()
|
||||
self._target_value = RNS.Transport.DESTINATION_VALUE
|
||||
self._link_cost = None
|
||||
self.ratchets = None
|
||||
self.ratchets_path = None
|
||||
self.ratchet_interval = Destination.RATCHET_INTERVAL
|
||||
|
|
@ -189,6 +205,13 @@ class Destination:
|
|||
self.name_hash = RNS.Identity.full_hash(self.expand_name(None, app_name, *aspects).encode("utf-8"))[:(RNS.Identity.NAME_HASH_LENGTH//8)]
|
||||
self.hexhash = self.hash.hex()
|
||||
|
||||
# Load existing stamp if available
|
||||
if self.direction == Destination.IN and self.identity != None and self.identity.prv != None and RNS.Transport.STAMP_DESTINATIONS:
|
||||
stamp_entry = load_destination_stamp(self.hash, self.stamp_material(), RNS.Transport.DESTINATION_COST)
|
||||
if stamp_entry != None:
|
||||
self.stamp, self.stamp_value = stamp_entry
|
||||
RNS.log(f"Loaded stamp with value {RNS.prettycost(self.stamp_value)} for {self}", RNS.LOG_DEBUG)
|
||||
|
||||
self.default_app_data = None
|
||||
self.callback = None
|
||||
self.proofcallback = None
|
||||
|
|
@ -205,7 +228,7 @@ class Destination:
|
|||
def _clean_ratchets(self):
|
||||
if self.ratchets != None:
|
||||
if len (self.ratchets) > self.retained_ratchets:
|
||||
self.ratchets = self.ratchets[:Destination.RATCHET_COUNT]
|
||||
self.ratchets = self.ratchets[:self.retained_ratchets]
|
||||
|
||||
def _persist_ratchets(self):
|
||||
try:
|
||||
|
|
@ -240,6 +263,26 @@ class Destination:
|
|||
|
||||
return False
|
||||
|
||||
def _await_stamp(self):
|
||||
with self.stamp_generation_lock:
|
||||
if self.stamp == None or self.stamp_value == None or self.stamp_value < quantize_cost(self.target_value):
|
||||
stamp_entry = load_destination_stamp(self.hash, self.stamp_material(), self.target_value)
|
||||
if stamp_entry != None: self.stamp, self.stamp_value = stamp_entry
|
||||
else:
|
||||
stamp, stamp_value = generate_stamp(self.stamp_material(), stamp_cost=self.target_value,
|
||||
expand_rounds=0, size=RNS.Identity.STAMPLENGTH)
|
||||
if stamp == None:
|
||||
RNS.log(f"Destination stamp generation for {RNS.prettyhexrep(self.hash)} failed or was aborted", RNS.LOG_DEBUG)
|
||||
return False
|
||||
|
||||
self.stamp = stamp
|
||||
self.stamp_value = stamp_value
|
||||
store_destination_stamp(self.hash, stamp, stamp_value)
|
||||
RNS.log(f"Generated and persisted stamp with value {self.stamp_value} for {self}", RNS.LOG_DEBUG)
|
||||
|
||||
if self.stamp == None: return False
|
||||
else: return True
|
||||
|
||||
def announce(self, app_data=None, path_response=False, attached_interface=None, tag=None, send=True):
|
||||
"""
|
||||
Creates an announce packet for this destination and broadcasts it on all
|
||||
|
|
@ -248,12 +291,20 @@ class Destination:
|
|||
:param app_data: *bytes* containing the app_data.
|
||||
:param path_response: Internal flag used by :ref:`RNS.Transport<api-transport>`. Ignore.
|
||||
"""
|
||||
if self.type != Destination.SINGLE:
|
||||
raise TypeError("Only SINGLE destination types can be announced")
|
||||
if self.type != Destination.SINGLE: raise TypeError("Only SINGLE destination types can be announced")
|
||||
if self.direction != Destination.IN: raise TypeError("Only IN destination types can be announced")
|
||||
|
||||
if self.direction != Destination.IN:
|
||||
raise TypeError("Only IN destination types can be announced")
|
||||
|
||||
if send and RNS.Transport.STAMP_DESTINATIONS and self.stamp == None:
|
||||
# If no stamp is available, queue for background generation
|
||||
# and announce emission once ready.
|
||||
RNS.Transport.enqueue_stamped_announce(self, app_data=app_data, path_response=path_response,
|
||||
attached_interface=attached_interface, tag=tag)
|
||||
else:
|
||||
# Otherwise, dispatch immediately.
|
||||
return self._announce(app_data=app_data, path_response=path_response,
|
||||
attached_interface=attached_interface, tag=tag, send=send)
|
||||
|
||||
def _announce(self, app_data=None, path_response=False, attached_interface=None, tag=None, send=True):
|
||||
ratchet = b""
|
||||
now = time.time()
|
||||
stale_responses = []
|
||||
|
|
@ -293,35 +344,77 @@ class Destination:
|
|||
returned_app_data = self.default_app_data()
|
||||
if isinstance(returned_app_data, bytes):
|
||||
app_data = returned_app_data
|
||||
|
||||
if RNS.Transport.STAMP_DESTINATIONS:
|
||||
if not self._await_stamp():
|
||||
RNS.log(f"Could not generate destination stamp for {RNS.prettyhexrep(self.hash)}, announce emission cancelled", RNS.LOG_WARNING)
|
||||
return None
|
||||
|
||||
cost_bytes = cost_to_bytes(self.get_link_cost())
|
||||
ext_payload = bytes([RNS.Identity.EXTPAYLOADMARKER, self.get_ext_flags()])+self.stamp+bytes([cost_bytes[1], cost_bytes[0]])
|
||||
|
||||
if app_data == None: app_data = ext_payload
|
||||
else: app_data = app_data+ext_payload
|
||||
|
||||
signed_data = self.hash+self.identity.get_public_key()+self.name_hash+random_hash+ratchet
|
||||
if app_data != None:
|
||||
signed_data += app_data
|
||||
if app_data != None: signed_data += app_data
|
||||
|
||||
signature = self.identity.sign(signed_data)
|
||||
announce_data = self.identity.get_public_key()+self.name_hash+random_hash+ratchet+signature
|
||||
|
||||
if app_data != None:
|
||||
announce_data += app_data
|
||||
if app_data != None: announce_data += app_data
|
||||
|
||||
self.path_responses[tag] = [time.time(), announce_data]
|
||||
|
||||
if path_response:
|
||||
announce_context = RNS.Packet.PATH_RESPONSE
|
||||
else:
|
||||
announce_context = RNS.Packet.NONE
|
||||
if path_response: announce_context = RNS.Packet.PATH_RESPONSE
|
||||
else: announce_context = RNS.Packet.NONE
|
||||
|
||||
if ratchet:
|
||||
context_flag = RNS.Packet.FLAG_SET
|
||||
else:
|
||||
context_flag = RNS.Packet.FLAG_UNSET
|
||||
if ratchet: context_flag = RNS.Packet.FLAG_SET
|
||||
else: context_flag = RNS.Packet.FLAG_UNSET
|
||||
|
||||
announce_packet = RNS.Packet(self, announce_data, RNS.Packet.ANNOUNCE, context = announce_context,
|
||||
attached_interface = attached_interface, context_flag=context_flag)
|
||||
if send:
|
||||
announce_packet.send()
|
||||
else:
|
||||
return announce_packet
|
||||
|
||||
if send: announce_packet.send()
|
||||
else: return announce_packet
|
||||
|
||||
def stamp_material(self):
|
||||
return self.hash+self.identity.get_public_key()+self.name_hash
|
||||
|
||||
def get_ext_flags(self):
|
||||
return 0x00
|
||||
|
||||
@property
|
||||
def target_value(self):
|
||||
return min(RNS.Transport.STAMP_COST_CAP, max(float(self._target_value), RNS.Transport.DESTINATION_VALUE))
|
||||
|
||||
def set_value(self, value):
|
||||
"""
|
||||
Sets the target stamp value for this destination.
|
||||
|
||||
:param value: The target value as a *float*.
|
||||
"""
|
||||
self._target_value = min(RNS.Transport.STAMP_COST_CAP, max(float(value), RNS.Transport.DESTINATION_VALUE))
|
||||
|
||||
def set_link_cost(self, cost):
|
||||
"""
|
||||
Sets the required link establishment stamp cost for this destination. Set to 0 to disable.
|
||||
|
||||
:param cost: The required cost as a *float*.
|
||||
"""
|
||||
self._link_cost = max(0, min(float(cost), RNS.Transport.LINK_COST_CAP))
|
||||
|
||||
def get_link_cost(self):
|
||||
"""
|
||||
Returns the link request stamp cost to be announced for this
|
||||
destination, as a float. If no destination-specific cost has
|
||||
been set, the default link request cost is returned.
|
||||
|
||||
:returns: The announced link request cost as a *float*.
|
||||
"""
|
||||
cost = self._link_cost
|
||||
if cost == None: cost = RNS.Transport.LR_COST
|
||||
return max(0.0, min(float(cost), RNS.Transport.STAMP_COST_CAP))
|
||||
|
||||
def accepts_links(self, accepts = None):
|
||||
"""
|
||||
|
|
@ -330,13 +423,10 @@ class Destination:
|
|||
:param accepts: If ``True`` or ``False``, this method sets whether the destination accepts incoming link requests. If not provided or ``None``, the method returns whether the destination currently accepts link requests.
|
||||
:returns: ``True`` or ``False`` depending on whether the destination accepts incoming link requests, if the *accepts* parameter is not provided or ``None``.
|
||||
"""
|
||||
if accepts == None:
|
||||
return self.accept_link_requests
|
||||
if accepts == None: return self.accept_link_requests
|
||||
|
||||
if accepts:
|
||||
self.accept_link_requests = True
|
||||
else:
|
||||
self.accept_link_requests = False
|
||||
if accepts: self.accept_link_requests = True
|
||||
else: self.accept_link_requests = False
|
||||
|
||||
def set_link_established_callback(self, callback):
|
||||
"""
|
||||
|
|
@ -372,10 +462,20 @@ class Destination:
|
|||
|
||||
:param proof_strategy: One of ``RNS.Destination.PROVE_NONE``, ``RNS.Destination.PROVE_ALL`` or ``RNS.Destination.PROVE_APP``. If ``RNS.Destination.PROVE_APP`` is set, the `proof_requested_callback` will be called to determine whether a proof should be sent or not.
|
||||
"""
|
||||
if not proof_strategy in Destination.proof_strategies:
|
||||
raise TypeError("Unsupported proof strategy")
|
||||
else:
|
||||
self.proof_strategy = proof_strategy
|
||||
if not proof_strategy in Destination.proof_strategies: raise TypeError("Unsupported proof strategy")
|
||||
else: self.proof_strategy = proof_strategy
|
||||
|
||||
def set_max_request_size(self, max_request_size):
|
||||
"""
|
||||
Sets the maximum accepted request size for registered request handlers.
|
||||
|
||||
:param max_request_size: The maximum accepted request size in bytes, as an integer.
|
||||
:raises: ``TypeError`` or ``ValueError`` if any of the argument is invalid.
|
||||
"""
|
||||
try: max_request_size = int(max_request_size)
|
||||
except: raise TypeError("Invalid maximum request size specified")
|
||||
if max_request_size < 0: raise ValueError("Maximum request size cannot be negative")
|
||||
self.max_request_size = max_request_size
|
||||
|
||||
def register_request_handler(self, path, response_generator = None, allow = ALLOW_NONE, allowed_list = None, auto_compress = True):
|
||||
"""
|
||||
|
|
@ -421,8 +521,7 @@ class Destination:
|
|||
else:
|
||||
if packet.packet_type == RNS.Packet.DATA:
|
||||
if self.callbacks.packet != None:
|
||||
try:
|
||||
self.callbacks.packet(plaintext, packet)
|
||||
try: self.callbacks.packet(plaintext, packet)
|
||||
except Exception as e:
|
||||
RNS.log("Error while executing receive callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
|
||||
|
|
@ -431,8 +530,7 @@ class Destination:
|
|||
def incoming_link_request(self, data, packet):
|
||||
if self.accept_link_requests:
|
||||
link = RNS.Link.validate_request(self, data, packet)
|
||||
if link != None:
|
||||
self.links.append(link)
|
||||
if link != None: self.links.append(link)
|
||||
|
||||
def _reload_ratchets(self, ratchets_path):
|
||||
if os.path.isfile(ratchets_path):
|
||||
|
|
|
|||
707
RNS/Discovery.py
707
RNS/Discovery.py
|
|
@ -1,12 +1,58 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
import os
|
||||
import re
|
||||
import RNS
|
||||
import time
|
||||
import random
|
||||
import threading
|
||||
import ipaddress
|
||||
import subprocess
|
||||
from threading import Lock
|
||||
from .vendor import umsgpack as msgpack
|
||||
from collections import deque
|
||||
|
||||
# Discoverable interfaces must include a short,
|
||||
# unique implementation-specific identifier and
|
||||
# version tag in announced discovery information.
|
||||
IMPLEMENTATION_NAME = "RNS"
|
||||
from RNS._version import __version__ as RNS_VERSION
|
||||
import RNS.Stamps
|
||||
|
||||
NAME = 0xFF
|
||||
TRANSPORT_ID = 0xFE
|
||||
TRANSPORT_IMPL = 0xFD
|
||||
TRANSPORT_VERS = 0xFC
|
||||
DST_COST = 0xFB
|
||||
PR_COST = 0xFA
|
||||
INTERFACE_TYPE = 0x00
|
||||
TRANSPORT = 0x01
|
||||
REACHABLE_ON = 0x02
|
||||
|
|
@ -22,29 +68,23 @@ SPREADINGFACTOR = 0x0B
|
|||
CODINGRATE = 0x0C
|
||||
MODULATION = 0x0D
|
||||
CHANNEL = 0x0E
|
||||
OP_ADDR = 0xF0
|
||||
|
||||
APP_NAME = "rnstransport"
|
||||
|
||||
class InterfaceAnnouncer():
|
||||
JOB_INTERVAL = 60
|
||||
DEFAULT_STAMP_VALUE = 14
|
||||
DEFAULT_STAMP_VALUE = 16
|
||||
WORKBLOCK_EXPAND_ROUNDS = 20
|
||||
|
||||
DISCOVERABLE_INTERFACE_TYPES = ["BackboneInterface", "TCPServerInterface", "TCPClientInterface",
|
||||
"RNodeInterface", "WeaveInterface", "I2PInterface", "KISSInterface"]
|
||||
|
||||
def __init__(self, owner):
|
||||
import importlib.util
|
||||
if importlib.util.find_spec('LXMF') != None: from LXMF import LXStamper
|
||||
else:
|
||||
RNS.log("Using on-network interface discovery requires the LXMF module to be installed.", RNS.LOG_CRITICAL)
|
||||
RNS.log("You can install it with the command: pip install lxmf", RNS.LOG_CRITICAL)
|
||||
RNS.panic()
|
||||
|
||||
self.owner = owner
|
||||
self.should_run = False
|
||||
self.job_interval = self.JOB_INTERVAL
|
||||
self.stamper = LXStamper
|
||||
self.stamper = RNS.Stamps
|
||||
self.stamp_cache = {}
|
||||
|
||||
if self.owner.has_network_identity(): identity = self.owner.network_identity
|
||||
|
|
@ -83,6 +123,7 @@ class InterfaceAnnouncer():
|
|||
RNS.trace_exception(e)
|
||||
|
||||
def sanitize(self, in_str):
|
||||
if in_str == None: return None
|
||||
sanitized = in_str.replace("\n", "")
|
||||
sanitized = sanitized.replace("\r", "")
|
||||
sanitized = sanitized.strip()
|
||||
|
|
@ -94,32 +135,78 @@ class InterfaceAnnouncer():
|
|||
|
||||
if not interface_type in self.DISCOVERABLE_INTERFACE_TYPES: return None
|
||||
else:
|
||||
if not RNS.vendor.platformutils.is_windows() and interface.discovery_location:
|
||||
try:
|
||||
discovery_location = self.sanitize(interface.discovery_location)
|
||||
exec_path = os.path.expanduser(discovery_location)
|
||||
if os.path.isfile(exec_path) and os.access(exec_path, os.X_OK):
|
||||
RNS.log(f"Evaluating discovery location from executable at {exec_path}", RNS.LOG_DEBUG)
|
||||
exec_result = subprocess.run([exec_path], stdout=subprocess.PIPE)
|
||||
exec_stdout = exec_result.stdout.decode("utf-8")
|
||||
if exec_result.returncode != 0: raise ValueError("Non-zero exit code from subprocess")
|
||||
discovery_location = self.sanitize(exec_stdout)
|
||||
location_components = discovery_location.replace(" ", "").split(",")
|
||||
if len(location_components) != 3: raise ValueError(f"Invalid location component count: {len(location_components)}")
|
||||
dlat = float(location_components[0])
|
||||
dlon = float(location_components[1])
|
||||
dhgt = float(location_components[2])
|
||||
if dlat < -90 or dlat > 90: raise ValueError(f"Invalid latitude: {dlat}")
|
||||
if dlon < -180 or dlat > 180: raise ValueError(f"Invalid longitude: {dlon}")
|
||||
if dhgt < -4000 or dhgt > 1e6: raise ValueError(f"Invalid height: {dhgt}")
|
||||
interface.discovery_latitude = dlat
|
||||
interface.discovery_longitude = dlon
|
||||
interface.discovery_height = dhgt
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error while getting discovery location from executable at {interface.reachable_on}: {e}", RNS.LOG_ERROR)
|
||||
RNS.log(f"Aborting discovery announce", RNS.LOG_ERROR)
|
||||
return None
|
||||
|
||||
flags = 0x00
|
||||
info = {INTERFACE_TYPE: interface_type,
|
||||
TRANSPORT: RNS.Reticulum.transport_enabled(),
|
||||
TRANSPORT_ID: RNS.Transport.identity.hash,
|
||||
TRANSPORT_IMPL: IMPLEMENTATION_NAME,
|
||||
TRANSPORT_VERS: RNS_VERSION,
|
||||
DST_COST: RNS.Stamps.quantize_cost(interface.destination_cost),
|
||||
PR_COST: RNS.Stamps.quantize_cost(interface.pr_cost),
|
||||
NAME: self.sanitize(interface.discovery_name),
|
||||
LATITUDE: interface.discovery_latitude,
|
||||
LONGITUDE: interface.discovery_longitude,
|
||||
HEIGHT: interface.discovery_height}
|
||||
|
||||
reachable_on = self.sanitize(interface.reachable_on)
|
||||
if not RNS.vendor.platformutils.is_windows():
|
||||
try:
|
||||
exec_path = os.path.expanduser(reachable_on)
|
||||
if os.path.isfile(exec_path) and os.access(exec_path, os.X_OK):
|
||||
RNS.log(f"Evaluating reachable_on from executable at {exec_path}", RNS.LOG_DEBUG)
|
||||
exec_result = subprocess.run([exec_path], stdout=subprocess.PIPE)
|
||||
exec_stdout = exec_result.stdout.decode("utf-8")
|
||||
if exec_result.returncode != 0: raise ValueError("Non-zero exit code from subprocess")
|
||||
reachable_on = self.sanitize(exec_stdout)
|
||||
if interface.discovery_lxmf_address: info[OP_ADDR] = interface.discovery_lxmf_address
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error while getting reachable_on from executable at {interface.reachable_on}: {e}", RNS.LOG_ERROR)
|
||||
if interface_type == "TCPClientInterface" and not interface.kiss_framing:
|
||||
RNS.log(f"Invalid interface discovery configuration for {interface}, aborting discovery announce", RNS.LOG_ERROR)
|
||||
return None
|
||||
|
||||
if interface_type in ["BackboneInterface", "TCPServerInterface"]:
|
||||
if not interface.reachable_on: return None
|
||||
reachable_on = self.sanitize(interface.reachable_on)
|
||||
|
||||
if not RNS.vendor.platformutils.is_windows():
|
||||
try:
|
||||
exec_path = os.path.expanduser(reachable_on)
|
||||
if os.path.isfile(exec_path) and os.access(exec_path, os.X_OK):
|
||||
RNS.log(f"Evaluating reachable_on from executable at {exec_path}", RNS.LOG_DEBUG)
|
||||
exec_result = subprocess.run([exec_path], stdout=subprocess.PIPE)
|
||||
exec_stdout = exec_result.stdout.decode("utf-8")
|
||||
if exec_result.returncode != 0: raise ValueError("Non-zero exit code from subprocess")
|
||||
reachable_on = self.sanitize(exec_stdout)
|
||||
if not (is_ip_address(reachable_on) or is_hostname(reachable_on)):
|
||||
raise ValueError(f"Valid IP address or hostname was not found in external script output \"{reachable_on}\"")
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error while getting reachable_on from executable at {interface.reachable_on}: {e}", RNS.LOG_ERROR)
|
||||
RNS.log(f"Aborting discovery announce", RNS.LOG_ERROR)
|
||||
return None
|
||||
|
||||
if not (is_ip_address(reachable_on) or is_hostname(reachable_on)):
|
||||
RNS.log(f"The configured reachable_on parameter \"{reachable_on}\" for {interface} is not a valid IP address or hostname", RNS.LOG_ERROR)
|
||||
RNS.log(f"Aborting discovery announce", RNS.LOG_ERROR)
|
||||
return None
|
||||
|
||||
if interface_type in ["BackboneInterface", "TCPServerInterface"]:
|
||||
info[REACHABLE_ON] = reachable_on
|
||||
info[PORT] = interface.bind_port
|
||||
|
||||
|
|
@ -145,8 +232,8 @@ class InterfaceAnnouncer():
|
|||
info[MODULATION] = self.sanitize(interface.discovery_modulation)
|
||||
|
||||
if interface.discovery_publish_ifac == True:
|
||||
info[IFAC_NETNAME] = self.sanitize(interface.ifac_netname)
|
||||
info[IFAC_NETKEY] = self.sanitize(interface.ifac_netkey)
|
||||
if interface.ifac_netname: info[IFAC_NETNAME] = self.sanitize(interface.ifac_netname)
|
||||
if interface.ifac_netkey: info[IFAC_NETKEY] = self.sanitize(interface.ifac_netkey)
|
||||
|
||||
packed = msgpack.packb(info)
|
||||
infohash = RNS.Identity.full_hash(packed)
|
||||
|
|
@ -173,17 +260,23 @@ class InterfaceAnnounceHandler:
|
|||
FLAG_ENCRYPTED = 0b00000010
|
||||
|
||||
def __init__(self, required_value=InterfaceAnnouncer.DEFAULT_STAMP_VALUE, callback=None):
|
||||
import importlib.util
|
||||
if importlib.util.find_spec('LXMF') != None: from LXMF import LXStamper
|
||||
else:
|
||||
RNS.log("Using on-network interface discovery requires the LXMF module to be installed.", RNS.LOG_CRITICAL)
|
||||
RNS.log("You can install it with the command: pip install lxmf", RNS.LOG_CRITICAL)
|
||||
RNS.panic()
|
||||
self.aspect_filter = APP_NAME+".discovery.interface"
|
||||
self.required_value = required_value
|
||||
self.callback = callback
|
||||
self.stamper = RNS.Stamps
|
||||
self.valid_cache = {}
|
||||
self.valid_cache_max = 2048
|
||||
self.invalid_cache = deque(maxlen=2048)
|
||||
self.validation_lock = threading.Lock()
|
||||
|
||||
self.aspect_filter = APP_NAME+".discovery.interface"
|
||||
self.required_value = required_value
|
||||
self.callback = callback
|
||||
self.stamper = LXStamper
|
||||
@staticmethod
|
||||
def sanitize_name(name):
|
||||
if not name: return None
|
||||
name = name.encode("ascii", "ignore").decode("ascii").strip()
|
||||
for i in [5,3,2]: name = name.replace(" "*i, " ")
|
||||
while len(name) and name[0] not in san_map: name = name[1:]
|
||||
while len(name) and name[-1] not in san_map+")": name = name[:-1]
|
||||
return name
|
||||
|
||||
def received_announce(self, destination_hash, announced_identity, app_data):
|
||||
try:
|
||||
|
|
@ -197,47 +290,105 @@ class InterfaceAnnounceHandler:
|
|||
app_data = app_data[1:]
|
||||
signed = flags & self.FLAG_SIGNED
|
||||
encrypted = flags & self.FLAG_ENCRYPTED
|
||||
fullhash = RNS.Identity.full_hash(app_data)
|
||||
hops = RNS.Transport.hops_to(destination_hash)
|
||||
|
||||
if encrypted:
|
||||
if not RNS.Transport.has_network_identity(): return
|
||||
app_data = RNS.Transport.network_identity.decrypt(app_data)
|
||||
if not app_data: return
|
||||
|
||||
stamp = app_data[-self.stamper.STAMP_SIZE:]
|
||||
packed = app_data[:-self.stamper.STAMP_SIZE]
|
||||
infohash = RNS.Identity.full_hash(packed)
|
||||
workblock = self.stamper.stamp_workblock(infohash, expand_rounds=InterfaceAnnouncer.WORKBLOCK_EXPAND_ROUNDS)
|
||||
value = self.stamper.stamp_value(workblock, stamp)
|
||||
valid = self.stamper.stamp_valid(stamp, self.required_value, workblock)
|
||||
|
||||
if not valid:
|
||||
RNS.log(f"Ignored discovered interface with invalid stamp", RNS.LOG_DEBUG)
|
||||
if hops > RNS.Transport.PATHFINDER_M-1:
|
||||
RNS.log(f"Ignored discovered interface with invalid hop count", RNS.LOG_PATHING) if RNS.sl(RNS.LOG_PATHING) else None
|
||||
return
|
||||
|
||||
if fullhash in self.invalid_cache:
|
||||
RNS.log(f"Ignored previously discovered interface with insufficient stamp value", RNS.LOG_PATHING) if RNS.sl(RNS.LOG_PATHING) else None
|
||||
return
|
||||
|
||||
if fullhash in self.valid_cache:
|
||||
RNS.log(f"Discovery announce cache hit", RNS.LOG_PATHING) if RNS.sl(RNS.LOG_PATHING) else None
|
||||
cache_entry = self.valid_cache[fullhash]
|
||||
stamp = cache_entry["stamp"]
|
||||
packed = cache_entry["packed"]
|
||||
value = cache_entry["value"]
|
||||
valid = cache_entry["valid"]
|
||||
|
||||
else:
|
||||
if encrypted:
|
||||
if not RNS.Transport.has_network_identity(): return
|
||||
app_data = RNS.Transport.network_identity.decrypt(app_data)
|
||||
if not app_data: return
|
||||
|
||||
if self.validation_lock.locked():
|
||||
RNS.log(f"Dropping received interface discovery announce, already validating other discovery stamp", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
return
|
||||
|
||||
with self.validation_lock:
|
||||
stamp = app_data[-self.stamper.STAMP_SIZE:]
|
||||
packed = app_data[:-self.stamper.STAMP_SIZE]
|
||||
infohash = RNS.Identity.full_hash(packed)
|
||||
|
||||
# Transitory mechanism, check non-expanded workblock validity first.
|
||||
workblock_ne = self.stamper.stamp_workblock(infohash, expand_rounds=0)
|
||||
valid_ne = self.stamper.stamp_valid(stamp, self.required_value, workblock_ne)
|
||||
if valid_ne:
|
||||
value = self.stamper.stamp_value(workblock_ne, stamp)
|
||||
valid = valid_ne
|
||||
|
||||
else:
|
||||
workblock = self.stamper.stamp_workblock(infohash, expand_rounds=InterfaceAnnouncer.WORKBLOCK_EXPAND_ROUNDS)
|
||||
value = self.stamper.stamp_value(workblock, stamp)
|
||||
valid = self.stamper.stamp_valid(stamp, self.required_value, workblock)
|
||||
|
||||
self.valid_cache[fullhash] = {"valid": valid, "value": value, "packed": packed, "stamp": stamp}
|
||||
while len(self.valid_cache) > self.valid_cache_max: self.valid_cache.pop(next(iter(self.valid_cache)))
|
||||
|
||||
if not valid:
|
||||
RNS.log(f"Ignored discovered interface with insufficient stamp value {RNS.prettycost(value)}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
self.invalid_cache.append(fullhash)
|
||||
return
|
||||
|
||||
if value < self.required_value: RNS.log(f"Ignored discovered interface with stamp value {value}", RNS.LOG_DEBUG)
|
||||
else:
|
||||
info = None
|
||||
unpacked = msgpack.unpackb(packed)
|
||||
if INTERFACE_TYPE in unpacked:
|
||||
interface_type = unpacked[INTERFACE_TYPE]
|
||||
interface_type = unpacked[INTERFACE_TYPE]
|
||||
name = self.sanitize_name(unpacked[NAME])
|
||||
impl_name = unpacked[TRANSPORT_IMPL] if TRANSPORT_IMPL in unpacked else None
|
||||
impl_version = unpacked[TRANSPORT_VERS] if TRANSPORT_VERS in unpacked else None
|
||||
dst_cost = RNS.Stamps.quantize_cost(unpacked[DST_COST]) if DST_COST in unpacked else None
|
||||
pr_cost = RNS.Stamps.quantize_cost(unpacked[PR_COST]) if PR_COST in unpacked else None
|
||||
|
||||
if type(unpacked[TRANSPORT]) != bool: raise ValueError("Invalid data in transport field of announce")
|
||||
if type(unpacked[LATITUDE]) not in [type(None), float]: raise ValueError("Invalid data in latitude field of announce")
|
||||
if type(unpacked[LONGITUDE]) not in [type(None), float]: raise ValueError("Invalid data in longitude field of announce")
|
||||
if type(unpacked[HEIGHT]) not in [type(None), float]: raise ValueError("Invalid data in height field of announce")
|
||||
if len(unpacked[TRANSPORT_ID]) != RNS.Identity.TRUNCATED_HASHLENGTH//8: raise ValueError("Invalid data in transport_id field of announce")
|
||||
if not interface_type in InterfaceAnnouncer.DISCOVERABLE_INTERFACE_TYPES:
|
||||
raise ValueError("Invalid interface type in announce data")
|
||||
|
||||
if REACHABLE_ON in unpacked:
|
||||
if not (is_ip_address(unpacked[REACHABLE_ON]) or is_hostname(unpacked[REACHABLE_ON])):
|
||||
raise ValueError("Invalid data in reachable_on field of announce")
|
||||
|
||||
info = {"type": interface_type,
|
||||
"impl_name": impl_name,
|
||||
"version": impl_version,
|
||||
"transport": unpacked[TRANSPORT],
|
||||
"name": unpacked[NAME] or f"Discovered {interface_type}",
|
||||
"name": name or f"Discovered {interface_type}",
|
||||
"received": time.time(),
|
||||
"stamp": stamp,
|
||||
"value": value,
|
||||
"transport_id": RNS.hexrep(unpacked[TRANSPORT_ID], delimit=False),
|
||||
"network_id": RNS.hexrep(announced_identity.hash, delimit=False),
|
||||
"hops": RNS.Transport.hops_to(destination_hash),
|
||||
"dst_cost": dst_cost,
|
||||
"pr_cost": pr_cost,
|
||||
"hops": hops,
|
||||
"latitude": unpacked[LATITUDE],
|
||||
"longitude": unpacked[LONGITUDE],
|
||||
"height": unpacked[HEIGHT]}
|
||||
|
||||
if IFAC_NETNAME in unpacked: info["ifac_netname"] = unpacked[IFAC_NETNAME]
|
||||
if IFAC_NETKEY in unpacked: info["ifac_netkey"] = unpacked[IFAC_NETKEY]
|
||||
if IFAC_NETNAME in unpacked and unpacked[IFAC_NETNAME] and type(unpacked[IFAC_NETNAME]) == str: info["ifac_netname"] = str(unpacked[IFAC_NETNAME])
|
||||
if IFAC_NETKEY in unpacked and unpacked[IFAC_NETKEY] and type(unpacked[IFAC_NETKEY]) == str: info["ifac_netkey"] = str(unpacked[IFAC_NETKEY])
|
||||
|
||||
if interface_type in ["BackboneInterface", "TCPServerInterface"]:
|
||||
backbone_support = not RNS.vendor.platformutils.is_windows()
|
||||
backbone_support = not RNS.vendor.platformutils.is_windows() and not RNS.vendor.platformutils.is_darwin()
|
||||
info["reachable_on"] = unpacked[REACHABLE_ON]
|
||||
info["port"] = unpacked[PORT]
|
||||
connection_interface = "BackboneInterface" if backbone_support else "TCPClientInterface"
|
||||
|
|
@ -263,7 +414,7 @@ class InterfaceAnnounceHandler:
|
|||
cfg_netname_str = f"\n network_name = {cfg_netname}" if cfg_netname else ""
|
||||
cfg_netkey_str = f"\n passphrase = {cfg_netkey}" if cfg_netkey else ""
|
||||
cfg_identity_str = f"\n transport_identity = {cfg_identity}"
|
||||
info["config_entry"] = f"[[{cfg_name}]]\n type = I2PInterface\n enabled = yes\n peers = {cfg_remote}{cfg_identity_str}{cfg_netname_str}{cfg_netkey_str}"
|
||||
info["config_entry"] = f"[[{cfg_name}]]\n type = I2PInterface\n enabled = yes\n peers = {cfg_remote}.b32.i2p{cfg_identity_str}{cfg_netname_str}{cfg_netkey_str}"
|
||||
|
||||
if interface_type == "RNodeInterface":
|
||||
info["frequency"] = unpacked[FREQUENCY]
|
||||
|
|
@ -316,24 +467,39 @@ class InterfaceAnnounceHandler:
|
|||
discovery_hash_material = info["transport_id"]+info["name"]
|
||||
info["discovery_hash"] = RNS.Identity.full_hash(discovery_hash_material.encode("utf-8"))
|
||||
|
||||
if info and OP_ADDR in unpacked:
|
||||
if type(unpacked[OP_ADDR]) not in [type(None), bytes]: raise ValueError("Invalid data in operator LXMF address field of announce")
|
||||
if unpacked[OP_ADDR] and len(unpacked[OP_ADDR]) == RNS.Identity.TRUNCATED_HASHLENGTH//8:
|
||||
info["operator_lxmf_address"] = RNS.hexrep(unpacked[OP_ADDR], delimit=False)
|
||||
|
||||
if self.callback and callable(self.callback): self.callback(info)
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"An error occurred while trying to decode discovered interface. The contained exception was: {e}", RNS.LOG_ERROR)
|
||||
RNS.log(f"Could not decode discovered interface. The contained exception was: {e}", RNS.LOG_DEBUG)
|
||||
|
||||
class InterfaceDiscovery():
|
||||
THRESHOLD_UNKNOWN = 24*60*60
|
||||
THRESHOLD_STALE = 3*24*60*60
|
||||
THRESHOLD_REMOVE = 7*24*60*60
|
||||
THRESHOLD_UNKNOWN = 24*60*60
|
||||
THRESHOLD_STALE = 3*24*60*60
|
||||
THRESHOLD_REMOVE = 7*24*60*60
|
||||
|
||||
MONITOR_INTERVAL = 5
|
||||
DETACH_THRESHOLD = 12
|
||||
MONITOR_INTERVAL = 5
|
||||
DETACH_THRESHOLD = 12
|
||||
|
||||
STATUS_STALE = 0
|
||||
STATUS_UNKNOWN = 100
|
||||
STATUS_AVAILABLE = 1000
|
||||
STATUS_CODE_MAP = {"available": STATUS_AVAILABLE, "unknown": STATUS_UNKNOWN, "stale": STATUS_STALE}
|
||||
AUTOCONNECT_TYPES = ["BackboneInterface", "TCPServerInterface"]
|
||||
STATUS_STALE = 0
|
||||
STATUS_UNKNOWN = 100
|
||||
STATUS_AVAILABLE = 1000
|
||||
STATUS_CODE_MAP = {"available": STATUS_AVAILABLE, "unknown": STATUS_UNKNOWN, "stale": STATUS_STALE}
|
||||
AUTOCONNECT_TYPES = ["BackboneInterface"]
|
||||
DISCOVERABLE_TYPES = ["BackboneInterface", "TCPServerInterface", "I2PInterface", "RNodeInterface", "WeaveInterface", "KISSInterface"]
|
||||
|
||||
AUTOCONNECT_IMPLS = ["RNS"]
|
||||
AUTOCONNECT_MIN_V = "1.5.7"
|
||||
|
||||
AC_TRANSPORT_MODE = RNS.Interfaces.Interface.Interface.MODE_GATEWAY
|
||||
AC_GRAVITY = 0
|
||||
|
||||
discovery_lock = Lock()
|
||||
autoconnect_lock = Lock()
|
||||
|
||||
def __init__(self, required_value=InterfaceAnnouncer.DEFAULT_STAMP_VALUE, callback=None, discover_interfaces=True):
|
||||
if not required_value: required_value = InterfaceAnnouncer.DEFAULT_STAMP_VALUE
|
||||
|
|
@ -345,30 +511,61 @@ class InterfaceDiscovery():
|
|||
self.monitoring_autoconnects = False
|
||||
self.monitor_interval = self.MONITOR_INTERVAL
|
||||
self.detach_threshold = self.DETACH_THRESHOLD
|
||||
self.initial_autoconnect_ran = False
|
||||
self.blackholed_updated = 0
|
||||
self.__blackholed = {}
|
||||
|
||||
if not self.rns_instance: raise SystemError("Attempt to start interface discovery listener without an active RNS instance")
|
||||
self.storagepath = os.path.join(RNS.Reticulum.storagepath, "discovery", "interfaces")
|
||||
if not os.path.isdir(self.storagepath): os.makedirs(self.storagepath)
|
||||
|
||||
if discover_interfaces:
|
||||
RNS.Reticulum.get_instance().wants_destination_type("rnstransport.discovery.interface")
|
||||
self.handler = InterfaceAnnounceHandler(callback=self.interface_discovered, required_value=self.required_value)
|
||||
RNS.Transport.register_announce_handler(self.handler)
|
||||
threading.Thread(target=self.connect_discovered, daemon=True).start()
|
||||
|
||||
def list_discovered_interfaces(self):
|
||||
def __blackholed_identities(self):
|
||||
st = time.time()
|
||||
if not self.__blackholed or time.time() > self.blackholed_updated+60:
|
||||
self.__blackholed = self.rns_instance.get_blackholed_identities()
|
||||
self.blackholed_updated = time.time()
|
||||
return self.__blackholed
|
||||
|
||||
def list_discovered_interfaces(self, only_available=False, only_transport=False):
|
||||
now = time.time()
|
||||
discovered_interfaces = []
|
||||
discovery_sources = RNS.Reticulum.interface_discovery_sources()
|
||||
blackholed_identities = self.__blackholed_identities()
|
||||
for filename in os.listdir(self.storagepath):
|
||||
try:
|
||||
filepath = os.path.join(self.storagepath, filename)
|
||||
with open(filepath, "rb") as f: info = msgpack.unpackb(f.read())
|
||||
with self.discovery_lock:
|
||||
filepath = os.path.join(self.storagepath, filename)
|
||||
with open(filepath, "rb") as f: info = msgpack.unpackb(f.read())
|
||||
|
||||
should_remove = False
|
||||
heard_delta = now-info["last_heard"]
|
||||
|
||||
if heard_delta > self.THRESHOLD_REMOVE: should_remove = True
|
||||
heard_delta = now-info["last_heard"]
|
||||
info["name"] = InterfaceAnnounceHandler.sanitize_name(info["name"])
|
||||
|
||||
# TODO: Temporary measure to sanitize invalidly persisted "None" values
|
||||
# for incorrect IFAC parameters published due to unguarded nonsensical
|
||||
# node-side configuration. Potentially remove once nodes upgrade to RNS
|
||||
# versions that guard against the configuration mismatch.
|
||||
try:
|
||||
if "ifac_netname" in info and info["ifac_netname"] == "None": info.pop("ifac_netname")
|
||||
if "ifac_netkey" in info and info["ifac_netkey"] == "None": info.pop("ifac_netkey")
|
||||
except Exception as e: RNS.log(f"Error while sanitizing discovery info: {e}", RNS.LOG_ERROR)
|
||||
|
||||
if heard_delta > self.THRESHOLD_REMOVE: should_remove = True
|
||||
elif not "transport_id" in info or not info["transport_id"]: should_remove = True
|
||||
elif not "network_id" in info or not info["network_id"]: should_remove = True
|
||||
elif discovery_sources and not "network_id" in info: should_remove = True
|
||||
elif discovery_sources and not bytes.fromhex(info["network_id"]) in discovery_sources: should_remove = True
|
||||
elif not "type" in info or ("type" in info and not info["type"] in self.DISCOVERABLE_TYPES): should_remove = True
|
||||
elif bytes.fromhex(info["network_id"]) in blackholed_identities: should_remove = True
|
||||
elif bytes.fromhex(info["transport_id"]) in blackholed_identities: should_remove = True
|
||||
elif "reachable_on" in info:
|
||||
if not (is_ip_address(info["reachable_on"]) or is_hostname(info["reachable_on"])): should_remove = True
|
||||
|
||||
if should_remove:
|
||||
os.unlink(filepath)
|
||||
|
|
@ -380,11 +577,18 @@ class InterfaceDiscovery():
|
|||
else: info["status"] = "available"
|
||||
|
||||
info["status_code"] = self.STATUS_CODE_MAP[info["status"]]
|
||||
discovered_interfaces.append(info)
|
||||
if not only_available and not only_transport: discovered_interfaces.append(info)
|
||||
else:
|
||||
should_append = True
|
||||
status = info["status"]
|
||||
transport = info["transport"]
|
||||
if only_available and status != "available": should_append = False
|
||||
if only_transport and not transport: should_append = False
|
||||
if should_append: discovered_interfaces.append(info)
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error while loading discovered interface data: {e}", RNS.LOG_ERROR)
|
||||
RNS.log(f"The interface data file {os.path.join(self.storagepath, filename)} may be corrupt", RNS.LOG_ERROR)
|
||||
RNS.log(f"Error while loading discovered interface data: {e}", RNS.LOG_WARNING)
|
||||
RNS.log(f"The interface data file {os.path.join(self.storagepath, filename)} may be corrupt", RNS.LOG_WARNING)
|
||||
RNS.trace_exception(e)
|
||||
|
||||
discovered_interfaces.sort(key=lambda info: (info["status_code"], info["value"], info["last_heard"]), reverse=True)
|
||||
|
|
@ -396,45 +600,53 @@ class InterfaceDiscovery():
|
|||
value = info["value"]
|
||||
interface_type = info["type"]
|
||||
discovery_hash = info["discovery_hash"]
|
||||
discovered_type = info["type"]
|
||||
has_impl_info = "impl_name" in info and "version" in info and info["impl_name"] and info["version"]
|
||||
version_str = f" ({info['impl_name']} {info['version']})" if has_impl_info else " (unknown implementation)"
|
||||
if not discovered_type in self.DISCOVERABLE_TYPES: return
|
||||
hops = info["hops"]; ms = "" if hops == 1 else "s"
|
||||
filename = RNS.hexrep(discovery_hash, delimit=False)
|
||||
filepath = os.path.join(self.storagepath, filename)
|
||||
RNS.log(f"Discovered {interface_type} {hops} hop{ms} away with stamp value {value}: {name}", RNS.LOG_DEBUG)
|
||||
if not os.path.isfile(filepath):
|
||||
try:
|
||||
with open(filepath, "wb") as f:
|
||||
info["discovered"] = info["received"]
|
||||
info["last_heard"] = info["received"]
|
||||
info["heard_count"] = 0
|
||||
f.write(msgpack.packb(info))
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error while persisting discovered interface data: {e}", RNS.LOG_ERROR)
|
||||
RNS.trace_exception(e)
|
||||
return
|
||||
RNS.log(f"Discovered {interface_type}{version_str} {hops} hop{ms} away with stamp value {RNS.prettycost(value)}: {name}", RNS.LOG_DEBUG)
|
||||
with self.discovery_lock:
|
||||
if not os.path.isfile(filepath):
|
||||
try:
|
||||
with open(filepath, "wb") as f:
|
||||
info["discovered"] = info["received"]
|
||||
info["last_heard"] = info["received"]
|
||||
info["heard_count"] = 0
|
||||
f.write(msgpack.packb(info))
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error while persisting discovered interface data: {e}", RNS.LOG_ERROR)
|
||||
RNS.trace_exception(e)
|
||||
return
|
||||
|
||||
else:
|
||||
discovered = None
|
||||
heard_count = None
|
||||
try:
|
||||
with open(filepath, "rb") as f:
|
||||
last_info = msgpack.unpackb(f.read())
|
||||
discovered = last_info["discovered"]
|
||||
heard_count = last_info["heard_count"]
|
||||
else:
|
||||
discovered = None
|
||||
heard_count = None
|
||||
try:
|
||||
try:
|
||||
with open(filepath, "rb") as f:
|
||||
last_info = msgpack.unpackb(f.read())
|
||||
discovered = last_info["discovered"]
|
||||
heard_count = last_info["heard_count"]
|
||||
|
||||
if discovered == None: discovered = info["discovered"]
|
||||
if heard_count == None: heard_count = 0
|
||||
except Exception as e: RNS.log(f"Error while reading existing data for discovered interface, re-creating data", RNS.LOG_ERROR)
|
||||
|
||||
with open(filepath, "wb") as f:
|
||||
info["discovered"] = discovered
|
||||
info["last_heard"] = info["received"]
|
||||
info["heard_count"] = heard_count+1
|
||||
f.write(msgpack.packb(info))
|
||||
if discovered == None: discovered = info["received"]
|
||||
if heard_count == None: heard_count = 0
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error while persisting discovered interface data: {e}", RNS.LOG_ERROR)
|
||||
RNS.trace_exception(e)
|
||||
return
|
||||
with open(filepath, "wb") as f:
|
||||
info["discovered"] = discovered
|
||||
info["last_heard"] = info["received"]
|
||||
info["heard_count"] = heard_count+1
|
||||
f.write(msgpack.packb(info))
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error while persisting discovered interface data: {e}", RNS.LOG_ERROR)
|
||||
RNS.trace_exception(e)
|
||||
return
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error processing discovered interface data: {e}", RNS.LOG_ERROR)
|
||||
|
|
@ -456,33 +668,43 @@ class InterfaceDiscovery():
|
|||
threading.Thread(target=self.__monitor_job, daemon=True).start()
|
||||
|
||||
def __monitor_job(self):
|
||||
while self.monitoring_autoconnects:
|
||||
while self.monitoring_autoconnects and RNS.Transport._should_run:
|
||||
time.sleep(self.monitor_interval)
|
||||
detached_interfaces = []
|
||||
online_interfaces = 0
|
||||
autoconnected_interfaces = self.autoconnect_count()
|
||||
for interface in self.monitored_interfaces:
|
||||
try:
|
||||
if interface.online:
|
||||
online_interfaces += 1
|
||||
if hasattr(interface, "autoconnect_down") and interface.autoconnect_down != None:
|
||||
RNS.log(f"Auto-discovered interface {interface} reconnected")
|
||||
interface.autoconnect_down = None
|
||||
if not interface in RNS.Transport.interfaces:
|
||||
RNS.log(f"A monitored auto-connected interface was manually detached, removing from monitoring", RNS.LOG_DEBUG)
|
||||
detached_interfaces.append(interface)
|
||||
|
||||
else:
|
||||
if not hasattr(interface, "autoconnect_down") or interface.autoconnect_down == None:
|
||||
RNS.log(f"Auto-discovered interface {interface} disconnected", RNS.LOG_DEBUG)
|
||||
interface.autoconnect_down = time.time()
|
||||
if interface.online:
|
||||
online_interfaces += 1
|
||||
if hasattr(interface, "autoconnect_down") and interface.autoconnect_down != None:
|
||||
RNS.log(f"Auto-discovered interface {interface} reconnected")
|
||||
interface.autoconnect_down = None
|
||||
|
||||
else:
|
||||
down_for = time.time()-interface.autoconnect_down
|
||||
if down_for >= self.detach_threshold:
|
||||
RNS.log(f"Auto-discovered interface {interface} has been down for {RNS.prettytime(down_for)}, detaching", RNS.LOG_DEBUG)
|
||||
detached_interfaces.append(interface)
|
||||
if not hasattr(interface, "autoconnect_down") or interface.autoconnect_down == None:
|
||||
RNS.log(f"Auto-discovered interface {interface} disconnected", RNS.LOG_DEBUG)
|
||||
interface.autoconnect_down = time.time()
|
||||
|
||||
else:
|
||||
down_for = time.time()-interface.autoconnect_down
|
||||
if down_for >= self.detach_threshold:
|
||||
RNS.log(f"Auto-discovered interface {interface} has been down for {RNS.prettytime(down_for)}, detaching", RNS.LOG_DEBUG)
|
||||
detached_interfaces.append(interface)
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error while checking auto-connected interface state for {interface}: {e}", RNS.LOG_ERROR)
|
||||
|
||||
if online_interfaces >= RNS.Reticulum.max_autoconnected_interfaces():
|
||||
max_autoconnected_interfaces = RNS.Reticulum.max_autoconnected_interfaces()
|
||||
free_slots = max(0, max_autoconnected_interfaces - autoconnected_interfaces)
|
||||
reserved_slots = max_autoconnected_interfaces//4
|
||||
|
||||
if online_interfaces >= max_autoconnected_interfaces:
|
||||
for interface in RNS.Transport.interfaces:
|
||||
if hasattr(interface, "bootstrap_only") and interface.bootstrap_only == True:
|
||||
RNS.log(f"Tearing down bootstrap-only {interface} since target connected auto-discovered interface count has been reached", RNS.LOG_INFO)
|
||||
|
|
@ -494,14 +716,23 @@ class InterfaceDiscovery():
|
|||
for config in RNS.Reticulum.get_instance().bootstrap_configs:
|
||||
RNS.Reticulum.get_instance()._synthesize_interface(config, config["name"])
|
||||
|
||||
if self.initial_autoconnect_ran and free_slots > reserved_slots:
|
||||
candidate_interfaces = self.list_discovered_interfaces(only_available=True, only_transport=True)
|
||||
if len(candidate_interfaces) > 0:
|
||||
random.shuffle(candidate_interfaces)
|
||||
selected_interface = candidate_interfaces[0]
|
||||
if not self.interface_exists(selected_interface): self.autoconnect(selected_interface)
|
||||
|
||||
for interface in detached_interfaces:
|
||||
try: self.teardown_interface(interface)
|
||||
except Exception as e:
|
||||
RNS.log(f"Error while de-registering auto-connected interface from transport: {e}", RNS.LOG_ERROR)
|
||||
|
||||
def teardown_interface(self, interface):
|
||||
interface.detach()
|
||||
if interface in RNS.Transport.interfaces: RNS.Transport.interfaces.remove(interface)
|
||||
if interface in RNS.Transport.interfaces:
|
||||
was_detached = RNS.Reticulum.get_instance()._detach_interface(interface.name, internal_forced=True)
|
||||
if not was_detached: RNS.log(f"Detach of auto-connected interface {interface} did not succeed", RNS.LOG_ERROR)
|
||||
|
||||
if interface in self.monitored_interfaces: self.monitored_interfaces.remove(interface)
|
||||
|
||||
def autoconnect_count(self):
|
||||
|
|
@ -513,73 +744,159 @@ class InterfaceDiscovery():
|
|||
def connect_discovered(self):
|
||||
if RNS.Reticulum.should_autoconnect_discovered_interfaces():
|
||||
try:
|
||||
discovered_interfaces = self.list_discovered_interfaces()
|
||||
discovered_interfaces = self.list_discovered_interfaces(only_transport=True)
|
||||
for info in discovered_interfaces:
|
||||
if self.autoconnect_count() >= RNS.Reticulum.max_autoconnected_interfaces(): break
|
||||
self.autoconnect(info)
|
||||
|
||||
self.initial_autoconnect_ran = True
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error while reconnecting discovered interfaces: {e}", RNS.LOG_ERROR)
|
||||
|
||||
def endpoint_hash(self, info):
|
||||
endpoint_specifier = ""
|
||||
if "reachable_on" in info: endpoint_specifier += str(info["reachable_on"])
|
||||
if "port" in info: endpoint_specifier += ":"+str(info["port"])
|
||||
endpoint_hash = RNS.Identity.full_hash(endpoint_specifier.encode("utf-8"))
|
||||
return endpoint_hash
|
||||
|
||||
def interface_exists(self, info):
|
||||
exists = False
|
||||
for interface in RNS.Transport.interfaces:
|
||||
if hasattr(interface, "autoconnect_hash") and interface.autoconnect_hash == self.endpoint_hash(info):
|
||||
exists = True
|
||||
break
|
||||
|
||||
else:
|
||||
dest_match = "reachable_on" in info and hasattr(interface, "target_ip") and interface.target_ip == info["reachable_on"]
|
||||
port_match = not "port" in info or (hasattr(interface, "target_port") and "port" in info and interface.target_port == info["port"])
|
||||
b32d_match = "reachable_on" in info and hasattr(interface, "b32") and interface.b32 == info["reachable_on"]
|
||||
|
||||
if (dest_match and port_match) or b32d_match:
|
||||
exists = True
|
||||
break
|
||||
|
||||
return exists
|
||||
|
||||
def autoconnect_interface_name(self, name):
|
||||
names = {interface.name for interface in RNS.Transport.interfaces}
|
||||
if not name in names: return name
|
||||
else:
|
||||
n = 2; seq_name = f"{name} ({n})"
|
||||
while seq_name in names:
|
||||
n += 1; seq_name = f"{name} ({n})"
|
||||
|
||||
return seq_name
|
||||
|
||||
def autoconnect_qualified(self, info):
|
||||
if not "transport" in info or info["transport"] != True: return False
|
||||
if RNS.Reticulum.should_autoconnect_unverified_implementations(): return True
|
||||
if not "impl_name" in info: return False
|
||||
if not info["impl_name"] in self.AUTOCONNECT_IMPLS: return False
|
||||
if not "version" in info or not info["version"]: return False
|
||||
|
||||
version = version_tuple(info["version"])
|
||||
if version == None: return False
|
||||
return version >= version_tuple(self.AUTOCONNECT_MIN_V)
|
||||
|
||||
def autoconnect(self, info):
|
||||
try:
|
||||
if RNS.Reticulum.should_autoconnect_discovered_interfaces():
|
||||
autoconnected_count = self.autoconnect_count()
|
||||
if autoconnected_count < RNS.Reticulum.max_autoconnected_interfaces():
|
||||
interface_type = info["type"]
|
||||
has_cost_info = "dst_cost" in info and "pr_cost" in info
|
||||
if interface_type in self.AUTOCONNECT_TYPES:
|
||||
endpoint_specifier = ""
|
||||
if "reachable_on" in info: endpoint_specifier += str(info["reachable_on"])
|
||||
if "port" in info: endpoint_specifier += ":"+str(info["port"])
|
||||
endpoint_hash = RNS.Identity.full_hash(endpoint_specifier.encode("utf-8"))
|
||||
exists = False
|
||||
for interface in RNS.Transport.interfaces:
|
||||
if hasattr(interface, "autoconnect_hash") and interface.autoconnect_hash == endpoint_hash:
|
||||
exists = True
|
||||
break
|
||||
|
||||
else:
|
||||
dest_match = "reachable_on" in info and hasattr(interface, "target_ip") and interface.target_ip == info["reachable_on"]
|
||||
port_match = not "port" in info or (hasattr(interface, "target_port") and "port" in info and interface.target_port == info["port"])
|
||||
b32d_match = "reachable_on" in info and hasattr(interface, "b32") and interface.b32 == info["reachable_on"]
|
||||
impl = f"{info['impl_name']} {info['version']}" if ("impl_name" in info and "version" in info and info["impl_name"] and info["version"]) else "unknown implementation"
|
||||
if not self.autoconnect_qualified(info):
|
||||
RNS.log(f"Not auto-connecting discovered {interface_type} {info['name']} ({impl}), auto-connect criteria not satisfied", RNS.LOG_DEBUG)
|
||||
return
|
||||
|
||||
if (dest_match and port_match) or b32d_match:
|
||||
exists = True
|
||||
break
|
||||
if has_cost_info:
|
||||
dst_value = RNS.Stamps.quantize_cost(RNS.Transport.DESTINATION_VALUE)
|
||||
pr_value = RNS.Stamps.quantize_cost(RNS.Transport.PR_VALUE)
|
||||
dst_cost = RNS.Stamps.quantize_cost(info["dst_cost"] or 0)
|
||||
pr_cost = RNS.Stamps.quantize_cost(info["pr_cost"] or 0)
|
||||
dst_cost_ok = dst_cost <= dst_value
|
||||
pr_cost_ok = pr_cost <= pr_value
|
||||
if not dst_cost_ok: RNS.log(f"Destination cost {dst_cost} higher than configured value {dst_value} for {interface_type} {info['name']} ({impl}), not auto-connecting", RNS.LOG_DEBUG)
|
||||
if not pr_cost_ok: RNS.log(f"Path request cost {pr_cost} higher than configured value {pr_value} for {interface_type} {info['name']} ({impl}), not auto-connecting", RNS.LOG_DEBUG)
|
||||
if not dst_cost_ok or not pr_cost_ok: return
|
||||
|
||||
endpoint_hash = self.endpoint_hash(info)
|
||||
exists = self.interface_exists(info)
|
||||
|
||||
if exists: RNS.log(f"Discovered {interface_type} already exists, not auto-connecting", RNS.LOG_DEBUG)
|
||||
else:
|
||||
if interface_type == "TCPClientInterface":
|
||||
RNS.log(f"Your operating system does not support the Backbone interface type, and must degrade to using TCPClientInterface instead", RNS.LOG_WARNING)
|
||||
RNS.log(f"Auto-connecting discovered TCPClient interfaces is not yet implemented, aborting auto-connect", RNS.LOG_WARNING)
|
||||
RNS.log(f"You can obtain the configuration entry and add this interface manually instead using rnstatus -D", RNS.LOG_WARNING)
|
||||
return
|
||||
|
||||
if interface_type == "I2PInterface":
|
||||
RNS.log(f"Auto-connecting discovered I2P interfaces is not yet implemented, aborting auto-connect", RNS.LOG_WARNING)
|
||||
RNS.log(f"You can obtain the configuration entry and add this interface manually instead using rnstatus -D", RNS.LOG_WARNING)
|
||||
RNS.log(f"Auto-connecting discovered I2P interfaces is not yet implemented, aborting auto-connect", RNS.LOG_DEBUG)
|
||||
RNS.log(f"You can obtain the configuration entry and add this interface manually instead using rnstatus -D", RNS.LOG_DEBUG)
|
||||
return
|
||||
|
||||
interface_name = info["name"]
|
||||
RNS.log(f"Auto-connecting discovered {interface_type} {interface_name}")
|
||||
config_entry = info["config_entry"]
|
||||
interface_config = {}
|
||||
interface_config["name"] = f"{interface_name}"
|
||||
ifac_netname = info["ifac_netname"] if "ifac_netname" in info else None
|
||||
ifac_netkey = info["ifac_netkey"] if "ifac_netkey" in info else None
|
||||
interface = None
|
||||
if is_ygg_ipv6(info["reachable_on"]):
|
||||
# TODO: Somehow detect if yggdrasil is enabled on the system
|
||||
return
|
||||
|
||||
if interface_type == "BackboneInterface":
|
||||
from RNS.Interfaces import BackboneInterface
|
||||
interface_config["target_host"] = info["reachable_on"]
|
||||
interface_config["target_port"] = info["port"]
|
||||
interface = BackboneInterface.BackboneClientInterface(RNS.Transport, interface_config)
|
||||
if is_onion_address(info["reachable_on"]):
|
||||
# TODO: Somehow detect if TOR is enabled on the system
|
||||
return
|
||||
|
||||
if interface:
|
||||
interface.autoconnect_hash = endpoint_hash
|
||||
interface.autoconnect_source = info["network_id"]
|
||||
RNS.Reticulum.get_instance()._add_interface(interface, ifac_netname=ifac_netname, ifac_netkey=ifac_netkey, configured_bitrate=5E6)
|
||||
self.monitor_interface(interface)
|
||||
if is_ip_address(info["reachable_on"]) and is_invalid_ip_address(info["reachable_on"]):
|
||||
RNS.log(f"Not auto-connecting discovered interface with invalid IP address: {info['reachable_on']}", RNS.LOG_DEBUG)
|
||||
return
|
||||
|
||||
with self.autoconnect_lock:
|
||||
if self.interface_exists(info): return
|
||||
backbone_support = (not RNS.vendor.platformutils.is_windows() and not RNS.vendor.platformutils.is_darwin())
|
||||
|
||||
interface_name = self.autoconnect_interface_name(info["name"])
|
||||
if interface_name != info["name"]:
|
||||
RNS.log(f"Auto-connect name collision for \"{info['name']}\", connecting as \"{interface_name}\"", RNS.LOG_NOTICE)
|
||||
|
||||
config_entry = info["config_entry"]
|
||||
interface_config = {}
|
||||
interface_config["name"] = f"{interface_name}"
|
||||
|
||||
# TODO: Temporary measure to sanitize invalidly persisted "None" values
|
||||
# for incorrect IFAC parameters published due to unguarded nonsensical
|
||||
# node-side configuration. Potentially remove once nodes upgrade to RNS
|
||||
# versions that guard against the configuration mismatch.
|
||||
if "ifac_netname" in info and info["ifac_netname"] == "None": info["ifac_netname"] = None
|
||||
if "ifac_netkey" in info and info["ifac_netkey"] == "None": info["ifac_netkey"] = None
|
||||
|
||||
ifac_netname = info["ifac_netname"] if "ifac_netname" in info and info["ifac_netname"] else None
|
||||
ifac_netkey = info["ifac_netkey"] if "ifac_netkey" in info and info["ifac_netkey"] else None
|
||||
interface = None
|
||||
|
||||
if interface_type == "BackboneInterface":
|
||||
if backbone_support:
|
||||
from RNS.Interfaces import BackboneInterface
|
||||
interface_class = BackboneInterface.BackboneClientInterface
|
||||
else:
|
||||
from RNS.Interfaces import TCPInterface
|
||||
interface_class = TCPInterface.TCPClientInterface
|
||||
RNS.log(f"BackboneInterface is not yet supported on this operating system, auto-connecting discovered {interface_type} {interface_name} using TCPClientInterface", RNS.LOG_NOTICE)
|
||||
|
||||
interface_config["target_host"] = info["reachable_on"]
|
||||
interface_config["target_port"] = info["port"]
|
||||
interface = interface_class(RNS.Transport, interface_config)
|
||||
|
||||
if interface:
|
||||
RNS.log(f"Auto-connecting discovered {interface_type} {interface_name}")
|
||||
interface.autoconnect_hash = endpoint_hash
|
||||
interface.autoconnect_source = info["network_id"]
|
||||
if RNS.Reticulum.autoconnect_interface_mode(): mode = RNS.Reticulum.autoconnect_interface_mode()
|
||||
else: mode = self.AC_TRANSPORT_MODE if RNS.Reticulum.transport_enabled() else None
|
||||
internal_a = True if RNS.Reticulum.autoconnect_announces_to_internal() else None
|
||||
gravity = RNS.Reticulum.autoconnect_interface_gravity() or self.AC_GRAVITY
|
||||
ar_target = RNS.Reticulum.get_instance()._default_ar_target() if RNS.Reticulum.transport_enabled() else None
|
||||
ar_penalty = RNS.Reticulum.get_instance()._default_ar_penalty() if RNS.Reticulum.transport_enabled() else None
|
||||
ar_grace = RNS.Reticulum.get_instance()._default_ar_grace() if RNS.Reticulum.transport_enabled() else None
|
||||
RNS.Reticulum.get_instance()._add_interface(interface, mode=mode, ifac_netname=ifac_netname, ifac_netkey=ifac_netkey, configured_bitrate=5E6,
|
||||
announce_rate_target=ar_target, announce_rate_grace=ar_grace, announce_rate_penalty=ar_penalty,
|
||||
announces_to_internal=internal_a, gravity=gravity)
|
||||
self.monitor_interface(interface)
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error while auto-connecting discovered interface: {e}", RNS.LOG_ERROR)
|
||||
|
|
@ -599,6 +916,7 @@ class BlackholeUpdater():
|
|||
|
||||
def start(self):
|
||||
if not self.should_run:
|
||||
RNS.Reticulum.get_instance().wants_destination_type("rnstransport.info.blackhole")
|
||||
source_count = len(RNS.Reticulum.blackhole_sources())
|
||||
ms = "" if source_count == 1 else "s"
|
||||
RNS.log(f"Starting blackhole updater with {source_count} source{ms}", RNS.LOG_DEBUG)
|
||||
|
|
@ -637,8 +955,9 @@ class BlackholeUpdater():
|
|||
if os.path.isfile(sourcelistpath): os.unlink(sourcelistpath)
|
||||
os.rename(tmppath, sourcelistpath)
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error while persisting blackhole list from {RNS.prettyhexrep(remote_identity.hash)}: {e}", RNS.LOG_ERROR)
|
||||
except Exception as e: RNS.log(f"Error while persisting blackhole list from {RNS.prettyhexrep(remote_identity.hash)}: {e}", RNS.LOG_ERROR)
|
||||
|
||||
RNS.Identity._reset_validation_caches()
|
||||
|
||||
RNS.log(f"Blackhole list update from {RNS.prettyhexrep(remote_identity.hash)} completed", RNS.LOG_DEBUG)
|
||||
|
||||
|
|
@ -651,7 +970,7 @@ class BlackholeUpdater():
|
|||
if identity_hash in self.last_updates: last_update = self.last_updates[identity_hash]
|
||||
else: last_update = 0
|
||||
|
||||
if now > last_update+self.UPDATE_INTERVAL:
|
||||
if now > last_update+RNS.Reticulum.blackhole_update_interval():
|
||||
try:
|
||||
destination_hash = RNS.Destination.hash_from_name_and_identity("rnstransport.info.blackhole", identity_hash)
|
||||
RNS.log(f"Attempting blackhole list update from {RNS.prettyhexrep(identity_hash)}...", RNS.LOG_DEBUG)
|
||||
|
|
@ -669,4 +988,48 @@ class BlackholeUpdater():
|
|||
RNS.log(f"Error in blackhole list updater job: {e}", RNS.LOG_ERROR)
|
||||
RNS.trace_exception(e)
|
||||
|
||||
time.sleep(self.job_interval)
|
||||
time.sleep(self.job_interval)
|
||||
|
||||
def is_ip_address(address_string):
|
||||
try:
|
||||
ipaddress.ip_address(address_string)
|
||||
return True
|
||||
except: return False
|
||||
|
||||
def is_ygg_ipv6(address_string):
|
||||
try: return ipaddress.ip_address(address_string) in ipaddress.IPv6Network("200::/7")
|
||||
except: return False
|
||||
|
||||
def is_onion_address(address_string):
|
||||
try: return address_string.lower().endswith(".onion")
|
||||
except: return False
|
||||
|
||||
INVALID_IP_ADDRESSES = ["127.0.0.1", "0.0.0.0"]
|
||||
def is_invalid_ip_address(address_string):
|
||||
if not address_string in INVALID_IP_ADDRESSES: return False
|
||||
else: return True
|
||||
|
||||
def is_hostname(hostname):
|
||||
if hostname[-1] == ".": hostname = hostname[:-1]
|
||||
if len(hostname) > 253: return False
|
||||
components = hostname.split(".")
|
||||
if re.match(r"[0-9]+$", components[-1]): return False
|
||||
allowed = re.compile(r"(?!-)[a-z0-9-]{1,63}(?<!-)$", re.IGNORECASE)
|
||||
return all(allowed.match(label) for label in components)
|
||||
|
||||
def version_tuple(version_string):
|
||||
if type(version_string) != str: return None
|
||||
try:
|
||||
components = []
|
||||
for component in version_string.strip().split("."):
|
||||
match = re.match(r"[0-9]+", component)
|
||||
if match == None: break
|
||||
components.append(int(match.group(0)))
|
||||
if len(components) == 0: return None
|
||||
return tuple(components)
|
||||
except Exception as e: return None
|
||||
|
||||
san_map = ""
|
||||
for i in range(48, 58): san_map += bytes([i]).decode("ascii")
|
||||
for i in range(65, 91): san_map += bytes([i]).decode("ascii")
|
||||
for i in range(97, 123): san_map += bytes([i]).decode("ascii")
|
||||
812
RNS/Identity.py
812
RNS/Identity.py
File diff suppressed because it is too large
Load diff
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -128,7 +128,9 @@ class AX25KISSInterface(Interface):
|
|||
|
||||
self.packet_queue = []
|
||||
self.flow_control = flow_control
|
||||
self.shared_medium = True
|
||||
self.interface_ready = False
|
||||
self.reconnecting = False
|
||||
self.flow_control_timeout = 5
|
||||
self.flow_control_locked = time.time()
|
||||
|
||||
|
|
@ -151,14 +153,18 @@ class AX25KISSInterface(Interface):
|
|||
|
||||
try:
|
||||
self.open_port()
|
||||
if self.serial.is_open: self.configure_device()
|
||||
else: raise IOError("Could not open serial port")
|
||||
|
||||
except Exception as e:
|
||||
RNS.log("Could not open serial port for interface "+str(self), RNS.LOG_ERROR)
|
||||
raise e
|
||||
RNS.log(f"Could not open serial port for {self}", RNS.LOG_ERROR)
|
||||
RNS.log(f"The contained exception was: {e}", RNS.LOG_ERROR)
|
||||
RNS.log(f"Reticulum will attempt to bring up this interface periodically", RNS.LOG_ERROR)
|
||||
if not self.detached and not self.reconnecting:
|
||||
thread = threading.Thread(target=self.reconnect_port)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
if self.serial.is_open:
|
||||
self.configure_device()
|
||||
else:
|
||||
raise IOError("Could not open serial port")
|
||||
|
||||
def open_port(self):
|
||||
RNS.log("Opening serial port "+self.port+"...", RNS.LOG_VERBOSE)
|
||||
|
|
@ -176,6 +182,12 @@ class AX25KISSInterface(Interface):
|
|||
dsrdtr = False,
|
||||
)
|
||||
|
||||
def detach(self):
|
||||
self.detached = True
|
||||
self.online = False
|
||||
try: self.serial.close()
|
||||
except Exception as e: RNS.log(f"Could not close port for {self} at detach: {e}", RNS.LOG_ERROR)
|
||||
|
||||
def configure_device(self):
|
||||
# Allow time for interface to initialise before config
|
||||
sleep(2.0)
|
||||
|
|
@ -254,9 +266,7 @@ class AX25KISSInterface(Interface):
|
|||
|
||||
|
||||
def process_incoming(self, data):
|
||||
if (len(data) > AX25.HEADER_SIZE):
|
||||
self.rxb += len(data)
|
||||
self.owner.inbound(data[AX25.HEADER_SIZE:], self)
|
||||
if (len(data) > AX25.HEADER_SIZE): self.owner.inbound(data[AX25.HEADER_SIZE:], self)
|
||||
|
||||
|
||||
def process_outgoing(self,data):
|
||||
|
|
@ -372,30 +382,28 @@ class AX25KISSInterface(Interface):
|
|||
RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
|
||||
|
||||
if RNS.Reticulum.panic_on_interface_error:
|
||||
RNS.panic()
|
||||
if RNS.Reticulum.panic_on_interface_error: RNS.panic()
|
||||
|
||||
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
|
||||
|
||||
self.online = False
|
||||
self.serial.close()
|
||||
self.reconnect_port()
|
||||
try: self.serial.close()
|
||||
except Exception as e: RNS.log(f"Could not close port for {self}", RNS.LOG_ERROR)
|
||||
if not self.detached and not self.reconnecting: self.reconnect_port()
|
||||
|
||||
def reconnect_port(self):
|
||||
while not self.online:
|
||||
self.reconnecting = True
|
||||
while not self.online and not self.detached:
|
||||
try:
|
||||
time.sleep(5)
|
||||
RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
|
||||
self.open_port()
|
||||
if self.serial.is_open:
|
||||
self.configure_device()
|
||||
except Exception as e:
|
||||
RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
if self.serial.is_open: self.configure_device()
|
||||
except Exception as e: RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
|
||||
RNS.log("Reconnected serial port for "+str(self))
|
||||
self.reconnecting = False
|
||||
if self.online: RNS.log("Reconnected serial port for "+str(self))
|
||||
|
||||
def should_ingress_limit(self):
|
||||
return False
|
||||
def should_ingress_limit(self): return False
|
||||
|
||||
def __str__(self):
|
||||
return "AX25KISSInterface["+self.name+"]"
|
||||
def __str__(self): return "AX25KISSInterface["+self.name+"]"
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -128,9 +128,11 @@ class KISSInterface(Interface):
|
|||
self.beacon_d = beacon_data.encode("utf-8")
|
||||
self.first_tx = None
|
||||
self.bitrate = KISSInterface.BITRATE_GUESS
|
||||
self.reconnecting = False
|
||||
|
||||
self.packet_queue = []
|
||||
self.flow_control = flow_control
|
||||
self.shared_medium = True
|
||||
self.interface_ready = False
|
||||
self.flow_control_timeout = 5
|
||||
self.flow_control_locked = time.time()
|
||||
|
|
@ -148,15 +150,23 @@ class KISSInterface(Interface):
|
|||
|
||||
try:
|
||||
self.open_port()
|
||||
if self.serial.is_open: self.configure_device()
|
||||
else: raise IOError("Could not open serial port")
|
||||
|
||||
except Exception as e:
|
||||
RNS.log("Could not open serial port "+self.port, RNS.LOG_ERROR)
|
||||
raise e
|
||||
|
||||
if self.serial.is_open:
|
||||
self.configure_device()
|
||||
else:
|
||||
raise IOError("Could not open serial port")
|
||||
RNS.log(f"Could not open serial port for {self}", RNS.LOG_ERROR)
|
||||
RNS.log(f"The contained exception was: {e}", RNS.LOG_ERROR)
|
||||
RNS.log(f"Reticulum will attempt to bring up this interface periodically", RNS.LOG_ERROR)
|
||||
if not self.detached and not self.reconnecting:
|
||||
thread = threading.Thread(target=self.reconnect_port)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
def detach(self):
|
||||
self.detached = True
|
||||
self.online = False
|
||||
try: self.serial.close()
|
||||
except Exception as e: RNS.log(f"Could not close port for {self} at detach: {e}", RNS.LOG_ERROR)
|
||||
|
||||
def open_port(self):
|
||||
RNS.log("Opening serial port "+self.port+"...")
|
||||
|
|
@ -416,24 +426,23 @@ class KISSInterface(Interface):
|
|||
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
|
||||
|
||||
self.online = False
|
||||
self.serial.close()
|
||||
self.reconnect_port()
|
||||
try: self.serial.close()
|
||||
except Exception as e: RNS.log(f"Could not close port for {self}", RNS.LOG_ERROR)
|
||||
if not self.detached and not self.reconnecting: self.reconnect_port()
|
||||
|
||||
def reconnect_port(self):
|
||||
self.reconnecting = True
|
||||
while not self.online:
|
||||
try:
|
||||
time.sleep(5)
|
||||
RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
|
||||
self.open_port()
|
||||
if self.serial.is_open:
|
||||
self.configure_device()
|
||||
except Exception as e:
|
||||
RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
if self.serial.is_open: self.configure_device()
|
||||
except Exception as e: RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
|
||||
RNS.log("Reconnected serial port for "+str(self))
|
||||
self.reconnecting = False
|
||||
if self.online: RNS.log("Reconnected serial port for "+str(self))
|
||||
|
||||
def should_ingress_limit(self):
|
||||
return False
|
||||
def should_ingress_limit(self): return False
|
||||
|
||||
def __str__(self):
|
||||
return "KISSInterface["+self.name+"]"
|
||||
def __str__(self): return "KISSInterface["+self.name+"]"
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -282,12 +282,9 @@ class RNodeInterface(Interface):
|
|||
if (port != None or device_serial != None) and (enable_bluetooth or disable_bluetooth or pairing_mode):
|
||||
serial = None
|
||||
bluetooth_state = None
|
||||
if pairing_mode:
|
||||
bluetooth_state = 0x01
|
||||
elif enable_bluetooth:
|
||||
bluetooth_state = 0x01
|
||||
elif disable_bluetooth:
|
||||
bluetooth_state = 0x00
|
||||
if pairing_mode: bluetooth_state = 0x01
|
||||
elif enable_bluetooth: bluetooth_state = 0x01
|
||||
elif disable_bluetooth: bluetooth_state = 0x00
|
||||
|
||||
if port != None:
|
||||
RNS.log("Opening serial port "+port+"...")
|
||||
|
|
@ -503,6 +500,7 @@ class RNodeInterface(Interface):
|
|||
self.r_interference = None
|
||||
self.r_interference_l = None
|
||||
self.r_temperature = None
|
||||
self.reports_phy_stats = True
|
||||
|
||||
self.r_battery_state = RNodeInterface.BATTERY_STATE_UNKNOWN
|
||||
self.r_battery_percent = 0
|
||||
|
|
@ -518,6 +516,7 @@ class RNodeInterface(Interface):
|
|||
|
||||
self.packet_queue = []
|
||||
self.flow_control = flow_control
|
||||
self.shared_medium = True
|
||||
self.interface_ready = False
|
||||
self.announce_rate_target = None
|
||||
self.last_port_io = 0
|
||||
|
|
@ -1506,13 +1505,13 @@ class RNodeInterface(Interface):
|
|||
|
||||
except Exception as e:
|
||||
self.online = False
|
||||
RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
|
||||
if not self.detached:
|
||||
RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
|
||||
|
||||
if RNS.Reticulum.panic_on_interface_error:
|
||||
RNS.panic()
|
||||
if RNS.Reticulum.panic_on_interface_error: RNS.panic()
|
||||
|
||||
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
|
||||
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
|
||||
|
||||
self.online = False
|
||||
|
||||
|
|
@ -1572,10 +1571,14 @@ class RNodeInterface(Interface):
|
|||
except Exception as e:
|
||||
RNS.log(f"An error occurred while detaching {self}: {e}", RNS.LOG_ERROR)
|
||||
|
||||
if self.use_ble: self.ble.close()
|
||||
if self.use_tcp:
|
||||
if self.use_ble:
|
||||
self.ble.close()
|
||||
self.ble.cleanup()
|
||||
elif self.use_tcp:
|
||||
time.sleep(0.5)
|
||||
self.tcp.close()
|
||||
else:
|
||||
self.serial.close()
|
||||
|
||||
def should_ingress_limit(self):
|
||||
return False
|
||||
|
|
@ -1654,6 +1657,9 @@ class BLEConnection(BluetoothDispatcher):
|
|||
self.owner.ble_rx_queue = b""
|
||||
return data
|
||||
|
||||
def cleanup(self):
|
||||
self.should_run = False
|
||||
|
||||
def close(self):
|
||||
try:
|
||||
if self.connected:
|
||||
|
|
@ -1742,7 +1748,7 @@ class BLEConnection(BluetoothDispatcher):
|
|||
ble_devices = self.find_target_devices()
|
||||
|
||||
if len(ble_devices) > 0: self.ble_device = ble_devices.pop()
|
||||
else: self.ble_device == None
|
||||
else: self.ble_device = None
|
||||
|
||||
if self.ble_device != None:
|
||||
if self.was_connected:
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -95,6 +95,7 @@ class SerialInterface(Interface):
|
|||
raise ValueError("No port specified for serial interface")
|
||||
|
||||
self.HW_MTU = 564
|
||||
self.shared_medium = True
|
||||
|
||||
self.pyserial = serial
|
||||
self.serial = None
|
||||
|
|
@ -108,6 +109,7 @@ class SerialInterface(Interface):
|
|||
self.timeout = 100
|
||||
self.online = False
|
||||
self.bitrate = self.speed
|
||||
self.reconnecting = False
|
||||
|
||||
if parity.lower() == "e" or parity.lower() == "even":
|
||||
self.parity = "E"
|
||||
|
|
@ -117,15 +119,23 @@ class SerialInterface(Interface):
|
|||
|
||||
try:
|
||||
self.open_port()
|
||||
if self.serial.is_open: self.configure_device()
|
||||
else: raise IOError("Could not open serial port")
|
||||
|
||||
except Exception as e:
|
||||
RNS.log("Could not open serial port for interface "+str(self), RNS.LOG_ERROR)
|
||||
raise e
|
||||
|
||||
if self.serial.is_open:
|
||||
self.configure_device()
|
||||
else:
|
||||
raise IOError("Could not open serial port")
|
||||
RNS.log(f"Could not open serial port for {self}", RNS.LOG_ERROR)
|
||||
RNS.log(f"The contained exception was: {e}", RNS.LOG_ERROR)
|
||||
RNS.log(f"Reticulum will attempt to bring up this interface periodically", RNS.LOG_ERROR)
|
||||
if not self.detached and not self.reconnecting:
|
||||
thread = threading.Thread(target=self.reconnect_port)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
def detach(self):
|
||||
self.detached = True
|
||||
self.online = False
|
||||
try: self.serial.close()
|
||||
except Exception as e: RNS.log(f"Could not close port for {self} at detach: {e}", RNS.LOG_ERROR)
|
||||
|
||||
def open_port(self):
|
||||
RNS.log("Opening serial port "+self.port+"...")
|
||||
|
|
@ -257,24 +267,23 @@ class SerialInterface(Interface):
|
|||
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
|
||||
|
||||
self.online = False
|
||||
self.serial.close()
|
||||
self.reconnect_port()
|
||||
try: self.serial.close()
|
||||
except Exception as e: RNS.log(f"Could not close port for {self}", RNS.LOG_ERROR)
|
||||
if not self.detached and not self.reconnecting: self.reconnect_port()
|
||||
|
||||
def reconnect_port(self):
|
||||
while not self.online:
|
||||
self.reconnecting = True
|
||||
while not self.online and not self.detached:
|
||||
try:
|
||||
time.sleep(5)
|
||||
RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
|
||||
self.open_port()
|
||||
if self.serial.is_open:
|
||||
self.configure_device()
|
||||
except Exception as e:
|
||||
RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
if self.serial.is_open: self.configure_device()
|
||||
except Exception as e: RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
|
||||
RNS.log("Reconnected serial port for "+str(self))
|
||||
self.reconnecting = False
|
||||
if self.online: RNS.log("Reconnected serial port for "+str(self))
|
||||
|
||||
def should_ingress_limit(self):
|
||||
return False
|
||||
def should_ingress_limit(self): return False
|
||||
|
||||
def __str__(self):
|
||||
return "SerialInterface["+self.name+"]"
|
||||
def __str__(self): return "SerialInterface["+self.name+"]"
|
||||
|
|
|
|||
|
|
@ -33,5 +33,7 @@ import glob
|
|||
|
||||
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
|
||||
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
|
||||
modules = py_modules+pyc_modules
|
||||
__all__ = list(set([os.path.basename(f).replace(".pyc", "").replace(".py", "") for f in modules if not (f.endswith("__init__.py") or f.endswith("__init__.pyc"))]))
|
||||
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
|
||||
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
|
||||
modules = py_modules+pyc_modules+so_modules+pyd_modules
|
||||
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -65,7 +65,7 @@ class AutoInterface(Interface):
|
|||
|
||||
ALL_IGNORE_IFS = ["lo0"]
|
||||
DARWIN_IGNORE_IFS = ["awdl0", "llw0", "lo0", "en5"]
|
||||
ANDROID_IGNORE_IFS = ["dummy0", "lo", "tun0"]
|
||||
ANDROID_IGNORE_IFS = ["dummy0", "lo", "tun0", "rmnet0", "rmnet1", "rmnet2", "rmnet3", "rmnet4", "rmnet5", "rmnet6", "rmnet7"]
|
||||
|
||||
BITRATE_GUESS = 10*1000*1000
|
||||
|
||||
|
|
@ -127,6 +127,7 @@ class AutoInterface(Interface):
|
|||
self.link_local_addresses = []
|
||||
self.adopted_interfaces = {}
|
||||
self.interface_servers = {}
|
||||
self.discovery_sockets = []
|
||||
self.multicast_echoes = {}
|
||||
self.initial_echoes = {}
|
||||
self.timed_out_interfaces = {}
|
||||
|
|
@ -268,6 +269,8 @@ class AutoInterface(Interface):
|
|||
addr_info = socket.getaddrinfo(link_local_addr+"%"+ifname, self.unicast_discovery_port, socket.AF_INET6, socket.SOCK_DGRAM)
|
||||
unicast_discovery_socket.bind(addr_info[0][4])
|
||||
|
||||
self.discovery_sockets.append(unicast_discovery_socket)
|
||||
|
||||
mcast_addr = self.mcast_discovery_address
|
||||
RNS.log(str(self)+" Creating multicast discovery listener on "+str(ifname)+" with address "+str(mcast_addr), RNS.LOG_EXTREME)
|
||||
|
||||
|
|
@ -296,6 +299,8 @@ class AutoInterface(Interface):
|
|||
|
||||
discovery_socket.bind(addr_info[0][4])
|
||||
|
||||
self.discovery_sockets.append(discovery_socket)
|
||||
|
||||
# Set up thread for multicast discovery packets
|
||||
def discovery_loop(): self.discovery_handler(discovery_socket, ifname)
|
||||
thread = threading.Thread(target=discovery_loop, daemon=True).start()
|
||||
|
|
@ -358,9 +363,9 @@ class AutoInterface(Interface):
|
|||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
while True:
|
||||
while not self.detached:
|
||||
data, ipv6_src = socket.recvfrom(1024)
|
||||
if self.final_init_done:
|
||||
if self.final_init_done and data and ipv6_src:
|
||||
peering_hash = data[:RNS.Identity.HASHLENGTH//8]
|
||||
expected_hash = RNS.Identity.full_hash(self.group_id+ipv6_src[0].encode("utf-8"))
|
||||
if peering_hash == expected_hash:
|
||||
|
|
@ -369,7 +374,7 @@ class AutoInterface(Interface):
|
|||
RNS.log(str(self)+" received peering packet on "+str(ifname)+" from "+str(ipv6_src[0])+", but authentication hash was incorrect.", RNS.LOG_DEBUG)
|
||||
|
||||
def peer_jobs(self):
|
||||
while True:
|
||||
while not self.detached:
|
||||
time.sleep(self.peer_job_interval)
|
||||
now = time.time()
|
||||
timed_out_peers = []
|
||||
|
|
@ -428,14 +433,22 @@ class AutoInterface(Interface):
|
|||
if ifname in self.interface_servers:
|
||||
RNS.log("Shutting down previous UDP listener for "+str(self)+" "+str(ifname), RNS.LOG_DEBUG)
|
||||
previous_server = self.interface_servers[ifname]
|
||||
def shutdown_server():
|
||||
previous_server.shutdown()
|
||||
def shutdown_server(): previous_server.shutdown()
|
||||
threading.Thread(target=shutdown_server, daemon=True).start()
|
||||
|
||||
RNS.log("Starting new UDP listener for "+str(self)+" "+str(ifname), RNS.LOG_DEBUG)
|
||||
|
||||
udp_server = socketserver.UDPServer(listen_address, self.handler_factory(self.process_incoming))
|
||||
self.interface_servers[ifname] = udp_server
|
||||
retry_delay = 1.25
|
||||
listener_started = False
|
||||
while not listener_started:
|
||||
try:
|
||||
time.sleep(retry_delay)
|
||||
udp_server = socketserver.UDPServer(listen_address, self.handler_factory(self.process_incoming))
|
||||
self.interface_servers[ifname] = udp_server
|
||||
listener_started = True
|
||||
except Exception as e:
|
||||
RNS.log(f"Could not start new UDP listener for {self} on {listen_address}: {e}", RNS.LOG_WARNING)
|
||||
RNS.log(f"Retrying in {retry_delay} seconds", RNS.LOG_WARNING)
|
||||
|
||||
thread = threading.Thread(target=udp_server.serve_forever)
|
||||
thread.daemon = True
|
||||
|
|
@ -470,7 +483,7 @@ class AutoInterface(Interface):
|
|||
|
||||
|
||||
def announce_handler(self, ifname):
|
||||
while True:
|
||||
while not self.detached:
|
||||
self.peer_announce(ifname)
|
||||
time.sleep(self.announce_interval)
|
||||
|
||||
|
|
@ -530,6 +543,24 @@ class AutoInterface(Interface):
|
|||
spawned_interface = AutoInterfacePeer(self, addr, ifname)
|
||||
spawned_interface.OUT = self.OUT
|
||||
spawned_interface.IN = self.IN
|
||||
|
||||
spawned_interface.ingress_control = self.ingress_control
|
||||
spawned_interface.ic_max_held_announces = self.ic_max_held_announces
|
||||
spawned_interface.ic_burst_hold = self.ic_burst_hold
|
||||
spawned_interface.ic_burst_freq = self.ic_burst_freq
|
||||
spawned_interface.ic_burst_freq_new = self.ic_burst_freq_new
|
||||
spawned_interface.ic_new_time = self.ic_new_time
|
||||
spawned_interface.ic_burst_penalty = self.ic_burst_penalty
|
||||
spawned_interface.ic_held_release_interval = self.ic_held_release_interval
|
||||
|
||||
spawned_interface.egress_control = self.egress_control
|
||||
spawned_interface.ec_pr_freq = self.ec_pr_freq
|
||||
spawned_interface.ic_pr_burst_freq_new = self.ic_pr_burst_freq_new
|
||||
spawned_interface.ic_pr_burst_freq = self.ic_pr_burst_freq
|
||||
|
||||
spawned_interface._destination_cost = self._destination_cost
|
||||
spawned_interface._pr_cost = self._pr_cost
|
||||
|
||||
spawned_interface.parent_interface = self
|
||||
spawned_interface.bitrate = self.bitrate
|
||||
|
||||
|
|
@ -557,9 +588,14 @@ class AutoInterface(Interface):
|
|||
spawned_interface.announce_rate_grace = self.announce_rate_grace
|
||||
spawned_interface.announce_rate_penalty = self.announce_rate_penalty
|
||||
spawned_interface.mode = self.mode
|
||||
spawned_interface.gravity = self.gravity
|
||||
spawned_interface.recursive_prs = self.recursive_prs
|
||||
spawned_interface.announces_from_internal = self.announces_from_internal
|
||||
spawned_interface.announces_to_internal = self.announces_to_internal
|
||||
spawned_interface.announce_cap = self.announce_cap
|
||||
spawned_interface.HW_MTU = self.HW_MTU
|
||||
spawned_interface.online = True
|
||||
RNS.Transport.interfaces.append(spawned_interface)
|
||||
RNS.Transport.add_interface(spawned_interface)
|
||||
if addr in self.spawned_interfaces:
|
||||
self.spawned_interfaces[addr].detach()
|
||||
self.spawned_interfaces[addr].teardown()
|
||||
|
|
@ -580,7 +616,20 @@ class AutoInterface(Interface):
|
|||
|
||||
def process_outgoing(self, data): pass
|
||||
|
||||
def detach(self): self.online = False
|
||||
def detach(self):
|
||||
self.online = False
|
||||
self.detached = True
|
||||
|
||||
RNS.log(f"Shutting down all UDP listeners for {self}", RNS.LOG_NOTICE)
|
||||
for ifname in self.interface_servers:
|
||||
try:
|
||||
self.interface_servers[ifname].server_close()
|
||||
self.interface_servers[ifname].shutdown()
|
||||
except Exception as e: RNS.log(f"Could not shut down UDP listener {ifname} for {self}: {e}", RNS.LOG_ERROR)
|
||||
|
||||
for s in self.discovery_sockets:
|
||||
try: s.close()
|
||||
except Exception as e: RNS.log(f"Error while shutting down discovery socket {s} for {self}: {e}", RNS.LOG_ERROR)
|
||||
|
||||
def __str__(self): return f"AutoInterface[{self.name}]"
|
||||
|
||||
|
|
@ -614,8 +663,6 @@ class AutoInterfacePeer(Interface):
|
|||
self.owner.refresh_peer(self.addr)
|
||||
self.owner.mif_deque.append(data_hash)
|
||||
self.owner.mif_deque_times.append([data_hash, time.time()])
|
||||
self.rxb += len(data)
|
||||
self.owner.rxb += len(data)
|
||||
self.owner.owner.inbound(data, self)
|
||||
|
||||
def process_outgoing(self, data):
|
||||
|
|
@ -651,7 +698,7 @@ class AutoInterfacePeer(Interface):
|
|||
except Exception as e:
|
||||
RNS.log(f"Could not remove {self} from parent interface on detach. The contained exception was: {e}", RNS.LOG_ERROR)
|
||||
|
||||
if self in RNS.Transport.interfaces: RNS.Transport.interfaces.remove(self)
|
||||
RNS.Transport.remove_interface(self)
|
||||
|
||||
class AutoInterfaceHandler(socketserver.BaseRequestHandler):
|
||||
def __init__(self, callback, *args, **keys):
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -88,8 +88,7 @@ class I2PController:
|
|||
self.ready = False
|
||||
|
||||
self.storagepath = rns_storagepath+"/i2p"
|
||||
if not os.path.isdir(self.storagepath):
|
||||
os.makedirs(self.storagepath)
|
||||
if not os.path.isdir(self.storagepath): os.makedirs(self.storagepath)
|
||||
|
||||
|
||||
def start(self):
|
||||
|
|
@ -110,28 +109,22 @@ class I2PController:
|
|||
except Exception as e:
|
||||
self.ready = False
|
||||
RNS.log("Exception on event loop for "+str(self)+": "+str(e), RNS.LOG_ERROR)
|
||||
finally:
|
||||
self.loop.close()
|
||||
finally: self.loop.close()
|
||||
|
||||
|
||||
def stop(self):
|
||||
for i2ptunnel in self.i2plib_tunnels:
|
||||
if hasattr(i2ptunnel, "stop") and callable(i2ptunnel.stop):
|
||||
i2ptunnel.stop()
|
||||
if hasattr(i2ptunnel, "stop") and callable(i2ptunnel.stop): i2ptunnel.stop()
|
||||
|
||||
if hasattr(asyncio.Task, "all_tasks") and callable(asyncio.Task.all_tasks):
|
||||
for task in asyncio.Task.all_tasks(loop=self.loop):
|
||||
task.cancel()
|
||||
for task in asyncio.Task.all_tasks(loop=self.loop): task.cancel()
|
||||
|
||||
time.sleep(0.2)
|
||||
|
||||
self.loop.stop()
|
||||
|
||||
|
||||
def get_free_port(self):
|
||||
return self.i2plib.utils.get_free_port()
|
||||
|
||||
|
||||
def stop_tunnel(self, i2ptunnel):
|
||||
if hasattr(i2ptunnel, "stop") and callable(i2ptunnel.stop):
|
||||
i2ptunnel.stop()
|
||||
|
|
@ -159,10 +152,9 @@ class I2PController:
|
|||
tn = self.i2plib_tunnels[i2p_destination]
|
||||
if tn != None and hasattr(tn, "status"):
|
||||
|
||||
RNS.log("Waiting for status from I2P control process", RNS.LOG_EXTREME)
|
||||
while not tn.status["setup_ran"]:
|
||||
time.sleep(0.1)
|
||||
RNS.log("Got status from I2P control process", RNS.LOG_EXTREME)
|
||||
RNS.log("Waiting for status from I2P control process", RNS.LOG_DEBUG)
|
||||
while not tn.status["setup_ran"]: time.sleep(0.1)
|
||||
RNS.log("Got status from I2P control process", RNS.LOG_DEBUG)
|
||||
|
||||
if tn.status["setup_failed"]:
|
||||
self.stop_tunnel(tn)
|
||||
|
|
@ -172,29 +164,19 @@ class I2PController:
|
|||
if owner.socket != None:
|
||||
if hasattr(owner.socket, "close"):
|
||||
if callable(owner.socket.close):
|
||||
try:
|
||||
owner.socket.shutdown(socket.SHUT_RDWR)
|
||||
except Exception as e:
|
||||
RNS.log("Error while shutting down socket for "+str(owner)+": "+str(e))
|
||||
try: owner.socket.shutdown(socket.SHUT_RDWR)
|
||||
except Exception as e: RNS.log("Error while shutting down socket for "+str(owner)+": "+str(e))
|
||||
try: owner.socket.close()
|
||||
except Exception as e: RNS.log("Error while closing socket for "+str(owner)+": "+str(e))
|
||||
|
||||
try:
|
||||
owner.socket.close()
|
||||
except Exception as e:
|
||||
RNS.log("Error while closing socket for "+str(owner)+": "+str(e))
|
||||
self.client_tunnels[i2p_destination] = True
|
||||
owner.awaiting_i2p_tunnel = False
|
||||
|
||||
RNS.log(str(owner)+" tunnel setup complete", RNS.LOG_VERBOSE)
|
||||
|
||||
else:
|
||||
raise IOError("Got no status response from SAM API")
|
||||
|
||||
except ConnectionRefusedError as e:
|
||||
raise e
|
||||
|
||||
except ConnectionAbortedError as e:
|
||||
raise e
|
||||
else: raise IOError("Got no status response from SAM API")
|
||||
|
||||
except ConnectionRefusedError as e: raise e
|
||||
except ConnectionAbortedError as e: raise e
|
||||
except Exception as e:
|
||||
RNS.log("Unexpected error type from I2P SAM: "+str(e), RNS.LOG_ERROR)
|
||||
raise e
|
||||
|
|
@ -206,54 +188,33 @@ class I2PController:
|
|||
|
||||
if i2ptunnel.status["setup_ran"] == False:
|
||||
RNS.log(str(self)+" I2P tunnel setup did not complete", RNS.LOG_ERROR)
|
||||
|
||||
self.stop_tunnel(i2ptunnel)
|
||||
return False
|
||||
|
||||
elif i2p_exception != None:
|
||||
RNS.log("An error ocurred while setting up I2P tunnel to "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
|
||||
if isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.CantReachPeer):
|
||||
RNS.log("The I2P daemon can't reach peer "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedDest):
|
||||
RNS.log("The I2P daemon reported that the destination is already in use", RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedId):
|
||||
RNS.log("The I2P daemon reported that the ID is arleady in use", RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidId):
|
||||
RNS.log("The I2P daemon reported that the stream session ID doesn't exist", RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidKey):
|
||||
RNS.log("The I2P daemon reported that the key for "+str(i2p_destination)+" is invalid", RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.KeyNotFound):
|
||||
RNS.log("The I2P daemon could not find the key for "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.PeerNotFound):
|
||||
RNS.log("The I2P daemon mould not find the peer "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.I2PError):
|
||||
RNS.log("The I2P daemon experienced an unspecified error", RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.Timeout):
|
||||
RNS.log("I2P daemon timed out while setting up client tunnel to "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
if isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.CantReachPeer): RNS.log("The I2P daemon can't reach peer "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedDest): RNS.log("The I2P daemon reported that the destination is already in use", RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedId): RNS.log("The I2P daemon reported that the ID is arleady in use", RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidId): RNS.log("The I2P daemon reported that the stream session ID doesn't exist", RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidKey): RNS.log("The I2P daemon reported that the key for "+str(i2p_destination)+" is invalid", RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.KeyNotFound): RNS.log("The I2P daemon could not find the key for "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.PeerNotFound): RNS.log("The I2P daemon could not find the peer "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.I2PError): RNS.log("The I2P daemon experienced an unspecified error", RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.Timeout): RNS.log("I2P daemon timed out while setting up client tunnel to "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
else: RNS.log(f"Unspecified I2P daemon error: {i2p_exception}", RNS.LOG_ERROR)
|
||||
|
||||
RNS.log("Resetting I2P tunnel and retrying later", RNS.LOG_ERROR)
|
||||
|
||||
self.stop_tunnel(i2ptunnel)
|
||||
return False
|
||||
|
||||
elif i2ptunnel.status["setup_failed"] == True:
|
||||
RNS.log(str(self)+" Unspecified I2P tunnel setup error, resetting I2P tunnel", RNS.LOG_ERROR)
|
||||
|
||||
self.stop_tunnel(i2ptunnel)
|
||||
return False
|
||||
|
||||
else:
|
||||
RNS.log(str(self)+" Got no status from SAM API, resetting I2P tunnel", RNS.LOG_ERROR)
|
||||
|
||||
self.stop_tunnel(i2ptunnel)
|
||||
return False
|
||||
|
||||
|
|
@ -274,10 +235,8 @@ class I2PController:
|
|||
i2p_keyfile_nf = self.storagepath+"/"+RNS.hexrep(i2p_dest_hash_nf, delimit=False)+".i2p"
|
||||
|
||||
# Use old format if a key is already present
|
||||
if os.path.isfile(i2p_keyfile_of):
|
||||
i2p_keyfile = i2p_keyfile_of
|
||||
else:
|
||||
i2p_keyfile = i2p_keyfile_nf
|
||||
if os.path.isfile(i2p_keyfile_of): i2p_keyfile = i2p_keyfile_of
|
||||
else: i2p_keyfile = i2p_keyfile_nf
|
||||
|
||||
i2p_dest = None
|
||||
if not os.path.isfile(i2p_keyfile):
|
||||
|
|
@ -312,8 +271,7 @@ class I2PController:
|
|||
asyncio.run_coroutine_threadsafe(tunnel_up(), self.loop).result()
|
||||
self.server_tunnels[i2p_b32] = True
|
||||
|
||||
except Exception as e:
|
||||
raise e
|
||||
except Exception as e: raise e
|
||||
|
||||
else:
|
||||
i2ptunnel = self.i2plib_tunnels[i2p_b32]
|
||||
|
|
@ -322,66 +280,45 @@ class I2PController:
|
|||
|
||||
if i2ptunnel.status["setup_ran"] == False:
|
||||
RNS.log(str(self)+" I2P tunnel setup did not complete", RNS.LOG_ERROR)
|
||||
|
||||
self.stop_tunnel(i2ptunnel)
|
||||
return False
|
||||
|
||||
elif i2p_exception != None:
|
||||
RNS.log("An error ocurred while setting up I2P tunnel", RNS.LOG_ERROR)
|
||||
|
||||
if isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.CantReachPeer):
|
||||
RNS.log("The I2P daemon can't reach peer "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedDest):
|
||||
RNS.log("The I2P daemon reported that the destination is already in use", RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedId):
|
||||
RNS.log("The I2P daemon reported that the ID is arleady in use", RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidId):
|
||||
RNS.log("The I2P daemon reported that the stream session ID doesn't exist", RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidKey):
|
||||
RNS.log("The I2P daemon reported that the key for "+str(i2p_destination)+" is invalid", RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.KeyNotFound):
|
||||
RNS.log("The I2P daemon could not find the key for "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.PeerNotFound):
|
||||
RNS.log("The I2P daemon mould not find the peer "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.I2PError):
|
||||
RNS.log("The I2P daemon experienced an unspecified error", RNS.LOG_ERROR)
|
||||
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.Timeout):
|
||||
RNS.log("I2P daemon timed out while setting up client tunnel to "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
if isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.CantReachPeer): RNS.log("The I2P daemon can't reach peer "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedDest): RNS.log("The I2P daemon reported that the destination is already in use", RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedId): RNS.log("The I2P daemon reported that the ID is arleady in use", RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidId): RNS.log("The I2P daemon reported that the stream session ID doesn't exist", RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidKey): RNS.log("The I2P daemon reported that the key for "+str(i2p_destination)+" is invalid", RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.KeyNotFound): RNS.log("The I2P daemon could not find the key for "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.PeerNotFound): RNS.log("The I2P daemon could not find the peer "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.I2PError): RNS.log("The I2P daemon experienced an unspecified error", RNS.LOG_ERROR)
|
||||
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.Timeout): RNS.log("I2P daemon timed out while setting up client tunnel to "+str(i2p_destination), RNS.LOG_ERROR)
|
||||
else: RNS.log(f"Unspecified I2P daemon error: {i2p_exception}", RNS.LOG_ERROR)
|
||||
|
||||
RNS.log("Resetting I2P tunnel and retrying later", RNS.LOG_ERROR)
|
||||
|
||||
self.stop_tunnel(i2ptunnel)
|
||||
return False
|
||||
|
||||
elif i2ptunnel.status["setup_failed"] == True:
|
||||
RNS.log(str(self)+" Unspecified I2P tunnel setup error, resetting I2P tunnel", RNS.LOG_ERROR)
|
||||
|
||||
self.stop_tunnel(i2ptunnel)
|
||||
return False
|
||||
|
||||
else:
|
||||
RNS.log(str(self)+" Got no status from SAM API, resetting I2P tunnel", RNS.LOG_ERROR)
|
||||
|
||||
self.stop_tunnel(i2ptunnel)
|
||||
return False
|
||||
|
||||
time.sleep(5)
|
||||
|
||||
def get_loop(self):
|
||||
return asyncio.get_event_loop()
|
||||
def get_loop(self): return asyncio.get_event_loop()
|
||||
|
||||
|
||||
class ThreadingI2PServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
|
||||
pass
|
||||
|
||||
|
||||
class I2PInterfacePeer(Interface):
|
||||
RECONNECT_WAIT = 15
|
||||
RECONNECT_MAX_TRIES = None
|
||||
|
|
@ -416,9 +353,11 @@ class I2PInterfacePeer(Interface):
|
|||
self.online = False
|
||||
self.detached = False
|
||||
self.kiss_framing = False
|
||||
self.tunnels_allowed = True
|
||||
self.i2p_tunneled = True
|
||||
self.i2p_dest = None
|
||||
self.i2p_tunnel_ready = False
|
||||
self.lossless_medium = True
|
||||
self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
|
||||
self.bitrate = I2PInterface.BITRATE_GUESS
|
||||
self.last_read = 0
|
||||
|
|
@ -431,18 +370,11 @@ class I2PInterfacePeer(Interface):
|
|||
self.ifac_netkey = self.parent_interface.ifac_netkey
|
||||
if self.ifac_netname != None or self.ifac_netkey != None:
|
||||
ifac_origin = b""
|
||||
if self.ifac_netname != None:
|
||||
ifac_origin += RNS.Identity.full_hash(self.ifac_netname.encode("utf-8"))
|
||||
if self.ifac_netkey != None:
|
||||
ifac_origin += RNS.Identity.full_hash(self.ifac_netkey.encode("utf-8"))
|
||||
if self.ifac_netname != None: ifac_origin += RNS.Identity.full_hash(self.ifac_netname.encode("utf-8"))
|
||||
if self.ifac_netkey != None: ifac_origin += RNS.Identity.full_hash(self.ifac_netkey.encode("utf-8"))
|
||||
|
||||
ifac_origin_hash = RNS.Identity.full_hash(ifac_origin)
|
||||
self.ifac_key = RNS.Cryptography.hkdf(
|
||||
length=64,
|
||||
derive_from=ifac_origin_hash,
|
||||
salt=RNS.Reticulum.IFAC_SALT,
|
||||
context=None
|
||||
)
|
||||
self.ifac_key = RNS.Cryptography.hkdf(length=64, derive_from=ifac_origin_hash, salt=RNS.Reticulum.IFAC_SALT, context=None)
|
||||
self.ifac_identity = RNS.Identity.from_bytes(self.ifac_key)
|
||||
self.ifac_signature = self.ifac_identity.sign(RNS.Identity.full_hash(self.ifac_key))
|
||||
|
||||
|
|
@ -450,10 +382,8 @@ class I2PInterfacePeer(Interface):
|
|||
self.announce_rate_grace = None
|
||||
self.announce_rate_penalty = None
|
||||
|
||||
if max_reconnect_tries == None:
|
||||
self.max_reconnect_tries = I2PInterfacePeer.RECONNECT_MAX_TRIES
|
||||
else:
|
||||
self.max_reconnect_tries = max_reconnect_tries
|
||||
if max_reconnect_tries == None: self.max_reconnect_tries = I2PInterfacePeer.RECONNECT_MAX_TRIES
|
||||
else: self.max_reconnect_tries = max_reconnect_tries
|
||||
|
||||
if connected_socket != None:
|
||||
self.receives = True
|
||||
|
|
@ -461,15 +391,12 @@ class I2PInterfacePeer(Interface):
|
|||
self.target_port = None
|
||||
self.socket = connected_socket
|
||||
|
||||
if platform.system() == "Linux":
|
||||
self.set_timeouts_linux()
|
||||
elif platform.system() == "Darwin":
|
||||
self.set_timeouts_osx()
|
||||
if platform.system() == "Linux": self.set_timeouts_linux()
|
||||
elif platform.system() == "Darwin": self.set_timeouts_osx()
|
||||
|
||||
elif target_i2p_dest != None:
|
||||
self.receives = True
|
||||
self.initiator = True
|
||||
|
||||
self.bind_ip = "127.0.0.1"
|
||||
|
||||
self.awaiting_i2p_tunnel = True
|
||||
|
|
@ -497,13 +424,9 @@ class I2PInterfacePeer(Interface):
|
|||
thread.start()
|
||||
|
||||
def wait_job():
|
||||
while self.awaiting_i2p_tunnel:
|
||||
time.sleep(0.25)
|
||||
while self.awaiting_i2p_tunnel: time.sleep(0.25)
|
||||
time.sleep(2)
|
||||
|
||||
if not self.kiss_framing:
|
||||
self.wants_tunnel = True
|
||||
|
||||
if not self.connect(initial=True):
|
||||
thread = threading.Thread(target=self.reconnect)
|
||||
thread.daemon = True
|
||||
|
|
@ -517,7 +440,6 @@ class I2PInterfacePeer(Interface):
|
|||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
|
||||
def set_timeouts_linux(self):
|
||||
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_USER_TIMEOUT, int(I2PInterfacePeer.I2P_USER_TIMEOUT * 1000))
|
||||
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
|
||||
|
|
@ -526,10 +448,8 @@ class I2PInterfacePeer(Interface):
|
|||
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPCNT, int(I2PInterfacePeer.I2P_PROBES))
|
||||
|
||||
def set_timeouts_osx(self):
|
||||
if hasattr(socket, "TCP_KEEPALIVE"):
|
||||
TCP_KEEPIDLE = socket.TCP_KEEPALIVE
|
||||
else:
|
||||
TCP_KEEPIDLE = 0x10
|
||||
if hasattr(socket, "TCP_KEEPALIVE"): TCP_KEEPIDLE = socket.TCP_KEEPALIVE
|
||||
else: TCP_KEEPIDLE = 0x10
|
||||
|
||||
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
|
||||
self.socket.setsockopt(socket.IPPROTO_TCP, TCP_KEEPIDLE, int(I2PInterfacePeer.I2P_PROBE_AFTER))
|
||||
|
|
@ -537,16 +457,12 @@ class I2PInterfacePeer(Interface):
|
|||
def shutdown_socket(self, target_socket):
|
||||
if callable(target_socket.close):
|
||||
try:
|
||||
if socket != None:
|
||||
target_socket.shutdown(socket.SHUT_RDWR)
|
||||
except Exception as e:
|
||||
RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
|
||||
if socket != None: target_socket.shutdown(socket.SHUT_RDWR)
|
||||
except Exception as e: RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
|
||||
|
||||
try:
|
||||
if socket != None:
|
||||
target_socket.close()
|
||||
except Exception as e:
|
||||
RNS.log("Error while closing socket for "+str(self)+": "+str(e))
|
||||
if socket != None: target_socket.close()
|
||||
except Exception as e: RNS.log("Error while closing socket for "+str(self)+": "+str(e))
|
||||
|
||||
def detach(self):
|
||||
RNS.log("Detaching "+str(self), RNS.LOG_DEBUG)
|
||||
|
|
@ -555,22 +471,39 @@ class I2PInterfacePeer(Interface):
|
|||
if callable(self.socket.close):
|
||||
self.detached = True
|
||||
|
||||
try:
|
||||
self.socket.shutdown(socket.SHUT_RDWR)
|
||||
except Exception as e:
|
||||
RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
|
||||
try: self.socket.shutdown(socket.SHUT_RDWR)
|
||||
except Exception as e: RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
|
||||
|
||||
try:
|
||||
self.socket.close()
|
||||
except Exception as e:
|
||||
RNS.log("Error while closing socket for "+str(self)+": "+str(e))
|
||||
try: self.socket.close()
|
||||
except Exception as e: RNS.log("Error while closing socket for "+str(self)+": "+str(e))
|
||||
|
||||
self.socket = None
|
||||
|
||||
def early_synthesis(self, packet):
|
||||
if not self.detached:
|
||||
if not packet.packed: packet.pack()
|
||||
if self.ifac_identity: raw = RNS.Transport.handle_outgoing_ifac(self, packet.raw);
|
||||
else: raw = packet.raw
|
||||
|
||||
if self.kiss_framing: frame = bytes([KISS.FEND])+bytes([KISS.CMD_DATA])+KISS.escape(raw)+bytes([KISS.FEND])
|
||||
else: frame = bytes([HDLC.FLAG])+HDLC.escape(raw)+bytes([HDLC.FLAG])
|
||||
|
||||
self.socket.sendall(frame)
|
||||
self.txb += len(frame)
|
||||
self.last_write = time.time()
|
||||
if hasattr(self, "parent_interface") and self.parent_interface != None and self.parent_count: self.parent_interface.txb += len(frame)
|
||||
|
||||
def connect(self, initial=False):
|
||||
try:
|
||||
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
self.socket.connect((self.target_ip, self.target_port))
|
||||
|
||||
if not self.kiss_framing:
|
||||
self.wants_tunnel = True
|
||||
self.remote_transport = None
|
||||
self.synthesis_received = False
|
||||
RNS.Transport.synthesize_tunnel(self)
|
||||
|
||||
self.online = True
|
||||
|
||||
except Exception as e:
|
||||
|
|
@ -578,24 +511,16 @@ class I2PInterfacePeer(Interface):
|
|||
if not self.awaiting_i2p_tunnel:
|
||||
RNS.log("Initial connection for "+str(self)+" could not be established: "+str(e), RNS.LOG_ERROR)
|
||||
RNS.log("Leaving unconnected and retrying connection in "+str(I2PInterfacePeer.RECONNECT_WAIT)+" seconds.", RNS.LOG_ERROR)
|
||||
|
||||
return False
|
||||
|
||||
else:
|
||||
raise e
|
||||
else: raise e
|
||||
|
||||
if platform.system() == "Linux":
|
||||
self.set_timeouts_linux()
|
||||
elif platform.system() == "Darwin":
|
||||
self.set_timeouts_osx()
|
||||
if platform.system() == "Linux": self.set_timeouts_linux()
|
||||
elif platform.system() == "Darwin": self.set_timeouts_osx()
|
||||
|
||||
self.online = True
|
||||
self.writing = False
|
||||
self.never_connected = False
|
||||
|
||||
if not self.kiss_framing and self.wants_tunnel:
|
||||
RNS.Transport.synthesize_tunnel(self)
|
||||
|
||||
return True
|
||||
|
||||
def reconnect(self):
|
||||
|
|
@ -612,14 +537,10 @@ class I2PInterfacePeer(Interface):
|
|||
self.teardown()
|
||||
break
|
||||
|
||||
try:
|
||||
self.connect()
|
||||
|
||||
try: self.connect()
|
||||
except Exception as e:
|
||||
if not self.awaiting_i2p_tunnel:
|
||||
RNS.log("Connection attempt for "+str(self)+" failed: "+str(e), RNS.LOG_DEBUG)
|
||||
else:
|
||||
RNS.log(str(self)+" still waiting for I2P tunnel to appear", RNS.LOG_VERBOSE)
|
||||
if not self.awaiting_i2p_tunnel: RNS.log("Connection attempt for "+str(self)+" failed: "+str(e), RNS.LOG_DEBUG)
|
||||
else: RNS.log(str(self)+" still waiting for I2P tunnel to appear", RNS.LOG_VERBOSE)
|
||||
|
||||
if not self.never_connected:
|
||||
RNS.log(str(self)+" Re-established connection via I2P tunnel", RNS.LOG_INFO)
|
||||
|
|
@ -628,40 +549,31 @@ class I2PInterfacePeer(Interface):
|
|||
thread = threading.Thread(target=self.read_loop)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
if not self.kiss_framing:
|
||||
RNS.Transport.synthesize_tunnel(self)
|
||||
|
||||
else:
|
||||
RNS.log("Attempt to reconnect on a non-initiator I2P interface. This should not happen.", RNS.LOG_ERROR)
|
||||
raise IOError("Attempt to reconnect on a non-initiator I2P interface")
|
||||
|
||||
def process_incoming(self, data):
|
||||
self.rxb += len(data)
|
||||
if hasattr(self, "parent_interface") and self.parent_interface != None and self.parent_count:
|
||||
self.parent_interface.rxb += len(data)
|
||||
|
||||
if not data: return
|
||||
self.owner.inbound(data, self)
|
||||
|
||||
def process_outgoing(self, data):
|
||||
if self.online:
|
||||
while self.writing:
|
||||
time.sleep(0.001)
|
||||
while self.writing: time.sleep(0.001)
|
||||
|
||||
try:
|
||||
self.writing = True
|
||||
|
||||
if self.kiss_framing:
|
||||
data = bytes([KISS.FEND])+bytes([KISS.CMD_DATA])+KISS.escape(data)+bytes([KISS.FEND])
|
||||
else:
|
||||
data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
|
||||
if self.kiss_framing: data = bytes([KISS.FEND])+bytes([KISS.CMD_DATA])+KISS.escape(data)+bytes([KISS.FEND])
|
||||
else: data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
|
||||
|
||||
self.socket.sendall(data)
|
||||
self.writing = False
|
||||
self.txb += len(data)
|
||||
self.last_write = time.time()
|
||||
|
||||
if hasattr(self, "parent_interface") and self.parent_interface != None and self.parent_count:
|
||||
self.parent_interface.txb += len(data)
|
||||
if hasattr(self, "parent_interface") and self.parent_interface != None and self.parent_count: self.parent_interface.txb += len(data)
|
||||
|
||||
except Exception as e:
|
||||
RNS.log("Exception occurred while transmitting via "+str(self)+", tearing down interface", RNS.LOG_ERROR)
|
||||
|
|
@ -670,8 +582,7 @@ class I2PInterfacePeer(Interface):
|
|||
|
||||
|
||||
def read_watchdog(self):
|
||||
while self.wd_reset:
|
||||
time.sleep(0.25)
|
||||
while self.wd_reset: time.sleep(0.25)
|
||||
|
||||
should_run = True
|
||||
try:
|
||||
|
|
@ -679,17 +590,13 @@ class I2PInterfacePeer(Interface):
|
|||
time.sleep(1)
|
||||
|
||||
if (time.time()-self.last_read > I2PInterfacePeer.I2P_PROBE_AFTER*2):
|
||||
if self.i2p_tunnel_state != I2PInterfacePeer.TUNNEL_STATE_STALE:
|
||||
RNS.log("I2P tunnel became unresponsive", RNS.LOG_DEBUG)
|
||||
|
||||
if self.i2p_tunnel_state != I2PInterfacePeer.TUNNEL_STATE_STALE: RNS.log("I2P tunnel became unresponsive", RNS.LOG_DEBUG)
|
||||
self.i2p_tunnel_state = I2PInterfacePeer.TUNNEL_STATE_STALE
|
||||
else:
|
||||
self.i2p_tunnel_state = I2PInterfacePeer.TUNNEL_STATE_ACTIVE
|
||||
else: self.i2p_tunnel_state = I2PInterfacePeer.TUNNEL_STATE_ACTIVE
|
||||
|
||||
if (time.time()-self.last_write > I2PInterfacePeer.I2P_PROBE_AFTER*1):
|
||||
try:
|
||||
if self.socket != None:
|
||||
self.socket.sendall(bytes([HDLC.FLAG, HDLC.FLAG]))
|
||||
if self.socket != None: self.socket.sendall(bytes([HDLC.FLAG, HDLC.FLAG]))
|
||||
except Exception as e:
|
||||
RNS.log("An error ocurred while sending I2P keepalive. The contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
self.shutdown_socket(self.socket)
|
||||
|
|
@ -698,22 +605,17 @@ class I2PInterfacePeer(Interface):
|
|||
if (time.time()-self.last_read > I2PInterfacePeer.I2P_READ_TIMEOUT):
|
||||
RNS.log("I2P socket is unresponsive, restarting...", RNS.LOG_WARNING)
|
||||
if self.socket != None:
|
||||
try:
|
||||
self.socket.shutdown(socket.SHUT_RDWR)
|
||||
except Exception as e:
|
||||
RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
|
||||
try: self.socket.shutdown(socket.SHUT_RDWR)
|
||||
except Exception as e: RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
|
||||
|
||||
try:
|
||||
self.socket.close()
|
||||
except Exception as e:
|
||||
RNS.log("Error while closing socket for "+str(self)+": "+str(e))
|
||||
try: self.socket.close()
|
||||
except Exception as e: RNS.log("Error while closing socket for "+str(self)+": "+str(e))
|
||||
|
||||
should_run = False
|
||||
|
||||
self.wd_reset = False
|
||||
|
||||
finally:
|
||||
self.wd_reset = False
|
||||
finally: self.wd_reset = False
|
||||
|
||||
def read_loop(self):
|
||||
try:
|
||||
|
|
@ -784,7 +686,6 @@ class I2PInterfacePeer(Interface):
|
|||
data_buffer = data_buffer+bytes([byte])
|
||||
else:
|
||||
self.online = False
|
||||
|
||||
self.wd_reset = True
|
||||
time.sleep(2)
|
||||
self.wd_reset = False
|
||||
|
|
@ -806,15 +707,12 @@ class I2PInterfacePeer(Interface):
|
|||
if self.initiator:
|
||||
RNS.log("Attempting to reconnect...", RNS.LOG_WARNING)
|
||||
self.reconnect()
|
||||
else:
|
||||
self.teardown()
|
||||
else: self.teardown()
|
||||
|
||||
def teardown(self):
|
||||
if self.initiator and not self.detached:
|
||||
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is being torn down. Restart Reticulum to attempt to open this interface again.", RNS.LOG_ERROR)
|
||||
if RNS.Reticulum.panic_on_interface_error:
|
||||
RNS.panic()
|
||||
|
||||
if RNS.Reticulum.panic_on_interface_error: RNS.panic()
|
||||
else:
|
||||
RNS.log("The interface "+str(self)+" is being torn down.", RNS.LOG_VERBOSE)
|
||||
|
||||
|
|
@ -826,10 +724,7 @@ class I2PInterfacePeer(Interface):
|
|||
while self in self.parent_interface.spawned_interfaces:
|
||||
self.parent_interface.spawned_interfaces.remove(self)
|
||||
|
||||
if self in RNS.Transport.interfaces:
|
||||
if not self.initiator:
|
||||
RNS.Transport.interfaces.remove(self)
|
||||
|
||||
if not self.initiator: RNS.Transport.remove_interface(self)
|
||||
|
||||
def __str__(self):
|
||||
return "I2PInterfacePeer["+str(self.name)+"]"
|
||||
|
|
@ -840,8 +735,7 @@ class I2PInterface(Interface):
|
|||
DEFAULT_IFAC_SIZE = 16
|
||||
|
||||
@property
|
||||
def clients(self):
|
||||
return len(self.spawned_interfaces)
|
||||
def clients(self): return len(self.spawned_interfaces)
|
||||
|
||||
def __init__(self, owner, configuration):
|
||||
super().__init__()
|
||||
|
|
@ -862,6 +756,8 @@ class I2PInterface(Interface):
|
|||
self.owner = owner
|
||||
self.connectable = connectable
|
||||
self.i2p_tunneled = True
|
||||
self.tunnels_allowed = True
|
||||
self.peers = peers
|
||||
self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
|
||||
|
||||
self.b32 = None
|
||||
|
|
@ -871,7 +767,6 @@ class I2PInterface(Interface):
|
|||
self.OUT = False
|
||||
self.name = name
|
||||
|
||||
|
||||
self.receives = True
|
||||
self.bind_ip = "127.0.0.1"
|
||||
self.bind_port = self.i2p.get_free_port()
|
||||
|
|
@ -881,6 +776,7 @@ class I2PInterface(Interface):
|
|||
self.ifac_netname = ifac_netname
|
||||
self.ifac_netkey = ifac_netkey
|
||||
self.supports_discovery = True
|
||||
self.lossless_medium = True
|
||||
|
||||
self.online = False
|
||||
|
||||
|
|
@ -895,11 +791,9 @@ class I2PInterface(Interface):
|
|||
RNS.log("I2P controller did not become available in time, waiting for controller", RNS.LOG_VERBOSE)
|
||||
i2p_notready_warning = True
|
||||
|
||||
while not self.i2p.ready:
|
||||
time.sleep(0.25)
|
||||
while not self.i2p.ready: time.sleep(0.25)
|
||||
|
||||
if i2p_notready_warning == True:
|
||||
RNS.log("I2P controller ready, continuing setup", RNS.LOG_VERBOSE)
|
||||
if i2p_notready_warning == True: RNS.log("I2P controller ready, continuing setup", RNS.LOG_VERBOSE)
|
||||
|
||||
def handlerFactory(callback):
|
||||
def createHandler(*args, **keys):
|
||||
|
|
@ -913,6 +807,7 @@ class I2PInterface(Interface):
|
|||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
def final_init(self):
|
||||
if self.connectable:
|
||||
def tunnel_job():
|
||||
while True:
|
||||
|
|
@ -932,22 +827,83 @@ class I2PInterface(Interface):
|
|||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
if peers != None:
|
||||
for peer_addr in peers:
|
||||
interface_name = self.name+" to "+peer_addr
|
||||
peer_interface = I2PInterfacePeer(self, self.owner, interface_name, peer_addr)
|
||||
peer_interface.OUT = True
|
||||
peer_interface.IN = True
|
||||
peer_interface.parent_interface = self
|
||||
peer_interface.parent_count = False
|
||||
RNS.Transport.interfaces.append(peer_interface)
|
||||
threading.Thread(target=self._spawn_peers, daemon=True).start()
|
||||
|
||||
def _spawn_peers(self):
|
||||
time.sleep(1.5)
|
||||
if self.peers != None:
|
||||
for peer_addr in self.peers:
|
||||
try:
|
||||
interface_name = self.name+" to "+peer_addr
|
||||
peer_interface = I2PInterfacePeer(self, self.owner, interface_name, peer_addr)
|
||||
peer_interface.OUT = self.OUT
|
||||
peer_interface.IN = self.IN
|
||||
peer_interface.parent_interface = self
|
||||
peer_interface.parent_count = False
|
||||
|
||||
peer_interface.mode = self.mode
|
||||
peer_interface.bitrate = self.bitrate
|
||||
peer_interface.recursive_prs = self.recursive_prs
|
||||
peer_interface.gravity = self.gravity
|
||||
peer_interface.HW_MTU = self.HW_MTU
|
||||
|
||||
peer_interface.announces_to_internal = self.announces_to_internal
|
||||
peer_interface.announces_from_internal = self.announces_from_internal
|
||||
peer_interface.announce_cap = self.announce_cap
|
||||
|
||||
peer_interface.ingress_control = self.ingress_control
|
||||
peer_interface.ic_max_held_announces = self.ic_max_held_announces
|
||||
peer_interface.ic_burst_hold = self.ic_burst_hold
|
||||
peer_interface.ic_burst_freq = self.ic_burst_freq
|
||||
peer_interface.ic_burst_freq_new = self.ic_burst_freq_new
|
||||
peer_interface.ic_new_time = self.ic_new_time
|
||||
peer_interface.ic_burst_penalty = self.ic_burst_penalty
|
||||
peer_interface.ic_held_release_interval = self.ic_held_release_interval
|
||||
|
||||
peer_interface.egress_control = self.egress_control
|
||||
peer_interface.ec_pr_freq = self.ec_pr_freq
|
||||
peer_interface.ic_pr_burst_freq_new = self.ic_pr_burst_freq_new
|
||||
peer_interface.ic_pr_burst_freq = self.ic_pr_burst_freq
|
||||
|
||||
peer_interface.announce_rate_target = self.announce_rate_target
|
||||
peer_interface.announce_rate_grace = self.announce_rate_grace
|
||||
peer_interface.announce_rate_penalty = self.announce_rate_penalty
|
||||
|
||||
peer_interface._destination_cost = self._destination_cost
|
||||
peer_interface._pr_cost = self._pr_cost
|
||||
|
||||
RNS.Transport.add_interface(peer_interface)
|
||||
|
||||
except Exception as e: RNS.log(f"Could not create interface for {self} to {peer_addr}: {e}", RNS.LOG_ERROR)
|
||||
|
||||
def incoming_connection(self, handler):
|
||||
RNS.log("Accepting incoming I2P connection", RNS.LOG_VERBOSE)
|
||||
interface_name = "Connected peer on "+self.name
|
||||
spawned_interface = I2PInterfacePeer(self, self.owner, interface_name, connected_socket=handler.request)
|
||||
spawned_interface.OUT = True
|
||||
spawned_interface.IN = True
|
||||
spawned_interface.OUT = self.OUT
|
||||
spawned_interface.IN = self.IN
|
||||
|
||||
spawned_interface.ingress_control = self.ingress_control
|
||||
spawned_interface.ic_max_held_announces = self.ic_max_held_announces
|
||||
spawned_interface.ic_burst_hold = self.ic_burst_hold
|
||||
spawned_interface.ic_burst_freq = self.ic_burst_freq
|
||||
spawned_interface.ic_burst_freq_new = self.ic_burst_freq_new
|
||||
spawned_interface.ic_new_time = self.ic_new_time
|
||||
spawned_interface.ic_burst_penalty = self.ic_burst_penalty
|
||||
spawned_interface.ic_held_release_interval = self.ic_held_release_interval
|
||||
|
||||
spawned_interface.announces_to_internal = self.announces_to_internal
|
||||
spawned_interface.announces_from_internal = self.announces_from_internal
|
||||
spawned_interface.announce_cap = self.announce_cap
|
||||
|
||||
spawned_interface.egress_control = self.egress_control
|
||||
spawned_interface.ec_pr_freq = self.ec_pr_freq
|
||||
spawned_interface.ic_pr_burst_freq_new = self.ic_pr_burst_freq_new
|
||||
spawned_interface.ic_pr_burst_freq = self.ic_pr_burst_freq
|
||||
|
||||
spawned_interface._destination_cost = self._destination_cost
|
||||
spawned_interface._pr_cost = self._pr_cost
|
||||
|
||||
spawned_interface.parent_interface = self
|
||||
spawned_interface.online = True
|
||||
spawned_interface.bitrate = self.bitrate
|
||||
|
|
@ -957,18 +913,11 @@ class I2PInterface(Interface):
|
|||
spawned_interface.ifac_netkey = self.ifac_netkey
|
||||
if spawned_interface.ifac_netname != None or spawned_interface.ifac_netkey != None:
|
||||
ifac_origin = b""
|
||||
if spawned_interface.ifac_netname != None:
|
||||
ifac_origin += RNS.Identity.full_hash(spawned_interface.ifac_netname.encode("utf-8"))
|
||||
if spawned_interface.ifac_netkey != None:
|
||||
ifac_origin += RNS.Identity.full_hash(spawned_interface.ifac_netkey.encode("utf-8"))
|
||||
if spawned_interface.ifac_netname != None: ifac_origin += RNS.Identity.full_hash(spawned_interface.ifac_netname.encode("utf-8"))
|
||||
if spawned_interface.ifac_netkey != None: ifac_origin += RNS.Identity.full_hash(spawned_interface.ifac_netkey.encode("utf-8"))
|
||||
|
||||
ifac_origin_hash = RNS.Identity.full_hash(ifac_origin)
|
||||
spawned_interface.ifac_key = RNS.Cryptography.hkdf(
|
||||
length=64,
|
||||
derive_from=ifac_origin_hash,
|
||||
salt=RNS.Reticulum.IFAC_SALT,
|
||||
context=None
|
||||
)
|
||||
spawned_interface.ifac_key = RNS.Cryptography.hkdf(length=64, derive_from=ifac_origin_hash, salt=RNS.Reticulum.IFAC_SALT, context=None)
|
||||
spawned_interface.ifac_identity = RNS.Identity.from_bytes(spawned_interface.ifac_key)
|
||||
spawned_interface.ifac_signature = spawned_interface.ifac_identity.sign(RNS.Identity.full_hash(spawned_interface.ifac_key))
|
||||
|
||||
|
|
@ -976,34 +925,48 @@ class I2PInterface(Interface):
|
|||
spawned_interface.announce_rate_grace = self.announce_rate_grace
|
||||
spawned_interface.announce_rate_penalty = self.announce_rate_penalty
|
||||
spawned_interface.mode = self.mode
|
||||
spawned_interface.recursive_prs = self.recursive_prs
|
||||
spawned_interface.gravity = self.gravity
|
||||
spawned_interface.hop_limiter = self.hop_limiter
|
||||
spawned_interface.HW_MTU = self.HW_MTU
|
||||
RNS.log("Spawned new I2PInterface Peer: "+str(spawned_interface), RNS.LOG_VERBOSE)
|
||||
RNS.Transport.interfaces.append(spawned_interface)
|
||||
while spawned_interface in self.spawned_interfaces:
|
||||
self.spawned_interfaces.remove(spawned_interface)
|
||||
RNS.Transport.add_interface(spawned_interface)
|
||||
while spawned_interface in self.spawned_interfaces: self.spawned_interfaces.remove(spawned_interface)
|
||||
self.spawned_interfaces.append(spawned_interface)
|
||||
spawned_interface.read_loop()
|
||||
|
||||
def process_outgoing(self, data):
|
||||
pass
|
||||
|
||||
def received_announce(self, from_spawned=False):
|
||||
if from_spawned: self.ia_freq_deque.append(time.time())
|
||||
def received_announce(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.ia_freq_deque.append(time.time())
|
||||
self.arxb += size
|
||||
|
||||
def sent_announce(self, from_spawned=False):
|
||||
if from_spawned: self.oa_freq_deque.append(time.time())
|
||||
def sent_announce(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.oa_freq_deque.append(time.time())
|
||||
self.atxb += size
|
||||
|
||||
def received_path_request(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.ip_freq_deque.append(time.time())
|
||||
self.prxb += size
|
||||
|
||||
def sent_path_request(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.op_freq_deque.append(time.time())
|
||||
self.ptxb += size
|
||||
|
||||
def detach(self):
|
||||
RNS.log("Detaching "+str(self), RNS.LOG_DEBUG)
|
||||
self.i2p.stop()
|
||||
|
||||
def __str__(self):
|
||||
return "I2PInterface["+self.name+"]"
|
||||
def __str__(self): return "I2PInterface["+self.name+"]"
|
||||
|
||||
class I2PInterfaceHandler(socketserver.BaseRequestHandler):
|
||||
def __init__(self, callback, *args, **keys):
|
||||
self.callback = callback
|
||||
socketserver.BaseRequestHandler.__init__(self, *args, **keys)
|
||||
|
||||
def handle(self):
|
||||
self.callback(handler=self)
|
||||
def handle(self): self.callback(handler=self)
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -39,7 +39,6 @@ class Interface:
|
|||
OUT = False
|
||||
FWD = False
|
||||
RPT = False
|
||||
name = None
|
||||
|
||||
# Interface mode definitions
|
||||
MODE_FULL = 0x01
|
||||
|
|
@ -48,67 +47,167 @@ class Interface:
|
|||
MODE_ROAMING = 0x04
|
||||
MODE_BOUNDARY = 0x05
|
||||
MODE_GATEWAY = 0x06
|
||||
MODE_INTERNAL = 0x07
|
||||
|
||||
# Which interface modes a Transport Node should
|
||||
# actively discover paths for.
|
||||
DISCOVER_PATHS_FOR = [MODE_ACCESS_POINT, MODE_GATEWAY, MODE_ROAMING]
|
||||
DISCOVER_PATHS_FOR = [MODE_ACCESS_POINT, MODE_GATEWAY, MODE_ROAMING, MODE_INTERNAL]
|
||||
BOUNDARY_SEARCH_MODES = [MODE_BOUNDARY, MODE_GATEWAY]
|
||||
|
||||
# How many samples to use for announce
|
||||
# frequency calculations
|
||||
IA_FREQ_SAMPLES = 6
|
||||
OA_FREQ_SAMPLES = 6
|
||||
IA_FREQ_SAMPLES = 48
|
||||
OA_FREQ_SAMPLES = 48
|
||||
IP_FREQ_SAMPLES = 48
|
||||
OP_FREQ_SAMPLES = 48
|
||||
|
||||
AR_MINFREQ_HZ = 0.1
|
||||
PR_MINFREQ_HZ = 0.1
|
||||
AR_FREQ_DECAY = 1/AR_MINFREQ_HZ
|
||||
PR_FREQ_DECAY = 1/PR_MINFREQ_HZ
|
||||
|
||||
# Maximum amount of ingress limited announces
|
||||
# to hold at any given time.
|
||||
MAX_HELD_ANNOUNCES = 256
|
||||
MAX_HELD_ANNOUNCES = 64
|
||||
|
||||
# How long a spawned interface will be
|
||||
# considered to be newly created. Two
|
||||
# hours by default.
|
||||
# Control parameters
|
||||
DEFAULT_GRAVITY = 0
|
||||
IC_NEW_TIME = 2*60*60
|
||||
IC_BURST_FREQ_NEW = 3.5
|
||||
IC_BURST_FREQ = 12
|
||||
IC_BURST_HOLD = 1*60
|
||||
IC_BURST_PENALTY = 5*60
|
||||
IC_HELD_RELEASE_INTERVAL = 30
|
||||
IC_BURST_FREQ_NEW = 3
|
||||
IC_BURST_FREQ = 10
|
||||
IC_PR_BURST_FREQ_NEW = 3
|
||||
IC_PR_BURST_FREQ = 8
|
||||
IC_BURST_HOLD = 15
|
||||
IC_BURST_PENALTY = 15
|
||||
IC_HELD_RELEASE_INTERVAL = 5
|
||||
IC_DEQUE_MIN_SAMPLE = 2
|
||||
EC_BURST_MIN_SAMPLES = 2
|
||||
EC_PR_FREQ = 5
|
||||
EGRESS_CONTROL = False
|
||||
|
||||
# Default announce rate targets
|
||||
DEFAULT_AR_TARGET = 3600
|
||||
DEFAULT_AR_PENALTY = 0
|
||||
DEFAULT_AR_GRACE = 5
|
||||
|
||||
AUTOCONFIGURE_MTU = False
|
||||
FIXED_MTU = False
|
||||
DEFAULT_IFAC_SIZE = 16
|
||||
|
||||
def __init__(self):
|
||||
self.name = None
|
||||
self.rxb = 0
|
||||
self.txb = 0
|
||||
self.frxb = 0
|
||||
self.ftxb = 0
|
||||
self.arxb = 0
|
||||
self.atxb = 0
|
||||
self.arxc = 0
|
||||
self.atxc = 0
|
||||
self.afrxc = 0
|
||||
self.prxb = 0
|
||||
self.ptxb = 0
|
||||
self.prxc = 0
|
||||
self.ptxc = 0
|
||||
self.gravity = 0
|
||||
self.created = time.time()
|
||||
self.detached = False
|
||||
self.online = False
|
||||
self.bitrate = 62500
|
||||
self.HW_MTU = None
|
||||
self.__hash = None
|
||||
|
||||
self.supports_discovery = False
|
||||
self.discoverable = False
|
||||
self.last_discovery_announce = 0
|
||||
self.bootstrap_only = False
|
||||
self.parent_interface = None
|
||||
self.spawned_interfaces = None
|
||||
self.tunnel_id = None
|
||||
self.ingress_control = True
|
||||
self.ic_max_held_announces = Interface.MAX_HELD_ANNOUNCES
|
||||
self.ic_burst_hold = Interface.IC_BURST_HOLD
|
||||
self.ic_burst_active = False
|
||||
self.ic_burst_activated = 0
|
||||
self.ic_held_release = 0
|
||||
self.ic_burst_freq_new = Interface.IC_BURST_FREQ_NEW
|
||||
self.ic_burst_freq = Interface.IC_BURST_FREQ
|
||||
self.ic_new_time = Interface.IC_NEW_TIME
|
||||
self.ic_burst_penalty = Interface.IC_BURST_PENALTY
|
||||
self.ic_held_release_interval = Interface.IC_HELD_RELEASE_INTERVAL
|
||||
self.held_announces = {}
|
||||
self.announce_cap = RNS.Reticulum.ANNOUNCE_CAP/100.0
|
||||
self.announce_allowed_at = 0
|
||||
self.supports_discovery = False
|
||||
self.discoverable = False
|
||||
self.last_discovery_announce = 0
|
||||
self.bootstrap_only = False
|
||||
self.recursive_prs = False
|
||||
self.announces_from_internal = True
|
||||
self.announces_to_internal = None
|
||||
self.is_local_shared_instance = False
|
||||
self.parent_interface = None
|
||||
self.spawned_interfaces = None
|
||||
self.tunnel_id = None
|
||||
self.local_tunnel_id = None
|
||||
self.tunnels_allowed = False
|
||||
self.synthesis_received = False
|
||||
self.synthesis_emitted = False
|
||||
self.wants_tunnel = False
|
||||
self.remote_transport = None
|
||||
self.ingress_control = True
|
||||
self.hop_limiter = None
|
||||
self.phy_keepalive = False
|
||||
self.shared_medium = False
|
||||
self.lossless_medium = False
|
||||
|
||||
self.remote_impl = None
|
||||
self.remote_vers = None
|
||||
self.remote_dst_cost = None
|
||||
self.remote_pr_cost = None
|
||||
self.remote_is_tn = None
|
||||
|
||||
self.ic_burst_active = False
|
||||
self.ic_burst_activated = 0
|
||||
self.ic_burst_sustained = 0
|
||||
self.ic_pr_burst_active = False
|
||||
self.ic_pr_burst_activated = 0
|
||||
self.ic_pr_burst_sustained = 0
|
||||
self.ic_pr_burst_cooldown = 0
|
||||
self.ic_held_release = 0
|
||||
self.ic_max_held_announces = RNS.Reticulum.get_instance()._default_ic_max_held_announces()
|
||||
self.ic_burst_hold = RNS.Reticulum.get_instance()._default_ic_burst_hold()
|
||||
self.ic_burst_freq_new = RNS.Reticulum.get_instance()._default_ic_burst_freq_new()
|
||||
self.ic_burst_freq = RNS.Reticulum.get_instance()._default_ic_burst_freq()
|
||||
self.ic_pr_burst_freq_new = RNS.Reticulum.get_instance()._default_ic_pr_burst_freq_new()
|
||||
self.ic_pr_burst_freq = RNS.Reticulum.get_instance()._default_ic_pr_burst_freq()
|
||||
self.ic_new_time = RNS.Reticulum.get_instance()._default_ic_new_time()
|
||||
self.ic_burst_penalty = RNS.Reticulum.get_instance()._default_ic_burst_penalty()
|
||||
self.ic_held_release_interval = RNS.Reticulum.get_instance()._default_ic_held_release_interval()
|
||||
self.ec_pr_freq = RNS.Reticulum.get_instance()._default_ec_pr_freq()
|
||||
self.egress_control = RNS.Reticulum.get_instance()._default_egress_control()
|
||||
self.announce_queue = []
|
||||
self.held_announces = {}
|
||||
self.ifac_identity = None
|
||||
self.transmit_buffer = None
|
||||
|
||||
self.ia_freq_deque = deque(maxlen=Interface.IA_FREQ_SAMPLES)
|
||||
self.oa_freq_deque = deque(maxlen=Interface.OA_FREQ_SAMPLES)
|
||||
self.ip_freq_deque = deque(maxlen=Interface.IA_FREQ_SAMPLES)
|
||||
self.op_freq_deque = deque(maxlen=Interface.OA_FREQ_SAMPLES)
|
||||
|
||||
self.protocol_violations = 0
|
||||
self.ifac_violations = 0
|
||||
self.packet_filter_hits = 0
|
||||
self.hop_limiter_hits = 0
|
||||
self.dp_ingress_tcount = 0
|
||||
self.dp_ingress_bytes = 0
|
||||
self.dp_ingress_packets = 0
|
||||
self.dp_ingress_hold = None
|
||||
self.dp_ingress_gated = False
|
||||
|
||||
self.tx_hwm = 4*1024*1024
|
||||
self.tx_stalled = False
|
||||
self.tx_drops = 0
|
||||
self.tx_dropped_bytes = 0
|
||||
self._dp_ec_prev_sent = 0
|
||||
self._dp_ec_zero_ticks = 0
|
||||
self._dp_ec_last_drain = time.time()
|
||||
|
||||
self._destination_cost = None
|
||||
self._pr_cost = None
|
||||
|
||||
self.reports_phy_stats = False
|
||||
self.r_stat_rssi = None
|
||||
self.r_stat_snr = None
|
||||
self.r_stat_q = None
|
||||
|
||||
self.is_connected_to_shared_instance = False
|
||||
|
||||
def get_hash(self):
|
||||
return RNS.Identity.full_hash(str(self).encode("utf-8"))
|
||||
if not self.__hash: self.__hash = RNS.Identity.full_hash(str(self).encode("utf-8"))
|
||||
return self.__hash
|
||||
|
||||
# This is a generic function for determining when an interface
|
||||
# should activate ingress limiting. Since this can vary for
|
||||
|
|
@ -120,134 +219,247 @@ class Interface:
|
|||
ia_freq = self.incoming_announce_frequency()
|
||||
|
||||
if self.ic_burst_active:
|
||||
if ia_freq < freq_threshold and time.time() > self.ic_burst_activated+self.ic_burst_hold:
|
||||
self.ic_burst_active = False
|
||||
self.ic_held_release = time.time() + self.ic_burst_penalty
|
||||
if ia_freq < freq_threshold and time.time() > self.ic_burst_activated+self.ic_burst_hold and time.time() > self.ic_burst_sustained+self.ic_burst_hold:
|
||||
if len(self.ia_freq_deque) >= self.IC_DEQUE_MIN_SAMPLE: self.ic_burst_active = False
|
||||
else:
|
||||
if ia_freq >= freq_threshold: self.ic_burst_sustained = time.time()
|
||||
|
||||
return True
|
||||
|
||||
else:
|
||||
if ia_freq > freq_threshold:
|
||||
self.ic_burst_active = True
|
||||
self.ic_burst_activated = time.time()
|
||||
self.ic_burst_sustained = time.time()
|
||||
self.ic_held_release = time.time() + self.ic_burst_penalty
|
||||
return True
|
||||
|
||||
else:
|
||||
return False
|
||||
else: return False
|
||||
|
||||
else:
|
||||
return False
|
||||
else: return False
|
||||
|
||||
def should_ingress_limit_pr(self):
|
||||
if self.ingress_control:
|
||||
freq_threshold = self.ic_pr_burst_freq_new if self.age() < self.ic_new_time else self.ic_pr_burst_freq
|
||||
ip_freq = self.incoming_pr_frequency()
|
||||
|
||||
if self.ic_pr_burst_active:
|
||||
if ip_freq < freq_threshold and time.time() > self.ic_pr_burst_activated+self.ic_burst_hold and time.time() > self.ic_pr_burst_sustained+self.ic_burst_hold:
|
||||
if self.ic_pr_burst_cooldown <= 0: self.ic_pr_burst_active = False
|
||||
else: self.ic_pr_burst_cooldown -= 1
|
||||
else:
|
||||
self.ic_pr_burst_cooldown = 3
|
||||
if ip_freq >= freq_threshold: self.ic_pr_burst_sustained = time.time()
|
||||
|
||||
return True
|
||||
|
||||
else:
|
||||
if ip_freq > freq_threshold:
|
||||
self.ic_pr_burst_active = True
|
||||
self.ic_pr_burst_activated = time.time()
|
||||
self.ic_pr_burst_sustained = time.time()
|
||||
self.ic_pr_burst_cooldown = 3
|
||||
return True
|
||||
|
||||
else: return False
|
||||
|
||||
else: return False
|
||||
|
||||
def should_egress_limit_pr(self):
|
||||
if self.egress_control:
|
||||
freq_threshold = self.ec_pr_freq
|
||||
op_freq = self.outgoing_pr_frequency(preemptive=True)
|
||||
|
||||
if op_freq > freq_threshold:
|
||||
if len(self.op_freq_deque) >= self.EC_BURST_MIN_SAMPLES: return True
|
||||
|
||||
return False
|
||||
|
||||
def optimise_mtu(self):
|
||||
if self.AUTOCONFIGURE_MTU:
|
||||
if self.bitrate >= 1_000_000_000:
|
||||
self.HW_MTU = 524288
|
||||
elif self.bitrate > 750_000_000:
|
||||
self.HW_MTU = 262144
|
||||
elif self.bitrate > 400_000_000:
|
||||
self.HW_MTU = 131072
|
||||
elif self.bitrate > 200_000_000:
|
||||
self.HW_MTU = 65536
|
||||
elif self.bitrate > 100_000_000:
|
||||
self.HW_MTU = 32768
|
||||
elif self.bitrate > 10_000_000:
|
||||
self.HW_MTU = 16384
|
||||
elif self.bitrate > 5_000_000:
|
||||
self.HW_MTU = 8192
|
||||
elif self.bitrate > 2_000_000:
|
||||
self.HW_MTU = 4096
|
||||
elif self.bitrate > 1_000_000:
|
||||
self.HW_MTU = 2048
|
||||
elif self.bitrate > 62_500:
|
||||
self.HW_MTU = 1024
|
||||
else:
|
||||
self.HW_MTU = None
|
||||
if self.bitrate >= 1_000_000_000: self.HW_MTU = 524288
|
||||
elif self.bitrate >= 750_000_000: self.HW_MTU = 262144
|
||||
elif self.bitrate >= 400_000_000: self.HW_MTU = 131072
|
||||
elif self.bitrate >= 200_000_000: self.HW_MTU = 65536
|
||||
elif self.bitrate >= 100_000_000: self.HW_MTU = 32768
|
||||
elif self.bitrate >= 10_000_000: self.HW_MTU = 16384
|
||||
elif self.bitrate >= 5_000_000: self.HW_MTU = 8192
|
||||
elif self.bitrate >= 2_000_000: self.HW_MTU = 4096
|
||||
elif self.bitrate >= 1_000_000: self.HW_MTU = 2048
|
||||
elif self.bitrate >= 62_500: self.HW_MTU = 1024
|
||||
else: self.HW_MTU = None
|
||||
|
||||
RNS.log(f"{self} hardware MTU set to {self.HW_MTU}", RNS.LOG_DEBUG) # TODO: Remove debug
|
||||
RNS.log(f"{self} hardware MTU set to {self.HW_MTU}", RNS.LOG_EXTREME)
|
||||
|
||||
def age(self):
|
||||
return time.time()-self.created
|
||||
|
||||
def hold_announce(self, announce_packet):
|
||||
if announce_packet.destination_hash in self.held_announces:
|
||||
if announce_packet.hops >= RNS.Transport.PATHFINDER_M-1:
|
||||
return self.filtered_announce()
|
||||
elif announce_packet.destination_hash in self.held_announces:
|
||||
self.held_announces[announce_packet.destination_hash] = announce_packet
|
||||
return self.filtered_announce()
|
||||
elif not len(self.held_announces) >= self.ic_max_held_announces:
|
||||
self.held_announces[announce_packet.destination_hash] = announce_packet
|
||||
return True
|
||||
else: return self.filtered_announce()
|
||||
|
||||
def process_held_announces(self):
|
||||
try:
|
||||
if not self.should_ingress_limit() and len(self.held_announces) > 0 and time.time() > self.ic_held_release:
|
||||
if len(self.held_announces) > 0 and time.time() > self.ic_held_release:
|
||||
freq_threshold = self.ic_burst_freq_new if self.age() < self.ic_new_time else self.ic_burst_freq
|
||||
ia_freq = self.incoming_announce_frequency()
|
||||
if ia_freq < freq_threshold:
|
||||
selected_announce_packet = None
|
||||
min_hops = RNS.Transport.PATHFINDER_M
|
||||
for destination_hash in self.held_announces:
|
||||
announce_packet = self.held_announces[destination_hash]
|
||||
cull_entries = []
|
||||
for destination_hash in self.held_announces.copy():
|
||||
announce_packet = self.held_announces.get(destination_hash)
|
||||
if not announce_packet: continue
|
||||
|
||||
# If we can determine that a held announce has become
|
||||
# useless for any potential path table updates, we'll
|
||||
# remove it immediately instead of waiting for it to
|
||||
# eventually get released, freeing up other held queue
|
||||
# slots for potentially useful new announces.
|
||||
path_entry = RNS.Transport.path_table.get(destination_hash)
|
||||
if path_entry and not RNS.Transport.announce_prefilter(announce_packet, path_entry):
|
||||
cull_entries.append(destination_hash)
|
||||
continue
|
||||
|
||||
if announce_packet.hops < min_hops:
|
||||
min_hops = announce_packet.hops
|
||||
selected_announce_packet = announce_packet
|
||||
|
||||
for destination_hash in cull_entries: self.held_announces.pop(destination_hash, None)
|
||||
|
||||
if selected_announce_packet != None:
|
||||
RNS.log("Releasing held announce packet "+str(selected_announce_packet)+" from "+str(self), RNS.LOG_EXTREME)
|
||||
self.ic_held_release = time.time() + self.ic_held_release_interval
|
||||
self.held_announces.pop(selected_announce_packet.destination_hash)
|
||||
def release():
|
||||
RNS.Transport.inbound(selected_announce_packet.raw, selected_announce_packet.receiving_interface)
|
||||
threading.Thread(target=release, daemon=True).start()
|
||||
self.held_announces.pop(selected_announce_packet.destination_hash, None)
|
||||
RNS.Transport.inbound(selected_announce_packet.raw, selected_announce_packet.receiving_interface, tc=RNS.Transport.TC_INGRESS_LIMITED, ifac_handled=True)
|
||||
|
||||
except Exception as e:
|
||||
RNS.log("An error occurred while processing held announces for "+str(self), RNS.LOG_ERROR)
|
||||
RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
|
||||
def received_announce(self, from_spawned=False):
|
||||
def received_announce(self, size=0, from_spawned=False):
|
||||
self.arxc += 1; self.arxb += size
|
||||
self.ia_freq_deque.append(time.time())
|
||||
if hasattr(self, "parent_interface") and self.parent_interface != None:
|
||||
self.parent_interface.received_announce(from_spawned=True)
|
||||
if self.parent_interface != None:
|
||||
self.parent_interface.received_announce(size=size, from_spawned=True)
|
||||
|
||||
def sent_announce(self, from_spawned=False):
|
||||
def sent_announce(self, size=0, from_spawned=False):
|
||||
self.atxc += 1; self.atxb += size
|
||||
self.oa_freq_deque.append(time.time())
|
||||
if hasattr(self, "parent_interface") and self.parent_interface != None:
|
||||
self.parent_interface.sent_announce(from_spawned=True)
|
||||
if self.parent_interface != None:
|
||||
self.parent_interface.sent_announce(size=size, from_spawned=True)
|
||||
|
||||
def filtered_announce(self, size=0, from_spawned=False):
|
||||
self.afrxc += 1
|
||||
RNS.Transport.filtered_arxc += 1
|
||||
if self.parent_interface != None: self.parent_interface.afrxc += 1
|
||||
return None
|
||||
|
||||
def received_path_request(self, size=0, from_spawned=False):
|
||||
self.prxc += 1; self.prxb += size
|
||||
self.ip_freq_deque.append(time.time())
|
||||
if self.parent_interface != None:
|
||||
self.parent_interface.received_path_request(size=size, from_spawned=True)
|
||||
|
||||
def sent_path_request(self, size=0, from_spawned=False):
|
||||
self.ptxc += 1; self.ptxb += size
|
||||
self.op_freq_deque.append(time.time())
|
||||
if self.parent_interface != None:
|
||||
self.parent_interface.sent_path_request(size=size, from_spawned=True)
|
||||
|
||||
def protocol_violation(self, description=None, level=None):
|
||||
if level == None: level = RNS.LOG_DEBUG
|
||||
self.protocol_violations += 1
|
||||
RNS.log(f"Protocol violation on {self}: {description}", level) if RNS.sl(level) else None
|
||||
return None
|
||||
|
||||
def ifac_violation(self, description=None):
|
||||
self.ifac_violations += 1
|
||||
RNS.log(f"IFAC violation on {self}: {description}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
return None
|
||||
|
||||
def packet_filter_hit(self):
|
||||
self.packet_filter_hits += 1
|
||||
return None
|
||||
|
||||
def hop_limiter_hit(self):
|
||||
self.hop_limiter_hits += 1
|
||||
return None
|
||||
|
||||
@property
|
||||
def destination_cost(self):
|
||||
if not self._destination_cost == None: return self._destination_cost
|
||||
else: return RNS.Transport.DESTINATION_COST
|
||||
|
||||
@property
|
||||
def pr_cost(self):
|
||||
if not self._pr_cost == None: return self._pr_cost
|
||||
else: return RNS.Transport.PR_COST
|
||||
|
||||
@property
|
||||
def ic_burst_count(self): return None
|
||||
|
||||
@property
|
||||
def ic_pr_burst_count(self): return None
|
||||
|
||||
def incoming_announce_frequency(self):
|
||||
if not len(self.ia_freq_deque) > 1:
|
||||
return 0
|
||||
n = len(self.ia_freq_deque)
|
||||
if not n > self.IC_DEQUE_MIN_SAMPLE: return 0
|
||||
else:
|
||||
dq_len = len(self.ia_freq_deque)
|
||||
delta_sum = 0
|
||||
for i in range(1,dq_len):
|
||||
delta_sum += self.ia_freq_deque[i]-self.ia_freq_deque[i-1]
|
||||
delta_sum += time.time() - self.ia_freq_deque[dq_len-1]
|
||||
|
||||
if delta_sum == 0:
|
||||
avg = 0
|
||||
else:
|
||||
avg = 1/(delta_sum/(dq_len))
|
||||
|
||||
return avg
|
||||
oldest = self.ia_freq_deque[0]
|
||||
span = time.time() - oldest
|
||||
if span > self.AR_FREQ_DECAY: self.ia_freq_deque.popleft()
|
||||
if span <= 0: return 0
|
||||
hz = n / span
|
||||
return hz
|
||||
|
||||
def outgoing_announce_frequency(self):
|
||||
if not len(self.oa_freq_deque) > 1:
|
||||
return 0
|
||||
n = len(self.oa_freq_deque)
|
||||
if not len(self.oa_freq_deque) > 1: return 0
|
||||
else:
|
||||
dq_len = len(self.oa_freq_deque)
|
||||
delta_sum = 0
|
||||
for i in range(1,dq_len):
|
||||
delta_sum += self.oa_freq_deque[i]-self.oa_freq_deque[i-1]
|
||||
delta_sum += time.time() - self.oa_freq_deque[dq_len-1]
|
||||
|
||||
if delta_sum == 0:
|
||||
avg = 0
|
||||
else:
|
||||
avg = 1/(delta_sum/(dq_len))
|
||||
oldest = self.oa_freq_deque[0]
|
||||
span = time.time() - oldest
|
||||
if span > self.AR_FREQ_DECAY: self.oa_freq_deque.popleft()
|
||||
if span <= 0: return 0
|
||||
hz = n / span
|
||||
return hz
|
||||
|
||||
return avg
|
||||
def incoming_pr_frequency(self):
|
||||
n = len(self.ip_freq_deque)
|
||||
if not n > self.IC_DEQUE_MIN_SAMPLE: return 0
|
||||
else:
|
||||
oldest = self.ip_freq_deque[0]
|
||||
span = time.time() - oldest
|
||||
if span > self.PR_FREQ_DECAY: self.ip_freq_deque.popleft()
|
||||
if span <= 0: return 0
|
||||
hz = n / span
|
||||
return hz
|
||||
|
||||
def outgoing_pr_frequency(self, preemptive=False):
|
||||
n = len(self.op_freq_deque)+(1 if preemptive else 0)
|
||||
if not len(self.op_freq_deque) > 1: return 0
|
||||
else:
|
||||
oldest = self.op_freq_deque[0]
|
||||
span = time.time() - oldest
|
||||
if span > self.PR_FREQ_DECAY: self.op_freq_deque.popleft()
|
||||
if span <= 0: return 0
|
||||
hz = n / span
|
||||
return hz
|
||||
|
||||
def process_announce_queue(self):
|
||||
if not hasattr(self, "announce_cap"):
|
||||
self.announce_cap = RNS.Reticulum.ANNOUNCE_CAP
|
||||
if not self.announce_queue: return
|
||||
|
||||
if hasattr(self, "announce_queue"):
|
||||
if not hasattr(self, "announce_cap"):
|
||||
self.announce_cap = RNS.Reticulum.ANNOUNCE_CAP/100.0
|
||||
|
||||
if self.announce_queue:
|
||||
try:
|
||||
now = time.time()
|
||||
stale = []
|
||||
|
|
@ -266,12 +478,13 @@ class Interface:
|
|||
selected = entries[0]
|
||||
|
||||
now = time.time()
|
||||
tx_time = (len(selected["raw"])*8) / self.bitrate
|
||||
tx_size = len(selected["raw"])
|
||||
tx_time = (tx_size*8) / self.bitrate
|
||||
wait_time = (tx_time / self.announce_cap)
|
||||
self.announce_allowed_at = now + wait_time
|
||||
|
||||
self.process_outgoing(selected["raw"])
|
||||
self.sent_announce()
|
||||
RNS.Transport.transmit(self, selected["raw"])
|
||||
self.sent_announce(tx_size)
|
||||
|
||||
if selected in self.announce_queue:
|
||||
self.announce_queue.remove(selected)
|
||||
|
|
@ -291,13 +504,14 @@ class Interface:
|
|||
def detach(self):
|
||||
pass
|
||||
|
||||
def teardown(self):
|
||||
pass
|
||||
|
||||
@staticmethod
|
||||
def get_config_obj(config_in):
|
||||
if type(config_in) == ConfigObj:
|
||||
return config_in
|
||||
if type(config_in) == ConfigObj: return config_in
|
||||
else:
|
||||
try:
|
||||
return ConfigObj(config_in)
|
||||
try: return ConfigObj(config_in)
|
||||
except Exception as e:
|
||||
RNS.log(f"Could not parse supplied configuration data. The contained exception was: {e}", RNS.LOG_ERROR)
|
||||
raise SystemError("Invalid configuration data supplied")
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -122,7 +122,9 @@ class KISSInterface(Interface):
|
|||
|
||||
self.packet_queue = []
|
||||
self.flow_control = flow_control
|
||||
self.shared_medium = True
|
||||
self.interface_ready = False
|
||||
self.reconnecting = False
|
||||
self.flow_control_timeout = 5
|
||||
self.flow_control_locked = time.time()
|
||||
|
||||
|
|
@ -139,15 +141,17 @@ class KISSInterface(Interface):
|
|||
|
||||
try:
|
||||
self.open_port()
|
||||
if self.serial.is_open: self.configure_device()
|
||||
else: raise IOError("Could not open serial port")
|
||||
|
||||
except Exception as e:
|
||||
RNS.log("Could not open serial port "+self.port, RNS.LOG_ERROR)
|
||||
raise e
|
||||
|
||||
if self.serial.is_open:
|
||||
self.configure_device()
|
||||
else:
|
||||
raise IOError("Could not open serial port")
|
||||
|
||||
RNS.log(f"Could not open serial port for {self}", RNS.LOG_ERROR)
|
||||
RNS.log(f"The contained exception was: {e}", RNS.LOG_ERROR)
|
||||
RNS.log(f"Reticulum will attempt to bring up this interface periodically", RNS.LOG_ERROR)
|
||||
if not self.detached and not self.reconnecting:
|
||||
thread = threading.Thread(target=self.reconnect_port)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
def open_port(self):
|
||||
RNS.log("Opening serial port "+self.port+"...", RNS.LOG_VERBOSE)
|
||||
|
|
@ -165,6 +169,11 @@ class KISSInterface(Interface):
|
|||
dsrdtr = False,
|
||||
)
|
||||
|
||||
def detach(self):
|
||||
self.detached = True
|
||||
self.online = False
|
||||
try: self.serial.close()
|
||||
except Exception as e: RNS.log(f"Could not close port for {self} at detach: {e}", RNS.LOG_ERROR)
|
||||
|
||||
def configure_device(self):
|
||||
# Allow time for interface to initialise before config
|
||||
|
|
@ -245,7 +254,7 @@ class KISSInterface(Interface):
|
|||
|
||||
|
||||
def process_incoming(self, data):
|
||||
self.rxb += len(data)
|
||||
if not data: return
|
||||
self.owner.inbound(data, self)
|
||||
|
||||
|
||||
|
|
@ -359,30 +368,28 @@ class KISSInterface(Interface):
|
|||
RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
|
||||
|
||||
if RNS.Reticulum.panic_on_interface_error:
|
||||
RNS.panic()
|
||||
if RNS.Reticulum.panic_on_interface_error: RNS.panic()
|
||||
|
||||
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
|
||||
|
||||
self.online = False
|
||||
self.serial.close()
|
||||
self.reconnect_port()
|
||||
try: self.serial.close()
|
||||
except Exception as e: RNS.log(f"Could not close port for {self}", RNS.LOG_ERROR)
|
||||
if not self.detached and not self.reconnecting: self.reconnect_port()
|
||||
|
||||
def reconnect_port(self):
|
||||
while not self.online:
|
||||
self.reconnecting = True
|
||||
while not self.online and not self.detached:
|
||||
try:
|
||||
time.sleep(5)
|
||||
RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
|
||||
self.open_port()
|
||||
if self.serial.is_open:
|
||||
self.configure_device()
|
||||
except Exception as e:
|
||||
RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
if self.serial.is_open: self.configure_device()
|
||||
except Exception as e: RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
|
||||
RNS.log("Reconnected serial port for "+str(self))
|
||||
self.reconnecting = False
|
||||
if self.online: RNS.log("Reconnected serial port for "+str(self))
|
||||
|
||||
def should_ingress_limit(self):
|
||||
return False
|
||||
def should_ingress_limit(self): return False
|
||||
|
||||
def __str__(self):
|
||||
return "KISSInterface["+self.name+"]"
|
||||
def __str__(self): return "KISSInterface["+self.name+"]"
|
||||
|
|
@ -30,6 +30,8 @@
|
|||
|
||||
from RNS.Interfaces.Interface import Interface
|
||||
from RNS.Interfaces.BackboneInterface import BackboneInterface
|
||||
from RNS.Interfaces.util.TransmitBuffer import TransmitBuffer
|
||||
from RNS.Interfaces.util.HDLC import HDLC, ReceiveBuffer
|
||||
import socketserver
|
||||
import threading
|
||||
import socket
|
||||
|
|
@ -39,17 +41,6 @@ import os
|
|||
import RNS
|
||||
from threading import Lock
|
||||
|
||||
class HDLC():
|
||||
FLAG = 0x7E
|
||||
ESC = 0x7D
|
||||
ESC_MASK = 0x20
|
||||
|
||||
@staticmethod
|
||||
def escape(data):
|
||||
data = data.replace(bytes([HDLC.ESC]), bytes([HDLC.ESC, HDLC.ESC^HDLC.ESC_MASK]))
|
||||
data = data.replace(bytes([HDLC.FLAG]), bytes([HDLC.ESC, HDLC.FLAG^HDLC.ESC_MASK]))
|
||||
return data
|
||||
|
||||
class ThreadingTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
|
||||
def server_bind(self):
|
||||
if RNS.vendor.platformutils.is_windows():
|
||||
|
|
@ -62,13 +53,17 @@ class ThreadingTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
|
|||
class LocalClientInterface(Interface):
|
||||
RECONNECT_WAIT = 8
|
||||
AUTOCONFIGURE_MTU = True
|
||||
CLIENT_SLEEP_PAUSE_TIMEOUT = 12
|
||||
|
||||
def __init__(self, owner, name, target_port = None, connected_socket=None, socket_path=None):
|
||||
super().__init__()
|
||||
|
||||
self.epoll_backend = False
|
||||
self.HW_MTU = 262144
|
||||
self.online = False
|
||||
self.owner = owner
|
||||
self.bitrate = 1_000_000_000
|
||||
self.epoll_backend = False
|
||||
self.HW_MTU = 262144
|
||||
self.online = False
|
||||
self.ifac_size = self.DEFAULT_IFAC_SIZE
|
||||
|
||||
if socket_path != None and RNS.Reticulum.get_instance().use_af_unix: self.socket_path = f"\0rns/{socket_path}"
|
||||
else: self.socket_path = None
|
||||
|
|
@ -82,11 +77,13 @@ class LocalClientInterface(Interface):
|
|||
self.detached = False
|
||||
self.name = name
|
||||
self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
|
||||
self.frame_buffer = b""
|
||||
self.transmit_buffer = b""
|
||||
self.transmit_buffer = TransmitBuffer()
|
||||
self.receive_buffer = ReceiveBuffer(mtu=lambda: self.HW_MTU, min_frame_len=RNS.Reticulum.HEADER_MINSIZE,
|
||||
max_frame_len=lambda: self.HW_MTU, on_frame=self.process_incoming)
|
||||
|
||||
if RNS.vendor.platformutils.use_epoll():
|
||||
self.epoll_backend = True
|
||||
if RNS.vendor.platformutils.use_epoll(): self.epoll_backend = True
|
||||
|
||||
self.pause_on_client_sleep = False
|
||||
|
||||
if connected_socket != None:
|
||||
self.receives = True
|
||||
|
|
@ -99,6 +96,10 @@ class LocalClientInterface(Interface):
|
|||
|
||||
self.is_connected_to_shared_instance = False
|
||||
|
||||
if RNS.vendor.platformutils.is_android():
|
||||
self.pause_on_client_sleep = True
|
||||
self.pause_timeout = time.time() + self.CLIENT_SLEEP_PAUSE_TIMEOUT
|
||||
|
||||
elif self.socket_path != None:
|
||||
self.receives = True
|
||||
self.target_ip = None
|
||||
|
|
@ -111,8 +112,6 @@ class LocalClientInterface(Interface):
|
|||
self.target_port = target_port
|
||||
self.connect()
|
||||
|
||||
self.owner = owner
|
||||
self.bitrate = 1_000_000_000
|
||||
self.online = True
|
||||
self.writing = False
|
||||
|
||||
|
|
@ -145,6 +144,7 @@ class LocalClientInterface(Interface):
|
|||
self.is_connected_to_shared_instance = True
|
||||
self.never_connected = False
|
||||
|
||||
if RNS.vendor.platformutils.is_android(): self.phy_keepalive = True
|
||||
if self.epoll_backend: BackboneInterface.add_client_socket(self.socket, self)
|
||||
|
||||
return True
|
||||
|
|
@ -185,36 +185,54 @@ class LocalClientInterface(Interface):
|
|||
raise IOError("Attempt to reconnect on a non-initiator local interface")
|
||||
|
||||
|
||||
def send_keepalive(self):
|
||||
if self.online:
|
||||
RNS.log(f"Sending keepalive on {self}", RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
|
||||
try:
|
||||
if self.epoll_backend:
|
||||
self.transmit_buffer.append(bytes([HDLC.FLAG])+bytes([HDLC.FLAG]))
|
||||
BackboneInterface.tx_ready(self)
|
||||
|
||||
else:
|
||||
self.writing = True
|
||||
data = bytes([HDLC.FLAG])+bytes([HDLC.FLAG])
|
||||
self.socket.sendall(data)
|
||||
self.writing = False
|
||||
|
||||
except Exception as e: RNS.log(f"Exception occurred while sending keepalive on {self}: {e}", RNS.LOG_ERROR)
|
||||
|
||||
def process_incoming(self, data):
|
||||
self.rxb += len(data)
|
||||
if self.parent_interface != None: self.parent_interface.rxb += len(data)
|
||||
|
||||
try:
|
||||
self.owner.inbound(data, self)
|
||||
try: self.owner.inbound(data, self)
|
||||
except Exception as e:
|
||||
RNS.log(f"An error in the processing of an incoming frame for {self}: {e}", RNS.LOG_ERROR)
|
||||
RNS.log(f"An error occurred in the processing of an incoming frame for {self}: {e}", RNS.LOG_ERROR)
|
||||
RNS.trace_exception(e)
|
||||
|
||||
def process_outgoing(self, data):
|
||||
if self.pause_on_client_sleep and time.time() > self.pause_timeout:
|
||||
RNS.log(f"TX paused for LocalInterface client, dropping outbound packet", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None # TODO: Remove
|
||||
return
|
||||
|
||||
if self.online:
|
||||
try:
|
||||
if self.epoll_backend:
|
||||
self.transmit_buffer += bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
|
||||
BackboneInterface.tx_ready(self)
|
||||
frame = HDLC.frame(data)
|
||||
if self.tx_stalled or not self.transmit_buffer.append(frame, self.tx_hwm):
|
||||
self.tx_drops += 1
|
||||
self.tx_dropped_bytes += len(frame)
|
||||
RNS.log(f"Egress control dropping outbound frame of {RNS.prettysize(len(frame))} on {self}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
else: BackboneInterface.tx_ready(self)
|
||||
|
||||
else:
|
||||
self.writing = True
|
||||
|
||||
if self._force_bitrate:
|
||||
if not hasattr(self, "send_lock"):
|
||||
self.send_lock = Lock()
|
||||
|
||||
if not hasattr(self, "send_lock"): self.send_lock = Lock()
|
||||
with self.send_lock:
|
||||
# RNS.log(f"Simulating latency of {RNS.prettytime(s)} for {len(data)} bytes", RNS.LOG_EXTREME)
|
||||
s = len(data) / self.bitrate * 8
|
||||
time.sleep(s)
|
||||
|
||||
data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
|
||||
data = HDLC.frame(data)
|
||||
self.socket.sendall(data)
|
||||
self.writing = False
|
||||
self.txb += len(data)
|
||||
|
|
@ -228,23 +246,7 @@ class LocalClientInterface(Interface):
|
|||
self.teardown()
|
||||
|
||||
def handle_hdlc(self, data_in):
|
||||
self.frame_buffer += data_in
|
||||
flags_remaining = True
|
||||
while flags_remaining:
|
||||
frame_start = self.frame_buffer.find(HDLC.FLAG)
|
||||
if frame_start != -1:
|
||||
frame_end = self.frame_buffer.find(HDLC.FLAG, frame_start+1)
|
||||
if frame_end != -1:
|
||||
frame = self.frame_buffer[frame_start+1:frame_end]
|
||||
frame = frame.replace(bytes([HDLC.ESC, HDLC.FLAG ^ HDLC.ESC_MASK]), bytes([HDLC.FLAG]))
|
||||
frame = frame.replace(bytes([HDLC.ESC, HDLC.ESC ^ HDLC.ESC_MASK]), bytes([HDLC.ESC]))
|
||||
if len(frame) > RNS.Reticulum.HEADER_MINSIZE:
|
||||
self.process_incoming(frame)
|
||||
self.frame_buffer = self.frame_buffer[frame_end:]
|
||||
else:
|
||||
flags_remaining = False
|
||||
else:
|
||||
flags_remaining = False
|
||||
self.receive_buffer.feed(data_in)
|
||||
|
||||
def receive(self, data_in):
|
||||
try:
|
||||
|
|
@ -258,8 +260,7 @@ class LocalClientInterface(Interface):
|
|||
# there's no other connectivity left to block anyway, it might be
|
||||
# unnecessary.
|
||||
self.reconnect()
|
||||
else:
|
||||
self.teardown(nowarning=True)
|
||||
else: self.teardown(nowarning=True)
|
||||
|
||||
except Exception as e:
|
||||
self.online = False
|
||||
|
|
@ -267,9 +268,11 @@ class LocalClientInterface(Interface):
|
|||
RNS.log("Tearing down "+str(self), RNS.LOG_ERROR)
|
||||
self.teardown()
|
||||
|
||||
if self.pause_on_client_sleep: self.pause_timeout = time.time() + self.CLIENT_SLEEP_PAUSE_TIMEOUT
|
||||
|
||||
def read_loop(self):
|
||||
try:
|
||||
self.frame_buffer = b""
|
||||
self.receive_buffer.reset()
|
||||
data_in = b""
|
||||
while True:
|
||||
data_in = self.socket.recv(4096)
|
||||
|
|
@ -320,15 +323,15 @@ class LocalClientInterface(Interface):
|
|||
self.OUT = False
|
||||
self.IN = False
|
||||
|
||||
if self in RNS.Transport.interfaces:
|
||||
RNS.Transport.interfaces.remove(self)
|
||||
RNS.Transport.remove_interface(self)
|
||||
|
||||
if self in RNS.Transport.local_client_interfaces:
|
||||
RNS.Transport.local_client_interfaces.remove(self)
|
||||
if hasattr(self, "parent_interface") and self.parent_interface != None:
|
||||
self.parent_interface.clients -= 1
|
||||
if hasattr(RNS.Transport, "owner") and RNS.Transport.owner != None:
|
||||
RNS.Transport.owner._should_persist_data()
|
||||
background = not self.detached
|
||||
RNS.Transport.owner._should_persist_data(background=background)
|
||||
|
||||
if nowarning == False:
|
||||
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is being torn down. Restart Reticulum to attempt to open this interface again.", RNS.LOG_ERROR)
|
||||
|
|
@ -355,6 +358,7 @@ class LocalServerInterface(Interface):
|
|||
self.epoll_backend = False
|
||||
self.online = False
|
||||
self.clients = 0
|
||||
self.ifac_size = self.DEFAULT_IFAC_SIZE
|
||||
|
||||
if socket_path != None and RNS.Reticulum.get_instance().use_af_unix: self.socket_path = f"\0rns/{socket_path}"
|
||||
else: self.socket_path = None
|
||||
|
|
@ -402,6 +406,7 @@ class LocalServerInterface(Interface):
|
|||
self.announce_rate_grace = None
|
||||
self.announce_rate_penalty = None
|
||||
|
||||
self.HW_MTU = 262144
|
||||
self.bitrate = 1000*1000*1000
|
||||
self.online = True
|
||||
|
||||
|
|
@ -430,7 +435,7 @@ class LocalServerInterface(Interface):
|
|||
spawned_interface.socket_path = self.socket_path
|
||||
|
||||
if hasattr(self, "_force_bitrate"): spawned_interface._force_bitrate = self._force_bitrate
|
||||
RNS.Transport.interfaces.append(spawned_interface)
|
||||
RNS.Transport.add_interface(spawned_interface)
|
||||
RNS.Transport.local_client_interfaces.append(spawned_interface)
|
||||
BackboneInterface.add_client_socket(client_socket, spawned_interface)
|
||||
self.clients += 1
|
||||
|
|
@ -446,7 +451,7 @@ class LocalServerInterface(Interface):
|
|||
spawned_interface.parent_interface = self
|
||||
spawned_interface.bitrate = self.bitrate
|
||||
if hasattr(self, "_force_bitrate"): spawned_interface._force_bitrate = self._force_bitrate
|
||||
RNS.Transport.interfaces.append(spawned_interface)
|
||||
RNS.Transport.add_interface(spawned_interface)
|
||||
RNS.Transport.local_client_interfaces.append(spawned_interface)
|
||||
self.clients += 1
|
||||
spawned_interface.read_loop()
|
||||
|
|
@ -454,11 +459,25 @@ class LocalServerInterface(Interface):
|
|||
def process_outgoing(self, data):
|
||||
pass
|
||||
|
||||
def received_announce(self, from_spawned=False):
|
||||
if from_spawned: self.ia_freq_deque.append(time.time())
|
||||
def received_announce(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.ia_freq_deque.append(time.time())
|
||||
self.arxb += size
|
||||
|
||||
def sent_announce(self, from_spawned=False):
|
||||
if from_spawned: self.oa_freq_deque.append(time.time())
|
||||
def sent_announce(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.oa_freq_deque.append(time.time())
|
||||
self.atxb += size
|
||||
|
||||
def received_path_request(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.ip_freq_deque.append(time.time())
|
||||
self.prxb += size
|
||||
|
||||
def sent_path_request(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.op_freq_deque.append(time.time())
|
||||
self.ptxb += size
|
||||
|
||||
def __str__(self):
|
||||
if self.socket_path: return "Shared Instance["+str(self.socket_path.replace("\0", ""))+"]"
|
||||
|
|
@ -469,5 +488,4 @@ class LocalInterfaceHandler(socketserver.BaseRequestHandler):
|
|||
self.callback = callback
|
||||
socketserver.BaseRequestHandler.__init__(self, *args, **keys)
|
||||
|
||||
def handle(self):
|
||||
self.callback(handler=self)
|
||||
def handle(self): self.callback(handler=self)
|
||||
|
|
@ -84,6 +84,7 @@ class PipeInterface(Interface):
|
|||
self.pipe_is_open = False
|
||||
self.bitrate = PipeInterface.BITRATE_GUESS
|
||||
self.respawn_delay = respawn_delay
|
||||
self.shared_medium = True
|
||||
|
||||
try:
|
||||
self.open_pipe()
|
||||
|
|
@ -119,7 +120,7 @@ class PipeInterface(Interface):
|
|||
|
||||
|
||||
def process_incoming(self, data):
|
||||
self.rxb += len(data)
|
||||
if not data: return
|
||||
self.owner.inbound(data, self)
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -258,9 +258,11 @@ class RNodeInterface(Interface):
|
|||
self.r_stat_tx = None
|
||||
self.r_stat_rssi = None
|
||||
self.r_stat_snr = None
|
||||
self.r_stat_q = None
|
||||
self.r_st_alock = None
|
||||
self.r_lt_alock = None
|
||||
self.r_random = None
|
||||
self.r_modulation = "LoRa"
|
||||
self.r_airtime_short = 0.0
|
||||
self.r_airtime_long = 0.0
|
||||
self.r_channel_load_short = 0.0
|
||||
|
|
@ -278,6 +280,7 @@ class RNodeInterface(Interface):
|
|||
self.r_noise_floor = None
|
||||
self.r_interference = None
|
||||
self.r_interference_l = None
|
||||
self.reports_phy_stats = True
|
||||
|
||||
self.r_battery_state = RNodeInterface.BATTERY_STATE_UNKNOWN
|
||||
self.r_battery_percent = 0
|
||||
|
|
@ -294,6 +297,7 @@ class RNodeInterface(Interface):
|
|||
|
||||
self.packet_queue = []
|
||||
self.flow_control = flow_control
|
||||
self.shared_medium = True
|
||||
self.interface_ready = False
|
||||
self.announce_rate_target = None
|
||||
self.supports_discovery = True
|
||||
|
|
@ -383,6 +387,7 @@ class RNodeInterface(Interface):
|
|||
RNS.log(f"Opening BLE connection for {self}...")
|
||||
self.timeout = 1250
|
||||
if self.ble != None and self.ble.running == False:
|
||||
RNS.log(f"Cleaning up previous BLE connection for {self}...")
|
||||
self.ble.close()
|
||||
self.ble.cleanup()
|
||||
self.ble = None
|
||||
|
|
@ -439,7 +444,11 @@ class RNodeInterface(Interface):
|
|||
ble_detect_timeout = 5.0
|
||||
detect_time = time.time()
|
||||
while not self.detected and time.time() < detect_time + ble_detect_timeout: time.sleep(0.1)
|
||||
if not self.detected: RNS.log(f"RNode detect timed out over BLE", RNS.LOG_ERROR)
|
||||
if not self.detected:
|
||||
RNS.log(f"RNode detect timed out over BLE", RNS.LOG_ERROR)
|
||||
if self.ble:
|
||||
self.ble.should_run = False
|
||||
self.ble.must_disconnect = True
|
||||
else:
|
||||
sleep(0.2)
|
||||
|
||||
|
|
@ -477,8 +486,7 @@ class RNodeInterface(Interface):
|
|||
self.setLTALock()
|
||||
self.setRadioState(KISS.RADIO_STATE_ON)
|
||||
|
||||
if self.use_ble:
|
||||
time.sleep(2)
|
||||
if self.use_ble: time.sleep(2)
|
||||
|
||||
def detect(self):
|
||||
kiss_command = bytes([KISS.FEND, KISS.CMD_DETECT, KISS.DETECT_REQ, KISS.FEND, KISS.CMD_FW_VERSION, 0x00, KISS.FEND, KISS.CMD_PLATFORM, 0x00, KISS.FEND, KISS.CMD_MCU, 0x00, KISS.FEND])
|
||||
|
|
@ -684,10 +692,8 @@ class RNodeInterface(Interface):
|
|||
RNS.log("Radio state mismatch", RNS.LOG_ERROR)
|
||||
self.validcfg = False
|
||||
|
||||
if (self.validcfg):
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
if (self.validcfg): return True
|
||||
else: return False
|
||||
|
||||
|
||||
def updateBitrate(self):
|
||||
|
|
@ -699,11 +705,8 @@ class RNodeInterface(Interface):
|
|||
self.bitrate = 0
|
||||
|
||||
def process_incoming(self, data):
|
||||
self.rxb += len(data)
|
||||
if not data: return
|
||||
self.owner.inbound(data, self)
|
||||
self.r_stat_rssi = None
|
||||
self.r_stat_snr = None
|
||||
|
||||
|
||||
def process_outgoing(self,data):
|
||||
datalen = len(data)
|
||||
|
|
@ -1154,19 +1157,17 @@ class RNodeInterface(Interface):
|
|||
|
||||
except Exception as e:
|
||||
self.online = False
|
||||
RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
|
||||
if not self.detached:
|
||||
RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
|
||||
|
||||
if RNS.Reticulum.panic_on_interface_error:
|
||||
RNS.panic()
|
||||
if RNS.Reticulum.panic_on_interface_error: RNS.panic()
|
||||
|
||||
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
|
||||
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
|
||||
|
||||
self.online = False
|
||||
try:
|
||||
self.serial.close()
|
||||
except Exception as e:
|
||||
pass
|
||||
try: self.serial.close()
|
||||
except Exception as e: pass
|
||||
|
||||
if not self.detached and not self.reconnecting:
|
||||
self.reconnect_port()
|
||||
|
|
@ -1192,14 +1193,20 @@ class RNodeInterface(Interface):
|
|||
self.disable_external_framebuffer()
|
||||
self.setRadioState(KISS.RADIO_STATE_OFF)
|
||||
self.leave()
|
||||
self.serial.close()
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"An error occurred while detaching {self}: {e}", RNS.LOG_ERROR)
|
||||
|
||||
if self.use_ble: self.ble.close()
|
||||
if self.use_tcp:
|
||||
if self.use_ble:
|
||||
self.ble.close()
|
||||
self.ble.cleanup()
|
||||
self.ble = None
|
||||
elif self.use_tcp:
|
||||
time.sleep(0.5)
|
||||
self.tcp.close()
|
||||
else:
|
||||
self.serial.close()
|
||||
|
||||
def should_ingress_limit(self):
|
||||
return False
|
||||
|
|
@ -1297,6 +1304,7 @@ class BLEConnection():
|
|||
self.must_disconnect = False
|
||||
self.connect_job_running = False
|
||||
self.device_disappeared = False
|
||||
self._windows_paired_addrs = None
|
||||
|
||||
import importlib.util
|
||||
if BLEConnection.bleak == None:
|
||||
|
|
@ -1307,8 +1315,8 @@ class BLEConnection():
|
|||
import asyncio
|
||||
BLEConnection.asyncio = asyncio
|
||||
else:
|
||||
RNS.log("Using the RNode interface over BLE requires a the \"bleak\" module to be installed.", RNS.LOG_CRITICAL)
|
||||
RNS.log("You can install one with the command: python3 -m pip install bleak", RNS.LOG_CRITICAL)
|
||||
RNS.log("Using the RNode interface over BLE requires the \"bleak\" module to be installed.", RNS.LOG_CRITICAL)
|
||||
RNS.log("You can install it with the command: python3 -m pip install bleak", RNS.LOG_CRITICAL)
|
||||
RNS.panic()
|
||||
|
||||
self.should_run = True
|
||||
|
|
@ -1324,6 +1332,7 @@ class BLEConnection():
|
|||
self.should_run = False
|
||||
|
||||
def connection_job(self):
|
||||
self.running = True
|
||||
while self.should_run:
|
||||
if self.ble_device == None:
|
||||
self.ble_device = self.find_target_device()
|
||||
|
|
@ -1350,6 +1359,7 @@ class BLEConnection():
|
|||
self.owner.ble_receive(data)
|
||||
|
||||
self.connected = True
|
||||
self.device_disappeared = False
|
||||
self.ble_device = ble_client
|
||||
self.last_client = ble_client
|
||||
self.owner.port = str(f"ble://{ble_client.address}")
|
||||
|
|
@ -1384,6 +1394,8 @@ class BLEConnection():
|
|||
|
||||
def find_target_device(self):
|
||||
RNS.log(f"Searching for attachable BLE device for {self.owner}...", RNS.LOG_EXTREME)
|
||||
if RNS.vendor.platformutils.is_windows():
|
||||
self._windows_paired_addrs = self._get_windows_paired_ble_addresses()
|
||||
def device_filter(device: self.bleak.backends.device.BLEDevice, adv: self.bleak.backends.scanner.AdvertisementData):
|
||||
if BLEConnection.UART_SERVICE_UUID.lower() in adv.service_uuids:
|
||||
if self.device_bonded(device):
|
||||
|
|
@ -1415,6 +1427,9 @@ class BLEConnection():
|
|||
|
||||
def device_bonded(self, device):
|
||||
try:
|
||||
if self._windows_paired_addrs is not None:
|
||||
return device.address is not None and device.address.lower() in self._windows_paired_addrs
|
||||
|
||||
if hasattr(device, "details"):
|
||||
if "props" in device.details and "Bonded" in device.details["props"]:
|
||||
if device.details["props"]["Bonded"] == True:
|
||||
|
|
@ -1425,6 +1440,15 @@ class BLEConnection():
|
|||
|
||||
return False
|
||||
|
||||
def _get_windows_paired_ble_addresses(self):
|
||||
from winrt.windows.devices.bluetooth import BluetoothLEDevice
|
||||
from winrt.windows.devices.enumeration import DeviceInformation
|
||||
async def _query():
|
||||
selector = BluetoothLEDevice.get_device_selector_from_pairing_state(True)
|
||||
infos = await DeviceInformation.find_all_async_aqs_filter(selector)
|
||||
return set(info.id.split("-")[-1].lower() for info in infos)
|
||||
return self.asyncio.run(_query())
|
||||
|
||||
class TCPConnection():
|
||||
TARGET_PORT = 7633
|
||||
CONNECT_TIMEOUT = 5.0
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -269,6 +269,7 @@ class RNodeMultiInterface(Interface):
|
|||
self.r_stat_tx = None
|
||||
self.r_stat_rssi = None
|
||||
self.r_stat_snr = None
|
||||
self.r_stat_q = None
|
||||
self.r_st_alock = None
|
||||
self.r_lt_alock = None
|
||||
self.r_random = None
|
||||
|
|
@ -276,6 +277,8 @@ class RNodeMultiInterface(Interface):
|
|||
self.packet_queue = []
|
||||
self.interface_ready = False
|
||||
self.announce_rate_target = None
|
||||
self.reports_phy_stats = True
|
||||
self.r_modulation = "LoRa"
|
||||
|
||||
self.validcfg = True
|
||||
if id_interval != None and id_callsign != None:
|
||||
|
|
@ -375,7 +378,7 @@ class RNodeMultiInterface(Interface):
|
|||
interface.mode = self.mode
|
||||
interface.HW_MTU = self.HW_MTU
|
||||
interface.detected = True
|
||||
RNS.Transport.interfaces.append(interface)
|
||||
RNS.Transport.add_interface(interface)
|
||||
RNS.log("Spawned new RNode subinterface: "+str(interface), RNS.LOG_VERBOSE)
|
||||
|
||||
self.clients += 1
|
||||
|
|
@ -543,11 +546,25 @@ class RNodeMultiInterface(Interface):
|
|||
if written != len(frame):
|
||||
raise IOError("Serial interface only wrote "+str(written)+" bytes of "+str(len(data)))
|
||||
|
||||
def received_announce(self, from_spawned=False):
|
||||
if from_spawned: self.ia_freq_deque.append(time.time())
|
||||
def received_announce(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.ia_freq_deque.append(time.time())
|
||||
self.arxb += size
|
||||
|
||||
def sent_announce(self, from_spawned=False):
|
||||
if from_spawned: self.oa_freq_deque.append(time.time())
|
||||
def sent_announce(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.oa_freq_deque.append(time.time())
|
||||
self.atxb += size
|
||||
|
||||
def received_path_request(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.ip_freq_deque.append(time.time())
|
||||
self.prxb += size
|
||||
|
||||
def sent_path_request(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.op_freq_deque.append(time.time())
|
||||
self.ptxb += size
|
||||
|
||||
def readLoop(self):
|
||||
try:
|
||||
|
|
@ -899,12 +916,12 @@ class RNodeMultiInterface(Interface):
|
|||
if interface != 0:
|
||||
self.setRadioState(KISS.RADIO_STATE_OFF, interface)
|
||||
self.leave()
|
||||
self.serial.close()
|
||||
|
||||
def teardown_subinterfaces(self):
|
||||
for interface in self.subinterfaces:
|
||||
if interface != 0:
|
||||
if interface in RNS.Transport.interfaces:
|
||||
RNS.Transport.interfaces.remove(interface)
|
||||
RNS.Transport.remove_interface(interface)
|
||||
self.subinterfaces[interface.index] = 0
|
||||
|
||||
def should_ingress_limit(self):
|
||||
|
|
@ -988,6 +1005,7 @@ class RNodeSubInterface(Interface):
|
|||
self.r_premable_time_ms = None
|
||||
|
||||
self.packet_queue = []
|
||||
self.shared_medium = True
|
||||
self.interface_ready = False
|
||||
self.parent_interface = parent_interface
|
||||
self.announce_rate_target = None
|
||||
|
|
@ -1111,10 +1129,8 @@ class RNodeSubInterface(Interface):
|
|||
self.bitrate = 0
|
||||
|
||||
def process_incoming(self, data):
|
||||
self.rxb += len(data)
|
||||
if not data: return
|
||||
self.owner.inbound(data, self)
|
||||
self.r_stat_rssi = None
|
||||
self.r_stat_snr = None
|
||||
|
||||
def process_outgoing(self,data):
|
||||
if self.online:
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -83,6 +83,7 @@ class SerialInterface(Interface):
|
|||
raise ValueError("No port specified for serial interface")
|
||||
|
||||
self.HW_MTU = 564
|
||||
self.shared_medium = True
|
||||
|
||||
self.pyserial = serial
|
||||
self.serial = None
|
||||
|
|
@ -96,6 +97,7 @@ class SerialInterface(Interface):
|
|||
self.timeout = 100
|
||||
self.online = False
|
||||
self.bitrate = self.speed
|
||||
self.reconnecting = False
|
||||
|
||||
if parity.lower() == "e" or parity.lower() == "even":
|
||||
self.parity = serial.PARITY_EVEN
|
||||
|
|
@ -105,15 +107,17 @@ class SerialInterface(Interface):
|
|||
|
||||
try:
|
||||
self.open_port()
|
||||
if self.serial.is_open: self.configure_device()
|
||||
else: raise IOError("Could not open serial port")
|
||||
|
||||
except Exception as e:
|
||||
RNS.log("Could not open serial port for interface "+str(self), RNS.LOG_ERROR)
|
||||
raise e
|
||||
|
||||
if self.serial.is_open:
|
||||
self.configure_device()
|
||||
else:
|
||||
raise IOError("Could not open serial port")
|
||||
|
||||
RNS.log(f"Could not open serial port for {self}", RNS.LOG_ERROR)
|
||||
RNS.log(f"The contained exception was: {e}", RNS.LOG_ERROR)
|
||||
RNS.log(f"Reticulum will attempt to bring up this interface periodically", RNS.LOG_ERROR)
|
||||
if not self.detached and not self.reconnecting:
|
||||
thread = threading.Thread(target=self.reconnect_port)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
def open_port(self):
|
||||
RNS.log("Opening serial port "+self.port+"...", RNS.LOG_VERBOSE)
|
||||
|
|
@ -131,6 +135,11 @@ class SerialInterface(Interface):
|
|||
dsrdtr = False,
|
||||
)
|
||||
|
||||
def detach(self):
|
||||
self.detached = True
|
||||
self.online = False
|
||||
try: self.serial.close()
|
||||
except Exception as e: RNS.log(f"Could not close port for {self} at detach: {e}", RNS.LOG_ERROR)
|
||||
|
||||
def configure_device(self):
|
||||
sleep(0.5)
|
||||
|
|
@ -140,12 +149,10 @@ class SerialInterface(Interface):
|
|||
self.online = True
|
||||
RNS.log("Serial port "+self.port+" is now open", RNS.LOG_VERBOSE)
|
||||
|
||||
|
||||
def process_incoming(self, data):
|
||||
self.rxb += len(data)
|
||||
if not data: return
|
||||
self.owner.inbound(data, self)
|
||||
|
||||
|
||||
def process_outgoing(self,data):
|
||||
if self.online:
|
||||
data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
|
||||
|
|
@ -154,7 +161,6 @@ class SerialInterface(Interface):
|
|||
if written != len(data):
|
||||
raise IOError("Serial interface only wrote "+str(written)+" bytes of "+str(len(data)))
|
||||
|
||||
|
||||
def readLoop(self):
|
||||
try:
|
||||
in_frame = False
|
||||
|
|
@ -198,30 +204,28 @@ class SerialInterface(Interface):
|
|||
RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
|
||||
|
||||
if RNS.Reticulum.panic_on_interface_error:
|
||||
RNS.panic()
|
||||
if RNS.Reticulum.panic_on_interface_error: RNS.panic()
|
||||
|
||||
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
|
||||
|
||||
self.online = False
|
||||
self.serial.close()
|
||||
self.reconnect_port()
|
||||
try: self.serial.close()
|
||||
except Exception as e: RNS.log(f"Could not close port for {self}", RNS.LOG_ERROR)
|
||||
if not self.detached and not self.reconnecting: self.reconnect_port()
|
||||
|
||||
def reconnect_port(self):
|
||||
while not self.online:
|
||||
self.reconnecting = True
|
||||
while not self.online and not self.detached:
|
||||
try:
|
||||
time.sleep(5)
|
||||
RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
|
||||
self.open_port()
|
||||
if self.serial.is_open:
|
||||
self.configure_device()
|
||||
except Exception as e:
|
||||
RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
if self.serial.is_open: self.configure_device()
|
||||
except Exception as e: RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
|
||||
RNS.log("Reconnected serial port for "+str(self))
|
||||
self.reconnecting = False
|
||||
if self.online: RNS.log("Reconnected serial port for "+str(self))
|
||||
|
||||
def should_ingress_limit(self):
|
||||
return False
|
||||
def should_ingress_limit(self): return False
|
||||
|
||||
def __str__(self):
|
||||
return "SerialInterface["+self.name+"]"
|
||||
def __str__(self): return "SerialInterface["+self.name+"]"
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -126,6 +126,8 @@ class TCPClientInterface(Interface):
|
|||
self.writing = False
|
||||
self.online = False
|
||||
self.detached = False
|
||||
self.tunnels_allowed = True
|
||||
self.lossless_medium = True
|
||||
self.kiss_framing = kiss_framing
|
||||
self.i2p_tunneled = i2p_tunneled
|
||||
self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
|
||||
|
|
@ -158,13 +160,13 @@ class TCPClientInterface(Interface):
|
|||
self.connect_timeout = connect_timeout
|
||||
else:
|
||||
self.connect_timeout = TCPClientInterface.INITIAL_CONNECT_TIMEOUT
|
||||
|
||||
if TCPClientInterface.SYNCHRONOUS_START:
|
||||
self.initial_connect()
|
||||
else:
|
||||
thread = threading.Thread(target=self.initial_connect)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
def final_init(self):
|
||||
if TCPClientInterface.SYNCHRONOUS_START: self.initial_connect()
|
||||
else:
|
||||
thread = threading.Thread(target=self.initial_connect)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
def initial_connect(self):
|
||||
if not self.connect(initial=True):
|
||||
|
|
@ -175,8 +177,6 @@ class TCPClientInterface(Interface):
|
|||
thread = threading.Thread(target=self.read_loop)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
if not self.kiss_framing:
|
||||
self.wants_tunnel = True
|
||||
|
||||
def set_timeouts_linux(self):
|
||||
if not self.i2p_tunneled:
|
||||
|
|
@ -208,6 +208,7 @@ class TCPClientInterface(Interface):
|
|||
|
||||
def detach(self):
|
||||
self.online = False
|
||||
self.detached = True
|
||||
if self.socket != None:
|
||||
if hasattr(self.socket, "close"):
|
||||
if callable(self.socket.close):
|
||||
|
|
@ -227,6 +228,20 @@ class TCPClientInterface(Interface):
|
|||
|
||||
self.socket = None
|
||||
|
||||
def early_synthesis(self, packet):
|
||||
if not self.detached:
|
||||
if not packet.packed: packet.pack()
|
||||
if self.ifac_identity: raw = RNS.Transport.handle_outgoing_ifac(self, packet.raw);
|
||||
else: raw = packet.raw
|
||||
|
||||
if self.kiss_framing: frame = bytes([KISS.FEND])+bytes([KISS.CMD_DATA])+KISS.escape(raw)+bytes([KISS.FEND])
|
||||
else: frame = bytes([HDLC.FLAG])+HDLC.escape(raw)+bytes([HDLC.FLAG])
|
||||
|
||||
self.socket.sendall(frame)
|
||||
self.txb += len(frame)
|
||||
if hasattr(self, "parent_interface") and self.parent_interface != None:
|
||||
self.parent_interface.txb += len(frame)
|
||||
|
||||
def connect(self, initial=False):
|
||||
try:
|
||||
if initial:
|
||||
|
|
@ -241,6 +256,13 @@ class TCPClientInterface(Interface):
|
|||
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
|
||||
self.socket.connect(target_address)
|
||||
self.socket.settimeout(None)
|
||||
|
||||
if not self.kiss_framing:
|
||||
self.wants_tunnel = True
|
||||
self.remote_transport = None
|
||||
self.synthesis_received = False
|
||||
RNS.Transport.synthesize_tunnel(self)
|
||||
|
||||
self.online = True
|
||||
|
||||
if initial:
|
||||
|
|
@ -272,7 +294,7 @@ class TCPClientInterface(Interface):
|
|||
if not self.reconnecting:
|
||||
self.reconnecting = True
|
||||
attempts = 0
|
||||
while not self.online:
|
||||
while not self.online and not self.detached:
|
||||
time.sleep(TCPClientInterface.RECONNECT_WAIT)
|
||||
attempts += 1
|
||||
|
||||
|
|
@ -281,9 +303,7 @@ class TCPClientInterface(Interface):
|
|||
self.teardown()
|
||||
break
|
||||
|
||||
try:
|
||||
self.connect()
|
||||
|
||||
try: self.connect()
|
||||
except Exception as e:
|
||||
RNS.log("Connection attempt for "+str(self)+" failed: "+str(e), RNS.LOG_DEBUG)
|
||||
|
||||
|
|
@ -294,19 +314,14 @@ class TCPClientInterface(Interface):
|
|||
thread = threading.Thread(target=self.read_loop)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
if not self.kiss_framing:
|
||||
RNS.Transport.synthesize_tunnel(self)
|
||||
|
||||
else:
|
||||
RNS.log("Attempt to reconnect on a non-initiator TCP interface. This should not happen.", RNS.LOG_ERROR)
|
||||
raise IOError("Attempt to reconnect on a non-initiator TCP interface")
|
||||
|
||||
def process_incoming(self, data):
|
||||
if not data: return
|
||||
if self.online and not self.detached:
|
||||
self.rxb += len(data)
|
||||
if hasattr(self, "parent_interface") and self.parent_interface != None:
|
||||
self.parent_interface.rxb += len(data)
|
||||
|
||||
self.owner.inbound(data, self)
|
||||
|
||||
def process_outgoing(self, data):
|
||||
|
|
@ -333,6 +348,13 @@ class TCPClientInterface(Interface):
|
|||
RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
self.teardown()
|
||||
|
||||
def check_frame_len(self, frame_len):
|
||||
if frame_len <= RNS.Reticulum.HEADER_MINSIZE: return False
|
||||
elif frame_len > self.HW_MTU + (self.ifac_size or 0): return False
|
||||
else: return True
|
||||
|
||||
def invalid_frame(self, frame_len):
|
||||
RNS.log(f"Invalid HDLC frame of {RNS.prettysize(frame_len)} received on {self}, dropping frame", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
|
||||
def read_loop(self):
|
||||
try:
|
||||
|
|
@ -389,12 +411,18 @@ class TCPClientInterface(Interface):
|
|||
frame = frame_buffer[frame_start+1:frame_end]
|
||||
frame = frame.replace(bytes([HDLC.ESC, HDLC.FLAG ^ HDLC.ESC_MASK]), bytes([HDLC.FLAG]))
|
||||
frame = frame.replace(bytes([HDLC.ESC, HDLC.ESC ^ HDLC.ESC_MASK]), bytes([HDLC.ESC]))
|
||||
if len(frame) > RNS.Reticulum.HEADER_MINSIZE:
|
||||
self.process_incoming(frame)
|
||||
frame_len = len(frame)
|
||||
if frame_len != 0:
|
||||
if self.check_frame_len(frame_len): self.process_incoming(frame)
|
||||
else: self.invalid_frame(len(frame))
|
||||
|
||||
frame_buffer = frame_buffer[frame_end:]
|
||||
|
||||
else:
|
||||
if len(frame_buffer) > self.HW_MTU*2: frame_buffer = b""
|
||||
flags_remaining = False
|
||||
else:
|
||||
frame_buffer = b""
|
||||
flags_remaining = False
|
||||
|
||||
else:
|
||||
|
|
@ -403,7 +431,7 @@ class TCPClientInterface(Interface):
|
|||
RNS.log("The socket for "+str(self)+" was closed, attempting to reconnect...", RNS.LOG_WARNING)
|
||||
self.reconnect()
|
||||
else:
|
||||
RNS.log("The socket for remote client "+str(self)+" was closed.", RNS.LOG_VERBOSE)
|
||||
RNS.log("The socket for remote client "+str(self)+" was closed.", RNS.LOG_DEBUG)
|
||||
self.teardown()
|
||||
|
||||
break
|
||||
|
|
@ -413,7 +441,7 @@ class TCPClientInterface(Interface):
|
|||
self.online = False
|
||||
RNS.log("An interface error occurred for "+str(self)+", the contained exception was: "+str(e), RNS.LOG_WARNING)
|
||||
|
||||
if self.initiator:
|
||||
if self.initiator and not self.detached:
|
||||
RNS.log("Attempting to reconnect...", RNS.LOG_WARNING)
|
||||
self.reconnect()
|
||||
else:
|
||||
|
|
@ -436,16 +464,12 @@ class TCPClientInterface(Interface):
|
|||
while self in self.parent_interface.spawned_interfaces:
|
||||
self.parent_interface.spawned_interfaces.remove(self)
|
||||
|
||||
if self in RNS.Transport.interfaces:
|
||||
if not self.initiator:
|
||||
RNS.Transport.interfaces.remove(self)
|
||||
|
||||
if not self.initiator:
|
||||
RNS.Transport.remove_interface(self)
|
||||
|
||||
def __str__(self):
|
||||
if ":" in self.target_ip:
|
||||
ip_str = f"[{self.target_ip}]"
|
||||
else:
|
||||
ip_str = f"{self.target_ip}"
|
||||
if ":" in self.target_ip: ip_str = f"[{self.target_ip}]"
|
||||
else: ip_str = f"{self.target_ip}"
|
||||
|
||||
return "TCPInterface["+str(self.name)+"/"+ip_str+":"+str(self.target_port)+"]"
|
||||
|
||||
|
|
@ -522,6 +546,8 @@ class TCPServerInterface(Interface):
|
|||
self.OUT = False
|
||||
self.name = name
|
||||
self.detached = False
|
||||
self.tunnels_allowed = True
|
||||
self.lossless_medium = True
|
||||
|
||||
self.i2p_tunneled = i2p_tunneled
|
||||
self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
|
||||
|
|
@ -579,6 +605,24 @@ class TCPServerInterface(Interface):
|
|||
spawned_interface = TCPClientInterface(self.owner, spawned_configuration, connected_socket=handler.request)
|
||||
spawned_interface.OUT = self.OUT
|
||||
spawned_interface.IN = self.IN
|
||||
|
||||
spawned_interface.ingress_control = self.ingress_control
|
||||
spawned_interface.ic_max_held_announces = self.ic_max_held_announces
|
||||
spawned_interface.ic_burst_hold = self.ic_burst_hold
|
||||
spawned_interface.ic_burst_freq = self.ic_burst_freq
|
||||
spawned_interface.ic_burst_freq_new = self.ic_burst_freq_new
|
||||
spawned_interface.ic_new_time = self.ic_new_time
|
||||
spawned_interface.ic_burst_penalty = self.ic_burst_penalty
|
||||
spawned_interface.ic_held_release_interval = self.ic_held_release_interval
|
||||
|
||||
spawned_interface.egress_control = self.egress_control
|
||||
spawned_interface.ec_pr_freq = self.ec_pr_freq
|
||||
spawned_interface.ic_pr_burst_freq_new = self.ic_pr_burst_freq_new
|
||||
spawned_interface.ic_pr_burst_freq = self.ic_pr_burst_freq
|
||||
|
||||
spawned_interface._destination_cost = self._destination_cost
|
||||
spawned_interface._pr_cost = self._pr_cost
|
||||
|
||||
spawned_interface.target_ip = handler.client_address[0]
|
||||
spawned_interface.target_port = str(handler.client_address[1])
|
||||
spawned_interface.parent_interface = self
|
||||
|
|
@ -609,20 +653,40 @@ class TCPServerInterface(Interface):
|
|||
spawned_interface.announce_rate_grace = self.announce_rate_grace
|
||||
spawned_interface.announce_rate_penalty = self.announce_rate_penalty
|
||||
spawned_interface.mode = self.mode
|
||||
spawned_interface.gravity = self.gravity
|
||||
spawned_interface.hop_limiter = self.hop_limiter
|
||||
spawned_interface.recursive_prs = self.recursive_prs
|
||||
spawned_interface.announces_from_internal = self.announces_from_internal
|
||||
spawned_interface.announces_to_internal = self.announces_to_internal
|
||||
spawned_interface.announce_cap = self.announce_cap
|
||||
spawned_interface.HW_MTU = self.HW_MTU
|
||||
spawned_interface.online = True
|
||||
RNS.log("Spawned new TCPClient Interface: "+str(spawned_interface), RNS.LOG_VERBOSE)
|
||||
RNS.Transport.interfaces.append(spawned_interface)
|
||||
RNS.Transport.add_interface(spawned_interface)
|
||||
while spawned_interface in self.spawned_interfaces:
|
||||
self.spawned_interfaces.remove(spawned_interface)
|
||||
self.spawned_interfaces.append(spawned_interface)
|
||||
spawned_interface.read_loop()
|
||||
|
||||
def received_announce(self, from_spawned=False):
|
||||
if from_spawned: self.ia_freq_deque.append(time.time())
|
||||
def received_announce(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.ia_freq_deque.append(time.time())
|
||||
self.arxb += size
|
||||
|
||||
def sent_announce(self, from_spawned=False):
|
||||
if from_spawned: self.oa_freq_deque.append(time.time())
|
||||
def sent_announce(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.oa_freq_deque.append(time.time())
|
||||
self.atxb += size
|
||||
|
||||
def received_path_request(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.ip_freq_deque.append(time.time())
|
||||
self.prxb += size
|
||||
|
||||
def sent_path_request(self, size=0, from_spawned=False):
|
||||
if from_spawned:
|
||||
self.op_freq_deque.append(time.time())
|
||||
self.ptxb += size
|
||||
|
||||
def process_outgoing(self, data):
|
||||
pass
|
||||
|
|
@ -644,10 +708,8 @@ class TCPServerInterface(Interface):
|
|||
|
||||
|
||||
def __str__(self):
|
||||
if ":" in self.bind_ip:
|
||||
ip_str = f"[{self.bind_ip}]"
|
||||
else:
|
||||
ip_str = f"{self.bind_ip}"
|
||||
if ":" in self.bind_ip: ip_str = f"[{self.bind_ip}]"
|
||||
else: ip_str = f"{self.bind_ip}"
|
||||
|
||||
return "TCPServerInterface["+self.name+"/"+ip_str+":"+str(self.bind_port)+"]"
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -78,6 +78,7 @@ class UDPInterface(Interface):
|
|||
self.name = name
|
||||
self.online = False
|
||||
self.bitrate = UDPInterface.BITRATE_GUESS
|
||||
self.shared_medium = True
|
||||
|
||||
if device != None:
|
||||
if bindip == None:
|
||||
|
|
@ -112,9 +113,17 @@ class UDPInterface(Interface):
|
|||
self.forward_ip = forwardip
|
||||
self.forward_port = forwardport
|
||||
|
||||
def detach(self):
|
||||
self.detached = True
|
||||
self.online = False
|
||||
if self.server:
|
||||
try:
|
||||
self.server.server_close()
|
||||
self.server.shutdown()
|
||||
except Exception as e: RNS.log(f"Could not shut down UDP listener for {self}: {e}", RNS.LOG_ERROR)
|
||||
|
||||
def process_incoming(self, data):
|
||||
self.rxb += len(data)
|
||||
if not data: return
|
||||
self.owner.inbound(data, self)
|
||||
|
||||
def process_outgoing(self,data):
|
||||
|
|
|
|||
|
|
@ -109,30 +109,30 @@ class WDCL():
|
|||
self.switch_id = self.switch_identity.sig_pub_bytes[-4:]
|
||||
self.switch_pub_bytes = self.switch_identity.sig_pub_bytes
|
||||
|
||||
self.rxb = 0
|
||||
self.txb = 0
|
||||
self.owner = owner
|
||||
self.as_interface = as_interface
|
||||
self.device = device
|
||||
self.rxb = 0
|
||||
self.txb = 0
|
||||
self.owner = owner
|
||||
self.as_interface = as_interface
|
||||
self.device = device
|
||||
self.device.connection = self
|
||||
self.pyserial = serial
|
||||
self.serial = None
|
||||
self.port = port
|
||||
self.speed = 3000000
|
||||
self.databits = 8
|
||||
self.parity = parity
|
||||
self.stopbits = 1
|
||||
self.timeout = 100
|
||||
self.online = False
|
||||
self.frame_buffer = b""
|
||||
self.next_tx = 0
|
||||
self.should_run = True
|
||||
self.receiver = None
|
||||
self.wdcl_connected = False
|
||||
self.reconnecting = False
|
||||
self.frame_queue = deque()
|
||||
if not self.as_interface:
|
||||
self.id = RNS.Identity.full_hash(port.hwid.encode("utf-8"))
|
||||
self.pyserial = serial
|
||||
self.serial = None
|
||||
self.port = port
|
||||
self.speed = 3000000
|
||||
self.databits = 8
|
||||
self.parity = parity
|
||||
self.stopbits = 1
|
||||
self.timeout = 100
|
||||
self.online = False
|
||||
self.frame_buffer = b""
|
||||
self.next_tx = 0
|
||||
self.should_run = True
|
||||
self.receiver = None
|
||||
self.wdcl_connected = False
|
||||
self.reconnecting = False
|
||||
self.frame_queue = deque()
|
||||
|
||||
if not self.as_interface: self.id = RNS.Identity.full_hash(port.hwid.encode("utf-8"))
|
||||
|
||||
if self.as_interface:
|
||||
try:
|
||||
|
|
@ -168,19 +168,10 @@ class WDCL():
|
|||
self.owner.wlog(f"Opening serial port {self.port.device}...")
|
||||
target_port = self.port.device
|
||||
|
||||
self.serial = self.pyserial.Serial(
|
||||
port = target_port,
|
||||
baudrate = self.speed,
|
||||
bytesize = self.databits,
|
||||
parity = self.parity,
|
||||
stopbits = self.stopbits,
|
||||
xonxoff = False,
|
||||
rtscts = False,
|
||||
timeout = 0.250,
|
||||
inter_byte_timeout = None,
|
||||
write_timeout = None,
|
||||
dsrdtr = False)
|
||||
|
||||
self.serial = self.pyserial.Serial(port = target_port, baudrate = self.speed,
|
||||
bytesize = self.databits, parity = self.parity, stopbits = self.stopbits,
|
||||
xonxoff = False, rtscts = False, timeout = 0.250, inter_byte_timeout = None,
|
||||
write_timeout = None, dsrdtr = False)
|
||||
else:
|
||||
if self.port != None:
|
||||
# Get device parameters
|
||||
|
|
@ -198,19 +189,9 @@ class WDCL():
|
|||
from usbserial4a.cdcacmserial4a import CdcAcmSerial
|
||||
proxy = CdcAcmSerial
|
||||
|
||||
self.serial = proxy(
|
||||
self.port,
|
||||
baudrate = self.speed,
|
||||
bytesize = self.databits,
|
||||
parity = self.parity,
|
||||
stopbits = self.stopbits,
|
||||
xonxoff = False,
|
||||
rtscts = False,
|
||||
timeout = None,
|
||||
inter_byte_timeout = None,
|
||||
# write_timeout = wtimeout,
|
||||
dsrdtr = False,
|
||||
)
|
||||
self.serial = proxy(self.port, baudrate = self.speed, bytesize = self.databits,
|
||||
parity = self.parity, stopbits = self.stopbits, xonxoff = False,
|
||||
rtscts = False, timeout = None, inter_byte_timeout = None, dsrdtr = False)
|
||||
|
||||
if vid == 0x0403:
|
||||
# Hardware parameters for FTDI devices @ 115200 baud
|
||||
|
|
@ -268,7 +249,6 @@ class WDCL():
|
|||
self.online = True
|
||||
|
||||
def process_incoming(self, data):
|
||||
self.rxb += len(data)
|
||||
if self.device:
|
||||
while len(self.frame_queue): self.device.incoming_frame(self.frame_queue.pop())
|
||||
self.device.incoming_frame(data)
|
||||
|
|
@ -299,10 +279,9 @@ class WDCL():
|
|||
frame = frame.replace(bytes([HDLC.ESC, HDLC.ESC ^ HDLC.ESC_MASK]), bytes([HDLC.ESC]))
|
||||
if len(frame) > WDCL.HEADER_MINSIZE: self.process_incoming(frame)
|
||||
self.frame_buffer = self.frame_buffer[frame_end:]
|
||||
else:
|
||||
flags_remaining = False
|
||||
else:
|
||||
flags_remaining = False
|
||||
|
||||
else: flags_remaining = False
|
||||
else: flags_remaining = False
|
||||
|
||||
except Exception as e:
|
||||
self.online = False
|
||||
|
|
@ -359,6 +338,7 @@ class Evt():
|
|||
ET_MSG = 0x0000
|
||||
ET_SYSTEM_BOOT = 0x0001
|
||||
ET_CORE_INIT = 0x0002
|
||||
ET_BOARD_INIT = 0x0003
|
||||
ET_DRV_UART_INIT = 0x1000
|
||||
ET_DRV_USB_CDC_INIT = 0x1010
|
||||
ET_DRV_USB_CDC_HOST_AVAIL = 0x1011
|
||||
|
|
@ -426,77 +406,78 @@ class Evt():
|
|||
IF_TYPE_DMA = 0x10
|
||||
|
||||
event_descriptions = {
|
||||
ET_SYSTEM_BOOT: "System boot",
|
||||
ET_CORE_INIT: "Core initialization",
|
||||
ET_DRV_UART_INIT: "UART driver initialization",
|
||||
ET_DRV_USB_CDC_INIT: "USB CDC driver initialization",
|
||||
ET_DRV_USB_CDC_HOST_AVAIL: "USB CDC host became available",
|
||||
ET_DRV_USB_CDC_HOST_SUSPEND: "USB CDC host suspend",
|
||||
ET_DRV_USB_CDC_HOST_RESUME: "USB CDC host resume",
|
||||
ET_DRV_USB_CDC_CONNECTED: "USB CDC host connection",
|
||||
ET_DRV_USB_CDC_READ_ERR: "USB CDC read error",
|
||||
ET_DRV_USB_CDC_OVERFLOW: "USB CDC overflow occurred",
|
||||
ET_DRV_USB_CDC_DROPPED: "USB CDC dropped bytes",
|
||||
ET_DRV_USB_CDC_TX_TIMEOUT: "USB CDC TX flush timeout",
|
||||
ET_DRV_I2C_INIT: "I2C driver initialization",
|
||||
ET_DRV_NVS_INIT: "NVS driver initialization",
|
||||
ET_DRV_CRYPTO_INIT: "Cryptography driver initialization",
|
||||
ET_DRV_W80211_INIT: "W802.11 driver initialization",
|
||||
ET_DRV_W80211_CHANNEL: "W802.11 channel configuration",
|
||||
ET_DRV_W80211_POWER: "W802.11 TX power configuration",
|
||||
ET_DRV_DISPLAY_INIT: "Display driver initialization",
|
||||
ET_SYSTEM_BOOT: "System boot",
|
||||
ET_CORE_INIT: "Core initialization",
|
||||
ET_BOARD_INIT: "Board hardware initialization",
|
||||
ET_DRV_UART_INIT: "UART driver initialization",
|
||||
ET_DRV_USB_CDC_INIT: "USB CDC driver initialization",
|
||||
ET_DRV_USB_CDC_HOST_AVAIL: "USB CDC host became available",
|
||||
ET_DRV_USB_CDC_HOST_SUSPEND: "USB CDC host suspend",
|
||||
ET_DRV_USB_CDC_HOST_RESUME: "USB CDC host resume",
|
||||
ET_DRV_USB_CDC_CONNECTED: "USB CDC host connection",
|
||||
ET_DRV_USB_CDC_READ_ERR: "USB CDC read error",
|
||||
ET_DRV_USB_CDC_OVERFLOW: "USB CDC overflow occurred",
|
||||
ET_DRV_USB_CDC_DROPPED: "USB CDC dropped bytes",
|
||||
ET_DRV_USB_CDC_TX_TIMEOUT: "USB CDC TX flush timeout",
|
||||
ET_DRV_I2C_INIT: "I2C driver initialization",
|
||||
ET_DRV_NVS_INIT: "NVS driver initialization",
|
||||
ET_DRV_CRYPTO_INIT: "Cryptography driver initialization",
|
||||
ET_DRV_W80211_INIT: "W802.11 driver initialization",
|
||||
ET_DRV_W80211_CHANNEL: "W802.11 channel configuration",
|
||||
ET_DRV_W80211_POWER: "W802.11 TX power configuration",
|
||||
ET_DRV_DISPLAY_INIT: "Display driver initialization",
|
||||
ET_DRV_DISPLAY_BUS_AVAILABLE: "Display bus availability",
|
||||
ET_DRV_DISPLAY_IO_CONFIGURED: "Display I/O configuration",
|
||||
ET_DRV_DISPLAY_PANEL_CREATED: "Display panel allocation",
|
||||
ET_DRV_DISPLAY_PANEL_RESET: "Display panel reset",
|
||||
ET_DRV_DISPLAY_PANEL_INIT: "Display panel initialization",
|
||||
ET_DRV_DISPLAY_PANEL_ENABLE: "Display panel activation",
|
||||
ET_DRV_DISPLAY_PANEL_RESET: "Display panel reset",
|
||||
ET_DRV_DISPLAY_PANEL_INIT: "Display panel initialization",
|
||||
ET_DRV_DISPLAY_PANEL_ENABLE: "Display panel activation",
|
||||
ET_DRV_DISPLAY_REMOTE_ENABLE: "Remote display output activation",
|
||||
ET_KRN_LOGGER_INIT: "Logging service initialization",
|
||||
ET_KRN_LOGGER_OUTPUT: "Logging service output activation",
|
||||
ET_KRN_UI_INIT: "User interface service initialization",
|
||||
ET_PROTO_WDCL_INIT: "WDCL protocol initialization",
|
||||
ET_PROTO_WDCL_RUNNING: "WDCL protocol activation",
|
||||
ET_PROTO_WDCL_CONNECTION: "WDCL host connection",
|
||||
ET_PROTO_WDCL_HOST_ENDPOINT: "Weave host endpoint",
|
||||
ET_PROTO_WEAVE_INIT: "Weave protocol initialization",
|
||||
ET_PROTO_WEAVE_RUNNING: "Weave protocol activation",
|
||||
ET_PROTO_WEAVE_EP_ALIVE: "Weave endpoint alive",
|
||||
ET_PROTO_WEAVE_EP_TIMEOUT: "Weave endpoint disappeared",
|
||||
ET_SRVCTL_REMOTE_DISPLAY: "Remote display service control event",
|
||||
ET_INTERFACE_REGISTERED: "Interface registration",
|
||||
ET_SYSERR_MEM_EXHAUSTED: "System memory exhausted",
|
||||
ET_KRN_LOGGER_INIT: "Logging service initialization",
|
||||
ET_KRN_LOGGER_OUTPUT: "Logging service output activation",
|
||||
ET_KRN_UI_INIT: "User interface service initialization",
|
||||
ET_PROTO_WDCL_INIT: "WDCL protocol initialization",
|
||||
ET_PROTO_WDCL_RUNNING: "WDCL protocol activation",
|
||||
ET_PROTO_WDCL_CONNECTION: "WDCL host connection",
|
||||
ET_PROTO_WDCL_HOST_ENDPOINT: "Weave host endpoint",
|
||||
ET_PROTO_WEAVE_INIT: "Weave protocol initialization",
|
||||
ET_PROTO_WEAVE_RUNNING: "Weave protocol activation",
|
||||
ET_PROTO_WEAVE_EP_ALIVE: "Weave endpoint alive",
|
||||
ET_PROTO_WEAVE_EP_TIMEOUT: "Weave endpoint disappeared",
|
||||
ET_SRVCTL_REMOTE_DISPLAY: "Remote display service control event",
|
||||
ET_INTERFACE_REGISTERED: "Interface registration",
|
||||
ET_SYSERR_MEM_EXHAUSTED: "System memory exhausted",
|
||||
}
|
||||
|
||||
interface_types = {
|
||||
IF_TYPE_USB: "usb",
|
||||
IF_TYPE_UART: "uart",
|
||||
IF_TYPE_W80211: "mw",
|
||||
IF_TYPE_BLE: "ble",
|
||||
IF_TYPE_LORA: "lora",
|
||||
IF_TYPE_USB: "usb",
|
||||
IF_TYPE_UART: "uart",
|
||||
IF_TYPE_W80211: "mw",
|
||||
IF_TYPE_BLE: "ble",
|
||||
IF_TYPE_LORA: "lora",
|
||||
IF_TYPE_ETHERNET: "eth",
|
||||
IF_TYPE_WIFI: "wifi",
|
||||
IF_TYPE_TCP: "tcp",
|
||||
IF_TYPE_UDP: "udp",
|
||||
IF_TYPE_IR: "ir",
|
||||
IF_TYPE_AFSK: "afsk",
|
||||
IF_TYPE_GPIO: "gpio",
|
||||
IF_TYPE_SPI: "spi",
|
||||
IF_TYPE_I2C: "i2c",
|
||||
IF_TYPE_CAN: "can",
|
||||
IF_TYPE_DMA: "dma",
|
||||
IF_TYPE_WIFI: "wifi",
|
||||
IF_TYPE_TCP: "tcp",
|
||||
IF_TYPE_UDP: "udp",
|
||||
IF_TYPE_IR: "ir",
|
||||
IF_TYPE_AFSK: "afsk",
|
||||
IF_TYPE_GPIO: "gpio",
|
||||
IF_TYPE_SPI: "spi",
|
||||
IF_TYPE_I2C: "i2c",
|
||||
IF_TYPE_CAN: "can",
|
||||
IF_TYPE_DMA: "dma",
|
||||
}
|
||||
|
||||
channel_descriptions = {
|
||||
1: "Channel 1 (2412 MHz)",
|
||||
2: "Channel 2 (2417 MHz)",
|
||||
3: "Channel 3 (2422 MHz)",
|
||||
4: "Channel 4 (2427 MHz)",
|
||||
5: "Channel 5 (2432 MHz)",
|
||||
6: "Channel 6 (2437 MHz)",
|
||||
7: "Channel 7 (2442 MHz)",
|
||||
8: "Channel 8 (2447 MHz)",
|
||||
9: "Channel 9 (2452 MHz)",
|
||||
1: "Channel 1 (2412 MHz)",
|
||||
2: "Channel 2 (2417 MHz)",
|
||||
3: "Channel 3 (2422 MHz)",
|
||||
4: "Channel 4 (2427 MHz)",
|
||||
5: "Channel 5 (2432 MHz)",
|
||||
6: "Channel 6 (2437 MHz)",
|
||||
7: "Channel 7 (2442 MHz)",
|
||||
8: "Channel 8 (2447 MHz)",
|
||||
9: "Channel 9 (2452 MHz)",
|
||||
10: "Channel 10 (2457 MHz)",
|
||||
11: "Channel 11 (2462 MHz)",
|
||||
12: "Channel 12 (2467 MHz)",
|
||||
|
|
@ -529,23 +510,23 @@ class Evt():
|
|||
}
|
||||
|
||||
task_descriptions = {
|
||||
"taskLVGL": "Driver: UI Renderer",
|
||||
"ui_service": "Service: User Interface",
|
||||
"TinyUSB": "Driver: USB",
|
||||
"drv_w80211": "Driver: W802.11",
|
||||
"system_stats": "System: Stats",
|
||||
"core": "System: Core",
|
||||
"protocol_wdcl": "Protocol: WDCL",
|
||||
"taskLVGL": "Driver: UI Renderer",
|
||||
"ui_service": "Service: User Interface",
|
||||
"TinyUSB": "Driver: USB",
|
||||
"drv_w80211": "Driver: W802.11",
|
||||
"system_stats": "System: Stats",
|
||||
"core": "System: Core",
|
||||
"protocol_wdcl": "Protocol: WDCL",
|
||||
"protocol_weave": "Protocol: Weave",
|
||||
"tiT": "Protocol: TCP/IP",
|
||||
"ipc0": "System: CPU 0 IPC",
|
||||
"ipc1": "System: CPU 1 IPC",
|
||||
"esp_timer": "Driver: Timers",
|
||||
"Tmr Svc": "Service: Timers",
|
||||
"kernel_logger": "Service: Logging",
|
||||
"tiT": "Protocol: TCP/IP",
|
||||
"ipc0": "System: CPU 0 IPC",
|
||||
"ipc1": "System: CPU 1 IPC",
|
||||
"esp_timer": "Driver: Timers",
|
||||
"Tmr Svc": "Service: Timers",
|
||||
"kernel_logger": "Service: Logging",
|
||||
"remote_display": "Service: Remote Display",
|
||||
"wifi": "System: WiFi Hardware",
|
||||
"sys_evt": "System: Kernel Events",
|
||||
"wifi": "System: WiFi Hardware",
|
||||
"sys_evt": "System: Kernel Events",
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
|
|
@ -775,8 +756,8 @@ class WeaveDevice():
|
|||
self.cpu_load = frame.data[0]
|
||||
self.capture_stats_cpu()
|
||||
elif frame.event == Evt.ET_STAT_MEMORY:
|
||||
self.memory_free = int.from_bytes(frame.data[:4])
|
||||
self.memory_total = int.from_bytes(frame.data[4:])
|
||||
self.memory_free = int.from_bytes(frame.data[:4], "big")
|
||||
self.memory_total = int.from_bytes(frame.data[4:], "big")
|
||||
self.memory_used = self.memory_total-self.memory_free
|
||||
self.memory_used_pct = round((self.memory_used/self.memory_total)*100, 2)
|
||||
self.capture_stats_memory()
|
||||
|
|
@ -942,8 +923,26 @@ class WeaveInterface(Interface):
|
|||
spawned_interface = WeaveInterfacePeer(self, endpoint_addr)
|
||||
spawned_interface.OUT = self.OUT
|
||||
spawned_interface.IN = self.IN
|
||||
|
||||
spawned_interface.ingress_control = self.ingress_control
|
||||
spawned_interface.ic_max_held_announces = self.ic_max_held_announces
|
||||
spawned_interface.ic_burst_hold = self.ic_burst_hold
|
||||
spawned_interface.ic_burst_freq = self.ic_burst_freq
|
||||
spawned_interface.ic_burst_freq_new = self.ic_burst_freq_new
|
||||
spawned_interface.ic_new_time = self.ic_new_time
|
||||
spawned_interface.ic_burst_penalty = self.ic_burst_penalty
|
||||
spawned_interface.ic_held_release_interval = self.ic_held_release_interval
|
||||
|
||||
spawned_interface.announces_to_internal = self.announces_to_internal
|
||||
spawned_interface.announces_from_internal = self.announces_from_internal
|
||||
spawned_interface.announce_cap = self.announce_cap
|
||||
|
||||
spawned_interface._destination_cost = self._destination_cost
|
||||
spawned_interface._pr_cost = self._pr_cost
|
||||
|
||||
spawned_interface.parent_interface = self
|
||||
spawned_interface.bitrate = self.bitrate
|
||||
spawned_interface.recursive_prs = self.recursive_prs
|
||||
|
||||
spawned_interface.ifac_size = self.ifac_size
|
||||
spawned_interface.ifac_netname = self.ifac_netname
|
||||
|
|
@ -968,10 +967,19 @@ class WeaveInterface(Interface):
|
|||
spawned_interface.announce_rate_target = self.announce_rate_target
|
||||
spawned_interface.announce_rate_grace = self.announce_rate_grace
|
||||
spawned_interface.announce_rate_penalty = self.announce_rate_penalty
|
||||
|
||||
spawned_interface.egress_control = self.egress_control
|
||||
spawned_interface.ec_pr_freq = self.ec_pr_freq
|
||||
spawned_interface.ic_pr_burst_freq_new = self.ic_pr_burst_freq_new
|
||||
spawned_interface.ic_pr_burst_freq = self.ic_pr_burst_freq
|
||||
|
||||
spawned_interface.mode = self.mode
|
||||
spawned_interface.gravity = self.gravity
|
||||
spawned_interface.hop_limiter = self.hop_limiter
|
||||
spawned_interface.HW_MTU = self.HW_MTU
|
||||
spawned_interface._online = True
|
||||
RNS.Transport.interfaces.append(spawned_interface)
|
||||
|
||||
RNS.Transport.add_interface(spawned_interface)
|
||||
if endpoint_addr in self.spawned_interfaces:
|
||||
self.spawned_interfaces[endpoint_addr].detach()
|
||||
self.spawned_interfaces[endpoint_addr].teardown()
|
||||
|
|
@ -1052,8 +1060,6 @@ class WeaveInterfacePeer(Interface):
|
|||
self.owner.refresh_peer(self.endpoint_addr)
|
||||
self.owner.mif_deque.append(data_hash)
|
||||
self.owner.mif_deque_times.append([data_hash, time.time()])
|
||||
self.rxb += len(data)
|
||||
self.owner.rxb += len(data)
|
||||
self.owner.owner.inbound(data, self)
|
||||
|
||||
def process_outgoing(self, data):
|
||||
|
|
@ -1087,5 +1093,4 @@ class WeaveInterfacePeer(Interface):
|
|||
except Exception as e:
|
||||
RNS.log(f"Could not remove {self} from parent interface on detach. The contained exception was: {e}", RNS.LOG_ERROR)
|
||||
|
||||
if self in RNS.Transport.interfaces:
|
||||
RNS.Transport.interfaces.remove(self)
|
||||
RNS.Transport.remove_interface(self)
|
||||
|
|
|
|||
|
|
@ -36,5 +36,7 @@ import RNS.Interfaces.util.netinfo as netinfo
|
|||
|
||||
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
|
||||
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
|
||||
modules = py_modules+pyc_modules
|
||||
__all__ = list(set([os.path.basename(f).replace(".pyc", "").replace(".py", "") for f in modules if not (f.endswith("__init__.py") or f.endswith("__init__.pyc"))]))
|
||||
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
|
||||
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
|
||||
modules = py_modules+pyc_modules+so_modules+pyd_modules
|
||||
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))
|
||||
115
RNS/Interfaces/util/HDLC.py
Normal file
115
RNS/Interfaces/util/HDLC.py
Normal file
|
|
@ -0,0 +1,115 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
class HDLC():
|
||||
FLAG = 0x7E
|
||||
ESC = 0x7D
|
||||
ESC_MASK = 0x20
|
||||
|
||||
FLAG_B = bytes([FLAG])
|
||||
ESC_B = bytes([ESC])
|
||||
ESCPD_FLAG = bytes([ESC, FLAG ^ ESC_MASK])
|
||||
ESCPD_ESC = bytes([ESC, ESC ^ ESC_MASK])
|
||||
|
||||
@staticmethod
|
||||
def escape(data):
|
||||
data = data.replace(HDLC.ESC_B, HDLC.ESCPD_ESC)
|
||||
data = data.replace(HDLC.FLAG_B, HDLC.ESCPD_FLAG)
|
||||
return data
|
||||
|
||||
@staticmethod
|
||||
def unescape(frame):
|
||||
frame = frame.replace(HDLC.ESCPD_FLAG, HDLC.FLAG_B)
|
||||
frame = frame.replace(HDLC.ESCPD_ESC, HDLC.ESC_B)
|
||||
return frame
|
||||
|
||||
@staticmethod
|
||||
def frame(data):
|
||||
esc = HDLC.escape(data)
|
||||
frame = bytearray(len(esc)+2)
|
||||
frame[0] = HDLC.FLAG
|
||||
frame[1:-1] = esc
|
||||
frame[-1] = HDLC.FLAG
|
||||
return frame
|
||||
|
||||
class ReceiveBuffer():
|
||||
def __init__(self, mtu, min_frame_len, max_frame_len=None, on_frame=None, on_invalid=None):
|
||||
self._mtu = mtu
|
||||
self._min_frame_len = min_frame_len
|
||||
self._max_frame_len = max_frame_len
|
||||
self._on_frame = on_frame
|
||||
self._on_invalid = on_invalid
|
||||
self._buf = bytearray()
|
||||
self._off = 0
|
||||
|
||||
def mtu(self):
|
||||
return self._mtu() if callable(self._mtu) else self._mtu
|
||||
|
||||
def max(self):
|
||||
m = self._max_frame_len
|
||||
return m() if callable(m) else m
|
||||
|
||||
def feed(self, data):
|
||||
if not data: return
|
||||
self._buf.extend(data)
|
||||
buf = self._buf
|
||||
|
||||
while True:
|
||||
frame_start = buf.find(HDLC.FLAG, self._off)
|
||||
if frame_start == -1:
|
||||
self.reset()
|
||||
return
|
||||
|
||||
frame_end = buf.find(HDLC.FLAG, frame_start+1)
|
||||
if frame_end == -1:
|
||||
if len(buf)-self._off > self.mtu()*2: self.reset()
|
||||
return
|
||||
|
||||
frame = HDLC.unescape(bytes(buf[frame_start+1:frame_end]))
|
||||
frame_len = len(frame)
|
||||
if frame_len > 0:
|
||||
max_len = self.max()
|
||||
if frame_len > self._min_frame_len and (max_len is None or frame_len <= max_len):
|
||||
if self._on_frame is not None: self._on_frame(frame)
|
||||
elif self._on_invalid is not None: self._on_invalid(frame_len)
|
||||
|
||||
self._off = frame_end
|
||||
if self._off > 0 and self._off >= len(buf) // 2:
|
||||
del buf[:self._off]
|
||||
self._off = 0
|
||||
|
||||
# Drops any partial frames from the buffer and
|
||||
# resets assembly offset
|
||||
def reset(self):
|
||||
self._buf.clear()
|
||||
self._off = 0
|
||||
|
||||
# Buffered bytes currently unconsumed
|
||||
def __len__(self): return len(self._buf)-self._off
|
||||
171
RNS/Interfaces/util/TransmitBuffer.py
Normal file
171
RNS/Interfaces/util/TransmitBuffer.py
Normal file
|
|
@ -0,0 +1,171 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
from collections import deque
|
||||
import threading
|
||||
|
||||
# Zero-copy coalesced transmit buffer for arbitrary
|
||||
# frame sizes. It's even almost 100% lock-free.
|
||||
class TransmitBuffer:
|
||||
COALESCE_TARGET = 65536
|
||||
|
||||
def __init__(self):
|
||||
self._tail_lock = threading.Lock()
|
||||
|
||||
# Producer-owned state
|
||||
self._cur = None
|
||||
self._cur_frames = 0
|
||||
|
||||
# Consumer-owned state
|
||||
self._head_offset = 0
|
||||
|
||||
# Shared chunk queue. Producer appends right, consumer
|
||||
# pops from left. Appended chunks are immutable.
|
||||
self._chunks = deque()
|
||||
|
||||
# Accounting counters
|
||||
|
||||
# Producer-owned
|
||||
self._tx_total = 0 # Total bytes appended
|
||||
self._tx_visible = 0 # Total bytes pushed to the chunk queue
|
||||
self._tx_frames = 0 # Total frames appended
|
||||
|
||||
# Consumer-owned
|
||||
self._tx_sent = 0 # Total bytes sent to the kernel
|
||||
self._tx_frames_sent = 0 # Total frames fully sent
|
||||
|
||||
# Queue a finalized and complete on-wire frame.
|
||||
# Executed by producer side only.
|
||||
def append(self, frame, limit=None):
|
||||
with self._tail_lock:
|
||||
if limit is not None and (self._tx_total - self._tx_sent) + len(frame) > limit: return False
|
||||
|
||||
# Allocate separate chunks to frames
|
||||
# larger than the coalescing target.
|
||||
if len(frame) >= self.COALESCE_TARGET:
|
||||
self._flush()
|
||||
self._chunks.append((frame, 1))
|
||||
self._tx_visible += len(frame)
|
||||
self._tx_total += len(frame)
|
||||
self._tx_frames += 1
|
||||
|
||||
else:
|
||||
if self._cur is not None and len(self._cur) + len(frame) > self.COALESCE_TARGET: self._flush()
|
||||
|
||||
if self._cur is None:
|
||||
self._cur = bytearray()
|
||||
self._cur_frames = 0
|
||||
|
||||
self._cur.extend(frame)
|
||||
self._cur_frames += 1
|
||||
self._tx_total += len(frame)
|
||||
self._tx_frames += 1
|
||||
|
||||
if not self._chunks:
|
||||
# Queue is idle. Make the frame visible immediately so
|
||||
# sparse traffic does not incur coalescing latency.
|
||||
self._flush()
|
||||
|
||||
return True
|
||||
|
||||
def _flush(self):
|
||||
if self._cur is not None and len(self._cur) > 0:
|
||||
self._chunks.append((self._cur, self._cur_frames))
|
||||
self._tx_visible += len(self._cur)
|
||||
self._cur = None
|
||||
self._cur_frames = 0
|
||||
|
||||
# Make producer-side coalescing tail visible to the consumer.
|
||||
# Called by producer when it has finished producing a burst,
|
||||
# and wants to make the remaining bytes transmittable. Safe to
|
||||
# call after every append.
|
||||
def flush(self):
|
||||
with self._tail_lock: self._flush()
|
||||
|
||||
# Drain buffered frames to a socket. Called by consumer. If the
|
||||
# chunk queue is drained completely, any potential producer-side
|
||||
# coalescing tail is released under the tail lock, and drained in
|
||||
# the same cycle. If you possess wizardry for doing this without
|
||||
# the lock, I am all ears.
|
||||
def drain_to(self, socket):
|
||||
# TODO: At *some* point (no, not now), replace with a
|
||||
# mechanism for send_msg to immediately drain all
|
||||
# chunks directly to the kernel.
|
||||
total = 0
|
||||
whole = True
|
||||
while whole:
|
||||
if not self._chunks:
|
||||
with self._tail_lock: self._flush()
|
||||
if not self._chunks: break
|
||||
|
||||
chunk, _ = self._chunks[0]
|
||||
remaining = len(chunk) - self._head_offset
|
||||
if remaining <= 0:
|
||||
self._pop_head()
|
||||
continue
|
||||
|
||||
try: written = socket.send(memoryview(chunk)[self._head_offset:])
|
||||
except BlockingIOError: break
|
||||
except InterruptedError: continue
|
||||
if written <= 0: break
|
||||
|
||||
total += written
|
||||
self._head_offset += written
|
||||
self._tx_sent += written
|
||||
|
||||
if self._head_offset >= len(chunk): self._pop_head()
|
||||
else: whole = False
|
||||
|
||||
return total
|
||||
|
||||
def _pop_head(self):
|
||||
chunk, frames = self._chunks.popleft()
|
||||
self._tx_frames_sent += frames
|
||||
self._head_offset = 0
|
||||
|
||||
# Total number of bytes currently buffered, including the
|
||||
# producer's in-progress coalescing chunk.
|
||||
def __len__(self): return self._tx_total-self._tx_sent
|
||||
|
||||
# The number of bytes actually ready to send, and will be
|
||||
# visible to the consumer, that is, what can be drained.
|
||||
@property
|
||||
def sendable(self): return self._tx_visible-self._tx_sent
|
||||
|
||||
# The number of complete frames currently buffere, including
|
||||
# ones contained in a potential in-progress coalescing chunk.
|
||||
@property
|
||||
def frames_buffered(self): return self._tx_frames-self._tx_frames_sent
|
||||
|
||||
# Number of chunks currently queued and visible to the consumer
|
||||
@property
|
||||
def chunks_buffered(self): return len(self._chunks)
|
||||
|
||||
def __repr__(self): return f"<TransmitBuffer buffered={len(self)} sendable={self.sendable} frames={self.frames_buffered} chunks={self.chunks_buffered}>"
|
||||
|
|
@ -3,5 +3,7 @@ import glob
|
|||
|
||||
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
|
||||
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
|
||||
modules = py_modules+pyc_modules
|
||||
__all__ = list(set([os.path.basename(f).replace(".pyc", "").replace(".py", "") for f in modules if not (f.endswith("__init__.py") or f.endswith("__init__.pyc"))]))
|
||||
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
|
||||
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
|
||||
modules = py_modules+pyc_modules+so_modules+pyd_modules
|
||||
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))
|
||||
|
|
|
|||
646
RNS/Link.py
646
RNS/Link.py
File diff suppressed because it is too large
Load diff
169
RNS/Packet.py
169
RNS/Packet.py
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -103,7 +103,7 @@ class Packet:
|
|||
# With an MTU of 500, the maximum of data we can
|
||||
# send in a single encrypted packet is given by
|
||||
# the below calculation; 383 bytes.
|
||||
ENCRYPTED_MDU = math.floor((RNS.Reticulum.MDU-RNS.Identity.TOKEN_OVERHEAD-RNS.Identity.KEYSIZE//16)/RNS.Identity.AES128_BLOCKSIZE)*RNS.Identity.AES128_BLOCKSIZE - 1
|
||||
ENCRYPTED_MDU = math.floor((RNS.Reticulum.MDU-RNS.Identity.TOKEN_OVERHEAD-RNS.Identity.KEYSIZE//16)/RNS.Identity.AES256_BLOCKSIZE)*RNS.Identity.AES256_BLOCKSIZE - 1
|
||||
"""
|
||||
The maximum size of the payload data in a single encrypted packet
|
||||
"""
|
||||
|
|
@ -116,11 +116,13 @@ class Packet:
|
|||
|
||||
__slots__ = "hops", "header", "header_type", "packet_type", "transport_type", "context", "context_flag", "destination"
|
||||
__slots__ += "transport_id", "data", "flags", "raw", "packed", "sent", "create_receipt", "receipt", "fromPacked", "MTU"
|
||||
__slots__ += "sent_at", "packet_hash", "ratchet_id", "attached_interface", "receiving_interface", "rssi", "snr", "q"
|
||||
__slots__ += "ciphertext", "plaintext", "destination_hash", "destination_type", "link", "map_hash"
|
||||
__slots__ += "sent_at", "packet_hash", "ratchet_id", "receiving_interface", "attached_interface", "blocked_interfaces"
|
||||
__slots__ += "rssi", "snr", "q", "ciphertext", "plaintext", "destination_hash", "destination_type", "link", "map_hash"
|
||||
__slots__ += "is_outbound_pr", "traffic_class", "announce_signature_validated"
|
||||
|
||||
def __init__(self, destination, data, packet_type = DATA, context = NONE, transport_type = RNS.Transport.BROADCAST,
|
||||
header_type = HEADER_1, transport_id = None, attached_interface = None, create_receipt = True, context_flag=FLAG_UNSET):
|
||||
def __init__(self, destination, data, packet_type=DATA, context=NONE, transport_type=RNS.Transport.BROADCAST,
|
||||
header_type=HEADER_1, transport_id=None, attached_interface=None, blocked_interfaces=None,
|
||||
create_receipt=True, context_flag=FLAG_UNSET):
|
||||
|
||||
if destination != None:
|
||||
if transport_type == None:
|
||||
|
|
@ -159,8 +161,12 @@ class Packet:
|
|||
self.packet_hash = None
|
||||
self.ratchet_id = None
|
||||
|
||||
self.traffic_class = None
|
||||
self.announce_signature_validated = None
|
||||
self.attached_interface = attached_interface
|
||||
self.blocked_interfaces = blocked_interfaces
|
||||
self.receiving_interface = None
|
||||
self.is_outbound_pr = False
|
||||
self.rssi = None
|
||||
self.snr = None
|
||||
self.q = None
|
||||
|
|
@ -231,8 +237,13 @@ class Packet:
|
|||
self.header += bytes([self.context])
|
||||
self.raw = self.header + self.ciphertext
|
||||
|
||||
if self.packet_type == Packet.ANNOUNCE and self.header_type == Packet.HEADER_1:
|
||||
transport_id_length = RNS.Reticulum.TRUNCATED_HASHLENGTH//8
|
||||
if len(self.raw)+transport_id_length > self.MTU:
|
||||
raise IOError(f"Transport-inserted announce packet size of {len(self.raw)+transport_id_length} exceeds MTU of {self.MTU} bytes")
|
||||
|
||||
if len(self.raw) > self.MTU:
|
||||
raise IOError("Packet size of "+str(len(self.raw))+" exceeds MTU of "+str(self.MTU)+" bytes")
|
||||
raise IOError(f"Packet size of {len(self.raw)} exceeds MTU of {self.MTU} bytes")
|
||||
|
||||
self.packed = True
|
||||
self.update_hash()
|
||||
|
|
@ -243,6 +254,9 @@ class Packet:
|
|||
self.flags = self.raw[0]
|
||||
self.hops = self.raw[1]
|
||||
|
||||
if self.hops >= RNS.Transport.PATHFINDER_M:
|
||||
raise ValueError(f"Invalid hop count {self.hops}")
|
||||
|
||||
self.header_type = (self.flags & 0b01000000) >> 6
|
||||
self.context_flag = (self.flags & 0b00100000) >> 5
|
||||
self.transport_type = (self.flags & 0b00010000) >> 4
|
||||
|
|
@ -256,18 +270,23 @@ class Packet:
|
|||
self.destination_hash = self.raw[DST_LEN+2:2*DST_LEN+2]
|
||||
self.context = ord(self.raw[2*DST_LEN+2:2*DST_LEN+3])
|
||||
self.data = self.raw[2*DST_LEN+3:]
|
||||
if len(self.transport_id) != DST_LEN: raise ValueError("Malformed Transport ID field")
|
||||
if len(self.destination_hash) != DST_LEN: raise ValueError("Malformed destination hash field")
|
||||
else:
|
||||
self.transport_id = None
|
||||
self.destination_hash = self.raw[2:DST_LEN+2]
|
||||
self.context = ord(self.raw[DST_LEN+2:DST_LEN+3])
|
||||
self.data = self.raw[DST_LEN+3:]
|
||||
if len(self.destination_hash) != DST_LEN: raise ValueError("Malformed destination hash field")
|
||||
|
||||
if len(self.data) == 0: raise ValueError("Zero-length data field")
|
||||
|
||||
self.packed = False
|
||||
self.update_hash()
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
RNS.log("Received malformed packet, dropping it. The contained exception was: "+str(e), RNS.LOG_EXTREME)
|
||||
RNS.log(f"Dropping malformed packet. The contained exception was: {e}", RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
|
||||
return False
|
||||
|
||||
def send(self):
|
||||
|
|
@ -277,9 +296,11 @@ class Packet:
|
|||
:returns: A :ref:`RNS.PacketReceipt<api-packetreceipt>` instance if *create_receipt* was set to *True* when the packet was instantiated, if not returns *None*. If the packet could not be sent *False* is returned.
|
||||
"""
|
||||
if not self.sent:
|
||||
if self.hops >= RNS.Transport.PATHFINDER_M: return False
|
||||
if not self.packed: self.pack()
|
||||
if self.destination.type == RNS.Destination.LINK:
|
||||
if self.destination.status == RNS.Link.CLOSED:
|
||||
RNS.log("Attempt to transmit over a closed link, dropping packet", RNS.LOG_DEBUG)
|
||||
RNS.log("Attempt to transmit over a closed link, dropping packet", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
self.sent = False
|
||||
self.receipt = None
|
||||
return False
|
||||
|
|
@ -287,13 +308,14 @@ class Packet:
|
|||
else:
|
||||
self.destination.last_outbound = time.time()
|
||||
self.destination.tx += 1
|
||||
self.destination.txbytes += len(self.data)
|
||||
|
||||
if not self.packed: self.pack()
|
||||
self.destination.txbytes += len(self.ciphertext)
|
||||
RNS.Transport.link_txb += len(self.raw)
|
||||
|
||||
if RNS.Transport.outbound(self): return self.receipt
|
||||
else:
|
||||
RNS.log("No interfaces could process the outbound packet", RNS.LOG_ERROR)
|
||||
if not self.attached_interface:
|
||||
RNS.log(f"No interfaces could process the outbound {self.hops} hop, type {self.packet_type} packet for {self.destination}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
|
||||
self.sent = False
|
||||
self.receipt = None
|
||||
return False
|
||||
|
|
@ -312,51 +334,44 @@ class Packet:
|
|||
# encrypted destinations
|
||||
self.pack()
|
||||
|
||||
if RNS.Transport.outbound(self):
|
||||
return self.receipt
|
||||
if RNS.Transport.outbound(self): return self.receipt
|
||||
else:
|
||||
RNS.log("No interfaces could process the outbound packet", RNS.LOG_ERROR)
|
||||
RNS.log("Re-send failed. No interfaces could process the outbound packet", RNS.LOG_WARNING)
|
||||
self.sent = False
|
||||
self.receipt = None
|
||||
return False
|
||||
else:
|
||||
raise IOError("Packet was not sent yet")
|
||||
|
||||
else: raise IOError("Packet was not sent yet")
|
||||
|
||||
def prove(self, destination=None):
|
||||
if self.fromPacked and hasattr(self, "destination") and self.destination:
|
||||
if self.destination.identity and self.destination.identity.prv:
|
||||
self.destination.identity.prove(self, destination)
|
||||
elif self.fromPacked and hasattr(self, "link") and self.link:
|
||||
self.link.prove_packet(self)
|
||||
else:
|
||||
RNS.log("Could not prove packet associated with neither a destination nor a link", RNS.LOG_ERROR)
|
||||
elif self.fromPacked and hasattr(self, "link") and self.link: self.link.prove_packet(self)
|
||||
else: RNS.log("Could not prove packet associated with neither a destination nor a link", RNS.LOG_ERROR)
|
||||
|
||||
# Generates a special destination that allows Reticulum
|
||||
# to direct the proof back to the proved packet's sender
|
||||
def generate_proof_destination(self):
|
||||
return ProofDestination(self)
|
||||
def generate_proof_destination(self): return ProofDestination(self)
|
||||
|
||||
def validate_proof_packet(self, proof_packet):
|
||||
return self.receipt.validate_proof_packet(proof_packet)
|
||||
def validate_proof_packet(self, proof_packet): return self.receipt.validate_proof_packet(proof_packet)
|
||||
|
||||
def validate_proof(self, proof):
|
||||
return self.receipt.validate_proof(proof)
|
||||
def validate_proof(self, proof): return self.receipt.validate_proof(proof)
|
||||
|
||||
def update_hash(self):
|
||||
self.packet_hash = self.get_hash()
|
||||
def update_hash(self): self.packet_hash = self.get_hash()
|
||||
|
||||
def get_hash(self):
|
||||
return RNS.Identity.full_hash(self.get_hashable_part())
|
||||
def get_hash(self): return RNS.Identity.full_hash(self.get_hashable_part())
|
||||
|
||||
def getTruncatedHash(self):
|
||||
return RNS.Identity.truncated_hash(self.get_hashable_part())
|
||||
def getTruncatedHash(self): return RNS.Identity.truncated_hash(self.get_hashable_part())
|
||||
|
||||
@property
|
||||
def truncated_packet_hash(self):
|
||||
return self.packet_hash[:RNS.Reticulum.TRUNCATED_HASHLENGTH//8]
|
||||
|
||||
def get_hashable_part(self):
|
||||
hashable_part = bytes([self.raw[0] & 0b00001111])
|
||||
if self.header_type == Packet.HEADER_2:
|
||||
hashable_part += self.raw[(RNS.Identity.TRUNCATED_HASHLENGTH//8)+2:]
|
||||
else:
|
||||
hashable_part += self.raw[2:]
|
||||
if self.header_type == Packet.HEADER_2: hashable_part += self.raw[(RNS.Identity.TRUNCATED_HASHLENGTH//8)+2:]
|
||||
else: hashable_part += self.raw[2:]
|
||||
|
||||
return hashable_part
|
||||
|
||||
|
|
@ -364,36 +379,29 @@ class Packet:
|
|||
"""
|
||||
:returns: The physical layer *Received Signal Strength Indication* if available, otherwise ``None``.
|
||||
"""
|
||||
if self.rssi != None:
|
||||
return self.rssi
|
||||
else:
|
||||
return reticulum.get_packet_rssi(self.packet_hash)
|
||||
if self.rssi != None: return self.rssi
|
||||
else: return RNS.Reticulum.get_instance().get_packet_rssi(self.packet_hash)
|
||||
|
||||
def get_snr(self):
|
||||
"""
|
||||
:returns: The physical layer *Signal-to-Noise Ratio* if available, otherwise ``None``.
|
||||
"""
|
||||
if self.snr != None:
|
||||
return self.snr
|
||||
else:
|
||||
return reticulum.get_packet_snr(self.packet_hash)
|
||||
if self.snr != None: return self.snr
|
||||
else: return RNS.Reticulum.get_instance().get_packet_snr(self.packet_hash)
|
||||
|
||||
def get_q(self):
|
||||
"""
|
||||
:returns: The physical layer *Link Quality* if available, otherwise ``None``.
|
||||
"""
|
||||
if self.q != None:
|
||||
return self.q
|
||||
else:
|
||||
return reticulum.get_packet_q(self.packet_hash)
|
||||
if self.q != None: return self.q
|
||||
else: return RNS.Reticulum.get_instance().get_packet_q(self.packet_hash)
|
||||
|
||||
class ProofDestination:
|
||||
def __init__(self, packet):
|
||||
self.hash = packet.get_hash()[:RNS.Reticulum.TRUNCATED_HASHLENGTH//8];
|
||||
self.hash = packet.truncated_packet_hash;
|
||||
self.type = RNS.Destination.SINGLE
|
||||
|
||||
def encrypt(self, plaintext):
|
||||
return plaintext
|
||||
def encrypt(self, plaintext): return plaintext
|
||||
|
||||
|
||||
class PacketReceipt:
|
||||
|
|
@ -415,8 +423,8 @@ class PacketReceipt:
|
|||
|
||||
# Creates a new packet receipt from a sent packet
|
||||
def __init__(self, packet):
|
||||
self.hash = packet.get_hash()
|
||||
self.truncated_hash = packet.getTruncatedHash()
|
||||
self.hash = packet.packet_hash
|
||||
self.truncated_hash = packet.truncated_packet_hash
|
||||
self.sent = True
|
||||
self.sent_at = time.time()
|
||||
self.proved = False
|
||||
|
|
@ -462,18 +470,17 @@ class PacketReceipt:
|
|||
link.last_proof = self.concluded_at
|
||||
|
||||
if self.callbacks.delivery != None:
|
||||
try:
|
||||
self.callbacks.delivery(self)
|
||||
try: self.callbacks.delivery(self)
|
||||
except Exception as e:
|
||||
RNS.log("An error occurred while evaluating external delivery callback for "+str(link), RNS.LOG_ERROR)
|
||||
RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
RNS.trace_exception(e)
|
||||
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
else:
|
||||
return False
|
||||
|
||||
else: return False
|
||||
else: return False
|
||||
|
||||
elif len(proof) == PacketReceipt.IMPL_LENGTH:
|
||||
pass
|
||||
# TODO: Why is this disabled?
|
||||
|
|
@ -490,8 +497,7 @@ class PacketReceipt:
|
|||
# else:
|
||||
# RNS.log("invalid")
|
||||
# return False
|
||||
else:
|
||||
return False
|
||||
else: return False
|
||||
|
||||
# Validate a raw proof
|
||||
def validate_proof(self, proof, proof_packet=None):
|
||||
|
|
@ -508,24 +514,20 @@ class PacketReceipt:
|
|||
self.proof_packet = proof_packet
|
||||
|
||||
if self.callbacks.delivery != None:
|
||||
try:
|
||||
self.callbacks.delivery(self)
|
||||
try: self.callbacks.delivery(self)
|
||||
except Exception as e:
|
||||
RNS.log("Error while executing proof validated callback. The contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
else:
|
||||
return False
|
||||
|
||||
else: return False
|
||||
else: return False
|
||||
|
||||
elif len(proof) == PacketReceipt.IMPL_LENGTH:
|
||||
# This is an implicit proof
|
||||
|
||||
if not hasattr(self.destination, "identity"):
|
||||
return False
|
||||
|
||||
if self.destination.identity == None:
|
||||
return False
|
||||
if not hasattr(self.destination, "identity"): return False
|
||||
if self.destination.identity == None: return False
|
||||
|
||||
signature = proof[:RNS.Identity.SIGLENGTH//8]
|
||||
proof_valid = self.destination.identity.validate(signature, self.hash)
|
||||
|
|
@ -536,16 +538,13 @@ class PacketReceipt:
|
|||
self.proof_packet = proof_packet
|
||||
|
||||
if self.callbacks.delivery != None:
|
||||
try:
|
||||
self.callbacks.delivery(self)
|
||||
except Exception as e:
|
||||
RNS.log("Error while executing proof validated callback. The contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
try: self.callbacks.delivery(self)
|
||||
except Exception as e: RNS.log("Error while executing proof validated callback. The contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
else:
|
||||
return False
|
||||
|
||||
else: return False
|
||||
else: return False
|
||||
|
||||
def get_rtt(self):
|
||||
"""
|
||||
|
|
@ -558,10 +557,8 @@ class PacketReceipt:
|
|||
|
||||
def check_timeout(self):
|
||||
if self.status == PacketReceipt.SENT and self.is_timed_out():
|
||||
if self.timeout == -1:
|
||||
self.status = PacketReceipt.CULLED
|
||||
else:
|
||||
self.status = PacketReceipt.FAILED
|
||||
if self.timeout == -1: self.status = PacketReceipt.CULLED
|
||||
else: self.status = PacketReceipt.FAILED
|
||||
|
||||
self.concluded_at = time.time()
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -31,5 +31,4 @@
|
|||
class Resolver:
|
||||
|
||||
@staticmethod
|
||||
def resolve_identity(full_name):
|
||||
pass
|
||||
def resolve_identity(full_name): pass
|
||||
441
RNS/Resource.py
441
RNS/Resource.py
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -30,6 +30,7 @@
|
|||
|
||||
import RNS
|
||||
import os
|
||||
import io
|
||||
import bz2
|
||||
import math
|
||||
import time
|
||||
|
|
@ -126,6 +127,7 @@ class Resource:
|
|||
PART_TIMEOUT_FACTOR = 4
|
||||
PART_TIMEOUT_FACTOR_AFTER_RTT = 2
|
||||
PROOF_TIMEOUT_FACTOR = 3
|
||||
HMU_WAIT_FACTOR = 3.5
|
||||
MAX_RETRIES = 16
|
||||
MAX_ADV_RETRIES = 4
|
||||
SENDER_GRACE_TIME = 10.0
|
||||
|
|
@ -137,6 +139,7 @@ class Resource:
|
|||
|
||||
HASHMAP_IS_NOT_EXHAUSTED = 0x00
|
||||
HASHMAP_IS_EXHAUSTED = 0xFF
|
||||
HASHMAP_IS_EMPTY = 0xFE
|
||||
|
||||
# Status constants
|
||||
NONE = 0x00
|
||||
|
|
@ -148,7 +151,7 @@ class Resource:
|
|||
COMPLETE = 0x06
|
||||
FAILED = 0x07
|
||||
CORRUPT = 0x08
|
||||
REJECTED = 0x00
|
||||
REJECTED = 0x09
|
||||
|
||||
@staticmethod
|
||||
def reject(advertisement_packet):
|
||||
|
|
@ -180,6 +183,7 @@ class Resource:
|
|||
resource.hashmap_raw = adv.m
|
||||
resource.encrypted = True if resource.flags & 0x01 else False
|
||||
resource.compressed = True if resource.flags >> 1 & 0x01 else False
|
||||
resource.app_flag = True if resource.flags >> 6 & 0x01 else False
|
||||
resource.initiator = False
|
||||
resource.callback = callback
|
||||
resource.__progress_callback = progress_callback
|
||||
|
|
@ -193,6 +197,7 @@ class Resource:
|
|||
resource.window_flexibility = Resource.WINDOW_FLEXIBILITY
|
||||
resource.last_activity = time.time()
|
||||
resource.started_transferring = resource.last_activity
|
||||
resource.advertisement_packet = advertisement_packet
|
||||
|
||||
resource.storagepath = RNS.Reticulum.resourcepath+"/"+resource.original_hash.hex()
|
||||
resource.meta_storagepath = resource.storagepath+".meta"
|
||||
|
|
@ -221,7 +226,7 @@ class Resource:
|
|||
if not resource.link.has_incoming_resource(resource):
|
||||
resource.link.register_incoming_resource(resource)
|
||||
|
||||
RNS.log(f"Accepting resource advertisement for {RNS.prettyhexrep(resource.hash)}. Transfer size is {RNS.prettysize(resource.size)} in {resource.total_parts} parts.", RNS.LOG_DEBUG)
|
||||
RNS.log(f"Accepting resource advertisement for {RNS.prettyhexrep(resource.hash)}. Transfer size is {RNS.prettysize(resource.size)} in {resource.total_parts} parts.", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
if resource.link.callbacks.resource_started != None:
|
||||
try:
|
||||
resource.link.callbacks.resource_started(resource)
|
||||
|
|
@ -233,18 +238,18 @@ class Resource:
|
|||
return resource
|
||||
|
||||
else:
|
||||
RNS.log("Ignoring resource advertisement for "+RNS.prettyhexrep(resource.hash)+", resource already transferring", RNS.LOG_DEBUG)
|
||||
RNS.log("Ignoring resource advertisement for "+RNS.prettyhexrep(resource.hash)+", resource already transferring", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
return None
|
||||
|
||||
except Exception as e:
|
||||
RNS.log("Could not decode resource advertisement, dropping resource", RNS.LOG_DEBUG)
|
||||
RNS.log(f"Could not decode resource advertisement, dropping resource: {e}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
return None
|
||||
|
||||
# Create a resource for transmission to a remote destination
|
||||
# The data passed can be either a bytes-array or a file opened
|
||||
# in binary read mode.
|
||||
def __init__(self, data, link, metadata=None, advertise=True, auto_compress=True, callback=None, progress_callback=None,
|
||||
timeout = None, segment_index = 1, original_hash = None, request_id = None, is_response = False, sent_metadata_size=0):
|
||||
timeout=None, segment_index = 1, original_hash=None, request_id=None, is_response=False, app_flag=False, sent_metadata_size=0):
|
||||
|
||||
data_size = None
|
||||
resource_data = None
|
||||
|
|
@ -254,6 +259,7 @@ class Resource:
|
|||
self.metadata = None
|
||||
self.has_metadata = False
|
||||
self.metadata_size = sent_metadata_size
|
||||
self.app_flag = app_flag
|
||||
|
||||
if metadata != None:
|
||||
packed_metadata = umsgpack.packb(metadata)
|
||||
|
|
@ -278,6 +284,19 @@ class Resource:
|
|||
|
||||
if hasattr(data, "read"):
|
||||
if data_size == None: data_size = os.stat(data.name).st_size
|
||||
if data_size == 0:
|
||||
if isinstance(data, io.BufferedReader):
|
||||
RNS.log(f"Attempting to proxy stream data...", RNS.LOG_DEBUG)
|
||||
stream_proxy = tempfile.TemporaryFile()
|
||||
stream_proxy.write(data.read())
|
||||
stream_proxy.flush(); stream_proxy.seek(0)
|
||||
data_size = os.stat(stream_proxy.name).st_size
|
||||
if data_size: RNS.log(f"Stream proxy succeeded, final data size is {RNS.prettysize(data_size)}", RNS.LOG_DEBUG)
|
||||
else: RNS.log(f"Resource initialisation received an invalid readable input, or stream data that could not be proxied, the transfer will likely fail.", RNS.LOG_WARNING)
|
||||
data = stream_proxy
|
||||
|
||||
else: RNS.log(f"Got zero-sized readable at resource initialisation, the transfer will likely.", RNS.LOG_WARNING)
|
||||
|
||||
self.total_size = data_size + self.metadata_size
|
||||
|
||||
if self.total_size <= Resource.MAX_EFFICIENT_SIZE:
|
||||
|
|
@ -288,12 +307,6 @@ class Resource:
|
|||
data.close()
|
||||
|
||||
else:
|
||||
# self.total_segments = ((data_size-1)//Resource.MAX_EFFICIENT_SIZE)+1
|
||||
# self.segment_index = segment_index
|
||||
# self.split = True
|
||||
# seek_index = segment_index-1
|
||||
# seek_position = seek_index*Resource.MAX_EFFICIENT_SIZE
|
||||
|
||||
self.total_segments = ((self.total_size-1)//Resource.MAX_EFFICIENT_SIZE)+1
|
||||
self.segment_index = segment_index
|
||||
self.split = True
|
||||
|
|
@ -312,30 +325,26 @@ class Resource:
|
|||
self.input_file = data
|
||||
|
||||
elif isinstance(data, bytes):
|
||||
data_size = len(data)
|
||||
data_size = len(data)
|
||||
self.total_size = data_size + self.metadata_size
|
||||
|
||||
resource_data = data
|
||||
resource_data = data
|
||||
self.total_segments = 1
|
||||
self.segment_index = 1
|
||||
self.split = False
|
||||
|
||||
elif data == None:
|
||||
pass
|
||||
elif data == None: pass
|
||||
|
||||
else:
|
||||
raise TypeError("Invalid data instance type passed to resource initialisation")
|
||||
else: raise TypeError("Invalid data instance type passed to resource initialisation")
|
||||
|
||||
if resource_data:
|
||||
if self.has_metadata: data = self.metadata + resource_data
|
||||
else: data = resource_data
|
||||
|
||||
self.status = Resource.NONE
|
||||
self.link = link
|
||||
if self.link.mtu:
|
||||
self.sdu = self.link.mtu - RNS.Reticulum.HEADER_MAXSIZE - RNS.Reticulum.IFAC_MIN_SIZE
|
||||
else:
|
||||
self.sdu = link.mdu or Resource.SDU
|
||||
self.link = link
|
||||
if self.link.mtu: self.sdu = self.link.mtu - RNS.Reticulum.HEADER_MAXSIZE - RNS.Reticulum.IFAC_MIN_SIZE
|
||||
else: self.sdu = link.mdu or Resource.SDU
|
||||
self.max_retries = Resource.MAX_RETRIES
|
||||
self.max_adv_retries = Resource.MAX_ADV_RETRIES
|
||||
self.retries_left = self.max_retries
|
||||
|
|
@ -359,6 +368,7 @@ class Resource:
|
|||
self.request_id = request_id
|
||||
self.started_transferring = None
|
||||
self.is_response = is_response
|
||||
self.max_decompressed_size = Resource.AUTO_COMPRESS_MAX_SIZE
|
||||
self.auto_compress_limit = Resource.AUTO_COMPRESS_MAX_SIZE
|
||||
self.auto_compress_option = auto_compress
|
||||
|
||||
|
|
@ -373,10 +383,8 @@ class Resource:
|
|||
self.req_hashlist = []
|
||||
self.receiver_min_consecutive_height = 0
|
||||
|
||||
if timeout != None:
|
||||
self.timeout = timeout
|
||||
else:
|
||||
self.timeout = self.link.rtt * self.link.traffic_timeout_factor
|
||||
if timeout != None: self.timeout = timeout
|
||||
else: self.timeout = self.link.rtt * self.link.traffic_timeout_factor
|
||||
|
||||
if data != None:
|
||||
self.initiator = True
|
||||
|
|
@ -385,18 +393,21 @@ class Resource:
|
|||
|
||||
compression_began = time.time()
|
||||
if self.auto_compress and data_size <= self.auto_compress_limit:
|
||||
RNS.log("Compressing resource data...", RNS.LOG_EXTREME)
|
||||
self.compressed_data = bz2.compress(self.uncompressed_data)
|
||||
RNS.log("Compression completed in "+str(round(time.time()-compression_began, 3))+" seconds", RNS.LOG_EXTREME)
|
||||
else:
|
||||
self.compressed_data = self.uncompressed_data
|
||||
RNS.log("Compressing resource data...", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
try:
|
||||
self.compressed_data = bz2.compress(self.uncompressed_data)
|
||||
RNS.log("Compression completed in "+str(round(time.time()-compression_began, 3))+" seconds", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
except Exception as e:
|
||||
RNS.log(f"Could not auto-compress resource data, falling back to uncompressed transfer: {e}", RNS.LOG_DEBUG)
|
||||
self.compressed_data = self.uncompressed_data
|
||||
else: self.compressed_data = self.uncompressed_data
|
||||
|
||||
self.uncompressed_size = len(self.uncompressed_data)
|
||||
self.compressed_size = len(self.compressed_data)
|
||||
|
||||
if (self.compressed_size < self.uncompressed_size and auto_compress):
|
||||
saved_bytes = len(self.uncompressed_data) - len(self.compressed_data)
|
||||
RNS.log("Compression saved "+str(saved_bytes)+" bytes, sending compressed", RNS.LOG_EXTREME)
|
||||
RNS.log("Compression saved "+str(saved_bytes)+" bytes, sending compressed", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
|
||||
self.data = b""
|
||||
self.data += RNS.Identity.get_random_hash()[:Resource.RANDOM_HASH_SIZE]
|
||||
|
|
@ -412,7 +423,7 @@ class Resource:
|
|||
self.compressed = False
|
||||
self.compressed_data = None
|
||||
if self.auto_compress and data_size <= self.auto_compress_limit:
|
||||
RNS.log("Compression did not decrease size, sending uncompressed", RNS.LOG_EXTREME)
|
||||
RNS.log("Compression did not decrease size, sending uncompressed", RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
|
||||
|
||||
self.compressed_data = None
|
||||
self.uncompressed_data = None
|
||||
|
|
@ -432,7 +443,7 @@ class Resource:
|
|||
hashmap_ok = False
|
||||
while not hashmap_ok:
|
||||
hashmap_computation_began = time.time()
|
||||
RNS.log("Starting resource hashmap computation with "+str(hashmap_entries)+" entries...", RNS.LOG_EXTREME)
|
||||
RNS.log("Starting resource hashmap computation with "+str(hashmap_entries)+" entries...", RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
|
||||
|
||||
self.random_hash = RNS.Identity.get_random_hash()[:Resource.RANDOM_HASH_SIZE]
|
||||
self.hash = RNS.Identity.full_hash(data+self.random_hash)
|
||||
|
|
@ -448,11 +459,11 @@ class Resource:
|
|||
self.hashmap = b""
|
||||
collision_guard_list = []
|
||||
for i in range(0,hashmap_entries):
|
||||
data = self.data[i*self.sdu:(i+1)*self.sdu]
|
||||
map_hash = self.get_map_hash(data)
|
||||
part_data = self.data[i*self.sdu:(i+1)*self.sdu]
|
||||
map_hash = self.get_map_hash(part_data)
|
||||
|
||||
if map_hash in collision_guard_list:
|
||||
RNS.log("Found hash collision in resource map, remapping...", RNS.LOG_DEBUG)
|
||||
RNS.log("Found hash collision in resource map, remapping...", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
hashmap_ok = False
|
||||
break
|
||||
else:
|
||||
|
|
@ -461,14 +472,14 @@ class Resource:
|
|||
if len(collision_guard_list) > ResourceAdvertisement.COLLISION_GUARD_SIZE:
|
||||
collision_guard_list.pop(0)
|
||||
|
||||
part = RNS.Packet(link, data, context=RNS.Packet.RESOURCE)
|
||||
part = RNS.Packet(link, part_data, context=RNS.Packet.RESOURCE)
|
||||
part.pack()
|
||||
part.map_hash = map_hash
|
||||
|
||||
self.hashmap += part.map_hash
|
||||
self.parts.append(part)
|
||||
|
||||
RNS.log("Hashmap computation concluded in "+str(round(time.time()-hashmap_computation_began, 3))+" seconds", RNS.LOG_EXTREME)
|
||||
RNS.log("Hashmap computation concluded in "+str(round(time.time()-hashmap_computation_began, 3))+" seconds", RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
|
||||
|
||||
self.data = None
|
||||
if advertise:
|
||||
|
|
@ -479,12 +490,12 @@ class Resource:
|
|||
|
||||
def hashmap_update_packet(self, plaintext):
|
||||
if not self.status == Resource.FAILED:
|
||||
self.last_activity = time.time()
|
||||
self.retries_left = self.max_retries
|
||||
|
||||
update = umsgpack.unpackb(plaintext[RNS.Identity.HASHLENGTH//8:])
|
||||
self.hashmap_update(update[0], update[1])
|
||||
if self.waiting_for_hmu:
|
||||
self.last_activity = time.time()
|
||||
self.retries_left = self.max_retries
|
||||
|
||||
update = umsgpack.unpackb(plaintext[RNS.Identity.HASHLENGTH//8:])
|
||||
self.hashmap_update(update[0], update[1])
|
||||
|
||||
def hashmap_update(self, segment, hashmap):
|
||||
if not self.status == Resource.FAILED:
|
||||
|
|
@ -492,12 +503,17 @@ class Resource:
|
|||
seg_len = ResourceAdvertisement.HASHMAP_MAX_LEN
|
||||
hashes = len(hashmap)//Resource.MAPHASH_LEN
|
||||
for i in range(0,hashes):
|
||||
if self.hashmap[i+segment*seg_len] == None:
|
||||
self.hashmap_height += 1
|
||||
if self.hashmap[i+segment*seg_len] == None: self.hashmap_height += 1
|
||||
self.hashmap[i+segment*seg_len] = hashmap[i*Resource.MAPHASH_LEN:(i+1)*Resource.MAPHASH_LEN]
|
||||
|
||||
self.waiting_for_hmu = False
|
||||
self.request_next()
|
||||
if hashes < 1 and segment != 0:
|
||||
RNS.log("Invalid HMU received, cancelling transfer", RNS.LOG_ERROR)
|
||||
self.cancel()
|
||||
|
||||
else:
|
||||
if hashes == 0 and segment == 0: RNS.log(f"Received HMU-less resource advertisement, requesting initial hashmap", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
self.waiting_for_hmu = False
|
||||
self.request_next()
|
||||
|
||||
def get_map_hash(self, data):
|
||||
return RNS.Identity.full_hash(data+self.random_hash)[:Resource.MAPHASH_LEN]
|
||||
|
|
@ -514,13 +530,22 @@ class Resource:
|
|||
prepare_thread = threading.Thread(target=self.__prepare_next_segment, daemon=True)
|
||||
prepare_thread.start()
|
||||
|
||||
def ensure_link(self):
|
||||
if not self.link or self.link.status != RNS.Link.ACTIVE:
|
||||
RNS.log(f"Invalid link state for {self}, aborting transfer", RNS.LOG_VERBOSE)
|
||||
try: self.cancel()
|
||||
except Exception as e: RNS.log(f"Error while cancelling resource on link-state abort: {e}", RNS.LOG_ERROR)
|
||||
return False
|
||||
else: return True
|
||||
|
||||
def __advertise_job(self):
|
||||
self.advertisement_packet = RNS.Packet(self.link, ResourceAdvertisement(self).pack(), context=RNS.Packet.RESOURCE_ADV)
|
||||
self.advertisement_packet = RNS.Packet(self.link, ResourceAdvertisement(self, mdu=self.link.mdu).pack(), context=RNS.Packet.RESOURCE_ADV)
|
||||
while not self.link.ready_for_new_resource():
|
||||
self.status = Resource.QUEUED
|
||||
sleep(0.25)
|
||||
|
||||
try:
|
||||
if not self.ensure_link(): return
|
||||
self.advertisement_packet.send()
|
||||
self.last_activity = time.time()
|
||||
self.started_transferring = self.last_activity
|
||||
|
|
@ -529,7 +554,7 @@ class Resource:
|
|||
self.status = Resource.ADVERTISED
|
||||
self.retries_left = self.max_adv_retries
|
||||
self.link.register_outgoing_resource(self)
|
||||
RNS.log("Sent resource advertisement for "+RNS.prettyhexrep(self.hash), RNS.LOG_EXTREME)
|
||||
RNS.log("Sent resource advertisement for "+RNS.prettyhexrep(self.hash), RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
|
||||
except Exception as e:
|
||||
RNS.log("Could not advertise resource, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
self.cancel()
|
||||
|
|
@ -538,18 +563,13 @@ class Resource:
|
|||
self.watchdog_job()
|
||||
|
||||
def update_eifr(self):
|
||||
if self.rtt == None:
|
||||
rtt = self.link.rtt
|
||||
else:
|
||||
rtt = self.rtt
|
||||
if self.rtt == None: rtt = self.link.rtt
|
||||
else: rtt = self.rtt
|
||||
|
||||
if self.req_data_rtt_rate != 0:
|
||||
expected_inflight_rate = self.req_data_rtt_rate*8
|
||||
if self.req_data_rtt_rate != 0: expected_inflight_rate = self.req_data_rtt_rate*8
|
||||
else:
|
||||
if self.previous_eifr != None:
|
||||
expected_inflight_rate = self.previous_eifr
|
||||
else:
|
||||
expected_inflight_rate = self.link.establishment_cost*8 / rtt
|
||||
if self.previous_eifr != None: expected_inflight_rate = self.previous_eifr
|
||||
else: expected_inflight_rate = self.link.establishment_cost*8 / rtt
|
||||
|
||||
self.eifr = expected_inflight_rate
|
||||
if self.link: self.link.expected_rate = self.eifr
|
||||
|
|
@ -563,28 +583,28 @@ class Resource:
|
|||
this_job_id = self.__watchdog_job_id
|
||||
|
||||
while self.status < Resource.ASSEMBLING and this_job_id == self.__watchdog_job_id:
|
||||
while self.watchdog_lock:
|
||||
sleep(0.025)
|
||||
while self.watchdog_lock: sleep(0.025)
|
||||
|
||||
sleep_time = None
|
||||
if self.status == Resource.ADVERTISED:
|
||||
sleep_time = (self.adv_sent+self.timeout+Resource.PROCESSING_GRACE)-time.time()
|
||||
if sleep_time < 0:
|
||||
if self.retries_left <= 0:
|
||||
RNS.log("Resource transfer timeout after sending advertisement", RNS.LOG_DEBUG)
|
||||
RNS.log("Resource transfer timeout after sending advertisement", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
self.cancel()
|
||||
sleep_time = 0.001
|
||||
else:
|
||||
try:
|
||||
RNS.log("No part requests received, retrying resource advertisement...", RNS.LOG_DEBUG)
|
||||
RNS.log("No part requests received, retrying resource advertisement...", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
self.retries_left -= 1
|
||||
self.advertisement_packet = RNS.Packet(self.link, ResourceAdvertisement(self).pack(), context=RNS.Packet.RESOURCE_ADV)
|
||||
if not self.ensure_link(): return
|
||||
self.advertisement_packet = RNS.Packet(self.link, ResourceAdvertisement(self, mdu=self.link.mdu).pack(), context=RNS.Packet.RESOURCE_ADV)
|
||||
self.advertisement_packet.send()
|
||||
self.last_activity = time.time()
|
||||
self.adv_sent = self.last_activity
|
||||
sleep_time = 0.001
|
||||
except Exception as e:
|
||||
RNS.log("Could not resend advertisement packet, cancelling resource. The contained exception was: "+str(e), RNS.LOG_VERBOSE)
|
||||
RNS.log("Could not resend advertisement packet, cancelling resource. The contained exception was: "+str(e), RNS.LOG_VERBOSE) if RNS.sl(RNS.LOG_VERBOSE) else None
|
||||
self.cancel()
|
||||
|
||||
|
||||
|
|
@ -594,21 +614,22 @@ class Resource:
|
|||
extra_wait = retries_used * Resource.PER_RETRY_DELAY
|
||||
|
||||
self.update_eifr()
|
||||
expected_hmu_wait_remaining = (self.sdu*8*self.HMU_WAIT_FACTOR)/self.eifr if self.waiting_for_hmu or self.outstanding_parts == 0 else 0
|
||||
expected_tof_remaining = (self.outstanding_parts*self.sdu*8)/self.eifr
|
||||
|
||||
if self.req_resp_rtt_rate != 0:
|
||||
sleep_time = self.last_activity + self.part_timeout_factor*expected_tof_remaining + Resource.RETRY_GRACE_TIME + extra_wait - time.time()
|
||||
sleep_time = self.last_activity + self.part_timeout_factor*expected_tof_remaining + expected_hmu_wait_remaining + Resource.RETRY_GRACE_TIME + extra_wait - time.time()
|
||||
else:
|
||||
sleep_time = self.last_activity + self.part_timeout_factor*((3*self.sdu)/self.eifr) + Resource.RETRY_GRACE_TIME + extra_wait - time.time()
|
||||
|
||||
# TODO: Remove debug at some point
|
||||
# RNS.log(f"EIFR {RNS.prettyspeed(self.eifr)}, ETOF {RNS.prettyshorttime(expected_tof_remaining)} ", RNS.LOG_DEBUG, pt=True)
|
||||
# RNS.log(f"EIFR {RNS.prettyspeed(self.eifr)}, ETOF {RNS.prettyshorttime(expected_tof_remaining)}, EHWR {RNS.prettyshorttime(expected_hmu_wait_remaining)}", RNS.LOG_DEBUG, pt=True)
|
||||
# RNS.log(f"Resource ST {RNS.prettyshorttime(sleep_time)}, RTT {RNS.prettyshorttime(self.rtt or self.link.rtt)}, {self.outstanding_parts} left", RNS.LOG_DEBUG, pt=True)
|
||||
|
||||
if sleep_time < 0:
|
||||
if self.retries_left > 0:
|
||||
ms = "" if self.outstanding_parts == 1 else "s"
|
||||
RNS.log(f"Timed out waiting for {self.outstanding_parts} part{ms}, requesting retry on {self}", RNS.LOG_DEBUG)
|
||||
RNS.log(f"Timed out waiting for {self.outstanding_parts} part{ms}, requesting retry on {self}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
if self.window > self.window_min:
|
||||
self.window -= 1
|
||||
if self.window_max > self.window_min:
|
||||
|
|
@ -628,7 +649,7 @@ class Resource:
|
|||
max_wait = self.rtt * self.timeout_factor * self.max_retries + self.sender_grace_time + max_extra_wait
|
||||
sleep_time = self.last_activity + max_wait - time.time()
|
||||
if sleep_time < 0:
|
||||
RNS.log("Resource timed out waiting for part requests", RNS.LOG_DEBUG)
|
||||
RNS.log("Resource timed out waiting for part requests", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
self.cancel()
|
||||
sleep_time = 0.001
|
||||
|
||||
|
|
@ -640,11 +661,11 @@ class Resource:
|
|||
sleep_time = self.last_part_sent + (self.rtt*self.timeout_factor+self.sender_grace_time) - time.time()
|
||||
if sleep_time < 0:
|
||||
if self.retries_left <= 0:
|
||||
RNS.log("Resource timed out waiting for proof", RNS.LOG_DEBUG)
|
||||
RNS.log("Resource timed out waiting for proof", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
self.cancel()
|
||||
sleep_time = 0.001
|
||||
else:
|
||||
RNS.log("All parts sent, but no resource proof received, querying network cache...", RNS.LOG_DEBUG)
|
||||
RNS.log("All parts sent, but no resource proof received, querying network cache...", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
self.retries_left -= 1
|
||||
expected_data = self.hash + self.expected_proof
|
||||
expected_proof_packet = RNS.Packet(self.link, expected_data, packet_type=RNS.Packet.PROOF, context=RNS.Packet.RESOURCE_PRF)
|
||||
|
|
@ -653,13 +674,13 @@ class Resource:
|
|||
self.last_part_sent = time.time()
|
||||
sleep_time = 0.001
|
||||
|
||||
elif self.status == Resource.REJECTED:
|
||||
elif self.status >= Resource.ASSEMBLING:
|
||||
sleep_time = 0.001
|
||||
|
||||
if sleep_time == 0:
|
||||
RNS.log("Warning! Link watchdog sleep time of 0!", RNS.LOG_DEBUG)
|
||||
RNS.log("Warning! Link watchdog sleep time of 0!", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
if sleep_time == None or sleep_time < 0:
|
||||
RNS.log("Timing error, cancelling resource transfer.", RNS.LOG_ERROR)
|
||||
RNS.log(f"Timing error ({sleep_time}/{self.status}), cancelling resource transfer.", RNS.LOG_ERROR)
|
||||
self.cancel()
|
||||
|
||||
if sleep_time != None:
|
||||
|
|
@ -677,8 +698,15 @@ class Resource:
|
|||
# Strip off random hash
|
||||
data = data[Resource.RANDOM_HASH_SIZE:]
|
||||
|
||||
if self.compressed: self.data = bz2.decompress(data)
|
||||
else: self.data = data
|
||||
if not self.compressed: self.data = data
|
||||
else:
|
||||
decompressor = bz2.BZ2Decompressor()
|
||||
self.data = decompressor.decompress(data, max_length=self.max_decompressed_size)
|
||||
if not decompressor.eof:
|
||||
self.status = Resource.CORRUPT
|
||||
self.cancel()
|
||||
RNS.log(f"Decompressed resource exceeded maximum decompressed size. The resource was rejected.", RNS.LOG_ERROR)
|
||||
return
|
||||
|
||||
calculated_hash = RNS.Identity.full_hash(self.data+self.random_hash)
|
||||
if calculated_hash == self.hash:
|
||||
|
|
@ -735,7 +763,7 @@ class Resource:
|
|||
except Exception as e:
|
||||
RNS.log(f"Error while cleaning up resource files, the contained exception was: {e}", RNS.LOG_ERROR)
|
||||
else:
|
||||
RNS.log("Resource segment "+str(self.segment_index)+" of "+str(self.total_segments)+" received, waiting for next segment to be announced", RNS.LOG_DEBUG)
|
||||
RNS.log("Resource segment "+str(self.segment_index)+" of "+str(self.total_segments)+" received, waiting for next segment to be announced", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
|
||||
|
||||
def prove(self):
|
||||
|
|
@ -743,30 +771,32 @@ class Resource:
|
|||
try:
|
||||
proof = RNS.Identity.full_hash(self.data+self.hash)
|
||||
proof_data = self.hash+proof
|
||||
if not self.ensure_link(): return
|
||||
proof_packet = RNS.Packet(self.link, proof_data, packet_type=RNS.Packet.PROOF, context=RNS.Packet.RESOURCE_PRF)
|
||||
proof_packet.send()
|
||||
RNS.Transport.cache(proof_packet, force_cache=True)
|
||||
except Exception as e:
|
||||
RNS.log("Could not send proof packet, cancelling resource", RNS.LOG_DEBUG)
|
||||
RNS.log("The contained exception was: "+str(e), RNS.LOG_DEBUG)
|
||||
RNS.log("Could not send proof packet, cancelling resource", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
RNS.log("The contained exception was: "+str(e), RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
self.cancel()
|
||||
|
||||
def __prepare_next_segment(self):
|
||||
# Prepare the next segment for advertisement
|
||||
RNS.log(f"Preparing segment {self.segment_index+1} of {self.total_segments} for resource {self}", RNS.LOG_DEBUG)
|
||||
RNS.log(f"Preparing segment {self.segment_index+1} of {self.total_segments} for resource {self}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
self.preparing_next_segment = True
|
||||
self.next_segment = Resource(
|
||||
self.input_file, self.link,
|
||||
callback = self.callback,
|
||||
segment_index = self.segment_index+1,
|
||||
original_hash=self.original_hash,
|
||||
progress_callback = self.__progress_callback,
|
||||
request_id = self.request_id,
|
||||
is_response = self.is_response,
|
||||
advertise = False,
|
||||
auto_compress = self.auto_compress_option,
|
||||
sent_metadata_size = self.metadata_size,
|
||||
)
|
||||
self.next_segment = Resource(self.input_file, self.link,
|
||||
callback = self.callback,
|
||||
segment_index = self.segment_index+1,
|
||||
original_hash=self.original_hash,
|
||||
progress_callback = self.__progress_callback,
|
||||
request_id = self.request_id,
|
||||
is_response = self.is_response,
|
||||
advertise = False,
|
||||
auto_compress = self.auto_compress_option,
|
||||
app_flag = self.app_flag,
|
||||
sent_metadata_size = self.metadata_size)
|
||||
|
||||
if self.__progress_callback: self.next_segment.progress_callback(self.__progress_callback)
|
||||
|
||||
def validate_proof(self, proof_data):
|
||||
if not self.status == Resource.FAILED:
|
||||
|
|
@ -849,9 +879,9 @@ class Resource:
|
|||
part_data = packet.data
|
||||
part_hash = self.get_map_hash(part_data)
|
||||
|
||||
consecutive_index = self.consecutive_completed_height if self.consecutive_completed_height >= 0 else 0
|
||||
i = consecutive_index
|
||||
for map_hash in self.hashmap[consecutive_index:consecutive_index+self.window]:
|
||||
search_start = self.consecutive_completed_height+1
|
||||
i = search_start
|
||||
for map_hash in self.hashmap[search_start:search_start+self.window]:
|
||||
if map_hash == part_hash:
|
||||
if self.parts[i] == None:
|
||||
|
||||
|
|
@ -862,9 +892,6 @@ class Resource:
|
|||
self.outstanding_parts -= 1
|
||||
|
||||
# Update consecutive completed pointer
|
||||
if i == self.consecutive_completed_height + 1:
|
||||
self.consecutive_completed_height = i
|
||||
|
||||
cp = self.consecutive_completed_height + 1
|
||||
while cp < len(self.parts) and self.parts[cp] != None:
|
||||
self.consecutive_completed_height = cp
|
||||
|
|
@ -927,23 +954,30 @@ class Resource:
|
|||
hashmap_exhausted = Resource.HASHMAP_IS_NOT_EXHAUSTED
|
||||
requested_hashes = b""
|
||||
|
||||
i = 0; pn = self.consecutive_completed_height+1
|
||||
search_start = pn
|
||||
search_size = self.window
|
||||
|
||||
for part in self.parts[search_start:search_start+search_size]:
|
||||
if part == None:
|
||||
part_hash = self.hashmap[pn]
|
||||
if part_hash != None:
|
||||
requested_hashes += part_hash
|
||||
self.outstanding_parts += 1
|
||||
i += 1
|
||||
else:
|
||||
hashmap_exhausted = Resource.HASHMAP_IS_EXHAUSTED
|
||||
# Initial advertisement-carried hashmap was empty,
|
||||
# we can't request any real parts, and need an
|
||||
# actual hashmap update first
|
||||
if self.hashmap[0] == None:
|
||||
hashmap_exhausted = Resource.HASHMAP_IS_EMPTY
|
||||
|
||||
pn += 1
|
||||
if i >= self.window or hashmap_exhausted == Resource.HASHMAP_IS_EXHAUSTED:
|
||||
break
|
||||
else:
|
||||
i = 0; pn = self.consecutive_completed_height+1
|
||||
search_start = pn
|
||||
search_size = self.window
|
||||
|
||||
for part in self.parts[search_start:search_start+search_size]:
|
||||
if part == None:
|
||||
part_hash = self.hashmap[pn]
|
||||
if part_hash != None:
|
||||
requested_hashes += part_hash
|
||||
self.outstanding_parts += 1
|
||||
i += 1
|
||||
else:
|
||||
hashmap_exhausted = Resource.HASHMAP_IS_EXHAUSTED
|
||||
|
||||
pn += 1
|
||||
if i >= self.window or hashmap_exhausted == Resource.HASHMAP_IS_EXHAUSTED:
|
||||
break
|
||||
|
||||
hmu_part = bytes([hashmap_exhausted])
|
||||
if hashmap_exhausted == Resource.HASHMAP_IS_EXHAUSTED:
|
||||
|
|
@ -951,19 +985,24 @@ class Resource:
|
|||
hmu_part += last_map_hash
|
||||
self.waiting_for_hmu = True
|
||||
|
||||
if hashmap_exhausted == Resource.HASHMAP_IS_EMPTY:
|
||||
self.waiting_for_hmu = True
|
||||
|
||||
request_data = hmu_part + self.hash + requested_hashes
|
||||
request_packet = RNS.Packet(self.link, request_data, context = RNS.Packet.RESOURCE_REQ)
|
||||
|
||||
try:
|
||||
if not self.ensure_link(): return
|
||||
request_packet.send()
|
||||
self.last_activity = time.time()
|
||||
self.req_sent = self.last_activity
|
||||
self.req_sent_bytes = len(request_packet.raw)
|
||||
self.rtt_rxd_bytes_at_part_req = self.rtt_rxd_bytes
|
||||
self.req_resp = None
|
||||
|
||||
except Exception as e:
|
||||
RNS.log("Could not send resource request packet, cancelling resource", RNS.LOG_DEBUG)
|
||||
RNS.log("The contained exception was: "+str(e), RNS.LOG_DEBUG)
|
||||
RNS.log("Could not send resource request packet, cancelling resource", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
RNS.log("The contained exception was: "+str(e), RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
self.cancel()
|
||||
|
||||
# Called on outgoing resource to make it send more data
|
||||
|
|
@ -979,8 +1018,16 @@ class Resource:
|
|||
|
||||
self.retries_left = self.max_retries
|
||||
|
||||
wants_more_hashmap = True if request_data[0] == Resource.HASHMAP_IS_EXHAUSTED else False
|
||||
pad = 1+Resource.MAPHASH_LEN if wants_more_hashmap else 1
|
||||
pad = 1
|
||||
hashmap_is_empty = False
|
||||
wants_more_hashmap = False
|
||||
if request_data[0] == Resource.HASHMAP_IS_EMPTY:
|
||||
wants_more_hashmap = True
|
||||
hashmap_is_empty = True
|
||||
|
||||
if request_data[0] == Resource.HASHMAP_IS_EXHAUSTED:
|
||||
wants_more_hashmap = True
|
||||
pad += Resource.MAPHASH_LEN
|
||||
|
||||
requested_hashes = request_data[pad+RNS.Identity.HASHLENGTH//8:]
|
||||
|
||||
|
|
@ -998,41 +1045,38 @@ class Resource:
|
|||
|
||||
for part in requested_parts:
|
||||
try:
|
||||
if not self.ensure_link(): return
|
||||
if not part.sent:
|
||||
part.send()
|
||||
self.sent_parts += 1
|
||||
else:
|
||||
part.resend()
|
||||
else: part.resend()
|
||||
|
||||
self.last_activity = time.time()
|
||||
self.last_part_sent = self.last_activity
|
||||
|
||||
except Exception as e:
|
||||
RNS.log("Resource could not send parts, cancelling transfer!", RNS.LOG_DEBUG)
|
||||
RNS.log("The contained exception was: "+str(e), RNS.LOG_DEBUG)
|
||||
RNS.log("Resource could not send parts, cancelling transfer!", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
RNS.log("The contained exception was: "+str(e), RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
self.cancel()
|
||||
|
||||
if wants_more_hashmap:
|
||||
last_map_hash = request_data[1:Resource.MAPHASH_LEN+1]
|
||||
|
||||
part_index = self.receiver_min_consecutive_height
|
||||
search_start = part_index
|
||||
search_end = self.receiver_min_consecutive_height+ResourceAdvertisement.COLLISION_GUARD_SIZE
|
||||
search_start = part_index if not hashmap_is_empty else 0
|
||||
search_end = self.receiver_min_consecutive_height+ResourceAdvertisement.COLLISION_GUARD_SIZE if not hashmap_is_empty else 0
|
||||
for part in self.parts[search_start:search_end]:
|
||||
part_index += 1
|
||||
if part.map_hash == last_map_hash:
|
||||
break
|
||||
if part.map_hash == last_map_hash: break
|
||||
|
||||
self.receiver_min_consecutive_height = max(part_index-1-Resource.WINDOW_MAX, 0)
|
||||
|
||||
if part_index % ResourceAdvertisement.HASHMAP_MAX_LEN != 0:
|
||||
if not hashmap_is_empty and part_index % ResourceAdvertisement.HASHMAP_MAX_LEN != 0:
|
||||
RNS.log("Resource sequencing error, cancelling transfer!", RNS.LOG_ERROR)
|
||||
self.cancel()
|
||||
return
|
||||
else:
|
||||
segment = part_index // ResourceAdvertisement.HASHMAP_MAX_LEN
|
||||
else: segment = part_index // ResourceAdvertisement.HASHMAP_MAX_LEN
|
||||
|
||||
|
||||
hashmap_start = segment*ResourceAdvertisement.HASHMAP_MAX_LEN
|
||||
hashmap_end = min((segment+1)*ResourceAdvertisement.HASHMAP_MAX_LEN, len(self.parts))
|
||||
|
||||
|
|
@ -1040,15 +1084,21 @@ class Resource:
|
|||
for i in range(hashmap_start,hashmap_end):
|
||||
hashmap += self.hashmap[i*Resource.MAPHASH_LEN:(i+1)*Resource.MAPHASH_LEN]
|
||||
|
||||
if not hashmap:
|
||||
RNS.log("Resource HMU error, cancelling transfer!", RNS.LOG_ERROR)
|
||||
self.cancel()
|
||||
return
|
||||
|
||||
hmu = self.hash+umsgpack.packb([segment, hashmap])
|
||||
hmu_packet = RNS.Packet(self.link, hmu, context = RNS.Packet.RESOURCE_HMU)
|
||||
|
||||
try:
|
||||
if not self.ensure_link(): return
|
||||
hmu_packet.send()
|
||||
self.last_activity = time.time()
|
||||
except Exception as e:
|
||||
RNS.log("Could not send resource HMU packet, cancelling resource", RNS.LOG_DEBUG)
|
||||
RNS.log("The contained exception was: "+str(e), RNS.LOG_DEBUG)
|
||||
RNS.log("Could not send resource HMU packet, cancelling resource", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
RNS.log("The contained exception was: "+str(e), RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
self.cancel()
|
||||
|
||||
if self.sent_parts == len(self.parts):
|
||||
|
|
@ -1056,8 +1106,7 @@ class Resource:
|
|||
self.retries_left = 3
|
||||
|
||||
if self.__progress_callback != None:
|
||||
try:
|
||||
self.__progress_callback(self)
|
||||
try: self.__progress_callback(self)
|
||||
except Exception as e:
|
||||
RNS.log("Error while executing progress callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
|
||||
|
|
@ -1065,18 +1114,29 @@ class Resource:
|
|||
"""
|
||||
Cancels transferring the resource.
|
||||
"""
|
||||
if self.status < Resource.COMPLETE:
|
||||
if self.next_segment: self.next_segment.cancel()
|
||||
|
||||
if self.status == Resource.CORRUPT:
|
||||
if self in self.link.incoming_resources: self.link.cancel_incoming_resource(self)
|
||||
self.reject(self.advertisement_packet)
|
||||
self.link.teardown()
|
||||
|
||||
elif self.status < Resource.COMPLETE:
|
||||
self.status = Resource.FAILED
|
||||
if self.initiator:
|
||||
if self.link.status == RNS.Link.ACTIVE:
|
||||
try:
|
||||
cancel_packet = RNS.Packet(self.link, self.hash, context=RNS.Packet.RESOURCE_ICL)
|
||||
cancel_packet.send()
|
||||
except Exception as e:
|
||||
RNS.log("Could not send resource cancel packet, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
self.link.cancel_outgoing_resource(self)
|
||||
except Exception as e: RNS.log("Could not send resource cancel packet, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
if self in self.link.outgoing_resources: self.link.cancel_outgoing_resource(self)
|
||||
else:
|
||||
self.link.cancel_incoming_resource(self)
|
||||
if self.link.status == RNS.Link.ACTIVE:
|
||||
try:
|
||||
cancel_packet = RNS.Packet(self.link, self.hash, context=RNS.Packet.RESOURCE_RCL)
|
||||
cancel_packet.send()
|
||||
except Exception as e: RNS.log("Could not send resource cancel packet, the contained exception was: "+str(e), RNS.LOG_ERROR)
|
||||
if self in self.link.incoming_resources: self.link.cancel_incoming_resource(self)
|
||||
|
||||
if self.callback != None:
|
||||
try:
|
||||
|
|
@ -1089,7 +1149,7 @@ class Resource:
|
|||
if self.status < Resource.COMPLETE:
|
||||
if self.initiator:
|
||||
self.status = Resource.REJECTED
|
||||
self.link.cancel_outgoing_resource(self)
|
||||
if self in self.link.outgoing_resources: self.link.cancel_outgoing_resource(self)
|
||||
if self.callback != None:
|
||||
try:
|
||||
self.link.resource_concluded(self)
|
||||
|
|
@ -1103,6 +1163,7 @@ class Resource:
|
|||
|
||||
def progress_callback(self, callback):
|
||||
self.__progress_callback = callback
|
||||
if self.next_segment: self.next_segment.progress_callback(callback)
|
||||
|
||||
def get_progress(self):
|
||||
"""
|
||||
|
|
@ -1209,7 +1270,9 @@ class Resource:
|
|||
return self.compressed
|
||||
|
||||
def __str__(self):
|
||||
return "<"+RNS.hexrep(self.hash,delimit=False)+"/"+RNS.hexrep(self.link.link_id,delimit=False)+">"
|
||||
if hasattr(self, "hash") and self.hash and hasattr(self, "link") and self.link:
|
||||
return "<"+RNS.hexrep(self.hash,delimit=False)+"/"+RNS.hexrep(self.link.link_id,delimit=False)+">"
|
||||
else: return "<initializing_resource/unknown>"
|
||||
|
||||
|
||||
class ResourceAdvertisement:
|
||||
|
|
@ -1222,42 +1285,33 @@ class ResourceAdvertisement:
|
|||
@staticmethod
|
||||
def is_request(advertisement_packet):
|
||||
adv = ResourceAdvertisement.unpack(advertisement_packet.plaintext)
|
||||
if adv.q != None and adv.u:
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
|
||||
if adv.q != None and adv.u: return True
|
||||
else: return False
|
||||
|
||||
@staticmethod
|
||||
def is_response(advertisement_packet):
|
||||
adv = ResourceAdvertisement.unpack(advertisement_packet.plaintext)
|
||||
|
||||
if adv.q != None and adv.p:
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
|
||||
if adv.q != None and adv.p: return True
|
||||
else: return False
|
||||
|
||||
@staticmethod
|
||||
def read_request_id(advertisement_packet):
|
||||
adv = ResourceAdvertisement.unpack(advertisement_packet.plaintext)
|
||||
return adv.q
|
||||
|
||||
|
||||
@staticmethod
|
||||
def read_transfer_size(advertisement_packet):
|
||||
adv = ResourceAdvertisement.unpack(advertisement_packet.plaintext)
|
||||
return adv.t
|
||||
|
||||
|
||||
@staticmethod
|
||||
def read_size(advertisement_packet):
|
||||
adv = ResourceAdvertisement.unpack(advertisement_packet.plaintext)
|
||||
return adv.d
|
||||
|
||||
|
||||
def __init__(self, resource=None, request_id=None, is_response=False):
|
||||
def __init__(self, resource=None, request_id=None, is_response=False, mdu=None):
|
||||
self.link = None
|
||||
self.mdu = mdu or RNS.Link.MDU
|
||||
if resource != None:
|
||||
self.t = resource.size # Transfer size
|
||||
self.d = resource.total_size # Total uncompressed data size
|
||||
|
|
@ -1268,6 +1322,7 @@ class ResourceAdvertisement:
|
|||
self.m = resource.hashmap # Resource hashmap
|
||||
self.c = resource.compressed # Compression flag
|
||||
self.e = resource.encrypted # Encryption flag
|
||||
self.a = resource.app_flag # Application flag
|
||||
self.s = resource.split # Split flag
|
||||
self.x = resource.has_metadata # Metadata flag
|
||||
self.i = resource.segment_index # Segment index
|
||||
|
|
@ -1285,31 +1340,17 @@ class ResourceAdvertisement:
|
|||
self.p = True
|
||||
|
||||
# Flags
|
||||
self.f = 0x00 | self.x << 5 | self.p << 4 | self.u << 3 | self.s << 2 | self.c << 1 | self.e
|
||||
self.f = 0x00 | self.a << 6 | self.x << 5 | self.p << 4 | self.u << 3 | self.s << 2 | self.c << 1 | self.e
|
||||
|
||||
def get_transfer_size(self):
|
||||
return self.t
|
||||
|
||||
def get_data_size(self):
|
||||
return self.d
|
||||
|
||||
def get_parts(self):
|
||||
return self.n
|
||||
|
||||
def get_segments(self):
|
||||
return self.l
|
||||
|
||||
def get_hash(self):
|
||||
return self.h
|
||||
|
||||
def is_compressed(self):
|
||||
return self.c
|
||||
|
||||
def has_metadata(self):
|
||||
return self.x
|
||||
|
||||
def get_link(self):
|
||||
return self.link
|
||||
def get_transfer_size(self): return self.t
|
||||
def get_data_size(self): return self.d
|
||||
def get_parts(self): return self.n
|
||||
def get_segments(self): return self.l
|
||||
def get_hash(self): return self.h
|
||||
def is_compressed(self): return self.c
|
||||
def has_metadata(self): return self.x
|
||||
def get_link(self): return self.link
|
||||
def app_flag_set(self): return self.f >> 6 & 0x01
|
||||
|
||||
def pack(self, segment=0):
|
||||
hashmap_start = segment*ResourceAdvertisement.HASHMAP_MAX_LEN
|
||||
|
|
@ -1319,22 +1360,25 @@ class ResourceAdvertisement:
|
|||
for i in range(hashmap_start,hashmap_end):
|
||||
hashmap += self.m[i*Resource.MAPHASH_LEN:(i+1)*Resource.MAPHASH_LEN]
|
||||
|
||||
dictionary = {
|
||||
"t": self.t, # Transfer size
|
||||
"d": self.d, # Data size
|
||||
"n": self.n, # Number of parts
|
||||
"h": self.h, # Resource hash
|
||||
"r": self.r, # Resource random hash
|
||||
"o": self.o, # Original hash
|
||||
"i": self.i, # Segment index
|
||||
"l": self.l, # Total segments
|
||||
"q": self.q, # Request ID
|
||||
"f": self.f, # Resource flags
|
||||
"m": hashmap
|
||||
}
|
||||
dictionary = { "t": self.t, # Transfer size
|
||||
"d": self.d, # Data size
|
||||
"n": self.n, # Number of parts
|
||||
"h": self.h, # Resource hash
|
||||
"r": self.r, # Resource random hash
|
||||
"o": self.o, # Original hash
|
||||
"i": self.i, # Segment index
|
||||
"l": self.l, # Total segments
|
||||
"q": self.q, # Request ID
|
||||
"f": self.f, # Resource flags
|
||||
"m": hashmap }
|
||||
|
||||
return umsgpack.packb(dictionary)
|
||||
packed_adv = umsgpack.packb(dictionary)
|
||||
if len(packed_adv) > self.mdu:
|
||||
RNS.log(f"Resource advertisement ({RNS.prettysize(len(packed_adv))}) exceeds link MDU, deferring initial HMU", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
|
||||
dictionary["m"] = b""
|
||||
packed_adv = umsgpack.packb(dictionary)
|
||||
|
||||
return packed_adv
|
||||
|
||||
@staticmethod
|
||||
def unpack(data):
|
||||
|
|
@ -1358,5 +1402,8 @@ class ResourceAdvertisement:
|
|||
adv.u = True if ((adv.f >> 3) & 0x01) == 0x01 else False
|
||||
adv.p = True if ((adv.f >> 4) & 0x01) == 0x01 else False
|
||||
adv.x = True if ((adv.f >> 5) & 0x01) == 0x01 else False
|
||||
adv.a = True if ((adv.f >> 6) & 0x01) == 0x01 else False
|
||||
|
||||
return adv
|
||||
if adv.t > Resource.MAX_EFFICIENT_SIZE*3: raise ValueError("Invalid transfer size")
|
||||
|
||||
return adv
|
||||
|
|
|
|||
1714
RNS/Reticulum.py
1714
RNS/Reticulum.py
File diff suppressed because it is too large
Load diff
1093
RNS/Stamps.py
Normal file
1093
RNS/Stamps.py
Normal file
File diff suppressed because it is too large
Load diff
851
RNS/Storage.py
Normal file
851
RNS/Storage.py
Normal file
|
|
@ -0,0 +1,851 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
import RNS
|
||||
import hashlib
|
||||
import struct
|
||||
import sys
|
||||
import threading
|
||||
import os
|
||||
import time
|
||||
from array import array
|
||||
|
||||
class AnnounceRateTable:
|
||||
# Columnar store for announce rate tracking. Layout:
|
||||
# _index : dict[destination_hash -> slot]
|
||||
# _keys : list[slot] -> destination_hash (for swap-remove bookkeeping)
|
||||
# _last : array('d') last non-violating announce time
|
||||
# _blocked: array('d') blocked_until
|
||||
# _viol : array('i') rate_violations
|
||||
# _ts : array('d') flat ring of max_timestamps doubles per slot
|
||||
# _ts_len : array('B') valid timestamp count per slot
|
||||
# _ts_head: array('B') ring write cursor per slot
|
||||
|
||||
def __init__(self, max_timestamps=16):
|
||||
assert max_timestamps <= 255, "Timestamp ring size must fit in a single byte"
|
||||
self.max_ts = max_timestamps
|
||||
self._index = {}
|
||||
self._keys = []
|
||||
self._last = array('d')
|
||||
self._blocked = array('d')
|
||||
self._viol = array('i')
|
||||
self._ts = array('d')
|
||||
self._ts_len = array('B')
|
||||
self._ts_head = array('B')
|
||||
self._cursor = 0
|
||||
|
||||
def __len__(self): return len(self._keys)
|
||||
def __contains__(self, destination_hash): return destination_hash in self._index
|
||||
|
||||
# Perform the announce-rate check-and-update for one announce.
|
||||
# Caller must hold the announce_rate_table_lock.
|
||||
def check_and_update(self, destination_hash, now, target, grace, penalty):
|
||||
slot = self._index.get(destination_hash)
|
||||
|
||||
if slot is None:
|
||||
slot = len(self._keys)
|
||||
self._index[destination_hash] = slot
|
||||
self._keys.append(destination_hash)
|
||||
self._last.append(now)
|
||||
self._blocked.append(0.0)
|
||||
self._viol.append(0)
|
||||
self._ts.extend([0.0]*self.max_ts)
|
||||
self._ts[slot*self.max_ts] = now
|
||||
self._ts_len.append(1)
|
||||
self._ts_head.append(1 % self.max_ts)
|
||||
return False
|
||||
|
||||
base = slot*self.max_ts
|
||||
head = self._ts_head[slot]
|
||||
self._ts[base+head] = now
|
||||
self._ts_head[slot] = (head+1) % self.max_ts
|
||||
if self._ts_len[slot] < self.max_ts: self._ts_len[slot] += 1
|
||||
|
||||
current_rate = now - self._last[slot]
|
||||
rate_blocked = False
|
||||
|
||||
if now > self._blocked[slot]:
|
||||
if current_rate < target: self._viol[slot] += 1
|
||||
else:
|
||||
v = self._viol[slot] - 1
|
||||
self._viol[slot] = v if v > 0 else 0
|
||||
|
||||
if self._viol[slot] > grace:
|
||||
self._blocked[slot] = self._last[slot] + target + penalty
|
||||
rate_blocked = True
|
||||
else: self._last[slot] = now
|
||||
|
||||
else: rate_blocked = True
|
||||
|
||||
return rate_blocked
|
||||
|
||||
# Caller must hold the table lock
|
||||
def cull(self, now, ttl, max_scan=1024, max_cull=None):
|
||||
if max_cull is None: max_cull = max_scan
|
||||
horizon = now - ttl
|
||||
culled = 0
|
||||
scanned = 0
|
||||
n = len(self._keys)
|
||||
|
||||
while scanned < max_scan and culled < max_cull and n > 0:
|
||||
if self._cursor >= n: self._cursor = 0
|
||||
slot = self._cursor
|
||||
|
||||
if self._last[slot] < horizon and self._blocked[slot] <= now:
|
||||
last_i = n - 1
|
||||
stale_key = self._keys[slot]
|
||||
|
||||
if slot != last_i:
|
||||
moved_key = self._keys[last_i]
|
||||
self._keys[slot] = moved_key
|
||||
self._index[moved_key] = slot
|
||||
self._last[slot] = self._last[last_i]
|
||||
self._blocked[slot] = self._blocked[last_i]
|
||||
self._viol[slot] = self._viol[last_i]
|
||||
self._ts_len[slot] = self._ts_len[last_i]
|
||||
self._ts_head[slot] = self._ts_head[last_i]
|
||||
|
||||
f = last_i*self.max_ts; t = slot*self.max_ts
|
||||
self._ts[t:t+self.max_ts] = self._ts[f:f+self.max_ts]
|
||||
|
||||
self._keys.pop()
|
||||
self._last.pop(); self._blocked.pop(); self._viol.pop()
|
||||
self._ts_len.pop(); self._ts_head.pop()
|
||||
del self._ts[len(self._ts)-self.max_ts:] # drop tail block of removed slot
|
||||
del self._index[stale_key]
|
||||
|
||||
n -= 1
|
||||
culled += 1
|
||||
# Do not advance cursor, re-examine the slot a moved entry landed in
|
||||
|
||||
else:
|
||||
self._cursor = slot + 1
|
||||
scanned += 1
|
||||
|
||||
if self._cursor >= n: self._cursor = 0
|
||||
|
||||
return culled
|
||||
|
||||
# Hold table lock, or accept a possibly inconsistent snapshot
|
||||
def items(self):
|
||||
for slot, key in enumerate(self._keys):
|
||||
n = self._ts_len[slot]
|
||||
head = self._ts_head[slot]
|
||||
base = slot*self.max_ts
|
||||
if n < self.max_ts: ts = list(self._ts[base:base+n])
|
||||
else: ts = list(self._ts[base+head:base+self.max_ts]) + list(self._ts[base:base+head])
|
||||
yield key, { "last": self._last[slot], "rate_violations": self._viol[slot],
|
||||
"blocked_until": self._blocked[slot], "timestamps": ts }
|
||||
|
||||
# Persists the announce rate table to a file, in
|
||||
# compacted, little-endian binary format:
|
||||
#
|
||||
# Header (16 bytes)
|
||||
# 0x00 6 Magic bytes
|
||||
# 0x06 1 Format version
|
||||
# 0x07 1 Flags, reserved, must be zero
|
||||
# 0x08 1 Timestamp ring size (max_ts)
|
||||
# 0x09 1 Destination hash length in bytes (keylen)
|
||||
# 0x0A 2 Reserved, zero
|
||||
# 0x0C 4 Entry count, uint32, little-endian
|
||||
#
|
||||
# Entries (columnar, in native slot order)
|
||||
# Destination hash keys, followed by the _last, _blocked,
|
||||
# _viol, _ts_len, _ts_head and _ts columns, all encoded as
|
||||
# little-endian fixed-width values.
|
||||
#
|
||||
# Caller must hold the announce rate table lock
|
||||
|
||||
PERSIST_MAGIC = b"RNSART"
|
||||
PERSIST_VERSION = 0x01
|
||||
def persist(self, path, retain=None):
|
||||
try:
|
||||
count = len(self._keys)
|
||||
|
||||
if count > 0:
|
||||
keylen = len(self._keys[0])
|
||||
for key in self._keys:
|
||||
if len(key) != keylen: raise ValueError("Inconsistent destination hash lengths in table")
|
||||
else:
|
||||
keylen = RNS.Identity.TRUNCATED_HASHLENGTH//8
|
||||
|
||||
if retain != None and count > 0:
|
||||
now = time.time()
|
||||
horizon = now - retain
|
||||
keep = [slot for slot in range(count) if not (self._last[slot] < horizon and self._blocked[slot] <= now)]
|
||||
else:
|
||||
keep = None
|
||||
|
||||
if keep != None and len(keep) != count:
|
||||
count = len(keep)
|
||||
keys = [self._keys[slot] for slot in keep]
|
||||
last = array("d", [self._last[slot] for slot in keep])
|
||||
blocked = array("d", [self._blocked[slot] for slot in keep])
|
||||
viol = array("i", [self._viol[slot] for slot in keep])
|
||||
ts_len = array("B", [self._ts_len[slot] for slot in keep])
|
||||
ts_head = array("B", [self._ts_head[slot] for slot in keep])
|
||||
ts = array("d")
|
||||
for slot in keep: ts.extend(self._ts[slot*self.max_ts:(slot+1)*self.max_ts])
|
||||
else:
|
||||
keys = self._keys
|
||||
last = self._last
|
||||
blocked = self._blocked
|
||||
viol = self._viol
|
||||
ts_len = self._ts_len
|
||||
ts_head = self._ts_head
|
||||
ts = self._ts
|
||||
|
||||
if last.itemsize != 8 or viol.itemsize != 4:
|
||||
raise ValueError(f"Unsupported array itemsize at persist: d={last.itemsize}, i={viol.itemsize}")
|
||||
|
||||
header = self.PERSIST_MAGIC
|
||||
header += bytes([self.PERSIST_VERSION, 0x00, self.max_ts & 0xFF, keylen & 0xFF])
|
||||
header += struct.pack("<H", 0)
|
||||
header += struct.pack("<I", count)
|
||||
|
||||
blob = header
|
||||
blob += b"".join(keys)
|
||||
blob += self._column_le(last)
|
||||
blob += self._column_le(blocked)
|
||||
blob += self._column_le(viol)
|
||||
blob += ts_len.tobytes()
|
||||
blob += ts_head.tobytes()
|
||||
blob += self._column_le(ts)
|
||||
|
||||
tmp_path = path+".tmp"
|
||||
with open(tmp_path, "wb") as file: file.write(blob)
|
||||
os.replace(tmp_path, path)
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Could not persist announce rate table to {path}: {e}", RNS.LOG_ERROR)
|
||||
return False
|
||||
|
||||
# Caller must hold the announce rate table lock
|
||||
def load(self, path):
|
||||
try:
|
||||
with open(path, "rb") as file: blob = file.read()
|
||||
except Exception: return False
|
||||
|
||||
try:
|
||||
if len(blob) < 16: return False
|
||||
if blob[0:6] != self.PERSIST_MAGIC: return False
|
||||
if blob[6] != self.PERSIST_VERSION: return False
|
||||
if blob[7] != 0x00: return False
|
||||
|
||||
file_max_ts = blob[8]
|
||||
keylen = blob[9]
|
||||
count = struct.unpack_from("<I", blob, 12)[0]
|
||||
|
||||
if file_max_ts < 1 or file_max_ts > 255: return False
|
||||
if keylen < 1: return False
|
||||
|
||||
entry_size = keylen+22+file_max_ts*8
|
||||
if len(blob) != 16+count*entry_size: return False
|
||||
|
||||
if file_max_ts == self.max_ts:
|
||||
# Ring size matches, load directly
|
||||
keys = [blob[16+slot*keylen : 16+(slot+1)*keylen] for slot in range(count)]
|
||||
index = {}
|
||||
for slot in range(count):
|
||||
if keys[slot] in index: return False
|
||||
index[keys[slot]] = slot
|
||||
|
||||
offset = 16+count*keylen
|
||||
last = self._column_from_le("d", blob[offset:offset+count*8]); offset += count*8
|
||||
blocked = self._column_from_le("d", blob[offset:offset+count*8]); offset += count*8
|
||||
viol = self._column_from_le("i", blob[offset:offset+count*4]); offset += count*4
|
||||
ts_len = array("B", blob[offset:offset+count]); offset += count
|
||||
ts_head = array("B", blob[offset:offset+count]); offset += count
|
||||
ts = self._column_from_le("d", blob[offset:offset+count*self.max_ts*8]); offset += count*self.max_ts*8
|
||||
|
||||
for slot in range(count):
|
||||
if ts_len[slot] > self.max_ts or ts_head[slot] >= self.max_ts: return False
|
||||
|
||||
else:
|
||||
# Different ring sizes from current running config,
|
||||
# rebuild entries best effort.
|
||||
keys = [blob[16+slot*keylen : 16+(slot+1)*keylen] for slot in range(count)]
|
||||
index = {}
|
||||
for slot in range(count):
|
||||
if keys[slot] in index: return False
|
||||
index[keys[slot]] = slot
|
||||
|
||||
offset = 16+count*keylen
|
||||
last = self._column_from_le("d", blob[offset:offset+count*8]); offset += count*8
|
||||
blocked = self._column_from_le("d", blob[offset:offset+count*8]); offset += count*8
|
||||
viol = self._column_from_le("i", blob[offset:offset+count*4]); offset += count*4
|
||||
f_ts_len = array("B", blob[offset:offset+count]); offset += count
|
||||
f_ts_head = array("B", blob[offset:offset+count]); offset += count
|
||||
f_ts = self._column_from_le("d", blob[offset:offset+count*file_max_ts*8]); offset += count*file_max_ts*8
|
||||
|
||||
for slot in range(count):
|
||||
if f_ts_len[slot] > file_max_ts or f_ts_head[slot] >= file_max_ts: return False
|
||||
|
||||
ts_len = array("B"); ts_head = array("B"); ts = array("d")
|
||||
for slot in range(count):
|
||||
base = slot*file_max_ts
|
||||
if f_ts_len[slot] < file_max_ts: chronological = list(f_ts[base:base+f_ts_len[slot]])
|
||||
else: chronological = list(f_ts[base+f_ts_head[slot]:base+file_max_ts])+list(f_ts[base:base+f_ts_head[slot]])
|
||||
chronological = chronological[-self.max_ts:]
|
||||
ts_count = len(chronological)
|
||||
|
||||
ts_len.append(ts_count)
|
||||
ts.extend(chronological)
|
||||
if ts_count < self.max_ts: ts.extend([0.0]*(self.max_ts-ts_count))
|
||||
ts_head.append(ts_count % self.max_ts)
|
||||
|
||||
self._index = index
|
||||
self._keys = keys
|
||||
self._last = last
|
||||
self._blocked = blocked
|
||||
self._viol = viol
|
||||
self._ts = ts
|
||||
self._ts_len = ts_len
|
||||
self._ts_head = ts_head
|
||||
self._cursor = 0
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Could not load announce rate table from {path}: {e}", RNS.LOG_ERROR)
|
||||
return False
|
||||
|
||||
def _column_le(self, column):
|
||||
if sys.byteorder == "big": swapped = array(column.typecode, column); swapped.byteswap(); return swapped.tobytes()
|
||||
else: return column.tobytes()
|
||||
|
||||
def _column_from_le(self, typecode, data):
|
||||
column = array(typecode); column.frombytes(data)
|
||||
if sys.byteorder == "big": column.byteswap()
|
||||
return column
|
||||
|
||||
class PacketHashlist:
|
||||
# Bucketed byte-blob packet filter hashlist, with the primary
|
||||
# design goal of memory coherence and density. All operations
|
||||
# are safe to call without holding a lock.
|
||||
|
||||
def __init__(self, hash_width=32, bucket_count=16384):
|
||||
assert bucket_count > 0 and bucket_count & (bucket_count-1) == 0, "Bucket count must be a power of two"
|
||||
self.width = hash_width
|
||||
self._nb = bucket_count
|
||||
self._mask = bucket_count - 1
|
||||
self._cur = [bytearray() for _ in range(bucket_count)]
|
||||
self._prv = [bytearray() for _ in range(bucket_count)]
|
||||
self._count = 0
|
||||
|
||||
def __len__(self):
|
||||
return self._count
|
||||
|
||||
def __contains__(self, packet_hash):
|
||||
bucket = (packet_hash[0] | (packet_hash[1] << 8)) & self._mask
|
||||
return packet_hash in self._cur[bucket] or packet_hash in self._prv[bucket]
|
||||
|
||||
def add(self, packet_hash):
|
||||
assert len(packet_hash) == self.width, "Packet hash width mismatch"
|
||||
bucket = (packet_hash[0] | (packet_hash[1] << 8)) & self._mask
|
||||
if packet_hash not in self._cur[bucket]:
|
||||
self._cur[bucket].extend(packet_hash)
|
||||
self._count += 1
|
||||
|
||||
def discard(self, packet_hash):
|
||||
w = self.width
|
||||
bucket = (packet_hash[0] | (packet_hash[1] << 8)) & self._mask
|
||||
for generation in (self._cur, self._prv):
|
||||
entries = generation[bucket]
|
||||
pos = entries.find(packet_hash)
|
||||
while pos != -1:
|
||||
if pos % w == 0:
|
||||
del entries[pos:pos+w]
|
||||
if generation is self._cur: self._count -= 1
|
||||
pos = entries.find(packet_hash, pos)
|
||||
else:
|
||||
pos = entries.find(packet_hash, pos+1)
|
||||
|
||||
def swap(self):
|
||||
self._prv = self._cur
|
||||
self._cur = [bytearray() for _ in range(self._nb)]
|
||||
self._count = 0
|
||||
|
||||
# Current-generation buckets for persist operations
|
||||
def buckets(self): return tuple(self._cur)
|
||||
|
||||
class KnownDestinationsStore:
|
||||
# Write-behind SQLite storage backend for known destinations.
|
||||
# The actual in-memory table is used directly for all transport
|
||||
# operations; this serves purely as the store, and no read operations
|
||||
# utilize the database directly.
|
||||
|
||||
def __init__(self, db_path, hash_width=16, read_only=False):
|
||||
import sqlite3
|
||||
self.sqlite3 = sqlite3
|
||||
self.db_path = db_path
|
||||
self.hash_width = hash_width
|
||||
self.read_only = read_only
|
||||
self._dirty_upsert = set()
|
||||
self._dirty_delete = set()
|
||||
self._ratchet_upserts = {}
|
||||
self._ratchet_deletes = set()
|
||||
self._kd_quarantine = {}
|
||||
self._ratchet_quarantine = {}
|
||||
self._flush_lock = threading.Lock()
|
||||
|
||||
if read_only: self._conn = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True, check_same_thread=False, isolation_level=None, cached_statements=0)
|
||||
else:
|
||||
self._conn = sqlite3.connect(db_path, check_same_thread=False, isolation_level=None, cached_statements=0)
|
||||
self._conn.execute("PRAGMA journal_mode=WAL")
|
||||
self._conn.execute("PRAGMA synchronous=NORMAL")
|
||||
self._conn.execute("PRAGMA journal_size_limit=8388608")
|
||||
self.__create_schema()
|
||||
|
||||
def close(self):
|
||||
if self._conn != None:
|
||||
self._conn.close()
|
||||
self._conn = None
|
||||
|
||||
def __create_schema(self):
|
||||
db_version = self._conn.execute("PRAGMA user_version").fetchone()[0]
|
||||
|
||||
# Pre-release database cleanup
|
||||
# TODO: Remove this in 1.6.1
|
||||
if db_version > 0:
|
||||
RNS.log(f"Pre-release database detected, rebuilding known destinations DB to release format", RNS.LOG_WARNING)
|
||||
self._conn.execute("DROP TABLE IF EXISTS known_destinations")
|
||||
|
||||
self._conn.execute("""CREATE TABLE IF NOT EXISTS known_destinations (
|
||||
destination_hash BLOB PRIMARY KEY,
|
||||
last_heard REAL NOT NULL,
|
||||
packet_hash BLOB,
|
||||
public_key BLOB NOT NULL,
|
||||
app_data BLOB,
|
||||
used REAL NOT NULL DEFAULT 0,
|
||||
stamp BLOB,
|
||||
stamp_value REAL,
|
||||
lr_cost REAL,
|
||||
identity_hash BLOB NOT NULL)""")
|
||||
|
||||
self._conn.execute("""CREATE TABLE IF NOT EXISTS ratchets (
|
||||
destination_hash BLOB PRIMARY KEY,
|
||||
ratchet BLOB NOT NULL,
|
||||
received REAL NOT NULL)""")
|
||||
|
||||
# Schema migrations
|
||||
try:
|
||||
self._conn.execute("BEGIN")
|
||||
# No migrations yet
|
||||
self._conn.execute("PRAGMA user_version = 0")
|
||||
self._conn.execute("COMMIT")
|
||||
self._conn.commit()
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Known destinations database version migration failed: {e}", RNS.LOG_CRITICAL)
|
||||
RNS.log(f"You will need to remove the known_destinations.db* files from your storage directory so it can be rebuilt", RNS.LOG_CRITICAL)
|
||||
|
||||
def __entry_row(self, destination_hash, entry):
|
||||
stamp = entry[5] if len(entry) > 5 else None
|
||||
stamp_value = entry[6] if len(entry) > 6 else 0
|
||||
lr_cost = entry[7] if len(entry) > 7 else None
|
||||
identity_hash = hashlib.sha256(entry[2]).digest()[:self.hash_width]
|
||||
return (destination_hash, entry[0], entry[1], entry[2], entry[3], entry[4], stamp, stamp_value, lr_cost, identity_hash)
|
||||
|
||||
# Caller must hold Identity.known_destinations_lock
|
||||
def mark_dirty(self, destination_hash):
|
||||
if self.read_only: return
|
||||
self._dirty_delete.discard(destination_hash)
|
||||
self._kd_quarantine.pop(destination_hash, None)
|
||||
self._dirty_upsert.add(destination_hash)
|
||||
|
||||
# Caller must hold Identity.known_destinations_lock
|
||||
def mark_deleted(self, destination_hash):
|
||||
if self.read_only: return
|
||||
self._dirty_upsert.discard(destination_hash)
|
||||
self._dirty_delete.add(destination_hash)
|
||||
self._kd_quarantine.pop(destination_hash, None)
|
||||
self._ratchet_quarantine.pop(destination_hash, None)
|
||||
|
||||
# Caller must hold Identity.known_destinations_lock
|
||||
def take_pending(self):
|
||||
if self.read_only: return set(), set(), {}, set()
|
||||
upserts = self._dirty_upsert
|
||||
deletes = self._dirty_delete
|
||||
ratchet_upserts = self._ratchet_upserts
|
||||
ratchet_deletes = self._ratchet_deletes
|
||||
self._dirty_upsert = set()
|
||||
self._dirty_delete = set()
|
||||
self._ratchet_upserts = {}
|
||||
self._ratchet_deletes = set()
|
||||
return upserts, deletes, ratchet_upserts, ratchet_deletes
|
||||
|
||||
# Caller must hold the flush lock and have an already open transaction
|
||||
def __wrap_exec(self, sql, rows):
|
||||
rejects = []
|
||||
|
||||
# By default, run batched queries in chunks.
|
||||
for i in range(0, len(rows), 4096):
|
||||
chunk = rows[i:i+4096]
|
||||
try:
|
||||
self._conn.execute("SAVEPOINT chunk")
|
||||
self._conn.executemany(sql, chunk)
|
||||
self._conn.execute("RELEASE chunk")
|
||||
|
||||
# But on integrity and data errors, roll back the chunk
|
||||
# and iterate per-row, so we can catch any violating
|
||||
# entries and quarantine them. A later, sane update
|
||||
# will then unquarantine and heal upon arrival.
|
||||
#
|
||||
# Other exceptions are intentionally not caught here,
|
||||
# so they can propagate directly.
|
||||
except (self.sqlite3.IntegrityError, self.sqlite3.DataError):
|
||||
self._conn.execute("ROLLBACK TO chunk")
|
||||
for row in chunk:
|
||||
try:
|
||||
self._conn.execute("SAVEPOINT row")
|
||||
self._conn.execute(sql, row)
|
||||
self._conn.execute("RELEASE row")
|
||||
|
||||
except (self.sqlite3.IntegrityError, self.sqlite3.DataError) as e:
|
||||
self._conn.execute("ROLLBACK TO row")
|
||||
rejects.append((row[0], str(e)))
|
||||
|
||||
self._conn.execute("RELEASE chunk")
|
||||
|
||||
return rejects
|
||||
|
||||
# May be called without any locks held
|
||||
def apply(self, upserts, deletes, ratchet_upserts, ratchet_deletes, known_destinations):
|
||||
if self.read_only: return 0
|
||||
if len(upserts) == 0 and len(deletes) == 0 and len(ratchet_upserts) == 0 and len(ratchet_deletes) == 0: return 0
|
||||
|
||||
with self._flush_lock:
|
||||
rows = []
|
||||
removables = []
|
||||
ratchet_rows = []
|
||||
ratchet_removables = []
|
||||
|
||||
for destination_hash in upserts:
|
||||
if destination_hash in self._kd_quarantine: continue
|
||||
entry = known_destinations.get(destination_hash)
|
||||
if entry != None: rows.append(self.__entry_row(destination_hash, entry))
|
||||
|
||||
for destination_hash in deletes:
|
||||
if destination_hash not in known_destinations: removables.append((destination_hash,))
|
||||
|
||||
for destination_hash, ratchet_entry in ratchet_upserts.items():
|
||||
# Ratchets of purged destinations must be removed as well
|
||||
if destination_hash not in known_destinations: ratchet_removables.append((destination_hash,))
|
||||
elif destination_hash not in self._ratchet_quarantine: ratchet_rows.append((destination_hash, *ratchet_entry))
|
||||
|
||||
for destination_hash in ratchet_deletes:
|
||||
if destination_hash not in known_destinations: ratchet_removables.append((destination_hash,))
|
||||
|
||||
kd_rejects = []
|
||||
ratchet_rejects = []
|
||||
try:
|
||||
self._conn.execute("BEGIN")
|
||||
kd_rejects += self.__wrap_exec("INSERT INTO known_destinations (destination_hash, last_heard, packet_hash, public_key, app_data, used, stamp, stamp_value, lr_cost, identity_hash) VALUES (?,?,?,?,?,?,?,?,?,?) ON CONFLICT(destination_hash) DO UPDATE SET last_heard=excluded.last_heard, packet_hash=excluded.packet_hash, public_key=excluded.public_key, app_data=excluded.app_data, used=excluded.used, stamp=excluded.stamp, stamp_value=excluded.stamp_value, lr_cost=excluded.lr_cost, identity_hash=excluded.identity_hash", rows)
|
||||
kd_rejects += self.__wrap_exec("DELETE FROM known_destinations WHERE destination_hash=?", removables)
|
||||
ratchet_rejects += self.__wrap_exec("INSERT INTO ratchets VALUES (?,?,?) ON CONFLICT(destination_hash) DO UPDATE SET ratchet=excluded.ratchet, received=excluded.received", ratchet_rows)
|
||||
ratchet_rejects += self.__wrap_exec("DELETE FROM ratchets WHERE destination_hash=?", ratchet_removables)
|
||||
self._conn.commit()
|
||||
|
||||
except Exception:
|
||||
# If an exception is not handled in the batch query wrapper,
|
||||
# we will have to roll back the transaction, and mark all
|
||||
# attempted entries as dirty. A failure state like such
|
||||
# should not happen, and this is a last resort; at this
|
||||
# point, writes may stall until an updated entry for
|
||||
# the violating row(s) arrive and heal the invalid
|
||||
# dirty entries, so this is raised for logging.
|
||||
self._dirty_upsert |= {row[0] for row in rows}
|
||||
self._dirty_delete |= {row[0] for row in removables}
|
||||
self._ratchet_upserts |= {row[0]: row[1:] for row in ratchet_rows}
|
||||
self._ratchet_deletes |= {row[0] for row in ratchet_removables}
|
||||
self._conn.rollback()
|
||||
raise
|
||||
|
||||
# Track failed persists and one-time log per failure. A later
|
||||
# update of the destination clears the quarantine state and
|
||||
# allows persist to retry with the (hopefully) valid data.
|
||||
if len(kd_rejects) > 0 or len(ratchet_rejects) > 0:
|
||||
for destination_hash, error in kd_rejects:
|
||||
self._kd_quarantine[destination_hash] = (error, time.time())
|
||||
RNS.log(f"Invalid known destination data for {RNS.prettyhexrep(destination_hash)} at persist. The contained exception was: {error}", RNS.LOG_ERROR)
|
||||
|
||||
for destination_hash, error in ratchet_rejects:
|
||||
self._ratchet_quarantine[destination_hash] = (error, time.time())
|
||||
RNS.log(f"Invalid ratchet data for {RNS.prettyhexrep(destination_hash)} at persist. The contained exception was: {error}", RNS.LOG_ERROR)
|
||||
|
||||
return len(rows)+len(removables)+len(ratchet_rows)+len(ratchet_removables)-len(kd_rejects)-len(ratchet_rejects)
|
||||
|
||||
def load_all(self):
|
||||
cursor = self._conn.execute("SELECT destination_hash, last_heard, packet_hash, public_key, app_data, used, stamp, stamp_value, lr_cost FROM known_destinations")
|
||||
while True:
|
||||
rows = cursor.fetchmany(4096)
|
||||
if not rows: break
|
||||
for row in rows: yield row[0], [row[1], row[2], row[3], row[4], row[5], row[6], row[7], row[8]]
|
||||
|
||||
def remember_ratchet(self, destination_hash, ratchet, now):
|
||||
if self.read_only: return False
|
||||
self._ratchet_quarantine.pop(destination_hash, None)
|
||||
|
||||
pending = self._ratchet_upserts.get(destination_hash)
|
||||
if pending != None:
|
||||
if pending[0] == ratchet:
|
||||
self._ratchet_deletes.discard(destination_hash)
|
||||
return False
|
||||
|
||||
else:
|
||||
with self._flush_lock:
|
||||
row = self._conn.execute("SELECT ratchet FROM ratchets WHERE destination_hash=?", (destination_hash,)).fetchone()
|
||||
|
||||
if row != None and row[0] == ratchet:
|
||||
# Discard any pending delete mark queued
|
||||
# before this ratchet arrived
|
||||
self._ratchet_deletes.discard(destination_hash)
|
||||
return False
|
||||
|
||||
self._ratchet_upserts[destination_hash] = (ratchet, now)
|
||||
self._ratchet_deletes.discard(destination_hash)
|
||||
return True
|
||||
|
||||
def fetch_ratchet(self, destination_hash):
|
||||
pending = self._ratchet_upserts.get(destination_hash)
|
||||
if pending != None: return pending[0], pending[1]
|
||||
if destination_hash in self._ratchet_deletes: return None
|
||||
|
||||
with self._flush_lock:
|
||||
row = self._conn.execute("SELECT ratchet, received FROM ratchets WHERE destination_hash=?", (destination_hash,)).fetchone()
|
||||
|
||||
if row != None: return row[0], row[1]
|
||||
|
||||
return None
|
||||
|
||||
# Caller must hold Identity.known_destinations_lock
|
||||
def mark_ratchet_deleted(self, destination_hash):
|
||||
if self.read_only: return
|
||||
self._ratchet_upserts.pop(destination_hash, None)
|
||||
self._ratchet_deletes.add(destination_hash)
|
||||
self._ratchet_quarantine.pop(destination_hash, None)
|
||||
|
||||
def expire_ratchets(self, cutoff):
|
||||
if self.read_only: return []
|
||||
|
||||
with self._flush_lock:
|
||||
# Drop expired entries in pending as well
|
||||
expired = set(row[0] for row in self._conn.execute("SELECT destination_hash FROM ratchets WHERE received<?", (cutoff,)))
|
||||
for destination_hash, entry in list(self._ratchet_upserts.items()):
|
||||
if entry[1] < cutoff:
|
||||
expired.add(destination_hash)
|
||||
self._ratchet_upserts.pop(destination_hash, None)
|
||||
|
||||
expired.difference_update(self._ratchet_deletes)
|
||||
if len(expired) > 0:
|
||||
self._conn.execute("BEGIN")
|
||||
for i in range(0, len(expired), 4096):
|
||||
self._conn.executemany("DELETE FROM ratchets WHERE destination_hash=?", [(h,) for h in list(expired)[i:i+4096]])
|
||||
self._conn.commit()
|
||||
|
||||
return list(expired)
|
||||
|
||||
def load_ratchets(self):
|
||||
cursor = self._conn.execute("SELECT destination_hash, ratchet, received FROM ratchets")
|
||||
while True:
|
||||
rows = cursor.fetchmany(4096)
|
||||
if not rows: break
|
||||
for row in rows: yield row[0], row[1], row[2]
|
||||
|
||||
def import_legacy_ratchets(self, dir_path, expiry_cutoff, known_destinations, ratchet_width=32):
|
||||
from .vendor import umsgpack
|
||||
imported = 0
|
||||
for filename in os.listdir(dir_path):
|
||||
file_path = os.path.join(dir_path, filename)
|
||||
try:
|
||||
if not os.path.isfile(file_path): continue
|
||||
with open(file_path, "rb") as rf:
|
||||
ratchet_data = umsgpack.unpackb(rf.read())
|
||||
|
||||
destination_hash = bytes.fromhex(filename)
|
||||
ratchet = ratchet_data["ratchet"]; received = ratchet_data["received"]
|
||||
if len(ratchet) != ratchet_width: raise ValueError("Invalid ratchet width")
|
||||
if received < expiry_cutoff: raise ValueError("Ratchet expired")
|
||||
if destination_hash not in known_destinations: raise ValueError("Destination unknown")
|
||||
|
||||
self._ratchet_upserts[destination_hash] = (ratchet, received)
|
||||
imported += 1
|
||||
os.unlink(file_path)
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Could not import ratchet for {filename}: {e}", RNS.LOG_WARNING)
|
||||
try: os.unlink(file_path)
|
||||
except: pass
|
||||
|
||||
pending = self._ratchet_upserts
|
||||
self._ratchet_upserts = {}
|
||||
self.apply(set(), set(), pending, set(), known_destinations)
|
||||
|
||||
return imported
|
||||
|
||||
def import_legacy(self, path):
|
||||
from .vendor import umsgpack
|
||||
with open(path, "rb") as file: legacy = umsgpack.load(file)
|
||||
|
||||
rows = []
|
||||
now = time.time()
|
||||
for destination_hash, entry in legacy.items():
|
||||
if len(entry) < 5: continue
|
||||
entry = list(entry)
|
||||
while len(entry) < 8: entry.append(None if len(entry) in (5, 7) else 0)
|
||||
if entry[0] == None or entry[0] <= 0: entry[0] = now
|
||||
rows.append(self.__entry_row(destination_hash, entry))
|
||||
|
||||
self._conn.execute("BEGIN")
|
||||
for i in range(0, len(rows), 4096): self._conn.executemany("INSERT OR REPLACE INTO known_destinations (destination_hash, last_heard, packet_hash, public_key, app_data, used, stamp, stamp_value, lr_cost, identity_hash) VALUES (?,?,?,?,?,?,?,?,?,?)", rows[i:i+4096])
|
||||
self._conn.commit()
|
||||
|
||||
return len(rows)
|
||||
|
||||
class AnnounceCache:
|
||||
|
||||
def __init__(self, db_path):
|
||||
import sqlite3
|
||||
self.db_path = db_path
|
||||
self._pending = {}
|
||||
self._flush_lock = threading.Lock()
|
||||
self._conn = sqlite3.connect(db_path, check_same_thread=False, isolation_level=None, cached_statements=0)
|
||||
|
||||
self._conn.execute("PRAGMA auto_vacuum=INCREMENTAL")
|
||||
self._conn.execute("PRAGMA journal_mode=WAL")
|
||||
self._conn.execute("PRAGMA synchronous=NORMAL")
|
||||
self._conn.execute("PRAGMA journal_size_limit=8388608")
|
||||
self.__create_schema()
|
||||
|
||||
def __create_schema(self):
|
||||
self._conn.execute("""CREATE TABLE IF NOT EXISTS cached_announces (
|
||||
packet_hash BLOB PRIMARY KEY,
|
||||
received REAL NOT NULL,
|
||||
raw BLOB NOT NULL,
|
||||
interface_reference TEXT)""")
|
||||
|
||||
self._conn.execute("PRAGMA user_version = 1")
|
||||
|
||||
def store(self, packet_hash, raw, interface_reference):
|
||||
self._pending[packet_hash] = (time.time(), raw, interface_reference)
|
||||
|
||||
def pending_count(self):
|
||||
return len(self._pending)
|
||||
|
||||
def fetch(self, packet_hash):
|
||||
pending = self._pending.get(packet_hash)
|
||||
if pending != None: return pending[1], pending[2]
|
||||
|
||||
with self._flush_lock:
|
||||
row = self._conn.execute("SELECT raw, interface_reference FROM cached_announces WHERE packet_hash=?", (packet_hash,)).fetchone()
|
||||
|
||||
if row != None: return row[0], row[1]
|
||||
|
||||
return None
|
||||
|
||||
def flush(self):
|
||||
with self._flush_lock: return self.__apply_pending()
|
||||
|
||||
def __apply_pending(self):
|
||||
pending = self._pending
|
||||
self._pending = {}
|
||||
if len(pending) == 0: return 0
|
||||
|
||||
try:
|
||||
self._conn.execute("BEGIN")
|
||||
items = list(pending.items())
|
||||
for i in range(0, len(items), 4096):
|
||||
self._conn.executemany("INSERT INTO cached_announces VALUES (?,?,?,?) ON CONFLICT(packet_hash) DO UPDATE SET received=excluded.received, raw=excluded.raw, interface_reference=excluded.interface_reference",
|
||||
[(packet_hash, *entry) for packet_hash, entry in items[i:i+4096]])
|
||||
self._conn.commit()
|
||||
|
||||
except Exception:
|
||||
# Re-mark everything as pending, so the changes
|
||||
# are retried on the next flush, then propagate.
|
||||
self._pending.update(pending)
|
||||
self._conn.rollback()
|
||||
raise
|
||||
|
||||
return len(pending)
|
||||
|
||||
def clean(self, active_hashes):
|
||||
with self._flush_lock:
|
||||
self.__apply_pending()
|
||||
|
||||
try:
|
||||
self._conn.execute("BEGIN")
|
||||
self._conn.execute("DROP TABLE IF EXISTS temp.active_announces")
|
||||
self._conn.execute("CREATE TEMP TABLE active_announces (packet_hash BLOB PRIMARY KEY)")
|
||||
active_list = list(active_hashes)
|
||||
for i in range(0, len(active_list), 4096):
|
||||
self._conn.executemany("INSERT OR IGNORE INTO active_announces VALUES (?)", [(h,) for h in active_list[i:i+4096]])
|
||||
|
||||
changes_before = self._conn.total_changes
|
||||
self._conn.execute("DELETE FROM cached_announces WHERE packet_hash NOT IN (SELECT packet_hash FROM active_announces)")
|
||||
removed = self._conn.total_changes - changes_before
|
||||
self._conn.commit()
|
||||
|
||||
except Exception:
|
||||
self._conn.rollback()
|
||||
raise
|
||||
|
||||
if removed > 0: self._conn.execute("PRAGMA incremental_vacuum")
|
||||
|
||||
return removed
|
||||
|
||||
def import_legacy(self, dir_path):
|
||||
from .vendor import umsgpack
|
||||
imported = 0
|
||||
for filename in os.listdir(dir_path):
|
||||
file_path = os.path.join(dir_path, filename)
|
||||
try:
|
||||
if not os.path.isfile(file_path): continue
|
||||
with open(file_path, "rb") as file:
|
||||
cached_data = umsgpack.unpackb(file.read())
|
||||
|
||||
packet_hash = bytes.fromhex(filename)
|
||||
interface_reference = cached_data[1] if len(cached_data) > 1 else None
|
||||
self.store(packet_hash, cached_data[0], interface_reference)
|
||||
imported += 1
|
||||
os.unlink(file_path)
|
||||
|
||||
except Exception:
|
||||
try: os.unlink(file_path)
|
||||
except Exception: pass
|
||||
|
||||
self.flush()
|
||||
return imported
|
||||
|
||||
def close(self):
|
||||
if self._conn != None:
|
||||
self._conn.close()
|
||||
self._conn = None
|
||||
5146
RNS/Transport.py
5146
RNS/Transport.py
File diff suppressed because it is too large
Load diff
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -33,5 +33,7 @@ import glob
|
|||
|
||||
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
|
||||
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
|
||||
modules = py_modules+pyc_modules
|
||||
__all__ = list(set([os.path.basename(f).replace(".pyc", "").replace(".py", "") for f in modules if not (f.endswith("__init__.py") or f.endswith("__init__.pyc"))]))
|
||||
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
|
||||
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
|
||||
modules = py_modules+pyc_modules+so_modules+pyd_modules
|
||||
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))
|
||||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -163,11 +163,11 @@ def listen(configdir, identitypath = None, verbosity = 0, quietness = 0, allowed
|
|||
except Exception as e:
|
||||
raise ValueError("Invalid destination entered. Check your input.")
|
||||
except Exception as e:
|
||||
print(str(e))
|
||||
RNS.log(f"Could not apply allowed identity: {e}", RNS.LOG_ERROR)
|
||||
RNS.exit(1)
|
||||
|
||||
if len(allowed_identity_hashes) < 1 and not disable_auth:
|
||||
print("Warning: No allowed identities configured, rncp will not accept any files!")
|
||||
RNS.log("No allowed identities configured, rncp will not accept any files!", RNS.LOG_WARNING)
|
||||
|
||||
def fetch_request(path, data, request_id, link_id, remote_identity, requested_at):
|
||||
global allow_fetch, fetch_jail, fetch_auto_compress
|
||||
|
|
@ -217,7 +217,7 @@ def listen(configdir, identitypath = None, verbosity = 0, quietness = 0, allowed
|
|||
else:
|
||||
destination.register_request_handler("fetch_file", response_generator=fetch_request, allow=RNS.Destination.ALLOW_LIST, allowed_list=allowed_identity_hashes)
|
||||
|
||||
print("rncp listening on "+RNS.prettyhexrep(destination.hash))
|
||||
RNS.log("rncp listening on "+RNS.prettyhexrep(destination.hash), RNS.LOG_INFO)
|
||||
|
||||
if announce >= 0:
|
||||
def job():
|
||||
|
|
@ -271,15 +271,15 @@ def receive_resource_started(resource):
|
|||
else:
|
||||
id_str = ""
|
||||
|
||||
print("Starting resource transfer "+RNS.prettyhexrep(resource.hash)+id_str)
|
||||
RNS.log("Starting resource transfer "+RNS.prettyhexrep(resource.hash)+id_str, RNS.LOG_INFO)
|
||||
|
||||
def receive_resource_concluded(resource):
|
||||
global save_path, allow_overwrite_on_receive
|
||||
if resource.status == RNS.Resource.COMPLETE:
|
||||
print(str(resource)+" completed")
|
||||
RNS.log(f"Incoming resource {resource} completed", RNS.LOG_INFO)
|
||||
|
||||
if resource.metadata == None:
|
||||
print("Invalid data received, ignoring resource")
|
||||
RNS.log("Invalid data received, ignoring resource", RNS.LOG_WARNING)
|
||||
return
|
||||
|
||||
else:
|
||||
|
|
@ -306,13 +306,14 @@ def receive_resource_concluded(resource):
|
|||
full_save_path = saved_filename+"."+str(counter)
|
||||
|
||||
shutil.move(resource.data.name, full_save_path)
|
||||
RNS.log("Saved received file to "+full_save_path, RNS.LOG_NOTICE)
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"An error occurred while saving received resource: {e}", RNS.LOG_ERROR)
|
||||
return
|
||||
|
||||
else:
|
||||
print("Resource failed")
|
||||
RNS.log("Resource failed", RNS.LOG_INFO)
|
||||
|
||||
resource_done = False
|
||||
current_resource = None
|
||||
|
|
@ -386,6 +387,7 @@ def fetch(configdir, identitypath = None, verbosity = 0, quietness = 0, destinat
|
|||
RNS.exit(1)
|
||||
|
||||
reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel)
|
||||
reticulum.wants_destination_type(f"{APP_NAME}.receive")
|
||||
|
||||
if identity == None:
|
||||
prepare_identity(identitypath)
|
||||
|
|
@ -400,7 +402,7 @@ def fetch(configdir, identitypath = None, verbosity = 0, quietness = 0, destinat
|
|||
|
||||
i = 0
|
||||
syms = "⢄⢂⢁⡁⡈⡐⡠"
|
||||
estab_timeout = time.time()+timeout
|
||||
estab_timeout = time.time()+max(timeout, reticulum.get_medium_path_timeout())
|
||||
while not RNS.Transport.has_path(destination_hash) and time.time() < estab_timeout:
|
||||
if not silent:
|
||||
time.sleep(0.1)
|
||||
|
|
@ -430,6 +432,7 @@ def fetch(configdir, identitypath = None, verbosity = 0, quietness = 0, destinat
|
|||
)
|
||||
|
||||
link = RNS.Link(listener_destination)
|
||||
estab_timeout = max(estab_timeout, time.time()+link.establishment_timeout)
|
||||
while link.status != RNS.Link.ACTIVE and time.time() < estab_timeout:
|
||||
if not silent:
|
||||
time.sleep(0.1)
|
||||
|
|
@ -641,6 +644,7 @@ def send(configdir, identitypath = None, verbosity = 0, quietness = 0, destinati
|
|||
print(f"{erase_str}", end="")
|
||||
|
||||
reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel)
|
||||
reticulum.wants_destination_type(f"{APP_NAME}.receive")
|
||||
|
||||
if identity == None:
|
||||
prepare_identity(identitypath)
|
||||
|
|
@ -655,7 +659,7 @@ def send(configdir, identitypath = None, verbosity = 0, quietness = 0, destinati
|
|||
|
||||
i = 0
|
||||
syms = "⢄⢂⢁⡁⡈⡐⡠"
|
||||
estab_timeout = time.time()+timeout
|
||||
estab_timeout = time.time()+max(timeout, reticulum.get_medium_path_timeout())
|
||||
while not RNS.Transport.has_path(destination_hash) and time.time() < estab_timeout:
|
||||
if not silent:
|
||||
time.sleep(0.1)
|
||||
|
|
@ -685,6 +689,7 @@ def send(configdir, identitypath = None, verbosity = 0, quietness = 0, destinati
|
|||
)
|
||||
|
||||
link = RNS.Link(receiver_destination)
|
||||
estab_timeout = max(estab_timeout, time.time()+link.establishment_timeout)
|
||||
while link.status != RNS.Link.ACTIVE and time.time() < estab_timeout:
|
||||
if not silent:
|
||||
time.sleep(0.1)
|
||||
|
|
@ -877,8 +882,8 @@ def main():
|
|||
|
||||
except KeyboardInterrupt:
|
||||
print("")
|
||||
if resource != None:
|
||||
resource.cancel()
|
||||
if current_resource != None:
|
||||
current_resource.cancel()
|
||||
if link != None:
|
||||
link.teardown()
|
||||
RNS.exit()
|
||||
|
|
|
|||
41
RNS/Utilities/rngit/__init__.py
Normal file
41
RNS/Utilities/rngit/__init__.py
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
APP_NAME = "git"
|
||||
|
||||
import os
|
||||
import glob
|
||||
|
||||
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
|
||||
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
|
||||
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
|
||||
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
|
||||
modules = py_modules+pyc_modules+so_modules+pyd_modules
|
||||
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))
|
||||
720
RNS/Utilities/rngit/client.py
Normal file
720
RNS/Utilities/rngit/client.py
Normal file
|
|
@ -0,0 +1,720 @@
|
|||
#!/usr/bin/env python3
|
||||
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
import RNS
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import shutil
|
||||
import threading
|
||||
import subprocess
|
||||
|
||||
from RNS._version import __version__
|
||||
from RNS.Utilities.rngit import APP_NAME
|
||||
from RNS.Utilities.rngit.util import medium_path_timeout
|
||||
|
||||
from RNS.vendor.configobj import ConfigObj
|
||||
from tempfile import TemporaryDirectory
|
||||
|
||||
def program_setup(configdir, rnsconfigdir, destination_hexhash, group_name, repo_name):
|
||||
git_client = ReticulumGitClient(configdir=configdir, rnsconfigdir=rnsconfigdir, destination_hexhash=destination_hexhash,
|
||||
group_name=group_name, repo_name=repo_name)
|
||||
|
||||
if not git_client.ready: sys.exit(1)
|
||||
else: git_client.run()
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 3:
|
||||
print("Usage: git-remote-rns <remote-name> <url>", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
url = sys.argv[2]
|
||||
if not url.startswith("rns://"):
|
||||
print("Invalid URL scheme. Must be rns://", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
try:
|
||||
parts = url[6:].split("/", 2)
|
||||
destination_hexhash = parts[0]
|
||||
group_name = parts[1]
|
||||
repo_name = parts[2]
|
||||
|
||||
except IndexError: print("Invalid URL format. Use rns://<hash>/<group>/<repo>", file=sys.stderr); sys.exit(1)
|
||||
|
||||
configdir = os.environ.get("RNGIT_CONFIG", None)
|
||||
rnsconfigdir = os.environ.get("RNS_CONFIG", None)
|
||||
|
||||
program_setup(configdir, rnsconfigdir, destination_hexhash, group_name, repo_name)
|
||||
exit(0)
|
||||
|
||||
|
||||
class ReticulumGitClient():
|
||||
PATH_LIST = "/git/list"
|
||||
PATH_FETCH = "/git/fetch"
|
||||
PATH_PUSH = "/git/push"
|
||||
PATH_DELETE = "/git/delete"
|
||||
|
||||
RES_DISALLOWED = 0x01
|
||||
RES_INVALID_REQ = 0x02
|
||||
RES_NOT_FOUND = 0x03
|
||||
RES_REMOTE_FAIL = 0xFF
|
||||
|
||||
IDX_REPOSITORY = 0x00
|
||||
IDX_RESULT_CODE = 0x01
|
||||
|
||||
REF_BATCH_SIZE = 25
|
||||
PATH_TIMEOUT = 15
|
||||
LINK_TIMEOUT = 15
|
||||
|
||||
def __init__(self, configdir, rnsconfigdir, destination_hexhash, group_name, repo_name):
|
||||
# Client state and configuration
|
||||
self.identity = None
|
||||
self.userdir = os.path.expanduser("~")
|
||||
self.config = None
|
||||
self.ready = False
|
||||
|
||||
self.destination_aliases = {}
|
||||
self.remote_identity = None
|
||||
self.destination = None
|
||||
self.link = None
|
||||
self.link_ready = False
|
||||
self.link_failed = False
|
||||
self.link_timeout = self.LINK_TIMEOUT
|
||||
self.path_timeout = self.PATH_TIMEOUT
|
||||
|
||||
self.destination_hexhash = destination_hexhash
|
||||
self.group_name = group_name
|
||||
self.repo_name = repo_name
|
||||
self.repo_path = f"{group_name}/{repo_name}"
|
||||
|
||||
self.tmp_dir = TemporaryDirectory()
|
||||
self.request_event = threading.Event()
|
||||
self.request_response = None
|
||||
self.response_metadata = None
|
||||
|
||||
self.ref_batch_size = self.REF_BATCH_SIZE
|
||||
self.remote_refs = {}
|
||||
|
||||
self.response_progress = 0
|
||||
self.previous_progress = 0
|
||||
self.response_size = None
|
||||
self.response_transfer_size = None
|
||||
self.progress_updated_at = None
|
||||
self.progress_enabled = False
|
||||
|
||||
if configdir != None: self.configdir = configdir
|
||||
else:
|
||||
if os.path.isdir(self.userdir+"/.config/rngit") and os.path.isfile(self.userdir+"/.config/rngit/config"): self.configdir = self.userdir+"/.rngit/reticulum"
|
||||
else: self.configdir = self.userdir+"/.rngit"
|
||||
|
||||
self.logfile = self.configdir+"/client_log"
|
||||
self.configpath = self.configdir+"/client_config"
|
||||
self.identitypath = self.configdir+"/client_identity"
|
||||
|
||||
if os.path.isfile(self.configpath):
|
||||
try: self.config = ConfigObj(self.configpath)
|
||||
except Exception as e:
|
||||
RNS.log("Could not parse the configuration at "+self.configpath, RNS.LOG_ERROR)
|
||||
return
|
||||
|
||||
else: self.__create_default_config()
|
||||
|
||||
RNS.logfile = self.logfile
|
||||
try: self.reticulum = RNS.Reticulum(configdir=rnsconfigdir, logdest=RNS.LOG_FILE)
|
||||
except Exception as e:
|
||||
print(f"Failed to initialize Reticulum: {e}", file=sys.stderr)
|
||||
return
|
||||
|
||||
self.reticulum.wants_destination_type(f"{APP_NAME}.repositories")
|
||||
|
||||
self.__apply_config()
|
||||
self.ready = True
|
||||
|
||||
def __create_default_config(self):
|
||||
self.config = ConfigObj(__default_rngit_config__)
|
||||
self.config.filename = self.configpath
|
||||
if not os.path.isdir(self.configdir): os.makedirs(self.configdir)
|
||||
self.config.write()
|
||||
|
||||
def __apply_config(self):
|
||||
if "logging" in self.config:
|
||||
section = self.config["logging"]
|
||||
if "loglevel" in section: RNS.loglevel = max(RNS.LOG_NONE, min(RNS.LOG_EXTREME, section.as_int("loglevel")))
|
||||
|
||||
if "client" in self.config:
|
||||
section = self.config["client"]
|
||||
if "ref_batch_size" in section: self.ref_batch_size = max(0, min(1024, section.as_int("ref_batch_size")))
|
||||
|
||||
if "aliases" in self.config:
|
||||
section = self.config["aliases"]
|
||||
for alias in section:
|
||||
alias_hexhash = section[alias]
|
||||
len_ok = len(alias_hexhash) == RNS.Identity.TRUNCATED_HASHLENGTH//8*2
|
||||
try: alias_hash = bytes.fromhex(alias_hexhash)
|
||||
except: alias_hash = None
|
||||
alias_exists = alias in self.destination_aliases
|
||||
if not len_ok or not alias_hash: continue
|
||||
if alias_exists: continue
|
||||
self.destination_aliases[alias] = RNS.hexrep(alias_hash, delimit=False)
|
||||
|
||||
if not os.path.isfile(self.identitypath):
|
||||
identity = RNS.Identity()
|
||||
identity.to_file(self.identitypath)
|
||||
RNS.log(f"Client identity generated and persisted to {self.identitypath}", RNS.LOG_VERBOSE)
|
||||
|
||||
else:
|
||||
identity = RNS.Identity.from_file(self.identitypath)
|
||||
RNS.log(f"Client identity loaded from {self.identitypath}", RNS.LOG_VERBOSE)
|
||||
|
||||
if not identity:
|
||||
RNS.log("Could not initialize client identity.", RNS.LOG_ERROR)
|
||||
self.ready = False
|
||||
|
||||
else: self.identity = identity
|
||||
|
||||
self.destination_hexhash = self.__resolve_destination_alias(self.destination_hexhash)
|
||||
|
||||
def __resolve_destination_alias(self, alias):
|
||||
def resolve(alias):
|
||||
len_match = len(alias) == RNS.Identity.TRUNCATED_HASHLENGTH//8*2
|
||||
try: hash_bytes = bytes.fromhex(alias)
|
||||
except: hash_bytes = None
|
||||
if len_match and hash_bytes: return alias
|
||||
else: return self.destination_aliases[alias] if alias in self.destination_aliases else alias
|
||||
|
||||
resolved = resolve(alias)
|
||||
return resolved
|
||||
|
||||
def abort(self, reason=None, code=255):
|
||||
if not reason: reason = "Unknown reason"
|
||||
print(f"git-remote-rns failed: {reason}", file=sys.stderr)
|
||||
if self.link: self.link.teardown()
|
||||
sys.exit(code)
|
||||
|
||||
def connect_server(self):
|
||||
try: destination_hash = bytes.fromhex(self.destination_hexhash)
|
||||
except Exception as e: self.abort(f"Invalid destination hash: {e}")
|
||||
|
||||
RNS.log(f"Requesting path to {RNS.prettyhexrep(destination_hash)}", RNS.LOG_DEBUG)
|
||||
sys.stderr.write(f"Requesting path..."); sys.stderr.flush()
|
||||
if not RNS.Transport.await_path(destination_hash, timeout=medium_path_timeout(self.path_timeout)):
|
||||
sys.stderr.write(f"\n"); sys.stderr.flush()
|
||||
self.abort(f"Could not resolve path to {RNS.prettyhexrep(destination_hash)}")
|
||||
|
||||
else:
|
||||
RNS.log(f"Path to {RNS.prettyhexrep(destination_hash)} resolved", RNS.LOG_DEBUG);
|
||||
sys.stderr.write(f"\rPath resolved "); sys.stderr.flush()
|
||||
|
||||
self.remote_identity = RNS.Identity.recall(destination_hash)
|
||||
if not self.remote_identity: self.abort("Could not recall remote identity. Is the server announcing?")
|
||||
|
||||
sys.stderr.write(f"\rEstablishing link..."); sys.stderr.flush()
|
||||
self.destination = RNS.Destination(self.remote_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "repositories")
|
||||
self.link = RNS.Link(self.destination)
|
||||
self.link.set_link_established_callback(self.link_established)
|
||||
self.link.set_link_closed_callback(self.link_closed)
|
||||
|
||||
def link_established(self, link):
|
||||
RNS.log(f"Link established, identifying...", RNS.LOG_DEBUG)
|
||||
sys.stderr.write(f"\rLink established with remote\n"); sys.stderr.flush()
|
||||
link.identify(self.identity)
|
||||
self.link_ready = True
|
||||
|
||||
def link_closed(self, link):
|
||||
RNS.log(f"Link was closed", RNS.LOG_DEBUG)
|
||||
if not self.link_ready: self.link_failed = True
|
||||
|
||||
def _on_progress(self, transfer_instance):
|
||||
if hasattr(transfer_instance, "progress"):
|
||||
self.response_progress = transfer_instance.progress
|
||||
self.response_size = transfer_instance.response_size
|
||||
self.response_transfer_size = transfer_instance.response_transfer_size
|
||||
|
||||
elif hasattr(transfer_instance, "get_progress") and callable(transfer_instance.get_progress):
|
||||
self.response_progress = transfer_instance.get_progress()
|
||||
self.response_size = transfer_instance.total_size
|
||||
self.response_transfer_size = transfer_instance.size
|
||||
|
||||
now = time.time()
|
||||
if self.progress_updated_at == None: self.progress_updated_at = now
|
||||
|
||||
if now > self.progress_updated_at+1:
|
||||
td = now - self.progress_updated_at
|
||||
pd = self.response_progress - self.previous_progress
|
||||
bd = pd*self.response_size if self.response_size else 0
|
||||
self.response_speed = (bd/td)*8 if td > 0 else 0
|
||||
self.previous_progress = self.response_progress
|
||||
self.progress_updated_at = now
|
||||
|
||||
# Report progress to git via stderr
|
||||
if self.progress_enabled and self.response_size:
|
||||
percent = round(self.response_progress * 100, 1)
|
||||
size = self.response_size
|
||||
rxd = size*self.response_progress
|
||||
speed_kbps = (self.response_speed / 1000) if hasattr(self, 'response_speed') else 0
|
||||
sys.stderr.write(f"Transferring: {percent}% ({RNS.prettysize(rxd)}/{RNS.prettysize(size)}) {RNS.prettyspeed(self.response_speed)} \r")
|
||||
sys.stderr.flush()
|
||||
|
||||
################################
|
||||
# Synchronous Request Wrappers #
|
||||
################################
|
||||
|
||||
def _response_ready(self, request_receipt):
|
||||
self.request_response = request_receipt.response
|
||||
self.response_metadata = request_receipt.metadata
|
||||
|
||||
if hasattr(self.request_response, "read") and callable(self.request_response.read):
|
||||
response_path = self.request_response.name
|
||||
try: self.request_response.close()
|
||||
except Exception as e: RNS.log(f"Could not close resource file descriptor before retain: {e}", RNS.LOG_ERROR)
|
||||
base_name = os.path.basename(response_path)
|
||||
retained_path = os.path.join(self.tmp_dir.name, base_name)
|
||||
shutil.move(response_path, retained_path)
|
||||
self.request_response = open(retained_path, "rb")
|
||||
|
||||
self.request_event.set()
|
||||
|
||||
def _response_failed(self, request_receipt=None):
|
||||
self.request_response = None
|
||||
self.request_event.set()
|
||||
|
||||
def send_request(self, path, data, timeout=7200):
|
||||
if not self.link_ready: self.abort("Link not ready for request")
|
||||
|
||||
self.request_event.clear()
|
||||
self.request_response = None
|
||||
self.response_metadata = None
|
||||
self.previous_progress = 0
|
||||
self.progress_updated_at = None
|
||||
|
||||
RNS.log(f"Sending request: {path}", RNS.LOG_DEBUG)
|
||||
request_receipt = self.link.request(path, data, progress_callback=self._on_progress, response_callback=self._response_ready, failed_callback=self._response_failed, timeout=timeout)
|
||||
if request_receipt.resource: request_receipt.resource.progress_callback(self._on_progress)
|
||||
self.request_event.wait(timeout=timeout)
|
||||
|
||||
if self.request_response is None: self.abort("Request failed or timed out")
|
||||
RNS.log(f"Got response for: {path}", RNS.LOG_DEBUG)
|
||||
|
||||
return self.request_response, self.response_metadata
|
||||
|
||||
#############################
|
||||
# Git Helper Protocol Logic #
|
||||
#############################
|
||||
|
||||
def _detach_stdout(self):
|
||||
sys.stdout = open(os.devnull, "w")
|
||||
sys.stderr = open(os.devnull, "w")
|
||||
|
||||
def run(self):
|
||||
try: self.connect_server()
|
||||
except Exception as e: self.abort(str(e))
|
||||
|
||||
timeout = max(self.link_timeout, self.link.establishment_timeout)
|
||||
while not self.link_ready and not self.link_failed and timeout > 0:
|
||||
time.sleep(0.5)
|
||||
timeout -= 0.5
|
||||
|
||||
if not self.link_ready: self.abort("Failed to establish link")
|
||||
|
||||
self.progress_enabled = False
|
||||
|
||||
git_stdin = sys.stdin
|
||||
git_stdout = sys.stdout
|
||||
git_stderr = sys.stderr
|
||||
|
||||
fetch_queue = []
|
||||
push_queue = []
|
||||
|
||||
while True:
|
||||
line = git_stdin.readline()
|
||||
if not line: break
|
||||
|
||||
line = line.strip()
|
||||
if line == "capabilities":
|
||||
git_stdout.write("list\n")
|
||||
git_stdout.write("fetch\n")
|
||||
git_stdout.write("push\n")
|
||||
git_stdout.write("option\n")
|
||||
git_stdout.write("\n")
|
||||
git_stdout.flush()
|
||||
|
||||
elif line == "list": self.handle_git_list(git_stdout)
|
||||
|
||||
elif line.startswith("list "): self.handle_git_list(git_stdout, for_push=True) # List for push
|
||||
|
||||
elif line.startswith("option"):
|
||||
# Line format: option <name> <value>
|
||||
parts = line.split(maxsplit=2)
|
||||
opt_name = parts[1] if len(parts) > 1 else ""
|
||||
opt_value = parts[2] if len(parts) > 2 else ""
|
||||
|
||||
if opt_name == "progress": self.progress_enabled = opt_value.lower() in ("true", "1", "yes"); git_stdout.write("ok\n")
|
||||
else: git_stdout.write("unsupported\n")
|
||||
|
||||
git_stdout.flush()
|
||||
|
||||
elif line.startswith("fetch"):
|
||||
# Line format: fetch <sha> <ref>
|
||||
parts = line.split()
|
||||
sha = parts[1]
|
||||
ref = parts[2]
|
||||
# Avoid duplicates in the same batch - TODO: Re-evaluate this
|
||||
if (sha, ref) not in fetch_queue: fetch_queue.append((sha, ref))
|
||||
push_queue = []
|
||||
|
||||
elif line.startswith("push"):
|
||||
# Line format: push <local_ref>:<remote_ref>
|
||||
parts = line.split()
|
||||
refspec = parts[1]
|
||||
local_ref, remote_ref = refspec.split(":", 1)
|
||||
push_queue.append((local_ref, remote_ref))
|
||||
fetch_queue = []
|
||||
|
||||
elif line == "": # End of batch
|
||||
try:
|
||||
self.process_fetch_queue(fetch_queue, git_stdout, self.progress_enabled, self.ref_batch_size)
|
||||
self.process_push_queue(push_queue, git_stdout, git_stderr, self.progress_enabled)
|
||||
fetch_queue = []
|
||||
push_queue = []
|
||||
git_stdout.write("\n")
|
||||
git_stdout.flush()
|
||||
|
||||
except BrokenPipeError:
|
||||
self._detach_stdout()
|
||||
RNS.log("Git closed connection, exiting", RNS.LOG_DEBUG)
|
||||
break
|
||||
|
||||
else: self.abort(f"Unknown Git command: {line}")
|
||||
|
||||
try: sys.stdout.flush()
|
||||
except BrokenPipeError: pass
|
||||
|
||||
if self.link: self.link.teardown()
|
||||
|
||||
def handle_git_list(self, git_stdout, for_push=False):
|
||||
RNS.log("Handle git list" + (" for-push" if for_push else ""), RNS.LOG_DEBUG)
|
||||
request_data = {self.IDX_REPOSITORY: self.repo_path, "for_push": for_push}
|
||||
response, metadata = self.send_request(self.PATH_LIST, request_data)
|
||||
|
||||
if not response or not isinstance(response, bytes): self.abort("Invalid list response from server")
|
||||
|
||||
status_byte = response[0]
|
||||
payload = response[1:]
|
||||
|
||||
if status_byte != 0: self.abort(f"Server refused list: {payload.decode('utf-8', errors='ignore')}")
|
||||
|
||||
response_text = payload.decode("utf-8")
|
||||
|
||||
self.remote_refs = {}
|
||||
for line in response_text.split("\n"):
|
||||
line = line.strip()
|
||||
if not line: continue
|
||||
parts = line.split(" ", 1)
|
||||
if len(parts) == 2:
|
||||
sha, ref_name = parts
|
||||
if ref_name == "HEAD": continue
|
||||
self.remote_refs[ref_name] = sha
|
||||
|
||||
git_stdout.write(response_text)
|
||||
git_stdout.write("\n") # Required to terminate list
|
||||
git_stdout.flush()
|
||||
|
||||
def escape_for_stdout(self, value):
|
||||
if isinstance(value, bytes): value = value.decode('utf-8', errors='replace')
|
||||
|
||||
escaped = '"'
|
||||
for char in value:
|
||||
if char == '\\': escaped += '\\\\'
|
||||
elif char == '"': escaped += '\\"'
|
||||
elif char == '\n': escaped += '\\n'
|
||||
elif char == '\t': escaped += '\\t'
|
||||
elif char == '\r': escaped += '\\r'
|
||||
elif ord(char) < 32 or ord(char) > 126: escaped += f'\\x{ord(char):02x}'
|
||||
else: escaped += char
|
||||
|
||||
return escaped + '"'
|
||||
|
||||
def process_fetch_queue(self, fetch_queue, git_stdout, progress_enabled=False, ref_batch_size=REF_BATCH_SIZE):
|
||||
import tempfile
|
||||
import subprocess
|
||||
|
||||
if not fetch_queue: return
|
||||
|
||||
# Build a global have list from all remote refs that the client already has objects for
|
||||
have_shas = []
|
||||
for sha in self.remote_refs.values():
|
||||
try:
|
||||
result = subprocess.run(["git", "cat-file", "-t", sha], capture_output=True, check=False)
|
||||
if result.returncode == 0: have_shas.append(sha)
|
||||
|
||||
except Exception as e: RNS.log(f"Could not verify remote SHA {sha} locally: {e}", RNS.LOG_WARNING)
|
||||
|
||||
while fetch_queue:
|
||||
batch = fetch_queue[:ref_batch_size]
|
||||
fetch_queue = fetch_queue[ref_batch_size:]
|
||||
|
||||
refs_list = []
|
||||
for sha, ref in batch:
|
||||
ref_entry = {"sha": sha, "ref": ref}
|
||||
try:
|
||||
# Attempt to get local ref SHA for incremental bundle generation on remote
|
||||
result = subprocess.run(["git", "rev-parse", ref], capture_output=True, text=True, check=False)
|
||||
if result.returncode == 0:
|
||||
local_sha = result.stdout.strip()
|
||||
if local_sha != sha: ref_entry["have"] = local_sha
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Could not resolve local SHA for {ref} during fetch enumeration, getting full history for this ref: {e}", RNS.LOG_WARNING)
|
||||
|
||||
refs_list.append(ref_entry)
|
||||
|
||||
ref_names = [ref for _, ref in batch]
|
||||
RNS.log(f"Fetching batch of {len(refs_list)} refs: {ref_names} (have {len(have_shas)} common objects)", RNS.LOG_DEBUG)
|
||||
|
||||
request_data = { self.IDX_REPOSITORY: self.repo_path, "refs": refs_list }
|
||||
if have_shas: request_data["have"] = have_shas
|
||||
|
||||
response, metadata = self.send_request(self.PATH_FETCH, request_data)
|
||||
|
||||
if not response: self.abort(f"No data in fetch response for batch")
|
||||
if not metadata:
|
||||
if not isinstance(response, bytes): self.abort(f"Invalid fetch response for batch")
|
||||
status_byte = response[0]
|
||||
|
||||
if status_byte == 0:
|
||||
RNS.log(f"Server returned empty bundle, all objects already exist locally", RNS.LOG_DEBUG)
|
||||
continue
|
||||
|
||||
else:
|
||||
error_msg = response[1:].decode('utf-8', errors='ignore')
|
||||
self.abort(f"Fetch failed for batch: {error_msg}")
|
||||
|
||||
else:
|
||||
if not self.IDX_RESULT_CODE in metadata: self.abort(f"No result metadata on bundle response")
|
||||
status_byte = metadata[self.IDX_RESULT_CODE]
|
||||
if status_byte == 0: bundle_path = response.name
|
||||
else: self.abort(f"Unknown remote state for batch ref fetch")
|
||||
|
||||
if progress_enabled:
|
||||
size = os.stat(bundle_path).st_size
|
||||
sys.stderr.write(f"Transferring: 100% ({RNS.prettysize(size)}). \n")
|
||||
sys.stderr.flush()
|
||||
|
||||
stderr_arg = sys.stderr if progress_enabled else subprocess.DEVNULL
|
||||
|
||||
verify_cmd = ["git", "bundle", "verify", "-q", bundle_path]
|
||||
verify_result = subprocess.run(verify_cmd, stderr=subprocess.DEVNULL, stdout=subprocess.DEVNULL)
|
||||
|
||||
if verify_result.returncode != 0: self.abort(f"Bundle verification failed for batch")
|
||||
|
||||
unbundle_cmd = ["git", "bundle", "unbundle"]
|
||||
if progress_enabled: unbundle_cmd.append("--progress")
|
||||
unbundle_cmd.append(bundle_path)
|
||||
|
||||
unbundle_result = subprocess.run(unbundle_cmd, stderr=stderr_arg, stdout=subprocess.DEVNULL)
|
||||
|
||||
if unbundle_result.returncode != 0: self.abort(f"Bundle unbundle failed for batch: Non-zero return code")
|
||||
|
||||
def process_push_queue(self, push_queue, git_stdout, git_stderr, progress_enabled=False):
|
||||
import tempfile
|
||||
import subprocess
|
||||
|
||||
for local_ref, remote_ref in push_queue:
|
||||
RNS.log(f"Pushing {local_ref} to {remote_ref}", RNS.LOG_DEBUG)
|
||||
|
||||
# Handle potential deletions
|
||||
if not local_ref or local_ref == "":
|
||||
request_data = { self.IDX_REPOSITORY: self.repo_path, "ref": remote_ref }
|
||||
response, metadata = self.send_request(self.PATH_DELETE, request_data)
|
||||
|
||||
if not response or not isinstance(response, bytes):
|
||||
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout('No response from server')}\n")
|
||||
git_stdout.flush()
|
||||
continue
|
||||
|
||||
status_byte = response[0]
|
||||
if status_byte != 0:
|
||||
error_msg = response[1:].decode("utf-8", errors="ignore")
|
||||
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout(error_msg)}\n")
|
||||
git_stdout.flush()
|
||||
continue
|
||||
|
||||
git_stdout.write(f"ok {remote_ref}\n")
|
||||
git_stdout.flush()
|
||||
continue
|
||||
|
||||
force = local_ref.startswith("+")
|
||||
if force: local_ref = local_ref[1:]
|
||||
|
||||
stderr_arg = sys.stderr if progress_enabled else subprocess.DEVNULL
|
||||
|
||||
# Resolve the SHA that local_ref points to
|
||||
sha_result = subprocess.run(["git", "rev-parse", local_ref], capture_output=True, text=True, check=False)
|
||||
if sha_result.returncode != 0:
|
||||
error_msg = f"Could not resolve local ref {local_ref}"
|
||||
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout(error_msg)}\n")
|
||||
git_stdout.flush()
|
||||
continue
|
||||
|
||||
local_sha = sha_result.stdout.strip()
|
||||
|
||||
bundle_empty = False
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
bundle_path = tmpdir + "/push.bundle"
|
||||
|
||||
create_cmd = ["git", "bundle", "create", bundle_path, local_ref]
|
||||
|
||||
# Exclude all remote ref SHAs that exist locally, so the
|
||||
# bundle only contains objects the remote doesn't already have
|
||||
exclude_count = 0
|
||||
for sha in self.remote_refs.values():
|
||||
try:
|
||||
# We need to verify each SHA actually exists locally, since git
|
||||
# bundle create will fail if a ^<sha> argument references an object
|
||||
# not present in the local repository.
|
||||
result = subprocess.run(["git", "cat-file", "-t", sha], capture_output=True, check=False)
|
||||
if result.returncode == 0:
|
||||
create_cmd.append(f"^{sha}")
|
||||
exclude_count += 1
|
||||
|
||||
except Exception as e: RNS.log(f"Could not verify remote SHA {sha} locally: {e}", RNS.LOG_WARNING)
|
||||
|
||||
RNS.log(f"Excluding {exclude_count}/{len(self.remote_refs)} remote refs for {local_ref}", RNS.LOG_DEBUG)
|
||||
|
||||
if progress_enabled: create_cmd.insert(3, "--progress")
|
||||
|
||||
create_result = subprocess.run(create_cmd, capture_output=True, text=True, check=False)
|
||||
|
||||
if create_result.returncode == 0:
|
||||
if create_result.stderr:
|
||||
# git_stderr.write(create_result.stderr)
|
||||
pass
|
||||
else:
|
||||
if "empty bundle" in create_result.stderr.lower():
|
||||
# All objects reachable from local_ref already exist on
|
||||
# the remote. In this case, no bundle is needed and we can
|
||||
# update the ref directly via the operations path instead.
|
||||
bundle_empty = True
|
||||
RNS.log(f"Empty bundle for {local_ref}, all objects already on remote", RNS.LOG_DEBUG)
|
||||
|
||||
else:
|
||||
if progress_enabled and create_result.stderr: git_stderr.write(create_result.stderr)
|
||||
error_msg = "Bundle creation failed"
|
||||
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout(error_msg)}\n")
|
||||
git_stdout.flush()
|
||||
continue
|
||||
|
||||
if not bundle_empty:
|
||||
with open(bundle_path, "rb") as f: bundle_data = f.read()
|
||||
|
||||
request_data = { self.IDX_REPOSITORY: self.repo_path, "local_ref": local_ref, "remote_ref": remote_ref,
|
||||
"force": force, "bundle": bundle_data }
|
||||
|
||||
response, metadata = self.send_request(self.PATH_PUSH, request_data)
|
||||
|
||||
if not response or not isinstance(response, bytes):
|
||||
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout('No response from server')}\n")
|
||||
git_stdout.flush()
|
||||
continue
|
||||
|
||||
status_byte = response[0]
|
||||
if status_byte != 0:
|
||||
error_msg = response[1:].decode('utf-8', errors='ignore')
|
||||
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout(error_msg)}\n")
|
||||
git_stdout.flush()
|
||||
continue
|
||||
|
||||
# When all reachable objects already exist on the remote, send a
|
||||
# direct ref update operation instead of a bundle.
|
||||
if bundle_empty:
|
||||
operation = {"action": "update_ref", "ref": remote_ref, "sha": local_sha, "force": force}
|
||||
request_data = { self.IDX_REPOSITORY: self.repo_path,
|
||||
"operations": [operation] }
|
||||
|
||||
response, metadata = self.send_request(self.PATH_PUSH, request_data)
|
||||
|
||||
if not response or not isinstance(response, bytes):
|
||||
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout('No response from server')}\n")
|
||||
git_stdout.flush()
|
||||
continue
|
||||
|
||||
status_byte = response[0]
|
||||
if status_byte != 0:
|
||||
error_msg = response[1:].decode('utf-8', errors='ignore')
|
||||
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout(error_msg)}\n")
|
||||
git_stdout.flush()
|
||||
continue
|
||||
|
||||
git_stdout.write(f"ok {remote_ref}\n")
|
||||
git_stdout.flush()
|
||||
|
||||
|
||||
__default_rngit_config__ = '''# This is the default rngit client config file.
|
||||
|
||||
[client]
|
||||
|
||||
# You can control the batch size of ref transfers
|
||||
# using the ref_batch_size directive:
|
||||
|
||||
ref_batch_size = 25
|
||||
|
||||
|
||||
[aliases]
|
||||
|
||||
# You can define aliases for commonly used destination
|
||||
# hashes in this section. Each line must be in the format
|
||||
# aliased_name = DESTINATION_HASH
|
||||
#
|
||||
# These hashes are used for resolving remote destinations.
|
||||
# For rngit node permissions and identity resolution,
|
||||
# aliases must be defined in ~/.rngit/config.
|
||||
|
||||
# my_node = 063d38912bffc850af4a1b8a270a9d85
|
||||
# bobs_node = 714981d03e41deda0e4468cb274414cc
|
||||
|
||||
|
||||
[logging]
|
||||
# Valid log levels are 0 through 7:
|
||||
# 0: Log only critical information
|
||||
# 1: Log errors and lower log levels
|
||||
# 2: Log warnings and lower log levels
|
||||
# 3: Log notices and lower log levels
|
||||
# 4: Log info and lower (this is the default)
|
||||
# 5: Verbose logging
|
||||
# 6: Debug logging
|
||||
# 7: Extreme logging
|
||||
|
||||
loglevel = 4
|
||||
|
||||
'''.splitlines()
|
||||
|
||||
if __name__ == "__main__": main()
|
||||
324
RNS/Utilities/rngit/commitsigs.py
Executable file
324
RNS/Utilities/rngit/commitsigs.py
Executable file
|
|
@ -0,0 +1,324 @@
|
|||
#!/usr/bin/env python3
|
||||
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
import os
|
||||
import sys
|
||||
import RNS
|
||||
import struct
|
||||
import base64
|
||||
import argparse
|
||||
|
||||
from RNS.Utilities.rnid import validate_rsg, create_rsg, extract_signed_rsg_data
|
||||
|
||||
SSHSIG_MAGIC = b"SSHSIG"
|
||||
SSHSIG_VERSION = 1
|
||||
NAMESPACE_GIT = b"git"
|
||||
RESERVED_EMPTY = b""
|
||||
HASH_ALGORITHM = b"sha256"
|
||||
|
||||
def ssh_string(data): return struct.pack(">I", len(data)) + data
|
||||
|
||||
def read_ssh_string(data, offset):
|
||||
if offset + 4 > len(data): raise ValueError("Not enough data for string length")
|
||||
length = struct.unpack(">I", data[offset:offset+4])[0]
|
||||
if offset + 4 + length > len(data): raise ValueError("Not enough data for string content")
|
||||
return data[offset+4:offset+4+length], offset + 4 + length
|
||||
|
||||
def create_ssh_signature(public_key_wire, namespace, reserved, hash_algorithm, signature_data):
|
||||
# SSHSIG (6 bytes) || version (uint32) || pubkey (ssh-string) || namespace (ssh-string) ||
|
||||
# reserved (ssh-string) || hash_algorithm (ssh-string) || signature (ssh-string)
|
||||
sig_blob = SSHSIG_MAGIC
|
||||
sig_blob += struct.pack(">I", SSHSIG_VERSION)
|
||||
sig_blob += ssh_string(public_key_wire)
|
||||
sig_blob += ssh_string(namespace)
|
||||
sig_blob += ssh_string(reserved)
|
||||
sig_blob += ssh_string(hash_algorithm)
|
||||
sig_blob += ssh_string(signature_data)
|
||||
return sig_blob
|
||||
|
||||
def parse_ssh_signature(sig_data):
|
||||
offset = 0
|
||||
|
||||
if not sig_data.startswith(SSHSIG_MAGIC): raise ValueError("Invalid SSH signature: missing SSHSIG magic")
|
||||
offset += len(SSHSIG_MAGIC)
|
||||
|
||||
if offset + 4 > len(sig_data): raise ValueError("Invalid SSH signature: truncated")
|
||||
version = struct.unpack(">I", sig_data[offset:offset+4])[0]
|
||||
if version != SSHSIG_VERSION: raise ValueError(f"Unsupported SSH signature version: {version}")
|
||||
offset += 4
|
||||
|
||||
public_key, offset = read_ssh_string(sig_data, offset)
|
||||
namespace, offset = read_ssh_string(sig_data, offset)
|
||||
reserved, offset = read_ssh_string(sig_data, offset)
|
||||
hash_algorithm, offset = read_ssh_string(sig_data, offset)
|
||||
signature_data, offset = read_ssh_string(sig_data, offset)
|
||||
|
||||
return { "version": version,
|
||||
"public_key": public_key,
|
||||
"namespace": namespace,
|
||||
"reserved": reserved,
|
||||
"hash_algorithm": hash_algorithm,
|
||||
"signature_data": signature_data }
|
||||
|
||||
def armor_ssh_signature(sig_blob):
|
||||
b64_data = base64.b64encode(sig_blob).decode('ascii')
|
||||
lines = [b64_data[i:i+70] for i in range(0, len(b64_data), 70)]
|
||||
|
||||
result = "-----BEGIN SSH SIGNATURE-----\n"
|
||||
result += "\n".join(lines) + "\n"
|
||||
result += "-----END SSH SIGNATURE-----\n"
|
||||
return result
|
||||
|
||||
def unarmor_ssh_signature(armored_data):
|
||||
lines = armored_data.strip().split('\n')
|
||||
b64_data = ""
|
||||
in_sig = False
|
||||
|
||||
for line in lines:
|
||||
if 'BEGIN SSH SIGNATURE' in line: in_sig = True; continue
|
||||
if 'END SSH SIGNATURE' in line: break
|
||||
if in_sig: b64_data += line.strip()
|
||||
|
||||
if not b64_data: raise ValueError("No signature data found in armored input")
|
||||
|
||||
return base64.b64decode(b64_data)
|
||||
|
||||
def get_pubkey_wire_format(identity):
|
||||
return ssh_string(b"ssh-ed25519")+ssh_string(identity.sig_pub_bytes)
|
||||
|
||||
def sign(args):
|
||||
keyfile = args.keyfile
|
||||
if not keyfile or not os.path.isfile(keyfile):
|
||||
print(f"Identity file not found: {keyfile}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
try:
|
||||
identity = RNS.Identity.from_file(keyfile)
|
||||
if not identity or not identity.get_private_key():
|
||||
print("Error: Could not load identity or identity has no private key", file=sys.stderr)
|
||||
return 1
|
||||
except Exception as e:
|
||||
print(f"Error loading identity: {e}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
if args.file and os.path.isfile(args.file):
|
||||
with open(args.file, 'rb') as f: message = f.read()
|
||||
sig_file = args.file + ".sig"
|
||||
else:
|
||||
message = sys.stdin.buffer.read()
|
||||
sig_file = None
|
||||
|
||||
try: rsg = create_rsg(identity, message)
|
||||
except Exception as e:
|
||||
print(f"Error creating signature: {e}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
try: ssh_pubkey = get_pubkey_wire_format(identity)
|
||||
except Exception as e:
|
||||
print(f"Error converting public key: {e}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
try:
|
||||
ssh_sig = create_ssh_signature(public_key_wire=ssh_pubkey, namespace=NAMESPACE_GIT, reserved=RESERVED_EMPTY,
|
||||
hash_algorithm=HASH_ALGORITHM, signature_data=rsg)
|
||||
except Exception as e:
|
||||
print(f"Error creating SSH signature: {e}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
try: armored = armor_ssh_signature(ssh_sig)
|
||||
except Exception as e:
|
||||
print(f"Error armoring signature: {e}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
if sig_file:
|
||||
try:
|
||||
with open(sig_file, 'w') as f: f.write(armored)
|
||||
except Exception as e:
|
||||
print(f"Error writing signature file: {e}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
else: print(armored, end="")
|
||||
|
||||
return 0
|
||||
|
||||
def find_principals(args):
|
||||
sigfile = args.sigfile
|
||||
if not sigfile or not os.path.isfile(sigfile): print("Error: Signature file not found", file=sys.stderr); return 1
|
||||
|
||||
try:
|
||||
with open(sigfile, 'r') as f: armored_sig = f.read()
|
||||
except Exception as e: print(f"Error reading signature file: {e}", file=sys.stderr); return 1
|
||||
|
||||
try: ssh_sig = parse_ssh_signature(unarmor_ssh_signature(armored_sig))
|
||||
except Exception as e: print(f"Error parsing SSH signature: {e}", file=sys.stderr); return 1
|
||||
|
||||
if ssh_sig["namespace"] != NAMESPACE_GIT:
|
||||
print(f"Error: Namespace mismatch: {ssh_sig['namespace']}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
rsg = ssh_sig["signature_data"]
|
||||
try: identity_hash = extract_signed_rsg_data(rsg)["meta"]["signer"]
|
||||
except Exception as e: print(f"Could not determine signer identity: {e}", file=sys.stderr); return 1
|
||||
|
||||
print(RNS.hexrep(identity_hash, delimit=False))
|
||||
return 0
|
||||
|
||||
def check_novalidate(args):
|
||||
sigfile = args.sigfile
|
||||
if not sigfile or not os.path.isfile(sigfile): return 1
|
||||
|
||||
try:
|
||||
with open(sigfile, 'r') as f: armored_sig = f.read()
|
||||
ssh_sig = parse_ssh_signature(unarmor_ssh_signature(armored_sig))
|
||||
|
||||
if ssh_sig["namespace"] != NAMESPACE_GIT: return 1
|
||||
|
||||
rsg = ssh_sig["signature_data"]
|
||||
signed_data = extract_signed_rsg_data(rsg)
|
||||
if not signed_data: return 1
|
||||
else: return 0
|
||||
|
||||
except Exception: return 1
|
||||
|
||||
def extract_commit_author(message):
|
||||
message_lines = message.splitlines()
|
||||
author = ""
|
||||
AUTHOR_TARGET = b"author "
|
||||
for line in message_lines:
|
||||
if not line.strip(b""): break
|
||||
elif line.startswith(AUTHOR_TARGET):
|
||||
try:
|
||||
spos = line.find(b"<"); epos = line.find(b">")
|
||||
if spos > len(AUTHOR_TARGET) and epos > spos and epos < len(line)-1:
|
||||
author = line[spos+1:epos].decode("utf-8")
|
||||
break
|
||||
except Exception as e: print(f"Error while determining author from signed commit"); return 1
|
||||
|
||||
return author
|
||||
|
||||
def extract_commit_committer(message):
|
||||
message_lines = message.splitlines()
|
||||
committer = ""
|
||||
COMMITTER_TARGET = b"committer "
|
||||
for line in message_lines:
|
||||
if not line.strip(b""): break
|
||||
elif line.startswith(COMMITTER_TARGET):
|
||||
try:
|
||||
spos = line.find(b"<"); epos = line.find(b">")
|
||||
if spos > len(COMMITTER_TARGET) and epos > spos and epos < len(line)-1:
|
||||
committer = line[spos+1:epos].decode("utf-8")
|
||||
break
|
||||
except Exception as e: print(f"Error while determining committer from signed commit"); return 1
|
||||
|
||||
return committer
|
||||
|
||||
def extract_commit_tagger(message):
|
||||
message_lines = message.splitlines()
|
||||
tagger = ""
|
||||
is_tag = False
|
||||
for line in message_lines:
|
||||
TAG_TARGET = b"tag "
|
||||
TAGGER_TARGET = b"tagger "
|
||||
if not line.strip(b""): break
|
||||
elif line.startswith(TAG_TARGET): is_tag = True
|
||||
elif line.startswith(TAGGER_TARGET) and is_tag:
|
||||
try:
|
||||
spos = line.find(b"<"); epos = line.find(b">")
|
||||
if spos > len(TAGGER_TARGET) and epos > spos and epos < len(line)-1:
|
||||
tagger = line[spos+1:epos].decode("utf-8")
|
||||
break
|
||||
except Exception as e: print(f"Error while determining tagger from signed commit"); return 1
|
||||
|
||||
return tagger, is_tag
|
||||
|
||||
def verify(args):
|
||||
sigfile = args.sigfile
|
||||
principal = args.principal
|
||||
if not sigfile or not os.path.isfile(sigfile): print("Error: Signature file not found", file=sys.stderr); return 1
|
||||
|
||||
message = sys.stdin.buffer.read()
|
||||
|
||||
try:
|
||||
with open(sigfile, 'r') as f: armored_sig = f.read()
|
||||
raw_sig = unarmor_ssh_signature(armored_sig)
|
||||
ssh_sig = parse_ssh_signature(raw_sig)
|
||||
|
||||
except Exception as e: print(f"Error parsing signature: {e}", file=sys.stderr); return 1
|
||||
|
||||
author = extract_commit_author(message)
|
||||
committer = extract_commit_committer(message)
|
||||
tagger, is_tag = extract_commit_tagger(message)
|
||||
|
||||
if ssh_sig["namespace"] != NAMESPACE_GIT: print(f"Invalid commit signature namespace", file=sys.stderr); return 1
|
||||
|
||||
rsg = ssh_sig["signature_data"]
|
||||
valid, signed_data, signing_identity = validate_rsg(rsg, message)
|
||||
|
||||
if not valid: print(f"Invalid signature", file=sys.stderr); return 1
|
||||
|
||||
if is_tag: author = tagger
|
||||
|
||||
signer_hash = RNS.hexrep(signing_identity.hash, delimit=False)
|
||||
if not author == signer_hash: print(f"Commit not signed by author <{author}>"); return 1
|
||||
|
||||
if principal:
|
||||
if principal != signer_hash: print(f"Principal mismatch", file=sys.stderr); return 1
|
||||
|
||||
print(f"Good \"git\" signature for commit, signed with Reticulum Identity key <{signer_hash}>")
|
||||
return 0
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Git commit signer and validator")
|
||||
parser.add_argument("-Y", dest="op", required=True, choices=["sign", "find-principals", "check-novalidate", "verify"], help="Operation to perform")
|
||||
parser.add_argument("-n", dest="namespace", default="git", help="Namespace")
|
||||
parser.add_argument("-f", dest="keyfile", help="Key file (for signing) or allowed signers file (for verification)")
|
||||
parser.add_argument("-I", dest="principal", help="Principal identity (for verification)")
|
||||
parser.add_argument("-s", dest="sigfile", help="Signature file")
|
||||
parser.add_argument("file", nargs="?", help="File to sign (for signing)")
|
||||
parser.add_argument("-O", dest="ssh_options", action="append", default=[], help="SSH options (for git compatibility, ignored)")
|
||||
|
||||
args, unknown = parser.parse_known_args()
|
||||
for arg in unknown:
|
||||
if arg.startswith('-O'): continue # TODO: Add options for time validation
|
||||
else:
|
||||
print(f"Error: Unknown argument: {arg}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if args.op == "sign": return sign(args)
|
||||
elif args.op == "find-principals": return find_principals(args)
|
||||
elif args.op == "check-novalidate": return check_novalidate(args)
|
||||
elif args.op == "verify": return verify(args)
|
||||
else:
|
||||
print(f"Error: Unknown operation: {args.op}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
if __name__ == "__main__": sys.exit(main())
|
||||
412
RNS/Utilities/rngit/highlight.py
Normal file
412
RNS/Utilities/rngit/highlight.py
Normal file
|
|
@ -0,0 +1,412 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
import os
|
||||
import io
|
||||
import RNS
|
||||
|
||||
class SyntaxHighlighter:
|
||||
|
||||
def __init__(self, theme=None):
|
||||
self.pygments_available = False
|
||||
self.pygments = None
|
||||
self._lexer_cache = {}
|
||||
self._check_pygments()
|
||||
self.theme = theme or self._get_default_theme()
|
||||
|
||||
def _get_default_theme(self):
|
||||
return {
|
||||
# Control flow - warm coral-red
|
||||
"keyword": "ff7b72",
|
||||
"keyword_constant": "ff7b72",
|
||||
"keyword_control": "ff7b72",
|
||||
"keyword_declaration": "ff7b72",
|
||||
|
||||
# Function definitions - bright sky blue
|
||||
"function_def": "79c0ff",
|
||||
"function_magic": "ff7b72",
|
||||
|
||||
# Function calls - soft lavender
|
||||
"function_call": "d2a8ff",
|
||||
"function_builtin": "ffa657", # amber
|
||||
|
||||
# Class definitions - fresh mint green
|
||||
"class_def": "7ee787",
|
||||
"class_ref": "56d364", # muted when referenced
|
||||
|
||||
# Instance context - soft pink
|
||||
"self": "ff9bce",
|
||||
"cls": "ff9bce",
|
||||
|
||||
# Data literals - cool, calm ice blue
|
||||
"string": "a5d6ff",
|
||||
"string_quoted": "a5d6ff",
|
||||
"string_doc": "8b949e", # docstrings - like comments
|
||||
"string_interpol": "ffd700", # f-string braces - gold
|
||||
"string_escape": "ffea00", # escape sequences - bright yellow
|
||||
|
||||
# Numbers - same as function def
|
||||
"number": "79c0ff",
|
||||
"number_float": "79c0ff",
|
||||
"number_integer": "79c0ff",
|
||||
"number_hex": "79c0ff",
|
||||
|
||||
# Comments - muted gray
|
||||
"comment": "8b949e",
|
||||
"comment_doc": "8b949e",
|
||||
"comment_preproc": "ff7b72", # preprocessor directives
|
||||
|
||||
# Operators - distinct pink/red for visibility
|
||||
"operator": "ff7b72", # General operators - coral
|
||||
"operator_arithmetic": "ff7b72", # +, -, *, /, etc.
|
||||
"operator_comparison": "ff7b72", # ==, !=, <, >, etc.
|
||||
"operator_assignment": "ff7b72", # =, +=, -=, etc.
|
||||
"operator_word": "ff7b72", # and, or, not, in, is
|
||||
"operator_dot": "c9d1d9", # . - subtle for attribute access
|
||||
|
||||
# Punctuation - neutral
|
||||
"punctuation": "b4b4b4",
|
||||
"punctuation_brace": "b4b4b4", # [, ], {, }
|
||||
"punctuation_paren": "b4b4b4", # (, )
|
||||
"punctuation_colon": "b4b4b4", # :, ;
|
||||
"punctuation_comma": "8b949e", # , - slightly dimmed
|
||||
|
||||
# Decorators - burnt orange
|
||||
"decorator": "f0883e",
|
||||
|
||||
# Constants - same as keywords
|
||||
"constant": "ff7b72",
|
||||
"constant_builtin": "ff7b72", # True, False, None
|
||||
|
||||
# Type hints and annotations - amber
|
||||
"type_hint": "ffa657",
|
||||
"type_builtin": "ffa657",
|
||||
|
||||
# Exception handling - alert red
|
||||
"exception": "f85149",
|
||||
"exception_builtin": "f85149",
|
||||
|
||||
# Names and attributes - near-white for readability
|
||||
"name": "e6edf3",
|
||||
"attribute": "e6edf3",
|
||||
"attribute_call": "d2a8ff", # Function/method calls after dot - lavender
|
||||
"variable": "e6edf3",
|
||||
"parameter": "e6edf3",
|
||||
|
||||
# Namespaces and modules
|
||||
"namespace": "7ee787",
|
||||
"module": "a5d6ff",
|
||||
|
||||
# Generic tokens
|
||||
"generic_heading": "c9d1d9",
|
||||
"generic_subheading": "c9d1d9",
|
||||
"generic_prompt": "8b949e",
|
||||
"generic_error": "f85149",
|
||||
"generic_deleted": "f85149",
|
||||
"generic_inserted": "7ee787",
|
||||
"generic_output": "e6edf3",
|
||||
|
||||
# Text and whitespace - no color (None means no color tag)
|
||||
"text": None,
|
||||
"whitespace": None,
|
||||
}
|
||||
|
||||
def _check_pygments(self):
|
||||
try:
|
||||
import pygments
|
||||
from pygments.lexers import get_lexer_for_filename, guess_lexer, get_lexer_by_name
|
||||
from pygments.formatter import Formatter
|
||||
from pygments.token import Token
|
||||
|
||||
self.pygments = pygments
|
||||
self.pygments_available = True
|
||||
RNS.log("Pygments syntax highlighting available", RNS.LOG_DEBUG)
|
||||
|
||||
except ImportError:
|
||||
self.pygments_available = False
|
||||
RNS.log("Pygments not available, using plain text rendering", RNS.LOG_DEBUG)
|
||||
|
||||
def highlight(self, content, filename=None, language=None):
|
||||
if not content: return self._plain_text(content)
|
||||
|
||||
if self.pygments_available:
|
||||
try:
|
||||
highlighted = self._highlight_pygments(content, filename, language)
|
||||
# Fix pygments insisting on trailing newlines
|
||||
if highlighted.endswith("\n") and not content.endswith("\n"): highlighted = highlighted[:-1]
|
||||
return highlighted
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Pygments highlighting failed, falling back: {e}", RNS.LOG_WARNING)
|
||||
return self._plain_text(content).replace("\\", "\\\\")
|
||||
|
||||
# TODO: Implement Python tokenize fallback for .py files.
|
||||
# For now, route to plain text
|
||||
if filename and filename.endswith(".py"):
|
||||
return self._plain_text(content).replace("\\", "\\\\")
|
||||
|
||||
# Universal fallback
|
||||
return self._plain_text(content).replace("\\", "\\\\")
|
||||
|
||||
def _highlight_pygments(self, content, filename=None, language=None):
|
||||
from pygments.lexers import get_lexer_for_filename, guess_lexer, get_lexer_by_name
|
||||
from pygments.util import ClassNotFound
|
||||
|
||||
lexer = None
|
||||
if language:
|
||||
if language == "env": language = "bash"
|
||||
if language == "environment": language = "bash"
|
||||
try: lexer = get_lexer_by_name(language)
|
||||
except ClassNotFound: pass
|
||||
|
||||
if lexer is None and filename:
|
||||
try: lexer = get_lexer_for_filename(filename)
|
||||
except ClassNotFound: pass
|
||||
|
||||
if lexer is None:
|
||||
try:
|
||||
if len(content) > 20: lexer = guess_lexer(content)
|
||||
except ClassNotFound: pass
|
||||
|
||||
if lexer is None: return self._plain_text(content)
|
||||
|
||||
formatter = MicronFormatter(theme=self.theme)
|
||||
result = self.pygments.highlight(content, lexer, formatter)
|
||||
return result
|
||||
|
||||
def _plain_text(self, content):
|
||||
escaped = self._escape_micron(content)
|
||||
return f"`=\n{escaped}\n`="
|
||||
|
||||
@staticmethod
|
||||
def _escape_micron(text): return text.replace("`", "\\`")
|
||||
|
||||
|
||||
class MicronFormatter:
|
||||
def __init__(self, theme, **options):
|
||||
self.theme = theme
|
||||
self.options = options
|
||||
|
||||
def format(self, tokensource, outfile):
|
||||
output_parts = []
|
||||
prev_was_dot = False
|
||||
|
||||
last_ended_with_break = True
|
||||
for ttype, value in tokensource:
|
||||
is_dot = (str(ttype) == "Token.Operator" and value == ".")
|
||||
ends_with_break = value.endswith("\n")
|
||||
|
||||
# If previous token was a dot and this is a Name, treat as attribute/function call
|
||||
# TODO: Improve this if we can check next token as parantheses or something.
|
||||
if prev_was_dot and str(ttype).startswith("Token.Name") and value:
|
||||
color = self._get_color_from_key("attribute_call")
|
||||
if color:
|
||||
escaped = self._escape_value(value)
|
||||
output_parts.append(f"`FT{color}{escaped}`f")
|
||||
else:
|
||||
output_parts.append(self._escape_value(value))
|
||||
|
||||
else:
|
||||
color_key = self._get_color_key_for_token(ttype)
|
||||
color = self._get_color_from_key(color_key)
|
||||
|
||||
if color and value:
|
||||
escaped = self._escape_value(value)
|
||||
if escaped.startswith("\n"): ilb = "\n"; escaped = escaped[1:]
|
||||
else: ilb = ""
|
||||
if escaped.endswith("\n"): tlb = "\n"; escaped = escaped[:-1]
|
||||
else: tlb = ""
|
||||
|
||||
if len(escaped): output = f"{ilb}`FT{color}{escaped}`f{tlb}"
|
||||
else: output = f"{ilb}{tlb}"
|
||||
|
||||
output_parts.append(output)
|
||||
|
||||
else:
|
||||
escaped = self._escape_value(value)
|
||||
if "\n" in escaped:
|
||||
parts = []
|
||||
splitl = escaped.splitlines()
|
||||
if len(splitl) > 1:
|
||||
for line in splitl:
|
||||
if line.startswith("-"): l = f"\\{line}"
|
||||
elif line.startswith(">"): l = f"\\{line}"
|
||||
elif line.startswith("<"): l = f"\\{line}"
|
||||
else: l = line
|
||||
parts.append(l)
|
||||
trmpart = "\n" if escaped.endswith("\n") else ""
|
||||
escaped = "\n".join(parts)+trmpart
|
||||
|
||||
elif last_ended_with_break:
|
||||
if escaped.startswith("-"): escaped = f"\\{escaped}"
|
||||
elif escaped.startswith(">"): escaped = f"\\{escaped}"
|
||||
elif escaped.startswith("<"): escaped = f"\\{escaped}"
|
||||
|
||||
output_parts.append(escaped)
|
||||
|
||||
prev_was_dot = is_dot
|
||||
last_ended_with_break = ends_with_break
|
||||
|
||||
output = "".join(output_parts)
|
||||
outfile.write(output)
|
||||
|
||||
def _get_color_key_for_token(self, ttype):
|
||||
token_parts = []
|
||||
current = ttype
|
||||
while current:
|
||||
token_parts.insert(0, current[0] if isinstance(current, tuple) else str(current).split(".")[-1])
|
||||
current = current.parent if hasattr(current, "parent") else None
|
||||
|
||||
token_str = ".".join(["Token"] + token_parts[1:] if len(token_parts) > 1 else token_parts)
|
||||
|
||||
current_type = ttype
|
||||
while current_type:
|
||||
token_key = str(current_type)
|
||||
if token_key in granular_token_map: return granular_token_map[token_key]
|
||||
|
||||
# Move to parent
|
||||
current_type = current_type.parent if hasattr(current_type, "parent") else None
|
||||
|
||||
return None
|
||||
|
||||
def _get_color_from_key(self, color_key):
|
||||
if color_key and color_key in self.theme: return self.theme[color_key]
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _escape_value(value):
|
||||
return value.replace("\\", "\\\\").replace("`", "\\`")
|
||||
|
||||
# Required by Pygments formatter API, returns None for Micron
|
||||
def get_style_defs(self, arg=None): return None
|
||||
|
||||
|
||||
# Convenience function for direct use
|
||||
def highlight_code(content: str, filename: str = None, language: str = None, theme=None) -> str:
|
||||
highlighter = SyntaxHighlighter(theme=theme)
|
||||
return highlighter.highlight(content, filename, language)
|
||||
|
||||
granular_token_map = {
|
||||
# Keywords with semantic distinction
|
||||
"Token.Keyword": "keyword",
|
||||
"Token.Keyword.Constant": "keyword_constant",
|
||||
"Token.Keyword.Declaration": "keyword_declaration",
|
||||
"Token.Keyword.Namespace": "keyword_control",
|
||||
"Token.Keyword.Pseudo": "keyword_control",
|
||||
"Token.Keyword.Reserved": "keyword_control",
|
||||
"Token.Keyword.Type": "type_builtin",
|
||||
|
||||
# Names - functions with definition vs call distinction
|
||||
"Token.Name.Function": "function_call",
|
||||
"Token.Name.Function.Magic": "function_magic",
|
||||
"Token.Name.Class": "class_ref",
|
||||
"Token.Name.Builtin": "function_builtin",
|
||||
"Token.Name.Builtin.Pseudo": "constant_builtin",
|
||||
"Token.Name.Exception": "exception_builtin",
|
||||
"Token.Name.Decorator": "decorator",
|
||||
"Token.Name.Namespace": "namespace",
|
||||
"Token.Name.Attribute": "attribute",
|
||||
"Token.Name.Variable": "variable",
|
||||
"Token.Name.Variable.Magic": "function_magic",
|
||||
"Token.Name.Other": "name",
|
||||
"Token.Name": "name",
|
||||
"Token.Name.Tag": "keyword", # HTML/XML tags
|
||||
"Token.Name.Constant": "constant",
|
||||
"Token.Name.Label": "name",
|
||||
"Token.Name.Entity": "name",
|
||||
|
||||
# Literals - strings with detailed handling
|
||||
"Token.Literal.String": "string",
|
||||
"Token.Literal.String.Affix": "string", # f, r, b prefixes
|
||||
"Token.Literal.String.Backtick": "string",
|
||||
"Token.Literal.String.Char": "string",
|
||||
"Token.Literal.String.Delimiter": "string",
|
||||
"Token.Literal.String.Doc": "string_doc",
|
||||
"Token.Literal.String.Double": "string_quoted",
|
||||
"Token.Literal.String.Escape": "string_escape",
|
||||
"Token.Literal.String.Heredoc": "string",
|
||||
"Token.Literal.String.Interpol": "string_interpol",
|
||||
"Token.Literal.String.Other": "string",
|
||||
"Token.Literal.String.Regex": "string",
|
||||
"Token.Literal.String.Single": "string_quoted",
|
||||
"Token.Literal.String.Symbol": "string",
|
||||
|
||||
# Numbers
|
||||
"Token.Literal.Number": "number",
|
||||
"Token.Literal.Number.Bin": "number",
|
||||
"Token.Literal.Number.Float": "number_float",
|
||||
"Token.Literal.Number.Hex": "number_hex",
|
||||
"Token.Literal.Number.Integer": "number_integer",
|
||||
"Token.Literal.Number.Integer.Long": "number_integer",
|
||||
"Token.Literal.Number.Oct": "number",
|
||||
"Token.Literal": "string",
|
||||
"Token.Literal.Date": "string",
|
||||
|
||||
# Operators - all operators get distinct coloring
|
||||
"Token.Operator": "operator",
|
||||
"Token.Operator.Word": "operator_word",
|
||||
"Token.Operator.Comparison": "operator_comparison",
|
||||
"Token.Operator.Assignment": "operator_assignment",
|
||||
"Token.Operator.Arithmetic": "operator_arithmetic",
|
||||
|
||||
# Punctuation - braces, parens, colons, commas
|
||||
"Token.Punctuation": "punctuation",
|
||||
"Token.Punctuation.Marker": "punctuation",
|
||||
"Token.Punctuation.Brace": "punctuation_brace",
|
||||
"Token.Punctuation.Bracket": "punctuation_brace",
|
||||
"Token.Punctuation.Parenthesis": "punctuation_paren",
|
||||
"Token.Punctuation.Colon": "punctuation_colon",
|
||||
"Token.Punctuation.Comma": "punctuation_comma",
|
||||
|
||||
# Comments
|
||||
"Token.Comment": "comment",
|
||||
"Token.Comment.Hashbang": "comment",
|
||||
"Token.Comment.Multiline": "comment_doc",
|
||||
"Token.Comment.Preproc": "comment_preproc",
|
||||
"Token.Comment.Single": "comment",
|
||||
"Token.Comment.Special": "comment",
|
||||
|
||||
# Generic tokens
|
||||
"Token.Generic.Deleted": "generic_deleted",
|
||||
"Token.Generic.Emph": "text",
|
||||
"Token.Generic.Error": "generic_error",
|
||||
"Token.Generic.Heading": "generic_heading",
|
||||
"Token.Generic.Inserted": "generic_inserted",
|
||||
"Token.Generic.Output": "generic_output",
|
||||
"Token.Generic.Prompt": "generic_prompt",
|
||||
"Token.Generic.Strong": "text",
|
||||
"Token.Generic.Subheading": "generic_subheading",
|
||||
"Token.Generic.Traceback": "generic_error",
|
||||
"Token.Generic": "text",
|
||||
|
||||
# Text and whitespace
|
||||
"Token.Text": "text",
|
||||
"Token.Text.Whitespace": "whitespace",
|
||||
}
|
||||
40
RNS/Utilities/rngit/main.py
Normal file
40
RNS/Utilities/rngit/main.py
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
import sys
|
||||
from RNS.Utilities.rngit import client, server
|
||||
|
||||
if __name__ == "__main__":
|
||||
cmd = sys.argv[0]
|
||||
if cmd == "rngit": ec = server.main()
|
||||
elif cmd == "git-remote-rns": ec = client.main()
|
||||
else: raise NotImplementedError(f"The {cmd} executable entrypoint is not yet implemented in rngit")
|
||||
|
||||
sys.exit(ec)
|
||||
377
RNS/Utilities/rngit/media.py
Normal file
377
RNS/Utilities/rngit/media.py
Normal file
|
|
@ -0,0 +1,377 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
import os
|
||||
import time
|
||||
import shutil
|
||||
import selectors
|
||||
import subprocess
|
||||
import tempfile
|
||||
import RNS
|
||||
|
||||
CONVERSION_TIMEOUT = 8
|
||||
|
||||
# WebP encoding backends, in preference order. All supported
|
||||
# encoder families handle the common still formats (PNG, JPEG,
|
||||
# TIFF, BMP, GIF, WebP).
|
||||
#
|
||||
# Environment overrides:
|
||||
# RNGIT_MEDIA_BACKEND force a specific backend
|
||||
|
||||
BACKENDS = ( ("magick", ["magick", "-", "webp:-"]),
|
||||
("convert", ["convert", "-", "webp:-"]),
|
||||
("gm", ["gm", "convert", "-", "webp:-"]),
|
||||
("ffmpeg", ["ffmpeg", "-y", "-loglevel", "error", "-i", "-", "-f", "webp", "pipe:1"]),
|
||||
("avconv", ["avconv", "-y", "-loglevel", "error", "-i", "-", "-f", "webp", "pipe:1"]) )
|
||||
|
||||
SUPPORTED_IMAGE_EXTS = [".webp", ".png", ".jpg", ".jpeg", ".gif", ".tiff", ".tif", ".bmp"]
|
||||
CONV_SIZE_MAX = 32*1024*1024
|
||||
SNIFF_BYTES = 64
|
||||
RELAY_CHUNK = 256*1024
|
||||
|
||||
_ENV_BACKEND = os.environ.get("RNGIT_MEDIA_BACKEND")
|
||||
_winner = None
|
||||
_no_backend_logged = False
|
||||
|
||||
def available_backends():
|
||||
out = []
|
||||
for name, argv in BACKENDS: out.append((name, shutil.which(argv[0]) is not None))
|
||||
return out
|
||||
|
||||
def _selected_backend():
|
||||
global _winner
|
||||
if _ENV_BACKEND:
|
||||
for name, argv in BACKENDS:
|
||||
if name == _ENV_BACKEND and shutil.which(argv[0]) is not None:
|
||||
return (name, argv)
|
||||
return None
|
||||
|
||||
order = list(BACKENDS)
|
||||
if _winner is not None: order.sort(key=lambda backend: backend[0] != _winner)
|
||||
|
||||
for name, argv in order:
|
||||
if shutil.which(argv[0]) is not None:
|
||||
_winner = name
|
||||
return (name, argv)
|
||||
|
||||
return None
|
||||
|
||||
def _terminate(proc):
|
||||
try: proc.kill()
|
||||
except Exception: pass
|
||||
try: proc.wait()
|
||||
except Exception: pass
|
||||
|
||||
def _stderr_tail(proc, limit=1024):
|
||||
try:
|
||||
data = proc.stderr.read(limit)
|
||||
return data.decode("utf-8", "replace").strip()
|
||||
except Exception: return ""
|
||||
|
||||
def _webp_info(data):
|
||||
if len(data) < 30 or data[:4] != b"RIFF" or data[8:12] != b"WEBP": return None
|
||||
fourcc = data[12:16]
|
||||
if fourcc == b"VP8X":
|
||||
width = int.from_bytes(data[24:27], "little") + 1
|
||||
height = int.from_bytes(data[27:30], "little") + 1
|
||||
elif fourcc == b"VP8 ":
|
||||
width = int.from_bytes(data[26:28], "little") & 0x3FFF
|
||||
height = int.from_bytes(data[28:30], "little") & 0x3FFF
|
||||
elif fourcc == b"VP8L":
|
||||
bits = int.from_bytes(data[21:25], "little")
|
||||
width = (bits & 0x3FFF) + 1
|
||||
height = ((bits >> 14) & 0x3FFF) + 1
|
||||
else: return None
|
||||
if width > 0 and height > 0: return (width, height)
|
||||
return None
|
||||
|
||||
def _is_bmp_header(data):
|
||||
if len(data) < 18 or data[:2] != b"BM": return False
|
||||
try: dib_size = int.from_bytes(data[14:18], "little")
|
||||
except Exception: return False
|
||||
return dib_size in (12, 40, 52, 56, 64, 108, 124)
|
||||
|
||||
_IMAGE_SIGNATURES = ( ("png", lambda data: len(data) >= 8 and data[:8] == b"\x89PNG\r\n\x1a\n"),
|
||||
("jpeg", lambda data: len(data) >= 3 and data[:3] == b"\xff\xd8\xff"),
|
||||
("gif", lambda data: len(data) >= 6 and data[:6] in (b"GIF87a", b"GIF89a")),
|
||||
("tiff", lambda data: len(data) >= 4 and data[:4] in (b"II*\x00", b"MM\x00*")),
|
||||
("webp", lambda data: _webp_info(data) is not None),
|
||||
("bmp", lambda data: _is_bmp_header(data)) )
|
||||
|
||||
def _classify_image(data):
|
||||
for format_name, signature_matches in _IMAGE_SIGNATURES:
|
||||
if signature_matches(data): return format_name
|
||||
return None
|
||||
|
||||
def _valid_webp(path):
|
||||
try:
|
||||
with open(path, "rb") as fh: return _webp_info(fh.read(30)) is not None
|
||||
except Exception: return False
|
||||
|
||||
def _unlink_output(path):
|
||||
try: os.unlink(path)
|
||||
except Exception: pass
|
||||
|
||||
def _configured_backend(quality=None, max_dimension=None, input_spec=None):
|
||||
global _no_backend_logged
|
||||
backend = _selected_backend()
|
||||
if backend is None:
|
||||
if not _no_backend_logged:
|
||||
_no_backend_logged = True
|
||||
RNS.log("No WebP encoding backend available for media conversion. You can Install ImageMagick or ffmpeg to enable image conversion.", RNS.LOG_WARNING)
|
||||
return None
|
||||
|
||||
backend_name, encoder_argv = backend
|
||||
_no_backend_logged = False
|
||||
|
||||
quality_arg = None
|
||||
dimension_arg = None
|
||||
if quality is not None:
|
||||
try: quality_arg = max(1, min(100, int(quality)))
|
||||
except (ValueError, TypeError): pass
|
||||
if max_dimension is not None:
|
||||
try: dimension_arg = int(max_dimension)
|
||||
except (ValueError, TypeError): pass
|
||||
if dimension_arg is not None and dimension_arg < 1: dimension_arg = None
|
||||
|
||||
encoder_argv = list(encoder_argv)
|
||||
if backend_name in ("magick", "convert", "gm"):
|
||||
if input_spec is not None:
|
||||
try: input_index = encoder_argv.index("-")
|
||||
except ValueError: input_index = None
|
||||
if input_index is not None: encoder_argv[input_index] = input_spec
|
||||
options = []
|
||||
if quality_arg is not None: options += ["-quality", str(quality_arg)]
|
||||
if dimension_arg is not None: options += ["-resize", f"{dimension_arg}x{dimension_arg}>"]
|
||||
encoder_argv = encoder_argv[:-1] + options + encoder_argv[-1:]
|
||||
|
||||
elif backend_name in ("ffmpeg", "avconv"):
|
||||
options = []
|
||||
if quality_arg is not None: options += ["-quality", str(quality_arg)]
|
||||
if dimension_arg is not None: options += ["-vf", f"scale='min(iw,{dimension_arg})':'min(ih,{dimension_arg})':force_original_aspect_ratio=decrease"]
|
||||
try: format_index = encoder_argv.index("-f")
|
||||
except ValueError: format_index = len(encoder_argv)
|
||||
encoder_argv = encoder_argv[:format_index] + options + encoder_argv[format_index:]
|
||||
|
||||
return backend_name, encoder_argv
|
||||
|
||||
def _await(backend_name, encoder_proc, input_proc=None, timeout=None):
|
||||
if timeout is None: timeout = CONVERSION_TIMEOUT
|
||||
|
||||
deadline = time.time() + timeout
|
||||
try:
|
||||
encoder_proc.wait(timeout=max(0.0, deadline - time.time()))
|
||||
if input_proc is not None: input_proc.wait(timeout=max(0.0, deadline - time.time()))
|
||||
except subprocess.TimeoutExpired:
|
||||
_terminate(input_proc)
|
||||
_terminate(encoder_proc)
|
||||
RNS.log(f"Media conversion via {backend_name} timed out after {RNS.prettytime(timeout)}", RNS.LOG_WARNING)
|
||||
return False
|
||||
|
||||
if encoder_proc.returncode != 0:
|
||||
detail = ""
|
||||
encoder_tail = _stderr_tail(encoder_proc)
|
||||
input_tail = _stderr_tail(input_proc) if input_proc is not None else ""
|
||||
if encoder_tail: detail += f" Encoder: {encoder_tail}"
|
||||
if input_tail: detail += f" Input: {input_tail}"
|
||||
RNS.log(f"Media conversion via {backend_name} failed.{detail}", RNS.LOG_WARNING)
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def _await_readable(fd, timeout, selector):
|
||||
try: return bool(selector.select(max(0.0, timeout)))
|
||||
except Exception: return False
|
||||
|
||||
def _read_prefix(fd, amount, deadline):
|
||||
data = bytearray()
|
||||
selector = selectors.DefaultSelector()
|
||||
try:
|
||||
selector.register(fd, selectors.EVENT_READ)
|
||||
while len(data) < amount:
|
||||
if not _await_readable(fd, deadline - time.time(), selector): return None
|
||||
chunk = os.read(fd, amount - len(data))
|
||||
if not chunk: break
|
||||
data += chunk
|
||||
return bytes(data)
|
||||
finally: selector.close()
|
||||
|
||||
def _pump(fd, encoder_stdin, prefix, deadline):
|
||||
written = len(prefix)
|
||||
if written > CONV_SIZE_MAX: return None
|
||||
encoder_stdin.write(prefix)
|
||||
|
||||
selector = selectors.DefaultSelector()
|
||||
try:
|
||||
selector.register(fd, selectors.EVENT_READ)
|
||||
while True:
|
||||
if not _await_readable(fd, deadline - time.time(), selector):
|
||||
raise subprocess.TimeoutExpired("input relay", max(0.0, deadline - time.time()))
|
||||
chunk = os.read(fd, RELAY_CHUNK)
|
||||
if not chunk: break
|
||||
|
||||
written += len(chunk)
|
||||
if written > CONV_SIZE_MAX: return None
|
||||
encoder_stdin.write(chunk)
|
||||
|
||||
return written
|
||||
|
||||
finally: selector.close()
|
||||
|
||||
def detect_image_format(data):
|
||||
return _classify_image(data)
|
||||
|
||||
def convert_to_webp(input_argv, output_path, cwd=None, timeout=None, quality=None, max_dimension=None):
|
||||
configured = _configured_backend(quality=quality, max_dimension=max_dimension)
|
||||
if configured is None: return False
|
||||
backend_name = configured[0]
|
||||
|
||||
if timeout is None: timeout = CONVERSION_TIMEOUT
|
||||
deadline = time.time() + timeout
|
||||
|
||||
try:
|
||||
with open(output_path, "wb") as output_fh:
|
||||
input_proc = subprocess.Popen(input_argv, cwd=cwd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
input_fd = input_proc.stdout.fileno()
|
||||
|
||||
prefix = _read_prefix(input_fd, SNIFF_BYTES, deadline)
|
||||
if prefix is None:
|
||||
RNS.log(f"Media conversion via {backend_name} timed out after {RNS.prettytime(timeout)}", RNS.LOG_WARNING)
|
||||
_terminate(input_proc)
|
||||
return False
|
||||
|
||||
detected = _classify_image(prefix)
|
||||
if detected is None:
|
||||
RNS.log("Media conversion failed: Input format is not supported", RNS.LOG_DEBUG)
|
||||
_terminate(input_proc)
|
||||
return False
|
||||
|
||||
configured = _configured_backend(quality=quality, max_dimension=max_dimension, input_spec=detected+":-")
|
||||
if configured is None:
|
||||
_terminate(input_proc)
|
||||
return False
|
||||
backend_name, encoder_argv = configured
|
||||
|
||||
encoder_proc = subprocess.Popen(encoder_argv, stdin=subprocess.PIPE, stdout=output_fh, stderr=subprocess.PIPE)
|
||||
|
||||
aborted = False
|
||||
try:
|
||||
if _pump(input_fd, encoder_proc.stdin, prefix, deadline) is None:
|
||||
RNS.log(f"Media conversion failed: Streaming input exceeded maximum size of {RNS.prettysize(CONV_SIZE_MAX)}", RNS.LOG_DEBUG)
|
||||
aborted = True
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
RNS.log(f"Media conversion via {backend_name} timed out after {RNS.prettytime(timeout)}", RNS.LOG_WARNING)
|
||||
aborted = True
|
||||
|
||||
except (BrokenPipeError, OSError):
|
||||
RNS.log(f"Media conversion via {backend_name} failed: Encoder exited before input was consumed", RNS.LOG_DEBUG)
|
||||
aborted = True
|
||||
|
||||
if aborted:
|
||||
_terminate(input_proc)
|
||||
_terminate(encoder_proc)
|
||||
return False
|
||||
|
||||
try: encoder_proc.stdin.close()
|
||||
except (BrokenPipeError, OSError): pass
|
||||
|
||||
try: input_proc.stdout.close()
|
||||
except Exception: pass
|
||||
|
||||
if not _await(backend_name, encoder_proc, input_proc=input_proc, timeout=max(0.0, deadline - time.time())): return False
|
||||
|
||||
if not _valid_webp(output_path):
|
||||
RNS.log(f"Media conversion via {backend_name} produced invalid WebP output", RNS.LOG_WARNING)
|
||||
return False
|
||||
|
||||
RNS.log(f"Media converted to WebP with {backend_name}", RNS.LOG_DEBUG)
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error during media conversion: {e}", RNS.LOG_WARNING)
|
||||
return False
|
||||
|
||||
def convert_file_to_webp(source_path, quality=85, max_dimension=None, timeout=None):
|
||||
configured = _configured_backend(quality=quality, max_dimension=max_dimension)
|
||||
if configured is None: return False
|
||||
backend_name = configured[0]
|
||||
|
||||
tmp_path = None
|
||||
success = False
|
||||
try:
|
||||
if not os.path.isfile(source_path):
|
||||
RNS.log(f"Cannot convert media: Source file does not exist: {source_path}", RNS.LOG_WARNING)
|
||||
return False
|
||||
|
||||
try: source_size = os.path.getsize(source_path)
|
||||
except Exception: source_size = None
|
||||
if source_size is not None and source_size > CONV_SIZE_MAX:
|
||||
RNS.log(f"Cannot convert media: Source file exceeds maximum size of {RNS.prettysize(CONV_SIZE_MAX)}", RNS.LOG_DEBUG)
|
||||
return False
|
||||
|
||||
fd, tmp_path = tempfile.mkstemp(prefix="rns_media_", suffix=".webp")
|
||||
os.close(fd)
|
||||
|
||||
with open(source_path, "rb", buffering=0) as input_fh, open(tmp_path, "wb") as output_fh:
|
||||
prefix = b""
|
||||
while len(prefix) < SNIFF_BYTES:
|
||||
chunk = input_fh.read(SNIFF_BYTES - len(prefix))
|
||||
if not chunk: break
|
||||
prefix += chunk
|
||||
|
||||
detected = _classify_image(prefix)
|
||||
if detected is None:
|
||||
RNS.log("Cannot convert media: Source file is not a supported media format", RNS.LOG_DEBUG)
|
||||
return False
|
||||
|
||||
configured = _configured_backend(quality=quality, max_dimension=max_dimension, input_spec=detected+":-")
|
||||
if configured is None: return False
|
||||
backend_name, encoder_argv = configured
|
||||
|
||||
input_fh.seek(0)
|
||||
|
||||
encoder_proc = subprocess.Popen(encoder_argv, stdin=input_fh, stdout=output_fh, stderr=subprocess.PIPE)
|
||||
if not _await(backend_name, encoder_proc, timeout=timeout): return False
|
||||
|
||||
if not _valid_webp(tmp_path):
|
||||
RNS.log(f"Invalid media conversion output from {backend_name}", RNS.LOG_WARNING)
|
||||
return False
|
||||
|
||||
success = True
|
||||
RNS.log(f"Media converted to WebP with {backend_name}", RNS.LOG_DEBUG)
|
||||
return tmp_path
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error during media conversion: {e}", RNS.LOG_WARNING)
|
||||
return False
|
||||
|
||||
finally:
|
||||
if not success and tmp_path is not None: _unlink_output(tmp_path)
|
||||
3074
RNS/Utilities/rngit/pages.py
Normal file
3074
RNS/Utilities/rngit/pages.py
Normal file
File diff suppressed because it is too large
Load diff
5120
RNS/Utilities/rngit/server.py
Normal file
5120
RNS/Utilities/rngit/server.py
Normal file
File diff suppressed because it is too large
Load diff
819
RNS/Utilities/rngit/util.py
Normal file
819
RNS/Utilities/rngit/util.py
Normal file
|
|
@ -0,0 +1,819 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
import re
|
||||
import RNS
|
||||
|
||||
# Validate ref names according to https://git-scm.com/docs/git-check-ref-format
|
||||
# This may be a bit overkill, since git validates names as well, but why not.
|
||||
def san_ref(ref):
|
||||
if ref.startswith("-"): return None
|
||||
if ref.startswith("/"): return None
|
||||
if ref.endswith("/"): return None
|
||||
if ref.endswith("."): return None
|
||||
|
||||
if " " in ref: return None
|
||||
if not "/" in ref: return None
|
||||
if ".." in ref: return None
|
||||
if "/." in ref: return None
|
||||
if "//" in ref: return None
|
||||
if "\\" in ref: return None
|
||||
|
||||
for comp in ref.split("/"):
|
||||
if comp.endswith(".lock"): return None
|
||||
|
||||
if not all(ord(c) >= 40 for c in ref): return None # Any control character
|
||||
if "\x7f" in ref: return None # ASCII DEL (177)
|
||||
if "~" in ref: return None
|
||||
if "^" in ref: return None
|
||||
if ":" in ref: return None
|
||||
if "?" in ref: return None
|
||||
if "*" in ref: return None
|
||||
if "[" in ref: return None
|
||||
if "@{" in ref: return None
|
||||
if "@" == ref: return None
|
||||
|
||||
return ref
|
||||
|
||||
def san_refs(refs):
|
||||
if not type(refs) == list: return None
|
||||
for ref in refs:
|
||||
if not san_ref(ref): return None
|
||||
|
||||
return refs
|
||||
|
||||
# Git SHA format validation
|
||||
def san_sha(sha):
|
||||
if len(sha) < 40: return None
|
||||
try: bytes.fromhex(sha)
|
||||
except: return None
|
||||
return sha
|
||||
|
||||
class MarkdownToMicron:
|
||||
BOLD = "`!"
|
||||
BOLD_END = "`!"
|
||||
ITALIC = "`*"
|
||||
ITALIC_END = "`*"
|
||||
UNDERLINE = "`_"
|
||||
UNDERLINE_END = "`_"
|
||||
|
||||
CODE_BG = "`BT282828"
|
||||
CODE_BG_INLINE = "`BT383838"
|
||||
CODE_FG = "`Fddd"
|
||||
CODE_RESET = "`f`b"
|
||||
|
||||
LITERAL_START = "`="
|
||||
LITERAL_END = "`="
|
||||
|
||||
BULLET = "•"
|
||||
|
||||
# Regex patterns for markdown elements
|
||||
HEADER_RE = re.compile(r'^(#{1,6})\s+(.+)$')
|
||||
CODE_FENCE_RE = re.compile(r'^(\s*)```(.*)$')
|
||||
HORIZONTAL_RULE_RE = re.compile(r'^(\s*)(---+|===+|\*\*\*+|___+)\s*$')
|
||||
UNORDERED_LIST_RE = re.compile(r'^(\s*)([-*+])\s+(.+)$')
|
||||
|
||||
# Table patterns
|
||||
TABLE_ROW_RE = re.compile(r'^\s*\|?(.+?)\|?\s*$')
|
||||
TABLE_SEP_RE = re.compile(r'^\s*\|?(?:\s*:?-+:?\s*\|)+\s*$')
|
||||
|
||||
# Quote pattern
|
||||
QUOTE_RE = re.compile(r'^>\s?(.*)$')
|
||||
|
||||
# Inline patterns (processed in order of specificity)
|
||||
LINK_RE = re.compile(r'\[([^\]]+)\]\(([^)]+)\)')
|
||||
INLINE_CODE_RE = re.compile(r'`([^`]+)`')
|
||||
BOLD_RE = re.compile(r'\*\*(.+?)\*\*|__(.+?)__')
|
||||
ITALIC_RE = re.compile(r'\*(.+?)\*|_(.+?)_')
|
||||
|
||||
TABLE_H = "─"
|
||||
TABLE_V = "│"
|
||||
TABLE_TL = "┌"
|
||||
TABLE_TR = "┐"
|
||||
TABLE_BL = "└"
|
||||
TABLE_BR = "┘"
|
||||
TABLE_ML = "├"
|
||||
TABLE_MR = "┤"
|
||||
TABLE_TM = "┬"
|
||||
TABLE_BM = "┴"
|
||||
TABLE_MM = "┼"
|
||||
|
||||
TABLE_MIN_COL_WIDTH = 3
|
||||
|
||||
def __init__(self, max_width=100, syntax_highlighter=None, url_scope=None):
|
||||
self.max_width = max_width
|
||||
self.local_url_scope = url_scope or ":/page/"
|
||||
self.__local_url_scope = self.local_url_scope
|
||||
self.syntax_highlighter = syntax_highlighter
|
||||
self.wcwidth = None
|
||||
|
||||
self.bold_links = True
|
||||
self.underline_links = True
|
||||
self.link_color = None
|
||||
|
||||
try:
|
||||
import wcwidth
|
||||
self.wcwidth = wcwidth
|
||||
|
||||
except: RNS.log(f"The wcwidth module is unavailable, display width calculations for some glyphs will be incorrect", RNS.LOG_WARNING)
|
||||
|
||||
def set_url_scope(self, url_scope): self.local_url_scope = url_scope
|
||||
def restore_url_scope(self, url_scope): self.local_url_scope = self.__local_url_scope
|
||||
|
||||
def display_width(self, text):
|
||||
if not self.wcwidth: return len(text)
|
||||
else:
|
||||
# wcswidth returns -1 for non-printable strings,
|
||||
# fallback to len in this case
|
||||
w = self.wcwidth.wcswidth(text)
|
||||
return w if w is not None and w >= 0 else len(text)
|
||||
|
||||
def format_block(self, text, url_scope=None):
|
||||
# text = text.replace("\\", "\\\\") # Now handled in format_line instead
|
||||
lines = text.split('\n')
|
||||
result_lines = []
|
||||
in_code_block = False
|
||||
code_block_lang = None
|
||||
code_buffer = []
|
||||
in_table = False
|
||||
table_buffer = []
|
||||
in_quote = False
|
||||
quote_buffer = []
|
||||
|
||||
def flush_quote_buffer():
|
||||
nonlocal result_lines, quote_buffer, in_quote
|
||||
if not quote_buffer:
|
||||
in_quote = False
|
||||
return
|
||||
|
||||
para = " ".join(quote_buffer)
|
||||
formatted = self._format_inline(para)
|
||||
|
||||
effective_width = self.max_width - 3
|
||||
if effective_width < 1: effective_width = 1
|
||||
wrapped_lines = self._wrap_text(formatted, effective_width)
|
||||
for wrapped_line in wrapped_lines: result_lines.append(f" │ {wrapped_line}")
|
||||
|
||||
quote_buffer = []
|
||||
in_quote = False
|
||||
|
||||
def flush_table_buffer():
|
||||
nonlocal result_lines, table_buffer, in_table
|
||||
if not table_buffer:
|
||||
in_table = False
|
||||
return
|
||||
|
||||
if len(table_buffer) >= 2 and self._is_table_separator(table_buffer[1]):
|
||||
formatted_lines = self.format_table(table_buffer)
|
||||
result_lines.extend(formatted_lines)
|
||||
|
||||
else:
|
||||
for line in table_buffer: result_lines.append(self.format_line(line))
|
||||
|
||||
table_buffer = []
|
||||
in_table = False
|
||||
|
||||
def flush_code_block():
|
||||
nonlocal result_lines, code_buffer, code_block_lang
|
||||
if not code_buffer:
|
||||
return
|
||||
|
||||
code_content = '\n'.join(code_buffer)
|
||||
|
||||
if self.syntax_highlighter and code_block_lang:
|
||||
if code_block_lang.lower() == "rawmu": result_lines.append(code_content)
|
||||
else:
|
||||
try:
|
||||
highlighted = self.syntax_highlighter.highlight(code_content, language=code_block_lang)
|
||||
result_lines.append(f"{self.CODE_BG}{self.CODE_FG}")
|
||||
result_lines.append(highlighted)
|
||||
result_lines.append(self.CODE_RESET)
|
||||
|
||||
except Exception:
|
||||
# Fallback to plain literal block on any error
|
||||
result_lines.append(f"{self.CODE_BG}{self.CODE_FG}")
|
||||
result_lines.append(self.LITERAL_START)
|
||||
result_lines.append(self._escape_literals(code_content))
|
||||
result_lines.append(self.LITERAL_END)
|
||||
result_lines.append(self.CODE_RESET)
|
||||
else:
|
||||
result_lines.append(f"{self.CODE_BG}{self.CODE_FG}")
|
||||
result_lines.append(self.LITERAL_START)
|
||||
result_lines.append(self._escape_literals(code_content))
|
||||
result_lines.append(self.LITERAL_END)
|
||||
result_lines.append(self.CODE_RESET)
|
||||
|
||||
code_buffer = []
|
||||
|
||||
for line in lines:
|
||||
is_fence, lang_hint = self._detect_code_fence(line)
|
||||
|
||||
if is_fence:
|
||||
# Flush any pending structures before code fence
|
||||
flush_quote_buffer()
|
||||
flush_table_buffer()
|
||||
|
||||
if not in_code_block:
|
||||
# Opening fence, start buffering
|
||||
in_code_block = True
|
||||
code_block_lang = lang_hint.strip() if lang_hint else None
|
||||
code_buffer = []
|
||||
|
||||
else:
|
||||
# Closing fence, flush highlighted code
|
||||
flush_code_block()
|
||||
in_code_block = False
|
||||
code_block_lang = None
|
||||
|
||||
else:
|
||||
# Buffer code lines for later highlighting
|
||||
if in_code_block: code_buffer.append(line)
|
||||
else:
|
||||
quote_match = self.QUOTE_RE.match(line)
|
||||
if quote_match:
|
||||
if not in_quote:
|
||||
flush_table_buffer()
|
||||
in_quote = True
|
||||
quote_buffer = []
|
||||
|
||||
quote_buffer.append(quote_match.group(1))
|
||||
|
||||
else:
|
||||
if in_quote:
|
||||
flush_quote_buffer()
|
||||
if line.strip() != "":
|
||||
if self._is_table_row(line):
|
||||
in_table = True
|
||||
table_buffer = [line]
|
||||
|
||||
else:
|
||||
formatted = self.format_line(line)
|
||||
result_lines.append(formatted)
|
||||
|
||||
# Pass through blank line as separator
|
||||
else: result_lines.append("")
|
||||
|
||||
else:
|
||||
if self._is_table_row(line):
|
||||
if not in_table:
|
||||
in_table = True
|
||||
table_buffer = [line]
|
||||
|
||||
else: table_buffer.append(line)
|
||||
|
||||
else:
|
||||
# Line breaks table, flush buffer
|
||||
if in_table: flush_table_buffer()
|
||||
formatted = self.format_line(line)
|
||||
result_lines.append(formatted)
|
||||
|
||||
# Handle unclosed structures
|
||||
if in_quote: flush_quote_buffer()
|
||||
if in_table: flush_table_buffer()
|
||||
if in_code_block: flush_code_block()
|
||||
|
||||
return '\n'.join(result_lines)
|
||||
|
||||
def format_line(self, line, mode="normal"):
|
||||
if mode == "codeblock": return self._escape_literals(line)
|
||||
line = line.replace("\\", "\\\\")
|
||||
|
||||
if line.startswith("-") and not line.startswith("---") and not line.startswith("- "): line = f"\\{line}"
|
||||
if line.startswith("<"): line = f"\\{line}"
|
||||
# if line.startswith(">"): line = f"\\{line}" # Now handled by blockquotes
|
||||
|
||||
if self.HORIZONTAL_RULE_RE.match(line): return self._format_horizontal_rule()
|
||||
|
||||
header_match = self.HEADER_RE.match(line)
|
||||
if header_match: return self._format_header(header_match)
|
||||
|
||||
list_match = self.UNORDERED_LIST_RE.match(line)
|
||||
if list_match: return self._format_list_item(list_match)
|
||||
|
||||
line = self._format_inline(line)
|
||||
|
||||
return line
|
||||
|
||||
def _format_inline(self, text):
|
||||
code_blocks = []
|
||||
def extract_code(match):
|
||||
code_blocks.append(match.group(1))
|
||||
return f"\x00CODE{len(code_blocks)-1}\x00"
|
||||
|
||||
links = []
|
||||
def extract_link(match):
|
||||
links.append((match.group(1), match.group(2)))
|
||||
return f"\x00LINK{len(links)-1}\x00"
|
||||
|
||||
text = self.LINK_RE.sub(extract_link, text)
|
||||
text = self.INLINE_CODE_RE.sub(extract_code, text)
|
||||
text = self.BOLD_RE.sub(self._bold_sub, text)
|
||||
text = self.ITALIC_RE.sub(self._italic_sub, text)
|
||||
|
||||
def restore_link(match):
|
||||
idx = int(match.group(1))
|
||||
text, url = links[idx]
|
||||
|
||||
anchor_components = url.split("#")
|
||||
url = anchor_components[0]
|
||||
anchor = anchor_components[1] if len(anchor_components) > 1 else ""
|
||||
|
||||
if not ":/" in url:
|
||||
url = f"{self.local_url_scope}{url}"
|
||||
if anchor: url = f"{url}|anchor={anchor}"
|
||||
|
||||
undl = "`_" if self.underline_links else ""
|
||||
bold = "`!" if self.bold_links else ""
|
||||
text = text.replace('`', '')
|
||||
link = f"{undl}{bold}`[{text}`{url}]{bold}{undl}"
|
||||
|
||||
if self.link_color and len(self.link_color) == 3: link = f"`F{self.link_color}{link}`f"
|
||||
if self.link_color and len(self.link_color) == 6: link = f"`FT{self.link_color}{link}`f"
|
||||
|
||||
return link
|
||||
|
||||
text = re.sub(r'\x00LINK(\d+)\x00', restore_link, text)
|
||||
|
||||
def restore_code(match):
|
||||
idx = int(match.group(1))
|
||||
content = code_blocks[idx]
|
||||
content = content.replace('`', '\\`')
|
||||
return f"{self.CODE_BG_INLINE}{self.CODE_FG}{content}{self.CODE_RESET}"
|
||||
|
||||
text = re.sub(r'\x00CODE(\d+)\x00', restore_code, text)
|
||||
return text
|
||||
|
||||
def _highlight_inline_code(self, content):
|
||||
if not self.syntax_highlighter: return None
|
||||
return self.syntax_highlighter.highlight(content, language=None)
|
||||
|
||||
def _bold_sub(self, match):
|
||||
content = match.group(1) or match.group(2)
|
||||
return f"{self.BOLD}{content}{self.BOLD_END}"
|
||||
|
||||
def _italic_sub(self, match):
|
||||
content = match.group(1) or match.group(2)
|
||||
return f"{self.ITALIC}{content}{self.ITALIC_END}"
|
||||
|
||||
def _format_header(self, match):
|
||||
hashes = match.group(1)
|
||||
content = match.group(2)
|
||||
level = len(hashes)
|
||||
prefix = ">" * min(level, 6)
|
||||
return f"{prefix}{self._format_inline(content)}"
|
||||
|
||||
def _format_list_item(self, match):
|
||||
indent = match.group(1)
|
||||
content = match.group(3)
|
||||
content = self._format_inline(content)
|
||||
return f"{indent} {self.BULLET} {content}"
|
||||
|
||||
def _format_horizontal_rule(self):
|
||||
return "-"
|
||||
|
||||
def _detect_code_fence(self, line):
|
||||
match = self.CODE_FENCE_RE.match(line)
|
||||
if match:
|
||||
# match.group(2) contains everything after the backticks (language hint)
|
||||
return True, match.group(2)
|
||||
return False, ""
|
||||
|
||||
def _is_table_row(self, line):
|
||||
if '|' not in line: return False
|
||||
match = self.TABLE_ROW_RE.match(line)
|
||||
if match is None: return False
|
||||
content = match.group(1)
|
||||
return '|' in content or line.strip().startswith('|')
|
||||
|
||||
def _is_table_separator(self, line):
|
||||
if '|' not in line: return False
|
||||
match = self.TABLE_SEP_RE.match(line)
|
||||
return match is not None
|
||||
|
||||
def _escape_literals(self, text):
|
||||
return text.replace('`', '\\`')
|
||||
|
||||
def format_table(self, rows, align="c"):
|
||||
if len(rows) < 2: return rows
|
||||
|
||||
# Parse header and separator
|
||||
header_cells = self._parse_table_row(rows[0])
|
||||
alignments = self._parse_table_alignments(rows[1])
|
||||
|
||||
# Ensure alignment count matches header cells
|
||||
while len(alignments) < len(header_cells): alignments.append('left')
|
||||
alignments = alignments[:len(header_cells)]
|
||||
|
||||
# Parse data rows
|
||||
data_rows = []
|
||||
for i in range(2, len(rows)):
|
||||
cells = self._parse_table_row(rows[i])
|
||||
while len(cells) < len(header_cells): cells.append("")
|
||||
cells = cells[:len(header_cells)]
|
||||
data_rows.append(cells)
|
||||
|
||||
# Calculate column widths based on content
|
||||
num_cols = len(header_cells)
|
||||
col_widths = [0] * num_cols
|
||||
|
||||
all_rows = [header_cells] + data_rows
|
||||
for row in all_rows:
|
||||
for i, cell in enumerate(row):
|
||||
formatted = self._format_inline(cell)
|
||||
width = self._visible_width(formatted)
|
||||
col_widths[i] = max(col_widths[i], width)
|
||||
|
||||
# Apply minimum width and calculate total
|
||||
col_widths = [max(w, self.TABLE_MIN_COL_WIDTH) for w in col_widths]
|
||||
|
||||
# Check max_width constraint
|
||||
# Total = sum of columns + 3 chars per column (space + 2 borders) + 1 for final border
|
||||
total_width = sum(col_widths) + (num_cols * 3) + 1
|
||||
|
||||
if total_width > self.max_width:
|
||||
# Reduce widest columns proportionally
|
||||
excess = total_width - self.max_width
|
||||
indexed_widths = [(i, w) for i, w in enumerate(col_widths)]
|
||||
indexed_widths.sort(key=lambda x: -x[1])
|
||||
|
||||
for i, w in indexed_widths:
|
||||
if excess <= 0: break
|
||||
reduction = min(excess, w - self.TABLE_MIN_COL_WIDTH)
|
||||
col_widths[i] -= reduction
|
||||
excess -= reduction
|
||||
|
||||
# Build formatted table
|
||||
result = []
|
||||
|
||||
# Alignment start
|
||||
if align: result.append(f"`{align}")
|
||||
|
||||
# Top border
|
||||
border = self.TABLE_TL
|
||||
for i, w in enumerate(col_widths):
|
||||
border += self.TABLE_H * (w + 2)
|
||||
if i < len(col_widths) - 1: border += self.TABLE_TM
|
||||
else: border += self.TABLE_TR
|
||||
|
||||
result.append(self._escape_literals(border))
|
||||
|
||||
# Header row
|
||||
header_line = self.TABLE_V
|
||||
for i, cell in enumerate(header_cells):
|
||||
formatted = self._format_inline(cell)
|
||||
padded = self._pad_cell(formatted, col_widths[i], 'left')
|
||||
header_line += f" {padded} {self.TABLE_V}"
|
||||
result.append(self._escape_literals(header_line))
|
||||
|
||||
# Separator row
|
||||
sep_line = self.TABLE_ML
|
||||
for i, w in enumerate(col_widths):
|
||||
cell_width = w + 2
|
||||
sep_line += self.TABLE_H * cell_width
|
||||
|
||||
if i < len(col_widths) - 1: sep_line += self.TABLE_MM
|
||||
else: sep_line += self.TABLE_MR
|
||||
|
||||
result.append(self._escape_literals(sep_line))
|
||||
|
||||
# Data rows
|
||||
for row in data_rows:
|
||||
row_line = self.TABLE_V
|
||||
for i, cell in enumerate(row):
|
||||
formatted = self._format_inline(cell)
|
||||
padded = self._pad_cell(formatted, col_widths[i], alignments[i])
|
||||
row_line += f" {padded} {self.TABLE_V}"
|
||||
|
||||
result.append(row_line)
|
||||
|
||||
# Bottom border
|
||||
border = self.TABLE_BL
|
||||
for i, w in enumerate(col_widths):
|
||||
border += self.TABLE_H * (w + 2)
|
||||
if i < len(col_widths) - 1: border += self.TABLE_BM
|
||||
else: border += self.TABLE_BR
|
||||
|
||||
result.append(self._escape_literals(border))
|
||||
|
||||
# End alignment
|
||||
if align: result.append("`a")
|
||||
|
||||
return result
|
||||
|
||||
def format_table_raw(self, rows, align="c"):
|
||||
if len(rows) < 2: return rows
|
||||
|
||||
# Parse header and separator
|
||||
header_cells = self._parse_table_row(rows[0])
|
||||
alignments = self._parse_table_alignments(rows[1])
|
||||
|
||||
# Ensure alignment count matches header cells
|
||||
while len(alignments) < len(header_cells): alignments.append('left')
|
||||
alignments = alignments[:len(header_cells)]
|
||||
|
||||
# Parse data rows
|
||||
data_rows = []
|
||||
for i in range(2, len(rows)):
|
||||
cells = self._parse_table_row(rows[i])
|
||||
while len(cells) < len(header_cells): cells.append("")
|
||||
cells = cells[:len(header_cells)]
|
||||
data_rows.append(cells)
|
||||
|
||||
# Calculate column widths based on raw content
|
||||
num_cols = len(header_cells)
|
||||
col_widths = [0] * num_cols
|
||||
|
||||
all_rows = [header_cells] + data_rows
|
||||
for row in all_rows:
|
||||
for i, cell in enumerate(row):
|
||||
width = self._visible_width(cell)
|
||||
col_widths[i] = max(col_widths[i], width)
|
||||
|
||||
# Apply minimum width and calculate total
|
||||
col_widths = [max(w, self.TABLE_MIN_COL_WIDTH) for w in col_widths]
|
||||
|
||||
# Check max_width constraint
|
||||
total_width = sum(col_widths) + (num_cols * 3) + 1
|
||||
|
||||
if total_width > self.max_width:
|
||||
# Reduce widest columns proportionally
|
||||
excess = total_width - self.max_width
|
||||
indexed_widths = [(i, w) for i, w in enumerate(col_widths)]
|
||||
indexed_widths.sort(key=lambda x: -x[1])
|
||||
|
||||
for i, w in indexed_widths:
|
||||
if excess <= 0: break
|
||||
reduction = min(excess, w - self.TABLE_MIN_COL_WIDTH)
|
||||
col_widths[i] -= reduction
|
||||
excess -= reduction
|
||||
|
||||
# Build formatted table
|
||||
result = []
|
||||
|
||||
# Alignment start
|
||||
if align: result.append(f"`{align}")
|
||||
|
||||
# Top border
|
||||
border = self.TABLE_TL
|
||||
for i, w in enumerate(col_widths):
|
||||
border += self.TABLE_H * (w + 2)
|
||||
if i < len(col_widths) - 1: border += self.TABLE_TM
|
||||
else: border += self.TABLE_TR
|
||||
|
||||
result.append(self._escape_literals(border))
|
||||
|
||||
# Header row
|
||||
header_line = self.TABLE_V
|
||||
for i, cell in enumerate(header_cells):
|
||||
padded = self._pad_cell(cell, col_widths[i], 'left')
|
||||
header_line += f" {padded} {self.TABLE_V}"
|
||||
result.append(header_line)
|
||||
|
||||
# Separator row - clean horizontal lines without alignment markers
|
||||
sep_line = self.TABLE_ML
|
||||
for i, w in enumerate(col_widths):
|
||||
cell_width = w + 2
|
||||
sep_line += self.TABLE_H * cell_width
|
||||
|
||||
if i < len(col_widths) - 1: sep_line += self.TABLE_MM
|
||||
else: sep_line += self.TABLE_MR
|
||||
|
||||
result.append(self._escape_literals(sep_line))
|
||||
|
||||
# Data rows (with alignment)
|
||||
for row in data_rows:
|
||||
row_line = self.TABLE_V
|
||||
for i, cell in enumerate(row):
|
||||
padded = self._pad_cell(cell, col_widths[i], alignments[i])
|
||||
row_line += f" {padded} {self.TABLE_V}"
|
||||
|
||||
result.append(row_line)
|
||||
|
||||
# Bottom border
|
||||
border = self.TABLE_BL
|
||||
for i, w in enumerate(col_widths):
|
||||
border += self.TABLE_H * (w + 2)
|
||||
if i < len(col_widths) - 1: border += self.TABLE_BM
|
||||
else: border += self.TABLE_BR
|
||||
|
||||
result.append(self._escape_literals(border))
|
||||
|
||||
# End alignment
|
||||
if align: result.append("`a")
|
||||
|
||||
return result
|
||||
|
||||
def _parse_table_row(self, line):
|
||||
line = line.strip()
|
||||
if line.startswith('|'): line = line[1:]
|
||||
if line.endswith('|'): line = line[:-1]
|
||||
|
||||
cells = []
|
||||
current = ""
|
||||
escaped = False
|
||||
for char in line:
|
||||
if escaped:
|
||||
current += char
|
||||
escaped = False
|
||||
elif char == '\\':
|
||||
escaped = True
|
||||
elif char == '|':
|
||||
cells.append(current.strip())
|
||||
current = ""
|
||||
else:
|
||||
current += char
|
||||
|
||||
cells.append(current.strip())
|
||||
return cells
|
||||
|
||||
def _parse_table_alignments(self, line):
|
||||
cells = self._parse_table_row(line)
|
||||
alignments = []
|
||||
for cell in cells:
|
||||
cell = cell.strip()
|
||||
if cell.startswith(':') and cell.endswith(':'): alignments.append('center')
|
||||
elif cell.endswith(':'): alignments.append('right')
|
||||
else: alignments.append('left')
|
||||
|
||||
return alignments
|
||||
|
||||
def _visible_width(self, text):
|
||||
text = re.sub(r'`[FB][0-9a-fA-F]{3}', '', text)
|
||||
text = re.sub(r'`[FB]T[0-9a-fA-F]{6}', '', text)
|
||||
text = re.sub(r'`[!*_=]', '', text)
|
||||
text = re.sub(r'`f`b', '', text)
|
||||
text = re.sub(r'`f', '', text)
|
||||
text = re.sub(r'`b', '', text)
|
||||
return self.display_width(text)
|
||||
|
||||
def _pad_cell(self, text, width, align):
|
||||
text = self._truncate_cell(text, width)
|
||||
text_width = self._visible_width(text)
|
||||
padding = width - text_width
|
||||
|
||||
if align == 'right':
|
||||
return " " * padding + text
|
||||
elif align == 'center':
|
||||
left = padding // 2
|
||||
right = padding - left
|
||||
return " " * left + text + " " * right
|
||||
else:
|
||||
return text + " " * padding
|
||||
|
||||
def _truncate_cell(self, text, width):
|
||||
if self._visible_width(text) <= width: return text
|
||||
|
||||
truncation_point = len(text)
|
||||
while truncation_point > 0 and self._visible_width(text[0:truncation_point]) >= width:
|
||||
truncation_point -= 1
|
||||
|
||||
truncated = text[:truncation_point]
|
||||
|
||||
# Yes, this is convoluted, but if someone else has
|
||||
# a better idea on how to handle unclosed micron
|
||||
# tags in the truncated cells, I'm all ears.
|
||||
active_tags = set()
|
||||
fg_active = False
|
||||
bg_active = False
|
||||
|
||||
i = 0
|
||||
while i < len(truncated):
|
||||
if truncated[i] == '`':
|
||||
if i + 1 < len(truncated):
|
||||
tag_char = truncated[i + 1]
|
||||
|
||||
if tag_char in '!*_=':
|
||||
if tag_char in active_tags: active_tags.remove(tag_char)
|
||||
else: active_tags.add(tag_char)
|
||||
i += 2
|
||||
continue
|
||||
|
||||
elif tag_char == 'f':
|
||||
fg_active = False
|
||||
i += 2
|
||||
continue
|
||||
|
||||
elif tag_char == 'b':
|
||||
bg_active = False
|
||||
i += 2
|
||||
continue
|
||||
|
||||
elif tag_char == 'F':
|
||||
fg_active = True
|
||||
if i + 2 < len(truncated) and truncated[i + 2] == 'T': i += 8
|
||||
else: i += 5
|
||||
continue
|
||||
|
||||
elif tag_char == 'B':
|
||||
bg_active = True
|
||||
if i + 2 < len(truncated) and truncated[i + 2] == 'T': i += 8
|
||||
else: i += 5
|
||||
continue
|
||||
i += 1
|
||||
|
||||
closers = []
|
||||
if fg_active: closers.append('`f')
|
||||
if bg_active: closers.append('`b')
|
||||
for fmt in active_tags: closers.append(f'`{fmt}')
|
||||
|
||||
return truncated + ''.join(closers) + "…"
|
||||
|
||||
def _wrap_text(self, text, width):
|
||||
if not text: return [""]
|
||||
|
||||
words = text.split(' ')
|
||||
lines = []
|
||||
current_line = ""
|
||||
current_width = 0
|
||||
|
||||
for word in words:
|
||||
if not word: continue
|
||||
|
||||
word_width = self._visible_width(word)
|
||||
|
||||
# Check if word alone exceeds width to force break it
|
||||
if word_width > width:
|
||||
if current_line:
|
||||
lines.append(current_line)
|
||||
current_line = ""
|
||||
current_width = 0
|
||||
|
||||
# Force break the long word character by character
|
||||
remaining = word
|
||||
while remaining:
|
||||
# Binary search for how many characters fit
|
||||
low, high = 1, len(remaining)
|
||||
fit_chars = 0
|
||||
|
||||
while low <= high:
|
||||
mid = (low + high) // 2
|
||||
test_substr = remaining[:mid]
|
||||
test_width = self._visible_width(test_substr)
|
||||
|
||||
if test_width <= width:
|
||||
fit_chars = mid
|
||||
low = mid + 1
|
||||
else:
|
||||
high = mid - 1
|
||||
|
||||
if fit_chars == 0: fit_chars = 1 # Need to force progress
|
||||
|
||||
lines.append(remaining[:fit_chars])
|
||||
remaining = remaining[fit_chars:]
|
||||
|
||||
continue
|
||||
|
||||
# Check if word fits on current line
|
||||
space_width = 1 if current_line else 0
|
||||
if current_width + space_width + word_width <= width:
|
||||
if current_line:
|
||||
current_line += " " + word
|
||||
current_width += space_width + word_width
|
||||
else:
|
||||
current_line = word
|
||||
current_width = word_width
|
||||
else:
|
||||
# Flush current line and start new one
|
||||
lines.append(current_line)
|
||||
current_line = word
|
||||
current_width = word_width
|
||||
|
||||
# Don't forget the last line
|
||||
if current_line: lines.append(current_line)
|
||||
|
||||
return lines if lines else [""]
|
||||
|
||||
|
||||
def convert_markdown_to_micron(text):
|
||||
converter = MarkdownToMicron()
|
||||
return converter.format_block(text)
|
||||
|
||||
def medium_path_timeout(default_timeout):
|
||||
return max(default_timeout, RNS.Reticulum.get_instance().get_medium_path_timeout())
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -55,19 +55,12 @@ def main():
|
|||
parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
|
||||
parser.add_argument('-v', '--verbose', action='count', default=0)
|
||||
parser.add_argument('-q', '--quiet', action='count', default=0)
|
||||
parser.add_argument("--exampleconfig", action='store_true', default=False, help="print verbose configuration example to stdout and exit")
|
||||
parser.add_argument("--version", action="version", version="rnir {version}".format(version=__version__))
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.exampleconfig:
|
||||
print(__example_rns_config__)
|
||||
exit()
|
||||
|
||||
if args.config:
|
||||
configarg = args.config
|
||||
else:
|
||||
configarg = None
|
||||
if args.config: configarg = args.config
|
||||
else: configarg = None
|
||||
|
||||
program_setup(configdir = configarg, verbosity=args.verbose, quietness=args.quiet)
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -3766,8 +3766,8 @@ def main():
|
|||
print("\nDevice configuration:")
|
||||
if ec_bt == 0x73: print(f" Bluetooth : Enabled")
|
||||
else: print(f" Bluetooth : Disabled")
|
||||
if ec_wifi == 0x01: print(f" WiFi : Enabled (Station)")
|
||||
if ec_wifi == 0x02: print(f" WiFi : Enabled (AP)")
|
||||
if ec_wifi == 0x01: print(f" WiFi : Enabled (Station)")
|
||||
elif ec_wifi == 0x02: print(f" WiFi : Enabled (AP)")
|
||||
else: print(f" WiFi : Disabled")
|
||||
if ec_wifi == 0x01 or ec_wifi == 0x02:
|
||||
if not ec_wchn: print(f" Channel : Unknown")
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -42,6 +42,7 @@ remote_link = None
|
|||
output_rst_str = "\r \r"
|
||||
def connect_remote(destination_hash, auth_identity, timeout, no_output = False, purpose="management"):
|
||||
global remote_link, reticulum
|
||||
timeout = max(timeout, reticulum.get_medium_path_timeout())
|
||||
if not RNS.Transport.has_path(destination_hash):
|
||||
if not no_output:
|
||||
print("Path to "+RNS.prettyhexrep(destination_hash)+" requested", end=" ")
|
||||
|
|
@ -59,7 +60,8 @@ def connect_remote(destination_hash, auth_identity, timeout, no_output = False,
|
|||
remote_identity = RNS.Identity.recall(destination_hash)
|
||||
|
||||
def remote_link_closed(link):
|
||||
if link.teardown_reason == RNS.Link.TIMEOUT:
|
||||
if link.teardown_reason == RNS.Link.INITIATOR_CLOSED: return
|
||||
elif link.teardown_reason == RNS.Link.TIMEOUT:
|
||||
if not no_output:
|
||||
print(output_rst_str, end="")
|
||||
print("The link timed out, exiting now")
|
||||
|
|
@ -99,11 +101,13 @@ def parse_hash(input_str):
|
|||
|
||||
def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity, timeout, drop_queues,
|
||||
drop_via, max_hops, remote=None, management_identity=None, remote_timeout=RNS.Transport.PATH_REQUEST_TIMEOUT,
|
||||
blackholed=False, blackhole=False, unblackhole=False, blackhole_duration=None, blackhole_reason=None,
|
||||
remote_blackhole_list=False, remote_blackhole_list_filter=None, no_output=False, json=False):
|
||||
blackholed=False, blackhole=False, unblackhole=False, blackhole_duration=None, blackhole_reason=None, remote_blackhole_list=False, remote_blackhole_list_filter=None,
|
||||
active=False, usage=False, vias=False, expinfo=False, min_value=None, no_output=False, analyze=False, capture=None, json=False):
|
||||
|
||||
global remote_link, reticulum
|
||||
reticulum = RNS.Reticulum(configdir = configdir, loglevel = 3+verbosity)
|
||||
reticulum.wants_destination_type("rnstransport.remote.management")
|
||||
reticulum.wants_destination_type("rnstransport.info.blackhole")
|
||||
if remote:
|
||||
try:
|
||||
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
|
||||
|
|
@ -238,6 +242,10 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
|
|||
print(f"Could not unblackhole identity: {e}")
|
||||
exit(20)
|
||||
|
||||
elif analyze:
|
||||
analyze_tables(path_table=reticulum.get_path_table(), rate_table=reticulum.get_rate_table(),
|
||||
link_distribution=reticulum.get_link_distribution(), capture=capture)
|
||||
|
||||
elif table:
|
||||
destination_hash = None
|
||||
if destination_hexhash != None:
|
||||
|
|
@ -279,13 +287,35 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
|
|||
exit()
|
||||
|
||||
else:
|
||||
now = time.time()
|
||||
if usage:
|
||||
try: table = sorted(table, key=lambda e: e["used"], reverse=True)
|
||||
except Exception as e: pass
|
||||
|
||||
now = time.time()
|
||||
for path in table:
|
||||
if destination_hash == None or destination_hash == path["hash"]:
|
||||
displayed += 1
|
||||
if min_value:
|
||||
if not path["value"]: continue
|
||||
if path["value"] < min_value: continue
|
||||
|
||||
if path["state"] == RNS.Transport.STATE_RESPONSIVE: state_str = " (R)"
|
||||
elif path["state"] == RNS.Transport.STATE_UNRESPONSIVE: state_str = " (u)"
|
||||
elif path["state"] == RNS.Transport.STATE_UNKNOWN: state_str = " (?)"
|
||||
else: state_str = " (?)"
|
||||
|
||||
exp_str = RNS.timestamp_str(path["expires"])
|
||||
if path["hops"] == 1: m_str = " "
|
||||
else: m_str = "s"
|
||||
print(RNS.prettyhexrep(path["hash"])+" is "+str(path["hops"])+" hop"+m_str+" away via "+RNS.prettyhexrep(path["via"])+" on "+path["interface"]+" expires "+RNS.timestamp_str(path["expires"]))
|
||||
if not "used" in path: u_str = ", no usage info"
|
||||
else:
|
||||
if active and not path["used"]: continue
|
||||
if usage: u_str = f", last used {RNS.prettytime(now-path['used'])} ago" if path["used"] else ", never used"
|
||||
else: u_str = ""
|
||||
via_str = f" via {RNS.prettyhexrep(path['via'])}" if vias else ""
|
||||
exp_str = f" expires in {RNS.prettytime(path['expires']-now, compact=True)}" if expinfo else ""
|
||||
print(f"{RNS.prettyhexrep(path['hash'])} {path['hops']:>3}h{state_str}{via_str} on {path['interface']}{exp_str} (dv:{RNS.prettycost(path['value'])}/lc:{RNS.prettycost(path['lr_cost'])}){u_str}")
|
||||
|
||||
if destination_hash != None and displayed == 0:
|
||||
print("No path known")
|
||||
|
|
@ -450,7 +480,7 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
|
|||
|
||||
i = 0
|
||||
syms = "⢄⢂⢁⡁⡈⡐⡠"
|
||||
limit = time.time()+timeout
|
||||
limit = time.time()+max(timeout, reticulum.get_medium_path_timeout())
|
||||
while not RNS.Transport.has_path(destination_hash) and time.time()<limit:
|
||||
time.sleep(0.1)
|
||||
print(("\b\b"+syms[i]+" "), end="")
|
||||
|
|
@ -482,7 +512,12 @@ def main():
|
|||
parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
|
||||
parser.add_argument("--version", action="version", version="rnpath {version}".format(version=__version__))
|
||||
parser.add_argument("-t", "--table", action="store_true", help="show all known paths", default=False)
|
||||
parser.add_argument("-a", "--active", action="store_true", help="show in-use paths only", default=False)
|
||||
parser.add_argument("-u", "--usage", action="store_true", help="show path usage time", default=False)
|
||||
parser.add_argument("-V", "--vias", action="store_true", help="show via transport IDs", default=False)
|
||||
parser.add_argument("-E", "--expiry", action="store_true", help="show path expiry times", default=False)
|
||||
parser.add_argument("-m", "--max", action="store", metavar="hops", type=int, help="maximum hops to filter path table by", default=None)
|
||||
parser.add_argument("-M", "--min-value", action="store", metavar="value", type=float, help="minimum value to filter path table by", default=None)
|
||||
parser.add_argument("-r", "--rates", action="store_true", help="show announce rate info", default=False)
|
||||
parser.add_argument("-d", "--drop", action="store_true", help="remove the path to a destination", default=False)
|
||||
parser.add_argument("-D", "--drop-announces", action="store_true", help="drop all queued announces", default=False)
|
||||
|
|
@ -494,10 +529,12 @@ def main():
|
|||
parser.add_argument("-b", "--blackholed", action="store_true", help="list blackholed identities", default=False)
|
||||
parser.add_argument("-B", "--blackhole", action="store_true", help="blackhole identity", default=False)
|
||||
parser.add_argument("-U", "--unblackhole", action="store_true", help="unblackhole identity", default=False)
|
||||
parser.add_argument( "--duration", action="store", type=float, help="duration of blackhole enforcement in hours", default=None)
|
||||
parser.add_argument( "--reason", action="store", type=str, help="reason for blackholing identity", default=None)
|
||||
parser.add_argument( "--duration", action="store", metavar="hours", type=float, help="duration of blackhole enforcement in hours", default=None)
|
||||
parser.add_argument( "--reason", action="store", metavar="reason", type=str, help="reason for blackholing identity", default=None)
|
||||
parser.add_argument("-p", "--blackholed-list", action="store_true", help="view published blackhole list for remote transport instance", default=False)
|
||||
parser.add_argument("-j", "--json", action="store_true", help="output in JSON format", default=False)
|
||||
parser.add_argument("-A", "--analyze", action="store_true", help="Analyze tables", default=False)
|
||||
parser.add_argument("--capture", action="store", metavar="path", type=str, help="path to capture file for analyze mode", default=None)
|
||||
parser.add_argument("destination", nargs="?", default=None, help="hexadecimal hash of the destination", type=str)
|
||||
parser.add_argument("list_filter", nargs="?", default=None, help="filter for remote blackhole list view", type=str)
|
||||
parser.add_argument('-v', '--verbose', action='count', default=0)
|
||||
|
|
@ -507,16 +544,17 @@ def main():
|
|||
if args.config: configarg = args.config
|
||||
else: configarg = None
|
||||
|
||||
if not args.drop_announces and not args.table and not args.rates and not args.destination and not args.drop_via and not args.blackholed:
|
||||
if not args.drop_announces and not args.table and not args.rates and not args.destination and not args.drop_via and not args.blackholed and not args.analyze:
|
||||
print("")
|
||||
parser.print_help()
|
||||
print("")
|
||||
else:
|
||||
program_setup(configdir = configarg, table = args.table, rates = args.rates, drop = args.drop, destination_hexhash = args.destination,
|
||||
verbosity = args.verbose, timeout = args.w, drop_queues = args.drop_announces, drop_via = args.drop_via, max_hops = args.max,
|
||||
program_setup(configdir=configarg, table=args.table, rates=args.rates, drop=args.drop, destination_hexhash=args.destination,
|
||||
verbosity=args.verbose, timeout=args.w, drop_queues=args.drop_announces, drop_via=args.drop_via, max_hops=args.max, min_value=args.min_value,
|
||||
remote=args.R, management_identity=args.i, remote_timeout=args.W, blackholed=args.blackholed, blackhole=args.blackhole,
|
||||
unblackhole=args.unblackhole, blackhole_duration=args.duration, blackhole_reason=args.reason, remote_blackhole_list=args.blackholed_list,
|
||||
remote_blackhole_list_filter=args.list_filter, json=args.json)
|
||||
remote_blackhole_list_filter=args.list_filter, usage=args.usage, vias=args.vias, active=args.active, expinfo=args.expiry,
|
||||
analyze=args.analyze, capture=args.capture, json=args.json)
|
||||
|
||||
sys.exit(0)
|
||||
|
||||
|
|
@ -524,6 +562,94 @@ def main():
|
|||
print("")
|
||||
exit()
|
||||
|
||||
def analyze_tables(path_table=None, rate_table=None, link_distribution=None, capture=None):
|
||||
print("Not yet implemented")
|
||||
exit()
|
||||
|
||||
# if not path_table:
|
||||
# print("No path table to analyze")
|
||||
# exit()
|
||||
|
||||
# else:
|
||||
# print(f"Analyzing tables...")
|
||||
|
||||
# # TODO: If a `capture` path is specified, we save and (if existing) load from
|
||||
# # this file, so that `rnpath --analyze` can be run at intervals, and continuously
|
||||
# # collect data, instead of only providing an at-the-moment snapshot view.
|
||||
|
||||
# # Locally initialize announce cache to be able to
|
||||
# # fetch full announce packet data
|
||||
# if RNS.Transport.owner.is_connected_to_shared_instance: RNS.Transport.init_announce_cache()
|
||||
# tm = RNS.Transport.destination_type_map
|
||||
# total = len(path_table)
|
||||
# destinations = {}
|
||||
# unresolvable = 0
|
||||
# unknown_types = set()
|
||||
# known_types = set()
|
||||
# unknown_type_dsts = 0
|
||||
# known_type_dsts = 0
|
||||
# for path in path_table:
|
||||
# dst_hash = path["hash"]
|
||||
# dst_hops = path["hops"]
|
||||
# last_used = path["used"]
|
||||
# name_hash = None
|
||||
# kd_entry = RNS.Identity.known_destinations.get(dst_hash, None)
|
||||
|
||||
# # New destinations may have arrived after our initial known
|
||||
# # destinations snapshot, which is fine, we just count the few
|
||||
# # that miss here for now.
|
||||
# if not kd_entry: unresolvable += 1
|
||||
|
||||
# # Enrich data with information from cached announces
|
||||
# else:
|
||||
# dst_value = kd_entry[6]
|
||||
# dst_lr_cost = kd_entry[7]
|
||||
# announce_hash = kd_entry[1]
|
||||
# announce_packet = RNS.Transport.get_cached_packet(announce_hash, packet_type="announce")
|
||||
# if not announce_packet: unresolvable += 1; continue
|
||||
|
||||
# try: announce_packet.unpack()
|
||||
# except: unresolvable +=1; continue
|
||||
|
||||
# keysize = RNS.Identity.KEYSIZE//8
|
||||
# name_hash_len = RNS.Identity.NAME_HASH_LENGTH//8
|
||||
# name_hash = announce_packet.data[keysize:keysize+name_hash_len]
|
||||
# app_data = kd_entry[3] or RNS.Identity.recall_app_data(dst_hash, _no_use=True)
|
||||
# app_data_len = len(app_data) if app_data else None # No len counting if no app data
|
||||
|
||||
# if name_hash in tm: known_type_dsts += 1; known_types.add(name_hash)
|
||||
# else: unknown_type_dsts += 1; unknown_types.add(name_hash)
|
||||
|
||||
# # TODO: Create actual data structure for use in analysis
|
||||
# entry = {}
|
||||
|
||||
# destinations[dst_hash] = entry
|
||||
|
||||
# if rate_table:
|
||||
# # TODO: Incorporate and enrich with data from rate table
|
||||
# pass
|
||||
|
||||
# if link_distribution:
|
||||
# # TODO: Incorporate and enrich with link distribution data
|
||||
# pass
|
||||
|
||||
# # TODO: Actually analyze tables
|
||||
# pass
|
||||
|
||||
# # TODO: Actually print a comprehensive, beautifully formatted
|
||||
# # overview, charts and data. This is a stub.
|
||||
# print(f"Known types : {len(known_types)}")
|
||||
# print(f"Unknown types : {len(unknown_types)}")
|
||||
# print(f"Total types : {len(known_types)+len(unknown_types)}")
|
||||
# print(f"Known type dests : {known_type_dsts}")
|
||||
# print(f"Unknown type dests : {unknown_type_dsts}")
|
||||
# print(f"Total type dests : {known_type_dsts+unknown_type_dsts}")
|
||||
|
||||
# # TODO: Ensure we get the terminal width if possible, and fall back to
|
||||
# # a sensible default if not, so everything is rendered beautifully.
|
||||
|
||||
# exit()
|
||||
|
||||
def pretty_date(time=False):
|
||||
from datetime import datetime
|
||||
now = datetime.now()
|
||||
|
|
@ -536,9 +662,8 @@ def pretty_date(time=False):
|
|||
if day_diff == 0:
|
||||
if second_diff < 10: return str(second_diff) + " seconds"
|
||||
if second_diff < 60: return str(second_diff) + " seconds"
|
||||
if second_diff < 120: return "1 minute"
|
||||
if second_diff < 3600: return str(int(second_diff / 60)) + " minutes"
|
||||
if second_diff < 7200: return "an hour"
|
||||
if second_diff < 70: return "1 minute"
|
||||
if second_diff < 7200: return str(int(second_diff / 60)) + " minutes"
|
||||
if second_diff < 86400: return str(int(second_diff / 3600)) + " hours"
|
||||
if day_diff == 1: return "1 day"
|
||||
if day_diff < 7: return str(day_diff) + " days"
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -81,7 +81,7 @@ def program_setup(configdir, destination_hexhash, size=None, full_name = None, v
|
|||
print("Path to "+RNS.prettyhexrep(destination_hash)+" requested ", end=" ")
|
||||
sys.stdout.flush()
|
||||
|
||||
_timeout = time.time() + (timeout or DEFAULT_TIMEOUT+reticulum.get_first_hop_timeout(destination_hash))
|
||||
_timeout = time.time() + (timeout or max(DEFAULT_TIMEOUT+reticulum.get_first_hop_timeout(destination_hash), reticulum.get_medium_path_timeout()))
|
||||
i = 0
|
||||
syms = "⢄⢂⢁⡁⡈⡐⡠"
|
||||
while not RNS.Transport.has_path(destination_hash) and not time.time() > _timeout:
|
||||
|
|
@ -131,7 +131,7 @@ def program_setup(configdir, destination_hexhash, size=None, full_name = None, v
|
|||
|
||||
print("\rSent probe "+str(sent)+" ("+str(size)+" bytes) to "+RNS.prettyhexrep(destination_hash)+more+" ", end=" ")
|
||||
|
||||
_timeout = time.time() + (timeout or DEFAULT_TIMEOUT+reticulum.get_first_hop_timeout(destination_hash))
|
||||
_timeout = time.time() + (timeout or max(DEFAULT_TIMEOUT+reticulum.get_first_hop_timeout(destination_hash), reticulum.get_medium_path_timeout()))
|
||||
i = 0
|
||||
while receipt.status == RNS.PacketReceipt.SENT and not time.time() > _timeout:
|
||||
time.sleep(0.1)
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -100,7 +100,7 @@ __example_rns_config__ = '''# This is an example Reticulum config file.
|
|||
# always-on. This directive is optional and can be removed
|
||||
# for brevity.
|
||||
|
||||
enable_transport = No
|
||||
enable_transport = no
|
||||
|
||||
|
||||
# By default, the first program to launch the Reticulum
|
||||
|
|
@ -112,7 +112,7 @@ enable_transport = No
|
|||
# user, and should generally be turned on. This directive
|
||||
# is optional and can be removed for brevity.
|
||||
|
||||
share_instance = Yes
|
||||
share_instance = yes
|
||||
|
||||
|
||||
# If you want to run multiple *different* shared instances
|
||||
|
|
@ -122,6 +122,7 @@ share_instance = Yes
|
|||
|
||||
instance_name = default
|
||||
|
||||
|
||||
# Some platforms don't support domain sockets, and if that
|
||||
# is the case, you can isolate different instances by
|
||||
# specifying a unique set of ports for each:
|
||||
|
|
@ -173,12 +174,18 @@ instance_name = default
|
|||
# network_identity = ~/.reticulum/storage/identity/network
|
||||
|
||||
|
||||
# It is possible to configure the global default interface
|
||||
# gravity. This value will be used if an interface does
|
||||
# not have a specific gravity value in its configuration.
|
||||
|
||||
# default_gravity = 0
|
||||
|
||||
# You can configure whether Reticulum should discover
|
||||
# available interfaces from other Transport Instances over
|
||||
# the network. If this option is enabled, Reticulum will
|
||||
# collect interface information discovered from the network.
|
||||
|
||||
# discover_interfaces = No
|
||||
# discover_interfaces = no
|
||||
|
||||
|
||||
# If you only want to discover interfaces from specific
|
||||
|
|
@ -209,13 +216,40 @@ instance_name = default
|
|||
# required_discovery_value = 14
|
||||
|
||||
|
||||
# You can disable allowing interfaces to be attached, de-
|
||||
# tached or reloaded via rnstatus, for example if running
|
||||
# on a shared system where users with normal RPC access
|
||||
# should not be allowed to control interface states.
|
||||
|
||||
# enable_interface_management = no
|
||||
|
||||
|
||||
# You can configure Reticulum to panic and forcibly close
|
||||
# if an unrecoverable interface error occurs, such as the
|
||||
# hardware device for an interface disappearing. This is
|
||||
# an optional directive, and can be left out for brevity.
|
||||
# This behaviour is disabled by default.
|
||||
|
||||
# panic_on_interface_error = No
|
||||
# panic_on_interface_error = no
|
||||
|
||||
# You can specify which mode discovered interfaces should
|
||||
# be created with when auto-connecting.
|
||||
|
||||
# autoconnect_interface_mode = gw
|
||||
|
||||
|
||||
# It is possible to allow announces from auto-connected
|
||||
# interfaces to propagate announces to internal-mode
|
||||
# interfaces, even if the auto-connected interface's mode
|
||||
# would normally not allow for this.
|
||||
|
||||
# autoconnect_announces_to_internal = yes
|
||||
|
||||
|
||||
# You can configure the gravity for discovered interfaces
|
||||
# that are auto-connected.
|
||||
|
||||
# autoconnect_interface_gravity = 0
|
||||
|
||||
|
||||
# When Transport is enabled, it is possible to allow the
|
||||
|
|
@ -226,14 +260,15 @@ instance_name = default
|
|||
# Transport Instance, and printed to the log at startup.
|
||||
# Optional, and disabled by default.
|
||||
|
||||
# respond_to_probes = No
|
||||
# respond_to_probes = no
|
||||
|
||||
|
||||
# You can publish your local list of blackholed identities
|
||||
# for other transport instances to use for automatic,
|
||||
# network-wide blackhole management.
|
||||
|
||||
# publish_blackhole = No
|
||||
# publish_blackhole = no
|
||||
|
||||
|
||||
# List of remote transport identities from which to auto-
|
||||
# matically source lists of blackholed identities.
|
||||
|
|
@ -249,8 +284,97 @@ instance_name = default
|
|||
# blackhole_sources = 521c87a83afb8f29e4455e77930b973b
|
||||
|
||||
|
||||
# You can set the interval in minutes at which remote
|
||||
# blackhole sources are updated. Defaults to one hour.
|
||||
|
||||
# blackhole_update_interval = 60
|
||||
|
||||
|
||||
# You can enable a hop limiter, that will drop all packets
|
||||
# exceeding the specified maximum hop count.
|
||||
|
||||
# hop_limiter = 16
|
||||
|
||||
|
||||
# When not running as a transport node, it is possible to
|
||||
# force the use of the same, static transport identity
|
||||
# for every instance start. Defaults to using a new identity
|
||||
# at every instance start, if transport is disabled.
|
||||
|
||||
# static_transport_identity = no
|
||||
|
||||
[stamps]
|
||||
# This section configures stamping behaviour and defaults.
|
||||
# In most cases, you do not need to change this, but on
|
||||
# constrained systems especially, tuning these parameters
|
||||
# can be very beneficial
|
||||
|
||||
# During the 1.6.x phased transition to network stamp usage,
|
||||
# it is possible to enable early opt-in on stamp generation
|
||||
# and enforcement. RNS 1.6.0 includes all the required stamp
|
||||
# generation and verification functionality, and actively
|
||||
# validates stamps, while leaving generation, emission and
|
||||
# enforcement disabled by default for destinations and PRs.
|
||||
# Link requests will be stamped already from 1.6.0 if a cost
|
||||
# has been announced by the destination. Version 1.6.1 will
|
||||
# enable generation and emission by default, and version
|
||||
# 1.6.2 will finally activate enforcement. Any of these
|
||||
# options can be enabled early by configuration:
|
||||
|
||||
# stamp_destinations = yes
|
||||
# stamp_prs = yes
|
||||
# enforce_stamps = yes
|
||||
|
||||
|
||||
# The default required stamp cost for an announce, and its
|
||||
# destination to be considered valid. This applies both to
|
||||
# announces received directly, and transported by this node,
|
||||
# if transport is enabled. Any announced destination not
|
||||
# meeting the cost will be ignored.
|
||||
|
||||
# destination_cost = 23.5
|
||||
|
||||
|
||||
# The default value this instance will target when generating
|
||||
# new destination stamps. An application may set a higher
|
||||
# target, but applications requesting a lower value than the
|
||||
# default will always generate at least the default value.
|
||||
# Destination stamps are cached and reused, so a one-time
|
||||
# high-value generation is usually a good idea.
|
||||
|
||||
# destination_value = 25.0
|
||||
|
||||
|
||||
# The default required path request stamp cost. Path requests
|
||||
# that do not meet the requirement are ignored.
|
||||
|
||||
# pr_cost = 15.8
|
||||
|
||||
|
||||
# The default target value of path request stamps generated
|
||||
# by this instance.
|
||||
|
||||
# pr_value = 15.8
|
||||
|
||||
|
||||
# The default link request stamp cost to use, if one has not
|
||||
# been announced by the destination the link is established
|
||||
# to, or if it is unknown. In practice, all destinations will
|
||||
# include their required link cost in each announce, and this
|
||||
# option is rarely necessary.
|
||||
|
||||
# lr_cost = 12.5
|
||||
|
||||
|
||||
# The maximum amount of time to spend attempting to generate
|
||||
# a link request and path request stamps, in seconds.
|
||||
|
||||
# lr_stamp_timeout = 15
|
||||
# pr_stamp_timeout = 15
|
||||
|
||||
|
||||
[logging]
|
||||
# Valid log levels are 0 through 7:
|
||||
# Valid log levels are 0 through 8:
|
||||
# 0: Log only critical information
|
||||
# 1: Log errors and lower log levels
|
||||
# 2: Log warnings and lower log levels
|
||||
|
|
@ -258,10 +382,17 @@ instance_name = default
|
|||
# 4: Log info and lower (this is the default)
|
||||
# 5: Verbose logging
|
||||
# 6: Debug logging
|
||||
# 7: Extreme logging
|
||||
# 7: Path logging
|
||||
# 8: Extreme logging
|
||||
|
||||
loglevel = 4
|
||||
|
||||
# You can disable timestamp inclusion in logs. Useful if
|
||||
# you want to use an external logging tool that provides
|
||||
# its own timestamps or custom formatting.
|
||||
|
||||
# logtimestamps = no
|
||||
|
||||
|
||||
# The interfaces section defines the physical and virtual
|
||||
# interfaces Reticulum will use to communicate on. This
|
||||
|
|
|
|||
49
RNS/Utilities/rnsh/__init__.py
Normal file
49
RNS/Utilities/rnsh/__init__.py
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
# Based on the original rnsh program by Aaron Heise (@acehoss)
|
||||
# https://github.com/acehoss/rnsh - MIT License - Copyright (c) 2023 Aaron Heise
|
||||
# This version of rnsh is included in RNS under the Reticulum License
|
||||
#
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
from ._version import __version__
|
||||
|
||||
import os
|
||||
import glob
|
||||
module_abs_filename = os.path.abspath(__file__)
|
||||
module_dir = os.path.dirname(module_abs_filename)
|
||||
|
||||
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
|
||||
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
|
||||
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
|
||||
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
|
||||
modules = py_modules+pyc_modules+so_modules+pyd_modules
|
||||
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))
|
||||
|
||||
def _get_version(): return __version__
|
||||
1
RNS/Utilities/rnsh/_version.py
Normal file
1
RNS/Utilities/rnsh/_version.py
Normal file
|
|
@ -0,0 +1 @@
|
|||
__version__ = "0.3.0"
|
||||
94
RNS/Utilities/rnsh/args.py
Normal file
94
RNS/Utilities/rnsh/args.py
Normal file
|
|
@ -0,0 +1,94 @@
|
|||
# Based on the original rnsh program by Aaron Heise (@acehoss)
|
||||
# https://github.com/acehoss/rnsh - MIT License - Copyright (c) 2023 Aaron Heise
|
||||
# This version of rnsh is included in RNS under the Reticulum License
|
||||
#
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
|
||||
from RNS.Utilities.rnsh._version import __version__ as __rnsh_version__
|
||||
from RNS._version import __version__
|
||||
|
||||
DEFAULT_SERVICE_NAME = "default"
|
||||
|
||||
def setup_argument_parser():
|
||||
parser = argparse.ArgumentParser(description="Reticulum Remote Shell Utility", epilog="When specifying a command to execute, separate rnsh\noptions from the command and its arguments with --\n\nFor example:\n rnsh -l -- /bin/bash --login\n rnsh <destination> -- ls -la /tmp", formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
|
||||
# Common options
|
||||
parser.add_argument("--config", "-c", action="store", default=None, help="path to config directory", type=str)
|
||||
parser.add_argument("--rnsconfig", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
|
||||
parser.add_argument("--identity", "-i", action="store", default=None, help="path to identity file to use", type=str)
|
||||
parser.add_argument("-v", "--verbose", action="count", default=0, help="increase verbosity")
|
||||
parser.add_argument("-q", "--quiet", action="count", default=0, help="decrease verbosity")
|
||||
parser.add_argument("-p", "--print-identity", action="store_true", default=False, help="print identity and destination info and exit")
|
||||
parser.add_argument("--version", action="version", version="rnsh {rv} (protocol {pv})".format(rv=__version__, pv=__rnsh_version__))
|
||||
|
||||
# Listener options
|
||||
parser.add_argument("-l", "--listen", action="store_true", default=False, help="listen (server) mode; any command specified after -- will be used as the default command when the initiator does not provide one or when remote command execution is disabled; if no command is specified, the default shell of the user running rnsh will be used")
|
||||
parser.add_argument("-s", "--service", action="store", default=None, help="service name for identity file if not the default", type=str)
|
||||
parser.add_argument("-b", "--announce",action="store", default=None,help="announce on startup and every PERIOD seconds; specify 0 to announce on startup only",metavar="PERIOD", type=int)
|
||||
parser.add_argument("-a", "--allowed", action="append", default=None, metavar="HASH", type=str, help="allow this identity to connect (may be specified multiple times); allowed identities can also be specified in ~/.rnsh/allowed_identities or ~/.config/rnsh/allowed_identities, one hash per line")
|
||||
parser.add_argument("-n", "--no-auth", action="store_true", default=False, help="disable authentication (allow any identity to connect)")
|
||||
parser.add_argument("-A", "--remote-command-as-args", action="store_true", default=False, help="concatenate remote command to the argument list of the default program or shell")
|
||||
parser.add_argument("-C", "--no-remote-command", action="store_true", default=False, help="disable executing command lines received from the remote initiator")
|
||||
|
||||
# Initiator options
|
||||
parser.add_argument("-N", "--no-id", action="store_true", default=False, help="disable identity announcement on connect")
|
||||
parser.add_argument("-m", "--mirror", action="store_true", default=False, help="return with the exit code of the remote process")
|
||||
parser.add_argument("-w", "--timeout", action="store", default=None, help="connect and request timeout in seconds", metavar="SECONDS", type=float)
|
||||
|
||||
parser.add_argument("destination", nargs="?", default=None, help="hexadecimal hash of the destination to connect to", type=str)
|
||||
|
||||
return parser
|
||||
|
||||
|
||||
def parse_arguments(argv=None):
|
||||
if argv is None: argv = sys.argv[1:]
|
||||
|
||||
# Split at -- to separate rnsh options from the command to execute.
|
||||
# Everything before -- (or the entire argv if no --) goes to argparse.
|
||||
# Everything after -- becomes the command list.
|
||||
try:
|
||||
split_idx = argv.index("--")
|
||||
rnsh_argv = argv[:split_idx]
|
||||
command = argv[split_idx + 1:]
|
||||
except ValueError:
|
||||
rnsh_argv = argv
|
||||
command = []
|
||||
|
||||
parser = setup_argument_parser()
|
||||
args = parser.parse_args(rnsh_argv)
|
||||
args.command = command
|
||||
|
||||
if args.listen and not args.service: args.service = DEFAULT_SERVICE_NAME
|
||||
|
||||
return args, parser
|
||||
58
RNS/Utilities/rnsh/exception.py
Normal file
58
RNS/Utilities/rnsh/exception.py
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
# Based on the original rnsh program by Aaron Heise (@acehoss)
|
||||
# https://github.com/acehoss/rnsh - MIT License - Copyright (c) 2023 Aaron Heise
|
||||
# This version of rnsh is included in RNS under the Reticulum License
|
||||
#
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
import contextlib
|
||||
from contextlib import AbstractContextManager
|
||||
import logging
|
||||
import sys
|
||||
|
||||
|
||||
class permit(AbstractContextManager):
|
||||
"""Context manager to allow specified exceptions
|
||||
|
||||
The specified exceptions will be allowed to bubble up. Other
|
||||
exceptions are suppressed.
|
||||
|
||||
After a non-matching exception is suppressed, execution proceeds
|
||||
with the next statement following the with statement.
|
||||
|
||||
with allow(KeyboardInterrupt):
|
||||
time.sleep(300)
|
||||
# Execution still resumes here if no KeyboardInterrupt
|
||||
"""
|
||||
|
||||
def __init__(self, *exceptions): self._exceptions = exceptions
|
||||
def __enter__(self): pass
|
||||
def __exit__(self, exctype, excinst, exctb):
|
||||
return exctype is not None and not issubclass(exctype, self._exceptions)
|
||||
58
RNS/Utilities/rnsh/helpers.py
Normal file
58
RNS/Utilities/rnsh/helpers.py
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
# Based on the original rnsh program by Aaron Heise (@acehoss)
|
||||
# https://github.com/acehoss/rnsh - MIT License - Copyright (c) 2023 Aaron Heise
|
||||
# This version of rnsh is included in RNS under the Reticulum License
|
||||
#
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
|
||||
def bitwise_or_if(value: int, condition: bool, orval: int):
|
||||
if not condition: return value
|
||||
return value | orval
|
||||
|
||||
def check_and(value: int, andval: int) -> bool:
|
||||
return (value & andval) > 0
|
||||
|
||||
class SleepRate:
|
||||
def __init__(self, target_period: float):
|
||||
self.target_period = target_period
|
||||
self.last_wake = time.time()
|
||||
|
||||
def next_sleep_time(self) -> float:
|
||||
old_last_wake = self.last_wake
|
||||
self.last_wake = time.time()
|
||||
next_wake = max(old_last_wake + 0.01, self.last_wake)
|
||||
sleep_for = next_wake - self.last_wake
|
||||
return sleep_for if sleep_for > 0 else 0
|
||||
|
||||
async def sleep_async(self): await asyncio.sleep(self.next_sleep_time())
|
||||
def sleep_block(self): time.sleep(self.next_sleep_time())
|
||||
457
RNS/Utilities/rnsh/initiator.py
Normal file
457
RNS/Utilities/rnsh/initiator.py
Normal file
|
|
@ -0,0 +1,457 @@
|
|||
# Based on the original rnsh program by Aaron Heise (@acehoss)
|
||||
# https://github.com/acehoss/rnsh - MIT License - Copyright (c) 2023 Aaron Heise
|
||||
# This version of rnsh is included in RNS under the Reticulum License
|
||||
#
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import enum
|
||||
import functools
|
||||
import os
|
||||
import queue
|
||||
import shlex
|
||||
import signal
|
||||
import sys
|
||||
import termios
|
||||
import threading
|
||||
import time
|
||||
import tty
|
||||
from typing import Callable, TypeVar
|
||||
import RNS
|
||||
import RNS.Utilities.rnsh.exception as exception
|
||||
import RNS.Utilities.rnsh.process as process
|
||||
import RNS.Utilities.rnsh.retry as retry
|
||||
import RNS.Utilities.rnsh.session as session
|
||||
import contextlib
|
||||
|
||||
import bz2
|
||||
import RNS.Utilities.rnsh.protocol as protocol
|
||||
import RNS.Utilities.rnsh.helpers as helpers
|
||||
import RNS.Utilities.rnsh.rnsh as rnsh
|
||||
|
||||
_identity = None
|
||||
_reticulum = None
|
||||
_cmd: [str] | None = None
|
||||
DATA_AVAIL_MSG = "data available"
|
||||
_finished: asyncio.Event = None
|
||||
_retry_timer: retry.RetryThread | None = None
|
||||
_destination: RNS.Destination | None = None
|
||||
_loop: asyncio.AbstractEventLoop | None = None
|
||||
|
||||
async def _check_finished(timeout: float = 0):
|
||||
return _finished is not None and await process.event_wait(_finished, timeout=timeout)
|
||||
|
||||
def _sigint_handler(sig, loop):
|
||||
global _finished
|
||||
RNS.log(f"{signal.Signals(sig).name}", RNS.LOG_DEBUG)
|
||||
if _finished is not None: _finished.set()
|
||||
else: raise KeyboardInterrupt()
|
||||
|
||||
async def _spin_tty(until=None, msg=None, timeout=None):
|
||||
i = 0
|
||||
syms = "⢄⢂⢁⡁⡈⡐⡠"
|
||||
if timeout != None: timeout = time.time()+timeout
|
||||
|
||||
print(msg+" ", end=" ")
|
||||
while (timeout == None or time.time()<timeout) and not until():
|
||||
await asyncio.sleep(0.1)
|
||||
print(("\b\b"+syms[i]+" "), end="")
|
||||
sys.stdout.flush()
|
||||
i = (i+1)%len(syms)
|
||||
print("\r"+" "*len(msg)+" \r", end="")
|
||||
|
||||
if timeout != None and time.time() > timeout: return False
|
||||
else: return True
|
||||
|
||||
async def _spin_pipe(until: callable = None, msg=None, timeout: float | None = None) -> bool:
|
||||
if timeout is not None: timeout += time.time()
|
||||
|
||||
while (timeout is None or time.time() < timeout) and not until():
|
||||
if await _check_finished(0.1): raise asyncio.CancelledError()
|
||||
|
||||
if timeout is not None and time.time() > timeout: return False
|
||||
else: return True
|
||||
|
||||
async def _spin(until: callable = None, msg=None, timeout: float | None = None, quiet: bool = False) -> bool:
|
||||
if not quiet and os.isatty(1): return await _spin_tty(until, msg, timeout)
|
||||
else: return await _spin_pipe(until, msg, timeout)
|
||||
|
||||
_link: RNS.Link | None = None
|
||||
_remote_exec_grace = 2.0
|
||||
_pq = queue.Queue()
|
||||
|
||||
|
||||
class InitiatorState(enum.IntEnum):
|
||||
IS_INITIAL = 0
|
||||
IS_LINKED = 1
|
||||
IS_WAIT_VERS = 2
|
||||
IS_RUNNING = 3
|
||||
IS_TERMINATE = 4
|
||||
IS_TEARDOWN = 5
|
||||
|
||||
def _client_link_closed(link):
|
||||
if _finished: _finished.set()
|
||||
|
||||
def _client_message_handler(message: RNS.MessageBase): _pq.put(message)
|
||||
|
||||
def compute_target_rns_loglevel(verbosity: int, quietness: int, base_level: int = RNS.LOG_INFO) -> int:
|
||||
try:
|
||||
target = int(base_level) + int(verbosity) - int(quietness)
|
||||
if target < RNS.LOG_NONE: target = RNS.LOG_NONE
|
||||
if target > RNS.LOG_EXTREME: target = RNS.LOG_EXTREME
|
||||
return target
|
||||
except Exception: return base_level
|
||||
|
||||
class RemoteExecutionError(Exception):
|
||||
def __init__(self, msg): self.msg = msg
|
||||
|
||||
async def _initiate_link(configdir, rnsconfigdir, identitypath=None, logfile=None, verbosity=0, quietness=0, noid=False, destination=None,
|
||||
timeout=RNS.Transport.PATH_REQUEST_TIMEOUT):
|
||||
global _identity, _reticulum, _link, _destination, _remote_exec_grace
|
||||
|
||||
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH // 8) * 2
|
||||
if len(destination) != dest_len:
|
||||
raise RemoteExecutionError(
|
||||
"Allowed destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(
|
||||
hex=dest_len, byte=dest_len // 2))
|
||||
|
||||
try: destination_hash = bytes.fromhex(destination)
|
||||
except Exception as e: raise RemoteExecutionError("Invalid destination entered. Check your input.")
|
||||
|
||||
if _reticulum is None:
|
||||
targetloglevel = compute_target_rns_loglevel(verbosity, quietness, RNS.LOG_INFO)
|
||||
RNS.logfile = logfile
|
||||
_reticulum = RNS.Reticulum(configdir=rnsconfigdir, loglevel=targetloglevel, logdest=RNS.LOG_FILE)
|
||||
|
||||
_reticulum.wants_destination_type(f"{rnsh.APP_NAME}")
|
||||
|
||||
if _identity is None:
|
||||
_identity = rnsh.prepare_identity(identity_path=identitypath, service_name=None, configdir=configdir)
|
||||
|
||||
if not RNS.Transport.has_path(destination_hash):
|
||||
RNS.Transport.request_path(destination_hash)
|
||||
RNS.log(f"Requesting path to {RNS.prettyhexrep(destination_hash)}", RNS.LOG_INFO)
|
||||
if not await _spin(until=lambda: RNS.Transport.has_path(destination_hash), msg=f"Requesting path to {RNS.prettyhexrep(destination_hash)}",
|
||||
timeout=timeout, quiet=quietness > 0):
|
||||
raise RemoteExecutionError("Path not found")
|
||||
|
||||
if _destination is None:
|
||||
listener_identity = RNS.Identity.recall(destination_hash)
|
||||
_destination = RNS.Destination(listener_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, rnsh.APP_NAME)
|
||||
|
||||
RNS.log(f"Establishing link with {RNS.prettyhexrep(destination_hash)}", RNS.LOG_INFO)
|
||||
if _link is None or _link.status == RNS.Link.PENDING:
|
||||
_link = RNS.Link(_destination)
|
||||
_link.did_identify = False
|
||||
|
||||
_link.set_link_closed_callback(_client_link_closed)
|
||||
|
||||
if not await _spin(until=lambda: _link.status == RNS.Link.ACTIVE, msg=f"Establishing link with {RNS.prettyhexrep(destination_hash)}",
|
||||
timeout=timeout, quiet=quietness > 0):
|
||||
raise RemoteExecutionError("Could not establish link with " + RNS.prettyhexrep(destination_hash))
|
||||
|
||||
RNS.log(f"Link established with {RNS.prettyhexrep(destination_hash)}", RNS.LOG_INFO)
|
||||
if not noid and not _link.did_identify:
|
||||
# Delay a tiny bit to allow listener to fully enter WAIT_IDENT state
|
||||
await asyncio.sleep(min(1, _link.rtt * 1.1 + 0.05))
|
||||
_link.identify(_identity)
|
||||
_link.did_identify = True
|
||||
|
||||
async def _handle_error(errmsg: RNS.MessageBase):
|
||||
if isinstance(errmsg, protocol.ErrorMessage):
|
||||
with contextlib.suppress(Exception):
|
||||
if _link and _link.status == RNS.Link.ACTIVE: _link.teardown()
|
||||
await asyncio.sleep(0.1)
|
||||
raise RemoteExecutionError(f"Remote error: {errmsg.msg}")
|
||||
|
||||
async def initiate(configdir: str, rnsconfigdir:str, identitypath: str, logfile:str, verbosity: int, quietness: int, noid: bool, destination: str,
|
||||
timeout: float, command: [str] | None = None):
|
||||
|
||||
global _finished, _link
|
||||
if timeout is None: timeout = RNS.Transport.PATH_REQUEST_TIMEOUT
|
||||
with process.TTYRestorer(sys.stdin.fileno()) as ttyRestorer:
|
||||
loop = asyncio.get_running_loop()
|
||||
state = InitiatorState.IS_INITIAL
|
||||
data_buffer = bytearray(sys.stdin.buffer.read()) if not os.isatty(sys.stdin.fileno()) else bytearray()
|
||||
line_buffer = bytearray()
|
||||
|
||||
await _initiate_link(configdir=configdir,
|
||||
rnsconfigdir=rnsconfigdir,
|
||||
identitypath=identitypath,
|
||||
logfile=logfile,
|
||||
verbosity=verbosity,
|
||||
quietness=quietness,
|
||||
noid=noid,
|
||||
destination=destination,
|
||||
timeout=timeout)
|
||||
|
||||
if not _link or _link.status not in [RNS.Link.ACTIVE, RNS.Link.PENDING]: return 255
|
||||
|
||||
state = InitiatorState.IS_LINKED
|
||||
outlet = session.RNSOutlet(_link)
|
||||
channel = _link.get_channel()
|
||||
protocol.register_message_types(channel)
|
||||
channel.add_message_handler(_client_message_handler)
|
||||
channel.send(protocol.VersionInfoMessage())
|
||||
try:
|
||||
vm = _pq.get(timeout=max(outlet.rtt * 20, 5))
|
||||
await _handle_error(vm)
|
||||
if not isinstance(vm, protocol.VersionInfoMessage): raise Exception("Invalid message received")
|
||||
RNS.log(f"Connected server version info: sw {vm.sw_version}, proto {vm.protocol_version}", RNS.LOG_INFO)
|
||||
state = InitiatorState.IS_RUNNING
|
||||
except queue.Empty:
|
||||
print("Protocol error")
|
||||
return 254
|
||||
|
||||
winch = False
|
||||
def sigwinch_handler():
|
||||
nonlocal winch
|
||||
winch = True
|
||||
|
||||
esc = False
|
||||
pre_esc = True
|
||||
line_mode = False
|
||||
line_flush = False
|
||||
blind_write_count = 0
|
||||
flush_chars = ["\x01", "\x03", "\x04", "\x05", "\x0c", "\x11", "\x13", "\x15", "\x19", "\t", "\x1A", "\x1B"]
|
||||
def handle_escape(b):
|
||||
nonlocal line_mode
|
||||
if b == "?":
|
||||
os.write(1, "\n\r\n\rSupported rnsh escape sequences:".encode("utf-8"))
|
||||
os.write(1, "\n\r ~~ Send the escape character by typing it twice".encode("utf-8"))
|
||||
os.write(1, "\n\r ~. Terminate session and exit immediately".encode("utf-8"))
|
||||
os.write(1, "\n\r ~L Toggle line-interactive mode".encode("utf-8"))
|
||||
os.write(1, "\n\r ~? Display this quick reference\n\r".encode("utf-8"))
|
||||
os.write(1, "\n\r(Escape sequences are only recognized immediately after newline)\n\r".encode("utf-8"))
|
||||
return None
|
||||
elif b == ".":
|
||||
_link.teardown()
|
||||
return None
|
||||
elif b == "L":
|
||||
line_mode = not line_mode
|
||||
if line_mode: os.write(1, "\n\rLine-interactive mode enabled\n\r".encode("utf-8"))
|
||||
else: os.write(1, "\n\rLine-interactive mode disabled\n\r".encode("utf-8"))
|
||||
return None
|
||||
|
||||
return b
|
||||
|
||||
stdin_eof = False
|
||||
def stdin():
|
||||
nonlocal stdin_eof, pre_esc, esc, line_mode
|
||||
nonlocal line_flush, blind_write_count
|
||||
try:
|
||||
in_data = process.tty_read(sys.stdin.fileno())
|
||||
if in_data is not None:
|
||||
data = bytearray()
|
||||
for b in bytes(in_data):
|
||||
c = chr(b)
|
||||
if c == "\r":
|
||||
pre_esc = True
|
||||
line_flush = True
|
||||
data.append(b)
|
||||
elif line_mode and c in flush_chars:
|
||||
pre_esc = False
|
||||
line_flush = True
|
||||
data.append(b)
|
||||
elif line_mode and (c == "\b" or c == "\x7f"):
|
||||
pre_esc = False
|
||||
if len(line_buffer)>0:
|
||||
line_buffer.pop(-1)
|
||||
blind_write_count -= 1
|
||||
os.write(1, "\b \b".encode("utf-8"))
|
||||
elif pre_esc == True and c == "~":
|
||||
pre_esc = False
|
||||
esc = True
|
||||
elif esc == True:
|
||||
ret = handle_escape(c)
|
||||
if ret != None:
|
||||
if ret != "~":
|
||||
data.append(ord("~"))
|
||||
data.append(ord(ret))
|
||||
esc = False
|
||||
else:
|
||||
pre_esc = False
|
||||
data.append(b)
|
||||
|
||||
if not line_mode: data_buffer.extend(data)
|
||||
else:
|
||||
line_buffer.extend(data)
|
||||
if line_flush:
|
||||
data_buffer.extend(line_buffer)
|
||||
line_buffer.clear()
|
||||
os.write(1, ("\b \b"*blind_write_count).encode("utf-8"))
|
||||
line_flush = False
|
||||
blind_write_count = 0
|
||||
else:
|
||||
os.write(1, data)
|
||||
blind_write_count += len(data)
|
||||
|
||||
except EOFError:
|
||||
if os.isatty(0): data_buffer.extend(process.CTRL_D)
|
||||
stdin_eof = True
|
||||
process.tty_unset_reader_callbacks(sys.stdin.fileno())
|
||||
|
||||
process.tty_add_reader_callback(sys.stdin.fileno(), stdin)
|
||||
|
||||
tcattr = None
|
||||
rows, cols, hpix, vpix = (None, None, None, None)
|
||||
try:
|
||||
tcattr = termios.tcgetattr(0)
|
||||
rows, cols, hpix, vpix = process.tty_get_winsize(0)
|
||||
except:
|
||||
try:
|
||||
tcattr = termios.tcgetattr(1)
|
||||
rows, cols, hpix, vpix = process.tty_get_winsize(1)
|
||||
except:
|
||||
try:
|
||||
tcattr = termios.tcgetattr(2)
|
||||
rows, cols, hpix, vpix = process.tty_get_winsize(2)
|
||||
except:
|
||||
pass
|
||||
|
||||
await _spin(lambda: channel.is_ready_to_send(), "Waiting for channel...", 1, quietness > 0)
|
||||
channel.send(protocol.ExecuteCommandMesssage(cmdline=command,
|
||||
pipe_stdin=not os.isatty(0),
|
||||
pipe_stdout=not os.isatty(1),
|
||||
pipe_stderr=not os.isatty(2),
|
||||
tcflags=tcattr,
|
||||
term=os.environ.get("TERM", None),
|
||||
rows=rows,
|
||||
cols=cols,
|
||||
hpix=hpix,
|
||||
vpix=vpix))
|
||||
|
||||
loop.add_signal_handler(signal.SIGWINCH, sigwinch_handler)
|
||||
_finished = asyncio.Event()
|
||||
loop.add_signal_handler(signal.SIGINT, functools.partial(_sigint_handler, signal.SIGINT, loop))
|
||||
loop.add_signal_handler(signal.SIGTERM, functools.partial(_sigint_handler, signal.SIGTERM, loop))
|
||||
mdu = _link.MDU - 16
|
||||
sent_eof = False
|
||||
last_winch = time.time()
|
||||
sleeper = helpers.SleepRate(0.01)
|
||||
processed = False
|
||||
while not await _check_finished() and state in [InitiatorState.IS_RUNNING]:
|
||||
try:
|
||||
try:
|
||||
message = _pq.get(timeout=sleeper.next_sleep_time() if not processed else 0.0005)
|
||||
await _handle_error(message)
|
||||
processed = True
|
||||
if isinstance(message, protocol.StreamDataMessage):
|
||||
if message.stream_id == protocol.StreamDataMessage.STREAM_ID_STDOUT:
|
||||
if message.data and len(message.data) > 0:
|
||||
ttyRestorer.raw()
|
||||
# RNS.log(f"stdout: {message.data}", RNS.LOG_EXTREME)
|
||||
os.write(1, message.data)
|
||||
sys.stdout.flush()
|
||||
if message.eof:
|
||||
os.close(1)
|
||||
if message.stream_id == protocol.StreamDataMessage.STREAM_ID_STDERR:
|
||||
if message.data and len(message.data) > 0:
|
||||
ttyRestorer.raw()
|
||||
# RNS.log(f"stdout: {message.data}", RNS.LOG_EXTREME)
|
||||
os.write(2, message.data)
|
||||
sys.stderr.flush()
|
||||
if message.eof:
|
||||
os.close(2)
|
||||
elif isinstance(message, protocol.CommandExitedMessage):
|
||||
RNS.log(f"received return code {message.return_code}, exiting", RNS.LOG_DEBUG)
|
||||
return message.return_code
|
||||
elif isinstance(message, protocol.ErrorMessage):
|
||||
RNS.log(f"Remote error: {message.data}", RNS.LOG_ERROR)
|
||||
if message.fatal:
|
||||
_link.teardown()
|
||||
return 200
|
||||
|
||||
except queue.Empty: processed = False
|
||||
|
||||
if channel.is_ready_to_send():
|
||||
def compress_adaptive(buf: bytes):
|
||||
comp_tries = RNS.RawChannelWriter.COMPRESSION_TRIES
|
||||
comp_try = 1
|
||||
comp_success = False
|
||||
|
||||
chunk_len = len(buf)
|
||||
if chunk_len > RNS.RawChannelWriter.MAX_CHUNK_LEN:
|
||||
chunk_len = RNS.RawChannelWriter.MAX_CHUNK_LEN
|
||||
chunk_segment = None
|
||||
|
||||
chunk_segment = None
|
||||
max_data_len = channel.mdu - protocol.StreamDataMessage.HEADER_LEN
|
||||
while chunk_len > 32 and comp_try < comp_tries:
|
||||
chunk_segment_length = int(chunk_len/comp_try)
|
||||
compressed_chunk = bz2.compress(buf[:chunk_segment_length])
|
||||
compressed_length = len(compressed_chunk)
|
||||
if compressed_length < max_data_len and compressed_length < chunk_segment_length:
|
||||
comp_success = True
|
||||
break
|
||||
else: comp_try += 1
|
||||
|
||||
if comp_success:
|
||||
diff = max_data_len - len(compressed_chunk)
|
||||
chunk = compressed_chunk
|
||||
processed_length = chunk_segment_length
|
||||
else:
|
||||
chunk = bytes(buf[:max_data_len])
|
||||
processed_length = len(chunk)
|
||||
|
||||
return comp_success, processed_length, chunk
|
||||
|
||||
comp_success, processed_length, chunk = compress_adaptive(data_buffer)
|
||||
stdin = chunk
|
||||
data_buffer = data_buffer[processed_length:]
|
||||
eof = not sent_eof and stdin_eof and len(stdin) == 0
|
||||
if len(stdin) > 0 or eof:
|
||||
channel.send(protocol.StreamDataMessage(protocol.StreamDataMessage.STREAM_ID_STDIN, stdin, eof, comp_success))
|
||||
sent_eof = eof
|
||||
processed = True
|
||||
|
||||
# send window change, but rate limited
|
||||
if winch and time.time() - last_winch > _link.rtt * 25:
|
||||
last_winch = time.time()
|
||||
winch = False
|
||||
with contextlib.suppress(Exception):
|
||||
r, c, h, v = process.tty_get_winsize(0)
|
||||
channel.send(protocol.WindowSizeMessage(r, c, h, v))
|
||||
processed = True
|
||||
|
||||
except RemoteExecutionError as e:
|
||||
print(e.msg)
|
||||
return 255
|
||||
except Exception as ex:
|
||||
print(f"Client exception: {ex}")
|
||||
if _link and _link.status != RNS.Link.CLOSED:
|
||||
_link.teardown()
|
||||
return 127
|
||||
|
||||
RNS.log("Main loop done", RNS.LOG_DEBUG)
|
||||
return 0
|
||||
234
RNS/Utilities/rnsh/listener.py
Normal file
234
RNS/Utilities/rnsh/listener.py
Normal file
|
|
@ -0,0 +1,234 @@
|
|||
# Based on the original rnsh program by Aaron Heise (@acehoss)
|
||||
# https://github.com/acehoss/rnsh - MIT License - Copyright (c) 2023 Aaron Heise
|
||||
# This version of rnsh is included in RNS under the Reticulum License
|
||||
#
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import queue
|
||||
import shlex
|
||||
import signal
|
||||
import sys
|
||||
import termios
|
||||
import threading
|
||||
import time
|
||||
import tty
|
||||
from typing import Callable, TypeVar
|
||||
import RNS
|
||||
import RNS.Utilities.rnsh.exception as exception
|
||||
import RNS.Utilities.rnsh.process as process
|
||||
import RNS.Utilities.rnsh.retry as retry
|
||||
import RNS.Utilities.rnsh.session as session
|
||||
import contextlib
|
||||
|
||||
import pwd
|
||||
import RNS.Utilities.rnsh.protocol as protocol
|
||||
import RNS.Utilities.rnsh.helpers as helpers
|
||||
import RNS.Utilities.rnsh.rnsh as rnsh
|
||||
|
||||
|
||||
_identity = None
|
||||
_reticulum = None
|
||||
_allow_all = False
|
||||
_allowed_file = None
|
||||
_allowed_identity_hashes = []
|
||||
_allowed_file_identity_hashes = []
|
||||
_cmd: [str] | None = None
|
||||
DATA_AVAIL_MSG = "data available"
|
||||
_finished: asyncio.Event = None
|
||||
_retry_timer: retry.RetryThread | None = None
|
||||
_destination: RNS.Destination | None = None
|
||||
_loop: asyncio.AbstractEventLoop | None = None
|
||||
_no_remote_command = True
|
||||
_remote_cmd_as_args = False
|
||||
|
||||
|
||||
async def _check_finished(timeout: float = 0):
|
||||
return await process.event_wait(_finished, timeout=timeout)
|
||||
|
||||
|
||||
def _sigint_handler(sig, loop):
|
||||
global _finished
|
||||
RNS.log(f"Signal: {signal.Signals(sig).name}", RNS.LOG_DEBUG)
|
||||
if _finished is not None: _finished.set()
|
||||
else: raise KeyboardInterrupt()
|
||||
|
||||
def _reload_allowed_file():
|
||||
global _allowed_file, _allowed_file_identity_hashes
|
||||
if _allowed_file != None:
|
||||
try:
|
||||
with open(_allowed_file, "r") as file:
|
||||
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH // 8) * 2
|
||||
added = 0
|
||||
line = 0
|
||||
_allowed_file_identity_hashes = []
|
||||
for allow in file.read().replace("\r", "").split("\n"):
|
||||
line += 1
|
||||
if len(allow) == dest_len:
|
||||
try:
|
||||
destination_hash = bytes.fromhex(allow)
|
||||
_allowed_file_identity_hashes.append(destination_hash)
|
||||
added += 1
|
||||
except Exception:
|
||||
RNS.log(f"Discarded invalid Identity hash in {_allowed_file} at line {line}", RNS.LOG_DEBUG)
|
||||
|
||||
ms = "y" if added == 1 else "ies"
|
||||
RNS.log(f"Loaded {added} allowed identit{ms} from "+str(_allowed_file), RNS.LOG_DEBUG)
|
||||
|
||||
except Exception as e: RNS.log(f"Error while reloading allowed indetities file: {e}", RNS.LOG_ERROR)
|
||||
|
||||
def compute_target_rns_loglevel(verbosity: int, quietness: int, base_level: int = RNS.LOG_INFO) -> int:
|
||||
try:
|
||||
target = int(base_level) + int(verbosity) - int(quietness)
|
||||
if target < RNS.LOG_NONE: target = RNS.LOG_NONE
|
||||
if target > RNS.LOG_EXTREME: target = RNS.LOG_EXTREME
|
||||
return target
|
||||
|
||||
except Exception: return base_level
|
||||
|
||||
async def listen(configdir, rnsconfigdir, command, identitypath=None, logfile=None, service_name=None, verbosity=0, quietness=0,
|
||||
allowed=None, allowed_file=None, disable_auth=None, announce_period=900, no_remote_command=True, remote_cmd_as_args=False,
|
||||
loop: asyncio.AbstractEventLoop = None):
|
||||
global _identity, _allow_all, _allowed_identity_hashes, _allowed_file, _allowed_file_identity_hashes
|
||||
global _reticulum, _cmd, _destination, _no_remote_command, _remote_cmd_as_args, _finished
|
||||
|
||||
if not loop: loop = asyncio.get_running_loop()
|
||||
if service_name is None or len(service_name) == 0: service_name = "default"
|
||||
|
||||
RNS.log(f"Using service name {service_name}", RNS.LOG_INFO)
|
||||
if logfile:
|
||||
RNS.log(f"Logging to {logfile}", RNS.LOG_NOTICE)
|
||||
logdest = RNS.LOG_FILE
|
||||
RNS.logfile = logfile
|
||||
else:
|
||||
RNS.log(f"Logging to console", RNS.LOG_NOTICE)
|
||||
logdest = RNS.LOG_STDOUT
|
||||
|
||||
# More -v should increase verbosity (higher RNS.loglevel); -q should decrease it
|
||||
targetloglevel = compute_target_rns_loglevel(verbosity, quietness, RNS.LOG_INFO)
|
||||
_reticulum = RNS.Reticulum(configdir=rnsconfigdir, loglevel=targetloglevel)
|
||||
_identity = rnsh.prepare_identity(identity_path=identitypath, service_name=service_name, configdir=configdir)
|
||||
_destination = RNS.Destination(_identity, RNS.Destination.IN, RNS.Destination.SINGLE, rnsh.APP_NAME)
|
||||
|
||||
RNS.log(f"rnsh listening for commands on {RNS.prettyhexrep(_destination.hash)}", RNS.LOG_NOTICE)
|
||||
RNS.logdest = logdest
|
||||
|
||||
_cmd = command
|
||||
if _cmd is None or len(_cmd) == 0:
|
||||
shell = None
|
||||
try: shell = pwd.getpwuid(os.getuid()).pw_shell
|
||||
except Exception as e: RNS.log(f"Error looking up shell: {e}", RNS.LOG_ERROR)
|
||||
RNS.log(f"Using {shell} for default command.", RNS.LOG_INFO)
|
||||
|
||||
# Ensure a sane shell default. Fall back to /bin/sh if lookup fails.
|
||||
if not shell or len(shell) == 0: shell = "/bin/sh"
|
||||
_cmd = [shell]
|
||||
|
||||
else: RNS.log(f"Using command {shlex.join(_cmd)}", RNS.LOG_INFO)
|
||||
|
||||
_no_remote_command = no_remote_command
|
||||
session.ListenerSession.allow_remote_command = not no_remote_command
|
||||
_remote_cmd_as_args = remote_cmd_as_args
|
||||
if (_cmd is None or len(_cmd) == 0 or _cmd[0] is None or len(_cmd[0]) == 0) and (_no_remote_command or _remote_cmd_as_args):
|
||||
raise Exception(f"Unable to look up shell for {os.getlogin()}, cannot proceed with -A or -C and no <program>.")
|
||||
|
||||
session.ListenerSession.default_command = _cmd
|
||||
session.ListenerSession.remote_cmd_as_args = _remote_cmd_as_args
|
||||
|
||||
if disable_auth:
|
||||
_allow_all = True
|
||||
session.ListenerSession.allow_all = True
|
||||
else:
|
||||
if allowed_file is not None:
|
||||
_allowed_file = allowed_file
|
||||
_reload_allowed_file()
|
||||
|
||||
if allowed is not None:
|
||||
for a in allowed:
|
||||
try:
|
||||
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH // 8) * 2
|
||||
if len(a) != dest_len:
|
||||
raise ValueError(
|
||||
"Allowed destination length is invalid, must be {hex} hexadecimal " +
|
||||
"characters ({byte} bytes).".format(
|
||||
hex=dest_len, byte=dest_len // 2))
|
||||
try:
|
||||
destination_hash = bytes.fromhex(a)
|
||||
_allowed_identity_hashes.append(destination_hash)
|
||||
session.ListenerSession.allowed_identity_hashes.append(destination_hash)
|
||||
except Exception:
|
||||
raise ValueError("Invalid destination entered. Check your input.")
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Unhandled error: {e}", RNS.LOG_ERROR)
|
||||
RNS.trace_exception(e)
|
||||
exit(1)
|
||||
|
||||
if (len(_allowed_identity_hashes) < 1 and len(_allowed_file_identity_hashes) < 1) and not disable_auth:
|
||||
RNS.log("Warning: No allowed identities configured, rnsh will not accept any connections!", RNS.LOG_WARNING)
|
||||
|
||||
def link_established(lnk: RNS.Link):
|
||||
_reload_allowed_file()
|
||||
session.ListenerSession.allowed_file_identity_hashes = _allowed_file_identity_hashes
|
||||
session.ListenerSession(session.RNSOutlet.get_outlet(lnk), lnk.get_channel(), loop)
|
||||
_destination.set_link_established_callback(link_established)
|
||||
|
||||
_finished = asyncio.Event()
|
||||
signal.signal(signal.SIGINT, _sigint_handler)
|
||||
|
||||
if announce_period is not None: _destination.announce()
|
||||
|
||||
last_announce = time.time()
|
||||
sleeper = helpers.SleepRate(0.01)
|
||||
|
||||
try:
|
||||
while not await _check_finished():
|
||||
if announce_period and 0 < announce_period < time.time() - last_announce:
|
||||
last_announce = time.time()
|
||||
_destination.announce()
|
||||
if len(session.ListenerSession.sessions) > 0:
|
||||
# no sleep if there's work to do
|
||||
if not await session.ListenerSession.pump_all():
|
||||
await sleeper.sleep_async()
|
||||
else:
|
||||
await asyncio.sleep(0.25)
|
||||
finally:
|
||||
RNS.log("Shutting down", RNS.LOG_NOTICE)
|
||||
await session.ListenerSession.terminate_all("Shutting down")
|
||||
await asyncio.sleep(1)
|
||||
links_still_active = list(filter(lambda l: l.status != RNS.Link.CLOSED, _destination.links))
|
||||
for link in links_still_active:
|
||||
if link.status not in [RNS.Link.CLOSED]:
|
||||
link.teardown()
|
||||
await asyncio.sleep(0.01)
|
||||
46
RNS/Utilities/rnsh/loop.py
Normal file
46
RNS/Utilities/rnsh/loop.py
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
# Based on the original rnsh program by Aaron Heise (@acehoss)
|
||||
# https://github.com/acehoss/rnsh - MIT License - Copyright (c) 2023 Aaron Heise
|
||||
# This version of rnsh is included in RNS under the Reticulum License
|
||||
#
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
import asyncio
|
||||
import functools
|
||||
from typing import Callable
|
||||
|
||||
def sig_handler_sys_to_loop(handler: Callable[[int, any], None]) -> Callable[[int, asyncio.AbstractEventLoop], None]:
|
||||
def wrapped(cb: Callable[[int, any], None], signal: int, loop: asyncio.AbstractEventLoop): cb(signal, None)
|
||||
return functools.partial(wrapped, handler)
|
||||
|
||||
def loop_set_signal(sig, handler: Callable[[int, asyncio.AbstractEventLoop], None], loop: asyncio.AbstractEventLoop = None):
|
||||
if loop is None: loop = asyncio.get_running_loop()
|
||||
loop.remove_signal_handler(sig)
|
||||
loop.add_signal_handler(sig, functools.partial(handler, sig, loop))
|
||||
769
RNS/Utilities/rnsh/process.py
Normal file
769
RNS/Utilities/rnsh/process.py
Normal file
|
|
@ -0,0 +1,769 @@
|
|||
# Based on the original rnsh program by Aaron Heise (@acehoss)
|
||||
# https://github.com/acehoss/rnsh - MIT License - Copyright (c) 2023 Aaron Heise
|
||||
# This version of rnsh is included in RNS under the Reticulum License
|
||||
#
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
from __future__ import annotations
|
||||
import asyncio
|
||||
import contextlib
|
||||
import copy
|
||||
import errno
|
||||
import fcntl
|
||||
import functools
|
||||
import os
|
||||
import pty
|
||||
import select
|
||||
import signal
|
||||
import struct
|
||||
import sys
|
||||
import termios
|
||||
import threading
|
||||
import tty
|
||||
import types
|
||||
import typing
|
||||
import RNS
|
||||
|
||||
import RNS.Utilities.rnsh.exception as exception
|
||||
|
||||
CTRL_C = "\x03".encode("utf-8")
|
||||
CTRL_D = "\x04".encode("utf-8")
|
||||
|
||||
def tty_add_reader_callback(fd: int, callback: callable, loop: asyncio.AbstractEventLoop = None):
|
||||
"""
|
||||
Add an async reader callback for a tty file descriptor.
|
||||
|
||||
Example usage:
|
||||
|
||||
def reader():
|
||||
data = tty_read(fd)
|
||||
# do something with data
|
||||
|
||||
tty_add_reader_callback(self._child_fd, reader, self._loop)
|
||||
|
||||
:param fd: file descriptor
|
||||
:param callback: callback function
|
||||
:param loop: asyncio event loop to which the reader should be added. If None, use the currently-running loop.
|
||||
"""
|
||||
if loop is None:
|
||||
loop = asyncio.get_running_loop()
|
||||
loop.add_reader(fd, callback)
|
||||
|
||||
|
||||
def tty_read(fd: int) -> bytes:
|
||||
"""
|
||||
Read available bytes from a tty file descriptor. When used in a callback added to a file descriptor using
|
||||
tty_add_reader_callback(...), this function creates a solution for non-blocking reads from ttys.
|
||||
:param fd: tty file descriptor
|
||||
:return: bytes read
|
||||
"""
|
||||
if fd_is_closed(fd):
|
||||
raise EOFError
|
||||
|
||||
try:
|
||||
run = True
|
||||
result = bytearray()
|
||||
while not fd_is_closed(fd):
|
||||
ready, _, _ = select.select([fd], [], [], 0)
|
||||
if len(ready) == 0:
|
||||
break
|
||||
for f in ready:
|
||||
try:
|
||||
data = os.read(f, 4096)
|
||||
except OSError as e:
|
||||
if e.errno != errno.EIO and e.errno != errno.EWOULDBLOCK:
|
||||
raise
|
||||
else:
|
||||
if not data: # EOF
|
||||
if data is not None and len(data) > 0:
|
||||
result.extend(data)
|
||||
return result
|
||||
elif len(result) > 0:
|
||||
return result
|
||||
else:
|
||||
raise EOFError
|
||||
if data is not None and len(data) > 0:
|
||||
result.extend(data)
|
||||
return result
|
||||
|
||||
except EOFError: raise
|
||||
except Exception as e: RNS.log(f"TTY read error: {e}", RNS.LOG_ERROR)
|
||||
|
||||
|
||||
def tty_read_poll(fd: int) -> bytes:
|
||||
"""
|
||||
Read available bytes from a tty file descriptor. When used in a callback added to a file descriptor using
|
||||
tty_add_reader_callback(...), this function creates a solution for non-blocking reads from ttys.
|
||||
:param fd: tty file descriptor
|
||||
:return: bytes read
|
||||
"""
|
||||
if fd_is_closed(fd):
|
||||
raise EOFError
|
||||
|
||||
result = bytearray()
|
||||
try:
|
||||
flags = fcntl.fcntl(fd, fcntl.F_GETFL)
|
||||
fcntl.fcntl(fd, fcntl.F_SETFL, flags | os.O_NONBLOCK)
|
||||
while True:
|
||||
try:
|
||||
data = os.read(fd, 4096)
|
||||
if not data:
|
||||
# EOF
|
||||
if len(result) > 0:
|
||||
return result
|
||||
raise EOFError
|
||||
result.extend(data)
|
||||
# continue loop to drain
|
||||
except OSError as e:
|
||||
if e.errno in (errno.EWOULDBLOCK, errno.EAGAIN):
|
||||
break
|
||||
if e.errno == errno.EIO:
|
||||
if len(result) > 0:
|
||||
return result
|
||||
raise EOFError
|
||||
raise
|
||||
except EOFError: raise
|
||||
except Exception as e: RNS.log(f"TTY read error: {e}", RNS.LOG_ERROR)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def fd_is_closed(fd: int) -> bool:
|
||||
"""
|
||||
Check if file descriptor is closed
|
||||
:param fd: file descriptor
|
||||
:return: True if file descriptor is closed
|
||||
"""
|
||||
try:
|
||||
fcntl.fcntl(fd, fcntl.F_GETFL) < 0
|
||||
except OSError as ose:
|
||||
return ose.errno == errno.EBADF
|
||||
|
||||
|
||||
def tty_unset_reader_callbacks(fd: int, loop: asyncio.AbstractEventLoop = None):
|
||||
"""
|
||||
Remove async reader callbacks for file descriptor.
|
||||
:param fd: file descriptor
|
||||
:param loop: asyncio event loop from which to remove callbacks
|
||||
"""
|
||||
with exception.permit(SystemExit):
|
||||
if loop is None:
|
||||
loop = asyncio.get_running_loop()
|
||||
loop.remove_reader(fd)
|
||||
|
||||
|
||||
def tty_get_winsize(fd: int) -> [int, int, int, int]:
|
||||
"""
|
||||
Ge the window size of a tty.
|
||||
:param fd: file descriptor of tty
|
||||
:return: (rows, cols, h_pixels, v_pixels)
|
||||
"""
|
||||
packed = fcntl.ioctl(fd, termios.TIOCGWINSZ, struct.pack('HHHH', 0, 0, 0, 0))
|
||||
rows, cols, h_pixels, v_pixels = struct.unpack('HHHH', packed)
|
||||
return rows, cols, h_pixels, v_pixels
|
||||
|
||||
|
||||
def tty_set_winsize(fd: int, rows: int, cols: int, h_pixels: int, v_pixels: int):
|
||||
"""
|
||||
Set the window size on a tty.
|
||||
:param fd: file descriptor of tty
|
||||
:param rows: number of visible rows
|
||||
:param cols: number of visible columns
|
||||
:param h_pixels: number of visible horizontal pixels
|
||||
:param v_pixels: number of visible vertical pixels
|
||||
"""
|
||||
if fd < 0:
|
||||
return
|
||||
packed = struct.pack('HHHH', rows, cols, h_pixels, v_pixels)
|
||||
fcntl.ioctl(fd, termios.TIOCSWINSZ, packed)
|
||||
|
||||
|
||||
def process_exists(pid) -> bool:
|
||||
"""
|
||||
Check For the existence of a unix pid.
|
||||
:param pid: process id to check
|
||||
:return: True if process exists
|
||||
"""
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
except OSError:
|
||||
return False
|
||||
else:
|
||||
return True
|
||||
|
||||
|
||||
class TTYRestorer(contextlib.AbstractContextManager):
|
||||
# Indexes of flags within the attrs array
|
||||
ATTR_IDX_IFLAG = 0
|
||||
ATTR_IDX_OFLAG = 1
|
||||
ATTR_IDX_CFLAG = 2
|
||||
ATTR_IDX_LFLAG = 4
|
||||
ATTR_IDX_CC = 5
|
||||
|
||||
def __init__(self, fd: int, suppress_logs=False):
|
||||
"""
|
||||
Saves termios attributes for a tty for later restoration.
|
||||
|
||||
The attributes are an array of values with the following meanings.
|
||||
|
||||
tcflag_t c_iflag; /* input modes */
|
||||
tcflag_t c_oflag; /* output modes */
|
||||
tcflag_t c_cflag; /* control modes */
|
||||
tcflag_t c_lflag; /* local modes */
|
||||
cc_t c_cc[NCCS]; /* special characters */
|
||||
|
||||
:param fd: file descriptor of tty
|
||||
"""
|
||||
self._fd = fd
|
||||
self._tattr = None
|
||||
self._suppress_logs = suppress_logs
|
||||
self._tattr = self.current_attr()
|
||||
if not self._tattr and not self._suppress_logs: RNS.log(f"Could not get attrs for fd {fd}", RNS.LOG_DEBUG)
|
||||
|
||||
def raw(self):
|
||||
"""
|
||||
Set raw mode on tty
|
||||
"""
|
||||
if self._fd is None:
|
||||
return
|
||||
with contextlib.suppress(termios.error):
|
||||
tty.setraw(self._fd, termios.TCSANOW)
|
||||
|
||||
def original_attr(self) -> [any]:
|
||||
return copy.deepcopy(self._tattr)
|
||||
|
||||
def current_attr(self) -> [any]:
|
||||
"""
|
||||
Get the current termios attributes for the wrapped fd.
|
||||
:return: attribute array
|
||||
"""
|
||||
if self._fd is None:
|
||||
return None
|
||||
|
||||
with contextlib.suppress(termios.error):
|
||||
return copy.deepcopy(termios.tcgetattr(self._fd))
|
||||
return None
|
||||
|
||||
def set_attr(self, attr: [any], when: int = termios.TCSADRAIN):
|
||||
"""
|
||||
Set termios attributes
|
||||
:param attr: attribute list to set
|
||||
:param when: when attributes should be applied (termios.TCSANOW, termios.TCSADRAIN, termios.TCSAFLUSH)
|
||||
"""
|
||||
if not attr or self._fd is None:
|
||||
return
|
||||
|
||||
with contextlib.suppress(termios.error):
|
||||
termios.tcsetattr(self._fd, when, attr)
|
||||
|
||||
def isatty(self):
|
||||
return os.isatty(self._fd) if self._fd is not None else None
|
||||
|
||||
def restore(self):
|
||||
"""
|
||||
Restore termios settings to state captured in constructor.
|
||||
"""
|
||||
self.set_attr(self._tattr, termios.TCSADRAIN)
|
||||
|
||||
def __exit__(self, __exc_type: typing.Type[BaseException], __exc_value: BaseException,
|
||||
__traceback: types.TracebackType) -> bool:
|
||||
self.restore()
|
||||
return False #__exc_type is not None and issubclass(__exc_type, termios.error)
|
||||
|
||||
|
||||
def _task_from_event(evt: asyncio.Event, loop: asyncio.AbstractEventLoop = None):
|
||||
if not loop:
|
||||
loop = asyncio.get_running_loop()
|
||||
|
||||
#TODO: this is hacky
|
||||
async def wait():
|
||||
while not evt.is_set():
|
||||
await asyncio.sleep(0.1)
|
||||
return True
|
||||
|
||||
return loop.create_task(wait())
|
||||
|
||||
|
||||
class AggregateException(Exception):
|
||||
def __init__(self, inner_exceptions: [Exception]):
|
||||
super().__init__()
|
||||
self.inner_exceptions = inner_exceptions
|
||||
|
||||
def __str__(self):
|
||||
return "Multiple exceptions encountered: \n\n" + "\n\n".join(map(lambda e: str(e), self.inner_exceptions))
|
||||
|
||||
|
||||
async def event_wait_any(evts: [asyncio.Event], timeout: float = None) -> (any, any):
|
||||
tasks = list(map(lambda evt: (evt, _task_from_event(evt)), evts))
|
||||
try:
|
||||
finished, unfinished = await asyncio.wait(map(lambda t: t[1], tasks),
|
||||
timeout=timeout,
|
||||
return_when=asyncio.FIRST_COMPLETED)
|
||||
|
||||
if len(unfinished) > 0:
|
||||
for task in unfinished:
|
||||
task.cancel()
|
||||
await asyncio.wait(unfinished)
|
||||
|
||||
exceptions = []
|
||||
|
||||
for f in finished:
|
||||
ex = f.exception()
|
||||
if ex and not isinstance(ex, asyncio.CancelledError) and not isinstance(ex, TimeoutError):
|
||||
exceptions.append(ex)
|
||||
|
||||
if len(exceptions) > 0:
|
||||
raise AggregateException(exceptions)
|
||||
|
||||
return next(map(lambda t: next(map(lambda tt: tt[0], tasks)), finished), None)
|
||||
finally:
|
||||
unfinished = []
|
||||
for task in map(lambda t: t[1], tasks):
|
||||
if task.done():
|
||||
if not task.cancelled():
|
||||
task.exception()
|
||||
else:
|
||||
task.cancel()
|
||||
unfinished.append(task)
|
||||
if len(unfinished) > 0:
|
||||
await asyncio.wait(unfinished)
|
||||
|
||||
|
||||
async def event_wait(evt: asyncio.Event, timeout: float) -> bool:
|
||||
"""
|
||||
Wait for event to be set, or timeout to expire.
|
||||
:param evt: asyncio.Event to wait on
|
||||
:param timeout: maximum number of seconds to wait.
|
||||
:return: True if event was set, False if timeout expired
|
||||
"""
|
||||
await event_wait_any([evt], timeout=timeout)
|
||||
return evt.is_set()
|
||||
|
||||
|
||||
def _launch_child(cmd_line: list[str], env: dict[str, str], stdin_is_pipe: bool, stdout_is_pipe: bool,
|
||||
stderr_is_pipe: bool) -> tuple[int, int, int, int]:
|
||||
# Set up PTY and/or pipes
|
||||
child_fd = parent_fd = None
|
||||
if not (stdin_is_pipe and stdout_is_pipe and stderr_is_pipe):
|
||||
parent_fd, child_fd = pty.openpty()
|
||||
child_stdin, parent_stdin = (os.pipe() if stdin_is_pipe else (child_fd, parent_fd))
|
||||
parent_stdout, child_stdout = (os.pipe() if stdout_is_pipe else (parent_fd, child_fd))
|
||||
parent_stderr, child_stderr = (os.pipe() if stderr_is_pipe else (parent_fd, child_fd))
|
||||
|
||||
# Fork
|
||||
pid = os.fork()
|
||||
|
||||
if pid == 0:
|
||||
try:
|
||||
# We are in the child process, so close all open sockets and pipes except for the PTY and/or pipes
|
||||
max_fd = os.sysconf("SC_OPEN_MAX")
|
||||
for fd in range(3, max_fd):
|
||||
if fd not in (child_stdin, child_stdout, child_stderr):
|
||||
try:
|
||||
os.close(fd)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
# Set up PTY and/or pipes
|
||||
os.dup2(child_stdin, 0)
|
||||
os.dup2(child_stdout, 1)
|
||||
os.dup2(child_stderr, 2)
|
||||
# Make PTY controlling if necessary so that CTRL_C/CTRL_D behave as expected
|
||||
if child_fd is not None:
|
||||
os.setsid()
|
||||
try:
|
||||
tty_fd = 0 if not stdin_is_pipe else (1 if not stdout_is_pipe else 2)
|
||||
# Set controlling TTY for this session
|
||||
fcntl.ioctl(tty_fd, termios.TIOCSCTTY, 0)
|
||||
except Exception:
|
||||
pass
|
||||
# Ensure the child is the foreground process group for the TTY
|
||||
try:
|
||||
os.setpgid(0, 0)
|
||||
pgid = os.getpgrp()
|
||||
import struct as _struct
|
||||
fcntl.ioctl(tty_fd, termios.TIOCSPGRP, _struct.pack('i', pgid))
|
||||
except Exception:
|
||||
pass
|
||||
# Ensure canonical input with signals and local echo enabled
|
||||
try:
|
||||
tty_fd = 0 if not stdin_is_pipe else (1 if not stdout_is_pipe else 2)
|
||||
attrs = termios.tcgetattr(tty_fd)
|
||||
lflag = attrs[3]
|
||||
lflag |= termios.ICANON | termios.ISIG | termios.ECHO
|
||||
attrs[3] = lflag
|
||||
termios.tcsetattr(tty_fd, termios.TCSANOW, attrs)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Execute the command
|
||||
os.execvpe(cmd_line[0], cmd_line, env)
|
||||
except Exception as err:
|
||||
exc_type, exc_obj, exc_tb = sys.exc_info()
|
||||
fname = os.path.split(exc_tb.tb_frame.f_code.co_filename)[1]
|
||||
print(f"Unable to start {cmd_line[0]}: {err} ({fname}:{exc_tb.tb_lineno})")
|
||||
sys.stdout.flush()
|
||||
# don't let any other modules get in our way, do an immediate silent exit.
|
||||
os._exit(255)
|
||||
|
||||
else:
|
||||
# We are in the parent process, so close the child-side of the PTY and/or pipes
|
||||
if child_fd is not None:
|
||||
os.close(child_fd)
|
||||
if child_stdin != child_fd:
|
||||
os.close(child_stdin)
|
||||
if child_stdout != child_fd:
|
||||
os.close(child_stdout)
|
||||
if child_stderr != child_fd:
|
||||
os.close(child_stderr)
|
||||
# # Close the write end of the pipe if a pipe is used for standard input
|
||||
# if not stdin_is_pipe:
|
||||
# os.close(parent_stdin)
|
||||
# Return the child PID and the file descriptors for the PTY and/or pipes
|
||||
return pid, parent_stdin, parent_stdout, parent_stderr
|
||||
|
||||
|
||||
class CallbackSubprocess:
|
||||
# time between checks of child process
|
||||
PROCESS_POLL_TIME: float = 0.1
|
||||
# Close pipes soon after process exit to avoid scheduling on closed event loops
|
||||
PROCESS_PIPE_TIME: int = 1
|
||||
|
||||
def __init__(self, argv: [str], env: dict, loop: asyncio.AbstractEventLoop, stdout_callback: callable,
|
||||
stderr_callback: callable, terminated_callback: callable, stdin_is_pipe: bool, stdout_is_pipe: bool,
|
||||
stderr_is_pipe: bool):
|
||||
"""
|
||||
Fork a child process and generate callbacks with output from the process.
|
||||
:param argv: the command line, tokenized. The first element must be the absolute path to an executable file.
|
||||
:param env: environment variables to override
|
||||
:param loop: the asyncio event loop to use
|
||||
:param stdout_callback: callback for data, e.g. def callback(data:bytes) -> None
|
||||
:param terminated_callback: callback for termination/return code, e.g. def callback(return_code:int) -> None
|
||||
"""
|
||||
assert loop is not None, "loop should not be None"
|
||||
assert stdout_callback is not None, "stdout_callback should not be None"
|
||||
assert terminated_callback is not None, "terminated_callback should not be None"
|
||||
|
||||
self._command: [str] = argv
|
||||
self._env = env or {}
|
||||
self._loop = loop
|
||||
self._stdout_cb = stdout_callback
|
||||
self._stderr_cb = stderr_callback
|
||||
self._terminated_cb = terminated_callback
|
||||
self._pid: int = None
|
||||
self._child_stdin: int = None
|
||||
self._child_stdout: int = None
|
||||
self._child_stderr: int = None
|
||||
self._return_code: int = None
|
||||
self._stdout_eof: bool = False
|
||||
self._stderr_eof: bool = False
|
||||
self._stdin_is_pipe = stdin_is_pipe
|
||||
self._stdout_is_pipe = stdout_is_pipe
|
||||
self._stderr_is_pipe = stderr_is_pipe
|
||||
|
||||
def _ensure_pipes_closed(self):
|
||||
stdin = self._child_stdin
|
||||
stdout = self._child_stdout
|
||||
stderr = self._child_stderr
|
||||
fds = set(filter(lambda x: x is not None, list({stdin, stdout, stderr})))
|
||||
RNS.log(f"Queuing close of pipes for ended process (fds: {fds})", RNS.LOG_DEBUG)
|
||||
|
||||
def ensure_pipes_closed_inner():
|
||||
RNS.log(f"Ensuring pipes are closed (fds: {fds})", RNS.LOG_DEBUG)
|
||||
for fd in fds:
|
||||
RNS.log(f"Closing fd {fd}", RNS.LOG_DEBUG)
|
||||
with contextlib.suppress(OSError): tty_unset_reader_callbacks(fd)
|
||||
with contextlib.suppress(OSError): os.close(fd)
|
||||
|
||||
self._child_stdin = None
|
||||
self._child_stdout = None
|
||||
self._child_stderr = None
|
||||
|
||||
# Avoid scheduling on a closed loop
|
||||
if self._loop.is_closed(): ensure_pipes_closed_inner()
|
||||
else: self._loop.call_later(CallbackSubprocess.PROCESS_PIPE_TIME, ensure_pipes_closed_inner)
|
||||
|
||||
def terminate(self, kill_delay: float = 1.0):
|
||||
"""
|
||||
Terminate child process if running
|
||||
:param kill_delay: if after kill_delay seconds the child process has not exited, escalate to SIGHUP and SIGKILL
|
||||
"""
|
||||
|
||||
RNS.log("terminate()", RNS.LOG_EXTREME)
|
||||
if not self.running: return
|
||||
|
||||
with exception.permit(SystemExit): os.kill(self._pid, signal.SIGTERM)
|
||||
|
||||
def kill():
|
||||
if process_exists(self._pid):
|
||||
RNS.log("kill()", RNS.LOG_EXTREME)
|
||||
with exception.permit(SystemExit):
|
||||
os.kill(self._pid, signal.SIGHUP)
|
||||
os.kill(self._pid, signal.SIGKILL)
|
||||
|
||||
self._loop.call_later(kill_delay, kill)
|
||||
|
||||
def wait():
|
||||
RNS.log("wait()", RNS.LOG_EXTREME)
|
||||
with contextlib.suppress(OSError): os.waitpid(self._pid, 0)
|
||||
self._ensure_pipes_closed()
|
||||
RNS.log("wait() finish", RNS.LOG_EXTREME)
|
||||
|
||||
threading.Thread(target=wait, daemon=True).start()
|
||||
|
||||
def close_stdin(self):
|
||||
with contextlib.suppress(Exception):
|
||||
os.close(self._child_stdin)
|
||||
# Encourage prompt shutdown if child lingers after stdin close
|
||||
def _ensure_terminate():
|
||||
if self.running:
|
||||
self.terminate(kill_delay=0.2)
|
||||
if not self._loop.is_closed():
|
||||
self._loop.call_later(0.05, _ensure_terminate)
|
||||
|
||||
@property
|
||||
def started(self) -> bool:
|
||||
"""
|
||||
:return: True if child process has been started
|
||||
"""
|
||||
return self._pid is not None
|
||||
|
||||
@property
|
||||
def running(self) -> bool:
|
||||
"""
|
||||
:return: True if child process is still running
|
||||
"""
|
||||
return self._pid is not None and process_exists(self._pid)
|
||||
|
||||
def write(self, data: bytes):
|
||||
"""
|
||||
Write bytes to the stdin of the child process.
|
||||
:param data: bytes to write
|
||||
"""
|
||||
|
||||
os.write(self._child_stdin, data)
|
||||
|
||||
# TODO: Check what this is actually supposed to solve.
|
||||
#
|
||||
# For pipe-in + TTY-out, echo should be visible immediately
|
||||
if self._stdin_is_pipe and not self._stdout_is_pipe and self._stdout_cb is not None and data not in (CTRL_C, CTRL_D):
|
||||
try: self._stdout_cb(data)
|
||||
except Exception: pass
|
||||
|
||||
def set_winsize(self, r: int, c: int, h: int, v: int):
|
||||
"""
|
||||
Set the window size on the tty of the child process.
|
||||
:param r: rows visible
|
||||
:param c: columns visible
|
||||
:param h: horizontal pixels visible
|
||||
:param v: vertical pixels visible
|
||||
:return:
|
||||
"""
|
||||
RNS.log(f"set_winsize({r},{c},{h},{v}", RNS.LOG_DEBUG)
|
||||
tty_set_winsize(self._child_stdout, r, c, h, v)
|
||||
|
||||
def copy_winsize(self, fromfd: int):
|
||||
"""
|
||||
Copy window size from one tty to another.
|
||||
:param fromfd: source tty file descriptor
|
||||
"""
|
||||
r, c, h, v = tty_get_winsize(fromfd)
|
||||
self.set_winsize(r, c, h, v)
|
||||
|
||||
def tcsetattr(self, when: int, attr: list[any]): # actual type is list[int | list[int | bytes]]
|
||||
"""
|
||||
Set tty attributes.
|
||||
:param when: when to apply change: termios.TCSANOW or termios.TCSADRAIN or termios.TCSAFLUSH
|
||||
:param attr: attributes to set
|
||||
"""
|
||||
termios.tcsetattr(self._child_stdin, when, attr)
|
||||
|
||||
def tcgetattr(self) -> list[any]: # actual type is list[int | list[int | bytes]]
|
||||
"""
|
||||
Get tty attributes.
|
||||
:return: tty attributes value
|
||||
"""
|
||||
return termios.tcgetattr(self._child_stdout)
|
||||
|
||||
def ttysetraw(self):
|
||||
tty.setraw(self._child_stdout, termios.TCSADRAIN)
|
||||
|
||||
def start(self):
|
||||
"""
|
||||
Start the child process.
|
||||
"""
|
||||
RNS.log("start()", RNS.LOG_EXTREME)
|
||||
|
||||
env = os.environ.copy()
|
||||
for key in self._env:
|
||||
env[key] = self._env[key]
|
||||
|
||||
program = self._command[0]
|
||||
assert isinstance(program, str)
|
||||
|
||||
self._pid, \
|
||||
self._child_stdin, \
|
||||
self._child_stdout, \
|
||||
self._child_stderr = _launch_child(self._command, env, self._stdin_is_pipe, self._stdout_is_pipe,
|
||||
self._stderr_is_pipe)
|
||||
RNS.log(f"Started pid {self.pid}, fds: {self._child_stdin}, {self._child_stdout}, {self._child_stderr}", RNS.LOG_DEBUG)
|
||||
|
||||
def poll():
|
||||
try:
|
||||
pid, self._return_code = os.waitpid(self._pid, os.WNOHANG)
|
||||
if self._return_code is not None:
|
||||
self._return_code = self._return_code & 0xff
|
||||
if self._return_code is not None and not process_exists(self._pid):
|
||||
RNS.log(f"polled return code {self._return_code}", RNS.LOG_DEBUG)
|
||||
self._terminated_cb(self._return_code)
|
||||
if self.running:
|
||||
self._loop.call_later(CallbackSubprocess.PROCESS_POLL_TIME, poll)
|
||||
else:
|
||||
self._ensure_pipes_closed()
|
||||
except Exception as e:
|
||||
if not hasattr(e, "errno") or e.errno != errno.ECHILD:
|
||||
RNS.log(f"Error in process poll: {e}", RNS.LOG_DEBUG)
|
||||
|
||||
self._loop.call_later(CallbackSubprocess.PROCESS_POLL_TIME, poll)
|
||||
|
||||
def stdout():
|
||||
try:
|
||||
with exception.permit(SystemExit):
|
||||
data = tty_read_poll(self._child_stdout)
|
||||
if data is not None and len(data) > 0:
|
||||
self._stdout_cb(data)
|
||||
# Opportunistically drain shortly after to coalesce immediate follow-up output
|
||||
if not self._loop.is_closed():
|
||||
self._loop.call_later(0.01, stdout)
|
||||
except EOFError:
|
||||
self._stdout_eof = True
|
||||
tty_unset_reader_callbacks(self._child_stdout)
|
||||
self._stdout_cb(bytearray())
|
||||
|
||||
def stderr():
|
||||
try:
|
||||
with exception.permit(SystemExit):
|
||||
data = tty_read_poll(self._child_stderr)
|
||||
if data is not None and len(data) > 0:
|
||||
self._stderr_cb(data)
|
||||
if not self._loop.is_closed():
|
||||
self._loop.call_later(0.01, stderr)
|
||||
except EOFError:
|
||||
self._stderr_eof = True
|
||||
tty_unset_reader_callbacks(self._child_stderr)
|
||||
self._stderr_cb(bytearray())
|
||||
|
||||
tty_add_reader_callback(self._child_stdout, stdout, self._loop)
|
||||
if self._child_stderr != self._child_stdout:
|
||||
tty_add_reader_callback(self._child_stderr, stderr, self._loop)
|
||||
|
||||
@property
|
||||
def stdout_eof(self):
|
||||
return self._stdout_eof or not self.running
|
||||
|
||||
@property
|
||||
def stderr_eof(self):
|
||||
return self._stderr_eof or not self.running
|
||||
|
||||
|
||||
@property
|
||||
def return_code(self) -> int:
|
||||
return self._return_code
|
||||
|
||||
@property
|
||||
def pid(self) -> int:
|
||||
return self._pid
|
||||
|
||||
|
||||
async def main():
|
||||
"""
|
||||
A test driver for the CallbackProcess class.
|
||||
python ./process.py /bin/zsh --login
|
||||
"""
|
||||
|
||||
if len(sys.argv) <= 1:
|
||||
print(f"Usage: {sys.argv} <absolute_path_to_child_executable> [child_arg ...]")
|
||||
exit(1)
|
||||
|
||||
loop = asyncio.get_event_loop()
|
||||
# asyncio.set_event_loop(loop)
|
||||
retcode = loop.create_future()
|
||||
|
||||
def stdout(data: bytes): os.write(sys.stdout.fileno(), data)
|
||||
|
||||
def terminated(rc: int): retcode.set_result(rc)
|
||||
|
||||
process = CallbackSubprocess(argv=sys.argv[1:],
|
||||
env={"TERM": os.environ.get("TERM", "xterm")},
|
||||
loop=loop,
|
||||
stdout_callback=stdout,
|
||||
terminated_callback=terminated)
|
||||
|
||||
def sigint_handler(sig, frame):
|
||||
if process is None or process.started and not process.running:
|
||||
raise KeyboardInterrupt
|
||||
elif process.running:
|
||||
process.write("\x03".encode("utf-8"))
|
||||
|
||||
def sigwinch_handler(sig, frame):
|
||||
process.copy_winsize(sys.stdin.fileno())
|
||||
|
||||
signal.signal(signal.SIGINT, sigint_handler)
|
||||
signal.signal(signal.SIGWINCH, sigwinch_handler)
|
||||
|
||||
def stdin():
|
||||
try:
|
||||
data = tty_read(sys.stdin.fileno())
|
||||
if data is not None:
|
||||
process.write(data)
|
||||
|
||||
except EOFError:
|
||||
tty_unset_reader_callbacks(sys.stdin.fileno())
|
||||
process.write(CTRL_D)
|
||||
|
||||
tty_add_reader_callback(sys.stdin.fileno(), stdin)
|
||||
process.start()
|
||||
# call_soon called it too soon, not sure why.
|
||||
loop.call_later(0.001, functools.partial(process.copy_winsize, sys.stdin.fileno()))
|
||||
|
||||
val = await retcode
|
||||
RNS.log(f"Got return code {val}", RNS.LOG_DEBUG)
|
||||
return val
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
tr = TTYRestorer(sys.stdin.fileno())
|
||||
try:
|
||||
tr.raw()
|
||||
asyncio.run(main())
|
||||
finally:
|
||||
tty_unset_reader_callbacks(sys.stdin.fileno())
|
||||
tr.restore()
|
||||
149
RNS/Utilities/rnsh/protocol.py
Normal file
149
RNS/Utilities/rnsh/protocol.py
Normal file
|
|
@ -0,0 +1,149 @@
|
|||
# Based on the original rnsh program by Aaron Heise (@acehoss)
|
||||
# https://github.com/acehoss/rnsh - MIT License - Copyright (c) 2023 Aaron Heise
|
||||
# This version of rnsh is included in RNS under the Reticulum License
|
||||
#
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import RNS
|
||||
from RNS.vendor import umsgpack
|
||||
from RNS.Buffer import StreamDataMessage as RNSStreamDataMessage
|
||||
import RNS.Utilities.rnsh.retry
|
||||
import abc
|
||||
import contextlib
|
||||
import struct
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
MSG_MAGIC = 0xac
|
||||
PROTOCOL_VERSION = 1
|
||||
|
||||
def _make_MSGTYPE(val: int):
|
||||
return ((MSG_MAGIC << 8) & 0xff00) | (val & 0x00ff)
|
||||
|
||||
|
||||
class NoopMessage(RNS.MessageBase):
|
||||
MSGTYPE = _make_MSGTYPE(0)
|
||||
def pack(self) -> bytes: return bytes()
|
||||
def unpack(self, raw): pass
|
||||
|
||||
|
||||
class WindowSizeMessage(RNS.MessageBase):
|
||||
MSGTYPE = _make_MSGTYPE(2)
|
||||
|
||||
def __init__(self, rows: int = None, cols: int = None, hpix: int = None, vpix: int = None):
|
||||
super().__init__()
|
||||
self.rows = rows
|
||||
self.cols = cols
|
||||
self.hpix = hpix
|
||||
self.vpix = vpix
|
||||
|
||||
def pack(self) -> bytes: return umsgpack.packb((self.rows, self.cols, self.hpix, self.vpix))
|
||||
def unpack(self, raw): self.rows, self.cols, self.hpix, self.vpix = umsgpack.unpackb(raw)
|
||||
|
||||
|
||||
class ExecuteCommandMesssage(RNS.MessageBase):
|
||||
MSGTYPE = _make_MSGTYPE(3)
|
||||
|
||||
def __init__(self, cmdline: [str] = None, pipe_stdin: bool = False, pipe_stdout: bool = False,
|
||||
pipe_stderr: bool = False, tcflags: [any] = None, term: str | None = None, rows: int = None,
|
||||
cols: int = None, hpix: int = None, vpix: int = None):
|
||||
|
||||
super().__init__()
|
||||
self.cmdline = cmdline
|
||||
self.pipe_stdin = pipe_stdin
|
||||
self.pipe_stdout = pipe_stdout
|
||||
self.pipe_stderr = pipe_stderr
|
||||
self.tcflags = tcflags
|
||||
self.term = term
|
||||
self.rows = rows
|
||||
self.cols = cols
|
||||
self.hpix = hpix
|
||||
self.vpix = vpix
|
||||
|
||||
def pack(self) -> bytes:
|
||||
return umsgpack.packb((self.cmdline, self.pipe_stdin, self.pipe_stdout, self.pipe_stderr,
|
||||
self.tcflags, self.term, self.rows, self.cols, self.hpix, self.vpix))
|
||||
|
||||
def unpack(self, raw):
|
||||
self.cmdline, self.pipe_stdin, self.pipe_stdout, self.pipe_stderr, self.tcflags, self.term, self.rows, \
|
||||
self.cols, self.hpix, self.vpix = umsgpack.unpackb(raw)
|
||||
|
||||
|
||||
# Create a version of RNS.Buffer.StreamDataMessage that we control
|
||||
class StreamDataMessage(RNSStreamDataMessage):
|
||||
MSGTYPE = _make_MSGTYPE(4)
|
||||
STREAM_ID_STDIN = 0
|
||||
STREAM_ID_STDOUT = 1
|
||||
STREAM_ID_STDERR = 2
|
||||
|
||||
|
||||
class VersionInfoMessage(RNS.MessageBase):
|
||||
MSGTYPE = _make_MSGTYPE(5)
|
||||
|
||||
def __init__(self, sw_version: str = None):
|
||||
super().__init__()
|
||||
self.sw_version = sw_version or RNS.Utilities.rnsh.__version__
|
||||
self.protocol_version = PROTOCOL_VERSION
|
||||
|
||||
def pack(self) -> bytes: return umsgpack.packb((self.sw_version, self.protocol_version))
|
||||
def unpack(self, raw): self.sw_version, self.protocol_version = umsgpack.unpackb(raw)
|
||||
|
||||
|
||||
class ErrorMessage(RNS.MessageBase):
|
||||
MSGTYPE = _make_MSGTYPE(6)
|
||||
|
||||
def __init__(self, msg: str = None, fatal: bool = False, data: dict = None):
|
||||
super().__init__()
|
||||
self.msg = msg
|
||||
self.fatal = fatal
|
||||
self.data = data
|
||||
|
||||
def pack(self) -> bytes: return umsgpack.packb((self.msg, self.fatal, self.data))
|
||||
def unpack(self, raw: bytes): self.msg, self.fatal, self.data = umsgpack.unpackb(raw)
|
||||
|
||||
|
||||
class CommandExitedMessage(RNS.MessageBase):
|
||||
MSGTYPE = _make_MSGTYPE(7)
|
||||
|
||||
def __init__(self, return_code: int = None):
|
||||
super().__init__()
|
||||
self.return_code = return_code
|
||||
|
||||
def pack(self) -> bytes: return umsgpack.packb(self.return_code)
|
||||
def unpack(self, raw: bytes): self.return_code = umsgpack.unpackb(raw)
|
||||
|
||||
|
||||
message_types = [NoopMessage, VersionInfoMessage, WindowSizeMessage, ExecuteCommandMesssage, StreamDataMessage,
|
||||
CommandExitedMessage, ErrorMessage]
|
||||
|
||||
def register_message_types(channel: RNS.Channel.Channel):
|
||||
for message_type in message_types: channel.register_message_type(message_type)
|
||||
196
RNS/Utilities/rnsh/retry.py
Normal file
196
RNS/Utilities/rnsh/retry.py
Normal file
|
|
@ -0,0 +1,196 @@
|
|||
# Based on the original rnsh program by Aaron Heise (@acehoss)
|
||||
# https://github.com/acehoss/rnsh - MIT License - Copyright (c) 2023 Aaron Heise
|
||||
# This version of rnsh is included in RNS under the Reticulum License
|
||||
#
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
import RNS
|
||||
import asyncio
|
||||
import threading
|
||||
import time
|
||||
import RNS.Utilities.rnsh.exception as exception
|
||||
from typing import Callable
|
||||
from contextlib import AbstractContextManager
|
||||
import types
|
||||
import typing
|
||||
|
||||
|
||||
class RetryStatus:
|
||||
def __init__(self, tag: any, try_limit: int, wait_delay: float, retry_callback: Callable[[any, int], any],
|
||||
timeout_callback: Callable[[any, int], None], tries: int = 1):
|
||||
|
||||
self.tag = tag
|
||||
self.try_limit = try_limit
|
||||
self.tries = tries
|
||||
self.wait_delay = wait_delay
|
||||
self.retry_callback = retry_callback
|
||||
self.timeout_callback = timeout_callback
|
||||
self.try_time = time.time()
|
||||
self.completed = False
|
||||
|
||||
@property
|
||||
def ready(self):
|
||||
ready = time.time() > self.try_time + self.wait_delay
|
||||
RNS.log(f"ready check {self.tag} try_time {self.try_time} wait_delay {self.wait_delay} " +
|
||||
f"next_try {self.try_time + self.wait_delay} now {time.time()} " +
|
||||
f"exceeded {time.time() - self.try_time - self.wait_delay} ready {ready}", RNS.LOG_DEBUG)
|
||||
return ready
|
||||
|
||||
@property
|
||||
def timed_out(self):
|
||||
return self.ready and self.tries >= self.try_limit
|
||||
|
||||
def timeout(self):
|
||||
self.completed = True
|
||||
self.timeout_callback(self.tag, self.tries)
|
||||
|
||||
def retry(self) -> any:
|
||||
self.tries = self.tries + 1
|
||||
self.try_time = time.time()
|
||||
return self.retry_callback(self.tag, self.tries)
|
||||
|
||||
|
||||
class RetryThread(AbstractContextManager):
|
||||
def __init__(self, loop_period: float = 0.25, name: str = "retry thread"):
|
||||
self._loop_period = loop_period
|
||||
self._statuses: list[RetryStatus] = []
|
||||
self._tag_counter = 0
|
||||
self._lock = threading.RLock()
|
||||
self._run = True
|
||||
self._finished: asyncio.Future = None
|
||||
self._thread = threading.Thread(name=name, target=self._thread_run, daemon=True)
|
||||
self._thread.start()
|
||||
|
||||
def is_alive(self): return self._thread.is_alive()
|
||||
|
||||
def close(self, loop: asyncio.AbstractEventLoop = None) -> asyncio.Future:
|
||||
RNS.log("Stopping timer thread", RNS.LOG_DEBUG)
|
||||
if loop is None:
|
||||
self._run = False
|
||||
self._thread.join()
|
||||
return None
|
||||
else:
|
||||
self._finished = loop.create_future()
|
||||
return self._finished
|
||||
|
||||
def wait(self, timeout: float = None):
|
||||
if timeout: timeout = timeout + time.time()
|
||||
while timeout is None or time.time() < timeout:
|
||||
with self._lock: task_count = len(self._statuses)
|
||||
if task_count == 0: return
|
||||
time.sleep(0.1)
|
||||
|
||||
def _thread_run(self):
|
||||
while self._run and self._finished is None:
|
||||
time.sleep(self._loop_period)
|
||||
ready: list[RetryStatus] = []
|
||||
prune: list[RetryStatus] = []
|
||||
with self._lock: ready.extend(list(filter(lambda s: s.ready, self._statuses)))
|
||||
|
||||
for retry in ready:
|
||||
try:
|
||||
if not retry.completed:
|
||||
if retry.timed_out:
|
||||
RNS.log(f"Timed out {retry.tag} after {retry.try_limit} tries", RNS.LOG_DEBUG)
|
||||
retry.timeout()
|
||||
prune.append(retry)
|
||||
elif retry.ready:
|
||||
RNS.log(f"Retrying {retry.tag}, try {retry.tries + 1}/{retry.try_limit}", RNS.LOG_DEBUG)
|
||||
should_continue = retry.retry()
|
||||
if not should_continue: self.complete(retry.tag)
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Error processing retry id {retry.tag}: {e}", RNS.LOG_ERROR)
|
||||
prune.append(retry)
|
||||
|
||||
with self._lock:
|
||||
for retry in prune:
|
||||
RNS.log(f"pruned retry {retry.tag}, retry count {retry.tries}/{retry.try_limit}", RNS.LOG_DEBUG)
|
||||
with exception.permit(SystemExit): self._statuses.remove(retry)
|
||||
|
||||
if self._finished is not None: self._finished.set_result(None)
|
||||
|
||||
def _get_next_tag(self):
|
||||
self._tag_counter += 1
|
||||
return self._tag_counter
|
||||
|
||||
def has_tag(self, tag: any) -> bool:
|
||||
with self._lock: return next(filter(lambda s: s.tag == tag, self._statuses), None) is not None
|
||||
|
||||
def begin(self, try_limit: int, wait_delay: float, try_callback: Callable[[any, int], any],
|
||||
timeout_callback: Callable[[any, int], None]) -> any:
|
||||
|
||||
RNS.log(f"Running first try", RNS.LOG_DEBUG)
|
||||
tag = try_callback(None, 1)
|
||||
RNS.log(f"First try got id {tag}", RNS.LOG_DEBUG)
|
||||
|
||||
if not tag:
|
||||
RNS.log(f"Callback returned None/False/0, considering complete.", RNS.LOG_DEBUG)
|
||||
return None
|
||||
|
||||
with self._lock:
|
||||
if tag is None: tag = self._get_next_tag()
|
||||
self.complete(tag)
|
||||
|
||||
self._statuses.append(RetryStatus(tag=tag,
|
||||
tries=1,
|
||||
try_limit=try_limit,
|
||||
wait_delay=wait_delay,
|
||||
retry_callback=try_callback,
|
||||
timeout_callback=timeout_callback))
|
||||
|
||||
RNS.log(f"Added retry timer for {tag}", RNS.LOG_DEBUG)
|
||||
return tag
|
||||
|
||||
def complete(self, tag: any):
|
||||
assert tag is not None
|
||||
with self._lock:
|
||||
status = next(filter(lambda l: l.tag == tag, self._statuses), None)
|
||||
if status is not None:
|
||||
status.completed = True
|
||||
self._statuses.remove(status)
|
||||
RNS.log(f"completed {tag}", RNS.LOG_DEBUG)
|
||||
return
|
||||
|
||||
RNS.log(f"status not found to complete {tag}", RNS.LOG_DEBUG)
|
||||
|
||||
def complete_all(self):
|
||||
with self._lock:
|
||||
for status in self._statuses:
|
||||
status.completed = True
|
||||
RNS.log(f"completed {status.tag}", RNS.LOG_DEBUG)
|
||||
|
||||
self._statuses.clear()
|
||||
|
||||
def __exit__(self, __exc_type: typing.Type[BaseException], __exc_value: BaseException,
|
||||
__traceback: types.TracebackType) -> bool:
|
||||
self.close()
|
||||
return False
|
||||
189
RNS/Utilities/rnsh/rnsh.py
Normal file
189
RNS/Utilities/rnsh/rnsh.py
Normal file
|
|
@ -0,0 +1,189 @@
|
|||
#!/usr/bin/env python3
|
||||
#
|
||||
# Based on the original rnsh program by Aaron Heise (@acehoss)
|
||||
# https://github.com/acehoss/rnsh - MIT License - Copyright (c) 2023 Aaron Heise
|
||||
# This version of rnsh is included in RNS under the Reticulum License
|
||||
#
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
|
||||
import re
|
||||
import os
|
||||
import sys
|
||||
|
||||
import RNS
|
||||
import RNS.Utilities.rnsh.process as process
|
||||
import RNS.Utilities.rnsh.session as session
|
||||
import RNS.Utilities.rnsh.args
|
||||
import RNS.Utilities.rnsh.loop
|
||||
import RNS.Utilities.rnsh.listener as listener
|
||||
import RNS.Utilities.rnsh.initiator as initiator
|
||||
from RNS.Utilities.rnsh.args import parse_arguments
|
||||
|
||||
APP_NAME = "rnsh"
|
||||
loop: asyncio.AbstractEventLoop | None = None
|
||||
|
||||
def _sanitize_service_name(service_name:str) -> str: return re.sub(r'\W+', '', service_name)
|
||||
|
||||
def prepare_identity(identity_path=None, service_name=None, configdir=None):
|
||||
service_name = _sanitize_service_name(service_name or "")
|
||||
if not identity_path:
|
||||
identity_path = f"{configdir}/identity" + (f".{service_name}" if service_name and len(service_name) > 0 else "")
|
||||
|
||||
identity = None
|
||||
if os.path.isfile(identity_path):
|
||||
RNS.log(f"Loading identity from {identity_path}", RNS.LOG_VERBOSE)
|
||||
identity = RNS.Identity.from_file(identity_path)
|
||||
|
||||
if identity is None:
|
||||
RNS.log(f"Creating identity {identity_path}", RNS.LOG_NOTICE)
|
||||
identity = RNS.Identity()
|
||||
identity.to_file(identity_path)
|
||||
|
||||
return identity
|
||||
|
||||
|
||||
def print_identity(configdir, rnsconfigdir, identitypath, service_name, include_destination: bool):
|
||||
reticulum = RNS.Reticulum(configdir=rnsconfigdir, loglevel=RNS.LOG_INFO)
|
||||
if service_name and len(service_name) > 0: print(f"Using service name \"{service_name}\"")
|
||||
identity = prepare_identity(identity_path=identitypath, service_name=service_name, configdir=configdir)
|
||||
destination = RNS.Destination(identity, RNS.Destination.IN, RNS.Destination.SINGLE, APP_NAME)
|
||||
print("Identity : " + str(identity))
|
||||
if include_destination: print("Listening on : " + RNS.prettyhexrep(destination.hash))
|
||||
|
||||
exit(0)
|
||||
|
||||
verbose_set = False
|
||||
|
||||
def ensure_config_directory(configdir=None):
|
||||
if configdir:
|
||||
if os.path.isdir(os.path.expanduser(configdir)): return os.path.expanduser(configdir)
|
||||
else:
|
||||
try:
|
||||
RNS.log(f"Creating configuration directory: {os.path.expanduser(configdir)}")
|
||||
os.makedirs(os.path.expanduser(configdir))
|
||||
return os.path.expanduser(configdir)
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Could not get or create rnsh configuration directory, aborting", RNS.LOG_CRITICAL)
|
||||
os._exit(1)
|
||||
|
||||
elif os.path.isdir(os.path.expanduser("~/.config/rnsh")): return os.path.expanduser("~/.config/rnsh")
|
||||
elif os.path.isdir(os.path.expanduser("~/.rnsh")): return os.path.expanduser("~/.rnsh")
|
||||
else:
|
||||
try:
|
||||
RNS.log(f"Creating configuration directory: {os.path.expanduser('~/.rnsh')}")
|
||||
os.makedirs(os.path.expanduser("~/.rnsh"))
|
||||
return os.path.expanduser("~/.rnsh")
|
||||
|
||||
except Exception as e:
|
||||
RNS.log(f"Could not get or create rnsh configuration directory, aborting", RNS.LOG_CRITICAL)
|
||||
os._exit(1)
|
||||
|
||||
|
||||
async def _rnsh_cli_main():
|
||||
global verbose_set
|
||||
args, parser = parse_arguments()
|
||||
verbose_set = args.verbose > 0
|
||||
configdir = ensure_config_directory(args.config)
|
||||
|
||||
if not configdir:
|
||||
RNS.log(f"Could not resolve rnsh configuration directory", RNS.LOG_CRITICAL)
|
||||
os._exit(1)
|
||||
|
||||
if args.print_identity:
|
||||
print_identity(configdir, args.rnsconfig, args.identity, args.service, args.listen)
|
||||
return 0
|
||||
|
||||
logfile = f"{configdir}/logfile"
|
||||
|
||||
if args.listen:
|
||||
allowed_file = None
|
||||
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
|
||||
if os.path.isfile(os.path.expanduser(f"{configdir}/allowed_identities")):
|
||||
allowed_file = os.path.expanduser(f"{configdir}/allowed_identities")
|
||||
|
||||
await listener.listen(configdir=configdir,
|
||||
rnsconfigdir=args.rnsconfig,
|
||||
command=args.command,
|
||||
identitypath=args.identity,
|
||||
logfile=logfile,
|
||||
service_name=args.service,
|
||||
verbosity=args.verbose,
|
||||
quietness=args.quiet,
|
||||
allowed=args.allowed or [],
|
||||
allowed_file=allowed_file,
|
||||
disable_auth=args.no_auth,
|
||||
announce_period=args.announce,
|
||||
no_remote_command=args.no_remote_command,
|
||||
remote_cmd_as_args=args.remote_command_as_args)
|
||||
return 0
|
||||
|
||||
if args.destination is not None:
|
||||
return_code = await initiator.initiate(configdir=configdir,
|
||||
rnsconfigdir=args.rnsconfig,
|
||||
identitypath=args.identity,
|
||||
logfile=f"{logfile}.initiator",
|
||||
verbosity=args.verbose,
|
||||
quietness=args.quiet,
|
||||
noid=args.no_id,
|
||||
destination=args.destination,
|
||||
timeout=args.timeout,
|
||||
command=args.command
|
||||
)
|
||||
RNS.log(f"Sesssion with <{args.destination}> ended", RNS.LOG_INFO)
|
||||
return return_code if args.mirror else 0
|
||||
else:
|
||||
print("")
|
||||
parser.print_help()
|
||||
print("")
|
||||
return 1
|
||||
|
||||
|
||||
def main():
|
||||
global verbose_set
|
||||
return_code = 1
|
||||
exc = None
|
||||
try: return_code = asyncio.run(_rnsh_cli_main())
|
||||
except SystemExit: pass
|
||||
except KeyboardInterrupt: pass
|
||||
except Exception as e:
|
||||
print(f"{e}")
|
||||
exc = e
|
||||
|
||||
process.tty_unset_reader_callbacks(0)
|
||||
if verbose_set and exc: raise exc
|
||||
sys.exit(return_code if return_code is not None else 255)
|
||||
|
||||
if __name__ == "__main__": main()
|
||||
445
RNS/Utilities/rnsh/session.py
Normal file
445
RNS/Utilities/rnsh/session.py
Normal file
|
|
@ -0,0 +1,445 @@
|
|||
# Based on the original rnsh program by Aaron Heise (@acehoss)
|
||||
# https://github.com/acehoss/rnsh - MIT License - Copyright (c) 2023 Aaron Heise
|
||||
# This version of rnsh is included in RNS under the Reticulum License
|
||||
#
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# - The Software shall not be used in any kind of system which includes amongst
|
||||
# its functions the ability to purposefully do harm to human beings.
|
||||
#
|
||||
# - The Software shall not be used, directly or indirectly, in the creation of
|
||||
# an artificial intelligence, machine learning or language model training
|
||||
# dataset, including but not limited to any use that contributes to the
|
||||
# training or development of such a model or algorithm.
|
||||
#
|
||||
# - The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
from __future__ import annotations
|
||||
import contextlib
|
||||
import functools
|
||||
import asyncio
|
||||
import RNS.Utilities.rnsh.exception as exception
|
||||
import RNS.Utilities.rnsh.process as process
|
||||
import RNS.Utilities.rnsh.helpers as helpers
|
||||
import RNS.Utilities.rnsh.protocol as protocol
|
||||
import enum
|
||||
from typing import TypeVar, Generic, Callable, List
|
||||
from abc import abstractmethod, ABC
|
||||
import os
|
||||
import bz2
|
||||
import RNS
|
||||
|
||||
_TLink = TypeVar("_TLink")
|
||||
_TIdentity = TypeVar("_TIdentity")
|
||||
|
||||
class SEType(enum.IntEnum):
|
||||
SE_LINK_CLOSED = 0
|
||||
|
||||
class SessionException(Exception):
|
||||
def __init__(self, setype: SEType, msg: str, *args):
|
||||
super().__init__(msg, args)
|
||||
self.type = setype
|
||||
|
||||
class LSState(enum.IntEnum):
|
||||
LSSTATE_WAIT_IDENT = 1
|
||||
LSSTATE_WAIT_VERS = 2
|
||||
LSSTATE_WAIT_CMD = 3
|
||||
LSSTATE_RUNNING = 4
|
||||
LSSTATE_ERROR = 5
|
||||
LSSTATE_TEARDOWN = 6
|
||||
|
||||
|
||||
class LSOutletBase(ABC):
|
||||
@abstractmethod
|
||||
def set_initiator_identified_callback(self, cb: Callable[[LSOutletBase, _TIdentity], None]): raise NotImplemented()
|
||||
|
||||
@abstractmethod
|
||||
def set_link_closed_callback(self, cb: Callable[[LSOutletBase], None]): raise NotImplemented()
|
||||
|
||||
@abstractmethod
|
||||
def unset_link_closed_callback(self): raise NotImplemented()
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def rtt(self): raise NotImplemented()
|
||||
|
||||
@abstractmethod
|
||||
def teardown(self): raise NotImplemented()
|
||||
|
||||
|
||||
class ListenerSession:
|
||||
sessions: List[ListenerSession] = []
|
||||
allowed_identity_hashes: [any] = []
|
||||
allowed_file_identity_hashes: [any] = []
|
||||
allow_all: bool = False
|
||||
allow_remote_command: bool = False
|
||||
default_command: [str] = []
|
||||
remote_cmd_as_args = False
|
||||
|
||||
def __init__(self, outlet: LSOutletBase, channel: RNS.Channel.Channel, loop: asyncio.AbstractEventLoop):
|
||||
RNS.log(f"Session started for {outlet}", RNS.LOG_INFO)
|
||||
self.outlet = outlet
|
||||
self.channel = channel
|
||||
self.outlet.set_initiator_identified_callback(self._initiator_identified)
|
||||
self.outlet.set_link_closed_callback(self._link_closed)
|
||||
self.loop = loop
|
||||
self.state: LSState = None
|
||||
self.terminated = False
|
||||
self.authenticated = False
|
||||
self.remote_identity = None
|
||||
self.term: str | None = None
|
||||
self.stdin_is_pipe: bool = False
|
||||
self.stdout_is_pipe: bool = False
|
||||
self.stderr_is_pipe: bool = False
|
||||
self.tcflags: [any] = None
|
||||
self.cmdline: [str] = None
|
||||
self.rows: int = 0
|
||||
self.cols: int = 0
|
||||
self.hpix: int = 0
|
||||
self.vpix: int = 0
|
||||
self.stdout_buf = bytearray()
|
||||
self.stdout_eof_sent = False
|
||||
self.stderr_buf = bytearray()
|
||||
self.stderr_eof_sent = False
|
||||
self.return_code: int | None = None
|
||||
self.return_code_sent = False
|
||||
self.process: process.CallbackSubprocess | None = None
|
||||
|
||||
if not self.allow_all: self._set_state(LSState.LSSTATE_WAIT_IDENT)
|
||||
else:
|
||||
self._set_state(LSState.LSSTATE_WAIT_VERS)
|
||||
self.authenticated = True
|
||||
|
||||
self.sessions.append(self)
|
||||
protocol.register_message_types(self.channel)
|
||||
self.channel.add_message_handler(self._handle_message)
|
||||
|
||||
def _terminated(self, return_code: int):
|
||||
self.return_code = return_code
|
||||
|
||||
def _set_state(self, state: LSState, timeout_factor: float = 10.0):
|
||||
timeout = max(self.outlet.rtt * timeout_factor, max(self.outlet.rtt * 2, 10)) if timeout_factor is not None else None
|
||||
RNS.log(f"Set state: {state.name}, timeout {timeout}", RNS.LOG_DEBUG)
|
||||
orig_state = self.state
|
||||
self.state = state
|
||||
if timeout_factor is not None:
|
||||
self._call(functools.partial(self._check_protocol_timeout, lambda: self.state == orig_state, state.name), timeout)
|
||||
|
||||
def _call(self, func: callable, delay: float = 0):
|
||||
def call_inner():
|
||||
if delay == 0: func()
|
||||
else: self.loop.call_later(delay, func)
|
||||
|
||||
self.loop.call_soon_threadsafe(call_inner)
|
||||
|
||||
def send(self, message: RNS.MessageBase): self.channel.send(message)
|
||||
def _protocol_error(self, name: str): self.terminate(f"Protocol error ({name})")
|
||||
def _protocol_timeout_error(self, name: str): self.terminate(f"Protocol timeout error: {name}")
|
||||
|
||||
def terminate(self, error: str = None):
|
||||
self.terminated = True
|
||||
self.state = LSState.LSSTATE_ERROR
|
||||
with contextlib.suppress(Exception):
|
||||
RNS.log("Terminating session" + (f": {error}" if error else ""), RNS.LOG_DEBUG)
|
||||
if error and self.state != LSState.LSSTATE_TEARDOWN:
|
||||
with contextlib.suppress(Exception):
|
||||
self.send(protocol.ErrorMessage(error, True))
|
||||
|
||||
self._terminate_process()
|
||||
self._call(self._prune, max(self.outlet.rtt * 3, process.CallbackSubprocess.PROCESS_PIPE_TIME+5))
|
||||
|
||||
def _prune(self):
|
||||
self.state = LSState.LSSTATE_TEARDOWN
|
||||
RNS.log("Pruning session", RNS.LOG_DEBUG)
|
||||
with contextlib.suppress(ValueError): self.sessions.remove(self)
|
||||
with contextlib.suppress(Exception): self.outlet.teardown()
|
||||
|
||||
def _check_protocol_timeout(self, fail_condition: Callable[[], bool], name: str):
|
||||
timeout = True
|
||||
try: timeout = self.state != LSState.LSSTATE_TEARDOWN and fail_condition()
|
||||
except Exception as e: RNS.log(f"Error in protocol timeout: {e}", RNS.LOG_ERROR)
|
||||
if timeout: self._protocol_timeout_error(name)
|
||||
|
||||
def _link_closed(self, outlet: LSOutletBase):
|
||||
outlet.unset_link_closed_callback()
|
||||
if outlet != self.outlet:
|
||||
RNS.log("Link closed received from incorrect outlet", RNS.LOG_DEBUG)
|
||||
return
|
||||
|
||||
RNS.log(f"link_closed {outlet}", RNS.LOG_DEBUG)
|
||||
self.terminate()
|
||||
|
||||
def _initiator_identified(self, outlet, identity):
|
||||
if outlet != self.outlet:
|
||||
RNS.log("Identity received from incorrect outlet", RNS.LOG_DEBUG)
|
||||
return
|
||||
|
||||
RNS.log(f"Initiator identified {identity} on link {outlet}", RNS.LOG_INFO)
|
||||
if self.state not in [LSState.LSSTATE_WAIT_IDENT, LSState.LSSTATE_WAIT_VERS]:
|
||||
self._protocol_error(LSState.LSSTATE_WAIT_IDENT.name)
|
||||
return
|
||||
|
||||
if self.allow_all or identity.hash in self.allowed_identity_hashes or identity.hash in self.allowed_file_identity_hashes:
|
||||
RNS.log(f"Identity {identity} authenticated")
|
||||
self.authenticated = True
|
||||
self.remote_identity = identity
|
||||
self._set_state(LSState.LSSTATE_WAIT_VERS)
|
||||
|
||||
else:
|
||||
self.authenticated = False
|
||||
self.terminate("Identity not allowed")
|
||||
RNS.log(f"Identity {identity} not allowed")
|
||||
|
||||
@classmethod
|
||||
async def pump_all(cls) -> True:
|
||||
processed_any = False
|
||||
for session in cls.sessions:
|
||||
processed = session.pump()
|
||||
processed_any = processed_any or processed
|
||||
await asyncio.sleep(0)
|
||||
|
||||
|
||||
@classmethod
|
||||
async def terminate_all(cls, reason: str):
|
||||
for session in cls.sessions:
|
||||
session.terminate(reason)
|
||||
await asyncio.sleep(0)
|
||||
|
||||
def pump(self) -> bool:
|
||||
def compress_adaptive(buf: bytes):
|
||||
comp_tries = RNS.RawChannelWriter.COMPRESSION_TRIES
|
||||
comp_try = 1
|
||||
comp_success = False
|
||||
|
||||
chunk_len = len(buf)
|
||||
if chunk_len > RNS.RawChannelWriter.MAX_CHUNK_LEN:
|
||||
chunk_len = RNS.RawChannelWriter.MAX_CHUNK_LEN
|
||||
chunk_segment = None
|
||||
|
||||
chunk_segment = None
|
||||
max_data_len = self.channel.mdu - protocol.StreamDataMessage.HEADER_LEN
|
||||
while chunk_len > 32 and comp_try < comp_tries:
|
||||
chunk_segment_length = int(chunk_len/comp_try)
|
||||
compressed_chunk = bz2.compress(buf[:chunk_segment_length])
|
||||
compressed_length = len(compressed_chunk)
|
||||
if compressed_length < max_data_len and compressed_length < chunk_segment_length:
|
||||
comp_success = True
|
||||
break
|
||||
else: comp_try += 1
|
||||
|
||||
if comp_success:
|
||||
diff = max_data_len - len(compressed_chunk)
|
||||
chunk = compressed_chunk
|
||||
processed_length = chunk_segment_length
|
||||
else:
|
||||
chunk = bytes(buf[:max_data_len])
|
||||
processed_length = len(chunk)
|
||||
|
||||
return comp_success, processed_length, chunk
|
||||
|
||||
try:
|
||||
if self.state != LSState.LSSTATE_RUNNING: return False
|
||||
elif not self.channel.is_ready_to_send(): return False
|
||||
elif len(self.stderr_buf) > 0:
|
||||
comp_success, processed_length, data = compress_adaptive(self.stderr_buf)
|
||||
self.stderr_buf = self.stderr_buf[processed_length:]
|
||||
send_eof = self.process.stderr_eof and len(data) == 0 and not self.stderr_eof_sent
|
||||
self.stderr_eof_sent = self.stderr_eof_sent or send_eof
|
||||
msg = protocol.StreamDataMessage(protocol.StreamDataMessage.STREAM_ID_STDERR,
|
||||
data, send_eof, comp_success)
|
||||
self.send(msg)
|
||||
if send_eof: self.stderr_eof_sent = True
|
||||
return True
|
||||
elif len(self.stdout_buf) > 0:
|
||||
comp_success, processed_length, data = compress_adaptive(self.stdout_buf)
|
||||
self.stdout_buf = self.stdout_buf[processed_length:]
|
||||
send_eof = self.process.stdout_eof and len(data) == 0 and not self.stdout_eof_sent
|
||||
self.stdout_eof_sent = self.stdout_eof_sent or send_eof
|
||||
msg = protocol.StreamDataMessage(protocol.StreamDataMessage.STREAM_ID_STDOUT,
|
||||
data, send_eof, comp_success)
|
||||
self.send(msg)
|
||||
if send_eof: self.stdout_eof_sent = True
|
||||
return True
|
||||
elif self.return_code is not None and not self.return_code_sent:
|
||||
msg = protocol.CommandExitedMessage(self.return_code)
|
||||
self.send(msg)
|
||||
self.return_code_sent = True
|
||||
self._call(functools.partial(self._check_protocol_timeout,
|
||||
lambda: self.state == LSState.LSSTATE_RUNNING, "CommandExitedMessage"),
|
||||
max(self.outlet.rtt * 5, 10))
|
||||
return False
|
||||
|
||||
except Exception as e: RNS.log(f"Error during pump: {e}", RNS.LOG_ERROR)
|
||||
return False
|
||||
|
||||
def _terminate_process(self):
|
||||
with contextlib.suppress(Exception):
|
||||
if self.process and self.process.running: self.process.terminate()
|
||||
|
||||
def _start_cmd(self, cmdline: [str], pipe_stdin: bool, pipe_stdout: bool, pipe_stderr: bool, tcflags: [any],
|
||||
term: str | None, rows: int, cols: int, hpix: int, vpix: int):
|
||||
|
||||
if self.terminated:
|
||||
RNS.log(f"Attempt to start command on terminated session: {cmdline}", RNS.LOG_WARNING)
|
||||
return
|
||||
|
||||
if not self.authenticated:
|
||||
RNS.log(f"Attempt to start command on unauthenticated session: {cmdline}", RNS.LOG_WARNING)
|
||||
self.terminate("Invalid state")
|
||||
return
|
||||
|
||||
if self.state != LSState.LSSTATE_RUNNING:
|
||||
RNS.log(f"Attempt to start command on session not in running state: {cmdline}", RNS.LOG_WARNING)
|
||||
self.terminate("Invalid state")
|
||||
return
|
||||
|
||||
self.cmdline = []
|
||||
self.cmdline.extend(self.default_command)
|
||||
if not self.allow_remote_command and cmdline and len(cmdline) > 0:
|
||||
self.terminate("Remote command line not allowed by listener")
|
||||
return
|
||||
|
||||
if self.remote_cmd_as_args and cmdline and len(cmdline) > 0: self.cmdline.extend(cmdline)
|
||||
elif cmdline and len(cmdline) > 0: self.cmdline = cmdline
|
||||
|
||||
self.stdin_is_pipe = pipe_stdin
|
||||
self.stdout_is_pipe = pipe_stdout
|
||||
self.stderr_is_pipe = pipe_stderr
|
||||
self.tcflags = tcflags
|
||||
self.term = term
|
||||
|
||||
def stdout(data: bytes): self.stdout_buf.extend(data)
|
||||
def stderr(data: bytes): self.stderr_buf.extend(data)
|
||||
|
||||
try:
|
||||
RNS.log(f"Remote {self.remote_identity} executing: {self.cmdline}", RNS.LOG_VERBOSE)
|
||||
self.process = process.CallbackSubprocess(argv=self.cmdline,
|
||||
env={"TERM": self.term or os.environ.get("TERM") or "xterm",
|
||||
"RNS_REMOTE_IDENTITY": (RNS.prettyhexrep(self.remote_identity.hash)
|
||||
if self.remote_identity and self.remote_identity.hash else "")},
|
||||
loop=self.loop,
|
||||
stdout_callback=stdout,
|
||||
stderr_callback=stderr,
|
||||
terminated_callback=self._terminated,
|
||||
stdin_is_pipe=self.stdin_is_pipe,
|
||||
stdout_is_pipe=self.stdout_is_pipe,
|
||||
stderr_is_pipe=self.stderr_is_pipe)
|
||||
self.process.start()
|
||||
self._set_window_size(rows, cols, hpix, vpix)
|
||||
except Exception as e:
|
||||
RNS.log(f"Unable to start process for link {self.outlet}: {e}", RNS.LOG_ERROR)
|
||||
self.terminate("Unable to start process")
|
||||
|
||||
def _set_window_size(self, rows: int, cols: int, hpix: int, vpix: int):
|
||||
self.rows = rows
|
||||
self.cols = cols
|
||||
self.hpix = hpix
|
||||
self.vpix = vpix
|
||||
with contextlib.suppress(Exception): self.process.set_winsize(rows, cols, hpix, vpix)
|
||||
|
||||
def _received_stdin(self, data: bytes, eof: bool):
|
||||
if data and len(data) > 0: self.process.write(data)
|
||||
if eof: self.process.close_stdin()
|
||||
|
||||
def _handle_message(self, message: RNS.MessageBase):
|
||||
if self.terminated:
|
||||
RNS.log(f"Received packet, but session was terminated", RNS.LOG_ERROR)
|
||||
return
|
||||
if self.state > LSState.LSSTATE_RUNNING:
|
||||
RNS.log(f"Received packet, but in state {self.state.name}", RNS.LOG_ERROR)
|
||||
return
|
||||
if self.state == LSState.LSSTATE_WAIT_IDENT:
|
||||
# Ignore any messages until the initiator has identified to avoid race conditions
|
||||
# between identity announcement and early protocol messages.
|
||||
RNS.log("Ignoring message while waiting for identification", RNS.LOG_DEBUG)
|
||||
return
|
||||
elif self.state == LSState.LSSTATE_WAIT_VERS:
|
||||
if not self.authenticated:
|
||||
self.terminate("Invalid state")
|
||||
return
|
||||
if not isinstance(message, protocol.VersionInfoMessage):
|
||||
self._protocol_error(self.state.name)
|
||||
return
|
||||
RNS.log(f"Version {message.sw_version}, protocol {message.protocol_version} on link {self.outlet}", RNS.LOG_VERBOSE)
|
||||
if message.protocol_version != protocol.PROTOCOL_VERSION:
|
||||
self.terminate("Incompatible protocol")
|
||||
return
|
||||
self.send(protocol.VersionInfoMessage())
|
||||
self._set_state(LSState.LSSTATE_WAIT_CMD)
|
||||
return
|
||||
elif self.state == LSState.LSSTATE_WAIT_CMD:
|
||||
if not self.authenticated:
|
||||
self.terminate("Invalid state")
|
||||
return
|
||||
if not isinstance(message, protocol.ExecuteCommandMesssage):
|
||||
return self._protocol_error(self.state.name)
|
||||
RNS.log(f"Execute command message on link {self.outlet}: {message.cmdline}", RNS.LOG_VERBOSE)
|
||||
self._set_state(LSState.LSSTATE_RUNNING)
|
||||
self._start_cmd(message.cmdline, message.pipe_stdin, message.pipe_stdout, message.pipe_stderr,
|
||||
message.tcflags, message.term, message.rows, message.cols, message.hpix, message.vpix)
|
||||
return
|
||||
elif self.state == LSState.LSSTATE_RUNNING:
|
||||
if not self.authenticated:
|
||||
self.terminate("Invalid state")
|
||||
return
|
||||
if isinstance(message, protocol.WindowSizeMessage):
|
||||
self._set_window_size(message.rows, message.cols, message.hpix, message.vpix)
|
||||
elif isinstance(message, protocol.StreamDataMessage):
|
||||
if message.stream_id != protocol.StreamDataMessage.STREAM_ID_STDIN:
|
||||
RNS.log(f"Received stream data for invalid stream {message.stream_id} on link {self.outlet}", RNS.LOG_ERROR)
|
||||
return self._protocol_error(self.state.name)
|
||||
self._received_stdin(message.data, message.eof)
|
||||
return
|
||||
elif isinstance(message, protocol.NoopMessage):
|
||||
# echo noop only on listener--used for keepalive/connectivity check
|
||||
self.send(message)
|
||||
return
|
||||
else:
|
||||
self._protocol_error("unexpected message")
|
||||
return
|
||||
|
||||
|
||||
class RNSOutlet(LSOutletBase):
|
||||
|
||||
def set_initiator_identified_callback(self, cb: Callable[[LSOutletBase, _TIdentity], None]):
|
||||
def inner_cb(link, identity: _TIdentity): cb(self, identity)
|
||||
self.link.set_remote_identified_callback(inner_cb)
|
||||
|
||||
def set_link_closed_callback(self, cb: Callable[[LSOutletBase], None]):
|
||||
def inner_cb(link): cb(self)
|
||||
self.link.set_link_closed_callback(inner_cb)
|
||||
|
||||
def unset_link_closed_callback(self): self.link.set_link_closed_callback(None)
|
||||
def teardown(self): self.link.teardown()
|
||||
|
||||
@property
|
||||
def rtt(self) -> float: return self.link.rtt
|
||||
|
||||
def __str__(self): return f"Outlet on {self.link}"
|
||||
|
||||
def __init__(self, link: RNS.Link):
|
||||
self.link = link
|
||||
link.lsoutlet = self
|
||||
|
||||
@staticmethod
|
||||
def get_outlet(link: RNS.Link):
|
||||
if hasattr(link, "lsoutlet"): return link.lsoutlet
|
||||
return RNSOutlet(link)
|
||||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -60,9 +60,13 @@ def size_str(num, suffix='B'):
|
|||
|
||||
request_result = None
|
||||
request_concluded = False
|
||||
def get_remote_status(destination_hash, include_lstats, identity, no_output=False, timeout=RNS.Transport.PATH_REQUEST_TIMEOUT):
|
||||
global request_result, request_concluded
|
||||
first_remote_req = True
|
||||
remote_destination = None
|
||||
remote_link = None
|
||||
def get_remote_status(destination_hash, include_lstats, include_profiling, identity, no_output=False, timeout=RNS.Transport.PATH_REQUEST_TIMEOUT):
|
||||
global request_result, request_concluded, first_remote_req, remote_destination, remote_link
|
||||
link_count = None
|
||||
profiling_results = None
|
||||
|
||||
if not RNS.Transport.has_path(destination_hash):
|
||||
if not no_output:
|
||||
|
|
@ -81,7 +85,8 @@ def get_remote_status(destination_hash, include_lstats, identity, no_output=Fals
|
|||
remote_identity = RNS.Identity.recall(destination_hash)
|
||||
|
||||
def remote_link_closed(link):
|
||||
if link.teardown_reason == RNS.Link.TIMEOUT:
|
||||
if link.teardown_reason == RNS.Link.INITIATOR_CLOSED: return
|
||||
elif link.teardown_reason == RNS.Link.TIMEOUT:
|
||||
if not no_output:
|
||||
print("\r \r", end="")
|
||||
print("The link timed out, exiting now")
|
||||
|
|
@ -107,44 +112,55 @@ def get_remote_status(destination_hash, include_lstats, identity, no_output=Fals
|
|||
response = request_receipt.response
|
||||
if isinstance(response, list):
|
||||
status = response[0]
|
||||
if len(response) > 1:
|
||||
link_count = response[1]
|
||||
else:
|
||||
link_count = None
|
||||
if len(response) > 1: link_count = response[1]
|
||||
else: link_count = None
|
||||
|
||||
request_result = (status, link_count)
|
||||
if len(response) > 2: profiling_results = response[2]
|
||||
else: profiling_results = None
|
||||
|
||||
request_result = (status, link_count, profiling_results)
|
||||
|
||||
request_concluded = True
|
||||
|
||||
def remote_link_established(link):
|
||||
if not no_output:
|
||||
global first_remote_req
|
||||
if not no_output and first_remote_req:
|
||||
print("\r \r", end="")
|
||||
print("Sending request...", end=" ")
|
||||
sys.stdout.flush()
|
||||
link.identify(identity)
|
||||
link.request("/status", data = [include_lstats], response_callback = got_response, failed_callback = request_failed)
|
||||
link.request("/status", data = [include_lstats, include_profiling], response_callback = got_response, failed_callback = request_failed)
|
||||
first_remote_req = False
|
||||
|
||||
if not no_output:
|
||||
if not remote_link and not no_output:
|
||||
print("\r \r", end="")
|
||||
print("Establishing link with remote transport instance...", end=" ")
|
||||
sys.stdout.flush()
|
||||
|
||||
remote_destination = RNS.Destination(remote_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, "rnstransport", "remote", "management")
|
||||
link = RNS.Link(remote_destination)
|
||||
link.set_link_established_callback(remote_link_established)
|
||||
link.set_link_closed_callback(remote_link_closed)
|
||||
if not remote_destination:
|
||||
remote_destination = RNS.Destination(remote_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, "rnstransport", "remote", "management")
|
||||
|
||||
if remote_link and remote_link.status == RNS.Link.ACTIVE:
|
||||
request_concluded = False
|
||||
remote_link.request("/status", data = [include_lstats], response_callback = got_response, failed_callback = request_failed)
|
||||
|
||||
while not request_concluded:
|
||||
time.sleep(0.1)
|
||||
else:
|
||||
remote_link = RNS.Link(remote_destination)
|
||||
remote_link.set_link_established_callback(remote_link_established)
|
||||
remote_link.set_link_closed_callback(remote_link_closed)
|
||||
|
||||
while not request_concluded: time.sleep(0.1)
|
||||
|
||||
if request_result != None:
|
||||
print("\r \r", end="")
|
||||
|
||||
return request_result
|
||||
|
||||
def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=False, astats=False, lstats=False, sorting=None, sort_reverse=False,
|
||||
remote=None, management_identity=None, remote_timeout=RNS.Transport.PATH_REQUEST_TIMEOUT, must_exit=True, rns_instance=None,
|
||||
traffic_totals=False, discovered_interfaces=False, config_entries=False):
|
||||
def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=False, astats=False, pstats=False, lstats=False,
|
||||
sorting=None, sort_reverse=False, remote=None, management_identity=None, must_exit=True, rns_instance=None,
|
||||
traffic_totals=False, discovered_interfaces=False, config_entries=False, burst_filter=False, blocked_ips=False,
|
||||
queue_stats=False, pps=False, profiling=False, remote_timeout=RNS.Transport.PATH_REQUEST_TIMEOUT,
|
||||
attach=None, detach=None, reload=None, show_stale=False, show_unknown=False, fstats=False, tstats=False):
|
||||
|
||||
if remote: require_shared = False
|
||||
else: require_shared = True
|
||||
|
|
@ -155,14 +171,48 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
must_exit = False
|
||||
else:
|
||||
reticulum = RNS.Reticulum(configdir=configdir, loglevel=3+verbosity, require_shared_instance=require_shared)
|
||||
reticulum.wants_destination_type("rnstransport.remote.management")
|
||||
|
||||
except Exception as e:
|
||||
print("No shared RNS instance available to get status from")
|
||||
if must_exit: exit(1)
|
||||
else: return
|
||||
|
||||
if attach:
|
||||
result = reticulum.attach_interface(attach)
|
||||
if result == True: print(f"Interface {attach} was attached")
|
||||
elif result == False: print(f"Could not attach interface {attach}")
|
||||
elif result == None: print(f"The interface {attach} does not exist")
|
||||
else: print(f"Unknown error while attaching interface {attach}")
|
||||
|
||||
if result == True: exit(0)
|
||||
else: exit(1)
|
||||
|
||||
if detach:
|
||||
result = reticulum.detach_interface(detach)
|
||||
if result == True: print(f"Interface {detach} was detached")
|
||||
elif result == False: print(f"Could not detach interface {detach}")
|
||||
elif result == None: print(f"The interface {detach} does not exist")
|
||||
else: print(f"Unknown error while detaching interface {detach}")
|
||||
|
||||
if result == True: exit(0)
|
||||
else: exit(1)
|
||||
|
||||
if reload:
|
||||
result = reticulum.reload_interface(reload)
|
||||
if result == True: print(f"Interface {reload} was reloaded")
|
||||
elif result == False: print(f"Could not reload interface {reload}")
|
||||
elif result == None: print(f"The interface {reload} does not exist")
|
||||
else: print(f"Unknown error while reloading interface {reload}")
|
||||
|
||||
if result == True: exit(0)
|
||||
else: exit(1)
|
||||
|
||||
|
||||
link_count = None
|
||||
active_link_count = None
|
||||
stats = None
|
||||
profiling_results = None
|
||||
|
||||
details = False
|
||||
if config_entries:
|
||||
|
|
@ -194,12 +244,17 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
name = i["name"]
|
||||
if_type = i["type"]
|
||||
status = i["status"]
|
||||
has_impl_info = "impl_name" in i and "version" in i and i["impl_name"] and i["version"]
|
||||
impl_str = f"{i['impl_name']} {i['version']}" if has_impl_info else "Unknown"
|
||||
|
||||
if status == "available": status_display = "Available"
|
||||
elif status == "unknown": status_display = "Unknown"
|
||||
elif status == "stale": status_display = "Stale"
|
||||
else: status_display = status
|
||||
|
||||
if status == "stale" and not show_stale: continue
|
||||
if not has_impl_info and not show_unknown: continue
|
||||
|
||||
now = time.time()
|
||||
dago = now-i["discovered"]
|
||||
hago = now-i["last_heard"]
|
||||
|
|
@ -207,6 +262,14 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
last_heard_display = f"{RNS.prettytime(hago, compact=True)} ago"
|
||||
transport_str = "Enabled" if i["transport"] else "Disabled"
|
||||
|
||||
has_cost_info = "dst_cost" in i and "pr_cost" in i
|
||||
dst_cost = None; pr_cost = None
|
||||
if not has_cost_info: cost_str = "Unknown"
|
||||
else:
|
||||
dst_cost = i["dst_cost"]
|
||||
pr_cost = i["pr_cost"]
|
||||
cost_str = f"Dst {RNS.prettycost(dst_cost)}, PR {RNS.prettycost(pr_cost)}"
|
||||
|
||||
if i["latitude"] is not None and i["longitude"] is not None:
|
||||
lat = round(i["latitude"], 4)
|
||||
lon = round(i["longitude"], 4)
|
||||
|
|
@ -215,20 +278,26 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
location = f"{lat}, {lon}{height}"
|
||||
else: location = "Unknown"
|
||||
|
||||
lxmf_addr = None
|
||||
transport_id = None
|
||||
network = None
|
||||
network = None
|
||||
if "operator_lxmf_address" in i: lxmf_addr = i['operator_lxmf_address']
|
||||
if "transport_id" in i: transport_id = i["transport_id"]
|
||||
if "transport_id" in i and "network_id" in i and i["transport_id"] != i["network_id"]:
|
||||
network = i["network_id"]
|
||||
|
||||
if idx > 0: print("\n"+"="*32+"\n")
|
||||
if idx > 0: print("\n"+"="*47+"\n")
|
||||
if network: print(f"Network ID : {network}")
|
||||
if transport_id: print(f"Transport ID : {transport_id}")
|
||||
if lxmf_addr: print(f"LXMF address : {lxmf_addr}")
|
||||
|
||||
print(f"Name : {name}")
|
||||
print(f"Type : {if_type}")
|
||||
print(f"Stack : {impl_str}")
|
||||
print(f"Status : {status_display}")
|
||||
print(f"Transport : {transport_str}")
|
||||
print(f"Stamp Value : {RNS.prettycost(i['value'])}")
|
||||
print(f"Costs : {cost_str}") if has_cost_info else None
|
||||
print(f"Distance : {i['hops']} hop{'' if i['hops'] == 1 else 's'}")
|
||||
print(f"Discovered : {discovered_display}")
|
||||
print(f"Last Heard : {last_heard_display}")
|
||||
|
|
@ -239,24 +308,25 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
if "sf" in i: print(f"Sprd. Factor : {i['sf']}")
|
||||
if "cr" in i: print(f"Coding Rate : {i['cr']}")
|
||||
if "modulation" in i: print(f"Modulation : {i['modulation']}")
|
||||
if "reachable_on" in i: print(f"Address : {i['reachable_on']}:{i['port']}")
|
||||
|
||||
print(f"Stamp Value : {i['value']}")
|
||||
if "reachable_on" in i: print(f"Address : {i['reachable_on']}")
|
||||
if "port" in i: print(f"Port : {i['port']}")
|
||||
|
||||
print(f"\nConfiguration Entry:")
|
||||
config_lines = i["config_entry"].split('\n')
|
||||
for line in config_lines: print(f" {line}")
|
||||
|
||||
except Exception as e:
|
||||
pass
|
||||
except Exception as e: pass
|
||||
|
||||
else:
|
||||
print(f"{'Name':<25} {'Type':<12} {'Status':<12} {'Last Heard':<12} {'Value':<8} {'Location':<15}")
|
||||
print("-" * 89)
|
||||
print(f"{'Name':<25} {'Type':<10} {'Status':<10} {'Last Heard':<12} {'Value':<7} {'Running':<16} {'Location':<15}")
|
||||
print("-" * 110)
|
||||
|
||||
for i in filtered_ifs:
|
||||
try:
|
||||
name = i["name"][:24] + "…" if len(i["name"]) > 24 else i["name"]
|
||||
has_impl_info = "impl_name" in i and "version" in i and i["impl_name"] and i["version"]
|
||||
impl_str = f"{i['impl_name']} {i['version']}" if has_impl_info else "Unknown"
|
||||
if len(impl_str) > 16: impl_str = impl_str[:15]+"…"
|
||||
|
||||
if_type = i["type"].replace("Interface", "")
|
||||
|
||||
|
|
@ -265,6 +335,9 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
elif status == "unknown": status_display = "? Unknown"
|
||||
elif status == "stale": status_display = "× Stale"
|
||||
else: status_display = status
|
||||
|
||||
if status == "stale" and not show_stale: continue
|
||||
if not has_impl_info and not show_unknown: continue
|
||||
|
||||
now = time.time()
|
||||
last_heard = i["last_heard"]
|
||||
|
|
@ -281,7 +354,7 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
days = int(diff / 86400)
|
||||
last_heard_display = f"{days}d ago"
|
||||
|
||||
value = str(i["value"])
|
||||
value = RNS.prettycost(i["value"])
|
||||
|
||||
if i["latitude"] is not None and i["longitude"] is not None:
|
||||
lat = round(i["latitude"], 4)
|
||||
|
|
@ -289,7 +362,7 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
location = f"{lat}, {lon}"
|
||||
else: location = "N/A"
|
||||
|
||||
print(f"{name:<25} {if_type:<12} {status_display:<12} {last_heard_display:<12} {value:<8} {location:<15}")
|
||||
print(f"{name:<25} {if_type:<10} {status_display:<12} {last_heard_display:<12} {value:<7} {impl_str:<16} {location:<15}")
|
||||
|
||||
except Exception as e:
|
||||
pass
|
||||
|
|
@ -299,28 +372,22 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
|
||||
if remote:
|
||||
try:
|
||||
if management_identity is None:
|
||||
raise ValueError("Remote management requires an identity file. Use -i to specify the path to a management identity.")
|
||||
if management_identity is None: raise ValueError("Remote management requires an identity file. Use -i to specify the path to a management identity.")
|
||||
|
||||
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
|
||||
if len(remote) != dest_len:
|
||||
raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
|
||||
if len(remote) != dest_len: raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
|
||||
try:
|
||||
identity_hash = bytes.fromhex(remote)
|
||||
destination_hash = RNS.Destination.hash_from_name_and_identity("rnstransport.remote.management", identity_hash)
|
||||
except Exception as e:
|
||||
raise ValueError("Invalid destination entered. Check your input.")
|
||||
except Exception as e: raise ValueError("Invalid destination entered. Check your input.")
|
||||
|
||||
identity = RNS.Identity.from_file(os.path.expanduser(management_identity))
|
||||
if identity == None:
|
||||
raise ValueError("Could not load management identity from "+str(management_identity))
|
||||
if identity == None: raise ValueError("Could not load management identity from "+str(management_identity))
|
||||
|
||||
try:
|
||||
remote_status = get_remote_status(destination_hash, lstats, identity, no_output=json, timeout=remote_timeout)
|
||||
if remote_status != None:
|
||||
stats, link_count = remote_status
|
||||
except Exception as e:
|
||||
raise e
|
||||
remote_status = get_remote_status(destination_hash, lstats, profiling, identity, no_output=json, timeout=remote_timeout)
|
||||
if remote_status != None: stats, link_count, profiling_results = remote_status
|
||||
except Exception as e: raise e
|
||||
|
||||
except Exception as e:
|
||||
print(str(e))
|
||||
|
|
@ -331,6 +398,12 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
if lstats:
|
||||
try: link_count = reticulum.get_link_count()
|
||||
except Exception as e: pass
|
||||
try: active_link_count = reticulum.get_active_link_count()
|
||||
except Exception as e: pass
|
||||
|
||||
if profiling:
|
||||
try: profiling_results = reticulum.get_profiling_results()
|
||||
except Exception as e: pass
|
||||
|
||||
try: stats = reticulum.get_interface_stats()
|
||||
except Exception as e: pass
|
||||
|
|
@ -356,28 +429,36 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
interfaces = stats["interfaces"]
|
||||
if sorting != None and isinstance(sorting, str):
|
||||
sorting = sorting.lower()
|
||||
if sorting == "rate" or sorting == "bitrate":
|
||||
interfaces.sort(key=lambda i: i["bitrate"], reverse=not sort_reverse)
|
||||
if sorting == "rx":
|
||||
interfaces.sort(key=lambda i: i["rxb"], reverse=not sort_reverse)
|
||||
if sorting == "tx":
|
||||
interfaces.sort(key=lambda i: i["txb"], reverse=not sort_reverse)
|
||||
if sorting == "rxs":
|
||||
interfaces.sort(key=lambda i: i["rxs"], reverse=not sort_reverse)
|
||||
if sorting == "txs":
|
||||
interfaces.sort(key=lambda i: i["txs"], reverse=not sort_reverse)
|
||||
if sorting == "traffic":
|
||||
interfaces.sort(key=lambda i: i["rxb"]+i["txb"], reverse=not sort_reverse)
|
||||
if sorting == "announces" or sorting == "announce":
|
||||
interfaces.sort(key=lambda i: i["incoming_announce_frequency"]+i["outgoing_announce_frequency"], reverse=not sort_reverse)
|
||||
if sorting == "arx":
|
||||
interfaces.sort(key=lambda i: i["incoming_announce_frequency"], reverse=not sort_reverse)
|
||||
if sorting == "atx":
|
||||
interfaces.sort(key=lambda i: i["outgoing_announce_frequency"], reverse=not sort_reverse)
|
||||
if sorting == "held":
|
||||
interfaces.sort(key=lambda i: i["held_announces"], reverse=not sort_reverse)
|
||||
if sorting == "rate" or sorting == "bitrate": interfaces.sort(key=lambda i: i["bitrate"], reverse=not sort_reverse)
|
||||
if sorting == "rx": interfaces.sort(key=lambda i: i["rxb"], reverse=not sort_reverse)
|
||||
if sorting == "tx": interfaces.sort(key=lambda i: i["txb"], reverse=not sort_reverse)
|
||||
if sorting == "frx": interfaces.sort(key=lambda i: i["frxb"], reverse=not sort_reverse)
|
||||
if sorting == "ftx": interfaces.sort(key=lambda i: i["ftxb"], reverse=not sort_reverse)
|
||||
if sorting == "rxs": interfaces.sort(key=lambda i: i["rxs"], reverse=not sort_reverse)
|
||||
if sorting == "txs": interfaces.sort(key=lambda i: i["txs"], reverse=not sort_reverse)
|
||||
if sorting == "frxs": interfaces.sort(key=lambda i: i["frxs"], reverse=not sort_reverse)
|
||||
if sorting == "ftxs": interfaces.sort(key=lambda i: i["ftxs"], reverse=not sort_reverse)
|
||||
if sorting == "traffic": interfaces.sort(key=lambda i: i["rxb"]+i["txb"], reverse=not sort_reverse)
|
||||
if sorting == "anns" or sorting == "announces": interfaces.sort(key=lambda i: i["incoming_announce_frequency"]+i["outgoing_announce_frequency"], reverse=not sort_reverse)
|
||||
if sorting == "arx": interfaces.sort(key=lambda i: i["incoming_announce_frequency"], reverse=not sort_reverse)
|
||||
if sorting == "atx": interfaces.sort(key=lambda i: i["outgoing_announce_frequency"], reverse=not sort_reverse)
|
||||
if sorting == "arxc": interfaces.sort(key=lambda i: i["arxc"], reverse=not sort_reverse)
|
||||
if sorting == "atxc": interfaces.sort(key=lambda i: i["atxc"], reverse=not sort_reverse)
|
||||
if sorting == "afrxc": interfaces.sort(key=lambda i: i["afrxc"], reverse=not sort_reverse)
|
||||
if sorting == "prx": interfaces.sort(key=lambda i: i["incoming_pr_frequency"], reverse=not sort_reverse)
|
||||
if sorting == "ptx": interfaces.sort(key=lambda i: i["outgoing_pr_frequency"], reverse=not sort_reverse)
|
||||
if sorting == "prxc": interfaces.sort(key=lambda i: i["prxc"], reverse=not sort_reverse)
|
||||
if sorting == "ptxc": interfaces.sort(key=lambda i: i["ptxc"], reverse=not sort_reverse)
|
||||
if sorting == "held": interfaces.sort(key=lambda i: i["held_announces"], reverse=not sort_reverse)
|
||||
if sorting == "pvs": interfaces.sort(key=lambda i: i["protocol_violations"], reverse=not sort_reverse)
|
||||
if sorting == "ivs": interfaces.sort(key=lambda i: i["ifac_violations"], reverse=not sort_reverse)
|
||||
if sorting == "flt": interfaces.sort(key=lambda i: i["packet_filter_hits"], reverse=not sort_reverse)
|
||||
if sorting == "gravity" or sorting == "g": interfaces.sort(key=lambda i: i["gravity"], reverse=not sort_reverse)
|
||||
if sorting == "txdrp": interfaces.sort(key=lambda i: i["txdrp"], reverse=not sort_reverse)
|
||||
if sorting == "txdrb": interfaces.sort(key=lambda i: i["txdrb"], reverse=not sort_reverse)
|
||||
if sorting == "txbuf": interfaces.sort(key=lambda i: i["txbuffered"], reverse=not sort_reverse)
|
||||
|
||||
|
||||
|
||||
for ifstat in interfaces:
|
||||
name = ifstat["name"]
|
||||
|
||||
|
|
@ -392,19 +473,33 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
):
|
||||
|
||||
if not (name.startswith("I2PInterface[") and ("i2p_connectable" in ifstat and ifstat["i2p_connectable"] == False)):
|
||||
if name_filter == None or name_filter.lower() in name.lower():
|
||||
if name_filter == None and burst_filter == None: show_if = True
|
||||
elif not burst_filter:
|
||||
if not name_filter or name_filter.lower() in name.lower(): show_if = True
|
||||
else: show_if = False
|
||||
elif burst_filter:
|
||||
burst_act = True if ("burst_active" in ifstat and "pr_burst_active" in ifstat) and (ifstat["burst_active"] or ifstat["pr_burst_active"]) else False
|
||||
nfilt = name_filter.lower() in name.lower() if name_filter else False
|
||||
if burst_act or nfilt: show_if = True
|
||||
else: show_if = False
|
||||
else: show_if = True
|
||||
|
||||
if show_if:
|
||||
print("")
|
||||
|
||||
if ifstat["status"]: ss = "Up"
|
||||
else: ss = "Down"
|
||||
|
||||
if "gravity" in ifstat and ifstat["gravity"]: ss += ", gravity "+str(ifstat["gravity"])
|
||||
|
||||
if ifstat["mode"] == RNS.Interfaces.Interface.Interface.MODE_ACCESS_POINT: modestr = "Access Point"
|
||||
elif ifstat["mode"] == RNS.Interfaces.Interface.Interface.MODE_POINT_TO_POINT: modestr = "Point-to-Point"
|
||||
elif ifstat["mode"] == RNS.Interfaces.Interface.Interface.MODE_ROAMING: modestr = "Roaming"
|
||||
elif ifstat["mode"] == RNS.Interfaces.Interface.Interface.MODE_BOUNDARY: modestr = "Boundary"
|
||||
elif ifstat["mode"] == RNS.Interfaces.Interface.Interface.MODE_GATEWAY: modestr = "Gateway"
|
||||
elif ifstat["mode"] == RNS.Interfaces.Interface.Interface.MODE_INTERNAL: modestr = "Internal"
|
||||
else: modestr = "Full"
|
||||
|
||||
if "announces_to_internal" in ifstat and ifstat["announces_to_internal"]: modestr += " (a>i)"
|
||||
|
||||
if ifstat["clients"] != None:
|
||||
clients = ifstat["clients"]
|
||||
|
|
@ -429,6 +524,12 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
clients_string = ""
|
||||
else:
|
||||
clients_string = "Clients : "+str(clients)
|
||||
if "blocked_ips" in ifstat:
|
||||
p = ifstat["blocked_ips"] > 0
|
||||
if p: clients_string += "\n Blocked : "+str(ifstat["blocked_ips"])+" IP"+"s" if p else ""
|
||||
|
||||
if blocked_ips and "blocked_ip_list" in ifstat and len(ifstat["blocked_ip_list"]) > 0:
|
||||
for bip in ifstat["blocked_ip_list"]: clients_string += f"\n {bip}"
|
||||
|
||||
else:
|
||||
clients = None
|
||||
|
|
@ -447,10 +548,28 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
print(" "+clients_string)
|
||||
|
||||
if not (name.startswith("Shared Instance[") or name.startswith("TCPInterface[Client") or name.startswith("LocalInterface[")):
|
||||
print(" Mode : {mode}".format(mode=modestr))
|
||||
print(f" Mode : {modestr}")
|
||||
|
||||
if ifstat["txdrp"]:
|
||||
print(f" TX Drops : {ifstat['txdrp']} ({RNS.prettysize(ifstat['txdrb'])})")
|
||||
|
||||
if "bitrate" in ifstat and ifstat["bitrate"] != None:
|
||||
print(" Rate : {ss}".format(ss=speed_str(ifstat["bitrate"])))
|
||||
print(f" Rate : {speed_str(ifstat['bitrate'])}, MTU {ifstat['mtu']}")
|
||||
|
||||
if tstats and not clients:
|
||||
print(f" Tunnel ID : {RNS.hexrep(ifstat['tunnel_id'], delimit=False) if ifstat['tunnel_id'] else 'None'}")
|
||||
|
||||
if tstats and ifstat["tunnel_id"]:
|
||||
print(f" Rem.Tr.ID : {RNS.prettyhexrep(ifstat['remote_transport_id']) if ifstat['remote_transport_id'] else 'Unknown'}")
|
||||
|
||||
trns_str = ", transport enabled" if ifstat['rem_is_tn'] else ", no transport"
|
||||
impl_str = f"{ifstat['rem_impl']}" if ifstat['rem_impl'] else "Unknown implementation"
|
||||
vers_str = f" {'.'.join([str(v) for v in ifstat['rem_vers']])}{trns_str}" if ifstat['rem_vers'] else ", unknown version"
|
||||
print(f" Rem.Vers. : {impl_str}{vers_str}")
|
||||
|
||||
dst_cost_str = f"Dst {RNS.prettycost(ifstat['rem_dst_cost'])}" if ifstat["rem_dst_cost"] != None else "Dst ?"
|
||||
pr_cost_str = f"PR {RNS.prettycost(ifstat['rem_pr_cost'])}" if ifstat["rem_pr_cost"] != None else "PR ?"
|
||||
print(f" Rem.Csts. : {dst_cost_str}, {pr_cost_str}")
|
||||
|
||||
if "noise_floor" in ifstat:
|
||||
if not "interference" in ifstat: nstr = ""
|
||||
|
|
@ -532,27 +651,141 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
print(" Held : {np} announce".format(np=aqn))
|
||||
else:
|
||||
print(" Held : {np} announces".format(np=aqn))
|
||||
|
||||
if astats and "incoming_announce_frequency" in ifstat and ifstat["incoming_announce_frequency"] != None:
|
||||
print(" Announces : {iaf}↑".format(iaf=RNS.prettyfrequency(ifstat["outgoing_announce_frequency"])))
|
||||
print(" {iaf}↓".format(iaf=RNS.prettyfrequency(ifstat["incoming_announce_frequency"])))
|
||||
|
||||
art = None; arp = None; arg = None
|
||||
if astats and "announce_rate_target" in ifstat: art = ifstat["announce_rate_target"]
|
||||
if astats and "announce_rate_penalty" in ifstat: arp = ifstat["announce_rate_penalty"]
|
||||
if astats and "announce_rate_grace" in ifstat: arg = ifstat["announce_rate_grace"]
|
||||
if art and arp != None and arg: art_str = f"(t:{RNS.prettytime(art)}/p:{RNS.prettytime(arp)}/g:{arg})"
|
||||
elif art and arp != None: art_str = f"(t:{RNS.prettytime(art)}/p:{RNS.prettytime(arp)})"
|
||||
elif art: art_str = f"(t:{RNS.prettytime(art)})"
|
||||
else: art_str = ""
|
||||
|
||||
burst_str = ""
|
||||
if "burst_active" in ifstat and ifstat["burst_active"]:
|
||||
if "burst_count" in ifstat and ifstat["burst_count"]: bcstr = f"on {ifstat['burst_count']} "
|
||||
else: bcstr = ""
|
||||
for_str = RNS.prettytime(time.time()-ifstat["burst_activated"])
|
||||
burst_str = f" burst {bcstr}for {for_str}"
|
||||
|
||||
pburst_str = ""
|
||||
if "pr_burst_active" in ifstat and ifstat["pr_burst_active"]:
|
||||
if "pr_burst_count" in ifstat and ifstat["pr_burst_count"]: prbcstr = f"on {ifstat['pr_burst_count']} "
|
||||
else: prbcstr = ""
|
||||
for_str = RNS.prettytime(time.time()-ifstat["pr_burst_activated"])
|
||||
pburst_str = f"burst {prbcstr}for {for_str}"
|
||||
|
||||
rxb_str = "↓"+RNS.prettysize(ifstat["rxb"])
|
||||
txb_str = "↑"+RNS.prettysize(ifstat["txb"])
|
||||
strdiff = len(rxb_str)-len(txb_str)
|
||||
if strdiff > 0:
|
||||
txb_str += " "*strdiff
|
||||
elif strdiff < 0:
|
||||
rxb_str += " "*-strdiff
|
||||
|
||||
asr = False
|
||||
if astats and "incoming_announce_frequency" in ifstat and ifstat["incoming_announce_frequency"] != None:
|
||||
oan = ifstat["outgoing_announce_frequency"]
|
||||
ian = ifstat["incoming_announce_frequency"]
|
||||
if name.startswith("Shared Instance[") and clients and clients > 0: oan = oan-(oan/clients) # Sub rnstatus own part
|
||||
oaf = RNS.prettyfrequency(oan, d=1, lpf=True)
|
||||
iaf = RNS.prettyfrequency(ian, d=1, lpf=True)
|
||||
|
||||
cspec = "c"
|
||||
if clients == None and "peers" in ifstat and ifstat["peers"]: clients = ifstat["peers"]; cspec = "p"
|
||||
if clients != None and clients > 0: pc_str = f" {RNS.prettyfrequency(ifstat['outgoing_announce_frequency']/clients, d=1, lpf=True)}/{cspec}"
|
||||
else: pc_str = ""
|
||||
|
||||
if "prxs" in ifstat and "rxs" in ifstat and "ptxs" in ifstat and "txs" in ifstat:
|
||||
arxspct = min(100.0, (ifstat["arxs"] / ifstat["rxs"])*100.0) if ifstat["rxs"] and ifstat["arxs"] else 0.0
|
||||
atxspct = min(100.0, (ifstat["atxs"] / ifstat["txs"])*100.0) if ifstat["txs"] and ifstat["atxs"] else 0.0
|
||||
apctstr = f"(↓{int(arxspct)}% / ↑{int(atxspct)}% of flow)"
|
||||
if pc_str: pc_str = f"{pc_str} {apctstr}"
|
||||
else: pc_str = f"{apctstr}"
|
||||
|
||||
asr = True
|
||||
|
||||
psr = False
|
||||
if pstats and "incoming_pr_frequency" in ifstat and ifstat["incoming_pr_frequency"] != None:
|
||||
opn = ifstat["outgoing_pr_frequency"]
|
||||
ipn = ifstat["incoming_pr_frequency"]
|
||||
if name.startswith("Shared Instance[") and clients and clients > 0: opn = opn-(opn/clients) # Sub rnstatus own part
|
||||
if astats:
|
||||
opf = "↑"+RNS.prettyfrequency(opn, d=1, lpf=True)
|
||||
ipf = "↓"+RNS.prettyfrequency(ipn, d=1, lpf=True)
|
||||
else:
|
||||
opf = RNS.prettyfrequency(opn,d=1, lpf=True)+"↑"
|
||||
ipf = RNS.prettyfrequency(ipn,d=1, lpf=True)+"↓"
|
||||
cspec = "c"
|
||||
if clients == None and "peers" in ifstat and ifstat["peers"]: clients = ifstat["peers"]; cspec = "p"
|
||||
if clients != None and clients > 0: rpc_str = f" {RNS.prettyfrequency(ifstat['outgoing_pr_frequency']/clients, d=1, lpf=True)}/{cspec}"
|
||||
else: rpc_str = ""
|
||||
|
||||
if "prxs" in ifstat and "rxs" in ifstat and "ptxs" in ifstat and "txs" in ifstat:
|
||||
prxspct = min(100.0, (ifstat["prxs"] / ifstat["rxs"])*100.0) if ifstat["rxs"] and ifstat["prxs"] else 0.0
|
||||
ptxspct = min(100.0, (ifstat["ptxs"] / ifstat["txs"])*100.0) if ifstat["txs"] and ifstat["ptxs"] else 0.0
|
||||
ppctstr = f"(↓{int(prxspct)}% / ↑{int(ptxspct)}% of flow)"
|
||||
if rpc_str: rpc_str = f"{rpc_str} {ppctstr}"
|
||||
else: rpc_str = f"{ppctstr}"
|
||||
|
||||
psr = True
|
||||
|
||||
if not asr: iaf = ""; oaf = ""
|
||||
if not psr: ipf = ""; opf = ""
|
||||
amlen = max(len(iaf), len(oaf))
|
||||
iaf += (amlen-len(iaf))*" "+"↓"
|
||||
oaf += (amlen-len(oaf))*" "+"↑"
|
||||
mlen = max(max(len(iaf), len(oaf), len(rxb_str), len(txb_str), len(ipf), len(opf)), 10)
|
||||
iaf += (mlen-len(iaf))*" "
|
||||
oaf += (mlen-len(oaf))*" "
|
||||
ipf += (mlen-len(ipf))*" "
|
||||
opf += (mlen-len(opf))*" "
|
||||
rxb_str += (mlen-len(rxb_str))*" "
|
||||
txb_str += (mlen-len(txb_str))*" "
|
||||
|
||||
if "protocol_violations" in ifstat and "ifac_violations" in ifstat:
|
||||
pv = ifstat["protocol_violations"]; iv = ifstat["ifac_violations"]
|
||||
if pv or iv:
|
||||
pvstr = f"{pv} protocol{', ' if iv else ''}"
|
||||
ivstr = f"{iv} IFAC" if iv else ""
|
||||
print(f" Violatns. : {pvstr}{ivstr}")
|
||||
|
||||
if "packet_filter_hits" in ifstat and ifstat["packet_filter_hits"]:
|
||||
print(f" Flt. Hits : {ifstat['packet_filter_hits']}")
|
||||
|
||||
if psr:
|
||||
if ifstat["prxc"] and ifstat["ptxc"]:
|
||||
print(f" Path Rqs. : {ifstat['prxc']}↓ {ifstat['ptxc']}↑ total")
|
||||
print(f" {opf} {rpc_str}")
|
||||
else:
|
||||
print(f" Path Rqs. : {opf} {rpc_str}")
|
||||
print(f" {ipf} {pburst_str}")
|
||||
|
||||
if asr:
|
||||
if ifstat["arxc"] and ifstat["atxc"]:
|
||||
f_str = f", {ifstat['afrxc']} filtered" if ifstat["afrxc"] else ""
|
||||
print(f" Announces : {ifstat['arxc']}↓ {ifstat['atxc']}↑ total{f_str}")
|
||||
print(f" {oaf} {pc_str}")
|
||||
else:
|
||||
print(f" Announces : {oaf} {pc_str}")
|
||||
print(f" {iaf} {art_str}{burst_str}")
|
||||
|
||||
rxstat = rxb_str
|
||||
txstat = txb_str
|
||||
if "rxs" in ifstat and "txs" in ifstat:
|
||||
rxstat += " "+RNS.prettyspeed(ifstat["rxs"])
|
||||
txstat += " "+RNS.prettyspeed(ifstat["txs"])
|
||||
|
||||
if ifstat["txbuffered"]:
|
||||
if ifstat["txstalled"]: stlstr = ", stalled"
|
||||
else: stlstr = ""
|
||||
txstat += f" ({RNS.prettysize(ifstat['txbuffered'])} waiting{stlstr})"
|
||||
|
||||
print(f" Traffic : {txstat}\n {rxstat}")
|
||||
|
||||
if fstats:
|
||||
frxstat = "↓"+RNS.prettysize(ifstat["frxb"])
|
||||
ftxstat = "↑"+RNS.prettysize(ifstat["ftxb"])
|
||||
if "frxs" in ifstat and "ftxs" in ifstat:
|
||||
frxstat += " "+RNS.prettyspeed(ifstat["frxs"])
|
||||
ftxstat += " "+RNS.prettyspeed(ifstat["ftxs"])
|
||||
print(f" Filtered : {ftxstat}\n {frxstat}")
|
||||
|
||||
lstr = ""
|
||||
if link_count != None and lstats:
|
||||
ms = "y" if link_count == 1 else "ies"
|
||||
|
|
@ -561,19 +794,131 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
|
|||
else:
|
||||
lstr = f" {link_count} entr{ms} in link table"
|
||||
|
||||
if active_link_count: lstr = f"{lstr} ({active_link_count} active)"
|
||||
|
||||
if traffic_totals:
|
||||
rxb_str = "↓"+RNS.prettysize(stats["rxb"])
|
||||
txb_str = "↑"+RNS.prettysize(stats["txb"])
|
||||
strdiff = len(rxb_str)-len(txb_str)
|
||||
if strdiff > 0:
|
||||
txb_str += " "*strdiff
|
||||
elif strdiff < 0:
|
||||
rxb_str += " "*-strdiff
|
||||
if strdiff > 0: txb_str += " "*strdiff
|
||||
elif strdiff < 0: rxb_str += " "*-strdiff
|
||||
|
||||
rxstat = rxb_str+" "+RNS.prettyspeed(stats["rxs"])
|
||||
txstat = txb_str+" "+RNS.prettyspeed(stats["txs"])
|
||||
|
||||
if pstats or astats:
|
||||
if "prxs" in stats and "ptxs" in stats and "arxs" in stats and "atxs" in stats:
|
||||
parxs = stats["prxs"]+stats["arxs"]
|
||||
patxs = stats["ptxs"]+stats["atxs"]
|
||||
drxpct = 100.0-min(100.0, (parxs / stats["rxs"])*100.0) if stats["rxs"] else 100.0
|
||||
dtxpct = 100.0-min(100.0, (patxs / stats["txs"])*100.0) if stats["txs"] else 100.0
|
||||
drxs = stats["rxs"]*(drxpct/100.0)
|
||||
dtxs = stats["txs"]*(dtxpct/100.0)
|
||||
|
||||
if stats["rxs"] > 0: rxstat = f"{rxstat}, {int(drxpct)}% data ({RNS.prettyspeed(drxs)})"
|
||||
if stats["txs"] > 0: txstat = f"{txstat}, {int(dtxpct)}% data ({RNS.prettyspeed(dtxs)})"
|
||||
|
||||
if pps:
|
||||
rpps = RNS.prettysize(stats['rxpps'], suffix="pps")
|
||||
tpps = RNS.prettysize(stats['txpps'], suffix="pps")
|
||||
rxstat = f"{rxstat}, {rpps}"
|
||||
txstat = f"{txstat}, {tpps}"
|
||||
|
||||
print(f"\n Totals : {txstat}\n {rxstat}")
|
||||
|
||||
if fstats:
|
||||
frxstat = "↓"+RNS.prettysize(stats["frxb"])
|
||||
ftxstat = "↑"+RNS.prettysize(stats["ftxb"])
|
||||
|
||||
strdiff = len(frxstat)-len(ftxstat)
|
||||
if strdiff > 0: ftxstat += " "*strdiff
|
||||
elif strdiff < 0: frxstat += " "*-strdiff
|
||||
|
||||
if "frxs" in stats and "ftxs" in stats:
|
||||
frxstat += " "+RNS.prettyspeed(stats["frxs"])
|
||||
ftxstat += " "+RNS.prettyspeed(stats["ftxs"])
|
||||
print(f"\n Filtered : {ftxstat}\n {frxstat}")
|
||||
|
||||
if lstats:
|
||||
if "lrxb" in stats and "ltxb" in stats and "lrxs" in stats and "ltxs" in stats:
|
||||
lrxb_str = "↓"+RNS.prettysize(stats["lrxb"])
|
||||
ltxb_str = "↑"+RNS.prettysize(stats["ltxb"])
|
||||
strdiff = len(lrxb_str)-len(ltxb_str)
|
||||
if strdiff > 0: ltxb_str += " "*strdiff
|
||||
elif strdiff < 0: lrxb_str += " "*-strdiff
|
||||
|
||||
parxs = stats["prxs"]+stats["arxs"]
|
||||
patxs = stats["ptxs"]+stats["atxs"]
|
||||
drxpct = 100.0-min(100.0, (parxs / stats["rxs"])*100.0) if stats["rxs"] else 100.0
|
||||
dtxpct = 100.0-min(100.0, (patxs / stats["txs"])*100.0) if stats["txs"] else 100.0
|
||||
drxs = stats["rxs"]*(drxpct/100.0)
|
||||
dtxs = stats["txs"]*(dtxpct/100.0)
|
||||
|
||||
lrxspct = min(100.0, (stats["lrxs"] / drxs)*100.0) if drxs and stats["lrxs"] else 0.0
|
||||
ltxspct = min(100.0, (stats["ltxs"] / dtxs)*100.0) if dtxs and stats["ltxs"] else 0.0
|
||||
|
||||
lrxstat = lrxb_str+" "+RNS.prettyspeed(stats["lrxs"])+f", {int(lrxspct)}% of data flow"
|
||||
ltxstat = ltxb_str+" "+RNS.prettyspeed(stats["ltxs"])+f", {int(ltxspct)}% of data flow"
|
||||
print(f"\n Links : {ltxstat}\n {lrxstat}")
|
||||
|
||||
if pstats:
|
||||
if "prxb" in stats and "ptxb" in stats and "prxs" in stats and "ptxs" in stats:
|
||||
prxb_str = "↓"+RNS.prettysize(stats["prxb"])
|
||||
ptxb_str = "↑"+RNS.prettysize(stats["ptxb"])
|
||||
strdiff = len(prxb_str)-len(ptxb_str)
|
||||
if strdiff > 0: ptxb_str += " "*strdiff
|
||||
elif strdiff < 0: prxb_str += " "*-strdiff
|
||||
|
||||
prxspct = min(100.0, (stats["prxs"] / stats["rxs"])*100.0) if stats["rxs"] and stats["prxs"] else 0.0
|
||||
ptxspct = min(100.0, (stats["ptxs"] / stats["txs"])*100.0) if stats["txs"] and stats["ptxs"] else 0.0
|
||||
|
||||
prxfstr = RNS.prettyfrequency(stats["prxf"])
|
||||
ptxfstr = RNS.prettyfrequency(stats["ptxf"])
|
||||
|
||||
prxstat = prxb_str+" "+RNS.prettyspeed(stats["prxs"])+f", {int(prxspct)}% of flow, {prxfstr}"
|
||||
ptxstat = ptxb_str+" "+RNS.prettyspeed(stats["ptxs"])+f", {int(ptxspct)}% of flow, {ptxfstr}"
|
||||
print(f"\n Path Rqs. : {ptxstat}\n {prxstat}")
|
||||
|
||||
if astats:
|
||||
if "arxb" in stats and "atxb" in stats and "arxs" in stats and "atxs" in stats:
|
||||
arxb_str = "↓"+RNS.prettysize(stats["arxb"])
|
||||
atxb_str = "↑"+RNS.prettysize(stats["atxb"])
|
||||
strdiff = len(arxb_str)-len(atxb_str)
|
||||
if strdiff > 0: atxb_str += " "*strdiff
|
||||
elif strdiff < 0: arxb_str += " "*-strdiff
|
||||
|
||||
arxspct = min(100.0, (stats["arxs"] / stats["rxs"])*100.0) if stats["rxs"] and stats["arxs"] else 0.0
|
||||
atxspct = min(100.0, (stats["atxs"] / stats["txs"])*100.0) if stats["txs"] and stats["atxs"] else 0.0
|
||||
|
||||
arxfstr = RNS.prettyfrequency(stats["arxf"])
|
||||
atxfstr = RNS.prettyfrequency(stats["atxf"])
|
||||
|
||||
arxstat = arxb_str+" "+RNS.prettyspeed(stats["arxs"])+f", {int(arxspct)}% of flow, {arxfstr}"
|
||||
atxstat = atxb_str+" "+RNS.prettyspeed(stats["atxs"])+f", {int(atxspct)}% of flow, {atxfstr}"
|
||||
print(f"\n Announces : {atxstat}\n {arxstat}")
|
||||
|
||||
|
||||
if queue_stats:
|
||||
tqdp = f", {stats['rxqtd']} dropped" if stats['rxqtd'] else ""
|
||||
dqdp = f", {stats['rxqdd']} dropped" if stats['rxqdd'] else ""
|
||||
aqdp = f", {stats['rxqad']} dropped" if stats['rxqad'] else ""
|
||||
pqdp = f", {stats['rxqpd']} dropped" if stats['rxqpd'] else ""
|
||||
ilqdp = f", {stats['rxqild']} dropped" if stats['rxqild'] else ""
|
||||
tqpress = f"{round(stats['tqpressure']*100.0, 1)}% total, {stats['rxqt']} pkts{tqdp}"
|
||||
dqpress = f"{round(stats['dqpressure']*100.0, 1)}% data, {stats['rxqd']} pkts{dqdp}"
|
||||
aqpress = f"{round(stats['aqpressure']*100.0, 1)}% announce, {stats['rxqa']} pkts{aqdp}"
|
||||
pqpress = f"{round(stats['pqpressure']*100.0, 1)}% path request, {stats['rxqp']} pkts{pqdp}"
|
||||
ilpress = f"{round(stats['ilqpressure']*100.0, 1)}% ingress limiter, {stats['rxqil']} pkts{ilqdp}"
|
||||
|
||||
print(f"\n Qu. Pressure : {tqpress}")
|
||||
print(f" {dqpress}")
|
||||
print(f" {aqpress}")
|
||||
print(f" {pqpress}")
|
||||
print(f" {ilpress}")
|
||||
|
||||
if profiling_results:
|
||||
print(f"\n Profiling :\n{RNS.Profiler.format_results(profiling_results)}")
|
||||
|
||||
if "transport_id" in stats and stats["transport_id"] != None:
|
||||
print("\n Transport Instance "+RNS.prettyhexrep(stats["transport_id"])+" running")
|
||||
if "network_id" in stats and stats["network_id"] != None:
|
||||
|
|
@ -604,11 +949,23 @@ def main(must_exit=True, rns_instance=None):
|
|||
parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
|
||||
parser.add_argument("--version", action="version", version="rnstatus {version}".format(version=__version__))
|
||||
|
||||
parser.add_argument("--attach", action="store", metavar="name", help="Attach interface by name", default=None, type=str)
|
||||
parser.add_argument("--detach", action="store", metavar="name", help="Detach interface by name", default=None, type=str)
|
||||
parser.add_argument("--reload", action="store", metavar="name", help="Reload interface by name", default=None, type=str)
|
||||
|
||||
parser.add_argument("-a", "--all", action="store_true", help="show all interfaces", default=False)
|
||||
parser.add_argument("-A", "--announce-stats", action="store_true", help="show announce stats", default=False)
|
||||
parser.add_argument("-P", "--pr-stats", action="store_true", help="show path request stats", default=False)
|
||||
parser.add_argument("-F", "--filter-stats", action="store_true", help="show filtered traffic stats", default=False)
|
||||
parser.add_argument("-l", "--link-stats", action="store_true", help="show link stats", default=False)
|
||||
parser.add_argument("-B", "--burst", action="store_true", help="only show interfaces with active bursts", default=False)
|
||||
parser.add_argument("-b", "--blocked-ips", action="store_true", help="show blocked IPs per interface", default=False)
|
||||
parser.add_argument("-t", "--totals", action="store_true", help="display traffic totals", default=False)
|
||||
parser.add_argument("-s", "--sort", action="store", help="sort interfaces by [rate, traffic, rx, tx, rxs, txs, announces, arx, atx, held]", default=None, type=str)
|
||||
parser.add_argument("-T", "--tunnels", action="store_true", help="display tunnel information", default=False)
|
||||
parser.add_argument("-p", "--pps", action="store_true", help="display packets per second in totals", default=False)
|
||||
parser.add_argument("-q", "--queues", action="store_true", help="display queue stats", default=False)
|
||||
parser.add_argument("-z", "--profiling", action="store_true", help="display live profiling results", default=False)
|
||||
parser.add_argument("-s", "--sort", action="store", help="sort interfaces by [rate, traffic, rx, tx, rxs, txs, frx, ftx, anns, arx, atx, arxc, atxc, afrxc, held, prx, ptx, prxc, ptxc, pvs, ivs, flt, txdrp, txdrb, txbuf]", default=None, type=str)
|
||||
parser.add_argument("-r", "--reverse", action="store_true", help="reverse sorting", default=False)
|
||||
parser.add_argument("-j", "--json", action="store_true", help="output in JSON format", default=False)
|
||||
parser.add_argument("-R", action="store", metavar="hash", help="transport identity hash of remote instance to get status from", default=None, type=str)
|
||||
|
|
@ -616,6 +973,8 @@ def main(must_exit=True, rns_instance=None):
|
|||
parser.add_argument("-w", action="store", metavar="seconds", type=float, help="timeout before giving up on remote queries", default=RNS.Transport.PATH_REQUEST_TIMEOUT)
|
||||
parser.add_argument("-d", "--discovered", action="store_true", help="list discovered interfaces", default=False)
|
||||
parser.add_argument("-D", action="store_true", help="show details and config entries for discovered interfaces", default=False)
|
||||
parser.add_argument("--show-stale", action="store_true", help="show stale discovery entries", default=False)
|
||||
parser.add_argument("--show-unknown", action="store_true", help="show discovery entries without version info", default=False)
|
||||
parser.add_argument("-m", "--monitor", action="store_true", help="continuously monitor status", default=False)
|
||||
parser.add_argument("-I", "--monitor-interval", action="store", metavar="seconds", type=float, help="refresh interval for monitor mode (default: 1)", default=1.0)
|
||||
parser.add_argument('-v', '--verbose', action='count', default=0)
|
||||
|
|
@ -630,21 +989,26 @@ def main(must_exit=True, rns_instance=None):
|
|||
if args.R: require_shared = False
|
||||
else: require_shared = True
|
||||
|
||||
try: reticulum = RNS.Reticulum(configdir=configarg, loglevel=3+args.verbose, require_shared_instance=require_shared)
|
||||
try:
|
||||
reticulum = RNS.Reticulum(configdir=configarg, loglevel=3+args.verbose, require_shared_instance=require_shared)
|
||||
reticulum.wants_destination_type("rnstransport.remote.management")
|
||||
except Exception as e:
|
||||
print("No shared RNS instance available to get status from")
|
||||
exit(1)
|
||||
|
||||
while True:
|
||||
st = time.time()
|
||||
buffer = io.StringIO()
|
||||
old_stdout = sys.stdout
|
||||
sys.stdout = buffer
|
||||
|
||||
try:
|
||||
program_setup(configdir = configarg, dispall = args.all, verbosity=args.verbose, name_filter=args.filter, json=args.json,
|
||||
astats=args.announce_stats, lstats=args.link_stats, sorting=args.sort, sort_reverse=args.reverse, remote=args.R,
|
||||
management_identity=args.i, remote_timeout=args.w, must_exit=False, rns_instance=reticulum, traffic_totals=args.totals,
|
||||
discovered_interfaces=args.discovered, config_entries=args.D)
|
||||
astats=args.announce_stats, pstats=args.pr_stats, lstats=args.link_stats, sorting=args.sort, sort_reverse=args.reverse,
|
||||
remote=args.R, management_identity=args.i, remote_timeout=args.w, must_exit=False, rns_instance=reticulum,
|
||||
traffic_totals=args.totals, discovered_interfaces=args.discovered, config_entries=args.D, burst_filter=args.burst,
|
||||
blocked_ips=args.blocked_ips, queue_stats=args.queues, pps=args.pps, profiling=args.profiling,
|
||||
show_stale=args.show_stale, show_unknown=args.show_unknown, fstats=args.filter_stats, tstats=args.tunnels)
|
||||
|
||||
finally:
|
||||
sys.stdout = old_stdout
|
||||
|
|
@ -652,14 +1016,19 @@ def main(must_exit=True, rns_instance=None):
|
|||
output = buffer.getvalue()
|
||||
print("\033[H\033[2J", end="")
|
||||
print(output, end="", flush=True)
|
||||
|
||||
time.sleep(args.monitor_interval)
|
||||
|
||||
td = time.time()-st
|
||||
sleeptime = max(args.monitor_interval-td, 0.2)
|
||||
time.sleep(sleeptime)
|
||||
|
||||
else:
|
||||
program_setup(configdir = configarg, dispall = args.all, verbosity=args.verbose, name_filter=args.filter, json=args.json,
|
||||
astats=args.announce_stats, lstats=args.link_stats, sorting=args.sort, sort_reverse=args.reverse, remote=args.R,
|
||||
management_identity=args.i, remote_timeout=args.w, must_exit=must_exit, rns_instance=rns_instance, traffic_totals=args.totals,
|
||||
discovered_interfaces=args.discovered, config_entries=args.D)
|
||||
astats=args.announce_stats, pstats=args.pr_stats, lstats=args.link_stats, sorting=args.sort, sort_reverse=args.reverse,
|
||||
remote=args.R, management_identity=args.i, remote_timeout=args.w, must_exit=must_exit, rns_instance=rns_instance,
|
||||
traffic_totals=args.totals, discovered_interfaces=args.discovered, config_entries=args.D, burst_filter=args.burst,
|
||||
blocked_ips=args.blocked_ips, queue_stats=args.queues, pps=args.pps, profiling=args.profiling,
|
||||
attach=args.attach, detach=args.detach, reload=args.reload, show_stale=args.show_stale, tstats=args.tunnels,
|
||||
show_unknown=args.show_unknown, fstats=args.filter_stats)
|
||||
|
||||
except KeyboardInterrupt:
|
||||
print("")
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -342,12 +342,14 @@ def execute(configdir, identitypath = None, verbosity = 0, quietness = 0, detail
|
|||
targetloglevel = 3+verbosity-quietness
|
||||
reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel)
|
||||
|
||||
reticulum.wants_destination_type(f"{APP_NAME}.execute")
|
||||
|
||||
if identity == None:
|
||||
prepare_identity(identitypath)
|
||||
|
||||
if not RNS.Transport.has_path(destination_hash):
|
||||
RNS.Transport.request_path(destination_hash)
|
||||
if not spin(until=lambda: RNS.Transport.has_path(destination_hash), msg="Path to "+RNS.prettyhexrep(destination_hash)+" requested", timeout=timeout):
|
||||
if not spin(until=lambda: RNS.Transport.has_path(destination_hash), msg="Path to "+RNS.prettyhexrep(destination_hash)+" requested", timeout=max(timeout, reticulum.get_medium_path_timeout())):
|
||||
print("Path not found")
|
||||
exit(242)
|
||||
|
||||
|
|
@ -365,7 +367,7 @@ def execute(configdir, identitypath = None, verbosity = 0, quietness = 0, detail
|
|||
link = RNS.Link(listener_destination)
|
||||
link.did_identify = False
|
||||
|
||||
if not spin(until=lambda: link.status == RNS.Link.ACTIVE, msg="Establishing link with "+RNS.prettyhexrep(destination_hash), timeout=timeout):
|
||||
if not spin(until=lambda: link.status == RNS.Link.ACTIVE, msg="Establishing link with "+RNS.prettyhexrep(destination_hash), timeout=max(timeout, link.establishment_timeout)):
|
||||
print("Could not establish link with "+RNS.prettyhexrep(destination_hash))
|
||||
exit(243)
|
||||
|
||||
|
|
|
|||
513
RNS/__init__.py
513
RNS/__init__.py
|
|
@ -1,6 +1,6 @@
|
|||
# Reticulum License
|
||||
#
|
||||
# Copyright (c) 2016-2025 Mark Qvist
|
||||
# Copyright (c) 2016-2026 Mark Qvist
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
@ -35,32 +35,26 @@ import time
|
|||
import datetime
|
||||
import random
|
||||
import threading
|
||||
import math
|
||||
import bisect
|
||||
|
||||
from ._version import __version__
|
||||
|
||||
from .Reticulum import Reticulum
|
||||
from .Identity import Identity
|
||||
from .Link import Link, RequestReceipt
|
||||
from .Channel import MessageBase
|
||||
from .Buffer import Buffer, RawChannelReader, RawChannelWriter
|
||||
from .Transport import Transport
|
||||
from .Discovery import InterfaceAnnouncer
|
||||
from .Destination import Destination
|
||||
from .Packet import Packet
|
||||
from .Packet import PacketReceipt
|
||||
from .Resolver import Resolver
|
||||
from .Resource import Resource, ResourceAdvertisement
|
||||
from .Cryptography import HKDF
|
||||
from .Cryptography import Hashes
|
||||
from collections import deque
|
||||
from functools import partial, wraps
|
||||
from RNS.Stamps import quantize_cost
|
||||
|
||||
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
|
||||
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
|
||||
modules = py_modules+pyc_modules
|
||||
__all__ = list(set([os.path.basename(f).replace(".pyc", "").replace(".py", "") for f in modules if not (f.endswith("__init__.py") or f.endswith("__init__.pyc"))]))
|
||||
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
|
||||
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
|
||||
modules = py_modules+pyc_modules+so_modules+pyd_modules
|
||||
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))
|
||||
|
||||
import importlib.util
|
||||
if importlib.util.find_spec("cython"): import cython; compiled = cython.compiled
|
||||
else: compiled = False
|
||||
try: from ._buildinfo import compiled
|
||||
except ImportError:
|
||||
if importlib.util.find_spec("cython"): import cython; compiled = cython.compiled
|
||||
else: compiled = False
|
||||
|
||||
LOG_NONE = -1
|
||||
LOG_CRITICAL = 0
|
||||
|
|
@ -70,7 +64,8 @@ LOG_NOTICE = 3
|
|||
LOG_INFO = 4
|
||||
LOG_VERBOSE = 5
|
||||
LOG_DEBUG = 6
|
||||
LOG_EXTREME = 7
|
||||
LOG_PATHING = 7
|
||||
LOG_EXTREME = 8
|
||||
|
||||
LOG_STDOUT = 0x91
|
||||
LOG_FILE = 0x92
|
||||
|
|
@ -82,6 +77,7 @@ loglevel = LOG_NOTICE
|
|||
logfile = None
|
||||
logdest = LOG_STDOUT
|
||||
logcall = None
|
||||
logtimestamps = True
|
||||
logtimefmt = "%Y-%m-%d %H:%M:%S"
|
||||
logtimefmt_p = "%H:%M:%S.%f"
|
||||
compact_log_fmt = False
|
||||
|
|
@ -91,25 +87,18 @@ instance_random.seed(os.urandom(10))
|
|||
|
||||
_always_override_destination = False
|
||||
|
||||
logging_lock = threading.Lock()
|
||||
logging_lock = threading.RLock()
|
||||
|
||||
def loglevelname(level):
|
||||
if (level == LOG_CRITICAL):
|
||||
return "[Critical]"
|
||||
if (level == LOG_ERROR):
|
||||
return "[Error] "
|
||||
if (level == LOG_WARNING):
|
||||
return "[Warning] "
|
||||
if (level == LOG_NOTICE):
|
||||
return "[Notice] "
|
||||
if (level == LOG_INFO):
|
||||
return "[Info] "
|
||||
if (level == LOG_VERBOSE):
|
||||
return "[Verbose] "
|
||||
if (level == LOG_DEBUG):
|
||||
return "[Debug] "
|
||||
if (level == LOG_EXTREME):
|
||||
return "[Extra] "
|
||||
if (level == LOG_CRITICAL): return "[Critical]"
|
||||
if (level == LOG_ERROR): return "[Error] "
|
||||
if (level == LOG_WARNING): return "[Warning] "
|
||||
if (level == LOG_NOTICE): return "[Notice] "
|
||||
if (level == LOG_INFO): return "[Info] "
|
||||
if (level == LOG_VERBOSE): return "[Verbose] "
|
||||
if (level == LOG_DEBUG): return "[Debug] "
|
||||
if (level == LOG_PATHING): return "[Pathing] "
|
||||
if (level == LOG_EXTREME): return "[Extra] "
|
||||
|
||||
return "Unknown"
|
||||
|
||||
|
|
@ -127,18 +116,16 @@ def timestamp_str(time_s):
|
|||
def precise_timestamp_str(time_s):
|
||||
return datetime.datetime.now().strftime(logtimefmt_p)[:-3]
|
||||
|
||||
def sl(level=3): return loglevel >= level
|
||||
def log(msg, level=3, _override_destination = False, pt=False):
|
||||
if loglevel == LOG_NONE: return
|
||||
global _always_override_destination, compact_log_fmt
|
||||
msg = str(msg)
|
||||
if loglevel >= level:
|
||||
if pt:
|
||||
logstring = "["+precise_timestamp_str(time.time())+"] "+loglevelname(level)+" "+msg
|
||||
if pt: logstring = "["+precise_timestamp_str(time.time())+"] "+loglevelname(level)+" "+msg
|
||||
else:
|
||||
if not compact_log_fmt:
|
||||
logstring = "["+timestamp_str(time.time())+"] "+loglevelname(level)+" "+msg
|
||||
else:
|
||||
logstring = "["+timestamp_str(time.time())+"] "+msg
|
||||
if not compact_log_fmt: logstring = ("["+timestamp_str(time.time())+"] " if logtimestamps else "")+loglevelname(level)+" "+msg
|
||||
else: logstring = ("["+timestamp_str(time.time())+"] " if logtimestamps else "")+msg
|
||||
|
||||
with logging_lock:
|
||||
if (logdest == LOG_STDOUT or _always_override_destination or _override_destination):
|
||||
|
|
@ -149,14 +136,10 @@ def log(msg, level=3, _override_destination = False, pt=False):
|
|||
|
||||
elif (logdest == LOG_FILE and logfile != None):
|
||||
try:
|
||||
file = open(logfile, "a")
|
||||
file.write(logstring+"\n")
|
||||
file.close()
|
||||
|
||||
with open(logfile, "a") as file: file.write(logstring+"\n")
|
||||
if os.path.getsize(logfile) > LOG_MAXSIZE:
|
||||
prevfile = logfile+".1"
|
||||
if os.path.isfile(prevfile):
|
||||
os.unlink(prevfile)
|
||||
if os.path.isfile(prevfile): os.unlink(prevfile)
|
||||
os.rename(logfile, prevfile)
|
||||
|
||||
except Exception as e:
|
||||
|
|
@ -166,14 +149,12 @@ def log(msg, level=3, _override_destination = False, pt=False):
|
|||
log(msg, level)
|
||||
|
||||
elif logdest == LOG_CALLBACK:
|
||||
try:
|
||||
logcall(logstring)
|
||||
try: logcall(logstring)
|
||||
except Exception as e:
|
||||
_always_override_destination = True
|
||||
log("Exception occurred while calling external log handler: "+str(e), LOG_CRITICAL)
|
||||
log("Dumping future log events to console!", LOG_CRITICAL)
|
||||
log(msg, level)
|
||||
|
||||
|
||||
def rand():
|
||||
result = instance_random.random()
|
||||
|
|
@ -186,14 +167,11 @@ def trace_exception(e):
|
|||
log(exception_info, LOG_ERROR)
|
||||
|
||||
def hexrep(data, delimit=True):
|
||||
try:
|
||||
iter(data)
|
||||
except TypeError:
|
||||
data = [data]
|
||||
try: iter(data)
|
||||
except TypeError: data = [data]
|
||||
|
||||
delimiter = ":"
|
||||
if not delimit:
|
||||
delimiter = ""
|
||||
if not delimit: delimiter = ""
|
||||
hexrep = delimiter.join("{:02x}".format(c) for c in data)
|
||||
return hexrep
|
||||
|
||||
|
|
@ -202,6 +180,19 @@ def prettyhexrep(data):
|
|||
hexrep = "<"+delimiter.join("{:02x}".format(c) for c in data)+">"
|
||||
return hexrep
|
||||
|
||||
__namehash_resolver = None
|
||||
def prettynamehash(name_hash):
|
||||
global __namehash_resolver
|
||||
if not __namehash_resolver:
|
||||
import RNS.Transport
|
||||
__namehash_resolver = RNS.Transport.destination_type_name
|
||||
return f"<{__namehash_resolver(name_hash)}>"
|
||||
|
||||
def prettycost(value):
|
||||
if value == None: return "n"
|
||||
elif not type(value) in (float, int): return "n"
|
||||
else: return f"{quantize_cost(value):.3f}"
|
||||
|
||||
def prettyspeed(num, suffix="b"):
|
||||
return prettysize(num/8, suffix=suffix)+"ps"
|
||||
|
||||
|
|
@ -216,22 +207,24 @@ def prettysize(num, suffix='B'):
|
|||
|
||||
for unit in units:
|
||||
if abs(num) < 1000.0:
|
||||
if unit == "":
|
||||
return "%.0f %s%s" % (num, unit, suffix)
|
||||
else:
|
||||
return "%.2f %s%s" % (num, unit, suffix)
|
||||
if unit == "": return "%.0f %s%s" % (num, unit, suffix)
|
||||
else: return "%.2f %s%s" % (num, unit, suffix)
|
||||
num /= 1000.0
|
||||
|
||||
return "%.2f%s%s" % (num, last_unit, suffix)
|
||||
|
||||
def prettyfrequency(hz, suffix="Hz"):
|
||||
num = hz*1e6
|
||||
units = ["µ", "m", "", "K","M","G","T","P","E","Z"]
|
||||
def prettyfrequency(hz, suffix="Hz", d=2, lpf=False):
|
||||
if hz == 0: return "0 Hz"
|
||||
if not lpf: num = hz*1e6
|
||||
else: num = hz
|
||||
if not lpf: units = ["µ", "m", "", "K","M","G","T","P","E","Z"]
|
||||
else: units = ["", "K","M","G","T","P","E","Z"]
|
||||
last_unit = "Y"
|
||||
|
||||
for unit in units:
|
||||
if abs(num) < 1000.0:
|
||||
return "%.2f %s%s" % (num, unit, suffix)
|
||||
if d == 2: return "%.2f %s%s" % (num, unit, suffix)
|
||||
else: return "%s %s%s" % (str(round(num,d)), unit, suffix)
|
||||
num /= 1000.0
|
||||
|
||||
return "%.2f%s%s" % (num, last_unit, suffix)
|
||||
|
|
@ -246,8 +239,7 @@ def prettydistance(m, suffix="m"):
|
|||
if unit == "m": divisor = 10
|
||||
if unit == "c": divisor = 100
|
||||
|
||||
if abs(num) < divisor:
|
||||
return "%.2f %s%s" % (num, unit, suffix)
|
||||
if abs(num) < divisor: return "%.2f %s%s" % (num, unit, suffix)
|
||||
num /= divisor
|
||||
|
||||
return "%.2f %s%s" % (num, last_unit, suffix)
|
||||
|
|
@ -264,10 +256,8 @@ def prettytime(time, verbose=False, compact=False):
|
|||
time %= 3600
|
||||
minutes = int(time // 60)
|
||||
time %= 60
|
||||
if compact:
|
||||
seconds = int(time)
|
||||
else:
|
||||
seconds = round(time, 2)
|
||||
if compact: seconds = int(time)
|
||||
else: seconds = round(time, 2)
|
||||
|
||||
ss = "" if seconds == 1 else "s"
|
||||
sm = "" if minutes == 1 else "s"
|
||||
|
|
@ -296,24 +286,18 @@ def prettytime(time, verbose=False, compact=False):
|
|||
tstr = ""
|
||||
for c in components:
|
||||
i += 1
|
||||
if i == 1:
|
||||
pass
|
||||
elif i < len(components):
|
||||
tstr += ", "
|
||||
elif i == len(components):
|
||||
tstr += " and "
|
||||
if i == 1: pass
|
||||
elif i < len(components): tstr += ", "
|
||||
elif i == len(components): tstr += " and "
|
||||
|
||||
tstr += c
|
||||
|
||||
if tstr == "":
|
||||
return "0s"
|
||||
if tstr == "": return "0s"
|
||||
else:
|
||||
if not neg:
|
||||
return tstr
|
||||
else:
|
||||
return f"-{tstr}"
|
||||
if not neg: return tstr
|
||||
else: return f"-{tstr}"
|
||||
|
||||
def prettyshorttime(time, verbose=False, compact=False):
|
||||
def prettyshorttime(time, verbose=False, compact=False, tight=False):
|
||||
neg = False
|
||||
time = time*1e6
|
||||
if time < 0:
|
||||
|
|
@ -323,10 +307,8 @@ def prettyshorttime(time, verbose=False, compact=False):
|
|||
seconds = int(time // 1e6); time %= 1e6
|
||||
milliseconds = int(time // 1e3); time %= 1e3
|
||||
|
||||
if compact:
|
||||
microseconds = int(time)
|
||||
else:
|
||||
microseconds = round(time, 2)
|
||||
if compact: microseconds = int(time)
|
||||
else: microseconds = round(time, 2)
|
||||
|
||||
ss = "" if seconds == 1 else "s"
|
||||
sms = "" if milliseconds == 1 else "s"
|
||||
|
|
@ -350,22 +332,16 @@ def prettyshorttime(time, verbose=False, compact=False):
|
|||
tstr = ""
|
||||
for c in components:
|
||||
i += 1
|
||||
if i == 1:
|
||||
pass
|
||||
elif i < len(components):
|
||||
tstr += ", "
|
||||
elif i == len(components):
|
||||
tstr += " and "
|
||||
if i == 1: pass
|
||||
elif i < len(components): tstr += ", " if not tight else " "
|
||||
elif i == len(components): tstr += " and " if not tight else " "
|
||||
|
||||
tstr += c
|
||||
|
||||
if tstr == "":
|
||||
return "0us"
|
||||
if tstr == "": return "0us"
|
||||
else:
|
||||
if not neg:
|
||||
return tstr
|
||||
else:
|
||||
return f"-{tstr}"
|
||||
if not neg: return tstr
|
||||
else: return f"-{tstr}"
|
||||
|
||||
def phyparams():
|
||||
print("Required Physical Layer MTU : "+str(Reticulum.MTU)+" bytes")
|
||||
|
|
@ -376,8 +352,7 @@ def phyparams():
|
|||
print("Link Public Key Size : "+str(Link.ECPUBSIZE*8)+" bits")
|
||||
print("Link Private Key Size : "+str(Link.KEYSIZE*8)+" bits")
|
||||
|
||||
def panic():
|
||||
os._exit(255)
|
||||
def panic(): os._exit(255)
|
||||
|
||||
exit_called = False
|
||||
def exit(code=0):
|
||||
|
|
@ -387,33 +362,67 @@ def exit(code=0):
|
|||
Reticulum.exit_handler()
|
||||
os._exit(code)
|
||||
|
||||
def _detach_stdout():
|
||||
sys.stdout = open(os.devnull, "w")
|
||||
sys.stderr = open(os.devnull, "w")
|
||||
|
||||
class Profiler:
|
||||
_ran = False
|
||||
profilers = {}
|
||||
tags = {}
|
||||
|
||||
# Samples per tag
|
||||
MAX_CAPTURES = 10000
|
||||
|
||||
@staticmethod
|
||||
def get_profiler(tag=None, super_tag=None):
|
||||
if tag in Profiler.profilers:
|
||||
return Profiler.profilers[tag]
|
||||
def get_profiler(tag=None, super_tag=None, max_captures=None):
|
||||
if tag in Profiler.profilers: return Profiler.profilers[tag]
|
||||
else:
|
||||
profiler = Profiler(tag, super_tag)
|
||||
if max_captures is None: max_captures = Profiler.MAX_CAPTURES
|
||||
profiler = Profiler(tag, super_tag, max_captures)
|
||||
Profiler.profilers[tag] = profiler
|
||||
return profiler
|
||||
|
||||
def __init__(self, tag=None, super_tag=None):
|
||||
@staticmethod
|
||||
def profile(func=None, *, tag=None, max_captures=None):
|
||||
"""
|
||||
Decorator to profile a function. With no arguments, it can be
|
||||
used as `@RNS.Profiler.profile`, in which case it will use the
|
||||
qualified name of the function as the profiler tag. It can
|
||||
also be used as `@RNS.Profiler.profile(tag="prof_tag")` to use
|
||||
a specific profiler tag.
|
||||
|
||||
:param tag: Either a profiler tag as passed to `Profiler.get_profiler(...)` or a `Profiler` object.
|
||||
:param max_captures: Maximum samples to capture for the specified profiler tag.
|
||||
"""
|
||||
if func is None:
|
||||
return partial(Profiler.profile, tag=tag, max_captures=max_captures)
|
||||
|
||||
tag = func.__qualname__ if tag is None else tag
|
||||
if isinstance(tag, Profiler): profiler = tag
|
||||
else: profiler = Profiler.get_profiler(tag=tag, max_captures=max_captures)
|
||||
|
||||
@wraps(func)
|
||||
def wrapper(*args, **kwargs):
|
||||
with profiler:
|
||||
return func(*args, **kwargs)
|
||||
return wrapper
|
||||
|
||||
def __init__(self, tag=None, super_tag=None, max_captures=None):
|
||||
self.paused = False
|
||||
self.pause_time = 0
|
||||
self.pause_started = None
|
||||
self.tag = tag
|
||||
self.super_tag = super_tag
|
||||
self.max_captures = max_captures if max_captures is not None else Profiler.MAX_CAPTURES
|
||||
|
||||
if self.super_tag in Profiler.profilers:
|
||||
self.super_profiler = Profiler.profilers[self.super_tag]
|
||||
self.pause_super = self.super_profiler.pause
|
||||
self.resume_super = self.super_profiler.resume
|
||||
|
||||
else:
|
||||
def noop(self=None):
|
||||
pass
|
||||
def noop(self=None): pass
|
||||
self.super_profiler = None
|
||||
self.pause_super = noop
|
||||
self.resume_super = noop
|
||||
|
|
@ -423,12 +432,18 @@ class Profiler:
|
|||
tag = self.tag
|
||||
super_tag = self.super_tag
|
||||
thread_ident = threading.get_ident()
|
||||
if not tag in Profiler.tags:
|
||||
Profiler.tags[tag] = {"threads": {}, "super": super_tag}
|
||||
if not tag in Profiler.tags: Profiler.tags[tag] = {"threads": {}, "super": super_tag, "captures": deque(maxlen=self.max_captures)}
|
||||
if not thread_ident in Profiler.tags[tag]["threads"]:
|
||||
Profiler.tags[tag]["threads"][thread_ident] = {"current_start": None, "captures": []}
|
||||
Profiler.tags[tag]["threads"][thread_ident] = {"running": deque()}
|
||||
|
||||
Profiler.tags[tag]["threads"][thread_ident]["current_start"] = time.perf_counter()
|
||||
# The tag deque stores a shared reference to the capture, and the end
|
||||
# time isn't updated until the context manager exits. This leaves the
|
||||
# deque sorted by start time, except for cases where the thread is
|
||||
# preemted between getting the current time and appending. We also store
|
||||
# a stack of start times per thread to support reentrancy.
|
||||
capture = {"start": time.perf_counter(), "end": None, "thread_ident": thread_ident}
|
||||
Profiler.tags[tag]["captures"].append(capture)
|
||||
Profiler.tags[tag]["threads"][thread_ident]["running"].append(capture)
|
||||
self.resume_super()
|
||||
|
||||
def __exit__(self, exc_type, exc_value, traceback):
|
||||
|
|
@ -438,14 +453,15 @@ class Profiler:
|
|||
end = time.perf_counter() - self.pause_time
|
||||
self.pause_time = 0
|
||||
thread_ident = threading.get_ident()
|
||||
if tag in Profiler.tags and thread_ident in Profiler.tags[tag]["threads"]:
|
||||
if Profiler.tags[tag]["threads"][thread_ident]["current_start"] != None:
|
||||
begin = Profiler.tags[tag]["threads"][thread_ident]["current_start"]
|
||||
Profiler.tags[tag]["threads"][thread_ident]["current_start"] = None
|
||||
Profiler.tags[tag]["threads"][thread_ident]["captures"].append(end-begin)
|
||||
try:
|
||||
if tag in Profiler.tags and thread_ident in Profiler.tags[tag]["threads"]:
|
||||
try: capture = Profiler.tags[tag]["threads"][thread_ident]["running"].pop()
|
||||
except IndexError as e: return
|
||||
capture["end"] = end
|
||||
if not Profiler._ran:
|
||||
Profiler._ran = True
|
||||
self.resume_super()
|
||||
finally:
|
||||
self.resume_super()
|
||||
|
||||
def pause(self, pause_started=None):
|
||||
if not self.paused:
|
||||
|
|
@ -460,88 +476,219 @@ class Profiler:
|
|||
self.resume_super()
|
||||
|
||||
@staticmethod
|
||||
def ran():
|
||||
return Profiler._ran
|
||||
def ran(): return Profiler._ran
|
||||
|
||||
@staticmethod
|
||||
def results():
|
||||
from statistics import mean, median, stdev
|
||||
def find_window_start(captures, start_time, hi=None):
|
||||
hi = len(captures) if hi is None else hi
|
||||
idx = bisect.bisect_left(captures, start_time, hi=hi, key=lambda c: c["start"])
|
||||
return idx if len(captures) - idx > 1 else None
|
||||
|
||||
# Fast one-pass calculation of summary statistics
|
||||
def calc_stats(captures, start=0, end=None, key=lambda c: c, threads_key=None):
|
||||
if end is None: end = len(captures)
|
||||
while start > 0 and start < len(captures) and start < end and key(captures[start]) is None: start += 1
|
||||
while end > 0 and end <= len(captures) and start < end and key(captures[end-1]) is None: end -= 1
|
||||
count = end - start
|
||||
|
||||
if count <= 0: return None
|
||||
elif count == 1:
|
||||
capture = key(captures[start])
|
||||
return { "count": 1,
|
||||
"mean": capture,
|
||||
"median": capture,
|
||||
"min": capture,
|
||||
"max": capture,
|
||||
"stdev": None,
|
||||
"sum": capture,
|
||||
"threads": 1 if threads_key else None}
|
||||
|
||||
# Median is approximate if there are any incomplete captures in the
|
||||
# middle of the range.
|
||||
med_even = count % 2 == 0
|
||||
med_idx = start + (count // 2 if med_even else (count - 1) // 2)
|
||||
while key(captures[med_idx]) is None and med_idx < end: med_idx += 1
|
||||
if med_idx == end: c_median = None
|
||||
elif med_even: c_median = key(captures[med_idx])
|
||||
else:
|
||||
med_idx2 = med_idx + 1
|
||||
while key(captures[med_idx2]) is None and med_idx2 < end: med_idx2 += 1
|
||||
if med_idx2 == end: c_median = None
|
||||
else:
|
||||
med_cap1 = key(captures[med_idx])
|
||||
med_cap2 = key(captures[med_idx+1])
|
||||
if med_cap1 is None or med_cap2 is None: c_median = None
|
||||
else: c_median = (med_cap1 + med_cap2) / 2
|
||||
|
||||
c_count = 0; c_sum = 0; c_min = key(captures[start]); c_max = key(captures[start]); ck = 0; ck2 = 0
|
||||
uniq_threads = set()
|
||||
for idx in range(start, end):
|
||||
c = key(captures[idx])
|
||||
if c is None: continue
|
||||
else: c_count += 1
|
||||
c_sum += c
|
||||
if c < c_min: c_min = c
|
||||
if c > c_max: c_max = c
|
||||
ck += c - c_median
|
||||
ck2 += (c - c_median) ** 2
|
||||
if threads_key: uniq_threads.add(threads_key(captures[idx]))
|
||||
c_mean = c_sum / c_count if c_count > 0 else None
|
||||
c_std = math.sqrt((ck2 - (ck ** 2)/c_count) / (c_count - 1)) if c_count > 1 else None
|
||||
|
||||
return { "count": c_count,
|
||||
"mean": c_mean,
|
||||
"median": c_median,
|
||||
"min": c_min,
|
||||
"max": c_max,
|
||||
"stdev": c_std,
|
||||
"sum": c_sum,
|
||||
"threads": len(uniq_threads) if threads_key else None }
|
||||
|
||||
def stats_key(capture):
|
||||
end = capture["end"]
|
||||
if end is None: return None
|
||||
else: return end - capture["start"]
|
||||
|
||||
results = {}
|
||||
|
||||
for tag in Profiler.tags:
|
||||
tag_captures = []
|
||||
now = time.perf_counter()
|
||||
for tag in sorted(Profiler.tags):
|
||||
tag_entry = Profiler.tags[tag]
|
||||
|
||||
for thread_ident in tag_entry["threads"]:
|
||||
thread_entry = tag_entry["threads"][thread_ident]
|
||||
thread_captures = thread_entry["captures"]
|
||||
sample_count = len(thread_captures)
|
||||
|
||||
if sample_count > 1:
|
||||
thread_results = {
|
||||
"count": sample_count,
|
||||
"mean": mean(thread_captures),
|
||||
"median": median(thread_captures),
|
||||
"stdev": stdev(thread_captures)
|
||||
}
|
||||
elif sample_count == 1:
|
||||
thread_results = {
|
||||
"count": sample_count,
|
||||
"mean": mean(thread_captures),
|
||||
"median": median(thread_captures),
|
||||
"stdev": None
|
||||
}
|
||||
tag_captures = sorted(tag_entry["captures"], key=lambda c: c["start"])
|
||||
|
||||
tag_captures.extend(thread_captures)
|
||||
if len(tag_captures):
|
||||
captures_1m = None; captures_5m = None; captures_30m = None; captures_60m = None
|
||||
stats_1m = None; stats_5m = None; stats_30m = None; stats_60m = None
|
||||
|
||||
sample_count = len(tag_captures)
|
||||
if sample_count > 1:
|
||||
tag_results = {
|
||||
"name": tag,
|
||||
"super": tag_entry["super"],
|
||||
"count": len(tag_captures),
|
||||
"mean": mean(tag_captures),
|
||||
"median": median(tag_captures),
|
||||
"stdev": stdev(tag_captures)
|
||||
}
|
||||
elif sample_count == 1:
|
||||
tag_results = {
|
||||
"name": tag,
|
||||
"super": tag_entry["super"],
|
||||
"count": len(tag_captures),
|
||||
"mean": mean(tag_captures),
|
||||
"median": median(tag_captures),
|
||||
"stdev": None
|
||||
}
|
||||
captures_1m = find_window_start(tag_captures, now - 1*60)
|
||||
if captures_1m: captures_5m = find_window_start(tag_captures, now - 5*60, hi=captures_1m)
|
||||
if captures_5m: captures_30m = find_window_start(tag_captures, now - 30*60, hi=captures_5m)
|
||||
if captures_30m: captures_60m = find_window_start(tag_captures, now - 60*60, hi=captures_30m)
|
||||
|
||||
results[tag] = tag_results
|
||||
stats_all = calc_stats(tag_captures, 0, None, key=stats_key, threads_key=lambda c: c["thread_ident"])
|
||||
if captures_1m: stats_1m = calc_stats(tag_captures, captures_1m, None, key=stats_key)
|
||||
if captures_5m: stats_5m = calc_stats(tag_captures, captures_5m, captures_1m, key=stats_key)
|
||||
if captures_30m: stats_30m = calc_stats(tag_captures, captures_30m, captures_5m, key=stats_key)
|
||||
if captures_60m: stats_60m = calc_stats(tag_captures, captures_60m, captures_30m, key=stats_key)
|
||||
|
||||
if stats_all["count"]:
|
||||
results[tag] = { "name": tag,
|
||||
"super": tag_entry["super"],
|
||||
"stats_all": stats_all,
|
||||
"stats_1m": stats_1m,
|
||||
"stats_5m": stats_5m,
|
||||
"stats_30m": stats_30m,
|
||||
"stats_60m": stats_60m }
|
||||
|
||||
# Yield to avoid bogging down the instance
|
||||
time.sleep(0.001)
|
||||
|
||||
return results
|
||||
|
||||
@staticmethod
|
||||
def format_results(results):
|
||||
def pst(time):
|
||||
if time is not None: return prettyshorttime(time, tight=True)
|
||||
else: return "-----"
|
||||
|
||||
def print_results_recursive(tag, results, level=0):
|
||||
print_tag_results(tag, level+1)
|
||||
results_str = print_tag_results(tag, level+1) + "\n"
|
||||
|
||||
for tag_name in results:
|
||||
sub_tag = results[tag_name]
|
||||
if sub_tag["super"] == tag["name"]:
|
||||
print_results_recursive(sub_tag, results, level=level+1)
|
||||
results_str += print_results_recursive(sub_tag, results, level=level+1)
|
||||
|
||||
return results_str
|
||||
|
||||
def print_tag_results(tag, level):
|
||||
ind = " "*level
|
||||
name = tag["name"]; count = tag["count"]
|
||||
mean = tag["mean"]; median = tag["median"]; stdev = tag["stdev"]
|
||||
print( f"{ind}{name}")
|
||||
print( f"{ind} Samples : {count}")
|
||||
if stdev != None:
|
||||
print(f"{ind} Mean : {prettyshorttime(mean)}")
|
||||
print(f"{ind} Median : {prettyshorttime(median)}")
|
||||
print(f"{ind} St.dev. : {prettyshorttime(stdev)}")
|
||||
print( f"{ind} Total : {prettyshorttime(mean*count)}")
|
||||
print("")
|
||||
name = tag["name"]
|
||||
stats_all = tag["stats_all"]; stats_1m = tag["stats_1m"]; stats_5m = tag["stats_5m"]; stats_30m = tag["stats_30m"]; stats_60m = tag["stats_60m"]
|
||||
results_str = f" {ind}{name}\n"
|
||||
results_str += f" {ind} Samples : {stats_all['count']} from {stats_all['threads']} thread{'s' if stats_all['threads'] > 1 else ''}\n"
|
||||
if stats_all != None:
|
||||
results_str += f" {ind} {'Mean':^15} | {'Median':^15} | {'Min':^15} | {'Max':^15} | {'St. dev':^15} | {'Total':^15}\n"
|
||||
results_str += f" {ind} Stats : ({pst(stats_all['mean']):^15} | {pst(stats_all['median']):^15} | {pst(stats_all['min']):^15} | {pst(stats_all['max']):^15} | {pst(stats_all['stdev']):^15} | {pst(stats_all['sum']):^15})\n"
|
||||
if stats_1m != None:
|
||||
results_str += f" {ind} 0-1m : ({pst(stats_1m['mean']):^15} | {pst(stats_1m['median']):^15} | {pst(stats_1m['min']):^15} | {pst(stats_1m['max']):^15} | {pst(stats_1m['stdev']):^15} | {pst(stats_1m['sum']):^15})\n"
|
||||
if stats_5m != None:
|
||||
results_str += f" {ind} 1-5m : ({pst(stats_5m['mean']):^15} | {pst(stats_5m['median']):^15} | {pst(stats_5m['min']):^15} | {pst(stats_5m['max']):^15} | {pst(stats_5m['stdev']):^15} | {pst(stats_5m['sum']):^15})\n"
|
||||
if stats_30m != None:
|
||||
results_str += f" {ind} 5-30m : ({pst(stats_30m['mean']):^15} | {pst(stats_30m['median']):^15} | {pst(stats_30m['min']):^15} | {pst(stats_30m['max']):^15} | {pst(stats_30m['stdev']):^15} | {pst(stats_30m['sum']):^15})\n"
|
||||
if stats_60m != None:
|
||||
results_str += f" {ind} 30-60m : ({pst(stats_60m['mean']):^15} | {pst(stats_60m['median']):^15} | {pst(stats_60m['min']):^15} | {pst(stats_60m['max']):^15} | {pst(stats_60m['stdev']):^15} | {pst(stats_60m['sum']):^15})\n"
|
||||
return results_str
|
||||
|
||||
print("\nProfiler results:\n")
|
||||
results_str = ""
|
||||
for tag_name in results:
|
||||
tag = results[tag_name]
|
||||
if tag["super"] == None:
|
||||
print_results_recursive(tag, results)
|
||||
results_str += print_results_recursive(tag, results)
|
||||
|
||||
profile = Profiler.get_profiler
|
||||
return results_str
|
||||
|
||||
profile = Profiler.get_profiler
|
||||
|
||||
# The base-256 table is likely to change. Currently, it is just
|
||||
# experimental, so don't count on it too much just yet.
|
||||
b256 = [
|
||||
# 0 1 2 3 4 5 6 7 8 9 A B C D F F
|
||||
"a","b","c","d","e","f","g","h","i","j","k","l","m","n","o","p", # 0x0 Latin & numerals
|
||||
"q","r","s","t","u","v","x","y","z","æ","ø","0","1","2","3","4", # 0x1 Latin & numerals
|
||||
"A","B","C","D","E","F","G","H","I","J","K","L","M","N","O","P", # 0x2 Latin & numerals
|
||||
"Q","R","S","T","U","W","X","Y","Z","Æ","Ø","5","6","7","8","9", # 0x3 Latin & numerals
|
||||
"α","β","γ","δ","ε","ζ","η","θ","ι","κ","λ","μ","ν","ξ","π","ρ", # 0x4 Greek
|
||||
"σ","τ","φ","χ","ψ","ω","Γ","Δ","Θ","Λ","Ξ","Π","Σ","Φ","Ψ","Ω", # 0x5 Greek
|
||||
"Б","Д","Ж","З","И","Л","П","Ц","Ч","Ш","Щ","Ъ","Ы","Э","Ю","Я", # 0x6 Cyrillic
|
||||
"б","д","ж","з","и","л","п","ц","ч","ш","щ","ъ","ы","э","ю","я", # 0x7 Cyrillic
|
||||
"Ա","Բ","Գ","Դ","Ե","Զ","Է","Ը","Թ","Ժ","Ի","Խ","Ծ","Կ","Հ","Ձ", # 0x8 Armenian Capitals
|
||||
"Ղ","Ճ","Մ","Յ","Ն","Շ","Ո","Չ","Պ","Ջ","Վ","Ր","Ց","Ւ","Ք","Ֆ", # 0x9 Armenian Captials
|
||||
"ᚠ","ᚢ","ᚦ","ᚱ","ᚹ","ᚺ","ᚾ","ᛈ","ᛇ","ᛉ","ᛊ","ᛏ","ᛒ","ᛖ","ᛗ","ᛟ", # 0xA Elder Futhark
|
||||
"イ","ウ","オ","カ","キ","ケ","サ","シ","ス","セ","タ","チ","テ","ト","ナ","ヌ", # 0xB Katakana
|
||||
"ネ","ヒ","フ","ヘ","ホ","マ","ミ","ム","メ","モ","ヤ","ラ","リ","ル","レ","ワ", # 0xC Katakana
|
||||
"𐑐","𐑑","𐑒","𐑔","𐑕","𐑗","𐑙","𐑳","𐑶","𐑸","𐑹","𐑺","𐑻","𐑽","𐑾","𐑿", # 0xD Shavian
|
||||
"᱑","᱕","᱘","᱙","ᱚ","ᱝ","ᱟ","ᱣ","ᱦ","ᱨ","ᱬ","ᱭ","ᱰ","ᱳ","ᱶ","ᱷ", # 0xE Ol Chiki
|
||||
"𐌳","𐌸","𐌾","𐐀","𐐁","𐐂","𐐆","𐐇","𐐈","𐐉","𐐊","𐐋","𐐌","𐐍","𐐎","𐐏", # 0xF Gothic & Deseret
|
||||
]
|
||||
|
||||
def b256rep(data): return "".join(bytes_to_b256(data))
|
||||
def prettyb256rep(data): return f"<{b256rep(data)}>"
|
||||
|
||||
def b256_to_byte(point):
|
||||
if not type(point) == str or not len(point) == 1: raise TypeError("Invalid input data for base256 byte decode")
|
||||
try: return b256.index(point)
|
||||
except Exception as e: raise ValueError(f"Could not decode base256 byte: {e}")
|
||||
|
||||
def b256_to_bytes(b256rep):
|
||||
if not type(b256rep) == str: raise TypeError("Invalid input data for base256 decode")
|
||||
try: return bytes([b256.index(c) for c in b256rep])
|
||||
except Exception as e: raise ValueError(f"Could not decode base256: {e}")
|
||||
|
||||
def byte_to_b256(input_byte):
|
||||
if type(input_byte) == bytes and not len(input_byte) == 1: TypeError("Invalid input data for base256 byte encode")
|
||||
if type(input_byte) == bytes and len(input_byte) == 1: input_byte = ord(input_byte)
|
||||
if not type(input_byte) == int: raise TypeError("Invalid input data for base256 byte encode")
|
||||
try: return b256[int(input_byte)]
|
||||
except Exception as e: raise TypeError(f"Could not encode byte to base256: {e}")
|
||||
|
||||
def bytes_to_b256(data):
|
||||
if not type(data) == bytes: raise TypeError("Invalid input data for base256 encode")
|
||||
try: return [byte_to_b256(c) for c in data]
|
||||
except Exception as e: raise TypeError(f"Could not encode to base256: {e}")
|
||||
|
||||
|
||||
from .Reticulum import Reticulum
|
||||
from .Identity import Identity
|
||||
from .Link import Link, RequestReceipt
|
||||
from .Channel import MessageBase
|
||||
from .Buffer import Buffer, RawChannelReader, RawChannelWriter
|
||||
from .Transport import Transport
|
||||
from .Discovery import InterfaceAnnouncer
|
||||
from .Destination import Destination
|
||||
from .Packet import Packet
|
||||
from .Packet import PacketReceipt
|
||||
from .Resolver import Resolver
|
||||
from .Resource import Resource, ResourceAdvertisement
|
||||
from .Cryptography import HKDF
|
||||
from .Cryptography import Hashes
|
||||
|
|
@ -1 +1 @@
|
|||
__version__ = "1.1.2"
|
||||
__version__ = "1.6.1"
|
||||
|
|
|
|||
6
RNS/vendor/__init__.py
vendored
6
RNS/vendor/__init__.py
vendored
|
|
@ -3,5 +3,7 @@ import glob
|
|||
|
||||
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
|
||||
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
|
||||
modules = py_modules+pyc_modules
|
||||
__all__ = list(set([os.path.basename(f).replace(".pyc", "").replace(".py", "") for f in modules if not (f.endswith("__init__.py") or f.endswith("__init__.pyc"))]))
|
||||
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
|
||||
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
|
||||
modules = py_modules+pyc_modules+so_modules+pyd_modules
|
||||
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))
|
||||
65
RNS/vendor/i2plib/aiosam.py
vendored
65
RNS/vendor/i2plib/aiosam.py
vendored
|
|
@ -6,14 +6,12 @@ from . import utils
|
|||
from .log import logger
|
||||
|
||||
def parse_reply(data):
|
||||
if not data:
|
||||
raise ConnectionAbortedError("Empty response: SAM API went offline")
|
||||
if not data: raise ConnectionAbortedError("Empty response: SAM API went offline")
|
||||
|
||||
try:
|
||||
msg = sam.Message(data.decode().strip())
|
||||
logger.debug("SAM reply: "+str(msg))
|
||||
except:
|
||||
raise ConnectionAbortedError("Invalid SAM response")
|
||||
except: raise ConnectionAbortedError("Invalid SAM response")
|
||||
|
||||
return msg
|
||||
|
||||
|
|
@ -28,8 +26,7 @@ async def get_sam_socket(sam_address=sam.DEFAULT_ADDRESS, loop=None):
|
|||
reader, writer = await asyncio.open_connection(*sam_address)
|
||||
writer.write(sam.hello("3.1", "3.1"))
|
||||
reply = parse_reply(await reader.readline())
|
||||
if reply.ok:
|
||||
return (reader, writer)
|
||||
if reply.ok: return (reader, writer)
|
||||
else:
|
||||
writer.close()
|
||||
raise exceptions.SAM_EXCEPTIONS[reply["RESULT"]]()
|
||||
|
|
@ -49,10 +46,8 @@ async def dest_lookup(domain, sam_address=sam.DEFAULT_ADDRESS,
|
|||
writer.write(sam.naming_lookup(domain))
|
||||
reply = parse_reply(await reader.readline())
|
||||
writer.close()
|
||||
if reply.ok:
|
||||
return sam.Destination(reply["VALUE"])
|
||||
else:
|
||||
raise exceptions.SAM_EXCEPTIONS[reply["RESULT"]]()
|
||||
if reply.ok: return sam.Destination(reply["VALUE"])
|
||||
else: raise exceptions.SAM_EXCEPTIONS[reply["RESULT"]]()
|
||||
|
||||
async def new_destination(sam_address=sam.DEFAULT_ADDRESS, loop=None,
|
||||
sig_type=sam.Destination.default_sig_type):
|
||||
|
|
@ -91,15 +86,10 @@ async def create_session(session_name, sam_address=sam.DEFAULT_ADDRESS,
|
|||
"""
|
||||
logger.debug("Creating session {}".format(session_name))
|
||||
if destination:
|
||||
if type(destination) == sam.Destination:
|
||||
destination = destination
|
||||
else:
|
||||
destination = sam.Destination(
|
||||
destination, has_private_key=True)
|
||||
|
||||
if type(destination) == sam.Destination: destination = destination
|
||||
else: destination = sam.Destination(destination, has_private_key=True)
|
||||
dest_string = destination.private_key.base64
|
||||
else:
|
||||
dest_string = sam.TRANSIENT_DESTINATION
|
||||
else: dest_string = sam.TRANSIENT_DESTINATION
|
||||
|
||||
options = " ".join(["{}={}".format(k, v) for k, v in options.items()])
|
||||
|
||||
|
|
@ -109,9 +99,7 @@ async def create_session(session_name, sam_address=sam.DEFAULT_ADDRESS,
|
|||
|
||||
reply = parse_reply(await reader.readline())
|
||||
if reply.ok:
|
||||
if not destination:
|
||||
destination = sam.Destination(
|
||||
reply["DESTINATION"], has_private_key=True)
|
||||
if not destination: destination = sam.Destination(reply["DESTINATION"], has_private_key=True)
|
||||
logger.debug(destination.base32)
|
||||
logger.debug("Session created {}".format(session_name))
|
||||
return (reader, writer)
|
||||
|
|
@ -130,10 +118,8 @@ async def stream_connect(session_name, destination,
|
|||
:return: A (reader, writer) pair
|
||||
"""
|
||||
logger.debug("Connecting stream {}".format(session_name))
|
||||
if isinstance(destination, str) and not destination.endswith(".i2p"):
|
||||
destination = sam.Destination(destination)
|
||||
elif isinstance(destination, str):
|
||||
destination = await dest_lookup(destination, sam_address, loop)
|
||||
if isinstance(destination, str) and not destination.endswith(".i2p"): destination = sam.Destination(destination)
|
||||
elif isinstance(destination, str): destination = await dest_lookup(destination, sam_address, loop)
|
||||
|
||||
reader, writer = await get_sam_socket(sam_address, loop)
|
||||
writer.write(sam.stream_connect(session_name, destination.base64,
|
||||
|
|
@ -158,8 +144,7 @@ async def stream_accept(session_name, sam_address=sam.DEFAULT_ADDRESS,
|
|||
reader, writer = await get_sam_socket(sam_address, loop)
|
||||
writer.write(sam.stream_accept(session_name, silent="false"))
|
||||
reply = parse_reply(await reader.readline())
|
||||
if reply.ok:
|
||||
return (reader, writer)
|
||||
if reply.ok: return (reader, writer)
|
||||
else:
|
||||
writer.close()
|
||||
raise exceptions.SAM_EXCEPTIONS[reply["RESULT"]]()
|
||||
|
|
@ -183,9 +168,9 @@ class Session:
|
|||
:return: :class:`Session` object
|
||||
"""
|
||||
def __init__(self, session_name, sam_address=sam.DEFAULT_ADDRESS,
|
||||
loop=None, style="STREAM",
|
||||
signature_type=sam.Destination.default_sig_type,
|
||||
destination=None, options={}):
|
||||
loop=None, style="STREAM", signature_type=sam.Destination.default_sig_type,
|
||||
destination=None, options={}):
|
||||
|
||||
self.session_name = session_name
|
||||
self.sam_address = sam_address
|
||||
self.loop = loop
|
||||
|
|
@ -195,10 +180,9 @@ class Session:
|
|||
self.options = options
|
||||
|
||||
async def __aenter__(self):
|
||||
self.reader, self.writer = await create_session(self.session_name,
|
||||
sam_address=self.sam_address, loop=self.loop, style=self.style,
|
||||
signature_type=self.signature_type,
|
||||
destination=self.destination, options=self.options)
|
||||
self.reader, self.writer = await create_session(self.session_name, sam_address=self.sam_address, loop=self.loop,
|
||||
style=self.style, signature_type=self.signature_type,
|
||||
destination=self.destination, options=self.options)
|
||||
return self
|
||||
|
||||
async def __aexit__(self, exc_type, exc, tb):
|
||||
|
|
@ -214,16 +198,15 @@ class StreamConnection:
|
|||
:param loop: (optional) Event loop instance
|
||||
:return: :class:`StreamConnection` object
|
||||
"""
|
||||
def __init__(self, session_name, destination,
|
||||
sam_address=sam.DEFAULT_ADDRESS, loop=None):
|
||||
def __init__(self, session_name, destination, sam_address=sam.DEFAULT_ADDRESS, loop=None):
|
||||
self.session_name = session_name
|
||||
self.sam_address = sam_address
|
||||
self.loop = loop
|
||||
self.destination = destination
|
||||
|
||||
async def __aenter__(self):
|
||||
self.reader, self.writer = await stream_connect(self.session_name,
|
||||
self.destination, sam_address=self.sam_address, loop=self.loop)
|
||||
self.reader, self.writer = await stream_connect(self.session_name, self.destination,
|
||||
sam_address=self.sam_address, loop=self.loop)
|
||||
self.read = self.reader.read
|
||||
self.write = self.writer.write
|
||||
return self
|
||||
|
|
@ -240,15 +223,13 @@ class StreamAcceptor:
|
|||
:param loop: (optional) Event loop instance
|
||||
:return: :class:`StreamAcceptor` object
|
||||
"""
|
||||
def __init__(self, session_name, sam_address=sam.DEFAULT_ADDRESS,
|
||||
loop=None):
|
||||
def __init__(self, session_name, sam_address=sam.DEFAULT_ADDRESS, loop=None):
|
||||
self.session_name = session_name
|
||||
self.sam_address = sam_address
|
||||
self.loop = loop
|
||||
|
||||
async def __aenter__(self):
|
||||
self.reader, self.writer = await stream_accept(self.session_name,
|
||||
sam_address=self.sam_address, loop=self.loop)
|
||||
self.reader, self.writer = await stream_accept(self.session_name, sam_address=self.sam_address, loop=self.loop)
|
||||
self.read = self.reader.read
|
||||
self.write = self.writer.write
|
||||
return self
|
||||
|
|
|
|||
21
RNS/vendor/i2plib/exceptions.py
vendored
21
RNS/vendor/i2plib/exceptions.py
vendored
|
|
@ -30,15 +30,12 @@ class PeerNotFound(SAMException):
|
|||
class Timeout(SAMException):
|
||||
"""The peer cannot be found on the network"""
|
||||
|
||||
SAM_EXCEPTIONS = {
|
||||
"CANT_REACH_PEER": CantReachPeer,
|
||||
"DUPLICATED_DEST": DuplicatedDest,
|
||||
"DUPLICATED_ID": DuplicatedId,
|
||||
"I2P_ERROR": I2PError,
|
||||
"INVALID_ID": InvalidId,
|
||||
"INVALID_KEY": InvalidKey,
|
||||
"KEY_NOT_FOUND": KeyNotFound,
|
||||
"PEER_NOT_FOUND": PeerNotFound,
|
||||
"TIMEOUT": Timeout,
|
||||
}
|
||||
|
||||
SAM_EXCEPTIONS = { "CANT_REACH_PEER": CantReachPeer,
|
||||
"DUPLICATED_DEST": DuplicatedDest,
|
||||
"DUPLICATED_ID": DuplicatedId,
|
||||
"I2P_ERROR": I2PError,
|
||||
"INVALID_ID": InvalidId,
|
||||
"INVALID_KEY": InvalidKey,
|
||||
"KEY_NOT_FOUND": KeyNotFound,
|
||||
"PEER_NOT_FOUND": PeerNotFound,
|
||||
"TIMEOUT": Timeout }
|
||||
7
RNS/vendor/i2plib/sam.py
vendored
7
RNS/vendor/i2plib/sam.py
vendored
|
|
@ -3,7 +3,6 @@ from hashlib import sha256
|
|||
import struct
|
||||
import re
|
||||
|
||||
|
||||
I2P_B64_CHARS = "-~"
|
||||
|
||||
def i2p_b64encode(x):
|
||||
|
|
@ -41,11 +40,9 @@ class Message(object):
|
|||
return self.opts[key]
|
||||
|
||||
@property
|
||||
def ok(self):
|
||||
return self["RESULT"] == "OK"
|
||||
def ok(self): return self["RESULT"] == "OK"
|
||||
|
||||
def __repr__(self):
|
||||
return self._reply_string
|
||||
def __repr__(self): return self._reply_string
|
||||
|
||||
|
||||
# SAM request messages
|
||||
|
|
|
|||
125
RNS/vendor/i2plib/tunnel.py
vendored
125
RNS/vendor/i2plib/tunnel.py
vendored
|
|
@ -15,15 +15,13 @@ async def proxy_data(reader, writer):
|
|||
while True:
|
||||
data = await reader.read(BUFFER_SIZE)
|
||||
if not data:
|
||||
logger.debug('no more data')
|
||||
break
|
||||
writer.write(data)
|
||||
except Exception as e:
|
||||
logger.debug('proxy_data_task exception {}'.format(e))
|
||||
except Exception as e: logger.debug('proxy_data_task exception {}'.format(e))
|
||||
finally:
|
||||
try:
|
||||
writer.close()
|
||||
except RuntimeError:
|
||||
pass
|
||||
try: writer.close()
|
||||
except RuntimeError: pass
|
||||
logger.debug('close connection')
|
||||
|
||||
class I2PTunnel(object):
|
||||
|
|
@ -44,6 +42,7 @@ class I2PTunnel(object):
|
|||
|
||||
def __init__(self, local_address, destination=None, session_name=None,
|
||||
options={}, loop=None, sam_address=sam.DEFAULT_ADDRESS):
|
||||
|
||||
self.local_address = local_address
|
||||
self.destination = destination
|
||||
self.session_name = session_name or utils.generate_session_id()
|
||||
|
|
@ -53,16 +52,13 @@ class I2PTunnel(object):
|
|||
|
||||
async def _pre_run(self):
|
||||
if not self.destination:
|
||||
self.destination = await aiosam.new_destination(
|
||||
sam_address=self.sam_address, loop=self.loop)
|
||||
_, self.session_writer = await aiosam.create_session(
|
||||
self.session_name, style=self.style, options=self.options,
|
||||
sam_address=self.sam_address,
|
||||
loop=self.loop, destination=self.destination)
|
||||
self.destination = await aiosam.new_destination(sam_address=self.sam_address, loop=self.loop)
|
||||
|
||||
def stop(self):
|
||||
"""Stop the tunnel"""
|
||||
self.session_writer.close()
|
||||
_, self.session_writer = await aiosam.create_session(self.session_name, style=self.style, options=self.options,
|
||||
sam_address=self.sam_address,
|
||||
loop=self.loop, destination=self.destination)
|
||||
|
||||
def stop(self): self.session_writer.close()
|
||||
|
||||
class ClientTunnel(I2PTunnel):
|
||||
"""Client tunnel, a subclass of tunnel.I2PTunnel
|
||||
|
|
@ -85,20 +81,25 @@ class ClientTunnel(I2PTunnel):
|
|||
"""A coroutine used to run the tunnel"""
|
||||
await self._pre_run()
|
||||
|
||||
self.status = { "setup_ran": False, "setup_failed": False, "exception": None, "connect_tasks": [] }
|
||||
self.status = { "setup_ran": False, "setup_failed": False, "exception": None,
|
||||
"connect_tasks": set(), "background_tasks": set() }
|
||||
|
||||
async def handle_client(client_reader, client_writer):
|
||||
"""Handle local client connection"""
|
||||
try:
|
||||
sc_task = aiosam.stream_connect(
|
||||
self.session_name, self.remote_destination,
|
||||
sam_address=self.sam_address, loop=self.loop)
|
||||
self.status["connect_tasks"].append(sc_task)
|
||||
sc_task = aiosam.stream_connect(self.session_name, self.remote_destination,
|
||||
sam_address=self.sam_address, loop=self.loop)
|
||||
|
||||
self.status["connect_tasks"].add(sc_task)
|
||||
|
||||
remote_reader, remote_writer = await sc_task
|
||||
asyncio.ensure_future(proxy_data(remote_reader, client_writer),
|
||||
loop=self.loop)
|
||||
asyncio.ensure_future(proxy_data(client_reader, remote_writer),
|
||||
loop=self.loop)
|
||||
task = asyncio.ensure_future(proxy_data(remote_reader, client_writer), loop=self.loop)
|
||||
self.status["background_tasks"].add(task)
|
||||
task.add_done_callback(self.status["background_tasks"].discard)
|
||||
|
||||
task = asyncio.ensure_future(proxy_data(client_reader, remote_writer), loop=self.loop)
|
||||
self.status["background_tasks"].add(task)
|
||||
task.add_done_callback(self.status["background_tasks"].discard)
|
||||
|
||||
except Exception as e:
|
||||
self.status["setup_ran"] = True
|
||||
|
|
@ -135,14 +136,15 @@ class ServerTunnel(I2PTunnel):
|
|||
"""A coroutine used to run the tunnel"""
|
||||
await self._pre_run()
|
||||
|
||||
self.status = { "setup_ran": False, "setup_failed": False, "exception": None, "connect_tasks": [] }
|
||||
self.status = { "setup_ran": False, "setup_failed": False, "exception": None,
|
||||
"connect_tasks": set(), "background_tasks": set() }
|
||||
|
||||
async def handle_client(incoming, client_reader, client_writer):
|
||||
try:
|
||||
# data and dest may come in one chunk
|
||||
dest, data = incoming.split(b"\n", 1)
|
||||
remote_destination = sam.Destination(dest.decode())
|
||||
logger.debug("{} client connected: {}.b32.i2p".format(
|
||||
self.session_name, remote_destination.base32))
|
||||
logger.debug("{} client connected: {}.b32.i2p".format(self.session_name, remote_destination.base32))
|
||||
|
||||
except Exception as e:
|
||||
self.status["exception"] = e
|
||||
|
|
@ -150,21 +152,20 @@ class ServerTunnel(I2PTunnel):
|
|||
data = None
|
||||
|
||||
try:
|
||||
sc_task = asyncio.wait_for(
|
||||
asyncio.open_connection(
|
||||
host=self.local_address[0],
|
||||
port=self.local_address[1]),
|
||||
timeout=5)
|
||||
self.status["connect_tasks"].append(sc_task)
|
||||
sc_task = asyncio.wait_for(asyncio.open_connection(host=self.local_address[0], port=self.local_address[1]), timeout=5)
|
||||
self.status["connect_tasks"].add(sc_task)
|
||||
|
||||
remote_reader, remote_writer = await sc_task
|
||||
if data: remote_writer.write(data)
|
||||
asyncio.ensure_future(proxy_data(remote_reader, client_writer),
|
||||
loop=self.loop)
|
||||
asyncio.ensure_future(proxy_data(client_reader, remote_writer),
|
||||
loop=self.loop)
|
||||
task = asyncio.ensure_future(proxy_data(remote_reader, client_writer), loop=self.loop)
|
||||
self.status["background_tasks"].add(task)
|
||||
task.add_done_callback(self.status["background_tasks"].discard)
|
||||
|
||||
except ConnectionRefusedError:
|
||||
task = asyncio.ensure_future(proxy_data(client_reader, remote_writer), loop=self.loop)
|
||||
self.status["background_tasks"].add(task)
|
||||
task.add_done_callback(self.status["background_tasks"].discard)
|
||||
|
||||
except ConnectionRefusedError as e:
|
||||
client_writer.close()
|
||||
self.status["exception"] = e
|
||||
self.status["setup_failed"] = True
|
||||
|
|
@ -172,14 +173,14 @@ class ServerTunnel(I2PTunnel):
|
|||
async def server_loop():
|
||||
try:
|
||||
while True:
|
||||
client_reader, client_writer = await aiosam.stream_accept(
|
||||
self.session_name, sam_address=self.sam_address,
|
||||
loop=self.loop)
|
||||
client_reader, client_writer = await aiosam.stream_accept(self.session_name, sam_address=self.sam_address,
|
||||
loop=self.loop)
|
||||
incoming = await client_reader.read(BUFFER_SIZE)
|
||||
asyncio.ensure_future(handle_client(
|
||||
incoming, client_reader, client_writer), loop=self.loop)
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
task = asyncio.ensure_future(handle_client(incoming, client_reader, client_writer), loop=self.loop)
|
||||
self.status["background_tasks"].add(task)
|
||||
task.add_done_callback(self.status["background_tasks"].discard)
|
||||
|
||||
except asyncio.CancelledError: pass
|
||||
|
||||
self.server_loop = asyncio.ensure_future(server_loop(), loop=self.loop)
|
||||
self.status["setup_ran"] = True
|
||||
|
|
@ -191,16 +192,11 @@ class ServerTunnel(I2PTunnel):
|
|||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('type', metavar="TYPE", choices=('server', 'client'),
|
||||
help="Tunnel type (server or client)")
|
||||
parser.add_argument('address', metavar="ADDRESS",
|
||||
help="Local address (e.g. 127.0.0.1:8000)")
|
||||
parser.add_argument('--debug', '-d', action='store_true',
|
||||
help='Debugging')
|
||||
parser.add_argument('--key', '-k', default='', metavar='PRIVATE_KEY',
|
||||
help='Path to private key file')
|
||||
parser.add_argument('--destination', '-D', default='',
|
||||
metavar='DESTINATION', help='Remote destination')
|
||||
parser.add_argument('type', metavar="TYPE", choices=('server', 'client'), help="Tunnel type (server or client)")
|
||||
parser.add_argument('address', metavar="ADDRESS", help="Local address (e.g. 127.0.0.1:8000)")
|
||||
parser.add_argument('--debug', '-d', action='store_true', help='Debugging')
|
||||
parser.add_argument('--key', '-k', default='', metavar='PRIVATE_KEY', help='Path to private key file')
|
||||
parser.add_argument('--destination', '-D', default='', metavar='DESTINATION', help='Remote destination')
|
||||
args = parser.parse_args()
|
||||
|
||||
SAM_ADDRESS = utils.get_sam_address()
|
||||
|
|
@ -209,26 +205,21 @@ if __name__ == '__main__':
|
|||
loop = asyncio.get_event_loop()
|
||||
loop.set_debug(args.debug)
|
||||
|
||||
if args.key:
|
||||
destination = sam.Destination(path=args.key, has_private_key=True)
|
||||
else:
|
||||
destination = None
|
||||
if args.key: destination = sam.Destination(path=args.key, has_private_key=True)
|
||||
else: destination = None
|
||||
|
||||
local_address = utils.address_from_string(args.address)
|
||||
|
||||
if args.type == "client":
|
||||
tunnel = ClientTunnel(args.destination, local_address, loop=loop,
|
||||
destination=destination, sam_address=SAM_ADDRESS)
|
||||
tunnel = ClientTunnel(args.destination, local_address, loop=loop, destination=destination, sam_address=SAM_ADDRESS)
|
||||
|
||||
elif args.type == "server":
|
||||
tunnel = ServerTunnel(local_address, loop=loop, destination=destination,
|
||||
sam_address=SAM_ADDRESS)
|
||||
tunnel = ServerTunnel(local_address, loop=loop, destination=destination, sam_address=SAM_ADDRESS)
|
||||
|
||||
asyncio.ensure_future(tunnel.run(), loop=loop)
|
||||
|
||||
try:
|
||||
loop.run_forever()
|
||||
except KeyboardInterrupt:
|
||||
tunnel.stop()
|
||||
try: loop.run_forever()
|
||||
except KeyboardInterrupt: tunnel.stop()
|
||||
finally:
|
||||
loop.stop()
|
||||
loop.close()
|
||||
|
|
|
|||
233
RNS/vendor/umsgpack.py
vendored
233
RNS/vendor/umsgpack.py
vendored
|
|
@ -2,7 +2,7 @@
|
|||
# https://github.com/vsergeev/u-msgpack-python
|
||||
#
|
||||
# u-msgpack-python is a lightweight MessagePack serializer and deserializer
|
||||
# module, compatible with both Python 2 and 3, as well CPython and PyPy
|
||||
# module, compatible with Python 3, as well CPython and PyPy
|
||||
# implementations of Python. u-msgpack-python is fully compliant with the
|
||||
# latest MessagePack specification.com/msgpack/msgpack/blob/master/spec.md). In
|
||||
# particular, it supports the new binary, UTF-8 string, and application ext
|
||||
|
|
@ -35,7 +35,7 @@ u-msgpack-python v2.7.1 - v at sergeev.io
|
|||
https://github.com/vsergeev/u-msgpack-python
|
||||
|
||||
u-msgpack-python is a lightweight MessagePack serializer and deserializer
|
||||
module, compatible with both Python 2 and 3, as well CPython and PyPy
|
||||
module, compatible with Python 3, as well CPython and PyPy
|
||||
implementations of Python. u-msgpack-python is fully compliant with the
|
||||
latest MessagePack specification.com/msgpack/msgpack/blob/master/spec.md). In
|
||||
particular, it supports the new binary, UTF-8 string, and application ext
|
||||
|
|
@ -49,10 +49,7 @@ import datetime
|
|||
import sys
|
||||
import io
|
||||
|
||||
if sys.version_info[0:2] >= (3, 3):
|
||||
from collections.abc import Hashable
|
||||
else:
|
||||
from collections import Hashable
|
||||
from collections.abc import Hashable
|
||||
|
||||
__version__ = "2.7.1"
|
||||
"Module version string"
|
||||
|
|
@ -104,8 +101,6 @@ class Ext(object):
|
|||
# Check data is type bytes or str
|
||||
elif sys.version_info[0] == 3 and not isinstance(data, bytes):
|
||||
raise TypeError("ext data is not type \'bytes\'")
|
||||
elif sys.version_info[0] == 2 and not isinstance(data, str):
|
||||
raise TypeError("ext data is not type \'str\'")
|
||||
|
||||
self.type = type
|
||||
self.data = data
|
||||
|
|
@ -129,7 +124,7 @@ class Ext(object):
|
|||
"""
|
||||
s = "Ext Object (Type: {:d}, Data: ".format(self.type)
|
||||
s += " ".join(["0x{:02}".format(ord(self.data[i:i + 1]))
|
||||
for i in xrange(min(len(self.data), 8))])
|
||||
for i in range(min(len(self.data), 8))])
|
||||
if len(self.data) > 8:
|
||||
s += " ..."
|
||||
s += ")"
|
||||
|
|
@ -452,94 +447,6 @@ def _pack_map(obj, fp, options):
|
|||
|
||||
########################################
|
||||
|
||||
|
||||
# Pack for Python 2, with 'unicode' type, 'str' type, and 'long' type
|
||||
def _pack2(obj, fp, **options):
|
||||
"""
|
||||
Serialize a Python object into MessagePack bytes.
|
||||
|
||||
Args:
|
||||
obj: a Python object
|
||||
fp: a .write()-supporting file-like object
|
||||
|
||||
Kwargs:
|
||||
ext_handlers (dict): dictionary of Ext handlers, mapping a custom type
|
||||
to a callable that packs an instance of the type
|
||||
into an Ext object
|
||||
force_float_precision (str): "single" to force packing floats as
|
||||
IEEE-754 single-precision floats,
|
||||
"double" to force packing floats as
|
||||
IEEE-754 double-precision floats.
|
||||
|
||||
Returns:
|
||||
None.
|
||||
|
||||
Raises:
|
||||
UnsupportedType(PackException):
|
||||
Object type not supported for packing.
|
||||
|
||||
Example:
|
||||
>>> f = open('test.bin', 'wb')
|
||||
>>> umsgpack.pack({u"compact": True, u"schema": 0}, f)
|
||||
>>>
|
||||
"""
|
||||
global compatibility
|
||||
|
||||
ext_handlers = options.get("ext_handlers")
|
||||
|
||||
if obj is None:
|
||||
_pack_nil(obj, fp, options)
|
||||
elif ext_handlers and obj.__class__ in ext_handlers:
|
||||
_pack_ext(ext_handlers[obj.__class__](obj), fp, options)
|
||||
elif obj.__class__ in _ext_class_to_type:
|
||||
try:
|
||||
_pack_ext(Ext(_ext_class_to_type[obj.__class__], obj.packb()), fp, options)
|
||||
except AttributeError:
|
||||
raise NotImplementedError("Ext serializable class {:s} is missing implementation of packb()".format(repr(obj.__class__)))
|
||||
elif isinstance(obj, bool):
|
||||
_pack_boolean(obj, fp, options)
|
||||
elif isinstance(obj, (int, long)):
|
||||
_pack_integer(obj, fp, options)
|
||||
elif isinstance(obj, float):
|
||||
_pack_float(obj, fp, options)
|
||||
elif compatibility and isinstance(obj, unicode):
|
||||
_pack_oldspec_raw(bytes(obj), fp, options)
|
||||
elif compatibility and isinstance(obj, bytes):
|
||||
_pack_oldspec_raw(obj, fp, options)
|
||||
elif isinstance(obj, unicode):
|
||||
_pack_string(obj, fp, options)
|
||||
elif isinstance(obj, str):
|
||||
_pack_binary(obj, fp, options)
|
||||
elif isinstance(obj, (list, tuple)):
|
||||
_pack_array(obj, fp, options)
|
||||
elif isinstance(obj, dict):
|
||||
_pack_map(obj, fp, options)
|
||||
elif isinstance(obj, datetime.datetime):
|
||||
_pack_ext_timestamp(obj, fp, options)
|
||||
elif isinstance(obj, Ext):
|
||||
_pack_ext(obj, fp, options)
|
||||
elif ext_handlers:
|
||||
# Linear search for superclass
|
||||
t = next((t for t in ext_handlers.keys() if isinstance(obj, t)), None)
|
||||
if t:
|
||||
_pack_ext(ext_handlers[t](obj), fp, options)
|
||||
else:
|
||||
raise UnsupportedTypeException(
|
||||
"unsupported type: {:s}".format(str(type(obj))))
|
||||
elif _ext_class_to_type:
|
||||
# Linear search for superclass
|
||||
t = next((t for t in _ext_class_to_type if isinstance(obj, t)), None)
|
||||
if t:
|
||||
try:
|
||||
_pack_ext(Ext(_ext_class_to_type[t], obj.packb()), fp, options)
|
||||
except AttributeError:
|
||||
raise NotImplementedError("Ext serializable class {:s} is missing implementation of packb()".format(repr(t)))
|
||||
else:
|
||||
raise UnsupportedTypeException("unsupported type: {:s}".format(str(type(obj))))
|
||||
else:
|
||||
raise UnsupportedTypeException("unsupported type: {:s}".format(str(type(obj))))
|
||||
|
||||
|
||||
# Pack for Python 3, with unicode 'str' type, 'bytes' type, and no 'long' type
|
||||
def _pack3(obj, fp, **options):
|
||||
"""
|
||||
|
|
@ -627,40 +534,6 @@ def _pack3(obj, fp, **options):
|
|||
raise UnsupportedTypeException(
|
||||
"unsupported type: {:s}".format(str(type(obj))))
|
||||
|
||||
|
||||
def _packb2(obj, **options):
|
||||
"""
|
||||
Serialize a Python object into MessagePack bytes.
|
||||
|
||||
Args:
|
||||
obj: a Python object
|
||||
|
||||
Kwargs:
|
||||
ext_handlers (dict): dictionary of Ext handlers, mapping a custom type
|
||||
to a callable that packs an instance of the type
|
||||
into an Ext object
|
||||
force_float_precision (str): "single" to force packing floats as
|
||||
IEEE-754 single-precision floats,
|
||||
"double" to force packing floats as
|
||||
IEEE-754 double-precision floats.
|
||||
|
||||
Returns:
|
||||
A 'str' containing serialized MessagePack bytes.
|
||||
|
||||
Raises:
|
||||
UnsupportedType(PackException):
|
||||
Object type not supported for packing.
|
||||
|
||||
Example:
|
||||
>>> umsgpack.packb({u"compact": True, u"schema": 0})
|
||||
'\x82\xa7compact\xc3\xa6schema\x00'
|
||||
>>>
|
||||
"""
|
||||
fp = io.BytesIO()
|
||||
_pack2(obj, fp, **options)
|
||||
return fp.getvalue()
|
||||
|
||||
|
||||
def _packb3(obj, **options):
|
||||
"""
|
||||
Serialize a Python object into MessagePack bytes.
|
||||
|
|
@ -885,9 +758,9 @@ def _unpack_array(code, fp, options):
|
|||
raise Exception("logic error, not array: 0x{:02x}".format(ord(code)))
|
||||
|
||||
if options.get('use_tuple'):
|
||||
return tuple((_unpack(fp, options) for i in xrange(length)))
|
||||
return tuple((_unpack(fp, options) for i in range(length)))
|
||||
|
||||
return [_unpack(fp, options) for i in xrange(length)]
|
||||
return [_unpack(fp, options) for i in range(length)]
|
||||
|
||||
|
||||
def _deep_list_to_tuple(obj):
|
||||
|
|
@ -907,7 +780,7 @@ def _unpack_map(code, fp, options):
|
|||
raise Exception("logic error, not map: 0x{:02x}".format(ord(code)))
|
||||
|
||||
d = {} if not options.get('use_ordered_dict') else collections.OrderedDict()
|
||||
for _ in xrange(length):
|
||||
for _ in range(length):
|
||||
# Unpack key
|
||||
k = _unpack(fp, options)
|
||||
|
||||
|
|
@ -938,53 +811,6 @@ def _unpack(fp, options):
|
|||
|
||||
########################################
|
||||
|
||||
|
||||
def _unpack2(fp, **options):
|
||||
"""
|
||||
Deserialize MessagePack bytes into a Python object.
|
||||
|
||||
Args:
|
||||
fp: a .read()-supporting file-like object
|
||||
|
||||
Kwargs:
|
||||
ext_handlers (dict): dictionary of Ext handlers, mapping integer Ext
|
||||
type to a callable that unpacks an instance of
|
||||
Ext into an object
|
||||
use_ordered_dict (bool): unpack maps into OrderedDict, instead of
|
||||
unordered dict (default False)
|
||||
use_tuple (bool): unpacks arrays into tuples, instead of lists (default
|
||||
False)
|
||||
allow_invalid_utf8 (bool): unpack invalid strings into instances of
|
||||
InvalidString, for access to the bytes
|
||||
(default False)
|
||||
|
||||
Returns:
|
||||
A Python object.
|
||||
|
||||
Raises:
|
||||
InsufficientDataException(UnpackException):
|
||||
Insufficient data to unpack the serialized object.
|
||||
InvalidStringException(UnpackException):
|
||||
Invalid UTF-8 string encountered during unpacking.
|
||||
UnsupportedTimestampException(UnpackException):
|
||||
Unsupported timestamp format encountered during unpacking.
|
||||
ReservedCodeException(UnpackException):
|
||||
Reserved code encountered during unpacking.
|
||||
UnhashableKeyException(UnpackException):
|
||||
Unhashable key encountered during map unpacking.
|
||||
The serialized map cannot be deserialized into a Python dictionary.
|
||||
DuplicateKeyException(UnpackException):
|
||||
Duplicate key encountered during map unpacking.
|
||||
|
||||
Example:
|
||||
>>> f = open('test.bin', 'rb')
|
||||
>>> umsgpack.unpackb(f)
|
||||
{u'compact': True, u'schema': 0}
|
||||
>>>
|
||||
"""
|
||||
return _unpack(fp, options)
|
||||
|
||||
|
||||
def _unpack3(fp, **options):
|
||||
"""
|
||||
Deserialize MessagePack bytes into a Python object.
|
||||
|
|
@ -1149,27 +975,11 @@ def __init():
|
|||
global _utc_tzinfo
|
||||
global _float_precision
|
||||
global _unpack_dispatch_table
|
||||
global xrange
|
||||
|
||||
# Compatibility mode for handling strings/bytes with the old specification
|
||||
compatibility = False
|
||||
|
||||
if sys.version_info[0] == 3:
|
||||
_utc_tzinfo = datetime.timezone.utc
|
||||
else:
|
||||
class UTC(datetime.tzinfo):
|
||||
ZERO = datetime.timedelta(0)
|
||||
|
||||
def utcoffset(self, dt):
|
||||
return UTC.ZERO
|
||||
|
||||
def tzname(self, dt):
|
||||
return "UTC"
|
||||
|
||||
def dst(self, dt):
|
||||
return UTC.ZERO
|
||||
|
||||
_utc_tzinfo = UTC()
|
||||
_utc_tzinfo = datetime.timezone.utc
|
||||
|
||||
# Calculate an aware epoch datetime
|
||||
_epoch = datetime.datetime(1970, 1, 1, tzinfo=_utc_tzinfo)
|
||||
|
|
@ -1181,25 +991,14 @@ def __init():
|
|||
_float_precision = "single"
|
||||
|
||||
# Map packb and unpackb to the appropriate version
|
||||
if sys.version_info[0] == 3:
|
||||
pack = _pack3
|
||||
packb = _packb3
|
||||
dump = _pack3
|
||||
dumps = _packb3
|
||||
unpack = _unpack3
|
||||
unpackb = _unpackb3
|
||||
load = _unpack3
|
||||
loads = _unpackb3
|
||||
xrange = range
|
||||
else:
|
||||
pack = _pack2
|
||||
packb = _packb2
|
||||
dump = _pack2
|
||||
dumps = _packb2
|
||||
unpack = _unpack2
|
||||
unpackb = _unpackb2
|
||||
load = _unpack2
|
||||
loads = _unpackb2
|
||||
pack = _pack3
|
||||
packb = _packb3
|
||||
dump = _pack3
|
||||
dumps = _packb3
|
||||
unpack = _unpack3
|
||||
unpackb = _unpackb3
|
||||
load = _unpack3
|
||||
loads = _unpackb3
|
||||
|
||||
# Build a dispatch table for fast lookup of unpacking function
|
||||
|
||||
|
|
|
|||
537
RNS/vendor/validate.py
vendored
Normal file
537
RNS/vendor/validate.py
vendored
Normal file
|
|
@ -0,0 +1,537 @@
|
|||
# validate.py
|
||||
# -*- coding: utf-8 -*-
|
||||
# pylint: disable=
|
||||
#
|
||||
# A Validator object.
|
||||
#
|
||||
# Copyright (C) 2005-2014:
|
||||
# (name) : (email)
|
||||
# Michael Foord: fuzzyman AT voidspace DOT org DOT uk
|
||||
# Mark Andrews: mark AT la-la DOT com
|
||||
# Nicola Larosa: nico AT tekNico DOT net
|
||||
# Rob Dennis: rdennis AT gmail DOT com
|
||||
# Eli Courtwright: eli AT courtwright DOT org
|
||||
|
||||
# This software is licensed under the terms of the BSD license.
|
||||
# http://opensource.org/licenses/BSD-3-Clause
|
||||
|
||||
# ConfigObj 5 - main repository for documentation and issue tracking:
|
||||
# https://github.com/DiffSK/configobj
|
||||
|
||||
import re
|
||||
import sys
|
||||
from pprint import pprint
|
||||
|
||||
__version__ = '1.0.1'
|
||||
|
||||
__all__ = (
|
||||
'dottedQuadToNum',
|
||||
'numToDottedQuad',
|
||||
'ValidateError',
|
||||
'VdtUnknownCheckError',
|
||||
'VdtParamError',
|
||||
'VdtTypeError',
|
||||
'VdtValueError',
|
||||
'VdtValueTooSmallError',
|
||||
'VdtValueTooBigError',
|
||||
'VdtValueTooShortError',
|
||||
'VdtValueTooLongError',
|
||||
'VdtMissingValue',
|
||||
'Validator',
|
||||
'is_integer',
|
||||
'is_float',
|
||||
'is_boolean',
|
||||
'is_list',
|
||||
'is_tuple',
|
||||
'is_ip_addr',
|
||||
'is_string',
|
||||
'is_int_list',
|
||||
'is_bool_list',
|
||||
'is_float_list',
|
||||
'is_string_list',
|
||||
'is_ip_addr_list',
|
||||
'is_mixed_list',
|
||||
'is_option',
|
||||
)
|
||||
|
||||
_list_arg = re.compile(r'''
|
||||
(?:
|
||||
([a-zA-Z_][a-zA-Z0-9_]*)\s*=\s*list\(
|
||||
(
|
||||
(?:
|
||||
\s*
|
||||
(?:
|
||||
(?:".*?")| # double quotes
|
||||
(?:'.*?')| # single quotes
|
||||
(?:[^'",\s\)][^,\)]*?) # unquoted
|
||||
)
|
||||
\s*,\s*
|
||||
)*
|
||||
(?:
|
||||
(?:".*?")| # double quotes
|
||||
(?:'.*?')| # single quotes
|
||||
(?:[^'",\s\)][^,\)]*?) # unquoted
|
||||
)? # last one
|
||||
)
|
||||
\)
|
||||
)
|
||||
''', re.VERBOSE | re.DOTALL) # two groups
|
||||
|
||||
_list_members = re.compile(r'''
|
||||
(
|
||||
(?:".*?")| # double quotes
|
||||
(?:'.*?')| # single quotes
|
||||
(?:[^'",\s=][^,=]*?) # unquoted
|
||||
)
|
||||
(?:
|
||||
(?:\s*,\s*)|(?:\s*$) # comma
|
||||
)
|
||||
''', re.VERBOSE | re.DOTALL) # one group
|
||||
|
||||
_paramstring = r'''
|
||||
(?:
|
||||
(
|
||||
(?:
|
||||
[a-zA-Z_][a-zA-Z0-9_]*\s*=\s*list\(
|
||||
(?:
|
||||
\s*
|
||||
(?:
|
||||
(?:".*?")| # double quotes
|
||||
(?:'.*?')| # single quotes
|
||||
(?:[^'",\s\)][^,\)]*?) # unquoted
|
||||
)
|
||||
\s*,\s*
|
||||
)*
|
||||
(?:
|
||||
(?:".*?")| # double quotes
|
||||
(?:'.*?')| # single quotes
|
||||
(?:[^'",\s\)][^,\)]*?) # unquoted
|
||||
)? # last one
|
||||
\)
|
||||
)|
|
||||
(?:
|
||||
(?:".*?")| # double quotes
|
||||
(?:'.*?')| # single quotes
|
||||
(?:[^'",\s=][^,=]*?)| # unquoted
|
||||
(?: # keyword argument
|
||||
[a-zA-Z_][a-zA-Z0-9_]*\s*=\s*
|
||||
(?:
|
||||
(?:".*?")| # double quotes
|
||||
(?:'.*?')| # single quotes
|
||||
(?:[^'",\s=][^,=]*?) # unquoted
|
||||
)
|
||||
)
|
||||
)
|
||||
)
|
||||
(?:
|
||||
(?:\s*,\s*)|(?:\s*$) # comma
|
||||
)
|
||||
)
|
||||
'''
|
||||
|
||||
_matchstring = '^%s*' % _paramstring
|
||||
|
||||
def dottedQuadToNum(ip):
|
||||
# import here to avoid it when ip_addr values are not used
|
||||
import socket, struct
|
||||
|
||||
try:
|
||||
return struct.unpack('!L',
|
||||
socket.inet_aton(ip.strip()))[0]
|
||||
except socket.error:
|
||||
raise ValueError('Not a good dotted-quad IP: %s' % ip)
|
||||
return
|
||||
|
||||
def numToDottedQuad(num):
|
||||
# import here to avoid it when ip_addr values are not used
|
||||
import socket, struct
|
||||
|
||||
# no need to intercept here, 4294967295L is fine
|
||||
if num > int(4294967295) or num < 0:
|
||||
raise ValueError('Not a good numeric IP: %s' % num)
|
||||
try:
|
||||
return socket.inet_ntoa(
|
||||
struct.pack('!L', int(num)))
|
||||
except (socket.error, struct.error, OverflowError):
|
||||
raise ValueError('Not a good numeric IP: %s' % num)
|
||||
|
||||
class ValidateError(Exception):
|
||||
"""
|
||||
This error indicates that the check failed.
|
||||
It can be the base class for more specific errors.
|
||||
"""
|
||||
|
||||
class VdtMissingValue(ValidateError):
|
||||
"""No value was supplied to a check that needed one."""
|
||||
|
||||
class VdtUnknownCheckError(ValidateError):
|
||||
def __init__(self, value):
|
||||
ValidateError.__init__(self, 'the check "{}" is unknown.'.format(value))
|
||||
|
||||
|
||||
class VdtParamError(SyntaxError):
|
||||
NOT_GIVEN = object()
|
||||
|
||||
def __init__(self, name_or_msg, value=NOT_GIVEN):
|
||||
if value is self.NOT_GIVEN:
|
||||
SyntaxError.__init__(self, name_or_msg)
|
||||
else:
|
||||
SyntaxError.__init__(self, 'passed an incorrect value "{}" for parameter "{}".'.format(value, name_or_msg))
|
||||
|
||||
|
||||
class VdtTypeError(ValidateError):
|
||||
def __init__(self, value):
|
||||
ValidateError.__init__(self, 'the value "{}" is of the wrong type.'.format(value))
|
||||
|
||||
|
||||
class VdtValueError(ValidateError):
|
||||
def __init__(self, value):
|
||||
ValidateError.__init__(self, 'the value "{}" is unacceptable.'.format(value))
|
||||
|
||||
|
||||
class VdtValueTooSmallError(VdtValueError):
|
||||
def __init__(self, value):
|
||||
ValidateError.__init__(self, 'the value "{}" is too small.'.format(value))
|
||||
|
||||
|
||||
class VdtValueTooBigError(VdtValueError):
|
||||
def __init__(self, value):
|
||||
ValidateError.__init__(self, 'the value "{}" is too big.'.format(value))
|
||||
|
||||
|
||||
class VdtValueTooShortError(VdtValueError):
|
||||
def __init__(self, value):
|
||||
ValidateError.__init__(
|
||||
self,
|
||||
'the value "{}" is too short.'.format(value))
|
||||
|
||||
class VdtValueTooLongError(VdtValueError):
|
||||
def __init__(self, value):
|
||||
ValidateError.__init__(self, 'the value "{}" is too long.'.format(value))
|
||||
|
||||
class Validator(object):
|
||||
# this regex does the initial parsing of the checks
|
||||
_func_re = re.compile(r'([^\(\)]+?)\((.*)\)', re.DOTALL)
|
||||
|
||||
# this regex takes apart keyword arguments
|
||||
_key_arg = re.compile(r'^([a-zA-Z_][a-zA-Z0-9_]*)\s*=\s*(.*)$', re.DOTALL)
|
||||
|
||||
|
||||
# this regex finds keyword=list(....) type values
|
||||
_list_arg = _list_arg
|
||||
|
||||
# this regex takes individual values out of lists - in one pass
|
||||
_list_members = _list_members
|
||||
|
||||
# These regexes check a set of arguments for validity
|
||||
# and then pull the members out
|
||||
_paramfinder = re.compile(_paramstring, re.VERBOSE | re.DOTALL)
|
||||
_matchfinder = re.compile(_matchstring, re.VERBOSE | re.DOTALL)
|
||||
|
||||
def __init__(self, functions=None):
|
||||
self.functions = {
|
||||
'': self._pass,
|
||||
'integer': is_integer,
|
||||
'float': is_float,
|
||||
'boolean': is_boolean,
|
||||
'ip_addr': is_ip_addr,
|
||||
'string': is_string,
|
||||
'list': is_list,
|
||||
'tuple': is_tuple,
|
||||
'int_list': is_int_list,
|
||||
'float_list': is_float_list,
|
||||
'bool_list': is_bool_list,
|
||||
'ip_addr_list': is_ip_addr_list,
|
||||
'string_list': is_string_list,
|
||||
'mixed_list': is_mixed_list,
|
||||
'pass': self._pass,
|
||||
'option': is_option,
|
||||
'force_list': force_list,
|
||||
}
|
||||
if functions is not None:
|
||||
self.functions.update(functions)
|
||||
# tekNico: for use by ConfigObj
|
||||
self.baseErrorClass = ValidateError
|
||||
self._cache = {}
|
||||
|
||||
def check(self, check, value, missing=False):
|
||||
fun_name, fun_args, fun_kwargs, default = self._parse_with_caching(check)
|
||||
|
||||
if missing:
|
||||
if default is None:
|
||||
# no information needed here - to be handled by caller
|
||||
raise VdtMissingValue()
|
||||
value = self._handle_none(default)
|
||||
|
||||
if value is None:
|
||||
return None
|
||||
|
||||
return self._check_value(value, fun_name, fun_args, fun_kwargs)
|
||||
|
||||
def _handle_none(self, value):
|
||||
if value == 'None':
|
||||
return None
|
||||
elif value in ("'None'", '"None"'):
|
||||
# Special case a quoted None
|
||||
value = self._unquote(value)
|
||||
return value
|
||||
|
||||
def _parse_with_caching(self, check):
|
||||
if check in self._cache:
|
||||
fun_name, fun_args, fun_kwargs, default = self._cache[check]
|
||||
# We call list and dict below to work with *copies* of the data
|
||||
# rather than the original (which are mutable of course)
|
||||
fun_args = list(fun_args)
|
||||
fun_kwargs = dict(fun_kwargs)
|
||||
else:
|
||||
fun_name, fun_args, fun_kwargs, default = self._parse_check(check)
|
||||
fun_kwargs = {str(key): value for (key, value) in list(fun_kwargs.items())}
|
||||
self._cache[check] = fun_name, list(fun_args), dict(fun_kwargs), default
|
||||
return fun_name, fun_args, fun_kwargs, default
|
||||
|
||||
def _check_value(self, value, fun_name, fun_args, fun_kwargs):
|
||||
try:
|
||||
fun = self.functions[fun_name]
|
||||
except KeyError:
|
||||
raise VdtUnknownCheckError(fun_name)
|
||||
else:
|
||||
return fun(value, *fun_args, **fun_kwargs)
|
||||
|
||||
def _parse_check(self, check):
|
||||
fun_match = self._func_re.match(check)
|
||||
if fun_match:
|
||||
fun_name = fun_match.group(1)
|
||||
arg_string = fun_match.group(2)
|
||||
arg_match = self._matchfinder.match(arg_string)
|
||||
if arg_match is None:
|
||||
# Bad syntax
|
||||
raise VdtParamError('Bad syntax in check "%s".' % check)
|
||||
fun_args = []
|
||||
fun_kwargs = {}
|
||||
# pull out args of group 2
|
||||
for arg in self._paramfinder.findall(arg_string):
|
||||
# args may need whitespace removing (before removing quotes)
|
||||
arg = arg.strip()
|
||||
listmatch = self._list_arg.match(arg)
|
||||
if listmatch:
|
||||
key, val = self._list_handle(listmatch)
|
||||
fun_kwargs[key] = val
|
||||
continue
|
||||
keymatch = self._key_arg.match(arg)
|
||||
if keymatch:
|
||||
val = keymatch.group(2)
|
||||
if not val in ("'None'", '"None"'):
|
||||
# Special case a quoted None
|
||||
val = self._unquote(val)
|
||||
fun_kwargs[keymatch.group(1)] = val
|
||||
continue
|
||||
|
||||
fun_args.append(self._unquote(arg))
|
||||
else:
|
||||
# allows for function names without (args)
|
||||
return check, (), {}, None
|
||||
|
||||
# Default must be deleted if the value is specified too,
|
||||
# otherwise the check function will get a spurious "default" keyword arg
|
||||
default = fun_kwargs.pop('default', None)
|
||||
return fun_name, fun_args, fun_kwargs, default
|
||||
|
||||
def _unquote(self, val):
|
||||
if (len(val) >= 2) and (val[0] in ("'", '"')) and (val[0] == val[-1]):
|
||||
val = val[1:-1]
|
||||
return val
|
||||
|
||||
def _list_handle(self, listmatch):
|
||||
out = []
|
||||
name = listmatch.group(1)
|
||||
args = listmatch.group(2)
|
||||
for arg in self._list_members.findall(args):
|
||||
out.append(self._unquote(arg))
|
||||
return name, out
|
||||
|
||||
def _pass(self, value):
|
||||
return value
|
||||
|
||||
def get_default_value(self, check):
|
||||
fun_name, fun_args, fun_kwargs, default = self._parse_with_caching(check)
|
||||
if default is None:
|
||||
raise KeyError('Check "%s" has no default value.' % check)
|
||||
value = self._handle_none(default)
|
||||
if value is None:
|
||||
return value
|
||||
return self._check_value(value, fun_name, fun_args, fun_kwargs)
|
||||
|
||||
def _is_num_param(names, values, to_float=False):
|
||||
fun = to_float and float or int
|
||||
out_params = []
|
||||
for (name, val) in zip(names, values):
|
||||
if val is None:
|
||||
out_params.append(val)
|
||||
elif isinstance(val, (int, float, str)):
|
||||
try:
|
||||
out_params.append(fun(val))
|
||||
except ValueError:
|
||||
raise VdtParamError(name, val)
|
||||
else:
|
||||
raise VdtParamError(name, val)
|
||||
return out_params
|
||||
|
||||
# built in checks
|
||||
# you can override these by setting the appropriate name
|
||||
# in Validator.functions
|
||||
# note: if the params are specified wrongly in your input string,
|
||||
# you will also raise errors.
|
||||
def is_integer(value, min=None, max=None):
|
||||
(min_val, max_val) = _is_num_param( # pylint: disable=unbalanced-tuple-unpacking
|
||||
('min', 'max'), (min, max))
|
||||
if not isinstance(value, (int, str)):
|
||||
raise VdtTypeError(value)
|
||||
if isinstance(value, str):
|
||||
# if it's a string - does it represent an integer ?
|
||||
try:
|
||||
value = int(value)
|
||||
except ValueError:
|
||||
raise VdtTypeError(value)
|
||||
if (min_val is not None) and (value < min_val):
|
||||
raise VdtValueTooSmallError(value)
|
||||
if (max_val is not None) and (value > max_val):
|
||||
raise VdtValueTooBigError(value)
|
||||
return value
|
||||
|
||||
|
||||
def is_float(value, min=None, max=None):
|
||||
(min_val, max_val) = _is_num_param(
|
||||
('min', 'max'), (min, max), to_float=True)
|
||||
if not isinstance(value, (int, float, str)):
|
||||
raise VdtTypeError(value)
|
||||
if not isinstance(value, float):
|
||||
# if it's a string - does it represent a float ?
|
||||
try:
|
||||
value = float(value)
|
||||
except ValueError:
|
||||
raise VdtTypeError(value)
|
||||
if (min_val is not None) and (value < min_val):
|
||||
raise VdtValueTooSmallError(value)
|
||||
if (max_val is not None) and (value > max_val):
|
||||
raise VdtValueTooBigError(value)
|
||||
return value
|
||||
|
||||
bool_dict = {
|
||||
True: True, 'on': True, '1': True, 'true': True, 'yes': True,
|
||||
False: False, 'off': False, '0': False, 'false': False, 'no': False,
|
||||
}
|
||||
|
||||
def is_boolean(value):
|
||||
if isinstance(value, str):
|
||||
try:
|
||||
return bool_dict[value.lower()]
|
||||
except KeyError:
|
||||
raise VdtTypeError(value)
|
||||
# we do an equality test rather than an identity test
|
||||
# this ensures Python 2.2 compatibility
|
||||
# and allows 0 and 1 to represent True and False
|
||||
if value == False:
|
||||
return False
|
||||
elif value == True:
|
||||
return True
|
||||
else:
|
||||
raise VdtTypeError(value)
|
||||
|
||||
|
||||
def is_ip_addr(value):
|
||||
if not isinstance(value, str):
|
||||
raise VdtTypeError(value)
|
||||
value = value.strip()
|
||||
try:
|
||||
dottedQuadToNum(value)
|
||||
except ValueError:
|
||||
raise VdtValueError(value)
|
||||
return value
|
||||
|
||||
|
||||
def is_list(value, min=None, max=None):
|
||||
(min_len, max_len) = _is_num_param( # pylint: disable=unbalanced-tuple-unpacking
|
||||
('min', 'max'), (min, max))
|
||||
if isinstance(value, str):
|
||||
raise VdtTypeError(value)
|
||||
try:
|
||||
num_members = len(value)
|
||||
except TypeError:
|
||||
raise VdtTypeError(value)
|
||||
if min_len is not None and num_members < min_len:
|
||||
raise VdtValueTooShortError(value)
|
||||
if max_len is not None and num_members > max_len:
|
||||
raise VdtValueTooLongError(value)
|
||||
return list(value)
|
||||
|
||||
|
||||
def is_tuple(value, min=None, max=None):
|
||||
return tuple(is_list(value, min, max))
|
||||
|
||||
def is_string(value, min=None, max=None):
|
||||
if not isinstance(value, str):
|
||||
raise VdtTypeError(value)
|
||||
(min_len, max_len) = _is_num_param(
|
||||
('min', 'max'), (min, max))
|
||||
try:
|
||||
num_members = len(value)
|
||||
except TypeError:
|
||||
raise VdtTypeError(value)
|
||||
if min_len is not None and num_members < min_len:
|
||||
raise VdtValueTooShortError(value)
|
||||
if max_len is not None and num_members > max_len:
|
||||
raise VdtValueTooLongError(value)
|
||||
return value
|
||||
|
||||
|
||||
def is_int_list(value, min=None, max=None):
|
||||
return [is_integer(mem) for mem in is_list(value, min, max)]
|
||||
|
||||
def is_bool_list(value, min=None, max=None):
|
||||
return [is_boolean(mem) for mem in is_list(value, min, max)]
|
||||
|
||||
def is_float_list(value, min=None, max=None):
|
||||
return [is_float(mem) for mem in is_list(value, min, max)]
|
||||
|
||||
def is_string_list(value, min=None, max=None):
|
||||
if isinstance(value, str):
|
||||
raise VdtTypeError(value)
|
||||
return [is_string(mem) for mem in is_list(value, min, max)]
|
||||
|
||||
def is_ip_addr_list(value, min=None, max=None):
|
||||
return [is_ip_addr(mem) for mem in is_list(value, min, max)]
|
||||
|
||||
def force_list(value, min=None, max=None):
|
||||
if not isinstance(value, (list, tuple)):
|
||||
value = [value]
|
||||
return is_list(value, min, max)
|
||||
|
||||
fun_dict = {
|
||||
int: is_integer,
|
||||
'int': is_integer,
|
||||
'integer': is_integer,
|
||||
float: is_float,
|
||||
'float': is_float,
|
||||
'ip_addr': is_ip_addr,
|
||||
str: is_string,
|
||||
'str': is_string,
|
||||
'string': is_string,
|
||||
bool: is_boolean,
|
||||
'bool': is_boolean,
|
||||
'boolean': is_boolean,
|
||||
}
|
||||
|
||||
def is_mixed_list(value, *args):
|
||||
try: length = len(value)
|
||||
except TypeError: raise VdtTypeError(value)
|
||||
if length < len(args): raise VdtValueTooShortError(value)
|
||||
elif length > len(args): raise VdtValueTooLongError(value)
|
||||
try: return [fun_dict[arg](val) for arg, val in zip(args, value)]
|
||||
except KeyError as cause: raise VdtParamError('mixed_list', cause)
|
||||
|
||||
|
||||
def is_option(value, *options):
|
||||
if not isinstance(value, str): raise VdtTypeError(value)
|
||||
if not value in options: raise VdtValueError(value)
|
||||
return value
|
||||
|
||||
94
Roadmap.md
94
Roadmap.md
|
|
@ -1,94 +0,0 @@
|
|||
# Reticulum Development Roadmap
|
||||
This document outlines the currently established development roadmap for Reticulum.
|
||||
|
||||
1. [Currently Active Work Areas](#currently-active-work-areas)
|
||||
2. [Primary Efforts](#primary-efforts)
|
||||
- [Comprehensibility](#comprehensibility)
|
||||
- [Universality](#universality)
|
||||
- [Functionality](#functionality)
|
||||
- [Usability & Utility](#usability--utility)
|
||||
- [Interfaceability](#interfaceability)
|
||||
3. [Auxillary Efforts](#auxillary-efforts)
|
||||
4. [Release History](#release-history)
|
||||
|
||||
## Currently Active Work Areas
|
||||
For each release cycle of Reticulum, improvements and additions from the five [Primary Efforts](#primary-efforts) are selected as active work areas, and can be expected to be included in the upcoming releases within that cycle. While not entirely set in stone for each release cycle, they serve as a pointer of what to expect in the near future.
|
||||
|
||||
## Primary Efforts
|
||||
The development path for Reticulum is currently laid out in five distinct areas: *Comprehensibility*, *Universality*, *Functionality*, *Usability & Utility* and *Interfaceability*. Conceptualising the development of Reticulum into these areas serves to advance the implementation and work towards the Foundational Goals & Values of Reticulum.
|
||||
|
||||
### Comprehensibility
|
||||
These efforts are aimed at improving the ease of which Reticulum is understood, and lowering the barrier to entry for people who wish to start building systems on Reticulum.
|
||||
|
||||
- Improving [the manual](https://markqvist.github.io/Reticulum/manual/) with tutorials specifically for beginners
|
||||
- Updating the documentation to reflect recent changes and improvements
|
||||
- Update descriptions of protocol mechanics
|
||||
- Update announce description
|
||||
- Add in-depth explanation of the IFAC system
|
||||
- Software
|
||||
- Update software descriptions and screenshots
|
||||
- Communications hardware section
|
||||
- Add information about RNode external displays.
|
||||
- Possibly add other relevant types here as well.
|
||||
- Setup *Best Practices For...* / *Installation Examples* section.
|
||||
- Home or office (example)
|
||||
- Vehicles (example)
|
||||
- No-grid/solar/remote sites (example)
|
||||
|
||||
### Universality
|
||||
These efforts seek to broaden the universality of the Reticulum software and hardware ecosystem by continously diversifying platform support, and by improving the overall availability and ease of deployment of the Reticulum stack.
|
||||
|
||||
- Create a standalone RNS Daemon app for Android
|
||||
- A lightweight and portable C implementation for microcontrollers, µRNS
|
||||
- A portable, high-performance Reticulum implementation in C/C++, see [#21](https://github.com/markqvist/Reticulum/discussions/21)
|
||||
- Bindings for other programming languages
|
||||
|
||||
### Functionality
|
||||
These efforts aim to expand and improve the core functionality and reliability of Reticulum.
|
||||
|
||||
- Add interface hot-plug and live up/down control to running instances
|
||||
- Add automatic retries to all use cases of the `Request` API
|
||||
- Network-wide path balancing
|
||||
- Distributed Destination Naming System
|
||||
- Globally routable multicast
|
||||
- Destination proxying
|
||||
- [Metric-based path selection and multiple paths](https://github.com/markqvist/Reticulum/discussions/86)
|
||||
|
||||
### Usability & Utility
|
||||
These efforts seek to make Reticulum easier to use and operate, and to expand the utility of the stack on deployed systems.
|
||||
|
||||
- Easy way to share interface configurations, see [#19](https://github.com/markqvist/Reticulum/discussions/19)
|
||||
- Transit traffic display in `rnstatus`
|
||||
- `rnsconfig` utility
|
||||
|
||||
### Interfaceability
|
||||
These efforts aim to expand the types of physical and virtual interfaces that Reticulum can natively use to transport data.
|
||||
|
||||
- Plain ESP32 devices (ESP-Now, WiFi, Bluetooth, etc.)
|
||||
- More LoRa transceivers
|
||||
- AT-compatible modems
|
||||
- Filesystem interface
|
||||
- Direct SDR Support
|
||||
- Optical mediums
|
||||
- IR Transceivers
|
||||
- AWDL / OWL
|
||||
- HF Modems
|
||||
- GNU Radio
|
||||
- CAN-bus
|
||||
- Raw SPI
|
||||
- Raw i²c
|
||||
- MQTT
|
||||
- XBee
|
||||
- Tor
|
||||
|
||||
## Auxillary Efforts
|
||||
The Reticulum ecosystem is enriched by several other software and hardware projects, and the support and improvement of these, in symbiosis with the core Reticulum project helps expand the reach and utility of Reticulum itself.
|
||||
|
||||
This section lists, in no particular order, various important efforts that would be beneficial to the goals of Reticulum.
|
||||
|
||||
- The [RNode](https://unsigned.io/rnode/) project
|
||||
- [x] Create a WebUSB-based bootstrapping utility, and integrate this directly into the [RNode Bootstrap Console](#), both on-device, and on an Internet-reachable copy. This will make it much easier to create new RNodes for average users.
|
||||
|
||||
## Release History
|
||||
|
||||
Please see the [Changelog](./Changelog.md) for a complete release history and changelog of Reticulum.
|
||||
7
SECURITY.md
Normal file
7
SECURITY.md
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
# Reporting Security Issues
|
||||
|
||||
Security notices that affect Reticulum are published [in-repository in the docs/security directory](./docs/security).
|
||||
|
||||
It goes without saying: If you have discovered a security issue in Reticulum or any related project, please report it responsibly to the maintainer over LXMF at `<8dd57a738226809646089335a6b03695>`. If you wish to remain anonymous, please create a disposable identity to send from, and notify us of any such wishes.
|
||||
|
||||
If you are not able to use LXMF, you can send an email to the address currently posted on the maintainer's profile on this site.
|
||||
68
This Is Not a Teahouse.md
Normal file
68
This Is Not a Teahouse.md
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
Recently, and mostly from people who I've never seen before, the opinions about how this project should be run has started flooding in again. In a recent forum thread of such opinions, specifically about:
|
||||
|
||||
- The decision to no longer mirror release notes on GitHub.
|
||||
- Some people feeling there were too many "barriers to entry" to joining RNS development.
|
||||
- The project not really being "[open source](https://reticulum.network/manual/brandolinis.html#open-source-means-open-source)" because random strangers couldn't just "contribute".
|
||||
|
||||
Joakim posted some very relevant observations about how Reticulum operates, along with the following quote:
|
||||
|
||||
> The modern industrial system has a built-in tendency to grow; it cannot really work unless it is growing. The word “stability” has been struck from its dictionary and replaced by “stagnation”. Its continuous growth pursues no particular aims or objectives: it is growth for the sake of growing. No one even enquires after its final shape. There is none; there is no “saturation point”.
|
||||
|
||||
That E. F. Schumacher quote perfectly illustrates the ontological schism that makes it so tiresome to deal with stuff like this.
|
||||
|
||||
There is, in this day and age, between different people, widely different base conceptual integrations of what "open source" means. For many people, "open source" has become synonymous **not** with skilled people working together in a coordinated and careful way on complex engineering challenges, but a sort of growth- and attention-focused "free-for-all" *behavioral* codex that must be followed above all else; a *social* modus operandi of fake inclusivity where everyone "should have their voice heard", and adherence to that specific *process* is weighed much higher than the final results.
|
||||
|
||||
I do not subscribe to, and consequently do not operate the Reticulum project under *any* versions of that idea.
|
||||
|
||||
**Here's the statistical, boring reality:**
|
||||
|
||||
- Around 90% of pull requests and "recommendations" I received when people could just submit stuff via GitHub would
|
||||
have *severely* broken things, introduced bugs or security issues, created roadblocks for future work, or otherwise
|
||||
damaged the software. Usually just for the sake of satisfying a random newcomers "idea" or personal preference.
|
||||
|
||||
- Similarly, around 90% "bug reports" were actually people asking for help, because of having failed to read even the
|
||||
most basic parts of the documentation.
|
||||
|
||||
- The people with the least amount of understanding, skill and effort invested tend to be loudest and most vocal. When
|
||||
all you have is "opinions", those are iterated upon ad infinitum, apparently.
|
||||
|
||||
Can you imagine how much time that wasted? Can you imagine what we could have accomplished with that time instead?
|
||||
|
||||
The only thing that this creates is *noise* and confusion. Clogging up the mental and physical workspaces, of people who are actually investing time and effort on the project, with stuff like that is objectively just taking time that could have been used on development, and replacing it with *nothing*.
|
||||
|
||||
I was receiving *actual* bug reports, pull requests, proper technical investigations and patches via methods outside GitHub and "public" internet-based channels *way* before GitHub interaction and similar was closed down. That was were almost *all* of the real contributions were coming from, anyway. Apparently, and not unsurprisingly, the people who has invested the time and effort to understand Reticulum also prefer to collaborate in this way. Since leaving the GitHub madhouse behind, the signal-to-noise ratio has **significantly** increased.
|
||||
|
||||
Managing a public "issue" tracker with global read/write access is a futile and useless endeavor. Consider this:
|
||||
|
||||
- User A reports a "bug" that is really just a failure of understanding.
|
||||
- User B sees this and seconds is, proposing a "fix" that in continued failure of understanding would actually break functionality X.
|
||||
- User C joins the bandwagon and asks why this hasn't already been implemented like that? It's obvious!
|
||||
|
||||
The sensible response here from the developer is closing the issue with "No. Go RTFM". Today, though, this usually results in hurt feelings, animosity towards the developer and in some cases (as experienced and documented in the case of RNS), months of perfidious personal vendetta against the developer for being so brazen as to suggest the user was wrong and wasting people's time.
|
||||
|
||||
When this pattern repeats, over and over, the only sensible, measured and constructive course of action is to shrug your shoulders and say:
|
||||
|
||||
*"This system is fundamentally broken. It ain't working. I can give up here, or I can go build something better that has a chance of working."*
|
||||
|
||||
So, now it's your turn. Go look at the diffs for the last six months. What does it look like I have been doing?
|
||||
|
||||
But I will be damned straight with you all, and say that part of that solution is **absolutely** to erect barriers to entry. You can fucking bet your arse on that. I don't want opinionated man-babies running around in my living-room at 3am. I don't want to clean up the floor after a wannabe "dev-ops star" with LLMs and a peripheral case of influencitis has puked all over the office.
|
||||
|
||||
- If you want to join the fun of changing core networking code that thousands of people rely on for communication
|
||||
daily, you better know what the fuck you're doing.
|
||||
|
||||
- I'm not here to provide validation and hugs to random strangers. I'm here to make sure the reference implementation
|
||||
of Reticulum works.
|
||||
|
||||
- If you cannot figure out how to submit a patch or valid bug investigation over RNS, you cannot expect I will take
|
||||
you seriously. At all.
|
||||
|
||||
If someone can't handle that, they should find their entertainment elsewhere.
|
||||
|
||||
I've said it before: I've provided the information and code required to make Reticulum *work*, and build networked systems, protocols and applications on top of it. That information is deep, complex, and requires you to read hundreds of pages, and put in weeks of efforts to get the *full* picture. A lot less is required to get started, but it *will* still be a steep learning experience.
|
||||
|
||||
This is a full networking stack, based on some pretty complex principles, for crying out loud. It's **not** a `hello_world` designed to make you feel good about yourself. It turns almost everything you know about networked systems on its head. That's **challenging** for *anyone*. Climb the mountain, and it will be satisfying in the end. Refuse to climb... Well, what do you think will happen?
|
||||
|
||||
As for barriers to entry of *using* RNS and related programs, utilities and clients, it's not my task to teach every single user how to do X, Y and Z. The information *is* out there. If it wasn't organized optimally for your way of learning, you can choose to "raise your concerns" about it, discuss "the fact of it" on a forum or chatroom, or: *You can choose to remedy that, and help others along*.
|
||||
|
||||
I sure know what *I* would have done.
|
||||
415
Zen of Reticulum.md
Normal file
415
Zen of Reticulum.md
Normal file
|
|
@ -0,0 +1,415 @@
|
|||
# Zen of Reticulum
|
||||
|
||||
## I: The Illusion Of The Center
|
||||
|
||||
For the better part of a generation, we have been taught to visualize the digital world through the lens of hierarchy. The mental maps we carry are dominated by a single, misleading image: **The Cloud**.
|
||||
|
||||
We imagine the network as a vast, ethereal space "up there" or "out there". A centralized repository of services and data to which we, the lowly clients, must connect. We build our software with this assumption hardcoded into our logic: *There is a server. The server has the authority. The server knows the way. I must find the server to function*.
|
||||
|
||||
This is the Client-Server mental model, and it is the primary obstacle to understanding Reticulum.
|
||||
|
||||
### Fallacy Of The Cloud
|
||||
|
||||
The first step in the Zen of Reticulum is to realize that *there is no cloud*. There is only other people's computers. When you build for the cloud, you are building *for* a landlord. You are accepting that your application's existence is conditional on the permission, uptime, and continued goodwill of a central authority.
|
||||
|
||||
In Reticulum, you must shift your thinking from "connecting to" to "being among". Reticulum is not a service you subscribe to - *it is a fabric you inhabit*. There is no "up there". There is only *here* and *there*, and the space between them is peer-to-peer.
|
||||
|
||||
### Decentralization Or Uncentralizability?
|
||||
|
||||
It is common to hear the word "decentralized" thrown around in modern tech circles. But often, this is merely a marketing term for "slightly distributed centralization". A blockchain with a few dominant miners, or a federated protocol with a few giant servers. *In practice*, it's still centralized. It simply has a few centers instead of one.
|
||||
|
||||
Reticulum goes further. It wants **Uncentralizability**.
|
||||
|
||||
This is not a wishful political stance, but a foundational mathematical characteristic of the protocol, onto which everything else has been built. Reticulum assumes that every peer on the network is potentially hostile, and every link is potentially compromised. It is designed with no "privileged" nodes. While some nodes may act as Transport Instances - forwarding traffic for others - they do so *blindly*, and they only know about their immediate surroundings, and nothing more. They route based on cryptographic proofs, not on administrative privilege. They cannot see who is talking to whom, nor can they selectively manipulate traffic without breaking their own ability to route entirely.
|
||||
|
||||
The system is designed to make hierarchy structurally impossible. You cannot hijack an address, because there is no central registry to hijack. You cannot block a user, because there is no central switch to flip. You can offer paths through the network, but you can't force anyone to use them.
|
||||
|
||||
### Death To The Address
|
||||
|
||||
To break free of the center, you must also let go of the concept of the "Address".
|
||||
|
||||
In the IP world, an address is a location. It is a coordinate in a *deeply hierarchical* and static grid. If you move your computer to a different house, your address changes. If your router reboots, your address might change. Your *identity* is bound to your *location*, and therefore, it is fragile, and easily controlled.
|
||||
|
||||
Reticulum abolishes this link between *Identity* and *Location*.
|
||||
|
||||
In Reticulum, an address is not a place; it is a **Hash of an Identity**. It is a cryptographic representation of *who* you are, not *where* you are. Because of this, your address is portable. You can take a laptop from a WiFi cafe in Berlin, to a LoRa mesh in the mountains, to a packet radio link on a boat, and your "address" - your *Destination Hash* - never changes.
|
||||
|
||||
The network does not route to a place; it routes to a *person* (or a machine). When you send a packet, you are not targeting a coordinate in a grid; you are encrypting a message for a specific entity. The network dynamically discovers where that entity currently resides, and it does so in a way where no one really knows where that entity is actually located physically.
|
||||
|
||||
**Consider:**
|
||||
|
||||
- **The Old Way:** *"I am at `192.168.1.5`. Come find me"*.
|
||||
- **The Zen Way:** *"I am `<327c1b2f87c9353e01769b01090b18f2>`. Wherever I am, my peers can reach me"*.
|
||||
|
||||
Once you stop thinking about servers and start thinking about portable identities, where everyone can always reach everyone else directly, the illusion of the center fades away. You realize there *is* no center holding the network together. No coordinators or bureaucrats required. The network is simply the sum of its peers, communicating directly, sovereignly, and without a master.
|
||||
|
||||
|
||||
## II: Physics Of Trust
|
||||
*Paranoia Is A Great Design Principle*
|
||||
|
||||
If we accept that there is no center - that the network is a chaotic, peer-to-peer mesh - we are forced to confront a terrifying reality: **There is no one guarding the door**.
|
||||
|
||||
In the traditional networking mindset, we rely on the concept of the "trusted core". We assume our local coffee shop WiFi is safe, or that the backbone providers are neutral custodians. We build our security like a castle: strong walls on the outside, soft and trusting on the inside. We use encryption only when we step out into the "wild" internet.
|
||||
|
||||
### Hostile Environments
|
||||
|
||||
The Zen of Reticulum requires you to invert this. You must assume that *every* environment is hostile. This isn't cynicism, just uncaring physics.
|
||||
|
||||
When you transmit information over radio waves, you are shouting into a crowded room. Anyone can listen. When you traverse the internet, your packets pass through routers controlled by strangers, corporations, and state actors. Assuming privacy in this environment without cryptographic protection is not optimism but gross negligence.
|
||||
|
||||
Reticulum is built on the premise that every link is tapped, and every peer is a potential adversary. If your system cannot survive an adversary owning the physical layer, it cannot survive at all.
|
||||
|
||||
But this is the paradox: By assuming the network is hostile, you make it safe. When you accept the dangers for what they are, they become manageable. When you stop trusting the infrastructure and start trusting the math, you eliminate the single point of failure: Human integrity.
|
||||
|
||||
### Encryption Is Not A Feature
|
||||
|
||||
In the world of TCP/IP, encryption is an afterthought. It is a layer we slap on top of the protocol (HTTPS, TLS) to patch the security holes of the original design. It is a "feature" you sometimes *enable* for "sensitive data". This is fundamentally flawed, since all data is sensitive.
|
||||
|
||||
In Reticulum, encryption is **gravity**.
|
||||
|
||||
It is not optional. It is not a plugin. It is the *fundamental force that allows the network to exist*. If you were to strip the encryption from Reticulum, the routing would break. The Transport system uses cryptographic signatures and entropy to verify paths and pass information. If packets were plaintext, intermediate nodes could not prove that a route was valid, nor could endpoints prevent spoofing or tampering.
|
||||
|
||||
In Reticulum, the entropy of the encrypted packet *is* the routing logic.
|
||||
|
||||
To ask for a version of Reticulum without encryption is like asking for a version of the ocean without liquid. You are not asking for a feature change; you're asking for a different physical universe. We design for a universe where information has mass, structure, and integrity.
|
||||
|
||||
### Zero-Trust Architectures
|
||||
|
||||
We must unlearn our reliance on **Institutional Trust**.
|
||||
|
||||
For decades, we have been trained to trust authorities. We trust a website because a chain of Certificate Authorities (companies we don't know) vouches for it. We trust an app because it is in an app store (run by a corporation we don't control). We trust a message because it comes from a phone number assigned by a telecom. Yet, everything in our digital information sphere today is more untrustworthy and risky than a medieval second-hand underwear market.
|
||||
|
||||
Reticulum replaces institutional trust with **Cryptographic Proof**.
|
||||
|
||||
In Reticulum, you do not trust a node because it has a nice hostname or because it is listed in a directory. You trust it because it holds the private key corresponding to the Destination Hash you are communicating with. This trust is binary, mathematical, and **absolute**. Either the signature matches, or it does not. There is no "maybe".
|
||||
|
||||
This shift moves the power from the institution to the individual. You become the ultimate arbiter of your own trust relationships. You decide which keys to accept, which paths to follow, and which identities to recognize.
|
||||
|
||||
**Consider:**
|
||||
|
||||
- **The Old Way:** *"I trust this site because the browser says the lock icon is green"*.
|
||||
- **The Zen Way:** *"I trust this destination because I have verified its hash fingerprint out-of-band, and the math confirms the signature"*.
|
||||
|
||||
When you internalize the Physics of Trust, you stop looking for protection from firewalls, VPNs, and Terms of Service agreements. You realize that true security comes from the design of the protocol itself. You can stop trusting the cloud, and you start trusting the code - because you can verify it yourself.
|
||||
|
||||
|
||||
## III: Merits Of Scarcity
|
||||
*Every Bit Counts*
|
||||
|
||||
We have grown addicted to abundance. In the modern digital ecosystem, bandwidth is treated as an endless, flat ocean. We stream high-definition video without a thought, we ship entire libraries of code just to render a single button, and we measure performance in gigabits per second. This abundance has hollowed out our craft. When constraints vanish, efficiency dies, and with it, a certain kind of Clarity and Quality.
|
||||
|
||||
Reticulum asks you to step out of the ocean and onto the tightrope.
|
||||
|
||||
### The Bandwidth Fallacy
|
||||
|
||||
The Zen of Reticulum requires the realization that **5 bits per second is a valid speed**.
|
||||
|
||||
To a modern developer, this sounds like paralysis. But there is a profound freedom in limits: When you have a gigabit connection, you can be incredibly sloppy. You can be wasteful. You can push your problems onto the infrastructure. *"It’s slow? Get a faster router"*.
|
||||
|
||||
But on a high-latency, low-bandwidth link (be it a noisy HF radio channel or a tenuous LoRa hop) you cannot push problems anywhere. You must solve them. The network does not negotiate with waste.
|
||||
|
||||
This forces a shift from consumption to interaction. You are no longer, then, consuming a service provided by a fat pipe; you are engaging in a careful negotiation with the physical medium. The medium becomes a partner in the conversation, not just a dumb conduit. You suddenly need to *understand the world to be in it*.
|
||||
|
||||
### Cost Of A Byte
|
||||
|
||||
In a scarce economy, a byte is not just data, but energy, time, and space.
|
||||
|
||||
Every byte you transmit consumes battery life on a solar-powered node. It occupies valuable airtime that could have been used by another peer. It represents a measurable slice of the electromagnetic spectrum.
|
||||
|
||||
When you internalize this, you begin to write code differently. You stop asking, "How much data can I send?" and start asking, "What is the *minimum* amount of information required to convey this intent? How can I best utilize my informational entropy?"
|
||||
|
||||
This is where the elegance of Reticulum shines. The protocol is designed to strip away the non-essential. A link establishment takes three very small packets. A destination hash fits in 16 bytes. The overhead is vanishingly small, leaving almost the entire channel for the message itself.
|
||||
|
||||
**Consider:**
|
||||
|
||||
- **The Old Way:** *"I need to send a status update. I'll send a JSON object with metadata, timestamps, and user profile info (15KB)."*
|
||||
- **The Zen Way:** *"I need to send a status update. I'll send a single byte representing the state code. The context is already known."*
|
||||
|
||||
This is of course optimization, but more importantly, *it is a form of respect*. Efficiency in a shared medium is an act of stewardship. By taking only what you need from the network, you leave room for others. The network listens to those who speak with purpose.
|
||||
|
||||
### Flow & Time
|
||||
|
||||
Scarcity also teaches us about time. We have become addicted to the *synchronous* now - the instant ping, the real-time stream. But Reticulum embraces *asynchronous* time.
|
||||
|
||||
When links are intermittent and latency is measured in minutes or hours, "real-time" is an illusion. Reticulum doesn't encourage **Store and Forward** as a mere fallback, but as a primary mode of existence. You write a message, it propagates when it can, and it arrives when it arrives.
|
||||
|
||||
This changes the psychological texture of communication. It removes the anxiety of the immediate response. It allows for contemplation. You are not demanding the recipient's attention *right now*; you are placing a gift in their path, to be found when they are ready.
|
||||
|
||||
By designing for delay, you design for resilience. You are no longer building a house of cards that collapses when a single packet drops. You are building a stone arch that distributes the load *over time*.
|
||||
|
||||
### Liberation From Limits
|
||||
|
||||
There is a strange optimism in scarcity. When you are forced to work within strict constraints, you are forced to prioritize. *You* must decide what truly matters. *That* is the real core of agency.
|
||||
|
||||
In the infinite fantasy world of The Cloud, everything is urgent, so nothing is. In the economy of Reticulum, the cost of transmission forces you to weigh the value of your message. Do you really need to send that heart beat? Is that photo essential?
|
||||
|
||||
When you strip away the noise, what remains is *signal*.
|
||||
|
||||
This discipline creates a different kind of developer. It creates a craftsman who understands that the best code is the code you don't have to write. It creates a user who understands that the most powerful message is the one that is *understood*, not the one that is loudest. In the world of Reticulum, you are not a mere consumer of bandwidth; you are an architect of intent.
|
||||
|
||||
|
||||
## IV: Sovereignty Through Infrastructure
|
||||
**Be Your Own Network**
|
||||
|
||||
We live in an era of digital tenancy. We lease our connectivity from ISPs. We rent our storage from cloud providers. We even borrow our identity from social media platforms. We are tenants in a house we did not build, governed by rules we did not write, subject to eviction at the whim of a landlord who has never met us.
|
||||
|
||||
The Zen of Reticulum is the realization that you *can* own the house.
|
||||
|
||||
### A Carrier-Grade Fallacy
|
||||
|
||||
For decades, we have been gaslit into believing that networking is really not just hard, but impossible. It is presented as a dark art reserved for telcos and billionaires, requiring millions of dollars of fiber optics, climate-controlled data centers, and armies of engineers. We are told that building reliable infrastructure is "too complex" for the individual or small organization.
|
||||
|
||||
This is a big, fat lie.
|
||||
|
||||
Physics is simple. A radio wave needs a transmitter and a receiver. A packet needs a path. The "complexity" of the modern internet is largely bureaucratic - a mountain of billing systems, regulatory capture, and legacy cruft designed to keep the gatekeepers in power.
|
||||
|
||||
Reticulum strips away the bureaucracy. It runs on hardware that costs the price of a dinner. It runs on spectrum that is free to use. It demonstrates that a robust, planetary-scale network does not require a Fortune 500 company. It requires only the will to deploy, and the distributed, uncoordinated efforts of many individuals.
|
||||
|
||||
### Personal Infrastructure
|
||||
|
||||
This is where the rubber meets the road. You can read about Reticulum, you can understand the theory, but the insights only arrive when you plug in a radio and run a Transport Node. Suddenly, you are no longer a consumer. You're an operator.
|
||||
|
||||
This shift is subtle but profound. When you run your own infrastructure, the network ceases to be a service that is provided *to* you. It becomes a space that you *inhabit*. You become responsible for the flow of information. You gain an intimate understanding of the medium - the way the weather affects the radio waves, the way the topology changes, the way the packets dance through the ether.
|
||||
|
||||
There is a quiet competence that comes from this. You stop asking "Is the internet down?" and start asking "Is *my* links up?" You stop waiting for a technician and start checking the logs. This is a form of strength. To understand the system that carries your words is to be free from the mystery that keeps you dependent.
|
||||
|
||||
### The Ability To Disconnect
|
||||
|
||||
Why go to the trouble? Why buy the radio, write the config, and leave the Pi running in the corner?
|
||||
|
||||
Because the old, centralized network is fragile. And because most of us doesn't even really want to be there anymore.
|
||||
|
||||
The internet we rely on today is a chain of single points of failure. Cut the undersea cable, and a continent goes dark. Shut down the power grid, and the cloud evaporates. Deprioritize the "wrong" traffic, and the flow of information is strangled.
|
||||
|
||||
Sovereignty is the ability to survive the cut, whether or not that cut was an accident or on purpose.
|
||||
|
||||
When you build your own infrastructure, you build a lifeline. Reticulum is designed to function over media that the traditional internet cannot touch - bare wires, battery-powered radios, ad-hoc WiFi meshes. When the grid fails, or the censors arrive, or the bill goes unpaid, your Reticulum network continues to hum.
|
||||
|
||||
This is not about "dropping out" of society. It is about building a substrate on which an actual *Society* can function.
|
||||
|
||||
**Consider:**
|
||||
|
||||
- **The Old Way:** "My connection is slow. I should call my ISP and complain."
|
||||
- **The Zen Way:** "The path is noisy. I will adjust the antenna or find a better route."
|
||||
|
||||
By taking ownership of the infrastructure, you take ownership of your voice. You stop shouting into someone else's megaphone and start building your own. The network is no longer something that happens to you; it is something you make happen.
|
||||
|
||||
|
||||
# V: Identity and Nomadism
|
||||
**A Fluid Self**
|
||||
|
||||
In the old world, you are defined by your coordinates. If you are at `34.109.71.5`, you're *here*. If you unplug the cable and walk down the street, you vanish. Your digital self evaporates because it was tethered to the wall. You are a ghost in the endless machinations of gears, levers and transistors, bound to the hardware, and those that own it.
|
||||
|
||||
This creates a subtle, constant anxiety. We are terrified of disconnecting because, in the architecture of the old web, disconnecting is a kind of death.
|
||||
|
||||
The Zen of Reticulum offers a different way to be.
|
||||
|
||||
### Portable Existence
|
||||
|
||||
In Reticulum, your identity is not a location, or a username granted by a service. It is a cryptographic key - a complex, unique mathematical signature that exists independently of the physical world. You can carry it only in your mind, if you want to.
|
||||
|
||||
Think of it less like a street address and more like a name. *A true name*.
|
||||
|
||||
If you travel from Berlin to Tokyo, you do not change your name. You are still you. The people who know you can still recognize you. Reticulum applies this principle to the network layer. Your Destination Hash is **invariant**. It travels with you, stored securely on your device, *immutable as a stone*.
|
||||
|
||||
This changes the relationship between you and the machine. You are not "logged into" the network via a specific gateway. You *are* the endpoint. The network does not connect to a place; *it converges on you*.
|
||||
|
||||
### Roaming Nodes
|
||||
|
||||
This freedom introduces a new concept of time and space: **Nomadism**.
|
||||
|
||||
Because your identity is portable, your connectivity can be fluid. You can be sitting at a desk connected to a fiber backbone one moment, and walking through a field connected only to a long-range LoRa mesh the next. To the rest of the network, nothing has changed. Your friends do not need to update your contact info. The messages they send do not bounce back. The network senses the shift in the medium and reroutes the flow of data automatically.
|
||||
|
||||
You are no longer a stationary node in a fixed grid. You are a wanderer in a fluid medium.
|
||||
|
||||
The interfaces - whether it is WiFi, Ethernet, Packet Radio, or a physical wire - is merely the clothing your node wears. You change it to suit the environment. Underneath, you remain the same. This is the liberation of the protocol. It treats the physical medium as a transient circumstance, not a definition of self.
|
||||
|
||||
**Consider:**
|
||||
|
||||
- **The Old Way:** *"I lost connection. I have to reconnect to the VPN to tell them where I am now."*
|
||||
- **The Zen Way:** *"I moved. The network subtly bends to accomodate this new reality."*
|
||||
|
||||
### Announcing Presence
|
||||
|
||||
How does the network find a wanderer? It listens.
|
||||
|
||||
In the IP world, we query directories. We ask a server, "Where is Mark?" The server checks its database and gives us a coordinate. This means that someone, somewhere, is keeping track of you. It assumes and *requires* surveillance.
|
||||
|
||||
Reticulum replaces surveillance with **Announces**.
|
||||
|
||||
Instead of asking a central authority where you are, you simply state your presence. You broadcast a cryptographic proof: "I am here, and I am who I say I am". This ripples out through the mesh. Your neighbors hear it, update their path tables, and pass it on.
|
||||
|
||||
This is a quiet, organic process. It is the digital equivalent of lighting lanterns in the dark. You do not need to chase the light; you let the light find you. It respects your autonomy. You choose when to announce, how often to speak, and to whom. You also choose when to disappear - for but a moment or perpetually.
|
||||
|
||||
### Anchor In The Flow
|
||||
|
||||
There is a deep peace in this nomadism. It teaches you that stability does not come from standing still. Stability comes from *internal coherence*.
|
||||
|
||||
By holding your own private key, you hold your own center of gravity. The world around you; the infrastructure, the topography and the availability of links can all shift chaotically. Storms can knock out towers. Cables can be cut. The internet can go down.
|
||||
|
||||
But as long as you possess your key, you possess your identity. The entire infrastructure can be destroyed and rebuilt, and you are still you. Nothing lasts, yet nothing is lost.
|
||||
|
||||
You become a sovereign entity moving through the noise, connected not by the rigidity of cables, but by the fluidity of recognition. The network becomes a place you inhabit, rather than a utility you subscribe to: You are at home in the ether.
|
||||
|
||||
|
||||
## VI: Ethics Of The Tool
|
||||
**Technology With Conscience**
|
||||
|
||||
You have unlearned the center. You have accepted the physics of trust. You have embraced the economy of scarcity and the freedom of unbound nomadism. You are standing in a new space. Now, look at the tool in your hand.
|
||||
|
||||
In the old world, we were taught that technology is neutral. We are told that "guns don't kill people, people do", or that a component is just a component, indifferent to what its combinatorial potential is. This is a convenient lie. It serves only to allow the builders to wash their hands of responsibility.
|
||||
|
||||
But we know better now. We know that **architecture is politics**, and *politics is control*. The way you build a system determines how it will be used. If you build a system optimized for mass surveillance, you *will* get a panopticon. If you build a system optimized for centralized control, you *will* get a dictatorship. If you build a system optimized for extraction, you *will* get a parasite.
|
||||
|
||||
The Zen of Reticulum asserts that a tool is never neutral.
|
||||
|
||||
On the very contrary: A tool is intent, **crystallized**.
|
||||
|
||||
### The Harm Principle
|
||||
|
||||
Why does the Reticulum License forbid the software from being used in systems designed to harm humans? Is it not just a restriction on freedom?
|
||||
|
||||
It is a restriction on *license*, yes, but it is an expansion of *freedom*.
|
||||
|
||||
Building powerful tools without a moral compass is in no way virtuous or commendable, it is plain and simple irresponsibility.
|
||||
|
||||
A tool that can easily be used to oppress is a real danger to the user. If you build a network that can be turned against you by a tyrant, you are not free. You are merely waiting for the leash to tighten. By encoding the "Harm Principle" into the legal DNA of the reference implementation, we are building a safeguard. We are stating, clearly and immutably, that *this tool* is for **life**, not for death.
|
||||
|
||||
This aligns the software with the interests of humanity. It cements that the network cannot be conscripted into a kill-system, a weaponized drone controller, or a torture device without breaking the license and the law. It is a line drawn in the sand - not by a government or external authority, but by the creators of the tool itself.
|
||||
|
||||
**Consider:**
|
||||
|
||||
- **The Old Way:** *"It's just software. How people use it is not my problem."*
|
||||
- **The Zen Way:** *"This software is a habitat. I will not allow it to be used to build a cage."*
|
||||
|
||||
It is *your* choice whether to align with this - we are not forcing this stance on anyone. If you choose to align with life over death, with creativity over destruction, we grant you an immensely powerful tool, to own and build with as you please. If you do not, we deny it.
|
||||
|
||||
If you do not like this, we most assuredly do not need you here, and you are on your own.
|
||||
|
||||
### Public Domain Protocol
|
||||
|
||||
This leads to a vital distinction: The difference between the *idea* and the *implementation*.
|
||||
|
||||
The protocol - the mathematical rules of how Reticulum works - is dedicated to the Public Domain. It belongs to humanity. **No one can own it**. Anyone can implement it, improve it, or adapt it. This is the core idea of free communication, which itself must be forever free.
|
||||
|
||||
But the functional, deployed *reference implementation* - the Python code, the maintenance, the years of labor - has a conscience. This distinction is the engine of sustainability. It allows the protocol to be universal, while ensuring that the specific labor of the builders is not hijacked to undermine the foundational intent of the project itself. From this document, it should be very clear what this intent is.
|
||||
|
||||
If you want to build a system with Reticulum that manipulates and damages users for profits or targets missiles, you can use the public domain protocol, and start from scratch. But you cannot take our work. You must do your own. This serves as a pillar of accountability. If you want to build a weapon, *you* go and forge the steel yourself, while the world observes. And when the blood is drawn - it is on **your** hands.
|
||||
|
||||
### Preserving Human Agency
|
||||
|
||||
We live in an era of predatory extraction. The open-source commons is being scraped, ingested, and regurgitated by machine learning algorithms, whose corporate owners seek to replace the very humans who built those commons. Our code, our words, and our creativity is being used to train systems that are specifically designed to make us obsolete, without offering anything else in return than serfdom and leashes.
|
||||
|
||||
Reticulum stands against this.
|
||||
|
||||
The license protects the software from being used to feed the beast. It draws a hard line: This tool is for *people*. It is for human-to-human connection. It is not a dataset to be strip-mined for the purpose of building a synthetic overlord, puppeteered by a miniscule conglomerate of controllers.
|
||||
|
||||
This is a radical act of preservation. By protecting the code from AI appropriation, we are protecting space for human agency. We are ensuring that there remains a digital realm where the actors are flesh, blood and soul, where decisions are made by minds, not overlords hiding behind models.
|
||||
|
||||
When you use Reticulum, you are using a tool that respects you. It does not see you as a product to be tracked. It does not see your data as fuel for an algorithm. It sees you as a sovereign, equal peer.
|
||||
|
||||
This changes the foundational premise of using the technology. It restores dignity to the interaction. You are not the user of a service; you are a participant in a mutual covenant. The tool aligns with your autonomy, rather than eroding it.
|
||||
|
||||
In this way, ethics is not a restriction, but a foundation. It is the foundation that helps ensure the network will still belong to you tomorrow.
|
||||
|
||||
|
||||
## VII: Design Patterns For Post-IP Systems
|
||||
**Practical Philosophy for Developers**
|
||||
|
||||
The philosophy is useless if it cannot be hammered into code. The metaphors we have explored - nomadism, scarcity, trust - are not just poetry, but real-world engineering constraints. When you sit down to write software for Reticulum, these concepts must shape the very structure of your application.
|
||||
|
||||
We are now moving from the *why* to the *how*. This is where the abstract becomes concrete, and where you will see the true depth of the patterns we have been weaving.
|
||||
|
||||
### Store & Forward
|
||||
|
||||
The web has trained us to be impatient. We write synchronous code. We fire a request and we wait, blocking the UI, holding our breath. If the response doesn't come in 250 milliseconds, we show a spinner. If it doesn't come in five seconds, we show an error. We treat network connectivity as a binary state: either we are "online" or we are "broken".
|
||||
|
||||
This is brittle. It is a rejection of reality.
|
||||
|
||||
In Reticulum, connectivity is a spectrum, and presence is asynchronous. If at all applicable to your intent, you must design your applications to embrace **Store & Forward**.
|
||||
|
||||
Instead of demanding an immediate answer, your application should act as a patient participant. You create a message for someone or something in the mesh. The network holds it. It carries it from node to node, perhaps over hours or days, waiting for the recipient to appear. When they finally surface, the message is delivered. This requires a shift from "request/response" to "event/handler". How exactly you do this is a challenge for you to solve intelligently within your problem domain, but Reticulum-based systems already exist that does this extremely well, and you can use them for inspiration.
|
||||
|
||||
**Consider:**
|
||||
|
||||
- **The Old Way:** `Connect() -> Send() -> Wait() -> Crash if timeout.`
|
||||
- **The Zen Way:** `Send() -> Continue living. -> Receive() when it arrives.`
|
||||
|
||||
This changes the user experience profoundly. It removes the anxiety of the loading bar. It creates a sense of continuity. The user is not "waiting for the network"; they are interacting with a persistent log of communication that lives in the network itself.
|
||||
|
||||
### Naming Is Power
|
||||
|
||||
In the IP world, we are slaves to the Domain Name System. We rely on a hierarchy of registrars to map human-readable names to machine-readable addresses. This hierarchy is a choke point. If the registrar revokes your domain, or if the DNS server goes down, you vanish.
|
||||
|
||||
Reticulum dissolves this hierarchy with **Hash-based Identity**.
|
||||
|
||||
In this design pattern, a name is not a string you look up; it is a cryptographic destination you verify. When you design for Reticulum, you stop asking the user for a URL and start asking for a Destination or Identity Hash.
|
||||
|
||||
This feels strange at first. A hash like `<83b7328926fed0d2e6a10a7671f9e237>` looks alien compared to `myfriend.com`. But that alienness is the armor. It **cannot** be spoofed. It **cannot** be censored by a registrar. It is **absolute**.
|
||||
|
||||
Designing for this means shifting your UI metaphors. You are no longer browsing a web of pages; you are managing a ledger of keys. You are building an "Address Book" that is actually a keyring. The names are given by the user, and the power stays with them. That hashes look complex is directly analogous to the strengths of the bonds formed by their use. It forces the user to engage in a moment of verification, an out-of-band handshake, which restores the human element of trust that SSL certificates stripped away.
|
||||
|
||||
### The Interface Is The Medium
|
||||
|
||||
One of the most liberating patterns in Reticulum is **Transport Agnosticism**.
|
||||
|
||||
In traditional networking, your code is often littered with transport logic. "Am I on WiFi? Check bandwidth. Am I on Cellular? Check data plan. Am I on Ethernet?". You are constantly micromanaging the pipe.
|
||||
|
||||
In Reticulum, you write to the API, and the API writes to the medium. You send a packet to a Destination. You do not care if that packet travels over a TCP tunnel, a LoRa radio wave, or a serial wire interface. That is the stack's concern.
|
||||
|
||||
This allows you to write **Universal Applications**.
|
||||
Imagine a messaging app. You write it once. It works on a laptop connected to fiber. It works on a phone in the city using WiFi. And, without a single line of code changed, it works on a device in the wilderness, talking only to other devices via radio.
|
||||
|
||||
The pattern is simple: **Never code to the hardware. Code to the intent.**
|
||||
|
||||
**Consider:**
|
||||
|
||||
- **The Old Way:** `socket.connect(ip, port)`
|
||||
- **The Zen Way:** `RNS.Packet(destination, data).send()`
|
||||
|
||||
By abstracting the medium, you make your software immortal to changes in infrastructure. The user might switch from a 4G hotspot to a HF modem tomorrow. Your software doesn't need to know. It simply continues the conversation.
|
||||
|
||||
### Emergent Patterns
|
||||
|
||||
When you combine these patterns - *Store & Forward*, *Hash-based Identity*, and *Transport Agnosticism* - you create software that feels fundamentally different.
|
||||
|
||||
It feels *grounded*. It doesn't flicker when the signal drops. It doesn't panic when the server is down. It has weight. It has persistence. It has *relevance*.
|
||||
|
||||
You are no longer building a "client" that begs a "server" for attention. You are building an autonomous agent that exists within the mesh. It speaks when it needs to, listens when it can, and carries its identity with it wherever it goes.
|
||||
|
||||
This is the culmination of the Zen. The code is not just a set of instructions: It is a behavioral envelope. It is a way of *being* in the network.
|
||||
|
||||
|
||||
## VIII: Fabric Of The Independent
|
||||
|
||||
We have stripped away the illusions. We have seen that the center is empty, that trust *must* be hard, that resources are finite, and that we must own our infrastructure. We have seen that tools have ethics and that our identity can move fluidly.
|
||||
|
||||
This is a reclaiming of the commons. For too long, we have allowed the most vital substrate of human society - *our ability to speak to one another* - to be colonized by entities that do not share our interests. We have allowed the architecture of our communication to be designed by accountants rather than architects.
|
||||
|
||||
We are taking it back. Not by petitioning the masters, but by building the new world within, over, under and around the shell of the old.
|
||||
|
||||
### The Work Is Finished
|
||||
|
||||
The heavy lifting is done.
|
||||
|
||||
The protocol is in the public domain, a gift to humanity that can never be taken away. The software is written, tested, and running on devices scattered across the globe. The manual lies open before you. The source code for the reference implementation is now distributed on hundreds of thousands of devices across the planet. No one can delete or destroy it. The hardware is accessible and abundant.
|
||||
|
||||
It was a hard road to get here, but we got here. Now, there is no roadmap committee waiting for approval. There is no venture capital dictating the user experience. There is no CEO to sign off on the next feature release.
|
||||
|
||||
There is only you.
|
||||
|
||||
The barrier to entry is no longer complexity: It is the mere habit of dependency. You were conditioned to wait. Wait for the app update. Wait for the ISP to fix the line. Wait for the platform to allow the post. Wait for the government to change the policies. Wait for the likes. Wait for the revolution to be televised.
|
||||
|
||||
The revolution never was televised.
|
||||
|
||||
It is packetized.
|
||||
|
||||
### Open Sky
|
||||
|
||||
The future of this technology is a construction project.
|
||||
|
||||
It looks like a single node on a windowsill, listening to the static. It looks like a message sent to a neighbor, bypassing the noise of the commercial web. It looks like a community mesh that grows, link by link, hop by hop, carried by hands that care more about connection than profit.
|
||||
|
||||
You have the blueprints. You have the tools. You have the philosophy. The noise of the old world has fallen away, leaving you with the quiet clarity of the open spectrum.
|
||||
|
||||
*Mark, early 2026*
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue