Compare commits

..

No commits in common. "master" and "0.8.2" have entirely different histories.

323 changed files with 28296 additions and 86359 deletions

View file

@ -12,15 +12,10 @@ Before creating a bug report on this issue tracker, you **must** read the [Contr
- The issue tracker is used by developers of this project. **Do not use it to ask general questions, or for support requests**.
- Ideas and feature requests can be made on the [Discussions](https://github.com/markqvist/Reticulum/discussions). **Only** feature requests accepted by maintainers and developers are tracked and included on the issue tracker. **Do not post feature requests here**.
- Do not submit code written using large language models (LLMs) or other generative 'AI' programs (see the [Generative AI Policy](/Contributing.md#generative-ai-policy) for details).
- After reading the [Contribution Guidelines](https://github.com/markqvist/Reticulum/blob/master/Contributing.md), **delete this section only** (*"Read the Contribution Guidelines"*) from your bug report, **and fill in all the other sections**.
- After reading the [Contribution Guidelines](https://github.com/markqvist/Reticulum/blob/master/Contributing.md), delete this section from your bug report.
**Describe the Bug**
First of all: Is this really a bug? Is it reproducible?
If this is a request for help because something is not working as you expected, stop right here, and go to the [discussions](https://github.com/markqvist/Reticulum/discussions) instead, where you can post your questions and get help from other users.
If this really is a bug or issue with the software, remove this section of the template, and provide **a clear and concise description of what the bug is**.
A clear and concise description of what the bug is.
**To Reproduce**
Describe in detail how to reproduce the bug.
@ -29,7 +24,7 @@ Describe in detail how to reproduce the bug.
A clear and concise description of what you expected to happen.
**Logs & Screenshots**
Please include any relevant log output. If applicable, also add screenshots to help explain your problem. In most cases, without any relevant log output, we will not be able to determine the cause of the bug, or reproduce it.
Please include any relevant log output. If applicable, also add screenshots to help explain your problem.
**System Information**
- OS and version

View file

@ -1,98 +0,0 @@
name: Build Reticulum
on:
push:
branches:
- '*'
tags:
- "[0-9]+.[0-9]+.[0-9]+*"
pull_request:
branches:
- master
paths-ignore:
- .gitignore
- LICENSE
permissions:
contents: write
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.11
- run: |
python -m pip install -q cryptography
make test
package:
needs: test
if: startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest
environment: ${{ contains(github.ref, '-') && 'development' || 'production' }}
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.11
- run: |
python -m pip install -q build wheel setuptools
make remove_symlinks
make build_wheel
make build_pure_wheel
make create_symlinks
- uses: actions/upload-artifact@v4
with:
name: package
path: dist/*.whl
# documentation:
# needs: test
# if: startsWith(github.ref, 'refs/tags/')
# runs-on: ubuntu-latest
# environment: ${{ contains(github.ref, '-') && 'development' || 'production' }}
# steps:
# - uses: actions/checkout@v4
# - uses: actions/setup-python@v5
# with:
# python-version: 3.x
# - run: |
# sudo apt-get -qq update && sudo apt-get -qq install latexmk texlive-latex-recommended texlive-latex-extra texlive-fonts-recommended
# python -m pip -q install sphinx sphinx-copybutton
# cd docs && make latexpdf && make epub
# - uses: actions/upload-artifact@v4
# with:
# name: documentation
# path: |
# docs/build/latex/*.pdf
# docs/build/epub/*.epub
release:
needs: [package]
if: startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest
environment: ${{ contains(github.ref, '-') && 'development' || 'production' }}
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
path: .artifacts
- uses: softprops/action-gh-release@v2
with:
files: |
# .artifacts/package/**.whl
# .artifacts/documentation/latex/reticulumnetworkstack.pdf
# .artifacts/documentation/epub/ReticulumNetworkStack.epub
draft: true
generate_release_notes: false
prerelease: ${{ contains(github.ref, '-') }}
fail_on_unmatched_files: true

28
.github/workflows/python-app.yml vendored Normal file
View file

@ -0,0 +1,28 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
name: Test suite
on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]
permissions:
contents: read
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.10
uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Test
run: |
make test

2
.gitignore vendored
View file

@ -13,5 +13,3 @@ tests/rnsconfig/storage
tests/rnsconfig/logfile*
*.data
*.result
.buildinfo.bak
docs/Reticulum Manual.*

1386
AGENTS.md

File diff suppressed because it is too large Load diff

View file

@ -1,292 +0,0 @@
compiling = False
noticed = False
notice_delay = 0.3
import os
import sys
import time
import threading
from importlib.util import find_spec
if find_spec("pyximport") and find_spec("cython"):
import json
import shutil
import subprocess
from importlib.abc import MetaPathFinder
from importlib.machinery import ExtensionFileLoader, SourceFileLoader
from importlib.machinery import EXTENSION_SUFFIXES
from importlib.util import spec_from_file_location
from distutils.extension import Extension
from pyximport import pyxbuild
COMPILE_DIRECTIVES = { "annotation_typing": False }
_WORKER_SCRIPT = r"""
import sys, os, json
from distutils.extension import Extension
from pyximport import pyxbuild
def main():
tasks_path = sys.argv[1]
with open(tasks_path, "r") as fh: tasks = json.load(fh)
results = []
for task in tasks:
try:
os.makedirs(task["scratch"], exist_ok=True)
extension = Extension(name=task["basename"], sources=[task["source"]])
extension.cython_directives = task["directives"]
so_path = pyxbuild.pyx_to_dll(task["source"], ext=extension, build_in_temp=True, pyxbuild_dir=task["scratch"])
os.makedirs(os.path.dirname(task["target"]), exist_ok=True)
os.replace(so_path, task["target"])
results.append({"fullname": task["fullname"], "ok": True, "error": None})
except Exception as e:
results.append({"fullname": task["fullname"], "ok": False,
"error": "{}: {}".format(type(e).__name__, e)})
with open(tasks_path + ".results", "w") as fh: json.dump(results, fh)
if __name__ == "__main__": main()
"""
def _compile_jobs():
jobs = os.environ.get("CRNS_JOBS")
if jobs: return max(1, int(jobs))
return max(1, min(os.cpu_count() or 1, 16))
class _ImportFinder(MetaPathFinder):
def __init__(self):
self.build_dir = os.path.join(os.path.expanduser("~"), ".pyxbld")
self.mirror_root = os.path.join(self.build_dir, "rnslib")
self.source_root = self._find_source_root()
self.abi_suffix = EXTENSION_SUFFIXES[0]
self.unbuildable = {}
self.unbuildable_path = os.path.join(self.build_dir, "unbuildable.json")
self._verify_mirror_directives()
try:
with open(self.unbuildable_path, "r") as fh: self.unbuildable = json.load(fh)
except Exception: self.unbuildable = {}
def _verify_mirror_directives(self):
# If the compiler directives have changed since the mirror was
# built, the entire mirror is invalidated and rebuilt.
stamp_path = os.path.join(self.build_dir, "rnslib.stamp")
directives = json.dumps(COMPILE_DIRECTIVES, sort_keys=True)
stamp = directives + "\n" + self.abi_suffix
previous = None
try:
with open(stamp_path, "r") as fh: previous = fh.read()
except Exception: pass
if previous != stamp:
shutil.rmtree(self.mirror_root, ignore_errors=True)
os.makedirs(self.mirror_root, exist_ok=True)
try:
os.makedirs(self.build_dir, exist_ok=True)
with open(stamp_path, "w") as fh: fh.write(stamp)
except Exception: pass
def _save_unbuildable(self):
try:
os.makedirs(os.path.dirname(self.unbuildable_path), exist_ok=True)
with open(self.unbuildable_path, "w") as fh: json.dump(self.unbuildable, fh, indent=1)
except Exception: pass
def _mark_unbuildable(self, fullname, source):
try:
self.unbuildable[fullname] = int(os.path.getmtime(source))
self._save_unbuildable()
except Exception: pass
def _unmark_buildable(self, fullname):
if fullname in self.unbuildable:
del self.unbuildable[fullname]
self._save_unbuildable()
def _is_unbuildable(self, fullname, source):
if fullname not in self.unbuildable: return False
try:
if int(os.path.getmtime(source)) == self.unbuildable[fullname]: return True
except Exception: pass
self.unbuildable.pop(fullname, None)
return False
def _find_source_root(self):
here = os.path.dirname(os.path.abspath(__file__))
candidate = os.path.abspath(os.path.join(here, "..", "RNS", "__init__.py"))
if os.path.exists(candidate): return os.path.dirname(os.path.dirname(candidate))
for entry in [os.getcwd()] + list(sys.path):
if not entry: continue
candidate = os.path.join(entry, "RNS", "__init__.py")
if os.path.exists(candidate): return os.path.dirname(candidate)
raise RuntimeError("CRNS could not locate the RNS source tree")
def _all_source_modules(self):
rns_root = os.path.join(self.source_root, "RNS")
for root, dirs, files in os.walk(rns_root):
dirs[:] = [d for d in dirs if d not in ("__pycache__", "i2plib")]
rel = os.path.relpath(root, self.source_root).replace(os.sep, ".")
for fn in sorted(files):
if not fn.endswith(".py"): continue
if fn == "__init__.py": fullname = rel
else: fullname = rel + "." + fn[:-3]
yield fullname
def _mirror_target(self, fullname, is_package):
rel = fullname.split(".")
basename = rel[-1]
if is_package: return os.path.join(self.mirror_root, *rel, "__init__" + self.abi_suffix)
return os.path.join(self.mirror_root, *rel[:-1], basename + self.abi_suffix)
def _source(self, fullname):
rel = fullname.split(".")
package_source = os.path.join(self.source_root, *rel, "__init__.py")
if os.path.exists(package_source): return package_source, True
module_source = os.path.join(self.source_root, *rel) + ".py"
if os.path.exists(module_source): return module_source, False
return None, None
def _compile(self, fullname, source):
basename = fullname.rsplit(".", 1)[-1]
extension = Extension(name=basename, sources=[source])
extension.cython_directives = COMPILE_DIRECTIVES
so_path = pyxbuild.pyx_to_dll(source, ext=extension, build_in_temp=True, pyxbuild_dir=self.build_dir)
abi_suffix = os.path.basename(so_path)[len(basename):]
return so_path, abi_suffix
def _is_current(self, fullname, source, is_package):
target = self._mirror_target(fullname, is_package)
if not os.path.exists(target): return False
try: return os.path.getmtime(target) >= os.path.getmtime(source)
except Exception: return False
def _ensure_compiled(self, fullname, source, is_package):
target = self._mirror_target(fullname, is_package)
if os.path.exists(target) and os.path.getmtime(target) >= os.path.getmtime(source): return target
so_path, abi_suffix = self._compile(fullname, source)
os.makedirs(os.path.dirname(target), exist_ok=True)
os.replace(so_path, target)
return target
def precompile(self):
if not os.path.isdir(self.mirror_root):
os.makedirs(self.mirror_root, exist_ok=True)
pending = []
for fullname in self._all_source_modules():
source, is_package = self._source(fullname)
if source is None:
continue
if self._is_current(fullname, source, is_package):
self._unmark_buildable(fullname)
continue
if self._is_unbuildable(fullname, source):
continue
pending.append((fullname, source, is_package))
if not pending: return
jobs = min(_compile_jobs(), len(pending))
if jobs > 1:
# Round-robin assignment spreads the large modules evenly
# across workers.
chunks = [[] for _ in range(jobs)]
for i, item in enumerate(pending): chunks[i % jobs].append(item)
workers_dir = os.path.join(self.build_dir, "workers")
os.makedirs(workers_dir, exist_ok=True)
procs = []
for w in range(jobs):
tasks = []
for fullname, source, is_package in chunks[w]:
tasks.append({ "fullname": fullname,
"basename": fullname.rsplit(".", 1)[-1],
"source": source,
"target": self._mirror_target(fullname, is_package),
"scratch": os.path.join(workers_dir, "w{}".format(w)),
"directives": COMPILE_DIRECTIVES })
task_path = os.path.join(workers_dir, "tasks{}.json".format(w))
with open(task_path, "w") as fh: json.dump(tasks, fh)
procs.append((w, task_path, subprocess.Popen([sys.executable, "-c", _WORKER_SCRIPT, task_path],
stdout=subprocess.PIPE, stderr=subprocess.PIPE)))
for w, task_path, proc in procs:
stdout, stderr = proc.communicate()
if proc.returncode != 0:
for fullname, _, _ in chunks[w]:
source, _ = self._source(fullname)
self._mark_unbuildable(fullname, source)
print(f"CRNS: compile worker failed for {fullname}, using interpreted module", file=sys.stderr)
if stderr:
tail = stderr.decode(errors="replace").strip().splitlines()[-5:]
print("CRNS: worker output:\n" + "\n".join(tail), file=sys.stderr)
continue
results = {}
try:
with open(task_path + ".results", "r") as fh:
for entry in json.load(fh): results[entry["fullname"]] = entry
except Exception: results = {}
for fullname, _, _ in chunks[w]:
entry = results.get(fullname)
if entry is not None and entry.get("ok"): self._unmark_buildable(fullname)
else:
error = "" if entry is None else entry.get("error", "")
source, _ = self._source(fullname)
self._mark_unbuildable(fullname, source)
print(f"Could not compile {fullname}, using interpreted module: {error}", file=sys.stderr)
else:
for fullname, source, is_package in pending:
try:
self._ensure_compiled(fullname, source, is_package)
self._unmark_buildable(fullname)
except Exception as e:
self._mark_unbuildable(fullname, source)
print(f"Could not compile {fullname}, using interpreted module: {e}", file=sys.stderr)
def _spec(self, fullname, source, is_package, compile_attempted):
if fullname == "RNS.vendor.i2plib" or fullname.startswith("RNS.vendor.i2plib."):
rel = fullname.split(".")
locations = [os.path.join(self.source_root, *rel)] if is_package else None
return spec_from_file_location(fullname, source, loader=SourceFileLoader(fullname, source), submodule_search_locations=locations)
if compile_attempted:
try:
target = self._ensure_compiled(fullname, source, is_package)
self._unmark_buildable(fullname)
if is_package: locations = [os.path.dirname(target)]
else: locations = None
return spec_from_file_location(fullname, target, loader=ExtensionFileLoader(fullname, target),
submodule_search_locations=locations)
except Exception as e:
self._mark_unbuildable(fullname, source)
print(f"Could not compile {fullname}, using interpreted module: {e}", file=sys.stderr)
rel = fullname.split(".")
locations = [os.path.join(self.source_root, *rel)] if is_package else None
return spec_from_file_location(fullname, source,loader=SourceFileLoader(fullname, source),submodule_search_locations=locations)
def find_spec(self, fullname, path, target=None):
if not isinstance(fullname, str): return None
if not (fullname == "RNS" or fullname.startswith("RNS.")): return None
source, is_package = self._source(fullname)
if source is None: return None
return self._spec(fullname, source, is_package, compile_attempted=not self._is_unbuildable(fullname, source))
_importfinder = _ImportFinder()
sys.meta_path.insert(0, _importfinder)
def notice_job():
global noticed
started = time.time()
while compiling:
if time.time() > started+notice_delay and compiling:
noticed = True
print("Compiling RNS object code...")
sys.stdout.flush()
break
time.sleep(0.1)
compiling = True
threading.Thread(target=notice_job, daemon=True).start()
_importfinder.precompile()
import RNS; compiling = False
if noticed: print("Compilation done"); sys.stdout.flush()

File diff suppressed because it is too large Load diff

View file

@ -2,66 +2,42 @@
Welcome, and thank you for your interest in contributing to Reticulum!
Before interacting with this community, read these short and simple guidelines.
If you didn't shit your pants after reading this document, you may well find that we are all really quite cuddly and nice to be around. Mostly.
Apart from writing code, there are many ways in which you can contribute. Before interacting with this community, read these short and simple guidelines.
## Expected Conduct
First and foremost, there is one simple requirement for taking part in the development of this technology: While we primarily interact virtually, your actions matter and have real consequences. Therefore: **Act like a responsible, civilized person** - especially in the face of disputes and heated disagreements. Speak your mind here; discussions are welcome. Just do so in the spirit of being face-to-face with everyone else. Thank you.
First and foremost, there is one simple requirement for taking part in this community: While we primarily interact virtually, your actions matter and have real consequences. Therefore: **Act like a responsible, civilized person** - also in the face of disputes and heated disagreements. Speak your mind here, discussions are welcome. Just do so in the spirit of being face-to-face with everyone else. Thank you.
In order to keep the discourse navigable and useful, the following types of posts and inquiries will be deleted and ignored:
## Asking Questions
- Spam.
- Questions that have already been adequately answered elsewhere. Utilize search functions.
- Low-effort posts or comments that contain no actual information or useful substance. [This is not a tea-house](./This%20Is%20Not%20a%20Teahouse.md).
- Post or comments solely containing personal opinions or beliefs without adding anything to the discussion. Facebook and X exists.
- Content that simply waste the developer's / maintainer's time with completely obvious "ideas", "insights" or "recommendations". Yes, we have *at least* 8 neurons ourselves.
- Posts that fail to understand that developing a highly complex software project with a very small amount of resources and people takes time. Imagining perfection on our behalf is useless.
If you want to ask a question, **do not open an issue**. The issue tracker is used by people *working on Reticulum* to track bugs, issues and improvements.
If you're new to the community and start out your engagement with any of the above transgressions, you will simply be banned permanently without notice or explanation, and any further communication from you will be ignored.
Instead, ask away on the [discussions](https://github.com/markqvist/Reticulum/discussions) or on the [Reticulum Matrix channel](https://matrix.to/#/#reticulum:matrix.org) at `#reticulum:matrix.org`
Except for the case of real, well-grounded, clearly documented, human-written security advisories, technical investigations or bug reports, if your inquiry or communication is not attached to a Reticulum Identity, expect it to be ignored.
## Providing Feedback & Ideas
If you find this "harsh", "unfair" or "unwelcoming", go somewhere else. This is not social club, but a work environment for the people contributing to the project.
Likewise, feedback, ideas and feature requests are a very welcome way to contribute, and should also be posted on the [discussions](https://github.com/markqvist/Reticulum/discussions), or on the [Reticulum Matrix channel](https://matrix.to/#/#reticulum:matrix.org) at `#reticulum:matrix.org`.
## Reporting Bugs
Please do not post feature requests or general ideas on the issue tracker, or in direct messages to the primary developers. You are much more likely to get a response and start a constructive discussion by posting your ideas in the public channels created for these purposes.
If you have found a bug or issue in this project, please report it to the maintainer over LXMF, or if strictly not possible, over email. Detailed and serious bug reports can also be posted in the various on-network forums and RRC channels that the developers frequent. Due to the realities described in [This Is Not a Teahouse](./This%20Is%20Not%20a%20Teahouse.md), there is no way to report issues or request features over the internet.
## Reporting Issues
If you have found a bug or issue in this project, please report it using the [issue tracker](https://github.com/markqvist/Reticulum/issues). If at all possible, be sure to include details on how to reproduce the bug.
Anything submitted to the issue tracker that does not follow these guidelines will be closed and removed without comments or explanation.
## Writing Code
Patches and code contributions are *generally* only welcomed from developers already known to the community, with a proven track-record of competence, skill, careful and considerate design and responsible conduct. If we do not know you yet, but you have created something that you believe lives up to such standards, we are eager to hear from you.
If you are interested in contributing code, fixing open issues or adding features, please coordinate the effort with the maintainer or one of the main developers **before** submitting a pull request. Before deciding to contribute, it is also a good idea to ensure your efforts are in alignment with the [Roadmap](./Roadmap.md) and current development focus.
Current development priorities and work documents are not publicly available on the internet, but shared and coordinated directly between the developers over Reticulum itself, following the principles outlined in [Distributed Development](https://reticulum.network/manual/distributed.html).
Pull requests have a high chance of being accepted if they are:
## Commit Signing
- In alignment with the [Roadmap](./Roadmap.md) or solve an open issue or feature request
- Sufficiently tested to work with all API functions, and pass the standard test suite
- Functionally and conceptually complete and well-designed
- Not simply formatting or code style changes
- Well-documented
All commits in Reticulum and related projects **must** be signed with a Reticulum identity. If you want your author credentials to be included directly in the commit log, ensure that you have [configured commit signing](https://reticulum.network/manual/git.html#commit-signing). Bare patches submitted manually or fetched remote commits without signatures are still very welcome, but we will review, squash merge and sign with the key of the maintainer that approved the changes. Attribution will be provided in the commit message instead (unless you wish to remain anonymous).
Even new ideas and proposals that have not been approved by a maintainer, or fall outside the established roadmap, are *occasionally* accepted - if they possess the remaining of the above qualities. If not, they will be closed and removed without comments or explanation.
## Machine Assistance & LLMs
For a highly complex, critical piece of cryptographically based networking infrastructure, we should not need to state the following, but alas, experience has proven otherwise. Therefore:
**Including material generated using LLMs or other "AI" programs in Reticulum is, almost without exception, prohibited.**
Before even *thinking* about using a generative software system to "create" something for Reticulum, read [Brandolini's Reference](https://reticulum.network/manual/brandolinis.html), in full. If you do not have the capacity for that, you have failed the first requirement for contributing: Basic literacy.
*If* you are not already a known contributor with a proven record of being able to responsibly handle these tools, a **full** audit trail of all ML-utilizing workflows, including prompts, LLM reasoning traces, tool calls and outputs (that is, the *entire*, unaltered session log) will need to be provided with any proposed contribution. Without that, we will not even *consider* looking at it.
This applies to all Reticulum-related projects and documentation, as well as all submitted issues, reports and discussion, **except** of course in cases where language translation, speech recogntion, or other assistive technologies are required.
In **all** circumstances, clear disclosure is **mandatory**. Any piece of material, file, code snippet, method, function or similar that was generated with machine assistance must be prepended with a disclosure header or comment that states the framework, exact model used and reviewer, such as:
```python
# Generated with lc/deepseek_v4_flash_q8_k_xl
# Reviewed and corrected by Mark Qvist
```
Additionally, any explanatory comments or doc-strings generated by software must be manually re-written in clear, concise language by the reviewer stated in the disclosure header.
## Contributor License Agreement
By contributing code to this project, you agree that copyright for the code is transferred to the Reticulum maintainers and that the code is irrevocably placed under the [Reticulum License](./LICENSE).
For a more in-depth elaboration on the current CLA position, see [this post](https://rns.recipes/forum/general/brandolinis-reference-or-alternatively-agentsmd?page=1#post-3002).
By contributing code to this project, you agree that copyright for the code is transferred to the Reticulum maintainers and that the code is irrevocably placed under the [MIT license](./LICENSE).

View file

@ -6,7 +6,6 @@
import argparse
import random
import sys
import RNS
# Let's define an app name. We'll use this for all
@ -169,4 +168,4 @@ if __name__ == "__main__":
except KeyboardInterrupt:
print("")
sys.exit(0)
exit()

View file

@ -118,4 +118,4 @@ if __name__ == "__main__":
except KeyboardInterrupt:
print("")
sys.exit(0)
exit()

View file

@ -157,7 +157,7 @@ def client(destination_hexhash, configpath):
destination_hash = bytes.fromhex(destination_hexhash)
except:
RNS.log("Invalid destination entered. Check your input!\n")
sys.exit(0)
exit()
# We must first initialise Reticulum
reticulum = RNS.Reticulum(configpath)
@ -254,8 +254,9 @@ def link_closed(link):
else:
RNS.log("Link closed, exiting now")
RNS.Reticulum.exit_handler()
time.sleep(1.5)
sys.exit(0)
os._exit(0)
# When the buffer has new data, read it and write it to the terminal.
def client_buffer_ready(ready_bytes: int):
@ -319,4 +320,4 @@ if __name__ == "__main__":
except KeyboardInterrupt:
print("")
sys.exit(0)
exit()

View file

@ -124,7 +124,7 @@ def server(configpath):
def server_loop(destination):
# Let the user know that everything is ready
RNS.log(
"Channel example "+
"Link example "+
RNS.prettyhexrep(destination.hash)+
" running, waiting for a connection."
)
@ -212,7 +212,7 @@ def client(destination_hexhash, configpath):
destination_hash = bytes.fromhex(destination_hexhash)
except:
RNS.log("Invalid destination entered. Check your input!\n")
sys.exit(0)
exit()
# We must first initialise Reticulum
reticulum = RNS.Reticulum(configpath)
@ -276,7 +276,7 @@ def client_loop():
packed_size = len(message.pack())
channel = server_link.get_channel()
if channel.is_ready_to_send():
if packed_size <= channel.mdu:
if packed_size <= channel.MDU:
channel.send(message)
else:
RNS.log(
@ -321,8 +321,9 @@ def link_closed(link):
else:
RNS.log("Link closed, exiting now")
RNS.Reticulum.exit_handler()
time.sleep(1.5)
sys.exit(0)
os._exit(0)
# When a packet is received over the channel, we
# simply print out the data.
@ -386,4 +387,4 @@ if __name__ == "__main__":
except KeyboardInterrupt:
print("")
sys.exit(0)
exit()

View file

@ -6,7 +6,6 @@
##########################################################
import argparse
import sys
import RNS
# Let's define an app name. We'll use this for all
@ -131,7 +130,7 @@ def client(destination_hexhash, configpath, timeout=None):
except Exception as e:
RNS.log("Invalid destination entered. Check your input!")
RNS.log(str(e)+"\n")
sys.exit(0)
exit()
# We must first initialise Reticulum
reticulum = RNS.Reticulum(configpath)
@ -329,4 +328,4 @@ if __name__ == "__main__":
client(args.destination, configarg, timeout=timeoutarg)
except KeyboardInterrupt:
print("")
sys.exit(0)
exit()

View file

@ -1,297 +0,0 @@
# This example illustrates creating a custom interface
# definition, that can be loaded and used by Reticulum at
# runtime. Any number of custom interfaces can be created
# and loaded. To use the interface place it in the folder
# ~/.reticulum/interfaces, and add an interface entry to
# your Reticulum configuration file similar to this:
# [[Example Custom Interface]]
# type = ExampleInterface
# enabled = no
# mode = gateway
# port = /dev/ttyUSB0
# speed = 115200
# databits = 8
# parity = none
# stopbits = 1
from time import sleep
import sys
import threading
import time
# This HDLC helper class is used by the interface
# to delimit and packetize data over the physical
# medium - in this case a serial connection.
class HDLC():
# This example interface packetizes data using
# simplified HDLC framing, similar to PPP
FLAG = 0x7E
ESC = 0x7D
ESC_MASK = 0x20
@staticmethod
def escape(data):
data = data.replace(bytes([HDLC.ESC]), bytes([HDLC.ESC, HDLC.ESC^HDLC.ESC_MASK]))
data = data.replace(bytes([HDLC.FLAG]), bytes([HDLC.ESC, HDLC.FLAG^HDLC.ESC_MASK]))
return data
# Let's define our custom interface class. It must
# be a sub-class of the RNS "Interface" class.
class ExampleInterface(Interface):
# All interface classes must define a default
# IFAC size, used in IFAC setup when the user
# has not specified a custom IFAC size. This
# option is specified in bytes.
DEFAULT_IFAC_SIZE = 8
# The following properties are local to this
# particular interface implementation.
owner = None
port = None
speed = None
databits = None
parity = None
stopbits = None
serial = None
# All Reticulum interfaces must have an __init__
# method that takes 2 positional arguments:
# The owner RNS Transport instance, and a dict
# of configuration values.
def __init__(self, owner, configuration):
# The following lines demonstrate handling
# potential dependencies required for the
# interface to function correctly.
import importlib
if importlib.util.find_spec('serial') != None:
import serial
else:
RNS.log("Using this interface requires a serial communication module to be installed.", RNS.LOG_CRITICAL)
RNS.log("You can install one with the command: python3 -m pip install pyserial", RNS.LOG_CRITICAL)
RNS.panic()
# We start out by initialising the super-class
super().__init__()
# To make sure the configuration data is in the
# correct format, we parse it through the following
# method on the generic Interface class. This step
# is required to ensure compatibility on all the
# platforms that Reticulum supports.
ifconf = Interface.get_config_obj(configuration)
# Read the interface name from the configuration
# and set it on our interface instance.
name = ifconf["name"]
self.name = name
# We read configuration parameters from the supplied
# configuration data, and provide default values in
# case any are missing.
port = ifconf["port"] if "port" in ifconf else None
speed = int(ifconf["speed"]) if "speed" in ifconf else 9600
databits = int(ifconf["databits"]) if "databits" in ifconf else 8
parity = ifconf["parity"] if "parity" in ifconf else "N"
stopbits = int(ifconf["stopbits"]) if "stopbits" in ifconf else 1
# In case no port is specified, we abort setup by
# raising an exception.
if port == None:
raise ValueError(f"No port specified for {self}")
# All interfaces must supply a hardware MTU value
# to the RNS Transport instance. This value should
# be the maximum data packet payload size that the
# underlying medium is capable of handling in all
# cases without any segmentation.
self.HW_MTU = 564
# We initially set the "online" property to false,
# since the interface has not actually been fully
# initialised and connected yet.
self.online = False
# In this case, we can also set the indicated bit-
# rate of the interface to the serial port speed.
self.bitrate = speed
# Configure internal properties on the interface
# according to the supplied configuration.
self.pyserial = serial
self.serial = None
self.owner = owner
self.port = port
self.speed = speed
self.databits = databits
self.parity = serial.PARITY_NONE
self.stopbits = stopbits
self.timeout = 100
if parity.lower() == "e" or parity.lower() == "even":
self.parity = serial.PARITY_EVEN
if parity.lower() == "o" or parity.lower() == "odd":
self.parity = serial.PARITY_ODD
# Since all required parameters are now configured,
# we will try opening the serial port.
try:
self.open_port()
except Exception as e:
RNS.log("Could not open serial port for interface "+str(self), RNS.LOG_ERROR)
raise e
# If opening the port succeeded, run any post-open
# configuration required.
if self.serial.is_open:
self.configure_device()
else:
raise IOError("Could not open serial port")
# Open the serial port with supplied configuration
# parameters and store a reference to the open port.
def open_port(self):
RNS.log("Opening serial port "+self.port+"...", RNS.LOG_VERBOSE)
self.serial = self.pyserial.Serial(
port = self.port,
baudrate = self.speed,
bytesize = self.databits,
parity = self.parity,
stopbits = self.stopbits,
xonxoff = False,
rtscts = False,
timeout = 0,
inter_byte_timeout = None,
write_timeout = None,
dsrdtr = False,
)
# The only thing required after opening the port
# is to wait a small amount of time for the
# hardware to initialise and then start a thread
# that reads any incoming data from the device.
def configure_device(self):
sleep(0.5)
thread = threading.Thread(target=self.read_loop)
thread.daemon = True
thread.start()
self.online = True
RNS.log("Serial port "+self.port+" is now open", RNS.LOG_VERBOSE)
# This method will be called from our read-loop
# whenever a full packet has been received over
# the underlying medium.
def process_incoming(self, data):
# Update our received bytes counter
self.rxb += len(data)
# And send the data packet to the Transport
# instance for processing.
self.owner.inbound(data, self)
# The running Reticulum Transport instance will
# call this method on the interface whenever the
# interface must transmit a packet.
def process_outgoing(self,data):
if self.online:
# First, escape and packetize the data
# according to HDLC framing.
data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
# Then write the framed data to the port
written = self.serial.write(data)
# Update the transmitted bytes counter
# and ensure that all data was written
self.txb += len(data)
if written != len(data):
raise IOError("Serial interface only wrote "+str(written)+" bytes of "+str(len(data)))
# This read loop runs in a thread and continously
# receives bytes from the underlying serial port.
# When a full packet has been received, it will
# be sent to the process_incoming methed, which
# will in turn pass it to the Transport instance.
def read_loop(self):
try:
in_frame = False
escape = False
data_buffer = b""
last_read_ms = int(time.time()*1000)
while self.serial.is_open:
if self.serial.in_waiting:
byte = ord(self.serial.read(1))
last_read_ms = int(time.time()*1000)
if (in_frame and byte == HDLC.FLAG):
in_frame = False
self.process_incoming(data_buffer)
elif (byte == HDLC.FLAG):
in_frame = True
data_buffer = b""
elif (in_frame and len(data_buffer) < self.HW_MTU):
if (byte == HDLC.ESC):
escape = True
else:
if (escape):
if (byte == HDLC.FLAG ^ HDLC.ESC_MASK):
byte = HDLC.FLAG
if (byte == HDLC.ESC ^ HDLC.ESC_MASK):
byte = HDLC.ESC
escape = False
data_buffer = data_buffer+bytes([byte])
else:
time_since_last = int(time.time()*1000) - last_read_ms
if len(data_buffer) > 0 and time_since_last > self.timeout:
data_buffer = b""
in_frame = False
escape = False
sleep(0.08)
except Exception as e:
self.online = False
RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
if RNS.Reticulum.panic_on_interface_error:
RNS.panic()
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
self.online = False
self.serial.close()
self.reconnect_port()
# This method handles serial port disconnects.
def reconnect_port(self):
while not self.online:
try:
time.sleep(5)
RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
self.open_port()
if self.serial.is_open:
self.configure_device()
except Exception as e:
RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
RNS.log("Reconnected serial port for "+str(self))
# Signal to Reticulum that this interface should
# not perform any ingress limiting.
def should_ingress_limit(self):
return False
# We must provide a string representation of this
# interface, that is used whenever the interface
# is printed in logs or external programs.
def __str__(self):
return "ExampleInterface["+self.name+"]"
# Finally, register the defined interface class as the
# target class for Reticulum to use as an interface
interface_class = ExampleInterface

View file

@ -224,7 +224,7 @@ def client(destination_hexhash, configpath):
destination_hash = bytes.fromhex(destination_hexhash)
except:
RNS.log("Invalid destination entered. Check your input!\n")
sys.exit(0)
exit()
# We must first initialise Reticulum
reticulum = RNS.Reticulum(configpath)
@ -462,7 +462,7 @@ def filelist_timeout_job():
global server_files
if len(server_files) == 0:
RNS.log("Timed out waiting for filelist, exiting")
sys.exit(0)
os._exit(0)
# When a link is closed, we'll inform the
@ -475,8 +475,9 @@ def link_closed(link):
else:
RNS.log("Link closed, exiting now")
RNS.Reticulum.exit_handler()
time.sleep(1.5)
sys.exit(0)
os._exit(0)
# When RNS detects that the download has
# started, we'll update our menu state
@ -600,4 +601,4 @@ if __name__ == "__main__":
except KeyboardInterrupt:
print("")
sys.exit(0)
exit()

View file

@ -133,7 +133,7 @@ def client(destination_hexhash, configpath):
destination_hash = bytes.fromhex(destination_hexhash)
except:
RNS.log("Invalid destination entered. Check your input!\n")
sys.exit(0)
exit()
# We must first initialise Reticulum
reticulum = RNS.Reticulum(configpath)
@ -245,8 +245,9 @@ def link_closed(link):
else:
RNS.log("Link closed, exiting now")
RNS.Reticulum.exit_handler()
time.sleep(1.5)
sys.exit(0)
os._exit(0)
# When a packet is received over the link, we
# simply print out the data.
@ -310,4 +311,4 @@ if __name__ == "__main__":
except KeyboardInterrupt:
print("")
sys.exit(0)
exit()

View file

@ -42,9 +42,6 @@ def server(configpath):
"linkexample"
)
# We can optionally configure a link request stamp cost
server_destination.set_link_cost(14.2)
# We configure a function that will get called every time
# a new client creates a link to this destination.
server_destination.set_link_established_callback(client_connected)
@ -122,7 +119,7 @@ def client(destination_hexhash, configpath):
destination_hash = bytes.fromhex(destination_hexhash)
except:
RNS.log("Invalid destination entered. Check your input!\n")
sys.exit(0)
exit()
# We must first initialise Reticulum
reticulum = RNS.Reticulum(configpath)
@ -225,8 +222,9 @@ def link_closed(link):
else:
RNS.log("Link closed, exiting now")
RNS.Reticulum.exit_handler()
time.sleep(1.5)
sys.exit(0)
os._exit(0)
# When a packet is received over the link, we
# simply print out the data.
@ -290,4 +288,4 @@ if __name__ == "__main__":
except KeyboardInterrupt:
print("")
sys.exit(0)
exit()

View file

@ -5,7 +5,6 @@
##########################################################
import argparse
import sys
import RNS
# Let's define an app name. We'll use this for all
@ -99,4 +98,4 @@ if __name__ == "__main__":
except KeyboardInterrupt:
print("")
sys.exit(0)
exit()

View file

@ -5,7 +5,6 @@
##########################################################
import argparse
import sys
import RNS
# Let's define an app name. We'll use this for all
@ -26,6 +25,9 @@ def server(configpath):
# We must first initialise Reticulum
reticulum = RNS.Reticulum(configpath)
# TODO: Remove
RNS.loglevel = RNS.LOG_DEBUG
# Randomly create a new identity for our echo server
server_identity = RNS.Identity()
@ -139,7 +141,7 @@ def client(destination_hexhash, configpath, timeout=None):
except Exception as e:
RNS.log("Invalid destination entered. Check your input!")
RNS.log(str(e)+"\n")
sys.exit(0)
exit()
# We must first initialise Reticulum
reticulum = RNS.Reticulum(configpath)
@ -338,4 +340,4 @@ if __name__ == "__main__":
client(args.destination, configarg, timeout=timeoutarg)
except KeyboardInterrupt:
print("")
sys.exit(0)
exit()

View file

@ -1,6 +1,6 @@
##########################################################
# This RNS example demonstrates how to perform requests #
# and receive responses over a link. #
# This RNS example demonstrates how to set perform #
# requests and receive responses over a link. #
##########################################################
import os
@ -119,7 +119,7 @@ def client(destination_hexhash, configpath):
destination_hash = bytes.fromhex(destination_hexhash)
except:
RNS.log("Invalid destination entered. Check your input!\n")
sys.exit(0)
exit()
# We must first initialise Reticulum
reticulum = RNS.Reticulum(configpath)
@ -226,8 +226,9 @@ def link_closed(link):
else:
RNS.log("Link closed, exiting now")
RNS.Reticulum.exit_handler()
time.sleep(1.5)
sys.exit(0)
os._exit(0)
##########################################################
@ -283,4 +284,4 @@ if __name__ == "__main__":
except KeyboardInterrupt:
print("")
sys.exit(0)
exit()

View file

@ -1,301 +0,0 @@
##########################################################
# This RNS example demonstrates how to transfer a #
# resource over an established link #
##########################################################
import os
import sys
import time
import random
import argparse
import RNS
# Let's define an app name. We'll use this for all
# destinations we create. Since this echo example
# is part of a range of example utilities, we'll put
# them all within the app namespace "example_utilities"
APP_NAME = "example_utilities"
##########################################################
#### Server Part #########################################
##########################################################
# A reference to the latest client link that connected
latest_client_link = None
# This initialisation is executed when the users chooses
# to run as a server
def server(configpath):
# We must first initialise Reticulum
reticulum = RNS.Reticulum(configpath)
# Randomly create a new identity for our link example
server_identity = RNS.Identity()
# We create a destination that clients can connect to. We
# want clients to create links to this destination, so we
# need to create a "single" destination type.
server_destination = RNS.Destination(
server_identity,
RNS.Destination.IN,
RNS.Destination.SINGLE,
APP_NAME,
"resourceexample"
)
# We configure a function that will get called every time
# a new client creates a link to this destination.
server_destination.set_link_established_callback(client_connected)
# Everything's ready!
# Let's Wait for client resources or user input
server_loop(server_destination)
def server_loop(destination):
# Let the user know that everything is ready
RNS.log(
"Resource example "+
RNS.prettyhexrep(destination.hash)+
" running, waiting for a connection."
)
RNS.log("Hit enter to manually send an announce (Ctrl-C to quit)")
# We enter a loop that runs until the users exits.
# If the user hits enter, we will announce our server
# destination on the network, which will let clients
# know how to create messages directed towards it.
while True:
entered = input()
destination.announce()
RNS.log("Sent announce from "+RNS.prettyhexrep(destination.hash))
# When a client establishes a link to our server
# destination, this function will be called with
# a reference to the link.
def client_connected(link):
global latest_client_link
RNS.log("Client connected")
# We configure the link to accept all resources
# and set a callback for completed resources
link.set_resource_strategy(RNS.Link.ACCEPT_ALL)
link.set_resource_concluded_callback(resource_concluded)
link.set_link_closed_callback(client_disconnected)
latest_client_link = link
def client_disconnected(link):
RNS.log("Client disconnected")
def resource_concluded(resource):
if resource.status == RNS.Resource.COMPLETE:
RNS.log(f"Resource {resource} received")
RNS.log(f"Metadata: {resource.metadata}")
RNS.log(f"Application flag is {'set' if resource.app_flag else 'not set'}")
RNS.log(f"Data length: {os.stat(resource.data.name).st_size}")
RNS.log(f"Data can be read directly from: {resource.data}")
RNS.log(f"Data can be moved or copied from: {resource.data.name}")
RNS.log(f"First 32 bytes of data: {RNS.hexrep(resource.data.read(32))}")
else:
RNS.log(f"Receiving resource {resource} failed")
##########################################################
#### Client Part #########################################
##########################################################
# A reference to the server link
server_link = None
def random_text_generator():
texts = ["They looked up", "On each full moon", "Becky was upset", "I’ll stay away from it", "The pet shop stocks everything"]
return texts[random.randint(0, len(texts)-1)]
# This initialisation is executed when the users chooses
# to run as a client
def client(destination_hexhash, configpath):
# We need a binary representation of the destination
# hash that was entered on the command line
try:
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
if len(destination_hexhash) != dest_len:
raise ValueError(
"Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2)
)
destination_hash = bytes.fromhex(destination_hexhash)
except:
RNS.log("Invalid destination entered. Check your input!\n")
sys.exit(0)
# We must first initialise Reticulum
reticulum = RNS.Reticulum(configpath)
# Check if we know a path to the destination
if not RNS.Transport.has_path(destination_hash):
RNS.log("Destination is not yet known. Requesting path and waiting for announce to arrive...")
RNS.Transport.request_path(destination_hash)
while not RNS.Transport.has_path(destination_hash):
time.sleep(0.1)
# Recall the server identity
server_identity = RNS.Identity.recall(destination_hash)
# Inform the user that we'll begin connecting
RNS.log("Establishing link with server...")
# When the server identity is known, we set
# up a destination
server_destination = RNS.Destination(
server_identity,
RNS.Destination.OUT,
RNS.Destination.SINGLE,
APP_NAME,
"resourceexample"
)
# And create a link
link = RNS.Link(server_destination)
# We'll set up functions to inform the
# user when the link is established or closed
link.set_link_established_callback(link_established)
link.set_link_closed_callback(link_closed)
# Everything is set up, so let's enter a loop
# for the user to interact with the example
client_loop()
def client_loop():
global server_link
# Wait for the link to become active
while not server_link:
time.sleep(0.1)
should_quit = False
while not should_quit:
try:
print("> ", end=" ")
text = input()
# Check if we should quit the example
if text == "quit" or text == "q" or text == "exit":
should_quit = True
server_link.teardown()
else:
# Generate 32 megabytes of random data
data = os.urandom(32*1024*1024)
RNS.log(f"Data length: {len(data)}")
RNS.log(f"First 32 bytes of data: {RNS.hexrep(data[:32])}")
# Generate some metadata
metadata = {"text": random_text_generator(),
"numbers": [1,2,3,4], "blob": os.urandom(16)}
# An optional application-specific flag can be set
flag_set = True
# Send the resource
resource = RNS.Resource(data, server_link, metadata=metadata, app_flag=flag_set,
callback=resource_concluded_sending, auto_compress=False)
# Alternatively, you can stream data
# directly from an open file descriptor
# with open("/path/to/file", "rb") as data_file:
# resource = RNS.Resource(data_file, server_link, metadata=metadata,
# callback=resource_concluded_sending, auto_compress=False)
except Exception as e:
RNS.log("Error while sending resource over the link: "+str(e))
should_quit = True
server_link.teardown()
def resource_concluded_sending(resource):
if resource.status == RNS.Resource.COMPLETE: RNS.log(f"The resource {resource} was sent successfully")
else: RNS.log(f"Sending the resource {resource} failed")
# This function is called when a link
# has been established with the server
def link_established(link):
# We store a reference to the link
# instance for later use
global server_link
server_link = link
# Inform the user that the server is
# connected
RNS.log("Link established with server, hit enter to send a resource, or type in \"quit\" to quit")
# When a link is closed, we'll inform the
# user, and exit the program
def link_closed(link):
if link.teardown_reason == RNS.Link.TIMEOUT:
RNS.log("The link timed out, exiting now")
elif link.teardown_reason == RNS.Link.DESTINATION_CLOSED:
RNS.log("The link was closed by the server, exiting now")
else:
RNS.log("Link closed, exiting now")
time.sleep(1.5)
sys.exit(0)
##########################################################
#### Program Startup #####################################
##########################################################
# This part of the program runs at startup,
# and parses input of from the user, and then
# starts up the desired program mode.
if __name__ == "__main__":
try:
parser = argparse.ArgumentParser(description="Simple resource example")
parser.add_argument(
"-s",
"--server",
action="store_true",
help="wait for incoming resources from clients"
)
parser.add_argument(
"--config",
action="store",
default=None,
help="path to alternative Reticulum config directory",
type=str
)
parser.add_argument(
"destination",
nargs="?",
default=None,
help="hexadecimal hash of the server destination",
type=str
)
args = parser.parse_args()
if args.config:
configarg = args.config
else:
configarg = None
if args.server:
server(configarg)
else:
if (args.destination == None):
print("")
parser.print_help()
print("")
else:
client(args.destination, configarg)
except KeyboardInterrupt:
print("")
sys.exit(0)

View file

@ -149,6 +149,8 @@ def server_packet_received(message, packet):
time.sleep(0.2)
rc = 0
received_data = 0
# latest_client_link.teardown()
# os._exit(0)
##########################################################
@ -157,7 +159,6 @@ def server_packet_received(message, packet):
# A reference to the server link
server_link = None
should_quit = False
# This initialisation is executed when the users chooses
# to run as a client
@ -174,7 +175,7 @@ def client(destination_hexhash, configpath):
destination_hash = bytes.fromhex(destination_hexhash)
except:
RNS.log("Invalid destination entered. Check your input!\n")
sys.exit(0)
exit()
# We must first initialise Reticulum
reticulum = RNS.Reticulum(configpath)
@ -215,7 +216,7 @@ def client(destination_hexhash, configpath):
client_loop()
def client_loop():
global server_link, should_quit
global server_link
# Wait for the link to become active
while not server_link:
@ -223,7 +224,16 @@ def client_loop():
should_quit = False
while not should_quit:
time.sleep(0.2)
try:
text = input()
# Check if we should quit the example
if text == "quit" or text == "q" or text == "exit":
should_quit = True
server_link.teardown()
except Exception as e:
raise e
# This function is called when a link
# has been established with the server
@ -236,10 +246,10 @@ def link_established(link):
# Inform the user that the server is
# connected
RNS.log("Link established with server, sending...")
rd = os.urandom(link.mdu)
RNS.log("Link established with server,sending...")
rd = os.urandom(RNS.Link.MDU)
started = time.time()
while link.status != RNS.Link.CLOSED and data_sent < data_cap*1.25:
while link.status == RNS.Link.ACTIVE and data_sent < data_cap*1.25:
RNS.Packet(server_link, rd, create_receipt=False).send()
data_sent += len(rd)
@ -266,17 +276,17 @@ def link_established(link):
# When a link is closed, we'll inform the
# user, and exit the program
def link_closed(link):
global should_quit
if link.teardown_reason == RNS.Link.TIMEOUT:
RNS.log("The link timed out, exiting now")
elif link.teardown_reason == RNS.Link.DESTINATION_CLOSED:
RNS.log("The link was closed by the server, exiting now")
else:
RNS.log("Link closed, exiting now")
RNS.Reticulum.exit_handler()
should_quit = True
time.sleep(1.5)
sys.exit(0)
os._exit(0)
def client_packet_received(message, packet):
pass
@ -334,4 +344,4 @@ if __name__ == "__main__":
except KeyboardInterrupt:
print("")
sys.exit(0)
exit()

View file

@ -1,7 +0,0 @@
{
"drips": {
"ethereum": {
"ownedBy": "0xae89F3B94fC4AD6563F0864a55F9a697a90261ff"
}
}
}

View file

@ -1,3 +1,2 @@
liberapay: Reticulum
ko_fi: markqvist
custom: "https://unsigned.io/donate"
custom: "https://unsigned.io/donate"

16
LICENSE
View file

@ -1,6 +1,6 @@
Reticulum License
MIT License, unless otherwise noted
Copyright (c) 2016-2026 Mark Qvist
Copyright (c) 2016-2024 Mark Qvist / unsigned.io
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
- The Software shall not be used in any kind of system which includes amongst
its functions the ability to purposefully do harm to human beings.
- The Software shall not be used, directly or indirectly, in the creation of
an artificial intelligence, machine learning or language model training
dataset, including but not limited to any use that contributes to the
training or development of such a model or algorithm.
- The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

View file

@ -1,33 +0,0 @@
This repository is a public mirror. All potential future development is happening elsewhere.
I am stepping back from all public-facing interaction with this project. Reticulum has always been primarily my work, and continuing in the current public, internet-facing model is no longer sustainable.
The software remains available for use as-is. Occasional updates may appear at unpredictable intervals, but there will be no support, no responses to issues, no discussions, and no community management in this or any other public venue. If it doesn't work for you, it doesn't work. That is the entire extent of available troubleshooting assistance I can offer you.
If you've followed this project for a while, you already know what this means. You know who designed, wrote and tested this, and you know how many years of my life it took. You'll also know about both my particular challenges and strengths, and how I believe anything worth building needs to be built and maintained with our own hands.
Seven months ago, I said I needed to step back, that I was exhausted, and that I needed to recover. I believed a public resolve would be enough to effectuate that, but while striving to get just a few more useful features and protocols out, the unproductive requests and demands also ramped up, and I got pulled back into the same patterns and draining interactions that I'd explicitly said I couldn't sustain anymore.
So here's what you might have already guessed: I'm done playing the game by rules I can't win at.
Everything you need is right here, and by any sensible measure, it's done. Anyone who wants to invest the time, skill and persistence can build on it, or completely re-imagine it with different priorities. That was always the point.
The people who actually contributed - you know who you are, and you know I mean it when I say: Thank you. All of you who've used this to build something real - that was the goal, and you did it without needing me to hold your hand.
The rest of you: You have what you need. Use it or don't. I am not going to be the person who explains it to you anymore.
This is not a temporary break. It's not "see you after some rest", but a recognition that the current model is fundamentally incompatible with my life, my health, and my reality.
If you want to support continued work, you can do so at the donation links listed in this repository. But please understand, that this is not purchasing support or guaranteeing updates. It is support for work that happens on my timeline, according to my capacity, which at the moment is not what it was.
If you want Reticulum to continue evolving, you have the power to make that happen. The protocol is public domain. The code is open source. Everything you need is right here. I've provided the tools, but building what comes next is not my responsibility anymore. It's yours.
To the small group of people who has actually been here, and understood what this work was and what it cost - you already know where to find me if it actually matters.
To everyone else: This is where we part ways. No hard feelings. It's just time.
---
असतो मा सद्गमय
तमसो मा ज्योतिर्गमय
मृत्योर्मा अमृतं गमय

View file

@ -21,29 +21,9 @@ clean:
@-rm -rf ./tests/__pycache__
@-rm -rf ./tests/rnsconfig/storage
@-rm -rf ./*.egg-info
@-rm -rf ./RNS/*.c
@-rm -rf ./RNS/Cryptography/*.c
@-rm -rf ./RNS/Cryptography/aes/*.c
@-rm -rf ./RNS/Cryptography/pure25519/*.c
@-rm -rf ./RNS/Interfaces/*.c
@-rm -rf ./RNS/Interfaces/util/*.c
@-rm -rf ./RNS/Interfaces/Android/*.c
@-rm -rf ./RNS/Utilities/*.c
@-rm -rf ./RNS/Utilities/rngit/*.c
@-rm -rf ./RNS/Utilities/rnsh/*.c
@-rm -rf ./RNS/vendor/*.c
@-rm -rf ./RNS/vendor/i2plib/*.c
@-rm -rf
@make -C docs clean
@echo Done
purge_docs:
@echo Purging documentation build...
@-rm -rf ./docs/manual
@-rm -rf ./docs/markdown
@-rm -rf ./docs/*.pdf
@-rm -rf ./docs/*.epub
remove_symlinks:
@echo Removing symlinks for build...
-rm Examples/RNS
@ -54,57 +34,28 @@ create_symlinks:
-ln -s ../RNS ./Examples/
-ln -s ../../RNS ./RNS/Utilities/
build_sdist: purge_docs
build_sdist_only:
python3 setup.py sdist
build_wheel:
python3 setup.py bdist_wheel
python3 setup.py sdist bdist_wheel
build_pure_wheel:
python3 setup.py bdist_wheel --pure
build_native:
python3 setup.py bdist_wheel --native
python3 setup.py sdist bdist_wheel --pure
documentation:
make -C docs html markdown
make -C docs html
manual:
make -C docs latexpdf epub
distcollect:
mv docs/Reticulum\ Manual.* dist
build_spkg: remove_symlinks build_sdist create_symlinks
release: test remove_symlinks build_sdist build_wheel build_pure_wheel documentation manual distcollect create_symlinks
release: test remove_symlinks build_wheel build_pure_wheel documentation manual create_symlinks
debug: remove_symlinks build_wheel build_pure_wheel create_symlinks
native: remove_symlinks build_native create_symlinks
local: release sign
sign:
rngit release rns://7649a50d84610232d1416b41d2896aff/reticulum/reticulum create $$(python setup.py --getversion):dist --name rns --local
upload:
@echo Ready to publish release over Reticulum
@read VOID
rngit release rns://7649a50d84610232d1416b41d2896aff/reticulum/reticulum create $$(python setup.py --getversion):dist --name rns
upload-pip: upload-rns-pip upload-rnspure-pip
upload-rns-pip:
@echo Ready to publish rns release, hit enter to continue
@echo Ready to publish release, hit enter to continue
@read VOID
@echo Uploading to PyPi...
twine upload dist/rns-*.whl dist/rns-*.tar.gz
twine upload dist/*
@echo Release published
upload-rnspure-pip:
@echo Ready to publish rnspure release, hit enter to continue
@read VOID
@echo Uploading to PyPi...
twine upload dist/rnspure-*.whl
@echo Release published

377
README.md
View file

@ -1,46 +1,47 @@
Reticulum Network Stack <img align="right" src="https://static.pepy.tech/personalized-badge/rns?period=month&units=international_system&left_color=grey&right_color=blue&left_text=Installs/month" style="padding-left:10px"/><a href="https://github.com/markqvist/Reticulum/actions/workflows/build.yml"><img align="right" src="https://github.com/markqvist/Reticulum/actions/workflows/build.yml/badge.svg"/></a>
Reticulum Network Stack β <img align="right" src="https://static.pepy.tech/personalized-badge/rns?period=total&units=international_system&left_color=grey&right_color=blue&left_text=Installs" style="padding-left:10px"/><a href="https://github.com/markqvist/reticulum/actions/workflows/python-app.yml"><img align="right" src="https://github.com/markqvist/reticulum/actions/workflows/python-app.yml/badge.svg"/></a>
==========
> [!WARNING]
> A significant number of hastily launched, incorrect, dubious, directly harmful and, in most cases, entirely LLM-generated fakes of Reticulum implementations, "documentation", "guides" and user-facing applications are currently being marketed under polished, but ultimately misleading or outright deceptive claims.
>
> For more detailed information, see the relevant sections of the [Brandolini's Reference](https://reticulum.network/manual/brandolinis.html) chapter of the manual. Known good community implementations are listed in the [Community Implementations](#community-implementations) section of this readme. For software and projects recognized by the Reticulum community, see the [Programs Using Reticulum](https://reticulum.network/manual/software.html) chapter of the manual.
>
> Machine-generated content derived from the RNS source code or documentation (such as "implementations", "guides" for sale, et cetera), are in several, well-documented cases violating the open source licenses that granted rights of copying and distribution.
>
> Any claimed assertion of copyright or grant of license (Apache, GPL, et cetera) that such projects are making are [void grants](https://reticulum.network/manual/brandolinis.html#void-grants-legal-foundations-of-machine-generated-code). Both the violators themselves, and all downstream projects using the infringing derivatives, are **directly** and personally liable for the legal consequences.
>
> Do **not** use these "projects", they are a hazard to the entire ecosystem we have been carefully building over the last ten years.
<p align="center"><img width="200" src="https://raw.githubusercontent.com/markqvist/Reticulum/master/docs/source/graphics/rns_logo_512.png"></p>
*This repository is [a public mirror](./MIRROR.md). All development is happening elsewhere.*
Reticulum is the cryptography-based networking stack for building local and wide-area
networks with readily available hardware. It can operate even with very high latency
and extremely low bandwidth. Reticulum allows you to build wide-area networks
with off-the-shelf tools, and offers end-to-end encryption and connectivity,
initiator anonymity, autoconfiguring cryptographically backed multi-hop
transport, efficient addressing, unforgeable delivery acknowledgements and
more.
To understand the foundational philosophy and goals of this system, read the [Zen of Reticulum](Zen%20of%20Reticulum.md).
The vision of Reticulum is to allow anyone to be their own network operator,
and to make it cheap and easy to cover vast areas with a myriad of independent,
inter-connectable and autonomous networks. Reticulum **is not** *one* network.
It is **a tool** for building *thousands of networks*. Networks without
kill-switches, surveillance, censorship and control. Networks that can freely
interoperate, associate and disassociate with each other, and require no
central oversight. Networks for human beings. *Networks for the people*.
Reticulum is the cryptography-based networking stack for building local and wide-area networks with readily available hardware. It can operate even with very high latency and extremely low bandwidth. Reticulum allows you to build wide-area networks with off-the-shelf tools, and offers end-to-end encryption and connectivity, initiator anonymity, autoconfiguring cryptographically backed multi-hop transport, efficient addressing, unforgeable delivery acknowledgements and more.
Reticulum is a complete networking stack, and does not rely on IP or higher
layers, but it is possible to use IP as the underlying carrier for Reticulum.
It is therefore trivial to tunnel Reticulum over the Internet or private IP
networks.
The vision of Reticulum is to allow anyone to be their own network operator, and to make it cheap and easy to cover vast areas with a myriad of independent, inter-connectable and autonomous networks. Reticulum **is not** *one* network. It is **a tool** for building *thousands of networks*. Networks without kill-switches, surveillance, censorship and control. Networks that can freely interoperate, associate and disassociate with each other, and require no central oversight. Networks for human beings. *Networks for the people*.
Having no dependencies on traditional networking stacks frees up overhead that
has been used to implement a networking stack built directly on cryptographic
principles, allowing resilience and stable functionality, even in open and
trustless networks.
Reticulum is a complete networking stack, and does not rely on IP or higher layers, but it is possible to use IP as the underlying carrier for Reticulum. It is therefore trivial to tunnel Reticulum over the Internet or private IP networks.
Having no dependencies on traditional networking stacks frees up overhead that has been used to implement a networking stack built directly on cryptographic principles, allowing resilience and stable functionality, even in open and trustless networks.
No kernel modules or drivers are required. Reticulum runs completely in userland, and can run on practically any system that runs Python 3.
No kernel modules or drivers are required. Reticulum runs completely in
userland, and can run on practically any system that runs Python 3.
## Read The Manual
The full documentation for Reticulum is available at [markqvist.github.io/Reticulum/manual/](https://markqvist.github.io/Reticulum/manual/).
You can also download the [Reticulum manual as a PDF](https://github.com/markqvist/Reticulum/releases/latest/download/Reticulum.Manual.pdf) or [as an e-book in EPUB format](https://github.com/markqvist/Reticulum/releases/latest/download/Reticulum.Manual.epub).
You can also download the [Reticulum manual as a PDF](https://github.com/markqvist/Reticulum/raw/master/docs/Reticulum%20Manual.pdf) or [as an e-book in EPUB format](https://github.com/markqvist/Reticulum/raw/master/docs/Reticulum%20Manual.epub).
For more info, see [reticulum.network](https://reticulum.network/) and [the FAQ section of the wiki](https://github.com/markqvist/Reticulum/wiki/Frequently-Asked-Questions).
## Notable Features
- Coordination-less globally unique addressing and identification
- Fully self-configuring multi-hop routing over heterogeneous carriers
- Flexible scalability over heterogeneous topologies
- Reticulum can carry data over any mixture of physical mediums and topologies
- Low-bandwidth networks can co-exist and interoperate with large, high-bandwidth networks
- Initiator anonymity, communicate without revealing your identity
- Reticulum does not include source addresses on any packets
- Asymmetric X25519 encryption and Ed25519 signatures as a basis for all communication
@ -48,18 +49,12 @@ For more info, see [reticulum.network](https://reticulum.network/) and [the FAQ
- Forward Secrecy is available for all communication types, both for single packets and over links
- Reticulum uses the following format for encrypted tokens:
- Ephemeral per-packet and link keys and derived from an ECDH key exchange on Curve25519
- AES-256 in CBC mode with PKCS7 padding
- AES-128 in CBC mode with PKCS7 padding
- HMAC using SHA256 for authentication
- IVs are generated through os.urandom()
- Unforgeable packet delivery confirmations
- Flexible and extensible interface system
- Reticulum includes a large variety of built-in interface types
- Ability to load and utilise custom user- or community-supplied interface types
- Easily create your own custom interfaces for communicating over anything
- Authentication and virtual network segmentation on all supported interface types
- A large variety of supported interface types
- An intuitive and easy-to-use API
- Simpler and easier to use than sockets APIs, but more powerful
- Makes building distributed and decentralised applications much simpler
- Reliable and efficient transfer of arbitrary amounts of data
- Reticulum can handle a few bytes of data or files of many gigabytes
- Sequencing, compression, transfer coordination and checksumming are automatic
@ -70,88 +65,65 @@ For more info, see [reticulum.network](https://reticulum.network/) and [the FAQ
- Low cost of keeping links open at only 0.44 bits per second
- Reliable sequential delivery with Channel and Buffer mechanisms
## Community Resources
## Roadmap
While Reticulum is already a fully featured and functional networking stack,
many improvements and additions are actively being worked on, and planned for the future.
The references listed here are not run, managed or moderated by the Reticulum project or its maintainer. They are provided by dedicated, independent volunteers in the community, for the benefit of anyone interested in learning more about Reticulum.
- [Reticulum Community Wiki](https://reticulum.miraheze.org/wiki)
- [RNS Recipes](https://rns.recipes/)
- [RNS Moscow](https://rns.moscow/)
- [RMAP.world](https://rmap.world/)
If you find anything that is no longer accurate, incorrect or simply not quite up to date, do not demand or complain towards the maintainers of these resources. Instead, take the task upon yourself to help out and improve things by working on, and offering the contributions you would like to see, like the creators of the above did themselves.
## Reference Implementation
The Python code in this repository is the Reference Implementation of Reticulum. The Reticulum Protocol is defined entirely and authoritatively by this reference implementation, and its associated manual. It is maintained by Mark Qvist, identified by the Reticulum Identity `<bc7291552be7a58f361522990465165c>`.
No second-order protocol specification derivatives currently do, or will ever exist. You can't get a better specification than one that is executable, functional, testable, deployed and operating 24/7 across the planet.
There is not, and never will be a separate, derived "formal spec", "RFC" or anything similar for Reticulum:
**The reference implementation *is* the final, authoritative specification.**
Compatibility with the Reticulum Protocol is defined as having full interoperability, and sufficient functional parity with this reference implementation. Any specific protocol implementation that achieves this is Reticulum. Any that does not is not Reticulum.
The reference implementation is licensed under the Reticulum License.
The Reticulum Protocol was dedicated to the Public Domain in 2016.
## Community Implementations
The implementations listed here have demonstrated wire-compatibility, sufficient feature parity, sensible security and development practices, long-term commitment and care for implementation quality. Additionally, they have a track record of at least 18 months of active development.
Most importantly, their developers and maintainers are recognized by the community to actually know what they are doing, and as having a commitment to improving the overall Reticulum ecosystem.
- [Chad Atterman's](https://github.com/attermann) [microReticulum](https://github.com/attermann/microReticulum/) is a C++ implementation targeting 32-bit micro-controllers.
- [Quad4's](https://quad4.io/) [Reticulum-Go](https://reticulum-go.quad4.io/) is a Go implementation targeting both desktop, mobile and embedded systems.
*If you are the maintainer of an implementation not listed above, and believe that it deserves inclusion by following the same standards of rigor and commitment, your proposal will be welcomed.*
---
**Warning!** A significant number of hastily launched, incorrect, dubious, directly harmful and, in most cases, entirely LLM-generated fakes of Reticulum implementations (and user facing applications) are currently being marketed and pushed under polished, but ultimately misleading or outright deceptive claims. Most of these are either quick attention or money grabs, completely unreviewed, incompatible, without maintenance or human oversight, actively trying to fragment the community, or a combination thereof.
---
For more extensive information on this topic, read the [Brandolini's Reference](https://reticulum.network/manual/brandolinis.html) chapter of the manual (for the legal foundations, see [Void Grants](https://reticulum.network/manual/brandolinis.html#void-grants-legal-foundations-of-machine-generated-code) specifically). The chapter is very long, very detailed, and is the polite, measured analysis, intended as an evolving, sectioned reference on these issues.
If you would rather just have the short, unfiltered and quite angry version, which consists of an enacted response to one particular manifestation of the phenomenon as a whole, used as an example, you can read [Yes, I am Angry](docs/history/2026_09_18_Yes_I_am_Angry.md). It gets the same point across, in a very different language, if you can handle that sort of communication.
The core Reticulum maintainers and developers **strongly advise against using any such "implementation"**. The [deceitful and self-serving behavior](https://reticulum.network/manual/brandolinis.html#a-persistent-but-fragile-triad) of a small number of either malintented, attention-seeking or incompetent individuals has caused significant [damage and waste of time and resources](https://reticulum.network/manual/brandolinis.html#intent-consequence) for both developers, node operators, users and the community and Reticulum project at large.
We kindly, but urgently ask all users to excercise caution, common sense and healthy skepticism when [evaluating](https://reticulum.network/manual/brandolinis.html#evaluating-a-reticulum-implementation) the potential use of any implementation not listed above; and to accept that **choosing to do so regardless translates the responsibility for potential damage from an irresponsible or malicious individual onto yourself**.
To learn more about the direction and future of Reticulum, please see the [Development Roadmap](./Roadmap.md).
## Examples of Reticulum Applications
If you want to quickly get an idea of what Reticulum can do, take a look at the [Programs Using Reticulum](https://reticulum.network/manual/software.html) section of the manual, or the following resources:
If you want to quickly get an idea of what Reticulum can do, take a look at the
following resources.
- You can use the [rnsh](https://github.com/acehoss/rnsh) program to establish remote shell sessions over Reticulum.
- [LXMF](https://github.com/markqvist/lxmf) is a distributed, delay and disruption tolerant message transfer protocol built on Reticulum
- The [LXST](https://github.com/markqvist/lxst) protocol and framework provides real-time audio and signals transport over Reticulum. It includes primitives and utilities for building voice-based applications and hardware devices, such as the `rnphone` program, that can be used to build hardware telephones.
- For an off-grid, encrypted and resilient mesh communications platform, see [Nomad Network](https://github.com/markqvist/NomadNet)
- The Android, Linux, macOS and Windows app [Sideband](https://github.com/markqvist/Sideband) has a graphical interface and many advanced features, such as file transfers, image and voice messages, real-time voice calls, a distributed telemetry system, mapping capabilities and full plugin extensibility.
- [MeshChatX](https://git.quad4.io/RNS-Things/MeshChatX) is a full-featured LXMF client with many built-in tools and functionalities, that also supports image and voice messages, file transfers and voice calls. It also includes a built-in page browser for browsing Nomad Network nodes.
- You can use the included [rnsh](https://reticulum.network/manual/using.html#the-rnsh-utility) program to establish remote shell sessions over Reticulum.
- The included [rngit](https://reticulum.network/manual/git.html) system provides a full solution for managing, hosting, collaborating on and interacting with Git repositories over Reticulum.
- The Android, Linux, macOS and Windows app [Sideband](https://github.com/markqvist/Sideband) has a graphical interface and focuses on ease of use.
- [MeshChat](https://github.com/liamcottle/reticulum-meshchat) is a user-friendly LXMF client, that also supports voice calls.
## Where can Reticulum be used?
Over practically any medium that can support at least a half-duplex channel with greater throughput than 5 bits per second, and an MTU of 500 bytes. Data radios, modems, LoRa radios, serial lines, AX.25 TNCs, amateur radio digital modes, WiFi and Ethernet devices, free-space optical links, and similar systems are all examples of the types of physical devices Reticulum can use.
Over practically any medium that can support at least a half-duplex channel
with greater throughput than 5 bits per second, and an MTU of 500 bytes. Data radios,
modems, LoRa radios, serial lines, AX.25 TNCs, amateur radio digital modes,
WiFi and Ethernet devices, free-space optical links, and similar systems are
all examples of the types of physical devices Reticulum can use.
An open-source LoRa-based interface called [RNode](https://markqvist.github.io/Reticulum/manual/hardware.html#rnode) has been designed specifically for use with Reticulum. It is possible to build yourself, or it can be purchased as a complete transceiver that just needs a USB, WiFi or Bluetooth connection to the host.
An open-source LoRa-based interface called
[RNode](https://markqvist.github.io/Reticulum/manual/hardware.html#rnode) has
been designed specifically for use with Reticulum. It is possible to build
yourself, or it can be purchased as a complete transceiver that just needs a
USB connection to the host.
Reticulum can also be encapsulated over existing IP networks, so there's nothing stopping you from using it over wired Ethernet, your local WiFi network or the Internet, where it'll work just as well. In fact, one of the strengths of Reticulum is how easily it allows you to connect different mediums into a self-configuring, resilient and encrypted mesh, using any available mixture of available infrastructure.
Reticulum can also be encapsulated over existing IP networks, so there's
nothing stopping you from using it over wired Ethernet, your local WiFi network
or the Internet, where it'll work just as well. In fact, one of the strengths
of Reticulum is how easily it allows you to connect different mediums into a
self-configuring, resilient and encrypted mesh, using any available mixture of
available infrastructure.
As an example, it's possible to set up a Raspberry Pi connected to both a LoRa radio, a packet radio TNC and a WiFi network. Once the interfaces are configured, Reticulum will take care of the rest, and any device on the WiFi network can communicate with nodes on the LoRa and packet radio sides of the network, and vice versa.
As an example, it's possible to set up a Raspberry Pi connected to both a LoRa
radio, a packet radio TNC and a WiFi network. Once the interfaces are
configured, Reticulum will take care of the rest, and any device on the WiFi
network can communicate with nodes on the LoRa and packet radio sides of the
network, and vice versa.
## How do I get started?
The best way to get started with the Reticulum Network Stack depends on what you want to do. For full details and examples, have a look at the [Getting Started Fast](https://markqvist.github.io/Reticulum/manual/gettingstartedfast.html) section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
The best way to get started with the Reticulum Network Stack depends on what
you want to do. For full details and examples, have a look at the
[Getting Started Fast](https://markqvist.github.io/Reticulum/manual/gettingstartedfast.html)
section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
To simply install Reticulum and related utilities on your system, the easiest way is via `pip`. You can then start any program that uses Reticulum, or start Reticulum as a system service with [the rnsd utility](https://markqvist.github.io/Reticulum/manual/using.html#the-rnsd-utility).
To simply install Reticulum and related utilities on your system, the easiest way is via `pip`.
You can then start any program that uses Reticulum, or start Reticulum as a system service with
[the rnsd utility](https://markqvist.github.io/Reticulum/manual/using.html#the-rnsd-utility).
```bash
pip install rns
```
If you are using an operating system that blocks normal user package installation via `pip`, you can return `pip` to normal behaviour by editing the `~/.config/pip/pip.conf` file, and adding the following directive in the `[global]` section:
If you are using an operating system that blocks normal user package installation via `pip`,
you can return `pip` to normal behaviour by editing the `~/.config/pip/pip.conf` file,
and adding the following directive in the `[global]` section:
```text
[global]
@ -164,14 +136,23 @@ Alternatively, you can use the `pipx` tool to install Reticulum in an isolated e
pipx install rns
```
When first started, Reticulum will create a default configuration file, providing basic connectivity to other Reticulum peers that might be locally reachable. The default config file contains a few examples, and references for creating a more complex configuration.
When first started, Reticulum will create a default configuration file,
providing basic connectivity to other Reticulum peers that might be locally
reachable. The default config file contains a few examples, and references for
creating a more complex configuration.
If you have an old version of `pip` on your system, you may need to upgrade it first with `pip install pip --upgrade`. If you no not already have `pip` installed, you can install it using the package manager of your system with `sudo apt install python3-pip` or similar.
For more detailed examples on how to expand communication over many mediums such as packet radio or LoRa, serial ports, or over fast IP links and the Internet using the UDP and TCP interfaces, take a look at the [Supported Interfaces](https://markqvist.github.io/Reticulum/manual/interfaces.html) section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
For more detailed examples on how to expand communication over many mediums such
as packet radio or LoRa, serial ports, or over fast IP links and the Internet using
the UDP and TCP interfaces, take a look at the [Supported Interfaces](https://markqvist.github.io/Reticulum/manual/interfaces.html)
section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
## Included Utilities
Reticulum includes a range of useful utilities for managing your networks, viewing status and information, and other tasks. You can read more about these programs in the [Included Utility Programs](https://markqvist.github.io/Reticulum/manual/using.html#included-utility-programs) section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
Reticulum includes a range of useful utilities for managing your networks,
viewing status and information, and other tasks. You can read more about these
programs in the [Included Utility Programs](https://markqvist.github.io/Reticulum/manual/using.html#included-utility-programs)
section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
- The system daemon `rnsd` for running Reticulum as an always-available service
- An interface status utility called `rnstatus`, that displays information about interfaces
@ -179,20 +160,23 @@ Reticulum includes a range of useful utilities for managing your networks, viewi
- A diagnostics tool called `rnprobe` for checking connectivity to destinations
- A simple file transfer program called `rncp` making it easy to transfer files between systems
- The identity management and encryption utility `rnid` let's you manage Identities and encrypt/decrypt files
- The `rnsh` program allows you to establish fully interactive shell session with remote systems
- The remote command execution program `rnx` let's you run simple commands and programs and retrieve output from remote systems
- The `rngit` program provides a full multi-repository Git node for serving repositories over Reticulum
- The included `git-remote-rns` helper allows you to interact with Git repositories over Reticulum
- The remote command execution program `rnx` let's you run commands and
programs and retrieve output from remote systems
All tools, including `rnx` and `rncp`, work reliably and well even over very low-bandwidth links like LoRa or Packet Radio. For full-featured remote shells over Reticulum, also have a look at the [rnsh](https://github.com/acehoss/rnsh) program.
All tools, including `rnx` and `rncp`, work reliably and well even over very
low-bandwidth links like LoRa or Packet Radio. For full-featured remote shells
over Reticulum, also have a look at the [rnsh](https://github.com/acehoss/rnsh)
program.
## Supported interface types and devices
Reticulum implements a range of generalised interface types that covers most of the communications hardware that Reticulum can run over. If your hardware is not supported, it's [simple to implement a custom interface module](https://markqvist.github.io/Reticulum/manual/interfaces.html#custom-interfaces).
Reticulum implements a range of generalised interface types that covers most of
the communications hardware that Reticulum can run over. If your hardware is
not supported, it's relatively simple to implement an interface class. I will
gratefully accept pull requests for custom interfaces if they are generally
useful.
Contributions of custom interfaces are gratefully accepted, provided they are generally useful and well-tested in real-world usage.
Currently, the following built-in interfaces are supported:
Currently, the following interfaces are supported:
- Any Ethernet device
- LoRa using [RNode](https://unsigned.io/rnode/)
@ -205,71 +189,120 @@ Currently, the following built-in interfaces are supported:
- Custom hardware via stdio or pipes
## Performance
*All performance numbers here assume `rnsd` running in fully interpreted Python mode, limited to a single CPU core.*
Reticulum targets a *very* wide usable performance envelope, but prioritises
functionality and performance on low-bandwidth mediums. The goal is to
provide a dynamic performance envelope from 250 bits per second, to 1 gigabit
per second on normal hardware.
Reticulum targets a *very* wide usable performance envelope, but prioritises functionality and performance on low-bandwidth mediums. The goal is to provide a dynamic performance envelope from 100 bits per second, to 10 gigabits per second on normal hardware.
Currently, the on-network, usable performance envelope is approximately 150 bits per second to 1 gigabit per second on modest hardware. Real-world performance will vary widely with the speed of your system, availability of cryptographic hardware acceleration, and other factors.
Raw routing throughput for a transport node hosted on a Raspberry Pi 5 is approximately 5 gigabits per second. Systems with modern crypto hardware acceleration can achieve transport throughput of 15 gigabits per second and higher.
While the reference implementation is perfectly capable of reaching 100+ Gbps, by utilizing multiple CPU cores and vectorized transport, performance beyond the current level is not higly prioritised, since RNS is already able to saturate available physical medium bandwidth in almost all real-world situations.
The primary objectives for RNS are *reliability*, *correctness*, *security*, *proper traffic management* and *actually practical performance*, not hollow on-paper "benchmarks" with little to no bearing on real-world usability.
All alternative-implementation maintainers are *highly* encouraged to adopt a similar approach, and especially to focus on getting their implementations working properly and reliably before publishing empirically dubious or factually wrong performance claims.
Currently, the usable performance envelope is approximately 150 bits per second
to 40 megabits per second, with physical mediums faster than that not being
saturated. Performance beyond the current level is intended for future
upgrades, but not highly prioritised at this point in time.
## Current Status
All core protocol features are implemented and functioning, but additions will probably occur as real-world use is explored and understood. The API and wire-format can be considered stable.
Reticulum should currently be considered beta software. All core protocol
features are implemented and functioning, but additions will probably occur as
real-world use is explored. There will be bugs. The API and wire-format can be
considered relatively stable at the moment, but could change if warranted.
## Dependencies
The installation of the default `rns` package requires only two external dependencies, listed below. Almost all systems and distributions have readily available packages for these dependencies, and when the `rns` package is installed with `pip`, they will be downloaded and installed as well.
The installation of the default `rns` package requires the dependencies listed
below. Almost all systems and distributions have readily available packages for
these dependencies, and when the `rns` package is installed with `pip`, they
will be downloaded and installed as well.
- [PyCA/cryptography](https://github.com/pyca/cryptography)
- [pyserial](https://github.com/pyserial/pyserial)
On more unusual systems, and in some rare cases, it might not be possible to install or even compile one or more of the above modules. In such situations, you can use the `rnspure` package instead, which require no external dependencies for installation. Please note that the contents of the `rns` and `rnspure` packages are *identical*. The only difference is that the `rnspure` package lists no dependencies required for installation.
On more unusual systems, and in some rare cases, it might not be possible to
install or even compile one or more of the above modules. In such situations,
you can use the `rnspure` package instead, which require no external
dependencies for installation. Please note that the contents of the `rns` and
`rnspure` packages are *identical*. The only difference is that the `rnspure`
package lists no dependencies required for installation.
No matter how Reticulum is installed and started, it will load external dependencies only if they are *needed* and *available*. If for example you want to use Reticulum on a system that cannot support [pyserial](https://github.com/pyserial/pyserial), it is perfectly possible to do so using the `rnspure` package, but Reticulum will not be able to use serial-based interfaces. All other available modules will still be loaded when needed.
No matter how Reticulum is installed and started, it will load external
dependencies only if they are *needed* and *available*. If for example you want
to use Reticulum on a system that cannot support
[pyserial](https://github.com/pyserial/pyserial), it is perfectly possible to
do so using the `rnspure` package, but Reticulum will not be able to use
serial-based interfaces. All other available modules will still be loaded when
needed.
**Please Note!** If you use the `rnspure` package to run Reticulum on systems that do not support [PyCA/cryptography](https://github.com/pyca/cryptography), it is important that you read and understand the [Cryptographic Primitives](#cryptographic-primitives) section of this document.
## Bootstrapping Connectivity
Reticulum is not a service you subscribe to, nor is it a single global network you "join". Reticulum provides functionality for discovering available public interfaces over the network itself, and the broader community has provided various directories of publicly available entrypoints to bootstrap connectivity.
To learn how to establish initial connectivity over Reticulum, read the [Bootstrapping Connectivity](https://reticulum.network/manual/gettingstartedfast.html#bootstrapping-connectivity) section of the manual.
If you already have a general idea of how this works, you can use community-run sites such as [directory.rns.recipes](https://directory.rns.recipes/) and [rmap.world](https://rmap.world) to find interface definitions for initial connectivity to the global distributed Reticulum backbone.
**Please Note!** If you use the `rnspure` package to run Reticulum on systems
that do not support [PyCA/cryptography](https://github.com/pyca/cryptography),
it is important that you read and understand the [Cryptographic
Primitives](#cryptographic-primitives) section of this document.
## Public Testnet
***Important!** Historically, a developer-targeted testnet was made available by the Reticulum project itself. As the amount of global Reticulum nodes and entrypoints have grown to a substantial quantity, this public testnet, including the Amsterdam Testnet entrypoint, has now been decommissioned. If your still have instances that relied on this entrypoint for connectivity, transition to using the distributed backbone instead. Reticulum now includes a full on-network interface discovery and connectivity bootstrapping system. Read the [Bootstrapping Connectivity](https://reticulum.network/manual/gettingstartedfast.html#bootstrapping-connectivity) section of the manual for pointers.*
If you just want to get started experimenting without building any physical
networks, you are welcome to join the RNS Development Testnet.
The testnet is just that, an informal network for testing and experimenting.
It will be up most of the time, and anyone can join, but it also means that
there's no guarantees for service availability.
It probably goes without saying, but *don't use the testnet entry-points as
hardcoded or default interfaces in any applications you ship to users*. When
shipping applications, the best practice is to provide your own default
connectivity solutions, if needed and applicable, or in most cases, simply
leave it up to the user which networks to connect to, and how.
The testnet runs the very latest version of Reticulum (often even a short while
before it is publicly released). Sometimes experimental versions of Reticulum
might be deployed to nodes on the testnet, which means strange behaviour might
occur. If none of that scares you, you can join the testnet via either TCP or
I2P. Just add one of the following interfaces to your Reticulum configuration
file:
```
# TCP/IP interface to the RNS Amsterdam Hub
[[RNS Testnet Amsterdam]]
type = TCPClientInterface
enabled = yes
target_host = amsterdam.connect.reticulum.network
target_port = 4965
# TCP/IP interface to the BetweenTheBorders Hub (community-provided)
[[RNS Testnet BetweenTheBorders]]
type = TCPClientInterface
enabled = yes
target_host = reticulum.betweentheborders.com
target_port = 4242
# Interface to Testnet I2P Hub
[[RNS Testnet I2P Hub]]
type = I2PInterface
enabled = yes
peers = g3br23bvx3lq5uddcsjii74xgmn6y5q325ovrkq2zw2wbzbqgbuq.b32.i2p
```
The testnet also contains a number of [Nomad Network](https://github.com/markqvist/nomadnet) nodes, and LXMF propagation nodes.
## Support Reticulum
For this to be possible, I need your help. Please support the continued development of open, free and private communications systems by donating via one of the following channels:
You can help support the continued development of open, free and private communications systems by donating via one of the following channels:
- Monero:
```
84FpY1QbxHcgdseePYNmhTHcrgMX4nFfBYtz2GKYToqHVVhJp8Eaw1Z1EedRnKD19b3B8NiLCGVxzKV17UMmmeEsCrPyA5w
```
- Bitcoin
```
bc1pgqgu8h8xvj4jtafslq396v7ju7hkgymyrzyqft4llfslz5vp99psqfk3a6
```
- Ethereum
```
0x91C421DdfB8a30a49A71d63447ddb54cEBe3465E
0xFDabC71AC4c0C78C95aDDDe3B4FA19d6273c5E73
```
- Bitcoin
```
35G9uWVzrpJJibzUwpNUQGQNFzLirhrYAH
```
- Liberapay: https://liberapay.com/Reticulum/
- Ko-Fi: https://ko-fi.com/markqvist
Are certain features in the development roadmap are important to you or your
organisation? Make them a reality quickly by sponsoring their implementation.
## Cryptographic Primitives
Reticulum uses a simple suite of efficient, strong and well-tested cryptographic primitives, with widely available implementations that can be used both on general-purpose CPUs and on microcontrollers.
One of the primary considerations for choosing this particular set of primitives is that they can be implemented *safely* with relatively few pitfalls, on practically all current computing platforms.
The primitives listed here **are authoritative**. Anything claiming to be Reticulum, but not using these exact primitives **is not** Reticulum, and possibly an intentionally compromised or weakened clone. The utilised primitives are:
Reticulum uses a simple suite of efficient, strong and well-tested cryptographic
primitives, with widely available implementations that can be used both on
general-purpose CPUs and on microcontrollers. The utilised primitives are:
- Reticulum Identity Keys are 512-bit Curve25519 keysets
- A 256-bit Ed25519 key for signatures
@ -277,44 +310,64 @@ The primitives listed here **are authoritative**. Anything claiming to be Reticu
- HKDF for key derivation
- Encrypted tokens are based on the [Fernet spec](https://github.com/fernet/spec/)
- Ephemeral keys derived from an ECDH key exchange on Curve25519
- AES-128 in CBC mode with PKCS7 padding
- HMAC using SHA256 for message authentication
- IVs must be generated through `os.urandom()` or better
- AES-256 in CBC mode with PKCS7 padding
- IVs are generated through os.urandom()
- No Fernet version and timestamp metadata fields
- SHA-256
- SHA-512
In the default installation configuration, the `X25519`, `Ed25519`, and `AES-256-CBC` primitives are provided by [OpenSSL](https://www.openssl.org/) (via the [PyCA/cryptography](https://github.com/pyca/cryptography) package). The hashing functions `SHA-256` and `SHA-512` are provided by the standard Python [hashlib](https://docs.python.org/3/library/hashlib.html). The `HKDF`, `HMAC`, `Token` primitives, and the `PKCS7` padding function are always provided by the following internal implementations:
In the default installation configuration, the `X25519`, `Ed25519` and
`AES-128-CBC` primitives are provided by [OpenSSL](https://www.openssl.org/)
(via the [PyCA/cryptography](https://github.com/pyca/cryptography) package).
The hashing functions `SHA-256` and `SHA-512` are provided by the standard
Python [hashlib](https://docs.python.org/3/library/hashlib.html). The `HKDF`,
`HMAC`, `Fernet` primitives, and the `PKCS7` padding function are always
provided by the following internal implementations:
- [HKDF.py](RNS/Cryptography/HKDF.py)
- [HMAC.py](RNS/Cryptography/HMAC.py)
- [Token.py](RNS/Cryptography/Token.py)
- [Fernet.py](RNS/Cryptography/Fernet.py)
- [PKCS7.py](RNS/Cryptography/PKCS7.py)
Reticulum also includes a complete implementation of all necessary primitives in pure Python. If OpenSSL and PyCA are not available on the system when Reticulum is started, Reticulum will instead use the internal pure-python primitives. A trivial consequence of this is performance, with the OpenSSL backend being *much* faster. The most important consequence however, is the potential loss of security by using primitives that has not seen the same amount of scrutiny, testing and review as those from OpenSSL.
Reticulum also includes a complete implementation of all necessary primitives
in pure Python. If OpenSSL & PyCA are not available on the system when
Reticulum is started, Reticulum will instead use the internal pure-python
primitives. A trivial consequence of this is performance, with the OpenSSL
backend being *much* faster. The most important consequence however, is the
potential loss of security by using primitives that has not seen the same
amount of scrutiny, testing and review as those from OpenSSL.
Please note that by default, installing Reticulum will **require** OpenSSL and PyCA to also be automatically installed if not already available. It is only possible to use the pure-python primitives if this requirement is specifically overridden by the user, for example by installing the `rnspure` package instead of the normal `rns` package, or by running directly from local source-code.
If you want to use the internal pure-python primitives, it is **highly
advisable** that you have a good understanding of the risks that this pose, and
make an informed decision on whether those risks are acceptable to you.
If you want to use the internal pure-python primitives, it is **highly advisable** that you have a good understanding of the risks that this pose, and make an informed decision on whether those risks are acceptable to you.
Having been in development for ten, and wider deployment for around five years, Reticulum is relatively young software, and should be considered as such. While it has been carefully built with cryptography best-practices very foremost in mind, there could very well still be privacy or security breaking bugs.
Reticulum is relatively young software, and should be considered as such. While
it has been built with cryptography best-practices very foremost in mind, it
_has not_ been externally security audited, and there could very well be
privacy or security breaking bugs. If you want to help out, or help sponsor an
audit, please do get in touch.
## Acknowledgements & Credits
Reticulum can only exist because of the mountain of Open Source work it was built on top of, the contributions of everyone involved, and everyone that has supported the project through the years. To everyone who has helped, thank you so much.
Reticulum can only exist because of the mountain of Open Source work it was
built on top of, the contributions of everyone involved, and everyone that has
supported the project through the years. To everyone who has helped, thank you
so much.
A number of other modules and projects are either part of, or used by Reticulum. Sincere thanks to the authors and contributors of the following projects:
A number of other modules and projects are either part of, or used by
Reticulum. Sincere thanks to the authors and contributors of the following
projects:
- [PyCA/cryptography](https://github.com/pyca/cryptography), *BSD License*
- [Pure-25519](https://github.com/warner/python-pure25519) by [Brian Warner](https://github.com/warner), *MIT License*
- [Pysha2](https://github.com/thomdixon/pysha2) by [Thom Dixon](https://github.com/thomdixon), *MIT License*
- [Python AES-128](https://github.com/orgurar/python-aes) by [Or Gur Arie](https://github.com/orgurar), *MIT License*
- [Python AES-256](https://github.com/boppreh/aes) by [BoppreH](https://github.com/boppreh), *MIT License*
- [Python-AES](https://github.com/orgurar/python-aes) by [Or Gur Arie](https://github.com/orgurar), *MIT License*
- [Curve25519.py](https://gist.github.com/nickovs/cc3c22d15f239a2640c185035c06f8a3#file-curve25519-py) by [Nicko van Someren](https://gist.github.com/nickovs), *Public Domain*
- [I2Plib](https://github.com/l-n-s/i2plib) by [Viktor Villainov](https://github.com/l-n-s)
- [PySerial](https://github.com/pyserial/pyserial) by Chris Liechti, *BSD License*
- [Configobj](https://github.com/DiffSK/configobj) by Michael Foord, Nicola Larosa, Rob Dennis & Eli Courtwright, *BSD License*
- [ifaddr](https://github.com/pydron/ifaddr) by Stefan C. Mueller, *MIT License*
- [Six](https://github.com/benjaminp/six) by [Benjamin Peterson](https://github.com/benjaminp), *MIT License*
- [ifaddr](https://github.com/pydron/ifaddr) by [Pydron](https://github.com/pydron), *MIT License*
- [Umsgpack.py](https://github.com/vsergeev/u-msgpack-python) by [Ivan A. Sergeev](https://github.com/vsergeev)
- [rnsh](https://github.com/acehoss/rnsh) by [Aaron Heise](https://github.com/acehoss)
- [Python](https://www.python.org)

291
README.mu
View file

@ -1,291 +0,0 @@
>> Reticulum Network Stack
To understand the foundational philosophy and goals of this system, read the `_`!`[Zen of Reticulum`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=Zen+of+Reticulum.md]`!`_.
Reticulum is the cryptography-based networking stack for building local and wide-area networks with readily available hardware. It can operate even with very high latency and extremely low bandwidth. Reticulum allows you to build wide-area networks with off-the-shelf tools, and offers end-to-end encryption and connectivity, initiator anonymity, autoconfiguring cryptographically backed multi-hop transport, efficient addressing, unforgeable delivery acknowledgements and more.
`(RNS Logo`w=18`a=c`:/media/reticulum/reticulum/HEAD/docs/source/graphics/rns_logo_256.webp)
The vision of Reticulum is to allow anyone to be their own network operator, and to make it cheap and easy to cover vast areas with a myriad of independent, inter-connectable and autonomous networks. Reticulum `!is not`! `*one`* network. It is `!a tool`! for building `*thousands of networks`*. Networks without kill-switches, surveillance, censorship and control. Networks that can freely interoperate, associate and disassociate with each other, and require no central oversight. Networks for human beings. `*Networks for the people`*.
Reticulum is a complete networking stack, and does not rely on IP or higher layers, but it is possible to use IP as the underlying carrier for Reticulum. It is therefore trivial to tunnel Reticulum over the Internet or private IP networks.
Having no dependencies on traditional networking stacks frees up overhead that has been used to implement a networking stack built directly on cryptographic principles, allowing resilience and stable functionality, even in open and trustless networks.
No kernel modules or drivers are required. Reticulum runs completely in userland, and can run on practically any system that runs Python.
>> Read The Manual
The full documentation for Reticulum is available on `_`!`[this node`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/index.md]`!`_.
You can also download the `_`!`[Reticulum manual as a PDF`:/file/artifact`g=reticulum|r=reticulum|t=latest|a=Reticulum+Manual.pdf]`!`_ or `_`!`[as an e-book in EPUB format`:/file/artifact`g=reticulum|r=reticulum|t=latest|a=Reticulum+Manual.epub]`!`_.
>> Notable Features
• Coordination-less globally unique addressing and identification
• Fully self-configuring multi-hop routing over heterogeneous carriers
• Flexible scalability over heterogeneous topologies
• Reticulum can carry data over any mixture of physical mediums and topologies
• Low-bandwidth networks can co-exist and interoperate with large, high-bandwidth networks
• Initiator anonymity, communicate without revealing your identity
• Reticulum does not include source addresses on any packets
• Asymmetric X25519 encryption and Ed25519 signatures as a basis for all communication
• The foundational Reticulum Identity Keys are 512-bit Elliptic Curve keysets
• Forward Secrecy is available for all communication types, both for single packets and over links
• Reticulum uses the following format for encrypted tokens:
• Ephemeral per-packet and link keys and derived from an ECDH key exchange on Curve25519
• AES-256 in CBC mode with PKCS7 padding
• HMAC using SHA256 for authentication
• IVs are generated through os.urandom()
• Unforgeable packet delivery confirmations
• Flexible and extensible interface system
• Reticulum includes a large variety of built-in interface types
• Ability to load and utilise custom user- or community-supplied interface types
• Easily create your own custom interfaces for communicating over anything
• Authentication and virtual network segmentation on all supported interface types
• An intuitive and easy-to-use API
• Simpler and easier to use than sockets APIs, but more powerful
• Makes building distributed and decentralised applications much simpler
• Reliable and efficient transfer of arbitrary amounts of data
• Reticulum can handle a few bytes of data or files of many gigabytes
• Sequencing, compression, transfer coordination and checksumming are automatic
• The API is very easy to use, and provides transfer progress
• Lightweight, flexible and expandable Request/Response mechanism
• Efficient link establishment
• Total cost of setting up an encrypted and verified link is only 3 packets, totalling 297 bytes
• Low cost of keeping links open at only 0.44 bits per second
• Reliable sequential delivery with Channel and Buffer mechanisms
>> Reference Implementation
The Python code in this repository is the Reference Implementation of Reticulum. The Reticulum Protocol is defined entirely and authoritatively by this reference implementation, and its associated manual. It is maintained by Mark Qvist, identified by the Reticulum Identity `B333<bc7291552be7a58f361522990465165c>`b.
No second-order protocol specification derivatives currently do, or will ever exist. You can't get a better specification than one that is executable, functional, testable, deployed and operating 24/7 across the planet.
There is not, and never will be a separate, derived "formal spec", "RFC" or anything similar for Reticulum:
`!The reference implementation `*is`* the final, authoritative specification.`!
Compatibility with the Reticulum Protocol is defined as having full interoperability, and sufficient functional parity with this reference implementation. Any specific protocol implementation that achieves this is Reticulum. Any that does not is not Reticulum.
The reference implementation is licensed under the Reticulum License.
The Reticulum Protocol was dedicated to the Public Domain in 2016.
>> Examples of Reticulum Applications
If you want to quickly get an idea of what Reticulum can do, take a look at the `_`!`[Programs Using Reticulum`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/software.md]`!`_ section of the manual, or the following resources:
• `_`!`[LXMF`a8d24177d946de4f1f0a0fe1af9a1338:/page/repo.mu`g=reticulum|r=lxmf]`!`_ is a distributed, delay and disruption tolerant message transfer protocol built on Reticulum
• The `_`!`[LXST`a8d24177d946de4f1f0a0fe1af9a1338:/page/repo.mu`g=reticulum|r=lxst]`!`_ protocol and framework provides real-time audio and signals transport over Reticulum. It
includes primitives and utilities for building voice-based applications and hardware devices,
such as the `B333rnphone`b program, that can be used to build hardware telephones.
• For an off-grid, encrypted and resilient mesh communications platform, see `_`!`[Nomad Network`a8d24177d946de4f1f0a0fe1af9a1338:/page/repo.mu`g=reticulum|r=nomadnet]`!`_.
• The Android, Linux, macOS and Windows app `_`!`[Sideband`a8d24177d946de4f1f0a0fe1af9a1338:/page/repo.mu`g=reticulum|r=sideband]`!`_ has a graphical interface and many advanced
features, such as file transfers, image and voice messages, real-time voice calls, a distributed
telemetry system, mapping capabilities and full plugin extensibility.
• `_`!`[MeshChatX`c10d80b1a42fa958c37a6cc30dc04f53]`!`_ (`_`!`[source`5399f5a0212477618821e91e88ce053b:/page/repo.mu`g=quad4|r=MeshChatX]`_`!) is a full-featured LXMF client with many built-in tools and functionalities,
that also supports image and voice messages, file transfers and voice calls. It also includes a
built-in page browser for browsing Nomad Network nodes.
• You can use the included `_`!`[rnsh`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/using.md|anchor=the-rnsh-utility]`!`_ program to establish remote shell sessions over Reticulum.
>> Where can Reticulum be used?
Over practically any medium that can support at least a half-duplex channel with greater throughput than 5 bits per second, and an MTU of 500 bytes. Data radios, modems, LoRa radios, serial lines, AX.25 TNCs, amateur radio digital modes, WiFi and Ethernet devices, free-space optical links, and similar systems are all examples of the types of physical devices Reticulum can use.
An open-source LoRa-based interface called `_`!`[RNode`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/hardware.md|anchor=rnode]`!`_ has been designed specifically for use with Reticulum. It is possible to build yourself, or it can be purchased as a complete transceiver that just needs a USB connection to the host.
Reticulum can also be encapsulated over existing IP networks, so there's nothing stopping you from using it over wired Ethernet, your local WiFi network or the Internet, where it'll work just as well. In fact, one of the strengths of Reticulum is how easily it allows you to connect different mediums into a self-configuring, resilient and encrypted mesh, using any available mixture of available infrastructure.
As an example, it's possible to set up a Raspberry Pi connected to both a LoRa radio, a packet radio TNC and a WiFi network. Once the interfaces are configured, Reticulum will take care of the rest, and any device on the WiFi network can communicate with nodes on the LoRa and packet radio sides of the network, and vice versa.
>> How do I get started?
The best way to get started with the Reticulum Network Stack depends on what you want to do. For full details and examples, have a look at the `_`!`[Getting Started Fast`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/gettingstartedfast.md]`!`_ section of the `_`!`[Reticulum Manual`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/index.md]`!`_.
To simply install Reticulum and related utilities on your system, the easiest way is via `B333pip`b. You can then start any program that uses Reticulum, or start Reticulum as a system service with `_`!`[the rnsd utility`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/using.md|anchor=the-rnsd-utility]`!`_.
`B333
`=
pip install rns
`=
`b
If you are using an operating system that blocks normal user package installation via `B333pip`b, you can return `B333pip`b to normal behaviour by editing the `B333~/.config/pip/pip.conf`b file, and adding the following directive in the `B333[global]`b section:
`B333
`=
[global]
break-system-packages = true
`=
`b
Alternatively, you can use the `B333pipx`b tool to install Reticulum in an isolated environment:
`B333
`=
pipx install rns
`=
`b
When first started, Reticulum will create a default configuration file, providing basic connectivity to other Reticulum peers that might be locally reachable. The default config file contains a few examples, and references for creating a more complex configuration.
If you have an old version of `B333pip`b on your system, you may need to upgrade it first with `B333pip install pip --upgrade`b. If you no not already have `B333pip`b installed, you can install it using the package manager of your system with `B333sudo apt install python3-pip`b or similar.
For more detailed examples on how to expand communication over many mediums such as packet radio or LoRa, serial ports, or over fast IP links and the Internet using the UDP and TCP interfaces, take a look at the `_`!`[Supported Interfaces`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/interfaces.md]`!`_ section of the `_`!`[Reticulum Manual`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/index.md]`!`_.
>> Included Utilities
Reticulum includes a range of useful utilities for managing your networks, viewing status and information, and other tasks. You can read more about these programs in the `_`!`[Included Utility Programs`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/using.md|anchor=included-utility-programs]`!`_ section of the `_`!`[Reticulum Manual`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/index.md]`!`_.
• The system daemon `B333rnsd`b for running Reticulum as an always-available service
• An interface status utility called `B333rnstatus`b, that displays information about interfaces
• The path lookup and management tool `B333rnpath`b letting you view and modify path tables
• A diagnostics tool called `B333rnprobe`b for checking connectivity to destinations
• A simple file transfer program called `B333rncp`b making it easy to transfer files between systems
• The identity management and encryption utility `B333rnid`b let's you manage Identities and encrypt/decrypt files
• The `B333rnsh`b program allows you to establish fully interactive shell session with remote systems
• The remote command execution program `B333rnx`b let's you run simple commands and programs and retrieve output from remote systems
• The `B333rngit`b program provides a full multi-repository Git node for serving repositories over Reticulum
• The included `B333git-remote-rns`b helper allows you to interact with Git repositories over Reticulum
>> Supported interface types and devices
Reticulum implements a range of generalised interface types that covers most of the communications hardware that Reticulum can run over. If your hardware is not supported, it's `_`!`[simple to implement a custom interface module`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/interfaces.md|anchor=custom-interfaces]`!`_.
Currently, the following built-in interfaces are supported:
• Any Ethernet device
• LoRa using `_`!`[RNode`a8d24177d946de4f1f0a0fe1af9a1338:/page/repo.mu`g=reticulum|r=rnode_firmware]`!`_
• Packet Radio TNCs (with or without AX.25)
• KISS-compatible hardware and software modems
• Any device with a serial port
• TCP over IP networks
• UDP over IP networks
• External programs via stdio or pipes
• Custom hardware via stdio or pipes
>> Performance
`*All performance numbers here assume `B333rnsd`b running in fully interpreted Python mode, limited to a single CPU core.`*
Reticulum targets a `*very`* wide usable performance envelope, but prioritises functionality and performance on low-bandwidth mediums. The goal is to provide a dynamic performance envelope from 100 bits per second, to 10 gigabits per second on normal hardware.
Currently, the on-network, usable performance envelope is approximately 150 bits per second to 1 gigabit per second on modest hardware. Real-world performance will vary widely with the speed of your system, availability of cryptographic hardware acceleration, and other factors.
Raw routing throughput for a transport node hosted on a Raspberry Pi 5 is approximately 4 gigabits per second. Systems with modern crypto hardware acceleration can achieve transport throughput of 15 gigabits per second and higher.
While the reference implementation is perfectly capable of reaching 100+ Gbps, by utilizing multiple CPU cores and vectorized transport, performance beyond the current level is not higly prioritised, since RNS is already able to saturate available physical medium bandwidth in almost all real-world situations.
The primary objectives for RNS are `*reliability`*, `*correctness`*, `*security`*, `*proper traffic management`* and `*actually practical performance`*, not hollow on-paper "benchmarks" with little to no bearing on real-world usability.
All alternative-implementation maintainers are `*highly`* encouraged to adopt a similar approach, and especially to focus on getting their implementations working properly before publishing empirically dubious or factually wrong performance claims.
>> Current Status
All core protocol features are implemented and functioning, but additions will probably occur as real-world use is explored and understood. The API and wire-format can be considered stable.
>> Dependencies
The installation of the default `B333rns`b package requires only two external dependencies, listed below. Almost all systems and distributions have readily available packages for these dependencies, and when the `B333rns`b package is installed with `B333pip`b, they will be downloaded and installed as well.
• PyCA/cryptography
• pyserial
On more unusual systems, and in some rare cases, it might not be possible to install or even compile one or more of the above modules. In such situations, you can use the `B333rnspure`b package instead, which require no external dependencies for installation. Please note that the contents of the `B333rns`b and `B333rnspure`b packages are `*identical`*. The only difference is that the `B333rnspure`b package lists no dependencies required for installation.
No matter how Reticulum is installed and started, it will load external dependencies only if they are `*needed`* and `*available`*. If for example you want to use Reticulum on a system that cannot support `B333pyserial`b, it is perfectly possible to do so using the `B333rnspure`b package, but Reticulum will not be able to use serial-based interfaces. All other available modules will still be loaded when needed.
`!Please Note!`! If you use the `B333rnspure`b package to run Reticulum on systems that do not support PyCA/cryptography, it is important that you read and understand the `!Cryptographic Primitives`! section of this document.
>> Bootstrapping Connectivity
Reticulum is not a service you subscribe to, nor is it a single global network you "join". Reticulum provides functionality for discovering available public interfaces over the network itself, and the broader community has provided various directories of publicly available entrypoints to bootstrap connectivity.
To learn how to establish initial connectivity over Reticulum, read the `_`!`[Bootstrapping Connectivity`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/gettingstartedfast.md|anchor=bootstrapping-connectivity]`!`_ section of the manual.
If you already have a general idea of how this works, you can use community-run sites such as `_`!`[rns.recipes`9ce92808be498e9e05590ff27cbfdfe4]`!`_ and `_`!`[rmap.world`a4a5e861626ce97c9aa544d9ecdf6d22]`!`_ to find interface definitions for initial connectivity to the global distributed Reticulum backbone.
>> Public Testnet
`!`*Important!`! Historically, a developer-targeted testnet was made available by the Reticulum project itself. As the amount of global Reticulum nodes and entrypoints have grown to a substantial quantity, this public testnet, including the Amsterdam Testnet entrypoint, has now been decommissioned. If you still have instances that relied on this entrypoint for connectivity, transition to using the distributed backbone instead. Reticulum now includes a full on-network interface discovery and connectivity bootstrapping system. Read the `_`[Bootstrapping Connectivity`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=docs/markdown/gettingstartedfast.md|anchor=bootstrapping-connectivity]`_ section of the manual for pointers.`*
>> Support Reticulum
For this to be possible, I need your help. Please support the continued development of open, free and private communications systems by donating via one of the following channels:
• `!Monero`!
84FpY1QbxHcgdseePYNmhTHcrgMX4nFfBYtz2GKYToqHVVhJp8Eaw1Z1EedRnKD19b3B8NiLCGVxzKV17UMmmeEsCrPyA5w
• `!Bitcoin`!
bc1pgqgu8h8xvj4jtafslq396v7ju7hkgymyrzyqft4llfslz5vp99psqfk3a6
• `!Ethereum`!
0x91C421DdfB8a30a49A71d63447ddb54cEBe3465E
• `!Liberapay`!
`[https://liberapay.com/Reticulum/]
• `!Ko-Fi`!
`[https://ko-fi.com/markqvist]
>> Cryptographic Primitives
Reticulum uses a simple suite of efficient, strong and well-tested cryptographic primitives, with widely available implementations that can be used both on general-purpose CPUs and on microcontrollers.
One of the primary considerations for choosing this particular set of primitives is that they can be implemented `*safely`* with relatively few pitfalls, on practically all current computing platforms.
The primitives listed here `!are authoritative`!. Anything `*claiming`* to be Reticulum, but not using these exact primitives `FTA35050`!is not`!`f Reticulum, and possibly an intentionally compromised or weakened clone. The utilised primitives are:
• Reticulum Identity Keys are 512-bit Curve25519 keysets
• A 256-bit Ed25519 key for signatures
• A 256-bit X22519 key for ECDH key exchanges
• HKDF for key derivation
• Encrypted tokens are based on the `_`!`[Fernet spec`https://github.com/fernet/spec/]`!`_
• Ephemeral keys derived from an ECDH key exchange on Curve25519
• HMAC using SHA256 for message authentication
• IVs must be generated through `B333os.urandom()`b or better
• AES-256 in CBC mode with PKCS7 padding
• No Fernet version and timestamp metadata fields
• SHA-256
• SHA-512
In the default installation configuration, the `B333X25519`b, `B333Ed25519`b, and `B333AES-256-CBC`b primitives are provided by `_`!`[OpenSSL`https://www.openssl.org/]`!`_ (via the `_`!`[PyCA/cryptography`https://github.com/pyca/cryptography]`!`_ package). The hashing functions `B333SHA-256`b and `B333SHA-512`b are provided by the standard Python `_`!`[hashlib`https://docs.python.org/3/library/hashlib.html]`!`_. The `B333HKDF`b, `B333HMAC`b, `B333Token`b primitives, and the `B333PKCS7`b padding function are always provided by the following internal implementations:
• `_`!`[HKDF.py`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=RNS/Cryptography/HKDF.py]`!`_
• `_`!`[HMAC.py`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=RNS/Cryptography/HMAC.py]`!`_
• `_`!`[Token.py`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=RNS/Cryptography/Token.py]`!`_
• `_`!`[PKCS7.py`:/page/blob.mu`g=reticulum|r=reticulum|ref=HEAD|path=RNS/Cryptography/PKCS7.py]`!`_
Reticulum also includes a complete implementation of all necessary primitives in pure Python. If OpenSSL and PyCA are not available on the system when Reticulum is started, Reticulum will instead use the internal pure-python primitives. A trivial consequence of this is performance, with the OpenSSL backend being `*much`* faster. The most important consequence however, is the potential loss of security by using primitives that has not seen the same amount of scrutiny, testing and review as those from OpenSSL.
Please note that by default, installing Reticulum will `!require`! OpenSSL and PyCA to also be automatically installed if not already available. It is only possible to use the pure-python primitives if this requirement is specifically overridden by the user, for example by installing the `B333rnspure`b package instead of the normal `B333rns`b package, or by running directly from local source-code.
If you want to use the internal pure-python primitives, it is `!highly advisable`! that you have a good understanding of the risks that this pose, and make an informed decision on whether those risks are acceptable to you.
Having been in development for ten, and wider deployment for around five years, Reticulum is relatively young software, and should be considered as such. While it has been carefully built with cryptography best-practices very foremost in mind, there could very well still be privacy or security breaking bugs.
>> Acknowledgements & Credits
Reticulum can only exist because of the mountain of Open Source work it was built on top of, the contributions of everyone involved, and everyone that has supported the project through the years. To everyone who has helped, thank you so much.
A number of other modules and projects are either part of, or used by Reticulum. Sincere thanks to the authors and contributors of the following projects:
• `_`!`[PyCA/cryptography`https://github.com/pyca/cryptography]`!`_, `*BSD License`*
• `_`!`[Pure-25519`https://github.com/warner/python-pure25519]`!`_, by `_`!`[Brian Warner`https://github.com/warner]`!`_, `*MIT License`*
• `_`!`[Pysha2`https://github.com/thomdixon/pysha2]`!`_ by `_`!`[Thom Dixon`https://github.com/thomdixon]`!`_, `*MIT License`*
• `_`!`[Python AES-128`https://github.com/orgurar/python-aes]`!`_ by `_`!`[Or Gur Arie`https://github.com/orgurar]`!`_, `*MIT License`*
• `_`!`[Python AES-256`https://github.com/boppreh/aes]`!`_ by `_`!`[BoppreH`https://github.com/boppreh]`!`_, `*MIT License`*
• `_`!`[Curve25519.py`https://gist.github.com/nickovs/cc3c22d15f239a2640c185035c06f8a3]`!`_ by `_`!`[Nicko van Someren`https://gist.github.com/nickovs]`!`_, `*Public Domain`*
• `_`!`[I2Plib`https://github.com/l-n-s/i2plib]`!`_ by `_`!`[Viktor Villainov`https://github.com/l-n-s]`!`_
• `_`!`[PySerial`https://github.com/pyserial/pyserial]`!`_ by Chris Liechti, `*BSD License`*
• `_`!`[Configobj`https://github.com/DiffSK/configobj]`!`_ by Michael Foord, Nicola Larosa, Rob Dennis & Eli Courtwright, `*BSD License`*
• `_`!`[ifaddr`https://github.com/pydron/ifaddr]`!`_ by Stefan C. Mueller, `*MIT License`*
• `_`!`[Umsgpack.py`https://github.com/vsergeev/u-msgpack-python]`!`_ by `_`!`[Ivan A. Sergeev`https://github.com/vsergeev]`!`_
• `_`!`[rnsh`https://github.com/acehoss/rnsh]`!`_ by `_`!`[Aaron Heise`https://github.com/acehoss]`!`_
• `_`!`[Python`https://www.python.org]`!`_

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -53,9 +45,7 @@ class StreamDataMessage(MessageBase):
The stream id is limited to 2 bytes - 2 bit
"""
HEADER_LEN = 2 # 2 for stream data message header
OVERHEAD = HEADER_LEN + 6 # 6 for channel envelope
MAX_DATA_LEN = RNS.Link.MDU - OVERHEAD
MAX_DATA_LEN = RNS.Link.MDU - 2 - 6 # 2 for stream data message header, 6 for channel envelope
"""
When the Buffer package is imported, this value is
calculcated based on the value of OVERHEAD
@ -93,9 +83,7 @@ class StreamDataMessage(MessageBase):
self.data = raw[2:]
if self.compressed:
decompressor = bz2.BZ2Decompressor()
self.data = decompressor.decompress(self.data, max_length=RawChannelWriter.MAX_CHUNK_LEN)
if not decompressor.eof: raise IOError("Decompressed buffer chunk exceeds maximum legitimate size")
self.data = bz2.decompress(self.data)
class RawChannelReader(RawIOBase, AbstractContextManager):
@ -227,7 +215,6 @@ class RawChannelWriter(RawIOBase, AbstractContextManager):
self._stream_id = stream_id
self._channel = channel
self._eof = False
self._mdu = channel.mdu - StreamDataMessage.HEADER_LEN
def write(self, __b: bytes) -> int | None:
try:
@ -243,7 +230,7 @@ class RawChannelWriter(RawIOBase, AbstractContextManager):
chunk_segment_length = int(chunk_len/comp_try)
compressed_chunk = bz2.compress(__b[:chunk_segment_length])
compressed_length = len(compressed_chunk)
if compressed_length < self._mdu and compressed_length < chunk_segment_length:
if compressed_length < StreamDataMessage.MAX_DATA_LEN and compressed_length < chunk_segment_length:
comp_success = True
break
else:
@ -253,7 +240,7 @@ class RawChannelWriter(RawIOBase, AbstractContextManager):
chunk = compressed_chunk
processed_length = chunk_segment_length
else:
chunk = bytes(__b[:self._mdu])
chunk = bytes(__b[:StreamDataMessage.MAX_DATA_LEN])
processed_length = len(chunk)
message = StreamDataMessage(self._stream_id, chunk, self._eof, comp_success)

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -45,6 +37,63 @@ TPacket = TypeVar("TPacket")
class SystemMessageTypes(enum.IntEnum):
SMT_STREAM_DATA = 0xff00
class ChannelOutletBase(ABC, Generic[TPacket]):
"""
An abstract transport layer interface used by Channel.
DEPRECATED: This was created for testing; eventually
Channel will use Link or a LinkBase interface
directly.
"""
@abstractmethod
def send(self, raw: bytes) -> TPacket:
raise NotImplemented()
@abstractmethod
def resend(self, packet: TPacket) -> TPacket:
raise NotImplemented()
@property
@abstractmethod
def mdu(self):
raise NotImplemented()
@property
@abstractmethod
def rtt(self):
raise NotImplemented()
@property
@abstractmethod
def is_usable(self):
raise NotImplemented()
@abstractmethod
def get_packet_state(self, packet: TPacket) -> MessageState:
raise NotImplemented()
@abstractmethod
def timed_out(self):
raise NotImplemented()
@abstractmethod
def __str__(self):
raise NotImplemented()
@abstractmethod
def set_packet_timeout_callback(self, packet: TPacket, callback: Callable[[TPacket], None] | None,
timeout: float | None = None):
raise NotImplemented()
@abstractmethod
def set_packet_delivered_callback(self, packet: TPacket, callback: Callable[[TPacket], None] | None):
raise NotImplemented()
@abstractmethod
def get_packet_id(self, packet: TPacket) -> any:
raise NotImplemented()
class CEType(enum.IntEnum):
"""
ChannelException type codes
@ -56,6 +105,7 @@ class CEType(enum.IntEnum):
ME_ALREADY_SENT = 4
ME_TOO_BIG = 5
class ChannelException(Exception):
"""
An exception thrown by Channel, with a type code.
@ -64,6 +114,7 @@ class ChannelException(Exception):
super().__init__(args)
self.type = ce_type
class MessageState(enum.IntEnum):
"""
Set of possible states for a Message
@ -73,6 +124,7 @@ class MessageState(enum.IntEnum):
MSGSTATE_DELIVERED = 2
MSGSTATE_FAILED = 3
class MessageBase(abc.ABC):
"""
Base type for any messages sent or received on a Channel.
@ -84,7 +136,7 @@ class MessageBase(abc.ABC):
MSGTYPE = None
"""
Defines a unique identifier for a message class.
* Must be unique within all classes registered with a ``Channel``
* Must be less than ``0xf000``. Values greater than or equal to ``0xf000`` are reserved.
"""
@ -110,6 +162,7 @@ class MessageBase(abc.ABC):
MessageCallbackType = NewType("MessageCallbackType", Callable[[MessageBase], bool])
class Envelope:
"""
Internal wrapper used to transport messages over a channel and
@ -119,7 +172,8 @@ class Envelope:
msgtype, self.sequence, length = struct.unpack(">HHH", self.raw[:6])
raw = self.raw[6:]
ctor = message_factories.get(msgtype, None)
if ctor is None: raise ChannelException(CEType.ME_NOT_REGISTERED, f"Unable to find constructor for Channel MSGTYPE {hex(msgtype)}")
if ctor is None:
raise ChannelException(CEType.ME_NOT_REGISTERED, f"Unable to find constructor for Channel MSGTYPE {hex(msgtype)}")
message = ctor()
message.unpack(raw)
self.unpacked = True
@ -128,13 +182,14 @@ class Envelope:
return message
def pack(self) -> bytes:
if self.message.__class__.MSGTYPE is None: raise ChannelException(CEType.ME_NO_MSG_TYPE, f"{self.message.__class__} lacks MSGTYPE")
if self.message.__class__.MSGTYPE is None:
raise ChannelException(CEType.ME_NO_MSG_TYPE, f"{self.message.__class__} lacks MSGTYPE")
data = self.message.pack()
self.raw = struct.pack(">HHH", self.message.MSGTYPE, self.sequence, len(data)) + data
self.packed = True
return self.raw
def __init__(self, outlet: LinkChannelOutlet, message: MessageBase = None, raw: bytes = None, sequence: int = None):
def __init__(self, outlet: ChannelOutletBase, message: MessageBase = None, raw: bytes = None, sequence: int = None):
self.ts = time.time()
self.id = id(self)
self.message = message
@ -192,11 +247,11 @@ class Channel(contextlib.AbstractContextManager):
# The maximum window size for transfers on fast links
WINDOW_MAX_FAST = 48
# For calculating maps and guard segments, this
# must be set to the global maximum window.
WINDOW_MAX = WINDOW_MAX_FAST
# If the fast rate is sustained for this many request
# rounds, the fast link window size will be allowed.
FAST_RATE_THRESHOLD = 10
@ -215,14 +270,13 @@ class Channel(contextlib.AbstractContextManager):
SEQ_MAX = 0xFFFF
SEQ_MODULUS = SEQ_MAX+1
def __init__(self, outlet: LinkChannelOutlet):
def __init__(self, outlet: ChannelOutletBase):
"""
@param outlet:
"""
self._outlet = outlet
self._lock = threading.RLock()
self._send_lock = threading.Lock()
self._tx_ring: collections.deque[Envelope] = collections.deque()
self._rx_ring: collections.deque[Envelope] = collections.deque()
self._message_callbacks: [MessageCallbackType] = []
@ -244,9 +298,11 @@ class Channel(contextlib.AbstractContextManager):
self.window_min = Channel.WINDOW_MIN
self.window_flexibility = Channel.WINDOW_FLEXIBILITY
def __enter__(self) -> Channel: return self
def __enter__(self) -> Channel:
return self
def __exit__(self, __exc_type: Type[BaseException] | None, __exc_value: BaseException | None, __traceback: TracebackType | None) -> bool | None:
def __exit__(self, __exc_type: Type[BaseException] | None, __exc_value: BaseException | None,
__traceback: TracebackType | None) -> bool | None:
self._shutdown()
return False
@ -262,13 +318,20 @@ class Channel(contextlib.AbstractContextManager):
def _register_message_type(self, message_class: Type[MessageBase], *, is_system_type: bool = False):
with self._lock:
if not issubclass(message_class, MessageBase): raise ChannelException(CEType.ME_INVALID_MSG_TYPE, f"{message_class} is not a subclass of {MessageBase}.")
if message_class.MSGTYPE is None: raise ChannelException(CEType.ME_INVALID_MSG_TYPE, f"{message_class} has invalid MSGTYPE class attribute.")
if message_class.MSGTYPE >= 0xf000 and not is_system_type: raise ChannelException(CEType.ME_INVALID_MSG_TYPE, f"{message_class} has system-reserved message type.")
try: message_class()
if not issubclass(message_class, MessageBase):
raise ChannelException(CEType.ME_INVALID_MSG_TYPE,
f"{message_class} is not a subclass of {MessageBase}.")
if message_class.MSGTYPE is None:
raise ChannelException(CEType.ME_INVALID_MSG_TYPE,
f"{message_class} has invalid MSGTYPE class attribute.")
if message_class.MSGTYPE >= 0xf000 and not is_system_type:
raise ChannelException(CEType.ME_INVALID_MSG_TYPE,
f"{message_class} has system-reserved message type.")
try:
message_class()
except Exception as ex:
raise ChannelException(CEType.ME_INVALID_MSG_TYPE, f"{message_class} raised an exception when constructed with no arguments: {ex}")
raise ChannelException(CEType.ME_INVALID_MSG_TYPE,
f"{message_class} raised an exception when constructed with no arguments: {ex}")
self._message_factories[message_class.MSGTYPE] = message_class
@ -311,30 +374,27 @@ class Channel(contextlib.AbstractContextManager):
if envelope.packet is not None:
self._outlet.set_packet_timeout_callback(envelope.packet, None)
self._outlet.set_packet_delivered_callback(envelope.packet, None)
envelope.tracked = False
for envelope in self._rx_ring:
envelope.tracked = False
self._tx_ring.clear()
self._rx_ring.clear()
def _emplace_envelope(self, envelope: Envelope, ring: collections.deque[Envelope]) -> bool:
with self._lock:
i = 0
for existing in ring:
if envelope.sequence == existing.sequence:
RNS.log(f"Envelope: Emplacement of duplicate envelope with sequence "+str(envelope.sequence), RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
return False
if envelope.sequence == existing.sequence:
RNS.log(f"Envelope: Emplacement of duplicate envelope with sequence "+str(envelope.sequence), RNS.LOG_EXTREME)
return False
if envelope.sequence < existing.sequence and not (self._next_rx_sequence - envelope.sequence) > (Channel.SEQ_MAX//2):
ring.insert(i, envelope)
envelope.tracked = True
return True
i += 1
envelope.tracked = True
ring.append(envelope)
@ -345,8 +405,10 @@ class Channel(contextlib.AbstractContextManager):
for cb in cbs:
try:
if cb(message): return
except Exception as e: RNS.log(f"Channel {str(self)} experienced an error while running a message callback. The contained exception was: {e}", RNS.LOG_ERROR)
if cb(message):
return
except Exception as e:
RNS.log("Channel "+str(self)+" experienced an error while running a message callback. The contained exception was: "+str(e), RNS.LOG_ERROR)
def _receive(self, raw: bytes):
try:
@ -358,20 +420,16 @@ class Channel(contextlib.AbstractContextManager):
window_overflow = (self._next_rx_sequence+Channel.WINDOW_MAX) % Channel.SEQ_MODULUS
if window_overflow < self._next_rx_sequence:
if envelope.sequence > window_overflow:
RNS.log(f"Invalid packet sequence ({envelope.sequence}) received on channel {self}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
RNS.log("Invalid packet sequence ("+str(envelope.sequence)+") received on channel "+str(self), RNS.LOG_EXTREME)
return
else:
RNS.log(f"Invalid packet sequence ({envelope.sequence}) received on channel {self}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
RNS.log("Invalid packet sequence ("+str(envelope.sequence)+") received on channel "+str(self), RNS.LOG_EXTREME)
return
elif envelope.sequence > self._next_rx_sequence + self.WINDOW_MAX:
RNS.log(f"Invalid packet sequence ({envelope.sequence}) received on channel {self}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
return
is_new = self._emplace_envelope(envelope, self._rx_ring)
if not is_new:
RNS.log("Duplicate message received on channel "+str(self), RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
RNS.log("Duplicate message received on channel "+str(self), RNS.LOG_EXTREME)
return
else:
with self._lock:
@ -387,9 +445,11 @@ class Channel(contextlib.AbstractContextManager):
self._next_rx_sequence = (self._next_rx_sequence + 1) % Channel.SEQ_MODULUS
for e in contigous:
if not e.unpacked: m = e.unpack(self._message_factories)
else: m = e.message
if not e.unpacked:
m = e.unpack(self._message_factories)
else:
m = e.message
self._rx_ring.remove(e)
self._run_callbacks(m)
@ -402,46 +462,67 @@ class Channel(contextlib.AbstractContextManager):
:return: True if ready
"""
if not self._outlet.is_usable: return False
if not self._outlet.is_usable:
return False
with self._lock:
outstanding = 0
for envelope in self._tx_ring:
if envelope.outlet == self._outlet:
if envelope.outlet == self._outlet:
if not envelope.packet or not self._outlet.get_packet_state(envelope.packet) == MessageState.MSGSTATE_DELIVERED:
outstanding += 1
if outstanding >= self.window: return False
if outstanding >= self.window:
return False
return True
def _packet_tx_op(self, packet: TPacket, op: Callable[[TPacket], bool]):
target_id = self._outlet.get_packet_id(packet)
with self._lock:
envelope = next(filter(lambda e: e.packet is not None and self._outlet.get_packet_id(e.packet) == target_id, self._tx_ring), None)
envelope = next(filter(lambda e: self._outlet.get_packet_id(e.packet) == self._outlet.get_packet_id(packet),
self._tx_ring), None)
if envelope and op(envelope):
envelope.tracked = False
if envelope in self._tx_ring:
self._tx_ring.remove(envelope)
if self.window < self.window_max: self.window += 1
if self.window < self.window_max:
self.window += 1
# TODO: Remove at some point
# RNS.log("Increased "+str(self)+" window to "+str(self.window), RNS.LOG_DEBUG)
if self._outlet.rtt != 0:
if self._outlet.rtt > Channel.RTT_FAST:
self.fast_rate_rounds = 0
if self._outlet.rtt > Channel.RTT_MEDIUM: self.medium_rate_rounds = 0
if self._outlet.rtt > Channel.RTT_MEDIUM:
self.medium_rate_rounds = 0
else:
self.medium_rate_rounds += 1
if self.window_max < Channel.WINDOW_MAX_MEDIUM and self.medium_rate_rounds == Channel.FAST_RATE_THRESHOLD:
self.window_max = Channel.WINDOW_MAX_MEDIUM
self.window_min = Channel.WINDOW_MIN_LIMIT_MEDIUM
# TODO: Remove at some point
# RNS.log("Increased "+str(self)+" max window to "+str(self.window_max), RNS.LOG_DEBUG)
# RNS.log("Increased "+str(self)+" min window to "+str(self.window_min), RNS.LOG_DEBUG)
else:
self.fast_rate_rounds += 1
if self.window_max < Channel.WINDOW_MAX_FAST and self.fast_rate_rounds == Channel.FAST_RATE_THRESHOLD:
self.window_max = Channel.WINDOW_MAX_FAST
self.window_min = Channel.WINDOW_MIN_LIMIT_FAST
# TODO: Remove at some point
# RNS.log("Increased "+str(self)+" max window to "+str(self.window_max), RNS.LOG_DEBUG)
# RNS.log("Increased "+str(self)+" min window to "+str(self.window_min), RNS.LOG_DEBUG)
else: RNS.log("Envelope not found in TX ring for "+str(self), RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
if not envelope: RNS.log("Spurious message received on "+str(self), RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
else:
RNS.log("Envelope not found in TX ring for "+str(self), RNS.LOG_EXTREME)
if not envelope:
RNS.log("Spurious message received on "+str(self), RNS.LOG_EXTREME)
def _packet_delivered(self, packet: TPacket):
self._packet_tx_op(packet, lambda env: True)
@ -450,47 +531,44 @@ class Channel(contextlib.AbstractContextManager):
for envelope in self._tx_ring:
updated_timeout = self._get_packet_timeout_time(envelope.tries)
if envelope.packet and hasattr(envelope.packet, "receipt") and envelope.packet.receipt and envelope.packet.receipt.timeout:
if updated_timeout > envelope.packet.receipt.timeout: envelope.packet.receipt.set_timeout(updated_timeout)
if updated_timeout > envelope.packet.receipt.timeout:
envelope.packet.receipt.set_timeout(updated_timeout)
def _get_packet_timeout_time(self, tries: int) -> float:
to = pow(1.5, tries - 1) * max(self._outlet.rtt*2.5, 0.025) * (len(self._tx_ring)+1.5)
return to
def _packet_timeout(self, packet: TPacket):
if self._outlet.get_packet_state(packet) == MessageState.MSGSTATE_DELIVERED: return
def retry_envelope(envelope: Envelope) -> bool:
if envelope.tries >= self._max_tries:
RNS.log("Retry count exceeded on "+str(self)+", tearing down Link.", RNS.LOG_ERROR)
self._shutdown() # start on separate thread?
self._outlet.timed_out()
return True
target_id = self._outlet.get_packet_id(packet)
envelope_to_resend: Envelope | None = None
should_teardown = False
with self._lock:
envelope = next(filter(lambda e: e.packet is not None and self._outlet.get_packet_id(e.packet) == target_id, self._tx_ring), None)
if envelope is None: return
envelope.tries += 1
self._outlet.resend(envelope.packet)
self._outlet.set_packet_delivered_callback(envelope.packet, self._packet_delivered)
self._outlet.set_packet_timeout_callback(envelope.packet, self._packet_timeout, self._get_packet_timeout_time(envelope.tries))
self._update_packet_timeouts()
if envelope.tries >= self._max_tries: should_teardown = True
else:
envelope.tries += 1
envelope_to_resend = envelope
if self.window > self.window_min:
self.window -= 1
# TODO: Remove at some point
# RNS.log("Decreased "+str(self)+" window to "+str(self.window), RNS.LOG_DEBUG)
if self.window > self.window_min:
self.window -= 1
if self.window_max > (self.window_min+self.window_flexibility):
self.window_max -= 1
if self.window_max > (self.window_min+self.window_flexibility):
self.window_max -= 1
# TODO: Remove at some point
# RNS.log("Decreased "+str(self)+" max window to "+str(self.window_max), RNS.LOG_DEBUG)
if should_teardown:
RNS.log("Retry count exceeded on "+str(self)+", tearing down Link.", RNS.LOG_ERROR)
self._shutdown()
self._outlet.timed_out()
return
# TODO: Remove at some point
# RNS.log("Decreased "+str(self)+" window to "+str(self.window), RNS.LOG_EXTREME)
if envelope_to_resend is not None:
self._outlet.resend(envelope_to_resend.packet)
with self._lock:
self._outlet.set_packet_delivered_callback(envelope_to_resend.packet, self._packet_delivered)
self._outlet.set_packet_timeout_callback(envelope_to_resend.packet, self._packet_timeout, self._get_packet_timeout_time(envelope_to_resend.tries))
self._update_packet_timeouts()
already_delivered = (self._outlet.get_packet_state(envelope_to_resend.packet) == MessageState.MSGSTATE_DELIVERED)
return False
if already_delivered: self._packet_delivered(envelope_to_resend.packet)
if self._outlet.get_packet_state(packet) != MessageState.MSGSTATE_DELIVERED:
self._packet_tx_op(packet, retry_envelope)
def send(self, message: MessageBase) -> Envelope:
"""
@ -499,38 +577,32 @@ class Channel(contextlib.AbstractContextManager):
:param message: an instance of a ``MessageBase`` subclass
"""
with self._send_lock:
with self._lock:
if not self.is_ready_to_send(): raise ChannelException(CEType.ME_LINK_NOT_READY, f"Link is not ready")
envelope: Envelope | None = None
with self._lock:
if not self.is_ready_to_send():
raise ChannelException(CEType.ME_LINK_NOT_READY, f"Link is not ready")
envelope = Envelope(self._outlet, message=message, sequence=self._next_sequence)
self._next_sequence = (self._next_sequence + 1) % Channel.SEQ_MODULUS
self._emplace_envelope(envelope, self._tx_ring)
reserved_sequence = self._next_sequence
envelope = Envelope(self._outlet, message=message, sequence=reserved_sequence)
envelope.pack()
if len(envelope.raw) > self._outlet.mdu:
raise ChannelException(CEType.ME_TOO_BIG, f"Packed message too big for packet: {len(envelope.raw)} > {self._outlet.mdu}")
self._next_sequence = (reserved_sequence + 1) % Channel.SEQ_MODULUS
if envelope is None:
raise BlockingIOError()
envelope.packet = self._outlet.send(envelope.raw)
if envelope.packet is None or getattr(envelope.packet, "raw", None) is None or (hasattr(envelope.packet, "receipt") and envelope.packet.receipt is None):
with self._lock: self._next_sequence = reserved_sequence
raise ChannelException(CEType.ME_LINK_NOT_READY, "Outlet did not transmit packet")
with self._lock:
self._emplace_envelope(envelope, self._tx_ring)
envelope.tries += 1
self._outlet.set_packet_delivered_callback(envelope.packet, self._packet_delivered)
self._outlet.set_packet_timeout_callback(envelope.packet, self._packet_timeout, self._get_packet_timeout_time(envelope.tries))
self._update_packet_timeouts()
already_delivered = (self._outlet.get_packet_state(envelope.packet) == MessageState.MSGSTATE_DELIVERED)
# Prevent _tx_ring envelope leak
if already_delivered: self._packet_delivered(envelope.packet)
envelope.pack()
if len(envelope.raw) > self._outlet.mdu:
raise ChannelException(CEType.ME_TOO_BIG, f"Packed message too big for packet: {len(envelope.raw)} > {self._outlet.mdu}")
envelope.packet = self._outlet.send(envelope.raw)
envelope.tries += 1
self._outlet.set_packet_delivered_callback(envelope.packet, self._packet_delivered)
self._outlet.set_packet_timeout_callback(envelope.packet, self._packet_timeout, self._get_packet_timeout_time(envelope.tries))
self._update_packet_timeouts()
return envelope
@property
def mdu(self):
def MDU(self):
"""
Maximum Data Unit: the number of bytes available
for a message to consume in a single send. This
@ -539,14 +611,12 @@ class Channel(contextlib.AbstractContextManager):
:return: number of bytes available
"""
mdu = self._outlet.mdu - 6 # sizeof(msgtype) + sizeof(length) + sizeof(sequence)
if mdu > 0xFFFF: mdu = 0xFFFF
return mdu
return self._outlet.mdu - 6 # sizeof(msgtype) + sizeof(length) + sizeof(sequence)
class LinkChannelOutlet(ABC, Generic[TPacket]):
class LinkChannelOutlet(ChannelOutletBase):
"""
A channel outlet implementation for RNS.Link.
An implementation of ChannelOutletBase for RNS.Link.
Allows Channel to send packets over an RNS Link with
Packets.
@ -557,51 +627,68 @@ class LinkChannelOutlet(ABC, Generic[TPacket]):
def send(self, raw: bytes) -> RNS.Packet:
packet = RNS.Packet(self.link, raw, context=RNS.Packet.CHANNEL)
if self.link.status == RNS.Link.ACTIVE: packet.send()
if self.link.status == RNS.Link.ACTIVE:
packet.send()
return packet
def resend(self, packet: RNS.Packet) -> RNS.Packet:
receipt = packet.resend()
if not receipt: RNS.log("Failed to resend packet", RNS.LOG_ERROR)
if not receipt:
RNS.log("Failed to resend packet", RNS.LOG_ERROR)
return packet
@property
def mdu(self): return self.link.mdu
def mdu(self):
return self.link.MDU
@property
def rtt(self): return self.link.rtt
def rtt(self):
return self.link.rtt
# TODO: Initial implementation had issues
# proxying this to link status. Should
# return False when link is not ACTIVE.
# Investigate what happened here.
@property
def is_usable(self): return True
def is_usable(self):
return True # had issues looking at Link.status
def get_packet_state(self, packet: TPacket) -> MessageState:
if packet.receipt == None:
return MessageState.MSGSTATE_FAILED
status = packet.receipt.get_status()
if status == RNS.PacketReceipt.SENT: return MessageState.MSGSTATE_SENT
if status == RNS.PacketReceipt.DELIVERED: return MessageState.MSGSTATE_DELIVERED
if status == RNS.PacketReceipt.FAILED: return MessageState.MSGSTATE_FAILED
else: raise Exception(f"Unexpected receipt state: {status}")
if status == RNS.PacketReceipt.SENT:
return MessageState.MSGSTATE_SENT
if status == RNS.PacketReceipt.DELIVERED:
return MessageState.MSGSTATE_DELIVERED
if status == RNS.PacketReceipt.FAILED:
return MessageState.MSGSTATE_FAILED
else:
raise Exception(f"Unexpected receipt state: {status}")
def set_packet_timeout_callback(self, packet: RNS.Packet, callback: Callable[[RNS.Packet], None] | None, timeout: float | None = None):
if timeout and packet.receipt: packet.receipt.set_timeout(timeout)
def inner(receipt: RNS.PacketReceipt): callback(packet)
if packet and packet.receipt: packet.receipt.set_timeout_callback(inner if callback else None)
def timed_out(self):
self.link.teardown()
def __str__(self):
return f"{self.__class__.__name__}({self.link})"
def set_packet_timeout_callback(self, packet: RNS.Packet, callback: Callable[[RNS.Packet], None] | None,
timeout: float | None = None):
if timeout and packet.receipt:
packet.receipt.set_timeout(timeout)
def inner(receipt: RNS.PacketReceipt):
callback(packet)
if packet and packet.receipt:
packet.receipt.set_timeout_callback(inner if callback else None)
def set_packet_delivered_callback(self, packet: RNS.Packet, callback: Callable[[RNS.Packet], None] | None):
def inner(receipt: RNS.PacketReceipt): callback(packet)
if packet and packet.receipt: packet.receipt.set_delivery_callback(inner if callback else None)
def inner(receipt: RNS.PacketReceipt):
callback(packet)
if packet and packet.receipt:
packet.receipt.set_delivery_callback(inner if callback else None)
def get_packet_id(self, packet: RNS.Packet) -> any:
if packet and getattr(packet, "raw", None) is not None and hasattr(packet, "get_hash") and callable(packet.get_hash):
if packet and hasattr(packet, "get_hash") and callable(packet.get_hash):
return packet.get_hash()
else: return None
def timed_out(self): self.link.teardown()
def __str__(self): return f"{self.__class__.__name__}({self.link})"
else:
return None

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2025 Mark Qvist
# Copyright (c) 2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -32,57 +24,23 @@ import RNS.Cryptography.Provider as cp
import RNS.vendor.platformutils as pu
if cp.PROVIDER == cp.PROVIDER_INTERNAL:
from .aes import AES128
from .aes import AES256
from .aes import AES
elif cp.PROVIDER == cp.PROVIDER_PYCA:
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
if pu.cryptography_old_api(): from cryptography.hazmat.backends import default_backend
if pu.cryptography_old_api():
from cryptography.hazmat.backends import default_backend
class AES_128_CBC:
@staticmethod
def encrypt(plaintext, key, iv):
if len(key) != 16: raise ValueError(f"Invalid key length {len(key)*8} for AES_128_CBC")
if cp.PROVIDER == cp.PROVIDER_INTERNAL:
cipher = AES128(key)
cipher = AES(key)
return cipher.encrypt(plaintext, iv)
elif cp.PROVIDER == cp.PROVIDER_PYCA:
if not pu.cryptography_old_api():
cipher = Cipher(algorithms.AES(key), modes.CBC(iv))
else:
cipher = Cipher(algorithms.AES(key), modes.CBC(iv), backend=default_backend())
encryptor = cipher.encryptor()
ciphertext = encryptor.update(plaintext) + encryptor.finalize()
return ciphertext
@staticmethod
def decrypt(ciphertext, key, iv):
if len(key) != 16: raise ValueError(f"Invalid key length {len(key)*8} for AES_128_CBC")
if cp.PROVIDER == cp.PROVIDER_INTERNAL:
cipher = AES128(key)
return cipher.decrypt(ciphertext, iv)
elif cp.PROVIDER == cp.PROVIDER_PYCA:
if not pu.cryptography_old_api():
cipher = Cipher(algorithms.AES(key), modes.CBC(iv))
else:
cipher = Cipher(algorithms.AES(key), modes.CBC(iv), backend=default_backend())
decryptor = cipher.decryptor()
plaintext = decryptor.update(ciphertext) + decryptor.finalize()
return plaintext
class AES_256_CBC:
@staticmethod
def encrypt(plaintext, key, iv):
if len(key) != 32: raise ValueError(f"Invalid key length {len(key)*8} for AES_256_CBC")
if cp.PROVIDER == cp.PROVIDER_INTERNAL:
cipher = AES256(key)
return cipher.encrypt_cbc(plaintext, iv)
elif cp.PROVIDER == cp.PROVIDER_PYCA:
if not pu.cryptography_old_api():
cipher = Cipher(algorithms.AES(key), modes.CBC(iv))
@ -95,10 +53,9 @@ class AES_256_CBC:
@staticmethod
def decrypt(ciphertext, key, iv):
if len(key) != 32: raise ValueError(f"Invalid key length {len(key)*8} for AES_256_CBC")
if cp.PROVIDER == cp.PROVIDER_INTERNAL:
cipher = AES256(key)
return cipher.decrypt_cbc(ciphertext, iv)
cipher = AES(key)
return cipher.decrypt(ciphertext, iv)
elif cp.PROVIDER == cp.PROVIDER_PYCA:
if not pu.cryptography_old_api():

View file

@ -1,33 +1,3 @@
# Reticulum License
#
# Copyright (c) 2016-2025 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import os
from .pure25519 import ed25519_oop as ed25519
@ -35,6 +5,7 @@ class Ed25519PrivateKey:
def __init__(self, seed):
self.seed = seed
self.sk = ed25519.SigningKey(self.seed)
#self.vk = self.sk.get_verifying_key()
@classmethod
def generate(cls):

110
RNS/Cryptography/Fernet.py Normal file
View file

@ -0,0 +1,110 @@
# MIT License
#
# Copyright (c) 2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import os
import time
from RNS.Cryptography import HMAC
from RNS.Cryptography import PKCS7
from RNS.Cryptography.AES import AES_128_CBC
class Fernet():
"""
This class provides a slightly modified implementation of the Fernet spec
found at: https://github.com/fernet/spec/blob/master/Spec.md
According to the spec, a Fernet token includes a one byte VERSION and
eight byte TIMESTAMP field at the start of each token. These fields are
not relevant to Reticulum. They are therefore stripped from this
implementation, since they incur overhead and leak initiator metadata.
"""
FERNET_OVERHEAD = 48 # Bytes
@staticmethod
def generate_key():
return os.urandom(32)
def __init__(self, key = None):
if key == None:
raise ValueError("Token key cannot be None")
if len(key) != 32:
raise ValueError("Token key must be 32 bytes, not "+str(len(key)))
self._signing_key = key[:16]
self._encryption_key = key[16:]
def verify_hmac(self, token):
if len(token) <= 32:
raise ValueError("Cannot verify HMAC on token of only "+str(len(token))+" bytes")
else:
received_hmac = token[-32:]
expected_hmac = HMAC.new(self._signing_key, token[:-32]).digest()
if received_hmac == expected_hmac:
return True
else:
return False
def encrypt(self, data = None):
iv = os.urandom(16)
current_time = int(time.time())
if not isinstance(data, bytes):
raise TypeError("Token plaintext input must be bytes")
ciphertext = AES_128_CBC.encrypt(
plaintext = PKCS7.pad(data),
key = self._encryption_key,
iv = iv,
)
signed_parts = iv+ciphertext
return signed_parts + HMAC.new(self._signing_key, signed_parts).digest()
def decrypt(self, token = None):
if not isinstance(token, bytes):
raise TypeError("Token must be bytes")
if not self.verify_hmac(token):
raise ValueError("Token HMAC was invalid")
iv = token[:16]
ciphertext = token[16:-32]
try:
plaintext = PKCS7.unpad(
AES_128_CBC.decrypt(
ciphertext,
self._encryption_key,
iv,
)
)
return plaintext
except Exception as e:
raise ValueError("Could not decrypt token")

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2025 Mark Qvist
# Copyright (c) 2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -32,59 +24,8 @@ import hashlib
from math import ceil
from RNS.Cryptography import HMAC
# Precomputed byte-permutation tables used to key the inner
# and outer hashing states of the HMAC construction
trans_5C = bytes((x ^ 0x5C) for x in range(256))
trans_36 = bytes((x ^ 0x36) for x in range(256))
# Precomputed HKDF counter bytes
_counters = [bytes([(i + 1) % 0x100]) for i in range(0x100)]
def hkdf(length=None, derive_from=None, salt=None, context=None):
hash_len = 32
hash_block_size = 64
if length == None or length < 1: raise ValueError("Invalid output key length")
if derive_from == None or derive_from == "": raise ValueError("Cannot derive key from empty input material")
if salt == None or len(salt) == 0: salt = bytes([0] * hash_len)
if context == None: context = b""
pseudorandom_key = HMAC.new(salt, derive_from).digest()
# The pseudorandom key is always a 32-byte SHA-256 digest, so the
# HMAC key schedule below never needs the long-key hashing branch.
key = pseudorandom_key.ljust(hash_block_size, b"\x00")
# Precompute the keyed inner and outer hashing states of the
# HMAC construction once. Each expansion block then only requires
# two cheap hash-object copies instead of constructing a complete
# HMAC object per 32 bytes of output. This is possible since the
# PRK is **always** 32-byte SHA-256 HMAC output.
inner_state = hashlib.sha256(key.translate(trans_36))
outer_state = hashlib.sha256(key.translate(trans_5C))
# Expand
has_context = len(context) > 0
block = b""
derived = bytearray()
for i in range(ceil(length / hash_len)):
inner = inner_state.copy()
inner.update(block)
if has_context: inner.update(context)
inner.update(_counters[i & 0xFF])
outer = outer_state.copy()
outer.update(inner.digest())
block = outer.digest()
derived += block
return bytes(derived[:length])
def hkdf_legacy(length=None, derive_from=None, salt=None, context=None):
hash_len = 32
def hmac_sha256(key, data):
return HMAC.new(key, data).digest()
@ -107,7 +48,7 @@ def hkdf_legacy(length=None, derive_from=None, salt=None, context=None):
derived = b""
for i in range(ceil(length / hash_len)):
block = hmac_sha256(pseudorandom_key, block + context + bytes([(i + 1)%(0xFF+1)]))
block = hmac_sha256(pseudorandom_key, block + context + bytes([i + 1]))
derived += block
return derived[:length]

View file

@ -180,9 +180,4 @@ def digest(key, msg, digest):
outer.update(key.translate(trans_5C))
inner.update(msg)
outer.update(inner.digest())
return outer.digest()
def _cd_internal(a, b): return a == b
try: from hmac import compare_digest as _cd
except ImportError: _cd = _cd_internal
compare_digest = _cd
return outer.digest()

View file

@ -1,34 +1,4 @@
# Reticulum License
#
# Copyright (c) 2016-2025 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import importlib.util
import importlib
if importlib.util.find_spec('hashlib') != None:
import hashlib
else:
@ -62,9 +32,3 @@ def sha512(data):
digest.update(data)
return digest.digest()
def file_sha256(file):
if not hashlib: raise SystemError("The hashlib module is not available on this system")
# TODO: Could implement fallback for old snakes here
if not hasattr(hashlib, "file_digest"): raise SystemError("The file_digest method is not available on this system. This functionality requires Python 3.11 or later.")
else: return hashlib.file_digest(file, "sha256").digest()

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2025 Mark Qvist
# Copyright (c) 2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

View file

@ -1,47 +1,16 @@
# Reticulum License
#
# Copyright (c) 2016-2025 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import importlib.util
import importlib
PROVIDER_NONE = 0x00
PROVIDER_INTERNAL = 0x01
PROVIDER_PYCA = 0x02
FORCE_INTERNAL = False
PROVIDER = PROVIDER_NONE
pyca_v = None
use_pyca = False
try:
if not FORCE_INTERNAL and importlib.util.find_spec('cryptography') != None:
if importlib.util.find_spec('cryptography') != None:
import cryptography
pyca_v = cryptography.__version__
v = pyca_v.split(".")

View file

@ -1,33 +1,3 @@
# Reticulum License
#
# Copyright (c) 2016-2025 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey, Ed25519PublicKey
from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey, X25519PublicKey

View file

@ -1,114 +0,0 @@
# Reticulum License
#
# Copyright (c) 2016-2025 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import os
import time
from RNS.Cryptography import HMAC
from RNS.Cryptography import PKCS7
from RNS.Cryptography import AES
from RNS.Cryptography.AES import AES_128_CBC
from RNS.Cryptography.AES import AES_256_CBC
class Token():
"""
This class provides a slightly modified implementation of the Fernet spec
found at: https://github.com/fernet/spec/blob/master/Spec.md
According to the spec, a Fernet token includes a one byte VERSION and
eight byte TIMESTAMP field at the start of each token. These fields are
not relevant to Reticulum. They are therefore stripped from this
implementation, since they incur overhead and leak initiator metadata.
"""
TOKEN_OVERHEAD = 48 # Bytes
@staticmethod
def generate_key(mode=AES_256_CBC):
if mode == AES_128_CBC: return os.urandom(32)
elif mode == AES_256_CBC: return os.urandom(64)
else: raise TypeError(f"Invalid token mode: {mode}")
def __init__(self, key=None, mode=AES):
if key == None: raise ValueError("Token key cannot be None")
if mode == AES:
if len(key) == 32:
self.mode = AES_128_CBC
self._signing_key = key[:16]
self._encryption_key = key[16:]
elif len(key) == 64:
self.mode = AES_256_CBC
self._signing_key = key[:32]
self._encryption_key = key[32:]
else: raise ValueError("Token key must be 128 or 256 bits, not "+str(len(key)*8))
else: raise TypeError(f"Invalid token mode: {mode}")
def verify_hmac(self, token):
if len(token) <= 32: raise ValueError("Cannot verify HMAC on token of only "+str(len(token))+" bytes")
else:
received_hmac = token[-32:]
expected_hmac = HMAC.new(self._signing_key, token[:-32]).digest()
if HMAC.compare_digest(received_hmac, expected_hmac): return True
else: return False
def encrypt(self, data = None):
if not isinstance(data, bytes): raise TypeError("Token plaintext input must be bytes")
iv = os.urandom(16)
ciphertext = self.mode.encrypt(
plaintext = PKCS7.pad(data),
key = self._encryption_key,
iv = iv)
signed_parts = iv+ciphertext
return signed_parts + HMAC.new(self._signing_key, signed_parts).digest()
def decrypt(self, token = None):
if not isinstance(token, bytes): raise TypeError("Token must be bytes")
if not self.verify_hmac(token): raise ValueError("Token HMAC was invalid")
iv = token[:16]
ciphertext = token[16:-32]
try:
return PKCS7.unpad(
self.mode.decrypt(
ciphertext = ciphertext,
key = self._encryption_key,
iv = iv))
except Exception as e: raise ValueError(f"Could not decrypt token: {e}")

View file

@ -82,13 +82,10 @@ def _fix_secret(n):
n |= 64 << 8 * 31
return n
def _fix_base_point(n):
n &= ~(2**255)
return n
def curve25519(base_point_raw, secret_raw):
"""Raise the base point to a given power"""
base_point = _fix_base_point(_unpack_number(base_point_raw))
base_point = _unpack_number(base_point_raw)
secret = _fix_secret(_unpack_number(secret_raw))
return _pack_number(_raw_curve25519(base_point, secret))

View file

@ -1,33 +1,3 @@
# Reticulum License
#
# Copyright (c) 2016-2025 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import os
import glob
@ -35,7 +5,7 @@ from .Hashes import sha256
from .Hashes import sha512
from .HKDF import hkdf
from .PKCS7 import PKCS7
from .Token import Token
from .Fernet import Fernet
from .Provider import backend
import RNS.Cryptography.Provider as cp
@ -50,9 +20,5 @@ elif cp.PROVIDER == cp.PROVIDER_PYCA:
from RNS.Cryptography.Proxies import Ed25519PrivateKeyProxy as Ed25519PrivateKey
from RNS.Cryptography.Proxies import Ed25519PublicKeyProxy as Ed25519PublicKey
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
modules = py_modules+pyc_modules+so_modules+pyd_modules
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))
modules = glob.glob(os.path.dirname(__file__)+"/*.py")
__all__ = [ os.path.basename(f)[:-3] for f in modules if not f.endswith('__init__.py')]

View file

@ -1,2 +1 @@
from .aes128 import AES128
from .aes256 import AES256
from .aes import AES

271
RNS/Cryptography/aes/aes.py Normal file
View file

@ -0,0 +1,271 @@
# MIT License
# Copyright (c) 2021 Or Gur Arie
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from .utils import *
class AES:
# AES-128 block size
block_size = 16
# AES-128 encrypts messages with 10 rounds
_rounds = 10
# initiate the AES objecy
def __init__(self, key):
"""
Initializes the object with a given key.
"""
# make sure key length is right
assert len(key) == AES.block_size
# ExpandKey
self._round_keys = self._expand_key(key)
# will perform the AES ExpandKey phase
def _expand_key(self, master_key):
"""
Expands and returns a list of key matrices for the given master_key.
"""
# Initialize round keys with raw key material.
key_columns = bytes2matrix(master_key)
iteration_size = len(master_key) // 4
# Each iteration has exactly as many columns as the key material.
i = 1
while len(key_columns) < (self._rounds + 1) * 4:
# Copy previous word.
word = list(key_columns[-1])
# Perform schedule_core once every "row".
if len(key_columns) % iteration_size == 0:
# Circular shift.
word.append(word.pop(0))
# Map to S-BOX.
word = [s_box[b] for b in word]
# XOR with first byte of R-CON, since the others bytes of R-CON are 0.
word[0] ^= r_con[i]
i += 1
elif len(master_key) == 32 and len(key_columns) % iteration_size == 4:
# Run word through S-box in the fourth iteration when using a
# 256-bit key.
word = [s_box[b] for b in word]
# XOR with equivalent word from previous iteration.
word = bytes(i^j for i, j in zip(word, key_columns[-iteration_size]))
key_columns.append(word)
# Group key words in 4x4 byte matrices.
return [key_columns[4*i : 4*(i+1)] for i in range(len(key_columns) // 4)]
# encrypt a single block of data with AES
def _encrypt_block(self, plaintext):
"""
Encrypts a single block of 16 byte long plaintext.
"""
# length of a single block
assert len(plaintext) == AES.block_size
# perform on a matrix
state = bytes2matrix(plaintext)
# AddRoundKey
add_round_key(state, self._round_keys[0])
# 9 main rounds
for i in range(1, self._rounds):
# SubBytes
sub_bytes(state)
# ShiftRows
shift_rows(state)
# MixCols
mix_columns(state)
# AddRoundKey
add_round_key(state, self._round_keys[i])
# last round, w/t AddRoundKey step
sub_bytes(state)
shift_rows(state)
add_round_key(state, self._round_keys[-1])
# return the encrypted matrix as bytes
return matrix2bytes(state)
# decrypt a single block of data with AES
def _decrypt_block(self, ciphertext):
"""
Decrypts a single block of 16 byte long ciphertext.
"""
# length of a single block
assert len(ciphertext) == AES.block_size
# perform on a matrix
state = bytes2matrix(ciphertext)
# in reverse order, last round is first
add_round_key(state, self._round_keys[-1])
inv_shift_rows(state)
inv_sub_bytes(state)
for i in range(self._rounds - 1, 0, -1):
# nain rounds
add_round_key(state, self._round_keys[i])
inv_mix_columns(state)
inv_shift_rows(state)
inv_sub_bytes(state)
# initial AddRoundKey phase
add_round_key(state, self._round_keys[0])
# return bytes
return matrix2bytes(state)
# will encrypt the entire data
def encrypt(self, plaintext, iv):
"""
Encrypts `plaintext` using CBC mode and PKCS#7 padding, with the given
initialization vector (iv).
"""
# iv length must be same as block size
assert len(iv) == AES.block_size
assert len(plaintext) % AES.block_size == 0
ciphertext_blocks = []
previous = iv
for plaintext_block in split_blocks(plaintext):
# in CBC mode every block is XOR'd with the previous block
xorred = xor_bytes(plaintext_block, previous)
# encrypt current block
block = self._encrypt_block(xorred)
previous = block
# append to ciphertext
ciphertext_blocks.append(block)
# return as bytes
return b''.join(ciphertext_blocks)
# will decrypt the entire data
def decrypt(self, ciphertext, iv):
"""
Decrypts `ciphertext` using CBC mode and PKCS#7 padding, with the given
initialization vector (iv).
"""
# iv length must be same as block size
assert len(iv) == AES.block_size
plaintext_blocks = []
previous = iv
for ciphertext_block in split_blocks(ciphertext):
# in CBC mode every block is XOR'd with the previous block
xorred = xor_bytes(previous, self._decrypt_block(ciphertext_block))
# append plaintext
plaintext_blocks.append(xorred)
previous = ciphertext_block
return b''.join(plaintext_blocks)
def test():
# modules and classes requiered for test only
import os
class bcolors:
OK = '\033[92m' #GREEN
WARNING = '\033[93m' #YELLOW
FAIL = '\033[91m' #RED
RESET = '\033[0m' #RESET COLOR
# will test AES class by performing an encryption / decryption
print("AES Tests")
print("=========")
# generate a secret key and print details
key = os.urandom(AES.block_size)
_aes = AES(key)
print(f"Algorithm: AES-CBC-{AES.block_size*8}")
print(f"Secret Key: {key.hex()}")
print()
# test single block encryption / decryption
iv = os.urandom(AES.block_size)
single_block_text = b"SingleBlock Text"
print("Single Block Tests")
print("------------------")
print(f"iv: {iv.hex()}")
print(f"plain text: '{single_block_text.decode()}'")
ciphertext_block = _aes._encrypt_block(single_block_text)
plaintext_block = _aes._decrypt_block(ciphertext_block)
print(f"Ciphertext Hex: {ciphertext_block.hex()}")
print(f"Plaintext: {plaintext_block.decode()}")
assert plaintext_block == single_block_text
print(bcolors.OK + "Single Block Test Passed Successfully" + bcolors.RESET)
print()
# test a less than a block length phrase
iv = os.urandom(AES.block_size)
short_text = b"Just Text"
print("Short Text Tests")
print("----------------")
print(f"iv: {iv.hex()}")
print(f"plain text: '{short_text.decode()}'")
ciphertext_short = _aes.encrypt(short_text, iv)
plaintext_short = _aes.decrypt(ciphertext_short, iv)
print(f"Ciphertext Hex: {ciphertext_short.hex()}")
print(f"Plaintext: {plaintext_short.decode()}")
assert short_text == plaintext_short
print(bcolors.OK + "Short Text Test Passed Successfully" + bcolors.RESET)
print()
# test an arbitrary length phrase
iv = os.urandom(AES.block_size)
text = b"This Text is longer than one block"
print("Arbitrary Length Tests")
print("----------------------")
print(f"iv: {iv.hex()}")
print(f"plain text: '{text.decode()}'")
ciphertext = _aes.encrypt(text, iv)
plaintext = _aes.decrypt(ciphertext, iv)
print(f"Ciphertext Hex: {ciphertext.hex()}")
print(f"Plaintext: {plaintext.decode()}")
assert text == plaintext
print(bcolors.OK + "Arbitrary Length Text Test Passed Successfully" + bcolors.RESET)
print()
if __name__ == "__main__":
# test AES class
test()

View file

@ -1,326 +0,0 @@
# MIT License
# Copyright (c) 2021 Or Gur Arie
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
## AES lookup tables
# resource: https://en.wikipedia.org/wiki/Rijndael_S-box
s_box = (
0x63, 0x7C, 0x77, 0x7B, 0xF2, 0x6B, 0x6F, 0xC5, 0x30, 0x01, 0x67, 0x2B, 0xFE, 0xD7, 0xAB, 0x76,
0xCA, 0x82, 0xC9, 0x7D, 0xFA, 0x59, 0x47, 0xF0, 0xAD, 0xD4, 0xA2, 0xAF, 0x9C, 0xA4, 0x72, 0xC0,
0xB7, 0xFD, 0x93, 0x26, 0x36, 0x3F, 0xF7, 0xCC, 0x34, 0xA5, 0xE5, 0xF1, 0x71, 0xD8, 0x31, 0x15,
0x04, 0xC7, 0x23, 0xC3, 0x18, 0x96, 0x05, 0x9A, 0x07, 0x12, 0x80, 0xE2, 0xEB, 0x27, 0xB2, 0x75,
0x09, 0x83, 0x2C, 0x1A, 0x1B, 0x6E, 0x5A, 0xA0, 0x52, 0x3B, 0xD6, 0xB3, 0x29, 0xE3, 0x2F, 0x84,
0x53, 0xD1, 0x00, 0xED, 0x20, 0xFC, 0xB1, 0x5B, 0x6A, 0xCB, 0xBE, 0x39, 0x4A, 0x4C, 0x58, 0xCF,
0xD0, 0xEF, 0xAA, 0xFB, 0x43, 0x4D, 0x33, 0x85, 0x45, 0xF9, 0x02, 0x7F, 0x50, 0x3C, 0x9F, 0xA8,
0x51, 0xA3, 0x40, 0x8F, 0x92, 0x9D, 0x38, 0xF5, 0xBC, 0xB6, 0xDA, 0x21, 0x10, 0xFF, 0xF3, 0xD2,
0xCD, 0x0C, 0x13, 0xEC, 0x5F, 0x97, 0x44, 0x17, 0xC4, 0xA7, 0x7E, 0x3D, 0x64, 0x5D, 0x19, 0x73,
0x60, 0x81, 0x4F, 0xDC, 0x22, 0x2A, 0x90, 0x88, 0x46, 0xEE, 0xB8, 0x14, 0xDE, 0x5E, 0x0B, 0xDB,
0xE0, 0x32, 0x3A, 0x0A, 0x49, 0x06, 0x24, 0x5C, 0xC2, 0xD3, 0xAC, 0x62, 0x91, 0x95, 0xE4, 0x79,
0xE7, 0xC8, 0x37, 0x6D, 0x8D, 0xD5, 0x4E, 0xA9, 0x6C, 0x56, 0xF4, 0xEA, 0x65, 0x7A, 0xAE, 0x08,
0xBA, 0x78, 0x25, 0x2E, 0x1C, 0xA6, 0xB4, 0xC6, 0xE8, 0xDD, 0x74, 0x1F, 0x4B, 0xBD, 0x8B, 0x8A,
0x70, 0x3E, 0xB5, 0x66, 0x48, 0x03, 0xF6, 0x0E, 0x61, 0x35, 0x57, 0xB9, 0x86, 0xC1, 0x1D, 0x9E,
0xE1, 0xF8, 0x98, 0x11, 0x69, 0xD9, 0x8E, 0x94, 0x9B, 0x1E, 0x87, 0xE9, 0xCE, 0x55, 0x28, 0xDF,
0x8C, 0xA1, 0x89, 0x0D, 0xBF, 0xE6, 0x42, 0x68, 0x41, 0x99, 0x2D, 0x0F, 0xB0, 0x54, 0xBB, 0x16,
)
inv_s_box = (
0x52, 0x09, 0x6A, 0xD5, 0x30, 0x36, 0xA5, 0x38, 0xBF, 0x40, 0xA3, 0x9E, 0x81, 0xF3, 0xD7, 0xFB,
0x7C, 0xE3, 0x39, 0x82, 0x9B, 0x2F, 0xFF, 0x87, 0x34, 0x8E, 0x43, 0x44, 0xC4, 0xDE, 0xE9, 0xCB,
0x54, 0x7B, 0x94, 0x32, 0xA6, 0xC2, 0x23, 0x3D, 0xEE, 0x4C, 0x95, 0x0B, 0x42, 0xFA, 0xC3, 0x4E,
0x08, 0x2E, 0xA1, 0x66, 0x28, 0xD9, 0x24, 0xB2, 0x76, 0x5B, 0xA2, 0x49, 0x6D, 0x8B, 0xD1, 0x25,
0x72, 0xF8, 0xF6, 0x64, 0x86, 0x68, 0x98, 0x16, 0xD4, 0xA4, 0x5C, 0xCC, 0x5D, 0x65, 0xB6, 0x92,
0x6C, 0x70, 0x48, 0x50, 0xFD, 0xED, 0xB9, 0xDA, 0x5E, 0x15, 0x46, 0x57, 0xA7, 0x8D, 0x9D, 0x84,
0x90, 0xD8, 0xAB, 0x00, 0x8C, 0xBC, 0xD3, 0x0A, 0xF7, 0xE4, 0x58, 0x05, 0xB8, 0xB3, 0x45, 0x06,
0xD0, 0x2C, 0x1E, 0x8F, 0xCA, 0x3F, 0x0F, 0x02, 0xC1, 0xAF, 0xBD, 0x03, 0x01, 0x13, 0x8A, 0x6B,
0x3A, 0x91, 0x11, 0x41, 0x4F, 0x67, 0xDC, 0xEA, 0x97, 0xF2, 0xCF, 0xCE, 0xF0, 0xB4, 0xE6, 0x73,
0x96, 0xAC, 0x74, 0x22, 0xE7, 0xAD, 0x35, 0x85, 0xE2, 0xF9, 0x37, 0xE8, 0x1C, 0x75, 0xDF, 0x6E,
0x47, 0xF1, 0x1A, 0x71, 0x1D, 0x29, 0xC5, 0x89, 0x6F, 0xB7, 0x62, 0x0E, 0xAA, 0x18, 0xBE, 0x1B,
0xFC, 0x56, 0x3E, 0x4B, 0xC6, 0xD2, 0x79, 0x20, 0x9A, 0xDB, 0xC0, 0xFE, 0x78, 0xCD, 0x5A, 0xF4,
0x1F, 0xDD, 0xA8, 0x33, 0x88, 0x07, 0xC7, 0x31, 0xB1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xEC, 0x5F,
0x60, 0x51, 0x7F, 0xA9, 0x19, 0xB5, 0x4A, 0x0D, 0x2D, 0xE5, 0x7A, 0x9F, 0x93, 0xC9, 0x9C, 0xEF,
0xA0, 0xE0, 0x3B, 0x4D, 0xAE, 0x2A, 0xF5, 0xB0, 0xC8, 0xEB, 0xBB, 0x3C, 0x83, 0x53, 0x99, 0x61,
0x17, 0x2B, 0x04, 0x7E, 0xBA, 0x77, 0xD6, 0x26, 0xE1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0C, 0x7D,
)
## AES AddRoundKey
# Round constants https://en.wikipedia.org/wiki/AES_key_schedule#Round_constants
r_con = (
0x00, 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40,
0x80, 0x1B, 0x36, 0x6C, 0xD8, 0xAB, 0x4D, 0x9A,
0x2F, 0x5E, 0xBC, 0x63, 0xC6, 0x97, 0x35, 0x6A,
0xD4, 0xB3, 0x7D, 0xFA, 0xEF, 0xC5, 0x91, 0x39,
)
def add_round_key(s, k):
for i in range(4):
for j in range(4):
s[i][j] ^= k[i][j]
## AES SubBytes
def sub_bytes(s):
for i in range(4):
for j in range(4):
s[i][j] = s_box[s[i][j]]
def inv_sub_bytes(s):
for i in range(4):
for j in range(4):
s[i][j] = inv_s_box[s[i][j]]
## AES ShiftRows
def shift_rows(s):
s[0][1], s[1][1], s[2][1], s[3][1] = s[1][1], s[2][1], s[3][1], s[0][1]
s[0][2], s[1][2], s[2][2], s[3][2] = s[2][2], s[3][2], s[0][2], s[1][2]
s[0][3], s[1][3], s[2][3], s[3][3] = s[3][3], s[0][3], s[1][3], s[2][3]
def inv_shift_rows(s):
s[0][1], s[1][1], s[2][1], s[3][1] = s[3][1], s[0][1], s[1][1], s[2][1]
s[0][2], s[1][2], s[2][2], s[3][2] = s[2][2], s[3][2], s[0][2], s[1][2]
s[0][3], s[1][3], s[2][3], s[3][3] = s[1][3], s[2][3], s[3][3], s[0][3]
## AES MixColumns
# learned from http://cs.ucsb.edu/~koc/cs178/projects/JT/aes.c
xtime = lambda a: (((a << 1) ^ 0x1B) & 0xFF) if (a & 0x80) else (a << 1)
def mix_single_column(a):
# see Sec 4.1.2 in The Design of Rijndael
t = a[0] ^ a[1] ^ a[2] ^ a[3]
u = a[0]
a[0] ^= t ^ xtime(a[0] ^ a[1])
a[1] ^= t ^ xtime(a[1] ^ a[2])
a[2] ^= t ^ xtime(a[2] ^ a[3])
a[3] ^= t ^ xtime(a[3] ^ u)
def mix_columns(s):
for i in range(4):
mix_single_column(s[i])
def inv_mix_columns(s):
# see Sec 4.1.3 in The Design of Rijndael
for i in range(4):
u = xtime(xtime(s[i][0] ^ s[i][2]))
v = xtime(xtime(s[i][1] ^ s[i][3]))
s[i][0] ^= u
s[i][1] ^= v
s[i][2] ^= u
s[i][3] ^= v
mix_columns(s)
## AES Bytes
def bytes2matrix(text):
""" Converts a 16-byte array into a 4x4 matrix. """
return [list(text[i:i+4]) for i in range(0, len(text), 4)]
def matrix2bytes(matrix):
""" Converts a 4x4 matrix into a 16-byte array. """
return bytes(sum(matrix, []))
def xor_bytes(a, b):
""" Returns a new byte array with the elements xor'ed. """
return bytes(i^j for i, j in zip(a, b))
def split_blocks(message, block_size=16, require_padding=True):
assert len(message) % block_size == 0 or not require_padding
return [message[i:i+16] for i in range(0, len(message), block_size)]
class AES128:
# AES-128 block size
block_size = 16
# AES-128 encrypts messages with 10 rounds
_rounds = 10
# initiate the AES objecy
def __init__(self, key):
"""
Initializes the object with a given key.
"""
# make sure key length is right
assert len(key) == AES128.block_size
# ExpandKey
self._round_keys = self._expand_key(key)
# will perform the AES ExpandKey phase
def _expand_key(self, master_key):
"""
Expands and returns a list of key matrices for the given master_key.
"""
# Initialize round keys with raw key material.
key_columns = bytes2matrix(master_key)
iteration_size = len(master_key) // 4
# Each iteration has exactly as many columns as the key material.
i = 1
while len(key_columns) < (self._rounds + 1) * 4:
# Copy previous word.
word = list(key_columns[-1])
# Perform schedule_core once every "row".
if len(key_columns) % iteration_size == 0:
# Circular shift.
word.append(word.pop(0))
# Map to S-BOX.
word = [s_box[b] for b in word]
# XOR with first byte of R-CON, since the others bytes of R-CON are 0.
word[0] ^= r_con[i]
i += 1
elif len(master_key) == 32 and len(key_columns) % iteration_size == 4:
# Run word through S-box in the fourth iteration when using a
# 256-bit key.
word = [s_box[b] for b in word]
# XOR with equivalent word from previous iteration.
word = bytes(i^j for i, j in zip(word, key_columns[-iteration_size]))
key_columns.append(word)
# Group key words in 4x4 byte matrices.
return [key_columns[4*i : 4*(i+1)] for i in range(len(key_columns) // 4)]
# encrypt a single block of data with AES
def _encrypt_block(self, plaintext):
"""
Encrypts a single block of 16 byte long plaintext.
"""
# length of a single block
assert len(plaintext) == AES128.block_size
# perform on a matrix
state = bytes2matrix(plaintext)
# AddRoundKey
add_round_key(state, self._round_keys[0])
# 9 main rounds
for i in range(1, self._rounds):
# SubBytes
sub_bytes(state)
# ShiftRows
shift_rows(state)
# MixCols
mix_columns(state)
# AddRoundKey
add_round_key(state, self._round_keys[i])
# last round, w/t AddRoundKey step
sub_bytes(state)
shift_rows(state)
add_round_key(state, self._round_keys[-1])
# return the encrypted matrix as bytes
return matrix2bytes(state)
# decrypt a single block of data with AES
def _decrypt_block(self, ciphertext):
"""
Decrypts a single block of 16 byte long ciphertext.
"""
# length of a single block
assert len(ciphertext) == AES128.block_size
# perform on a matrix
state = bytes2matrix(ciphertext)
# in reverse order, last round is first
add_round_key(state, self._round_keys[-1])
inv_shift_rows(state)
inv_sub_bytes(state)
for i in range(self._rounds - 1, 0, -1):
# nain rounds
add_round_key(state, self._round_keys[i])
inv_mix_columns(state)
inv_shift_rows(state)
inv_sub_bytes(state)
# initial AddRoundKey phase
add_round_key(state, self._round_keys[0])
# return bytes
return matrix2bytes(state)
# will encrypt the entire data
def encrypt(self, plaintext, iv):
"""
Encrypts `plaintext` using CBC mode and PKCS#7 padding, with the given
initialization vector (iv).
"""
# iv length must be same as block size
assert len(iv) == AES128.block_size
assert len(plaintext) % AES128.block_size == 0
ciphertext_blocks = []
previous = iv
for plaintext_block in split_blocks(plaintext):
# in CBC mode every block is XOR'd with the previous block
xorred = xor_bytes(plaintext_block, previous)
# encrypt current block
block = self._encrypt_block(xorred)
previous = block
# append to ciphertext
ciphertext_blocks.append(block)
# return as bytes
return b''.join(ciphertext_blocks)
# will decrypt the entire data
def decrypt(self, ciphertext, iv):
"""
Decrypts `ciphertext` using CBC mode and PKCS#7 padding, with the given
initialization vector (iv).
"""
# iv length must be same as block size
assert len(iv) == AES128.block_size
plaintext_blocks = []
previous = iv
for ciphertext_block in split_blocks(ciphertext):
# in CBC mode every block is XOR'd with the previous block
xorred = xor_bytes(previous, self._decrypt_block(ciphertext_block))
# append plaintext
plaintext_blocks.append(xorred)
previous = ciphertext_block
return b''.join(plaintext_blocks)

View file

@ -1,17 +1,17 @@
# MIT License
#
# Copyright (c) 2024 BoppreH
#
# Copyright (c) 2021 Or Gur Arie
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
@ -20,6 +20,12 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
'''
Utils class for AES encryption / decryption
'''
## AES lookup tables
# resource: https://en.wikipedia.org/wiki/Rijndael_S-box
s_box = (
0x63, 0x7C, 0x77, 0x7B, 0xF2, 0x6B, 0x6F, 0xC5, 0x30, 0x01, 0x67, 0x2B, 0xFE, 0xD7, 0xAB, 0x76,
0xCA, 0x82, 0xC9, 0x7D, 0xFA, 0x59, 0x47, 0xF0, 0xAD, 0xD4, 0xA2, 0xAF, 0x9C, 0xA4, 0x72, 0xC0,
@ -58,33 +64,53 @@ inv_s_box = (
0x17, 0x2B, 0x04, 0x7E, 0xBA, 0x77, 0xD6, 0x26, 0xE1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0C, 0x7D,
)
def sub_bytes(s):
for i in range(4):
for j in range(4):
s[i][j] = s_box[s[i][j]]
def inv_sub_bytes(s):
for i in range(4):
for j in range(4):
s[i][j] = inv_s_box[s[i][j]]
def shift_rows(s):
s[0][1], s[1][1], s[2][1], s[3][1] = s[1][1], s[2][1], s[3][1], s[0][1]
s[0][2], s[1][2], s[2][2], s[3][2] = s[2][2], s[3][2], s[0][2], s[1][2]
s[0][3], s[1][3], s[2][3], s[3][3] = s[3][3], s[0][3], s[1][3], s[2][3]
def inv_shift_rows(s):
s[0][1], s[1][1], s[2][1], s[3][1] = s[3][1], s[0][1], s[1][1], s[2][1]
s[0][2], s[1][2], s[2][2], s[3][2] = s[2][2], s[3][2], s[0][2], s[1][2]
s[0][3], s[1][3], s[2][3], s[3][3] = s[1][3], s[2][3], s[3][3], s[0][3]
## AES AddRoundKey
# Round constants https://en.wikipedia.org/wiki/AES_key_schedule#Round_constants
r_con = (
0x00, 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40,
0x80, 0x1B, 0x36, 0x6C, 0xD8, 0xAB, 0x4D, 0x9A,
0x2F, 0x5E, 0xBC, 0x63, 0xC6, 0x97, 0x35, 0x6A,
0xD4, 0xB3, 0x7D, 0xFA, 0xEF, 0xC5, 0x91, 0x39,
)
def add_round_key(s, k):
for i in range(4):
for j in range(4):
s[i][j] ^= k[i][j]
## AES SubBytes
def sub_bytes(s):
for i in range(4):
for j in range(4):
s[i][j] = s_box[s[i][j]]
def inv_sub_bytes(s):
for i in range(4):
for j in range(4):
s[i][j] = inv_s_box[s[i][j]]
## AES ShiftRows
def shift_rows(s):
s[0][1], s[1][1], s[2][1], s[3][1] = s[1][1], s[2][1], s[3][1], s[0][1]
s[0][2], s[1][2], s[2][2], s[3][2] = s[2][2], s[3][2], s[0][2], s[1][2]
s[0][3], s[1][3], s[2][3], s[3][3] = s[3][3], s[0][3], s[1][3], s[2][3]
def inv_shift_rows(s):
s[0][1], s[1][1], s[2][1], s[3][1] = s[3][1], s[0][1], s[1][1], s[2][1]
s[0][2], s[1][2], s[2][2], s[3][2] = s[2][2], s[3][2], s[0][2], s[1][2]
s[0][3], s[1][3], s[2][3], s[3][3] = s[1][3], s[2][3], s[3][3], s[0][3]
## AES MixColumns
# learned from http://cs.ucsb.edu/~koc/cs178/projects/JT/aes.c
xtime = lambda a: (((a << 1) ^ 0x1B) & 0xFF) if (a & 0x80) else (a << 1)
def mix_single_column(a):
# see Sec 4.1.2 in The Design of Rijndael
t = a[0] ^ a[1] ^ a[2] ^ a[3]
@ -94,10 +120,12 @@ def mix_single_column(a):
a[2] ^= t ^ xtime(a[2] ^ a[3])
a[3] ^= t ^ xtime(a[3] ^ u)
def mix_columns(s):
for i in range(4):
mix_single_column(s[i])
def inv_mix_columns(s):
# see Sec 4.1.3 in The Design of Rijndael
for i in range(4):
@ -110,127 +138,22 @@ def inv_mix_columns(s):
mix_columns(s)
r_con = (
0x00, 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40,
0x80, 0x1B, 0x36, 0x6C, 0xD8, 0xAB, 0x4D, 0x9A,
0x2F, 0x5E, 0xBC, 0x63, 0xC6, 0x97, 0x35, 0x6A,
0xD4, 0xB3, 0x7D, 0xFA, 0xEF, 0xC5, 0x91, 0x39,
)
## AES Bytes
def bytes2matrix(text):
""" Converts a 16-byte array into a 4x4 matrix. """
return [list(text[i:i+4]) for i in range(0, len(text), 4)]
def matrix2bytes(matrix):
""" Converts a 4x4 matrix into a 16-byte array. """
return bytes(sum(matrix, []))
def bytes2matrix(text): return [list(text[i:i+4]) for i in range(0, len(text), 4)]
def matrix2bytes(matrix): return bytes(sum(matrix, []))
def xor_bytes(a, b): return bytes(i^j for i, j in zip(a, b))
def xor_bytes(a, b):
""" Returns a new byte array with the elements xor'ed. """
return bytes(i^j for i, j in zip(a, b))
def inc_bytes(a):
out = list(a)
for i in reversed(range(len(out))):
if out[i] == 0xFF:
out[i] = 0
else:
out[i] += 1
break
return bytes(out)
def split_blocks(message, block_size=16, require_padding=True):
assert len(message) % block_size == 0 or not require_padding
return [message[i:i+16] for i in range(0, len(message), block_size)]
class AES256:
rounds_by_key_size = {32: 14}
def __init__(self, master_key):
assert len(master_key) in AES256.rounds_by_key_size
self.n_rounds = AES256.rounds_by_key_size[len(master_key)]
self._key_matrices = self._expand_key(master_key)
def _expand_key(self, master_key):
# Initialize round keys with raw key material.
key_columns = bytes2matrix(master_key)
iteration_size = len(master_key) // 4
i = 1
while len(key_columns) < (self.n_rounds + 1) * 4:
# Copy previous word.
word = list(key_columns[-1])
# Perform schedule_core once every "row".
if len(key_columns) % iteration_size == 0:
# Circular shift.
word.append(word.pop(0))
# Map to S-BOX.
word = [s_box[b] for b in word]
# XOR with first byte of R-CON, since the others bytes of R-CON are 0.
word[0] ^= r_con[i]
i += 1
elif len(master_key) == 32 and len(key_columns) % iteration_size == 4:
# Run word through S-box in the fourth iteration when using a
# 256-bit key.
word = [s_box[b] for b in word]
# XOR with equivalent word from previous iteration.
word = xor_bytes(word, key_columns[-iteration_size])
key_columns.append(word)
# Group key words in 4x4 byte matrices.
return [key_columns[4*i : 4*(i+1)] for i in range(len(key_columns) // 4)]
def encrypt_block(self, plaintext):
assert len(plaintext) == 16
plain_state = bytes2matrix(plaintext)
add_round_key(plain_state, self._key_matrices[0])
for i in range(1, self.n_rounds):
sub_bytes(plain_state)
shift_rows(plain_state)
mix_columns(plain_state)
add_round_key(plain_state, self._key_matrices[i])
sub_bytes(plain_state)
shift_rows(plain_state)
add_round_key(plain_state, self._key_matrices[-1])
return matrix2bytes(plain_state)
def decrypt_block(self, ciphertext):
assert len(ciphertext) == 16
cipher_state = bytes2matrix(ciphertext)
add_round_key(cipher_state, self._key_matrices[-1])
inv_shift_rows(cipher_state)
inv_sub_bytes(cipher_state)
for i in range(self.n_rounds - 1, 0, -1):
add_round_key(cipher_state, self._key_matrices[i])
inv_mix_columns(cipher_state)
inv_shift_rows(cipher_state)
inv_sub_bytes(cipher_state)
add_round_key(cipher_state, self._key_matrices[0])
return matrix2bytes(cipher_state)
def encrypt_cbc(self, plaintext, iv):
if len(iv) != 16: raise ValueError(f"Invalid IV length: {len(iv)}")
blocks = []
previous = iv
for plaintext_block in split_blocks(plaintext):
block = self.encrypt_block(xor_bytes(plaintext_block, previous))
blocks.append(block)
previous = block
return b''.join(blocks)
def decrypt_cbc(self, ciphertext, iv):
if len(iv) != 16: raise ValueError(f"Invalid IV length: {len(iv)}")
blocks = []
previous = iv
for ciphertext_block in split_blocks(ciphertext):
blocks.append(xor_bytes(previous, self.decrypt_block(ciphertext_block)))
previous = ciphertext_block
return b''.join(blocks)
__all__ = ["AES256"]
assert len(message) % block_size == 0 or not require_padding
return [message[i:i+16] for i in range(0, len(message), block_size)]

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2024 Mark Qvist / unsigned.io and contributors
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -34,9 +26,7 @@ import time
import threading
import RNS
from RNS.Cryptography import Token
from RNS.Stamps import generate_stamp, store_destination_stamp, load_destination_stamp
from RNS.Stamps import cost_to_bytes, quantize_cost
from RNS.Cryptography import Fernet
from .vendor import umsgpack as umsgpack
class Callbacks:
@ -148,14 +138,6 @@ class Destination:
return Destination.hash(identity, app_name, *aspects)
@staticmethod
def name_hash(full_name):
components = Destination.app_and_aspects_from_name(full_name)
app_name = str(components[0])
aspects = f".{'.'.join(components[1])}" if components[1] else ""
name = f"{app_name}{aspects}".encode("utf-8")
return RNS.Identity.full_hash(name)[:(RNS.Identity.NAME_HASH_LENGTH//8)]
def __init__(self, identity, direction, type, app_name, *aspects):
# Check input values and build name string
if "." in app_name: raise ValueError("Dots can't be used in app names")
@ -165,15 +147,9 @@ class Destination:
self.accept_link_requests = True
self.callbacks = Callbacks()
self.request_handlers = {}
self.max_request_size = None
self.type = type
self.direction = direction
self.proof_strategy = Destination.PROVE_NONE
self.stamp = None
self.stamp_value = None
self.stamp_generation_lock = threading.Lock()
self._target_value = RNS.Transport.DESTINATION_VALUE
self._link_cost = None
self.ratchets = None
self.ratchets_path = None
self.ratchet_interval = Destination.RATCHET_INTERVAL
@ -205,13 +181,6 @@ class Destination:
self.name_hash = RNS.Identity.full_hash(self.expand_name(None, app_name, *aspects).encode("utf-8"))[:(RNS.Identity.NAME_HASH_LENGTH//8)]
self.hexhash = self.hash.hex()
# Load existing stamp if available
if self.direction == Destination.IN and self.identity != None and self.identity.prv != None and RNS.Transport.STAMP_DESTINATIONS:
stamp_entry = load_destination_stamp(self.hash, self.stamp_material(), RNS.Transport.DESTINATION_COST)
if stamp_entry != None:
self.stamp, self.stamp_value = stamp_entry
RNS.log(f"Loaded stamp with value {RNS.prettycost(self.stamp_value)} for {self}", RNS.LOG_DEBUG)
self.default_app_data = None
self.callback = None
self.proofcallback = None
@ -223,26 +192,22 @@ class Destination:
"""
:returns: A human-readable representation of the destination including addressable hash and full name.
"""
return "<"+self.name+":"+self.hexhash+">"
return "<"+self.name+"/"+self.hexhash+">"
def _clean_ratchets(self):
if self.ratchets != None:
if len (self.ratchets) > self.retained_ratchets:
self.ratchets = self.ratchets[:self.retained_ratchets]
self.ratchets = self.ratchets[:Destination.RATCHET_COUNT]
def _persist_ratchets(self):
try:
with self.ratchet_file_lock:
temp_write_path = self.ratchets_path+".tmp"
packed_ratchets = umsgpack.packb(self.ratchets)
persisted_data = {"signature": self.sign(packed_ratchets), "ratchets": packed_ratchets}
ratchets_file = open(temp_write_path, "wb")
ratchets_file = open(self.ratchets_path, "wb")
ratchets_file.write(umsgpack.packb(persisted_data))
ratchets_file.close()
if os.path.isfile(self.ratchets_path): os.unlink(self.ratchets_path)
os.rename(temp_write_path, self.ratchets_path)
except Exception as e:
RNS.trace_exception(e)
self.ratchets = None
self.ratchets_path = None
raise OSError("Could not write ratchet file contents for "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
@ -263,26 +228,6 @@ class Destination:
return False
def _await_stamp(self):
with self.stamp_generation_lock:
if self.stamp == None or self.stamp_value == None or self.stamp_value < quantize_cost(self.target_value):
stamp_entry = load_destination_stamp(self.hash, self.stamp_material(), self.target_value)
if stamp_entry != None: self.stamp, self.stamp_value = stamp_entry
else:
stamp, stamp_value = generate_stamp(self.stamp_material(), stamp_cost=self.target_value,
expand_rounds=0, size=RNS.Identity.STAMPLENGTH)
if stamp == None:
RNS.log(f"Destination stamp generation for {RNS.prettyhexrep(self.hash)} failed or was aborted", RNS.LOG_DEBUG)
return False
self.stamp = stamp
self.stamp_value = stamp_value
store_destination_stamp(self.hash, stamp, stamp_value)
RNS.log(f"Generated and persisted stamp with value {self.stamp_value} for {self}", RNS.LOG_DEBUG)
if self.stamp == None: return False
else: return True
def announce(self, app_data=None, path_response=False, attached_interface=None, tag=None, send=True):
"""
Creates an announce packet for this destination and broadcasts it on all
@ -291,20 +236,12 @@ class Destination:
:param app_data: *bytes* containing the app_data.
:param path_response: Internal flag used by :ref:`RNS.Transport<api-transport>`. Ignore.
"""
if self.type != Destination.SINGLE: raise TypeError("Only SINGLE destination types can be announced")
if self.direction != Destination.IN: raise TypeError("Only IN destination types can be announced")
if self.type != Destination.SINGLE:
raise TypeError("Only SINGLE destination types can be announced")
if send and RNS.Transport.STAMP_DESTINATIONS and self.stamp == None:
# If no stamp is available, queue for background generation
# and announce emission once ready.
RNS.Transport.enqueue_stamped_announce(self, app_data=app_data, path_response=path_response,
attached_interface=attached_interface, tag=tag)
else:
# Otherwise, dispatch immediately.
return self._announce(app_data=app_data, path_response=path_response,
attached_interface=attached_interface, tag=tag, send=send)
def _announce(self, app_data=None, path_response=False, attached_interface=None, tag=None, send=True):
if self.direction != Destination.IN:
raise TypeError("Only IN destination types can be announced")
ratchet = b""
now = time.time()
stale_responses = []
@ -344,77 +281,35 @@ class Destination:
returned_app_data = self.default_app_data()
if isinstance(returned_app_data, bytes):
app_data = returned_app_data
if RNS.Transport.STAMP_DESTINATIONS:
if not self._await_stamp():
RNS.log(f"Could not generate destination stamp for {RNS.prettyhexrep(self.hash)}, announce emission cancelled", RNS.LOG_WARNING)
return None
cost_bytes = cost_to_bytes(self.get_link_cost())
ext_payload = bytes([RNS.Identity.EXTPAYLOADMARKER, self.get_ext_flags()])+self.stamp+bytes([cost_bytes[1], cost_bytes[0]])
if app_data == None: app_data = ext_payload
else: app_data = app_data+ext_payload
signed_data = self.hash+self.identity.get_public_key()+self.name_hash+random_hash+ratchet
if app_data != None: signed_data += app_data
if app_data != None:
signed_data += app_data
signature = self.identity.sign(signed_data)
announce_data = self.identity.get_public_key()+self.name_hash+random_hash+ratchet+signature
if app_data != None: announce_data += app_data
if app_data != None:
announce_data += app_data
self.path_responses[tag] = [time.time(), announce_data]
if path_response: announce_context = RNS.Packet.PATH_RESPONSE
else: announce_context = RNS.Packet.NONE
if path_response:
announce_context = RNS.Packet.PATH_RESPONSE
else:
announce_context = RNS.Packet.NONE
if ratchet: context_flag = RNS.Packet.FLAG_SET
else: context_flag = RNS.Packet.FLAG_UNSET
if ratchet:
context_flag = RNS.Packet.FLAG_SET
else:
context_flag = RNS.Packet.FLAG_UNSET
announce_packet = RNS.Packet(self, announce_data, RNS.Packet.ANNOUNCE, context = announce_context,
attached_interface = attached_interface, context_flag=context_flag)
if send: announce_packet.send()
else: return announce_packet
def stamp_material(self):
return self.hash+self.identity.get_public_key()+self.name_hash
def get_ext_flags(self):
return 0x00
@property
def target_value(self):
return min(RNS.Transport.STAMP_COST_CAP, max(float(self._target_value), RNS.Transport.DESTINATION_VALUE))
def set_value(self, value):
"""
Sets the target stamp value for this destination.
:param value: The target value as a *float*.
"""
self._target_value = min(RNS.Transport.STAMP_COST_CAP, max(float(value), RNS.Transport.DESTINATION_VALUE))
def set_link_cost(self, cost):
"""
Sets the required link establishment stamp cost for this destination. Set to 0 to disable.
:param cost: The required cost as a *float*.
"""
self._link_cost = max(0, min(float(cost), RNS.Transport.LINK_COST_CAP))
def get_link_cost(self):
"""
Returns the link request stamp cost to be announced for this
destination, as a float. If no destination-specific cost has
been set, the default link request cost is returned.
:returns: The announced link request cost as a *float*.
"""
cost = self._link_cost
if cost == None: cost = RNS.Transport.LR_COST
return max(0.0, min(float(cost), RNS.Transport.STAMP_COST_CAP))
if send:
announce_packet.send()
else:
return announce_packet
def accepts_links(self, accepts = None):
"""
@ -423,10 +318,13 @@ class Destination:
:param accepts: If ``True`` or ``False``, this method sets whether the destination accepts incoming link requests. If not provided or ``None``, the method returns whether the destination currently accepts link requests.
:returns: ``True`` or ``False`` depending on whether the destination accepts incoming link requests, if the *accepts* parameter is not provided or ``None``.
"""
if accepts == None: return self.accept_link_requests
if accepts == None:
return self.accept_link_requests
if accepts: self.accept_link_requests = True
else: self.accept_link_requests = False
if accepts:
self.accept_link_requests = True
else:
self.accept_link_requests = False
def set_link_established_callback(self, callback):
"""
@ -462,22 +360,12 @@ class Destination:
:param proof_strategy: One of ``RNS.Destination.PROVE_NONE``, ``RNS.Destination.PROVE_ALL`` or ``RNS.Destination.PROVE_APP``. If ``RNS.Destination.PROVE_APP`` is set, the `proof_requested_callback` will be called to determine whether a proof should be sent or not.
"""
if not proof_strategy in Destination.proof_strategies: raise TypeError("Unsupported proof strategy")
else: self.proof_strategy = proof_strategy
if not proof_strategy in Destination.proof_strategies:
raise TypeError("Unsupported proof strategy")
else:
self.proof_strategy = proof_strategy
def set_max_request_size(self, max_request_size):
"""
Sets the maximum accepted request size for registered request handlers.
:param max_request_size: The maximum accepted request size in bytes, as an integer.
:raises: ``TypeError`` or ``ValueError`` if any of the argument is invalid.
"""
try: max_request_size = int(max_request_size)
except: raise TypeError("Invalid maximum request size specified")
if max_request_size < 0: raise ValueError("Maximum request size cannot be negative")
self.max_request_size = max_request_size
def register_request_handler(self, path, response_generator = None, allow = ALLOW_NONE, allowed_list = None, auto_compress = True):
def register_request_handler(self, path, response_generator = None, allow = ALLOW_NONE, allowed_list = None):
"""
Registers a request handler.
@ -485,15 +373,17 @@ class Destination:
:param response_generator: A function or method with the signature *response_generator(path, data, request_id, link_id, remote_identity, requested_at)* to be called. Whatever this funcion returns will be sent as a response to the requester. If the function returns ``None``, no response will be sent.
:param allow: One of ``RNS.Destination.ALLOW_NONE``, ``RNS.Destination.ALLOW_ALL`` or ``RNS.Destination.ALLOW_LIST``. If ``RNS.Destination.ALLOW_LIST`` is set, the request handler will only respond to requests for identified peers in the supplied list.
:param allowed_list: A list of *bytes-like* :ref:`RNS.Identity<api-identity>` hashes.
:param auto_compress: If ``True`` or ``False``, determines whether automatic compression of responses should be carried out. If set to an integer value, responses will only be auto-compressed if under this size in bytes. If omitted, the default compression settings will be followed.
:raises: ``ValueError`` if any of the supplied arguments are invalid.
"""
if path == None or path == "": raise ValueError("Invalid path specified")
elif not callable(response_generator): raise ValueError("Invalid response generator specified")
elif not allow in Destination.request_policies: raise ValueError("Invalid request policy")
if path == None or path == "":
raise ValueError("Invalid path specified")
elif not callable(response_generator):
raise ValueError("Invalid response generator specified")
elif not allow in Destination.request_policies:
raise ValueError("Invalid request policy")
else:
path_hash = RNS.Identity.truncated_hash(path.encode("utf-8"))
request_handler = [path, response_generator, allow, allowed_list, auto_compress]
request_handler = [path, response_generator, allow, allowed_list]
self.request_handlers[path_hash] = request_handler
def deregister_request_handler(self, path):
@ -517,25 +407,24 @@ class Destination:
else:
plaintext = self.decrypt(packet.data)
packet.ratchet_id = self.latest_ratchet_id
if plaintext == None: return False
else:
if plaintext != None:
if packet.packet_type == RNS.Packet.DATA:
if self.callbacks.packet != None:
try: self.callbacks.packet(plaintext, packet)
try:
self.callbacks.packet(plaintext, packet)
except Exception as e:
RNS.log("Error while executing receive callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
return True
def incoming_link_request(self, data, packet):
if self.accept_link_requests:
link = RNS.Link.validate_request(self, data, packet)
if link != None: self.links.append(link)
if link != None:
self.links.append(link)
def _reload_ratchets(self, ratchets_path):
if os.path.isfile(ratchets_path):
with self.ratchet_file_lock:
def load_attempt():
try:
ratchets_file = open(ratchets_path, "rb")
persisted_data = umsgpack.unpackb(ratchets_file.read())
if "signature" in persisted_data and "ratchets" in persisted_data:
@ -544,28 +433,11 @@ class Destination:
self.ratchets_path = ratchets_path
else:
raise KeyError("Invalid ratchet file signature")
try:
try:
load_attempt()
except Exception as e:
RNS.trace_exception(e)
RNS.log(f"First ratchet reload attempt for {self} failed. Possible I/O conflict. Retrying in 500ms.", RNS.LOG_ERROR)
time.sleep(0.5)
load_attempt()
RNS.log(f"Ratchet reload retry succeeded", RNS.LOG_DEBUG)
except Exception as e:
self.ratchets = None
self.ratchets_path = None
RNS.trace_exception(e)
RNS.log(f"The ratchet file located at {ratchets_path} could not be loaded. This could indicate that the ratchet file has become corrupt.", RNS.LOG_CRITICAL)
RNS.log(f"You can attempt to manually recover the ratchet file, or simply remove it to have Reticulum recreate it on the next use.", RNS.LOG_CRITICAL)
RNS.log(f"If re-initialize this ratchet file, make sure to send an announce for the relevant destination as soon as possible,", RNS.LOG_CRITICAL)
RNS.log(f"so that the new ratchet information is synchronized to the network.", RNS.LOG_CRITICAL)
raise OSError("Could not read ratchet file contents for "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
else:
RNS.log("No existing ratchet data found, initialising new ratchet file for "+str(self), RNS.LOG_DEBUG)
self.ratchets = []
@ -591,6 +463,7 @@ class Destination:
self.latest_ratchet_time = 0
self._reload_ratchets(ratchets_path)
# TODO: Remove at some point
RNS.log("Ratchets enabled on "+str(self), RNS.LOG_DEBUG)
return True
@ -652,8 +525,8 @@ class Destination:
raise TypeError("A single destination holds keys through an Identity instance")
if self.type == Destination.GROUP:
self.prv_bytes = Token.generate_key()
self.prv = Token(self.prv_bytes)
self.prv_bytes = Fernet.generate_key()
self.prv = Fernet(self.prv_bytes)
def get_private_key(self):
"""
@ -683,7 +556,7 @@ class Destination:
if self.type == Destination.GROUP:
self.prv_bytes = key
self.prv = Token(self.prv_bytes)
self.prv = Fernet(self.prv_bytes)
def load_public_key(self, key):
if self.type != Destination.SINGLE:
@ -744,7 +617,7 @@ class Destination:
RNS.log(f"Decryption still failing after ratchet reload. The contained exception was: {e}", RNS.LOG_ERROR)
raise e
if decrypted: RNS.log("Decryption succeeded after ratchet reload", RNS.LOG_NOTICE)
RNS.log("Decryption succeeded after ratchet reload", RNS.LOG_NOTICE)
return decrypted

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -28,7 +20,7 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from RNS.Interfaces.Interface import Interface
from .Interface import Interface
from time import sleep
import sys
import threading
@ -67,7 +59,6 @@ class AX25():
class AX25KISSInterface(Interface):
MAX_CHUNK = 32768
BITRATE_GUESS = 1200
DEFAULT_IFAC_SIZE = 8
owner = None
port = None
@ -77,8 +68,8 @@ class AX25KISSInterface(Interface):
stopbits = None
serial = None
def __init__(self, owner, configuration):
import importlib.util
def __init__(self, owner, name, callsign, ssid, port, speed, databits, parity, stopbits, preamble, txtail, persistence, slottime, flow_control):
import importlib
if importlib.util.find_spec('serial') != None:
import serial
else:
@ -88,25 +79,6 @@ class AX25KISSInterface(Interface):
super().__init__()
c = Interface.get_config_obj(configuration)
name = c["name"]
preamble = int(c["preamble"]) if "preamble" in c else None
txtail = int(c["txtail"]) if "txtail" in c else None
persistence = int(c["persistence"]) if "persistence" in c else None
slottime = int(c["slottime"]) if "slottime" in c else None
flow_control = c.as_bool("flow_control") if "flow_control" in c else False
port = c["port"] if "port" in c else None
speed = int(c["speed"]) if "speed" in c else 9600
databits = int(c["databits"]) if "databits" in c else 8
parity = c["parity"] if "parity" in c else "N"
stopbits = int(c["stopbits"]) if "stopbits" in c else 1
callsign = c["callsign"] if "callsign" in c else ""
ssid = int(c["ssid"]) if "ssid" in c else -1
if port == None:
raise ValueError("No port specified for serial interface")
self.HW_MTU = 564
self.pyserial = serial
@ -128,9 +100,7 @@ class AX25KISSInterface(Interface):
self.packet_queue = []
self.flow_control = flow_control
self.shared_medium = True
self.interface_ready = False
self.reconnecting = False
self.flow_control_timeout = 5
self.flow_control_locked = time.time()
@ -153,18 +123,14 @@ class AX25KISSInterface(Interface):
try:
self.open_port()
if self.serial.is_open: self.configure_device()
else: raise IOError("Could not open serial port")
except Exception as e:
RNS.log(f"Could not open serial port for {self}", RNS.LOG_ERROR)
RNS.log(f"The contained exception was: {e}", RNS.LOG_ERROR)
RNS.log(f"Reticulum will attempt to bring up this interface periodically", RNS.LOG_ERROR)
if not self.detached and not self.reconnecting:
thread = threading.Thread(target=self.reconnect_port)
thread.daemon = True
thread.start()
RNS.log("Could not open serial port for interface "+str(self), RNS.LOG_ERROR)
raise e
if self.serial.is_open:
self.configure_device()
else:
raise IOError("Could not open serial port")
def open_port(self):
RNS.log("Opening serial port "+self.port+"...", RNS.LOG_VERBOSE)
@ -182,12 +148,6 @@ class AX25KISSInterface(Interface):
dsrdtr = False,
)
def detach(self):
self.detached = True
self.online = False
try: self.serial.close()
except Exception as e: RNS.log(f"Could not close port for {self} at detach: {e}", RNS.LOG_ERROR)
def configure_device(self):
# Allow time for interface to initialise before config
sleep(2.0)
@ -265,11 +225,13 @@ class AX25KISSInterface(Interface):
raise IOError("Could not enable AX.25 KISS interface flow control")
def process_incoming(self, data):
if (len(data) > AX25.HEADER_SIZE): self.owner.inbound(data[AX25.HEADER_SIZE:], self)
def processIncoming(self, data):
if (len(data) > AX25.HEADER_SIZE):
self.rxb += len(data)
self.owner.inbound(data[AX25.HEADER_SIZE:], self)
def process_outgoing(self,data):
def processOutgoing(self,data):
datalen = len(data)
if self.online:
if self.interface_ready:
@ -319,7 +281,7 @@ class AX25KISSInterface(Interface):
if len(self.packet_queue) > 0:
data = self.packet_queue.pop(0)
self.interface_ready = True
self.process_outgoing(data)
self.processOutgoing(data)
elif len(self.packet_queue) == 0:
self.interface_ready = True
@ -338,7 +300,7 @@ class AX25KISSInterface(Interface):
if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
in_frame = False
self.process_incoming(data_buffer)
self.processIncoming(data_buffer)
elif (byte == KISS.FEND):
in_frame = True
command = KISS.CMD_UNKNOWN
@ -382,28 +344,30 @@ class AX25KISSInterface(Interface):
RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
if RNS.Reticulum.panic_on_interface_error: RNS.panic()
if RNS.Reticulum.panic_on_interface_error:
RNS.panic()
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
self.online = False
try: self.serial.close()
except Exception as e: RNS.log(f"Could not close port for {self}", RNS.LOG_ERROR)
if not self.detached and not self.reconnecting: self.reconnect_port()
self.serial.close()
self.reconnect_port()
def reconnect_port(self):
self.reconnecting = True
while not self.online and not self.detached:
while not self.online:
try:
time.sleep(5)
RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
self.open_port()
if self.serial.is_open: self.configure_device()
except Exception as e: RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
if self.serial.is_open:
self.configure_device()
except Exception as e:
RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
self.reconnecting = False
if self.online: RNS.log("Reconnected serial port for "+str(self))
RNS.log("Reconnected serial port for "+str(self))
def should_ingress_limit(self): return False
def should_ingress_limit(self):
return False
def __str__(self): return "AX25KISSInterface["+self.name+"]"
def __str__(self):
return "AX25KISSInterface["+self.name+"]"

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -60,7 +52,6 @@ class KISS():
class KISSInterface(Interface):
MAX_CHUNK = 32768
BITRATE_GUESS = 1200
DEFAULT_IFAC_SIZE = 8
owner = None
port = None
@ -70,8 +61,8 @@ class KISSInterface(Interface):
stopbits = None
serial = None
def __init__(self, owner, configuration):
import importlib.util
def __init__(self, owner, name, port, speed, databits, parity, stopbits, preamble, txtail, persistence, slottime, flow_control, beacon_interval, beacon_data):
import importlib
if RNS.vendor.platformutils.is_android():
self.on_android = True
if importlib.util.find_spec('usbserial4a') != None:
@ -92,21 +83,6 @@ class KISSInterface(Interface):
raise SystemError("Android-specific interface was used on non-Android OS")
super().__init__()
c = Interface.get_config_obj(configuration)
name = c["name"]
preamble = int(c["preamble"]) if "preamble" in c else None
txtail = int(c["txtail"]) if "txtail" in c else None
persistence = int(c["persistence"]) if "persistence" in c else None
slottime = int(c["slottime"]) if "slottime" in c else None
flow_control = c.as_bool("flow_control") if "flow_control" in c else False
port = c["port"] if "port" in c else None
speed = int(c["speed"]) if "speed" in c else 9600
databits = int(c["databits"]) if "databits" in c else 8
parity = c["parity"] if "parity" in c else "N"
stopbits = int(c["stopbits"]) if "stopbits" in c else 1
beacon_interval = int(c["beacon_interval"]) if "beacon_interval" in c and c["beacon_interval"] != None else None
beacon_data = c["beacon_data"] if "beacon_data" in c else None
self.HW_MTU = 564
@ -128,11 +104,9 @@ class KISSInterface(Interface):
self.beacon_d = beacon_data.encode("utf-8")
self.first_tx = None
self.bitrate = KISSInterface.BITRATE_GUESS
self.reconnecting = False
self.packet_queue = []
self.flow_control = flow_control
self.shared_medium = True
self.interface_ready = False
self.flow_control_timeout = 5
self.flow_control_locked = time.time()
@ -150,23 +124,15 @@ class KISSInterface(Interface):
try:
self.open_port()
if self.serial.is_open: self.configure_device()
else: raise IOError("Could not open serial port")
except Exception as e:
RNS.log(f"Could not open serial port for {self}", RNS.LOG_ERROR)
RNS.log(f"The contained exception was: {e}", RNS.LOG_ERROR)
RNS.log(f"Reticulum will attempt to bring up this interface periodically", RNS.LOG_ERROR)
if not self.detached and not self.reconnecting:
thread = threading.Thread(target=self.reconnect_port)
thread.daemon = True
thread.start()
RNS.log("Could not open serial port "+self.port, RNS.LOG_ERROR)
raise e
if self.serial.is_open:
self.configure_device()
else:
raise IOError("Could not open serial port")
def detach(self):
self.detached = True
self.online = False
try: self.serial.close()
except Exception as e: RNS.log(f"Could not close port for {self} at detach: {e}", RNS.LOG_ERROR)
def open_port(self):
RNS.log("Opening serial port "+self.port+"...")
@ -301,13 +267,13 @@ class KISSInterface(Interface):
raise IOError("Could not enable KISS interface flow control")
def process_incoming(self, data):
def processIncoming(self, data):
self.rxb += len(data)
def af():
self.owner.inbound(data, self)
threading.Thread(target=af, daemon=True).start()
def process_outgoing(self,data):
def processOutgoing(self,data):
datalen = len(data)
if self.online:
if self.interface_ready:
@ -341,7 +307,7 @@ class KISSInterface(Interface):
if len(self.packet_queue) > 0:
data = self.packet_queue.pop(0)
self.interface_ready = True
self.process_outgoing(data)
self.processOutgoing(data)
elif len(self.packet_queue) == 0:
self.interface_ready = True
@ -362,7 +328,7 @@ class KISSInterface(Interface):
if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
in_frame = False
self.process_incoming(data_buffer)
self.processIncoming(data_buffer)
elif (byte == KISS.FEND):
in_frame = True
command = KISS.CMD_UNKNOWN
@ -407,13 +373,7 @@ class KISSInterface(Interface):
if time.time() > self.first_tx + self.beacon_i:
RNS.log("Interface "+str(self)+" is transmitting beacon data: "+str(self.beacon_d.decode("utf-8")), RNS.LOG_DEBUG)
self.first_tx = None
# Pad to minimum length
frame = bytearray(self.beacon_d)
while len(frame) < 15:
frame.append(0x00)
self.process_outgoing(bytes(frame))
self.processOutgoing(self.beacon_d)
except Exception as e:
self.online = False
@ -426,23 +386,24 @@ class KISSInterface(Interface):
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
self.online = False
try: self.serial.close()
except Exception as e: RNS.log(f"Could not close port for {self}", RNS.LOG_ERROR)
if not self.detached and not self.reconnecting: self.reconnect_port()
self.serial.close()
self.reconnect_port()
def reconnect_port(self):
self.reconnecting = True
while not self.online:
try:
time.sleep(5)
RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
self.open_port()
if self.serial.is_open: self.configure_device()
except Exception as e: RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
if self.serial.is_open:
self.configure_device()
except Exception as e:
RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
self.reconnecting = False
if self.online: RNS.log("Reconnected serial port for "+str(self))
RNS.log("Reconnected serial port for "+str(self))
def should_ingress_limit(self): return False
def should_ingress_limit(self):
return False
def __str__(self): return "KISSInterface["+self.name+"]"
def __str__(self):
return "KISSInterface["+self.name+"]"

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -32,7 +24,6 @@ from RNS.Interfaces.Interface import Interface
from time import sleep
import sys
import threading
import socket
import time
import math
import RNS
@ -73,13 +64,10 @@ class KISS():
CMD_STAT_CHTM = 0x25
CMD_STAT_PHYPRM = 0x26
CMD_STAT_BAT = 0x27
CMD_STAT_CSMA = 0x28
CMD_STAT_TEMP = 0x29
CMD_BLINK = 0x30
CMD_RANDOM = 0x40
CMD_FB_EXT = 0x41
CMD_FB_READ = 0x42
CMD_DISP_READ = 0x66
CMD_FB_WRITE = 0x43
CMD_BT_CTRL = 0x46
CMD_PLATFORM = 0x48
@ -95,16 +83,14 @@ class KISS():
RADIO_STATE_ON = 0x01
RADIO_STATE_ASK = 0xFF
CMD_ERROR = 0x90
ERROR_INITRADIO = 0x01
ERROR_TXFAILED = 0x02
ERROR_EEPROM_LOCKED = 0x03
ERROR_QUEUE_FULL = 0x04
ERROR_MEMORY_LOW = 0x05
ERROR_MODEM_TIMEOUT = 0x06
CMD_ERROR = 0x90
ERROR_INITRADIO = 0x01
ERROR_TXFAILED = 0x02
ERROR_EEPROM_LOCKED = 0x03
ERROR_QUEUE_FULL = 0x04
ERROR_MEMORY_LOW = 0x05
ERROR_INVALID_FIRMWARE = 0x10
ERROR_INVALID_BLE_MTU = 0x20
ERROR_INVALID_CONFIG = 0x40
PLATFORM_AVR = 0x90
PLATFORM_ESP32 = 0x80
@ -251,10 +237,9 @@ class AndroidBluetoothManager():
class RNodeInterface(Interface):
MAX_CHUNK = 32768
DEFAULT_IFAC_SIZE = 8
FREQ_MIN = 137000000
FREQ_MAX = 3000000000
FREQ_MAX = 1020000000
RSSI_OFFSET = 157
@ -275,16 +260,17 @@ class RNodeInterface(Interface):
BATTERY_STATE_CHARGING = 0x02
BATTERY_STATE_CHARGED = 0x03
DISPLAY_READ_INTERVAL = 1.0
@classmethod
def bluetooth_control(device_serial = None, port = None, enable_bluetooth = False, disable_bluetooth = False, pairing_mode = False):
if (port != None or device_serial != None) and (enable_bluetooth or disable_bluetooth or pairing_mode):
serial = None
bluetooth_state = None
if pairing_mode: bluetooth_state = 0x01
elif enable_bluetooth: bluetooth_state = 0x01
elif disable_bluetooth: bluetooth_state = 0x00
if pairing_mode:
bluetooth_state = 0x01
elif enable_bluetooth:
bluetooth_state = 0x01
elif disable_bluetooth:
bluetooth_state = 0x00
if port != None:
RNS.log("Opening serial port "+port+"...")
@ -354,30 +340,13 @@ class RNodeInterface(Interface):
serial.close()
def __init__(self, owner, configuration):
c = Interface.get_config_obj(configuration)
name = c["name"]
allow_bluetooth = c.as_bool("allow_bluetooth") if "allow_bluetooth" in c else False
target_device_name = c["target_device_name"] if "target_device_name" in c else None
target_device_address = c["target_device_address"] if "target_device_address" in c else None
ble_name = c["ble_name"] if "ble_name" in c else None
ble_addr = c["ble_addr"] if "ble_addr" in c else None
tcp_host = c["tcp_host"] if "tcp_host" in c else None
force_ble = c["force_ble"] if "force_ble" in c else False
force_tcp = c["force_tcp"] if "force_tcp" in c else False
frequency = int(c["frequency"]) if "frequency" in c else 0
bandwidth = int(c["bandwidth"]) if "bandwidth" in c else 0
txpower = int(c["txpower"]) if "txpower" in c else 0
sf = int(c["spreadingfactor"]) if "spreadingfactor" in c else 0
cr = int(c["codingrate"]) if "codingrate" in c else 0
flow_control = c.as_bool("flow_control") if "flow_control" in c else False
id_interval = int(c["id_interval"]) if "id_interval" in c and c["id_interval"] != None else None
id_callsign = c["id_callsign"] if "id_callsign" in c else None
st_alock = float(c["airtime_limit_short"]) if "airtime_limit_short" in c and c["airtime_limit_short"] != None else None
lt_alock = float(c["airtime_limit_long"]) if "airtime_limit_long" in c and c["airtime_limit_long"] != None else None
port = c["port"] if "port" in c else None
import importlib.util
def __init__(
self, owner, name, port, frequency = None, bandwidth = None, txpower = None,
sf = None, cr = None, flow_control = False, id_interval = None,
allow_bluetooth = False, target_device_name = None,
target_device_address = None, id_callsign = None, st_alock = None, lt_alock = None,
ble_addr = None, ble_name = None, force_ble=False):
import importlib
if RNS.vendor.platformutils.is_android():
self.on_android = True
if importlib.util.find_spec('usbserial4a') != None:
@ -436,14 +405,6 @@ class RNodeInterface(Interface):
self.ble_rx_queue= b""
self.ble_tx_queue= b""
self.tcp = None
self.use_tcp = False
self.tcp_host = tcp_host
self.tcp_rx_queue= b""
self.tcp_tx_queue= b""
self.tcp_rx_lock = threading.Lock()
self.tcp_tx_lock = threading.Lock()
self.frequency = frequency
self.bandwidth = bandwidth
self.txpower = txpower
@ -453,7 +414,6 @@ class RNodeInterface(Interface):
self.bitrate = 0
self.st_alock = st_alock
self.lt_alock = lt_alock
self.cpu_temp = None
self.platform = None
self.display = None
self.mcu = None
@ -466,7 +426,6 @@ class RNodeInterface(Interface):
self.first_tx = None
self.reconnect_w = RNodeInterface.RECONNECT_WAIT
self.reconnect_lock = threading.Lock()
self.awaiting_ble_reset = False
self.r_frequency = None
self.r_bandwidth = None
@ -486,52 +445,30 @@ class RNodeInterface(Interface):
self.r_airtime_long = 0.0
self.r_channel_load_short = 0.0
self.r_channel_load_long = 0.0
self.r_symbol_time_ms = None
self.r_symbol_rate = None
self.r_preamble_symbols = None
self.r_premable_time_ms = None
self.r_csma_slot_time_ms = None
self.r_csma_difs_ms = None
self.r_csma_cw_band = None
self.r_csma_cw_min = None
self.r_csma_cw_max = None
self.r_current_rssi = None
self.r_noise_floor = None
self.r_interference = None
self.r_interference_l = None
self.r_temperature = None
self.reports_phy_stats = True
self.r_symbol_time_ms = None
self.r_symbol_rate = None
self.r_preamble_symbols = None
self.r_premable_time_ms = None
self.r_battery_state = RNodeInterface.BATTERY_STATE_UNKNOWN
self.r_battery_percent = 0
self.r_framebuffer = b""
self.r_framebuffer_readtime = 0
self.r_framebuffer_latency = 0
self.r_disp = b""
self.r_disp_readtime = 0
self.r_disp_latency = 0
self.should_read_display = False
self.read_display_interval = RNodeInterface.DISPLAY_READ_INTERVAL
self.packet_queue = []
self.flow_control = flow_control
self.shared_medium = True
self.interface_ready = False
self.announce_rate_target = None
self.last_port_io = 0
self.port_io_timeout = RNodeInterface.PORT_IO_TIMEOUT
self.last_imagedata = None
if force_ble or self.ble_addr != None or self.ble_name != None: self.use_ble = True
if force_tcp or self.tcp_host != None: self.use_tcp = True
if force_ble or self.ble_addr != None or self.ble_name != None:
self.use_ble = True
self.validcfg = True
if (self.frequency < RNodeInterface.FREQ_MIN or self.frequency > RNodeInterface.FREQ_MAX):
RNS.log("Invalid frequency configured for "+str(self), RNS.LOG_ERROR)
self.validcfg = False
if (self.txpower < 0 or self.txpower > 37):
if (self.txpower < 0 or self.txpower > 22):
RNS.log("Invalid TX power configured for "+str(self), RNS.LOG_ERROR)
self.validcfg = False
@ -574,8 +511,10 @@ class RNodeInterface(Interface):
self.open_port()
if self.serial != None:
if self.serial.is_open: self.configure_device()
else: raise IOError("Could not open serial port")
if self.serial.is_open:
self.configure_device()
else:
raise IOError("Could not open serial port")
elif self.bt_manager != None:
if self.bt_manager.connected:
self.configure_device()
@ -593,9 +532,12 @@ class RNodeInterface(Interface):
def read_mux(self, len=None):
if self.serial != None: return self.serial.read()
elif self.bt_manager != None: return self.bt_manager.read()
else: raise IOError("No ports available for reading")
if self.serial != None:
return self.serial.read()
elif self.bt_manager != None:
return self.bt_manager.read()
else:
raise IOError("No ports available for reading")
def write_mux(self, data):
if self.serial != None:
@ -610,19 +552,17 @@ class RNodeInterface(Interface):
else:
raise IOError("No ports available for writing")
def reset_ble(self):
if not self.awaiting_ble_reset: return
else:
RNS.log(f"Clearing previous connection instance: "+str(self.ble), RNS.LOG_DEBUG)
self.ble = None
self.serial = None
self.ble = BLEConnection(owner=self, target_name=self.ble_name, target_bt_addr=self.ble_addr)
self.serial = self.ble
self.awaiting_ble_reset = False
RNS.log(f"New connection instance: "+str(self.ble), RNS.LOG_DEBUG)
# def reset_ble(self):
# RNS.log(f"Clearing previous connection instance: "+str(self.ble))
# del self.ble
# self.ble = None
# self.serial = None
# self.ble = BLEConnection(owner=self, target_name=self.ble_name, target_bt_addr=self.ble_addr)
# self.serial = self.ble
# RNS.log(f"New connection instance: "+str(self.ble))
def open_port(self):
if not self.use_ble and not self.use_tcp:
if not self.use_ble:
if self.port != None:
RNS.log("Opening serial port "+self.port+"...")
# Get device parameters
@ -690,7 +630,7 @@ class RNodeInterface(Interface):
if self.bt_manager != None:
self.bt_manager.connect_any_device()
elif self.use_ble:
else:
if self.ble == None:
self.ble = BLEConnection(owner=self, target_name=self.ble_name, target_bt_addr=self.ble_addr)
self.serial = self.ble
@ -699,24 +639,6 @@ class RNodeInterface(Interface):
while not self.ble.connected and time.time() < open_time + self.ble.CONNECT_TIMEOUT:
time.sleep(1)
elif self.use_tcp:
RNS.log(f"Opening TCP connection for {self}...")
if self.tcp != None and self.tcp.running == False:
self.tcp.close()
self.tcp.cleanup()
self.tcp = None
if self.tcp == None:
self.tcp = TCPConnection(owner=self, target_host=self.tcp_host)
self.serial = self.tcp
open_time = time.time()
while not self.tcp.connected and time.time() < open_time + self.tcp.CONNECT_TIMEOUT:
time.sleep(1)
else:
raise TypeError("No valid device connection type defined for RNode interface")
def configure_device(self):
self.resetRadioState()
@ -724,19 +646,17 @@ class RNodeInterface(Interface):
thread = threading.Thread(target=self.readLoop, daemon=True).start()
self.detect()
if self.use_tcp:
tcp_detect_timeout = 5.0
detect_time = time.time()
while not self.detected and time.time() < detect_time + tcp_detect_timeout: time.sleep(0.1)
if not self.detected: RNS.log(f"RNode detect timed out over TCP", RNS.LOG_ERROR)
elif self.use_ble:
if not self.use_ble:
sleep(0.5)
else:
ble_detect_timeout = 5
detect_time = time.time()
while not self.detected and time.time() < detect_time + ble_detect_timeout: time.sleep(0.1)
if self.detected: detect_time = RNS.prettytime(time.time()-detect_time)
else: RNS.log(f"RNode detect timed out over {self.port}", RNS.LOG_ERROR)
else:
sleep(0.2)
while not self.detected and time.time() < detect_time + ble_detect_timeout:
time.sleep(0.1)
if self.detected:
detect_time = RNS.prettytime(time.time()-detect_time)
else:
RNS.log(f"RNode detect timed out over {self.port}", RNS.LOG_ERROR)
if not self.detected:
raise IOError("Could not detect device")
@ -749,19 +669,11 @@ class RNodeInterface(Interface):
if self.serial != None and self.port != None:
self.timeout = 200
RNS.log(f"Serial port {self.port} is now open")
RNS.log("Serial port "+self.port+" is now open")
if self.bt_manager != None and self.bt_manager.connected:
self.timeout = 1500
RNS.log(f"Bluetooth connection to RNode now open")
if self.ble != None and self.ble.connected:
self.timeout = 1500
RNS.log(f"BLE connection {self.port} to RNode now open")
if self.tcp != None and self.tcp.connected:
self.timeout = 1500
RNS.log(f"TCP connection tcp://{self.tcp_host} to RNode now open")
RNS.log("Bluetooth connection to RNode now open")
RNS.log("Configuring RNode interface...", RNS.LOG_VERBOSE)
self.initRadio()
@ -774,7 +686,6 @@ class RNodeInterface(Interface):
RNS.log("After configuring "+str(self)+", the reported radio parameters did not match your configuration.", RNS.LOG_ERROR)
RNS.log("Make sure that your hardware actually supports the parameters specified in the configuration", RNS.LOG_ERROR)
RNS.log("Aborting RNode startup", RNS.LOG_ERROR)
self.hw_errors.append({"error": KISS.ERROR_INVALID_CONFIG, "description": "The configuration parameters were not validated by the device. Make sure that the device actually supports the TX power, frequency, bandwidth, spreading factor and coding rate you configured."})
if self.serial != None:
self.serial.close()
@ -881,33 +792,6 @@ class RNodeInterface(Interface):
if written != len(kiss_command):
raise IOError("An IO error occurred while writing framebuffer data device")
def read_framebuffer(self):
kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_FB_READ])+bytes([0x01])+bytes([KISS.FEND])
written = self.serial.write(kiss_command)
self.r_framebuffer_readtime = time.time()
if written != len(kiss_command):
raise IOError("An IO error occurred while sending framebuffer read command")
def read_display(self):
kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_DISP_READ])+bytes([0x01])+bytes([KISS.FEND])
written = self.serial.write(kiss_command)
self.r_disp_readtime = time.time()
if written != len(kiss_command):
raise IOError("An IO error occurred while sending display read command")
def _read_display_job(self):
while self.should_read_display:
self.read_display()
time.sleep(self.read_display_interval)
def start_display_updates(self):
if not self.should_read_display:
self.should_read_display = True
threading.Thread(target=self._read_display_job, daemon=True).start()
def stop_display_updates(self):
self.should_read_display = False
def hard_reset(self):
kiss_command = bytes([KISS.FEND, KISS.CMD_RESET, 0xf8, KISS.FEND])
written = self.write_mux(kiss_command)
@ -1013,8 +897,10 @@ class RNodeInterface(Interface):
def validateRadioState(self):
RNS.log("Waiting for radio configuration validation for "+str(self)+"...", RNS.LOG_VERBOSE)
if not self.platform == KISS.PLATFORM_ESP32: sleep(1.00);
else: sleep(2.00);
if not self.platform == KISS.PLATFORM_ESP32:
sleep(1.00);
else:
sleep(2.00);
self.validcfg = True
if (self.r_frequency != None and abs(self.frequency - int(self.r_frequency)) > 100):
@ -1054,7 +940,7 @@ class RNodeInterface(Interface):
except:
self.bitrate = 0
def process_incoming(self, data):
def processIncoming(self, data):
self.rxb += len(data)
def af():
@ -1062,7 +948,7 @@ class RNodeInterface(Interface):
threading.Thread(target=af, daemon=True).start()
def process_outgoing(self,data):
def processOutgoing(self,data):
datalen = len(data)
if self.online:
if self.interface_ready:
@ -1093,7 +979,7 @@ class RNodeInterface(Interface):
if len(self.packet_queue) > 0:
data = self.packet_queue.pop(0)
self.interface_ready = True
self.process_outgoing(data)
self.processOutgoing(data)
elif len(self.packet_queue) == 0:
self.interface_ready = True
@ -1120,7 +1006,7 @@ class RNodeInterface(Interface):
if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
in_frame = False
self.process_incoming(data_buffer)
self.processIncoming(data_buffer)
data_buffer = b""
command_buffer = b""
elif (byte == KISS.FEND):
@ -1296,33 +1182,16 @@ class RNodeInterface(Interface):
byte = KISS.FESC
escape = False
command_buffer = command_buffer+bytes([byte])
if (len(command_buffer) == 11):
if (len(command_buffer) == 8):
ats = command_buffer[0] << 8 | command_buffer[1]
atl = command_buffer[2] << 8 | command_buffer[3]
cus = command_buffer[4] << 8 | command_buffer[5]
cul = command_buffer[6] << 8 | command_buffer[7]
crs = command_buffer[8]
nfl = command_buffer[9]
ntf = command_buffer[10]
self.r_airtime_short = ats/100.0
self.r_airtime_long = atl/100.0
self.r_channel_load_short = cus/100.0
self.r_channel_load_long = cul/100.0
self.r_current_rssi = crs-RNodeInterface.RSSI_OFFSET
self.r_noise_floor = nfl-RNodeInterface.RSSI_OFFSET
if ntf == 0xFF:
self.r_interference = None
else:
self.r_interference = ntf-RNodeInterface.RSSI_OFFSET
self.r_interference_l = [time.time(), self.r_interference]
if self.r_interference != None:
RNS.log(f"{self} Radio detected interference at {self.r_interference} dBm", RNS.LOG_DEBUG)
# TODO: Remove debug
# RNS.log(f"RSSI: {self.r_current_rssi}, Noise floor: {self.r_noise_floor}, Interference: {self.r_interference}", RNS.LOG_EXTREME)
elif (command == KISS.CMD_STAT_PHYPRM):
if (byte == KISS.FESC):
escape = True
@ -1334,49 +1203,22 @@ class RNodeInterface(Interface):
byte = KISS.FESC
escape = False
command_buffer = command_buffer+bytes([byte])
if (len(command_buffer) == 12):
if (len(command_buffer) == 10):
lst = (command_buffer[0] << 8 | command_buffer[1])/1000.0
lsr = command_buffer[2] << 8 | command_buffer[3]
prs = command_buffer[4] << 8 | command_buffer[5]
prt = command_buffer[6] << 8 | command_buffer[7]
cst = command_buffer[8] << 8 | command_buffer[9]
dft = command_buffer[10] << 8 | command_buffer[11]
if lst != self.r_symbol_time_ms or lsr != self.r_symbol_rate or prs != self.r_preamble_symbols or prt != self.r_premable_time_ms or cst != self.r_csma_slot_time_ms or dft != self.r_csma_difs_ms:
if lst != self.r_symbol_time_ms or lsr != self.r_symbol_rate or prs != self.r_preamble_symbols or prt != self.r_premable_time_ms or cst != self.r_csma_slot_time_ms:
self.r_symbol_time_ms = lst
self.r_symbol_rate = lsr
self.r_preamble_symbols = prs
self.r_premable_time_ms = prt
self.r_csma_slot_time_ms = cst
self.r_csma_difs_ms = dft
RNS.log(f"{self} Radio reporting symbol time is "+str(round(self.r_symbol_time_ms,2))+"ms ("+str(self.r_symbol_rate)+" baud)", RNS.LOG_DEBUG)
RNS.log(f"{self} Radio reporting preamble is "+str(self.r_preamble_symbols)+" symbols ("+str(self.r_premable_time_ms)+"ms)", RNS.LOG_DEBUG)
RNS.log(f"{self} Radio reporting CSMA slot time is "+str(self.r_csma_slot_time_ms)+"ms", RNS.LOG_DEBUG)
RNS.log(f"{self} Radio reporting DIFS time is "+str(self.r_csma_difs_ms)+"ms", RNS.LOG_DEBUG)
elif (command == KISS.CMD_STAT_CSMA):
if (byte == KISS.FESC):
escape = True
else:
if (escape):
if (byte == KISS.TFEND):
byte = KISS.FEND
if (byte == KISS.TFESC):
byte = KISS.FESC
escape = False
command_buffer = command_buffer+bytes([byte])
if (len(command_buffer) == 3):
cbw = command_buffer[0]
cbl = command_buffer[1]
cbh = command_buffer[2]
if cbw != self.r_csma_cw_band or cbl != self.r_csma_cw_min or cbh != self.r_csma_cw_max:
self.r_csma_cw_band = cbw
self.r_csma_cw_min = cbl
self.r_csma_cw_max = cbh
# TODO: Remove debug
# RNS.log(f"{self} Radio reporting contention window band is {self.r_csma_cw_band}", RNS.LOG_EXTREME)
# RNS.log(f"{self} Radio reporting minimum contention window is {self.r_csma_cw_min}", RNS.LOG_EXTREME)
# RNS.log(f"{self} Radio reporting maximum contention window is {self.r_csma_cw_max}", RNS.LOG_EXTREME)
RNS.log(str(self)+" Radio reporting symbol time is "+str(round(self.r_symbol_time_ms,2))+"ms (at "+str(self.r_symbol_rate)+" baud)", RNS.LOG_DEBUG)
RNS.log(str(self)+" Radio reporting preamble is "+str(self.r_preamble_symbols)+" symbols ("+str(self.r_premable_time_ms)+"ms)", RNS.LOG_DEBUG)
RNS.log(str(self)+" Radio reporting CSMA slot time is "+str(self.r_csma_slot_time_ms)+"ms", RNS.LOG_DEBUG)
elif (command == KISS.CMD_STAT_BAT):
if (byte == KISS.FESC):
escape = True
@ -1396,22 +1238,6 @@ class RNodeInterface(Interface):
bat_percent = 0
self.r_battery_state = command_buffer[0]
self.r_battery_percent = bat_percent
elif (command == KISS.CMD_STAT_TEMP):
if (byte == KISS.FESC):
escape = True
else:
if (escape):
if (byte == KISS.TFEND):
byte = KISS.FEND
if (byte == KISS.TFESC):
byte = KISS.FESC
escape = False
command_buffer = command_buffer+bytes([byte])
if (len(command_buffer) == 1):
temp = command_buffer[0]-120
if temp >= -30 and temp <= 90: self.r_temperature = temp
else: self.r_temperature = None
self.cpu_temp = self.r_temperature
elif (command == KISS.CMD_RANDOM):
self.r_random = byte
elif (command == KISS.CMD_PLATFORM):
@ -1428,9 +1254,6 @@ class RNodeInterface(Interface):
elif (byte == KISS.ERROR_MEMORY_LOW):
RNS.log(str(self)+" hardware error (code "+RNS.hexrep(byte)+"): Memory exhausted", RNS.LOG_ERROR)
self.hw_errors.append({"error": KISS.ERROR_MEMORY_LOW, "description": "Memory exhausted on connected device"})
elif (byte == KISS.ERROR_MODEM_TIMEOUT):
RNS.log(str(self)+" hardware error (code "+RNS.hexrep(byte)+"): Modem communication timed out", RNS.LOG_ERROR)
self.hw_errors.append({"error": KISS.ERROR_MODEM_TIMEOUT, "description": "Modem communication timed out on connected device"})
else:
RNS.log(str(self)+" hardware error (code "+RNS.hexrep(byte)+")", RNS.LOG_ERROR)
raise IOError("Unknown hardware failure")
@ -1442,37 +1265,6 @@ class RNodeInterface(Interface):
raise IOError("ESP32 reset")
elif (command == KISS.CMD_READY):
self.process_queue()
elif (command == KISS.CMD_FB_READ):
if (byte == KISS.FESC):
escape = True
else:
if (escape):
if (byte == KISS.TFEND):
byte = KISS.FEND
if (byte == KISS.TFESC):
byte = KISS.FESC
escape = False
command_buffer = command_buffer+bytes([byte])
if (len(command_buffer) == 512):
self.r_framebuffer_latency = time.time() - self.r_framebuffer_readtime
self.r_framebuffer = command_buffer
elif (command == KISS.CMD_DISP_READ):
if (byte == KISS.FESC):
escape = True
else:
if (escape):
if (byte == KISS.TFEND):
byte = KISS.FEND
if (byte == KISS.TFESC):
byte = KISS.FESC
escape = False
command_buffer = command_buffer+bytes([byte])
if (len(command_buffer) == 1024):
self.r_disp_latency = time.time() - self.r_disp_readtime
self.r_disp = command_buffer
elif (command == KISS.CMD_DETECT):
if byte == KISS.DETECT_RESP:
self.detected = True
@ -1482,7 +1274,7 @@ class RNodeInterface(Interface):
if got == 0:
time_since_last = int(time.time()*1000) - last_read_ms
if len(data_buffer) > 0 and time_since_last > self.timeout:
RNS.log(f"{self} device read timeout in command {command} after {RNS.prettytime(self.timeout/1000.0)}", RNS.LOG_WARNING)
RNS.log(str(self)+" serial read timeout in command "+str(command), RNS.LOG_WARNING)
data_buffer = b""
in_frame = False
command = KISS.CMD_UNKNOWN
@ -1492,26 +1284,26 @@ class RNodeInterface(Interface):
if self.first_tx != None:
if time.time() > self.first_tx + self.id_interval:
RNS.log("Interface "+str(self)+" is transmitting beacon data: "+str(self.id_callsign.decode("utf-8")), RNS.LOG_DEBUG)
self.process_outgoing(self.id_callsign)
if self.use_tcp:
if self.tcp and self.tcp.connected:
if time.time() > self.tcp.last_write + self.tcp.ACTIVITY_KEEPALIVE:
self.detect()
self.processOutgoing(self.id_callsign)
if (time.time() - self.last_port_io > self.port_io_timeout): self.detect()
if (time.time() - self.last_port_io > self.port_io_timeout*3): raise IOError("Connected port for "+str(self)+" became unresponsive")
if self.bt_manager != None or self.ble != None: sleep(0.08)
if (time.time() - self.last_port_io > self.port_io_timeout):
self.detect()
if (time.time() - self.last_port_io > self.port_io_timeout*3):
raise IOError("Connected port for "+str(self)+" became unresponsive")
if self.bt_manager != None:
sleep(0.08)
except Exception as e:
self.online = False
if not self.detached:
RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
RNS.log("A serial port occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
if RNS.Reticulum.panic_on_interface_error: RNS.panic()
if RNS.Reticulum.panic_on_interface_error:
RNS.panic()
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
self.online = False
@ -1563,22 +1355,12 @@ class RNodeInterface(Interface):
def detach(self):
self.detached = True
try:
self.disable_external_framebuffer()
self.setRadioState(KISS.RADIO_STATE_OFF)
self.leave()
except Exception as e:
RNS.log(f"An error occurred while detaching {self}: {e}", RNS.LOG_ERROR)
self.disable_external_framebuffer()
self.setRadioState(KISS.RADIO_STATE_OFF)
self.leave()
if self.use_ble:
self.ble.close()
self.ble.cleanup()
elif self.use_tcp:
time.sleep(0.5)
self.tcp.close()
else:
self.serial.close()
def should_ingress_limit(self):
return False
@ -1599,17 +1381,6 @@ class RNodeInterface(Interface):
def get_battery_percent(self):
return self.r_battery_percent
def tcp_receive(self, data):
with self.tcp_rx_lock: self.tcp_rx_queue += data
def tcp_waiting(self): return len(self.tcp_tx_queue) > 0
def get_tcp_waiting(self, n):
with self.tcp_tx_lock:
data = self.tcp_tx_queue[:n]
self.tcp_tx_queue = self.tcp_tx_queue[n:]
return data
def ble_receive(self, data):
with self.ble_rx_lock:
self.ble_rx_queue += data
@ -1624,7 +1395,7 @@ class RNodeInterface(Interface):
return data
def __str__(self):
return f"RNodeInterface[{self.name}]"
return "RNodeInterface["+str(self.name)+"]"
class BLEConnection(BluetoothDispatcher):
UART_SERVICE_UUID = "6e400001-b5a3-f393-e0a9-e50e24dcca9e"
@ -1636,7 +1407,7 @@ class BLEConnection(BluetoothDispatcher):
MTU_TIMEOUT = 4.0
CONNECT_TIMEOUT = 7.0
RECONNECT_WAIT = 2.5
RECONNECT_WAIT = 1.0
@property
def is_open(self):
@ -1657,9 +1428,6 @@ class BLEConnection(BluetoothDispatcher):
self.owner.ble_rx_queue = b""
return data
def cleanup(self):
self.should_run = False
def close(self):
try:
if self.connected:
@ -1740,17 +1508,13 @@ class BLEConnection(BluetoothDispatcher):
self.write_thread = None
def connection_job(self):
ble_devices = []
while self.should_run:
if self.bt_manager.bt_enabled():
if not self.connected:
if len(ble_devices) == 0:
ble_devices = self.find_target_devices()
if len(ble_devices) > 0: self.ble_device = ble_devices.pop()
else: self.ble_device = None
if self.ble_device == None:
self.ble_device = self.find_target_device()
if self.ble_device != None:
if self.ble_device != None:
if not self.connected:
if self.was_connected:
RNS.log(f"Throttling BLE reconnect for {BLEConnection.RECONNECT_WAIT} seconds", RNS.LOG_DEBUG)
time.sleep(BLEConnection.RECONNECT_WAIT)
@ -1762,7 +1526,7 @@ class BLEConnection(BluetoothDispatcher):
RNS.log("Bluetooth was disabled, closing active BLE device connection", RNS.LOG_ERROR)
self.close()
time.sleep(1)
time.sleep(2)
def connect_device(self):
if self.ble_device != None and self.bt_manager.bt_enabled():
@ -1780,15 +1544,12 @@ class BLEConnection(BluetoothDispatcher):
else:
RNS.log(f"BLE device connection timed out for {self.owner}", RNS.LOG_DEBUG)
if self.mtu_requested_time:
RNS.log("MTU update timeout, tearing down connection and resetting BLE dispatcher")
RNS.log("MTU update timeout, tearing down connection")
self.owner.hw_errors.append({"error": KISS.ERROR_INVALID_BLE_MTU, "description": "The Bluetooth Low Energy transfer MTU could not be configured for the connected device, and communication has failed. Restart Reticulum and any connected applications to retry connecting."})
self.close()
self.close_gatt()
self.should_run = False
self.owner.awaiting_ble_reset = True
else:
self.close_gatt()
self.close_gatt()
self.connect_job_running = False
@ -1798,17 +1559,15 @@ class BLEConnection(BluetoothDispatcher):
self.ble_device = None
self.close_gatt()
def find_target_devices(self):
def find_target_device(self):
found_device = None
potential_devices = self.bt_manager.get_paired_devices()
suitable_devices = []
if self.target_bt_addr != None:
for device in potential_devices:
if (device.getType() == AndroidBluetoothManager.DEVICE_TYPE_LE) or (device.getType() == AndroidBluetoothManager.DEVICE_TYPE_DUAL):
if str(device.getAddress()).replace(":", "").lower() == str(self.target_bt_addr).replace(":", "").lower():
found_device = device
suitable_devices.append(device)
break
if not found_device and self.target_name != None:
@ -1816,7 +1575,6 @@ class BLEConnection(BluetoothDispatcher):
if (device.getType() == AndroidBluetoothManager.DEVICE_TYPE_LE) or (device.getType() == AndroidBluetoothManager.DEVICE_TYPE_DUAL):
if device.getName().lower() == self.target_name.lower():
found_device = device
suitable_devices.append(device)
break
if not found_device:
@ -1824,9 +1582,9 @@ class BLEConnection(BluetoothDispatcher):
if (device.getType() == AndroidBluetoothManager.DEVICE_TYPE_LE) or (device.getType() == AndroidBluetoothManager.DEVICE_TYPE_DUAL):
if device.getName().startswith("RNode "):
found_device = device
suitable_devices.append(device)
break
return suitable_devices
return found_device
def on_connection_state_change(self, status, state):
if status == GATT_SUCCESS and state:
@ -1870,139 +1628,4 @@ class BLEConnection(BluetoothDispatcher):
def on_characteristic_changed(self, characteristic):
if characteristic.getUuid().toString() == BLEConnection.UART_TX_CHAR_UUID:
recvd = bytes(characteristic.getValue())
self.owner.ble_receive(recvd)
class TCPConnection():
TARGET_PORT = 7633
CONNECT_TIMEOUT = 5.0
INITIAL_CONNECT_TIMEOUT = 5.0
RECONNECT_WAIT = 4.0
ACTIVITY_TIMEOUT = 6.0
ACTIVITY_KEEPALIVE = ACTIVITY_TIMEOUT-2.5
TCP_USER_TIMEOUT = 24
TCP_PROBE_AFTER = 5
TCP_PROBE_INTERVAL = 2
TCP_PROBES = 12
@property
def is_open(self):
return self.connected
@property
def in_waiting(self):
buflen = len(self.owner.tcp_rx_queue)
return buflen > 0
def write(self, data_bytes):
if self.connected and self.socket:
with self.owner.tcp_tx_lock:
if len(self.owner.tcp_tx_queue) > 0:
self.socket.sendall(self.owner.tcp_tx_queue)
self.owner.tcp_tx_queue = b""
self.socket.sendall(data_bytes)
self.last_write = time.time()
else:
with self.owner.tcp_tx_lock: self.owner.tcp_tx_queue += data_bytes
return len(data_bytes)
def read(self, n=4096):
with self.owner.tcp_rx_lock:
data = self.owner.tcp_rx_queue[:n]
self.owner.tcp_rx_queue = self.owner.tcp_rx_queue[n:]
return data
def close(self):
if self.connected:
RNS.log(f"Disconnecting TCP socket for {self.owner}", RNS.LOG_DEBUG)
self.must_disconnect = True
if self.socket: self.socket.close()
def __init__(self, owner=None, target_host=None):
self.owner = owner
self.target_host = target_host
self.connected = False
self.reconnecting = False
self.running = False
self.should_run = False
self.must_disconnect = False
self.connect_job_running = False
self.last_write = time.time()
self.should_run = True
self.connection_thread = threading.Thread(target=self.initial_connect, daemon=True).start()
def set_timeouts_linux(self):
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_USER_TIMEOUT, int(self.TCP_USER_TIMEOUT * 1000))
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, int(self.TCP_PROBE_AFTER))
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, int(self.TCP_PROBE_INTERVAL))
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPCNT, int(self.TCP_PROBES))
def set_timeouts_osx(self):
if hasattr(socket, "TCP_KEEPALIVE"): TCP_KEEPIDLE = socket.TCP_KEEPALIVE
else: TCP_KEEPIDLE = 0x10
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
self.socket.setsockopt(socket.IPPROTO_TCP, TCP_KEEPIDLE, int(self.TCP_PROBE_AFTER))
def cleanup(self):
try:
if self.socket: self.socket.close()
except Exception as e:
RNS.log(f"Error while disconnecting TCP socket on cleanup for {self.owner}", RNS.LOG_ERROR)
self.should_run = False
def initial_connect(self):
if self.connect(initial=True): threading.Thread(target=self.read_loop, daemon=True).start()
def connect(self, initial=False):
try:
if initial:
RNS.log(f"Establishing TCP connection to device for {self.owner}...", RNS.LOG_DEBUG)
address_info = socket.getaddrinfo(self.target_host, self.TARGET_PORT, proto=socket.IPPROTO_TCP)[0]
address_family = address_info[0]
target_address = address_info[4]
self.socket = socket.socket(address_family, socket.SOCK_STREAM)
self.socket.settimeout(self.INITIAL_CONNECT_TIMEOUT)
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
self.socket.connect(target_address)
self.socket.settimeout(None)
self.connected = True
self.last_write = time.time()
RNS.log(f"TCP connection to device for {self.owner} established", RNS.LOG_DEBUG)
if RNS.vendor.platformutils.is_linux(): self.set_timeouts_linux()
elif RNS.vendor.platformutils.is_darwin(): self.set_timeouts_osx()
return True
except Exception as e:
if initial:
RNS.log(f"TCP connection to device for {self.owner} could not be established: {e}", RNS.LOG_ERROR)
return False
else: raise e
def read_loop(self):
try:
data_in = b""
while not self.must_disconnect:
if self.socket: data_in = self.socket.recv(4096)
else: data_in = b""
if len(data_in) > 0: self.owner.tcp_receive(data_in)
else:
self.connected = False
RNS.log(f"The TCP socket for {self} was closed", RNS.LOG_WARNING)
break
except Exception as e:
self.connected = False
RNS.log(f"A TCP read error occurred for {self}, the contained exception was: {e}", RNS.LOG_WARNING)
self.owner.ble_receive(recvd)

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -50,7 +42,6 @@ class HDLC():
class SerialInterface(Interface):
MAX_CHUNK = 32768
DEFAULT_IFAC_SIZE = 8
owner = None
port = None
@ -60,8 +51,8 @@ class SerialInterface(Interface):
stopbits = None
serial = None
def __init__(self, owner, configuration):
import importlib.util
def __init__(self, owner, name, port, speed, databits, parity, stopbits):
import importlib
if RNS.vendor.platformutils.is_android():
self.on_android = True
if importlib.util.find_spec('usbserial4a') != None:
@ -83,19 +74,7 @@ class SerialInterface(Interface):
super().__init__()
c = Interface.get_config_obj(configuration)
name = c["name"]
port = c["port"] if "port" in c else None
speed = int(c["speed"]) if "speed" in c else 9600
databits = int(c["databits"]) if "databits" in c else 8
parity = c["parity"] if "parity" in c else "N"
stopbits = int(c["stopbits"]) if "stopbits" in c else 1
if port == None:
raise ValueError("No port specified for serial interface")
self.HW_MTU = 564
self.shared_medium = True
self.pyserial = serial
self.serial = None
@ -109,7 +88,6 @@ class SerialInterface(Interface):
self.timeout = 100
self.online = False
self.bitrate = self.speed
self.reconnecting = False
if parity.lower() == "e" or parity.lower() == "even":
self.parity = "E"
@ -119,23 +97,15 @@ class SerialInterface(Interface):
try:
self.open_port()
if self.serial.is_open: self.configure_device()
else: raise IOError("Could not open serial port")
except Exception as e:
RNS.log(f"Could not open serial port for {self}", RNS.LOG_ERROR)
RNS.log(f"The contained exception was: {e}", RNS.LOG_ERROR)
RNS.log(f"Reticulum will attempt to bring up this interface periodically", RNS.LOG_ERROR)
if not self.detached and not self.reconnecting:
thread = threading.Thread(target=self.reconnect_port)
thread.daemon = True
thread.start()
RNS.log("Could not open serial port for interface "+str(self), RNS.LOG_ERROR)
raise e
if self.serial.is_open:
self.configure_device()
else:
raise IOError("Could not open serial port")
def detach(self):
self.detached = True
self.online = False
try: self.serial.close()
except Exception as e: RNS.log(f"Could not close port for {self} at detach: {e}", RNS.LOG_ERROR)
def open_port(self):
RNS.log("Opening serial port "+self.port+"...")
@ -202,13 +172,13 @@ class SerialInterface(Interface):
RNS.log("Serial port "+self.port+" is now open", RNS.LOG_VERBOSE)
def process_incoming(self, data):
def processIncoming(self, data):
self.rxb += len(data)
def af():
self.owner.inbound(data, self)
threading.Thread(target=af, daemon=True).start()
def process_outgoing(self,data):
def processOutgoing(self,data):
if self.online:
data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
written = self.serial.write(data)
@ -232,7 +202,7 @@ class SerialInterface(Interface):
if (in_frame and byte == HDLC.FLAG):
in_frame = False
self.process_incoming(data_buffer)
self.processIncoming(data_buffer)
elif (byte == HDLC.FLAG):
in_frame = True
data_buffer = b""
@ -267,23 +237,24 @@ class SerialInterface(Interface):
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
self.online = False
try: self.serial.close()
except Exception as e: RNS.log(f"Could not close port for {self}", RNS.LOG_ERROR)
if not self.detached and not self.reconnecting: self.reconnect_port()
self.serial.close()
self.reconnect_port()
def reconnect_port(self):
self.reconnecting = True
while not self.online and not self.detached:
while not self.online:
try:
time.sleep(5)
RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
self.open_port()
if self.serial.is_open: self.configure_device()
except Exception as e: RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
if self.serial.is_open:
self.configure_device()
except Exception as e:
RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
self.reconnecting = False
if self.online: RNS.log("Reconnected serial port for "+str(self))
RNS.log("Reconnected serial port for "+str(self))
def should_ingress_limit(self): return False
def should_ingress_limit(self):
return False
def __str__(self): return "SerialInterface["+self.name+"]"
def __str__(self):
return "SerialInterface["+self.name+"]"

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2025 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -31,9 +23,5 @@
import os
import glob
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
modules = py_modules+pyc_modules+so_modules+pyd_modules
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))
modules = glob.glob(os.path.dirname(__file__)+"/*.py")
__all__ = [ os.path.basename(f)[:-3] for f in modules if not f.endswith('__init__.py')]

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -28,7 +20,7 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from RNS.Interfaces.Interface import Interface
from .Interface import Interface
from collections import deque
import socketserver
import threading
@ -41,13 +33,9 @@ import RNS
class AutoInterface(Interface):
HW_MTU = 1196
FIXED_MTU = True
DEFAULT_DISCOVERY_PORT = 29716
DEFAULT_DATA_PORT = 42671
DEFAULT_GROUP_ID = "reticulum".encode("utf-8")
DEFAULT_IFAC_SIZE = 16
SCOPE_LINK = "2"
SCOPE_ADMIN = "4"
@ -58,14 +46,11 @@ class AutoInterface(Interface):
MULTICAST_PERMANENT_ADDRESS_TYPE = "0"
MULTICAST_TEMPORARY_ADDRESS_TYPE = "1"
PEERING_TIMEOUT = 22.0
ANNOUNCE_INTERVAL = 1.6
PEER_JOB_INTERVAL = 4.0
MCAST_ECHO_TIMEOUT = 6.5
PEERING_TIMEOUT = 7.5
ALL_IGNORE_IFS = ["lo0"]
DARWIN_IGNORE_IFS = ["awdl0", "llw0", "lo0", "en5"]
ANDROID_IGNORE_IFS = ["dummy0", "lo", "tun0", "rmnet0", "rmnet1", "rmnet2", "rmnet3", "rmnet4", "rmnet5", "rmnet6", "rmnet7"]
ANDROID_IGNORE_IFS = ["dummy0", "lo", "tun0"]
BITRATE_GUESS = 10*1000*1000
@ -93,6 +78,7 @@ class AutoInterface(Interface):
return ifas
def interface_name_to_index(self, ifname):
# socket.if_nametoindex doesn't work with uuid interface names on windows, it wants the ethernet_0 style
# we will just get the index from netinfo instead as it seems to work
if RNS.vendor.platformutils.is_windows():
@ -100,56 +86,39 @@ class AutoInterface(Interface):
return socket.if_nametoindex(ifname)
def __init__(self, owner, configuration):
c = Interface.get_config_obj(configuration)
name = c["name"]
group_id = c["group_id"] if "group_id" in c else None
discovery_scope = c["discovery_scope"] if "discovery_scope" in c else None
discovery_port = int(c["discovery_port"]) if "discovery_port" in c else None
multicast_address_type = c["multicast_address_type"] if "multicast_address_type" in c else None
data_port = int(c["data_port"]) if "data_port" in c else None
allowed_interfaces = c.as_list("devices") if "devices" in c else None
ignored_interfaces = c.as_list("ignored_devices") if "ignored_devices" in c else None
configured_bitrate = c["configured_bitrate"] if "configured_bitrate" in c else None
from RNS.Interfaces import netinfo
def __init__(self, owner, name, group_id=None, discovery_scope=None, discovery_port=None, multicast_address_type=None, data_port=None, allowed_interfaces=None, ignored_interfaces=None, configured_bitrate=None):
from RNS.vendor.ifaddr import niwrapper
super().__init__()
self.netinfo = netinfo
self.netinfo = niwrapper
self.HW_MTU = 1064
self.HW_MTU = AutoInterface.HW_MTU
self.IN = True
self.OUT = False
self.name = name
self.owner = owner
self.online = False
self.final_init_done = False
self.peers = {}
self.link_local_addresses = []
self.adopted_interfaces = {}
self.interface_servers = {}
self.discovery_sockets = []
self.multicast_echoes = {}
self.initial_echoes = {}
self.timed_out_interfaces = {}
self.spawned_interfaces = {}
self.write_lock = threading.Lock()
self.mif_deque = deque(maxlen=AutoInterface.MULTI_IF_DEQUE_LEN)
self.mif_deque_times = deque(maxlen=AutoInterface.MULTI_IF_DEQUE_LEN)
self.carrier_changed = False
self.outbound_udp_socket = None
self.announce_rate_target = None
self.announce_interval = AutoInterface.ANNOUNCE_INTERVAL
self.peer_job_interval = AutoInterface.PEER_JOB_INTERVAL
self.peering_timeout = AutoInterface.PEERING_TIMEOUT
self.multicast_echo_timeout = AutoInterface.MCAST_ECHO_TIMEOUT
self.reverse_peering_interval = self.announce_interval*3.25
self.announce_rate_target = None
self.announce_interval = AutoInterface.PEERING_TIMEOUT/6.0
self.peer_job_interval = AutoInterface.PEERING_TIMEOUT*1.1
self.peering_timeout = AutoInterface.PEERING_TIMEOUT
self.multicast_echo_timeout = AutoInterface.PEERING_TIMEOUT/2
# Increase peering timeout on Android, due to potential
# low-power modes implemented on many chipsets.
if RNS.vendor.platformutils.is_android():
self.peering_timeout *= 1.25
self.peering_timeout *= 3
if allowed_interfaces == None:
self.allowed_interfaces = []
@ -171,8 +140,6 @@ class AutoInterface(Interface):
else:
self.discovery_port = discovery_port
self.unicast_discovery_port = self.discovery_port+1
if multicast_address_type == None:
self.multicast_address_type = AutoInterface.MULTICAST_TEMPORARY_ADDRESS_TYPE
elif str(multicast_address_type).lower() == "temporary":
@ -248,50 +215,33 @@ class AutoInterface(Interface):
if link_local_addr == None:
RNS.log(str(self)+" No link-local IPv6 address configured for "+str(ifname)+", skipping interface", RNS.LOG_EXTREME)
else:
RNS.log(str(self)+" Creating unicast discovery listener on "+str(ifname)+" with address "+str(link_local_addr), RNS.LOG_EXTREME)
mcast_addr = self.mcast_discovery_address
RNS.log(str(self)+" Creating multicast discovery listener on "+str(ifname)+" with address "+str(mcast_addr), RNS.LOG_EXTREME)
# Struct with interface index
if_struct = struct.pack("I", self.interface_name_to_index(ifname))
# Set up unicast discovery socket
unicast_discovery_socket = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
unicast_discovery_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
if hasattr(socket, "SO_REUSEPORT"): unicast_discovery_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
# Bind unicast discovery socket
if RNS.vendor.platformutils.is_windows():
# Windows throws "[WinError 10049] The requested address is not valid in its context"
# when trying to use the multicast address as host, or when providing interface index
# passing an empty host appears to work, but probably not exactly how we want it to...
unicast_discovery_socket.bind(('', self.unicast_discovery_port))
else:
addr_info = socket.getaddrinfo(link_local_addr+"%"+ifname, self.unicast_discovery_port, socket.AF_INET6, socket.SOCK_DGRAM)
unicast_discovery_socket.bind(addr_info[0][4])
self.discovery_sockets.append(unicast_discovery_socket)
mcast_addr = self.mcast_discovery_address
RNS.log(str(self)+" Creating multicast discovery listener on "+str(ifname)+" with address "+str(mcast_addr), RNS.LOG_EXTREME)
# Set up multicast discovery socket
# Set up multicast socket
discovery_socket = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
discovery_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
if hasattr(socket, "SO_REUSEPORT"): discovery_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
if hasattr(socket, "SO_REUSEPORT"):
discovery_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
discovery_socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_MULTICAST_IF, if_struct)
# Join multicast group
mcast_group = socket.inet_pton(socket.AF_INET6, mcast_addr) + if_struct
discovery_socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_JOIN_GROUP, mcast_group)
# Bind multicast socket
# Bind socket
if RNS.vendor.platformutils.is_windows():
# Windows throws "[WinError 10049] The requested address is not valid in its context"
# window throws "[WinError 10049] The requested address is not valid in its context"
# when trying to use the multicast address as host, or when providing interface index
# passing an empty host appears to work, but probably not exactly how we want it to...
discovery_socket.bind(('', self.discovery_port))
else:
if self.discovery_scope == AutoInterface.SCOPE_LINK:
addr_info = socket.getaddrinfo(mcast_addr+"%"+ifname, self.discovery_port, socket.AF_INET6, socket.SOCK_DGRAM)
else:
@ -299,15 +249,13 @@ class AutoInterface(Interface):
discovery_socket.bind(addr_info[0][4])
self.discovery_sockets.append(discovery_socket)
# Set up thread for discovery packets
def discovery_loop():
self.discovery_handler(discovery_socket, ifname)
# Set up thread for multicast discovery packets
def discovery_loop(): self.discovery_handler(discovery_socket, ifname)
thread = threading.Thread(target=discovery_loop, daemon=True).start()
# Set up thread for unicast discovery packets
def unicast_discovery_loop(): self.discovery_handler(unicast_discovery_socket, ifname, announce=False)
thread = threading.Thread(target=unicast_discovery_loop, daemon=True).start()
thread = threading.Thread(target=discovery_loop)
thread.daemon = True
thread.start()
suitable_interfaces += 1
@ -328,53 +276,51 @@ class AutoInterface(Interface):
else:
self.bitrate = AutoInterface.BITRATE_GUESS
def final_init(self):
peering_wait = self.announce_interval*1.2
RNS.log(str(self)+" discovering peers for "+str(round(peering_wait, 2))+" seconds...", RNS.LOG_VERBOSE)
peering_wait = self.announce_interval*1.2
RNS.log(str(self)+" discovering peers for "+str(round(peering_wait, 2))+" seconds...", RNS.LOG_VERBOSE)
socketserver.UDPServer.address_family = socket.AF_INET6
self.owner = owner
socketserver.UDPServer.address_family = socket.AF_INET6
for ifname in self.adopted_interfaces:
local_addr = self.adopted_interfaces[ifname]+"%"+str(self.interface_name_to_index(ifname))
addr_info = socket.getaddrinfo(local_addr, self.data_port, socket.AF_INET6, socket.SOCK_DGRAM)
address = addr_info[0][4]
for ifname in self.adopted_interfaces:
local_addr = self.adopted_interfaces[ifname]+"%"+str(self.interface_name_to_index(ifname))
addr_info = socket.getaddrinfo(local_addr, self.data_port, socket.AF_INET6, socket.SOCK_DGRAM)
address = addr_info[0][4]
udp_server = socketserver.UDPServer(address, self.handler_factory(self.process_incoming))
self.interface_servers[ifname] = udp_server
udp_server = socketserver.UDPServer(address, self.handler_factory(self.processIncoming))
self.interface_servers[ifname] = udp_server
thread = threading.Thread(target=udp_server.serve_forever)
thread.daemon = True
thread.start()
job_thread = threading.Thread(target=self.peer_jobs)
job_thread.daemon = True
job_thread.start()
time.sleep(peering_wait)
self.online = True
def discovery_handler(self, socket, ifname):
def announce_loop():
self.announce_handler(ifname)
thread = threading.Thread(target=udp_server.serve_forever)
thread.daemon = True
thread.start()
job_thread = threading.Thread(target=self.peer_jobs)
job_thread.daemon = True
job_thread.start()
time.sleep(peering_wait)
self.online = True
self.final_init_done = True
def discovery_handler(self, socket, ifname, announce=True):
def announce_loop(): self.announce_handler(ifname)
thread = threading.Thread(target=announce_loop)
thread.daemon = True
thread.start()
if announce:
thread = threading.Thread(target=announce_loop)
thread.daemon = True
thread.start()
while not self.detached:
while True:
data, ipv6_src = socket.recvfrom(1024)
if self.final_init_done and data and ipv6_src:
peering_hash = data[:RNS.Identity.HASHLENGTH//8]
expected_hash = RNS.Identity.full_hash(self.group_id+ipv6_src[0].encode("utf-8"))
if peering_hash == expected_hash:
self.add_peer(ipv6_src[0], ifname)
else:
RNS.log(str(self)+" received peering packet on "+str(ifname)+" from "+str(ipv6_src[0])+", but authentication hash was incorrect.", RNS.LOG_DEBUG)
expected_hash = RNS.Identity.full_hash(self.group_id+ipv6_src[0].encode("utf-8"))
if data == expected_hash:
self.add_peer(ipv6_src[0], ifname)
else:
RNS.log(str(self)+" received peering packet on "+str(ifname)+" from "+str(ipv6_src[0])+", but authentication hash was incorrect.", RNS.LOG_DEBUG)
def peer_jobs(self):
while not self.detached:
while True:
time.sleep(self.peer_job_interval)
now = time.time()
timed_out_peers = []
@ -389,24 +335,8 @@ class AutoInterface(Interface):
# Remove any timed out peers
for peer_addr in timed_out_peers:
removed_peer = self.peers.pop(peer_addr)
if peer_addr in self.spawned_interfaces:
spawned_interface = self.spawned_interfaces[peer_addr]
spawned_interface.detach()
spawned_interface.teardown()
RNS.log(str(self)+" removed peer "+str(peer_addr)+" on "+str(removed_peer[0]), RNS.LOG_DEBUG)
# Send reverse peering packets
for peer_addr in self.peers:
try:
peer = self.peers[peer_addr]
ifname = peer[0]
last_outbound = peer[2]
if now > last_outbound+self.reverse_peering_interval:
self.reverse_announce(ifname, peer_addr)
peer[2] = time.time()
except Exception as e:
RNS.log(f"Error while sending reverse peering packet to {peer_addr}: {e}", RNS.LOG_ERROR)
for ifname in self.adopted_interfaces:
# Check that the link-local address has not changed
try:
@ -433,22 +363,14 @@ class AutoInterface(Interface):
if ifname in self.interface_servers:
RNS.log("Shutting down previous UDP listener for "+str(self)+" "+str(ifname), RNS.LOG_DEBUG)
previous_server = self.interface_servers[ifname]
def shutdown_server(): previous_server.shutdown()
def shutdown_server():
previous_server.shutdown()
threading.Thread(target=shutdown_server, daemon=True).start()
RNS.log("Starting new UDP listener for "+str(self)+" "+str(ifname), RNS.LOG_DEBUG)
retry_delay = 1.25
listener_started = False
while not listener_started:
try:
time.sleep(retry_delay)
udp_server = socketserver.UDPServer(listen_address, self.handler_factory(self.process_incoming))
self.interface_servers[ifname] = udp_server
listener_started = True
except Exception as e:
RNS.log(f"Could not start new UDP listener for {self} on {listen_address}: {e}", RNS.LOG_WARNING)
RNS.log(f"Retrying in {retry_delay} seconds", RNS.LOG_WARNING)
udp_server = socketserver.UDPServer(listen_address, self.handler_factory(self.processIncoming))
self.interface_servers[ifname] = udp_server
thread = threading.Thread(target=udp_server.serve_forever)
thread.daemon = True
@ -460,10 +382,9 @@ class AutoInterface(Interface):
RNS.log("Could not get device information while updating link-local addresses for "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
# Check multicast echo timeouts
last_multicast_echo = 0
multicast_echo_received = False
if ifname in self.multicast_echoes: last_multicast_echo = self.multicast_echoes[ifname]
if ifname in self.initial_echoes: multicast_echo_received = True
last_multicast_echo = 0
if ifname in self.multicast_echoes:
last_multicast_echo = self.multicast_echoes[ifname]
if now - last_multicast_echo > self.multicast_echo_timeout:
if ifname in self.timed_out_interfaces and self.timed_out_interfaces[ifname] == False:
@ -475,32 +396,13 @@ class AutoInterface(Interface):
self.carrier_changed = True
RNS.log(str(self)+" Carrier recovered on "+str(ifname), RNS.LOG_WARNING)
self.timed_out_interfaces[ifname] = False
if not multicast_echo_received:
RNS.log(f"{self} No multicast echoes received on {ifname}. The networking hardware or a firewall may be blocking multicast traffic.", RNS.LOG_ERROR)
# else:
# RNS.log(f"{self} Initial multicast echo on {ifname} received {RNS.prettytime(time.time()-self.initial_echoes[ifname])} ago.", RNS.LOG_DEBUG)
def announce_handler(self, ifname):
while not self.detached:
while True:
self.peer_announce(ifname)
time.sleep(self.announce_interval)
def reverse_announce(self, ifname, peer_addr):
try:
link_local_address = self.adopted_interfaces[ifname]
discovery_token = RNS.Identity.full_hash(self.group_id+link_local_address.encode("utf-8"))
announce_socket = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
addr_info = socket.getaddrinfo(f"{peer_addr}%{ifname}", self.unicast_discovery_port, socket.AF_INET6, socket.SOCK_DGRAM)
ifis = struct.pack("I", self.interface_name_to_index(ifname))
announce_socket.sendto(discovery_token, addr_info[0][4])
announce_socket.close()
except Exception as e:
RNS.log(f"Could not send reverse peering packet to {peer_addr} on {ifname}: {e}", RNS.LOG_ERROR)
def peer_announce(self, ifname):
try:
link_local_address = self.adopted_interfaces[ifname]
@ -519,10 +421,6 @@ class AutoInterface(Interface):
else:
pass
@property
def peer_count(self):
return len(self.spawned_interfaces)
def add_peer(self, addr, ifname):
if addr in self.link_local_addresses:
ifname = None
@ -532,173 +430,58 @@ class AutoInterface(Interface):
if ifname != None:
self.multicast_echoes[ifname] = time.time()
if not ifname in self.initial_echoes: self.initial_echoes[ifname] = time.time()
else:
RNS.log(str(self)+" received multicast echo on unexpected interface "+str(ifname), RNS.LOG_WARNING)
else:
if not addr in self.peers:
self.peers[addr] = [ifname, time.time(), time.time()]
spawned_interface = AutoInterfacePeer(self, addr, ifname)
spawned_interface.OUT = self.OUT
spawned_interface.IN = self.IN
spawned_interface.ingress_control = self.ingress_control
spawned_interface.ic_max_held_announces = self.ic_max_held_announces
spawned_interface.ic_burst_hold = self.ic_burst_hold
spawned_interface.ic_burst_freq = self.ic_burst_freq
spawned_interface.ic_burst_freq_new = self.ic_burst_freq_new
spawned_interface.ic_new_time = self.ic_new_time
spawned_interface.ic_burst_penalty = self.ic_burst_penalty
spawned_interface.ic_held_release_interval = self.ic_held_release_interval
spawned_interface.egress_control = self.egress_control
spawned_interface.ec_pr_freq = self.ec_pr_freq
spawned_interface.ic_pr_burst_freq_new = self.ic_pr_burst_freq_new
spawned_interface.ic_pr_burst_freq = self.ic_pr_burst_freq
spawned_interface._destination_cost = self._destination_cost
spawned_interface._pr_cost = self._pr_cost
spawned_interface.parent_interface = self
spawned_interface.bitrate = self.bitrate
spawned_interface.ifac_size = self.ifac_size
spawned_interface.ifac_netname = self.ifac_netname
spawned_interface.ifac_netkey = self.ifac_netkey
if spawned_interface.ifac_netname != None or spawned_interface.ifac_netkey != None:
ifac_origin = b""
if spawned_interface.ifac_netname != None:
ifac_origin += RNS.Identity.full_hash(spawned_interface.ifac_netname.encode("utf-8"))
if spawned_interface.ifac_netkey != None:
ifac_origin += RNS.Identity.full_hash(spawned_interface.ifac_netkey.encode("utf-8"))
ifac_origin_hash = RNS.Identity.full_hash(ifac_origin)
spawned_interface.ifac_key = RNS.Cryptography.hkdf(
length=64,
derive_from=ifac_origin_hash,
salt=RNS.Reticulum.IFAC_SALT,
context=None
)
spawned_interface.ifac_identity = RNS.Identity.from_bytes(spawned_interface.ifac_key)
spawned_interface.ifac_signature = spawned_interface.ifac_identity.sign(RNS.Identity.full_hash(spawned_interface.ifac_key))
spawned_interface.announce_rate_target = self.announce_rate_target
spawned_interface.announce_rate_grace = self.announce_rate_grace
spawned_interface.announce_rate_penalty = self.announce_rate_penalty
spawned_interface.mode = self.mode
spawned_interface.gravity = self.gravity
spawned_interface.recursive_prs = self.recursive_prs
spawned_interface.announces_from_internal = self.announces_from_internal
spawned_interface.announces_to_internal = self.announces_to_internal
spawned_interface.announce_cap = self.announce_cap
spawned_interface.HW_MTU = self.HW_MTU
spawned_interface.online = True
RNS.Transport.add_interface(spawned_interface)
if addr in self.spawned_interfaces:
self.spawned_interfaces[addr].detach()
self.spawned_interfaces[addr].teardown()
if addr in self.spawned_interfaces: self.spawned_interfaces.pop(addr)
self.spawned_interfaces[addr] = spawned_interface
self.peers[addr] = [ifname, time.time()]
RNS.log(str(self)+" added peer "+str(addr)+" on "+str(ifname), RNS.LOG_DEBUG)
else:
self.refresh_peer(addr)
def refresh_peer(self, addr):
try: self.peers[addr][1] = time.time()
except Exception as e: RNS.log(f"An error occurred while refreshing peer {addr} on {self}: {e}", RNS.LOG_ERROR)
self.peers[addr][1] = time.time()
def process_incoming(self, data, addr=None):
if self.online and addr in self.spawned_interfaces:
self.spawned_interfaces[addr].process_incoming(data, addr)
def processIncoming(self, data):
data_hash = RNS.Identity.full_hash(data)
deque_hit = False
if data_hash in self.mif_deque:
for te in self.mif_deque_times:
if te[0] == data_hash and time.time() < te[1]+AutoInterface.MULTI_IF_DEQUE_TTL:
deque_hit = True
break
def process_outgoing(self, data): pass
if not deque_hit:
self.mif_deque.append(data_hash)
self.mif_deque_times.append([data_hash, time.time()])
self.rxb += len(data)
self.owner.inbound(data, self)
def detach(self):
self.online = False
self.detached = True
RNS.log(f"Shutting down all UDP listeners for {self}", RNS.LOG_NOTICE)
for ifname in self.interface_servers:
try:
self.interface_servers[ifname].server_close()
self.interface_servers[ifname].shutdown()
except Exception as e: RNS.log(f"Could not shut down UDP listener {ifname} for {self}: {e}", RNS.LOG_ERROR)
for s in self.discovery_sockets:
try: s.close()
except Exception as e: RNS.log(f"Error while shutting down discovery socket {s} for {self}: {e}", RNS.LOG_ERROR)
def __str__(self): return f"AutoInterface[{self.name}]"
class AutoInterfacePeer(Interface):
def __init__(self, owner, addr, ifname):
super().__init__()
self.owner = owner
self.parent_interface = owner
self.addr = addr
self.ifname = ifname
self.peer_addr = None
self.addr_info = None
self.HW_MTU = self.owner.HW_MTU
self.FIXED_MTU = self.owner.FIXED_MTU
def __str__(self):
return f"AutoInterfacePeer[{self.ifname}/{self.addr}]"
def process_incoming(self, data, addr=None):
if self.online and self.owner.online:
data_hash = RNS.Identity.full_hash(data)
deque_hit = False
if data_hash in self.owner.mif_deque:
for te in self.owner.mif_deque_times:
if te[0] == data_hash and time.time() < te[1]+AutoInterface.MULTI_IF_DEQUE_TTL:
deque_hit = True
break
if not deque_hit:
self.owner.refresh_peer(self.addr)
self.owner.mif_deque.append(data_hash)
self.owner.mif_deque_times.append([data_hash, time.time()])
self.owner.owner.inbound(data, self)
def process_outgoing(self, data):
if self.online:
with self.owner.write_lock:
def processOutgoing(self,data):
for peer in self.peers:
try:
if self.owner.outbound_udp_socket == None: self.owner.outbound_udp_socket = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
if self.peer_addr == None: self.peer_addr = str(self.addr)+"%"+str(self.owner.interface_name_to_index(self.ifname))
if self.addr_info == None: self.addr_info = socket.getaddrinfo(self.peer_addr, self.owner.data_port, socket.AF_INET6, socket.SOCK_DGRAM)
self.owner.outbound_udp_socket.sendto(data, self.addr_info[0][4])
self.txb += len(data)
self.owner.txb += len(data)
if self.outbound_udp_socket == None:
self.outbound_udp_socket = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
peer_addr = str(peer)+"%"+str(self.interface_name_to_index(self.peers[peer][0]))
addr_info = socket.getaddrinfo(peer_addr, self.data_port, socket.AF_INET6, socket.SOCK_DGRAM)
self.outbound_udp_socket.sendto(data, addr_info[0][4])
except Exception as e:
RNS.log("Could not transmit on "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
def detach(self):
self.online = False
self.detached = True
def teardown(self):
if not self.detached:
RNS.log(f"The interface {self} experienced an unrecoverable error and is being torn down.", RNS.LOG_ERROR)
if RNS.Reticulum.panic_on_interface_error: RNS.panic()
self.txb += len(data)
else: RNS.log(f"The interface {self} is being torn down.", RNS.LOG_VERBOSE)
# Until per-device sub-interfacing is implemented,
# ingress limiting should be disabled on AutoInterface
def should_ingress_limit(self):
return False
self.online = False
self.OUT = False
self.IN = False
if self.addr in self.owner.spawned_interfaces:
try: self.owner.spawned_interfaces.pop(self.addr)
except Exception as e:
RNS.log(f"Could not remove {self} from parent interface on detach. The contained exception was: {e}", RNS.LOG_ERROR)
RNS.Transport.remove_interface(self)
def __str__(self):
return "AutoInterface["+self.name+"]"
class AutoInterfaceHandler(socketserver.BaseRequestHandler):
def __init__(self, callback, *args, **keys):
@ -707,5 +490,4 @@ class AutoInterfaceHandler(socketserver.BaseRequestHandler):
def handle(self):
data = self.request[0]
addr = self.client_address[0]
self.callback(data, addr)
self.callback(data)

File diff suppressed because it is too large Load diff

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2025 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -28,7 +20,7 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from RNS.Interfaces.Interface import Interface
from .Interface import Interface
import socketserver
import threading
import platform
@ -88,7 +80,8 @@ class I2PController:
self.ready = False
self.storagepath = rns_storagepath+"/i2p"
if not os.path.isdir(self.storagepath): os.makedirs(self.storagepath)
if not os.path.isdir(self.storagepath):
os.makedirs(self.storagepath)
def start(self):
@ -109,22 +102,28 @@ class I2PController:
except Exception as e:
self.ready = False
RNS.log("Exception on event loop for "+str(self)+": "+str(e), RNS.LOG_ERROR)
finally: self.loop.close()
finally:
self.loop.close()
def stop(self):
for i2ptunnel in self.i2plib_tunnels:
if hasattr(i2ptunnel, "stop") and callable(i2ptunnel.stop): i2ptunnel.stop()
if hasattr(i2ptunnel, "stop") and callable(i2ptunnel.stop):
i2ptunnel.stop()
if hasattr(asyncio.Task, "all_tasks") and callable(asyncio.Task.all_tasks):
for task in asyncio.Task.all_tasks(loop=self.loop): task.cancel()
for task in asyncio.Task.all_tasks(loop=self.loop):
task.cancel()
time.sleep(0.2)
self.loop.stop()
def get_free_port(self):
return self.i2plib.utils.get_free_port()
def stop_tunnel(self, i2ptunnel):
if hasattr(i2ptunnel, "stop") and callable(i2ptunnel.stop):
i2ptunnel.stop()
@ -152,9 +151,10 @@ class I2PController:
tn = self.i2plib_tunnels[i2p_destination]
if tn != None and hasattr(tn, "status"):
RNS.log("Waiting for status from I2P control process", RNS.LOG_DEBUG)
while not tn.status["setup_ran"]: time.sleep(0.1)
RNS.log("Got status from I2P control process", RNS.LOG_DEBUG)
RNS.log("Waiting for status from I2P control process", RNS.LOG_EXTREME)
while not tn.status["setup_ran"]:
time.sleep(0.1)
RNS.log("Got status from I2P control process", RNS.LOG_EXTREME)
if tn.status["setup_failed"]:
self.stop_tunnel(tn)
@ -164,19 +164,29 @@ class I2PController:
if owner.socket != None:
if hasattr(owner.socket, "close"):
if callable(owner.socket.close):
try: owner.socket.shutdown(socket.SHUT_RDWR)
except Exception as e: RNS.log("Error while shutting down socket for "+str(owner)+": "+str(e))
try: owner.socket.close()
except Exception as e: RNS.log("Error while closing socket for "+str(owner)+": "+str(e))
try:
owner.socket.shutdown(socket.SHUT_RDWR)
except Exception as e:
RNS.log("Error while shutting down socket for "+str(owner)+": "+str(e))
try:
owner.socket.close()
except Exception as e:
RNS.log("Error while closing socket for "+str(owner)+": "+str(e))
self.client_tunnels[i2p_destination] = True
owner.awaiting_i2p_tunnel = False
RNS.log(str(owner)+" tunnel setup complete", RNS.LOG_VERBOSE)
else: raise IOError("Got no status response from SAM API")
else:
raise IOError("Got no status response from SAM API")
except ConnectionRefusedError as e:
raise e
except ConnectionAbortedError as e:
raise e
except ConnectionRefusedError as e: raise e
except ConnectionAbortedError as e: raise e
except Exception as e:
RNS.log("Unexpected error type from I2P SAM: "+str(e), RNS.LOG_ERROR)
raise e
@ -188,33 +198,54 @@ class I2PController:
if i2ptunnel.status["setup_ran"] == False:
RNS.log(str(self)+" I2P tunnel setup did not complete", RNS.LOG_ERROR)
self.stop_tunnel(i2ptunnel)
return False
elif i2p_exception != None:
RNS.log("An error ocurred while setting up I2P tunnel to "+str(i2p_destination), RNS.LOG_ERROR)
if isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.CantReachPeer): RNS.log("The I2P daemon can't reach peer "+str(i2p_destination), RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedDest): RNS.log("The I2P daemon reported that the destination is already in use", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedId): RNS.log("The I2P daemon reported that the ID is arleady in use", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidId): RNS.log("The I2P daemon reported that the stream session ID doesn't exist", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidKey): RNS.log("The I2P daemon reported that the key for "+str(i2p_destination)+" is invalid", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.KeyNotFound): RNS.log("The I2P daemon could not find the key for "+str(i2p_destination), RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.PeerNotFound): RNS.log("The I2P daemon could not find the peer "+str(i2p_destination), RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.I2PError): RNS.log("The I2P daemon experienced an unspecified error", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.Timeout): RNS.log("I2P daemon timed out while setting up client tunnel to "+str(i2p_destination), RNS.LOG_ERROR)
else: RNS.log(f"Unspecified I2P daemon error: {i2p_exception}", RNS.LOG_ERROR)
if isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.CantReachPeer):
RNS.log("The I2P daemon can't reach peer "+str(i2p_destination), RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedDest):
RNS.log("The I2P daemon reported that the destination is already in use", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedId):
RNS.log("The I2P daemon reported that the ID is arleady in use", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidId):
RNS.log("The I2P daemon reported that the stream session ID doesn't exist", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidKey):
RNS.log("The I2P daemon reported that the key for "+str(i2p_destination)+" is invalid", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.KeyNotFound):
RNS.log("The I2P daemon could not find the key for "+str(i2p_destination), RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.PeerNotFound):
RNS.log("The I2P daemon mould not find the peer "+str(i2p_destination), RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.I2PError):
RNS.log("The I2P daemon experienced an unspecified error", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.Timeout):
RNS.log("I2P daemon timed out while setting up client tunnel to "+str(i2p_destination), RNS.LOG_ERROR)
RNS.log("Resetting I2P tunnel and retrying later", RNS.LOG_ERROR)
self.stop_tunnel(i2ptunnel)
return False
elif i2ptunnel.status["setup_failed"] == True:
RNS.log(str(self)+" Unspecified I2P tunnel setup error, resetting I2P tunnel", RNS.LOG_ERROR)
self.stop_tunnel(i2ptunnel)
return False
else:
RNS.log(str(self)+" Got no status from SAM API, resetting I2P tunnel", RNS.LOG_ERROR)
self.stop_tunnel(i2ptunnel)
return False
@ -235,8 +266,10 @@ class I2PController:
i2p_keyfile_nf = self.storagepath+"/"+RNS.hexrep(i2p_dest_hash_nf, delimit=False)+".i2p"
# Use old format if a key is already present
if os.path.isfile(i2p_keyfile_of): i2p_keyfile = i2p_keyfile_of
else: i2p_keyfile = i2p_keyfile_nf
if os.path.isfile(i2p_keyfile_of):
i2p_keyfile = i2p_keyfile_of
else:
i2p_keyfile = i2p_keyfile_nf
i2p_dest = None
if not os.path.isfile(i2p_keyfile):
@ -271,7 +304,8 @@ class I2PController:
asyncio.run_coroutine_threadsafe(tunnel_up(), self.loop).result()
self.server_tunnels[i2p_b32] = True
except Exception as e: raise e
except Exception as e:
raise e
else:
i2ptunnel = self.i2plib_tunnels[i2p_b32]
@ -280,45 +314,66 @@ class I2PController:
if i2ptunnel.status["setup_ran"] == False:
RNS.log(str(self)+" I2P tunnel setup did not complete", RNS.LOG_ERROR)
self.stop_tunnel(i2ptunnel)
return False
elif i2p_exception != None:
RNS.log("An error ocurred while setting up I2P tunnel", RNS.LOG_ERROR)
if isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.CantReachPeer): RNS.log("The I2P daemon can't reach peer "+str(i2p_destination), RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedDest): RNS.log("The I2P daemon reported that the destination is already in use", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedId): RNS.log("The I2P daemon reported that the ID is arleady in use", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidId): RNS.log("The I2P daemon reported that the stream session ID doesn't exist", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidKey): RNS.log("The I2P daemon reported that the key for "+str(i2p_destination)+" is invalid", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.KeyNotFound): RNS.log("The I2P daemon could not find the key for "+str(i2p_destination), RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.PeerNotFound): RNS.log("The I2P daemon could not find the peer "+str(i2p_destination), RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.I2PError): RNS.log("The I2P daemon experienced an unspecified error", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.Timeout): RNS.log("I2P daemon timed out while setting up client tunnel to "+str(i2p_destination), RNS.LOG_ERROR)
else: RNS.log(f"Unspecified I2P daemon error: {i2p_exception}", RNS.LOG_ERROR)
if isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.CantReachPeer):
RNS.log("The I2P daemon can't reach peer "+str(i2p_destination), RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedDest):
RNS.log("The I2P daemon reported that the destination is already in use", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedId):
RNS.log("The I2P daemon reported that the ID is arleady in use", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidId):
RNS.log("The I2P daemon reported that the stream session ID doesn't exist", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidKey):
RNS.log("The I2P daemon reported that the key for "+str(i2p_destination)+" is invalid", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.KeyNotFound):
RNS.log("The I2P daemon could not find the key for "+str(i2p_destination), RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.PeerNotFound):
RNS.log("The I2P daemon mould not find the peer "+str(i2p_destination), RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.I2PError):
RNS.log("The I2P daemon experienced an unspecified error", RNS.LOG_ERROR)
elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.Timeout):
RNS.log("I2P daemon timed out while setting up client tunnel to "+str(i2p_destination), RNS.LOG_ERROR)
RNS.log("Resetting I2P tunnel and retrying later", RNS.LOG_ERROR)
self.stop_tunnel(i2ptunnel)
return False
elif i2ptunnel.status["setup_failed"] == True:
RNS.log(str(self)+" Unspecified I2P tunnel setup error, resetting I2P tunnel", RNS.LOG_ERROR)
self.stop_tunnel(i2ptunnel)
return False
else:
RNS.log(str(self)+" Got no status from SAM API, resetting I2P tunnel", RNS.LOG_ERROR)
self.stop_tunnel(i2ptunnel)
return False
time.sleep(5)
def get_loop(self): return asyncio.get_event_loop()
def get_loop(self):
return asyncio.get_event_loop()
class ThreadingI2PServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
pass
class I2PInterfacePeer(Interface):
RECONNECT_WAIT = 15
RECONNECT_MAX_TRIES = None
@ -353,11 +408,9 @@ class I2PInterfacePeer(Interface):
self.online = False
self.detached = False
self.kiss_framing = False
self.tunnels_allowed = True
self.i2p_tunneled = True
self.i2p_dest = None
self.i2p_tunnel_ready = False
self.lossless_medium = True
self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
self.bitrate = I2PInterface.BITRATE_GUESS
self.last_read = 0
@ -370,11 +423,18 @@ class I2PInterfacePeer(Interface):
self.ifac_netkey = self.parent_interface.ifac_netkey
if self.ifac_netname != None or self.ifac_netkey != None:
ifac_origin = b""
if self.ifac_netname != None: ifac_origin += RNS.Identity.full_hash(self.ifac_netname.encode("utf-8"))
if self.ifac_netkey != None: ifac_origin += RNS.Identity.full_hash(self.ifac_netkey.encode("utf-8"))
if self.ifac_netname != None:
ifac_origin += RNS.Identity.full_hash(self.ifac_netname.encode("utf-8"))
if self.ifac_netkey != None:
ifac_origin += RNS.Identity.full_hash(self.ifac_netkey.encode("utf-8"))
ifac_origin_hash = RNS.Identity.full_hash(ifac_origin)
self.ifac_key = RNS.Cryptography.hkdf(length=64, derive_from=ifac_origin_hash, salt=RNS.Reticulum.IFAC_SALT, context=None)
self.ifac_key = RNS.Cryptography.hkdf(
length=64,
derive_from=ifac_origin_hash,
salt=RNS.Reticulum.IFAC_SALT,
context=None
)
self.ifac_identity = RNS.Identity.from_bytes(self.ifac_key)
self.ifac_signature = self.ifac_identity.sign(RNS.Identity.full_hash(self.ifac_key))
@ -382,8 +442,10 @@ class I2PInterfacePeer(Interface):
self.announce_rate_grace = None
self.announce_rate_penalty = None
if max_reconnect_tries == None: self.max_reconnect_tries = I2PInterfacePeer.RECONNECT_MAX_TRIES
else: self.max_reconnect_tries = max_reconnect_tries
if max_reconnect_tries == None:
self.max_reconnect_tries = I2PInterfacePeer.RECONNECT_MAX_TRIES
else:
self.max_reconnect_tries = max_reconnect_tries
if connected_socket != None:
self.receives = True
@ -391,12 +453,15 @@ class I2PInterfacePeer(Interface):
self.target_port = None
self.socket = connected_socket
if platform.system() == "Linux": self.set_timeouts_linux()
elif platform.system() == "Darwin": self.set_timeouts_osx()
if platform.system() == "Linux":
self.set_timeouts_linux()
elif platform.system() == "Darwin":
self.set_timeouts_osx()
elif target_i2p_dest != None:
self.receives = True
self.initiator = True
self.bind_ip = "127.0.0.1"
self.awaiting_i2p_tunnel = True
@ -424,9 +489,13 @@ class I2PInterfacePeer(Interface):
thread.start()
def wait_job():
while self.awaiting_i2p_tunnel: time.sleep(0.25)
while self.awaiting_i2p_tunnel:
time.sleep(0.25)
time.sleep(2)
if not self.kiss_framing:
self.wants_tunnel = True
if not self.connect(initial=True):
thread = threading.Thread(target=self.reconnect)
thread.daemon = True
@ -440,6 +509,7 @@ class I2PInterfacePeer(Interface):
thread.daemon = True
thread.start()
def set_timeouts_linux(self):
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_USER_TIMEOUT, int(I2PInterfacePeer.I2P_USER_TIMEOUT * 1000))
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
@ -448,8 +518,10 @@ class I2PInterfacePeer(Interface):
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPCNT, int(I2PInterfacePeer.I2P_PROBES))
def set_timeouts_osx(self):
if hasattr(socket, "TCP_KEEPALIVE"): TCP_KEEPIDLE = socket.TCP_KEEPALIVE
else: TCP_KEEPIDLE = 0x10
if hasattr(socket, "TCP_KEEPALIVE"):
TCP_KEEPIDLE = socket.TCP_KEEPALIVE
else:
TCP_KEEPIDLE = 0x10
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
self.socket.setsockopt(socket.IPPROTO_TCP, TCP_KEEPIDLE, int(I2PInterfacePeer.I2P_PROBE_AFTER))
@ -457,12 +529,16 @@ class I2PInterfacePeer(Interface):
def shutdown_socket(self, target_socket):
if callable(target_socket.close):
try:
if socket != None: target_socket.shutdown(socket.SHUT_RDWR)
except Exception as e: RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
if socket != None:
target_socket.shutdown(socket.SHUT_RDWR)
except Exception as e:
RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
try:
if socket != None: target_socket.close()
except Exception as e: RNS.log("Error while closing socket for "+str(self)+": "+str(e))
if socket != None:
target_socket.close()
except Exception as e:
RNS.log("Error while closing socket for "+str(self)+": "+str(e))
def detach(self):
RNS.log("Detaching "+str(self), RNS.LOG_DEBUG)
@ -471,39 +547,22 @@ class I2PInterfacePeer(Interface):
if callable(self.socket.close):
self.detached = True
try: self.socket.shutdown(socket.SHUT_RDWR)
except Exception as e: RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
try:
self.socket.shutdown(socket.SHUT_RDWR)
except Exception as e:
RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
try: self.socket.close()
except Exception as e: RNS.log("Error while closing socket for "+str(self)+": "+str(e))
try:
self.socket.close()
except Exception as e:
RNS.log("Error while closing socket for "+str(self)+": "+str(e))
self.socket = None
def early_synthesis(self, packet):
if not self.detached:
if not packet.packed: packet.pack()
if self.ifac_identity: raw = RNS.Transport.handle_outgoing_ifac(self, packet.raw);
else: raw = packet.raw
if self.kiss_framing: frame = bytes([KISS.FEND])+bytes([KISS.CMD_DATA])+KISS.escape(raw)+bytes([KISS.FEND])
else: frame = bytes([HDLC.FLAG])+HDLC.escape(raw)+bytes([HDLC.FLAG])
self.socket.sendall(frame)
self.txb += len(frame)
self.last_write = time.time()
if hasattr(self, "parent_interface") and self.parent_interface != None and self.parent_count: self.parent_interface.txb += len(frame)
def connect(self, initial=False):
try:
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.connect((self.target_ip, self.target_port))
if not self.kiss_framing:
self.wants_tunnel = True
self.remote_transport = None
self.synthesis_received = False
RNS.Transport.synthesize_tunnel(self)
self.online = True
except Exception as e:
@ -511,16 +570,24 @@ class I2PInterfacePeer(Interface):
if not self.awaiting_i2p_tunnel:
RNS.log("Initial connection for "+str(self)+" could not be established: "+str(e), RNS.LOG_ERROR)
RNS.log("Leaving unconnected and retrying connection in "+str(I2PInterfacePeer.RECONNECT_WAIT)+" seconds.", RNS.LOG_ERROR)
return False
else: raise e
if platform.system() == "Linux": self.set_timeouts_linux()
elif platform.system() == "Darwin": self.set_timeouts_osx()
return False
else:
raise e
if platform.system() == "Linux":
self.set_timeouts_linux()
elif platform.system() == "Darwin":
self.set_timeouts_osx()
self.online = True
self.writing = False
self.never_connected = False
if not self.kiss_framing and self.wants_tunnel:
RNS.Transport.synthesize_tunnel(self)
return True
def reconnect(self):
@ -537,10 +604,14 @@ class I2PInterfacePeer(Interface):
self.teardown()
break
try: self.connect()
try:
self.connect()
except Exception as e:
if not self.awaiting_i2p_tunnel: RNS.log("Connection attempt for "+str(self)+" failed: "+str(e), RNS.LOG_DEBUG)
else: RNS.log(str(self)+" still waiting for I2P tunnel to appear", RNS.LOG_VERBOSE)
if not self.awaiting_i2p_tunnel:
RNS.log("Connection attempt for "+str(self)+" failed: "+str(e), RNS.LOG_DEBUG)
else:
RNS.log(str(self)+" still waiting for I2P tunnel to appear", RNS.LOG_VERBOSE)
if not self.never_connected:
RNS.log(str(self)+" Re-established connection via I2P tunnel", RNS.LOG_INFO)
@ -549,31 +620,40 @@ class I2PInterfacePeer(Interface):
thread = threading.Thread(target=self.read_loop)
thread.daemon = True
thread.start()
if not self.kiss_framing:
RNS.Transport.synthesize_tunnel(self)
else:
RNS.log("Attempt to reconnect on a non-initiator I2P interface. This should not happen.", RNS.LOG_ERROR)
raise IOError("Attempt to reconnect on a non-initiator I2P interface")
def process_incoming(self, data):
if not data: return
def processIncoming(self, data):
self.rxb += len(data)
if hasattr(self, "parent_interface") and self.parent_interface != None and self.parent_count:
self.parent_interface.rxb += len(data)
self.owner.inbound(data, self)
def process_outgoing(self, data):
def processOutgoing(self, data):
if self.online:
while self.writing: time.sleep(0.001)
while self.writing:
time.sleep(0.001)
try:
self.writing = True
if self.kiss_framing: data = bytes([KISS.FEND])+bytes([KISS.CMD_DATA])+KISS.escape(data)+bytes([KISS.FEND])
else: data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
if self.kiss_framing:
data = bytes([KISS.FEND])+bytes([KISS.CMD_DATA])+KISS.escape(data)+bytes([KISS.FEND])
else:
data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
self.socket.sendall(data)
self.writing = False
self.txb += len(data)
self.last_write = time.time()
if hasattr(self, "parent_interface") and self.parent_interface != None and self.parent_count: self.parent_interface.txb += len(data)
if hasattr(self, "parent_interface") and self.parent_interface != None and self.parent_count:
self.parent_interface.txb += len(data)
except Exception as e:
RNS.log("Exception occurred while transmitting via "+str(self)+", tearing down interface", RNS.LOG_ERROR)
@ -582,7 +662,8 @@ class I2PInterfacePeer(Interface):
def read_watchdog(self):
while self.wd_reset: time.sleep(0.25)
while self.wd_reset:
time.sleep(0.25)
should_run = True
try:
@ -590,13 +671,17 @@ class I2PInterfacePeer(Interface):
time.sleep(1)
if (time.time()-self.last_read > I2PInterfacePeer.I2P_PROBE_AFTER*2):
if self.i2p_tunnel_state != I2PInterfacePeer.TUNNEL_STATE_STALE: RNS.log("I2P tunnel became unresponsive", RNS.LOG_DEBUG)
if self.i2p_tunnel_state != I2PInterfacePeer.TUNNEL_STATE_STALE:
RNS.log("I2P tunnel became unresponsive", RNS.LOG_DEBUG)
self.i2p_tunnel_state = I2PInterfacePeer.TUNNEL_STATE_STALE
else: self.i2p_tunnel_state = I2PInterfacePeer.TUNNEL_STATE_ACTIVE
else:
self.i2p_tunnel_state = I2PInterfacePeer.TUNNEL_STATE_ACTIVE
if (time.time()-self.last_write > I2PInterfacePeer.I2P_PROBE_AFTER*1):
try:
if self.socket != None: self.socket.sendall(bytes([HDLC.FLAG, HDLC.FLAG]))
if self.socket != None:
self.socket.sendall(bytes([HDLC.FLAG, HDLC.FLAG]))
except Exception as e:
RNS.log("An error ocurred while sending I2P keepalive. The contained exception was: "+str(e), RNS.LOG_ERROR)
self.shutdown_socket(self.socket)
@ -605,17 +690,22 @@ class I2PInterfacePeer(Interface):
if (time.time()-self.last_read > I2PInterfacePeer.I2P_READ_TIMEOUT):
RNS.log("I2P socket is unresponsive, restarting...", RNS.LOG_WARNING)
if self.socket != None:
try: self.socket.shutdown(socket.SHUT_RDWR)
except Exception as e: RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
try:
self.socket.shutdown(socket.SHUT_RDWR)
except Exception as e:
RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
try: self.socket.close()
except Exception as e: RNS.log("Error while closing socket for "+str(self)+": "+str(e))
try:
self.socket.close()
except Exception as e:
RNS.log("Error while closing socket for "+str(self)+": "+str(e))
should_run = False
self.wd_reset = False
finally: self.wd_reset = False
finally:
self.wd_reset = False
def read_loop(self):
try:
@ -642,7 +732,7 @@ class I2PInterfacePeer(Interface):
# Read loop for KISS framing
if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
in_frame = False
self.process_incoming(data_buffer)
self.processIncoming(data_buffer)
elif (byte == KISS.FEND):
in_frame = True
command = KISS.CMD_UNKNOWN
@ -669,7 +759,7 @@ class I2PInterfacePeer(Interface):
# Read loop for HDLC framing
if (in_frame and byte == HDLC.FLAG):
in_frame = False
self.process_incoming(data_buffer)
self.processIncoming(data_buffer)
elif (byte == HDLC.FLAG):
in_frame = True
data_buffer = b""
@ -686,6 +776,7 @@ class I2PInterfacePeer(Interface):
data_buffer = data_buffer+bytes([byte])
else:
self.online = False
self.wd_reset = True
time.sleep(2)
self.wd_reset = False
@ -707,12 +798,15 @@ class I2PInterfacePeer(Interface):
if self.initiator:
RNS.log("Attempting to reconnect...", RNS.LOG_WARNING)
self.reconnect()
else: self.teardown()
else:
self.teardown()
def teardown(self):
if self.initiator and not self.detached:
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is being torn down. Restart Reticulum to attempt to open this interface again.", RNS.LOG_ERROR)
if RNS.Reticulum.panic_on_interface_error: RNS.panic()
if RNS.Reticulum.panic_on_interface_error:
RNS.panic()
else:
RNS.log("The interface "+str(self)+" is being torn down.", RNS.LOG_VERBOSE)
@ -721,10 +815,13 @@ class I2PInterfacePeer(Interface):
self.IN = False
if hasattr(self, "parent_interface") and self.parent_interface != None:
while self in self.parent_interface.spawned_interfaces:
self.parent_interface.spawned_interfaces.remove(self)
if self.parent_interface.clients > 0:
self.parent_interface.clients -= 1
if self in RNS.Transport.interfaces:
if not self.initiator:
RNS.Transport.interfaces.remove(self)
if not self.initiator: RNS.Transport.remove_interface(self)
def __str__(self):
return "I2PInterfacePeer["+str(self.name)+"]"
@ -732,32 +829,17 @@ class I2PInterfacePeer(Interface):
class I2PInterface(Interface):
BITRATE_GUESS = 256*1000
DEFAULT_IFAC_SIZE = 16
@property
def clients(self): return len(self.spawned_interfaces)
def __init__(self, owner, configuration):
def __init__(self, owner, name, rns_storagepath, peers, connectable = False, ifac_size = 16, ifac_netname = None, ifac_netkey = None):
super().__init__()
c = Interface.get_config_obj(configuration)
name = c["name"]
rns_storagepath = c["storagepath"]
peers = c.as_list("peers") if "peers" in c else None
connectable = c.as_bool("connectable") if "connectable" in c else False
ifac_size = c["ifac_size"] if "ifac_size" in c else None
ifac_netname = c["ifac_netname"] if "ifac_netname" in c else None
ifac_netkey = c["ifac_netkey"] if "ifac_netkey" in c else None
self.HW_MTU = 1064
self.online = False
self.spawned_interfaces = []
self.clients = 0
self.owner = owner
self.connectable = connectable
self.i2p_tunneled = True
self.tunnels_allowed = True
self.peers = peers
self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
self.b32 = None
@ -767,6 +849,7 @@ class I2PInterface(Interface):
self.OUT = False
self.name = name
self.receives = True
self.bind_ip = "127.0.0.1"
self.bind_port = self.i2p.get_free_port()
@ -775,8 +858,6 @@ class I2PInterface(Interface):
self.ifac_size = ifac_size
self.ifac_netname = ifac_netname
self.ifac_netkey = ifac_netkey
self.supports_discovery = True
self.lossless_medium = True
self.online = False
@ -791,9 +872,11 @@ class I2PInterface(Interface):
RNS.log("I2P controller did not become available in time, waiting for controller", RNS.LOG_VERBOSE)
i2p_notready_warning = True
while not self.i2p.ready: time.sleep(0.25)
while not self.i2p.ready:
time.sleep(0.25)
if i2p_notready_warning == True: RNS.log("I2P controller ready, continuing setup", RNS.LOG_VERBOSE)
if i2p_notready_warning == True:
RNS.log("I2P controller ready, continuing setup", RNS.LOG_VERBOSE)
def handlerFactory(callback):
def createHandler(*args, **keys):
@ -807,7 +890,6 @@ class I2PInterface(Interface):
thread.daemon = True
thread.start()
def final_init(self):
if self.connectable:
def tunnel_job():
while True:
@ -827,83 +909,22 @@ class I2PInterface(Interface):
thread.daemon = True
thread.start()
threading.Thread(target=self._spawn_peers, daemon=True).start()
def _spawn_peers(self):
time.sleep(1.5)
if self.peers != None:
for peer_addr in self.peers:
try:
interface_name = self.name+" to "+peer_addr
peer_interface = I2PInterfacePeer(self, self.owner, interface_name, peer_addr)
peer_interface.OUT = self.OUT
peer_interface.IN = self.IN
peer_interface.parent_interface = self
peer_interface.parent_count = False
peer_interface.mode = self.mode
peer_interface.bitrate = self.bitrate
peer_interface.recursive_prs = self.recursive_prs
peer_interface.gravity = self.gravity
peer_interface.HW_MTU = self.HW_MTU
peer_interface.announces_to_internal = self.announces_to_internal
peer_interface.announces_from_internal = self.announces_from_internal
peer_interface.announce_cap = self.announce_cap
peer_interface.ingress_control = self.ingress_control
peer_interface.ic_max_held_announces = self.ic_max_held_announces
peer_interface.ic_burst_hold = self.ic_burst_hold
peer_interface.ic_burst_freq = self.ic_burst_freq
peer_interface.ic_burst_freq_new = self.ic_burst_freq_new
peer_interface.ic_new_time = self.ic_new_time
peer_interface.ic_burst_penalty = self.ic_burst_penalty
peer_interface.ic_held_release_interval = self.ic_held_release_interval
peer_interface.egress_control = self.egress_control
peer_interface.ec_pr_freq = self.ec_pr_freq
peer_interface.ic_pr_burst_freq_new = self.ic_pr_burst_freq_new
peer_interface.ic_pr_burst_freq = self.ic_pr_burst_freq
peer_interface.announce_rate_target = self.announce_rate_target
peer_interface.announce_rate_grace = self.announce_rate_grace
peer_interface.announce_rate_penalty = self.announce_rate_penalty
peer_interface._destination_cost = self._destination_cost
peer_interface._pr_cost = self._pr_cost
RNS.Transport.add_interface(peer_interface)
except Exception as e: RNS.log(f"Could not create interface for {self} to {peer_addr}: {e}", RNS.LOG_ERROR)
if peers != None:
for peer_addr in peers:
interface_name = self.name+" to "+peer_addr
peer_interface = I2PInterfacePeer(self, self.owner, interface_name, peer_addr)
peer_interface.OUT = True
peer_interface.IN = True
peer_interface.parent_interface = self
peer_interface.parent_count = False
RNS.Transport.interfaces.append(peer_interface)
def incoming_connection(self, handler):
RNS.log("Accepting incoming I2P connection", RNS.LOG_VERBOSE)
interface_name = "Connected peer on "+self.name
spawned_interface = I2PInterfacePeer(self, self.owner, interface_name, connected_socket=handler.request)
spawned_interface.OUT = self.OUT
spawned_interface.IN = self.IN
spawned_interface.ingress_control = self.ingress_control
spawned_interface.ic_max_held_announces = self.ic_max_held_announces
spawned_interface.ic_burst_hold = self.ic_burst_hold
spawned_interface.ic_burst_freq = self.ic_burst_freq
spawned_interface.ic_burst_freq_new = self.ic_burst_freq_new
spawned_interface.ic_new_time = self.ic_new_time
spawned_interface.ic_burst_penalty = self.ic_burst_penalty
spawned_interface.ic_held_release_interval = self.ic_held_release_interval
spawned_interface.announces_to_internal = self.announces_to_internal
spawned_interface.announces_from_internal = self.announces_from_internal
spawned_interface.announce_cap = self.announce_cap
spawned_interface.egress_control = self.egress_control
spawned_interface.ec_pr_freq = self.ec_pr_freq
spawned_interface.ic_pr_burst_freq_new = self.ic_pr_burst_freq_new
spawned_interface.ic_pr_burst_freq = self.ic_pr_burst_freq
spawned_interface._destination_cost = self._destination_cost
spawned_interface._pr_cost = self._pr_cost
spawned_interface.OUT = True
spawned_interface.IN = True
spawned_interface.parent_interface = self
spawned_interface.online = True
spawned_interface.bitrate = self.bitrate
@ -913,11 +934,18 @@ class I2PInterface(Interface):
spawned_interface.ifac_netkey = self.ifac_netkey
if spawned_interface.ifac_netname != None or spawned_interface.ifac_netkey != None:
ifac_origin = b""
if spawned_interface.ifac_netname != None: ifac_origin += RNS.Identity.full_hash(spawned_interface.ifac_netname.encode("utf-8"))
if spawned_interface.ifac_netkey != None: ifac_origin += RNS.Identity.full_hash(spawned_interface.ifac_netkey.encode("utf-8"))
if spawned_interface.ifac_netname != None:
ifac_origin += RNS.Identity.full_hash(spawned_interface.ifac_netname.encode("utf-8"))
if spawned_interface.ifac_netkey != None:
ifac_origin += RNS.Identity.full_hash(spawned_interface.ifac_netkey.encode("utf-8"))
ifac_origin_hash = RNS.Identity.full_hash(ifac_origin)
spawned_interface.ifac_key = RNS.Cryptography.hkdf(length=64, derive_from=ifac_origin_hash, salt=RNS.Reticulum.IFAC_SALT, context=None)
spawned_interface.ifac_key = RNS.Cryptography.hkdf(
length=64,
derive_from=ifac_origin_hash,
salt=RNS.Reticulum.IFAC_SALT,
context=None
)
spawned_interface.ifac_identity = RNS.Identity.from_bytes(spawned_interface.ifac_key)
spawned_interface.ifac_signature = spawned_interface.ifac_identity.sign(RNS.Identity.full_hash(spawned_interface.ifac_key))
@ -925,48 +953,32 @@ class I2PInterface(Interface):
spawned_interface.announce_rate_grace = self.announce_rate_grace
spawned_interface.announce_rate_penalty = self.announce_rate_penalty
spawned_interface.mode = self.mode
spawned_interface.recursive_prs = self.recursive_prs
spawned_interface.gravity = self.gravity
spawned_interface.hop_limiter = self.hop_limiter
spawned_interface.HW_MTU = self.HW_MTU
RNS.log("Spawned new I2PInterface Peer: "+str(spawned_interface), RNS.LOG_VERBOSE)
RNS.Transport.add_interface(spawned_interface)
while spawned_interface in self.spawned_interfaces: self.spawned_interfaces.remove(spawned_interface)
self.spawned_interfaces.append(spawned_interface)
RNS.Transport.interfaces.append(spawned_interface)
self.clients += 1
spawned_interface.read_loop()
def process_outgoing(self, data):
def processOutgoing(self, data):
pass
def received_announce(self, size=0, from_spawned=False):
if from_spawned:
self.ia_freq_deque.append(time.time())
self.arxb += size
def received_announce(self, from_spawned=False):
if from_spawned: self.ia_freq_deque.append(time.time())
def sent_announce(self, size=0, from_spawned=False):
if from_spawned:
self.oa_freq_deque.append(time.time())
self.atxb += size
def received_path_request(self, size=0, from_spawned=False):
if from_spawned:
self.ip_freq_deque.append(time.time())
self.prxb += size
def sent_path_request(self, size=0, from_spawned=False):
if from_spawned:
self.op_freq_deque.append(time.time())
self.ptxb += size
def sent_announce(self, from_spawned=False):
if from_spawned: self.oa_freq_deque.append(time.time())
def detach(self):
RNS.log("Detaching "+str(self), RNS.LOG_DEBUG)
self.i2p.stop()
def __str__(self): return "I2PInterface["+self.name+"]"
def __str__(self):
return "I2PInterface["+self.name+"]"
class I2PInterfaceHandler(socketserver.BaseRequestHandler):
def __init__(self, callback, *args, **keys):
self.callback = callback
socketserver.BaseRequestHandler.__init__(self, *args, **keys)
def handle(self): self.callback(handler=self)
def handle(self):
self.callback(handler=self)

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2023 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -32,13 +24,13 @@ import RNS
import time
import threading
from collections import deque
from RNS.vendor.configobj import ConfigObj
class Interface:
IN = False
OUT = False
FWD = False
RPT = False
name = None
# Interface mode definitions
MODE_FULL = 0x01
@ -47,167 +39,55 @@ class Interface:
MODE_ROAMING = 0x04
MODE_BOUNDARY = 0x05
MODE_GATEWAY = 0x06
MODE_INTERNAL = 0x07
# Which interface modes a Transport Node should
# actively discover paths for.
DISCOVER_PATHS_FOR = [MODE_ACCESS_POINT, MODE_GATEWAY, MODE_ROAMING, MODE_INTERNAL]
BOUNDARY_SEARCH_MODES = [MODE_BOUNDARY, MODE_GATEWAY]
DISCOVER_PATHS_FOR = [MODE_ACCESS_POINT, MODE_GATEWAY, MODE_ROAMING]
# How many samples to use for announce
# frequency calculations
IA_FREQ_SAMPLES = 48
OA_FREQ_SAMPLES = 48
IP_FREQ_SAMPLES = 48
OP_FREQ_SAMPLES = 48
AR_MINFREQ_HZ = 0.1
PR_MINFREQ_HZ = 0.1
AR_FREQ_DECAY = 1/AR_MINFREQ_HZ
PR_FREQ_DECAY = 1/PR_MINFREQ_HZ
IA_FREQ_SAMPLES = 6
OA_FREQ_SAMPLES = 6
# Maximum amount of ingress limited announces
# to hold at any given time.
MAX_HELD_ANNOUNCES = 64
MAX_HELD_ANNOUNCES = 256
# Control parameters
DEFAULT_GRAVITY = 0
# How long a spawned interface will be
# considered to be newly created. Two
# hours by default.
IC_NEW_TIME = 2*60*60
IC_BURST_FREQ_NEW = 3
IC_BURST_FREQ = 10
IC_PR_BURST_FREQ_NEW = 3
IC_PR_BURST_FREQ = 8
IC_BURST_HOLD = 15
IC_BURST_PENALTY = 15
IC_HELD_RELEASE_INTERVAL = 5
IC_DEQUE_MIN_SAMPLE = 2
EC_BURST_MIN_SAMPLES = 2
EC_PR_FREQ = 5
EGRESS_CONTROL = False
# Default announce rate targets
DEFAULT_AR_TARGET = 3600
DEFAULT_AR_PENALTY = 0
DEFAULT_AR_GRACE = 5
AUTOCONFIGURE_MTU = False
FIXED_MTU = False
DEFAULT_IFAC_SIZE = 16
IC_BURST_FREQ_NEW = 3.5
IC_BURST_FREQ = 12
IC_BURST_HOLD = 1*60
IC_BURST_PENALTY = 5*60
IC_HELD_RELEASE_INTERVAL = 30
def __init__(self):
self.name = None
self.rxb = 0
self.txb = 0
self.frxb = 0
self.ftxb = 0
self.arxb = 0
self.atxb = 0
self.arxc = 0
self.atxc = 0
self.afrxc = 0
self.prxb = 0
self.ptxb = 0
self.prxc = 0
self.ptxc = 0
self.gravity = 0
self.created = time.time()
self.detached = False
self.online = False
self.bitrate = 62500
self.HW_MTU = None
self.__hash = None
self.rxb = 0
self.txb = 0
self.created = time.time()
self.online = False
self.bitrate = 1e6
self.announce_cap = RNS.Reticulum.ANNOUNCE_CAP/100.0
self.announce_allowed_at = 0
self.supports_discovery = False
self.discoverable = False
self.last_discovery_announce = 0
self.bootstrap_only = False
self.recursive_prs = False
self.announces_from_internal = True
self.announces_to_internal = None
self.is_local_shared_instance = False
self.parent_interface = None
self.spawned_interfaces = None
self.tunnel_id = None
self.local_tunnel_id = None
self.tunnels_allowed = False
self.synthesis_received = False
self.synthesis_emitted = False
self.wants_tunnel = False
self.remote_transport = None
self.ingress_control = True
self.hop_limiter = None
self.phy_keepalive = False
self.shared_medium = False
self.lossless_medium = False
self.remote_impl = None
self.remote_vers = None
self.remote_dst_cost = None
self.remote_pr_cost = None
self.remote_is_tn = None
self.ic_burst_active = False
self.ic_burst_activated = 0
self.ic_burst_sustained = 0
self.ic_pr_burst_active = False
self.ic_pr_burst_activated = 0
self.ic_pr_burst_sustained = 0
self.ic_pr_burst_cooldown = 0
self.ic_held_release = 0
self.ic_max_held_announces = RNS.Reticulum.get_instance()._default_ic_max_held_announces()
self.ic_burst_hold = RNS.Reticulum.get_instance()._default_ic_burst_hold()
self.ic_burst_freq_new = RNS.Reticulum.get_instance()._default_ic_burst_freq_new()
self.ic_burst_freq = RNS.Reticulum.get_instance()._default_ic_burst_freq()
self.ic_pr_burst_freq_new = RNS.Reticulum.get_instance()._default_ic_pr_burst_freq_new()
self.ic_pr_burst_freq = RNS.Reticulum.get_instance()._default_ic_pr_burst_freq()
self.ic_new_time = RNS.Reticulum.get_instance()._default_ic_new_time()
self.ic_burst_penalty = RNS.Reticulum.get_instance()._default_ic_burst_penalty()
self.ic_held_release_interval = RNS.Reticulum.get_instance()._default_ic_held_release_interval()
self.ec_pr_freq = RNS.Reticulum.get_instance()._default_ec_pr_freq()
self.egress_control = RNS.Reticulum.get_instance()._default_egress_control()
self.announce_queue = []
self.held_announces = {}
self.ifac_identity = None
self.transmit_buffer = None
self.ingress_control = True
self.ic_max_held_announces = Interface.MAX_HELD_ANNOUNCES
self.ic_burst_hold = Interface.IC_BURST_HOLD
self.ic_burst_active = False
self.ic_burst_activated = 0
self.ic_held_release = 0
self.ic_burst_freq_new = Interface.IC_BURST_FREQ_NEW
self.ic_burst_freq = Interface.IC_BURST_FREQ
self.ic_new_time = Interface.IC_NEW_TIME
self.ic_burst_penalty = Interface.IC_BURST_PENALTY
self.ic_held_release_interval = Interface.IC_HELD_RELEASE_INTERVAL
self.held_announces = {}
self.ia_freq_deque = deque(maxlen=Interface.IA_FREQ_SAMPLES)
self.oa_freq_deque = deque(maxlen=Interface.OA_FREQ_SAMPLES)
self.ip_freq_deque = deque(maxlen=Interface.IA_FREQ_SAMPLES)
self.op_freq_deque = deque(maxlen=Interface.OA_FREQ_SAMPLES)
self.protocol_violations = 0
self.ifac_violations = 0
self.packet_filter_hits = 0
self.hop_limiter_hits = 0
self.dp_ingress_tcount = 0
self.dp_ingress_bytes = 0
self.dp_ingress_packets = 0
self.dp_ingress_hold = None
self.dp_ingress_gated = False
self.tx_hwm = 4*1024*1024
self.tx_stalled = False
self.tx_drops = 0
self.tx_dropped_bytes = 0
self._dp_ec_prev_sent = 0
self._dp_ec_zero_ticks = 0
self._dp_ec_last_drain = time.time()
self._destination_cost = None
self._pr_cost = None
self.reports_phy_stats = False
self.r_stat_rssi = None
self.r_stat_snr = None
self.r_stat_q = None
self.is_connected_to_shared_instance = False
def get_hash(self):
if not self.__hash: self.__hash = RNS.Identity.full_hash(str(self).encode("utf-8"))
return self.__hash
return RNS.Identity.full_hash(str(self).encode("utf-8"))
# This is a generic function for determining when an interface
# should activate ingress limiting. Since this can vary for
@ -219,247 +99,107 @@ class Interface:
ia_freq = self.incoming_announce_frequency()
if self.ic_burst_active:
if ia_freq < freq_threshold and time.time() > self.ic_burst_activated+self.ic_burst_hold and time.time() > self.ic_burst_sustained+self.ic_burst_hold:
if len(self.ia_freq_deque) >= self.IC_DEQUE_MIN_SAMPLE: self.ic_burst_active = False
else:
if ia_freq >= freq_threshold: self.ic_burst_sustained = time.time()
if ia_freq < freq_threshold and time.time() > self.ic_burst_activated+self.ic_burst_hold:
self.ic_burst_active = False
self.ic_held_release = time.time() + self.ic_burst_penalty
return True
else:
if ia_freq > freq_threshold:
self.ic_burst_active = True
self.ic_burst_activated = time.time()
self.ic_burst_sustained = time.time()
self.ic_held_release = time.time() + self.ic_burst_penalty
return True
else: return False
else: return False
def should_ingress_limit_pr(self):
if self.ingress_control:
freq_threshold = self.ic_pr_burst_freq_new if self.age() < self.ic_new_time else self.ic_pr_burst_freq
ip_freq = self.incoming_pr_frequency()
if self.ic_pr_burst_active:
if ip_freq < freq_threshold and time.time() > self.ic_pr_burst_activated+self.ic_burst_hold and time.time() > self.ic_pr_burst_sustained+self.ic_burst_hold:
if self.ic_pr_burst_cooldown <= 0: self.ic_pr_burst_active = False
else: self.ic_pr_burst_cooldown -= 1
else:
self.ic_pr_burst_cooldown = 3
if ip_freq >= freq_threshold: self.ic_pr_burst_sustained = time.time()
return False
return True
else:
if ip_freq > freq_threshold:
self.ic_pr_burst_active = True
self.ic_pr_burst_activated = time.time()
self.ic_pr_burst_sustained = time.time()
self.ic_pr_burst_cooldown = 3
return True
else: return False
else: return False
def should_egress_limit_pr(self):
if self.egress_control:
freq_threshold = self.ec_pr_freq
op_freq = self.outgoing_pr_frequency(preemptive=True)
if op_freq > freq_threshold:
if len(self.op_freq_deque) >= self.EC_BURST_MIN_SAMPLES: return True
return False
def optimise_mtu(self):
if self.AUTOCONFIGURE_MTU:
if self.bitrate >= 1_000_000_000: self.HW_MTU = 524288
elif self.bitrate >= 750_000_000: self.HW_MTU = 262144
elif self.bitrate >= 400_000_000: self.HW_MTU = 131072
elif self.bitrate >= 200_000_000: self.HW_MTU = 65536
elif self.bitrate >= 100_000_000: self.HW_MTU = 32768
elif self.bitrate >= 10_000_000: self.HW_MTU = 16384
elif self.bitrate >= 5_000_000: self.HW_MTU = 8192
elif self.bitrate >= 2_000_000: self.HW_MTU = 4096
elif self.bitrate >= 1_000_000: self.HW_MTU = 2048
elif self.bitrate >= 62_500: self.HW_MTU = 1024
else: self.HW_MTU = None
RNS.log(f"{self} hardware MTU set to {self.HW_MTU}", RNS.LOG_EXTREME)
else:
return False
def age(self):
return time.time()-self.created
def hold_announce(self, announce_packet):
if announce_packet.hops >= RNS.Transport.PATHFINDER_M-1:
return self.filtered_announce()
elif announce_packet.destination_hash in self.held_announces:
if announce_packet.destination_hash in self.held_announces:
self.held_announces[announce_packet.destination_hash] = announce_packet
return self.filtered_announce()
elif not len(self.held_announces) >= self.ic_max_held_announces:
self.held_announces[announce_packet.destination_hash] = announce_packet
return True
else: return self.filtered_announce()
def process_held_announces(self):
try:
if len(self.held_announces) > 0 and time.time() > self.ic_held_release:
if not self.should_ingress_limit() and len(self.held_announces) > 0 and time.time() > self.ic_held_release:
freq_threshold = self.ic_burst_freq_new if self.age() < self.ic_new_time else self.ic_burst_freq
ia_freq = self.incoming_announce_frequency()
if ia_freq < freq_threshold:
selected_announce_packet = None
min_hops = RNS.Transport.PATHFINDER_M
cull_entries = []
for destination_hash in self.held_announces.copy():
announce_packet = self.held_announces.get(destination_hash)
if not announce_packet: continue
# If we can determine that a held announce has become
# useless for any potential path table updates, we'll
# remove it immediately instead of waiting for it to
# eventually get released, freeing up other held queue
# slots for potentially useful new announces.
path_entry = RNS.Transport.path_table.get(destination_hash)
if path_entry and not RNS.Transport.announce_prefilter(announce_packet, path_entry):
cull_entries.append(destination_hash)
continue
for destination_hash in self.held_announces:
announce_packet = self.held_announces[destination_hash]
if announce_packet.hops < min_hops:
min_hops = announce_packet.hops
selected_announce_packet = announce_packet
for destination_hash in cull_entries: self.held_announces.pop(destination_hash, None)
if selected_announce_packet != None:
RNS.log("Releasing held announce packet "+str(selected_announce_packet)+" from "+str(self), RNS.LOG_EXTREME)
self.ic_held_release = time.time() + self.ic_held_release_interval
self.held_announces.pop(selected_announce_packet.destination_hash, None)
RNS.Transport.inbound(selected_announce_packet.raw, selected_announce_packet.receiving_interface, tc=RNS.Transport.TC_INGRESS_LIMITED, ifac_handled=True)
self.held_announces.pop(selected_announce_packet.destination_hash)
def release():
RNS.Transport.inbound(selected_announce_packet.raw, selected_announce_packet.receiving_interface)
threading.Thread(target=release, daemon=True).start()
except Exception as e:
RNS.log("An error occurred while processing held announces for "+str(self), RNS.LOG_ERROR)
RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
def received_announce(self, size=0, from_spawned=False):
self.arxc += 1; self.arxb += size
def received_announce(self):
self.ia_freq_deque.append(time.time())
if self.parent_interface != None:
self.parent_interface.received_announce(size=size, from_spawned=True)
if hasattr(self, "parent_interface") and self.parent_interface != None:
self.parent_interface.received_announce(from_spawned=True)
def sent_announce(self, size=0, from_spawned=False):
self.atxc += 1; self.atxb += size
def sent_announce(self):
self.oa_freq_deque.append(time.time())
if self.parent_interface != None:
self.parent_interface.sent_announce(size=size, from_spawned=True)
def filtered_announce(self, size=0, from_spawned=False):
self.afrxc += 1
RNS.Transport.filtered_arxc += 1
if self.parent_interface != None: self.parent_interface.afrxc += 1
return None
def received_path_request(self, size=0, from_spawned=False):
self.prxc += 1; self.prxb += size
self.ip_freq_deque.append(time.time())
if self.parent_interface != None:
self.parent_interface.received_path_request(size=size, from_spawned=True)
def sent_path_request(self, size=0, from_spawned=False):
self.ptxc += 1; self.ptxb += size
self.op_freq_deque.append(time.time())
if self.parent_interface != None:
self.parent_interface.sent_path_request(size=size, from_spawned=True)
def protocol_violation(self, description=None, level=None):
if level == None: level = RNS.LOG_DEBUG
self.protocol_violations += 1
RNS.log(f"Protocol violation on {self}: {description}", level) if RNS.sl(level) else None
return None
def ifac_violation(self, description=None):
self.ifac_violations += 1
RNS.log(f"IFAC violation on {self}: {description}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
return None
def packet_filter_hit(self):
self.packet_filter_hits += 1
return None
def hop_limiter_hit(self):
self.hop_limiter_hits += 1
return None
@property
def destination_cost(self):
if not self._destination_cost == None: return self._destination_cost
else: return RNS.Transport.DESTINATION_COST
@property
def pr_cost(self):
if not self._pr_cost == None: return self._pr_cost
else: return RNS.Transport.PR_COST
@property
def ic_burst_count(self): return None
@property
def ic_pr_burst_count(self): return None
if hasattr(self, "parent_interface") and self.parent_interface != None:
self.parent_interface.sent_announce(from_spawned=True)
def incoming_announce_frequency(self):
n = len(self.ia_freq_deque)
if not n > self.IC_DEQUE_MIN_SAMPLE: return 0
if not len(self.ia_freq_deque) > 1:
return 0
else:
oldest = self.ia_freq_deque[0]
span = time.time() - oldest
if span > self.AR_FREQ_DECAY: self.ia_freq_deque.popleft()
if span <= 0: return 0
hz = n / span
return hz
dq_len = len(self.ia_freq_deque)
delta_sum = 0
for i in range(1,dq_len):
delta_sum += self.ia_freq_deque[i]-self.ia_freq_deque[i-1]
delta_sum += time.time() - self.ia_freq_deque[dq_len-1]
if delta_sum == 0:
avg = 0
else:
avg = 1/(delta_sum/(dq_len))
return avg
def outgoing_announce_frequency(self):
n = len(self.oa_freq_deque)
if not len(self.oa_freq_deque) > 1: return 0
if not len(self.oa_freq_deque) > 1:
return 0
else:
oldest = self.oa_freq_deque[0]
span = time.time() - oldest
if span > self.AR_FREQ_DECAY: self.oa_freq_deque.popleft()
if span <= 0: return 0
hz = n / span
return hz
dq_len = len(self.oa_freq_deque)
delta_sum = 0
for i in range(1,dq_len):
delta_sum += self.oa_freq_deque[i]-self.oa_freq_deque[i-1]
delta_sum += time.time() - self.oa_freq_deque[dq_len-1]
if delta_sum == 0:
avg = 0
else:
avg = 1/(delta_sum/(dq_len))
def incoming_pr_frequency(self):
n = len(self.ip_freq_deque)
if not n > self.IC_DEQUE_MIN_SAMPLE: return 0
else:
oldest = self.ip_freq_deque[0]
span = time.time() - oldest
if span > self.PR_FREQ_DECAY: self.ip_freq_deque.popleft()
if span <= 0: return 0
hz = n / span
return hz
def outgoing_pr_frequency(self, preemptive=False):
n = len(self.op_freq_deque)+(1 if preemptive else 0)
if not len(self.op_freq_deque) > 1: return 0
else:
oldest = self.op_freq_deque[0]
span = time.time() - oldest
if span > self.PR_FREQ_DECAY: self.op_freq_deque.popleft()
if span <= 0: return 0
hz = n / span
return hz
return avg
def process_announce_queue(self):
if not self.announce_queue: return
if not hasattr(self, "announce_cap"):
self.announce_cap = RNS.Reticulum.ANNOUNCE_CAP/100.0
self.announce_cap = RNS.Reticulum.ANNOUNCE_CAP
if self.announce_queue:
if hasattr(self, "announce_queue"):
try:
now = time.time()
stale = []
@ -478,13 +218,12 @@ class Interface:
selected = entries[0]
now = time.time()
tx_size = len(selected["raw"])
tx_time = (tx_size*8) / self.bitrate
tx_time = (len(selected["raw"])*8) / self.bitrate
wait_time = (tx_time / self.announce_cap)
self.announce_allowed_at = now + wait_time
RNS.Transport.transmit(self, selected["raw"])
self.sent_announce(tx_size)
self.processOutgoing(selected["raw"])
self.sent_announce()
if selected in self.announce_queue:
self.announce_queue.remove(selected)
@ -498,20 +237,5 @@ class Interface:
RNS.log("Error while processing announce queue on "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
RNS.log("The announce queue for this interface has been cleared.", RNS.LOG_ERROR)
def final_init(self):
pass
def detach(self):
pass
def teardown(self):
pass
@staticmethod
def get_config_obj(config_in):
if type(config_in) == ConfigObj: return config_in
else:
try: return ConfigObj(config_in)
except Exception as e:
RNS.log(f"Could not parse supplied configuration data. The contained exception was: {e}", RNS.LOG_ERROR)
raise SystemError("Invalid configuration data supplied")
pass

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -28,7 +20,7 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from RNS.Interfaces.Interface import Interface
from .Interface import Interface
from time import sleep
import sys
import threading
@ -60,7 +52,6 @@ class KISS():
class KISSInterface(Interface):
MAX_CHUNK = 32768
BITRATE_GUESS = 1200
DEFAULT_IFAC_SIZE = 8
owner = None
port = None
@ -70,8 +61,8 @@ class KISSInterface(Interface):
stopbits = None
serial = None
def __init__(self, owner, configuration):
import importlib.util
def __init__(self, owner, name, port, speed, databits, parity, stopbits, preamble, txtail, persistence, slottime, flow_control, beacon_interval, beacon_data):
import importlib
if importlib.util.find_spec('serial') != None:
import serial
else:
@ -80,24 +71,6 @@ class KISSInterface(Interface):
RNS.panic()
super().__init__()
c = Interface.get_config_obj(configuration)
name = c["name"]
preamble = int(c["preamble"]) if "preamble" in c else None
txtail = int(c["txtail"]) if "txtail" in c else None
persistence = int(c["persistence"]) if "persistence" in c else None
slottime = int(c["slottime"]) if "slottime" in c else None
flow_control = c.as_bool("flow_control") if "flow_control" in c else False
port = c["port"] if "port" in c else None
speed = int(c["speed"]) if "speed" in c else 9600
databits = int(c["databits"]) if "databits" in c else 8
parity = c["parity"] if "parity" in c else "N"
stopbits = int(c["stopbits"]) if "stopbits" in c else 1
beacon_interval = int(c["id_interval"]) if "id_interval" in c else None
beacon_data = c["id_callsign"] if "id_callsign" in c else None
if port == None:
raise ValueError("No port specified for serial interface")
self.HW_MTU = 564
@ -122,9 +95,7 @@ class KISSInterface(Interface):
self.packet_queue = []
self.flow_control = flow_control
self.shared_medium = True
self.interface_ready = False
self.reconnecting = False
self.flow_control_timeout = 5
self.flow_control_locked = time.time()
@ -141,17 +112,15 @@ class KISSInterface(Interface):
try:
self.open_port()
if self.serial.is_open: self.configure_device()
else: raise IOError("Could not open serial port")
except Exception as e:
RNS.log(f"Could not open serial port for {self}", RNS.LOG_ERROR)
RNS.log(f"The contained exception was: {e}", RNS.LOG_ERROR)
RNS.log(f"Reticulum will attempt to bring up this interface periodically", RNS.LOG_ERROR)
if not self.detached and not self.reconnecting:
thread = threading.Thread(target=self.reconnect_port)
thread.daemon = True
thread.start()
RNS.log("Could not open serial port "+self.port, RNS.LOG_ERROR)
raise e
if self.serial.is_open:
self.configure_device()
else:
raise IOError("Could not open serial port")
def open_port(self):
RNS.log("Opening serial port "+self.port+"...", RNS.LOG_VERBOSE)
@ -169,11 +138,6 @@ class KISSInterface(Interface):
dsrdtr = False,
)
def detach(self):
self.detached = True
self.online = False
try: self.serial.close()
except Exception as e: RNS.log(f"Could not close port for {self} at detach: {e}", RNS.LOG_ERROR)
def configure_device(self):
# Allow time for interface to initialise before config
@ -253,12 +217,12 @@ class KISSInterface(Interface):
raise IOError("Could not enable KISS interface flow control")
def process_incoming(self, data):
if not data: return
def processIncoming(self, data):
self.rxb += len(data)
self.owner.inbound(data, self)
def process_outgoing(self,data):
def processOutgoing(self,data):
datalen = len(data)
if self.online:
if self.interface_ready:
@ -292,7 +256,7 @@ class KISSInterface(Interface):
if len(self.packet_queue) > 0:
data = self.packet_queue.pop(0)
self.interface_ready = True
self.process_outgoing(data)
self.processOutgoing(data)
elif len(self.packet_queue) == 0:
self.interface_ready = True
@ -311,7 +275,7 @@ class KISSInterface(Interface):
if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
in_frame = False
self.process_incoming(data_buffer)
self.processIncoming(data_buffer)
elif (byte == KISS.FEND):
in_frame = True
command = KISS.CMD_UNKNOWN
@ -355,41 +319,37 @@ class KISSInterface(Interface):
if time.time() > self.first_tx + self.beacon_i:
RNS.log("Interface "+str(self)+" is transmitting beacon data: "+str(self.beacon_d.decode("utf-8")), RNS.LOG_DEBUG)
self.first_tx = None
# Pad to minimum length
frame = bytearray(self.beacon_d)
while len(frame) < 15:
frame.append(0x00)
self.process_outgoing(bytes(frame))
self.processOutgoing(self.beacon_d)
except Exception as e:
self.online = False
RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
if RNS.Reticulum.panic_on_interface_error: RNS.panic()
if RNS.Reticulum.panic_on_interface_error:
RNS.panic()
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
self.online = False
try: self.serial.close()
except Exception as e: RNS.log(f"Could not close port for {self}", RNS.LOG_ERROR)
if not self.detached and not self.reconnecting: self.reconnect_port()
self.serial.close()
self.reconnect_port()
def reconnect_port(self):
self.reconnecting = True
while not self.online and not self.detached:
while not self.online:
try:
time.sleep(5)
RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
self.open_port()
if self.serial.is_open: self.configure_device()
except Exception as e: RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
if self.serial.is_open:
self.configure_device()
except Exception as e:
RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
self.reconnecting = False
if self.online: RNS.log("Reconnected serial port for "+str(self))
RNS.log("Reconnected serial port for "+str(self))
def should_ingress_limit(self): return False
def should_ingress_limit(self):
return False
def __str__(self): return "KISSInterface["+self.name+"]"
def __str__(self):
return "KISSInterface["+self.name+"]"

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2025 Mark Qvist
# Copyright (c) 2016-2023 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -28,10 +20,7 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from RNS.Interfaces.Interface import Interface
from RNS.Interfaces.BackboneInterface import BackboneInterface
from RNS.Interfaces.util.TransmitBuffer import TransmitBuffer
from RNS.Interfaces.util.HDLC import HDLC, ReceiveBuffer
from .Interface import Interface
import socketserver
import threading
import socket
@ -41,6 +30,17 @@ import os
import RNS
from threading import Lock
class HDLC():
FLAG = 0x7E
ESC = 0x7D
ESC_MASK = 0x20
@staticmethod
def escape(data):
data = data.replace(bytes([HDLC.ESC]), bytes([HDLC.ESC, HDLC.ESC^HDLC.ESC_MASK]))
data = data.replace(bytes([HDLC.FLAG]), bytes([HDLC.ESC, HDLC.FLAG^HDLC.ESC_MASK]))
return data
class ThreadingTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
def server_bind(self):
if RNS.vendor.platformutils.is_windows():
@ -52,21 +52,16 @@ class ThreadingTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
class LocalClientInterface(Interface):
RECONNECT_WAIT = 8
AUTOCONFIGURE_MTU = True
CLIENT_SLEEP_PAUSE_TIMEOUT = 12
def __init__(self, owner, name, target_port = None, connected_socket=None, socket_path=None):
def __init__(self, owner, name, target_port = None, connected_socket=None):
super().__init__()
self.owner = owner
self.bitrate = 1_000_000_000
self.epoll_backend = False
self.HW_MTU = 262144
self.online = False
self.ifac_size = self.DEFAULT_IFAC_SIZE
# TODO: Remove at some point
# self.rxptime = 0
if socket_path != None and RNS.Reticulum.get_instance().use_af_unix: self.socket_path = f"\0rns/{socket_path}"
else: self.socket_path = None
self.HW_MTU = 1064
self.online = False
self.IN = True
self.OUT = False
@ -77,41 +72,24 @@ class LocalClientInterface(Interface):
self.detached = False
self.name = name
self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
self.transmit_buffer = TransmitBuffer()
self.receive_buffer = ReceiveBuffer(mtu=lambda: self.HW_MTU, min_frame_len=RNS.Reticulum.HEADER_MINSIZE,
max_frame_len=lambda: self.HW_MTU, on_frame=self.process_incoming)
if RNS.vendor.platformutils.use_epoll(): self.epoll_backend = True
self.pause_on_client_sleep = False
if connected_socket != None:
self.receives = True
self.target_ip = None
self.target_port = None
self.socket = connected_socket
if self.socket.family == socket.AF_INET:
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
self.is_connected_to_shared_instance = False
if RNS.vendor.platformutils.is_android():
self.pause_on_client_sleep = True
self.pause_timeout = time.time() + self.CLIENT_SLEEP_PAUSE_TIMEOUT
elif self.socket_path != None:
self.receives = True
self.target_ip = None
self.target_port = None
self.connect()
elif target_port != None:
self.receives = True
self.target_ip = "127.0.0.1"
self.target_port = target_port
self.connect()
self.owner = owner
self.bitrate = 1000*1000*1000
self.online = True
self.writing = False
@ -122,31 +100,22 @@ class LocalClientInterface(Interface):
self.announce_rate_penalty = None
if connected_socket == None:
if not self.epoll_backend:
thread = threading.Thread(target=self.read_loop)
thread.daemon = True
thread.start()
thread = threading.Thread(target=self.read_loop)
thread.daemon = True
thread.start()
def should_ingress_limit(self):
return False
def connect(self):
if self.socket_path != None:
self.socket = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
self.socket.connect(self.socket_path)
else:
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
self.socket.connect((self.target_ip, self.target_port))
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
self.socket.connect((self.target_ip, self.target_port))
self.online = True
self.is_connected_to_shared_instance = True
self.never_connected = False
if RNS.vendor.platformutils.is_android(): self.phy_keepalive = True
if self.epoll_backend: BackboneInterface.add_client_socket(self.socket, self)
return True
@ -170,11 +139,9 @@ class LocalClientInterface(Interface):
RNS.log("Reconnected socket for "+str(self)+".", RNS.LOG_INFO)
self.reconnecting = False
if not self.epoll_backend:
thread = threading.Thread(target=self.read_loop)
thread.daemon = True
thread.start()
thread = threading.Thread(target=self.read_loop)
thread.daemon = True
thread.start()
def job():
time.sleep(LocalClientInterface.RECONNECT_WAIT+2)
RNS.Transport.shared_connection_reappeared()
@ -185,112 +152,89 @@ class LocalClientInterface(Interface):
raise IOError("Attempt to reconnect on a non-initiator local interface")
def send_keepalive(self):
if self.online:
RNS.log(f"Sending keepalive on {self}", RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
try:
if self.epoll_backend:
self.transmit_buffer.append(bytes([HDLC.FLAG])+bytes([HDLC.FLAG]))
BackboneInterface.tx_ready(self)
def processIncoming(self, data):
self.rxb += len(data)
if hasattr(self, "parent_interface") and self.parent_interface != None:
self.parent_interface.rxb += len(data)
# TODO: Remove at some point
# processing_start = time.time()
self.owner.inbound(data, self)
else:
self.writing = True
data = bytes([HDLC.FLAG])+bytes([HDLC.FLAG])
self.socket.sendall(data)
self.writing = False
except Exception as e: RNS.log(f"Exception occurred while sending keepalive on {self}: {e}", RNS.LOG_ERROR)
def process_incoming(self, data):
try: self.owner.inbound(data, self)
except Exception as e:
RNS.log(f"An error occurred in the processing of an incoming frame for {self}: {e}", RNS.LOG_ERROR)
RNS.trace_exception(e)
def process_outgoing(self, data):
if self.pause_on_client_sleep and time.time() > self.pause_timeout:
RNS.log(f"TX paused for LocalInterface client, dropping outbound packet", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None # TODO: Remove
return
# TODO: Remove at some point
# duration = time.time() - processing_start
# self.rxptime += duration
def processOutgoing(self, data):
if self.online:
try:
if self.epoll_backend:
frame = HDLC.frame(data)
if self.tx_stalled or not self.transmit_buffer.append(frame, self.tx_hwm):
self.tx_drops += 1
self.tx_dropped_bytes += len(frame)
RNS.log(f"Egress control dropping outbound frame of {RNS.prettysize(len(frame))} on {self}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
else: BackboneInterface.tx_ready(self)
self.writing = True
else:
self.writing = True
if self._force_bitrate:
if not hasattr(self, "send_lock"):
self.send_lock = Lock()
if self._force_bitrate:
if not hasattr(self, "send_lock"): self.send_lock = Lock()
with self.send_lock:
# RNS.log(f"Simulating latency of {RNS.prettytime(s)} for {len(data)} bytes", RNS.LOG_EXTREME)
s = len(data) / self.bitrate * 8
time.sleep(s)
with self.send_lock:
s = len(data) / self.bitrate * 8
RNS.log(f"Simulating latency of {RNS.prettytime(s)} for {len(data)} bytes")
time.sleep(s)
data = HDLC.frame(data)
self.socket.sendall(data)
self.writing = False
self.txb += len(data)
if hasattr(self, "parent_interface") and self.parent_interface != None:
self.parent_interface.txb += len(data)
data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
self.socket.sendall(data)
self.writing = False
self.txb += len(data)
if hasattr(self, "parent_interface") and self.parent_interface != None:
self.parent_interface.txb += len(data)
except Exception as e:
RNS.log("Exception occurred while transmitting via "+str(self)+", tearing down interface", RNS.LOG_ERROR)
RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
RNS.trace_exception(e)
self.teardown()
def handle_hdlc(self, data_in):
self.receive_buffer.feed(data_in)
def receive(self, data_in):
try:
if len(data_in) > 0: self.handle_hdlc(data_in)
else:
self.online = False
if self.is_connected_to_shared_instance and not self.detached:
RNS.log("Socket for "+str(self)+" was closed, attempting to reconnect...", RNS.LOG_WARNING)
RNS.Transport.shared_connection_disappeared()
# TODO: Potentially run this in a thread, but since if we get here,
# there's no other connectivity left to block anyway, it might be
# unnecessary.
self.reconnect()
else: self.teardown(nowarning=True)
except Exception as e:
self.online = False
RNS.log("An interface error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
RNS.log("Tearing down "+str(self), RNS.LOG_ERROR)
self.teardown()
if self.pause_on_client_sleep: self.pause_timeout = time.time() + self.CLIENT_SLEEP_PAUSE_TIMEOUT
def read_loop(self):
try:
self.receive_buffer.reset()
data_in = b""
in_frame = False
escape = False
data_buffer = b""
while True:
data_in = self.socket.recv(4096)
if len(data_in) > 0: self.handle_hdlc(data_in)
if len(data_in) > 0:
pointer = 0
while pointer < len(data_in):
byte = data_in[pointer]
pointer += 1
if (in_frame and byte == HDLC.FLAG):
in_frame = False
self.processIncoming(data_buffer)
elif (byte == HDLC.FLAG):
in_frame = True
data_buffer = b""
elif (in_frame and len(data_buffer) < self.HW_MTU):
if (byte == HDLC.ESC):
escape = True
else:
if (escape):
if (byte == HDLC.FLAG ^ HDLC.ESC_MASK):
byte = HDLC.FLAG
if (byte == HDLC.ESC ^ HDLC.ESC_MASK):
byte = HDLC.ESC
escape = False
data_buffer = data_buffer+bytes([byte])
else:
self.online = False
if self.is_connected_to_shared_instance and not self.detached:
RNS.log("Socket for "+str(self)+" was closed, attempting to reconnect...", RNS.LOG_WARNING)
RNS.Transport.shared_connection_disappeared()
# TODO: Potentially run this in a thread, but since if we get here,
# there's no other connectivity left to block anyway, it might be
# unnecessary.
self.reconnect()
else:
self.teardown(nowarning=True)
break
except Exception as e:
self.online = False
RNS.log("An interface error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
@ -305,14 +249,12 @@ class LocalClientInterface(Interface):
self.detached = True
try:
if self.socket != None:
self.socket.shutdown(socket.SHUT_RDWR)
self.socket.shutdown(socket.SHUT_RDWR)
except Exception as e:
RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
try:
if self.socket != None:
self.socket.close()
self.socket.close()
except Exception as e:
RNS.log("Error while closing socket for "+str(self)+": "+str(e))
@ -323,15 +265,15 @@ class LocalClientInterface(Interface):
self.OUT = False
self.IN = False
RNS.Transport.remove_interface(self)
if self in RNS.Transport.interfaces:
RNS.Transport.interfaces.remove(self)
if self in RNS.Transport.local_client_interfaces:
RNS.Transport.local_client_interfaces.remove(self)
if hasattr(self, "parent_interface") and self.parent_interface != None:
self.parent_interface.clients -= 1
if hasattr(RNS.Transport, "owner") and RNS.Transport.owner != None:
background = not self.detached
RNS.Transport.owner._should_persist_data(background=background)
RNS.Transport.owner._should_persist_data()
if nowarning == False:
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is being torn down. Restart Reticulum to attempt to open this interface again.", RNS.LOG_ERROR)
@ -346,146 +288,84 @@ class LocalClientInterface(Interface):
def __str__(self):
if self.socket_path: return "LocalInterface["+str(self.socket_path.replace("\0", ""))+"]"
else: return "LocalInterface["+str(self.target_port)+"]"
return "LocalInterface["+str(self.target_port)+"]"
class LocalServerInterface(Interface):
AUTOCONFIGURE_MTU = True
def __init__(self, owner, bindport=None, socket_path=None):
def __init__(self, owner, bindport=None):
super().__init__()
self.epoll_backend = False
self.online = False
self.clients = 0
self.ifac_size = self.DEFAULT_IFAC_SIZE
if socket_path != None and RNS.Reticulum.get_instance().use_af_unix: self.socket_path = f"\0rns/{socket_path}"
else: self.socket_path = None
self.IN = True
self.OUT = False
self.name = "Reticulum"
self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
if RNS.vendor.platformutils.use_epoll():
self.epoll_backend = True
if socket_path != None and self.epoll_backend:
self.receives = True
self.bind_ip = None
self.bind_port = None
self.owner = owner
self.is_local_shared_instance = True
BackboneInterface.add_listener(self, self.socket_path, socket_type=socket.AF_UNIX)
elif bindport != None:
if (bindport != None):
self.receives = True
self.bind_ip = "127.0.0.1"
self.bind_port = bindport
def handlerFactory(callback):
def createHandler(*args, **keys):
return LocalInterfaceHandler(callback, *args, **keys)
return createHandler
self.owner = owner
self.is_local_shared_instance = True
address = (self.bind_ip, self.bind_port)
if self.epoll_backend: BackboneInterface.add_listener(self, address)
else:
def handlerFactory(callback):
def createHandler(*args, **keys):
return LocalInterfaceHandler(callback, *args, **keys)
return createHandler
self.server = ThreadingTCPServer(address, handlerFactory(self.incoming_connection))
self.server.daemon_threads = True
thread = threading.Thread(target=self.server.serve_forever)
thread.daemon = True
thread.start()
self.server = ThreadingTCPServer(address, handlerFactory(self.incoming_connection))
thread = threading.Thread(target=self.server.serve_forever)
thread.daemon = True
thread.start()
self.announce_rate_target = None
self.announce_rate_grace = None
self.announce_rate_penalty = None
self.bitrate = 1000*1000*1000
self.online = True
self.announce_rate_target = None
self.announce_rate_grace = None
self.announce_rate_penalty = None
self.HW_MTU = 262144
self.bitrate = 1000*1000*1000
self.online = True
def incoming_connection(self, handler):
if self.epoll_backend:
client_socket = handler
if client_socket.family == socket.AF_INET:
interface_name = str(str(client_socket.getpeername()[1]))
elif client_socket.family == socket.AF_UNIX:
interface_name = f"{self.clients}@{self.socket_path}"
interface_name = str(str(handler.client_address[1]))
spawned_interface = LocalClientInterface(self.owner, name=interface_name, connected_socket=handler.request)
spawned_interface.OUT = self.OUT
spawned_interface.IN = self.IN
spawned_interface.target_ip = handler.client_address[0]
spawned_interface.target_port = str(handler.client_address[1])
spawned_interface.parent_interface = self
spawned_interface.bitrate = self.bitrate
if hasattr(self, "_force_bitrate"):
spawned_interface._force_bitrate = self._force_bitrate
# RNS.log("Accepting new connection to shared instance: "+str(spawned_interface), RNS.LOG_EXTREME)
RNS.Transport.interfaces.append(spawned_interface)
RNS.Transport.local_client_interfaces.append(spawned_interface)
self.clients += 1
spawned_interface.read_loop()
spawned_interface = LocalClientInterface(self.owner, name=interface_name, connected_socket=client_socket)
spawned_interface.OUT = self.OUT
spawned_interface.IN = self.IN
spawned_interface.socket = client_socket
spawned_interface.parent_interface = self
spawned_interface.bitrate = self.bitrate
if client_socket.family == socket.AF_INET:
spawned_interface.target_ip = client_socket.getpeername()[0]
spawned_interface.target_port = str(client_socket.getpeername()[1])
elif client_socket.family == socket.AF_UNIX:
spawned_interface.target_ip = None
spawned_interface.target_port = interface_name
spawned_interface.socket_path = self.socket_path
if hasattr(self, "_force_bitrate"): spawned_interface._force_bitrate = self._force_bitrate
RNS.Transport.add_interface(spawned_interface)
RNS.Transport.local_client_interfaces.append(spawned_interface)
BackboneInterface.add_client_socket(client_socket, spawned_interface)
self.clients += 1
return True
else:
interface_name = str(str(handler.client_address[1]))
spawned_interface = LocalClientInterface(self.owner, name=interface_name, connected_socket=handler.request)
spawned_interface.OUT = self.OUT
spawned_interface.IN = self.IN
spawned_interface.target_ip = handler.client_address[0]
spawned_interface.target_port = str(handler.client_address[1])
spawned_interface.parent_interface = self
spawned_interface.bitrate = self.bitrate
if hasattr(self, "_force_bitrate"): spawned_interface._force_bitrate = self._force_bitrate
RNS.Transport.add_interface(spawned_interface)
RNS.Transport.local_client_interfaces.append(spawned_interface)
self.clients += 1
spawned_interface.read_loop()
def process_outgoing(self, data):
def processOutgoing(self, data):
pass
def received_announce(self, size=0, from_spawned=False):
if from_spawned:
self.ia_freq_deque.append(time.time())
self.arxb += size
def received_announce(self, from_spawned=False):
if from_spawned: self.ia_freq_deque.append(time.time())
def sent_announce(self, size=0, from_spawned=False):
if from_spawned:
self.oa_freq_deque.append(time.time())
self.atxb += size
def received_path_request(self, size=0, from_spawned=False):
if from_spawned:
self.ip_freq_deque.append(time.time())
self.prxb += size
def sent_path_request(self, size=0, from_spawned=False):
if from_spawned:
self.op_freq_deque.append(time.time())
self.ptxb += size
def sent_announce(self, from_spawned=False):
if from_spawned: self.oa_freq_deque.append(time.time())
def __str__(self):
if self.socket_path: return "Shared Instance["+str(self.socket_path.replace("\0", ""))+"]"
else: return "Shared Instance["+str(self.bind_port)+"]"
return "Shared Instance["+str(self.bind_port)+"]"
class LocalInterfaceHandler(socketserver.BaseRequestHandler):
def __init__(self, callback, *args, **keys):
self.callback = callback
socketserver.BaseRequestHandler.__init__(self, *args, **keys)
def handle(self): self.callback(handler=self)
def handle(self):
self.callback(handler=self)

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2025 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -28,7 +20,7 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from RNS.Interfaces.Interface import Interface
from .Interface import Interface
from time import sleep
import sys
import threading
@ -54,25 +46,16 @@ class HDLC():
class PipeInterface(Interface):
MAX_CHUNK = 32768
BITRATE_GUESS = 1*1000*1000
DEFAULT_IFAC_SIZE = 8
owner = None
command = None
def __init__(self, owner, configuration):
super().__init__()
c = Interface.get_config_obj(configuration)
name = c["name"]
command = c["command"] if "command" in c else None
respawn_delay = c.as_float("respawn_delay") if "respawn_delay" in c else None
if command == None:
raise ValueError("No command specified for PipeInterface")
def __init__(self, owner, name, command, respawn_delay):
if respawn_delay == None:
respawn_delay = 5
super().__init__()
self.HW_MTU = 1064
self.owner = owner
@ -84,7 +67,6 @@ class PipeInterface(Interface):
self.pipe_is_open = False
self.bitrate = PipeInterface.BITRATE_GUESS
self.respawn_delay = respawn_delay
self.shared_medium = True
try:
self.open_pipe()
@ -119,12 +101,12 @@ class PipeInterface(Interface):
RNS.log("Subprocess pipe for "+str(self)+" is now connected", RNS.LOG_VERBOSE)
def process_incoming(self, data):
if not data: return
def processIncoming(self, data):
self.rxb += len(data)
self.owner.inbound(data, self)
def process_outgoing(self,data):
def processOutgoing(self,data):
if self.online:
data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
written = self.process.stdin.write(data)
@ -152,7 +134,7 @@ class PipeInterface(Interface):
if (in_frame and byte == HDLC.FLAG):
in_frame = False
self.process_incoming(data_buffer)
self.processIncoming(data_buffer)
elif (byte == HDLC.FLAG):
in_frame = True
data_buffer = b""

File diff suppressed because it is too large Load diff

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2024 Jacob Eva. Adapted from the RNodeInterface by Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -28,7 +20,7 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from RNS.Interfaces.Interface import Interface
from .Interface import Interface
from time import sleep
import sys
import threading
@ -41,8 +33,6 @@ class KISS():
FESC = 0xDB
TFEND = 0xDC
TFESC = 0xDD
CMD_DATA = 0x00
CMD_UNKNOWN = 0xFE
CMD_FREQUENCY = 0x01
@ -74,7 +64,7 @@ class KISS():
CMD_FW_VERSION = 0x50
CMD_ROM_READ = 0x51
CMD_RESET = 0x55
CMD_INTERFACES = 0x71
CMD_INTERFACES = 0x64
CMD_INT0_DATA = 0x00
CMD_INT1_DATA = 0x10
@ -89,6 +79,19 @@ class KISS():
CMD_INT10_DATA = 0xE0
CMD_INT11_DATA = 0xF0
CMD_SEL_INT0 = 0x1E
CMD_SEL_INT1 = 0x1F
CMD_SEL_INT2 = 0x2F
CMD_SEL_INT3 = 0x74
CMD_SEL_INT4 = 0x7F
CMD_SEL_INT5 = 0x9F
CMD_SEL_INT6 = 0xAF
CMD_SEL_INT7 = 0xBF
CMD_SEL_INT8 = 0xCF
CMD_SEL_INT9 = 0xDF
CMD_SEL_INT10 = 0xEF
CMD_SEL_INT11 = 0xFF
DETECT_REQ = 0x73
DETECT_RESP = 0x46
@ -113,7 +116,33 @@ class KISS():
SX128X = 0x20
SX1280 = 0x21
CMD_SEL_INT = 0x1F
def int_data_cmd_to_index(int_data_cmd):
if int_data_cmd == KISS.CMD_INT0_DATA:
return 0
elif int_data_cmd == KISS.CMD_INT1_DATA:
return 1
elif int_data_cmd == KISS.CMD_INT2_DATA:
return 2
elif int_data_cmd == KISS.CMD_INT3_DATA:
return 3
elif int_data_cmd == KISS.CMD_INT4_DATA:
return 4
elif int_data_cmd == KISS.CMD_INT5_DATA:
return 5
elif int_data_cmd == KISS.CMD_INT6_DATA:
return 6
elif int_data_cmd == KISS.CMD_INT7_DATA:
return 7
elif int_data_cmd == KISS.CMD_INT8_DATA:
return 8
elif int_data_cmd == KISS.CMD_INT9_DATA:
return 9
elif int_data_cmd == KISS.CMD_INT10_DATA:
return 10
elif int_data_cmd == KISS.CMD_INT11_DATA:
return 11
else:
return 0
def interface_type_to_str(interface_type):
if interface_type == KISS.SX126X or interface_type == KISS.SX1262:
@ -134,7 +163,6 @@ class KISS():
class RNodeMultiInterface(Interface):
MAX_CHUNK = 32768
DEFAULT_IFAC_SIZE = 8
CALLSIGN_MAX_LEN = 32
@ -145,11 +173,11 @@ class RNodeMultiInterface(Interface):
MAX_SUBINTERFACES = 11
def __init__(self, owner, configuration):
def __init__(self, owner, name, port, subint_config, id_interval = None, id_callsign = None):
if RNS.vendor.platformutils.is_android():
raise SystemError("Invalid interface type. The Android-specific RNode interface must be used on Android")
import importlib.util
import importlib
if importlib.util.find_spec('serial') != None:
import serial
else:
@ -159,77 +187,6 @@ class RNodeMultiInterface(Interface):
super().__init__()
c = Interface.get_config_obj(configuration)
name = c["name"]
count = 0
enabled_count = 0
# Count how many interfaces are in the file
for subinterface in c:
# if the retrieved entry is not a string, it must be a dictionary, which is what we want
if isinstance(c[subinterface], dict):
count += 1
# Count how many interfaces are enabled to allow for appropriate matrix sizing
for subinterface in c:
if isinstance(c[subinterface], dict):
subinterface_config = c[subinterface]
if (("interface_enabled" in subinterface_config) and subinterface_config.as_bool("interface_enabled") == True) or (("enabled" in c) and c.as_bool("enabled") == True):
enabled_count += 1
# Create an array with a row for each subinterface
subint_config = [[None for x in range(11)] for y in range(enabled_count)]
subint_index = 0
for subinterface in c:
if isinstance(c[subinterface], dict):
subinterface_config = c[subinterface]
if (("interface_enabled" in subinterface_config) and subinterface_config.as_bool("interface_enabled") == True) or (("enabled" in c) and c.as_bool("enabled") == True):
subint_vport = subinterface_config["vport"] if "vport" in subinterface_config else None
subint_config[subint_index][0] = subinterface
subint_config[subint_index][1] = subint_vport
frequency = int(subinterface_config["frequency"]) if "frequency" in subinterface_config else None
subint_config[subint_index][2] = frequency
bandwidth = int(subinterface_config["bandwidth"]) if "bandwidth" in subinterface_config else None
subint_config[subint_index][3] = bandwidth
txpower = int(subinterface_config["txpower"]) if "txpower" in subinterface_config else None
subint_config[subint_index][4] = txpower
spreadingfactor = int(subinterface_config["spreadingfactor"]) if "spreadingfactor" in subinterface_config else None
subint_config[subint_index][5] = spreadingfactor
codingrate = int(subinterface_config["codingrate"]) if "codingrate" in subinterface_config else None
subint_config[subint_index][6] = codingrate
flow_control = subinterface_config.as_bool("flow_control") if "flow_control" in subinterface_config else False
subint_config[subint_index][7] = flow_control
st_alock = float(subinterface_config["airtime_limit_short"]) if "airtime_limit_short" in subinterface_config else None
subint_config[subint_index][8] = st_alock
lt_alock = float(subinterface_config["airtime_limit_long"]) if "airtime_limit_long" in subinterface_config else None
subint_config[subint_index][9] = lt_alock
if "outgoing" in subinterface_config and subinterface_config.as_bool("outgoing") == False:
subint_config[subint_index][10] = False
else:
subint_config[subint_index][10] = True
subint_index += 1
# if no subinterfaces are defined
if count == 0:
raise ValueError("No subinterfaces configured for "+name)
# if no subinterfaces are enabled
elif enabled_count == 0:
raise ValueError("No subinterfaces enabled for "+name)
id_interval = int(c["id_interval"]) if "id_interval" in c else None
id_callsign = c["id_callsign"] if "id_callsign" in c else None
port = c["port"] if "port" in c else None
if port == None:
raise ValueError("No port specified for "+name)
self.HW_MTU = 508
self.clients = 0
@ -269,7 +226,6 @@ class RNodeMultiInterface(Interface):
self.r_stat_tx = None
self.r_stat_rssi = None
self.r_stat_snr = None
self.r_stat_q = None
self.r_st_alock = None
self.r_lt_alock = None
self.r_random = None
@ -277,8 +233,6 @@ class RNodeMultiInterface(Interface):
self.packet_queue = []
self.interface_ready = False
self.announce_rate_target = None
self.reports_phy_stats = True
self.r_modulation = "LoRa"
self.validcfg = True
if id_interval != None and id_callsign != None:
@ -378,7 +332,7 @@ class RNodeMultiInterface(Interface):
interface.mode = self.mode
interface.HW_MTU = self.HW_MTU
interface.detected = True
RNS.Transport.add_interface(interface)
RNS.Transport.interfaces.append(interface)
RNS.log("Spawned new RNode subinterface: "+str(interface), RNS.LOG_VERBOSE)
self.clients += 1
@ -450,10 +404,11 @@ class RNodeMultiInterface(Interface):
c4 = frequency & 0xFF
data = KISS.escape(bytes([c1])+bytes([c2])+bytes([c3])+bytes([c4]))
kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_FREQUENCY])+data+bytes([KISS.FEND])
kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_FREQUENCY])+data+bytes([KISS.FEND])
written = self.serial.write(kiss_command)
if written != len(kiss_command):
raise IOError("An IO error occurred while configuring frequency for "+str(self))
self.selected_index = interface.index
def setBandwidth(self, bandwidth, interface):
c1 = bandwidth >> 24
@ -462,31 +417,35 @@ class RNodeMultiInterface(Interface):
c4 = bandwidth & 0xFF
data = KISS.escape(bytes([c1])+bytes([c2])+bytes([c3])+bytes([c4]))
kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_BANDWIDTH])+data+bytes([KISS.FEND])
kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_BANDWIDTH])+data+bytes([KISS.FEND])
written = self.serial.write(kiss_command)
if written != len(kiss_command):
raise IOError("An IO error occurred while configuring bandwidth for "+str(self))
self.selected_index = interface.index
def setTXPower(self, txpower, interface):
txp = txpower.to_bytes(1, byteorder="big", signed=True)
kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_TXPOWER])+txp+bytes([KISS.FEND])
kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_TXPOWER])+txp+bytes([KISS.FEND])
written = self.serial.write(kiss_command)
if written != len(kiss_command):
raise IOError("An IO error occurred while configuring TX power for "+str(self))
self.selected_index = interface.index
def setSpreadingFactor(self, sf, interface):
sf = bytes([sf])
kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_SF])+sf+bytes([KISS.FEND])
kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_SF])+sf+bytes([KISS.FEND])
written = self.serial.write(kiss_command)
if written != len(kiss_command):
raise IOError("An IO error occurred while configuring spreading factor for "+str(self))
self.selected_index = interface.index
def setCodingRate(self, cr, interface):
cr = bytes([cr])
kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_CR])+cr+bytes([KISS.FEND])
kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_CR])+cr+bytes([KISS.FEND])
written = self.serial.write(kiss_command)
if written != len(kiss_command):
raise IOError("An IO error occurred while configuring coding rate for "+str(self))
self.selected_index = interface.index
def setSTALock(self, st_alock, interface):
if st_alock != None:
@ -495,10 +454,11 @@ class RNodeMultiInterface(Interface):
c2 = at & 0xFF
data = KISS.escape(bytes([c1])+bytes([c2]))
kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_ST_ALOCK])+data+bytes([KISS.FEND])
kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_ST_ALOCK])+data+bytes([KISS.FEND])
written = self.serial.write(kiss_command)
if written != len(kiss_command):
raise IOError("An IO error occurred while configuring short-term airtime limit for "+str(self))
self.selected_index = interface.index
def setLTALock(self, lt_alock, interface):
if lt_alock != None:
@ -507,17 +467,19 @@ class RNodeMultiInterface(Interface):
c2 = at & 0xFF
data = KISS.escape(bytes([c1])+bytes([c2]))
kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_LT_ALOCK])+data+bytes([KISS.FEND])
kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_LT_ALOCK])+data+bytes([KISS.FEND])
written = self.serial.write(kiss_command)
if written != len(kiss_command):
raise IOError("An IO error occurred while configuring long-term airtime limit for "+str(self))
self.selected_index = interface.index
def setRadioState(self, state, interface):
#self.state = state
kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_RADIO_STATE])+bytes([state])+bytes([KISS.FEND])
kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_RADIO_STATE])+bytes([state])+bytes([KISS.FEND])
written = self.serial.write(kiss_command)
if written != len(kiss_command):
raise IOError("An IO error occurred while configuring radio state for "+str(self))
self.selected_index = interface.index
def validate_firmware(self):
if (self.maj_version >= RNodeMultiInterface.REQUIRED_FW_VER_MAJ):
@ -532,13 +494,13 @@ class RNodeMultiInterface(Interface):
RNS.log("Please update your RNode firmware with rnodeconf from https://github.com/markqvist/Reticulum/RNS/Utilities/rnodeconf.py")
RNS.panic()
def process_outgoing(self, data, interface = None):
def processOutgoing(self, data, interface = None):
if interface is None:
# do nothing if RNS tries to transmit on this interface directly
pass
else:
data = KISS.escape(data)
frame = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_DATA])+data+bytes([KISS.FEND])
frame = bytes([0xc0])+bytes([interface.data_cmd])+data+bytes([0xc0])
written = self.serial.write(frame)
self.txb += len(data)
@ -546,25 +508,11 @@ class RNodeMultiInterface(Interface):
if written != len(frame):
raise IOError("Serial interface only wrote "+str(written)+" bytes of "+str(len(data)))
def received_announce(self, size=0, from_spawned=False):
if from_spawned:
self.ia_freq_deque.append(time.time())
self.arxb += size
def received_announce(self, from_spawned=False):
if from_spawned: self.ia_freq_deque.append(time.time())
def sent_announce(self, size=0, from_spawned=False):
if from_spawned:
self.oa_freq_deque.append(time.time())
self.atxb += size
def received_path_request(self, size=0, from_spawned=False):
if from_spawned:
self.ip_freq_deque.append(time.time())
self.prxb += size
def sent_path_request(self, size=0, from_spawned=False):
if from_spawned:
self.op_freq_deque.append(time.time())
self.ptxb += size
def sent_announce(self, from_spawned=False):
if from_spawned: self.oa_freq_deque.append(time.time())
def readLoop(self):
try:
@ -581,9 +529,21 @@ class RNodeMultiInterface(Interface):
last_read_ms = int(time.time()*1000)
if (in_frame and byte == KISS.FEND and
(command == KISS.CMD_DATA)):
(command == KISS.CMD_INT0_DATA or
command == KISS.CMD_INT1_DATA or
command == KISS.CMD_INT2_DATA or
command == KISS.CMD_INT3_DATA or
command == KISS.CMD_INT4_DATA or
command == KISS.CMD_INT5_DATA or
command == KISS.CMD_INT6_DATA or
command == KISS.CMD_INT7_DATA or
command == KISS.CMD_INT8_DATA or
command == KISS.CMD_INT9_DATA or
command == KISS.CMD_INT10_DATA or
command == KISS.CMD_INT11_DATA)):
in_frame = False
self.subinterfaces[self.selected_index].process_incoming(data_buffer)
self.subinterfaces[KISS.int_data_cmd_to_index(command)].processIncoming(data_buffer)
self.selected_index = KISS.int_data_cmd_to_index(command)
data_buffer = b""
command_buffer = b""
elif (byte == KISS.FEND):
@ -648,9 +608,6 @@ class RNodeMultiInterface(Interface):
RNS.log(str(self.subinterfaces[self.selected_index])+" Radio reporting bandwidth is "+str(self.subinterfaces[self.selected_index].r_bandwidth/1000.0)+" KHz", RNS.LOG_DEBUG)
self.subinterfaces[self.selected_index].updateBitrate()
elif (command == KISS.CMD_SEL_INT):
self.selected_index = byte
elif (command == KISS.CMD_TXPOWER):
txp = byte - 256 if byte > 127 else byte
self.subinterfaces[self.selected_index].r_txpower = txp
@ -864,7 +821,7 @@ class RNodeMultiInterface(Interface):
for interface in self.subinterfaces:
if interface != 0 and interface.online:
interface_available = True
self.subinterfaces[interface.index].process_outgoing(self.id_callsign)
self.subinterfaces[interface.index].processOutgoing(self.id_callsign)
if interface_available:
RNS.log("Interface "+str(self)+" is transmitting beacon data on all subinterfaces: "+str(self.id_callsign.decode("utf-8")), RNS.LOG_DEBUG)
@ -916,12 +873,12 @@ class RNodeMultiInterface(Interface):
if interface != 0:
self.setRadioState(KISS.RADIO_STATE_OFF, interface)
self.leave()
self.serial.close()
def teardown_subinterfaces(self):
for interface in self.subinterfaces:
if interface != 0:
RNS.Transport.remove_interface(interface)
if interface in RNS.Transport.interfaces:
RNS.Transport.interfaces.remove(interface)
self.subinterfaces[interface.index] = 0
def should_ingress_limit(self):
@ -952,7 +909,7 @@ class RNodeSubInterface(Interface):
if RNS.vendor.platformutils.is_android():
raise SystemError("Invalid interface type. The Android-specific RNode interface must be used on Android")
import importlib.util
import importlib
if importlib.util.find_spec('serial') != None:
import serial
else:
@ -962,9 +919,51 @@ class RNodeSubInterface(Interface):
super().__init__()
if index == 0:
sel_cmd = KISS.CMD_SEL_INT0
data_cmd= KISS.CMD_INT0_DATA
elif index == 1:
sel_cmd = KISS.CMD_SEL_INT1
data_cmd= KISS.CMD_INT1_DATA
elif index == 2:
sel_cmd = KISS.CMD_SEL_INT2
data_cmd= KISS.CMD_INT2_DATA
elif index == 3:
sel_cmd = KISS.CMD_SEL_INT3
data_cmd= KISS.CMD_INT3_DATA
elif index == 4:
sel_cmd = KISS.CMD_SEL_INT4
data_cmd= KISS.CMD_INT4_DATA
elif index == 5:
sel_cmd = KISS.CMD_SEL_INT5
data_cmd= KISS.CMD_INT5_DATA
elif index == 6:
sel_cmd = KISS.CMD_SEL_INT6
data_cmd= KISS.CMD_INT6_DATA
elif index == 7:
sel_cmd = KISS.CMD_SEL_INT7
data_cmd= KISS.CMD_INT7_DATA
elif index == 8:
sel_cmd = KISS.CMD_SEL_INT8
data_cmd= KISS.CMD_INT8_DATA
elif index == 9:
sel_cmd = KISS.CMD_SEL_INT9
data_cmd= KISS.CMD_INT9_DATA
elif index == 10:
sel_cmd = KISS.CMD_SEL_INT10
data_cmd= KISS.CMD_INT10_DATA
elif index == 11:
sel_cmd = KISS.CMD_SEL_INT11
data_cmd= KISS.CMD_INT11_DATA
else:
sel_cmd = KISS.CMD_SEL_INT0
data_cmd= KISS.CMD_INT0_DATA
self.owner = owner
self.name = name
self.index = index
self.sel_cmd = sel_cmd
self.data_cmd = data_cmd
self.interface_type= interface_type
self.flow_control= flow_control
self.online = False
@ -1005,12 +1004,12 @@ class RNodeSubInterface(Interface):
self.r_premable_time_ms = None
self.packet_queue = []
self.shared_medium = True
self.interface_ready = False
self.parent_interface = parent_interface
self.announce_rate_target = None
self.mode = None
self.announce_cap = None
self.bitrate = None
self.ifac_size = None
@ -1030,7 +1029,7 @@ class RNodeSubInterface(Interface):
RNS.log("Invalid interface type configured for "+str(self), RNS.LOG_ERROR)
self.validcfg = False
if (self.txpower < -9 or self.txpower > 37):
if (self.txpower < -9 or self.txpower > 27):
RNS.log("Invalid TX power configured for "+str(self), RNS.LOG_ERROR)
self.validcfg = False
@ -1128,11 +1127,13 @@ class RNodeSubInterface(Interface):
except:
self.bitrate = 0
def process_incoming(self, data):
if not data: return
def processIncoming(self, data):
self.rxb += len(data)
self.owner.inbound(data, self)
self.r_stat_rssi = None
self.r_stat_snr = None
def process_outgoing(self,data):
def processOutgoing(self,data):
if self.online:
if self.interface_ready:
if self.flow_control:
@ -1144,7 +1145,7 @@ class RNodeSubInterface(Interface):
if self.parent_interface.first_tx == None:
self.parent_interface.first_tx = time.time()
self.txb += len(data)
self.parent_interface.process_outgoing(data, self)
self.parent_interface.processOutgoing(data, self)
else:
self.queue(data)
@ -1156,7 +1157,7 @@ class RNodeSubInterface(Interface):
if len(self.packet_queue) > 0:
data = self.packet_queue.pop(0)
self.interface_ready = True
self.process_outgoing(data)
self.processOutgoing(data)
elif len(self.packet_queue) == 0:
self.interface_ready = True

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -28,7 +20,7 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from RNS.Interfaces.Interface import Interface
from .Interface import Interface
from time import sleep
import sys
import threading
@ -50,7 +42,6 @@ class HDLC():
class SerialInterface(Interface):
MAX_CHUNK = 32768
DEFAULT_IFAC_SIZE = 8
owner = None
port = None
@ -60,8 +51,8 @@ class SerialInterface(Interface):
stopbits = None
serial = None
def __init__(self, owner, configuration):
import importlib.util
def __init__(self, owner, name, port, speed, databits, parity, stopbits):
import importlib
if importlib.util.find_spec('serial') != None:
import serial
else:
@ -71,19 +62,7 @@ class SerialInterface(Interface):
super().__init__()
c = Interface.get_config_obj(configuration)
name = c["name"]
port = c["port"] if "port" in c else None
speed = int(c["speed"]) if "speed" in c else 9600
databits = int(c["databits"]) if "databits" in c else 8
parity = c["parity"] if "parity" in c else "N"
stopbits = int(c["stopbits"]) if "stopbits" in c else 1
if port == None:
raise ValueError("No port specified for serial interface")
self.HW_MTU = 564
self.shared_medium = True
self.pyserial = serial
self.serial = None
@ -97,7 +76,6 @@ class SerialInterface(Interface):
self.timeout = 100
self.online = False
self.bitrate = self.speed
self.reconnecting = False
if parity.lower() == "e" or parity.lower() == "even":
self.parity = serial.PARITY_EVEN
@ -107,17 +85,15 @@ class SerialInterface(Interface):
try:
self.open_port()
if self.serial.is_open: self.configure_device()
else: raise IOError("Could not open serial port")
except Exception as e:
RNS.log(f"Could not open serial port for {self}", RNS.LOG_ERROR)
RNS.log(f"The contained exception was: {e}", RNS.LOG_ERROR)
RNS.log(f"Reticulum will attempt to bring up this interface periodically", RNS.LOG_ERROR)
if not self.detached and not self.reconnecting:
thread = threading.Thread(target=self.reconnect_port)
thread.daemon = True
thread.start()
RNS.log("Could not open serial port for interface "+str(self), RNS.LOG_ERROR)
raise e
if self.serial.is_open:
self.configure_device()
else:
raise IOError("Could not open serial port")
def open_port(self):
RNS.log("Opening serial port "+self.port+"...", RNS.LOG_VERBOSE)
@ -135,11 +111,6 @@ class SerialInterface(Interface):
dsrdtr = False,
)
def detach(self):
self.detached = True
self.online = False
try: self.serial.close()
except Exception as e: RNS.log(f"Could not close port for {self} at detach: {e}", RNS.LOG_ERROR)
def configure_device(self):
sleep(0.5)
@ -149,11 +120,13 @@ class SerialInterface(Interface):
self.online = True
RNS.log("Serial port "+self.port+" is now open", RNS.LOG_VERBOSE)
def process_incoming(self, data):
if not data: return
def processIncoming(self, data):
self.rxb += len(data)
self.owner.inbound(data, self)
def process_outgoing(self,data):
def processOutgoing(self,data):
if self.online:
data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
written = self.serial.write(data)
@ -161,6 +134,7 @@ class SerialInterface(Interface):
if written != len(data):
raise IOError("Serial interface only wrote "+str(written)+" bytes of "+str(len(data)))
def readLoop(self):
try:
in_frame = False
@ -175,7 +149,7 @@ class SerialInterface(Interface):
if (in_frame and byte == HDLC.FLAG):
in_frame = False
self.process_incoming(data_buffer)
self.processIncoming(data_buffer)
elif (byte == HDLC.FLAG):
in_frame = True
data_buffer = b""
@ -204,28 +178,30 @@ class SerialInterface(Interface):
RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
if RNS.Reticulum.panic_on_interface_error: RNS.panic()
if RNS.Reticulum.panic_on_interface_error:
RNS.panic()
RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
self.online = False
try: self.serial.close()
except Exception as e: RNS.log(f"Could not close port for {self}", RNS.LOG_ERROR)
if not self.detached and not self.reconnecting: self.reconnect_port()
self.serial.close()
self.reconnect_port()
def reconnect_port(self):
self.reconnecting = True
while not self.online and not self.detached:
while not self.online:
try:
time.sleep(5)
RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
self.open_port()
if self.serial.is_open: self.configure_device()
except Exception as e: RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
if self.serial.is_open:
self.configure_device()
except Exception as e:
RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
self.reconnecting = False
if self.online: RNS.log("Reconnected serial port for "+str(self))
RNS.log("Reconnected serial port for "+str(self))
def should_ingress_limit(self): return False
def should_ingress_limit(self):
return False
def __str__(self): return "SerialInterface["+self.name+"]"
def __str__(self):
return "SerialInterface["+self.name+"]"

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -28,7 +20,7 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from RNS.Interfaces.Interface import Interface
from .Interface import Interface
import socketserver
import threading
import platform
@ -38,9 +30,6 @@ import sys
import os
import RNS
class TCPInterface():
HW_MTU = 262144
class HDLC():
FLAG = 0x7E
ESC = 0x7D
@ -69,13 +58,8 @@ class KISS():
class ThreadingTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
pass
class ThreadingTCP6Server(socketserver.ThreadingMixIn, socketserver.TCPServer):
address_family = socket.AF_INET6
class TCPClientInterface(Interface):
BITRATE_GUESS = 10*1000*1000
DEFAULT_IFAC_SIZE = 16
AUTOCONFIGURE_MTU = True
RECONNECT_WAIT = 5
RECONNECT_MAX_TRIES = None
@ -94,26 +78,11 @@ class TCPClientInterface(Interface):
I2P_PROBE_INTERVAL = 9
I2P_PROBES = 5
def __init__(self, owner, configuration, connected_socket=None):
def __init__(self, owner, name, target_ip=None, target_port=None, connected_socket=None, max_reconnect_tries=None, kiss_framing=False, i2p_tunneled = False, connect_timeout = None):
super().__init__()
c = Interface.get_config_obj(configuration)
name = c["name"]
target_ip = c["target_host"] if "target_host" in c and c["target_host"] != None else None
target_port = int(c["target_port"]) if "target_port" in c and c["target_host"] != None else None
kiss_framing = False
if "kiss_framing" in c and c.as_bool("kiss_framing") == True:
kiss_framing = True
i2p_tunneled = c.as_bool("i2p_tunneled") if "i2p_tunneled" in c else False
connect_timeout = c.as_int("connect_timeout") if "connect_timeout" in c else None
max_reconnect_tries = c.as_int("max_reconnect_tries") if "max_reconnect_tries" in c else None
fixed_mtu = c.as_int("fixed_mtu") if "fixed_mtu" in c else None
if fixed_mtu:
if fixed_mtu < RNS.Reticulum.MTU: raise ValueError(f"Configured MTU of {fixed_mtu} bytes is too small")
self.AUTOCONFIGURE_MTU = False
self.FIXED_MTU = True
self.HW_MTU = TCPInterface.HW_MTU if not fixed_mtu else fixed_mtu
self.HW_MTU = 1064
self.IN = True
self.OUT = False
self.socket = None
@ -126,16 +95,15 @@ class TCPClientInterface(Interface):
self.writing = False
self.online = False
self.detached = False
self.tunnels_allowed = True
self.lossless_medium = True
self.kiss_framing = kiss_framing
self.i2p_tunneled = i2p_tunneled
self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
self.bitrate = TCPClientInterface.BITRATE_GUESS
self.supports_discovery = True
if max_reconnect_tries == None: self.max_reconnect_tries = TCPClientInterface.RECONNECT_MAX_TRIES
else: self.max_reconnect_tries = max_reconnect_tries
if max_reconnect_tries == None:
self.max_reconnect_tries = TCPClientInterface.RECONNECT_MAX_TRIES
else:
self.max_reconnect_tries = max_reconnect_tries
if connected_socket != None:
self.receives = True
@ -160,13 +128,13 @@ class TCPClientInterface(Interface):
self.connect_timeout = connect_timeout
else:
self.connect_timeout = TCPClientInterface.INITIAL_CONNECT_TIMEOUT
def final_init(self):
if TCPClientInterface.SYNCHRONOUS_START: self.initial_connect()
else:
thread = threading.Thread(target=self.initial_connect)
thread.daemon = True
thread.start()
if TCPClientInterface.SYNCHRONOUS_START:
self.initial_connect()
else:
thread = threading.Thread(target=self.initial_connect)
thread.daemon = True
thread.start()
def initial_connect(self):
if not self.connect(initial=True):
@ -177,6 +145,8 @@ class TCPClientInterface(Interface):
thread = threading.Thread(target=self.read_loop)
thread.daemon = True
thread.start()
if not self.kiss_framing:
self.wants_tunnel = True
def set_timeouts_linux(self):
if not self.i2p_tunneled:
@ -207,62 +177,34 @@ class TCPClientInterface(Interface):
self.socket.setsockopt(socket.IPPROTO_TCP, TCP_KEEPIDLE, int(TCPClientInterface.I2P_PROBE_AFTER))
def detach(self):
self.online = False
self.detached = True
if self.socket != None:
if hasattr(self.socket, "close"):
if callable(self.socket.close):
RNS.log("Detaching "+str(self), RNS.LOG_DEBUG)
self.detached = True
try:
if self.socket != None:
self.socket.shutdown(socket.SHUT_RDWR)
self.socket.shutdown(socket.SHUT_RDWR)
except Exception as e:
RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
try:
if self.socket != None:
self.socket.close()
self.socket.close()
except Exception as e:
RNS.log("Error while closing socket for "+str(self)+": "+str(e))
self.socket = None
def early_synthesis(self, packet):
if not self.detached:
if not packet.packed: packet.pack()
if self.ifac_identity: raw = RNS.Transport.handle_outgoing_ifac(self, packet.raw);
else: raw = packet.raw
if self.kiss_framing: frame = bytes([KISS.FEND])+bytes([KISS.CMD_DATA])+KISS.escape(raw)+bytes([KISS.FEND])
else: frame = bytes([HDLC.FLAG])+HDLC.escape(raw)+bytes([HDLC.FLAG])
self.socket.sendall(frame)
self.txb += len(frame)
if hasattr(self, "parent_interface") and self.parent_interface != None:
self.parent_interface.txb += len(frame)
def connect(self, initial=False):
try:
if initial:
RNS.log("Establishing TCP connection for "+str(self)+"...", RNS.LOG_DEBUG)
address_info = socket.getaddrinfo(self.target_ip, self.target_port, proto=socket.IPPROTO_TCP)[0]
address_family = address_info[0]
target_address = address_info[4]
self.socket = socket.socket(address_family, socket.SOCK_STREAM)
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.settimeout(TCPClientInterface.INITIAL_CONNECT_TIMEOUT)
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
self.socket.connect(target_address)
self.socket.connect((self.target_ip, self.target_port))
self.socket.settimeout(None)
if not self.kiss_framing:
self.wants_tunnel = True
self.remote_transport = None
self.synthesis_received = False
RNS.Transport.synthesize_tunnel(self)
self.online = True
if initial:
@ -294,7 +236,7 @@ class TCPClientInterface(Interface):
if not self.reconnecting:
self.reconnecting = True
attempts = 0
while not self.online and not self.detached:
while not self.online:
time.sleep(TCPClientInterface.RECONNECT_WAIT)
attempts += 1
@ -303,7 +245,9 @@ class TCPClientInterface(Interface):
self.teardown()
break
try: self.connect()
try:
self.connect()
except Exception as e:
RNS.log("Connection attempt for "+str(self)+" failed: "+str(e), RNS.LOG_DEBUG)
@ -314,18 +258,22 @@ class TCPClientInterface(Interface):
thread = threading.Thread(target=self.read_loop)
thread.daemon = True
thread.start()
if not self.kiss_framing:
RNS.Transport.synthesize_tunnel(self)
else:
RNS.log("Attempt to reconnect on a non-initiator TCP interface. This should not happen.", RNS.LOG_ERROR)
raise IOError("Attempt to reconnect on a non-initiator TCP interface")
def process_incoming(self, data):
if not data: return
if self.online and not self.detached:
self.owner.inbound(data, self)
def processIncoming(self, data):
self.rxb += len(data)
if hasattr(self, "parent_interface") and self.parent_interface != None:
self.parent_interface.rxb += len(data)
self.owner.inbound(data, self)
def process_outgoing(self, data):
if self.online and not self.detached:
def processOutgoing(self, data):
if self.online:
# while self.writing:
# time.sleep(0.01)
@ -348,35 +296,27 @@ class TCPClientInterface(Interface):
RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
self.teardown()
def check_frame_len(self, frame_len):
if frame_len <= RNS.Reticulum.HEADER_MINSIZE: return False
elif frame_len > self.HW_MTU + (self.ifac_size or 0): return False
else: return True
def invalid_frame(self, frame_len):
RNS.log(f"Invalid HDLC frame of {RNS.prettysize(frame_len)} received on {self}, dropping frame", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
def read_loop(self):
try:
in_frame = False
escape = False
frame_buffer = b""
data_in = b""
data_buffer = b""
command = KISS.CMD_UNKNOWN
while True:
if self.socket: data_in = self.socket.recv(4096)
else: data_in = b""
data_in = self.socket.recv(4096)
if len(data_in) > 0:
if self.kiss_framing:
# Read loop for KISS framing
pointer = 0
while pointer < len(data_in):
byte = data_in[pointer]
pointer += 1
pointer = 0
while pointer < len(data_in):
byte = data_in[pointer]
pointer += 1
if self.kiss_framing:
# Read loop for KISS framing
if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
in_frame = False
self.process_incoming(data_buffer)
self.processIncoming(data_buffer)
elif (byte == KISS.FEND):
in_frame = True
command = KISS.CMD_UNKNOWN
@ -399,39 +339,32 @@ class TCPClientInterface(Interface):
escape = False
data_buffer = data_buffer+bytes([byte])
else:
# Read loop for standard HDLC framing
frame_buffer += data_in
flags_remaining = True
while flags_remaining:
frame_start = frame_buffer.find(HDLC.FLAG)
if frame_start != -1:
frame_end = frame_buffer.find(HDLC.FLAG, frame_start+1)
if frame_end != -1:
frame = frame_buffer[frame_start+1:frame_end]
frame = frame.replace(bytes([HDLC.ESC, HDLC.FLAG ^ HDLC.ESC_MASK]), bytes([HDLC.FLAG]))
frame = frame.replace(bytes([HDLC.ESC, HDLC.ESC ^ HDLC.ESC_MASK]), bytes([HDLC.ESC]))
frame_len = len(frame)
if frame_len != 0:
if self.check_frame_len(frame_len): self.process_incoming(frame)
else: self.invalid_frame(len(frame))
frame_buffer = frame_buffer[frame_end:]
else:
# Read loop for HDLC framing
if (in_frame and byte == HDLC.FLAG):
in_frame = False
self.processIncoming(data_buffer)
elif (byte == HDLC.FLAG):
in_frame = True
data_buffer = b""
elif (in_frame and len(data_buffer) < self.HW_MTU):
if (byte == HDLC.ESC):
escape = True
else:
if len(frame_buffer) > self.HW_MTU*2: frame_buffer = b""
flags_remaining = False
else:
frame_buffer = b""
flags_remaining = False
if (escape):
if (byte == HDLC.FLAG ^ HDLC.ESC_MASK):
byte = HDLC.FLAG
if (byte == HDLC.ESC ^ HDLC.ESC_MASK):
byte = HDLC.ESC
escape = False
data_buffer = data_buffer+bytes([byte])
else:
self.online = False
if self.initiator and not self.detached:
RNS.log("The socket for "+str(self)+" was closed, attempting to reconnect...", RNS.LOG_WARNING)
self.reconnect()
else:
RNS.log("The socket for remote client "+str(self)+" was closed.", RNS.LOG_DEBUG)
RNS.log("The socket for remote client "+str(self)+" was closed.", RNS.LOG_VERBOSE)
self.teardown()
break
@ -441,7 +374,7 @@ class TCPClientInterface(Interface):
self.online = False
RNS.log("An interface error occurred for "+str(self)+", the contained exception was: "+str(e), RNS.LOG_WARNING)
if self.initiator and not self.detached:
if self.initiator:
RNS.log("Attempting to reconnect...", RNS.LOG_WARNING)
self.reconnect()
else:
@ -461,113 +394,55 @@ class TCPClientInterface(Interface):
self.IN = False
if hasattr(self, "parent_interface") and self.parent_interface != None:
while self in self.parent_interface.spawned_interfaces:
self.parent_interface.spawned_interfaces.remove(self)
self.parent_interface.clients -= 1
if self in RNS.Transport.interfaces:
if not self.initiator:
RNS.Transport.interfaces.remove(self)
if not self.initiator:
RNS.Transport.remove_interface(self)
def __str__(self):
if ":" in self.target_ip: ip_str = f"[{self.target_ip}]"
else: ip_str = f"{self.target_ip}"
return "TCPInterface["+str(self.name)+"/"+ip_str+":"+str(self.target_port)+"]"
return "TCPInterface["+str(self.name)+"/"+str(self.target_ip)+":"+str(self.target_port)+"]"
class TCPServerInterface(Interface):
BITRATE_GUESS = 10_000_000
DEFAULT_IFAC_SIZE = 16
AUTOCONFIGURE_MTU = True
BITRATE_GUESS = 10*1000*1000
@staticmethod
def get_address_for_if(name, bind_port, prefer_ipv6=False):
from RNS.Interfaces import netinfo
def get_address_for_if(name):
import RNS.vendor.ifaddr.niwrapper as netinfo
ifaddr = netinfo.ifaddresses(name)
if len(ifaddr) < 1:
raise SystemError(f"No addresses available on specified kernel interface \"{name}\" for TCPServerInterface to bind to")
if (prefer_ipv6 or not netinfo.AF_INET in ifaddr) and netinfo.AF_INET6 in ifaddr:
bind_ip = ifaddr[netinfo.AF_INET6][0]["addr"]
if bind_ip.lower().startswith("fe80::"):
# We'll need to add the interface as scope for link-local addresses
return TCPServerInterface.get_address_for_host(f"{bind_ip}%{name}", bind_port, prefer_ipv6)
else:
return TCPServerInterface.get_address_for_host(bind_ip, bind_port, prefer_ipv6)
elif netinfo.AF_INET in ifaddr:
bind_ip = ifaddr[netinfo.AF_INET][0]["addr"]
return (bind_ip, bind_port)
else:
raise SystemError(f"No addresses available on specified kernel interface \"{name}\" for TCPServerInterface to bind to")
return ifaddr[netinfo.AF_INET][0]["addr"]
@staticmethod
def get_address_for_host(name, bind_port, prefer_ipv6=False):
address_infos = socket.getaddrinfo(name, bind_port, proto=socket.IPPROTO_TCP)
address_info = address_infos[0]
for entry in address_infos:
if prefer_ipv6 and entry[0] == socket.AF_INET6:
address_info = entry; break
elif not prefer_ipv6 and entry[0] == socket.AF_INET:
address_info = entry; break
def get_broadcast_for_if(name):
import RNS.vendor.ifaddr.niwrapper as netinfo
ifaddr = netinfo.ifaddresses(name)
return ifaddr[netinfo.AF_INET][0]["broadcast"]
if address_info[0] == socket.AF_INET6:
return (name, bind_port, address_info[4][2], address_info[4][3])
elif address_info[0] == socket.AF_INET:
return (name, bind_port)
else:
raise SystemError(f"No suitable kernel interface available for address \"{name}\" for TCPServerInterface to bind to")
@property
def clients(self):
return len(self.spawned_interfaces)
def __init__(self, owner, configuration):
def __init__(self, owner, name, device=None, bindip=None, bindport=None, i2p_tunneled=False):
super().__init__()
c = Interface.get_config_obj(configuration)
name = c["name"]
device = c["device"] if "device" in c else None
port = int(c["port"]) if "port" in c else None
bindip = c["listen_ip"] if "listen_ip" in c else None
bindport = int(c["listen_port"]) if "listen_port" in c else None
i2p_tunneled = c.as_bool("i2p_tunneled") if "i2p_tunneled" in c else False
prefer_ipv6 = c.as_bool("prefer_ipv6") if "prefer_ipv6" in c else False
if port != None:
bindport = port
self.supports_discovery = True
self.HW_MTU = TCPInterface.HW_MTU
self.HW_MTU = 1064
self.online = False
self.spawned_interfaces = []
self.clients = 0
self.IN = True
self.OUT = False
self.name = name
self.detached = False
self.tunnels_allowed = True
self.lossless_medium = True
self.i2p_tunneled = i2p_tunneled
self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
if bindport == None:
raise SystemError(f"No TCP port configured for interface \"{name}\"")
else:
self.bind_port = bindport
bind_address = None
if device != None:
bind_address = TCPServerInterface.get_address_for_if(device, self.bind_port, prefer_ipv6)
else:
if bindip == None:
raise SystemError(f"No TCP bind IP configured for interface \"{name}\"")
bind_address = TCPServerInterface.get_address_for_host(bindip, self.bind_port, prefer_ipv6)
bindip = TCPServerInterface.get_address_for_if(device)
if bind_address != None:
if (bindip != None and bindport != None):
self.receives = True
self.bind_ip = bind_address[0]
self.bind_ip = bindip
self.bind_port = bindport
def handlerFactory(callback):
def createHandler(*args, **keys):
@ -575,18 +450,10 @@ class TCPServerInterface(Interface):
return createHandler
self.owner = owner
address = (self.bind_ip, self.bind_port)
if len(bind_address) == 4:
try:
ThreadingTCP6Server.allow_reuse_address = True
self.server = ThreadingTCP6Server(bind_address, handlerFactory(self.incoming_connection))
except Exception as e:
RNS.log(f"Error while binding IPv6 socket for interface, the contained exception was: {e}", RNS.LOG_ERROR)
raise SystemError("Could not bind IPv6 socket for interface. Please check the specified \"listen_ip\" configuration option")
else:
ThreadingTCPServer.allow_reuse_address = True
self.server = ThreadingTCPServer(bind_address, handlerFactory(self.incoming_connection))
self.server.daemon_threads = True
ThreadingTCPServer.allow_reuse_address = True
self.server = ThreadingTCPServer(address, handlerFactory(self.incoming_connection))
self.bitrate = TCPServerInterface.BITRATE_GUESS
@ -596,38 +463,17 @@ class TCPServerInterface(Interface):
self.online = True
else:
raise SystemError("Insufficient parameters to create TCP listener")
def incoming_connection(self, handler):
RNS.log("Accepting incoming TCP connection", RNS.LOG_VERBOSE)
spawned_configuration = {"name": "Client on "+self.name, "target_host": None, "target_port": None, "i2p_tunneled": self.i2p_tunneled}
spawned_interface = TCPClientInterface(self.owner, spawned_configuration, connected_socket=handler.request)
interface_name = "Client on "+self.name
spawned_interface = TCPClientInterface(self.owner, interface_name, target_ip=None, target_port=None, connected_socket=handler.request, i2p_tunneled=self.i2p_tunneled)
spawned_interface.OUT = self.OUT
spawned_interface.IN = self.IN
spawned_interface.ingress_control = self.ingress_control
spawned_interface.ic_max_held_announces = self.ic_max_held_announces
spawned_interface.ic_burst_hold = self.ic_burst_hold
spawned_interface.ic_burst_freq = self.ic_burst_freq
spawned_interface.ic_burst_freq_new = self.ic_burst_freq_new
spawned_interface.ic_new_time = self.ic_new_time
spawned_interface.ic_burst_penalty = self.ic_burst_penalty
spawned_interface.ic_held_release_interval = self.ic_held_release_interval
spawned_interface.egress_control = self.egress_control
spawned_interface.ec_pr_freq = self.ec_pr_freq
spawned_interface.ic_pr_burst_freq_new = self.ic_pr_burst_freq_new
spawned_interface.ic_pr_burst_freq = self.ic_pr_burst_freq
spawned_interface._destination_cost = self._destination_cost
spawned_interface._pr_cost = self._pr_cost
spawned_interface.target_ip = handler.client_address[0]
spawned_interface.target_port = str(handler.client_address[1])
spawned_interface.parent_interface = self
spawned_interface.bitrate = self.bitrate
spawned_interface.optimise_mtu()
spawned_interface.ifac_size = self.ifac_size
spawned_interface.ifac_netname = self.ifac_netname
@ -653,54 +499,31 @@ class TCPServerInterface(Interface):
spawned_interface.announce_rate_grace = self.announce_rate_grace
spawned_interface.announce_rate_penalty = self.announce_rate_penalty
spawned_interface.mode = self.mode
spawned_interface.gravity = self.gravity
spawned_interface.hop_limiter = self.hop_limiter
spawned_interface.recursive_prs = self.recursive_prs
spawned_interface.announces_from_internal = self.announces_from_internal
spawned_interface.announces_to_internal = self.announces_to_internal
spawned_interface.announce_cap = self.announce_cap
spawned_interface.HW_MTU = self.HW_MTU
spawned_interface.online = True
RNS.log("Spawned new TCPClient Interface: "+str(spawned_interface), RNS.LOG_VERBOSE)
RNS.Transport.add_interface(spawned_interface)
while spawned_interface in self.spawned_interfaces:
self.spawned_interfaces.remove(spawned_interface)
self.spawned_interfaces.append(spawned_interface)
RNS.Transport.interfaces.append(spawned_interface)
self.clients += 1
spawned_interface.read_loop()
def received_announce(self, size=0, from_spawned=False):
if from_spawned:
self.ia_freq_deque.append(time.time())
self.arxb += size
def received_announce(self, from_spawned=False):
if from_spawned: self.ia_freq_deque.append(time.time())
def sent_announce(self, size=0, from_spawned=False):
if from_spawned:
self.oa_freq_deque.append(time.time())
self.atxb += size
def sent_announce(self, from_spawned=False):
if from_spawned: self.oa_freq_deque.append(time.time())
def received_path_request(self, size=0, from_spawned=False):
if from_spawned:
self.ip_freq_deque.append(time.time())
self.prxb += size
def sent_path_request(self, size=0, from_spawned=False):
if from_spawned:
self.op_freq_deque.append(time.time())
self.ptxb += size
def process_outgoing(self, data):
def processOutgoing(self, data):
pass
def detach(self):
self.detached = True
self.online = False
if self.server != None:
if hasattr(self.server, "shutdown"):
if callable(self.server.shutdown):
try:
RNS.log("Detaching "+str(self), RNS.LOG_DEBUG)
self.server.shutdown()
self.server.server_close()
self.detached = True
self.server = None
except Exception as e:
@ -708,10 +531,7 @@ class TCPServerInterface(Interface):
def __str__(self):
if ":" in self.bind_ip: ip_str = f"[{self.bind_ip}]"
else: ip_str = f"{self.bind_ip}"
return "TCPServerInterface["+self.name+"/"+ip_str+":"+str(self.bind_port)+"]"
return "TCPServerInterface["+self.name+"/"+self.bind_ip+":"+str(self.bind_port)+"]"
class TCPInterfaceHandler(socketserver.BaseRequestHandler):

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -28,7 +20,7 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from RNS.Interfaces.Interface import Interface
from .Interface import Interface
import socketserver
import threading
import socket
@ -39,38 +31,22 @@ import RNS
class UDPInterface(Interface):
BITRATE_GUESS = 10*1000*1000
DEFAULT_IFAC_SIZE = 16
@staticmethod
def get_address_for_if(name):
from RNS.Interfaces import netinfo
import RNS.vendor.ifaddr.niwrapper as netinfo
ifaddr = netinfo.ifaddresses(name)
return ifaddr[netinfo.AF_INET][0]["addr"]
@staticmethod
def get_broadcast_for_if(name):
from RNS.Interfaces import netinfo
import RNS.vendor.ifaddr.niwrapper as netinfo
ifaddr = netinfo.ifaddresses(name)
return ifaddr[netinfo.AF_INET][0]["broadcast"]
def __init__(self, owner, configuration):
def __init__(self, owner, name, device=None, bindip=None, bindport=None, forwardip=None, forwardport=None):
super().__init__()
c = Interface.get_config_obj(configuration)
name = c["name"]
device = c["device"] if "device" in c else None
port = int(c["port"]) if "port" in c else None
bindip = c["listen_ip"] if "listen_ip" in c else None
bindport = int(c["listen_port"]) if "listen_port" in c else None
forwardip = c["forward_ip"] if "forward_ip" in c else None
forwardport = int(c["forward_port"]) if "forward_port" in c else None
if port != None:
if bindport == None:
bindport = port
if forwardport == None:
forwardport = port
self.HW_MTU = 1064
self.IN = True
@ -78,7 +54,6 @@ class UDPInterface(Interface):
self.name = name
self.online = False
self.bitrate = UDPInterface.BITRATE_GUESS
self.shared_medium = True
if device != None:
if bindip == None:
@ -100,7 +75,7 @@ class UDPInterface(Interface):
self.owner = owner
address = (self.bind_ip, self.bind_port)
socketserver.UDPServer.address_family = socket.AF_INET
self.server = socketserver.UDPServer(address, handlerFactory(self.process_incoming))
self.server = socketserver.UDPServer(address, handlerFactory(self.processIncoming))
thread = threading.Thread(target=self.server.serve_forever)
thread.daemon = True
@ -113,20 +88,12 @@ class UDPInterface(Interface):
self.forward_ip = forwardip
self.forward_port = forwardport
def detach(self):
self.detached = True
self.online = False
if self.server:
try:
self.server.server_close()
self.server.shutdown()
except Exception as e: RNS.log(f"Could not shut down UDP listener for {self}: {e}", RNS.LOG_ERROR)
def process_incoming(self, data):
if not data: return
def processIncoming(self, data):
self.rxb += len(data)
self.owner.inbound(data, self)
def process_outgoing(self,data):
def processOutgoing(self,data):
try:
udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
udp_socket.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)

File diff suppressed because it is too large Load diff

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2025 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -31,12 +23,6 @@
import os
import glob
import RNS.Interfaces.Android
import RNS.Interfaces.util
import RNS.Interfaces.util.netinfo as netinfo
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
modules = py_modules+pyc_modules+so_modules+pyd_modules
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))
modules = glob.glob(os.path.dirname(__file__)+"/*.py")
__all__ = [ os.path.basename(f)[:-3] for f in modules if not f.endswith('__init__.py')]

View file

@ -1,115 +0,0 @@
# Reticulum License
#
# Copyright (c) 2026 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
class HDLC():
FLAG = 0x7E
ESC = 0x7D
ESC_MASK = 0x20
FLAG_B = bytes([FLAG])
ESC_B = bytes([ESC])
ESCPD_FLAG = bytes([ESC, FLAG ^ ESC_MASK])
ESCPD_ESC = bytes([ESC, ESC ^ ESC_MASK])
@staticmethod
def escape(data):
data = data.replace(HDLC.ESC_B, HDLC.ESCPD_ESC)
data = data.replace(HDLC.FLAG_B, HDLC.ESCPD_FLAG)
return data
@staticmethod
def unescape(frame):
frame = frame.replace(HDLC.ESCPD_FLAG, HDLC.FLAG_B)
frame = frame.replace(HDLC.ESCPD_ESC, HDLC.ESC_B)
return frame
@staticmethod
def frame(data):
esc = HDLC.escape(data)
frame = bytearray(len(esc)+2)
frame[0] = HDLC.FLAG
frame[1:-1] = esc
frame[-1] = HDLC.FLAG
return frame
class ReceiveBuffer():
def __init__(self, mtu, min_frame_len, max_frame_len=None, on_frame=None, on_invalid=None):
self._mtu = mtu
self._min_frame_len = min_frame_len
self._max_frame_len = max_frame_len
self._on_frame = on_frame
self._on_invalid = on_invalid
self._buf = bytearray()
self._off = 0
def mtu(self):
return self._mtu() if callable(self._mtu) else self._mtu
def max(self):
m = self._max_frame_len
return m() if callable(m) else m
def feed(self, data):
if not data: return
self._buf.extend(data)
buf = self._buf
while True:
frame_start = buf.find(HDLC.FLAG, self._off)
if frame_start == -1:
self.reset()
return
frame_end = buf.find(HDLC.FLAG, frame_start+1)
if frame_end == -1:
if len(buf)-self._off > self.mtu()*2: self.reset()
return
frame = HDLC.unescape(bytes(buf[frame_start+1:frame_end]))
frame_len = len(frame)
if frame_len > 0:
max_len = self.max()
if frame_len > self._min_frame_len and (max_len is None or frame_len <= max_len):
if self._on_frame is not None: self._on_frame(frame)
elif self._on_invalid is not None: self._on_invalid(frame_len)
self._off = frame_end
if self._off > 0 and self._off >= len(buf) // 2:
del buf[:self._off]
self._off = 0
# Drops any partial frames from the buffer and
# resets assembly offset
def reset(self):
self._buf.clear()
self._off = 0
# Buffered bytes currently unconsumed
def __len__(self): return len(self._buf)-self._off

View file

@ -1,171 +0,0 @@
# Reticulum License
#
# Copyright (c) 2026 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from collections import deque
import threading
# Zero-copy coalesced transmit buffer for arbitrary
# frame sizes. It's even almost 100% lock-free.
class TransmitBuffer:
COALESCE_TARGET = 65536
def __init__(self):
self._tail_lock = threading.Lock()
# Producer-owned state
self._cur = None
self._cur_frames = 0
# Consumer-owned state
self._head_offset = 0
# Shared chunk queue. Producer appends right, consumer
# pops from left. Appended chunks are immutable.
self._chunks = deque()
# Accounting counters
# Producer-owned
self._tx_total = 0 # Total bytes appended
self._tx_visible = 0 # Total bytes pushed to the chunk queue
self._tx_frames = 0 # Total frames appended
# Consumer-owned
self._tx_sent = 0 # Total bytes sent to the kernel
self._tx_frames_sent = 0 # Total frames fully sent
# Queue a finalized and complete on-wire frame.
# Executed by producer side only.
def append(self, frame, limit=None):
with self._tail_lock:
if limit is not None and (self._tx_total - self._tx_sent) + len(frame) > limit: return False
# Allocate separate chunks to frames
# larger than the coalescing target.
if len(frame) >= self.COALESCE_TARGET:
self._flush()
self._chunks.append((frame, 1))
self._tx_visible += len(frame)
self._tx_total += len(frame)
self._tx_frames += 1
else:
if self._cur is not None and len(self._cur) + len(frame) > self.COALESCE_TARGET: self._flush()
if self._cur is None:
self._cur = bytearray()
self._cur_frames = 0
self._cur.extend(frame)
self._cur_frames += 1
self._tx_total += len(frame)
self._tx_frames += 1
if not self._chunks:
# Queue is idle. Make the frame visible immediately so
# sparse traffic does not incur coalescing latency.
self._flush()
return True
def _flush(self):
if self._cur is not None and len(self._cur) > 0:
self._chunks.append((self._cur, self._cur_frames))
self._tx_visible += len(self._cur)
self._cur = None
self._cur_frames = 0
# Make producer-side coalescing tail visible to the consumer.
# Called by producer when it has finished producing a burst,
# and wants to make the remaining bytes transmittable. Safe to
# call after every append.
def flush(self):
with self._tail_lock: self._flush()
# Drain buffered frames to a socket. Called by consumer. If the
# chunk queue is drained completely, any potential producer-side
# coalescing tail is released under the tail lock, and drained in
# the same cycle. If you possess wizardry for doing this without
# the lock, I am all ears.
def drain_to(self, socket):
# TODO: At *some* point (no, not now), replace with a
# mechanism for send_msg to immediately drain all
# chunks directly to the kernel.
total = 0
whole = True
while whole:
if not self._chunks:
with self._tail_lock: self._flush()
if not self._chunks: break
chunk, _ = self._chunks[0]
remaining = len(chunk) - self._head_offset
if remaining <= 0:
self._pop_head()
continue
try: written = socket.send(memoryview(chunk)[self._head_offset:])
except BlockingIOError: break
except InterruptedError: continue
if written <= 0: break
total += written
self._head_offset += written
self._tx_sent += written
if self._head_offset >= len(chunk): self._pop_head()
else: whole = False
return total
def _pop_head(self):
chunk, frames = self._chunks.popleft()
self._tx_frames_sent += frames
self._head_offset = 0
# Total number of bytes currently buffered, including the
# producer's in-progress coalescing chunk.
def __len__(self): return self._tx_total-self._tx_sent
# The number of bytes actually ready to send, and will be
# visible to the consumer, that is, what can be drained.
@property
def sendable(self): return self._tx_visible-self._tx_sent
# The number of complete frames currently buffere, including
# ones contained in a potential in-progress coalescing chunk.
@property
def frames_buffered(self): return self._tx_frames-self._tx_frames_sent
# Number of chunks currently queued and visible to the consumer
@property
def chunks_buffered(self): return len(self._chunks)
def __repr__(self): return f"<TransmitBuffer buffered={len(self)} sendable={self.sendable} frames={self.frames_buffered} chunks={self.chunks_buffered}>"

View file

@ -1,9 +0,0 @@
import os
import glob
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
modules = py_modules+pyc_modules+so_modules+pyd_modules
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))

View file

@ -1,325 +0,0 @@
# MIT License
#
# Copyright (c) 2014 Stefan C. Mueller
# Copyright (c) 2025 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import os
import socket
import ipaddress
import platform
import ctypes.util
import collections
from typing import List, Iterable, Optional, Tuple, Union
AF_INET6 = socket.AF_INET6.value
AF_INET = socket.AF_INET.value
def interfaces() -> List[str]:
adapters = get_adapters(include_unconfigured=True)
return [a.name for a in adapters]
def interface_names_to_indexes() -> dict:
adapters = get_adapters(include_unconfigured=True)
results = {}
for adapter in adapters:
results[adapter.name] = adapter.index
return results
def interface_name_to_nice_name(ifname) -> str:
try:
adapters = get_adapters(include_unconfigured=True)
for adapter in adapters:
if adapter.name == ifname:
if hasattr(adapter, "nice_name"):
return adapter.nice_name
except: return None
return None
def ifaddresses(ifname) -> dict:
adapters = get_adapters(include_unconfigured=True)
ifa = {}
for a in adapters:
if a.name == ifname:
ipv4s = []
ipv6s = []
for ip in a.ips:
t = {}
if ip.is_IPv4:
net = ipaddress.ip_network(str(ip.ip)+"/"+str(ip.network_prefix), strict=False)
t["addr"] = ip.ip
t["prefix"] = ip.network_prefix
t["broadcast"] = str(net.broadcast_address)
ipv4s.append(t)
if ip.is_IPv6:
t["addr"] = ip.ip[0]
ipv6s.append(t)
if len(ipv4s) > 0: ifa[AF_INET] = ipv4s
if len(ipv6s) > 0: ifa[AF_INET6] = ipv6s
return ifa
def get_adapters(include_unconfigured=False):
if os.name == "posix": return _get_adapters_posix(include_unconfigured=include_unconfigured)
elif os.name == "nt": return _get_adapters_win(include_unconfigured=include_unconfigured)
else: raise RuntimeError(f"Unsupported Operating System: {os.name}")
class Adapter(object):
def __init__(self, name: str, nice_name: str, ips: List["IP"], index: Optional[int] = None) -> None:
self.name = name
self.nice_name = nice_name
self.ips = ips
self.index = index
def __repr__(self) -> str:
return "Adapter(name={name}, nice_name={nice_name}, ips={ips}, index={index})".format(
name=repr(self.name), nice_name=repr(self.nice_name), ips=repr(self.ips), index=repr(self.index))
_IPv4Address = str
_IPv6Address = Tuple[str, int, int]
class IP(object):
def __init__(self, ip: Union[_IPv4Address, _IPv6Address], network_prefix: int, nice_name: str) -> None:
self.ip = ip
self.network_prefix = network_prefix
self.nice_name = nice_name
@property
def is_IPv4(self) -> bool: return not isinstance(self.ip, tuple)
@property
def is_IPv6(self) -> bool: return isinstance(self.ip, tuple)
def __repr__(self) -> str:
return "IP(ip={ip}, network_prefix={network_prefix}, nice_name={nice_name})".format(ip=repr(self.ip), network_prefix=repr(self.network_prefix), nice_name=repr(self.nice_name))
if platform.system() == "Darwin" or "BSD" in platform.system():
class sockaddr(ctypes.Structure):
_fields_ = [
("sa_len", ctypes.c_uint8),
("sa_familiy", ctypes.c_uint8),
("sa_data", ctypes.c_uint8 * 14)]
class sockaddr_in(ctypes.Structure):
_fields_ = [
("sa_len", ctypes.c_uint8),
("sa_familiy", ctypes.c_uint8),
("sin_port", ctypes.c_uint16),
("sin_addr", ctypes.c_uint8 * 4),
("sin_zero", ctypes.c_uint8 * 8)]
class sockaddr_in6(ctypes.Structure):
_fields_ = [
("sa_len", ctypes.c_uint8),
("sa_familiy", ctypes.c_uint8),
("sin6_port", ctypes.c_uint16),
("sin6_flowinfo", ctypes.c_uint32),
("sin6_addr", ctypes.c_uint8 * 16),
("sin6_scope_id", ctypes.c_uint32)]
else:
class sockaddr(ctypes.Structure): # type: ignore
_fields_ = [("sa_familiy", ctypes.c_uint16), ("sa_data", ctypes.c_uint8 * 14)]
class sockaddr_in(ctypes.Structure): # type: ignore
_fields_ = [
("sin_familiy", ctypes.c_uint16),
("sin_port", ctypes.c_uint16),
("sin_addr", ctypes.c_uint8 * 4),
("sin_zero", ctypes.c_uint8 * 8)]
class sockaddr_in6(ctypes.Structure): # type: ignore
_fields_ = [
("sin6_familiy", ctypes.c_uint16),
("sin6_port", ctypes.c_uint16),
("sin6_flowinfo", ctypes.c_uint32),
("sin6_addr", ctypes.c_uint8 * 16),
("sin6_scope_id", ctypes.c_uint32)]
def sockaddr_to_ip(sockaddr_ptr: "ctypes.pointer[sockaddr]") -> Optional[Union[_IPv4Address, _IPv6Address]]:
if sockaddr_ptr:
if sockaddr_ptr[0].sa_familiy == socket.AF_INET:
ipv4 = ctypes.cast(sockaddr_ptr, ctypes.POINTER(sockaddr_in))
ippacked = bytes(bytearray(ipv4[0].sin_addr))
ip = str(ipaddress.ip_address(ippacked))
return ip
elif sockaddr_ptr[0].sa_familiy == socket.AF_INET6:
ipv6 = ctypes.cast(sockaddr_ptr, ctypes.POINTER(sockaddr_in6))
flowinfo = ipv6[0].sin6_flowinfo
ippacked = bytes(bytearray(ipv6[0].sin6_addr))
ip = str(ipaddress.ip_address(ippacked))
scope_id = ipv6[0].sin6_scope_id
return (ip, flowinfo, scope_id)
return None
def ipv6_prefixlength(address: ipaddress.IPv6Address) -> int:
prefix_length = 0
for i in range(address.max_prefixlen):
if int(address) >> i & 1: prefix_length = prefix_length + 1
return prefix_length
if os.name == "posix":
class ifaddrs(ctypes.Structure): pass
ifaddrs._fields_ = [
("ifa_next", ctypes.POINTER(ifaddrs)),
("ifa_name", ctypes.c_char_p),
("ifa_flags", ctypes.c_uint),
("ifa_addr", ctypes.POINTER(sockaddr)),
("ifa_netmask", ctypes.POINTER(sockaddr)),]
libc = ctypes.CDLL(ctypes.util.find_library("socket" if os.uname()[0] == "SunOS" else "c"), use_errno=True) # type: ignore
def _get_adapters_posix(include_unconfigured: bool = False) -> Iterable[Adapter]:
addr0 = addr = ctypes.POINTER(ifaddrs)()
retval = libc.getifaddrs(ctypes.byref(addr))
if retval != 0:
eno = ctypes.get_errno()
raise OSError(eno, os.strerror(eno))
ips = collections.OrderedDict()
def add_ip(adapter_name: str, ip: Optional[IP]) -> None:
if adapter_name not in ips:
index = None # type: Optional[int]
try:
index = socket.if_nametoindex(adapter_name) # type: ignore
except (OSError, AttributeError): pass
ips[adapter_name] = Adapter(adapter_name, adapter_name, [], index=index)
if ip is not None:
ips[adapter_name].ips.append(ip)
while addr:
name = addr[0].ifa_name.decode(encoding="UTF-8")
ip_addr = sockaddr_to_ip(addr[0].ifa_addr)
if ip_addr:
if addr[0].ifa_netmask and not addr[0].ifa_netmask[0].sa_familiy:
addr[0].ifa_netmask[0].sa_familiy = addr[0].ifa_addr[0].sa_familiy
netmask = sockaddr_to_ip(addr[0].ifa_netmask)
if isinstance(netmask, tuple):
netmaskStr = str(netmask[0])
prefixlen = ipv6_prefixlength(ipaddress.IPv6Address(netmaskStr))
else:
assert netmask is not None, f"sockaddr_to_ip({addr[0].ifa_netmask}) returned None"
netmaskStr = str("0.0.0.0/" + netmask)
prefixlen = ipaddress.IPv4Network(netmaskStr).prefixlen
ip = IP(ip_addr, prefixlen, name)
add_ip(name, ip)
else:
if include_unconfigured:
add_ip(name, None)
addr = addr[0].ifa_next
libc.freeifaddrs(addr0)
return ips.values()
elif os.name == "nt":
from ctypes import wintypes
NO_ERROR = 0
ERROR_BUFFER_OVERFLOW = 111
MAX_ADAPTER_NAME_LENGTH = 256
MAX_ADAPTER_DESCRIPTION_LENGTH = 128
MAX_ADAPTER_ADDRESS_LENGTH = 8
AF_UNSPEC = 0
class SOCKET_ADDRESS(ctypes.Structure): _fields_ = [("lpSockaddr", ctypes.POINTER(sockaddr)), ("iSockaddrLength", wintypes.INT)]
class IP_ADAPTER_UNICAST_ADDRESS(ctypes.Structure): pass
IP_ADAPTER_UNICAST_ADDRESS._fields_ = [
("Length", wintypes.ULONG),
("Flags", wintypes.DWORD),
("Next", ctypes.POINTER(IP_ADAPTER_UNICAST_ADDRESS)),
("Address", SOCKET_ADDRESS),
("PrefixOrigin", ctypes.c_uint),
("SuffixOrigin", ctypes.c_uint),
("DadState", ctypes.c_uint),
("ValidLifetime", wintypes.ULONG),
("PreferredLifetime", wintypes.ULONG),
("LeaseLifetime", wintypes.ULONG),
("OnLinkPrefixLength", ctypes.c_uint8)]
class IP_ADAPTER_ADDRESSES(ctypes.Structure): pass
IP_ADAPTER_ADDRESSES._fields_ = [
("Length", wintypes.ULONG),
("IfIndex", wintypes.DWORD),
("Next", ctypes.POINTER(IP_ADAPTER_ADDRESSES)),
("AdapterName", ctypes.c_char_p),
("FirstUnicastAddress", ctypes.POINTER(IP_ADAPTER_UNICAST_ADDRESS)),
("FirstAnycastAddress", ctypes.c_void_p),
("FirstMulticastAddress", ctypes.c_void_p),
("FirstDnsServerAddress", ctypes.c_void_p),
("DnsSuffix", ctypes.c_wchar_p),
("Description", ctypes.c_wchar_p),
("FriendlyName", ctypes.c_wchar_p)]
iphlpapi = ctypes.windll.LoadLibrary("Iphlpapi") # type: ignore
def _enumerate_interfaces_of_adapter_win(nice_name: str, address: IP_ADAPTER_UNICAST_ADDRESS) -> Iterable[IP]:
# Iterate through linked list and fill list
addresses = [] # type: List[IP_ADAPTER_UNICAST_ADDRESS]
while True:
addresses.append(address)
if not address.Next: break
address = address.Next[0]
for address in addresses:
ip = sockaddr_to_ip(address.Address.lpSockaddr)
assert ip is not None, f"sockaddr_to_ip({address.Address.lpSockaddr}) returned None"
network_prefix = address.OnLinkPrefixLength
yield IP(ip, network_prefix, nice_name)
def _get_adapters_win(include_unconfigured: bool = False) -> Iterable[Adapter]:
addressbuffersize = wintypes.ULONG(15 * 1024)
retval = ERROR_BUFFER_OVERFLOW
while retval == ERROR_BUFFER_OVERFLOW:
addressbuffer = ctypes.create_string_buffer(addressbuffersize.value)
retval = iphlpapi.GetAdaptersAddresses(
wintypes.ULONG(AF_UNSPEC),
wintypes.ULONG(0),
None,
ctypes.byref(addressbuffer),
ctypes.byref(addressbuffersize))
if retval != NO_ERROR:
raise ctypes.WinError() # type: ignore
# Iterate through adapters and fill array
address_infos = [] # type: List[IP_ADAPTER_ADDRESSES]
address_info = IP_ADAPTER_ADDRESSES.from_buffer(addressbuffer)
while True:
address_infos.append(address_info)
if not address_info.Next: break
address_info = address_info.Next[0]
# Iterate through unicast addresses
result = [] # type: List[Adapter]
for adapter_info in address_infos:
name = adapter_info.AdapterName.decode()
nice_name = adapter_info.Description
index = adapter_info.IfIndex
if adapter_info.FirstUnicastAddress:
ips = _enumerate_interfaces_of_adapter_win(adapter_info.FriendlyName, adapter_info.FirstUnicastAddress[0])
ips = list(ips)
result.append(Adapter(name, nice_name, ips, index=index))
elif include_unconfigured: result.append(Adapter(name, nice_name, [], index=index))
return result

File diff suppressed because it is too large Load diff

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2024 Mark Qvist / unsigned.io and contributors.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -43,10 +35,10 @@ class Packet:
For ``RNS.Destination.GROUP`` destinations, Reticulum will use the
pre-shared key configured for the destination. All packets to group
destinations are encrypted with the same AES-256 key.
destinations are encrypted with the same AES-128 key.
For ``RNS.Destination.SINGLE`` destinations, Reticulum will use a newly
derived ephemeral AES-256 key for every packet.
derived ephemeral AES-128 key for every packet.
For :ref:`RNS.Link<api-link>` destinations, Reticulum will use per-link
ephemeral keys, and offers **Forward Secrecy**.
@ -103,7 +95,7 @@ class Packet:
# With an MTU of 500, the maximum of data we can
# send in a single encrypted packet is given by
# the below calculation; 383 bytes.
ENCRYPTED_MDU = math.floor((RNS.Reticulum.MDU-RNS.Identity.TOKEN_OVERHEAD-RNS.Identity.KEYSIZE//16)/RNS.Identity.AES256_BLOCKSIZE)*RNS.Identity.AES256_BLOCKSIZE - 1
ENCRYPTED_MDU = math.floor((RNS.Reticulum.MDU-RNS.Identity.FERNET_OVERHEAD-RNS.Identity.KEYSIZE//16)/RNS.Identity.AES128_BLOCKSIZE)*RNS.Identity.AES128_BLOCKSIZE - 1
"""
The maximum size of the payload data in a single encrypted packet
"""
@ -114,15 +106,8 @@ class Packet:
TIMEOUT_PER_HOP = RNS.Reticulum.DEFAULT_PER_HOP_TIMEOUT
__slots__ = "hops", "header", "header_type", "packet_type", "transport_type", "context", "context_flag", "destination"
__slots__ += "transport_id", "data", "flags", "raw", "packed", "sent", "create_receipt", "receipt", "fromPacked", "MTU"
__slots__ += "sent_at", "packet_hash", "ratchet_id", "receiving_interface", "attached_interface", "blocked_interfaces"
__slots__ += "rssi", "snr", "q", "ciphertext", "plaintext", "destination_hash", "destination_type", "link", "map_hash"
__slots__ += "is_outbound_pr", "traffic_class", "announce_signature_validated"
def __init__(self, destination, data, packet_type=DATA, context=NONE, transport_type=RNS.Transport.BROADCAST,
header_type=HEADER_1, transport_id=None, attached_interface=None, blocked_interfaces=None,
create_receipt=True, context_flag=FLAG_UNSET):
def __init__(self, destination, data, packet_type = DATA, context = NONE, transport_type = RNS.Transport.BROADCAST,
header_type = HEADER_1, transport_id = None, attached_interface = None, create_receipt = True, context_flag=FLAG_UNSET):
if destination != None:
if transport_type == None:
@ -152,21 +137,13 @@ class Packet:
self.fromPacked = True
self.create_receipt = False
if destination and destination.type == RNS.Destination.LINK:
self.MTU = destination.mtu
else:
self.MTU = RNS.Reticulum.MTU
self.MTU = RNS.Reticulum.MTU
self.sent_at = None
self.packet_hash = None
self.ratchet_id = None
self.traffic_class = None
self.announce_signature_validated = None
self.attached_interface = attached_interface
self.blocked_interfaces = blocked_interfaces
self.receiving_interface = None
self.is_outbound_pr = False
self.rssi = None
self.snr = None
self.q = None
@ -237,13 +214,8 @@ class Packet:
self.header += bytes([self.context])
self.raw = self.header + self.ciphertext
if self.packet_type == Packet.ANNOUNCE and self.header_type == Packet.HEADER_1:
transport_id_length = RNS.Reticulum.TRUNCATED_HASHLENGTH//8
if len(self.raw)+transport_id_length > self.MTU:
raise IOError(f"Transport-inserted announce packet size of {len(self.raw)+transport_id_length} exceeds MTU of {self.MTU} bytes")
if len(self.raw) > self.MTU:
raise IOError(f"Packet size of {len(self.raw)} exceeds MTU of {self.MTU} bytes")
raise IOError("Packet size of "+str(len(self.raw))+" exceeds MTU of "+str(self.MTU)+" bytes")
self.packed = True
self.update_hash()
@ -254,9 +226,6 @@ class Packet:
self.flags = self.raw[0]
self.hops = self.raw[1]
if self.hops >= RNS.Transport.PATHFINDER_M:
raise ValueError(f"Invalid hop count {self.hops}")
self.header_type = (self.flags & 0b01000000) >> 6
self.context_flag = (self.flags & 0b00100000) >> 5
self.transport_type = (self.flags & 0b00010000) >> 4
@ -270,23 +239,18 @@ class Packet:
self.destination_hash = self.raw[DST_LEN+2:2*DST_LEN+2]
self.context = ord(self.raw[2*DST_LEN+2:2*DST_LEN+3])
self.data = self.raw[2*DST_LEN+3:]
if len(self.transport_id) != DST_LEN: raise ValueError("Malformed Transport ID field")
if len(self.destination_hash) != DST_LEN: raise ValueError("Malformed destination hash field")
else:
self.transport_id = None
self.destination_hash = self.raw[2:DST_LEN+2]
self.context = ord(self.raw[DST_LEN+2:DST_LEN+3])
self.data = self.raw[DST_LEN+3:]
if len(self.destination_hash) != DST_LEN: raise ValueError("Malformed destination hash field")
if len(self.data) == 0: raise ValueError("Zero-length data field")
self.packed = False
self.update_hash()
return True
except Exception as e:
RNS.log(f"Dropping malformed packet. The contained exception was: {e}", RNS.LOG_EXTREME) if RNS.sl(RNS.LOG_EXTREME) else None
RNS.log("Received malformed packet, dropping it. The contained exception was: "+str(e), RNS.LOG_EXTREME)
return False
def send(self):
@ -296,26 +260,21 @@ class Packet:
:returns: A :ref:`RNS.PacketReceipt<api-packetreceipt>` instance if *create_receipt* was set to *True* when the packet was instantiated, if not returns *None*. If the packet could not be sent *False* is returned.
"""
if not self.sent:
if self.hops >= RNS.Transport.PATHFINDER_M: return False
if not self.packed: self.pack()
if self.destination.type == RNS.Destination.LINK:
if self.destination.status == RNS.Link.CLOSED:
RNS.log("Attempt to transmit over a closed link, dropping packet", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
self.sent = False
self.receipt = None
return False
raise IOError("Attempt to transmit over a closed link")
else:
self.destination.last_outbound = time.time()
self.destination.tx += 1
self.destination.txbytes += len(self.ciphertext)
RNS.Transport.link_txb += len(self.raw)
self.destination.txbytes += len(self.data)
if RNS.Transport.outbound(self): return self.receipt
if not self.packed:
self.pack()
if RNS.Transport.outbound(self):
return self.receipt
else:
if not self.attached_interface:
RNS.log(f"No interfaces could process the outbound {self.hops} hop, type {self.packet_type} packet for {self.destination}", RNS.LOG_DEBUG) if RNS.sl(RNS.LOG_DEBUG) else None
RNS.log("No interfaces could process the outbound packet", RNS.LOG_ERROR)
self.sent = False
self.receipt = None
return False
@ -334,74 +293,61 @@ class Packet:
# encrypted destinations
self.pack()
if RNS.Transport.outbound(self): return self.receipt
if RNS.Transport.outbound(self):
return self.receipt
else:
RNS.log("Re-send failed. No interfaces could process the outbound packet", RNS.LOG_WARNING)
RNS.log("No interfaces could process the outbound packet", RNS.LOG_ERROR)
self.sent = False
self.receipt = None
return False
else: raise IOError("Packet was not sent yet")
else:
raise IOError("Packet was not sent yet")
def prove(self, destination=None):
if self.fromPacked and hasattr(self, "destination") and self.destination:
if self.destination.identity and self.destination.identity.prv:
self.destination.identity.prove(self, destination)
elif self.fromPacked and hasattr(self, "link") and self.link: self.link.prove_packet(self)
else: RNS.log("Could not prove packet associated with neither a destination nor a link", RNS.LOG_ERROR)
elif self.fromPacked and hasattr(self, "link") and self.link:
self.link.prove_packet(self)
else:
RNS.log("Could not prove packet associated with neither a destination nor a link", RNS.LOG_ERROR)
# Generates a special destination that allows Reticulum
# to direct the proof back to the proved packet's sender
def generate_proof_destination(self): return ProofDestination(self)
def generate_proof_destination(self):
return ProofDestination(self)
def validate_proof_packet(self, proof_packet): return self.receipt.validate_proof_packet(proof_packet)
def validate_proof_packet(self, proof_packet):
return self.receipt.validate_proof_packet(proof_packet)
def validate_proof(self, proof): return self.receipt.validate_proof(proof)
def validate_proof(self, proof):
return self.receipt.validate_proof(proof)
def update_hash(self): self.packet_hash = self.get_hash()
def update_hash(self):
self.packet_hash = self.get_hash()
def get_hash(self): return RNS.Identity.full_hash(self.get_hashable_part())
def get_hash(self):
return RNS.Identity.full_hash(self.get_hashable_part())
def getTruncatedHash(self): return RNS.Identity.truncated_hash(self.get_hashable_part())
@property
def truncated_packet_hash(self):
return self.packet_hash[:RNS.Reticulum.TRUNCATED_HASHLENGTH//8]
def getTruncatedHash(self):
return RNS.Identity.truncated_hash(self.get_hashable_part())
def get_hashable_part(self):
hashable_part = bytes([self.raw[0] & 0b00001111])
if self.header_type == Packet.HEADER_2: hashable_part += self.raw[(RNS.Identity.TRUNCATED_HASHLENGTH//8)+2:]
else: hashable_part += self.raw[2:]
if self.header_type == Packet.HEADER_2:
hashable_part += self.raw[(RNS.Identity.TRUNCATED_HASHLENGTH//8)+2:]
else:
hashable_part += self.raw[2:]
return hashable_part
def get_rssi(self):
"""
:returns: The physical layer *Received Signal Strength Indication* if available, otherwise ``None``.
"""
if self.rssi != None: return self.rssi
else: return RNS.Reticulum.get_instance().get_packet_rssi(self.packet_hash)
def get_snr(self):
"""
:returns: The physical layer *Signal-to-Noise Ratio* if available, otherwise ``None``.
"""
if self.snr != None: return self.snr
else: return RNS.Reticulum.get_instance().get_packet_snr(self.packet_hash)
def get_q(self):
"""
:returns: The physical layer *Link Quality* if available, otherwise ``None``.
"""
if self.q != None: return self.q
else: return RNS.Reticulum.get_instance().get_packet_q(self.packet_hash)
class ProofDestination:
def __init__(self, packet):
self.hash = packet.truncated_packet_hash;
self.hash = packet.get_hash()[:RNS.Reticulum.TRUNCATED_HASHLENGTH//8];
self.type = RNS.Destination.SINGLE
def encrypt(self, plaintext): return plaintext
def encrypt(self, plaintext):
return plaintext
class PacketReceipt:
@ -423,8 +369,8 @@ class PacketReceipt:
# Creates a new packet receipt from a sent packet
def __init__(self, packet):
self.hash = packet.packet_hash
self.truncated_hash = packet.truncated_packet_hash
self.hash = packet.get_hash()
self.truncated_hash = packet.getTruncatedHash()
self.sent = True
self.sent_at = time.time()
self.proved = False
@ -470,17 +416,18 @@ class PacketReceipt:
link.last_proof = self.concluded_at
if self.callbacks.delivery != None:
try: self.callbacks.delivery(self)
try:
self.callbacks.delivery(self)
except Exception as e:
RNS.log("An error occurred while evaluating external delivery callback for "+str(link), RNS.LOG_ERROR)
RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
RNS.trace_exception(e)
return True
else: return False
else: return False
else:
return False
else:
return False
elif len(proof) == PacketReceipt.IMPL_LENGTH:
pass
# TODO: Why is this disabled?
@ -497,7 +444,8 @@ class PacketReceipt:
# else:
# RNS.log("invalid")
# return False
else: return False
else:
return False
# Validate a raw proof
def validate_proof(self, proof, proof_packet=None):
@ -514,20 +462,20 @@ class PacketReceipt:
self.proof_packet = proof_packet
if self.callbacks.delivery != None:
try: self.callbacks.delivery(self)
try:
self.callbacks.delivery(self)
except Exception as e:
RNS.log("Error while executing proof validated callback. The contained exception was: "+str(e), RNS.LOG_ERROR)
return True
else: return False
else: return False
else:
return False
else:
return False
elif len(proof) == PacketReceipt.IMPL_LENGTH:
# This is an implicit proof
if not hasattr(self.destination, "identity"): return False
if self.destination.identity == None: return False
if self.destination.identity == None:
return False
signature = proof[:RNS.Identity.SIGLENGTH//8]
proof_valid = self.destination.identity.validate(signature, self.hash)
@ -538,13 +486,16 @@ class PacketReceipt:
self.proof_packet = proof_packet
if self.callbacks.delivery != None:
try: self.callbacks.delivery(self)
except Exception as e: RNS.log("Error while executing proof validated callback. The contained exception was: "+str(e), RNS.LOG_ERROR)
try:
self.callbacks.delivery(self)
except Exception as e:
RNS.log("Error while executing proof validated callback. The contained exception was: "+str(e), RNS.LOG_ERROR)
return True
else: return False
else: return False
else:
return False
else:
return False
def get_rtt(self):
"""
@ -557,8 +508,10 @@ class PacketReceipt:
def check_timeout(self):
if self.status == PacketReceipt.SENT and self.is_timed_out():
if self.timeout == -1: self.status = PacketReceipt.CULLED
else: self.status = PacketReceipt.FAILED
if self.timeout == -1:
self.status = PacketReceipt.CULLED
else:
self.status = PacketReceipt.FAILED
self.concluded_at = time.time()

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -31,4 +23,5 @@
class Resolver:
@staticmethod
def resolve_identity(full_name): pass
def resolve_identity(full_name):
pass

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -1,851 +0,0 @@
# Reticulum License
#
# Copyright (c) 2016-2026 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import RNS
import hashlib
import struct
import sys
import threading
import os
import time
from array import array
class AnnounceRateTable:
# Columnar store for announce rate tracking. Layout:
# _index : dict[destination_hash -> slot]
# _keys : list[slot] -> destination_hash (for swap-remove bookkeeping)
# _last : array('d') last non-violating announce time
# _blocked: array('d') blocked_until
# _viol : array('i') rate_violations
# _ts : array('d') flat ring of max_timestamps doubles per slot
# _ts_len : array('B') valid timestamp count per slot
# _ts_head: array('B') ring write cursor per slot
def __init__(self, max_timestamps=16):
assert max_timestamps <= 255, "Timestamp ring size must fit in a single byte"
self.max_ts = max_timestamps
self._index = {}
self._keys = []
self._last = array('d')
self._blocked = array('d')
self._viol = array('i')
self._ts = array('d')
self._ts_len = array('B')
self._ts_head = array('B')
self._cursor = 0
def __len__(self): return len(self._keys)
def __contains__(self, destination_hash): return destination_hash in self._index
# Perform the announce-rate check-and-update for one announce.
# Caller must hold the announce_rate_table_lock.
def check_and_update(self, destination_hash, now, target, grace, penalty):
slot = self._index.get(destination_hash)
if slot is None:
slot = len(self._keys)
self._index[destination_hash] = slot
self._keys.append(destination_hash)
self._last.append(now)
self._blocked.append(0.0)
self._viol.append(0)
self._ts.extend([0.0]*self.max_ts)
self._ts[slot*self.max_ts] = now
self._ts_len.append(1)
self._ts_head.append(1 % self.max_ts)
return False
base = slot*self.max_ts
head = self._ts_head[slot]
self._ts[base+head] = now
self._ts_head[slot] = (head+1) % self.max_ts
if self._ts_len[slot] < self.max_ts: self._ts_len[slot] += 1
current_rate = now - self._last[slot]
rate_blocked = False
if now > self._blocked[slot]:
if current_rate < target: self._viol[slot] += 1
else:
v = self._viol[slot] - 1
self._viol[slot] = v if v > 0 else 0
if self._viol[slot] > grace:
self._blocked[slot] = self._last[slot] + target + penalty
rate_blocked = True
else: self._last[slot] = now
else: rate_blocked = True
return rate_blocked
# Caller must hold the table lock
def cull(self, now, ttl, max_scan=1024, max_cull=None):
if max_cull is None: max_cull = max_scan
horizon = now - ttl
culled = 0
scanned = 0
n = len(self._keys)
while scanned < max_scan and culled < max_cull and n > 0:
if self._cursor >= n: self._cursor = 0
slot = self._cursor
if self._last[slot] < horizon and self._blocked[slot] <= now:
last_i = n - 1
stale_key = self._keys[slot]
if slot != last_i:
moved_key = self._keys[last_i]
self._keys[slot] = moved_key
self._index[moved_key] = slot
self._last[slot] = self._last[last_i]
self._blocked[slot] = self._blocked[last_i]
self._viol[slot] = self._viol[last_i]
self._ts_len[slot] = self._ts_len[last_i]
self._ts_head[slot] = self._ts_head[last_i]
f = last_i*self.max_ts; t = slot*self.max_ts
self._ts[t:t+self.max_ts] = self._ts[f:f+self.max_ts]
self._keys.pop()
self._last.pop(); self._blocked.pop(); self._viol.pop()
self._ts_len.pop(); self._ts_head.pop()
del self._ts[len(self._ts)-self.max_ts:] # drop tail block of removed slot
del self._index[stale_key]
n -= 1
culled += 1
# Do not advance cursor, re-examine the slot a moved entry landed in
else:
self._cursor = slot + 1
scanned += 1
if self._cursor >= n: self._cursor = 0
return culled
# Hold table lock, or accept a possibly inconsistent snapshot
def items(self):
for slot, key in enumerate(self._keys):
n = self._ts_len[slot]
head = self._ts_head[slot]
base = slot*self.max_ts
if n < self.max_ts: ts = list(self._ts[base:base+n])
else: ts = list(self._ts[base+head:base+self.max_ts]) + list(self._ts[base:base+head])
yield key, { "last": self._last[slot], "rate_violations": self._viol[slot],
"blocked_until": self._blocked[slot], "timestamps": ts }
# Persists the announce rate table to a file, in
# compacted, little-endian binary format:
#
# Header (16 bytes)
# 0x00 6 Magic bytes
# 0x06 1 Format version
# 0x07 1 Flags, reserved, must be zero
# 0x08 1 Timestamp ring size (max_ts)
# 0x09 1 Destination hash length in bytes (keylen)
# 0x0A 2 Reserved, zero
# 0x0C 4 Entry count, uint32, little-endian
#
# Entries (columnar, in native slot order)
# Destination hash keys, followed by the _last, _blocked,
# _viol, _ts_len, _ts_head and _ts columns, all encoded as
# little-endian fixed-width values.
#
# Caller must hold the announce rate table lock
PERSIST_MAGIC = b"RNSART"
PERSIST_VERSION = 0x01
def persist(self, path, retain=None):
try:
count = len(self._keys)
if count > 0:
keylen = len(self._keys[0])
for key in self._keys:
if len(key) != keylen: raise ValueError("Inconsistent destination hash lengths in table")
else:
keylen = RNS.Identity.TRUNCATED_HASHLENGTH//8
if retain != None and count > 0:
now = time.time()
horizon = now - retain
keep = [slot for slot in range(count) if not (self._last[slot] < horizon and self._blocked[slot] <= now)]
else:
keep = None
if keep != None and len(keep) != count:
count = len(keep)
keys = [self._keys[slot] for slot in keep]
last = array("d", [self._last[slot] for slot in keep])
blocked = array("d", [self._blocked[slot] for slot in keep])
viol = array("i", [self._viol[slot] for slot in keep])
ts_len = array("B", [self._ts_len[slot] for slot in keep])
ts_head = array("B", [self._ts_head[slot] for slot in keep])
ts = array("d")
for slot in keep: ts.extend(self._ts[slot*self.max_ts:(slot+1)*self.max_ts])
else:
keys = self._keys
last = self._last
blocked = self._blocked
viol = self._viol
ts_len = self._ts_len
ts_head = self._ts_head
ts = self._ts
if last.itemsize != 8 or viol.itemsize != 4:
raise ValueError(f"Unsupported array itemsize at persist: d={last.itemsize}, i={viol.itemsize}")
header = self.PERSIST_MAGIC
header += bytes([self.PERSIST_VERSION, 0x00, self.max_ts & 0xFF, keylen & 0xFF])
header += struct.pack("<H", 0)
header += struct.pack("<I", count)
blob = header
blob += b"".join(keys)
blob += self._column_le(last)
blob += self._column_le(blocked)
blob += self._column_le(viol)
blob += ts_len.tobytes()
blob += ts_head.tobytes()
blob += self._column_le(ts)
tmp_path = path+".tmp"
with open(tmp_path, "wb") as file: file.write(blob)
os.replace(tmp_path, path)
return True
except Exception as e:
RNS.log(f"Could not persist announce rate table to {path}: {e}", RNS.LOG_ERROR)
return False
# Caller must hold the announce rate table lock
def load(self, path):
try:
with open(path, "rb") as file: blob = file.read()
except Exception: return False
try:
if len(blob) < 16: return False
if blob[0:6] != self.PERSIST_MAGIC: return False
if blob[6] != self.PERSIST_VERSION: return False
if blob[7] != 0x00: return False
file_max_ts = blob[8]
keylen = blob[9]
count = struct.unpack_from("<I", blob, 12)[0]
if file_max_ts < 1 or file_max_ts > 255: return False
if keylen < 1: return False
entry_size = keylen+22+file_max_ts*8
if len(blob) != 16+count*entry_size: return False
if file_max_ts == self.max_ts:
# Ring size matches, load directly
keys = [blob[16+slot*keylen : 16+(slot+1)*keylen] for slot in range(count)]
index = {}
for slot in range(count):
if keys[slot] in index: return False
index[keys[slot]] = slot
offset = 16+count*keylen
last = self._column_from_le("d", blob[offset:offset+count*8]); offset += count*8
blocked = self._column_from_le("d", blob[offset:offset+count*8]); offset += count*8
viol = self._column_from_le("i", blob[offset:offset+count*4]); offset += count*4
ts_len = array("B", blob[offset:offset+count]); offset += count
ts_head = array("B", blob[offset:offset+count]); offset += count
ts = self._column_from_le("d", blob[offset:offset+count*self.max_ts*8]); offset += count*self.max_ts*8
for slot in range(count):
if ts_len[slot] > self.max_ts or ts_head[slot] >= self.max_ts: return False
else:
# Different ring sizes from current running config,
# rebuild entries best effort.
keys = [blob[16+slot*keylen : 16+(slot+1)*keylen] for slot in range(count)]
index = {}
for slot in range(count):
if keys[slot] in index: return False
index[keys[slot]] = slot
offset = 16+count*keylen
last = self._column_from_le("d", blob[offset:offset+count*8]); offset += count*8
blocked = self._column_from_le("d", blob[offset:offset+count*8]); offset += count*8
viol = self._column_from_le("i", blob[offset:offset+count*4]); offset += count*4
f_ts_len = array("B", blob[offset:offset+count]); offset += count
f_ts_head = array("B", blob[offset:offset+count]); offset += count
f_ts = self._column_from_le("d", blob[offset:offset+count*file_max_ts*8]); offset += count*file_max_ts*8
for slot in range(count):
if f_ts_len[slot] > file_max_ts or f_ts_head[slot] >= file_max_ts: return False
ts_len = array("B"); ts_head = array("B"); ts = array("d")
for slot in range(count):
base = slot*file_max_ts
if f_ts_len[slot] < file_max_ts: chronological = list(f_ts[base:base+f_ts_len[slot]])
else: chronological = list(f_ts[base+f_ts_head[slot]:base+file_max_ts])+list(f_ts[base:base+f_ts_head[slot]])
chronological = chronological[-self.max_ts:]
ts_count = len(chronological)
ts_len.append(ts_count)
ts.extend(chronological)
if ts_count < self.max_ts: ts.extend([0.0]*(self.max_ts-ts_count))
ts_head.append(ts_count % self.max_ts)
self._index = index
self._keys = keys
self._last = last
self._blocked = blocked
self._viol = viol
self._ts = ts
self._ts_len = ts_len
self._ts_head = ts_head
self._cursor = 0
return True
except Exception as e:
RNS.log(f"Could not load announce rate table from {path}: {e}", RNS.LOG_ERROR)
return False
def _column_le(self, column):
if sys.byteorder == "big": swapped = array(column.typecode, column); swapped.byteswap(); return swapped.tobytes()
else: return column.tobytes()
def _column_from_le(self, typecode, data):
column = array(typecode); column.frombytes(data)
if sys.byteorder == "big": column.byteswap()
return column
class PacketHashlist:
# Bucketed byte-blob packet filter hashlist, with the primary
# design goal of memory coherence and density. All operations
# are safe to call without holding a lock.
def __init__(self, hash_width=32, bucket_count=16384):
assert bucket_count > 0 and bucket_count & (bucket_count-1) == 0, "Bucket count must be a power of two"
self.width = hash_width
self._nb = bucket_count
self._mask = bucket_count - 1
self._cur = [bytearray() for _ in range(bucket_count)]
self._prv = [bytearray() for _ in range(bucket_count)]
self._count = 0
def __len__(self):
return self._count
def __contains__(self, packet_hash):
bucket = (packet_hash[0] | (packet_hash[1] << 8)) & self._mask
return packet_hash in self._cur[bucket] or packet_hash in self._prv[bucket]
def add(self, packet_hash):
assert len(packet_hash) == self.width, "Packet hash width mismatch"
bucket = (packet_hash[0] | (packet_hash[1] << 8)) & self._mask
if packet_hash not in self._cur[bucket]:
self._cur[bucket].extend(packet_hash)
self._count += 1
def discard(self, packet_hash):
w = self.width
bucket = (packet_hash[0] | (packet_hash[1] << 8)) & self._mask
for generation in (self._cur, self._prv):
entries = generation[bucket]
pos = entries.find(packet_hash)
while pos != -1:
if pos % w == 0:
del entries[pos:pos+w]
if generation is self._cur: self._count -= 1
pos = entries.find(packet_hash, pos)
else:
pos = entries.find(packet_hash, pos+1)
def swap(self):
self._prv = self._cur
self._cur = [bytearray() for _ in range(self._nb)]
self._count = 0
# Current-generation buckets for persist operations
def buckets(self): return tuple(self._cur)
class KnownDestinationsStore:
# Write-behind SQLite storage backend for known destinations.
# The actual in-memory table is used directly for all transport
# operations; this serves purely as the store, and no read operations
# utilize the database directly.
def __init__(self, db_path, hash_width=16, read_only=False):
import sqlite3
self.sqlite3 = sqlite3
self.db_path = db_path
self.hash_width = hash_width
self.read_only = read_only
self._dirty_upsert = set()
self._dirty_delete = set()
self._ratchet_upserts = {}
self._ratchet_deletes = set()
self._kd_quarantine = {}
self._ratchet_quarantine = {}
self._flush_lock = threading.Lock()
if read_only: self._conn = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True, check_same_thread=False, isolation_level=None, cached_statements=0)
else:
self._conn = sqlite3.connect(db_path, check_same_thread=False, isolation_level=None, cached_statements=0)
self._conn.execute("PRAGMA journal_mode=WAL")
self._conn.execute("PRAGMA synchronous=NORMAL")
self._conn.execute("PRAGMA journal_size_limit=8388608")
self.__create_schema()
def close(self):
if self._conn != None:
self._conn.close()
self._conn = None
def __create_schema(self):
db_version = self._conn.execute("PRAGMA user_version").fetchone()[0]
# Pre-release database cleanup
# TODO: Remove this in 1.6.1
if db_version > 0:
RNS.log(f"Pre-release database detected, rebuilding known destinations DB to release format", RNS.LOG_WARNING)
self._conn.execute("DROP TABLE IF EXISTS known_destinations")
self._conn.execute("""CREATE TABLE IF NOT EXISTS known_destinations (
destination_hash BLOB PRIMARY KEY,
last_heard REAL NOT NULL,
packet_hash BLOB,
public_key BLOB NOT NULL,
app_data BLOB,
used REAL NOT NULL DEFAULT 0,
stamp BLOB,
stamp_value REAL,
lr_cost REAL,
identity_hash BLOB NOT NULL)""")
self._conn.execute("""CREATE TABLE IF NOT EXISTS ratchets (
destination_hash BLOB PRIMARY KEY,
ratchet BLOB NOT NULL,
received REAL NOT NULL)""")
# Schema migrations
try:
self._conn.execute("BEGIN")
# No migrations yet
self._conn.execute("PRAGMA user_version = 0")
self._conn.execute("COMMIT")
self._conn.commit()
except Exception as e:
RNS.log(f"Known destinations database version migration failed: {e}", RNS.LOG_CRITICAL)
RNS.log(f"You will need to remove the known_destinations.db* files from your storage directory so it can be rebuilt", RNS.LOG_CRITICAL)
def __entry_row(self, destination_hash, entry):
stamp = entry[5] if len(entry) > 5 else None
stamp_value = entry[6] if len(entry) > 6 else 0
lr_cost = entry[7] if len(entry) > 7 else None
identity_hash = hashlib.sha256(entry[2]).digest()[:self.hash_width]
return (destination_hash, entry[0], entry[1], entry[2], entry[3], entry[4], stamp, stamp_value, lr_cost, identity_hash)
# Caller must hold Identity.known_destinations_lock
def mark_dirty(self, destination_hash):
if self.read_only: return
self._dirty_delete.discard(destination_hash)
self._kd_quarantine.pop(destination_hash, None)
self._dirty_upsert.add(destination_hash)
# Caller must hold Identity.known_destinations_lock
def mark_deleted(self, destination_hash):
if self.read_only: return
self._dirty_upsert.discard(destination_hash)
self._dirty_delete.add(destination_hash)
self._kd_quarantine.pop(destination_hash, None)
self._ratchet_quarantine.pop(destination_hash, None)
# Caller must hold Identity.known_destinations_lock
def take_pending(self):
if self.read_only: return set(), set(), {}, set()
upserts = self._dirty_upsert
deletes = self._dirty_delete
ratchet_upserts = self._ratchet_upserts
ratchet_deletes = self._ratchet_deletes
self._dirty_upsert = set()
self._dirty_delete = set()
self._ratchet_upserts = {}
self._ratchet_deletes = set()
return upserts, deletes, ratchet_upserts, ratchet_deletes
# Caller must hold the flush lock and have an already open transaction
def __wrap_exec(self, sql, rows):
rejects = []
# By default, run batched queries in chunks.
for i in range(0, len(rows), 4096):
chunk = rows[i:i+4096]
try:
self._conn.execute("SAVEPOINT chunk")
self._conn.executemany(sql, chunk)
self._conn.execute("RELEASE chunk")
# But on integrity and data errors, roll back the chunk
# and iterate per-row, so we can catch any violating
# entries and quarantine them. A later, sane update
# will then unquarantine and heal upon arrival.
#
# Other exceptions are intentionally not caught here,
# so they can propagate directly.
except (self.sqlite3.IntegrityError, self.sqlite3.DataError):
self._conn.execute("ROLLBACK TO chunk")
for row in chunk:
try:
self._conn.execute("SAVEPOINT row")
self._conn.execute(sql, row)
self._conn.execute("RELEASE row")
except (self.sqlite3.IntegrityError, self.sqlite3.DataError) as e:
self._conn.execute("ROLLBACK TO row")
rejects.append((row[0], str(e)))
self._conn.execute("RELEASE chunk")
return rejects
# May be called without any locks held
def apply(self, upserts, deletes, ratchet_upserts, ratchet_deletes, known_destinations):
if self.read_only: return 0
if len(upserts) == 0 and len(deletes) == 0 and len(ratchet_upserts) == 0 and len(ratchet_deletes) == 0: return 0
with self._flush_lock:
rows = []
removables = []
ratchet_rows = []
ratchet_removables = []
for destination_hash in upserts:
if destination_hash in self._kd_quarantine: continue
entry = known_destinations.get(destination_hash)
if entry != None: rows.append(self.__entry_row(destination_hash, entry))
for destination_hash in deletes:
if destination_hash not in known_destinations: removables.append((destination_hash,))
for destination_hash, ratchet_entry in ratchet_upserts.items():
# Ratchets of purged destinations must be removed as well
if destination_hash not in known_destinations: ratchet_removables.append((destination_hash,))
elif destination_hash not in self._ratchet_quarantine: ratchet_rows.append((destination_hash, *ratchet_entry))
for destination_hash in ratchet_deletes:
if destination_hash not in known_destinations: ratchet_removables.append((destination_hash,))
kd_rejects = []
ratchet_rejects = []
try:
self._conn.execute("BEGIN")
kd_rejects += self.__wrap_exec("INSERT INTO known_destinations (destination_hash, last_heard, packet_hash, public_key, app_data, used, stamp, stamp_value, lr_cost, identity_hash) VALUES (?,?,?,?,?,?,?,?,?,?) ON CONFLICT(destination_hash) DO UPDATE SET last_heard=excluded.last_heard, packet_hash=excluded.packet_hash, public_key=excluded.public_key, app_data=excluded.app_data, used=excluded.used, stamp=excluded.stamp, stamp_value=excluded.stamp_value, lr_cost=excluded.lr_cost, identity_hash=excluded.identity_hash", rows)
kd_rejects += self.__wrap_exec("DELETE FROM known_destinations WHERE destination_hash=?", removables)
ratchet_rejects += self.__wrap_exec("INSERT INTO ratchets VALUES (?,?,?) ON CONFLICT(destination_hash) DO UPDATE SET ratchet=excluded.ratchet, received=excluded.received", ratchet_rows)
ratchet_rejects += self.__wrap_exec("DELETE FROM ratchets WHERE destination_hash=?", ratchet_removables)
self._conn.commit()
except Exception:
# If an exception is not handled in the batch query wrapper,
# we will have to roll back the transaction, and mark all
# attempted entries as dirty. A failure state like such
# should not happen, and this is a last resort; at this
# point, writes may stall until an updated entry for
# the violating row(s) arrive and heal the invalid
# dirty entries, so this is raised for logging.
self._dirty_upsert |= {row[0] for row in rows}
self._dirty_delete |= {row[0] for row in removables}
self._ratchet_upserts |= {row[0]: row[1:] for row in ratchet_rows}
self._ratchet_deletes |= {row[0] for row in ratchet_removables}
self._conn.rollback()
raise
# Track failed persists and one-time log per failure. A later
# update of the destination clears the quarantine state and
# allows persist to retry with the (hopefully) valid data.
if len(kd_rejects) > 0 or len(ratchet_rejects) > 0:
for destination_hash, error in kd_rejects:
self._kd_quarantine[destination_hash] = (error, time.time())
RNS.log(f"Invalid known destination data for {RNS.prettyhexrep(destination_hash)} at persist. The contained exception was: {error}", RNS.LOG_ERROR)
for destination_hash, error in ratchet_rejects:
self._ratchet_quarantine[destination_hash] = (error, time.time())
RNS.log(f"Invalid ratchet data for {RNS.prettyhexrep(destination_hash)} at persist. The contained exception was: {error}", RNS.LOG_ERROR)
return len(rows)+len(removables)+len(ratchet_rows)+len(ratchet_removables)-len(kd_rejects)-len(ratchet_rejects)
def load_all(self):
cursor = self._conn.execute("SELECT destination_hash, last_heard, packet_hash, public_key, app_data, used, stamp, stamp_value, lr_cost FROM known_destinations")
while True:
rows = cursor.fetchmany(4096)
if not rows: break
for row in rows: yield row[0], [row[1], row[2], row[3], row[4], row[5], row[6], row[7], row[8]]
def remember_ratchet(self, destination_hash, ratchet, now):
if self.read_only: return False
self._ratchet_quarantine.pop(destination_hash, None)
pending = self._ratchet_upserts.get(destination_hash)
if pending != None:
if pending[0] == ratchet:
self._ratchet_deletes.discard(destination_hash)
return False
else:
with self._flush_lock:
row = self._conn.execute("SELECT ratchet FROM ratchets WHERE destination_hash=?", (destination_hash,)).fetchone()
if row != None and row[0] == ratchet:
# Discard any pending delete mark queued
# before this ratchet arrived
self._ratchet_deletes.discard(destination_hash)
return False
self._ratchet_upserts[destination_hash] = (ratchet, now)
self._ratchet_deletes.discard(destination_hash)
return True
def fetch_ratchet(self, destination_hash):
pending = self._ratchet_upserts.get(destination_hash)
if pending != None: return pending[0], pending[1]
if destination_hash in self._ratchet_deletes: return None
with self._flush_lock:
row = self._conn.execute("SELECT ratchet, received FROM ratchets WHERE destination_hash=?", (destination_hash,)).fetchone()
if row != None: return row[0], row[1]
return None
# Caller must hold Identity.known_destinations_lock
def mark_ratchet_deleted(self, destination_hash):
if self.read_only: return
self._ratchet_upserts.pop(destination_hash, None)
self._ratchet_deletes.add(destination_hash)
self._ratchet_quarantine.pop(destination_hash, None)
def expire_ratchets(self, cutoff):
if self.read_only: return []
with self._flush_lock:
# Drop expired entries in pending as well
expired = set(row[0] for row in self._conn.execute("SELECT destination_hash FROM ratchets WHERE received<?", (cutoff,)))
for destination_hash, entry in list(self._ratchet_upserts.items()):
if entry[1] < cutoff:
expired.add(destination_hash)
self._ratchet_upserts.pop(destination_hash, None)
expired.difference_update(self._ratchet_deletes)
if len(expired) > 0:
self._conn.execute("BEGIN")
for i in range(0, len(expired), 4096):
self._conn.executemany("DELETE FROM ratchets WHERE destination_hash=?", [(h,) for h in list(expired)[i:i+4096]])
self._conn.commit()
return list(expired)
def load_ratchets(self):
cursor = self._conn.execute("SELECT destination_hash, ratchet, received FROM ratchets")
while True:
rows = cursor.fetchmany(4096)
if not rows: break
for row in rows: yield row[0], row[1], row[2]
def import_legacy_ratchets(self, dir_path, expiry_cutoff, known_destinations, ratchet_width=32):
from .vendor import umsgpack
imported = 0
for filename in os.listdir(dir_path):
file_path = os.path.join(dir_path, filename)
try:
if not os.path.isfile(file_path): continue
with open(file_path, "rb") as rf:
ratchet_data = umsgpack.unpackb(rf.read())
destination_hash = bytes.fromhex(filename)
ratchet = ratchet_data["ratchet"]; received = ratchet_data["received"]
if len(ratchet) != ratchet_width: raise ValueError("Invalid ratchet width")
if received < expiry_cutoff: raise ValueError("Ratchet expired")
if destination_hash not in known_destinations: raise ValueError("Destination unknown")
self._ratchet_upserts[destination_hash] = (ratchet, received)
imported += 1
os.unlink(file_path)
except Exception as e:
RNS.log(f"Could not import ratchet for {filename}: {e}", RNS.LOG_WARNING)
try: os.unlink(file_path)
except: pass
pending = self._ratchet_upserts
self._ratchet_upserts = {}
self.apply(set(), set(), pending, set(), known_destinations)
return imported
def import_legacy(self, path):
from .vendor import umsgpack
with open(path, "rb") as file: legacy = umsgpack.load(file)
rows = []
now = time.time()
for destination_hash, entry in legacy.items():
if len(entry) < 5: continue
entry = list(entry)
while len(entry) < 8: entry.append(None if len(entry) in (5, 7) else 0)
if entry[0] == None or entry[0] <= 0: entry[0] = now
rows.append(self.__entry_row(destination_hash, entry))
self._conn.execute("BEGIN")
for i in range(0, len(rows), 4096): self._conn.executemany("INSERT OR REPLACE INTO known_destinations (destination_hash, last_heard, packet_hash, public_key, app_data, used, stamp, stamp_value, lr_cost, identity_hash) VALUES (?,?,?,?,?,?,?,?,?,?)", rows[i:i+4096])
self._conn.commit()
return len(rows)
class AnnounceCache:
def __init__(self, db_path):
import sqlite3
self.db_path = db_path
self._pending = {}
self._flush_lock = threading.Lock()
self._conn = sqlite3.connect(db_path, check_same_thread=False, isolation_level=None, cached_statements=0)
self._conn.execute("PRAGMA auto_vacuum=INCREMENTAL")
self._conn.execute("PRAGMA journal_mode=WAL")
self._conn.execute("PRAGMA synchronous=NORMAL")
self._conn.execute("PRAGMA journal_size_limit=8388608")
self.__create_schema()
def __create_schema(self):
self._conn.execute("""CREATE TABLE IF NOT EXISTS cached_announces (
packet_hash BLOB PRIMARY KEY,
received REAL NOT NULL,
raw BLOB NOT NULL,
interface_reference TEXT)""")
self._conn.execute("PRAGMA user_version = 1")
def store(self, packet_hash, raw, interface_reference):
self._pending[packet_hash] = (time.time(), raw, interface_reference)
def pending_count(self):
return len(self._pending)
def fetch(self, packet_hash):
pending = self._pending.get(packet_hash)
if pending != None: return pending[1], pending[2]
with self._flush_lock:
row = self._conn.execute("SELECT raw, interface_reference FROM cached_announces WHERE packet_hash=?", (packet_hash,)).fetchone()
if row != None: return row[0], row[1]
return None
def flush(self):
with self._flush_lock: return self.__apply_pending()
def __apply_pending(self):
pending = self._pending
self._pending = {}
if len(pending) == 0: return 0
try:
self._conn.execute("BEGIN")
items = list(pending.items())
for i in range(0, len(items), 4096):
self._conn.executemany("INSERT INTO cached_announces VALUES (?,?,?,?) ON CONFLICT(packet_hash) DO UPDATE SET received=excluded.received, raw=excluded.raw, interface_reference=excluded.interface_reference",
[(packet_hash, *entry) for packet_hash, entry in items[i:i+4096]])
self._conn.commit()
except Exception:
# Re-mark everything as pending, so the changes
# are retried on the next flush, then propagate.
self._pending.update(pending)
self._conn.rollback()
raise
return len(pending)
def clean(self, active_hashes):
with self._flush_lock:
self.__apply_pending()
try:
self._conn.execute("BEGIN")
self._conn.execute("DROP TABLE IF EXISTS temp.active_announces")
self._conn.execute("CREATE TEMP TABLE active_announces (packet_hash BLOB PRIMARY KEY)")
active_list = list(active_hashes)
for i in range(0, len(active_list), 4096):
self._conn.executemany("INSERT OR IGNORE INTO active_announces VALUES (?)", [(h,) for h in active_list[i:i+4096]])
changes_before = self._conn.total_changes
self._conn.execute("DELETE FROM cached_announces WHERE packet_hash NOT IN (SELECT packet_hash FROM active_announces)")
removed = self._conn.total_changes - changes_before
self._conn.commit()
except Exception:
self._conn.rollback()
raise
if removed > 0: self._conn.execute("PRAGMA incremental_vacuum")
return removed
def import_legacy(self, dir_path):
from .vendor import umsgpack
imported = 0
for filename in os.listdir(dir_path):
file_path = os.path.join(dir_path, filename)
try:
if not os.path.isfile(file_path): continue
with open(file_path, "rb") as file:
cached_data = umsgpack.unpackb(file.read())
packet_hash = bytes.fromhex(filename)
interface_reference = cached_data[1] if len(cached_data) > 1 else None
self.store(packet_hash, cached_data[0], interface_reference)
imported += 1
os.unlink(file_path)
except Exception:
try: os.unlink(file_path)
except Exception: pass
self.flush()
return imported
def close(self):
if self._conn != None:
self._conn.close()
self._conn = None

File diff suppressed because it is too large Load diff

View file

@ -1,6 +1,6 @@
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -9,16 +9,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -31,9 +23,5 @@
import os
import glob
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
modules = py_modules+pyc_modules+so_modules+pyd_modules
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))
modules = glob.glob(os.path.dirname(__file__)+"/*.py")
__all__ = [ os.path.basename(f)[:-3] for f in modules if not f.endswith('__init__.py')]

View file

@ -1,8 +1,8 @@
#!/usr/bin/env python3
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -11,16 +11,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -33,7 +25,6 @@
import RNS
import argparse
import threading
import shutil
import time
import sys
import os
@ -43,45 +34,17 @@ from RNS._version import __version__
APP_NAME = "rncp"
allow_all = False
allow_fetch = False
allow_overwrite_on_receive = False
fetch_auto_compress = True
fetch_jail = None
save_path = None
show_phy_rates = False
allowed_identity_hashes = []
identity = None
def prepare_identity(identity_path):
global identity
if identity_path == None:
identity_path = RNS.Reticulum.identitypath+"/"+APP_NAME
if os.path.isfile(identity_path):
identity = RNS.Identity.from_file(identity_path)
if identity == None:
RNS.log(f"Could not load identity for rncp. The identity file at \"{identity_path}\" may be corrupt or unreadable.", RNS.LOG_ERROR)
RNS.exit(2)
if identity == None:
RNS.log("No valid saved identity found, creating new...", RNS.LOG_INFO)
identity = RNS.Identity()
identity.to_file(identity_path)
REQ_FETCH_NOT_ALLOWED = 0xF0
es = " "
erase_str = "\33[2K\r"
def listen(configdir, identitypath = None, verbosity = 0, quietness = 0, allowed = [], display_identity = False,
limit = None, disable_auth = None, fetch_allowed = False, no_compress=False,
jail = None, save = None, announce = False, allow_overwrite=False):
global allow_all, allow_fetch, allowed_identity_hashes, fetch_jail, save_path, identity
global fetch_auto_compress, allow_overwrite_on_receive
def listen(configdir, verbosity = 0, quietness = 0, allowed = [], display_identity = False,
limit = None, disable_auth = None, fetch_allowed = False, jail = None, announce = False):
global allow_all, allow_fetch, allowed_identity_hashes, fetch_jail
from tempfile import TemporaryFile
allow_fetch = fetch_allowed
fetch_auto_compress = not no_compress
allow_overwrite_on_receive = allow_overwrite
identity = None
if announce < 0:
announce = False
@ -93,28 +56,21 @@ def listen(configdir, identitypath = None, verbosity = 0, quietness = 0, allowed
fetch_jail = os.path.abspath(os.path.expanduser(jail))
RNS.log("Restricting fetch requests to paths under \""+fetch_jail+"\"", RNS.LOG_VERBOSE)
if save != None:
sp = os.path.abspath(os.path.expanduser(save))
if os.path.isdir(sp):
if os.access(sp, os.W_OK):
save_path = sp
else:
RNS.log("Output directory not writable", RNS.LOG_ERROR)
RNS.exit(4)
else:
RNS.log("Output directory not found", RNS.LOG_ERROR)
RNS.exit(3)
identity_path = RNS.Reticulum.identitypath+"/"+APP_NAME
if os.path.isfile(identity_path):
identity = RNS.Identity.from_file(identity_path)
RNS.log("Saving received files in \""+save_path+"\"", RNS.LOG_VERBOSE)
prepare_identity(identitypath)
if identity == None:
RNS.log("No valid saved identity found, creating new...", RNS.LOG_INFO)
identity = RNS.Identity()
identity.to_file(identity_path)
destination = RNS.Destination(identity, RNS.Destination.IN, RNS.Destination.SINGLE, APP_NAME, "receive")
if display_identity:
print("Identity : "+str(identity))
print("Listening on : "+RNS.prettyhexrep(destination.hash))
RNS.exit(0)
exit(0)
if disable_auth:
allow_all = True
@ -163,26 +119,21 @@ def listen(configdir, identitypath = None, verbosity = 0, quietness = 0, allowed
except Exception as e:
raise ValueError("Invalid destination entered. Check your input.")
except Exception as e:
RNS.log(f"Could not apply allowed identity: {e}", RNS.LOG_ERROR)
RNS.exit(1)
print(str(e))
exit(1)
if len(allowed_identity_hashes) < 1 and not disable_auth:
RNS.log("No allowed identities configured, rncp will not accept any files!", RNS.LOG_WARNING)
print("Warning: No allowed identities configured, rncp will not accept any files!")
def fetch_request(path, data, request_id, link_id, remote_identity, requested_at):
global allow_fetch, fetch_jail, fetch_auto_compress
global allow_fetch, fetch_jail
if not allow_fetch:
return REQ_FETCH_NOT_ALLOWED
file_path = os.path.abspath(os.path.expanduser(data))
if fetch_jail:
if data.startswith(fetch_jail+"/"):
data = data.replace(fetch_jail+"/", "")
file_path = os.path.abspath(os.path.expanduser(f"{fetch_jail}/{data}"))
if not file_path.startswith(fetch_jail+"/"):
RNS.log(f"Disallowing fetch request for {file_path} outside of fetch jail {fetch_jail}", RNS.LOG_WARNING)
if not file_path.startswith(jail):
return REQ_FETCH_NOT_ALLOWED
else:
file_path = os.path.abspath(os.path.expanduser(f"{data}"))
target_link = None
for link in RNS.Transport.active_links:
@ -196,28 +147,29 @@ def listen(configdir, identitypath = None, verbosity = 0, quietness = 0, allowed
if target_link != None:
RNS.log("Sending file "+str(file_path)+" to client", RNS.LOG_VERBOSE)
try:
metadata = {"name": os.path.basename(file_path).encode("utf-8") }
fetch_resource = RNS.Resource(open(file_path, "rb"), target_link, metadata=metadata, auto_compress=fetch_auto_compress)
return True
temp_file = TemporaryFile()
real_file = open(file_path, "rb")
filename_bytes = os.path.basename(file_path).encode("utf-8")
filename_len = len(filename_bytes)
except Exception as e:
RNS.log(f"Could not send file to client. The contained exception was: {e}", RNS.LOG_ERROR)
return False
if filename_len > 0xFFFF:
print("Filename exceeds max size, cannot send")
exit(1)
temp_file.write(filename_len.to_bytes(2, "big"))
temp_file.write(filename_bytes)
temp_file.write(real_file.read())
temp_file.seek(0)
fetch_resource = RNS.Resource(temp_file, target_link)
return True
else:
return None
destination.set_link_established_callback(client_link_established)
if allow_fetch:
if allow_all:
RNS.log("Allowing unauthenticated fetch requests", RNS.LOG_WARNING)
destination.register_request_handler("fetch_file", response_generator=fetch_request, allow=RNS.Destination.ALLOW_ALL)
else:
destination.register_request_handler("fetch_file", response_generator=fetch_request, allow=RNS.Destination.ALLOW_LIST, allowed_list=allowed_identity_hashes)
RNS.log("rncp listening on "+RNS.prettyhexrep(destination.hash), RNS.LOG_INFO)
destination.register_request_handler("fetch_file", response_generator=fetch_request, allow=RNS.Destination.ALLOW_LIST, allowed_list=allowed_identity_hashes)
print("rncp listening on "+RNS.prettyhexrep(destination.hash))
if announce >= 0:
def job():
@ -229,7 +181,8 @@ def listen(configdir, identitypath = None, verbosity = 0, quietness = 0, allowed
threading.Thread(target=job, daemon=True).start()
while True: time.sleep(1)
while True:
time.sleep(1)
def client_link_established(link):
RNS.log("Incoming link established", RNS.LOG_VERBOSE)
@ -271,108 +224,63 @@ def receive_resource_started(resource):
else:
id_str = ""
RNS.log("Starting resource transfer "+RNS.prettyhexrep(resource.hash)+id_str, RNS.LOG_INFO)
print("Starting resource transfer "+RNS.prettyhexrep(resource.hash)+id_str)
def receive_resource_concluded(resource):
global save_path, allow_overwrite_on_receive
if resource.status == RNS.Resource.COMPLETE:
RNS.log(f"Incoming resource {resource} completed", RNS.LOG_INFO)
print(str(resource)+" completed")
if resource.metadata == None:
RNS.log("Invalid data received, ignoring resource", RNS.LOG_WARNING)
return
if resource.total_size > 4:
filename_len = int.from_bytes(resource.data.read(2), "big")
filename = resource.data.read(filename_len).decode("utf-8")
counter = 0
saved_filename = filename
while os.path.isfile(saved_filename):
counter += 1
saved_filename = filename+"."+str(counter)
file = open(saved_filename, "wb")
file.write(resource.data.read())
file.close()
else:
try:
filename = os.path.basename(resource.metadata["name"].decode("utf-8"))
counter = 0
if save_path:
saved_filename = os.path.abspath(os.path.expanduser(save_path+"/"+filename))
if not saved_filename.startswith(save_path+"/"):
RNS.log(f"Invalid save path {saved_filename}, ignoring", RNS.LOG_ERROR)
return
else:
saved_filename = filename
full_save_path = saved_filename
if allow_overwrite_on_receive:
if os.path.isfile(full_save_path):
try: os.unlink(full_save_path)
except Exception as e:
RNS.log(f"Could not overwrite existing file {full_save_path}, renaming instead", RNS.LOG_ERROR)
while os.path.isfile(full_save_path):
counter += 1
full_save_path = saved_filename+"."+str(counter)
shutil.move(resource.data.name, full_save_path)
RNS.log("Saved received file to "+full_save_path, RNS.LOG_NOTICE)
except Exception as e:
RNS.log(f"An error occurred while saving received resource: {e}", RNS.LOG_ERROR)
return
print("Invalid data received, ignoring resource")
else:
RNS.log("Resource failed", RNS.LOG_INFO)
print("Resource failed")
resource_done = False
current_resource = None
stats = []
speed = 0.0
phy_speed = 0.0
phy_got_total = 0
def sender_progress(resource):
stats_max = 32
global current_resource, stats, speed, phy_speed, phy_got_total, resource_done
global current_resource, stats, speed, resource_done
current_resource = resource
now = time.time()
got = current_resource.get_progress()*current_resource.get_data_size()
phy_got = current_resource.get_segment_progress()*current_resource.get_transfer_size()
entry = [now, got, phy_got]
got = current_resource.get_progress()*current_resource.total_size
entry = [now, got]
stats.append(entry)
while len(stats) > stats_max:
stats.pop(0)
span = now - stats[0][0]
if span == 0:
speed = 0
phy_speed = 0
else:
diff = got - stats[0][1]
speed = diff/span
phy_diff = phy_got - stats[0][2]
if phy_diff > 0:
phy_speed = phy_diff/span
# phy_got_total += phy_diff
if resource.status < RNS.Resource.COMPLETE:
resource_done = False
else:
resource_done = True
link = None
def fetch(configdir, identitypath = None, verbosity = 0, quietness = 0, destination = None, file = None, timeout = RNS.Transport.PATH_REQUEST_TIMEOUT, silent=False, phy_rates=False, save=None, allow_overwrite=False):
global current_resource, resource_done, link, speed, show_phy_rates, save_path, allow_overwrite_on_receive, identity
def fetch(configdir, verbosity = 0, quietness = 0, destination = None, file = None, timeout = RNS.Transport.PATH_REQUEST_TIMEOUT, silent=False):
global current_resource, resource_done, link, speed
targetloglevel = 3+verbosity-quietness
show_phy_rates = phy_rates
allow_overwrite_on_receive = allow_overwrite
if save:
sp = os.path.abspath(os.path.expanduser(save))
if os.path.isdir(sp):
if os.access(sp, os.W_OK):
save_path = sp
else:
RNS.log("Output directory not writable", RNS.LOG_ERROR)
RNS.exit(4)
else:
RNS.log("Output directory not found", RNS.LOG_ERROR)
RNS.exit(3)
try:
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
@ -384,25 +292,35 @@ def fetch(configdir, identitypath = None, verbosity = 0, quietness = 0, destinat
raise ValueError("Invalid destination entered. Check your input.")
except Exception as e:
print(str(e))
RNS.exit(1)
exit(1)
reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel)
reticulum.wants_destination_type(f"{APP_NAME}.receive")
identity_path = RNS.Reticulum.identitypath+"/"+APP_NAME
if os.path.isfile(identity_path):
identity = RNS.Identity.from_file(identity_path)
if identity == None:
RNS.log("Could not load identity for rncp. The identity file at \""+str(identity_path)+"\" may be corrupt or unreadable.", RNS.LOG_ERROR)
exit(2)
else:
identity = None
if identity == None:
prepare_identity(identitypath)
RNS.log("No valid saved identity found, creating new...", RNS.LOG_INFO)
identity = RNS.Identity()
identity.to_file(identity_path)
if not RNS.Transport.has_path(destination_hash):
RNS.Transport.request_path(destination_hash)
if silent:
print("Path to "+RNS.prettyhexrep(destination_hash)+" requested")
else:
print("Path to "+RNS.prettyhexrep(destination_hash)+" requested ", end=es)
print("Path to "+RNS.prettyhexrep(destination_hash)+" requested ", end=" ")
sys.stdout.flush()
i = 0
syms = "⢄⢂⢁⡁⡈⡐⡠"
estab_timeout = time.time()+max(timeout, reticulum.get_medium_path_timeout())
estab_timeout = time.time()+timeout
while not RNS.Transport.has_path(destination_hash) and time.time() < estab_timeout:
if not silent:
time.sleep(0.1)
@ -414,13 +332,13 @@ def fetch(configdir, identitypath = None, verbosity = 0, quietness = 0, destinat
if silent:
print("Path not found")
else:
print(f"{erase_str}Path not found")
RNS.exit(1)
print("\r \rPath not found")
exit(1)
else:
if silent:
print("Establishing link with "+RNS.prettyhexrep(destination_hash))
else:
print(f"{erase_str}Establishing link with "+RNS.prettyhexrep(destination_hash)+" ", end=es)
print("\r \rEstablishing link with "+RNS.prettyhexrep(destination_hash)+" ", end=" ")
listener_identity = RNS.Identity.recall(destination_hash)
listener_destination = RNS.Destination(
@ -432,7 +350,6 @@ def fetch(configdir, identitypath = None, verbosity = 0, quietness = 0, destinat
)
link = RNS.Link(listener_destination)
estab_timeout = max(estab_timeout, time.time()+link.establishment_timeout)
while link.status != RNS.Link.ACTIVE and time.time() < estab_timeout:
if not silent:
time.sleep(0.1)
@ -444,13 +361,13 @@ def fetch(configdir, identitypath = None, verbosity = 0, quietness = 0, destinat
if silent:
print("Could not establish link with "+RNS.prettyhexrep(destination_hash))
else:
print(f"{erase_str}Could not establish link with "+RNS.prettyhexrep(destination_hash))
RNS.exit(1)
print("\r \rCould not establish link with "+RNS.prettyhexrep(destination_hash))
exit(1)
else:
if silent:
print("Requesting file from remote...")
else:
print(f"{erase_str}Requesting file from remote ", end=es)
print("\r \rRequesting file from remote ", end=" ")
link.identify(identity)
@ -459,7 +376,6 @@ def fetch(configdir, identitypath = None, verbosity = 0, quietness = 0, destinat
resource_resolved = False
resource_status = "unrequested"
current_resource = None
current_transfer_started = None
def request_response(request_receipt):
nonlocal request_resolved, request_status
if request_receipt.response == False:
@ -479,48 +395,32 @@ def fetch(configdir, identitypath = None, verbosity = 0, quietness = 0, destinat
request_resolved = True
def fetch_resource_started(resource):
nonlocal resource_status, current_transfer_started
nonlocal resource_status
current_resource = resource
current_resource.progress_callback(sender_progress)
resource_status = "started"
if not current_transfer_started: current_transfer_started = time.time()
def fetch_resource_concluded(resource):
nonlocal resource_resolved, resource_status
global save_path, allow_overwrite_on_receive
if resource.status == RNS.Resource.COMPLETE:
if resource.metadata == None:
print("Invalid data received, ignoring resource")
return
if resource.total_size > 4:
filename_len = int.from_bytes(resource.data.read(2), "big")
filename = resource.data.read(filename_len).decode("utf-8")
counter = 0
saved_filename = filename
while os.path.isfile(saved_filename):
counter += 1
saved_filename = filename+"."+str(counter)
file = open(saved_filename, "wb")
file.write(resource.data.read())
file.close()
resource_status = "completed"
else:
try:
filename = os.path.basename(resource.metadata["name"].decode("utf-8"))
counter = 0
if save_path:
saved_filename = os.path.abspath(os.path.expanduser(save_path+"/"+filename))
if not saved_filename.startswith(save_path+"/"):
print(f"Invalid save path {saved_filename}, ignoring")
return
else:
saved_filename = filename
full_save_path = saved_filename
if allow_overwrite_on_receive:
if os.path.isfile(full_save_path):
try: os.unlink(full_save_path)
except Exception as e:
print(f"Could not overwrite existing file {full_save_path}, renaming instead")
while os.path.isfile(full_save_path):
counter += 1
full_save_path = saved_filename+"."+str(counter)
shutil.move(resource.data.name, full_save_path)
except Exception as e:
print(f"An error occurred while saving received resource: {e}")
return
print("Invalid data received, ignoring resource")
resource_status = "invalid_data"
else:
print("Resource failed")
@ -542,31 +442,31 @@ def fetch(configdir, identitypath = None, verbosity = 0, quietness = 0, destinat
i = (i+1)%len(syms)
if request_status == "fetch_not_allowed":
if not silent: print(f"{erase_str}", end="")
if not silent: print("\r \r", end="")
print("Fetch request failed, fetching the file "+str(file)+" was not allowed by the remote")
link.teardown()
time.sleep(0.15)
RNS.exit(0)
exit(0)
elif request_status == "not_found":
if not silent: print(f"{erase_str}", end="")
if not silent: print("\r \r", end="")
print("Fetch request failed, the file "+str(file)+" was not found on the remote")
link.teardown()
time.sleep(0.15)
RNS.exit(0)
exit(0)
elif request_status == "remote_error":
if not silent: print(f"{erase_str}", end="")
if not silent: print("\r \r", end="")
print("Fetch request failed due to an error on the remote system")
link.teardown()
time.sleep(0.15)
RNS.exit(0)
exit(0)
elif request_status == "unknown":
if not silent: print(f"{erase_str}", end="")
if not silent: print("\r \r", end="")
print("Fetch request failed due to an unknown error (probably not authorised)")
link.teardown()
time.sleep(0.15)
RNS.exit(0)
exit(0)
elif request_status == "found":
if not silent: print(f"{erase_str}", end="")
if not silent: print("\r \r", end="")
while not resource_resolved:
if not silent:
@ -574,52 +474,40 @@ def fetch(configdir, identitypath = None, verbosity = 0, quietness = 0, destinat
if current_resource:
prg = current_resource.get_progress()
percent = round(prg * 100.0, 1)
if show_phy_rates:
pss = size_str(phy_speed, "b")
phy_str = f" ({pss}ps at physical layer)"
else:
phy_str = ""
ps = size_str(int(prg*current_resource.total_size))
ts = size_str(current_resource.total_size)
ss = size_str(speed, "b")
stat_str = f"{percent}% - {ps} of {ts} - {ss}ps{phy_str}"
stat_str = str(percent)+"% - " + size_str(int(prg*current_resource.total_size)) + " of " + size_str(current_resource.total_size) + " - " +size_str(speed, "b")+"ps"
if prg != 1.0:
print(f"{erase_str}Transferring file {syms[i]} {stat_str}", end=es)
print("\r \rTransferring file "+syms[i]+" "+stat_str, end=" ")
else:
end_time = time.time(); delta_time = end_time - current_transfer_started
speed = current_resource.total_size/delta_time; dt_str = RNS.prettytime(delta_time)
ss = size_str(speed, "b")
stat_str = f"{percent}% - {ps} of {ts} in {dt_str} - {ss}ps{phy_str}"
print(f"{erase_str}Transfer complete {stat_str}", end=es)
print("\r \rTransfer complete "+stat_str, end=" ")
else:
print(f"{erase_str}Waiting for transfer to start {syms[i]} ", end=es)
print("\r \rWaiting for transfer to start "+syms[i]+" ", end=" ")
sys.stdout.flush()
i = (i+1)%len(syms)
if not current_resource or current_resource.status != RNS.Resource.COMPLETE:
if current_resource.status != RNS.Resource.COMPLETE:
if silent:
print("The transfer failed")
else:
print(f"{erase_str}The transfer failed")
RNS.exit(1)
print("\r \rThe transfer failed")
exit(1)
else:
if silent:
print(str(file)+" fetched from "+RNS.prettyhexrep(destination_hash))
else:
#print("\r \r"+str(file)+" fetched from "+RNS.prettyhexrep(destination_hash))
print("\n"+str(file)+" fetched from "+RNS.prettyhexrep(destination_hash))
link.teardown()
time.sleep(0.1)
RNS.exit(0)
time.sleep(0.15)
exit(0)
link.teardown()
time.sleep(0.1)
RNS.exit(0)
exit(0)
def send(configdir, identitypath = None, verbosity = 0, quietness = 0, destination = None, file = None, timeout = RNS.Transport.PATH_REQUEST_TIMEOUT, silent=False, phy_rates=False, no_compress=False):
global current_resource, resource_done, link, speed, show_phy_rates, phy_got_total, phy_speed, identity
def send(configdir, verbosity = 0, quietness = 0, destination = None, file = None, timeout = RNS.Transport.PATH_REQUEST_TIMEOUT, silent=False):
global current_resource, resource_done, link, speed
from tempfile import TemporaryFile
targetloglevel = 3+verbosity-quietness
show_phy_rates = phy_rates
try:
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
@ -631,35 +519,59 @@ def send(configdir, identitypath = None, verbosity = 0, quietness = 0, destinati
raise ValueError("Invalid destination entered. Check your input.")
except Exception as e:
print(str(e))
RNS.exit(1)
exit(1)
file_path = os.path.expanduser(file)
if not os.path.isfile(file_path):
print("File not found")
sys.exit(1)
exit(1)
metadata = {"name": os.path.basename(file_path).encode("utf-8") }
temp_file = TemporaryFile()
real_file = open(file_path, "rb")
filename_bytes = os.path.basename(file_path).encode("utf-8")
filename_len = len(filename_bytes)
print(f"{erase_str}", end="")
if filename_len > 0xFFFF:
print("Filename exceeds max size, cannot send")
exit(1)
else:
print("Preparing file...", end=" ")
temp_file.write(filename_len.to_bytes(2, "big"))
temp_file.write(filename_bytes)
temp_file.write(real_file.read())
temp_file.seek(0)
print("\r \r", end="")
reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel)
reticulum.wants_destination_type(f"{APP_NAME}.receive")
identity_path = RNS.Reticulum.identitypath+"/"+APP_NAME
if os.path.isfile(identity_path):
identity = RNS.Identity.from_file(identity_path)
if identity == None:
RNS.log("Could not load identity for rncp. The identity file at \""+str(identity_path)+"\" may be corrupt or unreadable.", RNS.LOG_ERROR)
exit(2)
else:
identity = None
if identity == None:
prepare_identity(identitypath)
RNS.log("No valid saved identity found, creating new...", RNS.LOG_INFO)
identity = RNS.Identity()
identity.to_file(identity_path)
if not RNS.Transport.has_path(destination_hash):
RNS.Transport.request_path(destination_hash)
if silent:
print("Path to "+RNS.prettyhexrep(destination_hash)+" requested")
else:
print("Path to "+RNS.prettyhexrep(destination_hash)+" requested ", end=es)
print("Path to "+RNS.prettyhexrep(destination_hash)+" requested ", end=" ")
sys.stdout.flush()
i = 0
syms = "⢄⢂⢁⡁⡈⡐⡠"
estab_timeout = time.time()+max(timeout, reticulum.get_medium_path_timeout())
estab_timeout = time.time()+timeout
while not RNS.Transport.has_path(destination_hash) and time.time() < estab_timeout:
if not silent:
time.sleep(0.1)
@ -671,13 +583,13 @@ def send(configdir, identitypath = None, verbosity = 0, quietness = 0, destinati
if silent:
print("Path not found")
else:
print(f"{erase_str}Path not found")
RNS.exit(1)
print("\r \rPath not found")
exit(1)
else:
if silent:
print("Establishing link with "+RNS.prettyhexrep(destination_hash))
else:
print(f"{erase_str}Establishing link with "+RNS.prettyhexrep(destination_hash)+" ", end=es)
print("\r \rEstablishing link with "+RNS.prettyhexrep(destination_hash)+" ", end=" ")
receiver_identity = RNS.Identity.recall(destination_hash)
receiver_destination = RNS.Destination(
@ -689,7 +601,6 @@ def send(configdir, identitypath = None, verbosity = 0, quietness = 0, destinati
)
link = RNS.Link(receiver_destination)
estab_timeout = max(estab_timeout, time.time()+link.establishment_timeout)
while link.status != RNS.Link.ACTIVE and time.time() < estab_timeout:
if not silent:
time.sleep(0.1)
@ -701,28 +612,22 @@ def send(configdir, identitypath = None, verbosity = 0, quietness = 0, destinati
if silent:
print("Link establishment with "+RNS.prettyhexrep(destination_hash)+" timed out")
else:
print(f"{erase_str}Link establishment with "+RNS.prettyhexrep(destination_hash)+" timed out")
RNS.exit(1)
print("\r \rLink establishment with "+RNS.prettyhexrep(destination_hash)+" timed out")
exit(1)
elif not RNS.Transport.has_path(destination_hash):
if silent:
print("No path found to "+RNS.prettyhexrep(destination_hash))
else:
print(f"{erase_str}No path found to "+RNS.prettyhexrep(destination_hash))
RNS.exit(1)
print("\r \rNo path found to "+RNS.prettyhexrep(destination_hash))
exit(1)
else:
if silent:
print("Advertising file resource...")
else:
print(f"{erase_str}Advertising file resource ", end=es)
print("\r \rAdvertising file resource ", end=" ")
link.identify(identity)
auto_compress = True
if no_compress: auto_compress = False
try: resource = RNS.Resource(open(file_path, "rb"), link, metadata=metadata, callback = sender_progress, progress_callback = sender_progress, auto_compress = auto_compress)
except Exception as e:
print(f"Could not start transfer: {e}")
RNS.exit(1)
resource = RNS.Resource(temp_file, link, callback = sender_progress, progress_callback = sender_progress)
current_resource = resource
while resource.status < RNS.Resource.TRANSFERRING:
@ -732,38 +637,28 @@ def send(configdir, identitypath = None, verbosity = 0, quietness = 0, destinati
sys.stdout.flush()
i = (i+1)%len(syms)
resource_started_at = time.time()
if resource.status > RNS.Resource.COMPLETE:
if silent:
print("File was not accepted by "+RNS.prettyhexrep(destination_hash))
else:
print(f"{erase_str}File was not accepted by "+RNS.prettyhexrep(destination_hash))
RNS.exit(1)
print("\r \rFile was not accepted by "+RNS.prettyhexrep(destination_hash))
exit(1)
else:
if silent:
print("Transferring file...")
else:
print(f"{erase_str}Transferring file ", end=es)
print("\r \rTransferring file ", end=" ")
def progress_update(i, done=False):
time.sleep(0.1)
prg = current_resource.get_progress()
percent = round(prg * 100.0, 1)
if show_phy_rates and not resource_done:
pss = size_str(phy_speed, "b")
phy_str = f" ({pss}ps at physical layer)"
else:
phy_str = ""
es = " "
cs = size_str(int(prg*current_resource.total_size))
ts = size_str(current_resource.total_size)
ss = size_str(speed, "b")
stat_str = f"{percent}% - {cs} of {ts} - {ss}ps{phy_str}"
stat_str = str(percent)+"% - " + size_str(int(prg*current_resource.total_size)) + " of " + size_str(current_resource.total_size) + " - " +size_str(speed, "b")+"ps"
if not done:
print(f"{erase_str}Transferring file "+syms[i]+" "+stat_str, end=es)
print("\r \rTransferring file "+syms[i]+" "+stat_str, end=" ")
else:
print(f"{erase_str}Transfer complete "+stat_str, end=es)
print("\r \rTransfer complete "+stat_str, end=" ")
sys.stdout.flush()
i = (i+1)%len(syms)
return i
@ -772,11 +667,6 @@ def send(configdir, identitypath = None, verbosity = 0, quietness = 0, destinati
if not silent:
i = progress_update(i)
resource_concluded_at = time.time()
transfer_time = resource_concluded_at - resource_started_at
speed = current_resource.total_size/transfer_time
# phy_speed = phy_got_total/transfer_time
if not silent:
i = progress_update(i, done=True)
@ -784,16 +674,19 @@ def send(configdir, identitypath = None, verbosity = 0, quietness = 0, destinati
if silent:
print("The transfer failed")
else:
print(f"{erase_str}The transfer failed")
RNS.exit(1)
print("\r \rThe transfer failed")
exit(1)
else:
if silent:
print(str(file_path)+" copied to "+RNS.prettyhexrep(destination_hash))
else:
# print("\r \r"+str(file_path)+" copied to "+RNS.prettyhexrep(destination_hash))
print("\n"+str(file_path)+" copied to "+RNS.prettyhexrep(destination_hash))
link.teardown()
time.sleep(0.25)
RNS.exit(0)
real_file.close()
temp_file.close()
exit(0)
def main():
try:
@ -805,19 +698,14 @@ def main():
parser.add_argument('-q', '--quiet', action='count', default=0, help="decrease verbosity")
parser.add_argument("-S", '--silent', action='store_true', default=False, help="disable transfer progress output")
parser.add_argument("-l", '--listen', action='store_true', default=False, help="listen for incoming transfer requests")
parser.add_argument("-C", '--no-compress', action='store_true', default=False, help="disable automatic compression")
parser.add_argument("-F", '--allow-fetch', action='store_true', default=False, help="allow authenticated clients to fetch files")
parser.add_argument("-f", '--fetch', action='store_true', default=False, help="fetch file from remote listener instead of sending")
parser.add_argument("-j", "--jail", metavar="path", action="store", default=None, help="restrict fetch requests to specified path", type=str)
parser.add_argument("-s", "--save", metavar="path", action="store", default=None, help="save received files in specified path", type=str)
parser.add_argument('-O', '--overwrite', action='store_true', default=False, help="Allow overwriting received files, instead of adding postfix")
parser.add_argument("-b", action='store', metavar="seconds", default=-1, help="announce interval, 0 to only announce at startup", type=int)
parser.add_argument('-a', metavar="allowed_hash", dest="allowed", action='append', help="allow this identity (or add in ~/.rncp/allowed_identities)", type=str)
parser.add_argument('-a', metavar="allowed_hash", dest="allowed", action='append', help="allow this identity", type=str)
parser.add_argument('-n', '--no-auth', action='store_true', default=False, help="accept requests from anyone")
parser.add_argument('-p', '--print-identity', action='store_true', default=False, help="print identity and destination info and exit")
parser.add_argument('-i', metavar="identity", action='store', dest="identity", default=None, help="path to identity to use", type=str)
parser.add_argument("-w", action="store", metavar="seconds", type=float, help="sender timeout before giving up", default=RNS.Transport.PATH_REQUEST_TIMEOUT)
parser.add_argument('-P', '--phy-rates', action='store_true', default=False, help="display physical layer transfer rates")
# parser.add_argument("--limit", action="store", metavar="files", type=float, help="maximum number of files to accept", default=None)
parser.add_argument("--version", action="version", version="rncp {version}".format(version=__version__))
@ -826,35 +714,27 @@ def main():
if args.listen or args.print_identity:
listen(
configdir = args.config,
identitypath = args.identity,
verbosity=args.verbose,
quietness=args.quiet,
allowed = args.allowed,
fetch_allowed = args.allow_fetch,
no_compress = args.no_compress,
jail = args.jail,
save = args.save,
display_identity=args.print_identity,
# limit=args.limit,
disable_auth=args.no_auth,
announce=args.b,
allow_overwrite=args.overwrite,
)
elif args.fetch:
if args.destination != None and args.file != None:
fetch(
configdir = args.config,
identitypath = args.identity,
verbosity = args.verbose,
quietness = args.quiet,
destination = args.destination,
file = args.file,
timeout = args.w,
silent = args.silent,
phy_rates = args.phy_rates,
save = args.save,
allow_overwrite=args.overwrite,
)
else:
print("")
@ -864,15 +744,12 @@ def main():
elif args.destination != None and args.file != None:
send(
configdir = args.config,
identitypath = args.identity,
verbosity = args.verbose,
quietness = args.quiet,
destination = args.destination,
file = args.file,
timeout = args.w,
silent = args.silent,
phy_rates = args.phy_rates,
no_compress = args.no_compress,
)
else:
@ -882,11 +759,11 @@ def main():
except KeyboardInterrupt:
print("")
if current_resource != None:
current_resource.cancel()
if resource != None:
resource.cancel()
if link != None:
link.teardown()
RNS.exit()
exit()
def size_str(num, suffix='B'):
units = ['','K','M','G','T','P','E','Z']

View file

@ -1,41 +0,0 @@
# Reticulum License
#
# Copyright (c) 2016-2026 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
APP_NAME = "git"
import os
import glob
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
modules = py_modules+pyc_modules+so_modules+pyd_modules
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))

View file

@ -1,720 +0,0 @@
#!/usr/bin/env python3
# Reticulum License
#
# Copyright (c) 2016-2026 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import RNS
import os
import sys
import time
import shutil
import threading
import subprocess
from RNS._version import __version__
from RNS.Utilities.rngit import APP_NAME
from RNS.Utilities.rngit.util import medium_path_timeout
from RNS.vendor.configobj import ConfigObj
from tempfile import TemporaryDirectory
def program_setup(configdir, rnsconfigdir, destination_hexhash, group_name, repo_name):
git_client = ReticulumGitClient(configdir=configdir, rnsconfigdir=rnsconfigdir, destination_hexhash=destination_hexhash,
group_name=group_name, repo_name=repo_name)
if not git_client.ready: sys.exit(1)
else: git_client.run()
def main():
if len(sys.argv) < 3:
print("Usage: git-remote-rns <remote-name> <url>", file=sys.stderr)
sys.exit(1)
url = sys.argv[2]
if not url.startswith("rns://"):
print("Invalid URL scheme. Must be rns://", file=sys.stderr)
sys.exit(1)
try:
parts = url[6:].split("/", 2)
destination_hexhash = parts[0]
group_name = parts[1]
repo_name = parts[2]
except IndexError: print("Invalid URL format. Use rns://<hash>/<group>/<repo>", file=sys.stderr); sys.exit(1)
configdir = os.environ.get("RNGIT_CONFIG", None)
rnsconfigdir = os.environ.get("RNS_CONFIG", None)
program_setup(configdir, rnsconfigdir, destination_hexhash, group_name, repo_name)
exit(0)
class ReticulumGitClient():
PATH_LIST = "/git/list"
PATH_FETCH = "/git/fetch"
PATH_PUSH = "/git/push"
PATH_DELETE = "/git/delete"
RES_DISALLOWED = 0x01
RES_INVALID_REQ = 0x02
RES_NOT_FOUND = 0x03
RES_REMOTE_FAIL = 0xFF
IDX_REPOSITORY = 0x00
IDX_RESULT_CODE = 0x01
REF_BATCH_SIZE = 25
PATH_TIMEOUT = 15
LINK_TIMEOUT = 15
def __init__(self, configdir, rnsconfigdir, destination_hexhash, group_name, repo_name):
# Client state and configuration
self.identity = None
self.userdir = os.path.expanduser("~")
self.config = None
self.ready = False
self.destination_aliases = {}
self.remote_identity = None
self.destination = None
self.link = None
self.link_ready = False
self.link_failed = False
self.link_timeout = self.LINK_TIMEOUT
self.path_timeout = self.PATH_TIMEOUT
self.destination_hexhash = destination_hexhash
self.group_name = group_name
self.repo_name = repo_name
self.repo_path = f"{group_name}/{repo_name}"
self.tmp_dir = TemporaryDirectory()
self.request_event = threading.Event()
self.request_response = None
self.response_metadata = None
self.ref_batch_size = self.REF_BATCH_SIZE
self.remote_refs = {}
self.response_progress = 0
self.previous_progress = 0
self.response_size = None
self.response_transfer_size = None
self.progress_updated_at = None
self.progress_enabled = False
if configdir != None: self.configdir = configdir
else:
if os.path.isdir(self.userdir+"/.config/rngit") and os.path.isfile(self.userdir+"/.config/rngit/config"): self.configdir = self.userdir+"/.rngit/reticulum"
else: self.configdir = self.userdir+"/.rngit"
self.logfile = self.configdir+"/client_log"
self.configpath = self.configdir+"/client_config"
self.identitypath = self.configdir+"/client_identity"
if os.path.isfile(self.configpath):
try: self.config = ConfigObj(self.configpath)
except Exception as e:
RNS.log("Could not parse the configuration at "+self.configpath, RNS.LOG_ERROR)
return
else: self.__create_default_config()
RNS.logfile = self.logfile
try: self.reticulum = RNS.Reticulum(configdir=rnsconfigdir, logdest=RNS.LOG_FILE)
except Exception as e:
print(f"Failed to initialize Reticulum: {e}", file=sys.stderr)
return
self.reticulum.wants_destination_type(f"{APP_NAME}.repositories")
self.__apply_config()
self.ready = True
def __create_default_config(self):
self.config = ConfigObj(__default_rngit_config__)
self.config.filename = self.configpath
if not os.path.isdir(self.configdir): os.makedirs(self.configdir)
self.config.write()
def __apply_config(self):
if "logging" in self.config:
section = self.config["logging"]
if "loglevel" in section: RNS.loglevel = max(RNS.LOG_NONE, min(RNS.LOG_EXTREME, section.as_int("loglevel")))
if "client" in self.config:
section = self.config["client"]
if "ref_batch_size" in section: self.ref_batch_size = max(0, min(1024, section.as_int("ref_batch_size")))
if "aliases" in self.config:
section = self.config["aliases"]
for alias in section:
alias_hexhash = section[alias]
len_ok = len(alias_hexhash) == RNS.Identity.TRUNCATED_HASHLENGTH//8*2
try: alias_hash = bytes.fromhex(alias_hexhash)
except: alias_hash = None
alias_exists = alias in self.destination_aliases
if not len_ok or not alias_hash: continue
if alias_exists: continue
self.destination_aliases[alias] = RNS.hexrep(alias_hash, delimit=False)
if not os.path.isfile(self.identitypath):
identity = RNS.Identity()
identity.to_file(self.identitypath)
RNS.log(f"Client identity generated and persisted to {self.identitypath}", RNS.LOG_VERBOSE)
else:
identity = RNS.Identity.from_file(self.identitypath)
RNS.log(f"Client identity loaded from {self.identitypath}", RNS.LOG_VERBOSE)
if not identity:
RNS.log("Could not initialize client identity.", RNS.LOG_ERROR)
self.ready = False
else: self.identity = identity
self.destination_hexhash = self.__resolve_destination_alias(self.destination_hexhash)
def __resolve_destination_alias(self, alias):
def resolve(alias):
len_match = len(alias) == RNS.Identity.TRUNCATED_HASHLENGTH//8*2
try: hash_bytes = bytes.fromhex(alias)
except: hash_bytes = None
if len_match and hash_bytes: return alias
else: return self.destination_aliases[alias] if alias in self.destination_aliases else alias
resolved = resolve(alias)
return resolved
def abort(self, reason=None, code=255):
if not reason: reason = "Unknown reason"
print(f"git-remote-rns failed: {reason}", file=sys.stderr)
if self.link: self.link.teardown()
sys.exit(code)
def connect_server(self):
try: destination_hash = bytes.fromhex(self.destination_hexhash)
except Exception as e: self.abort(f"Invalid destination hash: {e}")
RNS.log(f"Requesting path to {RNS.prettyhexrep(destination_hash)}", RNS.LOG_DEBUG)
sys.stderr.write(f"Requesting path..."); sys.stderr.flush()
if not RNS.Transport.await_path(destination_hash, timeout=medium_path_timeout(self.path_timeout)):
sys.stderr.write(f"\n"); sys.stderr.flush()
self.abort(f"Could not resolve path to {RNS.prettyhexrep(destination_hash)}")
else:
RNS.log(f"Path to {RNS.prettyhexrep(destination_hash)} resolved", RNS.LOG_DEBUG);
sys.stderr.write(f"\rPath resolved "); sys.stderr.flush()
self.remote_identity = RNS.Identity.recall(destination_hash)
if not self.remote_identity: self.abort("Could not recall remote identity. Is the server announcing?")
sys.stderr.write(f"\rEstablishing link..."); sys.stderr.flush()
self.destination = RNS.Destination(self.remote_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "repositories")
self.link = RNS.Link(self.destination)
self.link.set_link_established_callback(self.link_established)
self.link.set_link_closed_callback(self.link_closed)
def link_established(self, link):
RNS.log(f"Link established, identifying...", RNS.LOG_DEBUG)
sys.stderr.write(f"\rLink established with remote\n"); sys.stderr.flush()
link.identify(self.identity)
self.link_ready = True
def link_closed(self, link):
RNS.log(f"Link was closed", RNS.LOG_DEBUG)
if not self.link_ready: self.link_failed = True
def _on_progress(self, transfer_instance):
if hasattr(transfer_instance, "progress"):
self.response_progress = transfer_instance.progress
self.response_size = transfer_instance.response_size
self.response_transfer_size = transfer_instance.response_transfer_size
elif hasattr(transfer_instance, "get_progress") and callable(transfer_instance.get_progress):
self.response_progress = transfer_instance.get_progress()
self.response_size = transfer_instance.total_size
self.response_transfer_size = transfer_instance.size
now = time.time()
if self.progress_updated_at == None: self.progress_updated_at = now
if now > self.progress_updated_at+1:
td = now - self.progress_updated_at
pd = self.response_progress - self.previous_progress
bd = pd*self.response_size if self.response_size else 0
self.response_speed = (bd/td)*8 if td > 0 else 0
self.previous_progress = self.response_progress
self.progress_updated_at = now
# Report progress to git via stderr
if self.progress_enabled and self.response_size:
percent = round(self.response_progress * 100, 1)
size = self.response_size
rxd = size*self.response_progress
speed_kbps = (self.response_speed / 1000) if hasattr(self, 'response_speed') else 0
sys.stderr.write(f"Transferring: {percent}% ({RNS.prettysize(rxd)}/{RNS.prettysize(size)}) {RNS.prettyspeed(self.response_speed)} \r")
sys.stderr.flush()
################################
# Synchronous Request Wrappers #
################################
def _response_ready(self, request_receipt):
self.request_response = request_receipt.response
self.response_metadata = request_receipt.metadata
if hasattr(self.request_response, "read") and callable(self.request_response.read):
response_path = self.request_response.name
try: self.request_response.close()
except Exception as e: RNS.log(f"Could not close resource file descriptor before retain: {e}", RNS.LOG_ERROR)
base_name = os.path.basename(response_path)
retained_path = os.path.join(self.tmp_dir.name, base_name)
shutil.move(response_path, retained_path)
self.request_response = open(retained_path, "rb")
self.request_event.set()
def _response_failed(self, request_receipt=None):
self.request_response = None
self.request_event.set()
def send_request(self, path, data, timeout=7200):
if not self.link_ready: self.abort("Link not ready for request")
self.request_event.clear()
self.request_response = None
self.response_metadata = None
self.previous_progress = 0
self.progress_updated_at = None
RNS.log(f"Sending request: {path}", RNS.LOG_DEBUG)
request_receipt = self.link.request(path, data, progress_callback=self._on_progress, response_callback=self._response_ready, failed_callback=self._response_failed, timeout=timeout)
if request_receipt.resource: request_receipt.resource.progress_callback(self._on_progress)
self.request_event.wait(timeout=timeout)
if self.request_response is None: self.abort("Request failed or timed out")
RNS.log(f"Got response for: {path}", RNS.LOG_DEBUG)
return self.request_response, self.response_metadata
#############################
# Git Helper Protocol Logic #
#############################
def _detach_stdout(self):
sys.stdout = open(os.devnull, "w")
sys.stderr = open(os.devnull, "w")
def run(self):
try: self.connect_server()
except Exception as e: self.abort(str(e))
timeout = max(self.link_timeout, self.link.establishment_timeout)
while not self.link_ready and not self.link_failed and timeout > 0:
time.sleep(0.5)
timeout -= 0.5
if not self.link_ready: self.abort("Failed to establish link")
self.progress_enabled = False
git_stdin = sys.stdin
git_stdout = sys.stdout
git_stderr = sys.stderr
fetch_queue = []
push_queue = []
while True:
line = git_stdin.readline()
if not line: break
line = line.strip()
if line == "capabilities":
git_stdout.write("list\n")
git_stdout.write("fetch\n")
git_stdout.write("push\n")
git_stdout.write("option\n")
git_stdout.write("\n")
git_stdout.flush()
elif line == "list": self.handle_git_list(git_stdout)
elif line.startswith("list "): self.handle_git_list(git_stdout, for_push=True) # List for push
elif line.startswith("option"):
# Line format: option <name> <value>
parts = line.split(maxsplit=2)
opt_name = parts[1] if len(parts) > 1 else ""
opt_value = parts[2] if len(parts) > 2 else ""
if opt_name == "progress": self.progress_enabled = opt_value.lower() in ("true", "1", "yes"); git_stdout.write("ok\n")
else: git_stdout.write("unsupported\n")
git_stdout.flush()
elif line.startswith("fetch"):
# Line format: fetch <sha> <ref>
parts = line.split()
sha = parts[1]
ref = parts[2]
# Avoid duplicates in the same batch - TODO: Re-evaluate this
if (sha, ref) not in fetch_queue: fetch_queue.append((sha, ref))
push_queue = []
elif line.startswith("push"):
# Line format: push <local_ref>:<remote_ref>
parts = line.split()
refspec = parts[1]
local_ref, remote_ref = refspec.split(":", 1)
push_queue.append((local_ref, remote_ref))
fetch_queue = []
elif line == "": # End of batch
try:
self.process_fetch_queue(fetch_queue, git_stdout, self.progress_enabled, self.ref_batch_size)
self.process_push_queue(push_queue, git_stdout, git_stderr, self.progress_enabled)
fetch_queue = []
push_queue = []
git_stdout.write("\n")
git_stdout.flush()
except BrokenPipeError:
self._detach_stdout()
RNS.log("Git closed connection, exiting", RNS.LOG_DEBUG)
break
else: self.abort(f"Unknown Git command: {line}")
try: sys.stdout.flush()
except BrokenPipeError: pass
if self.link: self.link.teardown()
def handle_git_list(self, git_stdout, for_push=False):
RNS.log("Handle git list" + (" for-push" if for_push else ""), RNS.LOG_DEBUG)
request_data = {self.IDX_REPOSITORY: self.repo_path, "for_push": for_push}
response, metadata = self.send_request(self.PATH_LIST, request_data)
if not response or not isinstance(response, bytes): self.abort("Invalid list response from server")
status_byte = response[0]
payload = response[1:]
if status_byte != 0: self.abort(f"Server refused list: {payload.decode('utf-8', errors='ignore')}")
response_text = payload.decode("utf-8")
self.remote_refs = {}
for line in response_text.split("\n"):
line = line.strip()
if not line: continue
parts = line.split(" ", 1)
if len(parts) == 2:
sha, ref_name = parts
if ref_name == "HEAD": continue
self.remote_refs[ref_name] = sha
git_stdout.write(response_text)
git_stdout.write("\n") # Required to terminate list
git_stdout.flush()
def escape_for_stdout(self, value):
if isinstance(value, bytes): value = value.decode('utf-8', errors='replace')
escaped = '"'
for char in value:
if char == '\\': escaped += '\\\\'
elif char == '"': escaped += '\\"'
elif char == '\n': escaped += '\\n'
elif char == '\t': escaped += '\\t'
elif char == '\r': escaped += '\\r'
elif ord(char) < 32 or ord(char) > 126: escaped += f'\\x{ord(char):02x}'
else: escaped += char
return escaped + '"'
def process_fetch_queue(self, fetch_queue, git_stdout, progress_enabled=False, ref_batch_size=REF_BATCH_SIZE):
import tempfile
import subprocess
if not fetch_queue: return
# Build a global have list from all remote refs that the client already has objects for
have_shas = []
for sha in self.remote_refs.values():
try:
result = subprocess.run(["git", "cat-file", "-t", sha], capture_output=True, check=False)
if result.returncode == 0: have_shas.append(sha)
except Exception as e: RNS.log(f"Could not verify remote SHA {sha} locally: {e}", RNS.LOG_WARNING)
while fetch_queue:
batch = fetch_queue[:ref_batch_size]
fetch_queue = fetch_queue[ref_batch_size:]
refs_list = []
for sha, ref in batch:
ref_entry = {"sha": sha, "ref": ref}
try:
# Attempt to get local ref SHA for incremental bundle generation on remote
result = subprocess.run(["git", "rev-parse", ref], capture_output=True, text=True, check=False)
if result.returncode == 0:
local_sha = result.stdout.strip()
if local_sha != sha: ref_entry["have"] = local_sha
except Exception as e:
RNS.log(f"Could not resolve local SHA for {ref} during fetch enumeration, getting full history for this ref: {e}", RNS.LOG_WARNING)
refs_list.append(ref_entry)
ref_names = [ref for _, ref in batch]
RNS.log(f"Fetching batch of {len(refs_list)} refs: {ref_names} (have {len(have_shas)} common objects)", RNS.LOG_DEBUG)
request_data = { self.IDX_REPOSITORY: self.repo_path, "refs": refs_list }
if have_shas: request_data["have"] = have_shas
response, metadata = self.send_request(self.PATH_FETCH, request_data)
if not response: self.abort(f"No data in fetch response for batch")
if not metadata:
if not isinstance(response, bytes): self.abort(f"Invalid fetch response for batch")
status_byte = response[0]
if status_byte == 0:
RNS.log(f"Server returned empty bundle, all objects already exist locally", RNS.LOG_DEBUG)
continue
else:
error_msg = response[1:].decode('utf-8', errors='ignore')
self.abort(f"Fetch failed for batch: {error_msg}")
else:
if not self.IDX_RESULT_CODE in metadata: self.abort(f"No result metadata on bundle response")
status_byte = metadata[self.IDX_RESULT_CODE]
if status_byte == 0: bundle_path = response.name
else: self.abort(f"Unknown remote state for batch ref fetch")
if progress_enabled:
size = os.stat(bundle_path).st_size
sys.stderr.write(f"Transferring: 100% ({RNS.prettysize(size)}). \n")
sys.stderr.flush()
stderr_arg = sys.stderr if progress_enabled else subprocess.DEVNULL
verify_cmd = ["git", "bundle", "verify", "-q", bundle_path]
verify_result = subprocess.run(verify_cmd, stderr=subprocess.DEVNULL, stdout=subprocess.DEVNULL)
if verify_result.returncode != 0: self.abort(f"Bundle verification failed for batch")
unbundle_cmd = ["git", "bundle", "unbundle"]
if progress_enabled: unbundle_cmd.append("--progress")
unbundle_cmd.append(bundle_path)
unbundle_result = subprocess.run(unbundle_cmd, stderr=stderr_arg, stdout=subprocess.DEVNULL)
if unbundle_result.returncode != 0: self.abort(f"Bundle unbundle failed for batch: Non-zero return code")
def process_push_queue(self, push_queue, git_stdout, git_stderr, progress_enabled=False):
import tempfile
import subprocess
for local_ref, remote_ref in push_queue:
RNS.log(f"Pushing {local_ref} to {remote_ref}", RNS.LOG_DEBUG)
# Handle potential deletions
if not local_ref or local_ref == "":
request_data = { self.IDX_REPOSITORY: self.repo_path, "ref": remote_ref }
response, metadata = self.send_request(self.PATH_DELETE, request_data)
if not response or not isinstance(response, bytes):
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout('No response from server')}\n")
git_stdout.flush()
continue
status_byte = response[0]
if status_byte != 0:
error_msg = response[1:].decode("utf-8", errors="ignore")
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout(error_msg)}\n")
git_stdout.flush()
continue
git_stdout.write(f"ok {remote_ref}\n")
git_stdout.flush()
continue
force = local_ref.startswith("+")
if force: local_ref = local_ref[1:]
stderr_arg = sys.stderr if progress_enabled else subprocess.DEVNULL
# Resolve the SHA that local_ref points to
sha_result = subprocess.run(["git", "rev-parse", local_ref], capture_output=True, text=True, check=False)
if sha_result.returncode != 0:
error_msg = f"Could not resolve local ref {local_ref}"
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout(error_msg)}\n")
git_stdout.flush()
continue
local_sha = sha_result.stdout.strip()
bundle_empty = False
with tempfile.TemporaryDirectory() as tmpdir:
bundle_path = tmpdir + "/push.bundle"
create_cmd = ["git", "bundle", "create", bundle_path, local_ref]
# Exclude all remote ref SHAs that exist locally, so the
# bundle only contains objects the remote doesn't already have
exclude_count = 0
for sha in self.remote_refs.values():
try:
# We need to verify each SHA actually exists locally, since git
# bundle create will fail if a ^<sha> argument references an object
# not present in the local repository.
result = subprocess.run(["git", "cat-file", "-t", sha], capture_output=True, check=False)
if result.returncode == 0:
create_cmd.append(f"^{sha}")
exclude_count += 1
except Exception as e: RNS.log(f"Could not verify remote SHA {sha} locally: {e}", RNS.LOG_WARNING)
RNS.log(f"Excluding {exclude_count}/{len(self.remote_refs)} remote refs for {local_ref}", RNS.LOG_DEBUG)
if progress_enabled: create_cmd.insert(3, "--progress")
create_result = subprocess.run(create_cmd, capture_output=True, text=True, check=False)
if create_result.returncode == 0:
if create_result.stderr:
# git_stderr.write(create_result.stderr)
pass
else:
if "empty bundle" in create_result.stderr.lower():
# All objects reachable from local_ref already exist on
# the remote. In this case, no bundle is needed and we can
# update the ref directly via the operations path instead.
bundle_empty = True
RNS.log(f"Empty bundle for {local_ref}, all objects already on remote", RNS.LOG_DEBUG)
else:
if progress_enabled and create_result.stderr: git_stderr.write(create_result.stderr)
error_msg = "Bundle creation failed"
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout(error_msg)}\n")
git_stdout.flush()
continue
if not bundle_empty:
with open(bundle_path, "rb") as f: bundle_data = f.read()
request_data = { self.IDX_REPOSITORY: self.repo_path, "local_ref": local_ref, "remote_ref": remote_ref,
"force": force, "bundle": bundle_data }
response, metadata = self.send_request(self.PATH_PUSH, request_data)
if not response or not isinstance(response, bytes):
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout('No response from server')}\n")
git_stdout.flush()
continue
status_byte = response[0]
if status_byte != 0:
error_msg = response[1:].decode('utf-8', errors='ignore')
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout(error_msg)}\n")
git_stdout.flush()
continue
# When all reachable objects already exist on the remote, send a
# direct ref update operation instead of a bundle.
if bundle_empty:
operation = {"action": "update_ref", "ref": remote_ref, "sha": local_sha, "force": force}
request_data = { self.IDX_REPOSITORY: self.repo_path,
"operations": [operation] }
response, metadata = self.send_request(self.PATH_PUSH, request_data)
if not response or not isinstance(response, bytes):
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout('No response from server')}\n")
git_stdout.flush()
continue
status_byte = response[0]
if status_byte != 0:
error_msg = response[1:].decode('utf-8', errors='ignore')
git_stdout.write(f"error {remote_ref} {self.escape_for_stdout(error_msg)}\n")
git_stdout.flush()
continue
git_stdout.write(f"ok {remote_ref}\n")
git_stdout.flush()
__default_rngit_config__ = '''# This is the default rngit client config file.
[client]
# You can control the batch size of ref transfers
# using the ref_batch_size directive:
ref_batch_size = 25
[aliases]
# You can define aliases for commonly used destination
# hashes in this section. Each line must be in the format
# aliased_name = DESTINATION_HASH
#
# These hashes are used for resolving remote destinations.
# For rngit node permissions and identity resolution,
# aliases must be defined in ~/.rngit/config.
# my_node = 063d38912bffc850af4a1b8a270a9d85
# bobs_node = 714981d03e41deda0e4468cb274414cc
[logging]
# Valid log levels are 0 through 7:
# 0: Log only critical information
# 1: Log errors and lower log levels
# 2: Log warnings and lower log levels
# 3: Log notices and lower log levels
# 4: Log info and lower (this is the default)
# 5: Verbose logging
# 6: Debug logging
# 7: Extreme logging
loglevel = 4
'''.splitlines()
if __name__ == "__main__": main()

View file

@ -1,324 +0,0 @@
#!/usr/bin/env python3
# Reticulum License
#
# Copyright (c) 2016-2026 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import os
import sys
import RNS
import struct
import base64
import argparse
from RNS.Utilities.rnid import validate_rsg, create_rsg, extract_signed_rsg_data
SSHSIG_MAGIC = b"SSHSIG"
SSHSIG_VERSION = 1
NAMESPACE_GIT = b"git"
RESERVED_EMPTY = b""
HASH_ALGORITHM = b"sha256"
def ssh_string(data): return struct.pack(">I", len(data)) + data
def read_ssh_string(data, offset):
if offset + 4 > len(data): raise ValueError("Not enough data for string length")
length = struct.unpack(">I", data[offset:offset+4])[0]
if offset + 4 + length > len(data): raise ValueError("Not enough data for string content")
return data[offset+4:offset+4+length], offset + 4 + length
def create_ssh_signature(public_key_wire, namespace, reserved, hash_algorithm, signature_data):
# SSHSIG (6 bytes) || version (uint32) || pubkey (ssh-string) || namespace (ssh-string) ||
# reserved (ssh-string) || hash_algorithm (ssh-string) || signature (ssh-string)
sig_blob = SSHSIG_MAGIC
sig_blob += struct.pack(">I", SSHSIG_VERSION)
sig_blob += ssh_string(public_key_wire)
sig_blob += ssh_string(namespace)
sig_blob += ssh_string(reserved)
sig_blob += ssh_string(hash_algorithm)
sig_blob += ssh_string(signature_data)
return sig_blob
def parse_ssh_signature(sig_data):
offset = 0
if not sig_data.startswith(SSHSIG_MAGIC): raise ValueError("Invalid SSH signature: missing SSHSIG magic")
offset += len(SSHSIG_MAGIC)
if offset + 4 > len(sig_data): raise ValueError("Invalid SSH signature: truncated")
version = struct.unpack(">I", sig_data[offset:offset+4])[0]
if version != SSHSIG_VERSION: raise ValueError(f"Unsupported SSH signature version: {version}")
offset += 4
public_key, offset = read_ssh_string(sig_data, offset)
namespace, offset = read_ssh_string(sig_data, offset)
reserved, offset = read_ssh_string(sig_data, offset)
hash_algorithm, offset = read_ssh_string(sig_data, offset)
signature_data, offset = read_ssh_string(sig_data, offset)
return { "version": version,
"public_key": public_key,
"namespace": namespace,
"reserved": reserved,
"hash_algorithm": hash_algorithm,
"signature_data": signature_data }
def armor_ssh_signature(sig_blob):
b64_data = base64.b64encode(sig_blob).decode('ascii')
lines = [b64_data[i:i+70] for i in range(0, len(b64_data), 70)]
result = "-----BEGIN SSH SIGNATURE-----\n"
result += "\n".join(lines) + "\n"
result += "-----END SSH SIGNATURE-----\n"
return result
def unarmor_ssh_signature(armored_data):
lines = armored_data.strip().split('\n')
b64_data = ""
in_sig = False
for line in lines:
if 'BEGIN SSH SIGNATURE' in line: in_sig = True; continue
if 'END SSH SIGNATURE' in line: break
if in_sig: b64_data += line.strip()
if not b64_data: raise ValueError("No signature data found in armored input")
return base64.b64decode(b64_data)
def get_pubkey_wire_format(identity):
return ssh_string(b"ssh-ed25519")+ssh_string(identity.sig_pub_bytes)
def sign(args):
keyfile = args.keyfile
if not keyfile or not os.path.isfile(keyfile):
print(f"Identity file not found: {keyfile}", file=sys.stderr)
return 1
try:
identity = RNS.Identity.from_file(keyfile)
if not identity or not identity.get_private_key():
print("Error: Could not load identity or identity has no private key", file=sys.stderr)
return 1
except Exception as e:
print(f"Error loading identity: {e}", file=sys.stderr)
return 1
if args.file and os.path.isfile(args.file):
with open(args.file, 'rb') as f: message = f.read()
sig_file = args.file + ".sig"
else:
message = sys.stdin.buffer.read()
sig_file = None
try: rsg = create_rsg(identity, message)
except Exception as e:
print(f"Error creating signature: {e}", file=sys.stderr)
return 1
try: ssh_pubkey = get_pubkey_wire_format(identity)
except Exception as e:
print(f"Error converting public key: {e}", file=sys.stderr)
return 1
try:
ssh_sig = create_ssh_signature(public_key_wire=ssh_pubkey, namespace=NAMESPACE_GIT, reserved=RESERVED_EMPTY,
hash_algorithm=HASH_ALGORITHM, signature_data=rsg)
except Exception as e:
print(f"Error creating SSH signature: {e}", file=sys.stderr)
return 1
try: armored = armor_ssh_signature(ssh_sig)
except Exception as e:
print(f"Error armoring signature: {e}", file=sys.stderr)
return 1
if sig_file:
try:
with open(sig_file, 'w') as f: f.write(armored)
except Exception as e:
print(f"Error writing signature file: {e}", file=sys.stderr)
return 1
else: print(armored, end="")
return 0
def find_principals(args):
sigfile = args.sigfile
if not sigfile or not os.path.isfile(sigfile): print("Error: Signature file not found", file=sys.stderr); return 1
try:
with open(sigfile, 'r') as f: armored_sig = f.read()
except Exception as e: print(f"Error reading signature file: {e}", file=sys.stderr); return 1
try: ssh_sig = parse_ssh_signature(unarmor_ssh_signature(armored_sig))
except Exception as e: print(f"Error parsing SSH signature: {e}", file=sys.stderr); return 1
if ssh_sig["namespace"] != NAMESPACE_GIT:
print(f"Error: Namespace mismatch: {ssh_sig['namespace']}", file=sys.stderr)
return 1
rsg = ssh_sig["signature_data"]
try: identity_hash = extract_signed_rsg_data(rsg)["meta"]["signer"]
except Exception as e: print(f"Could not determine signer identity: {e}", file=sys.stderr); return 1
print(RNS.hexrep(identity_hash, delimit=False))
return 0
def check_novalidate(args):
sigfile = args.sigfile
if not sigfile or not os.path.isfile(sigfile): return 1
try:
with open(sigfile, 'r') as f: armored_sig = f.read()
ssh_sig = parse_ssh_signature(unarmor_ssh_signature(armored_sig))
if ssh_sig["namespace"] != NAMESPACE_GIT: return 1
rsg = ssh_sig["signature_data"]
signed_data = extract_signed_rsg_data(rsg)
if not signed_data: return 1
else: return 0
except Exception: return 1
def extract_commit_author(message):
message_lines = message.splitlines()
author = ""
AUTHOR_TARGET = b"author "
for line in message_lines:
if not line.strip(b""): break
elif line.startswith(AUTHOR_TARGET):
try:
spos = line.find(b"<"); epos = line.find(b">")
if spos > len(AUTHOR_TARGET) and epos > spos and epos < len(line)-1:
author = line[spos+1:epos].decode("utf-8")
break
except Exception as e: print(f"Error while determining author from signed commit"); return 1
return author
def extract_commit_committer(message):
message_lines = message.splitlines()
committer = ""
COMMITTER_TARGET = b"committer "
for line in message_lines:
if not line.strip(b""): break
elif line.startswith(COMMITTER_TARGET):
try:
spos = line.find(b"<"); epos = line.find(b">")
if spos > len(COMMITTER_TARGET) and epos > spos and epos < len(line)-1:
committer = line[spos+1:epos].decode("utf-8")
break
except Exception as e: print(f"Error while determining committer from signed commit"); return 1
return committer
def extract_commit_tagger(message):
message_lines = message.splitlines()
tagger = ""
is_tag = False
for line in message_lines:
TAG_TARGET = b"tag "
TAGGER_TARGET = b"tagger "
if not line.strip(b""): break
elif line.startswith(TAG_TARGET): is_tag = True
elif line.startswith(TAGGER_TARGET) and is_tag:
try:
spos = line.find(b"<"); epos = line.find(b">")
if spos > len(TAGGER_TARGET) and epos > spos and epos < len(line)-1:
tagger = line[spos+1:epos].decode("utf-8")
break
except Exception as e: print(f"Error while determining tagger from signed commit"); return 1
return tagger, is_tag
def verify(args):
sigfile = args.sigfile
principal = args.principal
if not sigfile or not os.path.isfile(sigfile): print("Error: Signature file not found", file=sys.stderr); return 1
message = sys.stdin.buffer.read()
try:
with open(sigfile, 'r') as f: armored_sig = f.read()
raw_sig = unarmor_ssh_signature(armored_sig)
ssh_sig = parse_ssh_signature(raw_sig)
except Exception as e: print(f"Error parsing signature: {e}", file=sys.stderr); return 1
author = extract_commit_author(message)
committer = extract_commit_committer(message)
tagger, is_tag = extract_commit_tagger(message)
if ssh_sig["namespace"] != NAMESPACE_GIT: print(f"Invalid commit signature namespace", file=sys.stderr); return 1
rsg = ssh_sig["signature_data"]
valid, signed_data, signing_identity = validate_rsg(rsg, message)
if not valid: print(f"Invalid signature", file=sys.stderr); return 1
if is_tag: author = tagger
signer_hash = RNS.hexrep(signing_identity.hash, delimit=False)
if not author == signer_hash: print(f"Commit not signed by author <{author}>"); return 1
if principal:
if principal != signer_hash: print(f"Principal mismatch", file=sys.stderr); return 1
print(f"Good \"git\" signature for commit, signed with Reticulum Identity key <{signer_hash}>")
return 0
def main():
parser = argparse.ArgumentParser(description="Git commit signer and validator")
parser.add_argument("-Y", dest="op", required=True, choices=["sign", "find-principals", "check-novalidate", "verify"], help="Operation to perform")
parser.add_argument("-n", dest="namespace", default="git", help="Namespace")
parser.add_argument("-f", dest="keyfile", help="Key file (for signing) or allowed signers file (for verification)")
parser.add_argument("-I", dest="principal", help="Principal identity (for verification)")
parser.add_argument("-s", dest="sigfile", help="Signature file")
parser.add_argument("file", nargs="?", help="File to sign (for signing)")
parser.add_argument("-O", dest="ssh_options", action="append", default=[], help="SSH options (for git compatibility, ignored)")
args, unknown = parser.parse_known_args()
for arg in unknown:
if arg.startswith('-O'): continue # TODO: Add options for time validation
else:
print(f"Error: Unknown argument: {arg}", file=sys.stderr)
sys.exit(1)
if args.op == "sign": return sign(args)
elif args.op == "find-principals": return find_principals(args)
elif args.op == "check-novalidate": return check_novalidate(args)
elif args.op == "verify": return verify(args)
else:
print(f"Error: Unknown operation: {args.op}", file=sys.stderr)
return 1
if __name__ == "__main__": sys.exit(main())

View file

@ -1,412 +0,0 @@
# Reticulum License
#
# Copyright (c) 2016-2026 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import os
import io
import RNS
class SyntaxHighlighter:
def __init__(self, theme=None):
self.pygments_available = False
self.pygments = None
self._lexer_cache = {}
self._check_pygments()
self.theme = theme or self._get_default_theme()
def _get_default_theme(self):
return {
# Control flow - warm coral-red
"keyword": "ff7b72",
"keyword_constant": "ff7b72",
"keyword_control": "ff7b72",
"keyword_declaration": "ff7b72",
# Function definitions - bright sky blue
"function_def": "79c0ff",
"function_magic": "ff7b72",
# Function calls - soft lavender
"function_call": "d2a8ff",
"function_builtin": "ffa657", # amber
# Class definitions - fresh mint green
"class_def": "7ee787",
"class_ref": "56d364", # muted when referenced
# Instance context - soft pink
"self": "ff9bce",
"cls": "ff9bce",
# Data literals - cool, calm ice blue
"string": "a5d6ff",
"string_quoted": "a5d6ff",
"string_doc": "8b949e", # docstrings - like comments
"string_interpol": "ffd700", # f-string braces - gold
"string_escape": "ffea00", # escape sequences - bright yellow
# Numbers - same as function def
"number": "79c0ff",
"number_float": "79c0ff",
"number_integer": "79c0ff",
"number_hex": "79c0ff",
# Comments - muted gray
"comment": "8b949e",
"comment_doc": "8b949e",
"comment_preproc": "ff7b72", # preprocessor directives
# Operators - distinct pink/red for visibility
"operator": "ff7b72", # General operators - coral
"operator_arithmetic": "ff7b72", # +, -, *, /, etc.
"operator_comparison": "ff7b72", # ==, !=, <, >, etc.
"operator_assignment": "ff7b72", # =, +=, -=, etc.
"operator_word": "ff7b72", # and, or, not, in, is
"operator_dot": "c9d1d9", # . - subtle for attribute access
# Punctuation - neutral
"punctuation": "b4b4b4",
"punctuation_brace": "b4b4b4", # [, ], {, }
"punctuation_paren": "b4b4b4", # (, )
"punctuation_colon": "b4b4b4", # :, ;
"punctuation_comma": "8b949e", # , - slightly dimmed
# Decorators - burnt orange
"decorator": "f0883e",
# Constants - same as keywords
"constant": "ff7b72",
"constant_builtin": "ff7b72", # True, False, None
# Type hints and annotations - amber
"type_hint": "ffa657",
"type_builtin": "ffa657",
# Exception handling - alert red
"exception": "f85149",
"exception_builtin": "f85149",
# Names and attributes - near-white for readability
"name": "e6edf3",
"attribute": "e6edf3",
"attribute_call": "d2a8ff", # Function/method calls after dot - lavender
"variable": "e6edf3",
"parameter": "e6edf3",
# Namespaces and modules
"namespace": "7ee787",
"module": "a5d6ff",
# Generic tokens
"generic_heading": "c9d1d9",
"generic_subheading": "c9d1d9",
"generic_prompt": "8b949e",
"generic_error": "f85149",
"generic_deleted": "f85149",
"generic_inserted": "7ee787",
"generic_output": "e6edf3",
# Text and whitespace - no color (None means no color tag)
"text": None,
"whitespace": None,
}
def _check_pygments(self):
try:
import pygments
from pygments.lexers import get_lexer_for_filename, guess_lexer, get_lexer_by_name
from pygments.formatter import Formatter
from pygments.token import Token
self.pygments = pygments
self.pygments_available = True
RNS.log("Pygments syntax highlighting available", RNS.LOG_DEBUG)
except ImportError:
self.pygments_available = False
RNS.log("Pygments not available, using plain text rendering", RNS.LOG_DEBUG)
def highlight(self, content, filename=None, language=None):
if not content: return self._plain_text(content)
if self.pygments_available:
try:
highlighted = self._highlight_pygments(content, filename, language)
# Fix pygments insisting on trailing newlines
if highlighted.endswith("\n") and not content.endswith("\n"): highlighted = highlighted[:-1]
return highlighted
except Exception as e:
RNS.log(f"Pygments highlighting failed, falling back: {e}", RNS.LOG_WARNING)
return self._plain_text(content).replace("\\", "\\\\")
# TODO: Implement Python tokenize fallback for .py files.
# For now, route to plain text
if filename and filename.endswith(".py"):
return self._plain_text(content).replace("\\", "\\\\")
# Universal fallback
return self._plain_text(content).replace("\\", "\\\\")
def _highlight_pygments(self, content, filename=None, language=None):
from pygments.lexers import get_lexer_for_filename, guess_lexer, get_lexer_by_name
from pygments.util import ClassNotFound
lexer = None
if language:
if language == "env": language = "bash"
if language == "environment": language = "bash"
try: lexer = get_lexer_by_name(language)
except ClassNotFound: pass
if lexer is None and filename:
try: lexer = get_lexer_for_filename(filename)
except ClassNotFound: pass
if lexer is None:
try:
if len(content) > 20: lexer = guess_lexer(content)
except ClassNotFound: pass
if lexer is None: return self._plain_text(content)
formatter = MicronFormatter(theme=self.theme)
result = self.pygments.highlight(content, lexer, formatter)
return result
def _plain_text(self, content):
escaped = self._escape_micron(content)
return f"`=\n{escaped}\n`="
@staticmethod
def _escape_micron(text): return text.replace("`", "\\`")
class MicronFormatter:
def __init__(self, theme, **options):
self.theme = theme
self.options = options
def format(self, tokensource, outfile):
output_parts = []
prev_was_dot = False
last_ended_with_break = True
for ttype, value in tokensource:
is_dot = (str(ttype) == "Token.Operator" and value == ".")
ends_with_break = value.endswith("\n")
# If previous token was a dot and this is a Name, treat as attribute/function call
# TODO: Improve this if we can check next token as parantheses or something.
if prev_was_dot and str(ttype).startswith("Token.Name") and value:
color = self._get_color_from_key("attribute_call")
if color:
escaped = self._escape_value(value)
output_parts.append(f"`FT{color}{escaped}`f")
else:
output_parts.append(self._escape_value(value))
else:
color_key = self._get_color_key_for_token(ttype)
color = self._get_color_from_key(color_key)
if color and value:
escaped = self._escape_value(value)
if escaped.startswith("\n"): ilb = "\n"; escaped = escaped[1:]
else: ilb = ""
if escaped.endswith("\n"): tlb = "\n"; escaped = escaped[:-1]
else: tlb = ""
if len(escaped): output = f"{ilb}`FT{color}{escaped}`f{tlb}"
else: output = f"{ilb}{tlb}"
output_parts.append(output)
else:
escaped = self._escape_value(value)
if "\n" in escaped:
parts = []
splitl = escaped.splitlines()
if len(splitl) > 1:
for line in splitl:
if line.startswith("-"): l = f"\\{line}"
elif line.startswith(">"): l = f"\\{line}"
elif line.startswith("<"): l = f"\\{line}"
else: l = line
parts.append(l)
trmpart = "\n" if escaped.endswith("\n") else ""
escaped = "\n".join(parts)+trmpart
elif last_ended_with_break:
if escaped.startswith("-"): escaped = f"\\{escaped}"
elif escaped.startswith(">"): escaped = f"\\{escaped}"
elif escaped.startswith("<"): escaped = f"\\{escaped}"
output_parts.append(escaped)
prev_was_dot = is_dot
last_ended_with_break = ends_with_break
output = "".join(output_parts)
outfile.write(output)
def _get_color_key_for_token(self, ttype):
token_parts = []
current = ttype
while current:
token_parts.insert(0, current[0] if isinstance(current, tuple) else str(current).split(".")[-1])
current = current.parent if hasattr(current, "parent") else None
token_str = ".".join(["Token"] + token_parts[1:] if len(token_parts) > 1 else token_parts)
current_type = ttype
while current_type:
token_key = str(current_type)
if token_key in granular_token_map: return granular_token_map[token_key]
# Move to parent
current_type = current_type.parent if hasattr(current_type, "parent") else None
return None
def _get_color_from_key(self, color_key):
if color_key and color_key in self.theme: return self.theme[color_key]
return None
@staticmethod
def _escape_value(value):
return value.replace("\\", "\\\\").replace("`", "\\`")
# Required by Pygments formatter API, returns None for Micron
def get_style_defs(self, arg=None): return None
# Convenience function for direct use
def highlight_code(content: str, filename: str = None, language: str = None, theme=None) -> str:
highlighter = SyntaxHighlighter(theme=theme)
return highlighter.highlight(content, filename, language)
granular_token_map = {
# Keywords with semantic distinction
"Token.Keyword": "keyword",
"Token.Keyword.Constant": "keyword_constant",
"Token.Keyword.Declaration": "keyword_declaration",
"Token.Keyword.Namespace": "keyword_control",
"Token.Keyword.Pseudo": "keyword_control",
"Token.Keyword.Reserved": "keyword_control",
"Token.Keyword.Type": "type_builtin",
# Names - functions with definition vs call distinction
"Token.Name.Function": "function_call",
"Token.Name.Function.Magic": "function_magic",
"Token.Name.Class": "class_ref",
"Token.Name.Builtin": "function_builtin",
"Token.Name.Builtin.Pseudo": "constant_builtin",
"Token.Name.Exception": "exception_builtin",
"Token.Name.Decorator": "decorator",
"Token.Name.Namespace": "namespace",
"Token.Name.Attribute": "attribute",
"Token.Name.Variable": "variable",
"Token.Name.Variable.Magic": "function_magic",
"Token.Name.Other": "name",
"Token.Name": "name",
"Token.Name.Tag": "keyword", # HTML/XML tags
"Token.Name.Constant": "constant",
"Token.Name.Label": "name",
"Token.Name.Entity": "name",
# Literals - strings with detailed handling
"Token.Literal.String": "string",
"Token.Literal.String.Affix": "string", # f, r, b prefixes
"Token.Literal.String.Backtick": "string",
"Token.Literal.String.Char": "string",
"Token.Literal.String.Delimiter": "string",
"Token.Literal.String.Doc": "string_doc",
"Token.Literal.String.Double": "string_quoted",
"Token.Literal.String.Escape": "string_escape",
"Token.Literal.String.Heredoc": "string",
"Token.Literal.String.Interpol": "string_interpol",
"Token.Literal.String.Other": "string",
"Token.Literal.String.Regex": "string",
"Token.Literal.String.Single": "string_quoted",
"Token.Literal.String.Symbol": "string",
# Numbers
"Token.Literal.Number": "number",
"Token.Literal.Number.Bin": "number",
"Token.Literal.Number.Float": "number_float",
"Token.Literal.Number.Hex": "number_hex",
"Token.Literal.Number.Integer": "number_integer",
"Token.Literal.Number.Integer.Long": "number_integer",
"Token.Literal.Number.Oct": "number",
"Token.Literal": "string",
"Token.Literal.Date": "string",
# Operators - all operators get distinct coloring
"Token.Operator": "operator",
"Token.Operator.Word": "operator_word",
"Token.Operator.Comparison": "operator_comparison",
"Token.Operator.Assignment": "operator_assignment",
"Token.Operator.Arithmetic": "operator_arithmetic",
# Punctuation - braces, parens, colons, commas
"Token.Punctuation": "punctuation",
"Token.Punctuation.Marker": "punctuation",
"Token.Punctuation.Brace": "punctuation_brace",
"Token.Punctuation.Bracket": "punctuation_brace",
"Token.Punctuation.Parenthesis": "punctuation_paren",
"Token.Punctuation.Colon": "punctuation_colon",
"Token.Punctuation.Comma": "punctuation_comma",
# Comments
"Token.Comment": "comment",
"Token.Comment.Hashbang": "comment",
"Token.Comment.Multiline": "comment_doc",
"Token.Comment.Preproc": "comment_preproc",
"Token.Comment.Single": "comment",
"Token.Comment.Special": "comment",
# Generic tokens
"Token.Generic.Deleted": "generic_deleted",
"Token.Generic.Emph": "text",
"Token.Generic.Error": "generic_error",
"Token.Generic.Heading": "generic_heading",
"Token.Generic.Inserted": "generic_inserted",
"Token.Generic.Output": "generic_output",
"Token.Generic.Prompt": "generic_prompt",
"Token.Generic.Strong": "text",
"Token.Generic.Subheading": "generic_subheading",
"Token.Generic.Traceback": "generic_error",
"Token.Generic": "text",
# Text and whitespace
"Token.Text": "text",
"Token.Text.Whitespace": "whitespace",
}

View file

@ -1,40 +0,0 @@
# Reticulum License
#
# Copyright (c) 2016-2026 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import sys
from RNS.Utilities.rngit import client, server
if __name__ == "__main__":
cmd = sys.argv[0]
if cmd == "rngit": ec = server.main()
elif cmd == "git-remote-rns": ec = client.main()
else: raise NotImplementedError(f"The {cmd} executable entrypoint is not yet implemented in rngit")
sys.exit(ec)

View file

@ -1,239 +0,0 @@
# Reticulum License
#
# Copyright (c) 2016-2026 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import os
import time
import shutil
import subprocess
import tempfile
import RNS
CONVERSION_TIMEOUT = 8
# WebP encoding backends, in preference order. All supported
# encoder families handle the common still formats (PNG, JPEG,
# TIFF, BMP, GIF, WebP).
#
# Environment overrides:
# RNGIT_MEDIA_BACKEND force a specific backend
BACKENDS = ( ("magick", ["magick", "-", "webp:-"]),
("convert", ["convert", "-", "webp:-"]),
("gm", ["gm", "convert", "-", "webp:-"]),
("ffmpeg", ["ffmpeg", "-y", "-loglevel", "error", "-i", "-", "-f", "webp", "pipe:1"]),
("avconv", ["avconv", "-y", "-loglevel", "error", "-i", "-", "-f", "webp", "pipe:1"]) )
_ENV_BACKEND = os.environ.get("RNGIT_MEDIA_BACKEND")
_winner = None
_no_backend_logged = False
def available_backends():
out = []
for name, argv in BACKENDS: out.append((name, shutil.which(argv[0]) is not None))
return out
def _selected_backend():
global _winner
if _ENV_BACKEND:
for name, argv in BACKENDS:
if name == _ENV_BACKEND and shutil.which(argv[0]) is not None:
return (name, argv)
return None
order = list(BACKENDS)
if _winner is not None: order.sort(key=lambda backend: backend[0] != _winner)
for name, argv in order:
if shutil.which(argv[0]) is not None:
_winner = name
return (name, argv)
return None
def _terminate(proc):
try: proc.kill()
except Exception: pass
try: proc.wait()
except Exception: pass
def _stderr_tail(proc, limit=1024):
try:
data = proc.stderr.read(limit)
return data.decode("utf-8", "replace").strip()
except Exception: return ""
def _webp_info(data):
if len(data) < 30 or data[:4] != b"RIFF" or data[8:12] != b"WEBP": return None
fourcc = data[12:16]
if fourcc == b"VP8X":
width = int.from_bytes(data[24:27], "little") + 1
height = int.from_bytes(data[27:30], "little") + 1
elif fourcc == b"VP8 ":
width = int.from_bytes(data[26:28], "little") & 0x3FFF
height = int.from_bytes(data[28:30], "little") & 0x3FFF
elif fourcc == b"VP8L":
bits = int.from_bytes(data[21:25], "little")
width = (bits & 0x3FFF) + 1
height = ((bits >> 14) & 0x3FFF) + 1
else: return None
if width > 0 and height > 0: return (width, height)
return None
def _valid_webp(path):
try:
with open(path, "rb") as fh: return _webp_info(fh.read(30)) is not None
except Exception: return False
def _unlink_output(path):
try: os.unlink(path)
except Exception: pass
def _configured_backend(quality=None, max_dimension=None):
global _no_backend_logged
backend = _selected_backend()
if backend is None:
if not _no_backend_logged:
_no_backend_logged = True
RNS.log("No WebP encoding backend available for media conversion. You can Install ImageMagick or ffmpeg to enable image conversion.", RNS.LOG_WARNING)
return None
backend_name, encoder_argv = backend
_no_backend_logged = False
quality_arg = None
dimension_arg = None
if quality is not None:
try: quality_arg = max(1, min(100, int(quality)))
except (ValueError, TypeError): pass
if max_dimension is not None:
try: dimension_arg = int(max_dimension)
except (ValueError, TypeError): pass
if dimension_arg is not None and dimension_arg < 1: dimension_arg = None
encoder_argv = list(encoder_argv)
if backend_name in ("magick", "convert", "gm"):
options = []
if quality_arg is not None: options += ["-quality", str(quality_arg)]
if dimension_arg is not None: options += ["-resize", f"{dimension_arg}x{dimension_arg}>"]
encoder_argv = encoder_argv[:-1] + options + encoder_argv[-1:]
elif backend_name in ("ffmpeg", "avconv"):
options = []
if quality_arg is not None: options += ["-quality", str(quality_arg)]
if dimension_arg is not None: options += ["-vf", f"scale='min(iw,{dimension_arg})':'min(ih,{dimension_arg})':force_original_aspect_ratio=decrease"]
try: format_index = encoder_argv.index("-f")
except ValueError: format_index = len(encoder_argv)
encoder_argv = encoder_argv[:format_index] + options + encoder_argv[format_index:]
return backend_name, encoder_argv
def _await(backend_name, encoder_proc, input_proc=None, timeout=None):
if timeout is None: timeout = CONVERSION_TIMEOUT
deadline = time.time() + timeout
try:
encoder_proc.wait(timeout=max(0.0, deadline - time.time()))
if input_proc is not None: input_proc.wait(timeout=max(0.0, deadline - time.time()))
except subprocess.TimeoutExpired:
_terminate(input_proc)
_terminate(encoder_proc)
RNS.log(f"Media conversion via {backend_name} timed out after {timeout} seconds", RNS.LOG_WARNING)
return False
if encoder_proc.returncode != 0:
detail = ""
encoder_tail = _stderr_tail(encoder_proc)
input_tail = _stderr_tail(input_proc) if input_proc is not None else ""
if encoder_tail: detail += f" Encoder: {encoder_tail}"
if input_tail: detail += f" Input: {input_tail}"
RNS.log(f"Media conversion via {backend_name} failed.{detail}", RNS.LOG_WARNING)
return False
return True
def convert_to_webp(input_argv, output_path, cwd=None, timeout=None, quality=None, max_dimension=None):
configured = _configured_backend(quality=quality, max_dimension=max_dimension)
if configured is None: return False
backend_name, encoder_argv = configured
try:
with open(output_path, "wb") as output_fh:
input_proc = subprocess.Popen(input_argv, cwd=cwd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
encoder_proc = subprocess.Popen(encoder_argv, stdin=input_proc.stdout, stdout=output_fh, stderr=subprocess.PIPE)
# The parent must not retain a copy of the input pipe's write
# end, or the input process would never see EOF.
input_proc.stdout.close()
if not _await(backend_name, encoder_proc, input_proc=input_proc, timeout=timeout): return False
if not _valid_webp(output_path):
RNS.log(f"Media conversion via {backend_name} produced invalid WebP output", RNS.LOG_WARNING)
return False
RNS.log(f"Media converted to WebP with {backend_name}", RNS.LOG_DEBUG)
return True
except Exception as e:
RNS.log(f"Error during media conversion: {e}", RNS.LOG_WARNING)
return False
def convert_file_to_webp(source_path, quality=85, max_dimension=None, timeout=None):
configured = _configured_backend(quality=quality, max_dimension=max_dimension)
if configured is None: return False
backend_name, encoder_argv = configured
tmp_path = None
success = False
try:
if not os.path.isfile(source_path):
RNS.log(f"Cannot convert media: source file does not exist: {source_path}", RNS.LOG_WARNING)
return False
fd, tmp_path = tempfile.mkstemp(prefix="rns_media_", suffix=".webp")
os.close(fd)
with open(source_path, "rb") as input_fh, open(tmp_path, "wb") as output_fh:
encoder_proc = subprocess.Popen(encoder_argv, stdin=input_fh, stdout=output_fh, stderr=subprocess.PIPE)
if not _await(backend_name, encoder_proc, timeout=timeout): return False
if not _valid_webp(tmp_path):
RNS.log(f"Invalid media conversion output from {backend_name}", RNS.LOG_WARNING)
return False
success = True
RNS.log(f"Media converted to WebP with {backend_name}", RNS.LOG_DEBUG)
return tmp_path
except Exception as e:
RNS.log(f"Error during media conversion: {e}", RNS.LOG_WARNING)
return False
finally:
if not success and tmp_path is not None: _unlink_output(tmp_path)

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -1,819 +0,0 @@
# Reticulum License
#
# Copyright (c) 2016-2026 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import re
import RNS
# Validate ref names according to https://git-scm.com/docs/git-check-ref-format
# This may be a bit overkill, since git validates names as well, but why not.
def san_ref(ref):
if ref.startswith("-"): return None
if ref.startswith("/"): return None
if ref.endswith("/"): return None
if ref.endswith("."): return None
if " " in ref: return None
if not "/" in ref: return None
if ".." in ref: return None
if "/." in ref: return None
if "//" in ref: return None
if "\\" in ref: return None
for comp in ref.split("/"):
if comp.endswith(".lock"): return None
if not all(ord(c) >= 40 for c in ref): return None # Any control character
if "\x7f" in ref: return None # ASCII DEL (177)
if "~" in ref: return None
if "^" in ref: return None
if ":" in ref: return None
if "?" in ref: return None
if "*" in ref: return None
if "[" in ref: return None
if "@{" in ref: return None
if "@" == ref: return None
return ref
def san_refs(refs):
if not type(refs) == list: return None
for ref in refs:
if not san_ref(ref): return None
return refs
# Git SHA format validation
def san_sha(sha):
if len(sha) < 40: return None
try: bytes.fromhex(sha)
except: return None
return sha
class MarkdownToMicron:
BOLD = "`!"
BOLD_END = "`!"
ITALIC = "`*"
ITALIC_END = "`*"
UNDERLINE = "`_"
UNDERLINE_END = "`_"
CODE_BG = "`BT282828"
CODE_BG_INLINE = "`BT383838"
CODE_FG = "`Fddd"
CODE_RESET = "`f`b"
LITERAL_START = "`="
LITERAL_END = "`="
BULLET = "•"
# Regex patterns for markdown elements
HEADER_RE = re.compile(r'^(#{1,6})\s+(.+)$')
CODE_FENCE_RE = re.compile(r'^(\s*)```(.*)$')
HORIZONTAL_RULE_RE = re.compile(r'^(\s*)(---+|===+|\*\*\*+|___+)\s*$')
UNORDERED_LIST_RE = re.compile(r'^(\s*)([-*+])\s+(.+)$')
# Table patterns
TABLE_ROW_RE = re.compile(r'^\s*\|?(.+?)\|?\s*$')
TABLE_SEP_RE = re.compile(r'^\s*\|?(?:\s*:?-+:?\s*\|)+\s*$')
# Quote pattern
QUOTE_RE = re.compile(r'^>\s?(.*)$')
# Inline patterns (processed in order of specificity)
LINK_RE = re.compile(r'\[([^\]]+)\]\(([^)]+)\)')
INLINE_CODE_RE = re.compile(r'`([^`]+)`')
BOLD_RE = re.compile(r'\*\*(.+?)\*\*|__(.+?)__')
ITALIC_RE = re.compile(r'\*(.+?)\*|_(.+?)_')
TABLE_H = "─"
TABLE_V = "│"
TABLE_TL = "┌"
TABLE_TR = "┐"
TABLE_BL = "└"
TABLE_BR = "┘"
TABLE_ML = "├"
TABLE_MR = "┤"
TABLE_TM = "┬"
TABLE_BM = "┴"
TABLE_MM = "┼"
TABLE_MIN_COL_WIDTH = 3
def __init__(self, max_width=100, syntax_highlighter=None, url_scope=None):
self.max_width = max_width
self.local_url_scope = url_scope or ":/page/"
self.__local_url_scope = self.local_url_scope
self.syntax_highlighter = syntax_highlighter
self.wcwidth = None
self.bold_links = True
self.underline_links = True
self.link_color = None
try:
import wcwidth
self.wcwidth = wcwidth
except: RNS.log(f"The wcwidth module is unavailable, display width calculations for some glyphs will be incorrect", RNS.LOG_WARNING)
def set_url_scope(self, url_scope): self.local_url_scope = url_scope
def restore_url_scope(self, url_scope): self.local_url_scope = self.__local_url_scope
def display_width(self, text):
if not self.wcwidth: return len(text)
else:
# wcswidth returns -1 for non-printable strings,
# fallback to len in this case
w = self.wcwidth.wcswidth(text)
return w if w is not None and w >= 0 else len(text)
def format_block(self, text, url_scope=None):
# text = text.replace("\\", "\\\\") # Now handled in format_line instead
lines = text.split('\n')
result_lines = []
in_code_block = False
code_block_lang = None
code_buffer = []
in_table = False
table_buffer = []
in_quote = False
quote_buffer = []
def flush_quote_buffer():
nonlocal result_lines, quote_buffer, in_quote
if not quote_buffer:
in_quote = False
return
para = " ".join(quote_buffer)
formatted = self._format_inline(para)
effective_width = self.max_width - 3
if effective_width < 1: effective_width = 1
wrapped_lines = self._wrap_text(formatted, effective_width)
for wrapped_line in wrapped_lines: result_lines.append(f" │ {wrapped_line}")
quote_buffer = []
in_quote = False
def flush_table_buffer():
nonlocal result_lines, table_buffer, in_table
if not table_buffer:
in_table = False
return
if len(table_buffer) >= 2 and self._is_table_separator(table_buffer[1]):
formatted_lines = self.format_table(table_buffer)
result_lines.extend(formatted_lines)
else:
for line in table_buffer: result_lines.append(self.format_line(line))
table_buffer = []
in_table = False
def flush_code_block():
nonlocal result_lines, code_buffer, code_block_lang
if not code_buffer:
return
code_content = '\n'.join(code_buffer)
if self.syntax_highlighter and code_block_lang:
if code_block_lang.lower() == "rawmu": result_lines.append(code_content)
else:
try:
highlighted = self.syntax_highlighter.highlight(code_content, language=code_block_lang)
result_lines.append(f"{self.CODE_BG}{self.CODE_FG}")
result_lines.append(highlighted)
result_lines.append(self.CODE_RESET)
except Exception:
# Fallback to plain literal block on any error
result_lines.append(f"{self.CODE_BG}{self.CODE_FG}")
result_lines.append(self.LITERAL_START)
result_lines.append(self._escape_literals(code_content))
result_lines.append(self.LITERAL_END)
result_lines.append(self.CODE_RESET)
else:
result_lines.append(f"{self.CODE_BG}{self.CODE_FG}")
result_lines.append(self.LITERAL_START)
result_lines.append(self._escape_literals(code_content))
result_lines.append(self.LITERAL_END)
result_lines.append(self.CODE_RESET)
code_buffer = []
for line in lines:
is_fence, lang_hint = self._detect_code_fence(line)
if is_fence:
# Flush any pending structures before code fence
flush_quote_buffer()
flush_table_buffer()
if not in_code_block:
# Opening fence, start buffering
in_code_block = True
code_block_lang = lang_hint.strip() if lang_hint else None
code_buffer = []
else:
# Closing fence, flush highlighted code
flush_code_block()
in_code_block = False
code_block_lang = None
else:
# Buffer code lines for later highlighting
if in_code_block: code_buffer.append(line)
else:
quote_match = self.QUOTE_RE.match(line)
if quote_match:
if not in_quote:
flush_table_buffer()
in_quote = True
quote_buffer = []
quote_buffer.append(quote_match.group(1))
else:
if in_quote:
flush_quote_buffer()
if line.strip() != "":
if self._is_table_row(line):
in_table = True
table_buffer = [line]
else:
formatted = self.format_line(line)
result_lines.append(formatted)
# Pass through blank line as separator
else: result_lines.append("")
else:
if self._is_table_row(line):
if not in_table:
in_table = True
table_buffer = [line]
else: table_buffer.append(line)
else:
# Line breaks table, flush buffer
if in_table: flush_table_buffer()
formatted = self.format_line(line)
result_lines.append(formatted)
# Handle unclosed structures
if in_quote: flush_quote_buffer()
if in_table: flush_table_buffer()
if in_code_block: flush_code_block()
return '\n'.join(result_lines)
def format_line(self, line, mode="normal"):
if mode == "codeblock": return self._escape_literals(line)
line = line.replace("\\", "\\\\")
if line.startswith("-") and not line.startswith("---") and not line.startswith("- "): line = f"\\{line}"
if line.startswith("<"): line = f"\\{line}"
# if line.startswith(">"): line = f"\\{line}" # Now handled by blockquotes
if self.HORIZONTAL_RULE_RE.match(line): return self._format_horizontal_rule()
header_match = self.HEADER_RE.match(line)
if header_match: return self._format_header(header_match)
list_match = self.UNORDERED_LIST_RE.match(line)
if list_match: return self._format_list_item(list_match)
line = self._format_inline(line)
return line
def _format_inline(self, text):
code_blocks = []
def extract_code(match):
code_blocks.append(match.group(1))
return f"\x00CODE{len(code_blocks)-1}\x00"
links = []
def extract_link(match):
links.append((match.group(1), match.group(2)))
return f"\x00LINK{len(links)-1}\x00"
text = self.LINK_RE.sub(extract_link, text)
text = self.INLINE_CODE_RE.sub(extract_code, text)
text = self.BOLD_RE.sub(self._bold_sub, text)
text = self.ITALIC_RE.sub(self._italic_sub, text)
def restore_link(match):
idx = int(match.group(1))
text, url = links[idx]
anchor_components = url.split("#")
url = anchor_components[0]
anchor = anchor_components[1] if len(anchor_components) > 1 else ""
if not ":/" in url:
url = f"{self.local_url_scope}{url}"
if anchor: url = f"{url}|anchor={anchor}"
undl = "`_" if self.underline_links else ""
bold = "`!" if self.bold_links else ""
text = text.replace('`', '')
link = f"{undl}{bold}`[{text}`{url}]{bold}{undl}"
if self.link_color and len(self.link_color) == 3: link = f"`F{self.link_color}{link}`f"
if self.link_color and len(self.link_color) == 6: link = f"`FT{self.link_color}{link}`f"
return link
text = re.sub(r'\x00LINK(\d+)\x00', restore_link, text)
def restore_code(match):
idx = int(match.group(1))
content = code_blocks[idx]
content = content.replace('`', '\\`')
return f"{self.CODE_BG_INLINE}{self.CODE_FG}{content}{self.CODE_RESET}"
text = re.sub(r'\x00CODE(\d+)\x00', restore_code, text)
return text
def _highlight_inline_code(self, content):
if not self.syntax_highlighter: return None
return self.syntax_highlighter.highlight(content, language=None)
def _bold_sub(self, match):
content = match.group(1) or match.group(2)
return f"{self.BOLD}{content}{self.BOLD_END}"
def _italic_sub(self, match):
content = match.group(1) or match.group(2)
return f"{self.ITALIC}{content}{self.ITALIC_END}"
def _format_header(self, match):
hashes = match.group(1)
content = match.group(2)
level = len(hashes)
prefix = ">" * min(level, 6)
return f"{prefix}{self._format_inline(content)}"
def _format_list_item(self, match):
indent = match.group(1)
content = match.group(3)
content = self._format_inline(content)
return f"{indent} {self.BULLET} {content}"
def _format_horizontal_rule(self):
return "-"
def _detect_code_fence(self, line):
match = self.CODE_FENCE_RE.match(line)
if match:
# match.group(2) contains everything after the backticks (language hint)
return True, match.group(2)
return False, ""
def _is_table_row(self, line):
if '|' not in line: return False
match = self.TABLE_ROW_RE.match(line)
if match is None: return False
content = match.group(1)
return '|' in content or line.strip().startswith('|')
def _is_table_separator(self, line):
if '|' not in line: return False
match = self.TABLE_SEP_RE.match(line)
return match is not None
def _escape_literals(self, text):
return text.replace('`', '\\`')
def format_table(self, rows, align="c"):
if len(rows) < 2: return rows
# Parse header and separator
header_cells = self._parse_table_row(rows[0])
alignments = self._parse_table_alignments(rows[1])
# Ensure alignment count matches header cells
while len(alignments) < len(header_cells): alignments.append('left')
alignments = alignments[:len(header_cells)]
# Parse data rows
data_rows = []
for i in range(2, len(rows)):
cells = self._parse_table_row(rows[i])
while len(cells) < len(header_cells): cells.append("")
cells = cells[:len(header_cells)]
data_rows.append(cells)
# Calculate column widths based on content
num_cols = len(header_cells)
col_widths = [0] * num_cols
all_rows = [header_cells] + data_rows
for row in all_rows:
for i, cell in enumerate(row):
formatted = self._format_inline(cell)
width = self._visible_width(formatted)
col_widths[i] = max(col_widths[i], width)
# Apply minimum width and calculate total
col_widths = [max(w, self.TABLE_MIN_COL_WIDTH) for w in col_widths]
# Check max_width constraint
# Total = sum of columns + 3 chars per column (space + 2 borders) + 1 for final border
total_width = sum(col_widths) + (num_cols * 3) + 1
if total_width > self.max_width:
# Reduce widest columns proportionally
excess = total_width - self.max_width
indexed_widths = [(i, w) for i, w in enumerate(col_widths)]
indexed_widths.sort(key=lambda x: -x[1])
for i, w in indexed_widths:
if excess <= 0: break
reduction = min(excess, w - self.TABLE_MIN_COL_WIDTH)
col_widths[i] -= reduction
excess -= reduction
# Build formatted table
result = []
# Alignment start
if align: result.append(f"`{align}")
# Top border
border = self.TABLE_TL
for i, w in enumerate(col_widths):
border += self.TABLE_H * (w + 2)
if i < len(col_widths) - 1: border += self.TABLE_TM
else: border += self.TABLE_TR
result.append(self._escape_literals(border))
# Header row
header_line = self.TABLE_V
for i, cell in enumerate(header_cells):
formatted = self._format_inline(cell)
padded = self._pad_cell(formatted, col_widths[i], 'left')
header_line += f" {padded} {self.TABLE_V}"
result.append(self._escape_literals(header_line))
# Separator row
sep_line = self.TABLE_ML
for i, w in enumerate(col_widths):
cell_width = w + 2
sep_line += self.TABLE_H * cell_width
if i < len(col_widths) - 1: sep_line += self.TABLE_MM
else: sep_line += self.TABLE_MR
result.append(self._escape_literals(sep_line))
# Data rows
for row in data_rows:
row_line = self.TABLE_V
for i, cell in enumerate(row):
formatted = self._format_inline(cell)
padded = self._pad_cell(formatted, col_widths[i], alignments[i])
row_line += f" {padded} {self.TABLE_V}"
result.append(row_line)
# Bottom border
border = self.TABLE_BL
for i, w in enumerate(col_widths):
border += self.TABLE_H * (w + 2)
if i < len(col_widths) - 1: border += self.TABLE_BM
else: border += self.TABLE_BR
result.append(self._escape_literals(border))
# End alignment
if align: result.append("`a")
return result
def format_table_raw(self, rows, align="c"):
if len(rows) < 2: return rows
# Parse header and separator
header_cells = self._parse_table_row(rows[0])
alignments = self._parse_table_alignments(rows[1])
# Ensure alignment count matches header cells
while len(alignments) < len(header_cells): alignments.append('left')
alignments = alignments[:len(header_cells)]
# Parse data rows
data_rows = []
for i in range(2, len(rows)):
cells = self._parse_table_row(rows[i])
while len(cells) < len(header_cells): cells.append("")
cells = cells[:len(header_cells)]
data_rows.append(cells)
# Calculate column widths based on raw content
num_cols = len(header_cells)
col_widths = [0] * num_cols
all_rows = [header_cells] + data_rows
for row in all_rows:
for i, cell in enumerate(row):
width = self._visible_width(cell)
col_widths[i] = max(col_widths[i], width)
# Apply minimum width and calculate total
col_widths = [max(w, self.TABLE_MIN_COL_WIDTH) for w in col_widths]
# Check max_width constraint
total_width = sum(col_widths) + (num_cols * 3) + 1
if total_width > self.max_width:
# Reduce widest columns proportionally
excess = total_width - self.max_width
indexed_widths = [(i, w) for i, w in enumerate(col_widths)]
indexed_widths.sort(key=lambda x: -x[1])
for i, w in indexed_widths:
if excess <= 0: break
reduction = min(excess, w - self.TABLE_MIN_COL_WIDTH)
col_widths[i] -= reduction
excess -= reduction
# Build formatted table
result = []
# Alignment start
if align: result.append(f"`{align}")
# Top border
border = self.TABLE_TL
for i, w in enumerate(col_widths):
border += self.TABLE_H * (w + 2)
if i < len(col_widths) - 1: border += self.TABLE_TM
else: border += self.TABLE_TR
result.append(self._escape_literals(border))
# Header row
header_line = self.TABLE_V
for i, cell in enumerate(header_cells):
padded = self._pad_cell(cell, col_widths[i], 'left')
header_line += f" {padded} {self.TABLE_V}"
result.append(header_line)
# Separator row - clean horizontal lines without alignment markers
sep_line = self.TABLE_ML
for i, w in enumerate(col_widths):
cell_width = w + 2
sep_line += self.TABLE_H * cell_width
if i < len(col_widths) - 1: sep_line += self.TABLE_MM
else: sep_line += self.TABLE_MR
result.append(self._escape_literals(sep_line))
# Data rows (with alignment)
for row in data_rows:
row_line = self.TABLE_V
for i, cell in enumerate(row):
padded = self._pad_cell(cell, col_widths[i], alignments[i])
row_line += f" {padded} {self.TABLE_V}"
result.append(row_line)
# Bottom border
border = self.TABLE_BL
for i, w in enumerate(col_widths):
border += self.TABLE_H * (w + 2)
if i < len(col_widths) - 1: border += self.TABLE_BM
else: border += self.TABLE_BR
result.append(self._escape_literals(border))
# End alignment
if align: result.append("`a")
return result
def _parse_table_row(self, line):
line = line.strip()
if line.startswith('|'): line = line[1:]
if line.endswith('|'): line = line[:-1]
cells = []
current = ""
escaped = False
for char in line:
if escaped:
current += char
escaped = False
elif char == '\\':
escaped = True
elif char == '|':
cells.append(current.strip())
current = ""
else:
current += char
cells.append(current.strip())
return cells
def _parse_table_alignments(self, line):
cells = self._parse_table_row(line)
alignments = []
for cell in cells:
cell = cell.strip()
if cell.startswith(':') and cell.endswith(':'): alignments.append('center')
elif cell.endswith(':'): alignments.append('right')
else: alignments.append('left')
return alignments
def _visible_width(self, text):
text = re.sub(r'`[FB][0-9a-fA-F]{3}', '', text)
text = re.sub(r'`[FB]T[0-9a-fA-F]{6}', '', text)
text = re.sub(r'`[!*_=]', '', text)
text = re.sub(r'`f`b', '', text)
text = re.sub(r'`f', '', text)
text = re.sub(r'`b', '', text)
return self.display_width(text)
def _pad_cell(self, text, width, align):
text = self._truncate_cell(text, width)
text_width = self._visible_width(text)
padding = width - text_width
if align == 'right':
return " " * padding + text
elif align == 'center':
left = padding // 2
right = padding - left
return " " * left + text + " " * right
else:
return text + " " * padding
def _truncate_cell(self, text, width):
if self._visible_width(text) <= width: return text
truncation_point = len(text)
while truncation_point > 0 and self._visible_width(text[0:truncation_point]) >= width:
truncation_point -= 1
truncated = text[:truncation_point]
# Yes, this is convoluted, but if someone else has
# a better idea on how to handle unclosed micron
# tags in the truncated cells, I'm all ears.
active_tags = set()
fg_active = False
bg_active = False
i = 0
while i < len(truncated):
if truncated[i] == '`':
if i + 1 < len(truncated):
tag_char = truncated[i + 1]
if tag_char in '!*_=':
if tag_char in active_tags: active_tags.remove(tag_char)
else: active_tags.add(tag_char)
i += 2
continue
elif tag_char == 'f':
fg_active = False
i += 2
continue
elif tag_char == 'b':
bg_active = False
i += 2
continue
elif tag_char == 'F':
fg_active = True
if i + 2 < len(truncated) and truncated[i + 2] == 'T': i += 8
else: i += 5
continue
elif tag_char == 'B':
bg_active = True
if i + 2 < len(truncated) and truncated[i + 2] == 'T': i += 8
else: i += 5
continue
i += 1
closers = []
if fg_active: closers.append('`f')
if bg_active: closers.append('`b')
for fmt in active_tags: closers.append(f'`{fmt}')
return truncated + ''.join(closers) + "…"
def _wrap_text(self, text, width):
if not text: return [""]
words = text.split(' ')
lines = []
current_line = ""
current_width = 0
for word in words:
if not word: continue
word_width = self._visible_width(word)
# Check if word alone exceeds width to force break it
if word_width > width:
if current_line:
lines.append(current_line)
current_line = ""
current_width = 0
# Force break the long word character by character
remaining = word
while remaining:
# Binary search for how many characters fit
low, high = 1, len(remaining)
fit_chars = 0
while low <= high:
mid = (low + high) // 2
test_substr = remaining[:mid]
test_width = self._visible_width(test_substr)
if test_width <= width:
fit_chars = mid
low = mid + 1
else:
high = mid - 1
if fit_chars == 0: fit_chars = 1 # Need to force progress
lines.append(remaining[:fit_chars])
remaining = remaining[fit_chars:]
continue
# Check if word fits on current line
space_width = 1 if current_line else 0
if current_width + space_width + word_width <= width:
if current_line:
current_line += " " + word
current_width += space_width + word_width
else:
current_line = word
current_width = word_width
else:
# Flush current line and start new one
lines.append(current_line)
current_line = word
current_width = word_width
# Don't forget the last line
if current_line: lines.append(current_line)
return lines if lines else [""]
def convert_markdown_to_micron(text):
converter = MarkdownToMicron()
return converter.format_block(text)
def medium_path_timeout(default_timeout):
return max(default_timeout, RNS.Reticulum.get_instance().get_medium_path_timeout())

File diff suppressed because it is too large Load diff

View file

@ -1,8 +1,8 @@
#!/usr/bin/env python3
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2023 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -11,16 +11,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -55,12 +47,19 @@ def main():
parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
parser.add_argument('-v', '--verbose', action='count', default=0)
parser.add_argument('-q', '--quiet', action='count', default=0)
parser.add_argument("--version", action="version", version="rnir {version}".format(version=__version__))
parser.add_argument("--exampleconfig", action='store_true', default=False, help="print verbose configuration example to stdout and exit")
parser.add_argument("--version", action="version", version="ir {version}".format(version=__version__))
args = parser.parse_args()
if args.config: configarg = args.config
else: configarg = None
if args.exampleconfig:
print(__example_rns_config__)
exit()
if args.config:
configarg = args.config
else:
configarg = None
program_setup(configdir = configarg, verbosity=args.verbose, quietness=args.quiet)
@ -68,5 +67,8 @@ def main():
print("")
exit()
__example_rns_config__ = '''# This is an example Identity Resolver file.
'''
if __name__ == "__main__":
main()

File diff suppressed because it is too large Load diff

View file

@ -1,8 +1,8 @@
#!/usr/bin/env python3
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -11,16 +11,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -39,10 +31,8 @@ import argparse
from RNS._version import __version__
remote_link = None
output_rst_str = "\r \r"
def connect_remote(destination_hash, auth_identity, timeout, no_output = False, purpose="management"):
def connect_remote(destination_hash, auth_identity, timeout, no_output = False):
global remote_link, reticulum
timeout = max(timeout, reticulum.get_medium_path_timeout())
if not RNS.Transport.has_path(destination_hash):
if not no_output:
print("Path to "+RNS.prettyhexrep(destination_hash)+" requested", end=" ")
@ -53,226 +43,107 @@ def connect_remote(destination_hash, auth_identity, timeout, no_output = False,
time.sleep(0.1)
if time.time() - pr_time > timeout:
if not no_output:
print(output_rst_str, end="")
print("\r \r", end="")
print("Path request timed out")
exit(12)
remote_identity = RNS.Identity.recall(destination_hash)
def remote_link_closed(link):
if link.teardown_reason == RNS.Link.INITIATOR_CLOSED: return
elif link.teardown_reason == RNS.Link.TIMEOUT:
if link.teardown_reason == RNS.Link.TIMEOUT:
if not no_output:
print(output_rst_str, end="")
print("\r \r", end="")
print("The link timed out, exiting now")
elif link.teardown_reason == RNS.Link.DESTINATION_CLOSED:
if not no_output:
print(output_rst_str, end="")
print("\r \r", end="")
print("The link was closed by the server, exiting now")
else:
if not no_output:
print(output_rst_str, end="")
print("\r \r", end="")
print("Link closed unexpectedly, exiting now")
exit(10)
def remote_link_established(link):
global remote_link
if purpose == "management": link.identify(auth_identity)
link.identify(auth_identity)
remote_link = link
if not no_output:
print(output_rst_str, end="")
print("\r \r", end="")
print("Establishing link with remote transport instance...", end=" ")
sys.stdout.flush()
if purpose == "management": remote_destination = RNS.Destination(remote_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, "rnstransport", "remote", "management")
elif purpose == "blackhole": remote_destination = RNS.Destination(remote_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, "rnstransport", "info", "blackhole")
remote_destination = RNS.Destination(remote_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, "rnstransport", "remote", "management")
link = RNS.Link(remote_destination)
link.set_link_established_callback(remote_link_established)
link.set_link_closed_callback(remote_link_closed)
def parse_hash(input_str):
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
if len(input_str) != dest_len: raise ValueError("Hash length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
try:
hash_bytes = bytes.fromhex(input_str)
return hash_bytes
except Exception as e: raise ValueError("Invalid hash entered. Check your input.")
def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity, timeout, drop_queues,
drop_via, max_hops, remote=None, management_identity=None, remote_timeout=RNS.Transport.PATH_REQUEST_TIMEOUT,
blackholed=False, blackhole=False, unblackhole=False, blackhole_duration=None, blackhole_reason=None, remote_blackhole_list=False, remote_blackhole_list_filter=None,
active=False, usage=False, vias=False, expinfo=False, min_value=None, no_output=False, analyze=False, capture=None, json=False):
no_output=False, json=False):
global remote_link, reticulum
reticulum = RNS.Reticulum(configdir = configdir, loglevel = 3+verbosity)
reticulum.wants_destination_type("rnstransport.remote.management")
reticulum.wants_destination_type("rnstransport.info.blackhole")
if remote:
try:
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
if len(remote) != dest_len: raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
if len(remote) != dest_len:
raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
try:
identity_hash = bytes.fromhex(remote)
remote_hash = RNS.Destination.hash_from_name_and_identity("rnstransport.remote.management", identity_hash)
except Exception as e: raise ValueError("Invalid destination entered. Check your input.")
except Exception as e:
raise ValueError("Invalid destination entered. Check your input.")
identity = RNS.Identity.from_file(os.path.expanduser(management_identity))
if identity == None: raise ValueError("Could not load management identity from "+str(management_identity))
if identity == None:
raise ValueError("Could not load management identity from "+str(management_identity))
try: connect_remote(remote_hash, identity, remote_timeout, no_output)
except Exception as e: raise e
try:
connect_remote(remote_hash, identity, remote_timeout, no_output)
except Exception as e:
raise e
except Exception as e:
print(str(e))
exit(20)
while remote_link == None: time.sleep(0.1)
while remote_link == None:
time.sleep(0.1)
if blackholed or remote_blackhole_list:
blackholed_list = None
if blackholed:
if remote_link:
if not no_output:
print(output_rst_str, end="")
print("Listing blackholed identities on remote instances not yet implemented")
exit(255)
try: blackholed_list = reticulum.get_blackholed_identities()
except Exception as e:
print(f"Could not get blackholed identities from RNS instance: {e}")
exit(20)
elif remote_blackhole_list:
try: identity_hash = parse_hash(destination_hexhash)
except Exception as e:
print(f"{e}")
exit(20)
remote_hash = RNS.Destination.hash_from_name_and_identity("rnstransport.info.blackhole", identity_hash)
connect_remote(remote_hash, None, remote_timeout, no_output, purpose="blackhole")
while remote_link == None: time.sleep(0.1)
if not no_output:
print(output_rst_str, end="")
print("Sending request...", end=" ")
sys.stdout.flush()
receipt = remote_link.request("/list")
while not receipt.concluded(): time.sleep(0.1)
response = receipt.get_response()
if type(response) == dict:
blackholed_list = response
print(output_rst_str, end="")
else:
if not no_output:
print(output_rst_str, end="")
print("The remote request failed.")
exit(10)
else:
print(f"Nowhere to fetch blackhole list from")
exit(255)
if not blackholed_list:
print("No blackholed identity data available")
exit(20)
else:
rmlen = 64
def trunc(input_str):
if len(input_str) <= rmlen: return input_str
else: return f"{input_str[:rmlen-1]}…"
try:
now = time.time()
for identity_hash in blackholed_list:
until = blackholed_list[identity_hash]["until"]
reason = blackholed_list[identity_hash]["reason"]
source = blackholed_list[identity_hash]["source"]
until_str = f"for {RNS.prettytime(max(0, until-now))}" if until else "indefinitely"
reason_str = f" ({trunc(reason)})" if reason else ""
by_str = f" by {RNS.prettyhexrep(source)}" if source != RNS.Transport.identity.hash else ""
filter_str = f"{RNS.prettyhexrep(identity_hash)} {until_str} {reason_str} {by_str}"
if not remote_blackhole_list:
if destination_hexhash and not destination_hexhash in filter_str: continue
else:
if remote_blackhole_list_filter and not remote_blackhole_list_filter in filter_str: continue
print(f"{RNS.prettyhexrep(identity_hash)} blackholed {until_str}{reason_str}{by_str}")
except Exception as e:
print(f"Error while displaying collected blackhole data: {e}")
exit(20)
elif blackhole:
if remote_link:
if not no_output:
print(output_rst_str, end="")
print("Blackholing identity on remote instances not yet implemented")
exit(255)
try:
identity_hash = parse_hash(destination_hexhash)
until = time.time()+blackhole_duration*60*60 if blackhole_duration else None
result = reticulum.blackhole_identity(identity_hash, until=until, reason=blackhole_reason)
if result == True: print(f"Blackholed identity {destination_hexhash}")
elif result == None: print(f"Identity {destination_hexhash} already blackholed")
else: print(f"Could not blackhole identity {destination_hexhash}")
except Exception as e:
print(f"Could not blackhole identity: {e}")
exit(20)
elif unblackhole:
if remote_link:
if not no_output:
print(output_rst_str, end="")
print("Blackholing identity on remote instances not yet implemented")
exit(255)
try:
identity_hash = parse_hash(destination_hexhash)
result = reticulum.unblackhole_identity(identity_hash)
if result == True: print(f"Lifted blackhole for identity {destination_hexhash}")
elif result == None: print(f"Identity {destination_hexhash} not blackholed")
else: print(f"Could not unblackhole identity {destination_hexhash}")
except Exception as e:
print(f"Could not unblackhole identity: {e}")
exit(20)
elif analyze:
analyze_tables(path_table=reticulum.get_path_table(), rate_table=reticulum.get_rate_table(),
link_distribution=reticulum.get_link_distribution(), capture=capture)
elif table:
if table:
destination_hash = None
if destination_hexhash != None:
try:
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
if len(destination_hexhash) != dest_len: raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
try: destination_hash = bytes.fromhex(destination_hexhash)
except Exception as e: raise ValueError("Invalid destination entered. Check your input.")
if len(destination_hexhash) != dest_len:
raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
try:
destination_hash = bytes.fromhex(destination_hexhash)
except Exception as e:
raise ValueError("Invalid destination entered. Check your input.")
except Exception as e:
print(str(e))
sys.exit(1)
if not remote_link: table = sorted(reticulum.get_path_table(max_hops=max_hops), key=lambda e: (e["interface"], e["hops"]) )
if not remote_link:
table = sorted(reticulum.get_path_table(max_hops=max_hops), key=lambda e: (e["interface"], e["hops"]) )
else:
if not no_output:
print(output_rst_str, end="")
print("\r \r", end="")
print("Sending request...", end=" ")
sys.stdout.flush()
receipt = remote_link.request("/path", data = ["table", destination_hash, max_hops])
while not receipt.concluded(): time.sleep(0.1)
while not receipt.concluded():
time.sleep(0.1)
response = receipt.get_response()
if response:
table = response
print(output_rst_str, end="")
print("\r \r", end="")
else:
if not no_output:
print(output_rst_str, end="")
print("\r \r", end="")
print("The remote request failed. Likely authentication failure.")
exit(10)
@ -281,41 +152,21 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
import json
for p in table:
for k in p:
if isinstance(p[k], bytes): p[k] = RNS.hexrep(p[k], delimit=False)
if isinstance(p[k], bytes):
p[k] = RNS.hexrep(p[k], delimit=False)
print(json.dumps(table))
exit()
else:
now = time.time()
if usage:
try: table = sorted(table, key=lambda e: e["used"], reverse=True)
except Exception as e: pass
now = time.time()
for path in table:
if destination_hash == None or destination_hash == path["hash"]:
displayed += 1
if min_value:
if not path["value"]: continue
if path["value"] < min_value: continue
if path["state"] == RNS.Transport.STATE_RESPONSIVE: state_str = " (R)"
elif path["state"] == RNS.Transport.STATE_UNRESPONSIVE: state_str = " (u)"
elif path["state"] == RNS.Transport.STATE_UNKNOWN: state_str = " (?)"
else: state_str = " (?)"
exp_str = RNS.timestamp_str(path["expires"])
if path["hops"] == 1: m_str = " "
else: m_str = "s"
if not "used" in path: u_str = ", no usage info"
if path["hops"] == 1:
m_str = " "
else:
if active and not path["used"]: continue
if usage: u_str = f", last used {RNS.prettytime(now-path['used'])} ago" if path["used"] else ", never used"
else: u_str = ""
via_str = f" via {RNS.prettyhexrep(path['via'])}" if vias else ""
exp_str = f" expires in {RNS.prettytime(path['expires']-now, compact=True)}" if expinfo else ""
print(f"{RNS.prettyhexrep(path['hash'])} {path['hops']:>3}h{state_str}{via_str} on {path['interface']}{exp_str} (dv:{RNS.prettycost(path['value'])}/lc:{RNS.prettycost(path['lr_cost'])}){u_str}")
m_str = "s"
print(RNS.prettyhexrep(path["hash"])+" is "+str(path["hops"])+" hop"+m_str+" away via "+RNS.prettyhexrep(path["via"])+" on "+path["interface"]+" expires "+RNS.timestamp_str(path["expires"]))
if destination_hash != None and displayed == 0:
print("No path known")
@ -326,17 +177,21 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
if destination_hexhash != None:
try:
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
if len(destination_hexhash) != dest_len: raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
try: destination_hash = bytes.fromhex(destination_hexhash)
except Exception as e: raise ValueError("Invalid destination entered. Check your input.")
if len(destination_hexhash) != dest_len:
raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
try:
destination_hash = bytes.fromhex(destination_hexhash)
except Exception as e:
raise ValueError("Invalid destination entered. Check your input.")
except Exception as e:
print(str(e))
sys.exit(1)
if not remote_link: table = reticulum.get_rate_table()
if not remote_link:
table = reticulum.get_rate_table()
else:
if not no_output:
print(output_rst_str, end="")
print("\r \r", end="")
print("Sending request...", end=" ")
sys.stdout.flush()
receipt = remote_link.request("/path", data = ["rates", destination_hash])
@ -345,10 +200,10 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
response = receipt.get_response()
if response:
table = response
print(output_rst_str, end="")
print("\r \r", end="")
else:
if not no_output:
print(output_rst_str, end="")
print("\r \r", end="")
print("The remote request failed. Likely authentication failure.")
exit(10)
@ -357,12 +212,15 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
import json
for p in table:
for k in p:
if isinstance(p[k], bytes): p[k] = RNS.hexrep(p[k], delimit=False)
if isinstance(p[k], bytes):
p[k] = RNS.hexrep(p[k], delimit=False)
print(json.dumps(table))
exit()
else:
if len(table) == 0: print("No information available")
if len(table) == 0:
print("No information available")
else:
displayed = 0
for entry in table:
@ -408,7 +266,7 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
elif drop_queues:
if remote_link:
if not no_output:
print(output_rst_str, end="")
print("\r \r", end="")
print("Dropping announce queues on remote instances not yet implemented")
exit(255)
@ -418,20 +276,24 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
elif drop:
if remote_link:
if not no_output:
print(output_rst_str, end="")
print("\r \r", end="")
print("Dropping path on remote instances not yet implemented")
exit(255)
try:
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
if len(destination_hexhash) != dest_len: raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
try: destination_hash = bytes.fromhex(destination_hexhash)
except Exception as e: raise ValueError("Invalid destination entered. Check your input.")
if len(destination_hexhash) != dest_len:
raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
try:
destination_hash = bytes.fromhex(destination_hexhash)
except Exception as e:
raise ValueError("Invalid destination entered. Check your input.")
except Exception as e:
print(str(e))
sys.exit(1)
if reticulum.drop_path(destination_hash): print("Dropped path to "+RNS.prettyhexrep(destination_hash))
if reticulum.drop_path(destination_hash):
print("Dropped path to "+RNS.prettyhexrep(destination_hash))
else:
print("Unable to drop path to "+RNS.prettyhexrep(destination_hash)+". Does it exist?")
sys.exit(1)
@ -439,20 +301,24 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
elif drop_via:
if remote_link:
if not no_output:
print(output_rst_str, end="")
print("\r \r", end="")
print("Dropping all paths via specific transport instance on remote instances yet not implemented")
exit(255)
try:
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
if len(destination_hexhash) != dest_len: raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
try: destination_hash = bytes.fromhex(destination_hexhash)
except Exception as e: raise ValueError("Invalid destination entered. Check your input.")
if len(destination_hexhash) != dest_len:
raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
try:
destination_hash = bytes.fromhex(destination_hexhash)
except Exception as e:
raise ValueError("Invalid destination entered. Check your input.")
except Exception as e:
print(str(e))
sys.exit(1)
if reticulum.drop_all_via(destination_hash): print("Dropped all paths via "+RNS.prettyhexrep(destination_hash))
if reticulum.drop_all_via(destination_hash):
print("Dropped all paths via "+RNS.prettyhexrep(destination_hash))
else:
print("Unable to drop paths via "+RNS.prettyhexrep(destination_hash)+". Does the transport instance exist?")
sys.exit(1)
@ -460,15 +326,18 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
else:
if remote_link:
if not no_output:
print(output_rst_str, end="")
print("\r \r", end="")
print("Requesting paths on remote instances not implemented")
exit(255)
try:
dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
if len(destination_hexhash) != dest_len: raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
try: destination_hash = bytes.fromhex(destination_hexhash)
except Exception as e: raise ValueError("Invalid destination entered. Check your input.")
if len(destination_hexhash) != dest_len:
raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
try:
destination_hash = bytes.fromhex(destination_hexhash)
except Exception as e:
raise ValueError("Invalid destination entered. Check your input.")
except Exception as e:
print(str(e))
sys.exit(1)
@ -480,7 +349,7 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
i = 0
syms = "⢄⢂⢁⡁⡈⡐⡠"
limit = time.time()+max(timeout, reticulum.get_medium_path_timeout())
limit = time.time()+timeout
while not RNS.Transport.has_path(destination_hash) and time.time()<limit:
time.sleep(0.1)
print(("\b\b"+syms[i]+" "), end="")
@ -497,178 +366,207 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
next_hop = RNS.prettyhexrep(next_hop_bytes)
next_hop_interface = reticulum.get_next_hop_if_name(destination_hash)
if hops != 1: ms = "s"
else: ms = ""
if hops != 1:
ms = "s"
else:
ms = ""
print("\rPath found, destination "+RNS.prettyhexrep(destination_hash)+" is "+str(hops)+" hop"+ms+" away via "+next_hop+" on "+next_hop_interface)
else:
print("\r \rPath not found")
sys.exit(1)
def main():
try:
parser = argparse.ArgumentParser(description="Reticulum Path Management Utility")
parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
parser.add_argument("--version", action="version", version="rnpath {version}".format(version=__version__))
parser.add_argument("-t", "--table", action="store_true", help="show all known paths", default=False)
parser.add_argument("-a", "--active", action="store_true", help="show in-use paths only", default=False)
parser.add_argument("-u", "--usage", action="store_true", help="show path usage time", default=False)
parser.add_argument("-V", "--vias", action="store_true", help="show via transport IDs", default=False)
parser.add_argument("-E", "--expiry", action="store_true", help="show path expiry times", default=False)
parser.add_argument("-m", "--max", action="store", metavar="hops", type=int, help="maximum hops to filter path table by", default=None)
parser.add_argument("-M", "--min-value", action="store", metavar="value", type=float, help="minimum value to filter path table by", default=None)
parser.add_argument("-r", "--rates", action="store_true", help="show announce rate info", default=False)
parser.add_argument("-d", "--drop", action="store_true", help="remove the path to a destination", default=False)
parser.add_argument("-D", "--drop-announces", action="store_true", help="drop all queued announces", default=False)
parser.add_argument("-x", "--drop-via", action="store_true", help="drop all paths via specified transport instance", default=False)
parser.add_argument("-w", action="store", metavar="seconds", type=float, help="timeout before giving up", default=RNS.Transport.PATH_REQUEST_TIMEOUT)
parser.add_argument("-R", action="store", metavar="hash", help="transport identity hash of remote instance to manage", default=None, type=str)
parser.add_argument("-i", action="store", metavar="path", help="path to identity used for remote management", default=None, type=str)
parser.add_argument("-W", action="store", metavar="seconds", type=float, help="timeout before giving up on remote queries", default=RNS.Transport.PATH_REQUEST_TIMEOUT)
parser.add_argument("-b", "--blackholed", action="store_true", help="list blackholed identities", default=False)
parser.add_argument("-B", "--blackhole", action="store_true", help="blackhole identity", default=False)
parser.add_argument("-U", "--unblackhole", action="store_true", help="unblackhole identity", default=False)
parser.add_argument( "--duration", action="store", metavar="hours", type=float, help="duration of blackhole enforcement in hours", default=None)
parser.add_argument( "--reason", action="store", metavar="reason", type=str, help="reason for blackholing identity", default=None)
parser.add_argument("-p", "--blackholed-list", action="store_true", help="view published blackhole list for remote transport instance", default=False)
parser.add_argument("-j", "--json", action="store_true", help="output in JSON format", default=False)
parser.add_argument("-A", "--analyze", action="store_true", help="Analyze tables", default=False)
parser.add_argument("--capture", action="store", metavar="path", type=str, help="path to capture file for analyze mode", default=None)
parser.add_argument("destination", nargs="?", default=None, help="hexadecimal hash of the destination", type=str)
parser.add_argument("list_filter", nargs="?", default=None, help="filter for remote blackhole list view", type=str)
parser = argparse.ArgumentParser(description="Reticulum Path Discovery Utility")
parser.add_argument("--config",
action="store",
default=None,
help="path to alternative Reticulum config directory",
type=str
)
parser.add_argument(
"--version",
action="version",
version="rnpath {version}".format(version=__version__)
)
parser.add_argument(
"-t",
"--table",
action="store_true",
help="show all known paths",
default=False
)
parser.add_argument(
"-m",
"--max",
action="store",
metavar="hops",
type=int,
help="maximum hops to filter path table by",
default=None
)
parser.add_argument(
"-r",
"--rates",
action="store_true",
help="show announce rate info",
default=False
)
parser.add_argument(
"-d",
"--drop",
action="store_true",
help="remove the path to a destination",
default=False
)
parser.add_argument(
"-D",
"--drop-announces",
action="store_true",
help="drop all queued announces",
default=False
)
parser.add_argument(
"-x", "--drop-via",
action="store_true",
help="drop all paths via specified transport instance",
default=False
)
parser.add_argument(
"-w",
action="store",
metavar="seconds",
type=float,
help="timeout before giving up",
default=RNS.Transport.PATH_REQUEST_TIMEOUT
)
parser.add_argument(
"-R",
action="store",
metavar="hash",
help="transport identity hash of remote instance to manage",
default=None,
type=str
)
parser.add_argument(
"-i",
action="store",
metavar="path",
help="path to identity used for remote management",
default=None,
type=str
)
parser.add_argument(
"-W",
action="store",
metavar="seconds",
type=float,
help="timeout before giving up on remote queries",
default=RNS.Transport.PATH_REQUEST_TIMEOUT
)
parser.add_argument(
"-j",
"--json",
action="store_true",
help="output in JSON format",
default=False
)
parser.add_argument(
"destination",
nargs="?",
default=None,
help="hexadecimal hash of the destination",
type=str
)
parser.add_argument('-v', '--verbose', action='count', default=0)
args = parser.parse_args()
if args.config: configarg = args.config
else: configarg = None
if args.config:
configarg = args.config
else:
configarg = None
if not args.drop_announces and not args.table and not args.rates and not args.destination and not args.drop_via and not args.blackholed and not args.analyze:
if not args.drop_announces and not args.table and not args.rates and not args.destination and not args.drop_via:
print("")
parser.print_help()
print("")
else:
program_setup(configdir=configarg, table=args.table, rates=args.rates, drop=args.drop, destination_hexhash=args.destination,
verbosity=args.verbose, timeout=args.w, drop_queues=args.drop_announces, drop_via=args.drop_via, max_hops=args.max, min_value=args.min_value,
remote=args.R, management_identity=args.i, remote_timeout=args.W, blackholed=args.blackholed, blackhole=args.blackhole,
unblackhole=args.unblackhole, blackhole_duration=args.duration, blackhole_reason=args.reason, remote_blackhole_list=args.blackholed_list,
remote_blackhole_list_filter=args.list_filter, usage=args.usage, vias=args.vias, active=args.active, expinfo=args.expiry,
analyze=args.analyze, capture=args.capture, json=args.json)
program_setup(
configdir = configarg,
table = args.table,
rates = args.rates,
drop = args.drop,
destination_hexhash = args.destination,
verbosity = args.verbose,
timeout = args.w,
drop_queues = args.drop_announces,
drop_via = args.drop_via,
max_hops = args.max,
remote=args.R,
management_identity=args.i,
remote_timeout=args.W,
json=args.json,
)
sys.exit(0)
except KeyboardInterrupt:
print("")
exit()
def analyze_tables(path_table=None, rate_table=None, link_distribution=None, capture=None):
print("Not yet implemented")
exit()
# if not path_table:
# print("No path table to analyze")
# exit()
# else:
# print(f"Analyzing tables...")
# # TODO: If a `capture` path is specified, we save and (if existing) load from
# # this file, so that `rnpath --analyze` can be run at intervals, and continuously
# # collect data, instead of only providing an at-the-moment snapshot view.
# # Locally initialize announce cache to be able to
# # fetch full announce packet data
# if RNS.Transport.owner.is_connected_to_shared_instance: RNS.Transport.init_announce_cache()
# tm = RNS.Transport.destination_type_map
# total = len(path_table)
# destinations = {}
# unresolvable = 0
# unknown_types = set()
# known_types = set()
# unknown_type_dsts = 0
# known_type_dsts = 0
# for path in path_table:
# dst_hash = path["hash"]
# dst_hops = path["hops"]
# last_used = path["used"]
# name_hash = None
# kd_entry = RNS.Identity.known_destinations.get(dst_hash, None)
# # New destinations may have arrived after our initial known
# # destinations snapshot, which is fine, we just count the few
# # that miss here for now.
# if not kd_entry: unresolvable += 1
# # Enrich data with information from cached announces
# else:
# dst_value = kd_entry[6]
# dst_lr_cost = kd_entry[7]
# announce_hash = kd_entry[1]
# announce_packet = RNS.Transport.get_cached_packet(announce_hash, packet_type="announce")
# if not announce_packet: unresolvable += 1; continue
# try: announce_packet.unpack()
# except: unresolvable +=1; continue
# keysize = RNS.Identity.KEYSIZE//8
# name_hash_len = RNS.Identity.NAME_HASH_LENGTH//8
# name_hash = announce_packet.data[keysize:keysize+name_hash_len]
# app_data = kd_entry[3] or RNS.Identity.recall_app_data(dst_hash, _no_use=True)
# app_data_len = len(app_data) if app_data else None # No len counting if no app data
# if name_hash in tm: known_type_dsts += 1; known_types.add(name_hash)
# else: unknown_type_dsts += 1; unknown_types.add(name_hash)
# # TODO: Create actual data structure for use in analysis
# entry = {}
# destinations[dst_hash] = entry
# if rate_table:
# # TODO: Incorporate and enrich with data from rate table
# pass
# if link_distribution:
# # TODO: Incorporate and enrich with link distribution data
# pass
# # TODO: Actually analyze tables
# pass
# # TODO: Actually print a comprehensive, beautifully formatted
# # overview, charts and data. This is a stub.
# print(f"Known types : {len(known_types)}")
# print(f"Unknown types : {len(unknown_types)}")
# print(f"Total types : {len(known_types)+len(unknown_types)}")
# print(f"Known type dests : {known_type_dsts}")
# print(f"Unknown type dests : {unknown_type_dsts}")
# print(f"Total type dests : {known_type_dsts+unknown_type_dsts}")
# # TODO: Ensure we get the terminal width if possible, and fall back to
# # a sensible default if not, so everything is rendered beautifully.
# exit()
def pretty_date(time=False):
from datetime import datetime
now = datetime.now()
if type(time) is int: diff = now - datetime.fromtimestamp(time)
elif isinstance(time,datetime): diff = now - time
elif not time: diff = now - now
if type(time) is int:
diff = now - datetime.fromtimestamp(time)
elif isinstance(time,datetime):
diff = now - time
elif not time:
diff = now - now
second_diff = diff.seconds
day_diff = diff.days
if day_diff < 0: return ''
if day_diff < 0:
return ''
if day_diff == 0:
if second_diff < 10: return str(second_diff) + " seconds"
if second_diff < 60: return str(second_diff) + " seconds"
if second_diff < 70: return "1 minute"
if second_diff < 7200: return str(int(second_diff / 60)) + " minutes"
if second_diff < 86400: return str(int(second_diff / 3600)) + " hours"
if day_diff == 1: return "1 day"
if day_diff < 7: return str(day_diff) + " days"
if day_diff < 31: return str(int(day_diff / 7)) + " weeks"
if day_diff < 365: return str(int(day_diff / 30)) + " months"
if second_diff < 10:
return str(second_diff) + " seconds"
if second_diff < 60:
return str(second_diff) + " seconds"
if second_diff < 120:
return "1 minute"
if second_diff < 3600:
return str(int(second_diff / 60)) + " minutes"
if second_diff < 7200:
return "an hour"
if second_diff < 86400:
return str(int(second_diff / 3600)) + " hours"
if day_diff == 1:
return "1 day"
if day_diff < 7:
return str(day_diff) + " days"
if day_diff < 31:
return str(int(day_diff / 7)) + " weeks"
if day_diff < 365:
return str(int(day_diff / 30)) + " months"
return str(int(day_diff / 365)) + " years"
if __name__ == "__main__": main()
if __name__ == "__main__":
main()

View file

@ -1,78 +0,0 @@
#!/usr/bin/env python3
# Reticulum License
#
# Copyright (c) 2016-2026 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import RNS
import argparse
import time
from RNS._version import __version__
def program_setup(configdir, verbosity = 0, quietness = 0, service = False):
targetverbosity = verbosity-quietness
if service:
targetlogdest = RNS.LOG_FILE
targetverbosity = None
else:
targetlogdest = RNS.LOG_STDOUT
reticulum = RNS.Reticulum(configdir=configdir, verbosity=targetverbosity, logdest=targetlogdest)
exit(0)
def main():
try:
parser = argparse.ArgumentParser(description="Reticulum Meta Package Manager")
parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
parser.add_argument('-v', '--verbose', action='count', default=0)
parser.add_argument('-q', '--quiet', action='count', default=0)
parser.add_argument("--exampleconfig", action='store_true', default=False, help="print verbose configuration example to stdout and exit")
parser.add_argument("--version", action="version", version="rnpkg {version}".format(version=__version__))
args = parser.parse_args()
if args.exampleconfig:
print(__example_rnpkg_config__)
exit()
if args.config: configarg = args.config
else: configarg = None
program_setup(configdir = configarg, verbosity=args.verbose, quietness=args.quiet)
except KeyboardInterrupt:
print("")
exit()
__example_rnpkg_config__ = '''# This is an example package manager configuration file.
'''
if __name__ == "__main__": main()

View file

@ -1,8 +1,8 @@
#!/usr/bin/env python3
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -11,16 +11,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -81,7 +73,7 @@ def program_setup(configdir, destination_hexhash, size=None, full_name = None, v
print("Path to "+RNS.prettyhexrep(destination_hash)+" requested ", end=" ")
sys.stdout.flush()
_timeout = time.time() + (timeout or max(DEFAULT_TIMEOUT+reticulum.get_first_hop_timeout(destination_hash), reticulum.get_medium_path_timeout()))
_timeout = time.time() + (timeout or DEFAULT_TIMEOUT+reticulum.get_first_hop_timeout(destination_hash))
i = 0
syms = "⢄⢂⢁⡁⡈⡐⡠"
while not RNS.Transport.has_path(destination_hash) and not time.time() > _timeout:
@ -131,7 +123,7 @@ def program_setup(configdir, destination_hexhash, size=None, full_name = None, v
print("\rSent probe "+str(sent)+" ("+str(size)+" bytes) to "+RNS.prettyhexrep(destination_hash)+more+" ", end=" ")
_timeout = time.time() + (timeout or max(DEFAULT_TIMEOUT+reticulum.get_first_hop_timeout(destination_hash), reticulum.get_medium_path_timeout()))
_timeout = time.time() + (timeout or DEFAULT_TIMEOUT+reticulum.get_first_hop_timeout(destination_hash))
i = 0
while receipt.status == RNS.PacketReceipt.SENT and not time.time() > _timeout:
time.sleep(0.1)

View file

@ -1,8 +1,8 @@
#!/usr/bin/env python3
# Reticulum License
# MIT License
#
# Copyright (c) 2016-2026 Mark Qvist
# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
@ -11,16 +11,8 @@
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@ -37,7 +29,7 @@ import time
from RNS._version import __version__
def program_setup(configdir, verbosity = 0, quietness = 0, service = False, interactive=False):
def program_setup(configdir, verbosity = 0, quietness = 0, service = False):
targetverbosity = verbosity-quietness
if service:
@ -50,14 +42,10 @@ def program_setup(configdir, verbosity = 0, quietness = 0, service = False, inte
if reticulum.is_connected_to_shared_instance:
RNS.log("Started rnsd version {version} connected to another shared local instance, this is probably NOT what you want!".format(version=__version__), RNS.LOG_WARNING)
else:
# TODO: Rethink why this was added
# if RNS.Reticulum.get_instance().shared_instance_interface:
# RNS.Reticulum.get_instance().shared_instance_interface.server.daemon_threads = True
RNS.log("Started rnsd version {version}".format(version=__version__), RNS.LOG_NOTICE)
if interactive: import code; code.interact(local=globals())
else:
while True: time.sleep(1)
while True:
time.sleep(1)
def main():
try:
@ -66,7 +54,6 @@ def main():
parser.add_argument('-v', '--verbose', action='count', default=0)
parser.add_argument('-q', '--quiet', action='count', default=0)
parser.add_argument('-s', '--service', action='store_true', default=False, help="rnsd is running as a service and should log to file")
parser.add_argument('-i', '--interactive', action='store_true', default=False, help="drop into interactive shell after initialisation")
parser.add_argument("--exampleconfig", action='store_true', default=False, help="print verbose configuration example to stdout and exit")
parser.add_argument("--version", action="version", version="rnsd {version}".format(version=__version__))
@ -81,7 +68,7 @@ def main():
else:
configarg = None
program_setup(configdir = configarg, verbosity=args.verbose, quietness=args.quiet, service=args.service, interactive=args.interactive)
program_setup(configdir = configarg, verbosity=args.verbose, quietness=args.quiet, service=args.service)
except KeyboardInterrupt:
print("")
@ -100,7 +87,7 @@ __example_rns_config__ = '''# This is an example Reticulum config file.
# always-on. This directive is optional and can be removed
# for brevity.
enable_transport = no
enable_transport = No
# By default, the first program to launch the Reticulum
@ -112,30 +99,17 @@ enable_transport = no
# user, and should generally be turned on. This directive
# is optional and can be removed for brevity.
share_instance = yes
share_instance = Yes
# If you want to run multiple *different* shared instances
# on the same system, you will need to specify different
# instance names for each. On platforms supporting domain
# sockets, this can be done with the instance_name option:
# shared instance ports for each. The defaults are given
# below, and again, these options can be left out if you
# don't need them.
instance_name = default
# Some platforms don't support domain sockets, and if that
# is the case, you can isolate different instances by
# specifying a unique set of ports for each:
# shared_instance_port = 37428
# instance_control_port = 37429
# If you want to explicitly use TCP for shared instance
# communication, instead of domain sockets, this is also
# possible, by using the following option:
# shared_instance_type = tcp
shared_instance_port = 37428
instance_control_port = 37429
# On systems where running instances may not have access
@ -161,95 +135,13 @@ instance_name = default
# remote_management_allowed = 9fb6d773498fb3feda407ed8ef2c3229, 2d882c5586e548d79b5af27bca1776dc
# For easier management, discovery and configuration of
# networks with many individual transport instances,
# you can specify a network identity to be used across
# a set of instances. If sending interface discovery
# announces, these will all be signed by the specified
# network identity, and other nodes discovering your
# interfaces will be able to identify that they belong
# to the same network, even though they exist on different
# transport nodes.
# network_identity = ~/.reticulum/storage/identity/network
# It is possible to configure the global default interface
# gravity. This value will be used if an interface does
# not have a specific gravity value in its configuration.
# default_gravity = 0
# You can configure whether Reticulum should discover
# available interfaces from other Transport Instances over
# the network. If this option is enabled, Reticulum will
# collect interface information discovered from the network.
# discover_interfaces = no
# If you only want to discover interfaces from specific
# networks, you can provide a list of network identities
# from which to discover interfaces. If this option is not
# provided, interfaces will be discovered from all transport
# instances on all connected networks.
# interface_discovery_sources = 78616ff7c4b8d3886d67d494b440f333, cb127015e13aa6ea1e0a606cdc9123d0
# It is possible to automatically bring up and connect new
# interfaces discovered over the network. This option is
# disabled by default, but allows you to specify a maximum
# number of discovered interfaces to automatically connect.
# Additionally, if this option is enabled, Reticulum will
# also try to autoconnect available auto-discovered inter-
# faces on startup, up to the maximum number specified.
# autoconnect_discovered_interfaces = 0
# To prevent interface discovery spamming, a valid crypto-
# graphic stamp is required per announced interface. You
# can configure the minimum required value to accept as
# valid for discovered interfaces.
# required_discovery_value = 14
# You can disable allowing interfaces to be attached, de-
# tached or reloaded via rnstatus, for example if running
# on a shared system where users with normal RPC access
# should not be allowed to control interface states.
# enable_interface_management = no
# You can configure Reticulum to panic and forcibly close
# if an unrecoverable interface error occurs, such as the
# hardware device for an interface disappearing. This is
# an optional directive, and can be left out for brevity.
# This behaviour is disabled by default.
# panic_on_interface_error = no
# You can specify which mode discovered interfaces should
# be created with when auto-connecting.
# autoconnect_interface_mode = gw
# It is possible to allow announces from auto-connected
# interfaces to propagate announces to internal-mode
# interfaces, even if the auto-connected interface's mode
# would normally not allow for this.
# autoconnect_announces_to_internal = yes
# You can configure the gravity for discovered interfaces
# that are auto-connected.
# autoconnect_interface_gravity = 0
panic_on_interface_error = No
# When Transport is enabled, it is possible to allow the
@ -260,121 +152,11 @@ instance_name = default
# Transport Instance, and printed to the log at startup.
# Optional, and disabled by default.
# respond_to_probes = no
# You can publish your local list of blackholed identities
# for other transport instances to use for automatic,
# network-wide blackhole management.
# publish_blackhole = no
# List of remote transport identities from which to auto-
# matically source lists of blackholed identities.
#
# If you're connecting to a large external network, you
# can use one or more external blackhole list to block
# spammy and excessive announces onto your network. This
# funtionality is especially useful if you're hosting public
# entrypoints or gateways. The list source below provides a
# functional example, but better, more timely maintained
# lists probably exist in the community.
# blackhole_sources = 521c87a83afb8f29e4455e77930b973b
# You can set the interval in minutes at which remote
# blackhole sources are updated. Defaults to one hour.
# blackhole_update_interval = 60
# You can enable a hop limiter, that will drop all packets
# exceeding the specified maximum hop count.
# hop_limiter = 16
# When not running as a transport node, it is possible to
# force the use of the same, static transport identity
# for every instance start. Defaults to using a new identity
# at every instance start, if transport is disabled.
# static_transport_identity = no
[stamps]
# This section configures stamping behaviour and defaults.
# In most cases, you do not need to change this, but on
# constrained systems especially, tuning these parameters
# can be very beneficial
# During the 1.6.x phased transition to network stamp usage,
# it is possible to enable early opt-in on stamp generation
# and enforcement. RNS 1.6.0 includes all the required stamp
# generation and verification functionality, and actively
# validates stamps, while leaving generation, emission and
# enforcement disabled by default for destinations and PRs.
# Link requests will be stamped already from 1.6.0 if a cost
# has been announced by the destination. Version 1.6.1 will
# enable generation and emission by default, and version
# 1.6.2 will finally activate enforcement. Any of these
# options can be enabled early by configuration:
# stamp_destinations = yes
# stamp_prs = yes
# enforce_stamps = yes
# The default required stamp cost for an announce, and its
# destination to be considered valid. This applies both to
# announces received directly, and transported by this node,
# if transport is enabled. Any announced destination not
# meeting the cost will be ignored.
# destination_cost = 23.5
# The default value this instance will target when generating
# new destination stamps. An application may set a higher
# target, but applications requesting a lower value than the
# default will always generate at least the default value.
# Destination stamps are cached and reused, so a one-time
# high-value generation is usually a good idea.
# destination_value = 25.0
# The default required path request stamp cost. Path requests
# that do not meet the requirement are ignored.
# pr_cost = 15.8
# The default target value of path request stamps generated
# by this instance.
# pr_value = 15.8
# The default link request stamp cost to use, if one has not
# been announced by the destination the link is established
# to, or if it is unknown. In practice, all destinations will
# include their required link cost in each announce, and this
# option is rarely necessary.
# lr_cost = 12.5
# The maximum amount of time to spend attempting to generate
# a link request and path request stamps, in seconds.
# lr_stamp_timeout = 15
# pr_stamp_timeout = 15
respond_to_probes = No
[logging]
# Valid log levels are 0 through 8:
# Valid log levels are 0 through 7:
# 0: Log only critical information
# 1: Log errors and lower log levels
# 2: Log warnings and lower log levels
@ -382,17 +164,10 @@ instance_name = default
# 4: Log info and lower (this is the default)
# 5: Verbose logging
# 6: Debug logging
# 7: Path logging
# 8: Extreme logging
# 7: Extreme logging
loglevel = 4
# You can disable timestamp inclusion in logs. Useful if
# you want to use an external logging tool that provides
# its own timestamps or custom formatting.
# logtimestamps = no
# The interfaces section defines the physical and virtual
# interfaces Reticulum will use to communicate on. This

View file

@ -1,49 +0,0 @@
# Based on the original rnsh program by Aaron Heise (@acehoss)
# https://github.com/acehoss/rnsh - MIT License - Copyright (c) 2023 Aaron Heise
# This version of rnsh is included in RNS under the Reticulum License
#
# Reticulum License
#
# Copyright (c) 2016-2026 Mark Qvist
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# - The Software shall not be used in any kind of system which includes amongst
# its functions the ability to purposefully do harm to human beings.
#
# - The Software shall not be used, directly or indirectly, in the creation of
# an artificial intelligence, machine learning or language model training
# dataset, including but not limited to any use that contributes to the
# training or development of such a model or algorithm.
#
# - The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from ._version import __version__
import os
import glob
module_abs_filename = os.path.abspath(__file__)
module_dir = os.path.dirname(module_abs_filename)
py_modules = glob.glob(os.path.dirname(__file__)+"/*.py")
pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
so_modules = glob.glob(os.path.dirname(__file__)+"/*.so")
pyd_modules = glob.glob(os.path.dirname(__file__)+"/*.pyd")
modules = py_modules+pyc_modules+so_modules+pyd_modules
__all__ = list(set([os.path.basename(f).split(".")[0] for f in modules if not os.path.basename(f).split(".")[0] == "__init__"]))
def _get_version(): return __version__

Some files were not shown because too many files have changed in this diff Show more