Updated logging

README.md

@@ -1,10 +1,19 @@
 # Nomad Network - Communicate Freely
 
+> [!WARNING]  
+> Several hastily launched, incorrect, and in most cases entirely LLM-generated fakes of Nomad Network and associated tools are currently being circulated and marketed. Most of these "projects" violate the open source GPLv3 license that Nomad Network was published under, but claim independent ownership and license grants. Such claims or grants are **not** legally valid, and **not** recognized by the Nomad Network authors and copyright holders.
+>
+> Any claimed assertion of copyright or grant of license that such projects are making are [void grants](https://reticulum.network/manual/brandolinis.html#void-grants-legal-foundations-of-machine-generated-code). Both the violators themselves, and **all** downstream projects using the infringing derivatives, are **directly** and **personally** liable for the legal consequences.
+>
+> For more detailed information, see the relevant sections of the [Brandolini's Reference](https://reticulum.network/manual/brandolinis.html) chapter of the Reticulum Manual. For software and projects recognized by the Reticulum community, see the [Programs Using Reticulum](https://reticulum.network/manual/software.html) chapter.
+>
+> Do **not** use these "projects", they are a hazard to the entire ecosystem we have been carefully building over the last ten years.
+
 *This repository is [a public mirror](./MIRROR.md). All development is happening elsewhere.*
 
 Off-grid, resilient mesh communication with strong encryption, forward secrecy and extreme privacy.
 
-![Screenshot](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/1.png)
+![Screenshot](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/1a.png)
 
 Nomad Network allows you to build private and resilient communications platforms that are in complete control and ownership of the people that use them. No signups, no agreements, no handover of any data, no permissions and gatekeepers.
 
@@ -12,15 +21,23 @@ Nomad Network is build on [LXMF](https://github.com/markqvist/LXMF) and [Reticul
 
 Nomad Network does not need any connections to the public internet to work. In fact, it doesn't even need an IP or Ethernet network. You can use it entirely over packet radio, LoRa or even serial lines. But if you wish, you can bridge islanded networks over the Internet or private ethernet networks, or you can build networks running completely over the Internet. The choice is yours. Since Nomad Network uses Reticulum, it is efficient enough to run even over *extremely* low-bandwidth medium, and has been succesfully used over 300bps radio links.
 
-If you'd rather want to use an LXMF client with a graphical user interface, you may want to take a look at [Sideband](https://github.com/markqvist/sideband), which is available for Linux, Android and macOS.
+If you'd rather want to use an LXMF client with a graphical user interface, you may want to take a look at [Sideband](https://github.com/markqvist/sideband), which is available for Linux, Android, Windows and macOS.
+
+Other Nomad Network clients include:
+
+- [MeshChatX](https://meshchatx.com/)
+- [Columba (mobile)](https://github.com/torlando-tech/columba)
+- [Ren Browser](https://github.com/Quad4-Software/Ren-Browser)
 
 ## Notable Features
  - Encrypted messaging over packet-radio, LoRa, WiFi or anything else [Reticulum](https://github.com/markqvist/Reticulum) supports.
  - Zero-configuration, minimal-infrastructure mesh communication
  - Distributed and encrypted message store holds messages for offline users
  - Connectable nodes that can host pages and files
+ - Image rendering support in terminals that support it
  - Node-side generated pages with PHP, Python, bash or others
  - Built-in text-based browser for interacting with contents on nodes
+ - Built-in RRC client for live, many-to-many chat
  - An easy to use and bandwidth efficient markup language for writing pages
  - Page caching in browser
 
@@ -76,10 +93,10 @@ The first time the program is running, you will be presented with the **Guide se
 
 To use Nomad Network on packet radio or LoRa, you will need to configure your Reticulum installation to use any relevant packet radio TNCs or LoRa devices on your system. See the [Reticulum documentation](https://markqvist.github.io/Reticulum/manual/interfaces.html) for info. For a general introduction on how to set up such a system, take a look at [this post](https://unsigned.io/private-messaging-over-lora/).
 
-If you want to try Nomad Network without building your own physical network, you can connect to the [Unsigned.io RNS Testnet](https://github.com/markqvist/Reticulum#public-testnet) over the Internet, where there is already some Nomad Network and LXMF activity. If you connect to the testnet, you can leave nomadnet running for a while and wait for it to receive announces from other nodes on the network that host pages or services, or you can try connecting directly to some nodes listed here:
+If you want to try Nomad Network without building your own physical network, you can connect to the [distributed backbone](https://reticulum.network/manual/gettingstartedfast.html#bootstrapping-connectivity) over the Internet, where there is already quite a bit of Nomad Network and LXMF activity. If you connect to the testnet, you can leave nomadnet running for a while and wait for it to receive announces from other nodes on the network that host pages or services, or you can try connecting directly to some nodes listed here:
 
- - `abb3ebcd03cb2388a838e70c001291f9` Dublin Hub Testnet Node
+ - `9ce92808be498e9e05590ff27cbfdfe4` The rns.recipes forum
- - `ea6a715f814bdc37e56f80c34da6ad51` Frankfurt Hub Testnet Node
+ - `a4a5e861626ce97c9aa544d9ecdf6d22` rmap.world
 
 To browse pages on a node that is not currently known, open the URL dialog in the `Network` section of the program by pressing `Ctrl+U`, paste or enter the address and select `Go` or press enter. Nomadnet will attempt to discover and connect to the requested node.
 
@@ -88,42 +105,12 @@ You can install Nomad Network on Android using Termux, but there's a few more co
 
 For a native Android application with a graphical user interface, have a look at [Sideband](https://github.com/markqvist/Sideband).
 
-### Docker Images
-
-Nomad Network is automatically published as a docker image on Github Packages. Image tags are one of either `master` (for the very latest commit) or the version number (eg `0.2.0`) for a specific release.
-
-```sh
-$ docker pull ghcr.io/markqvist/nomadnet:master
-
-# Run nomadnet interactively in a container
-$ docker run -it ghcr.io/markqvist/nomadnet:master --textui
-
-# Run nomadnet as a daemon, using config stored on the host machine in specified
-# directories, and connect the containers network to the host network (which will
-# allow the default AutoInterface to automatically peer with other discovered
-# Reticulum instances).
-$ docker run -d \
-  -v /local/path/nomadnetconfigdir/:/root/.nomadnetwork/ \
-  -v /local/path/reticulumconfigdir/:/root/.reticulum/ \
-  --network host
-  ghcr.io/markqvist/nomadnet:master
-
-# You can also keep the network of the container isolated from the host, but you
-# will need to manually configure one or more Reticulum interfaces to reach other
-# nodes in a network, by editing the Reticulum configuration file.
-$ docker run -d \
-  -v /local/path/nomadnetconfigdir/:/root/.nomadnetwork/ \
-  -v /local/path/reticulumconfigdir/:/root/.reticulum/ \
-  ghcr.io/markqvist/nomadnet:master
-
-# Send daemon log output to console instead of file
-$ docker run -i ghcr.io/markqvist/nomadnet:master --daemon --console
-```
 
 ## Tools & Extensions
 
 Nomad Network is a very flexible and extensible platform, and a variety of community-provided tools, utilities and node-side extensions exist:
 
+- [Retipedia](https://github.com/RFnexus/Retipedia)
 - [NomadForum](https://codeberg.org/AutumnSpark1226/nomadForum) ([GitHub mirror](https://github.com/AutumnSpark1226/nomadForum))
 - [NomadForecast](https://github.com/faragher/NomadForecast)
 - [micron-blog](https://github.com/randogoth/micron-blog)
@@ -137,9 +124,6 @@ Nomad Network is a very flexible and extensible platform, and a variety of commu
 - [POPR](https://github.com/faragher/POPR)
 - [LXMF Tools](https://github.com/SebastianObi/LXMF-Tools)
 
-## Help & Discussion
-
-For help requests, discussion, sharing ideas or anything else related to Nomad Network, please have a look at the [Nomad Network discussions pages](https://github.com/markqvist/Reticulum/discussions/categories/nomad-network).
 
 ## Support Nomad Network
 You can help support the continued development of open, free and private communications systems by donating via one of the following channels:
@@ -161,32 +145,12 @@ You can help support the continued development of open, free and private communi
 - Ko-Fi: https://ko-fi.com/markqvist
 
 
-## Development Roadmap
-
-- New major features
-    - Network-wide propagated bulletins and discussion threads
-    - Collaborative maps and geospatial information sharing
-- Minor improvements and fixes
-    - Link status (RSSI and SNR) in conversation or conv list
-    - Ctrl-M shorcut for jumping to menu
-    - Share node with other users / send node info to user
-    - Fix internal editor failing on some OSes with no "editor" alias
-    - Possibly add a required-width header
-    - Improve browser handling of remote link close
-    - Better navigation handling when requests fail (also because of closed links)
-    - Retry failed messages mechanism
-    - Re-arrange buttons to be more consistent
-    - Term compatibility notice in readme
-    - Selected icon in conversation list
-    - Possibly a Search Local Nodes function
-    - Possibly add via entry in node info box, next to distance
-
 ## Caveat Emptor
-Nomad Network is beta software, and should be considered as such. While it has been built with cryptography best-practices very foremost in mind, it _has not_ been externally security audited, and there could very well be privacy-breaking bugs. If you want to help out, or help sponsor an audit, please do get in touch.
+Nomad Network is beta software, and should be considered as such. Use at your own risk and responsibility.
 
 ## Screenshots
 
-![Screenshot 1](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/1.png)
+![Screenshot 1](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/1a.png)
 
 ![Screenshot 2](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/2.png)
 
@@ -195,3 +159,5 @@ Nomad Network is beta software, and should be considered as such. While it has b
 ![Screenshot 4](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/4.png)
 
 ![Screenshot 5](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/5.png)
+
+![Screenshot 6](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/6.png)

README.mu

@@ -16,6 +16,7 @@ If you'd rather want to use an LXMF client with a graphical user interface, you
  • Zero-configuration, minimal-infrastructure mesh communication
  • Distributed and encrypted message store holds messages for offline users
  • Connectable nodes that can host pages and files
+ • Image rendering support in terminals that support it
  • Node-side generated pages with PHP, Python, bash or others
  • Built-in text-based browser for interacting with contents on nodes
  • Built-in RRC client for live, many-to-many chat
@@ -119,4 +120,4 @@ For this to be possible, I need your help. Please support the continued developm
 
 >> Caveat Emptor
 
-Nomad Network is experimental software, and should be considered as such. While it has been built with cryptography best-practices very foremost in mind, it `!has not`! been externally security audited, and there could very well be privacy-breaking bugs. Use at your own risk and responsibility.
+Nomad Network is experimental software, and should be considered as such. Use at your own risk and responsibility.

nomadnet/Directory.py

@@ -6,7 +6,7 @@ import nomadnet
 import threading
 import RNS.vendor.umsgpack as msgpack
 
-from LXMF import pn_announce_data_is_valid
+from LXMF import pn_announce_data_is_valid, PN_META_IMPL_NAME, PN_META_VERSION
 from nomadnet.util import strip_modifiers
 from nomadnet.util import sanitize_name
 
@@ -21,7 +21,10 @@ class PNAnnounceHandler:
                 data = msgpack.unpackb(app_data)
 
                 if data[2] == True:
-                    RNS.log("Received active propagation node announce from "+RNS.prettyhexrep(destination_hash), RNS.LOG_DEBUG)
+                    metadata = data[6]
+                    if not PN_META_IMPL_NAME in metadata or not PN_META_VERSION in metadata: impl_str = ""
+                    else: impl_str = f" running {metadata[PN_META_IMPL_NAME]} {metadata[PN_META_VERSION]}"
+                    RNS.log(f"Received active propagation node announce from {RNS.prettyhexrep(destination_hash)}{impl_str}", RNS.LOG_DEBUG)
 
                     associated_peer = RNS.Destination.hash_from_name_and_identity("lxmf.delivery", announced_identity)
                     associated_node = RNS.Destination.hash_from_name_and_identity("nomadnetwork.node", announced_identity)

nomadnet/RRC.py

@@ -118,7 +118,7 @@ def _msg_id():
 # greedy .+ intentionally captures nicks containing parens like "user (alt) (deadbeefcafe)"
 _WHO_ENTRY_RE = re.compile(
     r"(?:^|,\s)"
-    r"(?:(?P<bh>[0-9a-fA-F]{32})|(?P<nick>.+?)\s\((?P<np>[0-9a-fA-F]{12})\))"
+    r"(?:(?P<bh>[0-9a-fA-F]{64})|(?P<nick>.+?)\s\((?P<np>[0-9a-fA-F]{12})\))"
     r"(?=,\s|$)"
 )
 
@@ -170,6 +170,16 @@ def _parse_room_list_notice(text):
     return rooms
 
 
+# On low-MTU links the hub may deliver a /list reply as a sequence of
+# chunked NOTICE packets (one line per packet) instead of a single notice
+# or a resource transfer. The chunks carry no continuation marker, but each
+# line has a recognisable shape. Lines matching it, arriving within a short
+# window after a /list header, are treated as continuation fragments and
+# merged into the room list instead of being recorded as chat messages.
+_LIST_FRAGMENT_RE = re.compile(r"^  (?P<name>\S+?)(?: - (?P<topic>.*))?$")
+LIST_FRAGMENT_WINDOW = 60.0
+
+
 def _make_envelope(msg_type, src, room=None, body=None, nick=None, mid=None, ts=None):
     env = {
         K_V:   RRC_VERSION,
@@ -235,6 +245,11 @@ class RRCHub:
         self.mention_rooms = set()
         self.members = {}
         self.nicks = {}
+        # Display-only member entries derived from 12-hex identity prefixes
+        # in /who replies. Populated when the full identity hash of a nicked
+        # member is not yet known, and promoted to regular members as soon
+        # as the full hash is learned from room traffic or join fanouts.
+        self.prefix_members = {}
 
         self.auto_reconnect = False
         self.auto_list = False
@@ -259,6 +274,7 @@ class RRCHub:
         self.available_rooms = {}
         self._silent_list_pending = 0
         self._silent_who_rooms = set()
+        self._list_fragment_until = 0.0
 
         self.nick_override = None
         self._pending_joins = set()
@@ -290,6 +306,7 @@ class RRCHub:
             self.unread_rooms.discard(r)
             self.mention_rooms.discard(r)
             self.members.pop(r, None)
+            self.prefix_members.pop(r, None)
         self._delete_history(r)
         self.manager.save()
         self.manager._notify_change(self)
@@ -306,7 +323,23 @@ class RRCHub:
 
     def get_members(self, room):
         with self._lock:
-            return list(self.members.get(room, set()))
+            return list(self.members.get(room, set()) | self.prefix_members.get(room, set()))
+
+    def _upgrade_prefix_member(self, room, full_hash):
+        # Once a full identity hash is learned from room traffic or join
+        # fanouts, promote any display-only prefix member that matches it,
+        # so the entry becomes a full, addressable member. The prefix
+        # entry's nickname is carried over unless the promoting source
+        # provides one.
+        pset = self.prefix_members.get(room)
+        if pset:
+            for p in list(pset):
+                if full_hash.startswith(p):
+                    pset.discard(p)
+                    nick = self.nicks.pop(p, None)
+                    if nick and full_hash not in self.nicks:
+                        self.nicks[full_hash] = nick
+                    break
 
     def display_name_for(self, peer):
         if not isinstance(peer, (bytes, bytearray)):
@@ -467,6 +500,7 @@ class RRCHub:
             self.welcomed = False
             self.motd = None
             self.members.clear()
+            self.prefix_members.clear()
             self._resource_expectations.clear()
             self._pending_joins.clear()
             self._pending_parts.clear()
@@ -844,23 +878,50 @@ class RRCHub:
     def _process_notice_text(self, text):
         # Parse hub service notices (/list and /who replies) regardless of
         # whether they arrived as a packet or a resource transfer. Returns
-        # True when the notice was consumed silently (auto /list or /who)
+        # True when the notice was consumed silently (auto /list or /who,
-        # and should not be recorded to the message log.
+        # or a /list continuation fragment) and should not be recorded to
+        # the message log.
         parsed = _parse_room_list_notice(text)
         if parsed is not None:
             with self._lock:
                 self.available_rooms = {strip_modifiers(name): strip_modifiers(topic) for name, topic in parsed.items()}
+                # The first line of a /list reply opens a short window in
+                # which continuation fragments (chunked NOTICE packets on
+                # low-MTU links) are absorbed into the room list.
+                self._list_fragment_until = time.monotonic()+LIST_FRAGMENT_WINDOW
                 silent = self._silent_list_pending > 0
                 if silent:
                     self._silent_list_pending -= 1
             self.manager._notify_change(self)
-            if silent:
+            if silent: return True
+        with self._lock:
+            list_window_active = time.monotonic() < self._list_fragment_until
+        if list_window_active:
+            fm = _LIST_FRAGMENT_RE.match(text)
+            if fm is not None:
+                with self._lock:
+                    frag_name = fm.group("name").strip().lower()
+                    frag_topic = fm.group("topic")
+                    if frag_topic:
+                        frag_topic = strip_modifiers(frag_topic.strip()) or None
+                    else:
+                        frag_topic = None
+                    self.available_rooms[frag_name] = frag_topic
+                    self._list_fragment_until = time.monotonic()+LIST_FRAGMENT_WINDOW
+                self.manager._notify_change(self)
                 return True
         parsed_who = _parse_who_notice(text)
         if parsed_who is not None:
             who_room, who_entries = parsed_who
             with self._lock:
                 members = self.members.setdefault(who_room, set())
+                # Treat /who replies as fresh roster snapshots: replace any
+                # display-only prefix members from earlier replies before
+                # applying this one.
+                prefix_members = self.prefix_members.setdefault(who_room, set())
+                for p in list(prefix_members):
+                    self.nicks.pop(p, None)
+                prefix_members.clear()
                 for nick, hash_hex in who_entries:
                     try:
                         hash_bytes = bytes.fromhex(hash_hex)
@@ -868,11 +929,23 @@ class RRCHub:
                         continue
                     if nick is None:
                         members.add(hash_bytes)
+                        self._upgrade_prefix_member(who_room, hash_bytes)
                         continue
+                    known = None
                     for ph in members:
                         if ph.startswith(hash_bytes):
-                            self.nicks[ph] = nick
+                            known = ph
                             break
+                    if known is not None:
+                        self.nicks[known] = nick
+                    else:
+                        # A /who line only carries a 12-hex identity prefix
+                        # for nicked members. If the full hash is not yet
+                        # known, keep the prefix as a display-only member.
+                        # It is promoted to a full member as soon as the
+                        # hash is learned from room traffic or fanouts.
+                        prefix_members.add(hash_bytes)
+                        self.nicks[hash_bytes] = nick
                 silent_who = who_room in self._silent_who_rooms
                 if silent_who:
                     self._silent_who_rooms.discard(who_room)
@@ -987,8 +1060,10 @@ class RRCHub:
                     members = self.members.setdefault(r, set())
                     for h in body_hashes:
                         members.add(h)
+                        self._upgrade_prefix_member(r, h)
                     if own_hash is not None:
                         members.add(own_hash)
+                        self._upgrade_prefix_member(r, own_hash)
 
                     # rrcd 0.3.2 attaches advisory K_NICK to fanout JOINED; learn it
                     # so display_name_for() can render the nick instead of a hash prefix.
@@ -1043,12 +1118,19 @@ class RRCHub:
                             self.nicks[ph] = parter_nick
 
                     members = self.members.get(r)
+                    pset = self.prefix_members.get(r)
                     if members is not None:
                         for h in body_hashes:
                             members.discard(h)
+                            if pset:
+                                for p in list(pset):
+                                    if h.startswith(p):
+                                        pset.discard(p)
+                                        self.nicks.pop(p, None)
                     if self_part:
                         self.rooms.discard(r)
                         self.members.pop(r, None)
+                        self.prefix_members.pop(r, None)
 
                 if self_part:
                     self.manager.save()
@@ -1075,7 +1157,9 @@ class RRCHub:
                 with self._lock:
                     self.nicks[bytes(src)] = nick
                     if isinstance(room, str) and room:
-                        self.members.setdefault(room.strip().lower(), set()).add(bytes(src))
+                        rn = room.strip().lower()
+                        self.members.setdefault(rn, set()).add(bytes(src))
+                        self._upgrade_prefix_member(rn, bytes(src))
             if isinstance(body, str):
                 msg = RRCMessage(
                     "msg",
@@ -1108,7 +1192,9 @@ class RRCHub:
                 with self._lock:
                     self.nicks[bytes(src)] = nick
                     if isinstance(room, str) and room:
-                        self.members.setdefault(room.strip().lower(), set()).add(bytes(src))
+                        rn = room.strip().lower()
+                        self.members.setdefault(rn, set()).add(bytes(src))
+                        self._upgrade_prefix_member(rn, bytes(src))
             if isinstance(body, str):
                 msg = RRCMessage(
                     "action",
@@ -1132,12 +1218,47 @@ class RRCHub:
             room = env.get(K_ROOM)
             src  = env.get(K_SRC)
             if isinstance(body, str):
-                if self._process_notice_text(body):
+                #################### HACK ZONE #####################
-                    return
+                # This is the kind of crazy mess that happens when
+                # an LLM designs a chat protocol, and the client
+                # maintainer is too open-minded, decides to go with
+                # it, and now is responsible for the fundamental
+                # architectural chaos.
+                #
+                # Yes, I love RRC, but trying to figure out an
+                # elegant way to solve the underlying issues that
+                # make this kind of mess necessary revealed the
+                # following answer: There is none.
+                #
+                # There are so many bugs, weird cases, architectural
+                # mismatches, impossible situations and general
+                # weirdness here, that I am just not going to go and
+                # attempt to sort it all out. Sorry, I don't have the
+                # time, so a hack like this is what it will be. This
+                # makes things actually work semi-expectedly, even
+                # over low-bandwidth links and such, so hacky fix it
+                # is. Until we have something better, this will do
+                # just fine. On a related note, I was considering
+                # taking up maintenance of rrcd, now that it seems to
+                # not really be worked on or updated anymore, but
+                # looking at the entire system, I think it's better
+                # left as it; it works fine, albeit with a few quirks
+                # for it's purpose. My future work in chat-related
+                # protocols on Reticulum will be focused on a better
+                # architecture, incorporating everything we learned
+                # from RRC, and making it Reticulum-native from the
+                # start, instead of mirroring IRC too closely.
+                #  - Mark
+                was_parsed = self._process_notice_text(body)
                 room_n = room.strip().lower() if isinstance(room, str) else None
                 if room_n is None and isinstance(body, str) and body.strip():
-                    with self._lock:
+                    probably_motd = True
-                        self.motd = strip_modifiers(body)
+                    if was_parsed: probably_motd = False
+                    if body.lstrip().lower().startswith("registered public rooms:"): probably_motd = False; was_parsed = True
+                    if body.lstrip().lower().startswith("members in "): probably_motd = False; was_parsed = True
+                    if probably_motd:
+                        with self._lock: self.motd = strip_modifiers(body)
+                        was_parsed = False
                     self.manager._notify_change(self)
                 msg = RRCMessage(
                     "notice",
@@ -1147,7 +1268,7 @@ class RRCHub:
                     body,
                     _now_ms(),
                 )
-                self._record_notice(msg)
+                if not was_parsed: self._record_notice(msg)
             return
 
         if t == T_ERROR:

nomadnet/_version.py

@@ -1 +1 @@
-__version__ = "1.4.2"
+__version__ = "1.4.4"

nomadnet/ui/textui/Browser.py

@@ -93,14 +93,55 @@ class PageList(urwid.ListBox):
         self._invalidate()
 
     def _layout(self, maxcol):
-        if maxcol not in self._page_layouts:
+        offsets = self._page_layouts.get(maxcol)
+        if offsets is None or len(offsets) != len(self.body) + 1:
             layout_started = time.time()
             offsets = [0]
             for widget in self.body:
                 offsets.append(offsets[-1] + widget.rows((maxcol,), True))
             self._page_layouts[maxcol] = offsets
             # RNS.log(f" rebuilt {len(self.body)} widgets, {offsets[-1]} rows, width {maxcol}, {(time.time()-layout_started)*1000:.1f}ms", RNS.LOG_DEBUG)
-        return self._page_layouts[maxcol]
+        return offsets
+
+    def calculate_visible(self, size, focus=False):
+        visible = super().calculate_visible(size, focus)
+        if visible is not None and self.body:
+            self._sync_layout(size[0], visible)
+        return visible
+
+    def _sync_layout(self, maxcol, visible):
+        # Validate the cached layout against the live geometry of the
+        # currently visible widgets, and delta-update the cached offsets when a
+        # visible widget has changed height since the layout was built without
+        # rescanning the entire document on every render.
+
+        if not self.body or not self._page_layouts: return
+
+        offsets = self._page_layouts.get(maxcol)
+        if offsets is None or len(offsets) != len(self.body) + 1: return
+
+        def sync_position(pos, live_rows):
+            if pos < 0 or pos >= len(self.body) or live_rows < 0: return
+            cached_rows = offsets[pos + 1] - offsets[pos]
+            if cached_rows == live_rows: return
+
+            delta = live_rows - cached_rows
+            for i in range(pos + 1, len(offsets)): offsets[i] += delta
+
+            # Other cached widths may also be affected by this change. Sync
+            # them using the changed widget's live rows at their widths.
+            for width, other in list(self._page_layouts.items()):
+                if width == maxcol or len(other) != len(offsets): continue
+                other_cached = other[pos + 1] - other[pos]
+                other_live = self.body[pos].rows((width,), True)
+                if other_cached == other_live: continue
+                other_delta = other_live - other_cached
+                for i in range(pos + 1, len(other)): other[i] += other_delta
+
+        middle = visible.middle
+        sync_position(middle.focus_pos, middle.focus_rows)
+        for item in visible.top.fill:    sync_position(item.position, item.rows)
+        for item in visible.bottom.fill: sync_position(item.position, item.rows)
 
     def render(self, size, focus=False):
         self._page_size = size
@@ -133,7 +174,19 @@ class PageList(urwid.ListBox):
         position = max(0, min(int(position), max(0, offsets[-1] - maxrow)))
         pos = bisect.bisect_left(offsets, position, 0, len(self.body))
         if pos == len(self.body) or offsets[pos] - position >= maxrow: pos -= 1
-        self.change_focus(size, pos, offsets[pos] - position)
+
+        # Guard against the cached layout being inconsistent
+        # with the live widget geometry. Without it, change_focus
+        # can be called with an offset that the live target widget
+        # cannot satisfy, raising ListBoxError.
+        pos = max(0, min(pos, len(self.body) - 1))
+        offset_inset = offsets[pos] - position
+        if offset_inset > maxrow - 1: offset_inset = maxrow - 1
+        elif offset_inset < 0:
+            tgt_rows = max(1, self.body[pos].rows((maxcol,), True))
+            if offset_inset < 1 - tgt_rows: offset_inset = 1 - tgt_rows
+
+        self.change_focus(size, pos, offset_inset)
 
     def top_position(self, size=None):
         size = size or self._page_size

nomadnet/ui/textui/MicronParser.py

@@ -603,14 +603,14 @@ def parse_line(line, state, url_delegate):
                 if len(line) == 2:
                     divider_char = line[1]
                     # Control characters don't make sense here and otherwise crash nomadnet
-                    if ord(divider_char) < 32:
+                    if ord(divider_char) < 32: divider_char = "\u2500"
-                        divider_char = "\u2500"
+                    # Check if the divider char is renderable
-                else:
+                    try: urwid.Divider(divider_char).render((30,))
-                    divider_char = "\u2500"
+                    except Exception as e: divider_char = "\u2500"
-                if state["depth"] == 0:
+                else: divider_char = "\u2500"
-                    return [urwid.Divider(divider_char)]
+
-                else:
+                if state["depth"] == 0: return [urwid.Divider(divider_char)]
-                    return [urwid.Padding(urwid.Divider(divider_char), left=left_indent(state), right=right_indent(state))]
+                else:                   return [urwid.Padding(urwid.Divider(divider_char), left=left_indent(state), right=right_indent(state))]
 
         output = make_output(state, line, url_delegate, pre_escape)
 

nomadnet/ui/textui/images/_termlib.py

@@ -10,9 +10,11 @@ import os
 import select
 import shutil
 import sys
-import termios
 import time
 
+try: import termios
+except: termios = None
+
 from . import _ctlseqs
 
 QUERY_TIMEOUT = 0.3
@@ -43,7 +45,7 @@ def _open_tty():
 def query_tty(request, timeout=QUERY_TIMEOUT):
     # Sends a query to the terminal and returns the raw response.
     global _query_failed
-    if _query_failed: return b""
+    if _query_failed or not termios: return b""
 
     fd = _open_tty()
     if fd == -1:
@@ -121,6 +123,7 @@ def _cell_size_from_winsize(buf):
 
 def _query_cell_size():
     # Determines the current cell size in pixels, or None if unknown.
+    if not termios: return None
     fd = _open_tty()
     if fd != -1:
         # First, try the TIOCGWINSZ pixel fields (fast, no query)

nomadnet/vendor/Scrollable.py

@@ -407,6 +407,13 @@ class ScrollBar(urwid.WidgetDecoration):
         pos = ow_base.get_scrollpos(ow_size, focus)
         posmax = ow_rows_max - maxrow
 
+        # Clamp position to valid scroll range. The position
+        # reported by the underlying widget can transiently
+        # fall outside [0, posmax] if its layout state is
+        # inconsistent.
+        if pos < 0: pos = 0
+        elif pos > posmax: pos = posmax
+
         # Thumb shrinks/grows according to the ratio of
         # <number of visible lines> / <number of total lines>
         thumb_weight = min(1, maxrow / max(1, ow_rows_max))
@@ -418,9 +425,12 @@ class ScrollBar(urwid.WidgetDecoration):
         if top_height == 0 and top_weight > 0:
             top_height = 1
 
-        # Bottom part is remaining space
+        # Bottom part is remaining space. Guard against
+        # negative values so the scrollbar canvas can
+        # never be taller than maxrow.
         bottom_height = maxrow - thumb_height - top_height
-        assert thumb_height + top_height + bottom_height == maxrow
+        if bottom_height < 0: bottom_height = 0
+        assert thumb_height + top_height + bottom_height <= maxrow
 
         # Create scrollbar canvas
         # Creating SolidCanvases of correct height may result in "cviews do not

setup.py

@@ -36,6 +36,6 @@ setuptools.setup(
     entry_points= {
         'console_scripts': ['nomadnet=nomadnet.nomadnet:main']
     },
-    install_requires=["rns>=1.5.3", "lxmf>=1.1.1", "urwid>=3.0.5", "qrcode"],
+    install_requires=["rns>=1.5.5", "lxmf>=1.2.0", "urwid>=3.0.5", "qrcode"],
     python_requires=">=3.8",
 )