Research mode switches to nine kind-generic tools: search_files,
get_file, read_content, search_zims, get_zim_entry, get_inventory,
list_files, read_file, search_places. Tool definitions drop from 20
tools / 8,947 chars (~2,240 tokens) to 9 / 3,881 (~970) on every
request, a sixth of the 8k context the small tier runs with instead of
a quarter. The remaining descriptions are trimmed toward 300 chars.
The replaced per-kind endpoints are removed (four searches, three
getters, three readers, search_suggestions, the slug inventory route);
their tests move onto the generic routes. The research prompt, its
worked example, and the empty-search hint no longer tell the model to
list collections first: channels resolve by name from the library map
that is already in the prompt. list_collections, list_tags, and
get_map_overview stay as endpoints for the MCP wrappers but leave the
chat allowlist; the MCP search_suggestions tool goes since search_files
returns the same matches inline.
scripts/ai_eval.py expects search_files and treats "what channels do I
have" as answerable from context. On the 1.7B model against the dev
library both catalogs score 14/18, but the compact one answers with one
tool call where the old one needed up to five, no longer invents the
example's channel, and keeps kinds straight when asked for videos.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Response-quality work, all aimed at giving the model enough context to
answer accurately:
- Every chat's system prompt now carries a live library map (today's
date, counts, and the exact names of channels, domains, zims, and
inventories, capped and cached 60s). Exact spellings are in context
before the first token, and the list_collections round-trip — minutes
on slow hardware — is usually unnecessary.
- The request is budgeted against the model's real context: oldest
turns are trimmed first, the current question is never dropped, and
an over-budget turn gets a clear "start a new conversation" error
instead of a silently truncated prompt. cache_prompt reuses the KV
cache for the stable prefix, which is most of the latency on a Pi.
- context_size plumbing fixed: selecting a model adopts its catalog
default (the 4B was running at 8k instead of 16k), and llama-server
runs --parallel 1 so the single user gets the whole context.
- temperature is configurable in ai.yaml (default 0.3 — small models
call tools far more reliably near 0.2-0.3 than at server defaults).
- One worked tool-use example per mode is prepended to every
conversation; small models imitate far better than they obey.
- scripts/ai_eval.py scores the assistant against a live WROLPi on
mechanical checks (finished, called the right tool first, included a
WROLPi link, no external URLs) so prompt changes stop being validated
by anecdote.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
