ai: ground answers in a live library map, budget the context

Response-quality work, all aimed at giving the model enough context to
answer accurately:

- Every chat's system prompt now carries a live library map (today's
  date, counts, and the exact names of channels, domains, zims, and
  inventories, capped and cached 60s).  Exact spellings are in context
  before the first token, and the list_collections round-trip — minutes
  on slow hardware — is usually unnecessary.
- The request is budgeted against the model's real context: oldest
  turns are trimmed first, the current question is never dropped, and
  an over-budget turn gets a clear "start a new conversation" error
  instead of a silently truncated prompt.  cache_prompt reuses the KV
  cache for the stable prefix, which is most of the latency on a Pi.
- context_size plumbing fixed: selecting a model adopts its catalog
  default (the 4B was running at 8k instead of 16k), and llama-server
  runs --parallel 1 so the single user gets the whole context.
- temperature is configurable in ai.yaml (default 0.3 — small models
  call tools far more reliably near 0.2-0.3 than at server defaults).
- One worked tool-use example per mode is prepended to every
  conversation; small models imitate far better than they obey.
- scripts/ai_eval.py scores the assistant against a live WROLPi on
  mechanical checks (finished, called the right tool first, included a
  WROLPi link, no external URLs) so prompt changes stop being validated
  by anecdote.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Roland 2026-09-02 16:48:19 -06:00
parent 82e11cbaba
commit ced78e181f
No known key found for this signature in database
GPG key ID: 1BA620CE6C9A4680
11 changed files with 502 additions and 7 deletions

View file

@ -46,9 +46,11 @@ esac
# --jinja enables tool-call chat templates; mmap is llama.cpp's default so cold RAM cost is low.
# Localhost only on native installs -- never exposed through Caddy.
# --parallel 1: WROLPi is single-user; one slot gets the whole context.
exec llama-server \
--host "${LLAMA_HOST}" \
--port "${LLAMA_PORT}" \
--jinja \
--parallel 1 \
-c "${CONTEXT_SIZE}" \
-m "${MODEL}"