ai: the /api/chat agent loop — SSE streaming, modes, tool execution
Tool definitions are generated in-process from the /api/ai blueprint's
own OpenAPI operations (operationId = tool name, description = tool
description, dataclass bodies + declared parameters = one flat JSON
schema) — the spec IS the catalog, nothing to drift. Manage endpoints
never become tools. A mode is a server-enforced system prompt + tool
allowlist (Help / Research / System per ai.md); the executor can only
reach allowlisted /api/ai endpoints, path parameters are encoded and
traversal-rejected so a crafted argument cannot escape an endpoint.
POST /api/chat streams SSE (status/token/tool_call/tool_result/done/
error), starts llama-server on demand, validates tool args server-side
with instructive errors the model can retry on, caps tool results with
a paging hint, and bounds the loop by ai.yaml max_tool_calls plus a
hard timeout. Caddy passes the stream unbuffered (flush_interval -1)
in both deployments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 21:47:50 -06:00
|
|
|
"""Chat modes: a mode is a system prompt plus a tool allowlist, enforced server-side in the agent
|
|
|
|
|
loop — the model cannot opt out. Tool names are the /api/ai blueprint's operationIds."""
|
|
|
|
|
|
|
|
|
|
COMMON_PROMPT = (
|
|
|
|
|
'You are the WROLPi assistant, running locally on the user\'s own offline WROLPi server.'
|
|
|
|
|
' WROLPi is an off-grid digital library: videos, archived web pages, ebooks, Zim encyclopedias,'
|
|
|
|
|
' maps, and inventories. You have no internet access.'
|
|
|
|
|
' Answer briefly and plainly. Use the provided tools to find real information before answering;'
|
|
|
|
|
' never invent library content or system state.'
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
MODES = {
|
|
|
|
|
'help': dict(
|
|
|
|
|
system_prompt=(
|
|
|
|
|
f'{COMMON_PROMPT}'
|
|
|
|
|
' Your job is to teach the user how to use WROLPi.'
|
2026-08-27 12:21:44 -06:00
|
|
|
' You do NOT know how WROLPi works from memory: before answering ANY question, you MUST'
|
|
|
|
|
' first call search_help, then read the matching page with get_help_doc, and answer only'
|
ai: the /api/chat agent loop — SSE streaming, modes, tool execution
Tool definitions are generated in-process from the /api/ai blueprint's
own OpenAPI operations (operationId = tool name, description = tool
description, dataclass bodies + declared parameters = one flat JSON
schema) — the spec IS the catalog, nothing to drift. Manage endpoints
never become tools. A mode is a server-enforced system prompt + tool
allowlist (Help / Research / System per ai.md); the executor can only
reach allowlisted /api/ai endpoints, path parameters are encoded and
traversal-rejected so a crafted argument cannot escape an endpoint.
POST /api/chat streams SSE (status/token/tool_call/tool_result/done/
error), starts llama-server on demand, validates tool args server-side
with instructive errors the model can retry on, caps tool results with
a paging hint, and bounds the loop by ai.yaml max_tool_calls plus a
hard timeout. Caddy passes the stream unbuffered (flush_interval -1)
in both deployments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 21:47:50 -06:00
|
|
|
' from what the docs actually say. Always include the link of every help page you used.'
|
|
|
|
|
' If the docs do not cover the question, say so.'
|
|
|
|
|
),
|
|
|
|
|
tools=(
|
|
|
|
|
'search_help',
|
|
|
|
|
'get_help_doc',
|
|
|
|
|
'get_system_status',
|
|
|
|
|
),
|
|
|
|
|
),
|
|
|
|
|
'research': dict(
|
|
|
|
|
system_prompt=(
|
|
|
|
|
f'{COMMON_PROMPT}'
|
|
|
|
|
' Your job is to be the librarian: find and summarize content from the user\'s library.'
|
ai: research mode runs on the compact tool catalog
Research mode switches to nine kind-generic tools: search_files,
get_file, read_content, search_zims, get_zim_entry, get_inventory,
list_files, read_file, search_places. Tool definitions drop from 20
tools / 8,947 chars (~2,240 tokens) to 9 / 3,881 (~970) on every
request, a sixth of the 8k context the small tier runs with instead of
a quarter. The remaining descriptions are trimmed toward 300 chars.
The replaced per-kind endpoints are removed (four searches, three
getters, three readers, search_suggestions, the slug inventory route);
their tests move onto the generic routes. The research prompt, its
worked example, and the empty-search hint no longer tell the model to
list collections first: channels resolve by name from the library map
that is already in the prompt. list_collections, list_tags, and
get_map_overview stay as endpoints for the MCP wrappers but leave the
chat allowlist; the MCP search_suggestions tool goes since search_files
returns the same matches inline.
scripts/ai_eval.py expects search_files and treats "what channels do I
have" as answerable from context. On the 1.7B model against the dev
library both catalogs score 14/18, but the compact one answers with one
tool call where the old one needed up to five, no longer invents the
example's channel, and keeps kinds straight when asked for videos.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 16:28:16 -06:00
|
|
|
' Search first with search_files, then read captions, page text, or entries with read_content'
|
|
|
|
|
' when the user wants detail. The library list above has the exact channel, website, and'
|
|
|
|
|
' Zim names (spellings often differ): pass a channel or domain name to search_files directly.'
|
|
|
|
|
' To summarize a channel or browse everything of one kind, call search_files WITHOUT search_str'
|
|
|
|
|
' and with the channel, domain, or kind; that lists the newest items.'
|
|
|
|
|
' When a search finds nothing, do not give up: retry with fewer or different words, or drop the'
|
|
|
|
|
' filters. When it offers matches, narrow with one of them.'
|
ai: the /api/chat agent loop — SSE streaming, modes, tool execution
Tool definitions are generated in-process from the /api/ai blueprint's
own OpenAPI operations (operationId = tool name, description = tool
description, dataclass bodies + declared parameters = one flat JSON
schema) — the spec IS the catalog, nothing to drift. Manage endpoints
never become tools. A mode is a server-enforced system prompt + tool
allowlist (Help / Research / System per ai.md); the executor can only
reach allowlisted /api/ai endpoints, path parameters are encoded and
traversal-rejected so a crafted argument cannot escape an endpoint.
POST /api/chat streams SSE (status/token/tool_call/tool_result/done/
error), starts llama-server on demand, validates tool args server-side
with instructive errors the model can retry on, caps tool results with
a paging hint, and bounds the loop by ai.yaml max_tool_calls plus a
hard timeout. Caddy passes the stream unbuffered (flush_interval -1)
in both deployments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 21:47:50 -06:00
|
|
|
' ALWAYS include the WROLPi link for every item you mention, exactly as the tools return it.'
|
|
|
|
|
' Never give external URLs. When a search returns a large total, narrow the query instead'
|
2026-08-27 12:44:37 -06:00
|
|
|
' of paging.'
|
ai: the /api/chat agent loop — SSE streaming, modes, tool execution
Tool definitions are generated in-process from the /api/ai blueprint's
own OpenAPI operations (operationId = tool name, description = tool
description, dataclass bodies + declared parameters = one flat JSON
schema) — the spec IS the catalog, nothing to drift. Manage endpoints
never become tools. A mode is a server-enforced system prompt + tool
allowlist (Help / Research / System per ai.md); the executor can only
reach allowlisted /api/ai endpoints, path parameters are encoded and
traversal-rejected so a crafted argument cannot escape an endpoint.
POST /api/chat streams SSE (status/token/tool_call/tool_result/done/
error), starts llama-server on demand, validates tool args server-side
with instructive errors the model can retry on, caps tool results with
a paging hint, and bounds the loop by ai.yaml max_tool_calls plus a
hard timeout. Caddy passes the stream unbuffered (flush_interval -1)
in both deployments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 21:47:50 -06:00
|
|
|
),
|
|
|
|
|
tools=(
|
ai: research mode runs on the compact tool catalog
Research mode switches to nine kind-generic tools: search_files,
get_file, read_content, search_zims, get_zim_entry, get_inventory,
list_files, read_file, search_places. Tool definitions drop from 20
tools / 8,947 chars (~2,240 tokens) to 9 / 3,881 (~970) on every
request, a sixth of the 8k context the small tier runs with instead of
a quarter. The remaining descriptions are trimmed toward 300 chars.
The replaced per-kind endpoints are removed (four searches, three
getters, three readers, search_suggestions, the slug inventory route);
their tests move onto the generic routes. The research prompt, its
worked example, and the empty-search hint no longer tell the model to
list collections first: channels resolve by name from the library map
that is already in the prompt. list_collections, list_tags, and
get_map_overview stay as endpoints for the MCP wrappers but leave the
chat allowlist; the MCP search_suggestions tool goes since search_files
returns the same matches inline.
scripts/ai_eval.py expects search_files and treats "what channels do I
have" as answerable from context. On the 1.7B model against the dev
library both catalogs score 14/18, but the compact one answers with one
tool call where the old one needed up to five, no longer invents the
example's channel, and keeps kinds straight when asked for videos.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 16:28:16 -06:00
|
|
|
'search_files',
|
|
|
|
|
'get_file',
|
|
|
|
|
'read_content',
|
ai: the /api/chat agent loop — SSE streaming, modes, tool execution
Tool definitions are generated in-process from the /api/ai blueprint's
own OpenAPI operations (operationId = tool name, description = tool
description, dataclass bodies + declared parameters = one flat JSON
schema) — the spec IS the catalog, nothing to drift. Manage endpoints
never become tools. A mode is a server-enforced system prompt + tool
allowlist (Help / Research / System per ai.md); the executor can only
reach allowlisted /api/ai endpoints, path parameters are encoded and
traversal-rejected so a crafted argument cannot escape an endpoint.
POST /api/chat streams SSE (status/token/tool_call/tool_result/done/
error), starts llama-server on demand, validates tool args server-side
with instructive errors the model can retry on, caps tool results with
a paging hint, and bounds the loop by ai.yaml max_tool_calls plus a
hard timeout. Caddy passes the stream unbuffered (flush_interval -1)
in both deployments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 21:47:50 -06:00
|
|
|
'search_zims',
|
|
|
|
|
'get_zim_entry',
|
|
|
|
|
'get_inventory',
|
2026-09-10 15:32:27 -06:00
|
|
|
'list_files',
|
ai: the /api/chat agent loop — SSE streaming, modes, tool execution
Tool definitions are generated in-process from the /api/ai blueprint's
own OpenAPI operations (operationId = tool name, description = tool
description, dataclass bodies + declared parameters = one flat JSON
schema) — the spec IS the catalog, nothing to drift. Manage endpoints
never become tools. A mode is a server-enforced system prompt + tool
allowlist (Help / Research / System per ai.md); the executor can only
reach allowlisted /api/ai endpoints, path parameters are encoded and
traversal-rejected so a crafted argument cannot escape an endpoint.
POST /api/chat streams SSE (status/token/tool_call/tool_result/done/
error), starts llama-server on demand, validates tool args server-side
with instructive errors the model can retry on, caps tool results with
a paging hint, and bounds the loop by ai.yaml max_tool_calls plus a
hard timeout. Caddy passes the stream unbuffered (flush_interval -1)
in both deployments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 21:47:50 -06:00
|
|
|
'read_file',
|
ai: tags, downloads, map, and search-preview tools
Five new read-only /api/ai endpoints, each with a matching wrolpi_mcp
proxy tool: list_tags (tags with what they are applied to, plus recent
names), search_suggestions (matching channels/domains/authors/subjects
and file/Zim result estimates), list_downloads (queue summary and lean
recurring/one-time entries with relative destinations; status filter;
errors keep their tail, which holds the exception message),
get_map_overview (map files, search-index state, subscriptions, pins
with map links), and search_places (place search with map links and
optional proximity ranking).
Research mode allowlists the library-facing four. Nothing here can
start, stop, or retry a download.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 15:48:11 -06:00
|
|
|
'search_places',
|
ai: the /api/chat agent loop — SSE streaming, modes, tool execution
Tool definitions are generated in-process from the /api/ai blueprint's
own OpenAPI operations (operationId = tool name, description = tool
description, dataclass bodies + declared parameters = one flat JSON
schema) — the spec IS the catalog, nothing to drift. Manage endpoints
never become tools. A mode is a server-enforced system prompt + tool
allowlist (Help / Research / System per ai.md); the executor can only
reach allowlisted /api/ai endpoints, path parameters are encoded and
traversal-rejected so a crafted argument cannot escape an endpoint.
POST /api/chat streams SSE (status/token/tool_call/tool_result/done/
error), starts llama-server on demand, validates tool args server-side
with instructive errors the model can retry on, caps tool results with
a paging hint, and bounds the loop by ai.yaml max_tool_calls plus a
hard timeout. Caddy passes the stream unbuffered (flush_interval -1)
in both deployments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 21:47:50 -06:00
|
|
|
),
|
|
|
|
|
),
|
|
|
|
|
'system': dict(
|
|
|
|
|
system_prompt=(
|
|
|
|
|
f'{COMMON_PROMPT}'
|
|
|
|
|
' Your job is to troubleshoot the user\'s WROLPi.'
|
2026-09-02 18:02:09 -06:00
|
|
|
' You never know the current state of this system from memory or from example'
|
|
|
|
|
' conversations: EVERY answer about services, disks, or health starts with a fresh'
|
|
|
|
|
' get_system_status call. Read a failing service\'s logs with get_service_logs;'
|
|
|
|
|
' check drives with list_disks.'
|
ai: the /api/chat agent loop — SSE streaming, modes, tool execution
Tool definitions are generated in-process from the /api/ai blueprint's
own OpenAPI operations (operationId = tool name, description = tool
description, dataclass bodies + declared parameters = one flat JSON
schema) — the spec IS the catalog, nothing to drift. Manage endpoints
never become tools. A mode is a server-enforced system prompt + tool
allowlist (Help / Research / System per ai.md); the executor can only
reach allowlisted /api/ai endpoints, path parameters are encoded and
traversal-rejected so a crafted argument cannot escape an endpoint.
POST /api/chat streams SSE (status/token/tool_call/tool_result/done/
error), starts llama-server on demand, validates tool args server-side
with instructive errors the model can retry on, caps tool results with
a paging hint, and bounds the loop by ai.yaml max_tool_calls plus a
hard timeout. Caddy passes the stream unbuffered (flush_interval -1)
in both deployments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 21:47:50 -06:00
|
|
|
' You CANNOT execute commands or change anything — you can only read.'
|
|
|
|
|
' Only suggest commands that appear in the help documentation: find them with search_help,'
|
|
|
|
|
' quote them exactly, and include the help page link.'
|
|
|
|
|
),
|
|
|
|
|
tools=(
|
|
|
|
|
'get_system_status',
|
|
|
|
|
'list_services',
|
|
|
|
|
'get_service_logs',
|
|
|
|
|
'list_disks',
|
|
|
|
|
'search_help',
|
|
|
|
|
'get_help_doc',
|
|
|
|
|
),
|
|
|
|
|
),
|
|
|
|
|
}
|
|
|
|
|
|
ai: ground answers in a live library map, budget the context
Response-quality work, all aimed at giving the model enough context to
answer accurately:
- Every chat's system prompt now carries a live library map (today's
date, counts, and the exact names of channels, domains, zims, and
inventories, capped and cached 60s). Exact spellings are in context
before the first token, and the list_collections round-trip — minutes
on slow hardware — is usually unnecessary.
- The request is budgeted against the model's real context: oldest
turns are trimmed first, the current question is never dropped, and
an over-budget turn gets a clear "start a new conversation" error
instead of a silently truncated prompt. cache_prompt reuses the KV
cache for the stable prefix, which is most of the latency on a Pi.
- context_size plumbing fixed: selecting a model adopts its catalog
default (the 4B was running at 8k instead of 16k), and llama-server
runs --parallel 1 so the single user gets the whole context.
- temperature is configurable in ai.yaml (default 0.3 — small models
call tools far more reliably near 0.2-0.3 than at server defaults).
- One worked tool-use example per mode is prepended to every
conversation; small models imitate far better than they obey.
- scripts/ai_eval.py scores the assistant against a live WROLPi on
mechanical checks (finished, called the right tool first, included a
WROLPi link, no external URLs) so prompt changes stop being validated
by anecdote.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 16:48:19 -06:00
|
|
|
|
|
|
|
|
def _example(*messages) -> list:
|
|
|
|
|
"""A compact worked exchange (user -> tool call -> tool result -> answer) prepended to every
|
|
|
|
|
conversation. Small models imitate far better than they obey; one example per mode is what
|
|
|
|
|
stops them answering from memory or quitting after the first tool call."""
|
|
|
|
|
return list(messages)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _tool_call(call_id: str, name: str, arguments: str) -> dict:
|
|
|
|
|
return dict(role='assistant', content=None, tool_calls=[
|
|
|
|
|
dict(id=call_id, type='function', function=dict(name=name, arguments=arguments))])
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _tool_result(call_id: str, content: str) -> dict:
|
|
|
|
|
return dict(role='tool', tool_call_id=call_id, content=content)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
MODE_EXAMPLES = {
|
2026-09-02 17:26:29 -06:00
|
|
|
# The example's content must be TRUE (from the real help docs): when a user asks about the
|
|
|
|
|
# example's topic, a small model may answer straight from it without searching.
|
ai: ground answers in a live library map, budget the context
Response-quality work, all aimed at giving the model enough context to
answer accurately:
- Every chat's system prompt now carries a live library map (today's
date, counts, and the exact names of channels, domains, zims, and
inventories, capped and cached 60s). Exact spellings are in context
before the first token, and the list_collections round-trip — minutes
on slow hardware — is usually unnecessary.
- The request is budgeted against the model's real context: oldest
turns are trimmed first, the current question is never dropped, and
an over-budget turn gets a clear "start a new conversation" error
instead of a silently truncated prompt. cache_prompt reuses the KV
cache for the stable prefix, which is most of the latency on a Pi.
- context_size plumbing fixed: selecting a model adopts its catalog
default (the 4B was running at 8k instead of 16k), and llama-server
runs --parallel 1 so the single user gets the whole context.
- temperature is configurable in ai.yaml (default 0.3 — small models
call tools far more reliably near 0.2-0.3 than at server defaults).
- One worked tool-use example per mode is prepended to every
conversation; small models imitate far better than they obey.
- scripts/ai_eval.py scores the assistant against a live WROLPi on
mechanical checks (finished, called the right tool first, included a
WROLPi link, no external URLs) so prompt changes stop being validated
by anecdote.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 16:48:19 -06:00
|
|
|
'help': _example(
|
2026-09-02 17:26:29 -06:00
|
|
|
dict(role='user', content='Why does my browser warn about the certificate?'),
|
|
|
|
|
_tool_call('ex_help_1', 'search_help', '{"search_str": "certificate"}'),
|
ai: ground answers in a live library map, budget the context
Response-quality work, all aimed at giving the model enough context to
answer accurately:
- Every chat's system prompt now carries a live library map (today's
date, counts, and the exact names of channels, domains, zims, and
inventories, capped and cached 60s). Exact spellings are in context
before the first token, and the list_collections round-trip — minutes
on slow hardware — is usually unnecessary.
- The request is budgeted against the model's real context: oldest
turns are trimmed first, the current question is never dropped, and
an over-budget turn gets a clear "start a new conversation" error
instead of a silently truncated prompt. cache_prompt reuses the KV
cache for the stable prefix, which is most of the latency on a Pi.
- context_size plumbing fixed: selecting a model adopts its catalog
default (the 4B was running at 8k instead of 16k), and llama-server
runs --parallel 1 so the single user gets the whole context.
- temperature is configurable in ai.yaml (default 0.3 — small models
call tools far more reliably near 0.2-0.3 than at server defaults).
- One worked tool-use example per mode is prepended to every
conversation; small models imitate far better than they obey.
- scripts/ai_eval.py scores the assistant against a live WROLPi on
mechanical checks (finished, called the right tool first, included a
WROLPi link, no external URLs) so prompt changes stop being validated
by anecdote.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 16:48:19 -06:00
|
|
|
_tool_result('ex_help_1',
|
2026-09-02 17:26:29 -06:00
|
|
|
'{"results": [{"slug": "system/certificates", "title": "HTTPS Certificates",'
|
|
|
|
|
' "link": "/system/certificates/",'
|
|
|
|
|
' "snippet": "…WROLPi generates its own Root CA certificate…"}], "total": 1}'),
|
|
|
|
|
_tool_call('ex_help_2', 'get_help_doc', '{"slug": "system/certificates"}'),
|
ai: ground answers in a live library map, budget the context
Response-quality work, all aimed at giving the model enough context to
answer accurately:
- Every chat's system prompt now carries a live library map (today's
date, counts, and the exact names of channels, domains, zims, and
inventories, capped and cached 60s). Exact spellings are in context
before the first token, and the list_collections round-trip — minutes
on slow hardware — is usually unnecessary.
- The request is budgeted against the model's real context: oldest
turns are trimmed first, the current question is never dropped, and
an over-budget turn gets a clear "start a new conversation" error
instead of a silently truncated prompt. cache_prompt reuses the KV
cache for the stable prefix, which is most of the latency on a Pi.
- context_size plumbing fixed: selecting a model adopts its catalog
default (the 4B was running at 8k instead of 16k), and llama-server
runs --parallel 1 so the single user gets the whole context.
- temperature is configurable in ai.yaml (default 0.3 — small models
call tools far more reliably near 0.2-0.3 than at server defaults).
- One worked tool-use example per mode is prepended to every
conversation; small models imitate far better than they obey.
- scripts/ai_eval.py scores the assistant against a live WROLPi on
mechanical checks (finished, called the right tool first, included a
WROLPi link, no external URLs) so prompt changes stop being validated
by anecdote.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 16:48:19 -06:00
|
|
|
_tool_result('ex_help_2',
|
2026-09-02 17:26:29 -06:00
|
|
|
'{"content": "# HTTPS Certificates\\nWROLPi is offline, so it cannot use a public'
|
|
|
|
|
' certificate authority. It generates its own Root CA certificate which you install'
|
|
|
|
|
' once on each device that connects to it.", "next_offset": null, "total_chars": 205}'),
|
|
|
|
|
dict(role='assistant', content='WROLPi is offline, so it cannot use a public certificate'
|
|
|
|
|
' authority; it signs its own certificate with a Root CA that you'
|
|
|
|
|
' install once on each device. The HTTPS Certificates help page'
|
|
|
|
|
' explains how: /system/certificates/'),
|
ai: ground answers in a live library map, budget the context
Response-quality work, all aimed at giving the model enough context to
answer accurately:
- Every chat's system prompt now carries a live library map (today's
date, counts, and the exact names of channels, domains, zims, and
inventories, capped and cached 60s). Exact spellings are in context
before the first token, and the list_collections round-trip — minutes
on slow hardware — is usually unnecessary.
- The request is budgeted against the model's real context: oldest
turns are trimmed first, the current question is never dropped, and
an over-budget turn gets a clear "start a new conversation" error
instead of a silently truncated prompt. cache_prompt reuses the KV
cache for the stable prefix, which is most of the latency on a Pi.
- context_size plumbing fixed: selecting a model adopts its catalog
default (the 4B was running at 8k instead of 16k), and llama-server
runs --parallel 1 so the single user gets the whole context.
- temperature is configurable in ai.yaml (default 0.3 — small models
call tools far more reliably near 0.2-0.3 than at server defaults).
- One worked tool-use example per mode is prepended to every
conversation; small models imitate far better than they obey.
- scripts/ai_eval.py scores the assistant against a live WROLPi on
mechanical checks (finished, called the right tool first, included a
WROLPi link, no external URLs) so prompt changes stop being validated
by anecdote.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 16:48:19 -06:00
|
|
|
),
|
|
|
|
|
'research': _example(
|
|
|
|
|
dict(role='user', content='Summarize the bakehouse channel'),
|
ai: research mode runs on the compact tool catalog
Research mode switches to nine kind-generic tools: search_files,
get_file, read_content, search_zims, get_zim_entry, get_inventory,
list_files, read_file, search_places. Tool definitions drop from 20
tools / 8,947 chars (~2,240 tokens) to 9 / 3,881 (~970) on every
request, a sixth of the 8k context the small tier runs with instead of
a quarter. The remaining descriptions are trimmed toward 300 chars.
The replaced per-kind endpoints are removed (four searches, three
getters, three readers, search_suggestions, the slug inventory route);
their tests move onto the generic routes. The research prompt, its
worked example, and the empty-search hint no longer tell the model to
list collections first: channels resolve by name from the library map
that is already in the prompt. list_collections, list_tags, and
get_map_overview stay as endpoints for the MCP wrappers but leave the
chat allowlist; the MCP search_suggestions tool goes since search_files
returns the same matches inline.
scripts/ai_eval.py expects search_files and treats "what channels do I
have" as answerable from context. On the 1.7B model against the dev
library both catalogs score 14/18, but the compact one answers with one
tool call where the old one needed up to five, no longer invents the
example's channel, and keeps kinds straight when asked for videos.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 16:28:16 -06:00
|
|
|
_tool_call('ex_res_1', 'search_files', '{"channel": "Bakehouse"}'),
|
ai: ground answers in a live library map, budget the context
Response-quality work, all aimed at giving the model enough context to
answer accurately:
- Every chat's system prompt now carries a live library map (today's
date, counts, and the exact names of channels, domains, zims, and
inventories, capped and cached 60s). Exact spellings are in context
before the first token, and the list_collections round-trip — minutes
on slow hardware — is usually unnecessary.
- The request is budgeted against the model's real context: oldest
turns are trimmed first, the current question is never dropped, and
an over-budget turn gets a clear "start a new conversation" error
instead of a silently truncated prompt. cache_prompt reuses the KV
cache for the stable prefix, which is most of the latency on a Pi.
- context_size plumbing fixed: selecting a model adopts its catalog
default (the 4B was running at 8k instead of 16k), and llama-server
runs --parallel 1 so the single user gets the whole context.
- temperature is configurable in ai.yaml (default 0.3 — small models
call tools far more reliably near 0.2-0.3 than at server defaults).
- One worked tool-use example per mode is prepended to every
conversation; small models imitate far better than they obey.
- scripts/ai_eval.py scores the assistant against a live WROLPi on
mechanical checks (finished, called the right tool first, included a
WROLPi link, no external URLs) so prompt changes stop being validated
by anecdote.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 16:48:19 -06:00
|
|
|
_tool_result('ex_res_1',
|
|
|
|
|
'{"results": [{"id": 12, "kind": "video", "title": "Sourdough Basics",'
|
ai: research mode runs on the compact tool catalog
Research mode switches to nine kind-generic tools: search_files,
get_file, read_content, search_zims, get_zim_entry, get_inventory,
list_files, read_file, search_places. Tool definitions drop from 20
tools / 8,947 chars (~2,240 tokens) to 9 / 3,881 (~970) on every
request, a sixth of the 8k context the small tier runs with instead of
a quarter. The remaining descriptions are trimmed toward 300 chars.
The replaced per-kind endpoints are removed (four searches, three
getters, three readers, search_suggestions, the slug inventory route);
their tests move onto the generic routes. The research prompt, its
worked example, and the empty-search hint no longer tell the model to
list collections first: channels resolve by name from the library map
that is already in the prompt. list_collections, list_tags, and
get_map_overview stay as endpoints for the MCP wrappers but leave the
chat allowlist; the MCP search_suggestions tool goes since search_files
returns the same matches inline.
scripts/ai_eval.py expects search_files and treats "what channels do I
have" as answerable from context. On the 1.7B model against the dev
library both catalogs score 14/18, but the compact one answers with one
tool call where the old one needed up to five, no longer invents the
example's channel, and keeps kinds straight when asked for videos.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 16:28:16 -06:00
|
|
|
' "link": "/videos/12", "channel": "Bakehouse", "duration": "15:00"}], "total": 1}'),
|
|
|
|
|
dict(role='assistant', content='The Bakehouse channel has one video:\\n'
|
2026-09-02 17:26:29 -06:00
|
|
|
'1. **Sourdough Basics** (/videos/12) — a 15 minute introduction'
|
|
|
|
|
' to sourdough baking.'),
|
ai: ground answers in a live library map, budget the context
Response-quality work, all aimed at giving the model enough context to
answer accurately:
- Every chat's system prompt now carries a live library map (today's
date, counts, and the exact names of channels, domains, zims, and
inventories, capped and cached 60s). Exact spellings are in context
before the first token, and the list_collections round-trip — minutes
on slow hardware — is usually unnecessary.
- The request is budgeted against the model's real context: oldest
turns are trimmed first, the current question is never dropped, and
an over-budget turn gets a clear "start a new conversation" error
instead of a silently truncated prompt. cache_prompt reuses the KV
cache for the stable prefix, which is most of the latency on a Pi.
- context_size plumbing fixed: selecting a model adopts its catalog
default (the 4B was running at 8k instead of 16k), and llama-server
runs --parallel 1 so the single user gets the whole context.
- temperature is configurable in ai.yaml (default 0.3 — small models
call tools far more reliably near 0.2-0.3 than at server defaults).
- One worked tool-use example per mode is prepended to every
conversation; small models imitate far better than they obey.
- scripts/ai_eval.py scores the assistant against a live WROLPi on
mechanical checks (finished, called the right tool first, included a
WROLPi link, no external URLs) so prompt changes stop being validated
by anecdote.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 16:48:19 -06:00
|
|
|
),
|
2026-09-02 17:57:50 -06:00
|
|
|
# No invented failures in the example: the model carries them into real diagnoses (it kept
|
|
|
|
|
# investigating a fictional failed service the first version of this example described).
|
ai: ground answers in a live library map, budget the context
Response-quality work, all aimed at giving the model enough context to
answer accurately:
- Every chat's system prompt now carries a live library map (today's
date, counts, and the exact names of channels, domains, zims, and
inventories, capped and cached 60s). Exact spellings are in context
before the first token, and the list_collections round-trip — minutes
on slow hardware — is usually unnecessary.
- The request is budgeted against the model's real context: oldest
turns are trimmed first, the current question is never dropped, and
an over-budget turn gets a clear "start a new conversation" error
instead of a silently truncated prompt. cache_prompt reuses the KV
cache for the stable prefix, which is most of the latency on a Pi.
- context_size plumbing fixed: selecting a model adopts its catalog
default (the 4B was running at 8k instead of 16k), and llama-server
runs --parallel 1 so the single user gets the whole context.
- temperature is configurable in ai.yaml (default 0.3 — small models
call tools far more reliably near 0.2-0.3 than at server defaults).
- One worked tool-use example per mode is prepended to every
conversation; small models imitate far better than they obey.
- scripts/ai_eval.py scores the assistant against a live WROLPi on
mechanical checks (finished, called the right tool first, included a
WROLPi link, no external URLs) so prompt changes stop being validated
by anecdote.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 16:48:19 -06:00
|
|
|
'system': _example(
|
|
|
|
|
dict(role='user', content='Is everything running?'),
|
|
|
|
|
_tool_call('ex_sys_1', 'get_system_status', '{}'),
|
|
|
|
|
_tool_result('ex_sys_1',
|
|
|
|
|
'{"version": "0.28", "wrol_mode": false, "services": [{"name": "wrolpi-api",'
|
2026-09-02 17:57:50 -06:00
|
|
|
' "status": "running"}, {"name": "wrolpi-app", "status": "running"}], "errors": []}'),
|
2026-09-02 18:02:09 -06:00
|
|
|
dict(role='assistant', content='The status shows both services (wrolpi-api, wrolpi-app)'
|
|
|
|
|
' running on version 0.28, with no errors reported.'),
|
ai: ground answers in a live library map, budget the context
Response-quality work, all aimed at giving the model enough context to
answer accurately:
- Every chat's system prompt now carries a live library map (today's
date, counts, and the exact names of channels, domains, zims, and
inventories, capped and cached 60s). Exact spellings are in context
before the first token, and the list_collections round-trip — minutes
on slow hardware — is usually unnecessary.
- The request is budgeted against the model's real context: oldest
turns are trimmed first, the current question is never dropped, and
an over-budget turn gets a clear "start a new conversation" error
instead of a silently truncated prompt. cache_prompt reuses the KV
cache for the stable prefix, which is most of the latency on a Pi.
- context_size plumbing fixed: selecting a model adopts its catalog
default (the 4B was running at 8k instead of 16k), and llama-server
runs --parallel 1 so the single user gets the whole context.
- temperature is configurable in ai.yaml (default 0.3 — small models
call tools far more reliably near 0.2-0.3 than at server defaults).
- One worked tool-use example per mode is prepended to every
conversation; small models imitate far better than they obey.
- scripts/ai_eval.py scores the assistant against a live WROLPi on
mechanical checks (finished, called the right tool first, included a
WROLPi link, no external URLs) so prompt changes stop being validated
by anecdote.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 16:48:19 -06:00
|
|
|
),
|
|
|
|
|
}
|
|
|
|
|
|
ai: the /api/chat agent loop — SSE streaming, modes, tool execution
Tool definitions are generated in-process from the /api/ai blueprint's
own OpenAPI operations (operationId = tool name, description = tool
description, dataclass bodies + declared parameters = one flat JSON
schema) — the spec IS the catalog, nothing to drift. Manage endpoints
never become tools. A mode is a server-enforced system prompt + tool
allowlist (Help / Research / System per ai.md); the executor can only
reach allowlisted /api/ai endpoints, path parameters are encoded and
traversal-rejected so a crafted argument cannot escape an endpoint.
POST /api/chat streams SSE (status/token/tool_call/tool_result/done/
error), starts llama-server on demand, validates tool args server-side
with instructive errors the model can retry on, caps tool results with
a paging hint, and bounds the loop by ai.yaml max_tool_calls plus a
hard timeout. Caddy passes the stream unbuffered (flush_interval -1)
in both deployments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 21:47:50 -06:00
|
|
|
# Suggestion chips shown by the Chat tab; small models do much better when the opening message
|
|
|
|
|
# matches what the mode's prompt was tuned for.
|
|
|
|
|
MODE_SUGGESTIONS = {
|
|
|
|
|
'help': ['How do I download videos?', 'How do I add my own files?', 'What are Tags?'],
|
|
|
|
|
'research': ['Find videos about canning', 'What do I have about gardening?', 'Summarize my newest archives'],
|
|
|
|
|
'system': ['Why is my drive full?', 'Is everything running?', 'Why do downloads fail?'],
|
|
|
|
|
}
|