mirror of
https://github.com/lrnselfreliance/wrolpi.git
synced 2026-11-20 23:16:17 -05:00
ai: package llama-server for all three deployments, with idle unload
install_llama_cpp.sh builds llama-server from the SHA-pinned v0.3.0 source tarball (single source of truth: on-device upgrade, Pi image, Live ISO, and the new docker/llm container all call it; version-marker makes re-runs no-ops). start_llama_server.sh refuses to start unless ai.yaml enables AI and the active model file exists — which also covers unmounted media. wrolpi-llm.service is installed but deliberately never enabled at boot: the Controller starts it on demand (registered in managed_services; the llm compose service follows the samba pattern), and the API's perpetual idle-unload worker asks the Controller to stop it after the configured idle minutes. llama-server is localhost-only natively, internal-network-only in docker, and never proxied by Caddy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
256750e538
commit
d9924b6b53
18 changed files with 406 additions and 4 deletions
17
docker/llm/Dockerfile
Normal file
17
docker/llm/Dockerfile
Normal file
|
|
@ -0,0 +1,17 @@
|
|||
FROM debian:13
|
||||
ENV DOCKER=true
|
||||
|
||||
RUN apt-get update && apt-get install -y curl ca-certificates
|
||||
|
||||
# Build llama-server from the pinned release (single source of truth with native installs).
|
||||
COPY wrolpi/scripts/install_llama_cpp.sh /tmp/install_llama_cpp.sh
|
||||
RUN /tmp/install_llama_cpp.sh && rm /tmp/install_llama_cpp.sh \
|
||||
&& apt-get remove -y build-essential cmake && apt-get autoremove -y && apt-get clean
|
||||
|
||||
# The same startup scripts the native install uses.
|
||||
COPY scripts/start_llama_server.sh /opt/wrolpi/scripts/start_llama_server.sh
|
||||
COPY wrolpi/scripts/read_config_value.sh /opt/wrolpi/wrolpi/scripts/read_config_value.sh
|
||||
|
||||
COPY docker/llm/entrypoint.sh /entrypoint.sh
|
||||
|
||||
ENTRYPOINT ["/entrypoint.sh"]
|
||||
5
docker/llm/entrypoint.sh
Executable file
5
docker/llm/entrypoint.sh
Executable file
|
|
@ -0,0 +1,5 @@
|
|||
#! /usr/bin/env bash
|
||||
# Bind all interfaces so the api container can reach llama-server on the internal docker network.
|
||||
# The container publishes no host ports and is not proxied by Caddy.
|
||||
export LLAMA_HOST=0.0.0.0
|
||||
exec /opt/wrolpi/scripts/start_llama_server.sh
|
||||
Loading…
Add table
Add a link
Reference in a new issue