# Private llama-server — development candidate

CPU-only llama.cpp `b9860`, pinned to source revision
`fdb1db877c526ec90f668eca1b858da5dba85560`. The build uses the upstream-documented
GCC 14 toolchain with host-specific optimization disabled. No model is bundled;
no Hugging Face model or Web UI asset download is requested. The image and lifecycle
have been run on real nodes (archy-x250-pa3); the base image is pinned by
multi-arch index digest.

## Connection boundary

- No published host ports, no browser interface, no automatic FIPS exposure.
- Container DNS alias `llama-server` on `archy-net`; the candidate Routstr server
  can be configured with generic upstream base URL `http://llama-server:8080/v1`.
- A dedicated generated `llama-server-api-key` is injected through Podman secrets
  as `LLAMA_API_KEY`, an upstream-supported authentication option.
- The provider configuration must receive that key through a private trusted
  configuration path; it must never pass through AIUI, a model prompt or logs.
  Automatic key handoff/configuration RPC is not implemented yet.
- `archy-net` is shared. API authentication is required, but this is not dedicated
  network isolation. A private stack network remains an acceptance gate.
- The raw model server is not a payment boundary. Only the Routstr gateway should
  grant remote paid inference after peer authorization and accounting are complete.

## Model prerequisites

The operator must explicitly import a compatible GGUF as
`/var/lib/archipelago/llama-server/models/model.gguf` and verify its provenance,
license and checksum. The model mount is read-only. Nothing downloads a model
merely because the app is installed. Without a model it will fail to start; the
model-import/readiness UI and install prerequisite checks are still pending, so
this candidate is **not ready for catalog inclusion**.

Tested model: Qwen2.5-1.5B-Instruct `q4_k_m` (Apache-2.0) from
`Qwen/Qwen2.5-1.5B-Instruct-GGUF` revision
`91cad51170dc346986eccefdc2dd33a9da36ead9`, sha256
`6a1a2eb6d15622bf3c96857206351ba97e1af16c30d7a74ee38970e434e9407e`.
TinyLlama 1.1B was dropped: it is trained on 2048 tokens, llama-server silently
caps the context to that, and AIUI's instructions alone are about 2.4k tokens.

Initial limits: 8 GiB container memory, threads defaulting to the physical core
count, 8192-token context, one slot and a 1024-token generation setting. These are an initial small-model test
profile, not a promise that an arbitrary Llama model fits. Caller token bounds,
admission control and actual limit enforcement must be tested separately; do not
treat command-line defaults as the paid request budget ceiling.

The stable model alias is `archy-llama`. Test a non-streaming completion and exact
usage reporting before enabling it in Routstr. Tool support, streaming and GPU
variants remain unqualified. `/health` is the runtime's model-loading health
endpoint; it does not prove billing correctness.

## Source references

All references are to the pinned upstream revision:
- [server API and options](https://github.com/ggml-org/llama.cpp/blob/fdb1db877c526ec90f668eca1b858da5dba85560/tools/server/README.md)
- [upstream CPU build](https://github.com/ggml-org/llama.cpp/blob/fdb1db877c526ec90f668eca1b858da5dba85560/.devops/cpu.Dockerfile)
- [build options](https://github.com/ggml-org/llama.cpp/blob/fdb1db877c526ec90f668eca1b858da5dba85560/CMakeLists.txt)

Offline validation from the repository root:

```sh
bash scripts/validate-app-manifest.sh apps/llama-server/manifest.yml
python3 -m unittest discover -s tests/routstr-provider -p 'test_*.py' -v
```